Meet Drew Manderfeld!

I cannot believe that October is already coming to a close. The beautiful fall that we started with has suddenly slipped away and now heaters are turning on and winter coats coming out. We’ve been busy this week in the office with Halloween Treat Week and while in the past people have simply bought candy and the mound has grown to an enormous height by the end of the week, this year it seems people are branching out a bit more by baking cookies and muffins and buying cakes, pumpkin bread and cider. We’ve pretty much been on a sugar overload all week. Other office events include the info session we had last Wednesday at KU, from which we hope to gain some new talent to add to our growing company and the Halloween costume party and luncheon we will be having next Monday, which should be lots of fun.

As for now, it is time to meet our computer science intern from Overland Park, Drew Manderfeld. Drew will be graduating from KU this spring with a BS in Computer Science and after graduation Drew says he plans to go on full-time here at AllofE. When asked what Drew enjoyed the most about working at AllofE he said, “Being able to work alongside such talented and smart people. It is a huge advantage and keeps everyone striving to be at the top of their game.” Drew, like the rest of us, is involved in multiple projects, but his main focus is Matrix where he is translating assortments of student data into useful and readable information. Get to know Drew a bit better by how he answered the following questions:

What are some of your hobbies and interests?

I love soccer, but I rarely get to play anymore so instead I have season tickets to Sporting Kansas City. I enjoy other sports as well and am very disappointed I can’t play vAllOfEball this season due to class, but I think I’ll be back next season!

If you could do anything (your imagination is the limit) what would you do and why?

This might be a little out there but I believe there are multiple paths to happiness so I would love to be able to live two lives at once; one being my current life and another being a professional soccer player or something. There is always so much I want to do I feel like I need two lives.

What song would you choose to sing at a Karaoke bar if you had to sing something? I can’t sing well but I always end up singing and dancing to ‘When the Going Gets Tough’ by Billy Ocean.

Would you rather be on Dancing with the Stars or X-Factor (American Idol)?

I can’t dance or sing so I’d be better suited for Programming with the Stars or something.

If you had to trade lives with an animal for a day, which animal would you choose and why?

I think I’d go small like an ant or something. Stuff would be so amazing when you are so tiny. I’ve also always wanted to eat a giant Oatmeal Crème Pie like in Honey I Shrunk the Kids.

What is your favorite holiday?

Halloween with friends and Christmas with the family.

What is #1 on your Christmas wish list this year?

A long-sleeve Sporting KC home jersey.

Sky diving or bungee jumping?

I want to do both!

Extra pair of eyes or extra pair of ears?

Extra pair of eyes, my vision is horrible.

Eat 10 live worms or 20 dead flies?

10 live worms easy.

Naturally purple skin or naturally green skin?

Green I suppose.

Moped or Segway?

My girlfriend wants a moped so I’ll go with that, I’m happy with a car.

Mac or PC?

PC 100%

Thanks Drew! I hope everyone has a safe, fun and happy Halloween weekend!

Types of Student Growth Models

Everyone involved in K-12 education has probably heard of the term Adequate Yearly Progress (AYP). This is a report that determines the amount of positive influence (efficiency) of a school on a student. As student growth becomes more and more popular, AYPs have begun incorporating growth models in their calculations. Because these models differ from state to state, it is important to know what kinds of models your state is using to determine just how effective your school is.

Currently, there are three main categories of student growth models: growth-to-proficiency, transition-matrix, and projection models. Currently, Matrix clients include states that have adopted the growth-to-proficiency model. As Matrix’s user base grows, so too will the support for additional growth models including transition-matrix and projection.

All of these models use the idea of proficiency. Proficiency is a level of student achievement (on test scores) that illustrates appropriate student ability. Students are said to be sub-proficient if they are below this cutoff point. Some models make use of additional levels of student achievement, such as weak sub-proficiency (lowest level), low marginal sub-proficiency, high marginal sub-proficiency, proficient, and advanced (highest level), but the most important cutoff in these models is the proficient level.

The growth-to-proficiency or trajectory model asks that a student below proficiency will become proficient in some set amount of time. It compares current student achievement to recommended student achievement (e.g. the proficient cutoff) and asks that these two values meet in a few years (typically 3-4). This model awards AYP points if a student has met proficiency or is on track to proficiency. The problem with this model is that it expects sub-proficient students to make larger-than-typical gains every year to even be considered as “on track.”

As you can see, the sub-proficient student needs to make up quite a bit of additional ground each year to become proficient soon. This model thus presents a conundrum: if a teacher does just as well with their sub-proficient students as with their proficient students, they are actually punished for not achieving above-average growth results. Thus, this model may encourage teachers to create curricula that cater towards and pay disproportionate attention to sub-proficient students. Furthermore, because this model does not account for growth above the proficiency line, a teacher has no incentive to ensure a proficient or advanced student achieves any growth whatsoever. A teacher only has incentive to make sure they do not fall below proficiency.

The transition-matrix model uses a table of sub-proficiency levels and rewards students who grow from one achievement level to the next. Students who grow at least one level are considered “on track” for AYP purposes. Some versions of the transition-matrix model award full points to sub-proficient student growth, and some versions award some fraction of points to these students. Personally, a transition-matrix table that uses partial points, such as the Delaware model, misses the point of growth entirely -- that students should not be punished for starting at a low achievement level. A transition matrix table similar to the one used in Delaware’s model is shown below.

Transition-matrix models (at least, the ones that award full points for all growth) alleviate one of the problems of growth-to-proficiency models. Namely, that sub-proficient but normal growth is treated as the solid achievement that it is and does not punish a student for starting low.

One disadvantage to this model is similar to the growth-to-proficiency model in that proficient or higher students are not rewarded for growth at all. Again, this can cause teachers to pay less attention to these students.

Projection models are a more complicated, predictive analysis model. These models take into account cohort (previous classes) test scores, and compares them to this year’s class scores. This model properly takes into account growth regardless of the student achievement level (which reduces the ability to game the system), takes into account changes from one year to the next in the same subject area (for example, stunted growth in math across the board when students encounter algebra), and has the best predictive ability of all the models with upwards of 80% correct future proficiency prediction.

The s points represent the current student’s yearly scores, and the p points represent the proficiency cutoff for that year. The solid line takes the current student’s data (s points) along with some fancy regression analysis using the p points to determine where the student will land next year. As we can see, the student is currently in grade 5 and sub-proficient. However, this model will consider the student to be “on track”, since they are projected to achieve proficiency in grade 6.

The solid predictive power of projection models come at a large cost to complexity. While this graph may or may not be easily explainable to administrators, principals, and teachers, the regression analysis used in the background is not. This means that the model is now harder to explain and can make the model feel “less accountable” than a simple model. Furthermore, the projection in this model uses probability to determine the most likely outcome for a student. The fact that probability is involved means that there is uncertainty involved with the measurements. This means a student’s projection might be either too high or too low, which can place either too many or too few students in the “on track” category for AYP.

All three models have their upsides and downsides, and hopefully states new to student growth will choose a model that’s right for them.


Further Reading

Auty, William, et al. Implementor’s Guide to Growth Models. CCSSO: The Council of Chief State School Officers. June 2008. http://www.ccsso.org/Documents/2008/Implementers_Guide_to_Growth_2008.pdf

Hoffer, Thomas B., et. al. Final Report on the Evaluation of the Growth Model Pilot Project. U.S. Department of Education. January 2011. http://www2.ed.gov/rschstat/eval/disadv/growth-model-pilot/index.html

Meet Bryce Minter!

Another week is about to come to a close and that just means that we are one week closer to Halloween. While, I unfortunately do not have an elaborate Halloween costume planned, I do have a small October to-do list to get through. What is on that list? Well, firstly, I want to eat as much pumpkin-flavored food items as possible. I’ve kind of had a slow start, but it is about to pick up. I made some pumpkin-squash-coconut soup last week. I just bought pumpkin-spiced bagels and I’ve collected all the necessary ingredients to make some delicious pumpkin waffles this weekend. Of course I’ll have to make a pumpkin pie and maybe some pumpkin bread or muffins too before the month ends. I highly recommend pumpkin ice cream for those ice cream lovers out there, it is probably my favorite. Next thing is to visit the pumpkin patch and the apple cider mill. Our very own Louisburg Cider Mill is supposedly ranked as one of the top ten in the nation. It has been over a decade since the last time I was there and I think it deserves another visit. Third on the list is going to Worlds of Fun during their Halloween Haunt season. I’ve never been, but I hear great things. Sounds like I’m going to be busy for this weekend and all of next week!

But enough about me and my plans! It is time to meet another one of my coworkers! Today we are highlighting our Computer Science Intern, Bryce Minter. Bryce is a KU student from Kings Mountain, North Carolina and has been interning with us for just over a year. When asked what Bryce liked the most about working at AllofE, he said, “The sheer amount of projects and code base to work on and the calm, relaxing atmosphere.” He says he works on a little bit of everything. If anyone needs something done, Bryce is willing to do it. Check out how Bryce answered some of my other questions:

• What are some of your hobbies and interests? In my free time I like to do programming, mainly game programming, but also enjoy doing parkour when I have time outside of school and work.

• If you could have any talent what would it be and why? A photographic mind to remember any date, any day perfectly upon recalling information.

• If you could create a new law for the U.S. what would it be and why? It wouldn't be passed if I proposed it anyways...

• What is your favorite sport and/or sports team? Don't really have a favorite but I hope that detroit does pretty well this year because I have the quarterback for my fantasy team!

• What is your biggest pet peeve? People who don't respond when they are in front of any type of screen, especially texting.

• Stuck in the freezing mountains or the hot desert? Depends on the situation I was left in, if I had the tools to survive in the freezing mountains then the freezing mountains but if I didn't have any tools then the hot desert.

• Mercedes Benz or Porsche? Im fine with the car I have now lots of money has gone into it.

• Cake or Pie? I shall answer your question with a question, Is boston cream pie considered a pie or cake?

• Extra pair of arms or extra pair of legs? Arms, I think it would take to long to get used to an extra pair of legs. Also theres no place for the legs to really go.

• Most famous person in the world or completely unknown by anyone but your mother? Why not shoot for the most famous person, nothing to lose, at worst I end up being completely unknown by anyone.

• World’s longest finger nails or world’s biggest ears? World's biggest ears the better to hear you with!

• Mac or PC? PC

• iPad or Laptop? Laptop. Can't program on a iPad.

Thanks Bryce!

Everyone in the office is starting to plan out their costume ideas because Halloween falls on a Monday this year and that means we get to come to work as whoever we want to be! We’ll definitely be posting some pictures of how that turns out. Then it is on to Thanksgiving festivities! Oh, how I love the holiday season.

On The Drilldown

Introduction - History

At AllofE, we have clients in fields ranging from K-12 to higher education, from trucking to sports branding analysis. Every sector has its own challenges, and every project we complete gives us new perspectives on how to solve complex problems. The problem of aggregating massive amounts of data and displaying it in an understandable and intelligence-focused fashion is one of the hardest problems we've come across. After a lot of scrap pieces of paper, dried whiteboard markers, and long discussions, we came up with a solution that is easy to use, fast, and brings to light intelligence from data: the Drilldown. Although it was originally developed to solve a problem completely unrelated to student growth, the Drilldown is the best way to display a district's worth of information in an understandable and intuitive manner.

What the Drilldown Shows

Imagine a district's data as one giant table of raw data. Its columns would number in the hundreds, and its rows would number in the thousands. Because of the massive amount of data, extracting intelligence by looking at each and every piece of data is an impossible task. Different groups and cohorts of the data need to be displayed for the data to be truly understood.

The group people are most familiar with is the gender group. Matrix goes through every record in the data warehouse and places the record into one of two cohorts: male students or female students. The average scores for each type of test are then calculate across multiple years, and displayed for analysis. Being able to see the performance of males vs. females for an entire district, from clicking on the Drilldown to seeing the data, takes a matter of seconds. Matrix gives users access to all of their data, and allows them to understand all their data without having to wait hours or days.

The gender group is just one of many different groups the Drilldown uses. The ethnicity group breaks down data into even finer cohorts. Lets say a district administrator wants to know which ethnicity their teachers need to focus on more than others. By building a cohort for each ethnicity, Matrix can quickly show the performance for each ethnicity, and allow that administrator to quickly find which ethnicity teachers need to focus on.

The gender and ethnicity groups are two of the groups Matrix shows based on student demographics. Our data warehouse also connects together schedules and enrollments with students, and allows the Drilldown to show groups based on grade level and school. The district administrator from the ethnicity group may also want to know how much students are growing as they progress through the district. By looking at cohorts in the grade level group, they can see if there is positive growth through each grade level. They can also see the growth in each particular school to get an even deeper sense of how students are moving during their time in the district.

Combining Groups

Up to this point, the questions asked of the Drilldown have only been asked at a district level. Questions like "how well are we reaching 3rd grade males vs. 3rd grade females" or "which ethnicity is performing the best in an elementary schools" are questions that the Drilldown can also answer. Every cohort is clickable and allows you to zoom in or drilldown (where the name came from) to see a more focused picture of the district.

The first question, "how well are we reaching 3rd grade males vs. 3rd grade females," can be answered by one click of the mouse. By going to the grade level group and clicking on the 3rd grade cohort, the Drilldown refreshes itself to show data tied specifically to 3rd grades. Middle and High Schools will no longer show up under the school group, and 3rd grade will be the only entry in the grade level group. The data in the gender group shows performance of males and females in 3rd grade. In a manner of seconds, the Drilldown has answered a question some districts have spent days trying to answer. The Drilldown also allows for the performance of 4th grade males vs. 4th grade females to be answered in a similar manner.

Clicking on a cohort in the Drilldown allows the lens to be focused on specific parts of a district. It's easy to see that this method of filtering allows for finer and finer views of a district to be constructed and understood. Moving between cohorts like 3rd and 4th grade is also an easy process. Each filter is shown at the bottom of the Drilldown, and can be removed by clicking on it. The answer the previous question to about gender performance in 4th grade can be found by clicking on the existing 3rd grade filter, then filtering by the 4th grade cohort. The process of adding and removing filters allows users with any level of technical experience to understand student performance in their district in a manner of seconds.

Exports, Printing, and Detail Data

Matrix is an application that is accessible by anyone that has a computer and an Internet connection. It is also an application that recognizes the value of extracting data out of it. Being able to print a grid or save what's visible on the screen into a spreadsheet is possible throughout the system. The Drilldown takes the guesswork out of saving information for a presentation or for later analysis by offering both Export and Print buttons.

Along with printing and exporting, the Drilldown has the ability to display all of the underlying data used to create its filtered view. By clicking on the Detail Data button, users are shown the student level data used to create the Drilldown. Matrix makes it possible to move from a view of the entire district down to a set of 10 students within four or five clicks. The Detail Data is also exportable and printable, and can also display a student's entire history with just one click.

Conclusion - Answering Questions

The Drilldown was designed to answer questions a user may have about the performance in their district. Questions an administrator, principal, or teacher may have about student performance are answerable with a few clicks. Thanks to the Drilldown, the long, cumbersome process of answering one question at a time from a spreadsheet thousands of lines long is now fast, efficient, and possible for any user within a manner of seconds.

Matrix

There is a lot of controversy out there about whether or not we should be even using standardized testing as the best means of understanding a student’s academic performance ability. I am not going to make an argument for or against it, but I will say that it doesn’t appear to be going anywhere anytime soon. So I’d like to tell school districts how they can better use the data that they receive from those standardized tests.

The answer is that we have to find a way to make that data become useful information. Often times schools collect this information but find it too overwhelming or costly to make use of. We cannot have administrators and principals making important decisions for our schools based solely on intuition.

Enter “Matrix.”

Matrix is a web-based software program that acts as a data warehouse and student data analysis tool. With Matrix, school districts will be able to have all of a student’s information in one centralized location. Data from the district’s Student Information System (SIS) is integrated into Matrix right alongside the students’ assessment test scores, so that Matrix can make correlations between and generate reports for these two important data sets.

What does that mean?

Well the demographic information, like gender, ethnicity and meal-plan status from the SIS are correlated to assessment test scores for each student and then presented in an easy-to-read and -understand format for school officials to examine. This helps school officials look into factors that occur outside of the classroom, which may be playing a role in how well the student performs. For example, there may be a correlation between the students who receive free or reduced cost meals and their below average test scores. Of course there is a significant difference between correlation and causation, which must be kept in mind. Just because two things are correlated does not mean that one caused the other to occur, but on the other hand it doesn’t necessarily mean that it didn’t either. Matrix at least presents this sort of information to the school districts and gives them the ability to examine and understand teacher and student performance at a much deeper level.

Consider how a teacher may receive a really bad score (which is just the average of all of her students). Does this mean that the teacher should be let go? No, it means that more information needs to be taken into consideration.

Why did he or she get such a low score?

First of all, Matrix allows all aggregated values to be viewed by their raw components. This allows teachers to see which students are doing well and which are not.

Second, Matrix shows not only test scores, but also growth. Perhaps the teacher teaches a large number of academically failing students, but he or she gets a majority of them to improve their scores throughout the year. Even though the ending score is still not proficient, there was great progress made.

Third, Matrix was designed to accumulate data over a long period of time. This means that school officials can examine information for a teacher over the past several years that he or she has taught; the same goes for students. It is never wise to make critical decisions based on a one-time occurrence. Matrix shows when students or teachers consistently perform at a certain level.

Fourth, Matrix allows higher-up school officials to see which cohort of teachers produces the best students or the worst students. This may point out some key areas that are in need of improvement. This also presents school officials with the opportunity to inquire about the teaching methods of those teachers who seem to consistently performing well and then share those methods with the under performing teachers.

Lastly, Matrix presents all of this information in such an easy-to-read and easy-to-understand format that school districts wouldn’t need to waste money on hiring a data maven or training teachers how to use the product. Also reports can be easily generated and printed in order to share the information with other school officials and/or parents.

The bottom line is that if we ever hope to improve the education of our students we have to know what areas need help and what some of the reasons for that may be. Matrix assists in this process in a cost-efficient, easy and fast way. We know that schools are facing increasing budget cuts as well as increased pressure from the State to meet assessment standards and that is why we developed a tool for educators to use to start understanding where improvements need to be made. Matrix gets that conversation going.

Meet Zach Gardner!

As I was on my way to work this morning, I was catching up with my mother on the phone when all of the sudden a fat hairy spider fell onto my steering wheel. I freaked out a little; my mom asked if I was okay and at that moment the spider started to move so I hastily replied no and hung up in order to focus on how I was going to get him away from my hand. I ended up grabbing the white cardigan that I planned on wearing at work in order to kill him, because that was all I had around. I was at a stop light as I jabbed at the spider and ended up honking my horn really intensely. Then the light turned green and traffic started going, but I was distracted so I delayed a few seconds. When I did start to go I pressed too hard on the pedal in order to catch up with traffic and ended up screeching my tires. I’m pretty sure that everyone around me thought I was an unfit driver. When I got to work, I couldn’t find any spider remains on my cardigan or on the wheel. I did make it to work on time and as far as I can tell…I’m still an average human being. Guess it is going to be just another normal day…until I have to drive home again.

However, I want to shift the focus away from my arachnophobia and introduce you to another one of my co-workers, Zach Gardner. Zach was raised in Lawrence, Kansas and graduated from KU with a BS in Computer Science. He is a web developer at AllofE and is currently working on Matrix 5 (which means he will be my new partner in crime as Matrix is also my focus). When asked what he liked most about working at AllofE, he said, “The ability to create. Some people use paint, some use ink, some use steel, and some use food as their medium. I use computers.” When Zach is not working, he enjoys cooking and helping plan for his wedding which is coming up in June of 2012. Check out how Zach answered some of my other questions:

• If you could have any superpower what would it be and why? Foresight. A power that could help me go 24 hours and make every correct decision is worth its weight in gold.

• If you could go anywhere in the world, where would you go and why? Boleyn Ground, London. I've been a fan of West Ham United, a soccer team in England, for the past 4 years, and I've never seen them in person.

• What is one food that you could never live without? Pizza.

• What is your favorite music genre? Hip-hop.

• Night Owl or Early Bird? Early Bird.

• Taco Bell or Burger King? Neither.

• Christmas or Thanksgiving? Thanksgiving.

• 8 Babies or 12 Dogs? Dogs.

• No Legs or No Arms? Legs.

• No Taste or No Hearing? Taste.

• Mac or PC? PC.

• iPad or Laptop? Laptop.

I just want to point out how comical it is that Zach would choose between either having no legs vs. no arms and between no taste vs. no hearing, but he would not choose between Taco Bell or Burger King. Haha, anyway thanks Zach! Have a great week everyone!

Shopping for a Curriculum Management System

A couple weeks ago, I wrote about the need for curriculum management in higher education--specifically in programs with defined standards and competencies required for accreditation.

Since then, I’ve been considering how a program should shop for a curriculum management system. Choosing the right system is a heavy decision. You want to have a really good idea of what you’re getting into before you make the commitment. The system you choose will affect the lives of your faculty and your students daily.

Before you decide on a curriculum management system, I hope you will consider the following questions.


Does it do what your program needs it to do?
For the programs we’ve been working with, this means tracking the standards required for accreditation and simplifying the accreditation process. As we’ve been researching the current practices related to curriculum tracking and mapping within institutions of higher education, I’ve learned many institutions are spending amazing amounts of time tracking curriculum. Some programs meet every month, some meet every week (usually in addition to lengthy meetings at the beginning of each semester) to go over each course and ensure it has met the standards and competencies it was supposed to. Programs are also spending incredible amounts of time preparing for their accreditation reviews.

With the right system, much of this work can be eliminated. A lot can be automated; the rest can be made easier using a web-based system in which information is centralized, giving members of the faculty access to the curriculum information all the time. The right system will also simplify your accreditation review preparation with automated reports and other features to make your review much less daunting.

Can it be customized?
Every Physician Assistant program is different, as is every Dental Hygiene program and every Chiropractic program. Though each and every Physician Assistant program, for example, must meet the same standards to be accredited, each program finds unique ways to serve and educate. Your curriculum management system must recognize that. Customization should include more than changing the font or a title. You don’t want a cookie-cutter solution. You should not need to adapt to the system--the system should adapt to you.

Was it designed to be a curriculum management system?
A generic content management system cannot merely be tweaked to handle your curriculum. You need something designed specifically for the unique needs presented by curriculum management. A generic content management system will not generate reports. It will not consistently structure your information. You’re looking for something that will simplify your life, not complicate it.

Additionally, a system that tries to do everything (admissions, online assessments, demographics, delivering pizza...) misses the point. Because it tries to do everything, it risks not doing any of them well. A system with an all-in-one approach may lack the detail you need to efficiently manage your curriculum. Focus on the needs your program has. Do you need admissions, or does your school already have an admissions department? You know you need curriculum management. Find a system that handles your curriculum extremely well--not one that manages it ok but also has a million features you’ll never use.

This idea goes hand in hand with customization. Ideally, the company you choose to work with will have experience and expertise in a variety of areas--certainly in curriculum management, but also in content management, maybe online examinations or catalog management as well. The more experience the company has--the broader their range of expertise--the better that company will be able to ensure that the system does all you need. If you have an incredible system for tracking your demographic information, but need a little help with managing your curriculum alignment, an all-in-one system probably is not for you. Why would you trash what you’ve already got going for you? The company you choose should be willing to work with what you have, build on your strengths, and give your program a boost where it needs one.


Is the system easy for your faculty to learn and implement?
Your entire faculty will need to use the curriculum management system to some extent, so you need to be sure it’s easy to understand and to implement. Curriculum management should not require extensive technical knowledge nor extensive training--you have more important things to do than study a technical guide or attend countless training sessions. Check out your system before you buy it. Is it intuitive, or do you guess where to find the information you need? Learning the right system should be a natural, easy process.


Does the company know what it is doing?
The company you choose to work with should have experience in curriculum management. Having experience in the field is the only way the system they build will be able to do what you need it to do. A company that works only in general content management can’t know what educators need and will therefore either fall terribly short of those needs or will take forever to get it right.

Is the technology up to date?
Before buying a system, you should make sure you see a demonstration of how it works. In the demo, consider whether the technology seems modern. Does it look like something built in the early nineties? Is it direct, or does every action require a circuitous path? You should also study the company’s web site. If it is not presented well, if it is difficult to navigate, your system likely will be as well.