Beyond Standardized Tests: Student Survey Data in Matrix

We spend a lot of time in Matrix working on standardized test data. There is, after all, quite a lot of it, and almost all of our Matrix clients have standardized test data in our systems. Even so, I always enjoy getting a chance to work on data the districts are generating themselves. One of our clients periodically takes surveys of their students' experiences and attitudes, and they approached us about bringing this data into Matrix, so it would be available alongside the rest of their data.

There were different sets of questions for elementary, middle, and high school students at each survey administration, and though most of them were of the “Strongly Agree” to “Strongly Disagree” form, there were also questions asking about duration, frequency, or specific elements. We were able to teach Matrix how to distinguish the different answer scales and display the data as a grid or a chart, with all the filtering and drill down capabilities our Matrix clients are familiar with.

It's amazing how displaying data in a more engaging way can change the way you experience it. None of the information in these surveys was new to the client, but the first time they saw it in Matrix as a chart, it prompted a real discussion on policy between the administrators. That's a big goal of any data-driven interface: help users see where there are interesting questions to ask, then help them answer those questions and make decisions.

A Thousand Different Reports. Just Three Clicks.

Powerful, Fast, and Flexible
One of the most iconic reports within Matrix is our lightning fast Drilldown report, which allows school districts to dissect their entire set of data in any way they can imagine using just a few clicks. Our Drilldown report processes huge amounts of information, tens of thousands of data points, in just seconds and groups the data by school, grade, and student sub-groups like gender, race, IEP, and ELL/LEP. Here is a list of typical questions which can be answered using the Drilldown report in just a few movements of the mouse.

• In which school did female Hispanic students score the lowest on the state test? Two clicks.
• Do my 3rd grade free and reduced lunch students score better on the AIMSweb test or Acuity? Two clicks.
• Between all of my elementary schools, which teacher has the highest number of special education students? And how did they perform? Three clicks.

Details are Important
All of the scores in the Drilldown report are color-coded to highlight areas where the district is doing well and areas where the district is struggling. District administrators love to check into the Drilldown whenever new test data is uploaded, so they can get an overall look at how their district performed on the latest assessments. If they see any red-flags they can then instantly dive into the data and determine which student subgroup, teachers, or even classroom course section may have caused a drop in scores. Remember, context is everything, so Matrix will always show the number of students which make up each group and allow you to drilldown to the individual student scores which make up an average.
The Drilldown encompasses all of our main goals with Matrix:

1. Bring all types of data together in powerful, but easy to use reports.
2. See top level data, but be able to get to the details in a couple clicks.
3. Give districts the ability to make decisions by centralizing key performance data.

AP Predictor: Using Data to Drive Decisions

The AP Predictor report in Matrix is designed to identify students who are likely to do well in AP courses, so staff can be sure those students know about their AP course options. The predictive element is taken from research published by The College Board showing a strong correlation between PLAN scores and scores on specific AP tests. They provide a list of AP tests and their associated PLAN subscores and cut scores. For example, according to The College Board, students whose PLAN Math and Science scores average at least 26 have a 75% chance of scoring a 3 or better on the AP Microeconomics test.

Of course you could do this math by hand or in a spreadsheet, but your PLAN scores are already in Matrix, so why not let Matrix do the work for you? The AP Predictor report knows which PLAN subscores go with each AP test, and automatically calculates the relevant average when you select an AP test from the menu. You can filter by student name and minimum AP success category (e.g. only students with a 50% chance of scoring 3+ on the selected AP test), and sort by any of the scores.

Coming up with ways to display historical data that make it easier to absorb is a cornerstone of Matrix, but it's always exciting to build an interface that directly helps educators make decisions about the future.

Everyone has data. Our goal is to have everyone use it.

You have the data. We've spent the last few weeks getting it all from your piles of spreadsheets and student information system into Matrix. Now what do you do?
Luckily, Matrix isn't just a data warehouse. Yes, Matrix stores all kinds of data ranging from Fountas and Pinnell to the ACT to AIMSweb and even attendance, behavior, and assignment grades, but the real power of Matrix is in the dashboarding and reporting tools which allow districts to use all of their data to make game-changing decisions for their schools, teachers, and students.

Using Key Performance Indicators

An Index Measure is a numeric value assigned to a student based on various key performance indicators (KPIs) like attendance percentage or test proficiency. This allows us to normalize different academic areas which could never have been viewed on the same scale before. We can then use these student Index Measures to determine school, grade, teacher, and district performance.

Using Index Measures for College Readiness
An increasingly important and popular Index Measure is college/career readiness. Because every district does college readiness slightly differently, the Index Measure can be customized to include any set of KPIs and performance levels. For this example we will look at a college readiness measure which has two performance levels: “college ready” and “not college ready.”
Just showing up is a huge part of college and career success, so the first thing we are going to include is a student's attendance percentage. Let's say if a student has 90% attendance or better we will give them a point. These points will come into play when we determine how college ready a student is.
Next let's look at classroom and assessment performance. In the classroom, we can expect a student to have earned a C or better in their Algebra class to be at the level they need for college mathematics. We also expect the student to have scored at least a 21 on the ACT exam. Now combining all of these KPIs we can come up with the college/career readiness percentage for a student and aggregate it for a teacher or school.

Making it Easy
Index Measures are adaptive and completely customizable. Add new tests or academic indicators to any Index Measure at any time with no manual calculations. Matrix makes it easy: just pick the students and key performance indicators and let the system do the rest. Spend your time using data not finding it.

Behavior Documentation and Tracking for K-12

Helping districts use data to take more proactive steps towards behavior incidents. 

To effectively analyze and improve a student’s performance, teachers often need access to data beyond a student’s test scores. One example of such data is behavior. As with other types of data analysis, the more detailed and more rich the data is, the better we are at noticing trends and making decisions using it.

The Matrix Behavior Documentation module facilitates this process by providing a quick and easy way for teachers to track behavior incidents at a detailed level.

It gives administrators a comprehensive view of behavior across the entire district, at a building level, and at an individual student level. The district can look at this data in a number of different ways including yearly data, monthly data, referral type (proactive or reactive measure), physical location of the incident, time of day the incident occurred, and more.

 Behavior data is then aggregated at a student level for analysis with the student’s attendance, assessment or test scores, and demographics to give teachers and administrators a comprehensive picture of student progress and performance. This also allows the administration to make policy decisions like having additional adults at lunch or bus time to reduce the number of behavior incidents.

Are your students college ready?

In secondary education, there’s perhaps no bigger buzzword than “college readiness” right now. School districts are under enormous pressure to produce graduates that are prepared for college-level course work or career-ready skills that will give them an effective transition into the work force. It’s a constant battle every year, particularly after a 2013 ACT survey revealed that only 26 percent of college instructors believed high school graduates were ready for post-secondary classes.

We've got the data...

We’ve spoken to a number of district administrators who have emphasized the importance of being able to use data to help address this challenge and to ensure their graduates are ready for college. The good news is most districts already have a plethora of data they are collecting – whether it’s state assessments, common assessments, EXPLORE, PLAN, ACT, PSAT, SAT, AP test data, demographics, behavior, attendance, etc.

But now what?

Well, it’s a matter of getting the data into a usable format for teachers and administrators to use on a daily basis to guide instruction and improve student performance.  In order to effectively use data, districts need to be able to break it down and analyze trends, student growth, instruction effectiveness, etc.

Get it into the hands of teachers....

Better access to data that is in a usable format takes the guesswork (teachers' perceptions or anecdotal evidence) out of analyzing student performance. Teachers can spend less time trying to make sense of data and more time using it to drive instruction...

And improve performance.

Getting Past the Eye-Candy Test with Elementary Assessment Data

You can call it the “eye-candy test” problem.

It may not be as pronounced in education as it is in baseball. However, in almost every conversation I have with school district administrators we work with, they are focused on this core challenge: How can we make sure our data – not through some anecdotal perception or observation -- supports our decisions about how to improve student performance?

Brad Pitt played Oakland Athletics general manager Billy Beane in "Moneyball" a story about how the baseball team began using data to try to level the playing field with more profitable teams in the league.

When the concept comes up, I picture the scene in the movie “Moneyball” that depicts Brad Pitt – who plays general manager Billy Beane -- getting fed up with how his old-school baseball scouts are trying to identify the best players so their cash-strapped major league team can compete with the richer teams like the New York Yankees.

“He passes the eye-candy test. He's got the looks. He's great at playing the part. He just needs to get some playing time,” one scout says, while another questions the same player’s confidence because of how good-looking his girlfriend is.

Those more anecdotal observations could be important as a piece of the puzzle, but if you focus on that, decision-making becomes too arbitrary, which is the point of the book and the movie. Statistics and data add so much more.
Read more »