K-12 Challenges Related to Data Mining and Reporting

It is no secret that school districts are facing challenges from every which direction these days. It is also no secret that before a problem can be solved, one must know exactly what the problem is. So in this blog I want to focus on what exactly the problems are that school districts are having with data mining and reporting. These are important areas to focus on because schools look at data in order to make informed decisions on how to best educate their students. If school districts are having trouble accessing and understanding data, then how can they begin to improve their teaching methods and curricula?

The problems that most school districts are struggling with can be classified into three categories: Data Aggregation, Usefulness, and Resources.

Data Aggregation

No matter the size of the school district, data that schools collect can be overwhelming, especially if the data isn’t uniform in nature. Being able to see aggregated data allows the user to catch possible errors and better understand how the figure was produced.

• It is very difficult for school districts to aggregate data from multiple sources, including state assessments, MAP, EOC, Acuity, AIMSweb, ACT, Plan, Explore, and more.

• Data that is derived from multiple sources comes in different formats and structures, which means it can take hours to get the data from just one assessment into a format that is actually usable.

• Once data is aggregated, it is often hard to read and understand. Districts generally have to have a data administrator who can make sense of it all.

• Data is often not clean.

• If it is clean, administrators and teachers will often not even get to see it themselves. Generally reports only show what the final calculations have produced – perhaps a score, an average, or a graph – not the data that was used in the calculation.

Usefulness

Data only has value if it can be turned into information and knowledge, then action.

• Typically, districts end up with either very simplistic reports that don't meet their needs or the reports are so complex that it requires their staff to commit to months of training, and the reports still never get used.

• In a lot of cases districts end up with just one year’s worth of data that is tied to only a student. There is no correlation of that student’s score to the teacher, class, other students in the class, other assessments the student took, previous year’s assessment scores, demographics, grade level, etc. Looking at data in the context of several other related data sets can significantly affect the outcome of a decision.

• It is difficult to relate performance back to the state learning standards. How do districts know which standards need more attention when the resulting reports do not make clear which questions were associated to which standards?

• Data is used to determine teacher and student performance levels, but districts often only have the resources to take into account state-wide assessment data. The best way to indicate a student or teacher’s performance level is holistically and to do that many other data sets would need to come into play.

• Even when data and reports provide value, it is difficult to effectively share that information with each faculty member so that they know on a regular basis what their students’ performance levels are and where they need to improve.

Resources

Implementing a system that creates a greater strain in other areas may not provide the greatest value in the end.

• For districts that are able to make multiple correlations between data sets (i.e. student’s current score to their last test score, or the class’s average, gender’s average), it is an EXTREMELY time consuming process and requires someone that has a very good understanding of the data structure in their SIS as well as the technical ability to write queries to retrieve such data.

• Training staff and faculty to be able to use, read, and understand the resulting reports can take a long time and require constant support.

• Enterprise level systems typically come with an astronomical cost - making it very cost prohibitive, especially for smaller school districts, to take advantage.

• Deciding to integrate a data mining system into a school district can require a huge investment from everyone involved.

These challenges often result in district personnel spending a lot of internal resources (personnel, time, money) to just make sense of the basic data and then to try to use that data to build meaningful reports. Correlating this data back to the classroom and curriculum is almost impossible.

Yet, that is what is needed for the advancement of students.

And isn’t that the overall goal of our education system?