The Nebraska Department of Education (NDE) has been a pioneer in the open-source movement around modern predictive student outcome platforms, in pursuit of its strategic mission to “lead and support the preparation of all Nebraskans for learning, earning, and living.” The NDE is using Microsoft Azure to modernize its data platforms, leveraging tools such as Synapse Analytics to better provide actionable insights, while transforming its Statewide Longitudinal Data System (SLDS) to promote equitable outcomes for all students.

Schools in Nebraska have among the highest average graduation rates in the US, but the state’s Department of Education recognized that disparities existed across its K–12 system. “The last few years shined a light on inequities, whether it’s pandemic-related or in the civil protest movement. These experiences have fueled our passion for the equity agenda and data is critical for us to identify, measure, and prioritize the work we need to carry out,” says Dr. Dean Folkers, Chief Information Officer at the NDE [currently Director, Data and Technology, Council of Chief State School Officers (CCSSO)].

The NDE is accountable for implementing, monitoring, and evaluating education initiatives funded at both the state and federal levels. In fulfilling this responsibility, the agency has been a pioneer in adopting the Open Education Analytics architecture to improve student outcomes, and that’s fueled a desire to further modernize its data platforms. “We have to be more strategic in using data to inform best practices, policy approaches, and different types of interventions,” says Folkers, “to help all kids gain access to a quality education.”

“Not everyone needs the same support,” he adds. “If we give equally to districts that already have resources, we’re missing an opportunity to support kids in a historically marginalized situation or in districts that don’t have capacity to do it on their own.”

Building a data plan from the ground up

Schools across Nebraska have long been collecting copious data, such as grades, attendance, course-taking patterns, connections to teachers, and demographic info. “We found that our districts were spending about 655,000 staff hours per year sending us data—and they got little in return,” Folkers says. “We were consolidating, aggregating, and providing reports. We were also spending millions of dollars each year in staff time and licenses on software systems.”

The NDE ran a comprehensive survey of data collection and usage and, with this insight, developed a Statewide Longitudinal Data System (SLDS) plan. The goals of the plan were to improve how data was collected, process it more efficiently, and give it back to state, district, and school leaders, administrators, and educators in an actionable format. In 2020, the NDE launched a root-and-branch overhaul of the way it collects, refines, and disseminates information regarding its 350,000 students.

“We created a 10-year plan to establish priorities and we started down the path, leveraging different opportunities that came our way,” says Folkers. “One was to install an API in our collections process, letting us automate data collection directly from student information systems.”

Efficiency and efficacy of the system was one motivation. Another was ensuring state resources targeted those with the most need. The research revealed uneven access to digital services, with the smallest public school districts lacking capacity to use data effectively. Around 93 percent of Nebraska’s 244 school districts have fewer than 1,000 students, and many of those districts have not been able to fully access data for decision-making.

Making impactful decisions based on reliable data

The NDE had previously managed and supported multiple, disparate data sets across many different locations, including through hosting relationships with various state IT offices. This data often didn’t have documentation or a data dictionary associated with it, and sometimes even lacked security. The first phase of the SLDS plan was to move data to the cloud to begin addressing these issues, and also to enable data to flow in near-real time from hundreds of sources.

This meant automating data flows and ensuring secure interoperability between governmental agencies at all levels, as well as private sector organizations. It was a huge transformation, one that had to be performed in tandem with everyday operations.

“There is a growing need for interoperability with external systems and how we connect with them to send and receive data from our partners to expand our horizons,” says David Hefley, Administrator of the Office of Data Management and Application Development at the NDE. [currently Acting ISO]. “Decisions should be based not solely on data we collect, but also that which other entities hold, such as the Department of Labor and the University of Nebraska system.”

Working with a specialized team from Microsoft Consulting Services, the agency set out to create a deep and complex data lake on Microsoft’s Azure platform, to provide access to data in a more simplified, actionable format—thus giving individuals, schools, and districts the power to identify who needs what information and when. The NDE also chose Azure to help address technical debt, bringing more simplicity and scalability to what had become increasingly fragmented and manual processes.

“We felt like we had a pretty decent handle on the student data,” Folkers says. “What led to our deepened relationship with Microsoft and the transformation in data analytics—and Open Education Analytics—was an effort to begin to bring in other datasets.” For example, the NDE wanted to look at data associated with nutrition services, after-school programs, and budgets and expenditures, including per-pupil costs at the school level, not just the district level. Looking through this wider lens would allow more opportunities to introduce AI and machine learning, potentially even predictive analytics for timely interventions.

“As we were adopting the Azure cloud-based resources, one of the challenges was understanding the depth of offering that comes with the platform,” says Hefley. “Working with Microsoft team members has been fantastic, including the level of assistance in describing the opportunities, because the Azure ecosphere is deep and there are multiple ways to complete a task. Having a guide through the process is very helpful.”

“We are coalescing around Azure resources as we build the data lake, using Azure Data Factory, Synapse Analytics, App Service, and networking components,” Hefley adds. These tools have enabled them to “get more and more data and perform more complex analysis,” says Folkers. “[The Azure tools have] enabled the opportunity for us to do this predictive work and experiment with machine learning models adding value …. We’ve jumped into the whole Synapse platform in our Azure experience.”

We have also embraced DevOps from a project management and product life cycle point of view, optimizing the workflow of getting code where it needs to go. That really came into play at the start of the pandemic. We needed to find a cloud-based distributed management system that allowed us to do things efficiently online. It was great to move into an internet-based DevOps world.

David Hefley

Administrator of the Office of Data Management and Application Development, Nebraska Department of Education.

Simultaneous with the migration to Azure, the NDE has been enhancing its internal digital muscle, introducing department teams to Microsoft tools including Power BI, Dynamics, Microsoft 365, Teams, OneDrive and SharePoint to better support collaboration and remote work.

Not only did we go 100 percent remote in about six days, but during that … our offices were moved to a new location. Microsoft 365 was critical around the pandemic, with our complete remote environment.

Dr. Dean Folkers

Director, Data and Technology, Council of Chief State School Officers (CCSSO); Former Chief Information Officer, Nebraska Department of Education.

“We want to get systems and processes in place that help us make the most use of our hybrid environment moving forward,” says Folkers. “The collaborative tools—using Teams, our transition to OneDrive and SharePoint across the board—is all part of our implementation.”

This hybrid, partially remote work model is also offering new opportunities to support collaboration within the agency and across all stakeholders. “Things like Dynamics 365 and Power Apps, [we’re learning] about what we can do with those … and they’re pretty high on the radar to investigate further,” says Hefley.

Turning eyes toward digital transformation in the future

With the first phase of the SLDS complete and the data lake in place, focus has shifted to implementation, partnerships, and building new resources for educators while continuing to update the NDE’s internal systems.

For teachers, support staff, and administrators this will mean access to clear, concise data through Power BI analytics. The next step is to focus on education transformation. The NDE will introduce machine learning and predictive AI to create new opportunities for diagnostic and predictive insights, with recommendations that empower personalized learning and a whole-student approach for delivering education.

The NDE will continue to look to Microsoft for support and guidance throughout their digital transformation. “We legitimately consider Microsoft a partner,” says Folkers. “Ultimately, the Microsoft team has nothing but the best intentions for our project.”

Hefley adds, “The process isn’t easy, but that’s because of the inherent need to change our mindsets about how we do development work, handle data, and the opportunities that are available. There are a lot of new concepts that are different from our old SQL-based setup. Today, we can be agile in our environment infrastructure and try things out …. Every time we think we’ve had all of our ‘ah-ha!’ moments around Azure, we find something else. We’re like, ‘Oh, man, here’s another fantastic opportunity that we can re-architect this and reduce that technical debt yet again.’”

By resetting its data strategy and investigating the real-world opportunity presented by Azure, the NDE is pioneering a new approach to student outcomes, one in which all learners regardless of their background, will be supported to reach their full potential.

This post was originally published on the Microsoft Customer Stories website.