AI in education: how the technology is tested in Dutch schools
By Ioanna Lykiardopoulou
AI is no longer just an abstract concept in education — it’s being tested in classrooms through concrete projects. In the Netherlands, the National Education Lab AI (NOLAI) brings together researchers, schools, and teachers to explore how AI can meaningfully support learning and teaching.
At the center of this effort are three projects focusing on secondary education: tools that help students with self-regulation while writing, semi-automated scoring systems, and technologies that track student progress and recommend learning tasks.
Each of these projects addresses a pressing challenge in schools today, from reducing teacher workload to helping students take ownership of their learning. Together, they offer a window into what AI can — and cannot or should not — do for education.
We spoke with three experts involved in these projects:
Joep van der Graaf, Assistant Professor at NOLAI & the Behavioural Science Institute (BSI) at Radboud University
Eva de Schipper, Educational Researcher at CitoLab
Johan Jeuring, Professor of Software Technology of Learning and Teaching at Utrecht University & Chief Scientific Officer at DialogueTrainer
Self-regulation while writing
Launched in 2023, the project aims to improve students’ writing skills by developing a solution that supports self-regulation — the ability to monitor and direct one’s own learning process.
The project focuses on essay writing for six courses (geography, biology, history, Dutch, and English) in collaboration with teachers from Candea College in Duiven.
“We target self-regulated learning and we hope to see effects in the writing product,” says Joep van der Graaf, Assistant Professor at NOLAI, Radboud University, and scientific lead. To do this, the team developed a student dashboard which combines two main tools: an algorithm to track students’ learning process and NLP models that evaluate the essays.
Participating students work in a custom learning environment and complete a series of five lessons, two of which include essay writing followed by reflection.
From raw data to learning patterns:
A key component of the project is the Flora algorithm, which translates raw trace data (mouse clicks, keystrokes) into meaningful actions and learning patterns.
“If you click on a button to go to the instruction page, this is recorded as a simple click. The action label would be ‘going to the instruction page.’ Based on the action labels, we determine patterns of actions,” van der Graaf says. These patterns correspond to different learning processes, such as orientation and planning.
The dashboard shows students their actions as interpreted by the algorithm, which they can then use to reflect on their work.
"We started with a theoretical framework from the educational sciences that describes how the learning process happens in self-regulated learning,” van der Graaf says. The team then combined think-aloud and trace data and applied the Detect, Diagnose, Act framework.
While there were available coding schemes based on self-regulated learning theory to label think-aloud data, no such scheme for trace data existed at the time. “We developed it ourselves, ensuring it closely resembled the think-aloud scheme. We wanted to capture the same processes so we could also evaluate the labeling algorithm’s performance. And then we ran a comparison.”
“We identified some gaps based on the data, but we also saw some additional patterns that could be related to a process. We fine-tuned the algorithm and did another test, achieving a match rate of about 55%.”
NLP for essay evaluation
Van der Graaf explains that the writing evaluation feature was added later on, following feedback that students weren’t engaged enough in reflecting about the learning process alone.
The NLP models evaluate the content of the essays rather than the language and style. “We use three criteria provided to us by the teachers,” he says. “One of them is capturing the main points. We currently use NLP for history and biology.”
“Students can also see on the dashboard’s timeline at which part of the learning process (orientation, reading, or writing for instance) they covered one of the main points.” This way, they can understand how their entire writing process connects to their writing product.
Van der Graaf provides an example: “A student might reflect, ‘I didn’t plan enough and missed most of the main points — I should try a different approach next time.’ From there, they can set goals for the next essay, such as planning more reading time. If they follow through, it’s possible their essay will be better the next time.”
The project is currently in the validation phase, and the team is working on a prototype teacher dashboard.
Semi-automated grading support
On the teacher support front, CitoLab and NOLAI are working together on CheckMate: a digital grading environment that helps grading short, open-ended answers to test questions.
“We started researching the topic and built a prototype already in 2022,” says Eva de Schipper, Educational Researcher at Cito, who’s also leading the CheckMate project. In 2024, CitoLab joined a NOLAI co-creation project aiming to further develop the prototype with teachers from two schools.
The goal of the project is to address two main grading bottlenecks identified by teachers: the time-consuming nature of scoring and consistency in terms of fairness and accuracy.
De Schipper explains how CheckMate works: students see a question on the screen and type their answers into a text box. Once all answers are submitted, the system reviews them, groups similar responses, highlights key words, and suggests scores using different colors.
These suggestions show how other teachers evaluated similar answers, while the system also learns from a teacher’s own grading and adapts its recommendations accordingly.
“The teachers are still in charge of the final score and they don’t need to accept the suggestions ,” she says.
“We’re focusing on biology and geography courses for now, because the answers are shorter and there aren’t too many numericals,” de Schipper adds.
The team is currently working further on the clustering algorithm that groups student answers to also perform well in longer responses. “We feel it contributes the most to easing the burden of teachers, and we have received unanimously positive feedback.”
NLP processing
While parts of how CheckMate operates might change, the prototype is currently using NLP to process answers. “We first do pre-processing for the student answers to keep the parts of the responses that are relevant for scoring content quality,” de Schipper explains.
“Imagine if a student wrote down ‘the forest’. ‘Forest’ is a key component of the answer, but ‘the’ isn’t relevant. Similarly, a spelling mistake isn’t a problem when measuring their knowledge of biology.”
The team applies the same pre-processing technique for the one, or more, right answers in the scoring model.
"We then use vectors for the words in the answers, taken from an open-source language model. From these, we create an average representation of the meaning of a student’s answer and compare it to both the right answers and the answers of other students that have already been scored.”
This way, the system combines both reference-based and peer-based scoring.
According to de Schipper, one of the reasons behind choosing NLP is transparency. “We can precisely tell to users why we gave the score we did,” she says, adding that isn’t the case with generative AI.
Another reason is language. While generative AI models have strong scoring performance on large-scale English datasets, their performance in Dutch isn’t yet at the same level. A third reason is GenAI’s environmental impact.
Accuracy and automation
According to de Schipper, CheckMate is able to score a large number of student answers with high certainty. “For biology in vocational secondary education, our scoring agreement with the first grader is around 95%,” she says. But she also notes this accuracy is a somewhat elusive concept.
“Accuracy is only defined in comparison to what teachers do, and what we know from the data is that teachers don't always agree with each other even
on the same precise student answers.”
“What is the golden standard? That's one of the biggest questions for us,” she says. Another big question is to what extent scoring should be automated. De Schipper says that teacher feedback is so far mixed and ranges between semi- and full automation.
The project just completed the concept phase. The team will now start working on a renewed version of the prototype, incorporating teacher feedback. De Schipper hopes that, once the project finishes in 2027, the lessons from CheckMate can have widespread adoption.
Progress tracking and task recommendations for courses
Teacher shortages are another area where AI is helping provide (at least temporary) solutions. One such example is Co-Teach Informatica, a national program launched four years ago to address the lack of computer science teachers in secondary education.
Through the Co-Teach platform, students can still take computer science classes using fully online teaching materials, with class management support from teachers of other subjects, and learning support at a distance from universities. As a follow-up, a complementary NOLAI project addresses the challenge of setting appropriate learning goals and recommending suitable learning tasks without a certified computer teacher in the classroom.
“We’re focusing on tracking student progress and recommending tasks within a course,” explains Johan Jeuring — Professor of Software Technology for Learning and Teaching at Utrecht University — who’s also leading the research within the project.
The aim is to identify potential knowledge gaps before the end-of-chapter course exams, and remedy them by suggesting additional tasks.
At the core of the system lies an individual student model, which tracks a learner’s progress on knowledge and skills. “To track student progress, we annotated all tasks within a course with the learning goals they are addressing, such as the correct usage of the if-then-else statement,” Jeuring says.
As students complete tasks, their performance is recorded: some tasks are automatically assessed, while others involve self-assessment against a model solution. The results update the student model, allowing the system to monitor and analyze progress toward the set learning goals, and recommend suitable learning materials.
Bayesian knowledge tracing
The team developed its solution using bayesian knowledge tracing — an established method for intelligent tutoring systems, which models a student’s knowledge using performance data and probabilities.
“We calculate the probability that a student knows how to solve a particular task,” Jeuring explains. “If a student has a good result, the probability that they have mastered the subject increases. If they make an error, the probability decreases.”
They system updates accordingly a student’s knowledge network. Jeuring provides an example: suppose a student writes a correct if–then–else statement. Since this also requires the use of Boolean conditions, the system increases not only the probability of mastering if–then–else statements but also Booleans.
One remaining challenge is determining how effective the recommendations are, which the team plans to do in the final year of the project as it enters the validation phase.
“It is a non-trivial problem, because it’s not always guaranteed that the algorithm’s recommendations align with what students need or expect. What we do know is that if a student fails certain tasks, the system will recommend related tasks.”
“The harder question is if practicing those tasks helps the student improve and understand the material better. This is something we still need to look at.”
According to Jeuring, their solution is a platform technology. “There's nothing that makes it specific for computer science and actually the people who developed the Co-Teach platform are now also using it for some mathematics courses.”
The team is currently piloting its system in two schools, and Jeuring hopes that they will soon be able to roll out their results to more schools and courses.
Building AI in education: opportunities and risks
Beyond the benefits that personalization and adaptive learning can bring, Joep van der Graaf also sees AI as a way to create new kinds of educational situations that were previously impossible. “With our technology, for example, we can now track the learning process of 30 students at a time.”
Eva de Schipper emphasizes the potential of AI to support teachers more directly. She notes that teachers often need a more complete and up-to-date picture of their students — not only in individual subjects but also across subjects and on a more personal level.
For Johan Jeuring, the key opportunity of AI in education is its ability to collect student knowledge and use this information to provide teachers with insights into student needs for further progress.
However, he cautions that we should always be aware of data unfairness and bias. “Each time you learn something from data, you learn it from data from a particular group of students or a particular level and as soon as you use it for another group, it might not be representative any more.”
He also expresses concerns about the unrestricted use of generative AI by students, which might negatively affect the learning process by removing the difficulty one should experience to learn.
Van der Graaf shares a similar perspective. Citing a colleague, he refers to the risk of “metacognitive laziness.” When students rely on generative AI for good results, they stop thinking through tasks themselves and lose sight of what they are learning. The focus narrows to execution, leaving them unsure how to approach similar tasks on their own.
“As educational scientists, we should also keep in mind these questions: what do we want the future generation to learn? And what do we think is relevant?” he adds. The answers will define which learning goals — and at which stage — should remain independent of AI.
For Eva de Schipper, adopting AI in education calls for caution.“The introduction of automated scoring tools or scoring assistance into the educational environment already has so much potential to make an impact. But we don’t know what that impact will be and how it will change the didactics of school, or how it will be for students, parents, and all relevant stakeholders.”
💡 Want to learn more about the KickstartAI community and our mission to accelerate AI adoption in the Netherlands? Check out our website, listen to our podcast, or follow us on LinkedIn!
