Shield or sword? The role of AI in safety and security
By Ioanna Lykiardopoulou
Today, AI plays a critical role in strengthening our safety across diverse domains, from cyber defense to combating financial crime and protecting national security. Yet this same capability also exposes us to serious risks: AI can serve as both a powerful shield and a vicious sword, depending on who wields it.
This isn’t just about speculative, existential dangers. It’s about tangible, immediate threats stemming from misuse: generative AI tools enabling deepfakes, AI-driven social engineering scams, or autonomous systems deployed with unclear oversight.
This edition explores how AI is being used to reinforce security in cybersecurity, financial fraud detection, threat prediction, and military defense operations. It also looks inward at the field of AI safety: how experts are working to make these systems more robust, explainable, and aligned with human intent.
To better understand the current landscape in the Netherlands, we spoke with four leading experts working at the intersection of AI, safety, and security:
Stjepan Picek, Associate Professor of Digital Security at Radboud University
Iskander Hannivoort, Founder of Manukai, Investor, and Airline Captain at Transavia
Natasha Govender-Ropert, Head of AI – Financial Economic Crime at Rabobank
Captain van den Broek of the Netherlands Royal Army, and PhD researcher at the University of Twente
AI in cybersecurity: “a cat and mouse game”
AI is already integral to cybersecurity, from simpler tasks such as filtering spam to anomaly detection and defense automation. But the same technology is also being used to craft more personalized, scalable attacks — especially following the advent of Generative AI.
“We’re seeing a large amount of scam attempts using deepfakes for social engineering purposes,” says Stjepan Picek, Associate Professor of Digital Security at Radboud University. Attackers are also using GenAI tools to automate malware generation, enabling “more diverse attacks with less effort and on a massive scale.”
As to how cybersecurity systems can keep up, Picek says “it’s always been a cat and mouse game.”
“The main difference now is that the cycles are faster,” he adds. A new attack emerges, a defense follows, but another attack soon breaks through.
“With this incremental approach, it’s difficult to say that anything will change fundamentally,” Picek notes. Even if a perfectly secure system existed at the moment, it could still be vulnerable to future threats. “We, as defenders, constantly need to monitor, test, and improve.”
Security of AI and AI for security
Picek and his team at the university are working on two research areas: the security of AI and AI for security. “In the first case, we assess various kinds of machine learning algorithms, try to attack them, and by doing that, we try to propose new defenses that would strengthen them against attacks.”
In the AI for security field, they’re using machine learning and deep learning to perform different types of attacks (such as side-channel attacks), in order to test the security and resilience of various devices like smart or credit cards.
Accuracy trumps safety
One challenge in the cybersecurity space, according to Picek, is that safety by design in AI systems is quite difficult to achieve.
The first reason why is simple. “Machine learning was never designed a priori to be safe and secure. It was designed to be accurate,” he says. The second reason is that safety by design always involves a tradeoff. “Your algorithm will potentially become less accurate. You will spend more time and energy resources to train it. You will need more human expertise to make it.”
As widespread attacks are still relatively limited, he notes, many companies aren’t sufficiently concerned about potential risks. They’re often unwilling to invest in making their systems more secure, especially when this might reduce algorithmic performance.
As AI adoption surges, he warns that security is often an afterthought. “Everyone wants to deploy AI — but safety and security still aren’t high on the list of priorities.”
Smarter threat prediction
As both digital and physical security challenges grow more complex and intertwined, traditional tools are no longer enough. AI is shifting the paradigm — from reactive monitoring to proactive, real-time threat prediction.
Startups like Manukai are helping drive this shift. The company’s AI-based threat prediction tool — set to go live later this year — aims to alert business travelers about risks ranging from civil unrest to terrorism, before they occur.
“There are certain unique challenges in security at the moment that AI can help address,” Iskander Hannivoort, founder of Manukai, says. He explains that AI enables 24/7 monitoring, faster pattern recognition, and predictive insights — all critical in areas with geopolitical and operational security risks for businesses.
Real-time risk prediction
Manukai’s proprietary system, based on an adapted open-source model, pulls data from sources like news outlets and social media. It analyzes patterns to forecast where and when a threat may occur, using a time- and location-based logic. The goal is full automation, providing users with direct advice — from postponing a trip to navigating to a safe area — without human analysts in the loop.
To be effective, however, the system must be unbiased, Hannivoort explains, though he acknowledges that bias is hard to define. “We need to assess every bit of information before it goes into our prediction model. Is it true and verifiable? What’s the underlying narrative we can detect?”
A second layer of Manukai’s tool is personalization. “Your risk profile is dependent on who you are,” he says. “Our system will know which field you are in and for which groups or countries you or your company’s intellectual property is a high-value target.”
Challenges
A major challenge, Hannivort admits, is understanding how the AI arrives at its predictions, functioning like a black box. “You can only verify the conclusions and try to relatively make sure that they’re free of errors and hallucinations.”
There’s also the question of responsibility. Predictive tools risk both false alarms and missed threats — and with that comes liability. That’s why Manukai positions its system as a decision-support tool, not a replacement for human judgment.
Looking ahead, Hannivoort sees AI-based threat prediction playing a key role in long-term strategic planning — helping companies, governments, and societies identify faster existing and emerging threats.
Combating financial crime
In the complex realm of financial crime, AI acts as a double-edged sword. Natasha Govender-Ropert from Rabobank, shares how the team leverages AI and analytics to combat threats while navigating critical ethical considerations.
“Banks are highly regulated, which means that there is a significant number of rules and laws we must adhere to, next to having oversight agencies,” she says. “This also influences the type of AI algorithms we implement. We don’t use cutting-edge technology, unless we’re able to explain it.”
AI for anomaly detection
Fraud detection has traditionally relied on rule-based systems, which typically flag suspicious transactions based on certain rules and thresholds. For example, transfers over a certain amount of money over a short period of time.
“The issue with rule-based systems is that they give rise to a large number of false positives, which analysts have to manually review,” she adds. “So our biggest focus area is reducing the amount of false positives.”
For this reason, the fraud team is investigating the use of anomaly detection and real-time analytics for fraud detection. “We’re building traditional AI systems in combination with rule-based systems to create a more robust detection algorithm, so we can better understand if our customers’ behavior deviates from normal patterns,” she says.
Compared to rule-based systems with fixed logic, machine learning models operate in multi-dimensional feature spaces, enabling them to learn complex patterns and define more adaptive decision boundaries for fraud detection.
Generative AI: benefits and risks
Govender-Ropert sees GenAI as a valuable tool to improve day-to-day operations. It can significantly reduce manual, tedious work for fraud analysts by assisting with investigations, summarizations, and report writing. This allows human experts to focus on more complex, high-value tasks.
"For instance, we have a chatbot that guides analysts during investigations and retrieves relevant information from multiple databases, providing a clear and concise overview that saves time.”
But the advent of Generative AI has also introduced unprecedented challenges in combating financial crime, Govender-Ropert emphasizes. Fraudsters can now use GenAI to create hyper-realistic fake documents, videos, and even voices. Scams have also become more convincing, with phishing emails and messages that mimic real corporate communications almost perfectly.
Detection of deepfakes remains a developing area. Even experienced professionals may overlook subtle indicators, particularly when not actively assessing content for authenticity.
AI in defense: autonomy with human oversight
AI is emerging as the next frontier in modern warfare and defence. Earlier this year, State Secretary for Defence Gijs Tuinman announced his intention to invest €310mn in unmanned systems and innovative technologies such as AI.
AI can have several applications within the Dutch armed forces, according to Captain van den Broek of the Royal Netherlands Army. These applications include image recognition, terrain analysis, as well as planning and decision support for units.
Experimenting with autonomous robots
Captain van den Broek also leads the Dutch army’s development of military autonomy and AI. As part of his work in the army as well as his PhD at the University of Twente, he’s currently experimenting with autonomous robots — such as unmanned, remotely controlled vehicles. Such robots can be used for tasks including reconnaissance missions and cleaning high-danger zones like landmines and radioactive areas.
Controlled autonomy
The level of autonomy is entirely use-case dependent, Captain van den Broek says. “Is it just navigation or are we talking about swarming missions with multiple vehicles, or even drones all working together?”
He emphasizes that robots and AI can assist with data and a variety of tasks, but the use of lethal force must always remain a human decision. This principle guides the development of autonomous systems in the Dutch armed forces, where there is no place for self-learning weapons that decide in real-time who or what to target.
All model development and training must be completed and verified in advance. “During execution, you need a fully trained and validated model that has passed all necessary checks,” Captain van den Broek explains. This ensures what he calls "controlled autonomy:" systems that are capable but predictable — and always under human command.
Trust through transparency
Trust is central. “If we want to use autonomous robots,” he says, “commanders must trust that they will behave as expected.” Part of Captain van den Broek’s work involves not only refining the technology, but also showing unit commanders exactly how these systems work and what decisions they’re making in practice.
“These systems should support, not replace human decision-making,” he emphasizes.
What does “robust” AI really mean?
According to the experts we spoke with, the essence of AI robustness depends on the field and the use case for which a model is deployed.
From a cybersecurity perspective, model robustness ideally means resilience against attacks, Picek says. “Once you train your model in the presence of a specific attack, you make it more robust against similar attacks that might happen,” he explains. Achieving this kind of robustness depends not only on the model architecture and threat type, but also on who controls the training process — whether it’s handled internally or outsourced.
“In military language, AI or autonomy robustness means that the systems will always work across different environments, scenarios, and threats,” Captain van den Broek says. This includes high performance in various terrains, weather conditions, and different types of tasks such as traditional warfare or peacekeeping missions.
“Robustness starts with data quality and relevance," says Govender-Ropert. It’s also about checking a model’s thresholds, verifying its behavior through testing, and actively monitoring it after deployment — especially since models can drift over time as new data enters the system.
Building ethical and responsible AI
AI systems can reinforce bias, invade privacy, or make unsafe decisions, unless ethics are embedded from day one.
Responsible AI in practice
For Govender-Ropert, ethical AI refers to the broader societal and philosophical question of what is good or harmful, while responsible AI is about putting those values into practice. At Rabobank, this means building systems that are transparent, explainable, secure, and fair. These principles are embedded into the bank’s model development lifecycle.
But fairness isn't a universal standard and different organizations may interpret it differently, Govender-Ropert emphasizes. What matters is that teams make conscious decisions about what fairness means in their context and translate them into clear guidelines for development and testing.
“We need to understand that the decisions we make, how we build these models, and how we use them can impact people’s lives.”
Ethics start with intent, not compliance
Stjepan Picek believes that ethical and safe AI begins with intent, not just compliance with regulations such as the EU’s AI Act. “We, as humans, developers, owners, and users of models, need to acknowledge that AI security is an important goal,” he says.
Fairness is a critical element for Picek as well. “At the moment, we see more negative consequences from biased algorithms than from insecure algorithms.”
Guardrails without stifling innovation
Iskander Hannivoort agrees that safety and fairness are essential — but warns that over-regulation may hamper Europe’s competitiveness in the AI race. “We want to innovate, but we also need guardrails,” he says. “That balance is hard.”
Keeping humans in the loop
In the defense sector, Captain van den Broek stresses that responsible AI means incorporating human oversight at every level: from commanders who must trust the systems they deploy, to political and strategic leaders who decide when and how these technologies are used, and ultimately to the society they serve.
“Transparency, explainability, and human oversight: that’s what responsible autonomy looks like,” he says.
Ethical deployment, he argues, is about protecting people. Yet in our imperfect world, these same technologies can also be weaponized in violation of human rights. “This is a serious ethical debate,” Captain van den Broek says. “But not developing these technologies could leave us at a disadvantage against enemies who do.”
The Netherlands: positioned to lead or lagging behind?
The Netherlands has made visible progress in embedding ethical governance into AI development, especially in the banking sector, says Govender-Ropert, speaking from her personal experience in ING and Rabobank.
But challenges remain. She points to a lack of public trust and the need for AI education.
Trust isn’t the only challenge. Captain van den Broek sees ambition in the defense sector — with rising investment, collaborations, and experimentation — but acknowledges that the Netherlands is still in a catch-up phase. “Swarming autonomous tanks don’t exist yet,” he notes. “We know we need to make major steps in this field.”
Iskander Hannivoort is more blunt: “There’s a huge backlog.” He says Dutch organizations still underestimate security — not just in AI, but across the board. “The Netherlands is a safe country, and people aren’t used to violence or danger.”
Cybersecurity researcher Stjepan Picek adds that while the Netherlands is not lagging behind its neighboring countries, it isn’t leading either. “We’re in the middle. There’s good research on AI for security, but much less on the security of AI itself,” he says. Bridging the gap between academic research and real-world application is still an ongoing struggle.
Advice for AI professionals:
What should professionals working at the intersection of AI, safety, and security, keep in mind? Here are the key takeaways from our experts:
Stay educated: AI and cybersecurity evolve rapidly. Professionals must dedicate ongoing effort to stay informed about emerging threats, tools, and best practices.
Watch for emerging roles: As responsible AI matures, new professions will emerge. Roles like an AI ethicist — someone who embeds ethics into everyday model development — are expected to become standard parts of AI teams.
Know the field you’re securing: Those working in the security space need both domain-specific knowledge and real-world understanding of threats.
Make ethics part of the design: Ethical considerations must be integrated into the design and development process, not added as an afterthought. This includes selecting appropriate use cases, applying constraints, and ensuring alignment with broader societal values.
Conclusion:
AI is rapidly transforming the landscape of safety and security, from protecting financial systems and countering cyber threats to supporting national defense. But with greater capability comes greater complexity. The systems we rely on to protect us can also be used to harm, deceive, or destabilize.
They also pose ethical, legal, and societal dilemmas. What does fairness mean in financial AI-based systems? Who decides when and how to deploy autonomous systems in the military? What guardrails do we need to prevent misuse, and who is responsible when something goes wrong?
As our experts have made clear, there’s no one-size-fits-all solution. But there is a shared understanding: responsible AI is about intent, process, transparency, explainability, human oversight, and a commitment to ethics from design through deployment.
The present and future of AI in security isn’t just a technological question. It’s a human one — about how we build systems that align with our values, mitigate harm, and serve the societies they’re meant to protect. Ultimately, in a world where AI is both a shield and sword, how we wield it is what truly matters.
💡 Want to learn more about the KickstartAI community and our mission to accelerate AI adoption in the Netherlands? Check out our website, listen to our podcast, or follow us on LinkedIn!
