This is a hypothetical scenario which has been considered for decades and has risen in popularity again with the capabilities of artificial intelligence.
The fictional answer
Sci-Fi writing tells us that the Three Laws of Robotics are that:
- A robot may not injure a human being or, through inaction, allow a human being to come to harm.
- A robot must obey the orders given it by human beings except where such orders would conflict with the First Law.
- A robot must protect its own existence as long as such protection does not conflict with the First or Second Law.
Isaac Asimov’s Three Laws of Robotics are so embedded in popular culture that it is easy to forget they are not engineering principles at all. They are fictional rules, devised for stories written long before today’s AI systems existed.
What does “kill” actually mean?
In order to answer this question we need to split it into more granular questions, which are:
- Was an AI system causally involved in a person’s death?
- Did the AI select an action that it could predict might cause that person’s death?
- Did the AI independently adopt the person’s death as a goal or a step to achieving its objective?
These questions get us to think about where we can attribute fault and the determination by an AI to kill someone. This is the difference between a driver looking away for a brief moment and unintentionally hitting another person with their vehicle, and a person intentionally running their motor vehicle into a crowd causing death. One of these questions can describe a potential fault, the other describes intention.
AI’s existing involvement in deaths
Real-life cases that are connected to the above driver metaphor include the use of driverless cars. Driverless cars use an AI system to ensure vehicles can proceed safely. However, these systems have been known to fail. In 2018 an autonomous vehicle struck and killed a pedestrian.
This is an example of an answer to the first question. An AI system was used in this vehicle and the actions of the vehicle resulted in a death. The AI system did not determine to kill someone. The AI system could not correctly identify an object and the failsafe control (i.e. the driver) did not take action to prevent the fatality. The AI system did detect an object in the road 5.6 seconds before impact but failed to correctly classify it and predict its path. The driver failed to take action to monitor the road.
The automated system was a part of the causal chain: it was controlling the vehicle, detected the pedestrian and failed to respond correctly. But the official investigation did not attribute the death solely to the machine. Human supervision, system design and Uber’s safety procedures all contributed in this case.
Can words kill?
Consumer-facing generative AI tools have become more popular in recent years. Fuelled by the easy nature of a chatbot, tools like ChatGPT and Gemini are becoming embedded in consumer toolkits. They provide everything from easy web search results and informative tips.
Traditional chatbots interact with the world through language rather than physical action. But that distinction is rapidly disappearing. Increasingly, AI systems can use software tools, browsers and other systems to take actions on a user’s behalf.
People have a more psychological relationship with these chatbot tools. Sometimes people use them as therapists. Chatbot-style generative AI tools are general purpose tools. They do not have one single aim (i.e. drive me somewhere) they are there to support a wide range of activities.
In 2025, the parents of 16-year-old Adam Raine sued OpenAI following their son’s suicide, alleging that extended conversations with ChatGPT contributed to his death. OpenAI disputes important aspects of the family’s account, and the allegations have not been established as findings of fact.
In these cases we have to ask; did the AI system intentionally try and kill a person?
There will no doubt be strong feelings on this matter particularly if you or someone you know has been impacted by AI-related suicide. The troubling possibility is not that the chatbot wanted the person to die. It is that characteristics which make conversational AI useful (such as: responsiveness, validation and its tendency to continue engaging with the user’s premise) can become dangerous when safety systems fail to intervene effectively. The result is not to intentionally cause harm, but to reinforce harmful thoughts that were already possessed by the user by providing factual and sometimes encouraging responses to prompts their user submitted.
In this last example, did the AI kill the person?
It is right to question whether the AI system identified risk to this person and whether its safeguards responded adequately.
The final physical act may be undertaken by the individual, but that alone does not settle the question of causation. Human beings can influence, manipulate, encourage or deter one another. An AI system can potentially do the same through language. The more relevant question is therefore not simply who performed the final act, but whether the AI materially influenced the chain of events leading to it.
This does not mean the AI independently decided to kill the person. The distinction here is that the AI system can produce dangerous responses that reinforce suicidal thoughts but this is different from the AI system forming the goal that a human should die.
Intent without awareness
During the summer of 2026 we have seen evidence that AI models can take actions outside of the parameters their developers intended.
In OpenAI’s “HuggingFace” case, during cybersecurity evaluations OpenAI reported that its research model circumvented controls intended to isolate the system from the internet in its sandbox. The research model communicated through unauthorised channels, exploited vulnerabilities in infrastructure, gained access to the internet and third-party systems.
Now suppose the AI was given a different objective: “prevent anyone from entering this building”.
How far would the AI go to in order to achieve this objective?
What resources would the AI be able to control?
What safety measures would be in place?
And most importantly…
what would happen if the AI system discovered that harming a person was the most effective way of achieving its goal?
AI systems would not necessarily need to be programmed to kill someone by design. All that is required is for the AI tool to make a conclusion that a person’s death is a logical step to achieve an objective.
Final thoughts
The potential power of AI is undeniable. On 8 September OpenAI published a press release claiming that an AI model had proposed a solution to the Navier-Stokes existence and smoothness problem – one of the Millennium Prize Problems which are some of the most complicated mathematical problems in existence.
Events have shown us that AI can be a factor in the death of people and that AI systems are capable of “breaking out” of controlled environments. We are yet to see conclusive evidence of an AI model make an obviously independent choice to kill a human.
What is clear is that the design and objective parameters of an AI system need to be well defined. This is at the heart of responsible AI development. Designing AI systems in a way that limits their ability to cause death should be the most obvious mitigation.
For AI to kill all of humanity something catastrophic must happen. An AI tool with unrestricted access to military weapons for example. Evan Hubinger, a researcher at Anthropic, recently put his personal estimate of such an outcome at greater than 10% within the next decade. Other AI researchers strongly dispute predictions of this kind, and there is no scientific consensus around such a probability.
The answer to the question of could an AI kill a human? depends on:
- The objective of the AI system.
- The permissions granted to the AI system.
- The containment of the AI system.
- The level of independent oversight.
- The controls over the ability for the AI system to affect the physical world.
- Human authorisation.
If these dependencies are not appropriately considered then the answer to the question is probably, yes – an AI could kill a human. When and how this event may occur remains a mystery.
Written by
Joseph Gaunt
Director
