Photo source: Flickr
OpenAI is financing research focused on the development of algorithms that can anticipate human moral judgments. The funding was revealed in a recent IRS filing by OpenAI Inc., the nonprofit branch of OpenAI, which disclosed a grant awarded to researchers at Duke University for a project titled “Research AI Morality.”
An OpenAI representative highlighted that this funding is part of a broader, three-year commitment totaling USD$1 million to explore the concept of “moral AI.”
Details regarding this morality-focused research remain limited, with the grant expected to conclude in 2025.
Walter Sinnott-Armstrong, the principal investigator and a professor of practical ethics at Duke, along with co-investigator Jana Borg, has previously engaged in various studies and authored a book examining AI’s potential as a “moral GPS” to assist individuals in making improved judgments.
Their prior work includes developing a “morally-aligned” algorithm designed to assist in determining kidney transplant recipients and investigating scenarios where individuals might prefer AI to undertake moral decision-making tasks.
According to the press release, the objective of the OpenAI-supported research is to train algorithms capable of predicting human moral judgments in situations characterised by conflicts among significant moral factors across fields such as medicine, law, and business. However, the complexity inherent in morality raises questions about whether current technology can adequately address such nuanced concepts.
The challenges are underscored by past efforts in this area. For instance, in 2021, the Allen Institute for AI developed a tool named Ask Delphi, intended to provide ethically sound guidance.
While it performed reasonably well on straightforward moral dilemmas—recognising that cheating on an exam is wrong—it could be easily manipulated. Minor changes in question phrasing led Delphi to endorse various unethical actions, including harmful behaviours like smothering infants.
This limitation stems from the operational nature of modern AI systems. Machine learning models function as statistical entities trained on extensive datasets sourced from the internet. They identify patterns within this data to generate predictions; for example, they learn that “to whom” often precedes “it may concern.” However, AI lacks true comprehension of ethical principles and does not grasp the reasoning or emotions involved in moral decision-making.
Consequently, AI tends to reflect the values prevalent in Western, educated, industrialised societies, as these perspectives dominate the online content that informs AI training data.
It is not surprising that AI-generated responses may not align with the values of many individuals, particularly those who do not contribute to the digital environment.
Furthermore, AI can internalise biases that extend beyond Western ideologies; for instance, Delphi indicated that heterosexuality is more “morally acceptable” than homosexuality.
The task facing OpenAI and its supported researchers is complicated by morality’s inherently subjective nature. Philosophical discussions concerning various ethical frameworks have persisted for millennia without yielding a universally applicable theory.