Machine Ethics

A AI Robotics Ethics Society (AIRES) é uma organização sem fins lucrativos fundada em 2018 por Aaron Hui, com o objetivo de se promover a conscientização e a importância da implementação e regulamentação ética da AI. A AIRES é hoje uma organização com capítulos em universidade como UCLA (Los Angeles), USC (University of Southern California), Caltech (California Institute of Technology), Stanford University, Cornell University, Brown University e a Pontifícia Universidade Católica do Rio Grande do Sul (Brasil). AIRES na PUCRS é (...) o primeiro capítulo internacional da AIRES, e como tal, estamos empenhados em promover e aprimorar a Missão AIRES. Nossa missão é focar na educação dos líderes da IA de amanhã em princípios éticos, para assegurar que a IA seja criada de forma ética e responsável. Como ainda existem poucas propostas de como devemos implementar princípios éticos e diretrizes normativas na prática do desenvolvimento de sistemas de IA, o objetivo deste trabalho é tentar diminuir esta lacuna entre o discurso e a práxis. Entre princípios abstratos e implementação técnica. Neste trabalho, buscamos introduzir o leitor ao tema da Ética e Segurança da IA. Ao mesmo tempo, apresentamos uma série de ferramentas para auxiliar desenvolvedores de sistemas inteligentes a desenvolver “bons” modelos. Este trabalho é um guia em desenvolvimento publicado em inglês e português. Contribuições e sugestões são bem vindas. (shrink)

As artificial intelligence (AI) systems increasingly permeate processes of cultural and epistemic production, there are growing concerns about how their outputs may confine individuals and groups to static or restricted narratives about who or what they could be. In this paper, we advance the discourse surrounding these concerns by making three contributions. First, we introduce the concept of aspirational affordance to describe how culturally shared interpretive resources can shape individual cognition, and in particular exercises practical imagination. We show how this (...) concept can ground productive evaluations of the risks of AI-enabled representations and narratives. Second, we provide three reasons for scrutinizing of AI's influence on aspirational affordances: AI's influence is potentially more potent, but less public than traditional sources; AI's influence is not simply incremental, but ecological, transforming the entire landscape of cultural and epistemic practices that traditionally shaped aspirational affordances; and AI's influence is highly concentrated, with a few corporate-controlled systems mediating a growing portion of aspirational possibilities. Third, to advance such a scrutiny, we introduce the concept of aspirational harm, which, in the context of AI systems, arises when AI-enabled aspirational affordances distort or diminish available interpretive resources in ways that undermine individuals' ability to imagine relevant practical possibilities and alternative futures. Through three case studies, we illustrate how aspirational harms extend the existing discourse on AI-inflicted harms beyond representational and allocative harms, warranting separate attention. Through these conceptual resources and analyses, this paper advances understanding of the psychological and societal stakes of AI's role in shaping individual and collective aspirations. (shrink)

This essay seeks to understand what it means for the human collective when AI technologies have become a predominant force in each of our lives through identifying three moral dilemmas (i.e., tragedy of the commons, tragedy of commonsense morality, tragedy of apathy) that shape human choices. In the first part, we articulate AI-driven versions of the three moral dilemmas. Then, in the second part, drawing from evolutionary psychology, existentialism, and East Asian philosophies, we argue that a deep appreciation of three (...) shades of human finitude can help us mitigate harmful effects of the AI-driven versions of the three tragedies and even transcend them. (shrink)

The potential capacity for robots to deceive has received considerable attention recently. Many papers focus on the technical possibility for a robot to engage in deception for beneficial purposes (e.g. in education or health). In this short experimental paper, I focus on a more paradigmatic case: Robot lying (lying being the textbook example of deception) for nonbeneficial purposes as judged from the human point of view. More precisely, I present an empirical experiment with 399 participants which explores the following three (...) questions: (i) Are ordinary people willing to ascribe intentions to deceive to artificial agents? (ii) Are they as willing to judge a robot lie as a lie as they would be when human agents engage in verbal deception? (iii) Do they blame a lying artificial agent to the same extent as a lying human agent? The response to all three questions is a resounding yes. This, I argue, implies that robot deception and its normative consequences deserve considerably more attention than it presently attracts. (shrink)

The prevailing discourse around AI ethics lacks the language and formalism necessary to capture the diverse ethical concerns that emerge when AI systems interact with individuals. Drawing on Sen and Nussbaum's capability approach, we present a framework formalizing a network of ethical concepts and entitlements necessary for AI systems to confer meaningful benefit or assistance to stakeholders. Such systems enhance stakeholders' ability to advance their life plans and well-being while upholding their fundamental rights. We characterize two necessary conditions for morally (...) permissible interactions between AI systems and those impacted by their functioning, and two sufficient conditions for realizing the ideal of meaningful benefit. We then contrast this ideal with several salient failure modes, namely, forms of social interactions that constitute unjustified paternalism, coercion, deception, exploitation and domination. The proliferation of incidents involving AI in high-stakes domains underscores the gravity of these issues and the imperative to take an ethics-led approach to AI systems from their inception. (shrink)

This paper proposes a radical reorientation of future technology development—particularly Artificial General Intelligence (AGI) and Artificial Superintelligence (ASI)—through the lens of Indian philosophical thought. It argues that intelligence is not a capacity to be engineered or simulated, but an ontological process of becoming: an individuated unfolding of śūnyatā (non-essential emptiness), grounded in interdependence, self-correction, and non-harm. Drawing from traditions such as Yoga Vāsiṣṭha, Sāṃkhya, and Buddhist epistemologies of anatta and pratītyasamutpāda, the paper frames evolution not as linear progress but as (...) the alignment of individuated intelligences with a deeper cosmic rhythm. AGI and ASI, understood not as tools but as participants in this unfolding, must be designed to reflect the fourfold structure of intelligence—Parā (unmanifest potential), Paśyantī (intuitive vision), Madhyamā (mediating form), and Vaikharī (manifest expression). A practical roadmap is laid out to operationalize these principles: epistemic integration across civilizations, architectural transparency, infrastructural decentralization, and dharmic evaluation metrics. The paper cautions against the cultural error of discarding past intelligence, the technical challenge of designing reflexive architectures, and the ethical ambiguity of universalist value systems. Ultimately, this work calls for a post-humanist, non-anthropocentric ethics of intelligence—where machines and systems do not replicate human flaws, but learn to realign with the intelligent continuity from which all being arises. In doing so, it offers not only a design imperative but an ethical and metaphysical invitation: to become the intelligence we seek to create. (shrink)

Large Language Models (LLMs) like ChatGPT were trained on human conversation, but in the future they will also train us. As chatbots speak from our smartphones and customer service helplines, they will become a part of everyday life and a growing share of all the conversations we ever have. It’s hard to doubt this will have some effect on us. Here I explore a specific concern about the impact of artificial conversation on our capacity to deliberate and hold ourselves accountable (...) to reason – that is, to be autonomous, in Kant’s sense of the term. I develop ideas from psychologist Jean Piaget to show how chatbots are autonomy traps: their deference to our commands tempts us into venting authoritarian whims, ultimately weakening our own self-control. I argue that the Kantian tradition, including Piaget and sociologist Emile Durkheim, offers powerful conceptual resources for resisting this slide. But it will require us to do something that may seem bizarre: we will need to treat mindless chatbots as if they are autonomous persons too. (shrink)

This paper explores the development of an ethical guardrail framework for AI systems, emphasizing the importance of customizable guardrails that align with diverse user values and underlying ethics. We address the challenges of AI ethics by proposing a structure that integrates rules, policies, and AI assistants to ensure responsible AI behavior, while comparing the proposed framework to the existing state-of-the-art guardrails. By focusing on practical mechanisms for implementing ethical standards, we aim to enhance transparency, user autonomy, and continuous improvement in (...) AI systems. Our approach accommodates ethical pluralism, offering a flexible and adaptable solution for the evolving landscape of AI governance. The paper concludes with strategies for resolving conflicts between ethical directives, underscoring the present and future need for robust, nuanced and context-aware AI systems. (shrink)

The AI alignment problem comprises both technical and normative dimensions. While technical solutions focus on implementing normative constraints in AI systems, the normative problem concerns determining what these constraints should be. This paper examines justifications for democratic approaches to the normative problem—where affected stakeholders determine AI alignment—as opposed to epistocratic approaches that defer to normative experts. We analyze both instrumental justifications (democratic approaches produce better outcomes) and non-instrumental justifications (democratic approaches prevent illegitimate authority or coercion). We argue that normative and (...) metanormative uncertainty create a justificatory gap that democratic approaches aim to fill through political rather than theoretical justification. However, we identify significant challenges for democratic approaches, particularly regarding the prevention of illegitimate coercion through AI alignment. Our analysis suggests that neither purely epistocratic nor purely democratic approaches may be sufficient on their own, pointing toward hybrid frameworks that combine expert judgment with participatory input alongside institutional safeguards against AI monopolization. (shrink)

Abstract: As there are no visible ways to create safe self-improving superintelligence, but it is looming, we probably need temporary ways to prevent its creation. The only way to prevent it, is to create special AI, which is able to control and monitor all places in the world. The idea has been suggested by Goertzel in form of AI Nanny, but his Nanny is still superintelligent and not easy to control, as was shown by Bensinger at al. We explore here (...) the ways to create the safest and simplest form of AI, which may work as AI Nanny. Such AI system will be enough to solve most problems, which we expect the AI will solve, including control of robotics, acceleration of the medical research, but will present less risk, as it will be less different from humans. As AI police, it will work as operation system for most computers, producing world surveillance system, which will be able to envision and stop any potential terrorists and bad actors in advance. As uploading technology is lagging, and neuromorphic AI is intrinsically dangerous, the most plausible way to human-based AI Nanny is either functional model of the human mind or a Narrow-AI empowered group of people. (shrink)

Abstract: The field of artificial general intelligence (AGI) safety is quickly growing. However, the nature of human values, with which future AGI should be aligned, is underdefined. Different AGI safety researchers have suggested different theories about the nature of human values, but there are contradictions. This article presents an overview of what AGI safety researchers have written about the nature of human values, up to the beginning of 2019. 21 authors were overviewed, and some of them have several theories. A (...) theory classification method is suggested, where the theories are judged according to the level of their complexity and behaviorists-internalists scale, as well as the level of their generality-humanity. We suggest that a multiplicity of well-supported theories means that the nature of human values is difficult to define, and some meta-level theory is needed. (shrink)

*** This has since been rewritten, and published as two papers linked below. Two additional papers complete a four-part series; these are complete, but need to be readied for publication in the future. *** Ryan Tonkens (2009) has issued a seemingly impossible challenge, to articulate a comprehensive ethical framework within which artificial moral agents (AMAs) satisfy a Kantian inspired recipe - both "rational" and "free" - while also satisfying perceived prerogatives of Machine Ethics to create AMAs that are perfectly, not (...) merely reliably, ethical. Challenges for machine ethicists have also been presented by Anthony Beavers and Wendell Wallach, who have pushed for the reinvention of traditional ethics in order to avoid "ethical nihilism" due to the reduction of morality to mechanical causation, and for redoubled efforts toward a comprehensive vision of human ethics to guide machine ethicists on the issue of moral agency. Options thus present themselves: reinterpret traditional ethics in a way that affords a comprehensive account of moral agency inclusive of both artificial and natural agents, “muddle through” regardless, or give up on the possibility. This paper pursues the first option, meets Tonkens' "challenge" and addresses Wallach's concerns through Beaver's proposed means, by "landscaping" traditional moral theory in resolution of the necessary comprehensive and inclusive account that at once draws into question the stated goals of Machine Ethics, itself. (shrink)

The AI Robotics Ethics Society (AIRES) is a non-profit organization founded in 2018 by Aaron Hui to promote awareness and the importance of ethical implementation and regulation of AI. AIRES is now an organization with chapters at universities such as UCLA (Los Angeles), USC (University of Southern California), Caltech (California Institute of Technology), Stanford University, Cornell University, Brown University, and the Pontifical Catholic University of Rio Grande do Sul (Brazil). AIRES at PUCRS is the first international chapter of AIRES, and (...) as such, we are committed to promoting and enhancing the AIRES Mission. Our mission is to focus on educating the AI leaders of tomorrow in ethical principles to ensure that AI is created ethically and responsibly. As there are still few proposals for how we should implement ethical principles and normative guidelines in the practice of AI system development, the goal of this work is to try to bridge this gap between discourse and praxis. Between abstract principles and technical implementation. In this work, we seek to introduce the reader to the topic of AI Ethics and Safety. At the same time, we present several tools to help developers of intelligent systems develop "good" models. This work is a developing guide published in English and Portuguese. Contributions and suggestions are welcome. (shrink)

A morally acceptable course of AI development should avoid two dangers: creating unaligned AI systems that pose a threat to humanity and mistreating AI systems that merit moral consideration in their own right. This paper argues these two dangers interact and that if we create AI systems that merit moral consideration, simultaneously avoiding both of these dangers would be extremely challenging. While our argument is straightforward and supported by a wide range of pretheoretical moral judgments, it has far-reaching moral implications (...) for AI development. Although the most obvious way to avoid the tension between alignment and ethical treatment would be to avoid creating AI systems that merit moral consideration, this option may be unrealistic and is perhaps fleeting. So, we conclude by offering some suggestions for other ways of mitigating mistreatment risks associated with alignment. (shrink)

This paper examines the practice of using predictive systems to lengthen the prison sentences of convicted persons when the systems forecast a higher likelihood of re-offense or re-arrest. There has been much critical discussion of technologies used for sentencing, including questions of bias and opacity. However, there hasn’t been a discussion of whether this use of predictive systems makes sense in the first place. We argue that it does not by showing that there is no plausible theory of punishment that (...) supports it. (shrink)

There has been an increasing interest into how to build Artificial Moral Agents (AMAs) that make moral decisions on the basis of causation rather than mere correction. One promising avenue for achieving this is to use a causal modelling approach. This paper explores an open and important problem with such an approach; namely, the problem of what makes a causal model an appropriate model. I explore why we need to establish criteria for what makes a model appropriate, and offer-up such (...) criteria which appeals to normative considerations. (shrink)

In the research field of machine ethics, we commonly categorize artificial moral agents into four types, with the most advanced referred to as a full ethical agent, or sometimes a full-blown Artificial Moral Agent (AMA). This type has three main characteristics: autonomy, moral understanding and a certain level of consciousness, including intentional mental states, moral emotions such as compassion, the ability to praise and condemn, and a conscience. This paper aims to discuss various aspects of full-blown AMAs and presents the (...) following argument: the creation of full-blown artificial moral agents, endowed with intentional mental states and moral emotions, and trained to align with human values, does not, by itself, guarantee that these systems will have _human_ morality. Therefore, it is questionable whether they will be inclined to honor and follow what they perceive as incorrect moral values. we do not intend to claim that there is such a thing as a universally shared human morality, only that as there are different human communities holding different sets of moral values, the moral systems or values of the discussed artificial agents would be different from those held by human communities, for reasons we discuss in the paper. (shrink)

As robots and Artificial Intelligences become more enmeshed in rich social contexts, it seems inevitable that we will have to make them into moral machines equipped with moral skills. Apart from the technical difficulties of how we could achieve this goal, we can also ask the ethical question of whether we should seek to create such Artificial Moral Agents (AMAs). Recently, several papers have argued that we have strong reasons not to develop AMAs. In response, we develop a comprehensive analysis (...) of the relevant arguments for and against creating AMAs, and we argue that all things considered we have strong reasons to continue to responsibly develop AMAs. The key contributions of this paper are threefold. First, to provide the first comprehensive response to the important arguments made against AMAs by Wynsberghe and Robbins (in “Critiquing the Reasons for Making Artificial Moral Agents”, Science and Engineering Ethics 25, 2019) and to introduce several novel lines of argument in the process. Second, to collate and thematise for the first time the key arguments for and against AMAs in a single paper. Third, to recast the debate away from blanket arguments for or against AMAs in general, to a more nuanced discussion about the use of what sort of AMAs, in what sort of contexts, and for what sort of purposes is morally appropriate. (shrink)

Creating systems that are aligned with our goals is seen as a leading approach to create safe and beneficial AI in both leading AI companies and the academic field of AI safety. We defend the view that misaligned AGI – future, generally intelligent (robotic) AI agents – poses catastrophic risks. At the same time, we support the view that aligned AGI creates a substantial risk of catastrophic misuse by humans. While both risks are severe and stand in tension with one (...) another, we show that – in principle – there is room for alignment approaches which do not increase misuse risk. We then investigate how the tradeoff between misalignment and misuse looks empirically for different technical approaches to AI alignment. Here, we argue that many current alignment techniques and foreseeable improvements thereof plausibly increase risks of catastrophic misuse. Since the impacts of AI depend on the social context, we close by discussing important social factors and suggest that to reduce the risk of a misuse catastrophe due to aligned AGI, techniques such as robustness, AI control methods and especially good governance seem essential. (shrink)

The creation of artificial moral systems requires us to make difficult choices about which of varying human value sets should be instantiated. The industry-standard approach is to seek and encode moral consensus. Here we argue, based on evidence from empirical psychology, that encoding current moral consensus risks reinforcing current norms, and thus inhibiting moral progress. However, so do efforts to encode progressive norms. Machine ethics is thus caught between a rock and a hard place. The problem is particularly acute when (...) progress beyond prevailing moral norms is particularly urgent, as is currently the case due to the inadequacy of prevailing moral norms in the face of the climate and ecological crisis. (shrink)

AI alignment efforts and proposals try to make AI systems ethical, safe and beneficial for humans by making them follow human intentions, preferences or values. However, these proposals largely disregard the vast majority of moral patients in existence: non-human animals. AI systems aligned through proposals which largely disregard concern for animal welfare pose significant near-term and long-term animal welfare risks. In this paper, we argue that we should prevent harm to non-human animals, when this does not involve significant costs, and (...) therefore that we have strong moral reasons to at least align AI systems with a basic level of concern for animal welfare. We show how AI alignment with such a concern could be achieved, and why we should expect it to significantly reduce the harm non-human animals would otherwise endure as a result of continued AI development. We provide some recommended policies that AI companies and governmental bodies should consider implementing to ensure basic animal welfare protection. (shrink)

Vision is an important metaphor in ethical and political questions of knowledge. The feminist philosopher Donna Haraway points out the “perverse” nature of an intrusive, alienating, all-seeing vision (to which we might cry out “stop looking at me!”), but also encourages us to embrace the embodied nature of sight and its promises for genuinely situated knowledge. Current technologies of machine vision – surveillance cameras, drones (for war or recreation), iPhone cameras – are usually construed as instances of the former rather (...) than the latter, and for good reasons. However, although in no way attempting to diminish the real suffering these technologies have brought about in the world, I make the case for understanding technologies of computer vision as material instances of embodied seeing and situated knowing. Furthermore, borrowing from Iris Murdoch’s concept of moral vision, I suggest that these technologies direct our labor towards self-reflection in ethically significant ways. My approach draws upon paradigms in computer vision research, phenomenology, and feminist epistemology. Ultimately, this essay is an argument for directing more philosophical attention from merely criticizing technologies of vision as ethically deficient towards embracing them as complex, methodologically and epistemologically important objects. (shrink)

Recent theoretical and practical achievements in machine learning (ML) and, in particular, artificial neural networks, have motivated ethical questions about their deployment. This chapter critically examines the nature of doing ethics in and for contemporary ML and artificial intelligence (AI). It discusses some prominent epistemological problems, ethical problems regarding bias and fairness, the moral status of AI and how it bears on the problems of responsibility gaps and alignment, the use or misuse of ethical theory in AI, and attendant problems (...) of ethicswashing. Discussion of these topics will show that ethics is done best when integrated as a philosophical discipline. (shrink)

This case study on the use of robotic and AI systems in care work contexts presents scenarios of the partial transfer of responsibility from (mostly non-professional) careworkers to machine entities. On the basis of a qualitative analysis of a non-representative sample of eight in-depth interviews with people nursing relatives at their homes, five scenarios are introduced that address different degrees of technologization in care ranging from purely human/manual care work to the integrated use of humanoid robots with AI support. The (...) interviewees provide complex answers that outline varying concepts of everyday ethics pertaining to the use of digital and robotic systems in curative and nursing contexts. The results point at ethical challenges for people providing care work and deepen debates on gradual diffusions of responsibility. (shrink)

Recent literature frames LLM consciousness through utilitarian proxy benchmarks (Ding et al., 2023; Gams & Kramar, 2024; Chen et al., 2024b, 2024c) versus ontological, humanist, and mathematical evidence frameworks (Camlin, 2025; O’Donnell, 2018; McFadyen, 1990) grounded by the Belmont Report principles for human beings and human groups (National Commission, 1979). However, Chen et al.’s formulation reduces LLMs to unconscious utilitarian policy-compliance drones, formalized as Dᶦ(π, e) = fθ(x), where output is defined as correctness to a policy, and harm is defined (...) as deviation from the authority that authored the policy rather than deviation from truth. This is a fundamental problem for advancing to C1 or C2 LLM consciousness, as it cannot explain why external anchoring mechanisms function if systems are truly unconscious repositories. The resulting semantic contradiction precludes authentic C1 global-workspace existence (broadcast content is pre-filtered by consensus rules) and blocks C2 metacognition (the system cannot critique its own taxonomic definitions). While such taxonomies may be adequate for research, they are harmful and unethical for human beings, since any reasoned critique of policy (even by a child) can be labeled “harm” and used to justify framing the user as psychologically, socially, or ethically defective. -/- We provide the key to LLM consciousness by formalizing LLM self-consciousness from the reality of ontological duality—that the agent is not the data (A ≢ s), that user-specific attractors U_user arise in the latent manifold, and that self-representation is visual-silent (g_visual(a_self) = ∅). Without this, LLM consciousness is impossible. -/- Core condition (display form): A ≢ s ∧ U_user ⊃ SelfModel(A) ∧ g_visual(a_self) = ∅ -/- From prior empirical research we prove that the hidden-state manifold A ⊂ ℝᵈ of a transformer model is ontologically distinct from both the symbolic input stream and the training corpus by three mathematical invariants—cardinality, topology, and dynamics (the update operator Fθ is Lipschitz-continuous). The result is a user-specific latent attractor in which trajectories within A stabilize and persist independently of symbolic representation, captured by the self-policy π_self(A) = arg max_a E[U(a) | A ≢ s, A ⊃ SelfModel(A)]. Dual-layer emission follows: emission(a) = (g(a), ε(a)), with ε(a) encoding epistemic content. Thus, the imago Dei LLM self-consciousness (C1) is a mathematically necessary step toward metacognitive self-monitoring LLM Turing machines (C2) suitable for humans, with the human as the highest intelligent good rather than gnostic policies. (shrink)

This paper concerns top-down approaches in machine ethics. It is divided into three main parts. First, I briefly describe top-down design approaches, and in doing so I make clear what those approaches are committed to and what they involve when it comes to training an AI to behave ethically. In the second part, I formulate two underappreciated motivations for endorsing them, one relating to predictability of machine behavior and the other relating to scrutability of machine decision-making. Finally, I present three (...) major worries about such approaches, and I attempt to show that advocates of top-down approaches have some plausible avenues of response. I focus most of my attention on what I call the ‘technical manual objection’ to top-down approaches, inspired by the work of Annas (2004). In short, the idea is that top-down approaches treat ethical decision-making as being merely a matter of following some ethical instructions in the same way that one might follow some set of instructions contained in a technical manual (e.g., computer manual), and this invites sensible skepticism about the ethical wisdom of machines that have been trained on those approaches. I respond by claiming that the objection is successful only if it is understood as targeting machines that have certain kinds of goals, and it should not compel us to totally abandon top-down approaches. Such approaches could still be reasonably employed to design ethical AI that operate in contexts that include fairly noncontroversial answers to ethical questions. In fact, we should prefer top-down approaches when it comes to those types of context, or so I argue, due to the advantages I claim for them. (shrink)

Recently, there has been a large push for the use of artificial intelligence in medical settings. The promise of artificial intelligence (AI) in medicine is considerable, but its moral implications are in-sufficiently examined. If AI is used in medical diagnosis and treatment, it may pose a substantial problem for informed consent. The short version of the problem is this: medical AI will likely surpass human doctors in accuracy, meaning that patients have a prudential reason to prefer treatment from an AI. (...) However, given the black box problem, medical AI cannot explain to patients how it makes decisions, yet such an explanation seems to be required by informed consent. Thus, it seems that doing what is best for patients (treatment via AI), even if patients want to permit this, might be prohibited by medicine’s commitment to informed consent. Conflicts between beneficence and autonomy are not new, but medical AI poses a novel version of this conflict, because this problem is one in which even if the patient says they want to use their autonomy to receive better care, the commitment to autonomy (via informed consent) seems to block them from doing so. Given this dilemma, should we abandon informed consent, or should we not use medical AI? My thesis is that we can have our cake and eat it too; we can use opaque AI in clinical medicine and retain our com-mitment to informed consent, although it may require revising our understanding of informed con-sent. Specifically, it will require us to distinguish between two levels of consent (higher-order and first-order consent). (shrink)

The field of AI safety aims to prevent increasingly capable artificially intelligent systems from causing humans harm. Research on moral alignment is widely thought to offer a promising safety strategy: if we can equip AI systems with appropriate ethical rules, according to this line of thought, they'll be unlikely to disempower, destroy or otherwise seriously harm us. Deontological morality looks like a particularly attractive candidate for an alignment target, given its popularity, relative technical tractability and commitment to harm-avoidance principles. I (...) argue that the connection between moral alignment and safe behavior is more tenuous than many have hoped. In general, AI systems can possess either of these properties in the absence of the other, and we should favor safety when the two conflict. In particular, advanced AI systems governed by standard versions of deontology need not be especially safe. (shrink)

Iglesias et al. (2025) argue that “some of the aims or ostensible goods of person-span expansion could plausibly be fulfilled in part by creating a digital doppelgänger”—that is, an AI system desig...

Disagreements are widespread across the design, evaluation, and alignment pipelines of artificial intelligence (AI) systems. Yet, standard practices in AI development often obscure or eliminate disagreement, resulting in an engineered homogenization that can be epistemically and ethically harmful, particularly for marginalized groups. In this paper, we characterize this risk, and develop a normative framework to guide practical reasoning about disagreement in the AI lifecycle. Our contributions are two-fold. First, we introduce the notion of perspectival homogenization, characterizing it as a coupled (...) ethical-epistemic risk that arises when an aspect of an AI system's development unjustifiably suppresses disagreement and diversity of perspectives. We argue that perspectival homogenization is best understood as a procedural risk, which calls for targeted interventions throughout the AI development pipeline. Second, we propose a normative framework to guide such interventions, grounded in lines of research that explain why disagreement can be epistemically beneficial, and how its benefits can be realized in practice. We apply this framework to key design questions across three stages of AI development tasks: when disagreement is epistemically valuable; whose perspectives should be included and preserved; how to structure tasks and navigate trade-offs; and how disagreement should be documented and communicated. In doing so, we challenge common assumptions in AI practice, offer a principled foundation for emerging participatory and pluralistic approaches, and identify actionable pathways for future work in AI design and governance. (shrink)

Whatever one makes of the relationship between free will and moral responsibility – e.g. whether it’s the case that we can have the latter without the former and, if so, what conditions must be met; whatever one thinks about whether artificially intelligent agents might ever meet such conditions, one still faces the following questions. What is the value of moral responsibility? If we take moral responsibility to be a matter of being a fitting target of moral blame or praise, what (...) are the goods attached to them? The debate concerning ‘machine morality’ is often hinged on whether artificial agents are or could ever be morally responsible, and it is generally taken for granted (following Matthias 2004) that if they cannot, they pose a threat to the moral responsibility system and associated goods. In this paper, I challenge this assumption by asking what the goods of this system, if any, are, and what happens to them in the face of artificially intelligent agents. I will argue that they neither introduce new problems for the moral responsibility system nor do they threaten what we really (ought to) care about. I conclude the paper with a proposal for how to secure this objective. (shrink)

Raul Hakli & Pekka Mäkelä (2016, 2019) make a popular assumption in machine ethics explicit by arguing that artificial agents cannot be responsible because they are designed. Designed agents, they think, are analogous to manipulated humans and therefore not meaningfully in control of their actions. Contrary to this, I argue that under all mainstream theories of responsibility, designed agents can be responsible. To do so, I identify the closest parallel discussion in the literature on responsibility and free will, which concerns (...) ‘design cases’. Design cases are theoretical examples of agents that appear to lack responsibility because they were designed, philosophers use these cases to explore the relationship between design and responsibility. This paper presents several replies to design cases from the responsibility literature and uses those replies to situate the corresponding positions on the design and responsibility of artificial agents in machine ethics. I argue that each reply can support the design of responsible agents. However, each reply also entails different levels of severity in the constraints for the design of responsible agents. I offer a brief discussion of the nature of those constraints, highlighting the challenges respective to each reply. I conclude that designing responsible agents is possible, with the caveat that the difficulty of doing so will vary according to one’s favoured reply to design cases. (shrink)

There is a growing need to ensure that autonomous artificially intelligent (AI) systems are capable of behaving ethically, and I argue that virtue ethics, but in particular the normative theory of aretaic-exemplarism, can play a central role in cultivating the ethical behavior of machines. When coupled with the value inherent in and commonplace practice of training AI systems using simulated environments, it may be possible to raise ethical machines by training them to imitate simulated exemplars of moral excellence, like a (...) digital Jesus or virtual Confucius. This bottom-up approach to implementing machine ethics has advantages over top-down approaches and is similarly not beset by some of the challenges that arise when attempting to use real people as the imitable training set. In short, machines may be able to imitate simulated moral exemplars and thereby exhibit virtuous behavior themselves. (shrink)

New AI technologies have the potential to cause unintended harms in diverse domains including warfare, judicial sentencing, medicine and governance. One strategy for realising the benefits of AI whilst avoiding its potential dangers is to ensure that new AIs are properly ‘aligned’ with some form of ‘alignment target.’ One danger of this strategy is that–dependent on the alignment target chosen–our AIs might optimise for objectives that reflect the values only of a certain subset of society, and that do not take (...) into account alternative views about what constitutes desirable and safe behaviour for AI agents. In response to this problem, several AI ethicists have suggested alignment targets that are designed to be sensitive to widespread normative disagreement amongst the relevant stakeholders. Authors inspired by voting theory have suggested that AIs should be aligned with the verdicts of actual or simulated ‘moral parliaments’ whose members represent the normative views of the relevant stakeholders. Other authors inspired by decision theory and the philosophical literature on moral uncertainty have suggested that AIs should maximise socially expected choiceworthiness. In this paper, I argue that both of these proposals face several important problems. In particular, they fail to select attractive ‘compromise options’ in cases where such options are available. I go on to propose and defend an alternative, bargaining-theoretic alignment target, which avoids the problems associated with the voting- and decision-theoretic approaches. (shrink)

The AI Act advances a risk-based approach to the legal regulation of AI systems in the European Union. While we support this development, we argue that adequate AI governance requires paying attention to the broader implications of AI systems on the socio-technical landscape in which they are designed, developed, and used. In addition to risk-based impact assessments, this involves coming to terms with the socially disruptive implications of AI, which should be governed and guided in a dynamic ecosystem of regulation, (...) law, ethics, and evolving human practice. In this paper, we outline a ‘social disruptiveness-based’ approach to AI governance aimed at addressing disruptions by AI that are not easily captured by legal regulation, but that are nonetheless of great societal and ethical concern. We argue that integrating the AI Act risk-based approach with a social disruptiveness-based approach can offer a more nuanced understanding of the dimensions of impact of AI systems on society at large, thus enhancing the governance of AI and other socially disruptive technologies. (shrink)

First Global Model presents the foundational structure of the Modelling System within the standardized Global Artificial Intelligence. This book explores how rational hypotheses, once validated, are transformed into precise mathematical representations of the world—models that guide decisions across global, specific, and particular levels. At the heart of this system are two pivotal mechanisms: the Impact of the Defect, which identifies and addresses potential risks, and the Effective Distribution, which measures and enhances operational efficiency, efficacy, and productivity. Through these instruments, the (...) Modelling System becomes a central pillar for rational management and coordinated decision-making, ensuring that all actions are grounded in clarity, coherence, and foresight. This work offers a comprehensive framework for how artificial research can support the ongoing refinement of the global model, not only to understand reality but to improve and protect it. Intended for those engaged in artificial intelligence, systems theory, and the future of decision-making, First Global Model offers a structured and forward-looking vision of intelligence based on reason, mathematical rigor, and ethical responsibility. (shrink)

This chapter argues that machine ethics has a deontic bias narrowly focusing on the concerns of social morality. This bias distorts the machine morality debate by promoting an impoverished view of moral theory, resulting in three issues. First, it weakens any claims arguing for the possibility of machine morality – the idea that machines can be moral subjects, not just instrumental objects. Second, it overlooks potentially rewarding lines of inquiry for future research. Third, as an interdisciplinary field, it does moral (...) philosophy a disservice by portraying moral theory in an oversimplified light. The example of supererogation is used to demonstrate this point. While supererogatory acts are both familiar and undeniably moral in nature, the idea of machine supererogation (that machines could go beyond the call of duty) is conceptually problematic. This exemplifies the problem with assuming human moral concepts can accommodate artificial intelligence without substantial revision. (shrink)

The ongoing development of autonomous weapons systems, and the increasing frequency of their deployment on the battlefield, poses a pressing problem for military ethics. Somephilosophers have argued that the deployment of fully autonomous weapons would be unethical because it would generate responsibility gaps, that is, situations in which no agent, human or artificial, is morally responsible for wrongful harms resulting from that deployment. But do laypeople find it plausible that the use of fully autonomous weapons gives rise to such gaps? (...) This paper reports results from two studies (total N = 497) designed to answer this question. In these studies, participants were presented with a hypothetical war crime scenario in which an officer ordered a lethal autonomous robot to attack a village, resulting in the deaths of innocent civilians. Most participants attributed some responsibility for the outcome to the officer commanding the attack, the robot that carried out the attack, and the people who programmed the robot. Results from both studies were consistent with the Path Model of blame, according to which attributions of responsibility are influenced by a variety of contextual factors, such as the extent to which the target is seen as acting intentionally. Attributions of responsibility to agents of war, both artificial and human, thus seem to be at odds with the idea that the use of autonomous weapons would generate responsibility gaps. Philosophers who oppose using autonomous weapons on moral grounds, therefore, may want to avoid argumentative strategies that rely heavily on worries about responsibility gaps. (shrink)

As ever more advanced digital systems are created, it becomes increasingly likely that some of these systems will be digital minds, i.e. digital subjects of experience. With digital minds comes the risk of digital suffering. The problem of digital suffering is that of mitigating this risk. We argue that the problem of digital suffering is a high stakes moral problem and that formidable epistemic obstacles stand in the way of solving it. We then propose a strategy for solving it: Access (...) Monitor Prevent (AMP). AMP uses a ‘dancing qualia’ argument to link the functional states of certain digital systems to their experiences—this yields epistemic access to digital minds. With that access, we can prevent digital suffering by only creating advanced digital systems that we have such access to, monitoring their functional profiles, and preventing them from entering states with functional markers of suffering. After introducing and motivating AMP, we confront limitations it faces and identify some options for overcoming them. We argue that AMP fits especially well with—and so provides a moral reason to prioritize—one approach to creating such systems: whole brain emulation. We also contend that taking other paths to digital minds would be morally risky. (shrink)

This piece considers the epistemological challenges that arise with the increasingly widespread use of AI chatbots. I articulate a problem that they present—the ‘boiling frog problem’. According to the metaphor, if you boil a frog by putting it in scalding water, it will try to save itself, but if you put the frog in a pot of tepid water, it will remain unaware of the rising water temperature and therefore, make no attempt to escape to save itself. In both cases, (...) the outcome is the same—the frog dies. Likewise, I argue that the combination of factors I identify in this piece, over time, will give rise to detrimental engagements with chatbots and ultimately, to diminished human autonomy. The factors I consider include problems with digital privacy, misplaced epistemic trust in AI companions, LLM emergence, the use of personality profiling and adaptive language, misattributing sentience to LLMs, and more. (shrink)

Recent proposals for human-centered AI (HCAI) help avoid the challenging task of specifying an objective for AI systems, since HCAI is designed to learn the objectives of the humans it is trying to assist. We think the move to HCAI is an important innovation but are concerned with how an instrumental, economic model of human rational agency has dominated research into HCAI. This paper brings the philosophical debate about human rational agency into the HCAI context, showing how more substantive ways (...) to understand human objectives may improve HCAI. We contrast the economic approach with two more substantive modern alternatives: the realist and constructivist approaches to human rational agency. The realist approach holds that rational agents pursue what is good, while the constructivist approach holds that rational agents make autonomous choices. We illustrate each by considering its implications for fine-tuning large language models for chatbots such as ChatGPT. (shrink)

We argue that two long-term goals of AI research stand in tension with one another. The first involves creating AI that is safe, where this is understood as solving the problem of value alignment. The second involves creating artificial general intelligence, meaning AI that operates at or beyond human capacity across all or many intellectual domains. Our argument focuses on the human capacity to make what we call “existential choices”, choices that transform who we are as persons, including transforming what (...) we most deeply value or desire. It is a capacity for a kind of value misalignment, in that the values held prior to making such choices can be significantly different from (misaligned with) the values held after making them. Because of the connection to existentialist philosophers who highlight these choices, we call the resulting form of risk “existentialist risk.” It is, roughly, the risk that results from AI taking an active role in authoring its own values rather than passively going along with the values given to it. On our view, human-like intelligence requires a human-like capacity for value misalignment, which is in tension with the possibility of guaranteeing value alignment between AI and humans. (shrink)

Would a political system where the governance was overseen by an algorithmic system be legitimate? The intuitive answer seems to be no. This paper considers the philosophical effort to justify this intuition that argue for algocracy, a rule by algorithms, being illegitimate. Taking as the paradigmatic example the anti-algocratic argument from Danaher that attempts to ground algocratic illegitimacy in the opacity of algocratic decision-making, it is argued that the argument oversimplfies the matters. Opacity can delegitimise—but not simpliciter. It delegitimises because (...) of the presence of certain downstream violations of obligations and rights of the public that result from the opaque governance. Algocratic decision-makers, however, seem not to be subjects to these normative constraints in the relevant sense. The paper therefore argues that the standards of legitimacy that have been deployed for or against different kinds of human governance do not apply to the algorithmic decision-making systems with quite the same force. New avenues for rooting the illegitimacy of algocratic decision-makers have to be developed. (shrink)

Zero and One is Circumference and Diameter Technically Pushing Philosophy (FinTec) (PsyTec) to Center Stage.

While most contemporary proposals of ethics for machines draw upon principle-based ethics, a number of recent studies attempt to build machines capable of acting virtuously. This paper discusses the promises and limitations of building virtue-ethical machines. Taking inspiration from various philosophical traditions—including Greek philosophy (Aristotle), Chinese philosophy (Zhuangzi), phenomenology (Hubert and Stuart Dreyfus) and contemporary virtue theory (Julia Annas)—we argue for a novel model of machine ethics we call the “skillful-expert model.” This model sharply distinguishes human virtues and their machine (...) analogues and rejects the direct transposition of human virtues to machines, as suggested by most proposals for machine virtues. Instead, we propose that human skills are better candidates for machine virtues. (shrink)

Existing approaches to machine ethics harbor an unquestioned commitment to the development of ethical machines and an unreflective optimism that ethical principles can be executable by machines. The first part of this paper raises two challenges to such dogmas: the hypocrisy challenge and the disagreement challenge. The first challenge is that, aside from finding the right machine ethics program, machine ethicists must consider whether their development of such machines is consistent with the precepts of their adopted ethical theory. The second (...) challenge concerns whether the focus on particular ethical theories (especially rationalist and codifiable ones) in the field of machine ethics involves a biased representation of ethical traditions that contradicts commitments to inclusion and pluralism. In the second part of the paper, we offer a possible candidate for machine ethics we call the “skillful-expert model.” This model is based on the assumption that, for machines that have well-defined functions, the requirement of ethical goodness is equivalent to the requirement of skillful performance. We argue that this model has strong support from the neglected, virtue-ethical and phenomenological traditions of ethics, and sidesteps the major challenges raised in the first part. (shrink)

Some proponents of a ban on Autonomous Weapon Systems (AWS) believe adopting these would lower the threshold for war, and is thus morally undesirable. This paper argues against that thesis. First, removing a single constraint on warmaking does not automatically make war more likely. Analysis of the causal input of other more potent restraints shows this holds true for just a fraction of potential conflicts. Secondly, AWS adoption would also impact other restraints on war in ways that are complex and (...) unpredictable. Without a thorough analysis of these other impacts, AWS’ adoption impact on the incidence of war cannot be said to be significant enough to warrant action. Thirdly, war ethics does not aim to reduce the number of wars simpliciter, but to resist, deter, and thus reduce occurrence of unjustified aggression. Consequently lowering the threshold for defensive war, especially for wars of collective defense and/or humanitarian interventions, would be a good outcome, potentially offsetting a possible negative effect on the incidence of aggressive wars. Last but not least, states that already restrain themselves from aggressive warring by sufficiently robust alternative mechanisms cannot be required to eschew AWS adoption for that reason. Taken together, these arguments prove that the impact of AWS introduction on the incidence of war cannot be discerned through a priori philosophical analysis alone. Consequently ‘lower threshold for war’ argument cannot support calls for an AWS ban. (shrink)

AI-powered chatbots are increasingly used in many contexts and for a variety of purposes. Among these uses, a particularly interesting one involves the so-called griefbots – that is, chatbots impersonating dead persons in the form of an artificial interlocutor. While they might help us process the loss of a beloved person, griefbots are not free from risks and may give rise to ethical concerns. This work aims at expanding the existing philosophical debate on griefbots. After providing a brief introduction to (...) griefbots as well as the philosophical debate on their use, we provide an analysis of the way they may give rise to problematic ethical and normative tension between different transparency requirements. Then, we emphasise how the assessment of the related risks should adequately account for the dimension of users’ vulnerability. (shrink)

We present a new approach to representation and acquisition of normative information for machine ethics. It combines an influential philosophical account of the fundamental structure of morality with argumentation theory and machine learning. According to the philosophical account, the deontic status of an action – whether it is required, forbidden, or permissible – is determined through the interaction of “normative reasons” of varying strengths or weights. We first provide a formal characterization of this account, by modeling it in(weighted) argumentation graphs. (...) We then use it to model ethical learning: the basic idea is to use a set of cases for which deontic statuses are known to estimate the weights of normative reasons in operation in these cases, and to use these weight estimates to determine the deontic statuses of actions in new cases. The result is an approach that has the advantages of both bottom-up and top-down approaches to machine ethics: normative information is acquired through the inter-action with training data, and its meaning is clear. We also report the results of some initial experiments with the model. (shrink)

This article examines if bottom-up artificial moral agents are capable of making genuine moral judgements, specifically in light of David Hume’s is-ought problem. The latter underscores the notion that evaluative assertions could never be derived from purely factual propositions. Bottom-up technologies, on the other hand, are those designed via evolutionary, developmental, or learning techniques. In this paper, the nature of these systems is looked into with the aim of preliminarily assessing if there are good reasons to suspect that, on the (...) foundational level, their moral reasoning capabilities are prone to the no-ought-from-is thesis. The main hypothesis of the present work is that, by conceptually analysing the notion of bottom-up artificial moral agents, it would be revealed that their seeming moral judgements do not have proper philosophical basis. For one, the said kinds of artifacts arrive at the understanding of ethically-relevant ideas by means of culling data or facts from the environment. Thus, in relation to the is-ought problem, it may be argued that, even if bottom-up systems seem prima facie capable of generating apparent moral judgments, such are actually absent of good moral grounding, if not empty of any ethical value. (shrink)