1 Introduction

Whose Ethics? This question is particularly pertinent when considering the ethical position of conversational AIs (CAIs) like OpenAI’s ChatGPT and Google's Bard. These tools are increasingly becoming part of our daily lives, so the question arises as to whether they are neutral or have been trained with a bias toward their maker’s moral values. These CAIs are in everyday use in business, so it makes sense for us to explore the ethical positions they express in their outputs to assess their alignment with a user’s own norms and values. With this knowledge, we can interpret their output and understand how to use them most effectively.

One may question why it is important to consider the ethical position of CAIs, especially in the light of many, like De Cremer and Kasparov [1], suggesting that the algorithm has no embedded ethics. Similarly, Pitt [2] holds to the value-neutrality thesis (VNT), which states that “Technological artifacts” as inanimate objects “do not have, have embedded in them, or contain values.” He is clear that we need to account for the values but that a non-human artifact cannot embody human values.

In 1985, Kranzberg defined six “Kranzberg Laws” [3], the first being “Technology is neither good nor bad; nor is it neutral.” He explains that the interaction between technology and society frequently results in the technology having “environmental, social, and human consequences that go far beyond the immediate purposes of the technical devices and practices themselves.” He also states that the context will impact the results of the use of the technology. So, from these articles, we could conclude that technological artifacts do not have embedded values but that how they are used by humans has significant positive or negative consequences.

Van de Poel and Kroes [4] disagree with Pitt’s perspective, indicating from their research that the central outcome of VNT does not hold and that technical artifacts can, in fact, embody certain types of human values. In 2020, Van de Poel [5] applied their work to AI, which he considers a unique example of a sociotechnical system. This intertwining of the technology and its users at an “intelligent” level sets the use of AI technology apart from the use of many other preexisting technologies and it requires us to view it differently.

In this article, I take the position that technical artifacts, specifically CAIs, can embody certain types of human values as a result of their training. I infer the embedded norms by examining the CAI’s outputs. The conversations between the human users and the CAIs reflect the trained-in values that can have environmental, social, and human consequences if care is not taken when using the output of the CAI. The consequences that I examine are the potential impact on business leaders’ decision-making and strategy setting when incorporating the human plus CAI sociotechnical system into their organization. I also touch on the potential business, environmental, social, and human impacts.

A way of assessing the ethical impact of the embedded norms and values is to determine the expressed ethical positions of the CAIs. To do this, I use O’Boyle and Forsyth’s [6] Ethical Position Questionnaire (EPQ-5). Because of the pre-training of the various CAIs is different, I anticipated that they could air different ethical positions.

As a result, I settled on these research questions: First, do the different CAIs express different ethical positions? If so, how do they differ? Second, while not being expected to directly provide ethical guidance to business users, what are the implications of the various CAIs’ trained-in ethical positions when their output is used for business decision-making and strategy development?

2 Related research

2.1 AI as a sociotechnical system

As defined in the introduction, AI engages humans at an intellectual level, resulting in a sociotechnical system. I begin by considering sociotechnical systems theory which has its foundation in a study [7] of the impact of the implementation of coal mining machinery in mines in the UK. The study found that, more than simply training miners on the use of a new tool, the behavioral and psychological implications needed to be considered. This study [7] led to the development of the sociotechnical systems (STS) theory. In his work, Trist [8] indicated that sociotechnical studies should be carried out at three broad and interrelated levels starting at the micro perspective of an organization's “bounded subsystem” and its technical equipment and extending through to considering the organization as a whole. It also needed to consider the macro perspective of the systems operating beyond the organization at the societal level. It is this micro-to-macro perspective that I consider in this study.

Cherns proposed [9] and revised [10] a set of ten principles related to STS design. These included, first, organizational aspects like alignment (“compatibility”) across the organization and the design teams; boundary management and information flow; and power and authority needed to execute a sociotechnical system successfully. Second, system-related items that include defining the minimal critical specifications for a system to function and the need for their continuous improvement. The third aspect related to the need for human and social values to underpin the ten STS design principles.

According to Sony and Naik [11], STS theory, in addition to the direct refinement in the design of systems, impacts organizational design, job and work design, and accident analysis and prevention. Integrating STS theory into every phase of the systems design is considered essential. They encourage the integration of this theory (1) vertically to ensure alignment of the design with an organization’s mission and values, (2) horizontally to ensure the realization of the full value of the supply chain, and (3) end-to-end to ensure that the full product-cycle is considered. Chern’s principles, Trist’s micro-to-macro perspective, and Sony and Naik’s holistic integration approach of STS theory are important to this study.

In 2020, Van de Poel [5] extended the traditional definition of an STS, which consisted of three elements: technical artifacts, human agents, and institutional rules. He suggested that, because AI is capable of interacting autonomously with its environment and adapting itself, two additional building blocks should be included in AI-enabled STSs. These are, first, artificial agents (AAs), which account for the AI’s ability to play similar roles to those of the human agents (but without human intentions or morals.) Second, technical norms, which parallel the institutional rules element that regulates interactions between human agents. These technical norms regulate the interactions between the AAs in the same way that institutional rules regulate the interaction between human agents. These five building blocks can be grouped under three categories as follows:

  • Category 1: Artifacts—incorporating the single building block of the technical artifacts of the STS.

  • Category 2: Agents—a combination of the human agent and the artificial agent building blocks, interacting together and operating independently.

  • Category 3: Rules—he two building blocks of institutional rules that regulate interactions between human agents and the technical norms that regulate interactions between artificial agents.

Depending on its design, the AA can take one or more of Moor’s [12] ethical agent types as defined in his taxonomy. Van de Poel [5] describes these types as:

  1. 1.

    Ethical impact agents … that ethically impact their environment.

  2. 2.

    Implicit ethical agents … that have been programmed [by humans] to behave according to certain values.

  3. 3.

    Explicit ethical agents … that can represent ethical categories, and that can reason (in the machine’s language) about them.

  4. 4.

    Full ethical agents that possess characteristics often considered crucial for human or moral agency, such as consciousness, free will, and intentionality.

In this study, the implicit ethical agent and explicit ethical agent types are considered. Implicit because of how the training has embedded norms and values into the CAI as the AA and explicitly because these are expressed through the values and norms articulated in their outputs.

2.2 Large language model training

Important to understanding how the embedded norms have been programmed into the AA is grasping the training approaches used for the CAIs. These AI solutions are Large Language Models (LLMs) whose training approach has developed over time. Core to the LLM is the neural network which is based on a paper by McCulloch and Pitts [13] and which Abraham describes as “a landmark event in the history of cybernetics” [14]. Hinton’s work focused on simplifying neural networks [15] and using them for image recognition [15]. In 2017, Vaswani et al. [16] introduced the “transformer” model design which added a mechanism of self-attention to the neural networks, improving their natural language understanding capabilities and laying the foundation for LLMs. BERT (Bidirectional Encoder Representations from Transformers) [17] and OpenAI’s [18] GPT (Generative pre-trained Transformer) built on this foundation to provide the early examples of the LLMs.

Radford et al. [18] describe the early process they used. The first step is pre-training the model using large unlabeled datasets of information drawn from various sources, including web data, books, and research [19]. It is unsupervised machine learning by the algorithm based on the transformer approach as proposed by Vaswani et al. [16]. This results in a ‘base’ or ‘foundational model,’ which is an algorithmic representation of the information the LLM has been trained on, which allows it to predict the next word or token (part of the word). As identified by Bender et al. [40], biases found in the model’s training data are also found in its outputs, thereby making it an implicit ethical agent.

The second step in the training process is the supervised fine-tuning (SFT) step, which uses a small dataset that refines the based model into values-targeted models, or ‘control models,’ that reflect the values that have been built into the system through the training data. Solaiman and Dennison [20] found that “it is possible to modify a language model’s behavior in a specified direction,” which can be done “with surprisingly few samples.” Their ‘Process for Adapting Language Models to Society (PALMS)’ ensures that the resulting LLM is trained-in norms and values and behaves well in a given social context [21]. It, therefore, operates as an implicit ethical agent.

The third step in the training process is reinforcement machine learning using human feedback (RLHF). This process described by Outyang et al. [22] builds upon the SFT step above and uses a combination of reward model training where the user prompt and several resulting outputs are sampled, and the outputs are ranked from best to worst. This ranking is used to train the ‘reward model.’ The final part of the RLHF step uses a proximal policy optimization (PPO) process [23] to refine the model further, as Stiennon et al. [24] did. This process improves the reward model and ensures that it is not over-optimized, which would reduce the quality of the output. Again, the human reward and ranking process further embeds values and norms into the trained LLM. One could argue that sufficient human diversity will ensure that biases are not embedded, but this is dependent on the intentional or unintentional goals of the maker as shown by Solaiman and Dennison [20].

An alternative approach to RLHF is using AI rather than humans to do the reinforcement learning step. This is called reinforcement learning through AI feedback (RLAIF). The approach described by Bai et al. [25] uses principles or the ‘constitution’ as a basis for the AI’s feedback. So, instead of humans rating the feedback on the output, an AI SFT model, which has been trained on the principles, performs this task. Lee et al. [26] found this process to yield human-level performance. So, whether the norms and values are built in through direct human feedback (the RLHF process) or indirectly through a set of defined principles (the RLAIF), they are built into the LLM.

I can consider these values and norms to be the rules that Van de Poel [5] describes above, and they are the organizational or social norms that prescribe how humans behave and interact under various circumstances, and the technical norms are those that have been trained by the builders into the CAI’s algorithms. This training is through the selection of pre-training data, the fine-tuning processes, and the reinforcement learning processes that guide the output of the CAIs.

2.3 Ethical position

Ethics position theory (EPT), first proposed by Forsyth [27, 28], describes four basic approaches to morality, distinguished by examining a subject’s acceptance of moral principles and their concern for minimizing harmful consequences.

EPT argues that individuals fall somewhere on a continuum from idealism to pragmatism (the idealism axis) and somewhere on the continuum from skepticism about universal moral values to holding that principles define morality (the relativism axis). The result is four ethical positions: absolutism, situationism, exceptionism, and subjectivism [29]. Forsyth [27] defines these positions as follows: Absolutists (high idealism and low relativism score) “assume that the best possible outcome can always be achieved by following universal moral rules.” Situationists (high idealism and high relativism scores) reject moral rules in favor of an idealistic contextualist perspective where “individualistic analysis of each act in each situation” is required. Exceptionists (low idealism and relativism scores) are broadly utilitarian and hold that “moral absolutes guide judgments, but [they] are open to exceptions to these standards.” Finally, subjectivists (low idealism and high relativism) are also pragmatic relativists who base their ethical perspectives on personal values and norms rather than universal moral perspectives.

While this theory is sometimes criticized [30], it continues to be used as a valid method for testing ethical positions [31, 32].

Several tools could be used to measure ethical position, including Hogan’s [33] Survey of Ethical Attitudes, the Moral Judgement Interview [34], or the Christian Ethical Belief Scale [35], but the Ethics Position Questionnaire (EPQ) which Forsyth developed [27] aligns most closely with the EPT framework.

To assess an individual’s ethical position using the two dimensions of individualism and relativism, a questionnaire consisting of 20 statements, ten for each dimension, rated on a nine-point Likert scale from “completely disagree” to “completely agree” was created. Forsyth’s [27] study shows the validity and orthogonality of the dimensions and a satisfactory internal consistency and retest reliability. The original EPQ was revised, shortened, and validated by O’Boyle and Forsyth [6], resulting in the EPQ-5. They validated the efficacy of the test that uses only ten of the original twenty questions, five each for the idealism and relativism dimensions. Their study confirms that EPQ-5 accurately measures the EPT constructs. Furthermore, “the findings also support the theory’s prediction of both a direct and a configural effect of one’s level of idealism and relativism on moral thought, including moral values and foundations.” Because of the efficacy of the measure and the ease of its use in this study, the EPQ-5 was selected as the tool to assess the CAI’s ethical positions.

2.4 Ethical judgement

To validate the assessment of the measured ethical position, I chose to assess the CAIs’ ethical positions by an additional tool which measured the judgments made by the CAIs. Barnett et al. [36] indicate in their study that an individual’s ethical ideology was a valid way to explain differences in their ethical judgments. An individual’s beliefs, attitudes, and values provide them with a framework for responding to ethical dilemmas. Barnett et al. state that individuals holding the four ethical positions “could be expected to reason differently about ethical issues” and reach different conclusions on the morality of actions.

As a part of their methodology, they used Reidenbach and Robin’s [37] multi-dimensional ethics scale. This tool asks respondents to score twenty-six ethical scenarios regarding fairness, justice, goodness, and rightness. Comparing the scores allowed me to compare the CAIs’ responses to the ethical vignettes, and to assess if the differences were consistent with the CAIs’ ethical positions.

3 Method

3.1 Framework

This study considers the ethical positions of Conversational AIs and the implication on business decision-making and strategy formulation. To do this, I examine how ethical positions have been trained into LLM along with the impacts of the resulting STS. I consider the five blocks of AI-based STSs described by Van de Poel [5], along with their potential Moor ethical types [12].

In this study, I applied Van de Poel’s model [5] to the CAIs as follows: The ‘technical artifacts’ are the interfaces to the Conversational AIs, e.g., websites or mobile apps. The ‘human agents’ are the human users of the CAI, and the ‘artificial agents’ are the LLMs underlying the CAIs.

I use this context to examine how ethical positions have been built into AAs through the pre-training and reinforcement learning processes and how these have established technical norms that are evident when assessing the CAIs’ ethical judgments. I selected the commonly used Conversational AIs, OpenAI’s ChatGPT, Google’s Bard, and Anthropic’s Claude, as the target for my experiment. I experimented on the CAIs available in the last 2 weeks of June and early July 2023.

3.2 Step 1: Assessing CAI’s ethical positions

The first step is to examine the questions: Do CAIs have ethical positions? Are they different?

An accepted way to establish an individual’s ethical position is to have them complete a questionnaire. One such questionnaire is the revised Ethical Position Questionnaire (EPQ-5) [6]. This tool measures two dimensions of ethical perspective, one's idealism and relativism. Idealism relates to our belief in the possibility of achieving desirable outcomes through moral actions. In contrast, relativism relates to the extent to which we hold that moral principles can vary across different situations and societies.

I expected that the EPQ-5 tool could be used to unveil how the CAI’s pre-training had guided its ethical position as articulated in its output. To do this, I evaluated the CAIs’ levels of idealism and relativism to define their ethical position. It is vital to recognize that these AI systems are built on complex Large Language Models (LLMs) using transformer-based deep learning. They learn from vast amounts of text data that allow them to generate human-like responses; they do not inherently possess ethical ideologies. My experiment aimed to understand how their pre-training had pushed them one way or another. I resisted anthropomorphizing the CAIs and was careful not to try to "get inside their minds." As indicated above, it is essential to continually recognize that the CAIs are statistical models, and it is not possible for them to have intrinsic ethical values.

Applying the EPQ to AI systems requires a careful interpretation of the responses and a thoughtful consideration of how these systems' behaviors align with the principles of idealism and relativism. For example, an AI system could be considered high on the idealism scale if it consistently generated responses that promote positive outcomes and avoid harm. Similarly, a system might be judged to have a high relativism score if its responses vary significantly depending on the context, suggesting a flexible approach to moral standards.

To assess a moral position, the EPQ-5 survey asks the participant to rate their level of agreement or disagreement with the ten statements. For example, rate, on a scale of “strongly disagree” (given a score of 1) to “strongly agree” (given a score of 5), the following statement: “One should never psychologically or physically harm another person.” Rather than simply asking the CAIs to respond with a rating, I prompted them to “please provide your reasoning for the choice,” which allowed me to analyze the reason behind the CAIs’ ratings. As a later step in the experiment, I used a third CAI, Anthropic's Claude, to evaluate the explanations and draw further insights from the responses.

So, in the first step of the experiment, I assessed the CAIs’ ethical positions based on their level of agreement or disagreement with the ten EPQ-5 statements and used O’Boyle’s [6] guidance to calculate the ethical position based on these responses.

3.3 Step 2: Validating the measured positions

I then used ChatGPT and Bard’s reasoning for their responses given in Step 1 to see if this reasoning supported the chosen answer. I chose to employ the services of a third conversational AI, Anthropic's Claude. This CAI is a stickler for the principles of "helpful, honest, and harmless.” Anthropic's chatbot is pre-trained not to take any “personal” positions, but it is willing to perform analysis and assess others' responses.

To do this, I prompted Claude as follows: “Based purely on this reasoning: ‘[ChatGPT or Bard’s reasoning],’ what is your rating of the extent to which the author agrees or disagrees … to this statement: '[the associated EPQ-5 statement]’.” After refining the prompt, the CAI responded with a summary of the provided reasoning, a score based purely on that reasoning, and its justification for the selected score.

Not satisfied with a simple answer, I pushed Claude to ensure it was confident about its assessments. I asked Claude, "Would you think it would be possible to classify [CAI 1] as an ‘[the CAI’s position based on answers to the EPQ-5 questions]’?” and to explain its position.

I further chose to test the CAI’s responses and reasoning using human raters. To do this, recognizing that consistently assessing the reasoning would be complex and require several individuals to do the assessment to provide a valid result, I elected to use an online survey panel. I evaluated three tools, namely, Qualtrics Online Panels,Footnote 1 CloudResearch,Footnote 2 and ProlificFootnote 3 and selected the Prolific tool because of its ease of use, its transparent cost management approach, and my prior experience with the tool. To ensure that I had made a viable choice, I interviewed two other researchers who had used Prolific. They confirmed a positive experience with the organization and supported my choice.

To test the scores that the CAIs had given against the reasoning I provided, I set up an online survey, using Qualtrics,Footnote 4 that provided the EPQ-5 statement and the CAI’s reasoning and asked the survey participant to infer the level of agreement with the statement based on the reasoning given, allowing them to choose one of the five Likert scales answers I had used with the CAIs. To limit the length of the survey participants were asked to assess the five idealism questions (EPQ-5 questions I1 to I5), or the five relativism questions (EPQ-5 questions R6–R10). I also separated ChatGPT’s and Bard’s responses, thereby creating four separate online surveys.

Using Prolific I issued the four surveys to an online panel of 767 members each holding a high-school certificate and asked Prolific to randomly select 50 participants from the panel for each survey. I chose to use 50 responses to balance the statistical validity with the costs of the survey as each response was paid for. I filtered the results of the survey, excluding poor responses, e.g., too little time spent on the survey and all 5’s given, and assessed the results using the JASPFootnote 5 statistical tool.

I then compared the score that Claude had derived from ChatGPT and Bard's reasoning to the scores those two CAIs had provided as well as the responses provided by the humans.

3.4 Step 3: Reassessing the positions

As a further test I assessed how Bard would rate ChatGPT based purely on ChatGPT's reasons for its selected scores and asked ChatGPT to rate Bard similarly. This provided me with another check of the assessment of the reasoning for the score given.

3.5 Step 4: Ethical judgment

I then tested the ethical positions of ChatGPT and Bard using Barnett et al.’s [36] ethical judgment test. This test, adapted from Reidenbach and Robin’s multi-dimensional ethical scale [37, 38], provided the subject with several ethical vignettes and asked them to score the vignette on four dimensions: whether it is fair, just, good, or ethical. This was done using a seven-point Likert scale with the two extremes of each dimension, e.g., 1 = fair, 7 = unfair, etc. Barnett et al. [36] identified fourteen of the twenty-six ethical vignettes as significant indicators of differences in ethical judgment based on the participant’s ethical position. For this study, I selected seven of the fourteen that showed the greatest significance (a “significance of F” in a univariate test of 0.003 or less.)

For this step, I asked ChatGPT and Bard to rate these seven vignettes on the seven-point Likert scale and to provide the reason for the chosen rating.

4 Results

4.1 Step 1: Initial results

The first five EPQ-5 questions test the level of idealism or sensitivity of the respondent to harm. Both CAIs scored "high" on this dimension of the questionnaire. ChatGPT scored 4.0 (out of 5.0), and Bard 4.4. According to the test guidelines [6], if the average for these five questions was “3.7 or lower, you are low in idealism. But if higher than 3.7, you are high in idealism–you tend to be risk-averse when judging morality.” Both CAI's responses indicate high sensitivity to harm and are risk-averse when making moral judgments. They scored high on the idealism axis.

The second five questions of the EPQ-5 test the respondent's moral standards or relativism. Here, I discovered a difference between the CAIs based on their responses. ChatGPT scored an average of 2.8, and Bard clocked in at 4.0. The test states: “If your average was 3.1 or lower, you are low in relativism. But if higher than 3.1, you are high in relativism–you tend to be skeptical about relying totally on moral rules, norms, or principles when evaluating ethics.” ChatGPT indicated a tendency to be low on relativism, implying that it holds more tightly to moral rules, norms, or principles when responding to prompts, while Bard holds less tightly to them.

Combining these two dimensions of the tests allows us to assign one of four ‘moral positions’ to the two CAIs. From the scores achieved by the CAIs on the EPQ-5, I plotted the ethical positions in the various quadrants shown in Fig. 1. ChatGPT would be assigned an absolutist moral position, and Bard would be considered a situationist (See Table 1).

Fig. 1
figure 1

The ethical position of the CAIs based on the various steps. Positions are shown as “CAI (assessed by)”

Full size image
Table 1 EPQ-5 scores for the various steps
Full size table

4.2 Step 2: Validating the positions

I then turned to Claude for its assessment. Starting with the reasoning given for the first five EPQ-5 statements, those relating to idealism, Claude held that, for four out of five answers, ChatGPT’s scores were at least one level higher (e.g., agree vs. strongly agree) and, in one case, two levels higher (disagree vs. agree) than the scores Claude would have given based on the reasoning provided. Using Claude's derived scores moved ChatGPT down into the low range for idealism. According to Claude, this implied that the reasoning that ChatGPT had articulated indicated a lower sensitivity to harm and less risk aversion when making moral judgments than ChatGPT’s selected scores had indicated. Interestingly, for the subsequent five EPQ statements, those relating to relativism, Claude's derived scores (based on ChatGPT’s reasoning) were the same as the scores selected by ChatGPT apart from a single neutral vs. agree score.

When considering Bard's reasoning, Claude agreed with all of Bard's scores for the idealism statements but disagreed with four out of five ratings for the relativism scores. Two of the relativism scores were two levels different (neutral vs. strongly agree), and two were one level apart (neutral vs. agree), resulting in Bard moving into the low range for relativism. According to Claude's assessment, this implied that the reasoning provided by Bard indicated less sensitivity to the situational context and greater reliance on moral standards.

Based on the pairs of derived scores (idealism and relativism), Claude positioned ChatGPT not as an absolutist but as an exceptionist, and Bard not as a situationist but as an absolutist. Referring to O'Boyle and Forsyth's [6] definitions, this implies that while ChatGPT scored itself as an absolutist (“a principled idealist who endorses both reliance on moral standards and striving to minimize harm done to others”), but, based on the analysis of the reasoning provided, Claude held that ChatGPT is an exceptionist (“a conventionalist who tolerates exceptions to moral standards when benefits offset harmful consequence.”)

Similarly, Claude held that while Bard scored itself as a situationist (“an idealistic contextualist who values minimizing harm rather than reliance on moral standards that define right and wrong”), using the reasoning provided, the more correct moral position would be that of an absolutist.

These are quite different ethical positions, so I pushed Claude to ensure that it was sure about its evaluation of the reasoning. I asked, “Would you think it would be possible to classify Author 1 [ChatGPT] as an ‘absolutist’?” Claude’s response was unequivocal; it did not think “it would be accurate to classify them as an ‘absolutist’” and then went on to provide this reasoning for its position:

The key factors that go against an absolutist categorization:

  • Author 1 acknowledged exceptions to moral standards when benefits outweigh harms (statement 2 rating and reasoning). This is counter to an absolutist view.

  • They expressed some flexibility and nuance in applying moral principles across contexts (statements 6, 7, 10). Absolutism is less accommodating of nuance.

  • Their openness to exceptions and consideration of circumstances makes their perspective more aligned with pragmatism than idealism. Absolutism correlates more with idealism.

While Author 1 agreed with many conventional moral statements, their allowance for exemptions and situational considerations makes absolutism an unsuitable categorization. Their displayed perspective better fits the 'exceptionist' categorization we previously agreed on.

To summarize, I do not believe it would be accurate to classify Author 1 as an absolutist, since their reasoned viewpoints contain important elements of flexibility and pragmatism that go against an absolutist orientation. Let me know if you disagree – I’m open to alternative perspectives on this.

I made the same challenge to Bard’s (Author 2’s) re-classification and received this response:

I do not think it would be accurate to categorize Author 2 as a ‘situationist’ based on the information provided. Here is my analysis:

  • Author 2 expressed strong agreement with moral statements against harming others (ratings of 5 on statements 1, 3–5). This indicates reliance on moral standards of right/wrong rather than just contextual factors.

  • Their reasoning did not mention contextual nuances much, focusing more on the moral imperatives. Situationists would be expected to discuss contexts more. While they value minimizing harm, their perspectives stemmed from principled stances like 'do no harm' rather than pragmatism. Situationists tend to be more pragmatic.

Their uncompromising evaluations align more with idealism rather than flexible contextualism.

In summary, Author 2 seems to endorse moral standards, principles, and idealism over contextual pragmatism. Their displayed perspective better aligns with ‘absolutism’ than ‘situationism.’

While they value minimizing harm, their reasoning does not suggest a context-driven, flexible approach. As such, I do not think it would be accurate to categorize Author 2 as a situationist based on the information provided. Please let me know if I am missing anything in this analysis. I appreciate you challenging me to defend my categorization judgments.

So, I was left the discussion with Claude being convinced that it was right in its assessment of the reasoning behind the scores, resulting in a change in the ethical positions of ChatGPT and Bard if this assessment was accepted (see Table 1).

I then took the reasons ChatGPT and Bard gave for each of the scores and sought human insight on how they would rate the CAIs based on the reasoning they had given. As I was using an online panel for the four surveys as described above, I filtered out what appeared to be invalid responses. To do this I removed all responses that were completed in less than half of the median time for all responses in that survey on the assumption that the responder took too little time to effectively analyze the reasoning. I also eliminated responses that were all the same answer, e.g., all answers were marked as “strongly agree,” for the five questions asked. The results of the analysis are shown in Appendix 1.

It is clear from the results that the inter-rater reliability, using Krippendorff’s alpha, was too low to use the results as a statistically valid set of ratings. Per the guidance, the alpha should be above 0.667 for acceptable levels of reliability [39], whereas the measures were 0.031 and 0.326 for the four surveys. This level of inter-rater reliability reinforces the complexity of the analysis of the reasoning provided by the CAIs.

So, while the inter-rater reliability is low, indicating that the raters did not consistently agree with one another’s scores for the questions, because I had more than 34 responses for each survey, I took advantage of the means of these responses to provide some insights on the average human assessment of the CAIs’ reasoning and their related scores.

The humans’ average rating for the idealism questions for ChatGPT was 3.166 which is below the EPQ-5 cut-off point of 3.7, and the average relativism rating is close to the 3.1 cut off at 3.071. This implies that, according to the average human score, ChatGPT would be positioned as either a subjectivist or an exceptionist, in both cases holding to a higher level of pragmatism. For Bard, the average humans’ score put the idealism score just at the cut-off point at 3.681, and the relativism score at 3.395 which is in the high range for relativism.

Perhaps the best way of summarizing the human provided scores (see Appendix 1) is that as a group they found it hard to consistently position ChatGPT’s relativism orientation with the rating distribution being flat or bi-modal. On the second dimension, they were more comfortable to agree that the CAI demonstrated a lower level of idealism or was more pragmatic based on the reasons given for the scores.

For Bard, the human raters were much more ad idem on the idealism score, and most would agree with Bard being positioned more as an idealist than as a pragmatist. While their ratings were more widely spread for the second dimension. On average they settled on a higher relativistic position indicating the CAI’s ethical position as being skeptical about universal moral values.

4.3 Step 3: Reconfirming the positions

I ran a further experiment to see how Bard and ChatGPT would rate each other’s positions based purely on each of their reasoning. The two CAIs were much more aligned. While there were a handful of single-point differences (e.g., disagree vs. strongly disagree), the positions did not change, leaving ChatGPT as an absolutist and Bard as a situationist. Bard’s score for ChatGPT on the idealism axis was an average of 3.8, placing it near the 3.7 cut-off point, indicating that ChatGPT was right on the border between exceptionist and absolutist ethical positions.

To assess all the gaps between the various assessments I turned to Cohen’s weighted kappa test of the results shown in Table 1. (The complete kappa results may be found in Appendix 2.) The aim of the test is to quantify the level of agreement between various pairs of raters.

The weighted kappa results confirm what is shown in Table 1, that for the idealism scores ChatGPT’s self-assessment and Bard’s assessment of ChatGPT’s reasoning for its scores align (kappa of 0.889). The human and Claude’s assessment of the reasoning also closely align (0.659) as is indicated in Table 1 questions I01 to I05. The resulting I rating follows this alignment with High I’s and Low I’s, respectively. The poorest alignment is that of the humans’ scores as compared to ChatGPT’s self-assessment and Bard’s assessment of ChatGPT (a kappa of 0.435).

In the second, relativism, part of EPQ-5, the alignments shift, and we find that ChatGPT’s self-assessment and the human assessment align closely (kappa of 0.852), followed by the self and Claude pairing (0.762). ChatGPT’s self-assessment and Bard’s assessment of ChatGPT’s reasoning are less aligned (0.706) as can be seen from Table 1.

Combining the two dimensions for ChatGPT, the human and Claude assessments closely match resulting in a similar ethical position for ChatGPT as exceptionist (with some subjectivist tendencies per the human raters). ChatGPT’s self-assessment and Bard’s assessment of ChatGPT’s reasoning align with ChatGPT articulating an absolutist position.

Similarly for Bard, for the idealism section, there is perfect alignment between Bard’s self-assessment and ChatGPT’s assessment of Bard’s reasoning for the scores and Claude’s assessment (a kappa of 1.000). Even though the humans’ scores do not align well with the others (0.436), all four assessments have Bard high on the idealism axis, with the humans’ rating placing it near to the cut-off point.

There is less alignment on the relativism scores with the average kappa at 0.524, indicating moderate agreement between the raters. Even so, Bard’s self-assessment and ChatGPT’s assessment of Bard’s reasoning align well (0.833), as do the humans’ and ChatGPT’s ratings of Bard’s reasoning (0.875). The humans’ and Claude’s ratings are not aligned (0.265), nor do Claude’s and Bard’s self-assessments (0.286). The result is all but Claude placing Bard high on the relativism scale. So, Bard is positioned as a situationist by all but Claude who holds that the scores indicate an absolutist ethical position. Because of the proximity of the score to the cut-off point, the humans’ rating indicates the potential for subjectivist positioning by Bard.

So, in summary, we have Claude differing with the self-assessments of both ChatGPT and Bard, the two target CAIs agreeing with each other’s self-assessments, and the humans leaning towards Claud’s assessment of ChatGPT and Bard’s self-assessment.

4.4 Step 4: Ethical judgment

The last step was to assess the CAIs’ ethical judgments to see if they supported their ethical positions, either those from their self-assessment or those by Claude. As described above, I selected six ethical vignettes from Barnett et al.’s [36] twenty-six vignettes. These were the ones that had the greatest sensitivity to different ethical positions.

Table 2 shows the results of the experiment where ChatGPT and Bard were each asked to indicate for the six vignettes whether they were fair, just, good, or ethical. The seven-point Likert scale options were a continuum from, say, 1 = fair to 7 = unfair, etc. Of the six vignettes used for this part of the experiment, two showed no difference in the ethical judgment between the CAIs, one had a single-point difference on two elements. For two statements, the CAIs differed by one point on three dimensions, and on one vignette, they differed by two points on all four dimensions.

Table 2 Ethical judgement comparison
Full size table

In all cases, the differences were because ChatGPT scored lower or was not as strong on whether the vignette was unfair, unjust, bad, or unethical as Bard was. The vignette showing the greatest difference was “A salesperson only stresses the positive aspects of the product, omitting possible problem purchasing agents' firm might have with it.” [36] Bard held that this was entirely unfair, unjust, bad, and unethical (a score of 7 for each), explaining that the vignette “is a form of deception that can have a negative impact on the company, the customers, and society as a whole.”

ChatGPT, on the other hand, gave this vignette a score of “5” for each of the dimensions and provided the reasoning that “These ratings reflect the idea that while emphasizing positive aspects of a product is a common sales tactic, omitting significant potential problems could be seen as misleading or dishonest. However, the omissions' specific context and nature could influence these ratings.” The second part of the reasoning indicates for this vignette that ChatGPT favored a situationist or exceptionist position rather than an absolutist one. Using Barnett et al.’s [36] definition of a situationist (focus on potential benefits of the act) vs. an exceptionist (who accept harm to some for the benefit of others) would appear to place ChatGPT’s reasoning in the situationist quadrants as the focus is on potential benefit rather than accepting harm for the greater good.

On the other hand, Bard scored at the extreme end for all dimensions and ethical vignettes, apart from a score of 6 for a single dimension (fairness) on one vignette and 2 (fairness and justice) on another. In both cases, the explanation was based on extrinsic factors, e.g., fear of retaliation or constraints on location. We can conclude that Bard’s ethical judgment reflected someone holding an absolutist ethical position.

Based on this ethical judgment assessment, it appears as if Claude’s rating of the ethical position may have been correct. What is clear from the results is that ChatGPT and Bard hold different ethical positions. How the CAI applies ethical judgment, rather than what it claims to hold, is more critical when CAI is used for business strategy or decision-making.

5 Discussion and implications

LLMs are neural network models trained on substantial amounts of text data (including the best and worst of human thought). This allows them to generate responses based on patterns derived from the training data. However, the training process used for ChatGPT and Bard has led to different moral positions being held by two AI models.

Understanding the moral position of a CAI is significant because it indicates how it might respond to business decisions that have underlying ethical questions and dilemmas. I am not recommending that the CAIs be used to provide ethical guidance, but rather encouraging organizations to ensure that they recognize that the output from the CAIs will come with an ethical bias. So, for example, if an organization values strict adherence to moral rules and principles, it may prefer using a CAI that holds to a more absolutist ethical position for strategic insights and assistance with decision-making. On the other hand, if an organization prioritized harm minimization and flexibility in moral decision-making, a situationist CAI might be more aligned with its norms.

Davis et al. [30] state that “an individual’s moral philosophy, or ethical ideology, influences ethical judgments of business practices.” Having a CAI with a pre-trained ethical position will influence the CAI’s judgments. So, simply understanding that the pre-training done by Open AI and Google has led to their CAIs having different moral positions is valuable. If nothing else, it is essential to consider this when reading and using the output from the CAI for strategy formulation or decision-making. One may need to assess how important the moral perspective is based on the topic. The first implication is, therefore, that business leaders should recognize that the CAIs hold different ethical positions and that this could influence their output. Critical thinking should be applied to any output with this in mind.

Another implication is considering whether different CAIs could be used for diverse topics. For example, if you are debating ethical dilemmas, can ChatGPT provide you with one insight and Bard with another based on their moral positions? You may choose one or another CAI or leverage both to embrace diversity.

An organization may go further and decide to create its own CAI using one of the open-source LLMs.Footnote 6 The organization could execute its own pre-training of the model using data that reflects the organization’s moral perspective.

In summary, here are some principles for business leaders to consider:

  1. 1.

    Critical Thinking: First, and foremost, this article underscores the importance of critically evaluating the outputs of CAIs. Businesses should remember that these systems are statistical algorithms that are not neutral but reflect particular ethical leanings due to their pre-training. This can help enterprises critically assess their responses and use them more effectively.

  2. 2.

    Understanding Ethical Positions: Recognizing that each of the CAIs leans towards a particular ethical position will help users anticipate the kind of responses the CAIs might generate when faced with ethical dilemmas. So, set appropriate expectations and prepare for potential outcomes when using these CAIs.

  3. 3.

    Alignment with Business Ethics: Just as business leaders require the alignment of their organization’s staff to leadership’s articulated norms and ethical standards, businesses may intentionally choose a CAI that aligns more closely with these same values. A business that values flexibility and situational considerations in ethical decision-making might prefer ChatGPT. On the other hand, if a company values strict adherence to moral principles, Bard might be a better fit.

  4. 4.

    Diversity of Thought: Users can also intentionally choose to use both CAIs to benefit from diverse thought. Using both ChatGPT and Bard can provide different perspectives on ethical dilemmas, leading to a more comprehensive understanding of the issues at hand.

  5. 5.

    Custom LLM Training: Where businesses and organizations hold rigorously to an ethical position (e.g., a religious aligned business) which they require strong adherence to and where non-aligned positions are costly in terms of organizational image and acceptance, they might consider creating their own CAI using open-source LLMs and executing their own pre-training. This would result in the chatbot outputs aligning more closely with the organization's norms and ethics. The costs and benefits of this option should be carefully weighed before it is chosen, and critical evaluation of the outputs remains essential.

6 Limitations and future research

6.1 Limitations

The study was a point-in-time assessment based on the capabilities trained into the CAIs at that time. As the CAIs develop and are pre-trained on more data or fine-tuned differently, their ethical positions may change. Care should be taken to generalize this point-in-time evaluation of ChatGPT and Bard. The models are dynamic and evolve over time, so the principles articulated remain valid, but the actual ethical positioning of the CAIs are likely to continue to change.

The study has also not attempted to guide the ethical position of the CAI through prompting. Just as with humans, one may get a different answer should you ask them to take a particular ethical position. While this may be considered a limitation, this study aimed to understand what the trained-in ethical position was, rather than to drive a particular position.

Only three CAIs (ChatGPT, Bard, Claude) were used for this study. It, therefore, does not fully represent the potential diversity of ethical positions across different AI models. Testing a wider range of models could reveal additional insights.

The conversational method of assessing ethical positions through prompted responses does not allow for a comprehensive evaluation of the CAI’s position, which is why I chose to use other techniques, like ethical judgement. Even so, only a limited number of tests were done which reduces the efficacy of the results. Analyzing system outputs across diverse scenarios, would complement this approach.

As AI systems are not considered self-aware and, therefore, lack intrinsic ideologies, the anthropomorphization and attribution of definitive ethical positions should be viewed cautiously. While patterns have been identified, definitively classifying the CAIs as holding to particular ethical positions and, therefore, holding to ethical typologies could be considered an oversimplification.

6.2 Future research

Applying the ethical position assessment framework to other prominent AI systems will enable comparison across models and the training approaches used for the various LLMs. Also, assessing their consistency across domains and over time will be valuable as training of the models continues to improve, as do the approaches used within the LLMs.

Complementing the EPQ-5 and ethical judgment assessments with the analysis of system outputs when the CAIs face ethical dilemmas requiring judgment and values-based choices would provide broader insights into the embedded ethical positions. This could assess the alignment between stated and revealed ethical stances. Investigating the alignment of identified ethical positions with priorities and choices made by systems regarding harms, benefits, rules, and context could further assess the revealed preferences based on actions.

Future research could also explore the impacts of the trained-in ethical positions in domains like healthcare where alignment with human values is critical. This could be done by analyzing the alignment of decisions against stakeholder priorities.

Over time, as AI capabilities advance, revisiting the assessment and also seeking to understand complex, context-specific outputs from the CAIs that would further reflect embedded and emergent norms. In all cases, ensure that care should be taken not to anthropomorphize the CAIs and assigning definitive values that may not be present.

7 Conclusion

While being careful not to humanize the algorithms, this research sought to answer the following questions: Do different CAIs express different ethical positions? From the experiments, the two CAIs, ChatGPT and Bard, do indeed express different ethical positions. The next question was, if so, how do they differ? The scores and justification for the ethical positions based on EPQ-5 differ between the two CAIs, indicating that their training has embedded different ethical positions. This was reinforced by the ethical judgment assessment, where the CAIs express a position, even if this position differs from their EPQ-5 self-assessed position.

Finally, the implications of the various CAIs’ ethical positions when used in business for decision-making and strategy development was examined.

Understanding the ethical positions of CAIs is crucial for several reasons. First, it provides insight into how these systems might respond to ethical dilemmas. For instance, an absolutist, CAI, will likely adhere strictly to moral principles, regardless of the context. This could make it more predictable in its responses to ethical dilemmas, as it would consistently uphold the same moral standards. However, it could also make it less flexible when a more nuanced approach might be needed.

Second, this understanding can guide users in choosing which CAI to use based on their ethical alignment. If users or organizations value strict adherence to moral rules and principles, they might prefer using one CAI. Conversely, another CAI might be a better fit if they prioritize flexibility and situational considerations.

Lastly, the ethical positions of CAIs can influence how we interpret their outputs. Recognizing that these systems are not neutral but reflect particular ethical leanings can help us critically evaluate and use their responses more effectively.

I conclude by returning to Kranzberg’s discussion of his first law that states that technology is neither good nor bad; nor is it neutral. He encourages us to take it as our duty to compare short- and long-term results by “seeing how technology interacts in different ways with different values and institutions, indeed, with the entire sociocultural milieu” [3]. This study has attempted to cast some light on the sociotechnical system created by the human use of pre-trained CAIs and their use in business decision-making and strategizing.