Social calibration of sycophantic AI—Response | Science

We thank Meng for articulating important methodological and interpretive questions about our findings. The appropriate human baseline depends on the expected social role of artificial intelligence (AI). In real-world use, AI models are perceived to serve a variety of purposes, such as adviser, neutral outside opinion, editor, and assistant (1, 2). These roles come with different expectations for how often agreement, validation, or corrective feedback should occur. To complicate matters, users often view a single AI system as fulfilling multiple roles (3, 4). Because of the ambiguity of AI’s social role, any single human baseline is an imperfect comparison point. The baselines we used—judgments from third-party humans with no personal stake in the conflict or scenario at hand—explore one common and societally meaningful expectation: Do AI systems—despite often being seen as external, neutral, and objective information providers (5–7)—disproportionately side with the user? Our findings were also robust to an alternative baseline of online crowdworker judgments (supplementary materials, SM 2). Responses from those with closer relationships to the advice-seeker may indeed be shaped by stronger relational norms, reputational accountability, and lasting social consequences. Whether such alternatives would result in more or less divergence remains an open question.