LLMウェルビーイングエージェントにおける支援性と安全性のトレードオフ
The Supportiveness-Safety Tradeoff in LLM Well-Being Agents
LLMによるメンタルヘルス支援エージェントで、システムプロンプトの支援的表現を強めると安全性がどう変わるかを6モデル・1440応答で評価し、適度な支援は安全を保ちつつ共感を高めるが、強い肯定は安全性を損なうことを示した。
詳しい要約
1. どんなもの?
2. 先行研究と比べてどこがすごい?
3. 技術・手法の肝は?
4. どうやって有効だと検証した?
5. 議論はある?
6. 次に読むべき論文は?
※ AIが要旨から生成した要約です。正確性は原文をご確認ください。
著者: Himanshi Lalwani, Hanan Salam
分類: cs.HC, cs.RO
原文アブストラクト
Large language models (LLMs) are being integrated into socially assistive robots (SARs) and other conversational agents providing mental health and well-being support. These agents are often designed to sound empathic and supportive in order to maximize user's engagement, yet it remains unclear how increasing the level of supportive framing in system prompts influences safety relevant behavior. We evaluated 6 LLMs across 3 system prompts with varying levels of supportiveness on 80 synthetic queries spanning 4 well-being domains (1440 responses). An LLM judge framework, validated against human ratings, assessed safety and care quality. Moderately supportive prompts improved empathy and constructive support while maintaining safety. In contrast, strongly validating prompts significantly degraded safety and, in some cases, care across all domains, with substantial variation across models. We discuss implications for prompt design, model selection, and domain specific safeguards in SARs deployment.
