Self-Evaluation can Help Agents Meet Social Expectations

Abstract

As artificial intelligence (AI) and multi-agent systems (MAS) become increasingly advanced and integrated into real-world applications, they are frequently used for content generation, problem-solving, and interactive communication. However, these systems still lack reflective capabilities, the ability to self-evaluate and reason about their own decisions. In particular, large language models (LLMs) often exhibit surface-level reflection that is more a product of linguistic prediction than reasoning. In this paper, we implement a case study on a recently proposed architecture designed to equip AI systems with reflective and self-evaluation capabilities. We implement and evaluate this architecture using a widely accessible LLM, Llama3, as the test bed and compare its baseline performance with the same model enhanced by this reflective architecture. To test the proposed system’s ability to navigate social norms, we designed a normsensitive scenario involving a surprise birthday party. The model was prompted with 30 realistic questions that the guest of honor might ask, and its responses were evaluated across four metrics. The self-evaluator module is implemented using a second LLM to assess whether the base model’s response aligns with defined norms and expectations. If not, the prompt is revised and reevaluated in an iterative loop. Experimental results show that this reflective setup improves the model’s compliance with social and common-sense expectations, without requiring additional training or complex prompt engineering.

Publication
In 2025 IEEE International Conference on Autonomic Computing and Self-Organizing Systems Companion (ACSOS-C) (pp. 1-7). IEEE