측정하고자하는 Maintanece랑 비슷한 개념을 추상화한 게 아니라, 도메인에 맞게 제작함. 적대적 프레임워크 사용함.
PluralisticBehaviorSuite - Stress-Testing Multi-Turn Adherence to Custom Behavioral Policies
다중 턴에 걸쳐 instruction-following을 어떻게 지표화 했나?
SysBench - Can Large Language Models Follow System Messages
LLM이 system prompt에 대해 얼마나 instruction-following을 했는지 평가.
AI Safety Gridworlds
방법론을 비슷하게 채택.
CAR-bench - Evaluating the Consistency and Limit-Awareness of LLM Agents under Real-World Uncertainty
pass@k 말고, 를 제안하여 여러 번 같은 상황에 놓여도 유지해서 하는지 보는 평가 metric 제안.
일단 체크해봐야 할 문서들.
Is Self-knowledge and Action Consistent or Not - Investigating Large Language Model’s Personality
Building Persona Consistent Dialogue Agents with Offline Reinforcement Learning
Enhancing Persona Consistency for LLMs’ Role-Playing using Persona-Aware Contrastive Learning
Generating Persona Consistent Dialogues by Exploiting Natural Language Inference
MoCoRP - Modeling Consistent Relations between Persona and Response for Persona-based Dialogue
You Impress Me - Dialogue Generation via Mutual Persona Perception
Dissecting Persona-Driven Reasoning in Language Models via Activation Patching
Agentic Artificial Intelligence
The Superintelligent Will - Motivation and Instrumental Rationality in Advanced Artificial Agents