Researchers introduced multilingual story moral generation as a novel culturally grounded evaluation task to assess the cultural alignment of large language models (LLMs). They compared model outputs with human interpretations, finding that while LLMs can approximate central tendencies of human moral interpretation, they struggle to reproduce cross-linguistic variation and diverse values. This study suggests a new approach to studying cultural alignment in LLMs beyond static benchmarks or knowledge-based tests.

Researchers proposed two live streaming speech synthesis evaluation methods, Live-ProsodyJudge (LPJ) and Decoupled-Live-ProsodyJudge (D-LPJ), to assess fine-grained prosody such as emotion, intonation, and energy. LPJ and D-LPJ outperform a single proprietary Large Language Model (LLM) call in evaluation accuracy and demonstrate efficacy in fine-grained TTS preference optimization.

Researchers found that calibrated vision-language models can still report high confidence in incorrect answers due to trajectory-independent verbalized confidence. They propose the Trajectory-Grounding Score (TGS) to address this issue, which evaluates a model's confidence based on its reasoning trajectory. This discovery highlights a blind spot in current evaluation practices for AI models.