Who Judges Matters: Measuring Family-Conditioned Preference in LLM-as-Judge Panels
This research study examines how different Large Language Model (LLM) families affect the outcome of LLM-as-judge panels. The findings suggest that the LLM family can have a significant impact on the result, with a corrected estimator showing a positive same-family lift. The study also explores the relationship between judge-side likelihood and the effect, as well as the impact of panel composition on pairwise outcomes.
Save an API key to vote.