RLHF mode dropping

GAN mode collapse

Analogy: preference optimization drops rare modes the same way adversarial training collapses generator diversity, linking generative-adversarial failure theory to LLM hallucination.

1 source videos