offline human alignment

marketplace loop

Business-level A/B results and human-labeled offline evals are two layers of the same eval stack and must be reconciled when they disagree.

1 source videos