Soft Contamination Means Benchmarks Test Shallow Generalization

Study number couple-of-hundred by now showing that generative AI models are lossy storage and that benchmarks primarily measure storage and retrieval performanc...