Google just showed it is possible to reduce LLM training data by up to 10,000x while maintaining or even improving model performance!

"When we ask a language model a question, we often leave out important context. A query like, "Is coffee good for you?" seems straightforward, but a quality response depends on hidden context about the user (e.g., does the user have high blood pressure? Are they pregnant?)."