The 'data engine' idea of defensibility in AI may not be as defensible as we thought:
In SELF-INSTRUCT, authors get GPT-3 to generate it's *own* dataset for instruction tuning, outperforming vanilla GPT-3 and comparable to InstructGPT.
arxiv.org/pdf/2212.10560…
Here's how 👇
