History shows prediction capability doesn’t eliminate need for theory.
In 2008, excited from Google’s ability to predict the next click, WIRED magazine proclaimed ‘the end of theory,’ saying that all that was needed is data. That proved to be wrong. Now we have better models for the same approach with transformers doing next-token prediction, but we still have not seen the end of theory. This parallel shows that impressive prediction capabilities, whether for clicks or tokens, don’t replace the need for underlying theory and understanding. It’s a reminder that we’ve been at this place before with prediction hype.
— Healthcare’s AI Lesson: Autocomplete Isn’t Understanding · Forbes