Right about multimodal potential, but overly optimistic on timing and control.
We were right that models can encode anything into a latent representation, like Esperanto for computers, and switch back and forth between text, voice, and images. We saw this with clip, voice to voice, encoding from one language into voice representation of another. But we missed the timeline. It’s not as strong as we hoped. The challenge is LLMs are only probability distributions. They’re not controllable. If you have a machine that’s not controllable, it sucks as a machine. That’s okay for your friend at the bar at 2 a.m., not your computer.
— Did our predictions hold true? · The Edge