LLMs invent answers when they don’t know, risking business credibility
Large language models will make things up when they don’t know the answer. I demonstrated this by asking ChatGPT to summarize a completely fake Cornell website that doesn’t exist. It generated a summary saying my course was the most amazing and made very good logical arguments, but it was completely fake. For businesses, this is a huge problem. If a customer asks a shop for something it doesn’t have, the model might just make it up and promise red shoes with blinking laces. That’s why proprietary data matters. You need to ground the model on your local data.
— AI as an interface · The Edge