— Gary Marcusthe LLM doesn’t maintain structured symbolic systems like databases; it has no direct database of cities and populations. Aside from what it can retrieve with external tools like web searches (which provide a partial but only partial workaround) it just has a bunch of bits of text that it stochastically reconstructs. And that is why they hallucinate, frequently. They wouldn’t need to do so if they actually had reliable access to relevant databases.
Replicated under Fair Use from Generative AI’s crippling and widespread failure to induce robust models of the world by Gary Marcus.