— Gary Marcuspeople who have built neural networks such as LLMs have mostly tried to do without explicit models, hoping that intelligence would “emerge” from massive statistical analyses of big data. This by design. As a crude first approximation, what LLMs do is to extract correlations between bits of language (and in some cases images) - but they do this without the laborious and difficult working (once known as knowledge engineering) of creating explicit models of who did what to whom when and so forth.
Replicated under Fair Use from Generative AI’s crippling and widespread failure to induce robust models of the world by Gary Marcus.