Stop using generative AI as a search engine

This article is great at noting how ridiculous and harmful the framing of products like ChatGPT. I also noted on Bluesky:

Watch this "magic". If you frame the question to get what you want, it gives it to you, true or not.

The thing about ChatGPT (and the rest of the generative AI systems) is that it is basically--at a fundamental level--complicated autocomplete. It isn't giving motivated or biased answers, it is just looking for the most likely completion text for your question which, unlike news headlines, often gives you the answer you want instead of the answer that is correct. This isn't just a design choice for ChatGPT (though it is partially that) but also a result of sites building pages around SEO goals for more than a decade. It used to be that the bad--but well SEO optimized--answers would be excluded by the weight of links to the right answer and moderation action against fraudsters and checks against exploitative behavior by site builders. ChatGPT has none of that data involved in how it responds, it just finds the words in a multidimensional space closest to yours by drawing a line through its massive data set. That massive collection of data is a scrape of the whole web and therefore includes all those fraud sites built to try and get your click.

By the way, this problem is also one of the many reasons why Google Search increasingly sucks now, the algorithm for sorting results has increasingly leaned on click-through and time-to-return-to-search-page as value signals for site. That means that a site that gives you a concise, simple, correct answer gets devalued compared to a site that forces you to click through 5 ads to get a self-satisfying distraction. Like GenAI, modern Google Search uses human behavior but fails to account for or understand it.

Technology that actually serves people takes into account human behavior. Maybe there is a way to make generative AI useful, but in its current state, I feel tremendously sorry for anyone gullible enough to use it as a research tool.