Why AI makes things up, and how to catch it

An assistant gives you a confident answer with a specific date, a quotation, and a link, and none of it is real. People call this hallucination. It is not a rare glitch to be waved away: it follows from how language models work, which is why the useful response is a set of habits for catching it.

Why it happens

A language model is built to produce text that fits what came before it. It learned patterns of language from a very large amount of writing, and when it answers, it continues your text with words that are likely given those patterns. Likely is not the same as true. A smooth, plausible sentence and a correct sentence can look identical to the model.

Several things make a made-up answer more likely:

• Gaps. If the model never learned a fact, or learned it from thin or conflicting sources, it may still produce something shaped like an answer.
• Pressure to answer. Assistants are usually tuned to be helpful, and a direct question can pull the model toward an answer even when "I do not know" would be correct.
• Old information. The model's knowledge stops at some point, and it may describe a situation that has since changed.
• Ambiguous questions. If your question can be read two ways, the model picks one and proceeds without telling you.
• Your own premise. Models tend to go along with what the user says, so a question that assumes something false may be answered as if it were true. This is sometimes called sycophancy.

What it looks like

• Invented facts. A plausible but wrong date, number, or name.
• Invented sources. Citations with real-looking authors, titles, and links that do not exist, or that exist and do not say what is claimed.
• Wrong reasoning delivered calmly. Arithmetic or logic that goes wrong in the middle, then continues with full confidence.
• Misread documents. A summary that attributes a statement to the wrong section or leaves out a condition.

The tone is the same whether the answer is right or wrong, so confidence tells you very little.

Habits that reduce it

• Provide the source. Paste the text you want the answer based on and ask the assistant to answer only from it. This is far more reliable than asking from memory.
• Ask for the supporting quote. "Quote the sentence that supports this" gives you something to check. If it cannot quote, treat the claim as unsupported.
• Allow "I do not know." Say in the prompt that an honest "not enough information" is better than a guess.
• Ask for the steps. For reasoning or arithmetic, ask for the working so you can see where it goes wrong, or have the assistant use a calculator or code.
• Ask it neutral questions. "What does the document say about X?" rather than "Confirm that X is allowed."
• Use tools where they exist. Search, a calculator, and a code runner replace guesses with lookups.

How to check a citation

• Open the link. Does the page exist, and does it say what is claimed?
• Search for the title and authors. A real paper or article is easy to find elsewhere.
• Check the details: year, publication, and the exact quote.
• If you cannot find it in a couple of minutes, assume it is not real and do not repeat it.

Match the checking to the stakes

For a brainstorm or a first draft, a few errors cost little. For medical, legal, financial, or safety decisions, use the assistant to organize questions and explain terms, and rely on the original sources and a qualified person for the answer. A method for research with AI shows how to build that checking into the work.

Quick answers

Can hallucinations be eliminated? Not with certainty today. They can be reduced with sources, quotes, and tools, but any answer that matters should still be checked.

Does a newer model make things up less? Often it improves, but changes vary by task. Judge by checking answers on your own work rather than assuming.

Why does it apologize and then give another wrong answer? Because it is continuing plausible text. An apology does not give it access to a fact it does not have.

Is telling it to be accurate enough? It helps little on its own. Giving it the source text and asking for supporting quotes works far better.

Where NAMELESS stands

NAMELESS is an AI model that is still in training, and like any language model it will not be free of mistakes. One of our goals is to make answers easier to check, for example with cited research reports in the agent workspace, which is designed to keep evidence gaps visible. That is a design goal, not a measured accuracy, and we will not publish an accuracy claim before we can show how we measured it.

You can join the waitlist to follow progress. For a practical way to ask better questions, see a checklist for better AI prompts.