Most myth lists correct a belief by asserting the opposite, which is not much of an improvement. These six are each contradicted by something the companies involved have published themselves, so every correction below comes with a source and the date it was written.
Three of them are also the most common reasons people conclude AI “doesn’t work” after a bad week with it. The tool was fine. The expectation was wrong.
Myth 1: hallucinations are a bug that will be patched out
The belief: this is an early-days problem. A version or two from now, models will stop making things up.
What the source says: OpenAI published a piece called Why language models hallucinate arguing that the behaviour follows from how models are trained and graded, not from a defect in a particular model. When a model is scored only on how many answers it gets exactly right, guessing beats admitting ignorance. Asked for a birthday it does not know, a guess has some small chance of being right; “I don’t know” scores zero every time. Across thousands of benchmark questions, the guessing model posts the better number.
What is true instead: confident wrong answers are a predictable output of the current scoring regime, and the fix OpenAI points at is changing the benchmarks to reward saying “I don’t know” — an industry-wide change, not a patch. Treat fabrication as a permanent property to design around. Why AI makes things up covers the mechanism.
Myth 2: a bigger context window means it remembers your project
The belief: the model has a large context window, so it now knows the documents and conversations you gave it last week.
What is true instead: the context window is how much text the model can consider in a single request. It is a working desk, not a filing cabinet, and it is cleared between conversations. The “memory” features in consumer chat products are a separate layer that stores selected notes and feeds them back in — useful, but a different mechanism with much smaller capacity than the window itself.
Two practical consequences. A long conversation does not keep everything from the start of it equally available. And nothing you said in a different conversation is there unless a memory feature put it there. Why AI forgets mid-conversation has the detail and the current numbers.
Myth 3: fine-tuning is how you teach it your company’s facts
The belief: to get a model that knows your products, policies and prices, you fine-tune it on your documents.
What the source says: OpenAI’s own fine-tuning guide describes the result of supervised fine-tuning as “a customized model that more reliably produces your desired style and content” — form and behaviour, listing uses like classification, translation and generating content in a specific format. It is not described as a way to install facts.
What is true instead: fine-tuning adjusts how the model answers. Getting it to know your material means putting that material in front of it at the time of the question, which is retrieval. Fine-tuning on a document dump is an expensive way to produce a model with your tone of voice and the same gaps in knowledge. Fine-tuning vs RAG vs prompting works through which symptom needs which fix.
Myth 4: an AI detector can prove someone used AI
The belief: paste text into a detector and it will tell you, reliably enough to act on, whether a machine wrote it.
What the source says: OpenAI built one of these and withdrew it. Its AI Text Classifier launched on 31 January 2023 and was shut down on 20 July 2023 over what the company called a low rate of accuracy. By its own published figures, on a challenge set of English text the classifier correctly identified 26% of AI-written text while labelling 9% of human-written text as AI-generated.
What is true instead: the company with the most direct access to how its own model writes could not build a detector it was willing to stand behind, and pulled it. Those numbers are from 2023 and detectors have been marketed hard since, but no vendor has published evidence of the accuracy that accusations require. A false-positive rate anywhere near 9% means that in a class of a hundred honest students, several are accused. Detector output is not evidence.
Myth 5: everything you type into ChatGPT trains the model
The belief: paste a client document into ChatGPT and it is absorbed into the next version for everyone.
What the source says: it depends entirely on which product you are using, and both halves are documented. OpenAI’s help article on how your data is used states that for services for individuals such as ChatGPT and Codex, “we may use your content to train our models”, with an opt-out: turn off “Improve the model for everyone” under Settings, Data controls, or select “Do not train on my content” in the privacy portal. For business products the default is the other way round — the same article says OpenAI does not use inputs or outputs from ChatGPT Business, Enterprise, Edu or the API to improve its models, and the enterprise privacy page, updated 8 January 2026, states: “We do not train our models on your data by default.”
What is true instead: consumer tiers train on your conversations unless you turn it off; business tiers and the API do not unless you turn it on. And there is a detail worth knowing, because it catches people who think they have handled this: the same help article says that if you submit feedback on a response — a thumbs up or thumbs down — “the entire conversation associated with that feedback may be used to improve our models, even if you’ve opted out.” Temporary chats are excluded while they stay temporary.
So the myth is wrong, the reassurance is also wrong, and the answer is a setting you should go and look at.
Myth 6: reasoning models think before they answer
The belief: a reasoning model deliberates in something like the way a person does, which is why it gets harder questions right.
What is true instead: what these models do is generate more text before the final answer — working through intermediate steps in tokens, which gives the model more opportunity to catch its own errors and tends to improve results on maths, code and multi-step problems. That is a real and measurable improvement. It is also not introspection, and the visible “thinking” is not a reliable account of how the answer was reached: a model can show plausible working and still arrive somewhere wrong, or show working that does not match what actually drove the output.
Useful rule: treat the reasoning trace as more output to check, not as an explanation you can trust. What AI “thinking” actually is goes further into it.
The pattern behind all six
Five of these six myths are the same mistake in different clothes: assuming the model has a property it is merely imitating. Memory, knowledge of your business, deliberation, honesty about uncertainty — in each case the output looks like the product of that property, and in each case the underlying mechanism is something else.
Which gives you a usable heuristic. When a model appears to have done something that would require remembering, knowing or deciding, ask what it would be doing instead if it were only producing convincing text. Usually that is the answer, and it tells you where to put a check. If you want one habit rather than six facts, that is the verification routine.