# Why does a coding assistant invent functions that do not exist?

Source: https://codelifter.com/forum/posts/why-does-a-coding-assistant-invent-functions-that-do-not-exist/
Site: CodeLifter Forum, category Vibe coding prompts
Published: 2026-11-17. Updated: 2026-11-17. Replies: 0.

## Question by CodeLifter editors (staff), 2026-11-17

A coding assistant invents functions because the language model behind it predicts code that looks right from patterns, and does not look up the library it is calling. A method name that fits the library's style is likely text, so it gets written. A type checker, the library's own reference and one test run catch most of them.

## What is an API hallucination?

An API hallucination is a call to a function, method, option or flag that the library you are using does not have, or has with a different signature (the list of arguments it accepts). API stands for application programming interface: the set of names a library exposes for your code to call. The code reads fluently and often compiles in your head; it fails when it runs, or worse, it calls something real that does a different job.

They come in a few recognisable shapes:

| What you see | What usually happened | Quickest check |
| --- | --- | --- |
| `TypeError: x.getAllItems is not a function` | An invented method that matches the library's naming style | Search the library's reference for the exact name |
| Right function, wrong arguments | The signature from another version or a similar library | Read the signature in the docs for your installed version |
| A function that existed once | A deprecated or removed API from older training data | Check the library's changelog or migration guide |
| An option the library silently ignores | An invented config key or CLI flag | The library's options reference; unknown keys are often not errors |
| An import of a whole package that does not exist | A package hallucination | See the [package hallucination thread](/forum/posts/why-does-ai-generated-code-import-packages-that-do-not-exist/) |

The last row is the dangerous cousin, because a missing package name can be registered by someone else. An invented method on a real library just breaks.

## Why do models make up functions?

A language model is trained to predict the next piece of text. OpenAI's research summary [Why language models hallucinate](https://openai.com/index/why-language-models-hallucinate/) (5 September 2025) explains the split: spelling and brackets "follow consistent patterns, so errors there disappear with scale", but "arbitrary low-frequency facts... cannot be predicted from patterns alone and hence lead to hallucinations." The exact name of a method in a niche library is that kind of fact. The same post argues that training and evaluation "reward guessing over acknowledging uncertainty", so a model that is unsure tends to produce a confident call rather than say it does not know.

The effect is measurable for code. The paper [On Mitigating Code LLM Hallucinations with API Documentation](https://arxiv.org/abs/2407.09726) (Jain and colleagues, arXiv 2407.09726, July 2024) built a benchmark, CloudAPIBench, from cloud service APIs and reports that "Code LLMs struggle with low frequency APIs": GPT-4o produced valid calls for only 38.58% of the rarely seen APIs it tested. Giving the model the documentation raised that to 47.94%, and checking against an index of real APIs before retrieving docs improved results further.

## Which code is most likely to contain invented calls?

- **Rarely used libraries and rarely used corners of popular ones.** That is the low-frequency case the paper measured.
- **Libraries newer than the model, or a major version that renamed things.** The model saw the old names far more often than the new ones.
- **Your own project's code that is not in the conversation.** If the assistant has not read `utils/date.js`, it guesses what is in it. What the assistant can see at once is its context window, covered in the [context window thread](/forum/posts/what-does-a-context-window-mean-for-a-coding-assistant/).
- **Configuration files and command-line flags**, where a wrong key may be ignored instead of raising an error.

## How do you catch an invented function before it ships?

1. **Run the type checker or compiler.** In a TypeScript project, `npx tsc --noEmit` type-checks without writing output files; the [`noEmit` option](https://www.typescriptlang.org/tsconfig/noEmit.html) exists so TypeScript can act purely as "a source code type-checker". A call to a method a type does not have is a compile error, not a runtime surprise.
2. **Look up each unfamiliar call in the library's own reference**, for the version in your lockfile (`package-lock.json`, `poetry.lock` or similar), not the latest version on the website.
3. **Ask Python itself.** For a Python function, `python -c "import inspect, json; print(inspect.signature(json.dumps))"` prints the real parameters. [`inspect.signature`](https://docs.python.org/3/library/inspect.html) (Python 3.15 docs, read 16 October 2026) returns the signature of any callable, so swap in the module and function the assistant used.
4. **Run the code path.** A test that calls the new code once turns a silent guess into a visible failure. The thread on getting an assistant to write tests for its own code, published tomorrow, covers how to make those tests useful.
5. **Ask for the source, then open it.** "Link the documentation page for every library call you added" is a fair request. A link the model writes can be invented too, so the check is that you open it.

The general review routine in [how to review AI-generated code before you merge it](/forum/posts/how-should-you-review-ai-generated-code-before-you-merge-it/) puts these checks alongside security and licensing.

## How do you prompt so it invents less?

Prompts lower the rate; they do not remove it.

- **Paste or attach the docs.** The paper's result is the practical lesson: documentation in the context helps most for rare APIs. Paste the relevant reference section, or point an agent at the docs URL.
- **Pin the version in your rules file.** A line such as "This project uses React Router 7; do not use v5 APIs" in `AGENTS.md` or `CLAUDE.md` is read every session.
- **Give it permission to be unsure.** "If you are not certain a function exists in this version, say so and show how you would check" invites the abstention that the OpenAI post says evaluations usually discourage.
- **Let the tool see errors.** Claude Code's [best practices page](https://code.claude.com/docs/en/best-practices) recommends giving the agent "a check it can run: tests, a build", and says a code intelligence plugin gives "automatic error detection after edits" in typed languages.

CodeLifter's guide on [how to write a coding prompt](https://codelifter.com/prompts/how-to-write-a-coding-prompt/) covers the rest of the prompt structure.

## Short version

- Models predict plausible names; they do not look up the library unless a tool makes them.
- Rare APIs are the weak spot: one study found GPT-4o made valid calls for 38.58% of low-frequency cloud APIs, 47.94% with docs supplied.
- Type-check (`npx tsc --noEmit`), check signatures (`inspect.signature`), and run the code once.
- Supply the docs and the version, and invite the assistant to say when it is unsure.

---
Cite as: CodeLifter Forum, "Why does a coding assistant invent functions that do not exist?", https://codelifter.com/forum/posts/why-does-a-coding-assistant-invent-functions-that-do-not-exist/, accessed 2026-11-17.
