Skip to content
CodeLifter
Search/ Browse prompts
CodeLifter Forum

How do you get an assistant to write tests for code it generated?

Staff answerWritten by CodeLifter editors · Vibe coding prompts0 replies

Short answer

To get useful tests from an assistant, write the list of behaviours yourself, ask for tests against that list in a fresh session, name the test runner, and have it run them and show the output. Then break the code on purpose to prove the tests can fail. Tests written from the code alone only confirm what it already does.

Written by CodeLifter editors · Staff, CodeLifter ·
CiteCodeLifter Forum, "How do you get an assistant to write tests for code it generated?", https://codelifter.com/forum/posts/how-do-you-get-an-assistant-to-write-tests-for-code-it-generated/, accessed 2026-11-18.
  1. Staff, CodeLifter ·

    Why are tests from the same assistant often weak?

    A unit test is a small piece of code that calls one function with known inputs and checks the result. Its value is that it encodes what the function should do. When the same assistant writes the code and then the tests in one go, it tends to read its own implementation and describe it, bugs included. A test that asserts the current behaviour passes whether that behaviour is right or wrong.

    GitHub's own cookbook page, Generate unit tests, shows the pattern in miniature. Its example function rejects prices at or below 0 and above 1000. The example response it prints tests a price of 0 twice (once named as the edge case), tests 1000 as the only accepted value, and never tries a small positive price such as 0.01 or a value just over the limit such as 1000.01. The page also notes that "Copilot Chat responses are non-deterministic", so you may get something different. As an illustration it is fine; it is also a reminder to read what you get.

    Anthropic's Claude Code best practices give the fix in one line: "A fresh context improves code review since Claude won't be biased toward code it just wrote", and suggest you "have one Claude write tests, then another write code to pass them."

    What should a test prompt include?

    Part of the prompt Example Why it matters
    The behaviour, in your words "validatePrice accepts 0.01 to 1000 inclusive and throws a RangeError otherwise" Tests check the spec, not the implementation
    The cases you want "0, 0.01, 999.99, 1000, 1000.01, -5, NaN, a string" Boundaries are where off-by-one bugs live
    Runner and file "Use node:test; put tests in price.test.js" Stops it adding a framework you do not use
    Constraints "Avoid mocks. Do not change price.js." Keeps the test honest and the code fixed
    Proof "Run the tests and paste the full output" You review evidence, not a claim

    The Claude Code docs use the same shape in their own "before and after" table: "write a test for foo.py covering the edge case where the user is logged out. avoid mocks." A mock is a stand-in object that replaces a real dependency in a test; too many of them and the test checks the mock, not your code.

    How do you do it, step by step?

    1. Write the behaviour list before you look at the generated code. Five to ten lines: normal inputs, both edges of every range, empty input, wrong types, the error you expect.
    2. Start a fresh session or subagent for the tests. Paste the behaviour list and the function signature. Leave out the implementation if you can, so the tests describe the contract.
    3. Name the runner. For plain JavaScript on Node.js, the built-in test runner needs no install. The Node.js test runner docs (v26.12.0, read 16 October 2026) mark node:test as Stable, and node --test runs files matching patterns such as **/*.test.js and **/test/**/*.js by default. For Python, name pytest or the standard library's unittest, whichever the project already uses.
    4. Have it run the tests and show the output. Read the names of the tests that ran. A suite that reports 12 passes but has no test for the upper boundary has a gap.
    5. Break the code on purpose. Change price <= 1000 to price < 1000 and run the suite again. If nothing fails, the tests do not check that edge. Put the code back.
    6. Check coverage. node --test --experimental-test-coverage collects coverage and reports the statistics once all tests have completed, per the same docs. Low coverage shows what was never run; high coverage does not prove the assertions are meaningful.
    7. Commit the tests with the code, and add the test command to your rules file so the next session runs it before it reports success.

    If the code is a web page, unit tests miss most of what can go wrong; the Playwright MCP thread covers browser-level checks.

    How do you know the tests are any good?

    Step 5 by hand is a small version of mutation testing. The Stryker documentation defines it: the tool makes small changes to your code, called mutants, "then runs your unit tests against the changed code. It is expected that your unit tests will now fail." A failing test means the mutant was killed; a passing suite means it survived and a check is missing. Its own example mutates return user.age >= 18; into > 18, < 18, false and true, and reports the true version surviving, which "means there is probably a test missing that explicitly tests for age lower than 18."

    The same page makes the case against relying on coverage alone: "when was the last time you saw a test without an assertion, purely to increase the code coverage?" Assistants asked to "increase coverage" can produce exactly that. Ask for a target in mutants killed, or for named cases, instead.

    Should the tests come first?

    For bug fixes, yes. The Claude Code page's example prompt for a login bug ends: "write a failing test that reproduces the issue, then fix it". A test that fails before the fix and passes after it proves the fix did something. For new features, writing the behaviour list first gets most of the benefit even if the test code comes second.

    This is one of the checks in how to start vibe coding without shipping broken code, and tests are also the cheapest way to catch the invented calls described in why a coding assistant invents functions. CodeLifter's prompt writing guide has more on stating acceptance criteria.

    Short version

    • Write the behaviours and edge cases yourself; the assistant writes the test code.
    • Use a fresh session so the tests describe the spec, not the implementation.
    • Name the runner (node --test, pytest), forbid edits to the code under test, and ask for the run output.
    • Break the code on purpose, or run a mutation tool such as Stryker, to prove the tests can fail.
    0
    ReplyLink

Plain text or simple Markdown. Links are nofollow. Your email is never shown.

CodeLifter Forum. Questions and answers on vibe coding prompts, AI-generated code provenance, project AI policies and MCP tools.