How to Write Better Prompts for Local AI Models
A local model is not a smaller copy of a frontier chat product. It has a shorter useful context, weaker instruction following, and less patience for ambiguous wording. The fixes below are the ones that change the output on a normal home PC: put the job first, lock the format, keep the task small, write the prompt in plain language, add a check only when the task needs it, and cap the length.
Copy-ready versions of these patterns are in Best Prompt Templates for Local LLMs. The longer test notes are in What Actually Works.
1. Put the instructions before the source text
Many local models weigh the start of the prompt more heavily than the end. If you paste a long article and only then say “summarize this,” the model may continue the article, summarize only the opening, or answer a question that appeared inside the source. State the role, the task, and the rules first. Then paste the text under a clear label such as Text:.
Examples that work
Counterexample
Why it fails: the instruction is late, short, and vague. The model has already been “reading” as if the article is the task. “Make it good” does not say what good means. Move a specific summary instruction to the first lines and delete “make it good.”
2. Name the format, not the vibe
Words like “nice,” “professional,” and “insightful” are not a format. A format is a count, a shape, or a ban. Local models follow counts and bans more often than they follow taste.
Examples that work
Counterexample
Why it fails: there is no length, no number of points, and no ban on an introduction. You will often get a heading, a peppy intro, five uneven paragraphs, and a “hope this helps.” Replace it with a count and a sentence limit.
3. Ask for one deliverable
A single prompt that asks for a draft, a translation, a title list, and a social caption is four jobs. On a small model the later jobs get dropped or mixed into the first one. Split them. You can chain them yourself: take the draft, start a new prompt, and ask only for the translation.
Examples that work
Counterexample
Why it fails: four outputs, no length, and “be creative” fights consistency. The model usually writes a short English post and then loosely gestures at the rest. Run four prompts, or at most two if you have a larger model and you number the outputs.
4. Write the prompt in simpler language than the output
You can ask for a formal memo. The instructions themselves should still be short sentences. Nested clauses, stacked negatives, and jargon in the system prompt are where smaller models lose the thread.
Examples that work
Counterexample
Why it fails: the instruction is harder than the task. The model may echo the jargon or freeze into a vague essay. Say who it is for, how long it is, and what to include.
5. Add a self-check only for brittle tasks
A self-check helps when the output must match a shape: JSON, a fixed number of bullets, or code that should not include prose. It does not help a casual rewrite, and on small models it can make the model print the check instead of the answer. Use one line, at the end.
Examples that work
Counterexample
Why it fails: the check is heavier than the task. You wanted a note. You will often get a paragraph about reasoning and a stiff note at the end. For a birthday note, skip the self-check and set a word limit instead.
6. Cap the length in the prompt and in the runner
Local models ramble when neither the prompt nor the app sets a ceiling. Set both. In the prompt, use a word count or a bullet count. In Ollama, LM Studio, or Jan, set a max token limit a bit above that count so the model can finish the last sentence without being cut mid-word. If you are unsure how long the prompt itself is, run it through the Token Counter before you paste a long source document.
Examples that work
Counterexample
Why it fails: there is no end. A local model will keep listing generic tips until it hits the runner’s max tokens, often repeating itself. Name the audience and a length: “Give a home PC user 6 backup steps, one sentence each.”
A short before-and-after
This is the same request. The first is how people often type. The second is the version a 7B model can follow.
If you want a second pass, do not ask the same model to “also make it nicer” in the same prompt. Start a new prompt with the cleaned notes and a single new instruction.
What this does not fix
- A model that was never trained to write the language you asked for will not become fluent because the prompt is tidy.
- A 3B model will still struggle with long code review. Shrink the snippet or use a larger local model.
- Facts the model does not know cannot be forced by a stricter format. Ask it to use only the text you pasted.
- A high temperature will keep reshuffling wording even when the prompt is good. For repeatable tasks, see How to Get Consistent Results from Local Models.