Engineering Prompts for Local LLMs — From Plain Prompts to More Precise Results

I still remember the first time I ran local LLM on my used laptop. It's like just building a personal assistant in the corner of your room: no internet required, no API subscription required, and most importantly — the data doesn't go anywhere. But the euphoria quickly subsided once I started asking questions that were a little complex. "Explain the difference between REST and GraphQL," he answered distractedly. "Help me debug this," instead he was confused. It feels like telling a smart but forgetful friend to do homework without giving clear instructions.

This is where prompt engineering comes in. It's not magic, it's not a marketing trick. Prompt engineering is how we formulate instructions so that language models — especially those running on limited devices like local LLMs — can understand our intentions more accurately. In this article, I'll cover some of the techniques I use most often when interacting with Qwen 2.5 Coder 7B that I run at home.

What is Prompt Engineering?

In simple terms, prompt engineering is the art (and a bit of a science) of crafting input to a language model to produce better output. Imagine you order coffee at a shop. If you just say "coffee," the barista could be confused: black coffee? coffee milk? bitter? sweet? hot? cold? But if you say, "Coffee with milk, hot, a little sweet, use palm sugar," chances are the results will be as expected.

Language models work the same way. The more specific and structured your instructions are, the closer the results will be to what you want. This becomes more important for local LLMs because the model size is smaller, the context window is limited, and the reasoning capabilities are more deficient than GPT-4o class models.

Why Do Local LLMs Need This Technique?

Local models like Qwen 2.5 Coder 7B or Llama 3 8B are quite smart, but they have obvious weaknesses:

• Smaller parameter sizes — they do not store as many facts of the world as large models.
• Easily hallucinates — especially when asked questions that are outside the training domain.
• Context window is limited — we cannot insert many documents at once.
• Answer formats can be messy — without clear instructions, models often return a mixture of text, code, and explanations without structure.

That's why prompt engineering techniques are no longer a bonus, but a basic requirement.

1. Role Prompting: Model a Persona

One of the quickest and most effective techniques is to provide a role model. Tell him who he is and in what capacity he is responding.

Kamu adalah seorang sysadmin Linux berpengalaman yang sedang membimbing junior developer. Jelaskan perbedaan antara cron dan systemd timer dengan bahasa yang sederhana, tambahkan contoh perintah, dan sebutkan kapan sebaiknya memakai masing-masing.

With the prompt above, the model will adopt the persona and adjust the language style. The result is usually more structure, more relevance, and less nonsense. This technique works because the model is trained with many different writing styles, and assigning roles helps the model "slide" into the right mode.

2. Few-Shot Prompting: Teach by Example

Language models learn from patterns. If you provide one or two examples of the format you want, chances are the subsequent output will follow the same pattern.

Klasifikasikan sentiment kalimat berikut sebagai POSITIF, NEGATIF, atau NETRAL.

Kalimat: Aplikasi ini sangat membantu dan cepat.
Sentimen: POSITIF

Kalimat: Saya kecewa karena fitur pencarian sering error.
Sentimen: NEGATIF

Kalimat: Update terbaru cukup biasa saja, tidak ada yang istimewa.
Sentimen:

This technique is especially useful for repetitive tasks such as classification, data extraction, or certain JSON formats. For local LLMs, few-shot prompting is often more reliable than just giving abstract instructions.

3. Chain-of-Thought: Think Step by Step

Small models tend to jump to conclusions. So that he is more careful, we can ask him to write down his thought process.

Sebuah server memiliki 4 GB RAM. Aplikasi A memakai 1.2 GB, aplikasi B memakai 800 MB, dan sistem operasi membutuhkan 1 GB. Bisakah kita menjalankan aplikasi C yang membutuhkan 1.5 GB? Jelaskan langkah demi langkah.

With the addition of “step by step,” the model is forced to break down the problem: calculate total usage, compare with capacity, and then provide a conclusion. The results are usually more accurate, especially for math, logic, or troubleshooting problems.

4. Break Down Complex Tasks: Cut It Into Small Parts

Local LLMs are often overwhelmed if asked to do something big at once. The solution: break it into several steps.

Saya ingin membuat shell script untuk backup otomatis. Kerjakan dalam tiga langkah:
1. Buatkan struktur direktori dan nama file yang tepat.
2. Tuliskan skrip utama dengan restic.
3. Berikan contoh cron job untuk menjalankannya setiap hari pukul 02:00.

This method is similar to the way we cut bread: it's easier to chew, easier to check, and less remains on the cheek.

5. Provide Context and Constraints: Give Clear Boundaries

The model can't guess what's on your mind. The more context and constraints you provide, the more precise the output will be.

Konteks: Saya membuat API untuk home server menggunakan PHP 8.4 native tanpa framework.
Batasan: Tidak boleh pakai Composer, tidak boleh pakai ORM, dan query MySQL harus pakai prepared statement.
Tugas: Buatkan endpoint sederhana untuk login dengan JWT, lengkap dengan validasi input dan error handling.

With clear context and constraints, the model will not suggest Laravel, Eloquent, or external libraries. He will work within the framework you determine.

Antipattern: Prompts to Avoid

Before ending, there are several bad habits that I often include — and that you should not imitate:

• The prompt is too short and vague: "Coding, please." The model does not know the language, goal, or context.
• Prompt without output format: if you want JSON, say JSON. If you want to register, say register.
• A prompt that asks for many things at once: "Describe Linux, create a backup script, continue designing the database." Break it up into sessions.
• Does not provide domain context: "Fix this bug," without including code or error message.

Conclusion

Prompt engineering is not about making the model smarter, but about making the communication between us and the model clearer. For local LLMs running on limited devices, this technique is even more important because every token and every context is valuable.

Start simple: assign roles, add examples, ask for step-by-step explanations, break down big tasks, and always set boundaries. With practice, you will be surprised at how much difference a well-designed prompt can make compared to a sloppy one.

If you have your own favorite technique, write it in the comments column. I'm still learning, and every new trick is always interesting to try.