The Three Leverage Points for Building Better Systems with LLMs

An internal assistant provides a generic response. The team’s immediate reaction is to look for a larger model. After a few hours of comparison, the quality improves only slightly, and the cost doubles.

The problem, however, wasn’t with the model. The instruction didn’t specify the expected format, and the system didn’t provide the necessary documents. The team tried to fix an error in the most expensive layer that was actually in the simpler layers.

This pattern frequently arises in AI projects. To avoid it, it’s helpful to focus on three key areas: instructions, context, and the model.

The three levers represent different ways to alter the system’s behavior:

1. Instructions: change what we ask for and how we define the task.

2. Context: changing the information the model receives at runtime.

3. Model: swap or adjust the model’s weights to fine-tune it.

The order matters. Instructions are usually faster and less expensive. Context adds infrastructure but keeps the model intact. Modifying the model tends to require data, training, more extensive evaluation, and specialized operation.

The framework does not imply that fine-tuning is always the last step in chronological order. In some products, it is a strategic choice from the outset. The point is this: do not assume that the model needs to change before proving that instructions and context are insufficient.

In production, prompts should not exist solely in a person’s memory or in manually copied blocks. They need to be versioned, reviewed, associated with test cases, and evaluated before any changes are made.

A seemingly minor change—such as asking for more conciseness, changing the order of instructions, or adding an example—can improve one scenario and worsen another. Therefore, the prompt must be tested on a representative set of inputs.

It’s also helpful to separate responsibilities. The system instruction defines policies and the general role; the task template describes the specific work; and user data and retrieved context go into their own fields. This organization reduces ambiguities and facilitates auditing.

Few-shot: examples as specifications

When a textual description is not enough, examples can demonstrate the expected behavior. A small set of input–output pairs shows the model the relevant level of detail, structure, and exceptions.

Few-shot learning is especially useful in structured extraction, classification with company-specific categories, and text transformation. But bad examples also teach bad patterns. They need to represent real-world cases, including edge cases.

When instructions aren’t enough

Lever 2: Context

Context is everything the system provides in addition to the instruction: conversation history, documents, database records, search results, user preferences, and tool outputs.

In practical terms, context engineering is the discipline of selecting, organizing, and presenting to the model the information it needs at that moment.

The goal is not to put as much as possible into the input window. Too much context increases cost, can dilute important evidence, and introduce contradictions. The challenge is to provide the minimum sufficient set.

RAG: seek before responding

 The third lever involves two distinct decisions: choosing the most appropriate model and modifying the model through training or optimization.