Don’t Treat “Prompt Tuning” as an Engineering Skill: Practical Thoughts on Establishing “Deterministic Contracts” in AI Delivery

In the actual delivery process of our AI Lab, I have observed a very common misconception: many engineers devote most of their energy to so-called “prompt tuning.” They constantly try different phrasings, add examples (few-shot prompting), or even include psychological cues like “You are a senior expert” in the prompt, hoping to improve the stability of the model’s output through these methods.

However, from the perspective of engineering delivery, relying on prompt tuning to guarantee quality is an extremely risky approach. This is because a prompt is essentially a “soft constraint,” exhibiting strong randomness when faced with model version upgrades, shifts in input distribution, or interference from long texts.

True AI engineering capability lies not in writing better prompts, but in establishing a “deterministic contract” between uncertain model outputs and definite business requirements.

From “Hoping It’s Correct” to “Forcing It to Be Correct”

In a typical AI workflow, if your acceptance criterion is that the “output looks about right,” you will never achieve true automated delivery. We need to shift our focus from prompt wording to **structured contracts**.

1. Strongly Typed Output Contracts (Schema Enforcement)

Do not rely on the model to guarantee format simply because you wrote `Please output in JSON format`. In production environments, you must use tools like JSON Schema or Pydantic for strict validation. If the model’s output does not conform to the schema, directly trigger a retry mechanism or fall back to a predefined safe template, rather than attempting to parse a potentially incomplete JSON with complex regular expressions in your code.

2. State-Machine-Driven Pipeline Decomposition

If a complex AI task is crammed into a single massive prompt, its failure rate grows exponentially with logical complexity. Practical experience tells us that tasks should be broken down into multiple small, verifiable state nodes.

- **Node A**: Responsible only for entity extraction $\rightarrow$ Validate via regex/schema $\rightarrow$ Proceed to Node B.

- **Node B**: Perform logical reasoning based on entities $\rightarrow$ Compare against a predefined knowledge base $\rightarrow$ Proceed to Node C.

This approach transforms a huge “black box” into multiple observable “gray boxes,” allowing any failure point to be precisely located and optimized specifically.

“Negative Sample Sets” Are More Important Than “Positive Examples”

Many teams tend to provide the model with a few perfect correct examples when doing few-shot prompting. However, in actual engineering, **defining what is “wrong” is far more efficient than defining what is “correct.”**

We have established a mechanism called the `Negative-Case Library`: it records all real inputs that caused system crashes, hallucinations, or violations of safety policies, along with their corresponding erroneous outputs. During iteration, we do not strive for the model to produce stunning answers, but rather for it to stably avoid these known pitfalls.

When a new prompt version is released, it must pass regression testing against all historical negative samples (i.e., it must no longer produce previous errors) before it can be considered ready for deployment. This is the true baseline for quality.

Conclusion: The Essence of Engineering Is Eliminating Randomness

The charm of AI lies in its creativity (randomness), but the essence of engineering is eliminating randomness.

If you find yourself spending four hours a day tweaking a single adjective in your prompt, you may have fallen into the “alchemist’s trap.” Try shifting your focus to:

- How to define stricter output schemas?

- How to decompose long pipelines into verifiable state machines?

- How to build a negative sample regression set that covers all scenarios?

Only when we treat the AI model as an “unreliable but powerful component” and build a reliable engineering shell around it can AI Lab deliveries truly move from the “demo stage” to the “production stage.”