AI Engineering Guide: Prompts, Agents, Security, Testing

This guide is a map of what an engineer who builds with AI should understand. It starts with how language models work, moves to agents, then covers the software basics that decide whether an AI feature survives real users: process, access control, production readiness and testing.

In this guide: How LLMs work · Prompting · Prompts in software · Agents · SLMs and agent risks · How teams build software · Authentication and access control · Production readiness · AI code governance · Testing and releases · Engineering judgment

An LLM predicts the next token

A large language model (LLM) doesn't think. It reads all the text so far, split into small pieces called tokens, and predicts the most likely next token. Then it repeats. Four practical facts follow from this:

• It hallucinates. A confident prediction is not a checked fact. A made-up library name sounds exactly like a real one.
• Context is finite. The model only "remembers" what fits in its context window. Older text gets cut or ignored.
• Placement matters. Important instructions are usually followed better near the start of the prompt than buried in the middle.
• Answers vary. A setting called temperature adds randomness, so the same prompt can return different results.

The anatomy of a good prompt

Most bad AI output comes from under-specified prompts. A strong prompt has five parts:

Zero-shot and few-shot

Zero-shot means you give the instruction with no examples. It works for well-understood tasks. Few-shot means you show two or three input-to-output examples first. The model copies the pattern, which makes it the right choice when you need a specific format or a team convention.

Chain-of-thought

"Think step by step" works because it makes the model write out intermediate reasoning, and that reasoning steers the final answer. Use it for multi-step logic, debugging and design trade-offs. Skip it for simple lookups, where it only adds cost.

Common failure modes

• Hallucination: invented libraries, functions or facts. Ask for sources or for the model to say when it is unsure.
• Sycophancy: the model agrees with your wrong premise. Tell it to push back when you're wrong.
• Vagueness: too general an answer. Add constraints and a fixed format.
• Context leakage: earlier conversation bends the current answer. Start a fresh conversation.
• Anchoring on one cause: when debugging, the model commits to its first guess. Ask for several hypotheses and how to tell them apart.

Prompts inside software

System prompts and user prompts

In an API call, the system prompt sets the role, rules and output format for the whole conversation. The user prompt is each individual request. Put durable behaviour, such as tone and format, in the system prompt. Keep it short and specific, because very long system prompts dilute the model's attention.

Structured output

Programs can't reliably read free-form prose. Ask for JSON with a fixed schema, a table with fixed columns, or a set format. Many APIs have a JSON mode. Without one, a few examples of the target schema work well. Always validate the output with a parser, because models occasionally break the format.

Prompt chaining

Break a complex job into steps where each output feeds the next: extract the function names, then write a one-line description of each, then flag the ones with no error handling. Each step stays focused and you can check it. Chaining beats one giant prompt when the steps are fixed and you want to verify them.

Trim your context

More context is not better context. Irrelevant text weakens the signal and raises cost. For a code question, send the function, its types and the error, not the whole file. For a design question, send the relevant schema, not every migration.

Temperature

Near 0, output is nearly deterministic: use it for code, facts and structured output. Around 0.7 to 1.0, output is more varied: use it for brainstorming and alternative wordings. Top-p (nucleus sampling) controls diversity in a different way. For engineering work, low temperature almost always gives more consistent results.

Prompting for code

• Generating: state the language, the function signature, the expected behaviour, the edge cases and what not to do.
• Reviewing: give the code and the requirement it should meet. A model can only review against a known standard.
• Refactoring: name the actual problem (too complex, duplicated, hard to read), not just "make it better".

Agents: an LLM in a loop with tools

An agent is an LLM that can act. It reads input, decides what to do, calls a tool, looks at the result and repeats until a stopping condition is met. A single prompt only produces text. An agent can query systems, run code, read files and call APIs.

Tool use (function calling)

You describe each tool to the model: a name, a plain-language description and its parameters. The model decides when to call one. Your code runs the tool and returns the result. Typical tools for a helpdesk agent are get_ticket(id), search_knowledge_base(query) and update_ticket_status(id, status). Give each tool one responsibility.

The ReAct pattern

ReAct means Reason + Act. The model alternates between a short thought ("I should look up similar past tickets") and an action (the tool call). Because the thoughts are visible, you can trace why an agent did something, which is essential for debugging.

Memory

• In-context memory: everything in the current window. Cheap, but limited and gone when the session ends.
• External memory: a vector database, key-value store or SQL database the agent queries through a tool. Persistent and scalable, but it needs retrieval logic. Long-running agents need it, because no agent can hold a whole codebase in its window.

RAG: retrieval-augmented generation

Instead of hoping the model knows your domain, you fetch the relevant documents at question time and include them in the prompt. Before asking "is this sensor reading abnormal?", retrieve the last ten readings. Before asking "is this change a regression?", retrieve the function's history. RAG is the most reliable way to give a model domain knowledge without retraining it.

When not to use an agent

Agents add latency, cost and new ways to fail. Don't build one for a job a single prompt can do.

Multi-agent systems, small models and agent risks

Orchestrator and workers

For complex jobs, one orchestrator agent splits the task and hands pieces to narrow worker agents, for example one for security, one for readability and one for test coverage. It then combines their results. It mirrors how a senior engineer coordinates a team. It also multiplies failure modes, so start with one agent and split only when you need to.

SLMs and LLMs

LLMs run on cloud servers. Small language models (SLMs), roughly 1 to 13 billion parameters, run on a laptop or edge device. Choose an SLM when:

• latency matters and you can't afford a network round-trip,
• privacy matters and data must not leave the device,
• cost matters and per-token billing adds up,
• the system must work offline.

GGUF is the common file format for compressed (quantised) models that run on ordinary CPUs and GPUs. Ollama is a tool that downloads and serves such models locally through an API on your own machine. A Modelfile lets you bake a system prompt into a custom local model.

Prompt injection

Prompt injection happens when text that the agent reads, such as a ticket, web page or email, contains instructions that try to override its own. An agent that acts on what it reads is more exposed than a single prompt, because it processes untrusted text and has tools to act on it. Defences: treat every tool output as data, never as instructions; keep the instruction layer separate from the data layer; give tools the least power they need; and require human approval for risky actions.

Observability, cost and limits

• Trace every run: which tools were called, in what order, with what inputs and results. Without traces, a silent agent failure is almost impossible to debug.
• Cap the loop: set a maximum number of iterations, and raise an alert when it is hit rather than quietly stopping.
• Watch cost: ten tool calls per request with retries, across many users, can exhaust an API rate limit fast. Cache results that don't change, batch where you can, and use a cheap small model to filter before calling an expensive one.

How professional teams build software

Who does what

• Product owner: decides what to build and why, and owns the backlog.
• Business analyst: turns business intent into clear specifications.
• Developer: decides how to build it and is accountable for quality.
• QA / tester: checks it meets the spec and finds what was missed.
• DevOps: gets it deployed and keeps it running.

On a small team one person may hold several roles, but the responsibilities still exist.

User stories and acceptance criteria

A user story reads: "As a [role], I want [action] so that [benefit]." It describes what a person needs, not what the system does. The "so that" is required. If you can't state the benefit, the story is incomplete.

Acceptance criteria make the story testable, in the form Given (a starting condition), When (an action), Then (an observable result). If a criterion can't be tested, rewrite it. "The page should be fast" fails. "The list endpoint responds in under 200 ms for 95% of requests" passes.

Non-functional requirements and decision records

Functional requirements say what a system does. Non-functional ones say how well: speed, capacity, encryption, uptime. They get ignored until they become incidents, so write them down at design time.

An Architecture Decision Record (ADR) is a short note with four parts: the context, the options considered, the decision and the consequences. It answers "why is it built this way?" for anyone who joins later, including you in six months.

Git and code review

• Branches: main is production, develop is integration, and work happens on short-lived feature/…, fix/… and chore/… branches.
• Commits: use a convention such as feat: add ticket assignment endpoint or fix: return 403 for unauthorised access. "fix" and "update" tell a future debugger nothing.
• Pull requests: explain what changed, why and how to test it. Keep them under about 400 lines so they get reviewed properly.
• Branch protection: no direct pushes to main. Everything goes through a reviewed PR.
• Review is communication: check correctness against the story, security, readability and tests. Be specific and kind: "This can race when two requests arrive together. Would a database lock be clearer?" beats "this is wrong".

REST API design

URLs are nouns and HTTP methods are verbs: GET /tickets lists, POST /tickets creates, GET /tickets/42 reads one, PATCH /tickets/42 updates part of it, DELETE /tickets/42 removes it. Avoid verbs in URLs such as /getTickets. Version from day one (/api/v1/), return consistent response shapes, and use status codes meaningfully: 200, 201, 400, 401, 403, 404, 409, 500. PUT replaces a whole resource; PATCH changes only the fields you send.

Three-tier architecture

Database basics

Tables hold entities. Primary keys identify rows. Foreign keys enforce relationships, so a ticket's creator must be a real user. Indexes speed up searches on common columns. Migrations are versioned scripts that change the schema safely, and you never alter a shared database by hand.

Authentication and access control

Two different questions

• Authentication (AuthN): who are you? The user proves identity with something they know (a password), have (a one-time code) or are (a fingerprint). The result is a verified identity.
• Authorization (AuthZ): what may you do? Given a verified identity, can this person perform this action on this resource? A support agent is authenticated but may still be forbidden from deleting tickets.

Both checks must happen on every request.

Hiding a button is not security

The browser is a client you don't control. Anyone can open developer tools and re-enable a hidden button, or skip your frontend entirely and call the API with curl or Postman. The API never sees what your interface displayed. It only sees the request. So every permission check must run in the backend. The frontend is for user experience; the backend is for security.

Role-based access control (RBAC)

Instead of giving permissions to each person, you give permissions to roles and assign roles to people. A new hire just gets a role. Changing what managers can do means editing one role. The design artifact is a matrix: permissions down the side, roles across the top, allow or deny in each cell. Here is a small slice for a helpdesk app:

Write a one-line reason for every "No". If you can't justify a denial, you may be denying the wrong thing. Just as important, can you justify each "Yes"?

Least privilege and deny by default

Every role starts with zero permissions, and you grant only what is needed. A new permission category, such as "export data", stays denied for everyone until someone consciously opens it.

Ownership checks

RBAC says "employees can view tickets". It doesn't say which tickets. An ownership check compares the resource to the user, such as created_by = current user, and runs after the role check. Both checks are needed, and they are different checks.

JWT and sessions

• Session: the server remembers who is logged in and the browser holds a cookie. Simple and revocable, a good fit for server-rendered apps.
• JWT (JSON Web Token): the server signs a token containing the user ID and role. Each request carries it, and the server checks the signature without a database lookup. A good fit for APIs and mobile apps.

The main trade-off: a JWT can't be cancelled before it expires unless you keep a server-side denylist, so a stolen token works until expiry. Keep expiry short. Also know that HS256 uses one shared secret and RS256 uses a public/private key pair, which suits setups with several services.

Status codes: 401 vs 403

401 means "I don't know who you are" (missing or invalid credentials). 403 means "I know who you are, and you aren't allowed".

Middleware and audit trails

An authentication middleware runs before every protected route. It reads the token from the Authorization: Bearer header, verifies the signature and expiry, and attaches the user to the request. A second piece of middleware checks the role. Neither is optional per route.

An audit log records every sensitive action with five fields: who (user ID), what (the action), which resource (type and ID), when (UTC time) and from where (IP address). It is append-only. It lets you answer "who deleted that ticket on Tuesday?" without digging through backups. Write the log entry in the same database transaction as the action so the two can't disagree.

Passwords and the common security risks

• Hash passwords with a slow algorithm such as bcrypt or Argon2. MD5 and SHA-1 are fast by design and unsuitable.
• Never hard-code the token-signing secret in source code.
• Never grant everyone admin "temporarily" in development. Environments get copied.

The OWASP Top 10 is the standard list of web risks. Five matter on almost every backend: broken access control, cryptographic failures (weak hashing, unencrypted data), injection (such as SQL built from raw user input), security misconfiguration (default credentials, exposed debug endpoints) and authentication failures (weak policies, no expiry).

What separates a demo from production

Validate input in three places

• Frontend: helps users fill forms. It can be bypassed, so it is a convenience only.
• Backend: the real gate. Validate as if the frontend doesn't exist.
• Database constraints: the last line of defence if a bug slips past the backend.

Check type, presence, length, format, range and relationships (an assignee must be a real user with the right role). A validation library such as Zod, Joi or Pydantic does this cleanly.

Consistent errors that leak nothing

Return the same error shape everywhere, for example {"error": "Message for humans", "code": "MACHINE_CODE"}. Never return a raw database error (it reveals your schema), a stack trace (it reveals your code) or a 200 status with an error inside. A user seeing your column names in an error message is a security disclosure.

Constraints, migrations and transactions

• Constraints: NOT NULL (required), UNIQUE (no duplicate emails), FOREIGN KEY (the referenced row must exist) and CHECK (the value must meet a rule, such as an allowed status).
• Migrations: version-controlled, repeatable and reversible. Each has a rollback script, and you'll want it during an incident.
• Transactions: when one operation touches several tables, such as creating a ticket and writing its audit entry, wrap it so that if one step fails, all roll back. Otherwise you get orphaned or inconsistent records.

Structured logging

"Error occurred" is useless at 2 a.m. A structured log entry is machine-readable and complete:

{"timestamp":"2026-09-21T09:15Z","level":"error","requestId":"req-a7f2",
 "userId":"u-001","action":"CLOSE_TICKET","ticketId":"t-042",
 "error":"TICKET_NOT_FOUND","statusCode":404}

A request ID generated per request lets you follow one failing request across many log lines. Log levels have meanings: DEBUG is for local development only; INFO is normal business events; WARN is unexpected but not broken; ERROR means something broke. Only WARN and ERROR should alert a human. Never log passwords or tokens.

Secrets

Locally, use a .env file that is never committed, and commit a .env.example with placeholders. In production, the platform injects environment variables, or you use a secrets manager. A secret never appears in source code, logs, URLs or chat messages. If a secret was ever committed, treat it as compromised even if you delete the commit, because it stays in Git history. Rotate it.

Idempotency

If a user submits a form twice because the first request timed out, you don't want two tickets. An operation is idempotent if running it several times has the same effect as running it once. Two common approaches: the client sends a unique idempotency key that the server remembers, or the server detects duplicates (same user, same title, within 60 seconds). Return 409 Conflict when a duplicate is found.

AI code governance

As teams generate more code with AI, the question becomes how to keep quality, consistency and safety. The usual problems are:

• Dead code: unused functions, imports and unreachable branches.
• Documentation gaps: code with no docstrings or explanation.
• Security regressions: a pattern that looks right but opens a hole.
• Test gaps: features implemented but never tested.
• Style drift: ignoring team conventions.

A governance pipeline checks for these automatically in CI, where it runs on every change. Use two tiers: warnings for style and documentation, errors that fail the build only for serious findings. Blocking every build on any finding makes people ignore the tool.

A self-healing pipeline goes further and fixes simple issues itself, such as adding missing docstrings or formatting to the style guide. Mechanical fixes can be automatic. Judgment calls, such as deleting code that looks dead, should wait for a human to confirm.

Testing and releases

Three kinds of test

• Unit tests check one function in isolation with dependencies mocked. They're fast and focused, but they can't show the system works end to end.
• Integration tests call a real endpoint against a real test database. They're slower and catch wrong SQL, missing transactions and schema mismatches.
• Negative authorization tests prove forbidden actions are actually forbidden. For every "No" in your RBAC matrix there should be a test, for example employee sends DELETE /tickets/1 and expects 403.

CI/CD

Continuous integration runs the full test suite on every pull request, and a failing test blocks the merge. Continuous delivery deploys passing changes to staging automatically, with a manual approval before production. The goal is small, frequent, safe releases instead of rare, risky ones.

Environments

• Development: your machine, fake data, experiment freely.
• Staging: mirrors production's setup, with anonymised data. It's where you verify a release.
• Production: real users and real data, with controlled changes.

The three never share a database, and you never test on production data. Keep a short runbook with a pre-deploy checklist, deployment steps, a post-deploy smoke test and a rollback procedure.

Engineering judgment

A prototype proves an idea works. Production software can be operated by a team, recovered after failure, audited, maintained by someone who didn't write it and extended without a rewrite. Everything above is the gap between the two.

• Simpler is often right. A 50-user internal tool doesn't need Kubernetes. A well-indexed SQL query often beats adding a cache. A simple polling loop can be more reliable than a message queue when events are rare. Aim for the simplest thing that is correct, secure and maintainable.
• Explain decisions with reasons. "We use JWT because we need stateless authentication for an API-first design and tokens expire in 24 hours, which limits the damage from a leak" is an answer. "I've seen it used before" is not.
• Ask what's still missing. Before real users arrive, list the weaknesses you know about. A short honest list is a sign of a mature engineer.

One thing to do today

Pick one feature you've built or plan to build and write down its permission matrix: who can do what, and why each "No" is a No. Then write one test for each "No". It takes under an hour and it's the fastest way to find a security gap.

Taipreneur helps small businesses build AI and automation systems. Get in touch if you want help with yours.