First steps with AI agents: A practical guide for German SMEs

First steps with AI agents: A practical guide for German SMEs

Anyone with responsibility in a German mid-market company who is thinking about AI agents faces a confusing choice of tools, frameworks and promises. This guide walks you through the four phases every productive agent goes through. It is based on 18 projects between early 2024 and mid 2026, from a seven-person craft business to a 35-vehicle freight forwarder. If you work through the guide step by step, in eight weeks you have a productive agent that has a measurable effect on day-to-day operations.

Who should read this guide

This guide addresses three audiences who all want to solve the same problem but bring different tools:

Managing directors without an IT background. If you understand what your team spends 80 minutes a day on recurring tasks but do not know how to build an agent. This guide translates technical concepts into business logic.

Clerks with process responsibility. If you know which tickets, mails and requests come in daily but have not built an agent yet. This guide shows how domain knowledge becomes a workflow.

External IT leads without AI experience. If you maintain systems but have never run an LLM-based agent in production. This guide shows the minimum steps to a first productive system.

If you are a data scientist or already know agent frameworks, you will find little new here. But if you want to build an agent for the first time, you have a complete roadmap after reading.

Phase 1: pick a concrete task that does not need an architecture design

The most common mistake in a first agent project is choosing a task that is too complex. Anyone who starts with an open task space fails. Anyone who starts with a clearly defined, recurring task wins. This phase decides success and failure.

Choosing a task: three criteria. The right first task has three properties: it is measurable (what exactly should be automated), it is high-frequency (at least 20 times per week), and it is not safety-critical (no hospital, no stock trading). If you pick a task that meets all three criteria, you have a realistic first iteration.

Concrete examples from our projects. A tax advisor association automated client request classification. A craft business automated quote generation from notes. A freight forwarder automated dispatch for standard deliveries. A law firm automated contract clause extraction on incoming contracts. Four tasks that all meet the three criteria.

Setting realistic expectations. The first agent saves 60 to 80 percent of routine time for the specific task. It is not perfect. It needs HITL. It costs token fees. Anyone starting with the expectation of "100 percent autonomous, 0 errors, 0 costs" will be disappointed. Anyone starting with "60 percent time savings, 5 percent errors, 50 Euro per month for tokens" will win.

Three questions before you start. Anyone picking a task should answer three questions in writing: what exactly is being automated? Who checks the result before the final action? What is the acceptance time (too much correction, too little correction)? If you cannot answer these three questions, you need a better task definition, not more technology.

Phase 2: pick tools that are set up in two hours

Tool selection follows the task, not the other way around. Looking for the most modern framework before you have chosen the task builds a toy. This phase takes two hours and produces a toolchain that runs in the task.

Three tool layers that work together. A productive agent architecture needs three layers: an LLM layer (GPT-4, Claude or Gemini), an orchestration layer (Make, n8n or Zapier), a storage layer (cloud database, Google Sheets or file). If you control all three layers, you have a productive system. If you only have one layer, you have a toy prototype.

The LLM layer: three realistic options. OpenAI offers the lowest entry barrier with GPT-4o for text and image tasks, GPT-4o-mini for simple classifications. Anthropic Claude is stronger on longer texts and traceable reasoning. Google Gemini is more cost-efficient for large contexts. Anyone choosing a cloud LLM for their first agent is right with all three.

The orchestration layer: three realistic options. Make.com is visual and fast, n8n is code-flexible without programming knowledge, Zapier is the most conservative choice. If you have worked with Zapier before, stick with it. If you need structured workflows with real branching, go to Make or n8n. If you code, you can extend both frameworks via API.

The storage layer: from simple to complex. For simple classifications, you do not need a database. A Google Sheet is enough. For structured data over several weeks, Notion or Airtable works. For complex relationships, you need a real database. For the first eight weeks, any cloud storage with an API is sufficient.

Realistic tool choice for the first eight weeks. The combination we have most often seen succeed: GPT-4o-mini as the LLM, Make.com as orchestration, Google Sheets as storage. If you start with this three-way combination, you have a system that goes productive in eight weeks and shows the reality of the task space. If you need more later, migrate to Claude or Gemini, n8n and a real database.

Phase 3: build the first agent in 10 to 25 hours

The build phase is the shortest of the four phases, but in our experience it gets the most discussion. Those who confuse building with thinking spend weeks on architecture and weeks on the first productive run. This phase forces focus.

Hours 1 to 3: trigger and data input. A task has a trigger (incoming email, new ticket, new file) and a data point (text to classify, fields to extract, question to answer). The trigger comes from the orchestration layer. The data point is processed by the LLM. If you do not invest these three hours, you build an architecture without ground.

Hours 4 to 8: prompt iteration. The prompt is the instruction to the LLM. If you write the prompt well, you have a system that is right about 80 percent of the time. If you write it poorly, you have a system that is right about 30 percent of the time. Three versions are realistic. The first version is naive and delivers 50 to 60 percent. The second version uses the errors of the first. The third version is the productive system.

Hours 9 to 14: build in the HITL checkpoint. Every agent needs a checkpoint where a person reviews the result before it continues. For client request classification, the check is 5 to 10 seconds (a short read of the answer). For quote generation, the check is 30 to 60 seconds (detailed review of the line items). If you skip the HITL checkpoint, you build a system that automatically produces errors when it fails.

Hours 15 to 22: logging and audit trail. If you want to look back in a year and understand why a given agent made a given decision, you need a log. The log contains date, trigger content, prompt version, LLM response, HITL decision, final result. If you build this log, you have a permanent improvement lever.

Hours 23 to 25: end-to-end test. If you manually trigger, you see what happens. If you run five examples through, you see where the agent works well and where not. If you skip this test phase, you fall into the "let's just go live" mentality, which in agent projects typically has to be corrected after three weeks.

Realistic effort numbers. These numbers are not estimates but observations from 18 projects. A simple classification takes 8 to 12 hours of build time. A medium-complexity task (quote generation) takes 15 to 20 hours. A complex task with multi-step reasoning takes 30 to 40 hours. Anyone who needs less than 8 hours either has a trivial task or has skipped something.

Phase 4: continuous maintenance as an ongoing process

The most common mistake after the first productive run is assuming the agent will keep running on its own. Agents age out, models get updated, edge cases pile up, customer requirements change. This phase is what separates productive from wasted agent projects.

Weeks 1 to 2: observation. The agent runs, the HITL checkpoint points at every output. The person reviewing the HITL checkpoint logs in a spreadsheet which errors occur. These two weeks are the most valuable for system improvement. If you set up the logging mechanism in these weeks, you have a permanent improvement lever.

Weeks 3 to 6: prompt refinement. The error list shortens daily, the prompt becomes more precise. Realistically three to five prompt iterations, then manual correction rates are at 5 to 10 percent of inputs. If you switch the HITL person in this phase without explaining the error context to the new person, the rate gets worse again.

Weeks 7 to 12: HITL reduction. If you find the agent trustworthy in this phase, you can reduce the HITL checkpoint to sampling. Realistically a shift from "check every input" to "check every fifth input". If you keep the HITL checkpoint at 100 percent through this phase, you have an agent that does not actually add value.

Months 4 to 6: a second task. After three months of productive operation of the first task, the process is clear. A second task can be made productive with significantly less effort because the infrastructure is in place. If you wait for the second task because the first is not yet "perfect", you have misunderstood the process.

Months 7 to 12: model update. The model gets updated no earlier than after six months of productive operation. The agent code keeps working, the prompt gets adjusted, logging continues. If you switch the model monthly, you do not have a productive agent, you have a playground.

What needs continuous maintenance. Three items stay on the maintenance list permanently: the prompt (on model update), the error list (weekly), the HITL rate (monthly). If you keep these three in view, you have a low-maintenance system. If you neglect them, you have a zombie agent within twelve months that runs but nobody uses anymore.

What this guide does not cover

This guide covers the typical path to the first productive agent. It does not cover: multi-agent systems with several specialized agents, edge AI deployment on your own hardware, fully autonomous systems without HITL, regulatorily compliant high-risk applications with EU AI Act documentation. If you want to tackle one of these topics, you have a different project that starts with phase 1 of this guide but has a different complexity level.

Three points we got wrong on every project

Anyone who has worked through this guide should also know what we typically got wrong on these projects:

First: we built the first agent too complex. On seven projects we spent more time in the build phase than necessary. The simpler version would have delivered 80 percent of the benefit in 60 percent of the time.

Second: we built HITL checkpoints in too late. On five projects we introduced the HITL checkpoint only after the first productive run. The result was five weeks of trust problems that were not necessary.

Third: we pushed for full autonomy too early. On three projects the managing director removed the HITL checkpoint after four weeks of productive operation. The result was spectacular errors that damaged trust in the system for a year.

Conclusion

Anyone who wants to build a productive AI agent does not need the best framework but the right sequence: pick the task, pick the tools, build in two to three weeks, maintain continuously. This guide describes exactly that sequence. Anyone who persists through the eight weeks has a productive agent within a year that ten people in the team use. Anyone waiting for the perfect framework has no agent within a year.