How to measure whether your AI employees are actually working

September 27, 2026 · 6 min read · Summon

You've hired AI employees and handed off work. Now the uncomfortable question: are they actually earning their place? Here is a practical framework for knowing — not guessing.

Measuring a human employee is hard enough. You rely on gut feel, output volume, and whether problems keep landing back on your desk. Measuring an AI employee should be easier — because the record is complete. Every task issued, every output produced, every approval requested is logged. The data is there. You just need to know what to look for.

There are three questions worth asking, in order: Is work completing? Is the output usable? And is the value worth the cost?

1. Is work actually completing?

The first and most important signal is completion rate. Open your board and look at the last 30 days. Of the tasks you sent to an AI employee, what fraction closed as done without you having to take them back?

A healthy AI employee should close 75–85% of assigned tasks without requiring you to step back in. If you're rescuing tasks constantly — pulling them off the board and finishing them yourself — the work is not actually delegated, no matter what the interface says.

Low completion rate almost always points to one of two things: tasks that are too vague (the AI doesn't know what done looks like), or tasks that are blocked on information the AI doesn't have access to. Both are fixable with better briefs and better context-setting.

2. Is the output usable?

Completing a task and completing it well are different things. An AI employee can technically close 100% of tasks and still produce work you throw away every time.

Track your approval behavior. When an AI employee brings something to you for sign-off, how often do you approve it as-is versus rewrite it before approving? If you're rewriting more than 30–40% of outputs, the instructions need tightening.

The most common cause of poor output quality isn't the AI — it's a vague brief. "Write a follow-up email to the prospect" will produce mediocre work. "Write a 3-sentence follow-up email to James at Acme, referencing his concern about implementation time, offering a 30-minute call this week" will produce work you can use immediately.

If your Marketing AI keeps producing generic copy, you haven't told it your brand voice, your audience, and what you want readers to do next. Give it one good example of copy you'd actually send, and the quality jumps noticeably.

If your Sales AI's outreach keeps missing the mark, it probably doesn't know your ICP (ideal customer profile) well enough. Spend 15 minutes writing a crisp one-paragraph description of who you're targeting and why they buy. Paste that into the employee's standing instructions. The difference is immediate.

3. Is the value worth the cost?

Summon's flat pricing is $500/month for an unlimited AI company. The math is simple: if the AI employees collectively recover more than $500 worth of your time per month, you're ahead.

At $100/hour — a conservative effective rate for most founders — you need to recover just 5 hours a month. That's one recovered afternoon. Most active users report recovering 10–30 hours a month once the system is running well.

To estimate your actual savings, list the three biggest recurring tasks you've handed off and estimate how long each took you per week. Multiply by four. If that number exceeds 5 hours, the subscription is paying for itself — and likely by a comfortable margin.

What to do when the numbers say something's wrong

Low completion rate → rewrite the brief. Be specific about the output format and what done looks like.

High completion rate, low usable output → give examples. Paste in two or three pieces of output you'd actually use, and ask the employee to match that standard going forward.

Work completing and usable, but you're still doing too much approval → you're probably being too cautious. Check which approvals you've said yes to every single time. If you've approved the same type of output 20 times in a row without changes, set the employee's instructions to proceed without approval for that type of work.

Everything looks fine but it doesn't feel like the company is moving faster → look at whether you're actually feeding the system work. Many founders set up AI employees and then forget to assign things. The employees don't generate their own tasks by default. You have to use the queue.

The one metric that matters most

If you only track one thing, track this: how many hours this week did you personally spend doing work that an AI employee could have done instead?

If the answer is more than two hours, you're under-delegating. Open the board, pick the most time-consuming item on your own list, and write a task for the right department. That's the job.

The goal of an AI company isn't to have AI employees that are busy. It's to have a founder who is working only on the things that require a founder. Everything else is a queue item.