Syrup Labs · Sample Report
The Free AI Health Check
We audit what your AI agents actually did, not what they said they did. This is a real report on a real estate, not a mock-up.
Subject: Syrup Labs' own agent estate. Audited 2026-09-16. Eight findings. Every one is a green light that was lying, and every one is reproducible with a command printed in this report. We audited ourselves first because we will not sell you a check we have not survived.
8 false signals in one day
Three of them would have caused someone to act on a lie. The promise of this check is three findings in 48 hours. This estate produced eight in one afternoon, and it is an estate run by someone who cares about being right.
The findings
1. The security monitor reported all five safety hooks DEAD. All five were present.
Claimed
Five hooks "DEAD, SCRIPT MISSING", including the one that blocks credentials from being committed to source control.
True
All five files present on disk, 18 hook files tracked in version control.
Cause
The monitor substitutes a path variable by literal string match and missed the brace form, so it built a path that cannot exist and reported the absence as death.
Cost
Someone reinstalls working security controls, or worse, stops trusting the monitor and ignores a real failure later.
2. The sync doctor reported no database. The database is 1.5 MB and exactly where it belongs.
Claimed
BROKEN: no database at .data/mission-control.db
True
1,552,384 bytes, present.
Cause
The doctor was run from a secondary checkout and reported a fact about its own working directory as a fact about the system.
Cost
An emergency that is not one. This is the single most common false alarm in agent estates.
3. The config audit said 138 settings were unset. Twenty carry real values.
Claimed
138 UNSET, 0 CONFIGURED, environment "unreadable".
True
20 keys set with real lengths. 7 genuinely empty, and those 7 are the finding.
Cause
The audit reads the running process with a Linux service manager that does not exist on Windows. Unable to see, it assumed the worst and said so in the language of certainty.
Cost
The owner believed for months that nothing had ever been configured. It had.
4. The publishing surface returns success for files that do not exist.
Claimed
A published artifact URL answered 302, matching a known-good page exactly.
True
An invented filename answered 302 as well. The access gateway intercepts every request before the file is ever looked for, so no URL can fail.
Cause
Verification compared a response code against a control that was not a control.
Cost
Every "published and verified" claim on that surface is unproven. It was caught by opening the page in a browser and seeing the wrong page render. A status code cannot catch this. Eyes can.
5. An evaluation leaderboard reads from a table that does not exist.
Claimed
Three live endpoints serve agent evaluations and a leaderboard.
True
The schema defines nine tables and none of them stores evaluations.
Cost
A scoreboard nobody can audit, ranking agents on data with no home.
6. 45 self-tests exist and nothing runs them. Zero of twelve agents has one at all.
Claimed
"Every instrument takes a self-test and the self-tests are refusals."
True
True and irrelevant: 45 of them are invoked by nothing, and the twelve agents that actually build things have no evaluation criteria whatsoever.
Cost
Nothing grades the output of the things doing the work. This is why agent output cannot be trusted without a human reading all of it.
7. The monitoring canary has been dead for 19 days and reports INCONCLUSIVE, not FAILED.
Claimed
A 30-minute health sweep across 27 screens.
True
Last run 27,560 minutes ago against a 90-minute threshold. Its verdict language says "could not measure", which reads as neutral and is not.
Cost
An unmeasured screen is not a working screen. Nineteen days of unknown, presented as calm.
8. Ten of ten agents report online. The dispatcher sends work to agents that do not exist.
Claimed
"10/10 agents online, gateway connected", rendered on the live dashboard.
True
The recorded finding in this estate's own ledger: the fleet is idle because the scheduler dispatches to agent names with no runtime behind them. The heartbeat is real. The work going nowhere is also real.
Cost
The highest-stakes class of failure: the dashboard is green and the work is not happening.
What these eight have in common
Not one is a broken feature. Every one is a broken connection. The hooks work, the database exists, the config is set, the file was uploaded, the agents heartbeat. What failed in every case was the thing that reports on the thing.
Which means the danger is not that your agents are worse than you think. It is that your instruments are more confident than they are entitled to be, and confidence is what you have been reading.
The three questions this check answers
- Which of your green lights are lying? Every status indicator gets a negative control: we prove the check can fail before we believe it when it passes.
- What did your agents actually do? Every completion claim resolves to a file, a command output, or a measurement, or it does not count.
- What would it cost you if this one were wrong? Findings are ranked by consequence, not by severity labels.
How it runs
- 48 hours. Read-only. Nothing in your estate is changed, nothing is installed.
- You get this document, about your estate, with a reproducing command for every finding, so your own team can re-run every claim we make.
- No charge, and no pitch attached to it. If nothing is wrong you keep the report and we both learn something.
One sentence is all this needs to start:
"Has an agent ever told you something was done that turned out to be wrong?"
If the answer is yes, there is a version of this report with your name on it.