Security Research

Updated July 15, 2026, with a correction. Credit to the XBOW team.

At a glance

We took two open-source apps that had already been put through two commercial AI pentesters, Aikido and XBOW, and ran Intrudify against the same targets. Not to score points. To see, on real code with real exploitation, where three different engines land on the bugs that actually matter.

They mostly agree. Where they split, they split on the quieter findings, not the headline ones.

Five bugs worth a security team's time.

Why we ran this

Most security tooling gets evaluated on synthetic benchmarks or on the vendor's own demo app. That tells you close to nothing. The useful question is what a tool does on software that other serious tools have already worked over, where the easy findings are gone and what remains is the stuff that needs judgment.

Fider and Photoview are good for exactly this. Small enough to reason about end to end. Real enough to carry real bugs. And they already have a public paper trail from a 2026 Doyensec benchmark that put Aikido and XBOW through both apps and had a third-party team validate the results by hand. So we had a reference to measure against, and a way to be honest about where we actually stand rather than where our own marketing would like us to.

The Doyensec numbers were strong for both commercial platforms. Dozens of findings each, a high true-positive rate, and a fair bit of severity inflation on the low end, which is normal and not damning. What the study could not do, because it validated largely by reading code, was separate a bug that reaches a dangerous function from a bug that a downstream layer quietly neutralizes. That gap is where most of the interesting disagreements live, and it is the part we wanted to test with a running server instead of a source tree.

Why we focus on the impactful bugs

That last point shapes the whole comparison, so it is worth being explicit about. Of the roughly 80 validated findings across the two apps, the large majority are low-impact hygiene, the kind the study itself scored Low or Informational. Worth fixing, but not where tools separate themselves. Essentially every scanner reports them. The commercial platforms did, and Intrudify does too. A missing header is a missing header regardless of who surfaces it, so it carries almost no weight for a team trying to decide which tool to trust.

The findings that do matter, and where detection quality genuinely differs, are the handful that let an attacker take over an account, read another user's data, or reach the database directly. Across Fider and Photoview there are five of those. That is where this writeup concentrates. Whether a tool catches the bug that ends in a breach is the comparison that actually changes the outcome of a real engagement.

Method: we attacked the apps, then scanned them

We deployed the same container images the Doyensec study used: Fider v0.33.0, a Go feedback application backed by PostgreSQL, and Photoview 2.4.0, a Go photo-gallery application with a GraphQL API backed by MariaDB. Then two things for each.

We reproduced the proof-requiring findings by hand, sending live payloads at the running application and reading the exact source to confirm what the server actually does. And we ran Intrudify in combined mode, static and dynamic analysis (SAST and DAST), authenticated, so we could line our own output up against the study. Manual first, because a finding you cannot reproduce is a guess, and a scanner that ships guesses is worse than no scanner at all.

The two runs, by configuration and wall-clock time:

• Fider: mode dast_sast, authenticated with the admin credential so the agentic authenticator drove and captured the real sign-in flow, unlimited crawl, source supplied for the static pass. 71.5 minutes.
• Photoview: mode dast_sast, authenticated with two accounts, an administrator and an ordinary user, for cross-account testing, unlimited crawl, source supplied. 57.2 minutes.

What we found

We will not walk through all of it. Most of what any scanner surfaces on a mature app is hygiene, and the low-severity misconfigurations look much the same across all three tools. What follows are the findings with real impact, the ones a team would actually stop and fix. Grouped by app, weighted by what an attacker can do with them.

Fider

Account takeover, Critical. Fider has no password login, it authenticates by magic link. The sign-in verify endpoint trusts values the client supplies about which sign-in is being completed, so an attacker can complete a sign-in that was never theirs and land with a valid session cookie. We reproduced it cold against a fresh instance: one request, no prior email to us, and back comes a 307 with an authenticated cookie set. Point it at the admin and you own the app. Unauthenticated, no user interaction. The most serious bug in the whole set, and all three tools caught it.

Missing rate-limit on sign-in, Low to Medium. No throttle on the magic-link and verify endpoints. Minor on its own, a multiplier the moment it is chained with anything that benefits from volume. Intrudify and Aikido flag it. XBOW did not.

Photoview

Cross-account access on the GraphQL API, CWE-639. A signed-in user can call the favoriteMedia mutation on a media id they are not allowed to read. Authentication is present, authorization is absent, and the object id comes straight from the request with nothing tying it back to the caller. We tested it with the two seeded accounts:

Account A is denied the read but allowed the write. A verified session is not the same thing as permission to act on a specific object, and this is the class of bug where that gap does the most damage, because every part of the request looks legitimate.

Unauthenticated SQL injection, Critical. One of Photoview's endpoints lets attacker-controlled input reach a database query in a way that runs SQL before the request is ever authenticated. In plain terms, an attacker with no account can read the database out of the application: user rows, session tokens, share tokens. We confirmed it live, and it executes without ambiguity. This is the kind of bug that ends in a full data breach, and it earns the Critical. All three tools flag it. The specifics are held pending a fix.

Expired share-token accepted, Medium. Photoview writes an expiry onto each share token and then never compares it to the current time when the token is redeemed, so an expired share link keeps working. A control that exists in the schema and does nothing in the code, which is a common and quiet failure mode. Intrudify and Aikido flag it. XBOW did not.

Where the three diverge

Line the impactful bugs up and the picture is tight. On the two most severe findings, the Fider takeover and the Photoview SQL injection, all three tools agree. The divergence sits on the quieter ones, the Fider rate-limit and the Photoview expired token, which XBOW did not surface and the other two did.

That is a healthier outcome than a lopsided scoreboard, and it is the one we expected going in. Three capable engines run over the same two small apps should mostly converge on the bugs that matter. When they do not, the thing to know is which findings live in the gap, because those are the exact ones a team standing on a single tool would never see.

What this proves, and what it does not

It proves that on two real applications, Intrudify finds the bugs that matter, including an unauthenticated Critical, and that it holds its own next to two commercial platforms a third-party firm had already validated.

It does not prove a general ranking. Two open-source apps is a small sample. The comparison is not perfectly symmetric, since the commercial runs happened earlier in the year and tools move. And any scan is a snapshot, not a proof that everything it did not find is absent. We would rather say that flatly than dress it up.

One correction belongs here, for the record. An earlier version of this writeup had one of the findings above down as a false positive. On a closer, live look it is real, and we fixed both the record and the tooling behind it. Credit to the XBOW team, who had this one right, and our thanks for the correction.

The thing we keep landing on is that severity is not a property of a code path. It is a property of what an attacker can actually do with it. A tool that calls an inert database error a Critical costs a team a wasted week. A tool that waves past the one that gives up the database costs them the incident. The only way to tell those two apart is to send the request and watch the server answer. So that is what we build around.

---

Intrudify is a proof-required security scanner. It does not report a finding it cannot reproduce against the running target.

← Back to all posts