Notes

What Jev can and can't do for bug bounty hunting

TypeSafe's Jev answers typed questions in well under a second for a fraction of a cent. I ran it on the code of a real Cantina contest, where two models together put 5 of the 9 confirmed bugs in their top 15 functions. That makes it a decent way to pick what to read first. It missed every bug that needed a neighboring function and can't say why a function looks wrong, so I still wouldn't use it to find bugs. As a judge, I'd use it to sort duplicates.

Simon Willison on Jev

TypeSafe AI came out of stealth with Jev on September 15, 2026. It opened Jev to everyone five days later and had to pause signups on September 22 because of demand. I wanted to know whether it belongs in a bug bounty workflow, so I read the documentation and the first public tests, then ran it against the code of a real audit contest.

Jev turned out to be a decent way to decide which functions to read first and a poor way to find bugs, because it missed the bugs that span more than one function. It fits better on the judging side, sorting duplicate reports.

What Jev is

Jev doesn't generate text. You send it a state (a string, a JSON object or an array) and one or more typed questions, and it sends back numbers.

Question type What you ask What you get back
Choice Pick one of these options A probability for each option
Score Rate this on a defined scale A score and a probability for each level
Noul Is this statement true? A probability between 0 and 1

Several questions about the same state run in parallel in one call. Responses take 70 to 500 milliseconds, and pricing is $0.042 per million input tokens with output free. TypeSafe describes Jev as a transformer trained only on synthetic data, with a method it calls Reinforcement Learning for Calibrated Decisions. There is no paper and there are no weights.

The company's CEO, Diogo Almeida, worked on InstructGPT and ChatGPT at OpenAI. The name comes from the economist William Stanley Jevons, who observed that when a resource gets cheaper, people use more of it. TypeSafe calls Jev a "System One" model, after the fast, intuitive kind of judgment, as opposed to the slow reasoning that LLMs do. Because every answer has a fixed type, it can't drift outside the schema you give it.

Testing it on a real contest

The public tests I found are about alerts and documents, not code, so I ran my own. I took the VII Finance contest that Cantina ran from January 12 to 26, 2026, split the in-scope code (core-contracts and yield-harvesting-hook) into its 146 functions, and asked two decision models how likely each function was to contain a bug: Jev 1.13 and OpenAI's GPT-6 Luna decisions. Then I ranked the functions by that probability and checked where the 9 confirmed High and Medium findings from Cantina's audit report landed. A random ranking puts about 1.2 of them in the top 15.

Every in-scope function as one square, colored by AI suspicion, with the real bugs outlined and labeled
The whole codebase, one square per function. Darker squares are ranked higher; outlined squares hold the bugs from the audit report.

With one generic question ("is there a concrete, exploitable bug in this function?"), Jev put 3 of the 9 bugs in its top 15 and Luna up to 4. How I asked mattered more than how much code the model saw. Six checklist questions (reentrancy, access control, oracle manipulation, math, signatures and proxies), averaged, doubled Jev's average precision from 0.17 to 0.35, because each question caught different bugs: the oracle question pushed H2 and H6 up, and the access control question caught H1 and H4. Extra context helped less. Callers and callees took Luna from 2 to 4 bugs in its top 15, and the project's comments made both models slightly worse.

The best setup used both models in two stages. Jev's checklist keeps the top 30 functions, and Luna re-ranks them while seeing each function's callers and callees. That put 5 of the 9 bugs in the top 15, about four times a random pick, for roughly a cent per full scan. Each model covers a weakness of the other. Whatever it was asked, Luna rated ERC4626VaultWrapper's conversion functions at about 100% because they look swapped, although that is the intended design, and Jev's checklist never let those functions into the top 30. A shortlist of 20 gave the same 5 of 9, and a shortlist of 40 dropped to 3.

How many of the 9 real bugs made the top 15
Random pick (average) 1.2
Jev, one question 3
Luna, one question up to 4
Both models (Jev → Luna) 5

The misses follow a clear line. Both bugs visible inside a single function (H2, M1) made the two-stage top 15, and neither bug that needed a neighboring function did. H5, a denial of service when a pool currency is native ETH, never reached a top 15 in any of the 29 setups. M2, a salt shared by two factory functions, reached one once, and so did M3, an oracle that misprices non-18-decimal units. All three need knowledge from outside the function being scored.

This is one contest with 9 bugs, so treat the numbers as a first look. Several of the best setups, including the averaged checklist and the two-stage combination, were chosen after I saw the results, and new code will probably score lower. The report doesn't describe its 5 Low findings, so some of the "false positives" may be real Low issues. All 29 setups together cost about $0.37 in API calls.

Why it still doesn't find bugs

Speed is what Jev does best, and bug bounty hunting rarely needs it. A bug in a deployed contract usually stays there until someone reports it, and a valid finding often takes days of reading and testing. What decides the payout is whether the root cause is right. At a cent per scan, Jev is cheap enough to run as a first pass. The days after that are better spent with a model that works through the code and can tell me why it thinks something is broken.

The experiment and the public tests point to the same limits.

  • Jev returns a probability and nothing else. A 0.83 next to a function doesn't say which line, which caller or which invariant. A report needs all three, plus a working PoC, and a high rank doesn't even show that the model understood the bug.
  • The bugs it missed sit where Jev is weak. TypeSafe's documentation for version 1.13 lists arithmetic, date comparisons, indirection and numeric reasoning as things it can't do reliably. Rounding errors, stale timestamps and access control routed through a proxy or a delegatecall are exactly those, and so are bugs that only show up when you follow a call into the next function.
  • It is sensitive to the prompt. In my run the wording of the question moved the results more than anything else. In Torq's alert triage test, a single instruction moved Jev from labeling almost every alert malicious to labeling almost none, and with naive prompts it sometimes put every alert in one class. At a 0.8 threshold it reached 21.13% accuracy, against 46.48% for Gemini Flash.
  • The input can steer it. Check Point gave an attacker one section of a due diligence report and got Jev to rate an obvious Ponzi scheme as low risk in 25 of 27 runs, at about 50 cents per success. In code review the input includes the project's own comments and NatSpec, and in my run they made the scores worse without anyone trying. A comment like // only callable by the owner is a claim, and a model that can't trace the code will tend to believe it.

Where it fits best: duplicates

From the other side of the table, as a judge, I think Jev fits one job well: checking whether a report is a duplicate.

A competition or a busy bounty program collects many reports that describe the same root cause in different words. Someone has to read each new one, compare it with what is already filed and group it. That question is narrow. Do these two reports describe the same bug? It has a yes or no answer, it doesn't need arithmetic or a trace through the code, and both inputs are text. Jev is built for questions of that shape.

A Noul question per pair ("report A and report B describe the same root cause") or a Choice question per new report (which open issue it duplicates, or none) comes back in well under a second. Comparing everything stays cheap: two 2,000-token reports cost about $0.00017 per comparison, so 100,000 pairs come to around $17. Speed also matters more here than in hunting. Reports keep arriving during a judging period, and one that sits unsorted holds up every decision that depends on it.

I'd treat the scores as a list of which pairs a person should check first. Report text is written by the submitter, and the Check Point result shows how far crafted input can move Jev, so a report could be worded to slip past grouping or to land on someone else's issue. Even without bad intent, two reports can share a root cause and still read as unrelated. The final call, and closing any report, stays with the judge.

Sources: TypeSafe AI on X, Simon Willison, TechCrunch, Wikipedia, Torq, Check Point, Penligent.

Keep reading

View all