I recently tested Jev, a decision-focused model from TypeSafe AI, as a link-evaluation gatekeeper in a web crawler focused on "AI safety and alignment."
The idea was simple: instead of blindly following every link, let a model decide which links are actually worth crawling based on topical relevance.
The results
- 3,280 links evaluated
- 625 passed (19%)
- 2,655 rejected (81%)
- Only 204 flagged as "high-value" (6.2%)
That's a significant filter. The model rejected 4 out of 5 links it saw — keeping the crawler focused on what actually mattered.
The surprising finding
Same link. Different page. Different answer.
Context-sensitivity turned out to be the most interesting behavior. The same URL scored differently depending on the surrounding page context. The model wasn't just pattern-matching on URLs — it was evaluating relevance based on where the link appeared and what surrounded it.
The takeaway: Using a decision-focused model as a gatekeeper can dramatically improve crawler efficiency and relevance. But the real insight is that link evaluation isn't about the link itself — it's about the context the link sits in.