Collinear AI opens 100 public tasks in CWE-bench v2 security test
Collinear AI released CWE-bench v2, which opens a public set of 100 tasks that anyone can run on their own models. Each task gives an agent a real open-source repository and a short instruction to find and fix a security weakness without breaking existing functionality, and scoring is fully programmatic. The tasks cover 91 different CWEs. Of the 11 models evaluated, Gemini 4 Argon leads with 67.8% pass@1 and 81.0% pass@4, just ahead of Claude Opus 5.5 at 67.3% and 78.0%, and even the leader fails 19% of the set on every attempt.