theaicatalog
GitHub · Products

GitHub replaces LLM calls with small classifier in secret scanning

GitHub says it built a two-stage secret scanner. Regex filters find candidate strings, then a fine-tuned ModernBERT classifier reads the file path, secret type and surrounding code to separate real credentials from lookalikes such as placeholders and checksums. Against the previous LLM-based pipeline, GitHub reports 11.5% higher repository owner-weighted acceptance in a customer A/B test, 86% lower median post-push scan time and more than 98% lower serving cost. On the same public traffic, GitHub says post-push scanning had 13% higher precision and 19% higher recall. The company says the faster model made it practical to extend generic secret detection to push protection, which scans changes before a push is accepted.