YouTube Video
You bought an IoT device, extracted the firmware, and dropped the main runtime binary into your favorite reverse engineering tool. Now you're staring at thousands of decompiled functions with no source, no symbols, and no obvious place to start bug hunting. How might an LLM help find signal in the noise, even before you've clearly established what "signal" looks like?
The SiftRank algorithm reframes this uncertainty as an information retrieval problem. Instead of treating vulnerability discovery as an open-ended task for an interactive agent, SiftRank uses an LLM to repeatedly rank small batches of decompiled functions by their likelihood of containing a target vulnerability class. It aggregates each function's rank distribution, refines the candidate set across multiple rounds, and returns a fully ranked dataset with a calibrated top-k cutoff for focused analyst review.
On BinPool, a real-world binary vulnerability dataset containing 95 CVEs across 28 CWE classes, SiftRank achieves 2.26x greater precision in binary vulnerability discovery compared to zero-shot classification. With SiftRank, the relatively small GPT-5 Nano outperforms its much larger sibling, GPT-5, by 32% in precision at a model tier that is 25x cheaper per input token. I'll demonstrate this algorithm on firmware extracted from a commercial network power controller. SiftRank processed 5,710 decompiled functions and surfaced a hidden diagnostic endpoint at rank #1, allowing RCE and leading directly to CVE-2026-41446. This case study tangibly reflects the same pattern that the benchmark shows broadly, which is that small well-harnessed models can behave like serious vulnerability research tools.
This Briefing is a call for hackers to keep the hands-on imperative alive in the age of generative AI. Instead of surrendering the whole discovery process to an agent, we can get our hands dirty, decompose the problem, and invoke LLMs to make bounded, inspectable judgments as a basic research primitive.
Caleb Gross | Security Researcher
https://blackhat.com/us-26/briefings/schedule/index.html#sift-or-get-off-the-poc-applying-information-retrieval-to-vulnerability-research-52775
The SiftRank algorithm reframes this uncertainty as an information retrieval problem. Instead of treating vulnerability discovery as an open-ended task for an interactive agent, SiftRank uses an LLM to repeatedly rank small batches of decompiled functions by their likelihood of containing a target vulnerability class. It aggregates each function's rank distribution, refines the candidate set across multiple rounds, and returns a fully ranked dataset with a calibrated top-k cutoff for focused analyst review.
On BinPool, a real-world binary vulnerability dataset containing 95 CVEs across 28 CWE classes, SiftRank achieves 2.26x greater precision in binary vulnerability discovery compared to zero-shot classification. With SiftRank, the relatively small GPT-5 Nano outperforms its much larger sibling, GPT-5, by 32% in precision at a model tier that is 25x cheaper per input token. I'll demonstrate this algorithm on firmware extracted from a commercial network power controller. SiftRank processed 5,710 decompiled functions and surfaced a hidden diagnostic endpoint at rank #1, allowing RCE and leading directly to CVE-2026-41446. This case study tangibly reflects the same pattern that the benchmark shows broadly, which is that small well-harnessed models can behave like serious vulnerability research tools.
This Briefing is a call for hackers to keep the hands-on imperative alive in the age of generative AI. Instead of surrendering the whole discovery process to an agent, we can get our hands dirty, decompose the problem, and invoke LLMs to make bounded, inspectable judgments as a basic research primitive.
Caleb Gross | Security Researcher
https://blackhat.com/us-26/briefings/schedule/index.html#sift-or-get-off-the-poc-applying-information-retrieval-to-vulnerability-research-52775
SOCIAL SHARE CARD GENERATOR