How I Built an Evidence-Backed SaaS Opportunity Pipeline

A practical look at the adapters, evidence model, LLM analysis, deterministic scoring, and durable orchestration behind GripeRadar.

I started building GripeRadar in June 2026 because I kept running into the same problem: generating SaaS ideas was easy, but finding convincing reasons to build them was hard.

A complaint on Hacker News might reveal genuine frustration. A growing GitHub repository might show technical momentum. Google Trends can show increasing attention. Product Hunt can reveal launch activity. Revenue data can show commercial behavior.

But none of those signals means the same thing.

Ten complaints do not prove willingness to pay. GitHub stars do not prove unmet demand. Search growth does not prove that a useful product can be built. Revenue proves that someone is making money, but not necessarily that a nearby opportunity is still open.

So instead of building another idea generator, I built a multi-source research pipeline around a more useful question:

What evidence supports this opportunity, what does that evidence actually mean, and what is still uncertain?

This article explains how the pipeline works, the architectural decisions behind it, and the mistakes I would avoid if I were starting again.

TL;DR

The pipeline follows seven product phases:

Source adapters

    ↓

Raw signal ingestion

    ↓

Structured LLM analysis

    ↓

Opportunity clustering

    ↓

Classification and review

    ↓

Deterministic scoring

    ↓

Daily report and newsletter

The most important decisions were:

• Normalize evidence instead of normalizing platform popularity.


• Use an LLM to interpret signals, not to assign the final score.


• Keep opportunity quality separate from confidence.


• Preserve original evidence so every conclusion can be challenged.


• Treat scheduling as a durable workflow, not a set of loosely timed cron jobs.


• Treat technical access and permission to use a source as separate questions.

The real problem: signals are not votes

The tempting approach is to collect a lot of data, convert every metric into points, and rank the results.

That produces numbers quickly. It does not necessarily produce useful conclusions.

The pipeline stores both the signal and its bounded meaning.

A GitHub repository stays technical evidence. A search trend stays attention evidence. A revenue record stays commercial evidence. The system can combine them later, but it does not pretend they are interchangeable units.

This distinction became the foundation of the architecture.

Phase 1: Put every source behind an adapter

Each provider has different authentication, pagination, rate limits, identifiers, metadata, and failure modes. Letting those details spread through the application would make every new source a pipeline-wide change.

I instead defined a common adapter boundary. The TypeScript interface looks roughly like this:

interface SignalSourceAdapter<TRaw = unknown> {

  descriptor: SignalAdapterDescriptor;

  executionPolicy?: SignalAdapterExecutionPolicy;



  availability(

    config: SignalIngestionConfig

  ): AdapterAvailability | Promise<AdapterAvailability>;



  streams(config: SignalIngestionConfig): Promise<SignalAdapterStream[]>;



  fetchPage(

    context: SignalFetchPageContext

  ): Promise<SignalAdapterPage<TRaw>>;



  normalize(

    raw: TRaw,

    context: SignalNormalizeContext

  ): ConnectorSignalItem;

}

Each adapter answers four questions:

1. Is this source currently available?


2. Which independent streams should be fetched?


3. How should one page be retrieved?


4. How should a raw record become a normalized signal?

A stream might be a keyword, account, channel, trend feed, product category, or API query.

The ingestion runner handles the shared mechanics:

• Pagination and retries


• Rate-limit accounting


• Record validation


• Deduplication and content hashing


• Inserted, updated, unchanged, and failed counts


• Persistence of normalized and raw evidence

The registry currently contains 13 adapters at different maturity levels. Being registered does not automatically mean a source is enabled or included in production scheduling.

Some sources require credentials. Some require an explicit policy review. Some are deliberately disabled because their transport is too fragile. This lets me remove or pause one source without creating another downstream pipeline.

Normalize evidence, not meaning

The normalized contract includes shared fields such as:

• Source and external identifiers


• Canonical URL


• Title and content


• Publication and discovery timestamps


• Source reliability metadata


• Engagement or trend context


• Content hashes


• Raw source metadata

However, normalization should not erase what makes a source different.

I can store both GitHub stars and YouTube views as engagement metadata, but I should not add them together. They describe different actions, audiences, and levels of commitment.

The normalized record gives downstream phases a stable technical shape. Source-aware metadata preserves the meaning needed for later interpretation.

Phase 2: Use the LLM as an analyst, not a judge

Raw signals are noisy. A post can mention a problem without expressing real pain. A repository can be popular without representing a product opportunity. A trend can be driven by news rather than buyer demand.

Phase 2 uses an OpenRouter-compatible model to convert raw signals into structured analyses. It looks for grounded elements such as:

• The user or customer segment


• The affected workflow


• The problem or unmet need


• Existing workarounds


• Tool requests and urgency


• Commercial intent


• Competition or adoption context


• Direct excerpts supporting the interpretation

Candidates are ranked before reaching the model, and adaptive source quotas prevent one noisy provider from consuming the entire batch.

Every response is validated and assigned an explicit state:

accepted

needs_review

rejected

skipped

failed

That state model proved important. Treating every successfully parsed response as trustworthy would silently pass weak interpretations into clustering.

Structured output helps, but it is not magic. .

Each invocation leases and advances at most one bounded unit of work. The database stores:

• Current step


• Attempts and retry time


• Cursor and source run identifiers


• Result summary and error details


• Lease expiration

A crashed invocation can be resumed, and a slow phase can continue across multiple pulses.

The protected endpoint is implemented as a Next.js Route Handler—the standard App Router mechanism described in the , a project for researching SaaS opportunities using public market signals.

The product is the visible part, but most of the work has been underneath it: source isolation, evidence preservation, model validation, deterministic scoring, retries, policy gates, and making uncertainty visible.

I am still refining the thresholds and evidence model. That is why I wanted to share the architecture now—the interesting questions are not finished.

How would you handle confidence differently? Would you require cross-source corroboration before ranking an opportunity, or allow strong independent evidence from one source? Which signal types would you trust least?

Disclosure: AI tools helped with editing and structure. I reviewed and verified the technical content against the current implementation.