Beyond the Prompt: Building a High-Fidelity Discovery Engine

In the current era of “one-click” AI solutions, there is a growing misconception that complex systems can be prompted into existence. The marketing fluff suggests that if you describe a problem well enough, the AI will build the solution for you.

As an AI Operator, my reality is different. When I built the Startup Scouting engine, the goal was to transform raw, noisy signals from platforms like YC, LinkedIn, and Product Hunt into professional-grade intelligence. A generic prompt (“Find me 50 startups and make a spreadsheet”) resulted in a hallucinated mess of duplicate entries and broken data.

To solve this, I moved away from “black box” prompting and manually structured a three-tier architecture. Here’s what that actually looks like, and why each layer matters.

1. The Discovery Layer: Signal-First Search

Instead of relying on an AI’s internal knowledge (which is often outdated and confidently wrong) I codified advanced search heuristics and platform-specific query patterns into what I call a SKILL.md: a structured document that defines exactly what the AI should look for and how.

This “Signal-First” approach specifically targets high-quality founder signals. Not general company listings, but active indicators:

• Building in stealth (founders posting about problems, not products)
• Raising pre-seed (specific language patterns on LinkedIn and Twitter)
• MVP live (product hunts, beta announcements, App Store listings within the last 90 days)

The key insight here is that AI performs significantly better when you treat it as an execution layer for your own search logic rather than as the logic itself. You still need to understand what you’re looking for. The AI helps you scale the search, not define it.

2. The Logic Layer: Python for Precision

The “messy” reality of web data is that AI struggles with precise deduplication and classification. Two entries for the same startup, one listed as “FinTech Solutions Pvt Ltd” and one as “FinTech Solutions”: a human immediately recognises these as the same company. An LLM, especially in a long context window, often doesn’t.

I built a dedicated Python engine to process raw JSON data, ensuring 100% data integrity through deterministic code rather than probabilistic AI inference. The engine performs two core functions:

Strict deduplication: Merging entities with slightly different naming conventions using normalised string matching and fuzzy comparison algorithms. This eliminated 30-40% of duplicate entries that the raw AI output produced.

Sector classification: Categorising startups by stage and industry using pre-defined business logic rather than AI guesswork. When I tell the classifier that a company in “agricultural lending for kisan credit” belongs in “agritech x fintech,” that classification holds consistently across 500 entries. When I asked an LLM to do this classification directly, I got different results on different runs.

The principle: use AI where probabilistic output is acceptable, use code where you need deterministic results.

3. The Presentation Layer: Making Data Actionable

A professional-grade tracker needs to be more than a static CSV that nobody opens after the first week. I engineered a reporting system using openpyxl to generate an interactive, multi-sheet workbook.

This layer provides three things that changed how I actually use the output:

Interactive data views: Colour-coded rows (stage, sector, signal strength) and clickable founder/product links that open directly in the browser. The visual encoding reduces cognitive load: I can scan 200 entries in 10 minutes instead of 45.

Dashboard summaries: Auto-calculated statistics and visual bar charts for quick insights. How many pre-seed companies this week? Which sectors are producing the most new signals? These questions used to require manual counting.

Auditability: Automatic logging of applied search parameters and filters. This sounds boring but it’s operationally critical: when I’m reviewing a pipeline three weeks after building it, I need to know exactly what search logic produced those results.

The Actual Lesson

By taking back control of the logic, I achieved sub-second data views and a level of analytical precision that zero-shot AI simply cannot reach.

But the real lesson isn’t about architecture. It’s about the mindset behind it.

Most people using AI tools for research or deal sourcing are operating at layer zero: describing what they want and hoping the output is usable. The ceiling on that approach is low. The output quality is bounded by how well the AI interprets a natural language description.

The operators who get outsized results from AI are the ones who invest time upfront in defining their own logic (what signals matter, what quality looks like, what the edge cases are) and then use AI to execute that logic at scale. You don’t outsource the thinking. You use AI to scale the execution of your thinking.

That’s the difference between a prompt and an engine. One is a question. The other is a system.

---

Related reading: The AI Tools I Actually Use for Deal Sourcing, the broader stack of AI workflows I use daily at Delhi Angels, and what each one replaced. And: How I Evaluate Early-Stage Startups, what happens to the startups this engine surfaces, once they’re in the pipeline.