Building a High-Performance AI Lead Generation Engine with FastAPI and Google Gemini

Generating high-quality, personalized B2B leads is often hampered by slow manual research and generic outreach templates. This project, "Lead Generation Engine," addresses these challenges head-on by leveraging a modern, scalable architecture built on FastAPI and the powerful Google Gemini family of models.

This post delves into the architecture, the innovative two-step AI pipeline, and the implementation details that enable real-time, personalized lead profiling.

---

1. Project Goal and Architecture Overview

The primary goal was to create an intelligent system capable of:

1. Deeply researching a target company based on minimal input (name or URL).
2. Critically analyzing the company's online presence to identify specific pain points for digital solutions.
3. Generating a comprehensive, structured data profile (JSON).
4. Drafting hyper-personalized outreach messages (WhatsApp and Email).
5. Presenting the results in real-time via a modern web interface.

The solution employs a classic microservice approach:

• Backend: Python (FastAPI) handles routing, data persistence (in-memory caching and CSV export), and orchestrates the AI calls.
• AI Core: Google Gemini 2.5 models, using the official GenAI SDK, manage complex reasoning, tool use (Google Search and URL Context), and structured output generation.
• Frontend: A single-page application using HTML, vanilla JavaScript, and Tailwind CSS, utilizing Server-Sent Events (SSE) to stream the AI's thought process and report generation in real-time.

2. The Two-Step AI Pipeline: Research and Structure

The core innovation lies in separating the creative, research-intensive task from the rigid, data-formatting task.

Step 1: The Meticulous Researcher (Streaming)

The first AI call uses the gemini-2.5-flash-preview-05-20 model, guided by the incredibly detailed SYSTEM_PROMPT_LEAD_GEN. This prompt turns the AI into a "virtual business analyst" named 'Ayanda', instructing it to:

1. Use Tools Extensively: It must detail every step of its process, including which Google Search queries were executed and what specific insights were extracted from target URLs using the UrlContext tool.
2. Analyze Pain Points: It critically assesses the company's website (UX, technology stack, age, functionality) and social media presence to find concrete, verifiable opportunities for software services (Website Redesign, AI Automation, E-commerce, etc.).
3. Generate a Full Narrative: The output is a verbose, Markdown-formatted report, which includes its entire multi-phase thought process, the synthesized analysis, the chosen service pitch, and the drafted outreach messages.

Crucially, this entire step is handled via generate_content_stream. FastAPI captures this stream and relays it immediately to the frontend via SSE, allowing the user to watch the AI's research unfold in real-time.

Step 2: The Precise Formatter (Structured Output)

After the research report is complete, a second, non-streaming call is made to the same model, but with a different instruction: SYSTEM_PROMPT_JSON_FORMATTER.

This prompt forces the AI to act purely as a data extraction agent. By leveraging Pydantic schema definition (LeadProfileData) directly within the GenerateContentConfig using response_mime_type="application/json", we ensure the AI's output conforms strictly to the required sales lead structure.

This two-step process achieves high reliability: the first model focuses on complex, creative reasoning without worrying about schema, and the second model guarantees structured, parsable data extraction from the first model's output.

3. Real-Time Streaming Frontend (SSE & FastAPI)

To handle the continuous stream of text from the Researcher AI, the application uses Server-Sent Events (SSE).

1. Backend (main.py / backend_app.py): The /generate-lead-stream endpoint wraps the stream_and_extract_lead_data generator function in a FastAPI StreamingResponse (or similar SSE pattern in backend_app.py). This allows the server to keep the connection open and push chunks of text as soon as the Gemini API returns them.
2. Frontend (index.html): The client uses the native JavaScript EventSource object to listen for events. As text chunks arrive, the script processes them, appending the thought process and report content to separate collapsible sections in the chat log.
3. Dynamic UI: Once the final structured JSON is received (sent as a special lead_details_json event), the JavaScript validates the data using Pydantic's structure (implied in the frontend script) and populates the dedicated "Lead Details" pane instantly.

4. Key Features and Utilities

Beyond the AI core, several utilities enhance the tool's usability for sales teams:

• Pydantic Data Structure: Defines a robust schema (LeadProfileData) covering all necessary fields, from contact details to social media URLs and outreach templates.
• CSV Export: Leads are saved automatically upon successful generation. The /download-saved-csv/{request_id} endpoint allows the user to download a clean, structured CSV file for easy CRM integration.
• WhatsApp Link Generation: The /generate-whatsapp-link/{request_id} endpoint takes the generated cold_call_message_whatsapp and the cleaned phone number to create a direct wa.me link, simplifying immediate outreach.
• Detailed Logging: The chat interface retains the AI's entire thought process (Phase 1, 2, 3 analysis) in a readable, Markdown-rendered format, providing the sales rep with full context and justification for the suggested pitch.

5. Conclusion

The combination of FastAPI's asynchronous capabilities and the highly capable Gemini models, particularly with effective tool usage (Google Search, URL Context), creates a robust and uniquely powerful lead generation asset. By turning complex research and personalization into a fast, transparent, and structured process, this project demonstrates how AI can drastically enhance B2B sales development efficiency.