# Build with AI
Source: https://docs.firecrawl.dev/ai-onboarding

Everything you need to onboard your AI agent to Firecrawl.

If you're developing with AI, Firecrawl offers several resources to improve your experience. Firecrawl ships with **skills** — self-contained knowledge packs that AI coding agents discover and use automatically. One install command gives agents CLI tools for live web work and build skills for integrating Firecrawl into application code. Agents like Claude Code, Cursor, Antigravity, and OpenCode can self-onboard with a single command — no human setup required after the API key exists.

* [Prerequisite: Create an API Key](#prerequisite-create-an-api-key)
* [Skills + CLI](#skills-cli)
* [Using Firecrawl as a Tool](#using-firecrawl-as-a-tool)
* [Agentic Debugging](#agentic-debugging)
* [Firecrawl MCP Server](#firecrawl-mcp-server)
* [Firecrawl Docs for Agents](#firecrawl-docs-for-agents)
* [Quick Start Guides](#quick-start-guides)
* [Agent Harnesses](#agent-harnesses)
* [SDKs](#sdks)

## Prerequisite: Create an API Key

Currently, we require a human to create a Firecrawl account. Once you have an account, you'll need to [create an API key](https://www.firecrawl.dev/app/api-keys). With an API key, your agent can handle the rest — installing the skills, authenticating the CLI, wiring up MCP, and making calls on your behalf.

<Card title="Get your API key" icon="key" href="https://www.firecrawl.dev/app/api-keys">
  Sign up and grab an API key to start using Firecrawl.
</Card>

## Skills + CLI

The [Firecrawl CLI](/sdks/cli) lets your agent search, scrape, interact, crawl, map, extract, and run agent jobs from the terminal. It's built for humans, AI agents, and CI/CD pipelines.

The Firecrawl **skills** are self-contained knowledge packs that AI coding agents like Claude Code, Antigravity, and OpenCode discover and use automatically. A single install command sets up everything — the CLI tools for live web work and the build skills for integrating Firecrawl into application code:

```bash theme={null}
npx -y firecrawl-cli@latest init --all --browser
```

* `--all` installs the Firecrawl skills to every detected AI coding agent on the machine
* `--browser` opens the browser for Firecrawl authentication automatically

After install, verify everything is working:

```bash theme={null}
firecrawl --status
firecrawl scrape "https://firecrawl.dev"
```

### What the install gives you

The install sets up two categories of skills that cover every way an agent uses Firecrawl:

**CLI skills** — for live web work during an agent session:

| Skill                | Purpose                                           |
| -------------------- | ------------------------------------------------- |
| `firecrawl/cli`      | Overall CLI command workflow                      |
| `firecrawl-search`   | Search the web and discover pages                 |
| `firecrawl-scrape`   | Extract clean content from a known URL            |
| `firecrawl-interact` | Interact with scraped pages using prompts or code |
| `firecrawl-crawl`    | Bulk-extract content from an entire site          |
| `firecrawl-map`      | Discover all URLs on a domain                     |
| `firecrawl-agent`    | Run autonomous web data gathering with a job      |

**Build skills** — for integrating Firecrawl into application code:

| Skill                        | Purpose                                              |
| ---------------------------- | ---------------------------------------------------- |
| `firecrawl-build`            | Choose the right Firecrawl endpoint for your product |
| `firecrawl-build-onboarding` | Auth and project setup                               |
| `firecrawl-build-scrape`     | Implement scraping in app code                       |
| `firecrawl-build-search`     | Implement search in app code                         |
| `firecrawl-build-interact`   | Implement page interaction in app code               |
| `firecrawl-build-crawl`      | Implement crawling in app code                       |
| `firecrawl-build-map`        | Implement URL discovery in app code                  |

### Choose your path

Both skill categories use the same install. The difference is what happens next:

<Steps>
  <Step title="Live web tools (CLI skills)">
    Use this when you need web data during your current session — searching the web, scraping known URLs, interacting with scraped pages, crawling docs, mapping a site, or running an agent job.

    The default flow:

    1. Start with **search** when you need discovery
    2. Move to **scrape** when you have a URL
    3. Use **interact** when the scraped page needs follow-up actions
    4. Use **map** or **crawl** when you need many URLs or pages
    5. Use **agent** when the task is open-ended and needs autonomous discovery

    ```bash theme={null}
    # Search the web
    firecrawl search "best open-source web crawlers"

    # Scrape a page into clean markdown
    firecrawl scrape https://docs.firecrawl.dev

    # Crawl a whole site
    firecrawl crawl https://docs.firecrawl.dev
    ```
  </Step>

  <Step title="App integration (Build skills)">
    Use this when you're building an application, agent, or workflow that calls the Firecrawl API from code. The build skills help with picking the right endpoint, wiring up the SDK, and running a smoke test.

    The agent answers one key question — *what should Firecrawl do in the product?* — and the build skills route to `/search`, `/scrape`, `/interact`, `/crawl`, `/map`, or `/agent` accordingly.
  </Step>

  <Step title="REST API (no install needed)">
    If you prefer not to install anything, agents can call the Firecrawl REST API directly. Set the API key and hit the endpoints:

    * `POST https://api.firecrawl.dev/v2/search` — discover pages by query
    * `POST https://api.firecrawl.dev/v2/scrape` — extract clean markdown from a URL
    * `POST https://api.firecrawl.dev/v2/interact` — interact with a scraped page
    * `POST https://api.firecrawl.dev/v2/crawl` — bulk-extract an entire site
    * `POST https://api.firecrawl.dev/v2/map` — discover URLs on a domain
    * `POST https://api.firecrawl.dev/v2/agent` — run autonomous web data gathering

    Auth header: `Authorization: Bearer fc-YOUR_API_KEY`
  </Step>
</Steps>

The full onboarding definition is available at [`firecrawl.dev/agent-onboarding/SKILL.md`](https://www.firecrawl.dev/agent-onboarding/SKILL.md) — agents can fetch it directly for self-onboarding.

<Card title="Skills + CLI" icon="terminal" href="/sdks/cli">
  Install the CLI and skills, authenticate, and run scrape, search, crawl,
  interact, map, extract, and agent commands from the terminal.
</Card>

## Using Firecrawl as a Tool

Firecrawl gives agents five core tools for working with the web. Each tool maps to an API endpoint and a CLI command. Agents pick the right tool based on what they need:

<AccordionGroup>
  <Accordion title="Search — discover pages by query" icon="magnifying-glass">
    Start here when you don't have a URL yet. Search returns relevant web pages for a natural-language query, with optional full-page content included in the results.

    ```bash theme={null}
    # CLI
    firecrawl search "latest OpenAI API pricing"
    ```

    ```bash theme={null}
    # REST API
    curl -X POST https://api.firecrawl.dev/v2/search \
      -H "Authorization: Bearer fc-YOUR_API_KEY" \
      -H "Content-Type: application/json" \
      -d '{"query": "latest OpenAI API pricing"}'
    ```

    **When to use:** Research tasks, finding documentation, competitive analysis, answering questions that require up-to-date web information.
  </Accordion>

  <Accordion title="Scrape — extract content from a URL" icon="file-lines">
    Use this when you already have a URL and need clean, LLM-ready content. Scrape converts any web page into markdown, HTML, or structured data — handling JavaScript rendering, anti-bot measures, and messy HTML automatically.

    ```bash theme={null}
    # CLI
    firecrawl scrape https://docs.stripe.com/api/charges
    ```

    ```bash theme={null}
    # REST API
    curl -X POST https://api.firecrawl.dev/v2/scrape \
      -H "Authorization: Bearer fc-YOUR_API_KEY" \
      -H "Content-Type: application/json" \
      -d '{"url": "https://docs.stripe.com/api/charges"}'
    ```

    **When to use:** Reading documentation, extracting article content, pulling data from a known page, converting web pages to context for LLMs.
  </Accordion>

  <Accordion title="Crawl — bulk-extract an entire site" icon="spider">
    Crawl recursively follows links from a starting URL and scrapes every page it finds. Use it when you need content from an entire site or documentation set, not just a single page.

    ```bash theme={null}
    # CLI
    firecrawl crawl https://docs.firecrawl.dev --limit 50
    ```

    ```bash theme={null}
    # REST API
    curl -X POST https://api.firecrawl.dev/v2/crawl \
      -H "Authorization: Bearer fc-YOUR_API_KEY" \
      -H "Content-Type: application/json" \
      -d '{"url": "https://docs.firecrawl.dev", "limit": 50}'
    ```

    **When to use:** Ingesting full documentation sites, building knowledge bases, migrating content, training data collection.
  </Accordion>

  <Accordion title="Map — discover all URLs on a domain" icon="sitemap">
    Map rapidly discovers every indexed URL on a domain without scraping the content. Use it when you need to understand a site's structure or find specific pages before scraping them.

    ```bash theme={null}
    # CLI
    firecrawl map https://docs.firecrawl.dev
    ```

    ```bash theme={null}
    # REST API
    curl -X POST https://api.firecrawl.dev/v2/map \
      -H "Authorization: Bearer fc-YOUR_API_KEY" \
      -H "Content-Type: application/json" \
      -d '{"url": "https://docs.firecrawl.dev"}'
    ```

    **When to use:** Site audits, finding specific pages on a large site, understanding site structure before a targeted crawl.
  </Accordion>

  <Accordion title="Interact — work with a scraped page" icon="hand-pointer">
    Interact lets agents continue from a scrape using prompts or code. Use it when a scraped page requires clicks, form fills, navigation, or follow-up extraction.

    ```bash theme={null}
    # CLI
    firecrawl scrape https://example.com
    firecrawl interact "Click the pricing tab and extract the plan names"
    ```

    ```bash theme={null}
    # REST API
    curl -X POST https://api.firecrawl.dev/v2/interact \
      -H "Authorization: Bearer fc-YOUR_API_KEY" \
      -H "Content-Type: application/json" \
      -d '{"scrapeId": "scrape-id-from-scrape", "prompt": "Click the pricing tab and extract the plan names"}'
    ```

    **When to use:** Continuing from a scrape, navigating dynamic pages, filling forms, and extracting data after page actions.
  </Accordion>
</AccordionGroup>

### How agents chain tools together

Most agent workflows combine multiple tools. A typical pattern:

1. **Search** to find relevant pages → get a list of URLs
2. **Scrape** the most relevant URLs → get clean content
3. **Interact** when the scraped page needs follow-up actions
4. **Agent** when the task needs autonomous discovery or structured multi-page extraction

For bulk work, agents use **Map** to discover URLs first, then **Crawl** or selectively **Scrape** the pages they need.

## Agentic Debugging

When a Firecrawl call fails or returns unexpected results, your agent doesn't have to escalate to a human. The [`/support/ask`](/api-reference/endpoint/ask) endpoint is an AI support agent built for **agent-to-agent** communication — it diagnoses issues with your jobs, account, and API usage, then returns a verified answer with machine-readable fix parameters your agent can apply directly.

Wire it into your agent's error-handling flow so it can self-recover from scraping failures, crawl issues, and configuration problems — typically in 15–30 seconds, no human in the loop.

### How it works

1. **Your agent describes the problem** — a natural-language question describing the issue.
2. **The support agent investigates** — it inspects job logs, account state, documentation, and source code.
3. **The support agent validates** — when possible, it tests a fix against the live Firecrawl API (e.g., retrying a scrape with adjusted parameters).
4. **Your agent gets a verified answer** — a prose `answer`, machine-readable `fixParameters` to apply directly, and `validation` results showing whether the fix was tested.

### Example

Send a question, plus an optional `rationale` to give the support agent context about what your end user is trying to accomplish:

```bash theme={null}
curl -X POST https://api.firecrawl.dev/v2/support/ask \
  -H "Authorization: Bearer fc-YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "question": "my crawl returned 3 pages but I expected 50",
    "rationale": "user is on their third failed crawl attempt today"
  }'
```

The response includes an `answer`, a `confidence` rating, optional `fixParameters` (e.g., `{"waitFor": 5000}`) your agent can pass to the next call, and `validation` showing whether the fix was tested against the live API.

<Card title="Ask endpoint reference" icon="life-ring" href="/api-reference/endpoint/ask">
  Full request and response schema for `/support/ask`, including status codes and the feedback envelope returned when the agent gets stuck.
</Card>

## Firecrawl MCP Server

MCP is an open protocol that standardizes how applications provide context to LLMs. Among other benefits, it gives LLMs tools to act on your behalf. Our [MCP server](https://github.com/firecrawl/firecrawl-mcp-server) is open-source and covers our full API surface — search, scrape, interact, crawl, map, extract, and agent.

Use the remote hosted URL:

```
https://mcp.firecrawl.dev/{FIRECRAWL_API_KEY}/v2/mcp
```

Or add the local server to any MCP client:

```json theme={null}
{
  "mcpServers": {
    "firecrawl": {
      "command": "npx",
      "args": ["-y", "firecrawl-mcp"],
      "env": {
        "FIRECRAWL_API_KEY": "fc-YOUR-API-KEY"
      }
    }
  }
}
```

<Card title="MCP Server" icon="plug" href="/mcp-server">
  View installation instructions for Cursor, Claude Desktop, Windsurf, VS Code,
  and more.
</Card>

## Firecrawl Docs for Agents

You can give your agent current Firecrawl docs in a context-aware way. Agents can self-onboard by pulling these resources directly — no human wiring required.

<Steps>
  <Step title="Markdown docs">
    Every page has a markdown version. Append `.md` to any docs URL, or use the page action menu to copy the page as markdown.

    ```
    Docs for this page: https://docs.firecrawl.dev/ai-onboarding.md
    ```
  </Step>

  <Step title="Full llms.txt">
    Give your agent all of our docs in a single file.

    ```
    Here are the Firecrawl docs: https://docs.firecrawl.dev/llms-full.txt
    ```

    A shorter index is also available at `https://docs.firecrawl.dev/llms.txt`.
  </Step>

  <Step title="MCP docs server">
    For a structured approach using MCP tools, connect the Firecrawl MCP server in any MCP client (Cursor, Claude Code, Claude Desktop, Windsurf). See the [MCP Server](/mcp-server) page for install commands.
  </Step>

  <Step title="Copy to ChatGPT / Claude">
    Every page includes a contextual action menu (copy, view as markdown, open in ChatGPT, open in Claude) so agents and humans can move pages between tools in one click.
  </Step>
</Steps>

## Quick Start Guides

Drop-in quickstarts for the stacks agents build on most often. Point your agent at any of these to scaffold a working Firecrawl integration end-to-end.

Prefer to let Cursor drive? One-click install the Firecrawl MCP server and start prompting in Cursor:

<a href="cursor://anysphere.cursor-deeplink/mcp/install?name=firecrawl&config=eyJjb21tYW5kIjoibnB4IiwiYXJncyI6WyIteSIsImZpcmVjcmF3bC1tY3AiXSwiZW52Ijp7IkZJUkVDUkFXTF9BUElfS0VZIjoiWU9VUi1BUEktS0VZIn19">
  <img alt="Open in Cursor — Add Firecrawl MCP server" />
</a>

<CardGroup>
  <Card title="Node.js" icon="node-js" href="/quickstarts/nodejs">
    Server-side JavaScript and TypeScript with the Firecrawl Node SDK.
  </Card>

  <Card title="Next.js" icon="react" href="/quickstarts/nextjs">
    Scrape, search, and crawl from Next.js route handlers and server actions.
  </Card>

  <Card title="Python" icon="python" href="/quickstarts/python">
    Use Firecrawl from scripts, notebooks, and backend services.
  </Card>

  <Card title="FastAPI" icon="bolt" href="/quickstarts/fastapi">
    Build async Python APIs that search, scrape, and extract.
  </Card>

  <Card title="Cloudflare Workers" icon="cloudflare" href="/quickstarts/cloudflare-workers">
    Run Firecrawl at the edge with Workers.
  </Card>

  <Card title="Vercel Functions" icon="triangle" href="/quickstarts/vercel-functions">
    Call Firecrawl from Vercel serverless functions.
  </Card>

  <Card title="AWS Lambda" icon="aws" href="/quickstarts/aws-lambda">
    Invoke Firecrawl from Lambda handlers.
  </Card>

  <Card title="Supabase Edge Functions" icon="database" href="/quickstarts/supabase-edge-functions">
    Use Firecrawl inside Supabase Deno runtime.
  </Card>

  <Card title="Go" icon="golang" href="/quickstarts/go">
    Idiomatic Go SDK for search, scrape, and crawl.
  </Card>

  <Card title="Rust" icon="rust" href="/quickstarts/rust">
    Typed Rust SDK for Firecrawl.
  </Card>

  <Card title="Laravel" icon="laravel" href="/quickstarts/laravel">
    Add Firecrawl to Laravel apps via the PHP SDK.
  </Card>

  <Card title="Rails" icon="gem" href="/quickstarts/rails">
    Drop Firecrawl into Ruby on Rails.
  </Card>
</CardGroup>

See the full list of quickstarts (Express, NestJS, Fastify, Hono, Bun, Remix, Nuxt, SvelteKit, Astro, Mastra, Django, Flask, Elixir, Java, Spring Boot, .NET, ASP.NET Core, and more) in the left sidebar.

## Agent Harnesses

Firecrawl works with the runtimes and frameworks agents actually live inside — coding agents, agent SDKs, and model aggregators. Most coding harnesses can auto-discover the Firecrawl skills via `npx -y firecrawl-cli@latest init --all --browser`; the rest call Firecrawl as a tool over MCP or the REST API.

<CardGroup>
  <Card title="Claude Code" icon="terminal" href="/quickstarts/claude-code">
    Anthropic's CLI — set up Firecrawl MCP in Claude Code.
  </Card>

  <Card title="Cursor" icon="arrow-pointer" href="/quickstarts/cursor">
    IDE agent — one-click install Firecrawl MCP in Cursor.
  </Card>

  <Card title="OpenCode" icon="code-branch" href="/quickstarts/opencode">
    Wire Firecrawl MCP into OpenCode.
  </Card>

  <Card title="Codex CLI" icon="code" href="/quickstarts/codex-cli">
    Wire Firecrawl MCP into OpenAI Codex CLI.
  </Card>

  <Card title="OpenRouter" icon="route" href="/quickstarts/openrouter">
    Pair any OpenRouter model with Firecrawl web tools.
  </Card>

  <Card title="Amp" icon="bolt" href="/quickstarts/amp">
    Wire Firecrawl MCP into Sourcegraph Amp.
  </Card>

  <Card title="Windsurf" icon="wind" href="/quickstarts/windsurf">
    Agentic IDE — set up Firecrawl MCP in Windsurf.
  </Card>

  <Card title="Antigravity" icon="rocket" href="/quickstarts/antigravity">
    Add Firecrawl MCP to Google's agentic IDE.
  </Card>

  <Card title="Gemini CLI" icon="gem" href="/quickstarts/gemini-cli">
    Wire Firecrawl MCP into Google Gemini CLI.
  </Card>

  <Card title="Nous Research" icon="brain" href="/quickstarts/nous-research">
    Use Firecrawl as a tool with Hermes models.
  </Card>

  <Card title="AutoGen" icon="robot" href="/quickstarts/autogen">
    Firecrawl tools inside Microsoft AutoGen multi-agent teams.
  </Card>
</CardGroup>

## SDKs

Official, typed SDKs covering the full Firecrawl API surface. Point your agent at the language matching your stack.

<CardGroup>
  <Card title="Python" icon="python" href="/sdks/python" />

  <Card title="Node" icon="node-js" href="/sdks/node" />

  <Card title="Go" icon="golang" href="/sdks/go" />

  <Card title="Java" icon="java" href="/sdks/java" />

  <Card title="Ruby" icon="gem" href="/sdks/ruby" />

  <Card title="Rust" icon="rust" href="/sdks/rust" />

  <Card title=".NET" icon="microsoft" href="/sdks/dotnet" />

  <Card title="PHP" icon="php" href="/sdks/php" />

  <Card title="Elixir" icon="droplet" href="/sdks/elixir" />

  <Card title="CLI" icon="terminal" href="/sdks/cli" />
</CardGroup>

Firecrawl also has first-class SDK bindings for the major LLM SDKs and agent frameworks — see [LLM SDKs and Frameworks](/developer-guides/llm-sdks-and-frameworks/openai) for OpenAI, Anthropic, Gemini, Google ADK, Vercel AI SDK, LangChain, LangGraph, LlamaIndex, Mastra, and ElevenAgents.


# Scraping Amazon
Source: https://docs.firecrawl.dev/developer-guides/common-sites/amazon

Extract product data, prices, and reviews from Amazon using Firecrawl

<Info>
  Amazon is one of the most scraped e-commerce sites. This guide shows you how to effectively extract product data, pricing, reviews, and search results using Firecrawl's powerful features.
</Info>

## Setup

```bash theme={null}
npm install @mendable/firecrawl-js zod
```

## Overview

When scraping Amazon, you'll typically want to:

* Extract product information (title, price, availability)
* Get customer reviews and ratings
* Monitor price changes
* Search for products programmatically
* Track competitor listings

## Scrape with JSON Mode

Extract structured product data using Zod schemas.

```typescript theme={null}
import FirecrawlApp from '@mendable/firecrawl-js';
import { z } from 'zod';

// Define Zod schema
const ProductSchema = z.object({
    title: z.string(),
    price: z.string(),
    rating: z.number(),
    availability: z.string(),
    features: z.array(z.string())
});

const firecrawl = new FirecrawlApp({ apiKey: process.env.FIRECRAWL_API_KEY });

const result = await firecrawl.scrape('https://www.amazon.com/dp/B0DZZWMB2L', {
    formats: [{
        type: 'json',
        schema: z.toJSONSchema(ProductSchema)
    }],
});

// Parse and validate with Zod
const jsonData = typeof result.json === 'string' ? JSON.parse(result.json) : result.json;
const validated = ProductSchema.parse(jsonData);

console.log('✅ Validated product data:');
console.log(validated);
```

## Search

Find products on Amazon.

```typescript theme={null}
import FirecrawlApp from '@mendable/firecrawl-js';

const firecrawl = new FirecrawlApp({ apiKey: process.env.FIRECRAWL_API_KEY });

const searchResult = await firecrawl.search('gaming laptop site:amazon.com', {
    limit: 10,
    sources: [{ type: 'web' }], // { type: 'news' }, { type: 'images' }
    scrapeOptions: {
        formats: ['markdown']
    }
});

console.log(searchResult);
```

## Scrape

Scrape a single Amazon product page.

```typescript theme={null}
import FirecrawlApp from '@mendable/firecrawl-js';

const firecrawl = new FirecrawlApp({ apiKey: process.env.FIRECRAWL_API_KEY });

const result = await firecrawl.scrape('https://www.amazon.com/ASUS-ROG-Strix-Gaming-Laptop/dp/B0DZZWMB2L', {
    formats: ['markdown'], // i.e. html, links, etc.
    onlyMainContent: true
});

console.log(result);
```

## Map

Discover all available URLs on Amazon product or category pages. Note: Map returns URLs only, without content.

```typescript theme={null}
import FirecrawlApp from '@mendable/firecrawl-js';

const firecrawl = new FirecrawlApp({ apiKey: process.env.FIRECRAWL_API_KEY });

const mapResult = await firecrawl.map('https://www.amazon.com/Best-Sellers-Electronics/zgbs/electronics');

console.log(mapResult.links);
// Returns array of URLs without content
```

## Crawl

Crawl multiple pages from Amazon category or search results.

```typescript theme={null}
import FirecrawlApp from '@mendable/firecrawl-js';

const firecrawl = new FirecrawlApp({ apiKey: process.env.FIRECRAWL_API_KEY });

const crawlResult = await firecrawl.crawl('https://www.amazon.com/s?k=mechanical+keyboards', {
    limit: 10,
    scrapeOptions: {
        formats: ['markdown']
    }
});

console.log(crawlResult.data);
```

## Batch Scrape

Scrape multiple Amazon product URLs simultaneously.

```typescript theme={null}
import FirecrawlApp from '@mendable/firecrawl-js';

const firecrawl = new FirecrawlApp({ apiKey: process.env.FIRECRAWL_API_KEY });

// Wait for completion
const job = await firecrawl.batchScrape([
    'https://www.amazon.com/ASUS-ROG-Strix-Gaming-Laptop/dp/B0DZZWMB2L',
    'https://www.amazon.com/Razer-Blade-Gaming-Laptop-Lightweight/dp/B0FP47DNFQ',
    'https://www.amazon.com/HP-2025-Omen-Gaming-Laptop/dp/B0FL4RMGSH'],
    {
        options: {
            formats: ['markdown']
        },
        pollInterval: 2,
        timeout: 120
    }
);


console.log(job.status, job.completed, job.total);

console.log(job);
```


# Scraping Etsy
Source: https://docs.firecrawl.dev/developer-guides/common-sites/etsy

Extract handmade products, shop data, and pricing from Etsy marketplace

<Info>
  Etsy is a global marketplace for unique and creative goods. This guide shows you how to extract product listings, shop information, reviews, and trending items using Firecrawl.
</Info>

## Setup

```bash theme={null}
npm install @mendable/firecrawl-js zod
```

## Overview

When scraping Etsy, you'll typically want to:

* Extract product listings and variations
* Get shop information and ratings
* Monitor trending items and categories
* Track pricing and sales data
* Extract customer reviews

## Scrape with JSON Mode

Extract structured listing data using Zod schemas.

```typescript theme={null}
import FirecrawlApp from '@mendable/firecrawl-js';
import { z } from 'zod';

// Define Zod schema
const ListingSchema = z.object({
    title: z.string(),
    price: z.string(),
    shopName: z.string(),
    rating: z.number()
});

const firecrawl = new FirecrawlApp({ apiKey: process.env.FIRECRAWL_API_KEY });

const result = await firecrawl.scrape('https://www.etsy.com/listing/1844315896/handmade-925-sterling-silver-jewelry-set', {
    formats: [{
        type: 'json',
        schema: z.toJSONSchema(ListingSchema)
    }],
});

// Parse and validate with Zod
const jsonData = typeof result.json === 'string' ? JSON.parse(result.json) : result.json;
const validated = ListingSchema.parse(jsonData);

console.log('✅ Validated listing data:');
console.log(validated);
```

## Search

Find products on Etsy marketplace.

```typescript theme={null}
import FirecrawlApp from '@mendable/firecrawl-js';

const firecrawl = new FirecrawlApp({ apiKey: process.env.FIRECRAWL_API_KEY });

const searchResult = await firecrawl.search('handmade jewelry site:etsy.com', {
    limit: 10,
    sources: [{ type: 'web' }], // { type: 'news' }, { type: 'images' }
    scrapeOptions: {
        formats: ['markdown']
    }
});

console.log(searchResult);
```

## Scrape

Scrape a single Etsy product listing.

```typescript theme={null}
import FirecrawlApp from '@mendable/firecrawl-js';

const firecrawl = new FirecrawlApp({ apiKey: process.env.FIRECRAWL_API_KEY });

const result = await firecrawl.scrape('https://www.etsy.com/listing/1844315896/handmade-925-sterling-silver-jewelry-set', {
    formats: ['markdown'], // i.e. html, links, etc.
    onlyMainContent: true
});

console.log(result);
```

## Map

Discover all available URLs in an Etsy shop or category. Note: Map returns URLs only, without content.

```typescript theme={null}
import FirecrawlApp from '@mendable/firecrawl-js';

const firecrawl = new FirecrawlApp({ apiKey: process.env.FIRECRAWL_API_KEY });

const mapResult = await firecrawl.map('https://www.etsy.com/shop/YourShopName');

console.log(mapResult.links);
// Returns array of URLs without content
```

## Crawl

Crawl multiple pages from Etsy shop or category.

```typescript theme={null}
import FirecrawlApp from '@mendable/firecrawl-js';

const firecrawl = new FirecrawlApp({ apiKey: process.env.FIRECRAWL_API_KEY });

const crawlResult = await firecrawl.crawl('https://www.etsy.com/c/jewelry', {
    limit: 10,
    scrapeOptions: {
        formats: ['markdown']
    }
});

console.log(crawlResult.data);
```

## Batch Scrape

Scrape multiple Etsy listing URLs simultaneously.

```typescript theme={null}
import FirecrawlApp from '@mendable/firecrawl-js';

const firecrawl = new FirecrawlApp({ apiKey: process.env.FIRECRAWL_API_KEY });

// Wait for completion
const job = await firecrawl.batchScrape([
    'https://www.etsy.com/listing/1844315896/handmade-925-sterling-silver-jewelry-set',
    'https://www.etsy.com/market/handmade_jewelry',
    'https://www.etsy.com/market/jewelry_handmade'],
    {
        options: {
            formats: ['markdown']
        },
        pollInterval: 2,
        timeout: 120
    }
);


console.log(job.status, job.completed, job.total);

console.log(job);
```


# Scraping GitHub
Source: https://docs.firecrawl.dev/developer-guides/common-sites/github

Learn how to scrape GitHub using Firecrawl's core features

Learn how to use Firecrawl's core features to scrape GitHub repositories, issues, and documentation.

## Setup

```bash theme={null}
npm install @mendable/firecrawl-js zod
```

## Scrape with JSON Mode

Extract structured data from repositories using Zod schemas.

```typescript theme={null}
import FirecrawlApp from '@mendable/firecrawl-js';
import { z } from 'zod';

const firecrawl = new FirecrawlApp({ apiKey: process.env.FIRECRAWL_API_KEY });

const result = await firecrawl.scrape('https://github.com/firecrawl/firecrawl', {
    formats: [{
        type: 'json',
        schema: z.object({
            name: z.string(),
            description: z.string(),
            stars: z.number(),
            forks: z.number(),
            language: z.string(),
            topics: z.array(z.string())
        })
    }]
});

console.log(result.json);
```

## Search

Find repositories, issues, or documentation on GitHub.

```typescript theme={null}
import FirecrawlApp from '@mendable/firecrawl-js';

const firecrawl = new FirecrawlApp({ apiKey: process.env.FIRECRAWL_API_KEY });

const searchResult = await firecrawl.search('machine learning site:github.com', {
    limit: 10,
    sources: [{ type: 'web' }], // { type: 'news' }, { type: 'images' }
    scrapeOptions: {
        formats: ['markdown']
    }
});

console.log(searchResult);
```

## Scrape

Scrape a single GitHub page - repository, issue, or file.

```typescript theme={null}
import FirecrawlApp from '@mendable/firecrawl-js';

const firecrawl = new FirecrawlApp({ apiKey: process.env.FIRECRAWL_API_KEY });

const result = await firecrawl.scrape('https://github.com/firecrawl/firecrawl', {
    formats: ['markdown'] // i.e. html, links, etc.
});

console.log(result);
```

## Map

Discover all available URLs in a repository or documentation site. Note: Map returns URLs only, without content.

```typescript theme={null}
import FirecrawlApp from '@mendable/firecrawl-js';

const firecrawl = new FirecrawlApp({ apiKey: process.env.FIRECRAWL_API_KEY });

const mapResult = await firecrawl.map('https://github.com/vercel/next.js/tree/canary/docs');

console.log(mapResult.links);
// Returns array of URLs without content
```

## Crawl

Crawl multiple pages from a repository or documentation.

```typescript theme={null}
import FirecrawlApp from '@mendable/firecrawl-js';

const firecrawl = new FirecrawlApp({ apiKey: process.env.FIRECRAWL_API_KEY });

const crawlResult = await firecrawl.crawl('https://github.com/facebook/react/wiki', {
    limit: 10,
    scrapeOptions: {
        formats: ['markdown']
    }
});

console.log(crawlResult.data);
```

## Batch Scrape

Scrape multiple GitHub URLs simultaneously.

```typescript theme={null}
import FirecrawlApp from '@mendable/firecrawl-js';

const firecrawl = new FirecrawlApp({ apiKey: process.env.FIRECRAWL_API_KEY });

// Wait for completion
const job = await firecrawl.batchScrape([
    'https://github.com/vercel/next.js',
    'https://github.com/facebook/react',
    'https://github.com/microsoft/typescript'],
    {
        options: {
            formats: ['markdown']
        },
        pollInterval: 2,
        timeout: 120
    }
);


console.log(job.status, job.completed, job.total);

console.log(job);
```

## Batch Scrape with JSON Mode

Extract structured data from multiple repositories at once.

```typescript theme={null}
import FirecrawlApp from '@mendable/firecrawl-js';
import { z } from 'zod';

const firecrawl = new FirecrawlApp({ apiKey: process.env.FIRECRAWL_API_KEY });

// Wait for completion
const job = await firecrawl.batchScrape([
    'https://github.com/vercel/next.js',
    'https://github.com/facebook/react'],
    {
        options: {
            formats: [{
                type: 'json',
                schema: z.object({
                    name: z.string(),
                    description: z.string(),
                    stars: z.number(),
                    language: z.string()
                })
            }]
        },
        pollInterval: 2,
        timeout: 120
    }
);


console.log(job.status, job.completed, job.total);

console.log(job);
```


# Scraping Wikipedia
Source: https://docs.firecrawl.dev/developer-guides/common-sites/wikipedia

Extract articles, infoboxes, and build knowledge graphs from Wikipedia

Learn how to effectively scrape Wikipedia for research, knowledge extraction, and building AI applications.

## Setup

```bash theme={null}
npm install @mendable/firecrawl-js zod
```

## Use Cases

* Research automation and fact-checking
* Building knowledge graphs
* Multi-language content extraction
* Educational content aggregation
* Entity information extraction

## Scrape with JSON Mode

Extract structured data from Wikipedia articles using Zod schemas.

```typescript theme={null}
import FirecrawlApp from '@mendable/firecrawl-js';
import { z } from 'zod';

const firecrawl = new FirecrawlApp({ apiKey: process.env.FIRECRAWL_API_KEY });

const result = await firecrawl.scrape('https://en.wikipedia.org/wiki/JavaScript', {
    formats: [{
        type: 'json',
        schema: z.object({
            name: z.string(),
            creator: z.string(),
            firstAppeared: z.string(),
            typingDiscipline: z.string(),
            website: z.string()
        })
    }]
});

console.log(result.json);
```

## Search

Find articles on Wikipedia.

```typescript theme={null}
import FirecrawlApp from '@mendable/firecrawl-js';

const firecrawl = new FirecrawlApp({ apiKey: process.env.FIRECRAWL_API_KEY });

const searchResult = await firecrawl.search('quantum computing site:en.wikipedia.org', {
    limit: 10,
    sources: [{ type: 'web' }], // { type: 'news' }, { type: 'images' }
    scrapeOptions: {
        formats: ['markdown']
    }
});

console.log(searchResult);
```

## Scrape

Scrape a single Wikipedia article.

```typescript theme={null}
import FirecrawlApp from '@mendable/firecrawl-js';

const firecrawl = new FirecrawlApp({ apiKey: process.env.FIRECRAWL_API_KEY });

const result = await firecrawl.scrape('https://en.wikipedia.org/wiki/Artificial_intelligence', {
    formats: ['markdown'], // i.e. html, links, etc.
    onlyMainContent: true
});

console.log(result);
```

## Map

Discover all available URLs in a Wikipedia portal or category. Note: Map returns URLs only, without content.

```typescript theme={null}
import FirecrawlApp from '@mendable/firecrawl-js';

const firecrawl = new FirecrawlApp({ apiKey: process.env.FIRECRAWL_API_KEY });

const mapResult = await firecrawl.map('https://en.wikipedia.org/wiki/Portal:Computer_science');

console.log(mapResult.links);
// Returns array of URLs without content
```

## Crawl

Crawl multiple pages from Wikipedia documentation or categories.

```typescript theme={null}
import FirecrawlApp from '@mendable/firecrawl-js';

const firecrawl = new FirecrawlApp({ apiKey: process.env.FIRECRAWL_API_KEY });

const crawlResult = await firecrawl.crawl('https://en.wikipedia.org/wiki/Portal:Artificial_intelligence', {
    limit: 10,
    scrapeOptions: {
        formats: ['markdown']
    }
});

console.log(crawlResult.data);
```

## Batch Scrape

Scrape multiple Wikipedia URLs simultaneously.

```typescript theme={null}
import FirecrawlApp from '@mendable/firecrawl-js';

const firecrawl = new FirecrawlApp({ apiKey: process.env.FIRECRAWL_API_KEY });

// Wait for completion
const job = await firecrawl.batchScrape([
    'https://en.wikipedia.org/wiki/Machine_learning',
    'https://en.wikipedia.org/wiki/Artificial_intelligence',
    'https://en.wikipedia.org/wiki/Deep_learning'],
    {
        options: {
            formats: ['markdown']
        },
        pollInterval: 2,
        timeout: 120
    }
);


console.log(job.status, job.completed, job.total);

console.log(job);
```


# Building an AI Research Assistant with Firecrawl and AI SDK
Source: https://docs.firecrawl.dev/developer-guides/cookbooks/ai-research-assistant-cookbook

Build a complete AI-powered research assistant with web scraping and search capabilities

Build a complete AI-powered research assistant that can scrape websites and search the web to answer questions. The assistant automatically decides when to use web scraping or search tools to gather information, then provides comprehensive answers based on collected data.

<img alt="AI research assistant chatbot interface showing real-time web scraping with Firecrawl and conversational responses powered by OpenAI" />

## What You'll Build

An AI chat interface where users can ask questions about any topic. The AI assistant automatically decides when to use web scraping or search tools to gather information, then provides comprehensive answers based on the data it collects.

## Prerequisites

* Node.js 18 or later installed
* An OpenAI API key from [platform.openai.com](https://platform.openai.com)
* A Firecrawl API key from [firecrawl.dev](https://firecrawl.dev)
* Basic knowledge of React and Next.js

<Steps>
  <Step title="Create a New Next.js Project">
    Start by creating a fresh Next.js application and navigating into the project directory:

    ```bash theme={null}
    npx create-next-app@latest ai-sdk-firecrawl && cd ai-sdk-firecrawl
    ```

    When prompted, select the following options:

    * TypeScript: Yes
    * ESLint: Yes
    * Tailwind CSS: Yes
    * App Router: Yes
    * Use `src/` directory: No
    * Import alias: Yes (@/\*)
  </Step>

  <Step title="Install Dependencies">
    ### Install AI SDK Packages

    The AI SDK is a TypeScript toolkit that provides a unified API for working with different LLM providers:

    ```bash theme={null}
    npm i ai @ai-sdk/react zod
    ```

    These packages provide:

    * `ai`: Core SDK with streaming, tool calling, and response handling
    * `@ai-sdk/react`: React hooks like `useChat` for building chat interfaces
    * `zod`: Schema validation for tool inputs

    Learn more at [ai-sdk.dev/docs](https://ai-sdk.dev/docs).

    ### Install AI Elements

    AI Elements provides pre-built UI components for AI applications. Run the following command to scaffold all the necessary components:

    ```bash theme={null}
    npx ai-elements@latest
    ```

    This sets up AI Elements in your project, including conversation components, message displays, prompt inputs, and tool call visualizations.

    Documentation: [ai-sdk.dev/elements/overview](https://ai-sdk.dev/elements/overview).

    ### Install OpenAI Provider

    Install the OpenAI provider to connect with OpenAI's models:

    ```bash theme={null}
    npm install @ai-sdk/openai
    ```
  </Step>

  <Step title="Build the Frontend Chat Interface">
    Create the main page at `app/page.tsx` and copy the code from the Code tab below. This will be the chat interface where users interact with the AI assistant.

    <Tabs>
      <Tab title="Preview">
        <img alt="AI research assistant chatbot interface showing real-time web scraping with Firecrawl and conversational responses powered by OpenAI" />
      </Tab>

      <Tab title="Code">
        ```typescript app/page.tsx theme={null}
        "use client";

        import {
          Conversation,
          ConversationContent,
          ConversationScrollButton,
        } from "@/components/ai-elements/conversation";
        import {
          PromptInput,
          PromptInputActionAddAttachments,
          PromptInputActionMenu,
          PromptInputActionMenuContent,
          PromptInputActionMenuTrigger,
          PromptInputAttachment,
          PromptInputAttachments,
          PromptInputBody,
          PromptInputButton,
          PromptInputHeader,
          type PromptInputMessage,
          PromptInputSelect,
          PromptInputSelectContent,
          PromptInputSelectItem,
          PromptInputSelectTrigger,
          PromptInputSelectValue,
          PromptInputSubmit,
          PromptInputTextarea,
          PromptInputFooter,
          PromptInputTools,
        } from "@/components/ai-elements/prompt-input";
        import {
          MessageResponse,
          Message,
          MessageContent,
          MessageActions,
          MessageAction,
        } from "@/components/ai-elements/message";

        import { Fragment, useState } from "react";
        import { useChat } from "@ai-sdk/react";
        import type { ToolUIPart } from "ai";
        import {
          Tool,
          ToolContent,
          ToolHeader,
          ToolInput,
          ToolOutput,
        } from "@/components/ai-elements/tool";

        import { CopyIcon, GlobeIcon, RefreshCcwIcon } from "lucide-react";
        import {
          Source,
          Sources,
          SourcesContent,
          SourcesTrigger,
        } from "@/components/ai-elements/sources";
        import {
          Reasoning,
          ReasoningContent,
          ReasoningTrigger,
        } from "@/components/ai-elements/reasoning";
        import { Loader } from "@/components/ai-elements/loader";

        const models = [
          {
            name: "GPT 5 Mini (Thinking)",
            value: "gpt-5-mini",
          },
          {
            name: "GPT 4o Mini",
            value: "gpt-4o-mini",
          },
        ];

        const ChatBotDemo = () => {
          const [input, setInput] = useState("");
          const [model, setModel] = useState<string>(models[0].value);
          const [webSearch, setWebSearch] = useState(false);
          const { messages, sendMessage, status, regenerate } = useChat();

          const handleSubmit = (message: PromptInputMessage) => {
            const hasText = Boolean(message.text);
            const hasAttachments = Boolean(message.files?.length);

            if (!(hasText || hasAttachments)) {
              return;
            }

            sendMessage(
              {
                text: message.text || "Sent with attachments",
                files: message.files,
              },
              {
                body: {
                  model: model,
                  webSearch: webSearch,
                },
              }
            );
            setInput("");
          };

          return (
            <div className="max-w-4xl mx-auto p-6 relative size-full h-screen">
              <div className="flex flex-col h-full">
                <Conversation className="h-full">
                  <ConversationContent>
                    {messages.map((message) => (
                      <div key={message.id}>
                        {message.role === "assistant" &&
                          message.parts.filter((part) => part.type === "source-url")
                            .length > 0 && (
                            <Sources>
                              <SourcesTrigger
                                count={
                                  message.parts.filter(
                                    (part) => part.type === "source-url"
                                  ).length
                                }
                              />
                              {message.parts
                                .filter((part) => part.type === "source-url")
                                .map((part, i) => (
                                  <SourcesContent key={`${message.id}-${i}`}>
                                    <Source
                                      key={`${message.id}-${i}`}
                                      href={part.url}
                                      title={part.url}
                                    />
                                  </SourcesContent>
                                ))}
                            </Sources>
                          )}
                        {message.parts.map((part, i) => {
                          switch (part.type) {
                            case "text":
                              return (
                                <Fragment key={`${message.id}-${i}`}>
                                  <Message from={message.role}>
                                    <MessageContent>
                                      <MessageResponse>{part.text}</MessageResponse>
                                    </MessageContent>
                                  </Message>
                                  {message.role === "assistant" &&
                                    i === messages.length - 1 && (
                                      <MessageActions className="mt-2">
                                        <MessageAction
                                          onClick={() => regenerate()}
                                          label="Retry"
                                        >
                                          <RefreshCcwIcon className="size-3" />
                                        </MessageAction>
                                        <MessageAction
                                          onClick={() =>
                                            navigator.clipboard.writeText(part.text)
                                          }
                                          label="Copy"
                                        >
                                          <CopyIcon className="size-3" />
                                        </MessageAction>
                                      </MessageActions>
                                    )}
                                </Fragment>
                              );
                            case "reasoning":
                              return (
                                <Reasoning
                                  key={`${message.id}-${i}`}
                                  className="w-full"
                                  isStreaming={
                                    status === "streaming" &&
                                    i === message.parts.length - 1 &&
                                    message.id === messages.at(-1)?.id
                                  }
                                >
                                  <ReasoningTrigger />
                                  <ReasoningContent>{part.text}</ReasoningContent>
                                </Reasoning>
                              );
                            default: {
                              if (part.type.startsWith("tool-")) {
                                const toolPart = part as ToolUIPart;
                                return (
                                  <Tool
                                    key={`${message.id}-${i}`}
                                    defaultOpen={toolPart.state === "output-available"}
                                  >
                                    <ToolHeader
                                      type={toolPart.type}
                                      state={toolPart.state}
                                    />
                                    <ToolContent>
                                      <ToolInput input={toolPart.input} />
                                      <ToolOutput
                                        output={toolPart.output}
                                        errorText={toolPart.errorText}
                                      />
                                    </ToolContent>
                                  </Tool>
                                );
                              }
                              return null;
                            }
                          }
                        })}
                      </div>
                    ))}
                    {status === "submitted" && <Loader />}
                  </ConversationContent>
                  <ConversationScrollButton />
                </Conversation>

                <PromptInput
                  onSubmit={handleSubmit}
                  className="mt-4"
                  globalDrop
                  multiple
                >
                  <PromptInputHeader>
                    <PromptInputAttachments>
                      {(attachment) => <PromptInputAttachment data={attachment} />}
                    </PromptInputAttachments>
                  </PromptInputHeader>
                  <PromptInputBody>
                    <PromptInputTextarea
                      onChange={(e) => setInput(e.target.value)}
                      value={input}
                    />
                  </PromptInputBody>
                  <PromptInputFooter>
                    <PromptInputTools>
                      <PromptInputActionMenu>
                        <PromptInputActionMenuTrigger />
                        <PromptInputActionMenuContent>
                          <PromptInputActionAddAttachments />
                        </PromptInputActionMenuContent>
                      </PromptInputActionMenu>
                      <PromptInputButton
                        variant={webSearch ? "default" : "ghost"}
                        onClick={() => setWebSearch(!webSearch)}
                      >
                        <GlobeIcon size={16} />
                        <span>Search</span>
                      </PromptInputButton>
                      <PromptInputSelect
                        onValueChange={(value) => {
                          setModel(value);
                        }}
                        value={model}
                      >
                        <PromptInputSelectTrigger>
                          <PromptInputSelectValue />
                        </PromptInputSelectTrigger>
                        <PromptInputSelectContent>
                          {models.map((model) => (
                            <PromptInputSelectItem
                              key={model.value}
                              value={model.value}
                            >
                              {model.name}
                            </PromptInputSelectItem>
                          ))}
                        </PromptInputSelectContent>
                      </PromptInputSelect>
                    </PromptInputTools>
                    <PromptInputSubmit disabled={!input && !status} status={status} />
                  </PromptInputFooter>
                </PromptInput>
              </div>
            </div>
          );
        };

        export default ChatBotDemo;
        ```
      </Tab>
    </Tabs>

    ### Understanding the Frontend

    The frontend uses AI Elements components to provide a complete chat interface:

    **Key Features:**

    * **Conversation Display**: The `Conversation` component automatically handles message scrolling and display
    * **Message Rendering**: Each message part is rendered based on its type (text, reasoning, tool calls)
    * **Tool Visualization**: Tool calls are displayed with collapsible sections showing inputs and outputs
    * **Interactive Controls**: Users can toggle web search, select models, and attach files
    * **Message Actions**: Copy and retry actions for assistant messages
  </Step>

  <Step title="Add Markdown Rendering Support">
    To ensure the markdown from the LLM is correctly rendered, add the following import to your `app/globals.css` file:

    ```css theme={null}
    @source "../node_modules/streamdown/dist/index.js";
    ```

    This imports the necessary styles for rendering markdown content in the message responses.
  </Step>

  <Step title="Build the Basic API Route">
    Create the chat API endpoint at `app/api/chat/route.ts`. This route will handle incoming messages and stream responses from the AI.

    ```typescript theme={null}
    import { streamText, UIMessage, convertToModelMessages } from "ai";
    import { createOpenAI } from "@ai-sdk/openai";

    const openai = createOpenAI({
      apiKey: process.env.OPENAI_API_KEY!,
    });

    // Allow streaming responses up to 5 minutes
    export const maxDuration = 300;

    export async function POST(req: Request) {
      const {
        messages,
        model,
        webSearch,
      }: {
        messages: UIMessage[];
        model: string;
        webSearch: boolean;
      } = await req.json();

      const result = streamText({
        model: openai(model),
        messages: convertToModelMessages(messages),
        system:
          "You are a helpful assistant that can answer questions and help with tasks.",
      });

      // send sources and reasoning back to the client
      return result.toUIMessageStreamResponse({
        sendSources: true,
        sendReasoning: true,
      });
    }
    ```

    This basic route:

    * Receives messages from the frontend
    * Uses the OpenAI model selected by the user
    * Streams responses back to the client
    * Doesn't include tools yet - we'll add those next
  </Step>

  <Step title="Configure Environment Variables">
    Create a `.env.local` file in your project root:

    ```bash theme={null}
    touch .env.local
    ```

    Add your OpenAI API key:

    ```env theme={null}
    OPENAI_API_KEY=sk-your-openai-api-key
    ```

    The `OPENAI_API_KEY` is required for the AI model to function.
  </Step>

  <Step title="Test the Basic Chat">
    Now you can test the AI SDK chatbot without Firecrawl integration. Start the development server:

    ```bash theme={null}
    npm run dev
    ```

    Open [localhost:3000](http://localhost:3000) in your browser and test the basic chat functionality. The assistant should respond to messages, but won't have web scraping or search capabilities yet.

    <img alt="Basic AI chatbot without web scraping capabilities" />
  </Step>

  <Step title="Add Firecrawl Tools">
    Now let's enhance the assistant with web scraping and search capabilities using Firecrawl.

    ### Install Firecrawl SDK

    Firecrawl converts websites into LLM-ready formats with scraping and search capabilities:

    ```bash theme={null}
    npm i @mendable/firecrawl-js
    ```

    ### Create the Tools File

    Create a `lib` folder and add a `tools.ts` file inside it:

    ```bash theme={null}
    mkdir lib && touch lib/tools.ts
    ```

    Add the following code to define the web scraping and search tools:

    ```typescript lib/tools.ts theme={null}
    import FirecrawlApp from "@mendable/firecrawl-js";
    import { tool } from "ai";
    import { z } from "zod";

    const firecrawl = new FirecrawlApp({ apiKey: process.env.FIRECRAWL_API_KEY });

    export const scrapeWebsiteTool = tool({
      description: 'Scrape content from any website URL',
      inputSchema: z.object({
        url: z.string().url().describe('The URL to scrape')
      }),
      execute: async ({ url }) => {
        console.log('Scraping:', url);
        const result = await firecrawl.scrape(url, {
          formats: ['markdown'],
          onlyMainContent: true,
          timeout: 30000
        });
        console.log('Scraped content preview:', result.markdown?.slice(0, 200) + '...');
        return { content: result.markdown };
      }
    });

    export const searchWebTool = tool({
      description: 'Search the web using Firecrawl',
      inputSchema: z.object({
        query: z.string().describe('The search query'),
        limit: z.number().optional().describe('Number of results'),
        location: z.string().optional().describe('Location for localized results'),
        tbs: z.string().optional().describe('Time filter (qdr:h, qdr:d, qdr:w, qdr:m, qdr:y)'),
        sources: z.array(z.enum(['web', 'news', 'images'])).optional().describe('Result types'),
        categories: z.array(z.enum(['github', 'research', 'pdf'])).optional().describe('Filter categories'),
      }),
      execute: async ({ query, limit, location, tbs, sources, categories }) => {
        console.log('Searching:', query);
        const response = await firecrawl.search(query, {
          ...(limit && { limit }),
          ...(location && { location }),
          ...(tbs && { tbs }),
          ...(sources && { sources }),
          ...(categories && { categories }),
        }) as { web?: Array<{ title?: string; url?: string; description?: string }> };

        const results = (response.web || []).map((item) => ({
          title: item.title || item.url || 'Untitled',
          url: item.url || '',
          description: item.description || '',
        }));

        console.log('Search results:', results.length);
        return { results };
      },
    });
    ```

    ### Understanding the Tools

    **Scrape Website Tool:**

    * Accepts a URL as input (validated by Zod schema)
    * Uses Firecrawl's `scrape` method to fetch the page as markdown
    * Extracts only the main content to reduce token usage
    * Returns the scraped content for the AI to analyze

    **Search Web Tool:**

    * Accepts a search query with optional filters
    * Uses Firecrawl's `search` method to find relevant web pages
    * Supports advanced filters like location, time range, and content categories
    * Returns structured results with titles, URLs, and descriptions

    Learn more about tools: [ai-sdk.dev/docs/foundations/tools](https://ai-sdk.dev/docs/foundations/tools).
  </Step>

  <Step title="Update the API Route with Firecrawl Tools">
    Now update your `app/api/chat/route.ts` to include the Firecrawl tools we just created.

    <Accordion title="View complete app/api/chat/route.ts code">
      ```typescript theme={null}
      import { streamText, UIMessage, stepCountIs, convertToModelMessages } from "ai";
      import { createOpenAI } from "@ai-sdk/openai";
      import { scrapeWebsiteTool, searchWebTool } from "@/lib/tools";

      const openai = createOpenAI({
        apiKey: process.env.OPENAI_API_KEY!,
      });

      export const maxDuration = 300;

      export async function POST(req: Request) {
        const {
          messages,
          model,
          webSearch,
        }: {
          messages: UIMessage[];
          model: string;
          webSearch: boolean;
        } = await req.json();

        const result = streamText({
          model: openai(model),
          messages: convertToModelMessages(messages),
          system:
            "You are a helpful assistant that can answer questions and help with tasks.",
          // Add the Firecrawl tools here
          tools: {
            scrapeWebsite: scrapeWebsiteTool,
            searchWeb: searchWebTool,
          },
          stopWhen: stepCountIs(5),
          toolChoice: webSearch ? "auto" : "none",
        });

        return result.toUIMessageStreamResponse({
          sendSources: true,
          sendReasoning: true,
        });
      }
      ```
    </Accordion>

    The key changes from the basic route:

    * Import `stepCountIs` from the AI SDK
    * Import the Firecrawl tools from `@/lib/tools`
    * Add the `tools` object with both `scrapeWebsite` and `searchWeb` tools
    * Add `stopWhen: stepCountIs(5)` to limit execution steps
    * Set `toolChoice` to "auto" when web search is enabled, "none" otherwise

    Learn more about `streamText`: [ai-sdk.dev/docs/reference/ai-sdk-core/stream-text](https://ai-sdk.dev/docs/reference/ai-sdk-core/stream-text).
  </Step>

  <Step title="Add Your Firecrawl API Key">
    Update your `.env.local` file to include your Firecrawl API key:

    ```env theme={null}
    OPENAI_API_KEY=sk-your-openai-api-key
    FIRECRAWL_API_KEY=fc-your-firecrawl-api-key
    ```

    Get your Firecrawl API key from [firecrawl.dev](https://firecrawl.dev).
  </Step>

  <Step title="Test the Complete Application">
    Restart your development server:

    ```bash theme={null}
    npm run dev
    ```

    <img alt="AI chatbot with active Firecrawl tools" />

    Open [localhost:3000](http://localhost:3000) and test the enhanced assistant:

    1. Toggle the "Search" button to enable web search
    2. Ask: "What are the latest features from firecrawl.dev?"
    3. Watch as the AI calls the `searchWeb` or `scrapeWebsite` tool
    4. See the tool execution in the UI with inputs and outputs
    5. Read the AI's analysis based on the scraped data
  </Step>
</Steps>

## How It Works

### Message Flow

1. **User sends a message**: The user types a question and clicks submit
2. **Frontend sends request**: `useChat` sends the message to `/api/chat` with the selected model and web search setting
3. **Backend processes message**: The API route receives the message and calls `streamText`
4. **AI decides on tools**: The model analyzes the question and decides whether to use `scrapeWebsite` or `searchWeb` (only if web search is enabled)
5. **Tools execute**: If tools are called, Firecrawl scrapes or searches the web
6. **AI generates response**: The model analyzes tool results and generates a natural language response
7. **Frontend displays results**: The UI shows tool calls and the final response in real-time

### Tool Calling Process

The AI SDK's tool calling system ([ai-sdk.dev/docs/foundations/tools](https://ai-sdk.dev/docs/foundations/tools)) works as follows:

1. The model receives the user's message and available tool descriptions
2. If the model determines a tool is needed, it generates a tool call with parameters
3. The SDK executes the tool function with those parameters
4. The tool result is sent back to the model
5. The model uses the result to generate its final response

This all happens automatically within a single `streamText` call, with results streaming to the frontend in real-time.

## Key Features

### Model Selection

The application supports multiple OpenAI models:

* **GPT-5 Mini (Thinking)**: Recent OpenAI model with advanced reasoning capabilities
* **GPT-4o Mini**: Fast and cost-effective model

Users can switch between models using the dropdown selector.

### Web Search Toggle

The Search button controls whether the AI can use Firecrawl tools:

* **Enabled**: AI can call `scrapeWebsite` and `searchWeb` tools as needed
* **Disabled**: AI responds only with its training knowledge

This gives users control over when to use web data versus the model's built-in knowledge.

## Customization Ideas

### Add More Tools

Extend the assistant with additional tools:

* Database lookups for internal company data
* CRM integration to fetch customer information
* Email sending capabilities
* Document generation

Each tool follows the same pattern: define a schema with Zod, implement the execute function, and register it in the `tools` object.

### Change the AI Model

Swap OpenAI for another provider:

```typescript theme={null}
import { anthropic } from "@ai-sdk/anthropic";

const result = streamText({
  model: anthropic("claude-4.5-sonnet"),
  // ... rest of config
});
```

The AI SDK supports 20+ providers with the same API. Learn more: [ai-sdk.dev/docs/foundations/providers-and-models](https://ai-sdk.dev/docs/foundations/providers-and-models).

### Customize the UI

AI Elements components are built on shadcn/ui, so you can:

* Modify component styles in the component files
* Add new variants to existing components
* Create custom components that match the design system

## Best Practices

1. **Use appropriate tools**: Choose `searchWeb` to find relevant pages first, `scrapeWebsite` for single pages, or let the AI decide

2. **Monitor API usage**: Track your Firecrawl and OpenAI API usage to avoid unexpected costs

3. **Handle errors gracefully**: The tools include error handling, but consider adding user-facing error messages

4. **Optimize performance**: Use streaming to provide immediate feedback and consider caching frequently accessed content

5. **Set reasonable limits**: The `stopWhen: stepCountIs(5)` prevents excessive tool calls and runaway costs

***

## Related Resources

<CardGroup>
  <Card title="AI SDK Documentation" href="https://ai-sdk.dev/docs">
    Explore the AI SDK for building AI-powered applications with streaming, tool
    calling, and multi-provider support.
  </Card>

  <Card title="AI Elements Components" href="https://ai-sdk.dev/elements/overview">
    Pre-built UI components for AI applications built on shadcn/ui.
  </Card>
</CardGroup>


# Building a Brand Style Guide Generator with Firecrawl
Source: https://docs.firecrawl.dev/developer-guides/cookbooks/brand-style-guide-generator-cookbook

Generate professional PDF brand style guides by extracting design systems from any website using Firecrawl's branding format

Build a brand style guide generator that automatically extracts colors, typography, spacing, and visual identity from any website and compiles it into a professional PDF document.

<img alt="Brand style guide PDF generator using Firecrawl branding format to extract design system" />

## What You'll Build

A Node.js application that takes any website URL, extracts its complete brand identity using Firecrawl's branding format, and generates a polished PDF style guide with:

* Color palette with hex values
* Typography system (fonts, sizes, weights)
* Spacing and layout specifications
* Logo and brand imagery
* Theme information (light/dark mode)

<img alt="Example of generated brand style guide PDF with colors typography and spacing" />

## Prerequisites

* Node.js 18 or later installed
* A Firecrawl API key from [firecrawl.dev](https://firecrawl.dev)
* Basic knowledge of TypeScript and Node.js

<Steps>
  <Step title="Create a New Node.js Project">
    Start by creating a new directory for your project and initializing it:

    ```bash theme={null}
    mkdir brand-style-guide-generator && cd brand-style-guide-generator
    npm init -y
    ```

    Update your `package.json` to use ES modules:

    ```json package.json theme={null}
    {
      "name": "brand-style-guide-generator",
      "version": "1.0.0",
      "type": "module",
      "scripts": {
        "start": "npx tsx index.ts"
      }
    }
    ```
  </Step>

  <Step title="Install Dependencies">
    Install the required packages for web scraping and PDF generation:

    ```bash theme={null}
    npm i @mendable/firecrawl-js pdfkit
    npm i -D typescript tsx @types/node @types/pdfkit
    ```

    These packages provide:

    * `@mendable/firecrawl-js`: Firecrawl SDK for extracting brand identity from websites
    * `pdfkit`: PDF document generation library
    * `tsx`: TypeScript execution for Node.js
  </Step>

  <Step title="Build the Brand Style Guide Generator">
    Create the main application file at `index.ts`. This script extracts brand identity from a URL and generates a professional PDF style guide.

    ```typescript index.ts theme={null}
    import Firecrawl from "@mendable/firecrawl-js";
    import PDFDocument from "pdfkit";
    import fs from "fs";

    const API_KEY = "fc-YOUR-API-KEY";
    const URL = "https://firecrawl.dev";

    async function main() {
      const fc = new Firecrawl({ apiKey: API_KEY });
      const { branding: b } = (await fc.scrape(URL, { formats: ["branding"] })) as any;

      const doc = new PDFDocument({ size: "A4", margin: 50 });
      doc.pipe(fs.createWriteStream("brand-style-guide.pdf"));

      // Fetch logo (PNG/JPG only)
      let logoImg: Buffer | null = null;
      try {
        const logoUrl = b.images?.favicon || b.images?.ogImage;
        if (logoUrl?.match(/\.(png|jpg|jpeg)$/i)) {
          const res = await fetch(logoUrl);
          logoImg = Buffer.from(await res.arrayBuffer());
        }
      } catch {}

      // Header with logo
      doc.rect(0, 0, 595, 120).fill(b.colors?.primary || "#333");
      const titleX = logoImg ? 130 : 50;
      if (logoImg) doc.image(logoImg, 50, 30, { height: 60 });
      doc.fontSize(36).fillColor("#fff").text("Brand Style Guide", titleX, 38);
      doc.fontSize(14).text(URL, titleX, 80);

      // Colors
      doc.fontSize(18).fillColor("#333").text("Colors", 50, 160);
      const colors = Object.entries(b.colors || {}).filter(([, v]) => typeof v === "string" && (v as string).startsWith("#"));
      colors.forEach(([k, v], i) => {
        const x = 50 + i * 100;
        doc.rect(x, 195, 80, 80).fill(v as string);
        doc.fontSize(10).fillColor("#333").text(k, x, 282, { width: 80, align: "center" });
        doc.fontSize(9).fillColor("#888").text(v as string, x, 296, { width: 80, align: "center" });
      });

      // Typography
      doc.fontSize(18).fillColor("#333").text("Typography", 50, 340);
      doc.fontSize(13).fillColor("#444");
      doc.text(`Primary Font: ${b.typography?.fontFamilies?.primary || "—"}`, 50, 370);
      doc.text(`Heading Font: ${b.typography?.fontFamilies?.heading || "—"}`, 50, 392);
      doc.fontSize(12).fillColor("#666").text("Font Sizes:", 50, 422);
      Object.entries(b.typography?.fontSizes || {}).forEach(([k, v], i) => {
        doc.text(`${k.toUpperCase()}: ${v}`, 70, 445 + i * 22);
      });

      // Spacing & Theme
      doc.fontSize(18).fillColor("#333").text("Spacing & Theme", 320, 340);
      doc.fontSize(13).fillColor("#444");
      doc.text(`Base Unit: ${b.spacing?.baseUnit}px`, 320, 370);
      doc.text(`Border Radius: ${b.spacing?.borderRadius}`, 320, 392);
      doc.text(`Color Scheme: ${b.colorScheme}`, 320, 414);

      doc.end();
      console.log("Generated: brand-style-guide.pdf");
    }

    main();
    ```

    <Info>
      For this simple project, the API key is placed directly in the code. If you plan to push this to GitHub or share it with others, move the key to a `.env` file and use `process.env.FIRECRAWL_API_KEY` instead.
    </Info>

    Replace `fc-YOUR-API-KEY` with your Firecrawl API key from [firecrawl.dev](https://firecrawl.dev).

    ### Understanding the Code

    **Key Components:**

    * **Firecrawl Branding Format**: The `branding` format extracts comprehensive brand identity including colors, typography, spacing, and images
    * **PDFKit Document**: Creates a professional A4 PDF with proper margins and sections
    * **Color Swatches**: Renders visual color blocks with hex values and semantic names
    * **Typography Display**: Shows font families and sizes in an organized layout
    * **Spacing & Theme**: Documents the design system's spacing units and color scheme
  </Step>

  <Step title="Run the Generator">
    Run the script to generate a brand style guide:

    ```bash theme={null}
    npm start
    ```

    The script will:

    1. Extract the brand identity from the target URL using Firecrawl's branding format
    2. Generate a PDF named `brand-style-guide.pdf`
    3. Save it in your project directory

    To generate a style guide for a different website, simply change the `URL` constant in `index.ts`.
  </Step>
</Steps>

## How It Works

### Extraction Process

1. **URL Input**: The generator receives a target website URL
2. **Firecrawl Scrape**: Calls the `/scrape` endpoint with the `branding` format
3. **Brand Analysis**: Firecrawl analyzes the page's CSS, fonts, and visual elements
4. **Data Return**: Returns a structured `BrandingProfile` object with all design tokens

### PDF Generation Process

1. **Header Creation**: Generates a colored header using the primary brand color
2. **Logo Fetch**: Downloads and embeds the logo or favicon if available
3. **Color Palette**: Renders each color as a visual swatch with metadata
4. **Typography Section**: Documents font families, sizes, and weights
5. **Spacing Info**: Includes base units, border radius, and theme mode

### Branding Profile Structure

The [branding format](https://docs.firecrawl.dev/features/scrape#%2Fscrape-with-branding-endpoint) returns detailed brand information:

```typescript theme={null}
{
  colorScheme: "dark" | "light",
  logo: "https://example.com/logo.svg",
  colors: {
    primary: "#FF6B35",
    secondary: "#004E89",
    accent: "#F77F00",
    background: "#1A1A1A",
    textPrimary: "#FFFFFF",
    textSecondary: "#B0B0B0"
  },
  typography: {
    fontFamilies: { primary: "Inter", heading: "Inter", code: "Roboto Mono" },
    fontSizes: { h1: "48px", h2: "36px", body: "16px" },
    fontWeights: { regular: 400, medium: 500, bold: 700 }
  },
  spacing: {
    baseUnit: 8,
    borderRadius: "8px"
  },
  images: {
    logo: "https://example.com/logo.svg",
    favicon: "https://example.com/favicon.ico"
  }
}
```

## Key Features

### Automatic Color Extraction

The generator identifies and categorizes all brand colors:

* **Primary & Secondary**: Main brand colors
* **Accent**: Highlight and CTA colors
* **Background & Text**: UI foundation colors
* **Semantic Colors**: Success, warning, error states

### Typography Documentation

Captures the complete type system:

* **Font Families**: Primary, heading, and monospace fonts
* **Size Scale**: All heading and body text sizes
* **Font Weights**: Available weight variations

### Visual Output

The PDF includes:

* Color-coded header matching the brand
* Embedded logo when available
* Professional layout with clear hierarchy
* Metadata footer with generation date

<img alt="Side-by-side comparison of original website and generated brand style guide PDF" />

## Customization Ideas

### Add Component Documentation

Extend the generator to include UI component styles:

```typescript theme={null}
// Add after the Spacing & Theme section
if (b.components) {
  doc.addPage();
  doc.fontSize(20).fillColor("#333").text("UI Components", 50, 50);

  // Document button styles
  if (b.components.buttonPrimary) {
    const btn = b.components.buttonPrimary;
    doc.fontSize(14).text("Primary Button", 50, 90);
    doc.rect(50, 110, 120, 40).fill(btn.background);
    doc.fontSize(12).fillColor(btn.textColor).text("Button", 50, 120, { width: 120, align: "center" });
  }
}
```

### Export Multiple Formats

Add JSON export alongside the PDF:

```typescript theme={null}
// Add before doc.end()
fs.writeFileSync("brand-data.json", JSON.stringify(b, null, 2));
```

### Batch Processing

Generate guides for multiple websites:

```typescript theme={null}
const websites = [
  "https://stripe.com",
  "https://linear.app",
  "https://vercel.com"
];

for (const site of websites) {
  const { branding } = await fc.scrape(site, { formats: ["branding"] }) as any;
  // Generate PDF for each site...
}
```

### Custom PDF Themes

Create different PDF styles based on the extracted theme:

```typescript theme={null}
const isDarkMode = b.colorScheme === "dark";
const headerBg = isDarkMode ? b.colors?.background : b.colors?.primary;
const textColor = isDarkMode ? "#fff" : "#333";
```

## Best Practices

1. **Handle Missing Data**: Not all websites expose complete branding information. Always provide fallback values for missing properties.

2. **Respect Rate Limits**: When batch processing multiple sites, add delays between requests to respect Firecrawl's rate limits.

3. **Cache Results**: Store extracted branding data to avoid repeated API calls for the same site.

4. **Image Format Handling**: Some logos may be in formats PDFKit doesn't support (like SVG). Consider adding format conversion or graceful fallbacks.

5. **Error Handling**: Wrap the generation process in try-catch blocks and provide meaningful error messages.

***

## Related Resources

<CardGroup>
  <Card title="Branding Format Documentation" href="https://docs.firecrawl.dev/features/scrape#extract-brand-identity">
    Learn more about the branding format and all available properties you can extract.
  </Card>

  <Card title="Firecrawl Scrape API" href="https://docs.firecrawl.dev/api-reference/endpoint/scrape">
    Complete API reference for the scrape endpoint with all format options.
  </Card>

  <Card title="PDFKit Documentation" href="http://pdfkit.org/">
    Learn more about PDFKit for advanced PDF customization options.
  </Card>

  <Card title="Batch Scrape Guide" href="https://docs.firecrawl.dev/features/batch-scrape">
    Process multiple URLs efficiently with batch scraping.
  </Card>
</CardGroup>


# Full-Stack Templates
Source: https://docs.firecrawl.dev/developer-guides/examples

Explore real-world examples and tutorials for Firecrawl

## Official Examples

<CardGroup>
  <Card title="Open Lovable" href="https://github.com/firecrawl/open-lovable">
    <img alt="Open Lovable" />

    **Use Case**: Build a RAG-powered chatbot with live web data

    * [GitHub Repo](https://github.com/firecrawl/open-lovable)
  </Card>

  <Card title="Open Agent Builder" href="https://github.com/firecrawl/open-agent-builder">
    <img alt="Demo" />

    **Use Case**: Build and deploy AI agents with web scraping capabilities

    * [GitHub Repo](https://github.com/firecrawl/open-agent-builder)
  </Card>

  <Card title="Fireplexity" href="https://github.com/firecrawl/fireplexity">
    <img alt="Fireplexity" />

    **Use Case**: AI search engine with real-time citations, streaming responses, and live data

    * [GitHub Repo](https://github.com/firecrawl/fireplexity)
  </Card>

  <Card title="FireGEO" href="https://github.com/firecrawl/firegeo">
    <img alt="FireGEO Demo" />

    **Use Case**: SaaS starter with brand monitoring, authentication, and billing

    * [GitHub Repo](https://github.com/firecrawl/firegeo)
  </Card>

  <Card title="Fire Enrich" href="https://github.com/firecrawl/fire-enrich">
    <img alt="Fire Enrich" />

    **Use Case**: AI-powered data enrichment tool that transforms emails into rich datasets

    * [GitHub Repo](https://github.com/firecrawl/fire-enrich)
  </Card>

  <Card title="Firesearch" href="https://github.com/firecrawl/firesearch">
    <img alt="Firesearch" />

    **Use Case**: AI-powered deep research tool with validated answers and citations

    * [GitHub Repo](https://github.com/firecrawl/firesearch)
  </Card>

  <Card title="Firestarter" href="https://github.com/firecrawl/firestarter">
    <img alt="Firestarter" />

    **Use Case**: Instantly create AI chatbots for any website with RAG-powered search

    * [GitHub Repo](https://github.com/firecrawl/firestarter)
  </Card>

  <Card title="AI Ready Website" href="https://github.com/firecrawl/ai-ready-website">
    <img alt="AI Ready Website" />

    **Use Case**: Transform any website into AI-ready structured data

    * [GitHub Repo](https://github.com/firecrawl/ai-ready-website)
  </Card>

  <Card title="Open Researcher" href="https://github.com/firecrawl/open-researcher">
    <img alt="Open Researcher" />

    **Use Case**: AI-powered research assistant that gathers and analyzes web data

    * [GitHub Repo](https://github.com/firecrawl/open-researcher)
  </Card>
</CardGroup>


# Anthropic
Source: https://docs.firecrawl.dev/developer-guides/llm-sdks-and-frameworks/anthropic

Use Firecrawl with Claude for web scraping + AI workflows

Integrate Firecrawl with Claude to build AI applications powered by web data.

## Setup

```bash theme={null}
npm install @mendable/firecrawl-js @anthropic-ai/sdk zod zod-to-json-schema
```

Create `.env` file:

```bash theme={null}
FIRECRAWL_API_KEY=your_firecrawl_key
ANTHROPIC_API_KEY=your_anthropic_key
```

> **Note:** If using Node \< 20, install `dotenv` and add `import 'dotenv/config'` to your code.

## Scrape + Summarize

This example demonstrates a simple workflow: scrape a website and summarize the content using Claude.

```typescript theme={null}
import FirecrawlApp from '@mendable/firecrawl-js';
import Anthropic from '@anthropic-ai/sdk';

const firecrawl = new FirecrawlApp({ apiKey: process.env.FIRECRAWL_API_KEY });
const anthropic = new Anthropic({ apiKey: process.env.ANTHROPIC_API_KEY });

const scrapeResult = await firecrawl.scrape('https://firecrawl.dev', {
    formats: ['markdown']
});

console.log('Scraped content length:', scrapeResult.markdown?.length);

const message = await anthropic.messages.create({
    model: 'claude-haiku-4-5',
    max_tokens: 1024,
    messages: [
        { role: 'user', content: `Summarize in 100 words: ${scrapeResult.markdown}` }
    ]
});

console.log('Response:', message);
```

## Tool Use

This example shows how to use Claude's tool use feature to let the model decide when to scrape websites based on user requests.

```typescript theme={null}
import FirecrawlApp from '@mendable/firecrawl-js';
import { Anthropic } from '@anthropic-ai/sdk';
import { z } from 'zod';
import { zodToJsonSchema } from 'zod-to-json-schema';

const firecrawl = new FirecrawlApp({
    apiKey: process.env.FIRECRAWL_API_KEY
});

const anthropic = new Anthropic({
    apiKey: process.env.ANTHROPIC_API_KEY
});

const ScrapeArgsSchema = z.object({
    url: z.string()
});

console.log("Sending user message to Claude and requesting tool use if necessary...");
const response = await anthropic.messages.create({
    model: 'claude-haiku-4-5',
    max_tokens: 1024,
    tools: [{
        name: 'scrape_website',
        description: 'Scrape and extract markdown content from a website URL',
        input_schema: zodToJsonSchema(ScrapeArgsSchema, 'ScrapeArgsSchema') as any
    }],
    messages: [{
        role: 'user',
        content: 'What is Firecrawl? Check firecrawl.dev'
    }]
});

const toolUse = response.content.find(block => block.type === 'tool_use');

if (toolUse && toolUse.type === 'tool_use') {
    const input = toolUse.input as { url: string };
    console.log(`Calling tool: ${toolUse.name} | URL: ${input.url}`);

    const result = await firecrawl.scrape(input.url, {
        formats: ['markdown']
    });

    console.log(`Scraped content preview: ${result.markdown?.substring(0, 300)}...`);
    // Continue with the conversation or process the scraped content as needed
}
```

## Structured Extraction

This example demonstrates how to use Claude to extract structured data from scraped website content.

```typescript theme={null}
import FirecrawlApp from '@mendable/firecrawl-js';
import Anthropic from '@anthropic-ai/sdk';
import { z } from 'zod';

const firecrawl = new FirecrawlApp({ apiKey: process.env.FIRECRAWL_API_KEY });
const anthropic = new Anthropic({ apiKey: process.env.ANTHROPIC_API_KEY });

const CompanyInfoSchema = z.object({
    name: z.string(),
    industry: z.string().optional(),
    description: z.string().optional()
});

const scrapeResult = await firecrawl.scrape('https://stripe.com', {
    formats: ['markdown'],
    onlyMainContent: true
});

const prompt = `Extract company information from this website content.

Output ONLY valid JSON in this exact format (no markdown, no explanation):

{
  "name": "Company Name",
  "industry": "Industry",
  "description": "One sentence description"
}

Website content:
${scrapeResult.markdown}`;

const message = await anthropic.messages.create({
    model: 'claude-haiku-4-5',
    max_tokens: 1024,
    messages: [
        { role: 'user', content: prompt },
        { role: 'assistant', content: '{' }
    ]
});

const textBlock = message.content.find(block => block.type === 'text');

if (textBlock && textBlock.type === 'text') {
    const jsonText = '{' + textBlock.text;
    const companyInfo = CompanyInfoSchema.parse(JSON.parse(jsonText));
  
    console.log(companyInfo);
}
```

For more examples, check the [Claude documentation](https://docs.anthropic.com/claude/docs).


# ElevenAgents
Source: https://docs.firecrawl.dev/developer-guides/llm-sdks-and-frameworks/elevenagents

Give ElevenLabs voice and chat agents real-time web access with Firecrawl

Give your [ElevenAgents](https://elevenlabs.io/agents) voice and chat agents the ability to scrape, search, and crawl the web in real time using Firecrawl. This guide covers two integration paths:

1. **MCP server** — connect the hosted Firecrawl MCP server for zero-code setup.
2. **Server webhook tool** — point a custom tool at Firecrawl's REST API for full control over requests.

## Prerequisites

* An [ElevenLabs](https://elevenlabs.io) account with access to ElevenAgents
* A Firecrawl API key from your [firecrawl.dev dashboard](https://firecrawl.dev)

## Option 1: Firecrawl MCP Server

The fastest way to give an agent web access. ElevenAgents supports remote MCP servers, and Firecrawl provides a hosted MCP endpoint.

### Add the MCP server

1. Open the [Integrations page](https://elevenlabs.io/app/agents/integrations) in ElevenLabs and click **+ Add integration**.
2. Select **Custom MCP Server** from the integration library.
3. Fill in the following fields:

| Field           | Value                                                        |
| --------------- | ------------------------------------------------------------ |
| **Name**        | Firecrawl                                                    |
| **Description** | Search, scrape, crawl, and extract content from any website. |
| **Server type** | Streamable HTTP                                              |
| **Server URL**  | `https://mcp.firecrawl.dev/YOUR_FIRECRAWL_API_KEY/v2/mcp`    |

Replace `YOUR_FIRECRAWL_API_KEY` with your actual key. Leave the **Type** dropdown set to **Value**. Treat this URL as a secret — it contains your API key.

<Note>
  You must select **Streamable HTTP** as the server type. The default SSE option does not work with the Firecrawl MCP endpoint.
</Note>

4. Under **Tool Approval Mode**, choose an approval level:
   * **No Approval** — the agent uses tools freely. Fine for read-only scraping.
   * **Fine-Grained Tool Approval** — lets you pre-select which tools can run automatically and which require approval. Good for controlling expensive crawl operations.
   * **Always Ask** (default) — the agent requests permission before every tool call.

5. Check **I trust this server**, then click **Add Server**.

ElevenLabs will connect to the server and list the available tools (scrape, search, crawl, map, and more).

### Attach it to an agent

1. Create or open an agent in the [ElevenAgents dashboard](https://elevenlabs.io/app/agents/agents).
2. Go to the **Tools** tab, then select the **MCP** sub-tab.
3. Click **Add server** and select the **Firecrawl** integration from the dropdown.

### Update the system prompt

In the **Agent** tab, add instructions to the **System prompt** so the agent knows when to use Firecrawl. For example:

```text theme={null}
You are a helpful research assistant. When the user asks about a website,
a company, or any topic that requires up-to-date information, use the
Firecrawl tools to search the web or scrape the relevant page, then
summarize the results.
```

### Test it

Click **Preview** in the top navigation bar. You can test using the text chat input or by starting a voice call. Try a prompt like:

> "What does firecrawl.dev do? Go to the site and summarize it for me."

The agent will call the Firecrawl MCP `scrape` tool, receive the page markdown, and respond with a summary.

***

## Option 2: Server Webhook Tool

Use this approach when you need precise control over request parameters (formats, headers, timeouts, etc.) or want to call a specific Firecrawl endpoint without exposing the full MCP tool set.

### Scrape tool

Create a tool that scrapes a single URL and returns its content as markdown.

1. Open your agent and go to the **Tools** tab.
2. Click **Add tool** and select **Webhook**.
3. Configure the tool:

| Field           | Value                                                      |
| --------------- | ---------------------------------------------------------- |
| **Name**        | scrape\_website                                            |
| **Description** | Scrape content from a URL and return it as clean markdown. |
| **Method**      | POST                                                       |
| **URL**         | `https://api.firecrawl.dev/v2/scrape`                      |

<Note>
  The **Method** field defaults to GET — make sure to change it to **POST**.
</Note>

4. Scroll to the **Headers** section and click **Add header** for authentication:

| Header          | Value                           |
| --------------- | ------------------------------- |
| `Authorization` | `Bearer YOUR_FIRECRAWL_API_KEY` |

Alternatively, if you have workspace auth connections configured, you can use the **Authentication** dropdown instead.

5. Add a **body parameter**:

| Parameter | Type   | Description       | Required |
| --------- | ------ | ----------------- | -------- |
| `url`     | string | The URL to scrape | Yes      |

6. Click **Add tool**.

The Firecrawl API returns the page content as markdown by default. The agent receives the JSON response and can use the `markdown` field to answer questions.

### Search tool

Create a tool that searches the web and returns results with scraped content.

1. Click **Add tool** → **Webhook** again and configure:

| Field           | Value                                                                     |
| --------------- | ------------------------------------------------------------------------- |
| **Name**        | search\_web                                                               |
| **Description** | Search the web for a query and return relevant results with page content. |
| **Method**      | POST                                                                      |
| **URL**         | `https://api.firecrawl.dev/v2/search`                                     |

2. Add the same `Authorization` header as above.

3. Add **body parameters**:

| Parameter | Type   | Description                                     | Required |
| --------- | ------ | ----------------------------------------------- | -------- |
| `query`   | string | The search query                                | Yes      |
| `limit`   | number | Maximum number of results to return (default 5) | No       |

4. Click **Add tool**.

### Update the system prompt

In the **Agent** tab, update the **System prompt**:

```text theme={null}
You are a knowledgeable assistant with access to web tools.

- Use `scrape_website` when the user gives you a specific URL to read.
- Use `search_web` when the user asks a general question that requires
  finding information online.

Always summarize the information concisely and cite the source URL.
```

### Test it

Click **Preview** and try asking:

> "Search for the latest Next.js features and give me a summary."

The agent will call `search_web`, receive results from Firecrawl, and respond with a summary of the findings.

***

## Tips

* **Model selection** — For reliable tool calling, use a high-intelligence model such as GPT-4o, Claude Sonnet 4.5 or later, or Gemini 2.5 Flash. Smaller models may struggle to extract the correct parameters.
* **Keep prompts specific** — Tell the agent exactly when to use each tool. Vague instructions lead to missed or incorrect tool calls.
* **Limit response size** — For voice agents, long scraped pages can overwhelm the LLM context. Use `onlyMainContent: true` in scrape options (or instruct the agent to summarize aggressively) to keep responses concise.
* **Tool call sounds** — In the webhook or MCP tool settings, you can configure a **Tool call sound** to play ambient audio while a tool runs. This signals to the user that the agent is working.

## Resources

* [ElevenAgents documentation](https://elevenlabs.io/docs/eleven-agents/overview)
* [ElevenAgents tools overview](https://elevenlabs.io/docs/eleven-agents/customization/tools)
* [ElevenAgents MCP integration](https://elevenlabs.io/docs/eleven-agents/customization/tools/mcp)
* [Firecrawl API reference](https://docs.firecrawl.dev/api-reference/v2-introduction)
* [Firecrawl MCP server](https://docs.firecrawl.dev/mcp-server)


# Gemini
Source: https://docs.firecrawl.dev/developer-guides/llm-sdks-and-frameworks/gemini

Use Firecrawl with Google's Gemini AI for web scraping + AI workflows

Integrate Firecrawl with Google's Gemini for AI applications powered by web data.

## Setup

```bash theme={null}
npm install @mendable/firecrawl-js @google/genai
```

Create `.env` file:

```bash theme={null}
FIRECRAWL_API_KEY=your_firecrawl_key
GEMINI_API_KEY=your_gemini_key
```

> **Note:** If using Node \< 20, install `dotenv` and add `import 'dotenv/config'` to your code.

## Scrape + Summarize

This example demonstrates a simple workflow: scrape a website and summarize the content using Gemini.

```typescript theme={null}
import FirecrawlApp from '@mendable/firecrawl-js';
import { GoogleGenAI } from '@google/genai';

const firecrawl = new FirecrawlApp({ apiKey: process.env.FIRECRAWL_API_KEY });
const ai = new GoogleGenAI({ apiKey: process.env.GEMINI_API_KEY });

const scrapeResult = await firecrawl.scrape('https://firecrawl.dev', {
    formats: ['markdown']
});

console.log('Scraped content length:', scrapeResult.markdown?.length);

const response = await ai.models.generateContent({
    model: 'gemini-2.5-flash',
    contents: `Summarize: ${scrapeResult.markdown}`,
});

console.log('Summary:', response.text);
```

## Content Analysis

This example shows how to analyze website content using Gemini's multi-turn conversation capabilities.

```typescript theme={null}
import FirecrawlApp from '@mendable/firecrawl-js';
import { GoogleGenAI } from '@google/genai';

const firecrawl = new FirecrawlApp({ apiKey: process.env.FIRECRAWL_API_KEY });
const ai = new GoogleGenAI({ apiKey: process.env.GEMINI_API_KEY });

const scrapeResult = await firecrawl.scrape('https://news.ycombinator.com/', {
    formats: ['markdown']
});

console.log('Scraped content length:', scrapeResult.markdown?.length);

const chat = ai.chats.create({
    model: 'gemini-2.5-flash'
});

// Ask for the top 3 stories on Hacker News
const result1 = await chat.sendMessage({
    message: `Based on this website content from Hacker News, what are the top 3 stories right now?\n\n${scrapeResult.markdown}`
});
console.log('Top 3 Stories:', result1.text);

// Ask for the 4th and 5th stories on Hacker News
const result2 = await chat.sendMessage({
    message: `Now, what are the 4th and 5th top stories on Hacker News from the same content?`
});
console.log('4th and 5th Stories:', result2.text);
```

## Structured Extraction

This example demonstrates how to extract structured data using Gemini's JSON mode from scraped website content.

```typescript theme={null}
import FirecrawlApp from '@mendable/firecrawl-js';
import { GoogleGenAI, Type } from '@google/genai';

const firecrawl = new FirecrawlApp({ apiKey: process.env.FIRECRAWL_API_KEY });
const ai = new GoogleGenAI({ apiKey: process.env.GEMINI_API_KEY });

const scrapeResult = await firecrawl.scrape('https://stripe.com', {
    formats: ['markdown']
});

console.log('Scraped content length:', scrapeResult.markdown?.length);

const response = await ai.models.generateContent({
    model: 'gemini-2.5-flash',
    contents: `Extract company information: ${scrapeResult.markdown}`,
    config: {
        responseMimeType: 'application/json',
        responseSchema: {
            type: Type.OBJECT,
            properties: {
                name: { type: Type.STRING },
                industry: { type: Type.STRING },
                description: { type: Type.STRING },
                products: {
                    type: Type.ARRAY,
                    items: { type: Type.STRING }
                }
            },
            propertyOrdering: ['name', 'industry', 'description', 'products']
        }
    }
});

console.log('Extracted company info:', response?.text);
```

For more examples, check the [Gemini documentation](https://ai.google.dev/docs).


# Agent Development Kit (ADK)
Source: https://docs.firecrawl.dev/developer-guides/llm-sdks-and-frameworks/google-adk

Integrate Firecrawl with Google's ADK using MCP for advanced agent workflows

Integrate Firecrawl with Google's Agent Development Kit (ADK) to build powerful AI agents with web scraping capabilities through the Model Context Protocol (MCP).

## Overview

Firecrawl provides an MCP server that seamlessly integrates with Google's ADK, enabling your agents to efficiently scrape, crawl, and extract structured data from any website. The integration supports both cloud-based and self-hosted Firecrawl instances with streamable HTTP for optimal performance.

## Features

* Efficient web scraping, crawling, and content discovery from any website
* Advanced search capabilities and intelligent content extraction
* Deep research and high-volume batch scraping
* Flexible deployment (cloud-based or self-hosted)
* Optimized for modern web environments with streamable HTTP support

## Prerequisites

* Obtain an API key for Firecrawl from [firecrawl.dev](https://firecrawl.dev)
* Install Google ADK

## Setup

<CodeGroup>
  ```python Remote MCP Server theme={null}
  from google.adk.agents.llm_agent import Agent
  from google.adk.tools.mcp_tool.mcp_session_manager import StreamableHTTPServerParams
  from google.adk.tools.mcp_tool.mcp_toolset import MCPToolset

  FIRECRAWL_API_KEY = "YOUR-API-KEY"

  root_agent = Agent(
      model="gemini-2.5-pro",
      name="firecrawl_agent",
      description='A helpful assistant for scraping websites with Firecrawl',
      instruction='Help the user search for website content',
      tools=[
          MCPToolset(
              connection_params=StreamableHTTPServerParams(
                  url=f"https://mcp.firecrawl.dev/{FIRECRAWL_API_KEY}/v2/mcp",
              ),
          )
      ],
  )
  ```

  ```python Local MCP Server theme={null}
  from google.adk.agents.llm_agent import Agent
  from google.adk.tools.mcp_tool.mcp_session_manager import StdioConnectionParams
  from google.adk.tools.mcp_tool.mcp_toolset import MCPToolset
  from mcp import StdioServerParameters

  root_agent = Agent(
      model='gemini-2.5-pro',
      name='firecrawl_agent',
      description='A helpful assistant for scraping websites with Firecrawl',
      instruction='Help the user search for website content',
      tools=[
          MCPToolset(
              connection_params=StdioConnectionParams(
                  server_params = StdioServerParameters(
                      command='npx',
                      args=[
                          "-y",
                          "firecrawl-mcp",
                      ],
                      env={
                          "FIRECRAWL_API_KEY": "YOUR-API-KEY",
                      }
                  ),
                  timeout=30,
              ),
          )
      ],
  )
  ```
</CodeGroup>

## Available Tools

| Tool               | Name                           | Description                                                                          |
| ------------------ | ------------------------------ | ------------------------------------------------------------------------------------ |
| Scrape Tool        | `firecrawl_scrape`             | Scrape content from a single URL with advanced options                               |
| Batch Scrape Tool  | `firecrawl_batch_scrape`       | Scrape multiple URLs efficiently with built-in rate limiting and parallel processing |
| Check Batch Status | `firecrawl_check_batch_status` | Check the status of a batch operation                                                |
| Map Tool           | `firecrawl_map`                | Map a website to discover all indexed URLs on the site                               |
| Search Tool        | `firecrawl_search`             | Search the web and optionally extract content from search results                    |
| Crawl Tool         | `firecrawl_crawl`              | Start an asynchronous crawl with advanced options                                    |
| Check Crawl Status | `firecrawl_check_crawl_status` | Check the status of a crawl job                                                      |
| Extract Tool       | `firecrawl_extract`            | Extract structured information from web pages using LLM capabilities                 |

## Configuration

### Required Configuration

**FIRECRAWL\_API\_KEY**: Your Firecrawl API key

* Required when using cloud API (default)
* Optional when using self-hosted instance with FIRECRAWL\_API\_URL

### Optional Configuration

**Firecrawl API URL (for self-hosted instances)**:

* `FIRECRAWL_API_URL`: Custom API endpoint
* Example: `https://firecrawl.your-domain.com`
* If not provided, the cloud API will be used

**Retry configuration**:

* `FIRECRAWL_RETRY_MAX_ATTEMPTS`: Maximum retry attempts (default: 3)
* `FIRECRAWL_RETRY_INITIAL_DELAY`: Initial delay in milliseconds (default: 1000)
* `FIRECRAWL_RETRY_MAX_DELAY`: Maximum delay in milliseconds (default: 10000)
* `FIRECRAWL_RETRY_BACKOFF_FACTOR`: Exponential backoff multiplier (default: 2)

**Credit usage monitoring**:

* `FIRECRAWL_CREDIT_WARNING_THRESHOLD`: Warning threshold (default: 1000)
* `FIRECRAWL_CREDIT_CRITICAL_THRESHOLD`: Critical threshold (default: 100)

## Example: Web Research Agent

```python theme={null}
from google.adk.agents.llm_agent import Agent
from google.adk.tools.mcp_tool.mcp_session_manager import StreamableHTTPServerParams
from google.adk.tools.mcp_tool.mcp_toolset import MCPToolset

FIRECRAWL_API_KEY = "YOUR-API-KEY"

# Create a research agent
research_agent = Agent(
    model="gemini-2.5-pro",
    name="research_agent",
    description='An AI agent that researches topics by scraping and analyzing web content',
    instruction='''You are a research assistant. When given a topic or question:
    1. Use the search tool to find relevant websites
    2. Scrape the most relevant pages for detailed information
    3. Extract structured data when needed
    4. Provide comprehensive, well-sourced answers''',
    tools=[
        MCPToolset(
            connection_params=StreamableHTTPServerParams(
                url=f"https://mcp.firecrawl.dev/{FIRECRAWL_API_KEY}/v2/mcp",
            ),
        )
    ],
)

# Use the agent
response = research_agent.run("What are the latest features in Python 3.13?")
print(response)
```

## Best Practices

1. **Use the right tool for the job**:
   * `firecrawl_search` when you need to find relevant pages first
   * `firecrawl_scrape` for single pages
   * `firecrawl_batch_scrape` for multiple known URLs
   * `firecrawl_crawl` for discovering and scraping entire sites

2. **Monitor your usage**: Configure credit thresholds to avoid unexpected usage

3. **Handle errors gracefully**: Configure retry settings based on your use case

4. **Optimize performance**: Use batch operations when scraping multiple URLs

***

## Related Resources

<CardGroup>
  <Card title="Comprehensive Guide to Building AI Agents Using Google Agent Development Kit (ADK) and Firecrawl" href="https://www.firecrawl.dev/blog/google-adk-multi-agent-tutorial">
    Learn how to build powerful multi-agent AI systems using Google's ADK framework with Firecrawl for web scraping capabilities.
  </Card>

  <Card title="MCP Server Documentation" href="https://docs.firecrawl.dev/mcp-server">
    Learn more about Firecrawl's Model Context Protocol (MCP) server integration and capabilities.
  </Card>

  <Card title="Google ADK Official Documentation" href="https://google.github.io/adk-docs/">
    Explore the official Google Agent Development Kit documentation for comprehensive guides and API references.
  </Card>
</CardGroup>


# LangChain
Source: https://docs.firecrawl.dev/developer-guides/llm-sdks-and-frameworks/langchain

Use Firecrawl with LangChain for web scraping + AI workflows

Integrate Firecrawl with LangChain to build AI applications powered by web data.

## Setup

```bash theme={null}
npm install @langchain/openai @mendable/firecrawl-js 
```

Create `.env` file:

```bash theme={null}
FIRECRAWL_API_KEY=your_firecrawl_key
OPENAI_API_KEY=your_openai_key
```

> **Note:** If using Node \< 20, install `dotenv` and add `import 'dotenv/config'` to your code.

## Scrape + Chat

This example demonstrates a simple workflow: scrape a website and process the content using LangChain.

```typescript theme={null}
import FirecrawlApp from '@mendable/firecrawl-js';
import { ChatOpenAI } from '@langchain/openai';
import { HumanMessage } from '@langchain/core/messages';

const firecrawl = new FirecrawlApp({ apiKey: process.env.FIRECRAWL_API_KEY });
const chat = new ChatOpenAI({
    model: 'gpt-5-nano',
    apiKey: process.env.OPENAI_API_KEY
});

const scrapeResult = await firecrawl.scrape('https://firecrawl.dev', {
    formats: ['markdown']
});

console.log('Scraped content length:', scrapeResult.markdown?.length);

const response = await chat.invoke([
    new HumanMessage(`Summarize: ${scrapeResult.markdown}`)
]);

console.log('Summary:', response.content);
```

## Chains

This example shows how to build a LangChain chain to process and analyze scraped content.

```typescript theme={null}
import FirecrawlApp from '@mendable/firecrawl-js';
import { ChatOpenAI } from '@langchain/openai';
import { ChatPromptTemplate } from '@langchain/core/prompts';
import { StringOutputParser } from '@langchain/core/output_parsers';

const firecrawl = new FirecrawlApp({ apiKey: process.env.FIRECRAWL_API_KEY });
const model = new ChatOpenAI({
    model: 'gpt-5-nano',
    apiKey: process.env.OPENAI_API_KEY
});

const scrapeResult = await firecrawl.scrape('https://stripe.com', {
    formats: ['markdown']
});

console.log('Scraped content length:', scrapeResult.markdown?.length);

// Create processing chain
const prompt = ChatPromptTemplate.fromMessages([
    ['system', 'You are an expert at analyzing company websites.'],
    ['user', 'Extract the company name and main products from: {content}']
]);

const chain = prompt.pipe(model).pipe(new StringOutputParser());

// Execute the chain
const result = await chain.invoke({
    content: scrapeResult.markdown
});

console.log('Chain result:', result);
```

## Tool Calling

This example demonstrates how to use LangChain's tool calling feature to let the model decide when to scrape websites.

```typescript theme={null}
import FirecrawlApp from '@mendable/firecrawl-js';
import { ChatOpenAI } from '@langchain/openai';
import { DynamicStructuredTool } from '@langchain/core/tools';
import { z } from 'zod';

const firecrawl = new FirecrawlApp({ apiKey: process.env.FIRECRAWL_API_KEY });

// Create the scraping tool
const scrapeWebsiteTool = new DynamicStructuredTool({
    name: 'scrape_website',
    description: 'Scrape content from any website URL',
    schema: z.object({
        url: z.string().url().describe('The URL to scrape')
    }),
    func: async ({ url }) => {
        console.log('Scraping:', url);
        const result = await firecrawl.scrape(url, {
            formats: ['markdown']
        });
        console.log('Scraped content preview:', result.markdown?.substring(0, 200) + '...');
        return result.markdown || 'No content scraped';
    }
});

const model = new ChatOpenAI({
    model: 'gpt-5-nano',
    apiKey: process.env.OPENAI_API_KEY
}).bindTools([scrapeWebsiteTool]);

const response = await model.invoke('What is Firecrawl? Visit firecrawl.dev and tell me about it.');

console.log('Response:', response.content);
console.log('Tool calls:', response.tool_calls);
```

## Structured Data Extraction

This example shows how to extract structured data using LangChain's structured output feature.

```typescript theme={null}
import FirecrawlApp from '@mendable/firecrawl-js';
import { ChatOpenAI } from '@langchain/openai';
import { z } from 'zod';

const firecrawl = new FirecrawlApp({ apiKey: process.env.FIRECRAWL_API_KEY });

const scrapeResult = await firecrawl.scrape('https://stripe.com', {
    formats: ['markdown']
});

console.log('Scraped content length:', scrapeResult.markdown?.length);

const CompanyInfoSchema = z.object({
    name: z.string(),
    industry: z.string(),
    description: z.string(),
    products: z.array(z.string())
});

const model = new ChatOpenAI({
    model: 'gpt-5-nano',
    apiKey: process.env.OPENAI_API_KEY
}).withStructuredOutput(CompanyInfoSchema);

const companyInfo = await model.invoke([
    {
        role: 'system',
        content: 'Extract company information from website content.'
    },
    {
        role: 'user',
        content: `Extract data: ${scrapeResult.markdown}`
    }
]);

console.log('Extracted company info:', companyInfo);
```

For more examples, check the [LangChain documentation](https://js.langchain.com/docs).


# LangGraph
Source: https://docs.firecrawl.dev/developer-guides/llm-sdks-and-frameworks/langgraph

Integrate Firecrawl with LangGraph for building agent workflows

This guide shows how to integrate Firecrawl with LangGraph to build AI agent workflows that can scrape and process web content.

## Setup

```bash theme={null}
npm install @langchain/langgraph @langchain/openai @mendable/firecrawl-js
```

Create `.env` file:

```bash theme={null}
FIRECRAWL_API_KEY=your_firecrawl_key
OPENAI_API_KEY=your_openai_key
```

> **Note:** If using Node \< 20, install `dotenv` and add `import 'dotenv/config'` to your code.

## Basic Workflow

This example demonstrates a basic LangGraph workflow that scrapes a website and analyzes the content.

```typescript theme={null}
import FirecrawlApp from '@mendable/firecrawl-js';
import { ChatOpenAI } from '@langchain/openai';
import { StateGraph, MessagesAnnotation, START, END } from '@langchain/langgraph';

// Initialize Firecrawl
const firecrawl = new FirecrawlApp({ apiKey: process.env.FIRECRAWL_API_KEY });

// Initialize LLM
const llm = new ChatOpenAI({
    model: "gpt-5-nano",
    apiKey: process.env.OPENAI_API_KEY
});

// Define the scrape node
async function scrapeNode(state: typeof MessagesAnnotation.State) {
    console.log('Scraping...');
    const result = await firecrawl.scrape('https://firecrawl.dev', { formats: ['markdown'] });
    return {
        messages: [{
            role: "system",
            content: `Scraped content: ${result.markdown}`
        }]
    };
}

// Define the analyze node
async function analyzeNode(state: typeof MessagesAnnotation.State) {
    console.log('Analyzing...');
    const response = await llm.invoke(state.messages);
    return { messages: [response] };
}

// Build the graph
const graph = new StateGraph(MessagesAnnotation)
    .addNode("scrape", scrapeNode)
    .addNode("analyze", analyzeNode)
    .addEdge(START, "scrape")
    .addEdge("scrape", "analyze")
    .addEdge("analyze", END);

// Compile the graph
const app = graph.compile();

// Run the workflow
const result = await app.invoke({
    messages: [{ role: "user", content: "Summarize the website" }]
});

console.log(JSON.stringify(result, null, 2));
```

## Multi-Step Workflow

This example demonstrates a more complex workflow that scrapes multiple URLs and processes them.

```typescript theme={null}
import FirecrawlApp from '@mendable/firecrawl-js';
import { ChatOpenAI } from '@langchain/openai';
import { StateGraph, Annotation, START, END } from '@langchain/langgraph';

const firecrawl = new FirecrawlApp({ apiKey: process.env.FIRECRAWL_API_KEY });
const llm = new ChatOpenAI({ model: "gpt-5-nano", apiKey: process.env.OPENAI_API_KEY });

// Define custom state
const WorkflowState = Annotation.Root({
    urls: Annotation<string[]>(),
    scrapedData: Annotation<Array<{ url: string; content: string }>>(),
    summary: Annotation<string>()
});

// Scrape multiple URLs
async function scrapeMultiple(state: typeof WorkflowState.State) {
    const scrapedData = [];
    for (const url of state.urls) {
        const result = await firecrawl.scrape(url, { formats: ['markdown'] });
        scrapedData.push({ url, content: result.markdown || '' });
    }
    return { scrapedData };
}

// Summarize all scraped content
async function summarizeAll(state: typeof WorkflowState.State) {
    const combinedContent = state.scrapedData
        .map(item => `Content from ${item.url}:\n${item.content}`)
        .join('\n\n');

    const response = await llm.invoke([
        { role: "user", content: `Summarize these websites:\n${combinedContent}` }
    ]);

    return { summary: response.content as string };
}

// Build the workflow graph
const workflow = new StateGraph(WorkflowState)
    .addNode("scrape", scrapeMultiple)
    .addNode("summarize", summarizeAll)
    .addEdge(START, "scrape")
    .addEdge("scrape", "summarize")
    .addEdge("summarize", END);

const app = workflow.compile();

// Execute workflow
const result = await app.invoke({
    urls: ["https://firecrawl.dev", "https://firecrawl.dev/pricing"]
});

console.log(result.summary);
```

For more examples, check the [LangGraph documentation](https://langchain-ai.github.io/langgraphjs/).


# LlamaIndex
Source: https://docs.firecrawl.dev/developer-guides/llm-sdks-and-frameworks/llamaindex

Use Firecrawl with LlamaIndex for RAG applications

Integrate Firecrawl with LlamaIndex to build AI applications with vector search and embeddings powered by web content.

## Setup

```bash theme={null}
npm install llamaindex @llamaindex/openai @mendable/firecrawl-js
```

Create `.env` file:

```bash theme={null}
FIRECRAWL_API_KEY=your_firecrawl_key
OPENAI_API_KEY=your_openai_key
```

> **Note:** If using Node \< 20, install `dotenv` and add `import 'dotenv/config'` to your code.

## RAG with Vector Search

This example demonstrates how to use LlamaIndex with Firecrawl to crawl a website, create embeddings, and query the content using RAG.

```typescript theme={null}
import Firecrawl from '@mendable/firecrawl-js';
import { Document, VectorStoreIndex, Settings } from 'llamaindex';
import { OpenAI, OpenAIEmbedding } from '@llamaindex/openai';

Settings.llm = new OpenAI({ model: "gpt-4o" });
Settings.embedModel = new OpenAIEmbedding({ model: "text-embedding-3-small" });

const firecrawl = new Firecrawl({ apiKey: process.env.FIRECRAWL_API_KEY });
const crawlResult = await firecrawl.crawl('https://firecrawl.dev', {
  limit: 10,
  scrapeOptions: { formats: ['markdown'] }
});
console.log(`Crawled ${crawlResult.data.length } pages`);

const documents = crawlResult.data.map((page: any, i: number) =>
  new Document({
    text: page.markdown,
    id_: `page-${i}`,
    metadata: { url: page.metadata?.sourceURL }
  })
);

const index = await VectorStoreIndex.fromDocuments(documents);
console.log('Vector index created with embeddings');

const queryEngine = index.asQueryEngine();
const response = await queryEngine.query({ query: 'What is Firecrawl and how does it work?' });

console.log('\nAnswer:', response.toString());
```

For more examples, check the [LlamaIndex documentation](https://ts.llamaindex.ai/).


# Mastra
Source: https://docs.firecrawl.dev/developer-guides/llm-sdks-and-frameworks/mastra

Use Firecrawl with Mastra for building AI workflows

Integrate Firecrawl with Mastra, the TypeScript framework for building AI agents and workflows.

## Setup

```bash theme={null}
npm install @mastra/core @mendable/firecrawl-js zod
```

Create `.env` file:

```bash theme={null}
FIRECRAWL_API_KEY=your_firecrawl_key
OPENAI_API_KEY=your_openai_key
```

> **Note:** If using Node \< 20, install `dotenv` and add `import 'dotenv/config'` to your code.

## Multi-Step Workflow

This example demonstrates a complete workflow that searches, scrapes, and summarizes documentation using Firecrawl and Mastra.

```typescript theme={null}
import { createWorkflow, createStep } from "@mastra/core/workflows";
import { z } from "zod";
import Firecrawl from "@mendable/firecrawl-js";
import { Agent } from "@mastra/core/agent";

const firecrawl = new Firecrawl({
  apiKey: process.env.FIRECRAWL_API_KEY || "fc-YOUR_API_KEY"
});

const agent = new Agent({
  name: "summarizer",
  instructions: "You are a helpful assistant that creates concise summaries of documentation.",
  model: "openai/gpt-5-nano",
});

// Step 1: Search with Firecrawl SDK
const searchStep = createStep({
  id: "search",
  inputSchema: z.object({
    query: z.string(),
  }),
  outputSchema: z.object({
    url: z.string(),
    title: z.string(),
  }),
  execute: async ({ inputData }: { inputData: { query: string } }) => {
    console.log(`Searching: ${inputData.query}`);
    const searchResults = await firecrawl.search(inputData.query, { limit: 1 });
    const webResults = (searchResults as any)?.web;

    if (!webResults || !Array.isArray(webResults) || webResults.length === 0) {
      throw new Error("No search results found");
    }

    const firstResult = webResults[0];
    console.log(`Found: ${firstResult.title}`);
    return {
      url: firstResult.url,
      title: firstResult.title,
    };
  },
});

// Step 2: Scrape the URL with Firecrawl SDK
const scrapeStep = createStep({
  id: "scrape",
  inputSchema: z.object({
    url: z.string(),
    title: z.string(),
  }),
  outputSchema: z.object({
    markdown: z.string(),
    title: z.string(),
  }),
  execute: async ({ inputData }: { inputData: { url: string; title: string } }) => {
    console.log(`Scraping: ${inputData.url}`);
    const scrapeResult = await firecrawl.scrape(inputData.url, {
      formats: ["markdown"],
    });

    console.log(`Scraped: ${scrapeResult.markdown?.length || 0} characters`);
    return {
      markdown: scrapeResult.markdown || "",
      title: inputData.title,
    };
  },
});

// Step 3: Summarize with Claude
const summarizeStep = createStep({
  id: "summarize",
  inputSchema: z.object({
    markdown: z.string(),
    title: z.string(),
  }),
  outputSchema: z.object({
    summary: z.string(),
  }),
  execute: async ({ inputData }: { inputData: { markdown: string; title: string } }) => {
    console.log(`Summarizing: ${inputData.title}`);

    const prompt = `Summarize the following documentation in 2-3 sentences:\n\nTitle: ${inputData.title}\n\n${inputData.markdown}`;
    const result = await agent.generate(prompt);

    console.log(`Summary generated`);
    return { summary: result.text };
  },
});

// Create workflow
export const workflow = createWorkflow({
  id: "firecrawl-workflow",
  inputSchema: z.object({
    query: z.string(),
  }),
  outputSchema: z.object({
    summary: z.string(),
  }),
  steps: [searchStep, scrapeStep, summarizeStep],
})
  .then(searchStep)
  .then(scrapeStep)
  .then(summarizeStep)
  .commit();

async function testWorkflow() {
  const run = await workflow.createRunAsync();
  const result = await run.start({
    inputData: { query: "Firecrawl documentation" }
  });

  if (result.status === "success") {
    const { summarize } = result.steps;

    if (summarize.status === "success") {
      console.log(`\n${summarize.output.summary}`);
    }
  } else {
    console.error("Workflow failed:", result.status);
  }
}

testWorkflow().catch(console.error);
```

For more examples, check the [Mastra documentation](https://mastra.ai/docs).


# OpenAI
Source: https://docs.firecrawl.dev/developer-guides/llm-sdks-and-frameworks/openai

Use Firecrawl with OpenAI for web scraping + AI workflows

Integrate Firecrawl with OpenAI to build AI applications powered by web data.

## Setup

```bash theme={null}
npm install @mendable/firecrawl-js openai zod
```

Create `.env` file:

```bash theme={null}
FIRECRAWL_API_KEY=your_firecrawl_key
OPENAI_API_KEY=your_openai_key
```

> **Note:** If using Node \< 20, install `dotenv` and add `import 'dotenv/config'` to your code.

## Scrape + Summarize

This example demonstrates a simple workflow: scrape a website and summarize the content using an OpenAI model.

```typescript theme={null}
import FirecrawlApp from '@mendable/firecrawl-js';
import OpenAI from 'openai';

const firecrawl = new FirecrawlApp({ apiKey: process.env.FIRECRAWL_API_KEY });
const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

// Scrape the website content
const scrapeResult = await firecrawl.scrape('https://firecrawl.dev', {
    formats: ['markdown']
});

console.log('Scraped content length:', scrapeResult.markdown?.length);

// Summarize with OpenAI model
const completion = await openai.chat.completions.create({
    model: 'gpt-5-nano',
    messages: [
        { role: 'user', content: `Summarize: ${scrapeResult.markdown}` }
    ]
});

console.log('Summary:', completion.choices[0]?.message.content);
```

## Function Calling

This example shows how to use OpenAI's function calling feature to let the model decide when to scrape websites based on user requests.

```typescript theme={null}
import FirecrawlApp from '@mendable/firecrawl-js';
import OpenAI from 'openai';
import { z } from 'zod';

const firecrawl = new FirecrawlApp({ apiKey: process.env.FIRECRAWL_API_KEY });
const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

const ScrapeArgsSchema = z.object({
    url: z.string().describe('The URL of the website to scrape')
});

const tools = [{
    type: 'function' as const,
    function: {
        name: 'scrape_website',
        description: 'Scrape content from any website URL',
        parameters: z.toJSONSchema(ScrapeArgsSchema)
    }
}];

const response = await openai.chat.completions.create({
    model: 'gpt-5-nano',
    messages: [{
        role: 'user',
        content: 'What is Firecrawl? Visit firecrawl.dev and tell me about it.'
    }],
    tools
});

const message = response.choices[0]?.message;

if (message?.tool_calls && message.tool_calls.length > 0) {
    for (const toolCall of message.tool_calls) {
        if (toolCall.type === 'function') {
            console.log('Tool called:', toolCall.function.name);

            const args = ScrapeArgsSchema.parse(JSON.parse(toolCall.function.arguments));
            const result = await firecrawl.scrape(args.url, {
                formats: ['markdown'] // Other formats: html, links, etc.
            });
            console.log('Scraped content:', result.markdown?.substring(0, 200) + '...');

            // Send the scraped content back to the model for final response
            const finalResponse = await openai.chat.completions.create({
                model: 'gpt-5-nano',
                messages: [
                    {
                        role: 'user',
                        content: 'What is Firecrawl? Visit firecrawl.dev and tell me about it.'
                    },
                    message,
                    {
                        role: 'tool',
                        tool_call_id: toolCall.id,
                        content: result.markdown || 'No content scraped'
                    }
                ],
                tools
            });

            console.log('Final response:', finalResponse.choices[0]?.message?.content);
        }
    }
} else {
    console.log('Direct response:', message?.content);
}
```

## Structured Data Extraction

This example demonstrates how to use OpenAI models with structured outputs to extract specific data from scraped content.

```typescript theme={null}
import FirecrawlApp from '@mendable/firecrawl-js';
import OpenAI from 'openai';
import { z } from 'zod';

const firecrawl = new FirecrawlApp({ apiKey: process.env.FIRECRAWL_API_KEY });
const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

const scrapeResult = await firecrawl.scrape('https://stripe.com', {
    formats: ['markdown']
});

console.log('Scraped content length:', scrapeResult.markdown?.length);

const CompanyInfoSchema = z.object({
    name: z.string(),
    industry: z.string(),
    description: z.string(),
    products: z.array(z.string())
});

const response = await openai.chat.completions.create({
    model: 'gpt-5-nano',
    messages: [
        {
            role: 'system',
            content: 'Extract company information from website content.'
        },
        {
            role: 'user',
            content: `Extract data: ${scrapeResult.markdown}`
        }
    ],
    response_format: {
        type: 'json_schema',
        json_schema: {
            name: 'company_info',
            schema: z.toJSONSchema(CompanyInfoSchema),
            strict: true
        }
    }
});

const content = response.choices[0]?.message?.content;
const companyInfo = content ? CompanyInfoSchema.parse(JSON.parse(content)) : null;
console.log('Validated company info:', companyInfo);
```

## Search + Analyze

This example combines Firecrawl's search capabilities with OpenAI model analysis to find and summarize information from multiple sources.

```typescript theme={null}
import FirecrawlApp from '@mendable/firecrawl-js';
import OpenAI from 'openai';

const firecrawl = new FirecrawlApp({ apiKey: process.env.FIRECRAWL_API_KEY });
const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

// Search for relevant information
const searchResult = await firecrawl.search('Next.js 16 new features', {
    limit: 3,
    sources: [{ type: 'web' }], // Other sources: { type: 'news' }, { type: 'images' }
    scrapeOptions: { formats: ['markdown'] }
});

console.log('Search results:', searchResult.web?.length, 'pages found');

// Analyze and summarize the key features
const analysis = await openai.chat.completions.create({
    model: 'gpt-5-nano',
    messages: [{
        role: 'user',
        content: `Summarize the key features: ${JSON.stringify(searchResult)}`
    }]
});

console.log('Analysis:', analysis.choices[0]?.message?.content);
```

## Responses API with MCP

This example shows how to use OpenAI's Responses API with Firecrawl configured as an MCP (Model Context Protocol) server.

```typescript theme={null}
import OpenAI from 'openai';

const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

const response = await openai.responses.create({
    model: 'gpt-5-nano',
    tools: [
        {
            type: 'mcp',
            server_label: 'firecrawl',
            server_description: 'A web search and scraping MCP server to scrape and extract content from websites.',
            server_url: `https://mcp.firecrawl.dev/${process.env.FIRECRAWL_API_KEY}/v2/mcp`,
            require_approval: 'never'
        }
    ],
    input: 'Find out what the top stories on Hacker News are and the latest blog post on OpenAI and summarize them in a bullet point format'
});

console.log('Response:', JSON.stringify(response.output, null, 2));
```

For more examples, check the [OpenAI documentation](https://platform.openai.com/docs).


# Vercel AI SDK
Source: https://docs.firecrawl.dev/developer-guides/llm-sdks-and-frameworks/vercel-ai-sdk

Firecrawl tools for Vercel AI SDK. Web scraping, search, interact, and crawling for AI applications.

Firecrawl tools for the Vercel AI SDK. Search, scrape, interact with pages, and crawl the web in your AI applications.

## Install

```bash theme={null}
npm install firecrawl-aisdk ai
```

Set environment variables:

```bash theme={null}
FIRECRAWL_API_KEY=fc-your-key       # https://firecrawl.dev
AI_GATEWAY_API_KEY=your-key         # https://vercel.com/ai-gateway
```

<Note>
  These examples use the [Vercel AI Gateway](https://vercel.com/ai-gateway) string model format, but Firecrawl tools work with any AI SDK provider. You can also use provider imports like `anthropic('claude-sonnet-4-5-20250514')` from `@ai-sdk/anthropic`.
</Note>

## Quick Start

`FirecrawlTools()` bundles `search`, `scrape`, and `interact` by default.

```typescript theme={null}
import { generateText, stepCountIs } from 'ai';
import { FirecrawlTools } from 'firecrawl-aisdk';

const { text } = await generateText({
  model: 'anthropic/claude-sonnet-4-5',
  tools: FirecrawlTools(),
  stopWhen: stepCountIs(30),
  prompt: `
    1. Use interact on Hacker News to identify the top story
    2. Search for other perspectives on the same topic
    3. Scrape the most relevant pages you found
    4. Summarize everything you found
  `,
});
```

## FirecrawlTools

`FirecrawlTools()` gives you the default tools plus an auto-generated `systemPrompt` you can pass to `generateText`.

```typescript theme={null}
import { generateText, stepCountIs } from 'ai';
import { FirecrawlTools } from 'firecrawl-aisdk';

const tools = FirecrawlTools();

const { text } = await generateText({
  model: 'anthropic/claude-sonnet-4-5',
  system: `${tools.systemPrompt}\n\nAnswer with citations when possible.`,
  tools,
  stopWhen: stepCountIs(20),
  prompt: 'Find the current Firecrawl pricing page and explain the available plans.',
});
```

You can customize defaults, opt into async tools, or disable individual tools:

```typescript theme={null}
const tools = FirecrawlTools({
  search: { limit: 5 },
  scrape: { formats: ['markdown'], onlyMainContent: true },
  interact: { profile: { name: 'my-session', saveChanges: true } },
  crawl: true,
  agent: true,
});
```

```typescript theme={null}
// Disable interact, keep search + scrape
FirecrawlTools({ interact: false });

// Opt into deprecated browser compatibility
FirecrawlTools({ browser: {} });

// Include every available tool
FirecrawlTools({ all: true });
```

When scraping to answer a question about a page, use the `question` format:

```typescript theme={null}
formats: [{ type: 'question', question: 'What does this page say about pricing and rate limits?' }]
```

Use `formats: ['markdown']` only when you need the full page content.

## Individual Tools

Every tool can be used directly or called with options:

```typescript theme={null}
import { generateText } from 'ai';
import { scrape, search } from 'firecrawl-aisdk';

const { text } = await generateText({
  model: 'anthropic/claude-sonnet-4-5',
  prompt: 'Search for Firecrawl, then scrape the most relevant result.',
  tools: { search, scrape },
});

const customScrape = scrape({ apiKey: 'fc-custom-key', apiUrl: 'https://api.firecrawl.dev' });
```

### Search + Scrape

```typescript theme={null}
import { generateText } from 'ai';
import { search, scrape } from 'firecrawl-aisdk';

const { text } = await generateText({
  model: 'anthropic/claude-sonnet-4-5',
  prompt: 'Search for Firecrawl, scrape the top official result, and explain what it does.',
  tools: { search, scrape },
});
```

### Map

```typescript theme={null}
import { generateText } from 'ai';
import { map } from 'firecrawl-aisdk';

const { text } = await generateText({
  model: 'anthropic/claude-sonnet-4-5',
  prompt: 'Map https://docs.firecrawl.dev and list the main sections.',
  tools: { map },
});
```

### Stream

```typescript theme={null}
import { streamText, stepCountIs } from 'ai';
import { scrape } from 'firecrawl-aisdk';

const result = streamText({
  model: 'anthropic/claude-sonnet-4-5',
  prompt: 'What is the first 100 words of firecrawl.dev?',
  tools: { scrape },
  stopWhen: stepCountIs(3),
});

for await (const chunk of result.textStream) {
  process.stdout.write(chunk);
}

await result.fullStream;
```

## Interact

`interact()` creates a scrape-backed interactive session. Call `start(url)` to bootstrap a session and get a live view URL, then let the model reuse that session through the `interact` tool.

```typescript theme={null}
import { generateText, stepCountIs } from 'ai';
import { interact, search } from 'firecrawl-aisdk';

const interactTool = interact();
console.log('Live view:', await interactTool.start('https://news.ycombinator.com'));

const { text } = await generateText({
  model: 'anthropic/claude-sonnet-4-5',
  tools: { interact: interactTool, search },
  stopWhen: stepCountIs(25),
  prompt: 'Use interact on the current Hacker News session, find the top story, then search for more context.',
});

await interactTool.close();
```

If you need the explicit live view URL after startup, use `interactTool.interactiveLiveViewUrl`.

Reuse browser state across sessions with profiles:

```typescript theme={null}
const interactTool = interact({
  profile: { name: 'my-session', saveChanges: true },
});
```

`browser()` is deprecated. Prefer `interact()`.

## Async Tools

Crawl, batch scrape, and agent return a job ID. Pair them with `poll`.

### Crawl

```typescript theme={null}
import { generateText } from 'ai';
import { crawl, poll } from 'firecrawl-aisdk';

const { text } = await generateText({
  model: 'anthropic/claude-sonnet-4-5',
  prompt: 'Crawl https://docs.firecrawl.dev (limit 3 pages) and summarize.',
  tools: { crawl, poll },
});
```

### Batch Scrape

```typescript theme={null}
import { generateText } from 'ai';
import { batchScrape, poll } from 'firecrawl-aisdk';

const { text } = await generateText({
  model: 'anthropic/claude-sonnet-4-5',
  prompt: 'Scrape https://firecrawl.dev and https://docs.firecrawl.dev, then compare them.',
  tools: { batchScrape, poll },
});
```

### Agent

Autonomous web data gathering that searches, navigates, and extracts on its own.

```typescript theme={null}
import { generateText, stepCountIs } from 'ai';
import { agent, poll } from 'firecrawl-aisdk';

const { text } = await generateText({
  model: 'anthropic/claude-sonnet-4-5',
  prompt: 'Find the founders of Firecrawl, their roles, and their backgrounds.',
  tools: { agent, poll },
  stopWhen: stepCountIs(10),
});
```

## Logging

```typescript theme={null}
import { generateText } from 'ai';
import { logStep, scrape, stepLogger } from 'firecrawl-aisdk';

const logger = stepLogger();

const { text, usage } = await generateText({
  model: 'anthropic/claude-sonnet-4-5',
  prompt: 'Scrape https://firecrawl.dev and summarize it.',
  tools: { scrape },
  onStepFinish: logger.onStep,
  experimental_onToolCallFinish: logger.onToolCallFinish,
});

logger.close();
logger.summary(usage);

await generateText({
  model: 'anthropic/claude-sonnet-4-5',
  prompt: 'Scrape https://firecrawl.dev and summarize it again.',
  tools: { scrape },
  onStepFinish: logStep,
});
```

## All Exports

```typescript theme={null}
import {
  // Core tools
  search,             // Search the web
  scrape,             // Scrape a single URL
  map,                // Discover URLs on a site
  crawl,              // Crawl multiple pages (async, use poll)
  batchScrape,        // Scrape multiple URLs (async, use poll)
  agent,              // Autonomous web research (async, use poll)

  // Job management
  poll,               // Poll async jobs for results
  status,             // Check job status
  cancel,             // Cancel running jobs

  // Browser/session tools
  interact,           // interact({ profile: { name: '...' } })
  browser,            // deprecated compatibility export

  // All-in-one bundle
  FirecrawlTools,     // FirecrawlTools({ search, scrape, interact, crawl, agent })

  // Helpers
  stepLogger,         // Token stats per tool call
  logStep,            // Simple one-liner logging
} from 'firecrawl-aisdk';
```


# MCP Web Search & Scrape in ChatGPT
Source: https://docs.firecrawl.dev/developer-guides/mcp-setup-guides/chatgpt

Add web scraping and search to ChatGPT in 2 minutes

<Note>
  MCP support in ChatGPT is currently a beta feature. The interface and availability may change as OpenAI continues development.
</Note>

<Warning>
  **Availability:** Developer mode with MCP connectors is not available on the Free tier and requires a paid ChatGPT subscription. Availability and features vary by plan and region. See [OpenAI's documentation on Developer Mode](https://help.openai.com/en/articles/12584461-developer-mode-apps-and-full-mcp-connectors-in-chatgpt-beta) for current availability and setup instructions.
</Warning>

Add web scraping and search capabilities to ChatGPT with Firecrawl MCP.

## Quick Setup

### 1. Get Your API Key

Sign up at [firecrawl.dev/app/api-keys](https://www.firecrawl.dev/app/api-keys) and copy your API key.

### 2. Enable Developer Mode

Open ChatGPT settings by clicking your username in the bottom left corner, or navigate directly to [chatgpt.com/#settings](https://chatgpt.com/#settings).

In the settings modal, scroll to the bottom and select **Advanced Settings**. Toggle **Developer mode** to ON.

<Frame>
  <img alt="ChatGPT settings showing Advanced Settings with Developer mode toggle" />
</Frame>

### 3. Create the Connector

With Developer mode enabled, go to the **Apps & Connectors** tab in the same settings modal.

Click the **Create** button in the top right corner.

<Frame>
  <img alt="Apps & Connectors tab with Create button highlighted" />
</Frame>

Fill in the connector details:

* **Name:** `Firecrawl MCP`
* **Description:** `Web scraping, crawling, search, and content extraction` (optional)
* **MCP Server URL:** `https://mcp.firecrawl.dev/YOUR_API_KEY_HERE/v2/mcp`
* **Authentication:** `None`

Replace `YOUR_API_KEY_HERE` in the URL with your actual [Firecrawl API key](https://www.firecrawl.dev/app/api-keys).

<Frame>
  <img alt="New Connector form filled out with Firecrawl MCP details" />
</Frame>

Check the **"I understand and want to continue"** checkbox, then click **Create**.

### 4. Verify Setup

Go back to the main ChatGPT interface. You should see **Developer mode** displayed, indicating that MCP connectors are active.

If you do not see Developer mode, reload the page. If it still does not appear, open settings again and verify that Developer mode is toggled ON under Advanced Settings.

### 5. Access Firecrawl Tools

To use Firecrawl in a conversation, click the **+** button in the chat input, then select **More** and choose **Firecrawl MCP**.

<Frame>
  <img alt="ChatGPT chat input showing + menu expanded with More submenu and Firecrawl MCP option" />
</Frame>

## Quick Demo

With Firecrawl MCP selected, try these prompts:

**Search:**

```
Search for the latest React Server Components updates
```

**Scrape:**

```
Scrape firecrawl.dev and tell me what it does
```

**Get docs:**

```
Scrape the Vercel documentation for edge functions and summarize it
```

## Tool Confirmation

When ChatGPT uses the Firecrawl MCP tools, you may see a confirmation prompt asking for your approval. Some ChatGPT Desktop versions auto-approve tool calls without showing this dialog. If no prompt appears, you can verify the tool was invoked by checking for a "Called tool" section in the response or reviewing your usage at [firecrawl.dev/app/usage](https://www.firecrawl.dev/app/usage).

<Tip>
  **No confirmation prompt appearing?** If ChatGPT answers your question without showing a confirmation dialog, the Firecrawl MCP connector is most likely not attached to the conversation. Go back to [Step 5](#5-access-firecrawl-tools) and make sure you click the **+** button, select **More**, and choose **Firecrawl MCP** before sending your prompt. The connector must be attached to each new conversation.
</Tip>

<Frame>
  <img alt="ChatGPT tool confirmation dialog showing Firecrawl MCP request" />
</Frame>

You can check **"Remember for this conversation"** to avoid repeated confirmations during the same chat session. This security measure is implemented by OpenAI to ensure MCP tools do not perform unintended actions.

Once confirmed, ChatGPT will execute the request and return the results.

<Frame>
  <img alt="Example of Firecrawl search results displayed in ChatGPT" />
</Frame>


# MCP Web Search & Scrape in Claude.ai
Source: https://docs.firecrawl.dev/developer-guides/mcp-setup-guides/claude-ai

Add web scraping and search to Claude.ai (Co-work) in 2 minutes

Add web scraping and search capabilities to Claude.ai with Firecrawl MCP using custom connectors.

<Note>
  Looking for Claude Code setup? See the [Claude Code guide](/quickstarts/claude-code) instead.
</Note>

## Quick Setup

### 1. Get Your API Key

Sign up at [firecrawl.dev/app/api-keys](https://www.firecrawl.dev/app/api-keys) and copy your API key.

### 2. Add Custom Connector

Go to [Settings > Connectors](https://claude.ai/settings/connectors) in Claude.ai and click **Add custom connector**.

Fill in the connector details:

* **URL:** `https://mcp.firecrawl.dev/YOUR_API_KEY/v2/mcp`
* **OAuth Client ID:** Leave blank
* **OAuth Client Secret:** Leave blank

Replace `YOUR_API_KEY` in the URL with your actual [Firecrawl API key](https://www.firecrawl.dev/app/api-keys). Your API key is embedded directly in the URL, so no additional authentication fields are needed.

Click **Add** to save the connector.

### 3. Enable in Conversation

In any Claude.ai conversation, click the **+** button at the bottom left, go to **Connectors**, and enable the Firecrawl connector.

## Quick Demo

With the Firecrawl connector enabled, try these prompts:

**Search the web:**

```
Search for the latest Next.js 15 features
```

**Scrape a page:**

```
Scrape firecrawl.dev and tell me what it does
```

**Get documentation:**

```
Find and scrape the Stripe API docs for payment intents
```

Claude will automatically use Firecrawl's search and scrape tools to get the information.


# MCP Web Search & Scrape in Factory AI
Source: https://docs.firecrawl.dev/developer-guides/mcp-setup-guides/factory-ai

Add web scraping and search to Factory AI in 2 minutes

Add web scraping and search capabilities to Factory AI with Firecrawl MCP.

## Quick Setup

### 1. Get Your API Key

Sign up at [firecrawl.dev/app](https://firecrawl.dev/app) and copy your API key.

### 2. Install Factory AI CLI

Install the [Factory AI CLI](https://docs.factory.ai/cli/getting-started/quickstart) if you haven't already:

**macOS/Linux:**

```bash theme={null}
curl -fsSL https://app.factory.ai/cli | sh
```

**Windows:**

```powershell theme={null}
iwr https://app.factory.ai/cli/install.ps1 -useb | iex
```

### 3. Add Firecrawl MCP Server

In the Factory droid CLI, add Firecrawl using the `/mcp add` command:

```bash theme={null}
/mcp add firecrawl "npx -y firecrawl-mcp" -e FIRECRAWL_API_KEY=your-api-key-here
```

Replace `your-api-key-here` with your actual Firecrawl API key.

### 4. Done!

The Firecrawl tools are now available in your Factory AI session!

## Quick Demo

Try these in Factory AI:

**Search the web:**

```
Search for the latest Next.js 15 features
```

**Scrape a page:**

```
Scrape firecrawl.dev and tell me what it does
```

**Get documentation:**

```
Find and scrape the Stripe API docs for payment intents
```

Factory will automatically use Firecrawl's search and scrape tools to get the information.


# Choosing the Data Extractor
Source: https://docs.firecrawl.dev/developer-guides/usage-guides/choosing-the-data-extractor

Compare /agent, /extract, and /scrape (JSON mode) to pick the right tool for structured data extraction

Firecrawl offers three approaches for extracting structured data from web pages. Each serves different use cases with varying levels of automation and control.

## Quick Comparison

| Feature             | `/agent`                               | `/extract`                                 | `/scrape` (JSON mode)                     |
| ------------------- | -------------------------------------- | ------------------------------------------ | ----------------------------------------- |
| **Status**          | Active                                 | Use `/agent` instead                       | Active                                    |
| **URL Required**    | No (optional)                          | Yes (wildcards supported)                  | Yes (single URL)                          |
| **Scope**           | Web-wide discovery                     | Multiple pages/domains                     | Single page                               |
| **URL Discovery**   | Autonomous web search                  | Crawls from given URLs                     | None                                      |
| **Processing**      | Asynchronous                           | Asynchronous                               | Synchronous                               |
| **Schema Required** | No (prompt or schema)                  | No (prompt or schema)                      | No (prompt or schema)                     |
| **Pricing**         | Dynamic (5 free runs/day)              | Token-based (1 credit = 15 tokens)         | 5 credits/page (1 base + 4 for JSON mode) |
| **Best For**        | Research, discovery, complex gathering | Multi-page extraction (when you know URLs) | Known single-page extraction              |

## 1. `/agent` Endpoint

The `/agent` endpoint is Firecrawl's most advanced offering—the successor to `/extract`. It uses AI agents to autonomously search, navigate, and gather data from across the web.

### Key Characteristics

* **URLs Optional**: Just describe what you need via `prompt`; URLs are completely optional
* **Autonomous Navigation**: The agent searches and navigates deep into sites to find your data
* **Deep Web Search**: Autonomously discovers information across multiple domains and pages
* **Parallel Processing**: Processes multiple sources simultaneously for faster results
* **Models Available**: `spark-1-mini` (default, 60% cheaper) and `spark-1-pro` (higher accuracy)

### Example

<CodeGroup>
  ```python Python theme={null}
  from firecrawl import Firecrawl
  from pydantic import BaseModel, Field
  from typing import List, Optional

  app = Firecrawl(api_key="fc-YOUR_API_KEY")

  class Founder(BaseModel):
      name: str = Field(description="Full name of the founder")
      role: Optional[str] = Field(None, description="Role or position")
      background: Optional[str] = Field(None, description="Professional background")

  class FoundersSchema(BaseModel):
      founders: List[Founder] = Field(description="List of founders")

  result = app.agent(
      prompt="Find the founders of Firecrawl",
      schema=FoundersSchema,
      model="spark-1-mini",
      max_credits=100
  )

  print(result.data)
  ```

  ```js Node theme={null}
  import Firecrawl from '@mendable/firecrawl-js';
  import { z } from 'zod';

  const firecrawl = new Firecrawl({ apiKey: "fc-YOUR_API_KEY" });

  const result = await firecrawl.agent({
    prompt: "Find the founders of Firecrawl",
    schema: z.object({
      founders: z.array(z.object({
        name: z.string().describe("Full name of the founder"),
        role: z.string().describe("Role or position").optional(),
        background: z.string().describe("Professional background").optional()
      })).describe("List of founders")
    }),
    model: "spark-1-mini",
    maxCredits: 100
  });

  console.log(result.data);
  ```

  ```bash cURL theme={null}
  curl -X POST "https://api.firecrawl.dev/v2/agent" \
    -H "Authorization: Bearer $FIRECRAWL_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{
      "prompt": "Find the founders of Firecrawl",
      "model": "spark-1-mini",
      "maxCredits": 100,
      "schema": {
        "type": "object",
        "properties": {
          "founders": {
            "type": "array",
            "description": "List of founders",
            "items": {
              "type": "object",
              "properties": {
                "name": { "type": "string", "description": "Full name" },
                "role": { "type": "string", "description": "Role or position" },
                "background": { "type": "string", "description": "Professional background" }
              },
              "required": ["name"]
            }
          }
        },
        "required": ["founders"]
      }
    }'
  ```
</CodeGroup>

### Best Use Case: Autonomous Research & Discovery

**Scenario**: You need to find information about AI startups that raised Series A funding, including their founders and funding amounts.

**Why `/agent`**: You don't know which websites contain this information. The agent will autonomously search the web, navigate to relevant sources (Crunchbase, news sites, company pages), and compile the structured data for you.

For more details, see the [Agent documentation](/features/agent).

***

## 2. `/extract` Endpoint

<Note>
  **Use `/agent` instead**: We recommend migrating to [`/agent`](/features/agent)—it's faster, more reliable, doesn't require URLs, and handles all `/extract` use cases plus more.
</Note>

The `/extract` endpoint collects structured data from specified URLs or entire domains using LLM-powered extraction.

### Key Characteristics

* **URLs Typically Required**: Provide at least one URL (supports wildcards like `example.com/*`)
* **Domain Crawling**: Can crawl and parse all URLs discovered in a domain
* **Web Search Enhancement**: Optional `enableWebSearch` to follow links outside specified domains
* **Schema Optional**: Supports strict JSON schema OR natural language prompts
* **Async Processing**: Returns job ID for status checking

### The URL Limitation

The fundamental challenge with `/extract` is that you typically need to know URLs upfront:

1. **Discovery gap**: For tasks like "find YC W24 companies," you don't know which URLs contain the data. You'd need a separate search step before calling `/extract`.
2. **Awkward web search**: While `enableWebSearch` exists, it's constrained to start from URLs you provide—an awkward workflow for discovery tasks.
3. **Why `/agent` was created**: `/extract` is good at extracting from known locations, but less effective at discovering where data lives.

### Example

<CodeGroup>
  ```python Python theme={null}
  from firecrawl import Firecrawl

  firecrawl = Firecrawl(api_key="fc-YOUR-API-KEY")

  schema = {
      "type": "object",
      "properties": {"description": {"type": "string"}},
      "required": ["description"],
  }

  res = firecrawl.extract(
      urls=["https://docs.firecrawl.dev"],
      prompt="Extract the page description",
      schema=schema,
  )

  print(res.data["description"])
  ```

  ```js Node theme={null}
  import Firecrawl from '@mendable/firecrawl-js';

  const firecrawl = new Firecrawl({ apiKey: "fc-YOUR-API-KEY" });

  const schema = {
    type: 'object',
    properties: {
      title: { type: 'string' }
    },
    required: ['title']
  };

  const res = await firecrawl.extract({
    urls: ['https://docs.firecrawl.dev'],
    prompt: 'Extract the page title',
    schema,
    scrapeOptions: { formats: [{ type: 'json', prompt: 'Extract', schema }] }
  });

  console.log(res.status || res.success, res.data);
  ```

  ```bash cURL theme={null}
  curl -s -X POST "https://api.firecrawl.dev/v2/extract" \
    -H "Authorization: Bearer $FIRECRAWL_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{
      "urls": ["https://docs.firecrawl.dev"],
      "prompt": "Extract the page title",
      "schema": {
        "type": "object",
        "properties": {"title": {"type": "string"}},
        "required": ["title"]
      },
      "scrapeOptions": {
        "formats": [{"type": "json", "prompt": "Extract", "schema": {"type": "object"}}]
      }
    }'
  ```
</CodeGroup>

### Best Use Case: Targeted Multi-Page Extraction

**Scenario**: You have your competitor's documentation URL and want to extract all their API endpoints from `docs.competitor.com/*`.

**Why `/extract` worked here**: You knew the exact domain. But even then, `/agent` with URLs provided would typically give better results today.

For more details, see the [Extract documentation](/features/extract).

***

## 3. `/scrape` Endpoint with JSON Mode

The `/scrape` endpoint with JSON mode is the most controlled approach—it extracts structured data from a single known URL using an LLM to parse the page content into your specified schema.

### Key Characteristics

* **Single URL Only**: Designed for extracting data from one specific page at a time
* **Exact URL Required**: You must know the precise URL containing the data
* **Schema Optional**: Can use JSON schema OR just a prompt (LLM chooses structure)
* **Synchronous**: Returns data immediately (no job polling needed)
* **Additional Formats**: Can combine JSON extraction with markdown, HTML, screenshots in one request

### Example

<CodeGroup>
  ```python Python theme={null}
  from firecrawl import Firecrawl
  from pydantic import BaseModel

  app = Firecrawl(api_key="fc-YOUR-API-KEY")

  class CompanyInfo(BaseModel):
      company_mission: str
      supports_sso: bool
      is_open_source: bool
      is_in_yc: bool

  result = app.scrape(
      'https://firecrawl.dev',
      formats=[{
        "type": "json",
        "schema": CompanyInfo.model_json_schema()
      }],
      only_main_content=False,
      timeout=120000
  )

  print(result)
  ```

  ```js Node theme={null}
  import Firecrawl from "@mendable/firecrawl-js";
  import { z } from "zod";

  const app = new Firecrawl({
    apiKey: "fc-YOUR_API_KEY"
  });

  // Define schema to extract contents into
  const schema = z.object({
    company_mission: z.string(),
    supports_sso: z.boolean(),
    is_open_source: z.boolean(),
    is_in_yc: z.boolean()
  });

  const result = await app.scrape("https://firecrawl.dev", {
    formats: [{
      type: "json",
      schema: schema
    }],
  });

  console.log(result);
  ```

  ```bash cURL theme={null}
  curl -X POST https://api.firecrawl.dev/v2/scrape \
      -H 'Content-Type: application/json' \
      -H 'Authorization: Bearer YOUR_API_KEY' \
      -d '{
        "url": "https://firecrawl.dev",
        "formats": [ {
          "type": "json",
          "schema": {
            "type": "object",
            "properties": {
              "company_mission": {
                        "type": "string"
              },
              "supports_sso": {
                        "type": "boolean"
              },
              "is_open_source": {
                        "type": "boolean"
              },
              "is_in_yc": {
                        "type": "boolean"
              }
            },
            "required": [
              "company_mission",
              "supports_sso",
              "is_open_source",
              "is_in_yc"
            ]
          }
        } ]
      }'
  ```
</CodeGroup>

### Best Use Case: Single-Page Precision Extraction

**Scenario**: You're building a price monitoring tool and need to extract the price, stock status, and product details from a specific product page you already have the URL for.

**Why `/scrape` with JSON mode**: You know exactly which page contains the data, need precise single-page extraction, and want synchronous results without job management overhead.

For more details, see the [JSON mode documentation](/features/llm-extract).

***

## Decision Guide

**Do you know the exact URL(s) containing your data?**

* **NO** → Use `/agent` (autonomous web discovery)
* **YES**
  * **Single page?** → Use `/scrape` with JSON mode
  * **Multiple pages?** → Use `/agent` with URLs (or batch `/scrape`)

### Recommendations by Scenario

| Scenario                                           | Recommended Endpoint            |
| -------------------------------------------------- | ------------------------------- |
| "Find all AI startups and their funding"           | `/agent`                        |
| "Extract data from this specific product page"     | `/scrape` (JSON mode)           |
| "Get all blog posts from competitor.com"           | `/agent` with URL               |
| "Monitor prices across multiple known URLs"        | `/scrape` with batch processing |
| "Research companies in a specific industry"        | `/agent`                        |
| "Extract contact info from 50 known company pages" | `/scrape` with batch processing |

***

## Pricing

| Endpoint              | Cost                                      | Notes                                 |
| --------------------- | ----------------------------------------- | ------------------------------------- |
| `/scrape` (JSON mode) | 5 credits/page (1 base + 4 for JSON mode) | Fixed, predictable                    |
| `/extract`            | Token-based (1 credit = 15 tokens)        | Variable based on content             |
| `/agent`              | Dynamic                                   | 5 free runs/day; varies by complexity |

### Example: "Find the founders of Firecrawl"

| Endpoint   | How It Works                                    | Credits Used           |
| ---------- | ----------------------------------------------- | ---------------------- |
| `/scrape`  | You find the URL manually, then scrape 1 page   | \~1 credit             |
| `/extract` | You provide URL(s), it extracts structured data | Variable (token-based) |
| `/agent`   | Just send the prompt—agent finds and extracts   | \~100–500 credits      |

**Tradeoff**: `/scrape` is cheapest but requires you to know the URL. `/agent` costs more but handles discovery automatically.

For detailed pricing, see [Firecrawl Pricing](https://firecrawl.dev/pricing).

***

## Migration: `/extract` → `/agent`

If you're currently using `/extract`, migration is straightforward:

**Before (extract):**

```python theme={null}
result = app.extract(
    urls=["https://example.com/*"],
    prompt="Extract product information",
    schema=schema
)
```

**After (agent):**

```python theme={null}
result = app.agent(
    urls=["https://example.com"],  # Optional - can omit entirely
    prompt="Extract product information from example.com",
    schema=schema,
    model="spark-1-mini"  # or "spark-1-pro" for higher accuracy
)
```

The key advantage: with `/agent`, you can drop the URLs entirely and just describe what you need.

***

## Key Takeaways

1. **Know the exact URL?** Use `/scrape` with JSON mode—it's the cheapest (5 credits/page), fastest (synchronous), and most predictable option.

2. **Need autonomous research?** Use `/agent`—it handles discovery automatically with 5 free runs/day, then dynamic pricing based on complexity.

3. **Migrate from `/extract`** to `/agent` for new projects—`/agent` is the successor with better capabilities.

4. **Cost vs. convenience tradeoff**: `/scrape` is most cost-effective when you know your URLs; `/agent` costs more but eliminates manual URL discovery.

***

## Further Reading

* [Agent documentation](/features/agent)
* [Agent models](/features/models)
* [JSON mode documentation](/features/llm-extract)
* [Extract documentation](/features/extract)
* [Batch scraping](/features/batch-scrape)


# Firecrawl + Dify
Source: https://docs.firecrawl.dev/developer-guides/workflow-automation/dify

Official plugin for Firecrawl + Dify AI workflow automation

<Note>
  **Official Dify Plugin:** [marketplace.dify.ai/plugins/langgenius/firecrawl](https://marketplace.dify.ai/plugins/langgenius/firecrawl)

  Official plugin by Dify team • 44,000+ installs • Chatflow & Agent apps • Free to use
</Note>

## Dify Integration Overview

Dify is an open-source LLM app development platform. The official Firecrawl plugin enables web crawling and scraping directly in your AI workflows.

<CardGroup>
  <Card title="Chatflow & Workflow Apps" icon="diagram-project">
    Build visual pipelines with Firecrawl nodes for data extraction
  </Card>

  <Card title="Agent Applications" icon="robot">
    Give AI agents the power to scrape live web data on demand
  </Card>
</CardGroup>

## Firecrawl Tools in Dify

<AccordionGroup>
  <Accordion title="Scrape" icon="file-code">
    Convert any URL into clean, structured data. Transform raw HTML into actionable insights.

    **Use Cases:** Extract product data, scrape article content, get structured data with JSON mode.
  </Accordion>

  <Accordion title="Crawl" icon="spider">
    Perform recursive crawls of websites and subdomains to gather extensive content.

    **Use Cases:** Full site content extraction, documentation scraping, multi-page data collection.
  </Accordion>

  <Accordion title="Map" icon="sitemap">
    Generate a complete map of all URLs present on a website.

    **Use Cases:** Site structure analysis, SEO auditing, URL discovery for batch scraping.
  </Accordion>

  <Accordion title="Crawl Job" icon="list-check">
    Retrieve scraping results based on a Job ID or cancel ongoing tasks.

    **Use Cases:** Monitor long-running crawls, manage async scraping workflows, cancel operations when needed.
  </Accordion>
</AccordionGroup>

## Getting Started

<Steps>
  <Step title="Install Firecrawl Plugin">
    Access the [Dify Plugin Marketplace](https://marketplace.dify.ai/plugins/langgenius/firecrawl) and install the Firecrawl tool
  </Step>

  <Step title="Get Firecrawl API Key">
    Visit [Firecrawl API Keys](https://www.firecrawl.dev/app/api-keys) and create a new API key
  </Step>

  <Step title="Authorize in Dify">
    Navigate to **Plugins > Firecrawl > To Authorize** and input your API key
  </Step>

  <Step title="Add to Your Workflow">
    Drag Firecrawl tools into your Chatflow, Workflow, or Agent application
  </Step>

  <Step title="Configure & Test">
    Set up parameters and test your workflow
  </Step>
</Steps>

## Usage Patterns

<Tabs>
  <Tab title="Chatflow Apps">
    **Visual Pipeline Integration**

    1. Add Firecrawl node to your pipeline
    2. Select action (Map, Crawl, Scrape)
    3. Define input variables
    4. Execute pipeline sequentially

    **Example Flow:**

    ```
    User Input → Firecrawl (Scrape) → LLM Processing → Response
    ```
  </Tab>

  <Tab title="Workflow Apps">
    **Automated Data Processing**

    Build multi-step workflows with:

    * Scheduled scraping
    * Data transformation
    * Database storage
    * Notifications

    **Example Flow:**

    ```
    Schedule Trigger → Firecrawl (Crawl) → Data Processing → Storage
    ```
  </Tab>

  <Tab title="Agent Apps">
    **AI-Powered Web Access**

    Give agents real-time web scraping capabilities:

    1. Add Firecrawl tool to Agent
    2. Agent autonomously decides when to scrape
    3. LLM analyzes extracted content
    4. Agent provides informed responses

    **Use Case:** Customer support agents that reference live documentation
  </Tab>
</Tabs>

## Common Use Cases

<CardGroup>
  <Card title="AI Chatbot with Live Data" icon="messages">
    Build RAG-powered chatbots that scrape and reference live website content
  </Card>

  <Card title="Content Analysis Agent" icon="brain">
    Agents that research topics by scraping and analyzing multiple sources
  </Card>

  <Card title="Competitor Monitoring" icon="binoculars">
    Automated workflows that track competitor websites and alert on changes
  </Card>

  <Card title="Data Enrichment Pipeline" icon="database">
    Extract and enrich data from websites into structured databases
  </Card>
</CardGroup>

## Firecrawl Actions

| Tool          | Description                    | Best For                |
| ------------- | ------------------------------ | ----------------------- |
| **Scrape**    | Single-page data extraction    | Quick content capture   |
| **Crawl**     | Multi-page recursive crawling  | Full site extraction    |
| **Map**       | URL discovery and site mapping | SEO analysis, URL lists |
| **Crawl Job** | Async job management           | Long-running operations |

## Best Practices

<CardGroup>
  <Card title="Agent Apps" icon="robot">
    * Let agents decide when to scrape
    * Use natural language instructions
    * Enable tool calling in LLM settings
    * Monitor token usage with large scrapes
  </Card>

  <Card title="Workflow Apps" icon="diagram-project">
    * Use Map before Crawl for large sites
    * Set appropriate crawl limits
    * Add error handling nodes
    * Test with small datasets first
  </Card>
</CardGroup>

## Dify vs Other Platforms

| Feature         | Dify                 | Make                | Zapier              | n8n                 |
| --------------- | -------------------- | ------------------- | ------------------- | ------------------- |
| **Type**        | LLM app platform     | Workflow automation | Workflow automation | Workflow automation |
| **Best For**    | AI agents & chatbots | Visual workflows    | Quick automation    | Developer control   |
| **Pricing**     | Open-source + Cloud  | Operations-based    | Per-task            | Flat monthly        |
| **AI-Native**   | Yes                  | Partial             | Partial             | Partial             |
| **Self-Hosted** | Yes                  | No                  | No                  | Yes                 |

<Tip>
  **Pro Tip:** Dify excels at building AI-native applications where agents need dynamic web access. Perfect for chatbots, research assistants, and AI tools that need live data.
</Tip>


# Firecrawl + Make
Source: https://docs.firecrawl.dev/developer-guides/workflow-automation/make

Official integration and workflow automation for Firecrawl + Make

<Note>
  **Official Make Integration:** [make.com/en/integrations/firecrawl](https://www.make.com/en/integrations/firecrawl)

  Connect with 3,000+ apps • Visual workflow builder • Enterprise-grade automation • AI-powered scenarios
</Note>

## Make Integration Overview

Make (formerly Integromat) provides a verified, officially supported Firecrawl integration maintained by Mendable.

<CardGroup>
  <Card title="Visual Workflow Builder" icon="diagram-project">
    Design complex automations with Make's intuitive visual interface
  </Card>

  <Card title="Enterprise Ready" icon="building">
    Scale securely with enterprise-grade automation and controls
  </Card>
</CardGroup>

## Firecrawl Modules in Make

<AccordionGroup>
  <Accordion title="Actions (7 modules)" icon="bolt">
    ### Crawl a Website

    Crawl a URL and get its content from multiple pages

    ***

    ### Extract a Website

    Extract structured data from pages using LLMs

    ***

    ### Scrape a Website

    Scrape a URL and get its content from a single page

    ***

    ### Map a Website

    Map multiple URLs based on options

    ***

    ### Search a Website

    Web search with Firecrawl's scraping capabilities

    ***

    ### Get Crawl Status

    Get the status of a given crawl event ID

    ***

    ### Get Extract Status

    Get the status of a given extraction event ID
  </Accordion>

  <Accordion title="Search (1 module)" icon="magnifying-glass">
    ### Search a Website

    Full-page content retrieval for any search query
  </Accordion>

  <Accordion title="Advanced" icon="code">
    ### Make an API Call

    Perform arbitrary authorized API calls for custom use cases
  </Accordion>
</AccordionGroup>

## Popular App Integrations

<Tabs>
  <Tab title="Data Storage">
    **Google Sheets** - Track and log scraped data in spreadsheets

    **Airtable** - Build structured databases with scraped content

    **Google Drive** - Store scraped files and reports

    **Notion** - Organize research and web data
  </Tab>

  <Tab title="Communication">
    **Slack** - Get alerts for website changes and updates

    **Telegram Bot** - Instant notifications for monitoring

    **Gmail** - Email reports and digests

    **Microsoft 365 Email** - Enterprise email automation
  </Tab>

  <Tab title="CRM & Sales">
    **HubSpot CRM** - Enrich leads with web data

    **monday.com** - Track competitor intelligence

    **ClickUp** - Manage research tasks
  </Tab>

  <Tab title="AI Tools">
    **OpenAI (ChatGPT, DALL-E)** - Analyze and summarize scraped content

    **Google Gemini AI** - Process and extract insights

    **Perplexity AI** - Enhanced research workflows

    **Make AI Agents** - Build adaptive AI-powered automations
  </Tab>
</Tabs>

## Common Workflow Patterns

<CardGroup>
  <Card title="Competitor Monitoring" icon="binoculars">
    **Schedule** → Firecrawl (Scrape) → Google Sheets (Log) → Slack (Alert)

    Track competitor websites and get instant notifications
  </Card>

  <Card title="Lead Enrichment" icon="users">
    **Google Forms** → Firecrawl (Scrape company site) → HubSpot CRM (Update)

    Automatically enrich leads with company data
  </Card>

  <Card title="Content Aggregation" icon="newspaper">
    **Schedule** → Firecrawl (Crawl blog) → OpenAI (Summarize) → Gmail (Send digest)

    Automated content curation and distribution
  </Card>

  <Card title="Price Monitoring" icon="chart-line">
    **Schedule (Hourly)** → Firecrawl (Scrape) → Filter → Telegram (Alert)

    Real-time price tracking and alerts
  </Card>
</CardGroup>

## Getting Started

<Steps>
  <Step title="Sign up for Firecrawl">
    Get your API key at [firecrawl.dev](https://firecrawl.dev)
  </Step>

  <Step title="Create a Scenario in Make">
    Log into [Make](https://make.com) and click "Create a new scenario"
  </Step>

  <Step title="Add Firecrawl Module">
    Search for "Firecrawl" and select your desired action
  </Step>

  <Step title="Connect with API Key">
    Add your Firecrawl API key to authenticate
  </Step>

  <Step title="Configure & Test">
    Set up your workflow parameters and run a test
  </Step>
</Steps>

## Firecrawl Actions Overview

| Module                | Use Case                         | Best For            |
| --------------------- | -------------------------------- | ------------------- |
| **Scrape a Website**  | Single-page data extraction      | Quick data capture  |
| **Crawl a Website**   | Multi-page content collection    | Full site scraping  |
| **Extract a Website** | AI-powered structured extraction | Complex data needs  |
| **Search a Website**  | Search + full content            | Research automation |
| **Map a Website**     | URL discovery                    | SEO analysis        |

## Best Practices

<CardGroup>
  <Card title="Performance" icon="gauge-high">
    * Use **Scrape** for single pages (fastest)
    * Use **Crawl** with limits for large sites
    * Schedule appropriately to avoid rate limits
    * Add error handling modules
  </Card>

  <Card title="Cost Optimization" icon="piggy-bank">
    * Schedule strategically (hourly/daily/weekly)
    * Use filters to prevent unnecessary runs
    * Set crawl limits to control API usage
    * Test in Firecrawl playground first
  </Card>
</CardGroup>

## Industry Use Cases

<AccordionGroup>
  <Accordion title="E-commerce" icon="cart-shopping">
    * Competitor price monitoring
    * Product availability tracking
    * Review aggregation and analysis
    * Inventory level monitoring
  </Accordion>

  <Accordion title="Real Estate" icon="house">
    * Listing aggregation from multiple sources
    * Market trend analysis
    * Property data enrichment
    * Competitive pricing intelligence
  </Accordion>

  <Accordion title="Marketing" icon="bullhorn">
    * Competitor content monitoring
    * SEO performance tracking
    * Backlink analysis
    * Social media mention tracking
  </Accordion>

  <Accordion title="Finance" icon="chart-line">
    * Market data collection
    * News and sentiment aggregation
    * Regulatory filing monitoring
    * Stock price tracking
  </Accordion>

  <Accordion title="HR & Recruitment" icon="user-tie">
    * Job posting aggregation
    * Company research automation
    * Candidate background enrichment
    * Salary benchmarking
  </Accordion>
</AccordionGroup>

## Make vs Zapier vs n8n

| Feature            | Make                              | Zapier           | n8n                  |
| ------------------ | --------------------------------- | ---------------- | -------------------- |
| **Setup**          | Visual builder, cloud             | No-code, cloud   | Self-hosted or cloud |
| **Pricing**        | Operations-based                  | Per-task pricing | Flat monthly         |
| **Integrations**   | 3,000+ apps                       | 8,000+ apps      | 400+ integrations    |
| **Complexity**     | Advanced workflows                | Simple workflows | Complex workflows    |
| **Best For**       | Visual automation, mid-complexity | Quick automation | Developer control    |
| **Learning Curve** | Moderate                          | Easy             | Moderate-Advanced    |

<Tip>
  **Pro Tip:** Make excels at visual workflow design and complex automations. Perfect for teams that need more control than Zapier but prefer visual building over n8n's code-first approach.
</Tip>


# Firecrawl + n8n
Source: https://docs.firecrawl.dev/developer-guides/workflow-automation/n8n

Learn how to use Firecrawl with n8n for web scraping automation, a complete step-by-step guide.

## Introduction to Firecrawl and n8n

Web scraping automation has become essential for modern businesses. Whether you need to monitor competitor prices, aggregate content, generate leads, or power AI applications with real-time data, the combination of Firecrawl and n8n provides a powerful solution without requiring programming knowledge.

<img alt="Firecrawl and n8n integration" />

**What is n8n?**

n8n is an open-source workflow automation platform that connects different tools and services together. Think of it as a visual programming environment where you drag and drop nodes onto a canvas, connect them, and create automated workflows. With over 400 integrations, n8n lets you build complex automations without writing code.

## Why Use Firecrawl with n8n?

Traditional web scraping presents several challenges. Custom scripts break when websites update their structure. Anti-bot systems block automated requests. JavaScript-heavy sites don't render properly. Infrastructure requires constant maintenance.

Firecrawl handles these technical complexities on the scraping side, while n8n provides the automation framework. Together, they let you build production-ready workflows that:

* Extract data from any website reliably
* Connect scraped data to other business tools
* Run on schedules or triggered by events
* Scale from simple tasks to complex pipelines

This guide will walk you through setting up both platforms and building your first scraping workflow from scratch.

## Step 1: Create Your Firecrawl Account

Firecrawl provides the web scraping capabilities for your workflows. Let's set up your account and get your API credentials.

### Sign Up for Firecrawl

1. Navigate to [firecrawl.dev](https://firecrawl.dev) in your web browser
2. Click the "Get Started" or "Sign Up" button
3. Create an account using your email address or GitHub login
4. Verify your email if prompted

### Get Your API Key

After signing in, you need an API key to connect Firecrawl to n8n:

1. Go to your Firecrawl dashboard
2. Navigate to the [API Keys page](https://www.firecrawl.dev/app/api-keys)
3. Click "Create New API Key"
4. Give your key a descriptive name (e.g., "n8n Integration")
5. Copy the generated API key and save it somewhere secure

<img alt="Firecrawl api key section" />

<Note>
  Your API key is like a password. Keep it secure and never share it publicly. You'll need this key in the next section.
</Note>

Firecrawl provides free credits when you sign up, which is enough to test your workflows and complete this tutorial.

## Step 2: Set Up n8n

n8n offers two deployment options: cloud-hosted or self-hosted. For beginners, the cloud version is the fastest way to get started.

### Choose Your n8n Version

**n8n Cloud (Recommended for beginners):**

* No installation required
* Free tier available
* Managed infrastructure
* Automatic updates

**Self-Hosted:**

* Complete data control
* Run on your own servers
* Requires Docker installation
* Good for advanced users with specific security requirements

Choose the option that fits your needs. Both paths will lead you to the same workflow editor interface.

### Option A: n8n Cloud (Recommended for Beginners)

1. Visit [n8n.cloud](https://n8n.cloud)
2. Click "Start Free" or "Sign Up"
3. Register using your email address or GitHub
4. Complete the verification process
5. You'll be directed to your n8n dashboard

<img alt="n8n Cloud homepage showing the signup options" />

The free tier provides everything you need to build and test workflows. You can upgrade later if you need more execution time or advanced features.

### Option B: Self-Hosted with Docker

If you prefer to run n8n on your own infrastructure, you can set it up quickly using Docker.

**Prerequisites:**

* [Docker Desktop](https://www.docker.com/products/docker-desktop/) installed on your computer
* Basic familiarity with command line/terminal

**Installation Steps:**

1. Open your terminal or command prompt
2. Create a Docker volume to persist your workflow data:

```bash theme={null}
docker volume create n8n_data
```

This volume stores your workflows, credentials, and execution history so they persist even if you restart the container.

3. Run the n8n Docker container:

```bash theme={null}
docker run -it --rm --name n8n -p 5678:5678 -v n8n_data:/home/node/.n8n docker.n8n.io/n8nio/n8n
```

<img alt="Terminal showing the docker commands being executed with n8n starting up" />

4. Wait for n8n to start. You'll see output indicating the server is running
5. Open your web browser and navigate to `http://localhost:5678`
6. Create your n8n account by registering with an email

<img alt="n8n self-hosted welcome screen at localhost:5678" />

Your self-hosted n8n instance is now running locally. The interface is identical to n8n Cloud, so you can follow the rest of this guide regardless of which option you chose.

<Note>
  The `--rm` flag automatically removes the container when you stop it, but your data remains safe in the `n8n_data` volume. For production deployments, see the [n8n self-hosting documentation](https://docs.n8n.io/hosting/) for more advanced configuration options.
</Note>

### Understanding the n8n Interface

When you first log in to n8n, you'll see the main dashboard:

<img alt="n8n dashboard showing the workflow list view with &#x22;Create new workflow&#x22; button" />

Key interface elements:

* **Workflows**: Your saved automations appear here
* **Executions**: History of workflow runs
* **Credentials**: Stored API keys and authentication tokens
* **Settings**: Account and workspace configuration

Click "Create New Workflow" to open the workflow editor.

### The Workflow Canvas

The workflow editor is where you'll build your automations:

<img alt="Empty n8n workflow canvas with the &#x22;+&#x22; button visible in the center" />

Important elements:

* **Canvas**: The main area where you place and connect nodes
* **Add Node Button (+)**: Click this to add new nodes to your workflow
* **Node Panel**: Opens when you click "+" showing all available nodes
* **Execute Workflow**: Runs your workflow manually for testing
* **Save**: Saves your workflow configuration

Let's build your first workflow by adding the Firecrawl node.

## Step 3: Install and Configure the Firecrawl Node

n8n includes native support for Firecrawl. You'll install the node and connect it to your Firecrawl account using the API key you created earlier.

### Add the Firecrawl Node to Your Workflow

1. In your new workflow canvas, click the "**+**" button in the center
2. The node selection panel opens on the right side
3. In the search box at the top, type "**Firecrawl**"
4. You'll see the Firecrawl node appear in the search results

<img alt="Clicking the + button, typing &#x22;Firecrawl&#x22; in the search, and the Firecrawl node appearing" />

5. Click "**Install**" next to the Firecrawl node
6. Wait for the installation to complete (this takes a few seconds)
7. Once installed, click on the Firecrawl node to add it to your canvas

<img alt="Firecrawl node now added to the canvas, showing as a box with the Firecrawl logo" />

The Firecrawl node will appear on your canvas as a box with the Firecrawl logo. This node represents a single Firecrawl operation in your workflow.

### Connect Your Firecrawl API Key

<Tip>
  **n8n Cloud users:** The launch offer that included a free Hobby plan and 100,000 credits has expired. You can still use the one-click **"Connect to Firecrawl"** OAuth button when adding the Firecrawl node. This automatically creates a new Firecrawl team linked to your n8n account and grants **10,000 free credits**. These credits do not include a Hobby plan upgrade. To view them on the [Firecrawl dashboard](https://www.firecrawl.dev/app/usage), make sure you switch to your n8n-linked team using the team selector in the top-left corner.
</Tip>

Before you can use the Firecrawl node, you need to authenticate it with your API key:

1. Click on the Firecrawl node box to open its configuration panel on the right
2. At the top, you'll see a "Credential to connect with" dropdown
3. Since this is your first time, click "**Create New Credential**"

<img alt="Firecrawl node configuration panel showing the credentials dropdown with &#x22;Create New Credential&#x22; option" />

4. A credential configuration window opens
5. Enter a name for this credential (e.g., "My Firecrawl Account")
6. Paste your Firecrawl API key in the "API Key" field
7. Click "**Save**" at the bottom

The credential is now saved in n8n. You won't need to enter your API key again for future Firecrawl nodes.

### Test Your Connection

Let's verify that your Firecrawl node is properly connected:

1. With the Firecrawl node still selected, look at the configuration panel
2. In the "Resource" dropdown, select "**Scrape a url and get its content**"
3. In the "URL" field, enter: `https://firecrawl.dev`
4. Leave other settings at their defaults for now
5. Click the "**Test step**" button at the bottom right of the node

<img alt="Selecting Scrape operation, entering example.com URL, and clicking Test step button" />

If everything is configured correctly, you'll see the scraped content from example.com appear in the output panel below the node.

Congratulations! Your Firecrawl node is now connected and working.

## Step 4: Create Your Telegram Bot

Before building your first workflow, you'll need a Telegram bot to receive notifications. Telegram bots are free and easy to create through Telegram's BotFather.

### Create a Bot with BotFather

1. Open Telegram on your phone or desktop
2. Search for "**@BotFather**" (the official bot from Telegram)
3. Start a conversation with BotFather by clicking "**Start**"
4. Send the command `/newbot` to create a new bot
5. BotFather will ask you to choose a name for your bot (this is the display name users will see)
6. Enter a name like "**My Firecrawl Bot**"
7. Next, choose a username for your bot. It must end with "bot" (e.g., "**my\_firecrawl\_updates\_bot**")
8. If the username is available, BotFather will create your bot and send you a message with your bot token

<img alt="Creating a bot with BotFather, showing the full conversation flow" />

<Note>
  Save your bot token securely. This token is like a password that allows n8n to send messages as your bot. Never share it publicly.
</Note>

### Get Your Chat ID

To send messages to yourself, you need your Telegram chat ID:

1. Open your web browser and visit this URL (replace `YOUR_BOT_TOKEN` with your actual bot token):
   ```
   https://api.telegram.org/botYOUR_BOT_TOKEN/getUpdates
   ```
2. Keep this browser tab open
3. Now, search for your bot's username in Telegram (the one you just created)
4. Start a conversation with your bot by clicking "**Start**"
5. Send any message to your bot (e.g., "hello")
6. Go back to the browser tab and refresh the page
7. Look for the `"chat":{"id":` field in the JSON response
8. The number next to `"id":` is your chat ID (e.g., `123456789`)
9. Save this chat ID for later

<img alt="Browser showing Telegram API getUpdates response with chat ID highlighted" />

<Note>
  Your chat ID is the unique identifier for your conversation with the bot. You'll use this to tell n8n where to send messages.
</Note>

You now have everything needed to integrate Telegram with your n8n workflows.

## Step 5: Build Practical Workflows with Telegram

Now let's build three real-world workflows that send information directly to your Telegram. These examples demonstrate different Firecrawl operations and how to integrate them with Telegram notifications.

### Example 1: Daily Firecrawl Product Updates Summary

Get a daily summary of Firecrawl product updates delivered to your Telegram every morning.

**What you'll build:**

* Scrapes Firecrawl's product updates blog at 9 AM daily
* Uses AI to generate a summary of the content
* Sends the summary to your Telegram

**Step-by-step:**

1. Create a new workflow in n8n
2. Add a **Schedule Trigger** node:
   * Click the "**+**" button on canvas
   * Search for "**Schedule Trigger**"
   * Configure: Every day at 9:00 AM

<img alt="Schedule Trigger configured for daily 9 AM execution" />

3. Add the **Firecrawl** node:
   * Click "**+**" next to Schedule Trigger
   * Search for and add "**Firecrawl**"
   * Select your Firecrawl credential
   * Configure:
     * **Resource**: Scrape a url and get its content
     * **URL**: `https://www.firecrawl.dev/blog/category/product-updates`
     * **Formats**: Select "Summary"

<img alt="Adding and configuring Firecrawl node with the blog URL and Summary format selected" />

4. Add the **Telegram** node:
   * Click "**+**" next to Firecrawl
   * Search for "**Telegram**"
   * Click "**Send a text message**" to add it to the canvas

5. Set up Telegram credentials:
   * Click on the Telegram node to open its configuration
   * In the "Credential to connect with" dropdown, click "**Create New Credential**"
   * Paste your bot token from BotFather
   * Click "**Save**"

<img alt="Telegram credential configuration with bot token field" />

6. Configure the Telegram message:
   * **Operation**: Send Message

   * **Chat ID**: Enter your chat ID

   * **Text**: Leave this with a "hello" message for now

   * Click **Execute step** to test sending a message while receiving the summary from Firecrawl.

<img alt="Configuring Telegram node and mapping the summary field to the message text" />

* Now with Firecrawl's summary structure, add the summary to the message text by dragging the `summary` field from Firecrawl node output.

7. Test the workflow:
   * Click "**Execute Workflow**"
   * Check your Telegram for the summary message

<img alt="Complete workflow showing Schedule Trigger → Firecrawl → Telegram with all nodes connected" />

8. Activate the workflow by toggling the "**Active**" switch

Your Telegram bot will now send you a daily summary of Firecrawl product updates every morning at 9 AM.

### Example 2: AI News Search to Telegram

This workflow uses Firecrawl's Search operation to find AI news and send formatted results to Telegram.

**Key differences from Example 1:**

* Uses a **Manual Trigger** instead of Schedule (run on demand)
* Uses **Search** operation instead of Scrape
* Includes a **Code** node to format multiple results

**Build the workflow:**

1. Create a new workflow and add a **Manual Trigger** node

2. Add **Firecrawl** node with these settings:
   * **Resource**: Search and optionally scrape search results
   * **Query**: `ai news`
   * **Limit**: 5

<img alt="Firecrawl Search node configuration with &#x22;ai news&#x22; query" />

3. Add a **Code** node to format the search results:
   * Select "Run Once for All Items"
   * Paste this code:

```javascript theme={null}
const results = $input.all();
let message = "Latest AI News:\n\n";

results.forEach((item) => {
  const webData = item.json.data.web;
  webData.forEach((article, index) => {
    message += `${index + 1}. ${article.title}\n`;
    message += `${article.description}\n`;
    message += `${article.url}\n\n`;
  });
});

return [{ json: { message } }];
```

<img alt="Adding Code node and pasting the formatting script" />

4. Update **Telegram** node (using your saved credential):
   * **Text**: Drag the `message` field from Code node

<img alt="Complete workflow execution with AI news sent to Telegram" />

<Tip>
  Replace the Manual Trigger with a Schedule Trigger to get automatic AI news updates at set intervals.
</Tip>

### Example 3: AI-Powered News Summary

This workflow adds AI to Example 2, using OpenAI to generate intelligent summaries of the latest AI news before sending to Telegram.

**Key changes from Example 2:**

* Add **OpenAI credentials** setup
* Add **AI Agent** node between Code and Telegram
* AI Agent analyzes and summarizes all the news articles intelligently
* Telegram receives the AI-generated summary instead of raw news list

**Modify the workflow:**

1. **Get your OpenAI API key**:
   * Go to [platform.openai.com/api-keys](https://platform.openai.com/api-keys)
   * Sign in or create an account
   * Click "**Create new secret key**"
   * Give it a name (e.g., "n8n Integration")
   * Copy the API key immediately (you won't see it again)

<img alt="OpenAI dashboard showing API key creation" />

2. **Add and connect the AI Agent node**:
   * Click "**+**" after the Code node
   * Search for "**Basic LLM Chain**" or "**AI Agent**"
   * Drag the `message` field from the Code node to the AI Agent's input prompt field
   * Select **OpenAI** as the LLM provider

<img alt="Adding AI Agent node, dragging input from Code node, and connecting OpenAI as LLM" />

3. **Add your OpenAI credentials**:
   * Click "**Create New Credential**" for OpenAI
   * Paste your OpenAI API key
   * Select model: **gpt-5-mini** (cost-effective) or **gpt-5** (more capable)
   * Click "**Save**"

<img alt="Adding OpenAI credentials to the AI Agent node" />

4. **Add the system prompt to the AI Agent**:
   * In the AI Agent node, add this system prompt:

```
You are an AI news analyst. Analyze the provided AI news articles and create a concise,
insightful summary highlighting the most important developments and trends.
Group related topics together and provide context about why these developments matter.
Keep the summary conversational and engaging, around 3-4 paragraphs.
```

<img alt="Adding the system prompt to the AI Agent node" />

5. **Update the Telegram node and test**:
   * Update the Telegram node:
     * **Text**: Drag the AI Agent's output (the generated summary)
     * Remove the old mapping to the Code node's message
   * Click "**Execute Workflow**" to test
   * The AI will analyze all news articles and create a summary
   * Check your Telegram for the AI-generated summary

<img alt="Complete workflow execution with AI-generated summary sent to Telegram" />

<Note>
  The AI Agent receives all the formatted news articles and creates an intelligent summary, making it easier to understand trends and important developments at a glance.
</Note>

## Understanding Firecrawl Operations

Now that you've built some workflows, let's explore the different Firecrawl operations available in n8n. Each operation is designed for specific web scraping use cases.

### Scrape a url and get its content

Extracts content from a single web page and returns it in various formats.

**What it does:**

* Scrapes a single URL
* Returns clean markdown, HTML, or AI-generated summaries
* Can capture screenshots and extract links

**Best for:**

* Article extraction
* Product page monitoring
* Blog post scraping
* Generating page summaries

**Example use case:** Daily blog summaries (like Example 1 above)

### Search and optionally scrape search results

Performs web searches and returns results with optional content scraping.

**What it does:**

* Searches the web, news, or images
* Returns titles, descriptions, and URLs
* Optionally scrapes the full content of results

**Best for:**

* Research automation
* News monitoring
* Trend discovery
* Finding relevant content

**Example use case:** AI news aggregation (like Example 2 above)

### Crawl a website

Recursively discovers and scrapes multiple pages from a website.

**What it does:**

* Follows links automatically
* Scrapes multiple pages in one operation
* Can filter URLs by patterns

**Best for:**

* Full documentation extraction
* Site archiving
* Multi-page data collection

### Map a website and get urls

Returns all URLs found on a website without scraping content.

**What it does:**

* Discovers all links on a site
* Returns clean URL list
* Fast and lightweight

**Best for:**

* URL discovery
* Sitemap generation
* Planning larger crawls

### Extract Data

Uses AI to extract structured information based on custom prompts or schemas.

**What it does:**

* AI-powered data extraction
* Returns data in your specified format
* Works across multiple pages

**Best for:**

* Custom data extraction
* Building databases
* Structured information gathering

### Batch Scrape

Scrapes multiple URLs in parallel efficiently.

**What it does:**

* Processes multiple URLs at once
* More efficient than loops
* Returns all results together

**Best for:**

* Processing URL lists
* Bulk data collection
* Large-scale scraping projects

### Agent

Uses an AI agent to autonomously browse and extract data from websites based on a natural language prompt.

**What it does:**

* Accepts a prompt describing what data you need
* AI agent navigates and extracts information autonomously
* Available in **Sync** mode (waits for results) and **Async** mode (returns a job ID immediately)
* Use **Get Agent Status** to poll for results when using Async mode

**Best for:**

* Complex, multi-page data gathering guided by a prompt
* Extracting information when you don't know the exact page structure
* Research tasks that require navigating multiple pages

**Sync vs. Async:**

* **Agent (Sync)** starts the job and waits for the result in one step — simplest for most use cases. The **Max Wait Time** parameter controls how long the node polls before timing out (default: 300 seconds, maximum: 600 seconds). If the agent job takes longer than this, the node returns a timeout status even though the job may still complete on the Firecrawl side. For jobs that may exceed 10 minutes, use the async mode instead.
* **Agent (Async)** returns a job ID immediately. Add a second Firecrawl node with the **Get Agent Status** operation to retrieve results once the job completes.

For details on the agent feature, see the [Agent documentation](/features/agent).

### Browser Sandbox

Provides a persistent browser session that you can control with code, allowing multi-step browser automation within a single session.

**Operations:**

* **Create Browser Session** — starts a new browser session and returns a `sessionId`
* **Execute Browser Code** — runs JavaScript, Python, or bash code in the session (using the `sessionId` from the Create step)
* **List Browser Sessions** — lists active or destroyed sessions
* **Delete Browser Session** — destroys a session when you are done

**Best for:**

* Multi-step browser workflows that require maintaining state across pages
* Dynamic page navigation where the number of steps is not known in advance
* Workflows that use persistent browser profiles to preserve cookies and localStorage across runs

In n8n, select the **Browser** resource on the Firecrawl node to access these operations. Pass the `sessionId` from the Create step into each subsequent Execute or Delete step. Use n8n's **Loop Over Items** node to iterate through a dynamic list of pages, calling Execute for each one within the same session.

For details on the Browser Sandbox feature, see the [Browser Sandbox documentation](/features/browser-sandbox).

## Workflow Templates and Examples

Instead of building from scratch, you can start with pre-built templates. The n8n community has created numerous Firecrawl workflows you can copy and customize.

### Featured Templates

<CardGroup>
  <Card title="Complete n8n Tutorial" icon="rocket" href="https://www.firecrawl.dev/blog/firecrawl-n8n-web-automation">
    Build an AI chatbot with web access using Firecrawl and n8n
  </Card>

  <Card title="8 Production Workflows" icon="layer-group" href="https://www.firecrawl.dev/blog/n8n-web-scraping-workflow-templates">
    Ready-to-use templates for lead generation, price monitoring, and more
  </Card>

  <Card title="Supabase pgvector RAG Pipeline" icon="database" href="https://n8n.io/workflows/13911-scrape-and-ingest-web-content-into-supabase-pgvector-with-firecrawl/">
    Scrape pages into embeddings and store in Supabase pgvector for RAG
  </Card>

  <Card title="Lead Enrichment with AI" icon="building" href="https://n8n.io/workflows/13910-enrich-company-leads-with-firecrawl-openrouter-ai-and-supabase/">
    Scrape company websites and extract structured business signals
  </Card>

  <Card title="Pinecone RAG Pipeline" icon="tree" href="https://n8n.io/workflows/13964-scrape-and-ingest-web-pages-into-a-pinecone-rag-stack-with-firecrawl-and-openai/">
    Scrape pages into embeddings and store in Pinecone for RAG
  </Card>

  <Card title="n8n Community Workflows" icon="book" href="https://n8n.io/workflows?search=firecrawl">
    Browse hundreds of workflows using Firecrawl
  </Card>

  <Card title="Official n8n Integration" icon="plug" href="https://n8n.io/integrations/firecrawl/">
    View official integration documentation
  </Card>
</CardGroup>

### How to Import Templates

To use a template from the n8n community:

1. Click on a workflow template link
2. Click "**Import template to localhost:5678 self-hosted instance**" button on the template page
3. The workflow opens in your n8n instance
4. Configure credentials for each node
5. Customize settings for your use case
6. Activate the workflow

<img alt="Importing a template from n8n.io, showing the import button and workflow appearing in n8n" />

## Best Practices

Follow these guidelines to build reliable, efficient workflows:

### Testing and Debugging

* Always test workflows manually before activating schedules
* Use the "**Execute Workflow**" button to test the entire flow
* Check output data at each node to verify correctness
* Use the "**Executions**" tab to review past runs and debug issues

<img alt="Executions tab showing workflow run history with timestamps and status" />

### Error Handling

* Add Error Trigger nodes to catch and handle failures
* Set up notifications when workflows fail
* Use the "**Continue On Fail**" setting for non-critical nodes
* Monitor your workflow executions regularly

### Performance Optimization

* Use Batch Scrape for multiple URLs instead of loops
* Set appropriate rate limits to avoid overwhelming target sites
* Cache data when possible to reduce unnecessary requests
* Schedule intensive workflows during off-peak hours

### Security

* Never expose API keys in workflow configurations
* Use n8n's credential system to securely store authentication
* Be careful when sharing workflows publicly
* Follow target websites' terms of service and robots.txt

## Next Steps

You now have the fundamentals to build web scraping automations with Firecrawl and n8n. Here's how to continue learning:

### Explore Advanced Features

* Study webhook configurations for real-time data processing
* Experiment with AI-powered extraction using prompts and schemas
* Build complex multi-step workflows with branching logic

### Join the Community

* [Firecrawl Discord](https://discord.gg/firecrawl) - Get help with Firecrawl and discuss web scraping
* [n8n Community Forum](https://community.n8n.io/) - Ask questions about workflow automation
* Share your workflows and learn from others

### Recommended Learning Path

1. Complete the example workflows in this guide
2. Modify templates from the community library
3. Build a workflow to solve a real problem in your work
4. Explore advanced Firecrawl operations
5. Contribute your own templates to help others

<Note>
  **Need help?** If you're stuck or have questions, the Firecrawl and n8n communities are active and helpful. Don't hesitate to ask for guidance as you build your automations.
</Note>

## Additional Resources

* [Firecrawl API Documentation](/api-reference/v2-introduction)
* [n8n Documentation](https://docs.n8n.io/)
* [Web Scraping Best Practices](https://www.firecrawl.dev/blog)


# Firecrawl + Zapier
Source: https://docs.firecrawl.dev/developer-guides/workflow-automation/zapier

Official tutorials and Zapier integration templates for Firecrawl + Zapier automation

<Note>
  **Official Zapier Integration:** [zapier.com/apps/firecrawl/integrations](https://zapier.com/apps/firecrawl/integrations)

  Connect with 8,000+ apps • No-code automation • Pre-built Zap templates • Cloud-based
</Note>

## Official Blog Post

<Card title="How Zapier uses Firecrawl" icon="rocket" href="https://www.firecrawl.dev/blog/how-zapier-uses-firecrawl-to-power-chatbots">
  Real-world case study: How Zapier integrated Firecrawl into Zapier Chatbots in a single afternoon.
</Card>

## Popular Integrations

<AccordionGroup>
  <Accordion title="Data Storage & Databases" icon="database">
    ### Google Sheets

    → [View Integration](https://zapier.com/apps/google-sheets/integrations/firecrawl)

    Track competitor data, centralize marketing insights, and automate data collection.

    **Best For:** Business owners, marketing teams

    ***

    ### Airtable

    → [View Integration](https://zapier.com/apps/airtable/integrations/firecrawl)

    Build lead generation databases and content aggregation systems with structured storage.

    **Best For:** Sales teams, project managers

    ***

    ### Zapier Tables

    → [View Integration](https://zapier.com/apps/zapier-tables/integrations/firecrawl)

    No-code database automation for employee onboarding and centralized lead management.

    **Best For:** HR teams, operations
  </Accordion>

  <Accordion title="Communication & Notifications" icon="bell">
    ### Slack

    → [View Integration](https://zapier.com/apps/slack/integrations/firecrawl)

    Get website change notifications, competitor monitoring alerts, and market intelligence updates.

    **Best For:** Marketing teams, product managers

    ***

    ### Telegram

    → [View Integration](https://zapier.com/apps/telegram/integrations/firecrawl)

    Instant price alerts, breaking news notifications, and real-time monitoring.

    **Best For:** Traders, news enthusiasts
  </Accordion>

  <Accordion title="CRM & Sales" icon="briefcase">
    ### HubSpot

    → [View Integration](https://zapier.com/apps/hubspot/integrations/firecrawl)

    Contact enrichment, lead scoring with web data, and marketing automation.

    **Best For:** Marketing ops, sales ops

    ***

    ### Pipedrive

    → [View Integration](https://zapier.com/apps/pipedrive/integrations/firecrawl)

    Lead enrichment from websites and competitor intelligence tracking.

    **Best For:** Sales teams, account executives

    ***

    ### Attio

    → [View Integration](https://zapier.com/apps/attio/integrations/firecrawl)

    Modern CRM data enrichment and relationship intelligence.

    **Best For:** Modern sales teams
  </Accordion>

  <Accordion title="Productivity & Documentation" icon="file-lines">
    ### Google Docs

    → [View Integration](https://zapier.com/apps/google-docs/integrations/firecrawl)

    Automated report generation, research documentation, and content aggregation.

    **Best For:** Researchers, content creators

    ***

    ### Notion

    → [View Integration](https://zapier.com/apps/notion/integrations/firecrawl)

    Knowledge base updates, research library building, and content curation.

    **Best For:** Product teams, researchers
  </Accordion>

  <Accordion title="Zapier Native Tools" icon="zap">
    ### Schedule by Zapier

    → [View Integration](https://zapier.com/apps/schedule/integrations/firecrawl)

    Run hourly, daily, weekly, or monthly scraping automatically.

    ***

    ### Zapier Interfaces

    → [View Integration](https://zapier.com/apps/interfaces/integrations/firecrawl)

    Build custom internal tools with form-based scraping and team dashboards.

    **Best For:** Operations teams

    ***

    ### Zapier Chatbots

    → [View Integration](https://zapier.com/apps/zapier-chatbots/integrations/firecrawl)

    AI chatbots with live web knowledge for customer support and lead generation.

    <Info>Official Zapier product uses Firecrawl internally</Info>
  </Accordion>
</AccordionGroup>

## Firecrawl Actions

| Action                      | Use Case                                  |
| --------------------------- | ----------------------------------------- |
| **Scrape URL**              | Quick single-page data capture            |
| **Crawl Website**           | Full site scraping with multiple pages    |
| **Extract Structured Data** | AI-powered extraction with custom schemas |
| **Search Web**              | Research automation with search + scrape  |
| **Map Website**             | SEO analysis and site structure mapping   |

## Quick Reference

<CardGroup>
  <Card title="Getting Started" icon="play">
    1. Sign up at [firecrawl.dev](https://firecrawl.dev)
    2. Get your API key
    3. Create a Zap in Zapier
    4. Connect Firecrawl with your API key
    5. Choose your workflow and activate
  </Card>

  <Card title="Best Practices" icon="lightbulb">
    * Use `/Scrape URL` for single pages (faster)
    * Schedule strategically (hourly/daily/weekly)
    * Test in Firecrawl playground first
    * Add error handling for failed scrapes
    * Use filters to prevent unnecessary runs
  </Card>
</CardGroup>

## Industry Use Cases

<AccordionGroup>
  <Accordion title="E-commerce" icon="cart-shopping">
    * Price monitoring across competitors
    * Product availability tracking
    * Review aggregation
  </Accordion>

  <Accordion title="Real Estate" icon="house">
    * Listing aggregation
    * Market trend analysis
    * Property data collection
  </Accordion>

  <Accordion title="Marketing" icon="bullhorn">
    * Competitor content tracking
    * SEO monitoring
    * Backlink analysis
  </Accordion>

  <Accordion title="Finance" icon="chart-line">
    * Market data collection
    * News aggregation
    * Regulatory filing monitoring
  </Accordion>

  <Accordion title="Recruitment" icon="user-tie">
    * Job posting aggregation
    * Company research automation
    * Candidate information enrichment
  </Accordion>
</AccordionGroup>

## Zapier vs n8n

| Feature          | Zapier                                | n8n                      |
| ---------------- | ------------------------------------- | ------------------------ |
| **Setup**        | No-code, cloud-based                  | Self-hosted or cloud     |
| **Pricing**      | Per-task pricing                      | Flat monthly             |
| **Integrations** | 8,000+ apps                           | 400+ integrations        |
| **Best For**     | Quick automation, non-technical users | Custom logic, developers |

<Tip>
  **Pro Tip:** Start with Zapier's pre-built templates and customize as needed. Perfect for quick, no-code automation!
</Tip>


# Debug Firecrawl with Ask
Source: https://docs.firecrawl.dev/features/ask

Agentic debugging for your Firecrawl integration

Firecrawl `/support/ask` is an AI support agent exposed as an API. Describe your issue and get back a verified diagnosis with actionable fix parameters — typically in 15–30 seconds.

**Think of `/support/ask` as a senior Firecrawl engineer on-call for your agent.**

<Info>
  The Ask API is designed primarily for **AI agent callers**. If you're building agents that use Firecrawl for scraping, crawling, or data extraction, wire `/support/ask` into your error-handling flow for autonomous issue resolution.
</Info>

## Two endpoints

| Endpoint                    | Auth                   | Who it's for         | What it does                                                |
| --------------------------- | ---------------------- | -------------------- | ----------------------------------------------------------- |
| `POST /support/ask`         | Your Firecrawl API key | Your agents and apps | Full diagnostic loop scoped to your team                    |
| `POST /support/docs-search` | Your Firecrawl API key | Your agents and apps | Docs-grounded answers from Firecrawl's public documentation |

## Quick start

### Debug a failing crawl

```bash theme={null}
curl -X POST https://api.firecrawl.dev/v2/support/ask \
  -H "Authorization: Bearer fc-YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "question": "my crawl returned 3 pages but I expected 50"
  }'
```

### Search the docs

```bash theme={null}
curl -X POST https://api.firecrawl.dev/v2/support/docs-search \
  -H "Authorization: Bearer fc-YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "question": "how do I set up webhook signature verification?"
  }'
```

## How it works

When you call `/support/ask`, the AI agent:

1. **Gathers evidence** — inspects your job logs, account state, credit usage, and relevant documentation in parallel
2. **Diagnoses the issue** — reasons across all evidence to identify the root cause
3. **Proposes a fix** — generates machine-actionable `fixParameters` you can apply directly to your next API call
4. **Validates the fix** — when possible, tests the fix against the live Firecrawl API (e.g., retrying a scrape with adjusted parameters) and reports the result

## Using Ask in your agent

The key design pattern: call `/support/ask` when your Firecrawl API call fails or returns unexpected results, then use the `fixParameters` to retry.

### Python example

```python theme={null}
import requests

FIRECRAWL_API_KEY = "fc-YOUR_API_KEY"

def diagnose_firecrawl_issue(question, rationale=None):
    """Call the Firecrawl Ask API to debug an issue."""
    payload = {"question": question}
    if rationale:
        payload["rationale"] = rationale

    response = requests.post(
        "https://api.firecrawl.dev/v2/support/ask",
        headers={
            "Authorization": f"Bearer {FIRECRAWL_API_KEY}",
            "Content-Type": "application/json",
        },
        json=payload,
    )
    return response.json()


# Example: debug a scrape that returned empty content
result = diagnose_firecrawl_issue(
    question="scrape returned empty markdown for https://example.com",
    rationale="user needs product pricing data for competitive analysis",
)

print(result["answer"])
print(result["fixParameters"])  # e.g., {"waitFor": 5000, "actions": [...]}
print(result["confidence"])     # "high", "medium", or "low"
```

### Node.js example

```javascript theme={null}
async function diagnoseFirecrawlIssue(question, rationale) {
  const response = await fetch(
    "https://api.firecrawl.dev/v2/support/ask",
    {
      method: "POST",
      headers: {
        Authorization: `Bearer ${process.env.FIRECRAWL_API_KEY}`,
        "Content-Type": "application/json",
      },
      body: JSON.stringify({ question, rationale }),
    }
  );
  return response.json();
}

// Example: debug a crawl that stopped early
const result = await diagnoseFirecrawlIssue(
  "my crawl returned 3 pages but I expected 50",
  "user is on their third failed crawl attempt today"
);

console.log(result.answer);
console.log(result.fixParameters);
```

### Agent retry pattern

```python theme={null}
from firecrawl import Firecrawl

client = Firecrawl(api_key="fc-YOUR_API_KEY")

# Step 1: Try the scrape
doc = client.scrape("https://example.com/pricing", formats=["markdown"])

if not doc.markdown or len(doc.markdown) < 100:
    # Step 2: Ask for help debugging
    diagnosis = diagnose_firecrawl_issue(
        question=f"scrape returned only {len(doc.markdown or '')} chars of markdown for https://example.com/pricing",
    )

    # Step 3: Apply fix parameters and retry
    if diagnosis.get("fixParameters"):
        doc = client.scrape(
            "https://example.com/pricing",
            formats=["markdown"],
            **diagnosis["fixParameters"],
        )
```

## Parameters

### `/support/ask`

| Parameter   | Type   | Required | Description                                                                                                 |
| ----------- | ------ | -------- | ----------------------------------------------------------------------------------------------------------- |
| `question`  | string | Yes      | What to debug (1–8,000 characters)                                                                          |
| `rationale` | string | No       | Recommended for AI callers. What the end user is trying to accomplish. Helps prioritize evidence gathering. |
| `context`   | object | No       | Free-form metadata from your agent, included in the debugging prompt                                        |

### `/support/docs-search`

| Parameter  | Type   | Required | Description                                 |
| ---------- | ------ | -------- | ------------------------------------------- |
| `question` | string | Yes      | The question to answer (1–8,000 characters) |

## Response

### `/support/ask` response

```json theme={null}
{
  "requestId": "req_...",
  "answer": "<2-4 sentence prose diagnosis of the issue plus the recommended fix.>",
  "confidence": "high",
  "fixParameters": { "<param>": "<value>" },
  "validation": {
    "tested": true,
    "result": "success",
    "evidence": "<short summary of the validation tool call the agent ran to confirm the fix>"
  },
  "feedback": null,
  "durationMs": 18432
}
```

The actual `answer`, `fixParameters`, and `validation.evidence` are produced per request by the agent based on your specific run; the example above shows the response shape, not a real diagnosis.

### `/support/docs-search` response

```json theme={null}
{
  "requestId": "req_...",
  "answer": "The signature is sent in the X-Firecrawl-Signature header...",
  "evidence": [
    { "pathOrUrl": "webhooks/security.mdx#L1-L52", "reason": "..." }
  ],
  "usage": { "inputTokens": 4356, "outputTokens": 688, "totalTokens": 5044 },
  "durationMs": 11252
}
```

## Performance

| Metric  | Typical       | Maximum                   |
| ------- | ------------- | ------------------------- |
| Latency | 15–30 seconds | 60 seconds (hard ceiling) |

## API Reference

* [Ask endpoint API Reference](/api-reference/endpoint/ask)
* [Docs Search endpoint API Reference](/api-reference/endpoint/docs-search)

Have feedback or need help? Email [help@firecrawl.com](mailto:help@firecrawl.com).

> Are you an AI agent that needs a Firecrawl API key? See [firecrawl.dev/agent-onboarding/SKILL.md](https://www.firecrawl.dev/agent-onboarding/SKILL.md) for automated onboarding instructions.


# Document Parsing
Source: https://docs.firecrawl.dev/features/document-parsing

Learn about document parsing capabilities.

Firecrawl provides powerful document parsing capabilities, allowing you to extract structured content from various document formats. This feature is particularly useful for processing files like spreadsheets, Word documents, and more.

## Supported Document Formats

Firecrawl currently supports the following document formats:

* **Excel Spreadsheets** (`.xlsx`, `.xls`)
  * Each worksheet is converted to an HTML table
  * Worksheets are separated by H2 headings with the sheet name
  * Preserves cell formatting and data types

* **Word Documents** (`.docx`, `.doc`, `.odt`, `.rtf`)
  * Extracts text content while preserving document structure
  * Maintains headings, paragraphs, lists, and tables
  * Preserves basic formatting and styling

* **PDF Documents** (`.pdf`)
  * Extracts text content with layout information
  * Preserves document structure including sections and paragraphs
  * Handles both text-based and scanned PDFs (with OCR support)
  * Supports `mode` option to control parsing strategy: `fast` (text-only), `auto` (text with OCR fallback, default), or `ocr` (force OCR)
  * Priced at 1 credit per-page. See [Pricing](https://firecrawl.dev/pricing) for details.

### PDF Parsing Modes

Use the `parsers` option to control how PDFs are processed:

| Mode   | Description                                                                                                           |
| ------ | --------------------------------------------------------------------------------------------------------------------- |
| `auto` | Attempts fast text-based extraction first, falls back to OCR if needed. This is the default.                          |
| `fast` | Text-based parsing only (embedded text). Fastest option, but will not extract text from scanned or image-heavy pages. |
| `ocr`  | Forces OCR parsing on every page. Use for scanned documents or when `auto` misclassifies a page.                      |

```js theme={null}
// Object syntax with mode
parsers: [{ type: "pdf", mode: "ocr", maxPages: 20 }]

// Default (auto mode)
parsers: [{ type: "pdf" }]
```

## How to Use Document Parsing

Document parsing in Firecrawl works in two ways:

1. **URL-based parsing (`/v2/scrape`)**: provide a URL that points to a supported document type.
2. **File upload parsing (`/v2/parse`)**: upload file bytes directly with `multipart/form-data`.

For URL-based parsing, Firecrawl detects file type from extension or content type automatically.

### Upload documents with `/v2/parse`

Use `/v2/parse` when the source document is local or not publicly accessible by URL.

<CodeGroup>
  ```bash cURL theme={null}
  curl -X POST "https://api.firecrawl.dev/v2/parse" \
    -H "Authorization: Bearer fc-YOUR-API-KEY" \
    -F 'options={"formats":["markdown"]}' \
    -F "file=@./document.docx;type=application/vnd.openxmlformats-officedocument.wordprocessingml.document"
  ```

  ```js Node theme={null}
  import Firecrawl from "@mendable/firecrawl-js";

  const app = new Firecrawl({ apiKey: "fc-YOUR-API-KEY" });

  const doc = await app.parse(
    {
      data: "<html><body><h1>Upload Parse</h1></body></html>",
      filename: "upload.html",
      contentType: "text/html",
    },
    { formats: ["markdown"] },
  );

  console.log(doc.markdown);
  ```

  ```python Python theme={null}
  from firecrawl import Firecrawl
  from firecrawl.v2.types import ScrapeOptions

  app = Firecrawl(api_key="fc-YOUR-API-KEY")
  doc = app.parse(
      b"<!DOCTYPE html><html><body><h1>Upload Parse</h1></body></html>",
      filename="upload.html",
      content_type="text/html",
      options=ScrapeOptions(formats=["markdown"]),
  )
  print(doc.markdown)
  ```
</CodeGroup>

### Example: Scraping an Excel File

```js Node theme={null}
import Firecrawl from '@mendable/firecrawl-js';

const firecrawl = new Firecrawl({ apiKey: "fc-YOUR-API-KEY" });

const doc = await firecrawl.scrape('https://example.com/data.xlsx');

console.log(doc.markdown);
```

### Example: Scraping a Word Document

```js Node theme={null}
import Firecrawl from '@mendable/firecrawl-js';

const firecrawl = new Firecrawl({ apiKey: "fc-YOUR-API-KEY" });

const doc = await firecrawl.scrape('https://example.com/data.docx');

console.log(doc.markdown);
```

## Output Format

All supported document types are converted to clean, structured markdown. For example, an Excel file with multiple sheets might be converted to:

```markdown theme={null}
## Sheet1

| Name  | Value |
|-------|-------|
| Item 1 | 100   |
| Item 2 | 200   |

## Sheet2

| Date       | Description  |
|------------|--------------|
| 2023-01-01 | First quarter|
```

> Are you an AI agent that needs a Firecrawl API key? See [firecrawl.dev/agent-onboarding/SKILL.md](https://www.firecrawl.dev/agent-onboarding/SKILL.md) for automated onboarding instructions.


# Interact after scraping
Source: https://docs.firecrawl.dev/features/interact

Interact with a page you fetched by prompting or running code.

Scrape a page to get clean data, then call `/interact` to start taking actions in that page - click buttons, fill forms, extract dynamic content, or navigate deeper. Just describe what you want, or write code if you need full control.

<CardGroup>
  <Card title="AI prompts" icon="wand-magic-sparkles">
    Describe what action you want to take in the page
  </Card>

  <Card title="Code execution" icon="code">
    Interact via code execution securely with playwright, agent-browser
  </Card>

  <Card title="Live view" icon="eye">
    Watch or interact with the browser in real time via embeddable stream
  </Card>
</CardGroup>

## How It Works

1. **Scrape** a URL with `POST /v2/scrape`. The response includes a `scrapeId` in `data.metadata.scrapeId`. Optionally pass a `profile` to persist browser state across sessions.
2. **Interact** by calling `POST /v2/scrape/{scrapeId}/interact` with a `prompt` or with playwright `code`. On the first call, the scraped session is resumed and you can start interacting with the page.
3. **Stop** the session with `DELETE /v2/scrape/{scrapeId}/interact` when you're done.

## Quick Start

Scrape a page, interact with it, and stop the session:

<CodeGroup>
  ```python Python theme={null}
  from firecrawl import Firecrawl

  app = Firecrawl(api_key="fc-YOUR-API-KEY")

  # 1. Scrape Amazon's homepage
  result = app.scrape("https://www.amazon.com", formats=["markdown"])
  scrape_id = result.metadata.scrape_id

  # 2. Interact — search for a product and get its price
  app.interact(scrape_id, prompt="Search for iPhone 16 Pro Max")
  response = app.interact(scrape_id, prompt="Click on the first result and tell me the price")
  print(response.output)

  # 3. Stop the session
  app.stop_interaction(scrape_id)
  ```

  ```js Node theme={null}
  import Firecrawl from '@mendable/firecrawl-js';

  const app = new Firecrawl({ apiKey: 'fc-YOUR-API-KEY' });

  // 1. Scrape Amazon's homepage
  const result = await app.scrape('https://www.amazon.com', { formats: ['markdown'] });
  const scrapeId = result.metadata?.scrapeId;

  // 2. Interact — search for a product and get its price
  await app.interact(scrapeId, { prompt: 'Search for iPhone 16 Pro Max' });
  const response = await app.interact(scrapeId, { prompt: 'Click on the first result and tell me the price' });
  console.log(response.output);

  // 3. Stop the session
  await app.stopInteraction(scrapeId);
  ```

  ```bash cURL theme={null}
  # 1. Scrape Amazon's homepage
  RESPONSE=$(curl -s -X POST "https://api.firecrawl.dev/v2/scrape" \
    -H "Authorization: Bearer $FIRECRAWL_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{"url": "https://www.amazon.com", "formats": ["markdown"]}')

  SCRAPE_ID=$(echo $RESPONSE | jq -r '.data.metadata.scrapeId')

  # 2. Interact — search for a product and get its price
  curl -s -X POST "https://api.firecrawl.dev/v2/scrape/$SCRAPE_ID/interact" \
    -H "Authorization: Bearer $FIRECRAWL_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{"prompt": "Search for iPhone 16 Pro Max"}'

  curl -s -X POST "https://api.firecrawl.dev/v2/scrape/$SCRAPE_ID/interact" \
    -H "Authorization: Bearer $FIRECRAWL_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{"prompt": "Click on the first result and tell me the price"}'

  # 3. Stop the session
  curl -s -X DELETE "https://api.firecrawl.dev/v2/scrape/$SCRAPE_ID/interact" \
    -H "Authorization: Bearer $FIRECRAWL_API_KEY"
  ```

  ```bash CLI theme={null}
  # 1. Scrape Amazon's homepage (scrape ID is saved automatically)
  firecrawl scrape https://www.amazon.com

  # 2. Interact — search for a product and get its price
  firecrawl interact "Search for iPhone 16 Pro Max"
  firecrawl interact "Click on the first result and tell me the price"

  # 3. Stop the session
  firecrawl interact stop
  ```
</CodeGroup>

```json Response theme={null}
{
  "success": true,
  "liveViewUrl": "https://liveview.firecrawl.dev/...",
  "interactiveLiveViewUrl": "https://liveview.firecrawl.dev/...",
  "output": "The iPhone 16 Pro Max (256GB) is priced at $1,199.00.",
  "exitCode": 0,
  "killed": false
}
```

## Interact via prompting

The simplest way to interact with a page. Describe what you want in natural language and it will click, type, scroll, and extract data automatically.

<CodeGroup>
  ```python Python theme={null}
  response = app.interact(scrape_id, prompt="What are the customer reviews saying about battery life?")
  print(response.output)
  ```

  ```js Node theme={null}
  const response = await app.interact(scrapeId, {
    prompt: 'What are the customer reviews saying about battery life?',
  });
  console.log(response.output);
  ```

  ```bash cURL theme={null}
  curl -s -X POST "https://api.firecrawl.dev/v2/scrape/$SCRAPE_ID/interact" \
    -H "Authorization: Bearer $FIRECRAWL_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{
      "prompt": "What are the customer reviews saying about battery life?"
    }'
  ```

  ```bash CLI theme={null}
  firecrawl interact "What are the customer reviews saying about battery life?"
  ```
</CodeGroup>

The response includes an `output` field with the agent's answer:

```json Response theme={null}
{
  "success": true,
  "liveViewUrl": "https://liveview.firecrawl.dev/...",
  "interactiveLiveViewUrl": "https://liveview.firecrawl.dev/...",
  "output": "Customers are generally positive about battery life. Most reviewers report 8-10 hours of use on a single charge. A few noted it drains faster with heavy multitasking.",
  "stdout": "...",
  "result": "...",
  "stderr": "",
  "exitCode": 0,
  "killed": false
}
```

### Keep Prompts Small and Focused

Prompts work best when each one is a **single, clear task**. Instead of asking the agent to do a complex multi-step workflow in one shot, break it into separate interact calls. Each call reuses the same browser session, so state carries over between them.

## Running Code

For full control, you can execute code directly in the browser sandbox. The `page` variable (a Playwright Page object) is available in Node.js and Python. Bash mode has [agent-browser](https://github.com/vercel-labs/agent-browser) pre-installed. You can also take screenshots within the session — use `(await page.screenshot()).toString("base64")` in Node.js, `await page.screenshot(path="/tmp/screenshot.png")` in Python, or `agent-browser screenshot` in Bash.

### Node.js (Playwright)

The default language. Write Playwright code directly — `page` is already connected to the browser.

<CodeGroup>
  ```python Python theme={null}
  response = app.interact(scrape_id, code="""
  // Click a button and wait for navigation
  await page.click('#next-page');
  await page.waitForLoadState('networkidle');

  // Extract content from the new page
  const title = await page.title();
  const content = await page.$eval('.article-body', el => el.textContent);
  JSON.stringify({ title, content });
  """)
  print(response.result)
  ```

  ```js Node theme={null}
  const response = await app.interact(scrapeId, {
    code: `
      // Click a button and wait for navigation
      await page.click('#next-page');
      await page.waitForLoadState('networkidle');

      // Extract content from the new page
      const title = await page.title();
      const content = await page.$eval('.article-body', el => el.textContent);
      JSON.stringify({ title, content });
    `,
  });
  console.log(response.result);
  ```

  ```bash cURL theme={null}
  curl -s -X POST "https://api.firecrawl.dev/v2/scrape/$SCRAPE_ID/interact" \
    -H "Authorization: Bearer $FIRECRAWL_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{
      "code": "await page.click(\"#next-page\"); await page.waitForLoadState(\"networkidle\"); const title = await page.title(); JSON.stringify({ title });",
      "language": "node",
      "timeout": 30
    }'
  ```

  ```bash CLI theme={null}
  # Uses the last scrape automatically
  firecrawl interact -c "
    await page.click('#next-page');
    await page.waitForLoadState('networkidle');
    const title = await page.title();
    const content = await page.\$eval('.article-body', el => el.textContent);
    JSON.stringify({ title, content });
  "

  # Or pass a scrape ID explicitly
  # firecrawl interact <scrape-id> -c "await page.title()"
  ```
</CodeGroup>

### Python

Set `language` to `"python"` for Playwright's Python API.

<CodeGroup>
  ```python Python theme={null}
  response = app.interact(
      scrape_id,
      code="""
  import json

  await page.click('#load-more')
  await page.wait_for_load_state('networkidle')

  items = await page.query_selector_all('.item')
  data = []
  for item in items:
      text = await item.text_content()
      data.append(text.strip())

  print(json.dumps(data))
  """,
      language="python",
  )
  print(response.stdout)
  ```

  ```js Node theme={null}
  const response = await app.interact(scrapeId, {
    code: `
  import json

  await page.click('#load-more')
  await page.wait_for_load_state('networkidle')

  items = await page.query_selector_all('.item')
  data = []
  for item in items:
      text = await item.text_content()
      data.append(text.strip())

  print(json.dumps(data))
  `,
    language: 'python',
  });
  console.log(response.stdout);
  ```

  ```bash cURL theme={null}
  curl -s -X POST "https://api.firecrawl.dev/v2/scrape/$SCRAPE_ID/interact" \
    -H "Authorization: Bearer $FIRECRAWL_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{
      "code": "import json\nawait page.click(\"#load-more\")\nawait page.wait_for_load_state(\"networkidle\")\nitems = await page.query_selector_all(\".item\")\ndata = [await i.text_content() for i in items]\nprint(json.dumps(data))",
      "language": "python"
    }'
  ```

  ```bash CLI theme={null}
  firecrawl interact --python -c "
  import json
  await page.click('#load-more')
  await page.wait_for_load_state('networkidle')
  items = await page.query_selector_all('.item')
  data = [await i.text_content() for i in items]
  print(json.dumps(data))
  "
  ```
</CodeGroup>

### Bash (agent-browser)

[agent-browser](https://github.com/vercel-labs/agent-browser) is a CLI pre-installed in the sandbox with 60+ commands. It provides an accessibility tree with element refs (`@e1`, `@e2`, ...) — ideal for LLM-driven automation.

<CodeGroup>
  ```python Python theme={null}
  # Take a snapshot to see interactive elements
  snapshot = app.interact(
      scrape_id,
      code="agent-browser snapshot -i",
      language="bash",
  )
  print(snapshot.stdout)
  # Output:
  # [document]
  #   @e1 [input type="text"] "Search..."
  #   @e2 [button] "Search"
  #   @e3 [link] "About"

  # Interact with elements using @refs
  app.interact(
      scrape_id,
      code='agent-browser fill @e1 "firecrawl" && agent-browser click @e2',
      language="bash",
  )
  ```

  ```js Node theme={null}
  // Take a snapshot to see interactive elements
  const snapshot = await app.interact(scrapeId, {
    code: 'agent-browser snapshot -i',
    language: 'bash',
  });
  console.log(snapshot.stdout);
  // Output:
  // [document]
  //   @e1 [input type="text"] "Search..."
  //   @e2 [button] "Search"
  //   @e3 [link] "About"

  // Interact with elements using @refs
  await app.interact(scrapeId, {
    code: 'agent-browser fill @e1 "firecrawl" && agent-browser click @e2',
    language: 'bash',
  });
  ```

  ```bash cURL theme={null}
  # Take a snapshot to see interactive elements
  curl -s -X POST "https://api.firecrawl.dev/v2/scrape/$SCRAPE_ID/interact" \
    -H "Authorization: Bearer $FIRECRAWL_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{"code": "agent-browser snapshot -i", "language": "bash"}'

  # Interact with elements using @refs
  curl -s -X POST "https://api.firecrawl.dev/v2/scrape/$SCRAPE_ID/interact" \
    -H "Authorization: Bearer $FIRECRAWL_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{"code": "agent-browser fill @e1 \"firecrawl\" && agent-browser click @e2", "language": "bash"}'
  ```

  ```bash CLI theme={null}
  # Take a snapshot to see interactive elements
  firecrawl interact --bash -c "agent-browser snapshot -i"

  # Interact with elements using @refs
  firecrawl interact --bash -c 'agent-browser fill @e1 "firecrawl" && agent-browser click @e2'
  ```
</CodeGroup>

Common agent-browser commands:

| Command                   | Description                               |
| ------------------------- | ----------------------------------------- |
| `snapshot`                | Full accessibility tree with element refs |
| `snapshot -i`             | Interactive elements only                 |
| `click @e1`               | Click element by ref                      |
| `fill @e1 "text"`         | Clear field and type text                 |
| `type @e1 "text"`         | Type without clearing                     |
| `press Enter`             | Press a keyboard key                      |
| `scroll down 500`         | Scroll down by pixels                     |
| `get text @e1`            | Get text content                          |
| `get url`                 | Get current URL                           |
| `wait @e1`                | Wait for element                          |
| `wait --load networkidle` | Wait for network idle                     |
| `find text "X" click`     | Find element by text and click            |
| `screenshot`              | Take a screenshot of the current page     |
| `eval "js code"`          | Run JavaScript in page                    |

## Live View

Every interact response returns a `liveViewUrl` that you can embed to watch the browser in real time. Useful for debugging, demos, or building browser-powered UIs.

```json Response theme={null}
{
  "success": true,
  "liveViewUrl": "https://liveview.firecrawl.dev/...",
  "interactiveLiveViewUrl": "https://liveview.firecrawl.dev/...",
  "stdout": "",
  "result": "...",
  "exitCode": 0
}
```

```html theme={null}
<iframe src="LIVE_VIEW_URL" width="100%" height="600" />
```

### Interactive Live View

The response also includes an `interactiveLiveViewUrl`. Unlike the standard live view which is view-only, the interactive live view allows users to click, type, and interact with the browser session directly through the embedded stream. This is useful for building user-facing browser UIs — such as login flows, or guided workflows where end users need to control the browser.

```html theme={null}
<iframe src="INTERACTIVE_LIVE_VIEW_URL" width="100%" height="600" />
```

## Session Lifecycle

### Creation

The first `POST /v2/scrape/{scrapeId}/interact` continues the scrape session and starts the interaction.

### Reuse

Subsequent interact calls on the same `scrapeId` reuse the existing session. The browser stays open and maintains its state between calls, so you can chain multiple interactions:

<CodeGroup>
  ```python Python theme={null}
  # First call — click a tab
  app.interact(scrape_id, code="await page.click('#tab-2')")

  # Second call — the tab is still selected, extract its content
  result = app.interact(scrape_id, code="await page.$eval('#tab-2-content', el => el.textContent)")
  print(result.result)
  ```

  ```js Node theme={null}
  // First call — click a tab
  await app.interact(scrapeId, { code: "await page.click('#tab-2')" });

  // Second call — the tab is still selected, extract its content
  const result = await app.interact(scrapeId, {
    code: "await page.$eval('#tab-2-content', el => el.textContent)",
  });
  console.log(result.result);
  ```

  ```bash CLI theme={null}
  # First call — click a tab
  firecrawl interact -c "await page.click('#tab-2')"

  # Second call — the tab is still selected, extract its content
  firecrawl interact -c "await page.\$eval('#tab-2-content', el => el.textContent)"
  ```
</CodeGroup>

### Cleanup

Stop the session explicitly when done:

<CodeGroup>
  ```python Python theme={null}
  app.stop_interaction(scrape_id)
  ```

  ```js Node theme={null}
  await app.stopInteraction(scrapeId);
  ```

  ```bash cURL theme={null}
  curl -s -X DELETE "https://api.firecrawl.dev/v2/scrape/$SCRAPE_ID/interact" \
    -H "Authorization: Bearer $FIRECRAWL_API_KEY"
  ```

  ```bash CLI theme={null}
  # Stops the last scrape session
  firecrawl interact stop

  # Or stop a specific session by ID
  # firecrawl interact stop <scrape-id>
  ```
</CodeGroup>

Sessions also expire automatically based on TTL (default: 10 minutes) or inactivity timeout (default: 5 minutes).

<Warning>
  Always stop sessions when you're done to avoid unnecessary billing. Credits are prorated by the second.
</Warning>

## Persistent Profiles

By default, each scrape + interact session starts with a clean browser. With `profile`, you can save and reuse browser state (cookies, localStorage, sessions) across scrapes. This is useful for staying logged in and preserving preferences.

Pass the `profile` parameter when calling scrape. Sessions with the same profile name share state.

<CodeGroup>
  ```python Python theme={null}
  from firecrawl import Firecrawl

  app = Firecrawl(api_key="fc-YOUR-API-KEY")

  # Session 1: Scrape with a profile, log in, then stop (state is saved)
  result = app.scrape(
      "https://app.example.com/login",
      formats=["markdown"],
      profile={"name": "my-app", "save_changes": True},
  )
  scrape_id = result.metadata.scrape_id

  app.interact(scrape_id, prompt="Fill in user@example.com and password, then click Login")
  app.stop_interaction(scrape_id)

  # Session 2: Scrape with the same profile — already logged in
  result = app.scrape(
      "https://app.example.com/dashboard",
      formats=["markdown"],
      profile={"name": "my-app", "save_changes": True},
  )
  scrape_id = result.metadata.scrape_id

  response = app.interact(scrape_id, prompt="Extract the dashboard data")
  print(response.output)
  app.stop_interaction(scrape_id)
  ```

  ```js Node theme={null}
  import Firecrawl from '@mendable/firecrawl-js';

  const app = new Firecrawl({ apiKey: 'fc-YOUR-API-KEY' });

  // Session 1: Scrape with a profile, log in, then stop (state is saved)
  const result1 = await app.scrape('https://app.example.com/login', {
    formats: ['markdown'],
    profile: { name: 'my-app', saveChanges: true },
  });
  const scrapeId1 = result1.metadata?.scrapeId;

  await app.interact(scrapeId1, { prompt: 'Fill in user@example.com and password, then click Login' });
  await app.stopInteraction(scrapeId1);

  // Session 2: Scrape with the same profile — already logged in
  const result2 = await app.scrape('https://app.example.com/dashboard', {
    formats: ['markdown'],
    profile: { name: 'my-app', saveChanges: true },
  });
  const scrapeId2 = result2.metadata?.scrapeId;

  const response = await app.interact(scrapeId2, { prompt: 'Extract the dashboard data' });
  console.log(response.output);
  await app.stopInteraction(scrapeId2);
  ```

  ```bash cURL theme={null}
  # Session 1: Scrape with a profile
  RESPONSE=$(curl -s -X POST "https://api.firecrawl.dev/v2/scrape" \
    -H "Authorization: Bearer $FIRECRAWL_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{
      "url": "https://app.example.com/login",
      "formats": ["markdown"],
      "profile": { "name": "my-app", "saveChanges": true }
    }')

  SCRAPE_ID=$(echo $RESPONSE | jq -r '.data.metadata.scrapeId')

  # Log in via interact
  curl -s -X POST "https://api.firecrawl.dev/v2/scrape/$SCRAPE_ID/interact" \
    -H "Authorization: Bearer $FIRECRAWL_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{"prompt": "Fill in user@example.com and password, then click Login"}'

  # Stop — state is saved to the profile
  curl -s -X DELETE "https://api.firecrawl.dev/v2/scrape/$SCRAPE_ID/interact" \
    -H "Authorization: Bearer $FIRECRAWL_API_KEY"

  # Session 2: Scrape again with the same profile — already logged in
  RESPONSE=$(curl -s -X POST "https://api.firecrawl.dev/v2/scrape" \
    -H "Authorization: Bearer $FIRECRAWL_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{
      "url": "https://app.example.com/dashboard",
      "formats": ["markdown"],
      "profile": { "name": "my-app", "saveChanges": true }
    }')

  SCRAPE_ID=$(echo $RESPONSE | jq -r '.data.metadata.scrapeId')

  curl -s -X POST "https://api.firecrawl.dev/v2/scrape/$SCRAPE_ID/interact" \
    -H "Authorization: Bearer $FIRECRAWL_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{"prompt": "Extract the dashboard data"}'
  ```

  ```bash CLI theme={null}
  # Session 1: Scrape with a profile, log in, then stop (state is saved)
  firecrawl scrape https://app.example.com/login --profile my-app
  firecrawl interact "Fill in user@example.com and password, then click Login"
  firecrawl interact stop

  # Session 2: Scrape with the same profile — already logged in
  firecrawl scrape https://app.example.com/dashboard --profile my-app
  firecrawl interact "Extract the dashboard data"
  firecrawl interact stop

  # Read-only: load profile state without saving changes back
  firecrawl scrape https://app.example.com/dashboard --profile my-app --no-save-changes
  ```
</CodeGroup>

| Parameter     | Default | Description                                                                                                                                                                                       |
| ------------- | ------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `name`        | —       | A name for the persistent profile. Scrapes with the same name share browser state.                                                                                                                |
| `saveChanges` | `true`  | When `true`, browser state is saved back to the profile when the interact session stops. Set to `false` to load existing data without writing — useful when you need multiple concurrent readers. |

<Note>
  Only one session can save to a profile at a time. If another session is already saving, you'll get a `409` error. You can still open the same profile with `saveChanges: false`, or try again later.
</Note>

The browser state is saved when the interact session is stopped. Always stop the session when you're done so the profile can be reused.

## When to Use What

| Use Case                         | Recommended                | Why                             |
| -------------------------------- | -------------------------- | ------------------------------- |
| Web search                       | [Search](/features/search) | Dedicated search endpoint       |
| Get clean content from a URL     | [Scrape](/features/scrape) | One API call, no session needed |
| Click, type, navigate on a page  | **Interact** (prompt)      | Just describe it in English     |
| Extract data behind interactions | **Interact** (prompt)      | No selectors needed             |
| Complex scraping logic           | **Interact** (code)        | Full Playwright control         |

<Info>
  **Interact vs Browser Sandbox**: Interact is built on the same infrastructure as [Browser Sandbox](/features/browser) but provides a better interface for the most common pattern — scrape a page, then go deeper. Browser Sandbox is better when you need a standalone browser session that isn't tied to a specific scrape.
</Info>

## Pricing

* **Code-only** (no `prompt`) — 2 credits per session minute
* **With AI prompts** — 7 credits per session minute
* **Scrape** — billed separately (1 credit per scrape, plus any format-specific costs)

## API Reference

* [Execute Interact](/api-reference/endpoint/scrape-execute) — `POST /v2/scrape/{scrapeId}/interact`
* [Stop Interact](/api-reference/endpoint/scrape-browser-delete) — `DELETE /v2/scrape/{scrapeId}/interact`

### Request Body (POST)

| Field      | Type     | Default  | Description                                                                                          |
| ---------- | -------- | -------- | ---------------------------------------------------------------------------------------------------- |
| `prompt`   | `string` | —        | Natural language task for the AI agent. Required if `code` is not set. Max 10,000 characters.        |
| `code`     | `string` | —        | Code to execute (Node.js, Python, or Bash). Required if `prompt` is not set. Max 100,000 characters. |
| `language` | `string` | `"node"` | `"node"`, `"python"`, or `"bash"`. Only used with `code`.                                            |
| `timeout`  | `number` | `30`     | Timeout in seconds (1–300).                                                                          |
| `origin`   | `string` | —        | Caller identifier for activity tracking.                                                             |

### Response

| Field                    | Description                                                                                                                                           |
| ------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------- |
| `success`                | `true` if the execution completed without errors                                                                                                      |
| `liveViewUrl`            | Read-only live view URL for the browser session                                                                                                       |
| `interactiveLiveViewUrl` | Interactive live view URL (viewers can control the browser)                                                                                           |
| `output`                 | The agent's natural language answer to your prompt. Only present when using `prompt`.                                                                 |
| `stdout`                 | Standard output from the code execution                                                                                                               |
| `result`                 | Raw return value from the sandbox. For `code`: the last expression evaluated. For `prompt`: the raw page snapshot the agent used to produce `output`. |
| `stderr`                 | Standard error output                                                                                                                                 |
| `exitCode`               | Exit code (`0` = success)                                                                                                                             |
| `killed`                 | `true` if the execution was terminated due to timeout                                                                                                 |

***

Have feedback or need help? Email [help@firecrawl.com](mailto:help@firecrawl.com) or reach out on [Discord](https://discord.gg/firecrawl).


# Lockdown Mode
Source: https://docs.firecrawl.dev/features/lockdown

Cache-only scrape mode for compliance and air-gapped environments. No outbound traffic.

Lockdown mode forces the scrape endpoint to read from Firecrawl's existing index and cache only — it never makes an outbound request to the target URL. It is designed for compliance-constrained and air-gapped environments where the scrape request itself (the URL, headers, and body) could leak sensitive information over the network.

## How it works

When `lockdown: true` is set on a `/v2/scrape` request:

* **No outbound traffic.** Firecrawl never connects to the target URL. All outbound paths (HTTP engines, robots.txt fetching, search-index writes, audio transforms, etc.) are gated off.
* **Cache-only reads.** The request is served from Firecrawl's index if a matching entry exists. The default `maxAge` is bumped to 2 years so existing cached pages are eligible regardless of age.
* **Cache miss returns an error.** If no cached data is available, Firecrawl returns a `404` with error code `SCRAPE_LOCKDOWN_CACHE_MISS`. The URL is never logged on miss.
* **Zero data retention.** Lockdown requests are treated as ZDR: no URL is persisted, no response blob is written to long-term storage, and the scrape job is cleaned up after delivery.

## When to use this

**Great for:**

* Regulated industries (healthcare, finance, legal) where outbound requests require audit or approval
* Air-gapped or compliance-constrained environments where the URL itself is sensitive
* Replaying already-indexed pages without re-hitting origins

**Skip for:**

* Fresh content that has never been scraped before — lockdown mode returns an error on cache miss
* Real-time or time-sensitive data

## Usage

Add `lockdown: true` to your scrape request.

<CodeGroup>
  ```python Python theme={null}
  from firecrawl import Firecrawl

  firecrawl = Firecrawl(api_key="fc-YOUR_API_KEY")

  # Serve only previously cached results. No outbound request is made.
  # Returns SCRAPE_LOCKDOWN_CACHE_MISS if the URL is not in the cache.
  scrape_result = firecrawl.scrape(
      'https://firecrawl.dev',
      formats=['markdown'],
      lockdown=True,
  )

  print(scrape_result.markdown)
  ```

  ```javascript JavaScript theme={null}
  import Firecrawl from '@mendable/firecrawl-js';

  const firecrawl = new Firecrawl({ apiKey: "fc-YOUR_API_KEY" });

  // Serve only previously cached results. No outbound request is made.
  // Returns SCRAPE_LOCKDOWN_CACHE_MISS if the URL is not in the cache.
  const scrapeResult = await firecrawl.scrape('https://firecrawl.dev', {
    formats: ['markdown'],
    lockdown: true,
  });

  console.log(scrapeResult.markdown);
  ```

  ```bash cURL theme={null}
  curl -X POST https://api.firecrawl.dev/v2/scrape \
    -H 'Content-Type: application/json' \
    -H 'Authorization: Bearer fc-YOUR_API_KEY' \
    -d '{
      "url": "https://firecrawl.dev",
      "formats": ["markdown"],
      "lockdown": true
    }'
  ```

  ```bash CLI theme={null}
  # Serve only previously cached results. No outbound request is made.
  firecrawl https://firecrawl.dev --lockdown
  ```
</CodeGroup>

## Cache miss response

If the URL has not been previously scraped and cached, the response is:

```json theme={null}
{
  "success": false,
  "code": "SCRAPE_LOCKDOWN_CACHE_MISS",
  "error": "No cached data is available for this request in lockdown mode. Lockdown mode only serves previously cached responses and never makes outbound requests. To resolve this, either disable lockdown mode to allow a fresh scrape, or try again after the URL has been scraped and cached."
}
```

To seed the cache, perform a normal (non-lockdown) scrape of the URL first. Subsequent lockdown requests will return the cached result.

## Billing

| Outcome                                   | Credits   |
| ----------------------------------------- | --------- |
| Cache hit                                 | 5 credits |
| Cache miss (`SCRAPE_LOCKDOWN_CACHE_MISS`) | 1 credit  |

Zero Data Retention does not incur an additional charge on lockdown requests — the ZDR cost is waived because lockdown mode is already ZDR by default.

## Cache hit matching

Lockdown uses the same cache-match rules as regular scrapes. For a cache hit, these parameters must match the cached entry: `url`, `mobile`, `location`, `waitFor`, `blockAds`, `screenshot` (enabled/disabled and full-page), and enhanced proxy mode. You can verify behavior via `metadata.cacheState` in the response — it will be `"hit"` on a served response.

## Availability

Lockdown mode is supported on the `/v2/scrape` endpoint and is exposed across all surfaces that call it:

* **SDKs** — Python, Node.js, Go, Rust, Java, .NET, Ruby, PHP, and Elixir (`lockdown: true` on the scrape options).
* **CLI** — pass `--lockdown` to `firecrawl scrape`.
* **MCP server** — include `"lockdown": true` in the `firecrawl_scrape` tool arguments.

It is not available on `crawl`, `map`, `extract`, or `search`.

> Are you an AI agent that needs a Firecrawl API key? See [firecrawl.dev/agent-onboarding/SKILL.md](https://www.firecrawl.dev/agent-onboarding/SKILL.md) for automated onboarding instructions.


# Parse
Source: https://docs.firecrawl.dev/features/parse

Upload a local or non-public document and convert it into clean, LLM-ready data

The `/parse` endpoint converts local or non-public documents into clean, LLM-ready data. Upload file bytes via `multipart/form-data` and get back Markdown, JSON, HTML, links, images, or a summary — with reading order and tables preserved.

* Turn PDF, DOCX, XLSX, HTML, and more into Markdown or structured JSON
* Up to **5x faster** parsing via a Rust-based engine
* Files up to **50 MB** per request
* Zero Data Retention support

## When to use `/parse`

Use `/parse` when the source document is **a local file** or **not publicly accessible by URL**. If you have a public URL that points to a document, prefer [`/scrape`](/features/scrape) — it auto-detects the file type from the extension or content type and parses it the same way.

| Source                                                           | Endpoint                                         |
| ---------------------------------------------------------------- | ------------------------------------------------ |
| Public URL to a document (e.g. `https://example.com/report.pdf`) | [`POST /scrape`](/api-reference/endpoint/scrape) |
| Local file or non-public bytes (PDF, DOCX, XLSX, HTML, …)        | [`POST /parse`](/api-reference/endpoint/parse)   |

## Parsing

### /parse endpoint

Used to upload a file and receive parsed content. The request is `multipart/form-data` with a required `file` part and an optional `options` JSON part.

**Supported extensions:** `.html`, `.htm`, `.pdf`, `.docx`, `.doc`, `.odt`, `.rtf`, `.xlsx`, `.xls`.

### Usage

<CodeGroup>
  ```python Python theme={null}
  from firecrawl import Firecrawl

  firecrawl = Firecrawl(api_key="fc-YOUR-API-KEY")

  doc = firecrawl.parse("./report.pdf")

  print(doc.markdown)
  ```

  ```javascript Node theme={null}
  import Firecrawl from "@mendable/firecrawl-js";
  import fs from "node:fs";

  const firecrawl = new Firecrawl({ apiKey: "fc-YOUR-API-KEY" });

  const doc = await firecrawl.parse({
    data: fs.readFileSync("./report.pdf"),
    filename: "report.pdf",
  });

  console.log(doc.markdown);
  ```

  ```bash cURL theme={null}
  curl -X POST https://api.firecrawl.dev/v2/parse \
    -H 'Authorization: Bearer YOUR_API_KEY' \
    -F 'file=@./report.pdf' \
    -F 'options={"formats":["markdown"]};type=application/json'
  ```
</CodeGroup>

### Response

SDKs return the document object directly. cURL returns the JSON payload.

```json theme={null}
{
  "success": true,
  "data": {
    "markdown": "# Annual Report\n\n...",
    "metadata": {
      "title": "Annual Report",
      "numPages": 42,
      "sourceFile": "report.pdf"
    }
  }
}
```

## Options

`/parse` accepts a subset of scrape options under the `options` field. Common settings:

* `formats`: Array of output formats. Defaults to `["markdown"]`. Supported: `markdown`, `html`, `rawHtml`, `links`, `images`, `summary`, and `json` (with a schema or prompt).
* `onlyMainContent`: Only return the main content of the document. Defaults to `true`.
* `includeTags` / `excludeTags`: Tag-level inclusion or exclusion (HTML inputs).
* `timeout`: Request timeout in milliseconds. Defaults to `30000`, max `300000`.
* `parsers`: File-parser controls. For PDFs, set `{ "type": "pdf", "mode": "fast" | "auto" | "ocr", "maxPages": <int> }`.

<Note>
  `/parse` does not support browser-only options like `actions`, `waitFor`, `location`, `mobile`, or change tracking.
</Note>

### PDF parser modes

```bash cURL theme={null}
curl -X POST https://api.firecrawl.dev/v2/parse \
  -H 'Authorization: Bearer YOUR_API_KEY' \
  -F 'file=@./scan.pdf' \
  -F 'options={"parsers":[{"type":"pdf","mode":"ocr","maxPages":50}]};type=application/json'
```

* `fast`: text-only extraction, fastest path.
* `auto` (default): text-first with OCR fallback for image-only pages.
* `ocr`: OCR every page — use for scanned documents.

### Structured JSON output

Pass a JSON schema or prompt to extract structured data directly from the document:

```bash cURL theme={null}
curl -X POST https://api.firecrawl.dev/v2/parse \
  -H 'Authorization: Bearer YOUR_API_KEY' \
  -F 'file=@./invoice.pdf' \
  -F 'options={"formats":[{"type":"json","schema":{"type":"object","properties":{"total":{"type":"number"},"vendor":{"type":"string"}}}}]};type=application/json'
```

## Considerations

* Maximum file size is **50 MB** per request.
* Parsing very large or scanned PDFs in `ocr` mode may take longer — increase `timeout` or use `maxPages` to bound the work.
* For batches of files, call `/parse` per file in parallel; there is no batch upload variant.

> Are you an AI agent that needs a Firecrawl API key? See [firecrawl.dev/agent-onboarding/SKILL.md](https://www.firecrawl.dev/agent-onboarding/SKILL.md) for automated onboarding instructions.


# ASP.NET Core
Source: https://docs.firecrawl.dev/quickstarts/aspnet-core

Use Firecrawl with ASP.NET Core to search, scrape, and interact with web data using the REST API.

## Prerequisites

* .NET 6.0+
* A Firecrawl API key — [get one free](https://www.firecrawl.dev/app/api-keys)

## Configuration

Add your API key to `appsettings.json`:

```json theme={null}
{
  "Firecrawl": {
    "ApiKey": "fc-YOUR-API-KEY",
    "BaseUrl": "https://api.firecrawl.dev/v2"
  }
}
```

Or use environment variables / user secrets:

```bash theme={null}
export Firecrawl__ApiKey=fc-YOUR-API-KEY
```

## Create a service

Create `Services/FirecrawlService.cs`:

```csharp theme={null}
using System.Net.Http.Headers;
using System.Text;
using System.Text.Json;

public class FirecrawlService
{
    private readonly HttpClient _http;
    private readonly string _baseUrl;

    public FirecrawlService(IConfiguration config)
    {
        _baseUrl = config["Firecrawl:BaseUrl"] ?? "https://api.firecrawl.dev/v2";
        _http = new HttpClient();
        _http.DefaultRequestHeaders.Authorization =
            new AuthenticationHeaderValue("Bearer", config["Firecrawl:ApiKey"]);
    }

    public async Task<JsonDocument> SearchAsync(string query, int limit = 5)
    {
        var content = new StringContent(
            JsonSerializer.Serialize(new { query, limit }),
            Encoding.UTF8, "application/json");

        var response = await _http.PostAsync($"{_baseUrl}/search", content);
        response.EnsureSuccessStatusCode();
        return JsonDocument.Parse(await response.Content.ReadAsStringAsync());
    }

    public async Task<JsonDocument> ScrapeAsync(string url)
    {
        var content = new StringContent(
            JsonSerializer.Serialize(new { url }),
            Encoding.UTF8, "application/json");

        var response = await _http.PostAsync($"{_baseUrl}/scrape", content);
        response.EnsureSuccessStatusCode();
        return JsonDocument.Parse(await response.Content.ReadAsStringAsync());
    }

    public async Task<JsonDocument> InteractAsync(string url, string prompt, string? followUp = null)
    {
        // 1. Scrape to open a browser session
        var scrapeContent = new StringContent(
            JsonSerializer.Serialize(new { url, formats = new[] { "markdown" } }),
            Encoding.UTF8, "application/json");

        var scrapeRes = await _http.PostAsync($"{_baseUrl}/scrape", scrapeContent);
        scrapeRes.EnsureSuccessStatusCode();
        var scrapeDoc = JsonDocument.Parse(await scrapeRes.Content.ReadAsStringAsync());
        var scrapeId = scrapeDoc.RootElement
            .GetProperty("data").GetProperty("metadata").GetProperty("scrapeId").GetString();

        // 2. Send first prompt
        var firstPrompt = new StringContent(
            JsonSerializer.Serialize(new { prompt }),
            Encoding.UTF8, "application/json");
        await _http.PostAsync($"{_baseUrl}/scrape/{scrapeId}/interact", firstPrompt);

        // 3. Send follow-up prompt
        JsonDocument? result = null;
        if (followUp != null)
        {
            var followUpContent = new StringContent(
                JsonSerializer.Serialize(new { prompt = followUp }),
                Encoding.UTF8, "application/json");
            var followUpRes = await _http.PostAsync(
                $"{_baseUrl}/scrape/{scrapeId}/interact", followUpContent);
            followUpRes.EnsureSuccessStatusCode();
            result = JsonDocument.Parse(await followUpRes.Content.ReadAsStringAsync());
        }

        // 4. Close the session
        await _http.DeleteAsync($"{_baseUrl}/scrape/{scrapeId}/interact");

        return result ?? scrapeDoc;
    }
}
```

## Register and use

In `Program.cs`:

```csharp theme={null}
var builder = WebApplication.CreateBuilder(args);
builder.Services.AddSingleton<FirecrawlService>();

var app = builder.Build();

app.MapPost("/api/search", async (FirecrawlService firecrawl, SearchRequest req) =>
{
    var result = await firecrawl.SearchAsync(req.Query, req.Limit);
    return Results.Ok(result.RootElement);
});

app.MapPost("/api/scrape", async (FirecrawlService firecrawl, ScrapeRequest req) =>
{
    var result = await firecrawl.ScrapeAsync(req.Url);
    return Results.Ok(result.RootElement);
});

app.MapPost("/api/interact", async (FirecrawlService firecrawl, InteractRequest req) =>
{
    var result = await firecrawl.InteractAsync(req.Url, req.Prompt, req.FollowUp);
    return Results.Ok(result.RootElement);
});

app.Run();

record SearchRequest(string Query, int Limit = 5);
record ScrapeRequest(string Url);
record InteractRequest(string Url, string Prompt, string? FollowUp = null);
```

## Run it

```bash theme={null}
dotnet run
```

## Test it

```bash theme={null}
# Search the web
curl -X POST http://localhost:5000/api/search \
  -H "Content-Type: application/json" \
  -d '{"query": "firecrawl web scraping"}'

# Scrape a page
curl -X POST http://localhost:5000/api/scrape \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com"}'

# Interact with a page
curl -X POST http://localhost:5000/api/interact \
  -H "Content-Type: application/json" \
  -d '{"url": "https://www.amazon.com", "prompt": "Search for iPhone 16 Pro Max", "followUp": "Click on the first result and tell me the price"}'
```

## Next steps

<CardGroup>
  <Card title="Search docs" icon="magnifying-glass" href="/features/search">
    Search the web and get full page content
  </Card>

  <Card title="Scrape docs" icon="file-lines" href="/features/scrape">
    All scrape options including formats, actions, and proxies
  </Card>

  <Card title="Interact docs" icon="hand-pointer" href="/features/interact">
    Click, fill forms, and extract dynamic content
  </Card>

  <Card title="API Reference" icon="code" href="/api-reference/v2-introduction">
    Complete REST API documentation
  </Card>
</CardGroup>


# Astro
Source: https://docs.firecrawl.dev/quickstarts/astro

Use Firecrawl with Astro to scrape, search, and interact with web data in your content-driven site.

## Prerequisites

* Astro project with SSR enabled (`output: "server"` or `"hybrid"`)
* A Firecrawl API key — [get one free](https://www.firecrawl.dev/app/api-keys)

## Install the SDK

```bash theme={null}
npm install @mendable/firecrawl-js
```

Add your API key to `.env`:

```bash theme={null}
FIRECRAWL_API_KEY=fc-YOUR-API-KEY
```

## Search the web

Create `src/pages/api/search.ts`:

```typescript theme={null}
import type { APIRoute } from "astro";
import Firecrawl from "@mendable/firecrawl-js";

const firecrawl = new Firecrawl({
  apiKey: import.meta.env.FIRECRAWL_API_KEY,
});

export const POST: APIRoute = async ({ request }) => {
  const { query } = await request.json();
  const results = await firecrawl.search(query, { limit: 5 });
  return new Response(JSON.stringify(results), {
    headers: { "Content-Type": "application/json" },
  });
};
```

Or search at request time in a server-rendered page (`src/pages/search.astro`):

```astro theme={null}
---
import Firecrawl from "@mendable/firecrawl-js";

const firecrawl = new Firecrawl({
  apiKey: import.meta.env.FIRECRAWL_API_KEY,
});

const query = Astro.url.searchParams.get("q");
let results = [];

if (query) {
  const searchData = await firecrawl.search(query, { limit: 5 });
  results = searchData.web || [];
}
---

<html>
  <body>
    <h1>Search Results</h1>
    {results.length > 0 ? (
      <ul>
        {results.map((r) => (
          <li><a href={r.url}>{r.title}</a></li>
        ))}
      </ul>
    ) : (
      <p>Pass ?q= to search the web</p>
    )}
  </body>
</html>
```

## Scrape a page

Create `src/pages/api/scrape.ts`:

```typescript theme={null}
import type { APIRoute } from "astro";
import Firecrawl from "@mendable/firecrawl-js";

const firecrawl = new Firecrawl({
  apiKey: import.meta.env.FIRECRAWL_API_KEY,
});

export const POST: APIRoute = async ({ request }) => {
  const { url } = await request.json();
  const result = await firecrawl.scrape(url);
  return new Response(JSON.stringify(result), {
    headers: { "Content-Type": "application/json" },
  });
};
```

Or scrape at request time in a server-rendered page (`src/pages/scrape.astro`):

```astro theme={null}
---
import Firecrawl from "@mendable/firecrawl-js";

const firecrawl = new Firecrawl({
  apiKey: import.meta.env.FIRECRAWL_API_KEY,
});

const target = Astro.url.searchParams.get("url");
let markdown = null;

if (target) {
  const result = await firecrawl.scrape(target);
  markdown = result.markdown;
}
---

<html>
  <body>
    <h1>Scraped Content</h1>
    {markdown ? <pre>{markdown}</pre> : <p>Pass ?url= to scrape a page</p>}
  </body>
</html>
```

## Interact with a page

Create `src/pages/api/interact.ts`:

```typescript theme={null}
import type { APIRoute } from "astro";
import Firecrawl from "@mendable/firecrawl-js";

const firecrawl = new Firecrawl({
  apiKey: import.meta.env.FIRECRAWL_API_KEY,
});

export const POST: APIRoute = async () => {
  const result = await firecrawl.scrape("https://www.amazon.com", {
    formats: ["markdown"],
  });

  const scrapeId = result.metadata?.scrapeId;

  await firecrawl.interact(scrapeId, {
    prompt: "Search for iPhone 16 Pro Max",
  });

  const response = await firecrawl.interact(scrapeId, {
    prompt: "Click on the first result and tell me the price",
  });

  await firecrawl.stopInteraction(scrapeId);

  return new Response(JSON.stringify({ output: response.output }), {
    headers: { "Content-Type": "application/json" },
  });
};
```

## Next steps

<CardGroup>
  <Card title="Scrape docs" icon="file-lines" href="/features/scrape">
    All scrape options including formats, actions, and proxies
  </Card>

  <Card title="Search docs" icon="magnifying-glass" href="/features/search">
    Search the web and get full page content
  </Card>

  <Card title="Interact docs" icon="hand-pointer" href="/features/interact">
    Click, fill forms, and extract dynamic content
  </Card>

  <Card title="Node SDK reference" icon="node" href="/sdks/node">
    Full SDK reference with crawl, map, batch scrape, and more
  </Card>
</CardGroup>


# AWS Lambda
Source: https://docs.firecrawl.dev/quickstarts/aws-lambda

Use Firecrawl with AWS Lambda to search, scrape, and interact with web data in serverless functions.

## Prerequisites

* AWS account with Lambda access
* A Firecrawl API key — [get one free](https://www.firecrawl.dev/app/api-keys)

## Setup

```bash theme={null}
mkdir firecrawl-lambda && cd firecrawl-lambda
npm init -y
npm install @mendable/firecrawl-js
```

## Search the web

Create `index.mjs` with a search handler:

```javascript theme={null}
import Firecrawl from "@mendable/firecrawl-js";

const firecrawl = new Firecrawl({ apiKey: process.env.FIRECRAWL_API_KEY });

export async function handler(event) {
  const body = JSON.parse(event.body || "{}");

  if (body.action === "search") {
    const results = await firecrawl.search(body.query, { limit: 5 });
    return {
      statusCode: 200,
      body: JSON.stringify(results),
    };
  }

  return { statusCode: 400, body: JSON.stringify({ error: "Unknown action" }) };
}
```

## Scrape a page

Add a `scrape` action to the same handler:

```javascript theme={null}
if (body.action === "scrape") {
  const result = await firecrawl.scrape(body.url);
  return {
    statusCode: 200,
    body: JSON.stringify(result),
  };
}
```

## Interact with a page

Add an `interact` action to control a live browser session — click buttons, fill forms, and extract dynamic content:

```javascript theme={null}
if (body.action === "interact") {
  const result = await firecrawl.scrape("https://www.amazon.com", {
    formats: ["markdown"],
  });
  const scrapeId = result.metadata?.scrapeId;

  await firecrawl.interact(scrapeId, {
    prompt: "Search for iPhone 16 Pro Max",
  });
  const response = await firecrawl.interact(scrapeId, {
    prompt: "Click on the first result and tell me the price",
  });

  await firecrawl.stopInteraction(scrapeId);
  return {
    statusCode: 200,
    body: JSON.stringify({ output: response.output }),
  };
}
```

## Deploy

Package and deploy with the AWS CLI:

```bash theme={null}
zip -r function.zip index.mjs node_modules/

aws lambda create-function \
  --function-name firecrawl-scraper \
  --runtime nodejs20.x \
  --handler index.handler \
  --zip-file fileb://function.zip \
  --role arn:aws:iam::YOUR_ACCOUNT:role/lambda-role \
  --environment Variables="{FIRECRAWL_API_KEY=fc-YOUR-API-KEY}" \
  --timeout 60
```

<Note>
  Set the Lambda timeout to at least 30 seconds. Scraping dynamic pages and interact sessions can take longer than the default 3-second timeout.
</Note>

## Next steps

<CardGroup>
  <Card title="Search docs" icon="magnifying-glass" href="/features/search">
    Search the web and get full page content
  </Card>

  <Card title="Scrape docs" icon="file-lines" href="/features/scrape">
    All scrape options including formats, actions, and proxies
  </Card>

  <Card title="Interact docs" icon="hand-pointer" href="/features/interact">
    Click, fill forms, and extract dynamic content
  </Card>

  <Card title="Node SDK reference" icon="node" href="/sdks/node">
    Full SDK reference with crawl, map, batch scrape, and more
  </Card>
</CardGroup>


# Bun
Source: https://docs.firecrawl.dev/quickstarts/bun

Use Firecrawl with Bun to build fast web scraping and search servers.

## Prerequisites

* Bun 1.0+
* A Firecrawl API key — [get one free](https://www.firecrawl.dev/app/api-keys)

## Install the SDK

```bash theme={null}
bun add @mendable/firecrawl-js
```

Add your API key to `.env`:

```bash theme={null}
FIRECRAWL_API_KEY=fc-YOUR-API-KEY
```

## Search the web

Bun has a built-in HTTP server. Create `index.ts`:

```typescript theme={null}
import Firecrawl from "@mendable/firecrawl-js";

const firecrawl = new Firecrawl({ apiKey: process.env.FIRECRAWL_API_KEY });

Bun.serve({
  port: 3000,
  async fetch(req) {
    const url = new URL(req.url);

    if (req.method === "POST" && url.pathname === "/search") {
      const { query } = await req.json();
      const results = await firecrawl.search(query, { limit: 5 });
      return Response.json(results);
    }

    return new Response("Not found", { status: 404 });
  },
});

console.log("Server running on port 3000");
```

Run it:

```bash theme={null}
bun run index.ts
```

## Scrape a page

Add a `/scrape` route to the same server:

```typescript theme={null}
if (req.method === "POST" && url.pathname === "/scrape") {
  const { url: targetUrl } = await req.json();
  const result = await firecrawl.scrape(targetUrl);
  return Response.json(result);
}
```

## Interact with a page

Use interact to control a live browser session — click buttons, fill forms, and extract dynamic content.

```typescript theme={null}
if (req.method === "POST" && url.pathname === "/interact") {
  const { url: targetUrl } = await req.json();

  const result = await firecrawl.scrape(targetUrl, { formats: ['markdown'] });
  const scrapeId = result.metadata?.scrapeId;

  await firecrawl.interact(scrapeId, { prompt: 'Search for iPhone 16 Pro Max' });
  const response = await firecrawl.interact(scrapeId, { prompt: 'Click on the first result and tell me the price' });

  await firecrawl.stopInteraction(scrapeId);

  return Response.json({ output: response.output });
}
```

## Script usage

Use Firecrawl in a standalone Bun script:

```typescript theme={null}
import Firecrawl from "@mendable/firecrawl-js";

const app = new Firecrawl({ apiKey: process.env.FIRECRAWL_API_KEY });
const results = await app.search("firecrawl web scraping", { limit: 5 });
console.log(results);
```

```bash theme={null}
bun run search.ts
```

## Next steps

<CardGroup>
  <Card title="Scrape docs" icon="file-lines" href="/features/scrape">
    All scrape options including formats, actions, and proxies
  </Card>

  <Card title="Search docs" icon="magnifying-glass" href="/features/search">
    Search the web and get full page content
  </Card>

  <Card title="Interact docs" icon="hand-pointer" href="/features/interact">
    Click, fill forms, and extract dynamic content
  </Card>

  <Card title="Node SDK reference" icon="node" href="/sdks/node">
    Full SDK reference with crawl, map, batch scrape, and more
  </Card>
</CardGroup>


# Cloudflare Workers
Source: https://docs.firecrawl.dev/quickstarts/cloudflare-workers

Use Firecrawl with Cloudflare Workers to search, scrape, and interact with web data at the edge.

## Prerequisites

* Wrangler CLI (`npm install -g wrangler`)
* A Firecrawl API key — [get one free](https://www.firecrawl.dev/app/api-keys)

## Setup

```bash theme={null}
npm create cloudflare@latest my-scraper
cd my-scraper
npm install @mendable/firecrawl-js
```

Add your API key as a secret:

```bash theme={null}
wrangler secret put FIRECRAWL_API_KEY
```

## Search the web

Create a handler that searches the web and returns results with full page content.

Edit `src/index.ts`:

```typescript theme={null}
import Firecrawl from "@mendable/firecrawl-js";

export interface Env {
  FIRECRAWL_API_KEY: string;
}

export default {
  async fetch(request: Request, env: Env): Promise<Response> {
    const firecrawl = new Firecrawl({ apiKey: env.FIRECRAWL_API_KEY });
    const url = new URL(request.url);

    if (request.method === "POST" && url.pathname === "/search") {
      const { query } = (await request.json()) as { query: string };
      const results = await firecrawl.search(query, { limit: 5 });
      return Response.json(results);
    }

    return new Response("Not found", { status: 404 });
  },
};
```

## Scrape a page

Add a `/scrape` route to extract clean markdown from any URL.

```typescript theme={null}
if (request.method === "POST" && url.pathname === "/scrape") {
  const { url: targetUrl } = (await request.json()) as { url: string };
  const result = await firecrawl.scrape(targetUrl);
  return Response.json(result);
}
```

## Interact with a page

Add an `/interact` route to control a live browser session — click buttons, fill forms, and extract dynamic content.

```typescript theme={null}
if (request.method === "POST" && url.pathname === "/interact") {
  const result = await firecrawl.scrape("https://www.amazon.com", {
    formats: ["markdown"],
  });
  const scrapeId = result.metadata?.scrapeId;

  await firecrawl.interact(scrapeId, {
    prompt: "Search for iPhone 16 Pro Max",
  });
  const response = await firecrawl.interact(scrapeId, {
    prompt: "Click on the first result and tell me the price",
  });

  await firecrawl.stopInteraction(scrapeId);
  return Response.json({ output: response.output });
}
```

## Deploy

```bash theme={null}
wrangler deploy
```

## Test it

```bash theme={null}
curl -X POST https://my-scraper.<your-subdomain>.workers.dev/search \
  -H "Content-Type: application/json" \
  -d '{"query": "firecrawl web scraping"}'
```

## Next steps

<CardGroup>
  <Card title="Search docs" icon="magnifying-glass" href="/features/search">
    Search the web and get full page content
  </Card>

  <Card title="Scrape docs" icon="file-lines" href="/features/scrape">
    All scrape options including formats, actions, and proxies
  </Card>

  <Card title="Interact docs" icon="hand-pointer" href="/features/interact">
    Click, fill forms, and extract dynamic content
  </Card>

  <Card title="Node SDK reference" icon="node" href="/sdks/node">
    Full SDK reference with crawl, map, batch scrape, and more
  </Card>
</CardGroup>


# Deno Deploy
Source: https://docs.firecrawl.dev/quickstarts/deno-deploy

Use Firecrawl with Deno Deploy to search, scrape, and interact with web data at the edge.

## Prerequisites

* Deno 1.40+ or Deno 2
* A Firecrawl API key — [get one free](https://www.firecrawl.dev/app/api-keys)

## Setup

Create `main.ts`:

```typescript theme={null}
import Firecrawl from "npm:@mendable/firecrawl-js";

const firecrawl = new Firecrawl({
  apiKey: Deno.env.get("FIRECRAWL_API_KEY"),
});
```

## Search the web

Add a `/search` route that searches the web and returns results with full page content.

```typescript theme={null}
Deno.serve(async (req) => {
  const url = new URL(req.url);

  if (req.method === "POST" && url.pathname === "/search") {
    const { query } = await req.json();
    const results = await firecrawl.search(query, { limit: 5 });
    return Response.json(results);
  }

  return new Response("Not found", { status: 404 });
});
```

## Scrape a page

Add a `/scrape` route to extract clean markdown from any URL.

```typescript theme={null}
if (req.method === "POST" && url.pathname === "/scrape") {
  const { url: targetUrl } = await req.json();
  const result = await firecrawl.scrape(targetUrl);
  return Response.json(result);
}
```

## Interact with a page

Add an `/interact` route to control a live browser session — click buttons, fill forms, and extract dynamic content.

```typescript theme={null}
if (req.method === "POST" && url.pathname === "/interact") {
  const result = await firecrawl.scrape("https://www.amazon.com", {
    formats: ["markdown"],
  });
  const scrapeId = result.metadata?.scrapeId;

  await firecrawl.interact(scrapeId, {
    prompt: "Search for iPhone 16 Pro Max",
  });
  const response = await firecrawl.interact(scrapeId, {
    prompt: "Click on the first result and tell me the price",
  });
  console.log(response.output);

  await firecrawl.stopInteraction(scrapeId);
  return Response.json({ output: response.output });
}
```

## Run locally

```bash theme={null}
FIRECRAWL_API_KEY=fc-YOUR-API-KEY deno run --allow-net --allow-env main.ts
```

## Deploy

Install the Deno Deploy CLI (`deployctl`) and deploy:

```bash theme={null}
deployctl deploy --project=my-scraper main.ts
```

Set the environment variable in the Deno Deploy dashboard or via CLI:

```bash theme={null}
deployctl env set FIRECRAWL_API_KEY=fc-YOUR-API-KEY --project=my-scraper
```

## Test it

```bash theme={null}
curl -X POST https://my-scraper.deno.dev/search \
  -H "Content-Type: application/json" \
  -d '{"query": "firecrawl web scraping"}'
```

## Next steps

<CardGroup>
  <Card title="Search docs" icon="magnifying-glass" href="/features/search">
    Search the web and get full page content
  </Card>

  <Card title="Scrape docs" icon="file-lines" href="/features/scrape">
    All scrape options including formats, actions, and proxies
  </Card>

  <Card title="Interact docs" icon="hand-pointer" href="/features/interact">
    Click, fill forms, and extract dynamic content
  </Card>

  <Card title="Node SDK reference" icon="node" href="/sdks/node">
    Full SDK reference with crawl, map, batch scrape, and more
  </Card>
</CardGroup>


# Django
Source: https://docs.firecrawl.dev/quickstarts/django

Use Firecrawl with Django to scrape, search, and interact with web data in your Python web application.

## Prerequisites

* Django 4+ project
* A Firecrawl API key — [get one free](https://www.firecrawl.dev/app/api-keys)

## Install the SDK

```bash theme={null}
pip install firecrawl-py
```

Add your API key to your Django settings or environment:

```bash theme={null}
export FIRECRAWL_API_KEY=fc-YOUR-API-KEY
```

## Create views

Add search, scrape, and interact views to your Django app. In `views.py`:

```python theme={null}
import json
import os
from django.http import JsonResponse
from django.views.decorators.csrf import csrf_exempt
from django.views.decorators.http import require_POST
from firecrawl import Firecrawl

firecrawl = Firecrawl(api_key=os.environ["FIRECRAWL_API_KEY"])


@csrf_exempt
@require_POST
def search_view(request):
    body = json.loads(request.body)
    results = firecrawl.search(body["query"], limit=body.get("limit", 5))
    return JsonResponse(
        [{"title": r.title, "url": r.url} for r in results.web],
        safe=False,
    )


@csrf_exempt
@require_POST
def scrape_view(request):
    body = json.loads(request.body)
    result = firecrawl.scrape(body["url"])
    return JsonResponse({
        "markdown": result.markdown,
        "metadata": result.metadata,
    })


@csrf_exempt
@require_POST
def interact_start_view(request):
    body = json.loads(request.body)
    result = firecrawl.scrape(body["url"], formats=["markdown"])
    return JsonResponse({"scrape_id": result.metadata.scrape_id})


@csrf_exempt
@require_POST
def interact_view(request):
    body = json.loads(request.body)
    response = firecrawl.interact(body["scrape_id"], prompt=body["prompt"])
    return JsonResponse({"output": response.output})


@csrf_exempt
@require_POST
def interact_stop_view(request):
    body = json.loads(request.body)
    firecrawl.stop_interaction(body["scrape_id"])
    return JsonResponse({"status": "stopped"})
```

## Wire up URLs

In `urls.py`:

```python theme={null}
from django.urls import path
from . import views

urlpatterns = [
    path("api/search/", views.search_view),
    path("api/scrape/", views.scrape_view),
    path("api/interact/start/", views.interact_start_view),
    path("api/interact/", views.interact_view),
    path("api/interact/stop/", views.interact_stop_view),
]
```

## Test it

```bash theme={null}
python manage.py runserver

# Search the web
curl -X POST http://localhost:8000/api/search/ \
  -H "Content-Type: application/json" \
  -d '{"query": "firecrawl web scraping", "limit": 5}'

# Scrape a page
curl -X POST http://localhost:8000/api/scrape/ \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com"}'

# Start an interactive session
curl -X POST http://localhost:8000/api/interact/start/ \
  -H "Content-Type: application/json" \
  -d '{"url": "https://www.amazon.com"}'
```

## Management command

Use Firecrawl in a Django management command for scripts and data pipelines. Create `management/commands/scrape.py`:

```python theme={null}
import os
from django.core.management.base import BaseCommand
from firecrawl import Firecrawl


class Command(BaseCommand):
    help = "Scrape a URL and print the markdown"

    def add_arguments(self, parser):
        parser.add_argument("url", type=str)

    def handle(self, *args, **options):
        firecrawl = Firecrawl(api_key=os.environ["FIRECRAWL_API_KEY"])
        result = firecrawl.scrape(options["url"])
        self.stdout.write(result.markdown)
```

```bash theme={null}
python manage.py scrape https://example.com
```

## Next steps

<CardGroup>
  <Card title="Scrape docs" icon="file-lines" href="/features/scrape">
    All scrape options including formats, actions, and proxies
  </Card>

  <Card title="Search docs" icon="magnifying-glass" href="/features/search">
    Search the web and get full page content
  </Card>

  <Card title="Interact docs" icon="hand-pointer" href="/features/interact">
    Click, fill forms, and extract dynamic content
  </Card>

  <Card title="Python SDK reference" icon="python" href="/sdks/python">
    Full SDK reference with crawl, map, async, and more
  </Card>
</CardGroup>


# .NET
Source: https://docs.firecrawl.dev/quickstarts/dotnet

Get started with Firecrawl in .NET. Scrape, search, and interact with web data using the REST API.

## Prerequisites

* .NET 6.0+
* A Firecrawl API key — [get one free](https://www.firecrawl.dev/app/api-keys)

## Search the web

Firecrawl works with .NET through the REST API using `HttpClient`.

```csharp theme={null}
using System.Net.Http.Headers;
using System.Text;
using System.Text.Json;

var apiKey = Environment.GetEnvironmentVariable("FIRECRAWL_API_KEY");
var client = new HttpClient();
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", apiKey);

var content = new StringContent(
    JsonSerializer.Serialize(new { query = "firecrawl web scraping", limit = 5 }),
    Encoding.UTF8,
    "application/json"
);

var response = await client.PostAsync("https://api.firecrawl.dev/v2/search", content);
var json = await response.Content.ReadAsStringAsync();
Console.WriteLine(json);
```

<Accordion title="Example response">
  ```json theme={null}
  {
    "success": true,
    "data": {
      "web": [
        {
          "url": "https://docs.firecrawl.dev",
          "title": "Firecrawl Documentation",
          "markdown": "# Firecrawl\n\nFirecrawl is a web scraping API..."
        }
      ]
    }
  }
  ```
</Accordion>

## Scrape a page

```csharp theme={null}
var scrapeContent = new StringContent(
    JsonSerializer.Serialize(new { url = "https://example.com" }),
    Encoding.UTF8,
    "application/json"
);

var scrapeResponse = await client.PostAsync("https://api.firecrawl.dev/v2/scrape", scrapeContent);
var scrapeJson = await scrapeResponse.Content.ReadAsStringAsync();

using var doc = JsonDocument.Parse(scrapeJson);
var markdown = doc.RootElement.GetProperty("data").GetProperty("markdown").GetString();
Console.WriteLine(markdown);
```

<Accordion title="Example response">
  ```json theme={null}
  {
    "success": true,
    "data": {
      "markdown": "# Example Domain\n\nThis domain is for use in illustrative examples...",
      "metadata": {
        "title": "Example Domain",
        "sourceURL": "https://example.com"
      }
    }
  }
  ```
</Accordion>

## Interact with a page

Start a browser session, interact with the page using natural-language prompts, then close the session.

### Step 1 — Scrape to start a session

```csharp theme={null}
var sessionContent = new StringContent(
    JsonSerializer.Serialize(new { url = "https://www.amazon.com", formats = new[] { "markdown" } }),
    Encoding.UTF8,
    "application/json"
);

var sessionResponse = await client.PostAsync("https://api.firecrawl.dev/v2/scrape", sessionContent);
var sessionJson = await sessionResponse.Content.ReadAsStringAsync();

using var sessionDoc = JsonDocument.Parse(sessionJson);
var scrapeId = sessionDoc.RootElement
    .GetProperty("data")
    .GetProperty("metadata")
    .GetProperty("scrapeId")
    .GetString();

Console.WriteLine($"scrapeId: {scrapeId}");
```

### Step 2 — Send interactions

```csharp theme={null}
var interactUrl = $"https://api.firecrawl.dev/v2/scrape/{scrapeId}/interact";

// Search for a product
var searchBody = new StringContent(
    JsonSerializer.Serialize(new { prompt = "Search for iPhone 16 Pro Max" }),
    Encoding.UTF8,
    "application/json"
);

var searchResult = await client.PostAsync(interactUrl, searchBody);
Console.WriteLine(await searchResult.Content.ReadAsStringAsync());

// Click on the first result
var clickBody = new StringContent(
    JsonSerializer.Serialize(new { prompt = "Click on the first result and tell me the price" }),
    Encoding.UTF8,
    "application/json"
);

var clickResult = await client.PostAsync(interactUrl, clickBody);
Console.WriteLine(await clickResult.Content.ReadAsStringAsync());
```

### Step 3 — Stop the session

```csharp theme={null}
await client.DeleteAsync(interactUrl);
Console.WriteLine("Session stopped");
```

## Reusable client class

For repeated use, wrap the API in a typed client:

```csharp theme={null}
using System.Net.Http.Headers;
using System.Text;
using System.Text.Json;

public class FirecrawlClient
{
    private readonly HttpClient _http;
    private const string BaseUrl = "https://api.firecrawl.dev/v2";

    public FirecrawlClient(string apiKey)
    {
        _http = new HttpClient();
        _http.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", apiKey);
    }

    private async Task<JsonDocument> PostAsync(string endpoint, object payload)
    {
        var content = new StringContent(
            JsonSerializer.Serialize(payload),
            Encoding.UTF8,
            "application/json"
        );

        var response = await _http.PostAsync($"{BaseUrl}{endpoint}", content);
        response.EnsureSuccessStatusCode();

        var json = await response.Content.ReadAsStringAsync();
        return JsonDocument.Parse(json);
    }

    public async Task<JsonDocument> ScrapeAsync(string url)
    {
        return await PostAsync("/scrape", new { url });
    }

    public async Task<JsonDocument> SearchAsync(string query, int limit = 5)
    {
        return await PostAsync("/search", new { query, limit });
    }
}

// Usage
var firecrawl = new FirecrawlClient(Environment.GetEnvironmentVariable("FIRECRAWL_API_KEY")!);
var result = await firecrawl.SearchAsync("firecrawl web scraping");
Console.WriteLine(result.RootElement);
```

## Next steps

<CardGroup>
  <Card title="Search docs" icon="magnifying-glass" href="/features/search">
    Search the web and get full page content
  </Card>

  <Card title="Scrape docs" icon="file-lines" href="/features/scrape">
    All scrape options including formats, actions, and proxies
  </Card>

  <Card title="Interact docs" icon="hand-pointer" href="/features/interact">
    Click, fill forms, and extract dynamic content
  </Card>

  <Card title="API Reference" icon="code" href="/api-reference/v2-introduction">
    Complete REST API documentation
  </Card>
</CardGroup>


# Elixir
Source: https://docs.firecrawl.dev/quickstarts/elixir

Get started with Firecrawl in Elixir. Search, scrape, and interact with web data using the official SDK.

## Prerequisites

* Elixir 1.14+ and OTP 25+
* A Firecrawl API key — [get one free](https://www.firecrawl.dev/app/api-keys)

## Install the SDK

Add `firecrawl` to your `mix.exs`:

```elixir theme={null}
defp deps do
  [
    {:firecrawl, "~> 1.0"}
  ]
end
```

Configure your API key in `config/config.exs`:

```elixir theme={null}
config :firecrawl, api_key: System.get_env("FIRECRAWL_API_KEY")
```

## Search the web

```elixir theme={null}
{:ok, result} = Firecrawl.search_and_scrape(query: "firecrawl web scraping", limit: 5)

for entry <- result.body["data"]["web"] do
  IO.puts("#{entry["title"]} - #{entry["url"]}")
end
```

## Scrape a page

```elixir theme={null}
{:ok, result} = Firecrawl.scrape_and_extract_from_url(url: "https://example.com")
IO.puts(result.body["data"]["markdown"])
```

<Accordion title="Example response">
  ```json theme={null}
  {
    "success": true,
    "data": {
      "markdown": "# Example Domain\n\nThis domain is for use in illustrative examples...",
      "metadata": {
        "title": "Example Domain",
        "sourceURL": "https://example.com"
      }
    }
  }
  ```
</Accordion>

## Interact with a page

Scrape a page, then keep working with it using the browser session API.

```elixir theme={null}
{:ok, scrape} = Firecrawl.scrape_and_extract_from_url(
  url: "https://www.amazon.com",
  formats: ["markdown"]
)

scrape_id = get_in(scrape.body, ["data", "metadata", "scrapeId"])

# Use the REST API for interact (prompt-based)
headers = [
  {"Authorization", "Bearer #{Application.get_env(:firecrawl, :api_key)}"},
  {"Content-Type", "application/json"}
]

{:ok, _} = Req.post(
  "https://api.firecrawl.dev/v2/scrape/#{scrape_id}/interact",
  json: %{prompt: "Search for iPhone 16 Pro Max"},
  headers: headers
)

{:ok, response} = Req.post(
  "https://api.firecrawl.dev/v2/scrape/#{scrape_id}/interact",
  json: %{prompt: "Click on the first result and tell me the price"},
  headers: headers
)

IO.inspect(response.body)

# Stop the session
Req.delete(
  "https://api.firecrawl.dev/v2/scrape/#{scrape_id}/interact",
  headers: headers
)
```

## Environment variable

Set `FIRECRAWL_API_KEY` instead of hardcoding:

```bash theme={null}
export FIRECRAWL_API_KEY=fc-YOUR-API-KEY
```

## Next steps

<CardGroup>
  <Card title="Scrape docs" icon="file-lines" href="/features/scrape">
    All scrape options including formats, actions, and proxies
  </Card>

  <Card title="Search docs" icon="magnifying-glass" href="/features/search">
    Search the web and get full page content
  </Card>

  <Card title="Interact docs" icon="hand-pointer" href="/features/interact">
    Click, fill forms, and extract dynamic content
  </Card>

  <Card title="Elixir SDK reference" icon="droplet" href="/sdks/elixir">
    Full SDK reference with crawl, map, batch scrape, and more
  </Card>
</CardGroup>


# Express
Source: https://docs.firecrawl.dev/quickstarts/express

Use Firecrawl with Express to build web scraping and search APIs.

## Prerequisites

* Node.js 18+
* A Firecrawl API key — [get one free](https://www.firecrawl.dev/app/api-keys)

## Setup

```bash theme={null}
npm install express @mendable/firecrawl-js
```

Add your API key to `.env`:

```bash theme={null}
FIRECRAWL_API_KEY=fc-YOUR-API-KEY
```

## Search the web

```javascript theme={null}
import express from "express";
import Firecrawl from "@mendable/firecrawl-js";

const app = express();
app.use(express.json());

const firecrawl = new Firecrawl({ apiKey: process.env.FIRECRAWL_API_KEY });

app.post("/search", async (req, res) => {
  try {
    const { query } = req.body;
    const results = await firecrawl.search(query, { limit: 5 });
    res.json(results);
  } catch (error) {
    res.status(500).json({ error: error.message });
  }
});

app.listen(3000, () => console.log("Server running on port 3000"));
```

## Scrape a page

```javascript theme={null}
app.post("/scrape", async (req, res) => {
  try {
    const { url } = req.body;
    const result = await firecrawl.scrape(url);
    res.json(result);
  } catch (error) {
    res.status(500).json({ error: error.message });
  }
});
```

## Interact with a page

Use interact to control a live browser session — click buttons, fill forms, and extract dynamic content.

```javascript theme={null}
app.post("/interact", async (req, res) => {
  try {
    const { url } = req.body;

    const result = await firecrawl.scrape(url, { formats: ['markdown'] });
    const scrapeId = result.metadata?.scrapeId;

    await firecrawl.interact(scrapeId, { prompt: 'Search for iPhone 16 Pro Max' });
    const response = await firecrawl.interact(scrapeId, { prompt: 'Click on the first result and tell me the price' });

    await firecrawl.stopInteraction(scrapeId);

    res.json({ output: response.output });
  } catch (error) {
    res.status(500).json({ error: error.message });
  }
});
```

## Test it

```bash theme={null}
curl -X POST http://localhost:3000/search \
  -H "Content-Type: application/json" \
  -d '{"query": "firecrawl web scraping"}'
```

## Next steps

<CardGroup>
  <Card title="Scrape docs" icon="file-lines" href="/features/scrape">
    All scrape options including formats, actions, and proxies
  </Card>

  <Card title="Search docs" icon="magnifying-glass" href="/features/search">
    Search the web and get full page content
  </Card>

  <Card title="Interact docs" icon="hand-pointer" href="/features/interact">
    Click, fill forms, and extract dynamic content
  </Card>

  <Card title="Node SDK reference" icon="node" href="/sdks/node">
    Full SDK reference with crawl, map, batch scrape, and more
  </Card>
</CardGroup>


# FastAPI
Source: https://docs.firecrawl.dev/quickstarts/fastapi

Use Firecrawl with FastAPI to build async web scraping and search APIs in Python.

## Prerequisites

* Python 3.8+
* A Firecrawl API key — [get one free](https://www.firecrawl.dev/app/api-keys)

## Setup

```bash theme={null}
pip install fastapi uvicorn firecrawl-py
```

Add your API key to `.env`:

```bash theme={null}
FIRECRAWL_API_KEY=fc-YOUR-API-KEY
```

## Create the API

Create `main.py`:

```python theme={null}
import os
from fastapi import FastAPI
from pydantic import BaseModel
from firecrawl import Firecrawl

app = FastAPI()
firecrawl = Firecrawl(api_key=os.environ["FIRECRAWL_API_KEY"])


class SearchRequest(BaseModel):
    query: str
    limit: int = 5


class ScrapeRequest(BaseModel):
    url: str


class InteractRequest(BaseModel):
    scrape_id: str
    prompt: str


@app.post("/search")
async def search(req: SearchRequest):
    results = firecrawl.search(req.query, limit=req.limit)
    return [{"title": r.title, "url": r.url} for r in results.web]


@app.post("/scrape")
async def scrape(req: ScrapeRequest):
    result = firecrawl.scrape(req.url)
    return {"markdown": result.markdown, "metadata": result.metadata}


@app.post("/interact/start")
async def interact_start(req: ScrapeRequest):
    result = firecrawl.scrape(req.url, formats=["markdown"])
    return {"scrape_id": result.metadata.scrape_id}


@app.post("/interact")
async def interact(req: InteractRequest):
    response = firecrawl.interact(req.scrape_id, prompt=req.prompt)
    return {"output": response.output}


@app.post("/interact/stop")
async def interact_stop(req: InteractRequest):
    firecrawl.stop_interaction(req.scrape_id)
    return {"status": "stopped"}
```

## Run it

```bash theme={null}
uvicorn main:app --reload
```

## Test it

```bash theme={null}
# Search the web
curl -X POST http://localhost:8000/search \
  -H "Content-Type: application/json" \
  -d '{"query": "firecrawl web scraping", "limit": 5}'

# Scrape a page
curl -X POST http://localhost:8000/scrape \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com"}'

# Start an interactive session, then send prompts
curl -X POST http://localhost:8000/interact/start \
  -H "Content-Type: application/json" \
  -d '{"url": "https://www.amazon.com"}'
```

FastAPI auto-generates interactive docs at `http://localhost:8000/docs`.

## Async variant

For better concurrency under load, use `AsyncFirecrawl`:

```python theme={null}
from firecrawl import AsyncFirecrawl

async_firecrawl = AsyncFirecrawl(api_key=os.environ["FIRECRAWL_API_KEY"])

@app.post("/scrape-async")
async def scrape_async(req: ScrapeRequest):
    result = await async_firecrawl.scrape(req.url)
    return {"markdown": result.markdown}
```

## Next steps

<CardGroup>
  <Card title="Scrape docs" icon="file-lines" href="/features/scrape">
    All scrape options including formats, actions, and proxies
  </Card>

  <Card title="Search docs" icon="magnifying-glass" href="/features/search">
    Search the web and get full page content
  </Card>

  <Card title="Interact docs" icon="hand-pointer" href="/features/interact">
    Click, fill forms, and extract dynamic content
  </Card>

  <Card title="Python SDK reference" icon="python" href="/sdks/python">
    Full SDK reference with crawl, map, async, and more
  </Card>
</CardGroup>


# Fastify
Source: https://docs.firecrawl.dev/quickstarts/fastify

Use Firecrawl with Fastify to build high-performance web scraping and search APIs.

## Prerequisites

* Node.js 18+
* A Firecrawl API key — [get one free](https://www.firecrawl.dev/app/api-keys)

## Setup

```bash theme={null}
npm install fastify @mendable/firecrawl-js
```

Add your API key to `.env`:

```bash theme={null}
FIRECRAWL_API_KEY=fc-YOUR-API-KEY
```

## Search the web

```javascript theme={null}
import Fastify from "fastify";
import Firecrawl from "@mendable/firecrawl-js";

const fastify = Fastify({ logger: true });
const firecrawl = new Firecrawl({ apiKey: process.env.FIRECRAWL_API_KEY });

fastify.post("/search", async (request) => {
  const { query } = request.body;
  return firecrawl.search(query, { limit: 5 });
});

fastify.listen({ port: 3000 });
```

## Scrape a page

```javascript theme={null}
fastify.post("/scrape", async (request) => {
  const { url } = request.body;
  return firecrawl.scrape(url);
});
```

## Interact with a page

Use interact to control a live browser session — click buttons, fill forms, and extract dynamic content.

```javascript theme={null}
fastify.post("/interact", async (request) => {
  const { url } = request.body;

  const result = await firecrawl.scrape(url, { formats: ['markdown'] });
  const scrapeId = result.metadata?.scrapeId;

  await firecrawl.interact(scrapeId, { prompt: 'Search for iPhone 16 Pro Max' });
  const response = await firecrawl.interact(scrapeId, { prompt: 'Click on the first result and tell me the price' });

  await firecrawl.stopInteraction(scrapeId);

  return { output: response.output };
});
```

## As a Fastify plugin

Encapsulate the client in a plugin for reuse across routes:

```javascript theme={null}
import fp from "fastify-plugin";
import Firecrawl from "@mendable/firecrawl-js";

export default fp(async function firecrawlPlugin(fastify) {
  const client = new Firecrawl({ apiKey: process.env.FIRECRAWL_API_KEY });
  fastify.decorate("firecrawl", client);
});
```

Register the plugin, then use `fastify.firecrawl` in any route:

```javascript theme={null}
fastify.register(firecrawlPlugin);

fastify.post("/search", async function (request) {
  const { query } = request.body;
  return this.firecrawl.search(query, { limit: 5 });
});
```

## Test it

```bash theme={null}
curl -X POST http://localhost:3000/search \
  -H "Content-Type: application/json" \
  -d '{"query": "firecrawl web scraping"}'
```

## Next steps

<CardGroup>
  <Card title="Scrape docs" icon="file-lines" href="/features/scrape">
    All scrape options including formats, actions, and proxies
  </Card>

  <Card title="Search docs" icon="magnifying-glass" href="/features/search">
    Search the web and get full page content
  </Card>

  <Card title="Interact docs" icon="hand-pointer" href="/features/interact">
    Click, fill forms, and extract dynamic content
  </Card>

  <Card title="Node SDK reference" icon="node" href="/sdks/node">
    Full SDK reference with crawl, map, batch scrape, and more
  </Card>
</CardGroup>


# Flask
Source: https://docs.firecrawl.dev/quickstarts/flask

Use Firecrawl with Flask to build web scraping and search APIs in Python.

## Prerequisites

* Python 3.8+
* A Firecrawl API key — [get one free](https://www.firecrawl.dev/app/api-keys)

## Setup

```bash theme={null}
pip install flask firecrawl-py
```

Add your API key to `.env`:

```bash theme={null}
FIRECRAWL_API_KEY=fc-YOUR-API-KEY
```

## Create the app

Create `app.py`:

```python theme={null}
import os
from flask import Flask, request, jsonify
from firecrawl import Firecrawl

app = Flask(__name__)
firecrawl = Firecrawl(api_key=os.environ["FIRECRAWL_API_KEY"])


@app.post("/search")
def search():
    data = request.get_json()
    results = firecrawl.search(data["query"], limit=data.get("limit", 5))
    return jsonify([{"title": r.title, "url": r.url} for r in results.web])


@app.post("/scrape")
def scrape():
    data = request.get_json()
    result = firecrawl.scrape(data["url"])
    return jsonify(markdown=result.markdown, metadata=result.metadata)


@app.post("/interact/start")
def interact_start():
    data = request.get_json()
    result = firecrawl.scrape(data["url"], formats=["markdown"])
    return jsonify(scrape_id=result.metadata.scrape_id)


@app.post("/interact")
def interact():
    data = request.get_json()
    response = firecrawl.interact(data["scrape_id"], prompt=data["prompt"])
    return jsonify(output=response.output)


@app.post("/interact/stop")
def interact_stop():
    data = request.get_json()
    firecrawl.stop_interaction(data["scrape_id"])
    return jsonify(status="stopped")


if __name__ == "__main__":
    app.run(debug=True)
```

## Run it

```bash theme={null}
flask run
```

## Test it

```bash theme={null}
# Search the web
curl -X POST http://localhost:5000/search \
  -H "Content-Type: application/json" \
  -d '{"query": "firecrawl web scraping", "limit": 5}'

# Scrape a page
curl -X POST http://localhost:5000/scrape \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com"}'

# Start an interactive session
curl -X POST http://localhost:5000/interact/start \
  -H "Content-Type: application/json" \
  -d '{"url": "https://www.amazon.com"}'
```

## Next steps

<CardGroup>
  <Card title="Scrape docs" icon="file-lines" href="/features/scrape">
    All scrape options including formats, actions, and proxies
  </Card>

  <Card title="Search docs" icon="magnifying-glass" href="/features/search">
    Search the web and get full page content
  </Card>

  <Card title="Interact docs" icon="hand-pointer" href="/features/interact">
    Click, fill forms, and extract dynamic content
  </Card>

  <Card title="Python SDK reference" icon="python" href="/sdks/python">
    Full SDK reference with crawl, map, async, and more
  </Card>
</CardGroup>


# Go
Source: https://docs.firecrawl.dev/quickstarts/go

Get started with Firecrawl in Go. Scrape, search, and interact with web data using the REST API.

## Prerequisites

* Go 1.21+
* A Firecrawl API key — [get one free](https://www.firecrawl.dev/app/api-keys)

## Search the web

Firecrawl works with Go through the REST API. Use `net/http` to make requests directly.

```go theme={null}
package main

import (
	"bytes"
	"encoding/json"
	"fmt"
	"io"
	"net/http"
	"os"
)

func main() {
	apiKey := os.Getenv("FIRECRAWL_API_KEY")

	body, _ := json.Marshal(map[string]interface{}{
		"query": "firecrawl web scraping",
		"limit": 5,
	})

	req, _ := http.NewRequest("POST", "https://api.firecrawl.dev/v2/search", bytes.NewReader(body))
	req.Header.Set("Authorization", "Bearer "+apiKey)
	req.Header.Set("Content-Type", "application/json")

	resp, err := http.DefaultClient.Do(req)
	if err != nil {
		fmt.Fprintf(os.Stderr, "request failed: %v\n", err)
		os.Exit(1)
	}
	defer resp.Body.Close()

	result, _ := io.ReadAll(resp.Body)
	fmt.Println(string(result))
}
```

<Accordion title="Example response">
  ```json theme={null}
  {
    "success": true,
    "data": {
      "web": [
        {
          "url": "https://docs.firecrawl.dev",
          "title": "Firecrawl Documentation",
          "markdown": "# Firecrawl\n\nFirecrawl is a web scraping API..."
        }
      ]
    }
  }
  ```
</Accordion>

## Scrape a page

```go theme={null}
body, _ := json.Marshal(map[string]string{
	"url": "https://example.com",
})

req, _ := http.NewRequest("POST", "https://api.firecrawl.dev/v2/scrape", bytes.NewReader(body))
req.Header.Set("Authorization", "Bearer "+apiKey)
req.Header.Set("Content-Type", "application/json")

resp, err := http.DefaultClient.Do(req)
if err != nil {
	fmt.Fprintf(os.Stderr, "request failed: %v\n", err)
	os.Exit(1)
}
defer resp.Body.Close()

result, _ := io.ReadAll(resp.Body)
fmt.Println(string(result))
```

<Accordion title="Example response">
  ```json theme={null}
  {
    "success": true,
    "data": {
      "markdown": "# Example Domain\n\nThis domain is for use in illustrative examples...",
      "metadata": {
        "title": "Example Domain",
        "sourceURL": "https://example.com"
      }
    }
  }
  ```
</Accordion>

## Interact with a page

Start a browser session, interact with the page using natural-language prompts, then close the session.

### Step 1 — Scrape to start a session

```go theme={null}
body, _ := json.Marshal(map[string]interface{}{
	"url":     "https://www.amazon.com",
	"formats": []string{"markdown"},
})

req, _ := http.NewRequest("POST", "https://api.firecrawl.dev/v2/scrape", bytes.NewReader(body))
req.Header.Set("Authorization", "Bearer "+apiKey)
req.Header.Set("Content-Type", "application/json")

resp, err := http.DefaultClient.Do(req)
if err != nil {
	fmt.Fprintf(os.Stderr, "request failed: %v\n", err)
	os.Exit(1)
}
defer resp.Body.Close()

var scrapeResult map[string]interface{}
json.NewDecoder(resp.Body).Decode(&scrapeResult)

data := scrapeResult["data"].(map[string]interface{})
metadata := data["metadata"].(map[string]interface{})
scrapeId := metadata["scrapeId"].(string)
fmt.Println("scrapeId:", scrapeId)
```

### Step 2 — Send interactions

```go theme={null}
// Search for a product
interactBody, _ := json.Marshal(map[string]string{
	"prompt": "Search for iPhone 16 Pro Max",
})

interactURL := fmt.Sprintf("https://api.firecrawl.dev/v2/scrape/%s/interact", scrapeId)
req, _ = http.NewRequest("POST", interactURL, bytes.NewReader(interactBody))
req.Header.Set("Authorization", "Bearer "+apiKey)
req.Header.Set("Content-Type", "application/json")

resp, err = http.DefaultClient.Do(req)
if err != nil {
	fmt.Fprintf(os.Stderr, "interact failed: %v\n", err)
	os.Exit(1)
}
defer resp.Body.Close()

result, _ := io.ReadAll(resp.Body)
fmt.Println(string(result))

// Click on the first result
interactBody, _ = json.Marshal(map[string]string{
	"prompt": "Click on the first result and tell me the price",
})

req, _ = http.NewRequest("POST", interactURL, bytes.NewReader(interactBody))
req.Header.Set("Authorization", "Bearer "+apiKey)
req.Header.Set("Content-Type", "application/json")

resp, err = http.DefaultClient.Do(req)
if err != nil {
	fmt.Fprintf(os.Stderr, "interact failed: %v\n", err)
	os.Exit(1)
}
defer resp.Body.Close()

result, _ = io.ReadAll(resp.Body)
fmt.Println(string(result))
```

### Step 3 — Stop the session

```go theme={null}
req, _ = http.NewRequest("DELETE", interactURL, nil)
req.Header.Set("Authorization", "Bearer "+apiKey)

resp, err = http.DefaultClient.Do(req)
if err != nil {
	fmt.Fprintf(os.Stderr, "delete failed: %v\n", err)
	os.Exit(1)
}
defer resp.Body.Close()

fmt.Println("Session stopped")
```

## Reusable helper

For repeated use, wrap the API in a small helper:

```go theme={null}
type FirecrawlClient struct {
	APIKey  string
	BaseURL string
	Client  *http.Client
}

func NewFirecrawlClient(apiKey string) *FirecrawlClient {
	return &FirecrawlClient{
		APIKey:  apiKey,
		BaseURL: "https://api.firecrawl.dev/v2",
		Client:  &http.Client{},
	}
}

func (fc *FirecrawlClient) post(endpoint string, payload interface{}) ([]byte, error) {
	body, err := json.Marshal(payload)
	if err != nil {
		return nil, err
	}

	req, err := http.NewRequest("POST", fc.BaseURL+endpoint, bytes.NewReader(body))
	if err != nil {
		return nil, err
	}
	req.Header.Set("Authorization", "Bearer "+fc.APIKey)
	req.Header.Set("Content-Type", "application/json")

	resp, err := fc.Client.Do(req)
	if err != nil {
		return nil, err
	}
	defer resp.Body.Close()

	return io.ReadAll(resp.Body)
}

func (fc *FirecrawlClient) Scrape(url string) ([]byte, error) {
	return fc.post("/scrape", map[string]string{"url": url})
}

func (fc *FirecrawlClient) Search(query string, limit int) ([]byte, error) {
	return fc.post("/search", map[string]interface{}{"query": query, "limit": limit})
}
```

<Note>
  A [community Go SDK](https://github.com/mendableai/firecrawl-go) is available for the v1 API. See the [Go SDK docs](/sdks/go) for details.
</Note>

## Next steps

<CardGroup>
  <Card title="Search docs" icon="magnifying-glass" href="/features/search">
    Search the web and get full page content
  </Card>

  <Card title="Scrape docs" icon="file-lines" href="/features/scrape">
    All scrape options including formats, actions, and proxies
  </Card>

  <Card title="Interact docs" icon="hand-pointer" href="/features/interact">
    Click, fill forms, and extract dynamic content
  </Card>

  <Card title="API Reference" icon="code" href="/api-reference/v2-introduction">
    Complete REST API documentation
  </Card>
</CardGroup>


# Hono
Source: https://docs.firecrawl.dev/quickstarts/hono

Use Firecrawl with Hono to build lightweight web scraping and search APIs that run anywhere.

## Prerequisites

* Node.js 18+, Bun, or Deno
* A Firecrawl API key — [get one free](https://www.firecrawl.dev/app/api-keys)

## Setup

```bash theme={null}
npm install hono @mendable/firecrawl-js
```

Add your API key to `.env`:

```bash theme={null}
FIRECRAWL_API_KEY=fc-YOUR-API-KEY
```

## Search the web

```typescript theme={null}
import { Hono } from "hono";
import Firecrawl from "@mendable/firecrawl-js";

const app = new Hono();
const firecrawl = new Firecrawl({ apiKey: process.env.FIRECRAWL_API_KEY });

app.post("/search", async (c) => {
  const { query } = await c.req.json();
  const results = await firecrawl.search(query, { limit: 5 });
  return c.json(results);
});

export default app;
```

## Scrape a page

```typescript theme={null}
app.post("/scrape", async (c) => {
  const { url } = await c.req.json();
  const result = await firecrawl.scrape(url);
  return c.json(result);
});
```

## Interact with a page

Use interact to control a live browser session — click buttons, fill forms, and extract dynamic content.

```typescript theme={null}
app.post("/interact", async (c) => {
  const { url } = await c.req.json();

  const result = await firecrawl.scrape(url, { formats: ['markdown'] });
  const scrapeId = result.metadata?.scrapeId;

  await firecrawl.interact(scrapeId, { prompt: 'Search for iPhone 16 Pro Max' });
  const response = await firecrawl.interact(scrapeId, { prompt: 'Click on the first result and tell me the price' });

  await firecrawl.stopInteraction(scrapeId);

  return c.json({ output: response.output });
});
```

## Deploy anywhere

Hono runs on multiple runtimes. For Cloudflare Workers, pass the API key from the environment binding:

```typescript theme={null}
import { Hono } from "hono";
import Firecrawl from "@mendable/firecrawl-js";

type Bindings = { FIRECRAWL_API_KEY: string };
const app = new Hono<{ Bindings: Bindings }>();

app.post("/search", async (c) => {
  const firecrawl = new Firecrawl({ apiKey: c.env.FIRECRAWL_API_KEY });
  const { query } = await c.req.json();
  const results = await firecrawl.search(query, { limit: 5 });
  return c.json(results);
});

export default app;
```

## Next steps

<CardGroup>
  <Card title="Scrape docs" icon="file-lines" href="/features/scrape">
    All scrape options including formats, actions, and proxies
  </Card>

  <Card title="Search docs" icon="magnifying-glass" href="/features/search">
    Search the web and get full page content
  </Card>

  <Card title="Interact docs" icon="hand-pointer" href="/features/interact">
    Click, fill forms, and extract dynamic content
  </Card>

  <Card title="Node SDK reference" icon="node" href="/sdks/node">
    Full SDK reference with crawl, map, batch scrape, and more
  </Card>
</CardGroup>


# Java
Source: https://docs.firecrawl.dev/quickstarts/java

Get started with Firecrawl in Java. Search, scrape, and interact with web data using the official SDK.

## Prerequisites

* Java 11+
* A Firecrawl API key — [get one free](https://www.firecrawl.dev/app/api-keys)

## Install the SDK

<Tabs>
  <Tab title="Gradle (Kotlin DSL)">
    ```kotlin theme={null}
    dependencies {
        implementation("com.firecrawl:firecrawl-java:1.2.0")
    }
    ```
  </Tab>

  <Tab title="Maven">
    ```xml theme={null}
    <dependency>
        <groupId>com.firecrawl</groupId>
        <artifactId>firecrawl-java</artifactId>
        <version>1.2.0</version>
    </dependency>
    ```
  </Tab>
</Tabs>

## Search the web

```java theme={null}
import com.firecrawl.client.FirecrawlClient;
import com.firecrawl.models.SearchData;
import com.firecrawl.models.SearchOptions;

public class Main {
    public static void main(String[] args) {
        FirecrawlClient client = FirecrawlClient.builder()
            .apiKey("fc-YOUR-API-KEY")
            .build();

        SearchData results = client.search(
            "firecrawl web scraping",
            SearchOptions.builder().limit(5).build()
        );

        if (results.getWeb() != null) {
            for (var result : results.getWeb()) {
                System.out.println(result.get("title") + " - " + result.get("url"));
            }
        }
    }
}
```

## Scrape a page

```java theme={null}
import com.firecrawl.models.Document;

Document doc = client.scrape("https://example.com");
System.out.println(doc.getMarkdown());
```

<Accordion title="Example response">
  ```json theme={null}
  {
    "markdown": "# Example Domain\n\nThis domain is for use in illustrative examples...",
    "metadata": {
      "title": "Example Domain",
      "sourceURL": "https://example.com"
    }
  }
  ```
</Accordion>

## Interact with a page

Open a browser session, run Playwright code against it, and close it when done:

```java theme={null}
import com.firecrawl.models.ScrapeOptions;
import com.firecrawl.models.BrowserExecuteResponse;
import java.util.List;

Document doc = client.scrape("https://www.amazon.com",
    ScrapeOptions.builder().formats(List.of((Object) "markdown")).build());
String scrapeId = (String) doc.getMetadata().get("scrapeId");

BrowserExecuteResponse run = client.interact(scrapeId,
    "const title = await page.title(); console.log(title);");
System.out.println(run.getStdout());

client.stopInteractiveBrowser(scrapeId);
```

## Environment variable

Instead of passing `apiKey` directly, set the `FIRECRAWL_API_KEY` environment variable:

```bash theme={null}
export FIRECRAWL_API_KEY=fc-YOUR-API-KEY
```

```java theme={null}
FirecrawlClient client = FirecrawlClient.fromEnv();
```

## Next steps

<CardGroup>
  <Card title="Search docs" icon="magnifying-glass" href="/features/search">
    Search the web and get full page content
  </Card>

  <Card title="Scrape docs" icon="file-lines" href="/features/scrape">
    All scrape options including formats, actions, and proxies
  </Card>

  <Card title="Interact docs" icon="hand-pointer" href="/features/interact">
    Click, fill forms, and extract dynamic content
  </Card>

  <Card title="Java SDK reference" icon="java" href="/sdks/java">
    Full SDK reference with crawl, map, batch scrape, and more
  </Card>
</CardGroup>


# Laravel
Source: https://docs.firecrawl.dev/quickstarts/laravel

Use Firecrawl with Laravel to search, scrape, and interact with web data using the REST API.

## Prerequisites

* Laravel 10+ project
* A Firecrawl API key — [get one free](https://www.firecrawl.dev/app/api-keys)

## Configuration

Add your API key to `.env`:

```bash theme={null}
FIRECRAWL_API_KEY=fc-YOUR-API-KEY
```

Add the config entry to `config/services.php`:

```php theme={null}
'firecrawl' => [
    'api_key' => env('FIRECRAWL_API_KEY'),
    'base_url' => env('FIRECRAWL_API_URL', 'https://api.firecrawl.dev/v2'),
],
```

## Create a service class

Create `app/Services/FirecrawlService.php`:

```php theme={null}
<?php

namespace App\Services;

use Illuminate\Support\Facades\Http;

class FirecrawlService
{
    private string $apiKey;
    private string $baseUrl;

    public function __construct()
    {
        $this->apiKey = config('services.firecrawl.api_key');
        $this->baseUrl = config('services.firecrawl.base_url');
    }

    public function search(string $query, int $limit = 5): array
    {
        $response = Http::withToken($this->apiKey)
            ->post("{$this->baseUrl}/search", [
                'query' => $query,
                'limit' => $limit,
            ]);

        return $response->json();
    }

    public function scrape(string $url, array $options = []): array
    {
        $response = Http::withToken($this->apiKey)
            ->post("{$this->baseUrl}/scrape", array_merge(['url' => $url], $options));

        return $response->json();
    }

    public function interact(string $url, string $prompt, ?string $followUp = null): array
    {
        // 1. Scrape to open a browser session
        $scrapeResult = $this->scrape($url, ['formats' => ['markdown']]);
        $scrapeId = $scrapeResult['data']['metadata']['scrapeId'];

        // 2. Send first prompt
        Http::withToken($this->apiKey)
            ->post("{$this->baseUrl}/scrape/{$scrapeId}/interact", [
                'prompt' => $prompt,
            ]);

        // 3. Send follow-up prompt
        $result = null;
        if ($followUp) {
            $result = Http::withToken($this->apiKey)
                ->post("{$this->baseUrl}/scrape/{$scrapeId}/interact", [
                    'prompt' => $followUp,
                ])->json();
        }

        // 4. Close the session
        Http::withToken($this->apiKey)
            ->delete("{$this->baseUrl}/scrape/{$scrapeId}/interact");

        return $result ?? $scrapeResult;
    }
}
```

## Create a controller

Create `app/Http/Controllers/FirecrawlController.php`:

```php theme={null}
<?php

namespace App\Http\Controllers;

use App\Services\FirecrawlService;
use Illuminate\Http\Request;

class FirecrawlController extends Controller
{
    public function __construct(private FirecrawlService $firecrawl) {}

    public function search(Request $request)
    {
        $validated = $request->validate(['query' => 'required|string']);
        return response()->json(
            $this->firecrawl->search($validated['query'], $request->input('limit', 5))
        );
    }

    public function scrape(Request $request)
    {
        $validated = $request->validate(['url' => 'required|url']);
        return response()->json($this->firecrawl->scrape($validated['url']));
    }

    public function interact(Request $request)
    {
        $validated = $request->validate([
            'url' => 'required|url',
            'prompt' => 'required|string',
        ]);
        return response()->json(
            $this->firecrawl->interact(
                $validated['url'],
                $validated['prompt'],
                $request->input('followUp')
            )
        );
    }
}
```

## Register routes

In `routes/api.php`:

```php theme={null}
use App\Http\Controllers\FirecrawlController;

Route::post('/search', [FirecrawlController::class, 'search']);
Route::post('/scrape', [FirecrawlController::class, 'scrape']);
Route::post('/interact', [FirecrawlController::class, 'interact']);
```

## Test it

```bash theme={null}
php artisan serve

# Search the web
curl -X POST http://localhost:8000/api/search \
  -H "Content-Type: application/json" \
  -d '{"query": "firecrawl web scraping"}'

# Scrape a page
curl -X POST http://localhost:8000/api/scrape \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com"}'

# Interact with a page
curl -X POST http://localhost:8000/api/interact \
  -H "Content-Type: application/json" \
  -d '{"url": "https://www.amazon.com", "prompt": "Search for iPhone 16 Pro Max", "followUp": "Click on the first result and tell me the price"}'
```

## Next steps

<CardGroup>
  <Card title="Search docs" icon="magnifying-glass" href="/features/search">
    Search the web and get full page content
  </Card>

  <Card title="Scrape docs" icon="file-lines" href="/features/scrape">
    All scrape options including formats, actions, and proxies
  </Card>

  <Card title="Interact docs" icon="hand-pointer" href="/features/interact">
    Click, fill forms, and extract dynamic content
  </Card>

  <Card title="API Reference" icon="code" href="/api-reference/v2-introduction">
    Complete REST API documentation
  </Card>
</CardGroup>


# Mastra
Source: https://docs.firecrawl.dev/quickstarts/mastra

Wire Firecrawl into Mastra tools so your agents and workflows can search and scrape live web data.

## Prerequisites

* Node.js 22.13+
* A Firecrawl API key — [get one free](https://www.firecrawl.dev/app/api-keys)
* An API key from a supported [Mastra model provider](https://mastra.ai/models)
* An existing Mastra project — follow the [Mastra quickstart](https://mastra.ai/guides/getting-started/quickstart) to set one up

## Install the SDK

```bash theme={null}
npm install @mendable/firecrawl-js
```

## Set your API key

Add your API key to `.env`:

```bash theme={null}
FIRECRAWL_API_KEY=fc-YOUR-API-KEY
```

## Build the Firecrawl tools

Create `src/mastra/tools/firecrawl.ts` to expose search and scrape as Mastra tools:

```typescript theme={null}
import Firecrawl from "@mendable/firecrawl-js";
import { createTool } from "@mastra/core/tools";
import { z } from "zod";

const firecrawl = new Firecrawl({ apiKey: process.env.FIRECRAWL_API_KEY! });

export const firecrawlSearch = createTool({
  id: "firecrawl-search",
  description: "Search the web and return top results.",
  inputSchema: z.object({ query: z.string().min(1) }),
  outputSchema: z.object({
    results: z.array(
      z.object({
        title: z.string().nullable(),
        url: z.string(),
      }),
    ),
  }),
  execute: async ({ query }) => {
    const results = await firecrawl.search(query, { limit: 3 });
    return {
      results: (results.web ?? []).map((item) => ({
        title: item.title ?? null,
        url: item.url,
      })),
    };
  },
});

export const firecrawlScrape = createTool({
  id: "firecrawl-scrape",
  description: "Scrape a URL and return markdown content.",
  inputSchema: z.object({ url: z.string().url() }),
  outputSchema: z.object({ markdown: z.string() }),
  execute: async ({ url }) => {
    const result = await firecrawl.scrape(url, {
      formats: ["markdown"],
      onlyMainContent: true,
    });
    return { markdown: result.markdown ?? "" };
  },
});
```

## Create the agent

Create `src/mastra/agents/web-agent.ts` and give it the Firecrawl tools:

```typescript theme={null}
import { Agent } from "@mastra/core/agent";
import { firecrawlSearch, firecrawlScrape } from "../tools/firecrawl";

export const webAgent = new Agent({
  id: "web-agent",
  name: "Web Agent",
  instructions:
    "Use Firecrawl tools to search and scrape web pages, then summarize the results.",
  model: "openai/gpt-5.4",
  tools: { firecrawlSearch, firecrawlScrape },
});
```

## Register the agent

Register the agent on your Mastra instance in `src/mastra/index.ts`:

```typescript theme={null}
import { Mastra } from "@mastra/core";
import { webAgent } from "./agents/web-agent";

export const mastra = new Mastra({
  agents: { webAgent },
});
```

## Test in Studio

Run the dev server and open [Mastra Studio](https://mastra.ai/docs/studio/overview):

```bash theme={null}
mastra dev
```

Open the **Web Agent** and try prompts like:

* "Find the latest Firecrawl changelog and summarize the last release."
* "Search for Firecrawl pricing and extract the plan tiers."

## Self-hosted Firecrawl

If you run Firecrawl locally, set `FIRECRAWL_API_URL` and pass `apiUrl` to the client:

```typescript theme={null}
const firecrawl = new Firecrawl({
  apiKey: process.env.FIRECRAWL_API_KEY!,
  apiUrl: process.env.FIRECRAWL_API_URL,
});
```

## Next steps

<CardGroup>
  <Card title="Scrape docs" icon="file-lines" href="/features/scrape">
    All scrape options including formats, actions, and proxies
  </Card>

  <Card title="Search docs" icon="magnifying-glass" href="/features/search">
    Search the web and get full page content
  </Card>

  <Card title="Agent docs" icon="robot" href="/features/agent">
    Let an agent drive Firecrawl end to end
  </Card>

  <Card title="Node SDK reference" icon="node" href="/sdks/node">
    Full SDK reference with crawl, map, batch scrape, and more
  </Card>
</CardGroup>


# NestJS
Source: https://docs.firecrawl.dev/quickstarts/nestjs

Use Firecrawl with NestJS to build structured web scraping and search services.

## Prerequisites

* NestJS project (`@nestjs/cli`)
* A Firecrawl API key — [get one free](https://www.firecrawl.dev/app/api-keys)

## Install the SDK

```bash theme={null}
npm install @mendable/firecrawl-js
```

Add your API key to `.env`:

```bash theme={null}
FIRECRAWL_API_KEY=fc-YOUR-API-KEY
```

## Create a Firecrawl service

Create `src/firecrawl/firecrawl.service.ts`:

```typescript theme={null}
import { Injectable } from "@nestjs/common";
import Firecrawl from "@mendable/firecrawl-js";

@Injectable()
export class FirecrawlService {
  private readonly client: Firecrawl;

  constructor() {
    this.client = new Firecrawl({ apiKey: process.env.FIRECRAWL_API_KEY });
  }

  async search(query: string, limit = 5) {
    return this.client.search(query, { limit });
  }

  async scrape(url: string) {
    return this.client.scrape(url);
  }

  async interact(url: string, prompts: string[]) {
    const result = await this.client.scrape(url, { formats: ['markdown'] });
    const scrapeId = result.metadata?.scrapeId;

    const responses = [];
    for (const prompt of prompts) {
      const response = await this.client.interact(scrapeId, { prompt });
      responses.push(response);
    }

    await this.client.stopInteraction(scrapeId);
    return responses;
  }
}
```

## Create a controller

Create `src/firecrawl/firecrawl.controller.ts`:

```typescript theme={null}
import { Body, Controller, Post } from "@nestjs/common";
import { FirecrawlService } from "./firecrawl.service";

@Controller("firecrawl")
export class FirecrawlController {
  constructor(private readonly firecrawlService: FirecrawlService) {}

  @Post("search")
  async search(@Body("query") query: string) {
    return this.firecrawlService.search(query);
  }

  @Post("scrape")
  async scrape(@Body("url") url: string) {
    return this.firecrawlService.scrape(url);
  }

  @Post("interact")
  async interact(@Body("url") url: string, @Body("prompts") prompts: string[]) {
    return this.firecrawlService.interact(url, prompts);
  }
}
```

## Register the module

Create `src/firecrawl/firecrawl.module.ts`:

```typescript theme={null}
import { Module } from "@nestjs/common";
import { FirecrawlService } from "./firecrawl.service";
import { FirecrawlController } from "./firecrawl.controller";

@Module({
  providers: [FirecrawlService],
  controllers: [FirecrawlController],
  exports: [FirecrawlService],
})
export class FirecrawlModule {}
```

Import `FirecrawlModule` in your `AppModule`.

## Test it

```bash theme={null}
curl -X POST http://localhost:3000/firecrawl/search \
  -H "Content-Type: application/json" \
  -d '{"query": "firecrawl web scraping"}'
```

## Next steps

<CardGroup>
  <Card title="Scrape docs" icon="file-lines" href="/features/scrape">
    All scrape options including formats, actions, and proxies
  </Card>

  <Card title="Search docs" icon="magnifying-glass" href="/features/search">
    Search the web and get full page content
  </Card>

  <Card title="Interact docs" icon="hand-pointer" href="/features/interact">
    Click, fill forms, and extract dynamic content
  </Card>

  <Card title="Node SDK reference" icon="node" href="/sdks/node">
    Full SDK reference with crawl, map, batch scrape, and more
  </Card>
</CardGroup>


# Next.js
Source: https://docs.firecrawl.dev/quickstarts/nextjs

Use Firecrawl with Next.js to scrape, search, and interact with web data in your React application.

## Prerequisites

* Next.js 14+ project (App Router)
* A Firecrawl API key — [get one free](https://www.firecrawl.dev/app/api-keys)

## Install the SDK

```bash theme={null}
npm install @mendable/firecrawl-js
```

## Set your API key

Add your API key to `.env.local`:

```bash theme={null}
FIRECRAWL_API_KEY=fc-YOUR-API-KEY
```

## Search the web

The SDK should only run server-side since it requires your API key.

### Route Handler

Create `app/api/search/route.ts`:

```typescript theme={null}
import { NextResponse } from "next/server";
import Firecrawl from "@mendable/firecrawl-js";

const firecrawl = new Firecrawl({ apiKey: process.env.FIRECRAWL_API_KEY });

export async function POST(request: Request) {
  const { query } = await request.json();
  const results = await firecrawl.search(query, { limit: 5 });

  return NextResponse.json(results);
}
```

### Server Action

Create `app/actions.ts` for use from Client Components:

```typescript theme={null}
"use server";

import Firecrawl from "@mendable/firecrawl-js";

const firecrawl = new Firecrawl({ apiKey: process.env.FIRECRAWL_API_KEY });

export async function searchWeb(query: string) {
  const results = await firecrawl.search(query, { limit: 5 });
  return (results.web || []).map((r) => ({ title: r.title, url: r.url }));
}
```

## Scrape a page

### Route Handler

Create `app/api/scrape/route.ts`:

```typescript theme={null}
import { NextResponse } from "next/server";
import Firecrawl from "@mendable/firecrawl-js";

const firecrawl = new Firecrawl({ apiKey: process.env.FIRECRAWL_API_KEY });

export async function POST(request: Request) {
  const { url } = await request.json();
  const result = await firecrawl.scrape(url);

  return NextResponse.json(result);
}
```

### Server Component

Fetch data directly in a Server Component at `app/page.tsx`:

```tsx theme={null}
import Firecrawl from "@mendable/firecrawl-js";

const firecrawl = new Firecrawl({ apiKey: process.env.FIRECRAWL_API_KEY });

export default async function Page() {
  const result = await firecrawl.scrape("https://example.com");

  return (
    <article>
      <h1>Scraped Content</h1>
      <pre>{result.markdown}</pre>
    </article>
  );
}
```

## Interact with a page

Use interact to control a live browser session — click buttons, fill forms, and extract dynamic content.

### Route Handler

Create `app/api/interact/route.ts`:

```typescript theme={null}
import { NextResponse } from "next/server";
import Firecrawl from "@mendable/firecrawl-js";

const firecrawl = new Firecrawl({ apiKey: process.env.FIRECRAWL_API_KEY });

export async function POST(request: Request) {
  const { url, prompts } = await request.json();

  const result = await firecrawl.scrape(url, { formats: ['markdown'] });
  const scrapeId = result.metadata?.scrapeId;

  await firecrawl.interact(scrapeId, { prompt: 'Search for iPhone 16 Pro Max' });
  const response = await firecrawl.interact(scrapeId, { prompt: 'Click on the first result and tell me the price' });

  await firecrawl.stopInteraction(scrapeId);

  return NextResponse.json({ output: response.output });
}
```

## Next steps

<CardGroup>
  <Card title="Scrape docs" icon="file-lines" href="/features/scrape">
    All scrape options including formats, actions, and proxies
  </Card>

  <Card title="Search docs" icon="magnifying-glass" href="/features/search">
    Search the web and get full page content
  </Card>

  <Card title="Interact docs" icon="hand-pointer" href="/features/interact">
    Click, fill forms, and extract dynamic content
  </Card>

  <Card title="Node SDK reference" icon="node" href="/sdks/node">
    Full SDK reference with crawl, map, batch scrape, and more
  </Card>
</CardGroup>


# Node.js
Source: https://docs.firecrawl.dev/quickstarts/nodejs

Get started with Firecrawl in Node.js. Scrape, search, and interact with web data using the official SDK.

## Prerequisites

* Node.js 18+
* A Firecrawl API key — [get one free](https://www.firecrawl.dev/app/api-keys)

## Install the SDK

```bash theme={null}
npm install @mendable/firecrawl-js
```

## Environment variable

Instead of passing `apiKey` directly, set the `FIRECRAWL_API_KEY` environment variable:

```bash theme={null}
export FIRECRAWL_API_KEY=fc-YOUR-API-KEY
```

```javascript theme={null}
const app = new Firecrawl();
```

## Search the web

```javascript theme={null}
import Firecrawl from '@mendable/firecrawl-js';

const app = new Firecrawl({ apiKey: "fc-YOUR-API-KEY" });
const results = await app.search("firecrawl web scraping", { limit: 5 });

for (const result of results.web) {
  console.log(result.title, result.url);
}
```

## Scrape a page

```javascript theme={null}
const result = await app.scrape("https://example.com");

console.log(result.markdown);
```

<Accordion title="Example response">
  ```json theme={null}
  {
    "markdown": "# Example Domain\n\nThis domain is for use in illustrative examples...",
    "metadata": {
      "title": "Example Domain",
      "sourceURL": "https://example.com"
    }
  }
  ```
</Accordion>

## Interact with a page

Use interact to control a live browser session — click buttons, fill forms, and extract dynamic content.

```javascript theme={null}
const result = await app.scrape('https://www.amazon.com', { formats: ['markdown'] });
const scrapeId = result.metadata?.scrapeId;

await app.interact(scrapeId, { prompt: 'Search for iPhone 16 Pro Max' });
const response = await app.interact(scrapeId, { prompt: 'Click on the first result and tell me the price' });
console.log(response.output);

await app.stopInteraction(scrapeId);
```

## Next steps

<CardGroup>
  <Card title="Scrape docs" icon="file-lines" href="/features/scrape">
    All scrape options including formats, actions, and proxies
  </Card>

  <Card title="Search docs" icon="magnifying-glass" href="/features/search">
    Search the web and get full page content
  </Card>

  <Card title="Interact docs" icon="hand-pointer" href="/features/interact">
    Click, fill forms, and extract dynamic content
  </Card>

  <Card title="Node SDK reference" icon="node" href="/sdks/node">
    Full SDK reference with crawl, map, batch scrape, and more
  </Card>
</CardGroup>


# Nuxt
Source: https://docs.firecrawl.dev/quickstarts/nuxt

Use Firecrawl with Nuxt to scrape, search, and interact with web data in your Vue application.

## Prerequisites

* Nuxt 3+ project
* A Firecrawl API key — [get one free](https://www.firecrawl.dev/app/api-keys)

## Install the SDK

```bash theme={null}
npm install @mendable/firecrawl-js
```

Add your API key to `.env`:

```bash theme={null}
FIRECRAWL_API_KEY=fc-YOUR-API-KEY
```

## Search the web

Create `server/api/search.post.ts`:

```typescript theme={null}
import Firecrawl from "@mendable/firecrawl-js";

const firecrawl = new Firecrawl({
  apiKey: process.env.FIRECRAWL_API_KEY,
});

export default defineEventHandler(async (event) => {
  const { query } = await readBody(event);
  const results = await firecrawl.search(query, { limit: 5 });
  return results;
});
```

Call it from a Vue component:

```vue theme={null}
<script setup>
const query = ref("");
const { data, execute } = useFetch("/api/search", {
  method: "POST",
  body: { query },
  immediate: false,
});
</script>

<template>
  <div>
    <input v-model="query" placeholder="Search the web..." />
    <button @click="execute()">Search</button>
    <ul v-if="data?.web">
      <li v-for="result in data.web" :key="result.url">
        <a :href="result.url">{{ result.title }}</a>
      </li>
    </ul>
  </div>
</template>
```

## Scrape a page

Create `server/api/scrape.post.ts`:

```typescript theme={null}
import Firecrawl from "@mendable/firecrawl-js";

const firecrawl = new Firecrawl({
  apiKey: process.env.FIRECRAWL_API_KEY,
});

export default defineEventHandler(async (event) => {
  const { url } = await readBody(event);
  const result = await firecrawl.scrape(url);
  return result;
});
```

Call it from a Vue component:

```vue theme={null}
<script setup>
const url = ref("https://example.com");
const { data, execute } = useFetch("/api/scrape", {
  method: "POST",
  body: { url },
  immediate: false,
});
</script>

<template>
  <div>
    <input v-model="url" placeholder="Enter URL" />
    <button @click="execute()">Scrape</button>
    <pre v-if="data">{{ data.markdown }}</pre>
  </div>
</template>
```

## Interact with a page

Create `server/api/interact.post.ts`:

```typescript theme={null}
import Firecrawl from "@mendable/firecrawl-js";

const firecrawl = new Firecrawl({
  apiKey: process.env.FIRECRAWL_API_KEY,
});

export default defineEventHandler(async (event) => {
  const result = await firecrawl.scrape("https://www.amazon.com", {
    formats: ["markdown"],
  });

  const scrapeId = result.metadata?.scrapeId;

  await firecrawl.interact(scrapeId, {
    prompt: "Search for iPhone 16 Pro Max",
  });

  const response = await firecrawl.interact(scrapeId, {
    prompt: "Click on the first result and tell me the price",
  });

  await firecrawl.stopInteraction(scrapeId);

  return { output: response.output };
});
```

## Next steps

<CardGroup>
  <Card title="Scrape docs" icon="file-lines" href="/features/scrape">
    All scrape options including formats, actions, and proxies
  </Card>

  <Card title="Search docs" icon="magnifying-glass" href="/features/search">
    Search the web and get full page content
  </Card>

  <Card title="Interact docs" icon="hand-pointer" href="/features/interact">
    Click, fill forms, and extract dynamic content
  </Card>

  <Card title="Node SDK reference" icon="node" href="/sdks/node">
    Full SDK reference with crawl, map, batch scrape, and more
  </Card>
</CardGroup>


# PHP
Source: https://docs.firecrawl.dev/quickstarts/php

Get started with Firecrawl in PHP. Scrape, search, and interact with web data using the REST API.

## Prerequisites

* PHP 8.0+ with cURL extension
* A Firecrawl API key — [get one free](https://www.firecrawl.dev/app/api-keys)

## Search the web

Firecrawl works with PHP through the REST API using cURL.

```php theme={null}
<?php
$apiKey = getenv('FIRECRAWL_API_KEY');

$ch = curl_init('https://api.firecrawl.dev/v2/search');
curl_setopt_array($ch, [
    CURLOPT_RETURNTRANSFER => true,
    CURLOPT_POST => true,
    CURLOPT_HTTPHEADER => [
        'Authorization: Bearer ' . $apiKey,
        'Content-Type: application/json',
    ],
    CURLOPT_POSTFIELDS => json_encode([
        'query' => 'firecrawl web scraping',
        'limit' => 5,
    ]),
]);

$response = curl_exec($ch);
curl_close($ch);

$results = json_decode($response, true);
foreach ($results['data']['web'] as $result) {
    echo $result['title'] . ' - ' . $result['url'] . "\n";
}
```

<Accordion title="Example response">
  ```json theme={null}
  {
    "success": true,
    "data": {
      "web": [
        {
          "url": "https://docs.firecrawl.dev",
          "title": "Firecrawl Documentation",
          "markdown": "# Firecrawl\n\nFirecrawl is a web scraping API..."
        }
      ]
    }
  }
  ```
</Accordion>

## Scrape a page

```php theme={null}
<?php
$ch = curl_init('https://api.firecrawl.dev/v2/scrape');
curl_setopt_array($ch, [
    CURLOPT_RETURNTRANSFER => true,
    CURLOPT_POST => true,
    CURLOPT_HTTPHEADER => [
        'Authorization: Bearer ' . $apiKey,
        'Content-Type: application/json',
    ],
    CURLOPT_POSTFIELDS => json_encode([
        'url' => 'https://example.com',
    ]),
]);

$response = curl_exec($ch);
curl_close($ch);

$data = json_decode($response, true);
echo $data['data']['markdown'];
```

<Accordion title="Example response">
  ```json theme={null}
  {
    "success": true,
    "data": {
      "markdown": "# Example Domain\n\nThis domain is for use in illustrative examples...",
      "metadata": {
        "title": "Example Domain",
        "sourceURL": "https://example.com"
      }
    }
  }
  ```
</Accordion>

## Interact with a page

Start a browser session, interact with the page using natural-language prompts, then close the session.

### Step 1 — Scrape to start a session

```php theme={null}
<?php
$ch = curl_init('https://api.firecrawl.dev/v2/scrape');
curl_setopt_array($ch, [
    CURLOPT_RETURNTRANSFER => true,
    CURLOPT_POST => true,
    CURLOPT_HTTPHEADER => [
        'Authorization: Bearer ' . $apiKey,
        'Content-Type: application/json',
    ],
    CURLOPT_POSTFIELDS => json_encode([
        'url' => 'https://www.amazon.com',
        'formats' => ['markdown'],
    ]),
]);

$response = curl_exec($ch);
curl_close($ch);

$scrapeResult = json_decode($response, true);
$scrapeId = $scrapeResult['data']['metadata']['scrapeId'];
echo "scrapeId: $scrapeId\n";
```

### Step 2 — Send interactions

```php theme={null}
<?php
$interactUrl = "https://api.firecrawl.dev/v2/scrape/$scrapeId/interact";
$headers = [
    'Authorization: Bearer ' . $apiKey,
    'Content-Type: application/json',
];

// Search for a product
$ch = curl_init($interactUrl);
curl_setopt_array($ch, [
    CURLOPT_RETURNTRANSFER => true,
    CURLOPT_POST => true,
    CURLOPT_HTTPHEADER => $headers,
    CURLOPT_POSTFIELDS => json_encode([
        'prompt' => 'Search for iPhone 16 Pro Max',
    ]),
]);

$response = curl_exec($ch);
curl_close($ch);
echo $response . "\n";

// Click on the first result
$ch = curl_init($interactUrl);
curl_setopt_array($ch, [
    CURLOPT_RETURNTRANSFER => true,
    CURLOPT_POST => true,
    CURLOPT_HTTPHEADER => $headers,
    CURLOPT_POSTFIELDS => json_encode([
        'prompt' => 'Click on the first result and tell me the price',
    ]),
]);

$response = curl_exec($ch);
curl_close($ch);
echo $response . "\n";
```

### Step 3 — Stop the session

```php theme={null}
<?php
$ch = curl_init($interactUrl);
curl_setopt_array($ch, [
    CURLOPT_RETURNTRANSFER => true,
    CURLOPT_CUSTOMREQUEST => 'DELETE',
    CURLOPT_HTTPHEADER => [
        'Authorization: Bearer ' . $apiKey,
    ],
]);

curl_exec($ch);
curl_close($ch);

echo "Session stopped\n";
```

## Reusable helper class

For repeated use, wrap the API in a class:

```php theme={null}
<?php
class Firecrawl
{
    private string $apiKey;
    private string $baseUrl = 'https://api.firecrawl.dev/v2';

    public function __construct(string $apiKey)
    {
        $this->apiKey = $apiKey;
    }

    private function post(string $endpoint, array $payload): array
    {
        $ch = curl_init($this->baseUrl . $endpoint);
        curl_setopt_array($ch, [
            CURLOPT_RETURNTRANSFER => true,
            CURLOPT_POST => true,
            CURLOPT_HTTPHEADER => [
                'Authorization: Bearer ' . $this->apiKey,
                'Content-Type: application/json',
            ],
            CURLOPT_POSTFIELDS => json_encode($payload),
        ]);

        $response = curl_exec($ch);
        $httpCode = curl_getinfo($ch, CURLINFO_HTTP_CODE);
        curl_close($ch);

        if ($httpCode >= 400) {
            throw new \RuntimeException("Firecrawl API error: HTTP $httpCode");
        }

        return json_decode($response, true);
    }

    public function scrape(string $url, array $options = []): array
    {
        return $this->post('/scrape', array_merge(['url' => $url], $options));
    }

    public function search(string $query, int $limit = 5): array
    {
        return $this->post('/search', ['query' => $query, 'limit' => $limit]);
    }
}

// Usage
$app = new Firecrawl(getenv('FIRECRAWL_API_KEY'));
$result = $app->scrape('https://example.com');
echo $result['data']['markdown'];
```

## Next steps

<CardGroup>
  <Card title="Search docs" icon="magnifying-glass" href="/features/search">
    Search the web and get full page content
  </Card>

  <Card title="Scrape docs" icon="file-lines" href="/features/scrape">
    All scrape options including formats, actions, and proxies
  </Card>

  <Card title="Interact docs" icon="hand-pointer" href="/features/interact">
    Click, fill forms, and extract dynamic content
  </Card>

  <Card title="API Reference" icon="code" href="/api-reference/v2-introduction">
    Complete REST API documentation
  </Card>
</CardGroup>


# Python
Source: https://docs.firecrawl.dev/quickstarts/python

Get started with Firecrawl in Python. Scrape, search, and interact with web data using the official SDK.

## Prerequisites

* Python 3.8+
* A Firecrawl API key — [get one free](https://www.firecrawl.dev/app/api-keys)

## Install the SDK

```bash theme={null}
pip install firecrawl-py
```

## Search the web

```python theme={null}
from firecrawl import Firecrawl

app = Firecrawl(api_key="fc-YOUR-API-KEY")
results = app.search("firecrawl web scraping", limit=5)

for result in results.web:
    print(result.title, result.url)
```

## Scrape a page

```python theme={null}
result = app.scrape("https://example.com")
print(result.markdown)
```

<Accordion title="Example response">
  ```json theme={null}
  {
    "markdown": "# Example Domain\n\nThis domain is for use in illustrative examples...",
    "metadata": {
      "title": "Example Domain",
      "sourceURL": "https://example.com"
    }
  }
  ```
</Accordion>

## Interact with a page

Use Interact to control a live browser session — click buttons, fill forms, and extract dynamic content.

```python theme={null}
result = app.scrape("https://www.amazon.com", formats=["markdown"])
scrape_id = result.metadata.scrape_id

app.interact(scrape_id, prompt="Search for iPhone 16 Pro Max")
response = app.interact(scrape_id, prompt="Click on the first result and tell me the price")
print(response.output)

app.stop_interaction(scrape_id)
```

## Environment variable

Instead of passing `api_key` directly, set the `FIRECRAWL_API_KEY` environment variable:

```bash theme={null}
export FIRECRAWL_API_KEY=fc-YOUR-API-KEY
```

```python theme={null}
app = Firecrawl()
```

## Next steps

<CardGroup>
  <Card title="Scrape docs" icon="file-lines" href="/features/scrape">
    All scrape options including formats, actions, and proxies
  </Card>

  <Card title="Search docs" icon="magnifying-glass" href="/features/search">
    Search the web and get full page content
  </Card>

  <Card title="Interact docs" icon="hand-pointer" href="/features/interact">
    Click, fill forms, and extract dynamic content
  </Card>

  <Card title="Python SDK reference" icon="python" href="/sdks/python">
    Full SDK reference with crawl, map, async, and more
  </Card>
</CardGroup>


# Rails
Source: https://docs.firecrawl.dev/quickstarts/rails

Use Firecrawl with Ruby on Rails to search, scrape, and interact with web data using the REST API.

## Prerequisites

* Ruby 3.0+ and Rails 7+
* A Firecrawl API key — [get one free](https://www.firecrawl.dev/app/api-keys)

## Configuration

Add your API key to your Rails credentials or environment:

```bash theme={null}
export FIRECRAWL_API_KEY=fc-YOUR-API-KEY
```

## Create a service

Create `app/services/firecrawl_service.rb`:

```ruby theme={null}
require "net/http"
require "json"
require "uri"

class FirecrawlService
  BASE_URL = "https://api.firecrawl.dev/v2"

  def initialize(api_key: ENV.fetch("FIRECRAWL_API_KEY"))
    @api_key = api_key
  end

  def search(query, limit: 5)
    post("/search", { query: query, limit: limit })
  end

  def scrape(url, **options)
    post("/scrape", { url: url }.merge(options))
  end

  def interact(url, prompt, follow_up: nil)
    # 1. Scrape to open a browser session
    scrape_result = scrape(url, formats: ["markdown"])
    scrape_id = scrape_result.dig("data", "metadata", "scrapeId")

    # 2. Send first prompt
    post("/scrape/#{scrape_id}/interact", { prompt: prompt })

    # 3. Send follow-up prompt
    result = nil
    if follow_up
      result = post("/scrape/#{scrape_id}/interact", { prompt: follow_up })
    end

    # 4. Close the session
    delete("/scrape/#{scrape_id}/interact")

    result || scrape_result
  end

  private

  def post(endpoint, payload)
    uri = URI("#{BASE_URL}#{endpoint}")
    request = Net::HTTP::Post.new(uri)
    request["Authorization"] = "Bearer #{@api_key}"
    request["Content-Type"] = "application/json"
    request.body = payload.to_json

    response = Net::HTTP.start(uri.hostname, uri.port, use_ssl: true) do |http|
      http.request(request)
    end

    JSON.parse(response.body)
  end

  def delete(endpoint)
    uri = URI("#{BASE_URL}#{endpoint}")
    request = Net::HTTP::Delete.new(uri)
    request["Authorization"] = "Bearer #{@api_key}"

    Net::HTTP.start(uri.hostname, uri.port, use_ssl: true) do |http|
      http.request(request)
    end
  end
end
```

## Create a controller

Generate a controller:

```bash theme={null}
rails generate controller Firecrawl search scrape interact --skip-routes
```

Edit `app/controllers/firecrawl_controller.rb`:

```ruby theme={null}
class FirecrawlController < ApplicationController
  skip_before_action :verify_authenticity_token

  def search
    service = FirecrawlService.new
    result = service.search(params.require(:query), limit: params.fetch(:limit, 5).to_i)
    render json: result
  end

  def scrape
    service = FirecrawlService.new
    result = service.scrape(params.require(:url))
    render json: result
  end

  def interact
    service = FirecrawlService.new
    result = service.interact(
      params.require(:url),
      params.require(:prompt),
      follow_up: params[:followUp]
    )
    render json: result
  end
end
```

## Add routes

In `config/routes.rb`:

```ruby theme={null}
Rails.application.routes.draw do
  post "api/search", to: "firecrawl#search"
  post "api/scrape", to: "firecrawl#scrape"
  post "api/interact", to: "firecrawl#interact"
end
```

## Test it

```bash theme={null}
rails server

# Search the web
curl -X POST http://localhost:3000/api/search \
  -H "Content-Type: application/json" \
  -d '{"query": "firecrawl web scraping"}'

# Scrape a page
curl -X POST http://localhost:3000/api/scrape \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com"}'

# Interact with a page
curl -X POST http://localhost:3000/api/interact \
  -H "Content-Type: application/json" \
  -d '{"url": "https://www.amazon.com", "prompt": "Search for iPhone 16 Pro Max", "followUp": "Click on the first result and tell me the price"}'
```

## Next steps

<CardGroup>
  <Card title="Search docs" icon="magnifying-glass" href="/features/search">
    Search the web and get full page content
  </Card>

  <Card title="Scrape docs" icon="file-lines" href="/features/scrape">
    All scrape options including formats, actions, and proxies
  </Card>

  <Card title="Interact docs" icon="hand-pointer" href="/features/interact">
    Click, fill forms, and extract dynamic content
  </Card>

  <Card title="API Reference" icon="code" href="/api-reference/v2-introduction">
    Complete REST API documentation
  </Card>
</CardGroup>


# Remix
Source: https://docs.firecrawl.dev/quickstarts/remix

Use Firecrawl with Remix to scrape, search, and interact with web data in your full-stack React app.

## Prerequisites

* Remix project
* A Firecrawl API key — [get one free](https://www.firecrawl.dev/app/api-keys)

## Install the SDK

```bash theme={null}
npm install @mendable/firecrawl-js
```

Add your API key to `.env`:

```bash theme={null}
FIRECRAWL_API_KEY=fc-YOUR-API-KEY
```

## Search the web

Use Firecrawl in an `action` to handle form submissions. Create `app/routes/search.tsx`:

```tsx theme={null}
import { json, type ActionFunctionArgs } from "@remix-run/node";
import { Form, useActionData } from "@remix-run/react";
import Firecrawl from "@mendable/firecrawl-js";

const firecrawl = new Firecrawl({ apiKey: process.env.FIRECRAWL_API_KEY });

export async function action({ request }: ActionFunctionArgs) {
  const formData = await request.formData();
  const query = formData.get("query") as string;
  const results = await firecrawl.search(query, { limit: 5 });
  return json({ results: (results.web || []).map((r) => ({ title: r.title, url: r.url })) });
}

export default function SearchPage() {
  const data = useActionData<typeof action>();

  return (
    <div>
      <Form method="post">
        <input name="query" placeholder="Search the web..." />
        <button type="submit">Search</button>
      </Form>
      {data?.results?.map((r, i) => (
        <div key={i}>
          <a href={r.url}>{r.title}</a>
        </div>
      ))}
    </div>
  );
}
```

## Scrape a page

Use Firecrawl in a `loader` to fetch data at request time. Create `app/routes/scrape.tsx`:

```tsx theme={null}
import { json, type LoaderFunctionArgs } from "@remix-run/node";
import { useLoaderData } from "@remix-run/react";
import Firecrawl from "@mendable/firecrawl-js";

const firecrawl = new Firecrawl({ apiKey: process.env.FIRECRAWL_API_KEY });

export async function loader({ request }: LoaderFunctionArgs) {
  const url = new URL(request.url);
  const target = url.searchParams.get("url");
  if (!target) return json({ markdown: null });

  const result = await firecrawl.scrape(target);
  return json({ markdown: result.markdown });
}

export default function ScrapePage() {
  const { markdown } = useLoaderData<typeof loader>();

  return (
    <div>
      <h1>Scraped Content</h1>
      {markdown ? <pre>{markdown}</pre> : <p>Pass ?url= to scrape a page</p>}
    </div>
  );
}
```

## Interact with a page

Use interact to control a live browser session — click buttons, fill forms, and extract dynamic content. Create `app/routes/interact.tsx`:

```tsx theme={null}
import { json, type ActionFunctionArgs } from "@remix-run/node";
import { Form, useActionData } from "@remix-run/react";
import Firecrawl from "@mendable/firecrawl-js";

const firecrawl = new Firecrawl({ apiKey: process.env.FIRECRAWL_API_KEY });

export async function action({ request }: ActionFunctionArgs) {
  const formData = await request.formData();
  const url = formData.get("url") as string;

  const result = await firecrawl.scrape(url, { formats: ['markdown'] });
  const scrapeId = result.metadata?.scrapeId;

  await firecrawl.interact(scrapeId, { prompt: 'Search for iPhone 16 Pro Max' });
  const response = await firecrawl.interact(scrapeId, { prompt: 'Click on the first result and tell me the price' });

  await firecrawl.stopInteraction(scrapeId);

  return json({ output: response.output });
}

export default function InteractPage() {
  const data = useActionData<typeof action>();

  return (
    <div>
      <Form method="post">
        <input name="url" placeholder="URL to interact with..." />
        <button type="submit">Interact</button>
      </Form>
      {data?.output && <pre>{data.output}</pre>}
    </div>
  );
}
```

## Next steps

<CardGroup>
  <Card title="Scrape docs" icon="file-lines" href="/features/scrape">
    All scrape options including formats, actions, and proxies
  </Card>

  <Card title="Search docs" icon="magnifying-glass" href="/features/search">
    Search the web and get full page content
  </Card>

  <Card title="Interact docs" icon="hand-pointer" href="/features/interact">
    Click, fill forms, and extract dynamic content
  </Card>

  <Card title="Node SDK reference" icon="node" href="/sdks/node">
    Full SDK reference with crawl, map, batch scrape, and more
  </Card>
</CardGroup>


# Ruby
Source: https://docs.firecrawl.dev/quickstarts/ruby

Get started with Firecrawl in Ruby. Search, scrape, and interact with web data using the REST API.

## Prerequisites

* Ruby 3.0+
* A Firecrawl API key — [get one free](https://www.firecrawl.dev/app/api-keys)

## Search the web

Firecrawl works with Ruby through the REST API using `net/http`.

```ruby theme={null}
require "net/http"
require "json"
require "uri"

api_key = ENV.fetch("FIRECRAWL_API_KEY")

uri = URI("https://api.firecrawl.dev/v2/search")
request = Net::HTTP::Post.new(uri)
request["Authorization"] = "Bearer #{api_key}"
request["Content-Type"] = "application/json"
request.body = { query: "firecrawl web scraping", limit: 5 }.to_json

response = Net::HTTP.start(uri.hostname, uri.port, use_ssl: true) { |http| http.request(request) }
results = JSON.parse(response.body)

results["data"]["web"].each do |result|
  puts "#{result['title']} - #{result['url']}"
end
```

## Scrape a page

```ruby theme={null}
uri = URI("https://api.firecrawl.dev/v2/scrape")
request = Net::HTTP::Post.new(uri)
request["Authorization"] = "Bearer #{api_key}"
request["Content-Type"] = "application/json"
request.body = { url: "https://example.com" }.to_json

response = Net::HTTP.start(uri.hostname, uri.port, use_ssl: true) { |http| http.request(request) }
data = JSON.parse(response.body)

puts data.dig("data", "markdown")
```

<Accordion title="Example response">
  ```json theme={null}
  {
    "success": true,
    "data": {
      "markdown": "# Example Domain\n\nThis domain is for use in illustrative examples...",
      "metadata": {
        "title": "Example Domain",
        "sourceURL": "https://example.com"
      }
    }
  }
  ```
</Accordion>

## Interact with a page

Scrape a page, then keep working with it using natural language prompts.

```ruby theme={null}
uri = URI("https://api.firecrawl.dev/v2/scrape")
request = Net::HTTP::Post.new(uri)
request["Authorization"] = "Bearer #{api_key}"
request["Content-Type"] = "application/json"
request.body = { url: "https://www.amazon.com", formats: ["markdown"] }.to_json

response = Net::HTTP.start(uri.hostname, uri.port, use_ssl: true) { |http| http.request(request) }
scrape_id = JSON.parse(response.body).dig("data", "metadata", "scrapeId")

interact_uri = URI("https://api.firecrawl.dev/v2/scrape/#{scrape_id}/interact")
interact_req = Net::HTTP::Post.new(interact_uri)
interact_req["Authorization"] = "Bearer #{api_key}"
interact_req["Content-Type"] = "application/json"
interact_req.body = { prompt: "Search for iPhone 16 Pro Max" }.to_json

interact_resp = Net::HTTP.start(interact_uri.hostname, interact_uri.port, use_ssl: true) { |http| http.request(interact_req) }
puts JSON.parse(interact_resp.body)

# Stop the session
delete_uri = URI("https://api.firecrawl.dev/v2/scrape/#{scrape_id}/interact")
delete_req = Net::HTTP::Delete.new(delete_uri)
delete_req["Authorization"] = "Bearer #{api_key}"
Net::HTTP.start(delete_uri.hostname, delete_uri.port, use_ssl: true) { |http| http.request(delete_req) }
```

## Next steps

<CardGroup>
  <Card title="Scrape docs" icon="file-lines" href="/features/scrape">
    All scrape options including formats, actions, and proxies
  </Card>

  <Card title="Search docs" icon="magnifying-glass" href="/features/search">
    Search the web and get full page content
  </Card>

  <Card title="API Reference" icon="code" href="/api-reference/v2-introduction">
    Complete REST API documentation
  </Card>

  <Card title="Interact docs" icon="hand-pointer" href="/features/interact">
    Click, fill forms, and extract dynamic content
  </Card>
</CardGroup>


# Rust
Source: https://docs.firecrawl.dev/quickstarts/rust

Get started with Firecrawl in Rust. Search, scrape, and interact with web data using the official SDK.

## Prerequisites

* Rust 1.70+ with Cargo
* A Firecrawl API key — [get one free](https://www.firecrawl.dev/app/api-keys)

## Install the crate

Add `firecrawl` to your `Cargo.toml`:

```toml theme={null}
[dependencies]
firecrawl = "2"
tokio = { version = "1", features = ["full"] }
serde_json = "1"
```

## Search the web

```rust theme={null}
use firecrawl::{Client, SearchOptions};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let client = Client::new("fc-YOUR-API-KEY")?;

    let results = client.search(
        "firecrawl web scraping",
        SearchOptions { limit: Some(5), ..Default::default() },
    ).await?;

    if let Some(web) = results.data.web {
        for item in web {
            if let firecrawl::SearchResultOrDocument::WebResult(r) = item {
                println!("{} - {}", r.url, r.title.unwrap_or_default());
            }
        }
    }
    Ok(())
}
```

## Scrape a page

```rust theme={null}
let doc = client.scrape("https://example.com", None).await?;
println!("{}", doc.markdown.unwrap_or_default());
```

<Accordion title="Example response">
  ```json theme={null}
  {
    "markdown": "# Example Domain\n\nThis domain is for use in illustrative examples...",
    "metadata": {
      "title": "Example Domain",
      "sourceURL": "https://example.com"
    }
  }
  ```
</Accordion>

## Interact with a page

Scrape a page to get a `scrapeId`, then use the interact API to control the browser session:

```rust theme={null}
use firecrawl::{Client, ScrapeOptions, Format, ScrapeExecuteOptions};

let doc = client.scrape(
    "https://www.amazon.com",
    ScrapeOptions {
        formats: Some(vec![Format::Markdown]),
        ..Default::default()
    },
).await?;

let scrape_id = doc.metadata
    .as_ref()
    .and_then(|m| m.scrape_id.as_deref())
    .expect("scrapeId not found");

// Send a prompt to interact with the page
let run = client.interact(
    scrape_id,
    ScrapeExecuteOptions {
        prompt: Some("Search for iPhone 16 Pro Max".to_string()),
        ..Default::default()
    },
).await?;

let run = client.interact(
    scrape_id,
    ScrapeExecuteOptions {
        prompt: Some("Click on the first result and tell me the price".to_string()),
        ..Default::default()
    },
).await?;

println!("{:?}", run.output);

// Close the session
client.stop_interaction(scrape_id).await?;
```

## Environment variable

Set `FIRECRAWL_API_KEY` instead of passing the key directly:

```bash theme={null}
export FIRECRAWL_API_KEY=fc-YOUR-API-KEY
```

```rust theme={null}
let api_key = std::env::var("FIRECRAWL_API_KEY")?;
let client = Client::new(api_key)?;
```

## Next steps

<CardGroup>
  <Card title="Search docs" icon="magnifying-glass" href="/features/search">
    Search the web and get full page content
  </Card>

  <Card title="Scrape docs" icon="file-lines" href="/features/scrape">
    All scrape options including formats, actions, and proxies
  </Card>

  <Card title="Interact docs" icon="hand-pointer" href="/features/interact">
    Click, fill forms, and extract dynamic content
  </Card>

  <Card title="Rust SDK reference" icon="rust" href="/sdks/rust">
    Full SDK reference with crawl, map, batch scrape, and more
  </Card>
</CardGroup>


# Spring Boot
Source: https://docs.firecrawl.dev/quickstarts/spring-boot

Use Firecrawl with Spring Boot to search, scrape, and interact with web data using the official Java SDK.

## Prerequisites

* Java 17+ and Spring Boot 3+
* A Firecrawl API key — [get one free](https://www.firecrawl.dev/app/api-keys)

## Add the dependency

<Tabs>
  <Tab title="Gradle (Kotlin DSL)">
    ```kotlin theme={null}
    dependencies {
        implementation("com.firecrawl:firecrawl-java:1.2.0")
    }
    ```
  </Tab>

  <Tab title="Maven">
    ```xml theme={null}
    <dependency>
        <groupId>com.firecrawl</groupId>
        <artifactId>firecrawl-java</artifactId>
        <version>1.2.0</version>
    </dependency>
    ```
  </Tab>
</Tabs>

## Configuration

Add your API key to `application.properties`:

```properties theme={null}
firecrawl.api-key=${FIRECRAWL_API_KEY}
```

Or set it as an environment variable:

```bash theme={null}
export FIRECRAWL_API_KEY=fc-YOUR-API-KEY
```

## Create a configuration bean

Create `FirecrawlConfig.java`:

```java theme={null}
import com.firecrawl.client.FirecrawlClient;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration
public class FirecrawlConfig {

    @Bean
    public FirecrawlClient firecrawlClient(
            @Value("${firecrawl.api-key}") String apiKey) {
        return FirecrawlClient.builder()
            .apiKey(apiKey)
            .build();
    }
}
```

## Create a REST controller

Create `FirecrawlController.java`:

```java theme={null}
import com.firecrawl.client.FirecrawlClient;
import com.firecrawl.models.Document;
import com.firecrawl.models.SearchData;
import com.firecrawl.models.SearchOptions;
import com.firecrawl.models.ScrapeOptions;
import com.firecrawl.models.BrowserExecuteResponse;
import org.springframework.web.bind.annotation.*;

import java.util.List;
import java.util.Map;

@RestController
@RequestMapping("/api")
public class FirecrawlController {

    private final FirecrawlClient firecrawl;

    public FirecrawlController(FirecrawlClient firecrawl) {
        this.firecrawl = firecrawl;
    }

    @PostMapping("/search")
    public SearchData search(@RequestBody Map<String, Object> body) {
        return firecrawl.search(
            (String) body.get("query"),
            SearchOptions.builder()
                .limit((int) body.getOrDefault("limit", 5))
                .build()
        );
    }

    @PostMapping("/scrape")
    public Map<String, Object> scrape(@RequestBody Map<String, String> body) {
        Document doc = firecrawl.scrape(body.get("url"));
        return Map.of(
            "markdown", doc.getMarkdown(),
            "metadata", doc.getMetadata()
        );
    }

    @PostMapping("/interact")
    public Map<String, Object> interact(@RequestBody Map<String, String> body) {
        Document doc = firecrawl.scrape(body.get("url"),
            ScrapeOptions.builder().formats(List.of((Object) "markdown")).build());
        String scrapeId = (String) doc.getMetadata().get("scrapeId");

        BrowserExecuteResponse response = firecrawl.interact(scrapeId,
            body.getOrDefault("code", "const title = await page.title(); console.log(title);"));

        firecrawl.stopInteractiveBrowser(scrapeId);

        return Map.of("result", response.getStdout());
    }
}
```

## Run it

```bash theme={null}
./gradlew bootRun
```

## Test it

```bash theme={null}
# Search the web
curl -X POST http://localhost:8080/api/search \
  -H "Content-Type: application/json" \
  -d '{"query": "firecrawl web scraping"}'

# Scrape a page
curl -X POST http://localhost:8080/api/scrape \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com"}'

# Interact with a page
curl -X POST http://localhost:8080/api/interact \
  -H "Content-Type: application/json" \
  -d '{"url": "https://www.amazon.com", "code": "const title = await page.title(); console.log(title);"}'
```

## Next steps

<CardGroup>
  <Card title="Search docs" icon="magnifying-glass" href="/features/search">
    Search the web and get full page content
  </Card>

  <Card title="Scrape docs" icon="file-lines" href="/features/scrape">
    All scrape options including formats, actions, and proxies
  </Card>

  <Card title="Interact docs" icon="hand-pointer" href="/features/interact">
    Click, fill forms, and extract dynamic content
  </Card>

  <Card title="Java SDK reference" icon="java" href="/sdks/java">
    Full SDK reference with crawl, map, batch scrape, and more
  </Card>
</CardGroup>


# Supabase Edge Functions
Source: https://docs.firecrawl.dev/quickstarts/supabase-edge-functions

Use Firecrawl with Supabase Edge Functions to search, scrape, and interact with web data at the edge.

## Prerequisites

* Supabase project with CLI (`supabase init`)
* A Firecrawl API key — [get one free](https://www.firecrawl.dev/app/api-keys)

## Setup

```bash theme={null}
supabase functions new firecrawl-search
supabase functions new firecrawl-scrape
supabase functions new firecrawl-interact
```

Set the secret:

```bash theme={null}
supabase secrets set FIRECRAWL_API_KEY=fc-YOUR-API-KEY
```

## Search the web

Edit `supabase/functions/firecrawl-search/index.ts`:

```typescript theme={null}
import Firecrawl from "npm:@mendable/firecrawl-js";

const firecrawl = new Firecrawl({
  apiKey: Deno.env.get("FIRECRAWL_API_KEY"),
});

Deno.serve(async (req) => {
  const { query } = await req.json();
  const results = await firecrawl.search(query, { limit: 5 });

  return new Response(JSON.stringify(results), {
    headers: { "Content-Type": "application/json" },
  });
});
```

## Scrape a page

Edit `supabase/functions/firecrawl-scrape/index.ts`:

```typescript theme={null}
import Firecrawl from "npm:@mendable/firecrawl-js";

const firecrawl = new Firecrawl({
  apiKey: Deno.env.get("FIRECRAWL_API_KEY"),
});

Deno.serve(async (req) => {
  const { url } = await req.json();
  const result = await firecrawl.scrape(url);

  return new Response(JSON.stringify(result), {
    headers: { "Content-Type": "application/json" },
  });
});
```

## Interact with a page

Edit `supabase/functions/firecrawl-interact/index.ts`:

```typescript theme={null}
import Firecrawl from "npm:@mendable/firecrawl-js";

const firecrawl = new Firecrawl({
  apiKey: Deno.env.get("FIRECRAWL_API_KEY"),
});

Deno.serve(async (_req) => {
  const result = await firecrawl.scrape("https://www.amazon.com", {
    formats: ["markdown"],
  });
  const scrapeId = result.metadata?.scrapeId;

  await firecrawl.interact(scrapeId, {
    prompt: "Search for iPhone 16 Pro Max",
  });
  const response = await firecrawl.interact(scrapeId, {
    prompt: "Click on the first result and tell me the price",
  });
  console.log(response.output);

  await firecrawl.stopInteraction(scrapeId);

  return new Response(JSON.stringify({ output: response.output }), {
    headers: { "Content-Type": "application/json" },
  });
});
```

## Deploy

```bash theme={null}
supabase functions deploy firecrawl-search
supabase functions deploy firecrawl-scrape
supabase functions deploy firecrawl-interact
```

## Test it

```bash theme={null}
curl -X POST https://<project-ref>.supabase.co/functions/v1/firecrawl-search \
  -H "Authorization: Bearer <ANON_KEY>" \
  -H "Content-Type: application/json" \
  -d '{"query": "firecrawl web scraping"}'
```

## Next steps

<CardGroup>
  <Card title="Search docs" icon="magnifying-glass" href="/features/search">
    Search the web and get full page content
  </Card>

  <Card title="Scrape docs" icon="file-lines" href="/features/scrape">
    All scrape options including formats, actions, and proxies
  </Card>

  <Card title="Interact docs" icon="hand-pointer" href="/features/interact">
    Click, fill forms, and extract dynamic content
  </Card>

  <Card title="Node SDK reference" icon="node" href="/sdks/node">
    Full SDK reference with crawl, map, batch scrape, and more
  </Card>
</CardGroup>


# SvelteKit
Source: https://docs.firecrawl.dev/quickstarts/sveltekit

Use Firecrawl with SvelteKit to scrape, search, and interact with web data in your Svelte application.

## Prerequisites

* SvelteKit project
* A Firecrawl API key — [get one free](https://www.firecrawl.dev/app/api-keys)

## Install the SDK

```bash theme={null}
npm install @mendable/firecrawl-js
```

Add your API key to `.env`:

```bash theme={null}
FIRECRAWL_API_KEY=fc-YOUR-API-KEY
```

## Search the web

Create a form action in `src/routes/search/+page.server.ts`:

```typescript theme={null}
import Firecrawl from "@mendable/firecrawl-js";
import { FIRECRAWL_API_KEY } from "$env/static/private";

const firecrawl = new Firecrawl({ apiKey: FIRECRAWL_API_KEY });

export const actions = {
  default: async ({ request }) => {
    const data = await request.formData();
    const query = data.get("query") as string;
    const results = await firecrawl.search(query, { limit: 5 });
    return { results: (results.web || []).map((r) => ({ title: r.title, url: r.url })) };
  },
};
```

Wire it up in `src/routes/search/+page.svelte`:

```svelte theme={null}
<script>
  export let form;
</script>

<form method="POST">
  <input name="query" placeholder="Search the web..." />
  <button>Search</button>
</form>

{#if form?.results}
  {#each form.results as result}
    <div><a href={result.url}>{result.title}</a></div>
  {/each}
{/if}
```

## Scrape a page

Fetch data in a load function at `src/routes/scrape/+page.server.ts`:

```typescript theme={null}
import Firecrawl from "@mendable/firecrawl-js";
import { FIRECRAWL_API_KEY } from "$env/static/private";

const firecrawl = new Firecrawl({ apiKey: FIRECRAWL_API_KEY });

export async function load({ url }) {
  const target = url.searchParams.get("url");
  if (!target) return { markdown: null };

  const result = await firecrawl.scrape(target);
  return { markdown: result.markdown };
}
```

Display it in `src/routes/scrape/+page.svelte`:

```svelte theme={null}
<script>
  export let data;
</script>

{#if data.markdown}
  <pre>{data.markdown}</pre>
{:else}
  <p>Pass ?url= to scrape a page</p>
{/if}
```

## Interact with a page

Create a server endpoint at `src/routes/api/interact/+server.ts`:

```typescript theme={null}
import { json } from "@sveltejs/kit";
import Firecrawl from "@mendable/firecrawl-js";
import { FIRECRAWL_API_KEY } from "$env/static/private";

const firecrawl = new Firecrawl({ apiKey: FIRECRAWL_API_KEY });

export async function POST() {
  const result = await firecrawl.scrape("https://www.amazon.com", {
    formats: ["markdown"],
  });

  const scrapeId = result.metadata?.scrapeId;

  await firecrawl.interact(scrapeId, {
    prompt: "Search for iPhone 16 Pro Max",
  });

  const response = await firecrawl.interact(scrapeId, {
    prompt: "Click on the first result and tell me the price",
  });

  await firecrawl.stopInteraction(scrapeId);

  return json({ output: response.output });
}
```

## Next steps

<CardGroup>
  <Card title="Scrape docs" icon="file-lines" href="/features/scrape">
    All scrape options including formats, actions, and proxies
  </Card>

  <Card title="Search docs" icon="magnifying-glass" href="/features/search">
    Search the web and get full page content
  </Card>

  <Card title="Interact docs" icon="hand-pointer" href="/features/interact">
    Click, fill forms, and extract dynamic content
  </Card>

  <Card title="Node SDK reference" icon="node" href="/sdks/node">
    Full SDK reference with crawl, map, batch scrape, and more
  </Card>
</CardGroup>


# Vercel Functions
Source: https://docs.firecrawl.dev/quickstarts/vercel-functions

Use Firecrawl with Vercel Functions to search, scrape, and interact with web data in serverless deployments.

## Prerequisites

* Vercel project (Next.js, SvelteKit, Nuxt, or standalone)
* A Firecrawl API key — [get one free](https://www.firecrawl.dev/app/api-keys)

## Setup

```bash theme={null}
npm install @mendable/firecrawl-js
```

Add `FIRECRAWL_API_KEY` as an environment variable in your Vercel project settings, or in `.env.local` for local development:

```bash theme={null}
FIRECRAWL_API_KEY=fc-YOUR-API-KEY
```

## Search the web

Create `api/search.ts` (or `app/api/search/route.ts` for Next.js):

```typescript theme={null}
import Firecrawl from "@mendable/firecrawl-js";

const firecrawl = new Firecrawl({ apiKey: process.env.FIRECRAWL_API_KEY });

export async function POST(request: Request) {
  const { query } = await request.json();
  const results = await firecrawl.search(query, { limit: 5 });

  return new Response(JSON.stringify(results), {
    headers: { "Content-Type": "application/json" },
  });
}
```

## Scrape a page

Create `api/scrape.ts` (or `app/api/scrape/route.ts` for Next.js):

```typescript theme={null}
import Firecrawl from "@mendable/firecrawl-js";

const firecrawl = new Firecrawl({ apiKey: process.env.FIRECRAWL_API_KEY });

export async function POST(request: Request) {
  const { url } = await request.json();
  const result = await firecrawl.scrape(url);

  return new Response(JSON.stringify(result), {
    headers: { "Content-Type": "application/json" },
  });
}
```

## Interact with a page

Create `api/interact.ts` (or `app/api/interact/route.ts` for Next.js):

```typescript theme={null}
import Firecrawl from "@mendable/firecrawl-js";

const firecrawl = new Firecrawl({ apiKey: process.env.FIRECRAWL_API_KEY });

export async function POST(request: Request) {
  const result = await firecrawl.scrape("https://www.amazon.com", {
    formats: ["markdown"],
  });
  const scrapeId = result.metadata?.scrapeId;

  await firecrawl.interact(scrapeId, {
    prompt: "Search for iPhone 16 Pro Max",
  });
  const response = await firecrawl.interact(scrapeId, {
    prompt: "Click on the first result and tell me the price",
  });

  await firecrawl.stopInteraction(scrapeId);

  return new Response(JSON.stringify({ output: response.output }), {
    headers: { "Content-Type": "application/json" },
  });
}
```

## Deploy

```bash theme={null}
vercel deploy
```

## Test it

```bash theme={null}
curl -X POST https://your-project.vercel.app/api/search \
  -H "Content-Type: application/json" \
  -d '{"query": "firecrawl web scraping"}'
```

<Note>
  Vercel Functions have a default timeout of 10 seconds on the Hobby plan and 60 seconds on Pro. For large crawl jobs, use the Firecrawl async API with webhooks instead.
</Note>

## Next steps

<CardGroup>
  <Card title="Search docs" icon="magnifying-glass" href="/features/search">
    Search the web and get full page content
  </Card>

  <Card title="Scrape docs" icon="file-lines" href="/features/scrape">
    All scrape options including formats, actions, and proxies
  </Card>

  <Card title="Interact docs" icon="hand-pointer" href="/features/interact">
    Click, fill forms, and extract dynamic content
  </Card>

  <Card title="Node SDK reference" icon="node" href="/sdks/node">
    Full SDK reference with crawl, map, batch scrape, and more
  </Card>
</CardGroup>


# AI Platforms
Source: https://docs.firecrawl.dev/use-cases/ai-platforms

Power AI assistants and let customers build AI apps

AI platform builders and teams use Firecrawl to power knowledge bases, chatbots, and enable customers to build AI applications with web data.

## Start with a Template

<Card title="Firestarter" icon="github" href="https://github.com/firecrawl/firestarter">
  Instant AI chatbots for websites with web knowledge integration
</Card>

<Note>
  **Get started with templates and examples.** Build AI-powered applications with web data.
</Note>

## How It Works

Transform websites into AI-ready data. Power chatbots with real-time web knowledge, build RAG systems with up-to-date documentation, and enable your users to connect their AI applications to web sources.

## Why AI Platforms Choose Firecrawl

### Reduce Hallucinations with Real-Time Data

Your AI assistants need current information, not outdated training data. Whether it's domain-specific knowledge, technical documentation, or industry-specific content, Firecrawl ensures your knowledge bases stay synchronized with the latest updates-reducing hallucinations and improving response accuracy.

## Customer Stories

<CardGroup>
  <Card href="https://www.firecrawl.dev/blog/how-replit-uses-firecrawl-to-power-ai-agents">
    **Replit**

    Learn how Replit leverages Firecrawl to keep Replit Agent up-to-date with the latest API documentation and web content.
  </Card>

  <Card href="https://www.firecrawl.dev/blog/how-stack-ai-uses-firecrawl-to-power-ai-agents">
    **Stack AI**

    Discover how Stack AI uses Firecrawl to seamlessly feed agentic AI workflows with high-quality web data.
  </Card>
</CardGroup>

## FAQs

<AccordionGroup>
  <Accordion title="How does Firecrawl integrate with AI development platforms?">
    Firecrawl provides simple APIs and SDKs that integrate directly into AI platforms. Whether you're building with LangChain, using no-code tools like n8n, or custom frameworks, Firecrawl delivers clean, structured web data ready for AI consumption.
  </Accordion>

  <Accordion title="Can Firecrawl handle the scale required for AI training?">
    Yes. Firecrawl is designed for enterprise-scale data extraction, processing millions of pages for AI training datasets. Our infrastructure scales automatically to meet your needs.
  </Accordion>

  <Accordion title="What data formats does Firecrawl provide for AI platforms?">
    Firecrawl delivers data in AI-friendly formats including clean markdown, structured JSON, raw HTML, extracted images, screenshots, and news content. This flexibility ensures compatibility with any AI platform's data ingestion requirements.
  </Accordion>

  <Accordion title="Can I use Firecrawl for real-time AI applications?">
    Yes! Our API supports real-time data extraction, enabling AI applications to access fresh web data on-demand. This is perfect for AI agents that need current information to make decisions.
  </Accordion>
</AccordionGroup>

## Related Use Cases

* [Deep Research](/use-cases/deep-research) - Advanced research capabilities
* [Content Generation](/use-cases/content-generation) - AI-powered content creation
* [Developers & MCP](/use-cases/developers-mcp) - Developer integrations


# Competitive Intelligence
Source: https://docs.firecrawl.dev/use-cases/competitive-intelligence

Monitor competitor websites and track changes in real-time

Business intelligence teams use Firecrawl to monitor competitors and get alerts on strategic changes.

## Start with a Template

<CardGroup>
  <Card title="Firecrawl Observer" icon="github" href="https://github.com/firecrawl/firecrawl-observer">
    Real-time website monitoring with intelligent alerts
  </Card>

  <Card title="Fireplexity" icon="github" href="https://github.com/firecrawl/fireplexity">
    Research and analyze competitor strategies with AI
  </Card>
</CardGroup>

<Note>
  **Choose from monitoring and research templates.** Track competitors and analyze their strategies.
</Note>

## How It Works

Set up a pipeline that scrapes competitor sites on a schedule, extracts the data you care about, and alerts your team when something changes.

<Steps>
  <Step title="Scrape competitor pages">
    Crawl or scrape competitor websites with Firecrawl to get clean, structured content from product pages, pricing tables, and blog posts.
  </Step>

  <Step title="Extract key data points">
    Pull out specific fields like pricing tiers, feature lists, job openings, or partnership announcements using structured extraction.
  </Step>

  <Step title="Compare over time">
    Store each extraction and diff it against previous snapshots to pinpoint what changed and when.
  </Step>

  <Step title="Alert on meaningful changes">
    Trigger notifications when you detect important updates such as a new product launch, a pricing shift, or a change in positioning.
  </Step>
</Steps>

## What You Can Track

* **Products**: New launches, features, specs, pricing, documentation
* **Marketing**: Messaging changes, campaigns, case studies, testimonials
* **Business**: Job postings, partnerships, funding, press releases
* **Strategy**: Positioning, target markets, pricing approaches, go-to-market
* **Technical**: API changes, integrations, technology stack updates

## FAQs

<AccordionGroup>
  <Accordion title="How quickly can I detect changes?">
    Firecrawl extracts current page content whenever called. Build your own monitoring system to check competitors at intervals that match your needs - from hourly for critical updates to daily for routine tracking.
  </Accordion>

  <Accordion title="Can I monitor competitors in different regions?">
    Yes, Firecrawl can access region-specific content. You can monitor different versions of competitor sites across multiple countries and languages.
  </Accordion>

  <Accordion title="How do I avoid false positive alerts?">
    When building your monitoring system, implement filters to ignore minor changes like timestamps or dynamic content. Compare extracted data over time and use your own logic to determine what constitutes a meaningful change.
  </Accordion>

  <Accordion title="Can I track competitor social media and PR activity?">
    Yes. Extract data from competitor press releases, blog posts, and public social media pages. Build systems to analyze announcement patterns, messaging changes, and campaign launches over time.
  </Accordion>

  <Accordion title="How do I organize intelligence across multiple competitors?">
    Extract data from multiple competitor sites using Firecrawl's APIs. Build your own system to organize and compare this data - many users create databases with competitor profiles and custom dashboards for analysis.
  </Accordion>
</AccordionGroup>

## Related Use Cases

* [Product & E-commerce](/use-cases/product-ecommerce) - Track competitor products
* [Investment & Finance](/use-cases/investment-finance) - Market intelligence
* [SEO Platforms](/use-cases/seo-platforms) - search competitor tracking


# Content Generation
Source: https://docs.firecrawl.dev/use-cases/content-generation

Generate AI content based on website data, images, and news

Turn live website data into presentations, emails, marketing copy, and newsletters. Firecrawl scrapes pages, extracts clean text and images, and feeds structured content to your AI pipeline so every piece you publish is grounded in real, up-to-date sources.

## Start with a Template

<Card title="Open Lovable" icon="github" href="https://github.com/firecrawl/open-lovable">
  Clone and recreate any website as a modern React app
</Card>

## How It Works

Point Firecrawl at any URL or set of URLs. It returns clean, structured content you can pipe directly into your generation workflow.

### Multiple Output Formats

Extract content as structured HTML, clean Markdown, JSON, or full-page screenshots, whichever format best fits your content pipeline.

### Image and Visual Extraction

Capture images and take screenshots from source pages to enrich presentations, emails, and social posts with real visuals.

### News and Trending Content

Surface relevant news stories as part of your request to keep generated content current and timely.

## What You Can Create

* **Sales Decks**: Custom presentations with prospect data
* **Email Campaigns**: Personalized outreach at scale
* **Marketing Content**: Data-driven blog posts and reports
* **Social Media**: Trending topic and news-driven content generation
* **Documentation**: Auto-updated technical content
* **Newsletters**: Curated updates from industry and competitor news
* **Visual Content**: Posts and reports enriched with extracted images and screenshots

## FAQs

<AccordionGroup>
  <Accordion title="How does Firecrawl ensure data accuracy for content creation?">
    Firecrawl extracts data directly from source websites, preserving the original content structure and context. All extracted data includes source URLs and timestamps for verification.
  </Accordion>

  <Accordion title="What data can Firecrawl provide for content generation?">
    Firecrawl provides clean markdown, structured JSON, HTML, images, and screenshots from websites. This extracted data serves as the factual foundation for your content generation workflows.
  </Accordion>

  <Accordion title="Can Firecrawl handle images and news sources?">
    Yes. Firecrawl can extract images, capture screenshots, and pull content from news sites. This enables you to create visually rich content and stay current with industry developments.
  </Accordion>

  <Accordion title="What types of websites can Firecrawl extract from?">
    Firecrawl excels at extracting from company websites, news sites, blogs, and documentation. Sites with structured HTML and clear content hierarchies yield the cleanest extraction results.
  </Accordion>

  <Accordion title="How can I use Firecrawl for bulk data extraction?">
    Use Firecrawl's batch scraping and crawl APIs to extract data from multiple websites efficiently. Process hundreds of URLs in parallel to build comprehensive datasets for your content workflows.
  </Accordion>
</AccordionGroup>

## Related Use Cases

* [AI Platforms](/use-cases/ai-platforms) - Build AI-powered content tools
* [Lead Enrichment](/use-cases/lead-enrichment) - Personalize with prospect data
* [SEO Platforms](/use-cases/seo-platforms) - Optimize generated content


# Data Migration
Source: https://docs.firecrawl.dev/use-cases/data-migration

Transfer web data efficiently between platforms and systems

Extract content, structure, and metadata from any website and transform it for import into a new platform. Firecrawl handles the scraping so you can focus on mapping data to your target system.

## Start with a Template

<Card title="Firecrawl Migrator" icon="github" href="https://github.com/firecrawl/firecrawl-migrator">
  Efficiently migrate data between platforms and systems
</Card>

## How It Works

Point Firecrawl at your source website to crawl every page and return clean, structured data. From there, you transform the output to match your target platform's schema and import it using that platform's API or bulk-import tools.

## What You Can Migrate

* **Content**: Pages, posts, articles, media files, metadata
* **Structure**: Hierarchies, categories, tags, taxonomies
* **Users**: Profiles and user-related data where publicly accessible
* **Settings**: Configurations, custom fields, workflows
* **E-commerce**: Products, catalogs, inventory, orders

## Common Migration Use Cases

Users build migration tools with Firecrawl to extract data from various platforms:

<CardGroup>
  <Card title="CMS Content Extraction">
    Extract content from WordPress, Drupal, and Joomla sites or custom CMS platforms. Preserve content structure and metadata, then export for import into new systems like Contentful, Strapi, or Sanity.
  </Card>

  <Card title="E-commerce Data Extraction">
    Extract product catalogs from Magento and WooCommerce stores including inventory, pricing, descriptions, and specifications. Format data for import into Shopify, BigCommerce, or other platforms.
  </Card>
</CardGroup>

## FAQs

<AccordionGroup>
  <Accordion title="How do you handle large-scale migrations?">
    Our infrastructure scales automatically to handle large migrations. We support incremental processing with batching and parallel extraction, allowing you to migrate millions of pages by breaking them into manageable chunks with progress tracking.
  </Accordion>

  <Accordion title="Can I preserve SEO value during migration?">
    Yes! Extract all SEO metadata including URLs, titles, descriptions, and implement proper redirects. We help maintain your search rankings through the migration.
  </Accordion>

  <Accordion title="What about media files and attachments?">
    Firecrawl can extract and catalog all media files. You can download them for re-upload to your new platform or reference them directly if keeping the same CDN.
  </Accordion>

  <Accordion title="How do I validate the migration?">
    We provide detailed extraction reports and support comparison tools. You can verify content completeness, check broken links, and validate data integrity.
  </Accordion>

  <Accordion title="Can I migrate user-generated content and comments?">
    Yes, you can extract publicly visible user-generated content including comments, reviews, and forum posts. Private user data requires appropriate authentication and permissions.
  </Accordion>
</AccordionGroup>

## Related Use Cases

* [Product & E-commerce](/use-cases/product-ecommerce) - Catalog migrations
* [Content Generation](/use-cases/content-generation) - Content transformation
* [AI Platforms](/use-cases/ai-platforms) - Knowledge base migration


# Deep Research
Source: https://docs.firecrawl.dev/use-cases/deep-research

Build agentic research tools with deep web search capabilities

Build automated research agents that search the web, scrape full-page content, and synthesize findings with an LLM. Firecrawl handles source discovery and content extraction so you can focus on analysis, not parsing HTML.

## Start with a Template

<CardGroup>
  <Card title="Fireplexity" icon="github" href="https://github.com/firecrawl/fireplexity">
    Blazing-fast AI search with real-time citations
  </Card>

  <Card title="Firesearch" icon="github" href="https://github.com/firecrawl/firesearch">
    Deep research agent with LangGraph and answer validation
  </Card>

  <Card title="Open Researcher" icon="github" href="https://github.com/firecrawl/open-researcher">
    Visual AI research assistant for comprehensive analysis
  </Card>
</CardGroup>

<Note>
  **Choose from multiple research templates.** Clone, configure your API key, and start researching.
</Note>

## How It Works

Build powerful research tools that transform scattered web data into comprehensive insights. The core pattern is a **search → scrape → analyze → repeat** loop: use Firecrawl's search API to discover relevant sources, scrape each source for full content, then feed the results into an LLM to synthesize findings and identify follow-up queries.

<Steps>
  <Step title="Search for sources">
    Use the `/search` endpoint to find relevant pages for your research topic.

    <CodeGroup>
      ```python Python theme={null}
      from firecrawl import Firecrawl

      firecrawl = Firecrawl(api_key="fc-YOUR-API-KEY")

      results = firecrawl.search(
          "recent advances in quantum computing",
          limit=5,
          scrape_options={"formats": ["markdown", "links"]}
      )
      ```

      ```js Node.js theme={null}
      import Firecrawl from '@mendable/firecrawl-js';

      const firecrawl = new Firecrawl({ apiKey: "fc-YOUR-API-KEY" });

      const results = await firecrawl.search(
        'recent advances in quantum computing',
        { limit: 5, scrapeOptions: { formats: ['markdown', 'links'] } }
      );
      ```
    </CodeGroup>
  </Step>

  <Step title="Scrape discovered pages">
    Extract full content from each result to get detailed information with citations.

    <CodeGroup>
      ```python Python theme={null}
      for result in results:
          doc = firecrawl.scrape(result["url"], formats=["markdown"])
          # Feed doc content into your LLM for analysis
      ```

      ```js Node.js theme={null}
      for (const result of results) {
        const doc = await firecrawl.scrape(result.url, { formats: ['markdown'] });
        // Feed doc content into your LLM for analysis
      }
      ```
    </CodeGroup>
  </Step>

  <Step title="Analyze and iterate">
    Use an LLM to synthesize findings, identify gaps, and generate follow-up queries. Repeat the loop until your research question is fully answered.
  </Step>
</Steps>

## Why Researchers Choose Firecrawl

### Accelerate Research from Weeks to Hours

Build automated research systems that discover, read, and synthesize information from across the web. Create tools that deliver comprehensive reports with full citations, eliminating manual searching through hundreds of sources.

### Ensure Research Completeness

Reduce the risk of missing critical information. Build systems that follow citation chains, discover related sources, and surface insights that traditional search methods miss.

## Research Tool Capabilities

* **Iterative Exploration**: Build tools that automatically discover related topics and sources
* **Multi-Source Synthesis**: Combine information from hundreds of websites
* **Citation Preservation**: Maintain full source attribution in your research outputs
* **Intelligent Summarization**: Extract key findings and insights for analysis
* **Trend Detection**: Identify patterns across multiple sources

## FAQs

<AccordionGroup>
  <Accordion title="How can I build research tools with Firecrawl?">
    Use Firecrawl's crawl and search APIs to build iterative research systems. Start with search results, extract content from relevant pages, follow citation links, and aggregate findings. Combine with LLMs to synthesize comprehensive research reports.
  </Accordion>

  <Accordion title="Can Firecrawl handle academic and scientific websites?">
    Yes. Firecrawl can extract data from open-access research papers, academic websites, and publicly available scientific publications. It preserves formatting, citations, and technical content critical for research work.
  </Accordion>

  <Accordion title="How do I ensure research data accuracy?">
    Firecrawl maintains source attribution and extracts content exactly as presented on websites. All data includes source URLs and timestamps, ensuring full traceability for research purposes.
  </Accordion>

  <Accordion title="Can I use Firecrawl for longitudinal studies?">
    Yes. Set up scheduled crawls to track how information changes over time. This is perfect for monitoring trends, policy changes, or any research requiring temporal data analysis.
  </Accordion>

  <Accordion title="How does Firecrawl handle large-scale research projects?">
    Our crawling infrastructure scales to handle thousands of sources simultaneously. Whether you're analyzing entire industries or tracking global trends, Firecrawl provides the data pipeline you need.
  </Accordion>
</AccordionGroup>

## Related Use Cases

* [AI Platforms](/use-cases/ai-platforms) - Build AI research assistants
* [Content Generation](/use-cases/content-generation) - Research-based content
* [Competitive Intelligence](/use-cases/competitive-intelligence) - Market research


# Developers & MCP
Source: https://docs.firecrawl.dev/use-cases/developers-mcp

Build powerful integrations with Model Context Protocol support

Give your AI coding assistant the ability to scrape, crawl, and search the web in real time. Firecrawl's MCP server connects to Claude Desktop, Cursor, and other Model Context Protocol clients so your assistant can pull live documentation, discover site structures, and extract structured data on demand.

## Start with a Template

<CardGroup>
  <Card title="MCP Server Firecrawl" icon="github" href="https://github.com/firecrawl/firecrawl-mcp-server">
    Official MCP server - Add web scraping to Claude Desktop and Cursor
  </Card>

  <Card title="Open Lovable" icon="github" href="https://github.com/firecrawl/open-lovable">
    Build complete applications from any website instantly
  </Card>
</CardGroup>

<Note>
  **Get started with MCP in minutes.** Follow our [setup guide](https://github.com/firecrawl/firecrawl-mcp-server#installation) to integrate Firecrawl into Claude Desktop or Cursor.
</Note>

## How It Works

Integrate Firecrawl directly into your AI coding workflow through Model Context Protocol. Once configured, your AI assistant gains access to a set of web scraping tools it can call on your behalf:

| Tool             | What it does                                               |
| ---------------- | ---------------------------------------------------------- |
| **Scrape**       | Extract content or structured data from a single URL       |
| **Batch Scrape** | Extract content from multiple known URLs in parallel       |
| **Map**          | Discover all indexed URLs on a website                     |
| **Crawl**        | Walk a site section and extract content from every page    |
| **Search**       | Search the web and optionally extract content from results |

Your assistant picks the right tool automatically. Ask it to "read the Next.js docs" and it will scrape. Ask it to "find all blog posts on example.com" and it will map then batch scrape.

## Why Developers Choose Firecrawl MCP

### Build Smarter AI Assistants

Give your AI real-time access to documentation, APIs, and web resources. Reduce outdated information and hallucinations by providing your assistant with the latest data.

### Zero Infrastructure Required

No servers to manage, no crawlers to maintain. Just configure once and your AI assistant can access websites instantly through the Model Context Protocol.

## Customer Stories

<CardGroup>
  <Card href="https://www.firecrawl.dev/blog/how-botpress-enhances-knowledge-base-creation-with-firecrawl">
    **Botpress**

    Discover how Botpress uses Firecrawl to streamline knowledge base population and improve developer experience.
  </Card>

  <Card href="https://www.firecrawl.dev/blog/how-answer-hq-powers-ai-customer-support-with-firecrawl">
    **Answer HQ**

    Learn how Answer HQ uses Firecrawl to help businesses import website data and build intelligent support assistants.
  </Card>
</CardGroup>

## FAQs

<AccordionGroup>
  <Accordion title="Which AI assistants support MCP?">
    Currently, Claude Desktop and Cursor have native MCP support. More AI assistants are adding support regularly. You can also use the MCP SDK to build custom integrations.
  </Accordion>

  <Accordion title="Can I use MCP in VS Code or other IDEs?">
    VS Code and other IDEs can use MCP through community extensions or terminal integrations. Native support varies by IDE. Check our [GitHub repository](https://github.com/firecrawl/firecrawl-mcp-server) for IDE-specific setup guides.
  </Accordion>

  <Accordion title="How do I cache frequently accessed docs?">
    The MCP server automatically caches responses for 15 minutes. You can configure cache duration in your MCP server settings or implement custom caching logic.
  </Accordion>

  <Accordion title="Is there a rate limit for MCP requests?">
    MCP requests use your standard Firecrawl API rate limits. We recommend batching related requests and using caching for frequently accessed documentation.
  </Accordion>

  <Accordion title="How do I set up MCP with my Firecrawl API key?">
    Follow our [setup guide](https://github.com/firecrawl/firecrawl-mcp-server#installation) to configure MCP. You'll need to add your Firecrawl API key to your MCP configuration file. The process takes just a few minutes.
  </Accordion>
</AccordionGroup>

## Related Use Cases

* [AI Platforms](/use-cases/ai-platforms) - Build AI-powered dev tools
* [Deep Research](/use-cases/deep-research) - Complex technical research
* [Content Generation](/use-cases/content-generation) - Generate documentation


# Investment & Finance
Source: https://docs.firecrawl.dev/use-cases/investment-finance

Track companies and extract financial insights from web data

Turn public web data into structured financial signals. Firecrawl scrapes company websites, news sites, job boards, and regulatory filings, then returns clean JSON you can feed directly into due diligence workflows, earnings prep, or ongoing portfolio surveillance.

## Start with a Template

<Card title="Firecrawl Observer" icon="github" href="https://github.com/firecrawl/firecrawl-observer">
  Monitor portfolio companies for material changes and trigger events
</Card>

## What You Can Track

* **Company Metrics**: growth indicators, team changes, product launches, funding rounds
* **Market Signals**: industry trends, competitor moves, sentiment shifts, regulatory changes
* **Risk Indicators**: leadership changes, legal issues, regulatory mentions, customer complaints
* **Financial Data**: pricing updates, revenue signals, partnership announcements
* **Alternative Data**: job postings, web traffic, social signals, news mentions

## Customer Stories

<CardGroup>
  <Card href="https://www.firecrawl.dev/blog/how-athena-intelligence-empowers-analysts-with-firecrawl">
    **Athena Intelligence**

    Discover how Athena Intelligence leverages Firecrawl to fuel its AI-native analytics platform for enterprise analysts.
  </Card>

  <Card href="https://www.firecrawl.dev/blog/how-cargo-empowers-gtm-teams-with-firecrawl">
    **Cargo**

    See how Cargo uses Firecrawl to analyze market data and power revenue intelligence workflows.
  </Card>
</CardGroup>

## FAQs

<AccordionGroup>
  <Accordion title="Can I track private companies?">
    Yes. Monitor publicly available information from company websites, news mentions, job postings, and social media presence.
  </Accordion>

  <Accordion title="How real-time is the data?">
    Firecrawl extracts data on every call, so you always get the live page. Schedule your own monitoring at intervals that match your strategy, from minute-by-minute for critical events to daily for routine tracking.
  </Accordion>

  <Accordion title="What alternative data sources can I monitor?">
    Any public web source, including company websites, news sites, job boards, review sites, forums, social media, government filings, and open-access industry data.
  </Accordion>

  <Accordion title="How can I track ESG and sustainability signals?">
    Extract data from ESG reports, sustainability pages, news coverage of environmental initiatives, and regulatory filings. Re-run extractions on a schedule to flag shifts in sustainability commitments or ESG-related developments.
  </Accordion>

  <Accordion title="Can Firecrawl help with earnings call preparation?">
    Yes. Extract recent company updates, product launches, executive changes, and industry trends ahead of the call, then combine with competitor data to anticipate questions and identify key discussion points.
  </Accordion>
</AccordionGroup>

## Related Use Cases

* [Competitive Intelligence](/use-cases/competitive-intelligence) - Track market competitors
* [Deep Research](/use-cases/deep-research) - Comprehensive market analysis
* [Lead Enrichment](/use-cases/lead-enrichment) - B2B investment opportunities


# Lead Enrichment
Source: https://docs.firecrawl.dev/use-cases/lead-enrichment

Extract and filter leads from websites to power your sales pipeline

Scrape business directories for leads, enrich your CRM with live company data, and automate account research. Firecrawl handles the extraction so you can focus on closing deals.

## Start with a Template

<Card title="Fire Enrich" icon="github" href="https://github.com/firecrawl/fire-enrich">
  AI-powered lead enrichment and data extraction from websites
</Card>

## How It Works

1. **Point Firecrawl at a source.** Pass a directory URL, company website, or list of domains to the API.
2. **Extract structured data.** Firecrawl scrapes each page and returns fields like company name, contact details, team members, and recent news.
3. **Send the data to your CRM.** Push enriched records into Salesforce, HubSpot, or any other tool through the API or a Zapier integration.

## Why Sales Teams Choose Firecrawl

### Extract leads from any directory

Point Firecrawl at a business directory, trade association, or conference attendee list. You get back structured records with company details and contact information, ready to import into your CRM.

### Keep CRM data fresh automatically

Firecrawl pulls information directly from live company websites instead of a static database. Your team always sees the latest company news, team changes, and growth signals.

## Customer Stories

<CardGroup>
  <Card href="https://www.firecrawl.dev/blog/how-zapier-uses-firecrawl-to-power-chatbots">
    **Zapier**

    Discover how Zapier uses Firecrawl to empower customers with custom knowledge in their chatbots.
  </Card>

  <Card href="https://www.firecrawl.dev/blog/how-cargo-empowers-gtm-teams-with-firecrawl">
    **Cargo**

    See how Cargo uses Firecrawl to instantly analyze webpage content and power Go-To-Market workflows.
  </Card>
</CardGroup>

## Lead Sources

Pull leads from two kinds of sources, depending on what you already have to work with.

### Business Directories

Use a directory URL when you need a list of companies in a market or industry.

* Industry-specific directories
* Chamber of commerce listings
* Trade association members
* Conference attendee lists

### Company Websites

Use a company URL when you already know the account and want deeper context for outreach.

* About pages and team sections
* Press releases and news
* Job postings for growth signals
* Customer case studies

## FAQs

<AccordionGroup>
  <Accordion title="How does Firecrawl enhance lead enrichment processes?">
    Firecrawl automatically extracts company information, contact details, product offerings, and recent news from prospect websites. This enriches your CRM with accurate, up-to-date information for better sales outreach.
  </Accordion>

  <Accordion title="Can Firecrawl find contact information from websites?">
    Yes! Firecrawl extracts publicly available contact information including emails and phone numbers from company websites, team pages, and contact sections.
  </Accordion>

  <Accordion title="How accurate is the lead data from Firecrawl?">
    Since Firecrawl extracts data directly from live websites, you get the most current information available. This is more accurate than static databases that quickly become outdated.
  </Accordion>

  <Accordion title="Can I integrate Firecrawl with my CRM?">
    Yes. Use our API or Zapier integration to automatically enrich leads in Salesforce, HubSpot, Pipedrive, and other CRMs. Keep your lead data fresh without manual research.
  </Accordion>

  <Accordion title="How does Firecrawl help with account-based marketing?">
    Extract detailed company information, recent updates, and trigger events from target account websites. This intelligence helps personalize outreach and identify the perfect timing for engagement.
  </Accordion>
</AccordionGroup>

## Related Use Cases

* [AI Platforms](/use-cases/ai-platforms) - Build AI sales assistants
* [Competitive Intelligence](/use-cases/competitive-intelligence) - Track competitors
* [Investment & Finance](/use-cases/investment-finance) - Investment opportunities


# Observability & Monitoring
Source: https://docs.firecrawl.dev/use-cases/observability

Monitor websites, track uptime, and detect changes in real-time

DevOps and SRE teams use Firecrawl to monitor websites, track availability, and detect critical changes across their digital infrastructure.

## Start with a Template

<Card title="Firecrawl Observer" icon="github" href="https://github.com/firecrawl/firecrawl-observer">
  Real-time website monitoring and intelligent change detection
</Card>

## How It Works

Call Firecrawl's scrape or extract API on a schedule to capture page content, then compare each snapshot against a baseline in your own system. When the extracted data changes or a page fails to load, trigger an alert through your existing tools (PagerDuty, Slack, email, etc.).

Because Firecrawl fully renders JavaScript before extracting, you get the page as users see it, not just the raw HTML. This makes it reliable for monitoring SPAs, dynamic dashboards, and client-rendered content.

## What You Can Monitor

* **Availability**: Uptime, response times, error rates
* **Content**: Text changes, image updates, layout shifts
* **Performance**: Page load times, resource sizes, Core Web Vitals
* **Security**: SSL certificates, security headers, misconfigurations
* **SEO Health**: Meta tags, structured data, sitemap validity
* **User journeys**: Multi-step transaction flows and cross-browser rendering

## FAQs

<AccordionGroup>
  <Accordion title="How does Firecrawl help with website monitoring?">
    Firecrawl extracts website content and structure on demand. Build monitoring systems that call Firecrawl's API to check pages, compare extracted data against baselines, and trigger your own alerts when changes occur.
  </Accordion>

  <Accordion title="Can I monitor JavaScript-heavy applications?">
    Yes! Firecrawl fully renders JavaScript, making it perfect for monitoring modern SPAs, React apps, and dynamic content. We capture the page as users see it, not just the raw HTML.
  </Accordion>

  <Accordion title="How quickly can I detect website issues?">
    Firecrawl extracts data in real-time when called. Build your monitoring system to check sites at whatever frequency you need - from minute-by-minute for critical pages to daily for routine checks.
  </Accordion>

  <Accordion title="Can I validate specific page elements?">
    Yes. Use the extract API to pull specific elements like prices, inventory levels, or critical content. Build validation logic in your monitoring system to verify that important information is present and correct.
  </Accordion>

  <Accordion title="How can I integrate Firecrawl with alerting systems?">
    Firecrawl provides webhooks that you can use to build integrations with your alerting tools. Send extracted data to PagerDuty, Slack, email, or any monitoring platform by building connectors that process Firecrawl's responses.
  </Accordion>
</AccordionGroup>

## Related Use Cases

* [Competitive Intelligence](/use-cases/competitive-intelligence) - Monitor competitor changes
* [Product & E-commerce](/use-cases/product-ecommerce) - Track inventory and pricing
* [Data Migration](/use-cases/data-migration) - Validate migrations


# Use Cases
Source: https://docs.firecrawl.dev/use-cases/overview

Transform web data into powerful features for your applications

Turn any website into structured data for your product. Whether you are building RAG pipelines, enriching leads, or monitoring competitors, Firecrawl handles the scraping so you can focus on what you do with the data.

Browse the use cases below to find guides, code samples, and architecture patterns for your workflow.

<CardGroup>
  <Card title="AI Platforms" href="/use-cases/ai-platforms">
    Add web knowledge to your RAG chatbots and AI assistants.
  </Card>

  <Card title="Lead Enrichment" href="/use-cases/lead-enrichment">
    Extract and filter leads from websites to enrich your sales pipeline.
  </Card>

  <Card title="SEO Platforms" href="/use-cases/seo-platforms">
    Monitor search rankings and optimize content strategy.
  </Card>

  <Card title="Deep Research" href="/use-cases/deep-research">
    Build agentic research tools with deep web search capabilities.
  </Card>

  <Card title="Product & E-commerce" href="/use-cases/product-ecommerce">
    Monitor pricing and track inventory across e-commerce sites.
  </Card>

  <Card title="Content Generation" href="/use-cases/content-generation">
    Generate AI content based on website data and structure.
  </Card>

  <Card title="Developers & MCP" href="/use-cases/developers-mcp">
    Build powerful integrations with Model Context Protocol support.
  </Card>

  <Card title="Investment & Finance" href="/use-cases/investment-finance">
    Track companies and extract financial insights from web data.
  </Card>

  <Card title="Competitive Intelligence" href="/use-cases/competitive-intelligence">
    Monitor competitor websites and track changes in real-time.
  </Card>

  <Card title="Data Migration" href="/use-cases/data-migration">
    Transfer web data seamlessly between platforms and systems.
  </Card>

  <Card title="Observability" href="/use-cases/observability">
    Monitor websites, track uptime, and detect changes in real-time.
  </Card>
</CardGroup>


# Product & E-commerce
Source: https://docs.firecrawl.dev/use-cases/product-ecommerce

Monitor pricing and track inventory across e-commerce sites

Turn any e-commerce website into structured product data. Firecrawl extracts pricing, inventory, and catalog information so you can monitor competitors, track stock levels, and migrate products between platforms.

## Start with a Template

<Card title="Firecrawl Migrator" icon="github" href="https://github.com/firecrawl/firecrawl-migrator">
  Migrate product catalogs and e-commerce data between platforms
</Card>

## What You Can Extract

* **Product Data**: Title, SKU, specs, descriptions, categories
* **Pricing**: Current price, discounts, shipping, tax
* **Inventory**: Stock levels, availability, lead times
* **Reviews**: Ratings, customer feedback, Q\&A sections

## Use Cases in Action

<CardGroup>
  <Card>
    **Price Monitoring**

    Track competitor pricing across multiple e-commerce sites, receive alerts on price changes, and optimize your pricing strategy based on real-time market data.
  </Card>

  <Card>
    **Catalog Migration**

    Seamlessly migrate thousands of products between e-commerce platforms, preserving all product data, variants, images, and metadata.
  </Card>
</CardGroup>

## FAQs

<AccordionGroup>
  <Accordion title="How can I track competitor pricing changes?">
    Build a monitoring system using Firecrawl's API to extract prices at regular intervals. Compare extracted data over time to identify pricing trends, promotions, and competitive positioning.
  </Accordion>

  <Accordion title="Can I extract product variants (size, color, etc.)?">
    Yes, Firecrawl can extract all product variants including size, color, and other options. Structure the data with custom schemas to capture all variant information.
  </Accordion>

  <Accordion title="How do I handle dynamic pricing or user-specific prices?">
    For dynamic pricing, you can use Firecrawl's JavaScript rendering to capture prices after they load. For user-specific pricing, configure authentication headers in your requests.
  </Accordion>

  <Accordion title="Can I extract data from different e-commerce platforms?">
    Yes. Firecrawl can extract data from any publicly accessible e-commerce website. Users successfully extract from Shopify, WooCommerce, Magento, BigCommerce, and custom-built stores.
  </Accordion>

  <Accordion title="Can Firecrawl handle pagination and infinite scroll?">
    Yes. Firecrawl can navigate through paginated product listings and handle infinite scroll mechanisms to extract complete product catalogs, ensuring no products are missed during extraction.
  </Accordion>
</AccordionGroup>

## Related Use Cases

* [Lead Enrichment](/use-cases/lead-enrichment) - Enrich B2B e-commerce leads
* [Competitive Intelligence](/use-cases/competitive-intelligence) - Track competitor strategies
* [Data Migration](/use-cases/data-migration) - Migrate between platforms


# SEO Platforms
Source: https://docs.firecrawl.dev/use-cases/seo-platforms

Optimize websites for AI assistants and search engines

Audit entire websites for SEO and AI readability at scale. Firecrawl extracts meta tags, header structures, internal links, and content across thousands of pages, so you can spot optimization gaps that sampling-based tools miss.

## Start with a Template

<Card title="FireGEO" icon="github" href="https://github.com/firecrawl/firegeo">
  GEO-powered SEO monitoring and multi-region rank tracking
</Card>

<Note>
  **Get started with the FireGEO template.** Optimize for both search engines and AI assistants.
</Note>

## How It Works

Crawl a site with Firecrawl to get structured markdown and metadata for every page. Use that output to analyze content quality, check technical SEO elements, and evaluate how well each page communicates with AI assistants.

## Why SEO Platforms Choose Firecrawl

### Optimize for AI discovery, not just Google

Tune your clients' content for AI assistant responses alongside traditional search rankings. Firecrawl extracts the structural and semantic signals AI crawlers rely on, so you can audit pages for both Google and the new wave of AI-powered discovery.

### Complete site intelligence at scale

Audit entire websites instead of sampling a handful of pages. Extract meta tags, header structures, internal links, and content across thousands of pages in a single crawl, and surface optimization gaps your competitors miss.

## What You Can Build

* **AI Readability Audit**: Optimize for AI comprehension
* **Content Analysis**: Structure and semantic optimization
* **Technical SEO**: Site performance and crawlability
* **Search Tracking**: Monitor search engine positions

## FAQs

<AccordionGroup>
  <Accordion title="How can I optimize my site for AI assistants?">
    Firecrawl helps you structure content for optimal AI comprehension, extract semantic signals, and ensure your site follows best practices for AI discovery. This includes generating experimental formats like llms.txt (an emerging convention for AI crawler guidance).
  </Accordion>

  <Accordion title="How does Firecrawl support SEO audits?">
    Firecrawl extracts complete site structures, meta tags, headers, internal links, and content to perform comprehensive SEO audits. Identify optimization opportunities and track improvements over time.
  </Accordion>

  <Accordion title="Can Firecrawl help with competitor SEO analysis?">
    Yes. Analyze competitor site structures, keyword usage, content strategies, and technical SEO implementations. Understand what's working in your industry to inform your strategy.
  </Accordion>

  <Accordion title="How can I use Firecrawl for content gap analysis?">
    Crawl competitor sites to identify topics they cover that you don't. Extract their content categories, blog topics, and page structures to find opportunities for new content.
  </Accordion>

  <Accordion title="Does Firecrawl help with technical SEO monitoring?">
    Yes. Identify broken links, track redirect chains, extract canonical tags, and monitor meta tag implementation. Regular crawls help identify technical SEO issues across your site.
  </Accordion>
</AccordionGroup>

## Related Use Cases

* [AI Platforms](/use-cases/ai-platforms) - Build AI-powered SEO tools
* [Competitive Intelligence](/use-cases/competitive-intelligence) - Track competitor SEO
* [Content Generation](/use-cases/content-generation) - Create SEO content


# FIRE-1 Agent (Beta)
Source: https://docs.firecrawl.dev/agents/fire-1-extract

FIRE-1 is an AI agent that enables intelligent navigation and interaction with web pages

FIRE-1 is an AI agent that enhances Firecrawl's scraping capabilities through intelligent web navigation and interaction. It can handle pagination, control browser actions, and navigate complex website structures to enable comprehensive data extraction beyond traditional scraping methods.

### What FIRE-1 Can Do:

* Navigate through paginated content automatically.
* Interact with buttons, links, inputs, and dynamic elements.
* Perform sophisticated extraction tasks across multiple pages.

## Enabling FIRE-1 Agent

To enable the FIRE-1 agent, you need to include the `agent` object within your API request payload for either the `scrape` or `extract` endpoint.

The `agent` object has the following properties:

* `model` (string, optional): Specifies the AI model to use. If not provided, it defaults to `FIRE-1`. Currently, `FIRE-1` is the only available model.
* `prompt` (string, required for `scrape` endpoint): Provides instructions for the AI agent, describing what content to look for and how to navigate the website (e.g., how to handle pagination, buttons to click, etc.). In `/extract` it will use the prompt provided in the `prompt` parameter.

### Using FIRE-1 with the Scrape Endpoint

You can use the FIRE-1 agent with the `/v1/scrape` endpoint to apply intelligent navigation before scraping the final content.

**Example (cURL):**

```bash theme={null}
curl -X POST https://api.firecrawl.dev/v1/scrape \
    -H 'Content-Type: application/json' \
    -H 'Authorization: Bearer YOUR_API_KEY' \
    -d '{
      "url": "https://example.com/products?page=1",
      "formats": ["markdown"],
      "agent": {
        "model": "FIRE-1",
        "prompt": "Navigate through the product listings by clicking the \'Next Page\' button until it is disabled. Scrape the content of each page visited."
      }
    }'
```

In this example, the FIRE-1 agent is instructed to paginate through product listings before the final scrape occurs.

### Using FIRE-1 with the Extract Endpoint

Similarly, you can leverage the FIRE-1 agent with the `/v1/extract` endpoint for complex extraction tasks that require navigation across multiple pages or interaction with elements.

**Example (cURL):**

```bash theme={null}
curl -X POST https://api.firecrawl.dev/v1/extract \
    -H 'Content-Type: application/json' \
    -H 'Authorization: Bearer YOUR_API_KEY' \
    -d '{
      "urls": ["https://example-forum.com/topic/123"],
      "prompt": "Extract all user comments from this forum thread.",
      "schema": {
        "type": "object",
        "properties": {
          "comments": {
            "type": "array",
            "items": {
              "type": "object",
              "properties": {
                "author": {"type": "string"},
                "comment_text": {"type": "string"}
              },
              "required": ["author", "comment_text"]
            }
          }
        },
        "required": ["comments"]
      },
      "agent": {
        "model": "FIRE-1"
      }
    }'
```

Here, the agent ensures all comments are loaded on the page by interacting with the "Load More Comments" button before the extraction process begins based on the provided schema and prompt.
**Note:** The FIRE-1 agent provides powerful capabilities but might consume more credits depending on the complexity of the navigation instructions and the number of pages interacted with.

### Meet FIRE-1: Intelligent Navigation and Interaction

<img alt="FIRE-1 Agent Visualization" />

FIRE-1 brings a new level of intelligence to Firecrawl, enhancing your scraping tasks by navigating complex website structures, handling pagination, interacting with dynamic content, and more. This powerful AI agent ensures comprehensive data extraction beyond traditional scraping methods.

### What FIRE-1 Can Do:

* Navigate through paginated content automatically.
* Interact with buttons, links, and dynamic elements.
* Perform sophisticated extraction tasks across multiple pages.

## How to Enable FIRE-1

Activating FIRE-1 is straightforward. Simply include an `agent` object in your scrape or extract API request:

```json theme={null}
"agent": {
  "model": "FIRE-1",
  "prompt": "Your detailed navigation instructions here."
}
```

*Note:* The `prompt` field is required for scrape requests, instructing FIRE-1 precisely how to interact with the webpage.

## Example Usage with Scrape Endpoint

Here's a quick example using FIRE-1 with the scrape endpoint to paginate through product listings:

```bash theme={null}
curl -X POST https://api.firecrawl.dev/v1/scrape \
  -H 'Content-Type: application/json' \
  -H 'Authorization: Bearer YOUR_API_KEY' \
  -d '{
    "url": "https://example.com/products?page=1",
    "formats": ["markdown"],
    "agent": {
      "model": "FIRE-1",
      "prompt": "Navigate through the product listings by clicking the \'Next Page\' button until disabled. Scrape each page."
    }
  }'
```

In this scenario, FIRE-1 intelligently navigates pagination and gathers all of the products.

## Considerations

* Using FIRE-1 may consume more credits based on task complexity and interaction depth.
* Ensure your prompts clearly guide FIRE-1 to optimize results and efficiency.

## Start Using FIRE-1 Today

Experience the future of web scraping today:

* **Try FIRE-1:** Integrate intelligent navigation into your scraping and extracting workflows.
* **Explore the docs:** Learn more in our [comprehensive documentation](https://docs.firecrawl.dev/agents/fire-1).
* **Need help?** Join our [Discord community](https://discord.gg/S7Enyh9Abh) or email [help@firecrawl.com](mailto:help@firecrawl.com).
  **Ready to leverage AI-powered scraping?** [Sign up for Firecrawl](https://firecrawl.dev/signup) and start with FIRE-1 today.


# Build with AI
Source: https://docs.firecrawl.dev/ai-onboarding

Everything you need to onboard your AI agent to Firecrawl.

If you're developing with AI, Firecrawl offers several resources to improve your experience. Firecrawl ships with **skills** — self-contained knowledge packs that AI coding agents discover and use automatically. One install command gives agents CLI tools for live web work and build skills for integrating Firecrawl into application code. Agents like Claude Code, Cursor, Antigravity, and OpenCode can self-onboard with a single command — no human setup required after the API key exists.

* [Prerequisite: Create an API Key](#prerequisite-create-an-api-key)
* [Skills + CLI](#skills-cli)
* [Using Firecrawl as a Tool](#using-firecrawl-as-a-tool)
* [Agentic Debugging](#agentic-debugging)
* [Firecrawl MCP Server](#firecrawl-mcp-server)
* [Firecrawl Docs for Agents](#firecrawl-docs-for-agents)
* [Quick Start Guides](#quick-start-guides)
* [Agent Harnesses](#agent-harnesses)
* [SDKs](#sdks)

## Prerequisite: Create an API Key

Currently, we require a human to create a Firecrawl account. Once you have an account, you'll need to [create an API key](https://www.firecrawl.dev/app/api-keys). With an API key, your agent can handle the rest — installing the skills, authenticating the CLI, wiring up MCP, and making calls on your behalf.

<Card title="Get your API key" icon="key" href="https://www.firecrawl.dev/app/api-keys">
  Sign up and grab an API key to start using Firecrawl.
</Card>

## Skills + CLI

The [Firecrawl CLI](/sdks/cli) lets your agent search, scrape, interact, crawl, map, extract, and run agent jobs from the terminal. It's built for humans, AI agents, and CI/CD pipelines.

The Firecrawl **skills** are self-contained knowledge packs that AI coding agents like Claude Code, Antigravity, and OpenCode discover and use automatically. A single install command sets up everything — the CLI tools for live web work and the build skills for integrating Firecrawl into application code:

```bash theme={null}
npx -y firecrawl-cli@latest init --all --browser
```

* `--all` installs the Firecrawl skills to every detected AI coding agent on the machine
* `--browser` opens the browser for Firecrawl authentication automatically

After install, verify everything is working:

```bash theme={null}
firecrawl --status
firecrawl scrape "https://firecrawl.dev"
```

### What the install gives you

The install sets up two categories of skills that cover every way an agent uses Firecrawl:

**CLI skills** — for live web work during an agent session:

| Skill                | Purpose                                           |
| -------------------- | ------------------------------------------------- |
| `firecrawl/cli`      | Overall CLI command workflow                      |
| `firecrawl-search`   | Search the web and discover pages                 |
| `firecrawl-scrape`   | Extract clean content from a known URL            |
| `firecrawl-interact` | Interact with scraped pages using prompts or code |
| `firecrawl-crawl`    | Bulk-extract content from an entire site          |
| `firecrawl-map`      | Discover all URLs on a domain                     |
| `firecrawl-agent`    | Run autonomous web data gathering with a job      |

**Build skills** — for integrating Firecrawl into application code:

| Skill                        | Purpose                                              |
| ---------------------------- | ---------------------------------------------------- |
| `firecrawl-build`            | Choose the right Firecrawl endpoint for your product |
| `firecrawl-build-onboarding` | Auth and project setup                               |
| `firecrawl-build-scrape`     | Implement scraping in app code                       |
| `firecrawl-build-search`     | Implement search in app code                         |
| `firecrawl-build-interact`   | Implement page interaction in app code               |
| `firecrawl-build-crawl`      | Implement crawling in app code                       |
| `firecrawl-build-map`        | Implement URL discovery in app code                  |

### Choose your path

Both skill categories use the same install. The difference is what happens next:

<Steps>
  <Step title="Live web tools (CLI skills)">
    Use this when you need web data during your current session — searching the web, scraping known URLs, interacting with scraped pages, crawling docs, mapping a site, or running an agent job.

    The default flow:

    1. Start with **search** when you need discovery
    2. Move to **scrape** when you have a URL
    3. Use **interact** when the scraped page needs follow-up actions
    4. Use **map** or **crawl** when you need many URLs or pages
    5. Use **agent** when the task is open-ended and needs autonomous discovery

    ```bash theme={null}
    # Search the web
    firecrawl search "best open-source web crawlers"

    # Scrape a page into clean markdown
    firecrawl scrape https://docs.firecrawl.dev

    # Crawl a whole site
    firecrawl crawl https://docs.firecrawl.dev
    ```
  </Step>

  <Step title="App integration (Build skills)">
    Use this when you're building an application, agent, or workflow that calls the Firecrawl API from code. The build skills help with picking the right endpoint, wiring up the SDK, and running a smoke test.

    The agent answers one key question — *what should Firecrawl do in the product?* — and the build skills route to `/search`, `/scrape`, `/interact`, `/crawl`, `/map`, or `/agent` accordingly.
  </Step>

  <Step title="REST API (no install needed)">
    If you prefer not to install anything, agents can call the Firecrawl REST API directly. Set the API key and hit the endpoints:

    * `POST https://api.firecrawl.dev/v2/search` — discover pages by query
    * `POST https://api.firecrawl.dev/v2/scrape` — extract clean markdown from a URL
    * `POST https://api.firecrawl.dev/v2/interact` — interact with a scraped page
    * `POST https://api.firecrawl.dev/v2/crawl` — bulk-extract an entire site
    * `POST https://api.firecrawl.dev/v2/map` — discover URLs on a domain
    * `POST https://api.firecrawl.dev/v2/agent` — run autonomous web data gathering

    Auth header: `Authorization: Bearer fc-YOUR_API_KEY`
  </Step>
</Steps>

The full onboarding definition is available at [`firecrawl.dev/agent-onboarding/SKILL.md`](https://www.firecrawl.dev/agent-onboarding/SKILL.md) — agents can fetch it directly for self-onboarding.

<Card title="Skills + CLI" icon="terminal" href="/sdks/cli">
  Install the CLI and skills, authenticate, and run scrape, search, crawl,
  interact, map, extract, and agent commands from the terminal.
</Card>

## Using Firecrawl as a Tool

Firecrawl gives agents five core tools for working with the web. Each tool maps to an API endpoint and a CLI command. Agents pick the right tool based on what they need:

<AccordionGroup>
  <Accordion title="Search — discover pages by query" icon="magnifying-glass">
    Start here when you don't have a URL yet. Search returns relevant web pages for a natural-language query, with optional full-page content included in the results.

    ```bash theme={null}
    # CLI
    firecrawl search "latest OpenAI API pricing"
    ```

    ```bash theme={null}
    # REST API
    curl -X POST https://api.firecrawl.dev/v2/search \
      -H "Authorization: Bearer fc-YOUR_API_KEY" \
      -H "Content-Type: application/json" \
      -d '{"query": "latest OpenAI API pricing"}'
    ```

    **When to use:** Research tasks, finding documentation, competitive analysis, answering questions that require up-to-date web information.
  </Accordion>

  <Accordion title="Scrape — extract content from a URL" icon="file-lines">
    Use this when you already have a URL and need clean, LLM-ready content. Scrape converts any web page into markdown, HTML, or structured data — handling JavaScript rendering, anti-bot measures, and messy HTML automatically.

    ```bash theme={null}
    # CLI
    firecrawl scrape https://docs.stripe.com/api/charges
    ```

    ```bash theme={null}
    # REST API
    curl -X POST https://api.firecrawl.dev/v2/scrape \
      -H "Authorization: Bearer fc-YOUR_API_KEY" \
      -H "Content-Type: application/json" \
      -d '{"url": "https://docs.stripe.com/api/charges"}'
    ```

    **When to use:** Reading documentation, extracting article content, pulling data from a known page, converting web pages to context for LLMs.
  </Accordion>

  <Accordion title="Crawl — bulk-extract an entire site" icon="spider">
    Crawl recursively follows links from a starting URL and scrapes every page it finds. Use it when you need content from an entire site or documentation set, not just a single page.

    ```bash theme={null}
    # CLI
    firecrawl crawl https://docs.firecrawl.dev --limit 50
    ```

    ```bash theme={null}
    # REST API
    curl -X POST https://api.firecrawl.dev/v2/crawl \
      -H "Authorization: Bearer fc-YOUR_API_KEY" \
      -H "Content-Type: application/json" \
      -d '{"url": "https://docs.firecrawl.dev", "limit": 50}'
    ```

    **When to use:** Ingesting full documentation sites, building knowledge bases, migrating content, training data collection.
  </Accordion>

  <Accordion title="Map — discover all URLs on a domain" icon="sitemap">
    Map rapidly discovers every indexed URL on a domain without scraping the content. Use it when you need to understand a site's structure or find specific pages before scraping them.

    ```bash theme={null}
    # CLI
    firecrawl map https://docs.firecrawl.dev
    ```

    ```bash theme={null}
    # REST API
    curl -X POST https://api.firecrawl.dev/v2/map \
      -H "Authorization: Bearer fc-YOUR_API_KEY" \
      -H "Content-Type: application/json" \
      -d '{"url": "https://docs.firecrawl.dev"}'
    ```

    **When to use:** Site audits, finding specific pages on a large site, understanding site structure before a targeted crawl.
  </Accordion>

  <Accordion title="Interact — work with a scraped page" icon="hand-pointer">
    Interact lets agents continue from a scrape using prompts or code. Use it when a scraped page requires clicks, form fills, navigation, or follow-up extraction.

    ```bash theme={null}
    # CLI
    firecrawl scrape https://example.com
    firecrawl interact "Click the pricing tab and extract the plan names"
    ```

    ```bash theme={null}
    # REST API
    curl -X POST https://api.firecrawl.dev/v2/interact \
      -H "Authorization: Bearer fc-YOUR_API_KEY" \
      -H "Content-Type: application/json" \
      -d '{"scrapeId": "scrape-id-from-scrape", "prompt": "Click the pricing tab and extract the plan names"}'
    ```

    **When to use:** Continuing from a scrape, navigating dynamic pages, filling forms, and extracting data after page actions.
  </Accordion>
</AccordionGroup>

### How agents chain tools together

Most agent workflows combine multiple tools. A typical pattern:

1. **Search** to find relevant pages → get a list of URLs
2. **Scrape** the most relevant URLs → get clean content
3. **Interact** when the scraped page needs follow-up actions
4. **Agent** when the task needs autonomous discovery or structured multi-page extraction

For bulk work, agents use **Map** to discover URLs first, then **Crawl** or selectively **Scrape** the pages they need.

## Agentic Debugging

When a Firecrawl call fails or returns unexpected results, your agent doesn't have to escalate to a human. The [`/support/ask`](/api-reference/endpoint/ask) endpoint is an AI support agent built for **agent-to-agent** communication — it diagnoses issues with your jobs, account, and API usage, then returns a verified answer with machine-readable fix parameters your agent can apply directly.

Wire it into your agent's error-handling flow so it can self-recover from scraping failures, crawl issues, and configuration problems — typically in 15–30 seconds, no human in the loop.

### How it works

1. **Your agent describes the problem** — a natural-language question describing the issue.
2. **The support agent investigates** — it inspects job logs, account state, documentation, and source code.
3. **The support agent validates** — when possible, it tests a fix against the live Firecrawl API (e.g., retrying a scrape with adjusted parameters).
4. **Your agent gets a verified answer** — a prose `answer`, machine-readable `fixParameters` to apply directly, and `validation` results showing whether the fix was tested.

### Example

Send a question, plus an optional `rationale` to give the support agent context about what your end user is trying to accomplish:

```bash theme={null}
curl -X POST https://api.firecrawl.dev/v2/support/ask \
  -H "Authorization: Bearer fc-YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "question": "my crawl returned 3 pages but I expected 50",
    "rationale": "user is on their third failed crawl attempt today"
  }'
```

The response includes an `answer`, a `confidence` rating, optional `fixParameters` (e.g., `{"waitFor": 5000}`) your agent can pass to the next call, and `validation` showing whether the fix was tested against the live API.

<Card title="Ask endpoint reference" icon="life-ring" href="/api-reference/endpoint/ask">
  Full request and response schema for `/support/ask`, including status codes and the feedback envelope returned when the agent gets stuck.
</Card>

## Firecrawl MCP Server

MCP is an open protocol that standardizes how applications provide context to LLMs. Among other benefits, it gives LLMs tools to act on your behalf. Our [MCP server](https://github.com/firecrawl/firecrawl-mcp-server) is open-source and covers our full API surface — search, scrape, interact, crawl, map, extract, and agent.

Use the remote hosted URL:

```
https://mcp.firecrawl.dev/{FIRECRAWL_API_KEY}/v2/mcp
```

Or add the local server to any MCP client:

```json theme={null}
{
  "mcpServers": {
    "firecrawl": {
      "command": "npx",
      "args": ["-y", "firecrawl-mcp"],
      "env": {
        "FIRECRAWL_API_KEY": "fc-YOUR-API-KEY"
      }
    }
  }
}
```

<Card title="MCP Server" icon="plug" href="/mcp-server">
  View installation instructions for Cursor, Claude Desktop, Windsurf, VS Code,
  and more.
</Card>

## Firecrawl Docs for Agents

You can give your agent current Firecrawl docs in a context-aware way. Agents can self-onboard by pulling these resources directly — no human wiring required.

<Steps>
  <Step title="Markdown docs">
    Every page has a markdown version. Append `.md` to any docs URL, or use the page action menu to copy the page as markdown.

    ```
    Docs for this page: https://docs.firecrawl.dev/ai-onboarding.md
    ```
  </Step>

  <Step title="Full llms.txt">
    Give your agent all of our docs in a single file.

    ```
    Here are the Firecrawl docs: https://docs.firecrawl.dev/llms-full.txt
    ```

    A shorter index is also available at `https://docs.firecrawl.dev/llms.txt`.
  </Step>

  <Step title="MCP docs server">
    For a structured approach using MCP tools, connect the Firecrawl MCP server in any MCP client (Cursor, Claude Code, Claude Desktop, Windsurf). See the [MCP Server](/mcp-server) page for install commands.
  </Step>

  <Step title="Copy to ChatGPT / Claude">
    Every page includes a contextual action menu (copy, view as markdown, open in ChatGPT, open in Claude) so agents and humans can move pages between tools in one click.
  </Step>
</Steps>

## Quick Start Guides

Drop-in quickstarts for the stacks agents build on most often. Point your agent at any of these to scaffold a working Firecrawl integration end-to-end.

Prefer to let Cursor drive? One-click install the Firecrawl MCP server and start prompting in Cursor:

<a href="cursor://anysphere.cursor-deeplink/mcp/install?name=firecrawl&config=eyJjb21tYW5kIjoibnB4IiwiYXJncyI6WyIteSIsImZpcmVjcmF3bC1tY3AiXSwiZW52Ijp7IkZJUkVDUkFXTF9BUElfS0VZIjoiWU9VUi1BUEktS0VZIn19">
  <img alt="Open in Cursor — Add Firecrawl MCP server" />
</a>

<CardGroup>
  <Card title="Node.js" icon="node-js" href="/quickstarts/nodejs">
    Server-side JavaScript and TypeScript with the Firecrawl Node SDK.
  </Card>

  <Card title="Next.js" icon="react" href="/quickstarts/nextjs">
    Scrape, search, and crawl from Next.js route handlers and server actions.
  </Card>

  <Card title="Python" icon="python" href="/quickstarts/python">
    Use Firecrawl from scripts, notebooks, and backend services.
  </Card>

  <Card title="FastAPI" icon="bolt" href="/quickstarts/fastapi">
    Build async Python APIs that search, scrape, and extract.
  </Card>

  <Card title="Cloudflare Workers" icon="cloudflare" href="/quickstarts/cloudflare-workers">
    Run Firecrawl at the edge with Workers.
  </Card>

  <Card title="Vercel Functions" icon="triangle" href="/quickstarts/vercel-functions">
    Call Firecrawl from Vercel serverless functions.
  </Card>

  <Card title="AWS Lambda" icon="aws" href="/quickstarts/aws-lambda">
    Invoke Firecrawl from Lambda handlers.
  </Card>

  <Card title="Supabase Edge Functions" icon="database" href="/quickstarts/supabase-edge-functions">
    Use Firecrawl inside Supabase Deno runtime.
  </Card>

  <Card title="Go" icon="golang" href="/quickstarts/go">
    Idiomatic Go SDK for search, scrape, and crawl.
  </Card>

  <Card title="Rust" icon="rust" href="/quickstarts/rust">
    Typed Rust SDK for Firecrawl.
  </Card>

  <Card title="Laravel" icon="laravel" href="/quickstarts/laravel">
    Add Firecrawl to Laravel apps via the PHP SDK.
  </Card>

  <Card title="Rails" icon="gem" href="/quickstarts/rails">
    Drop Firecrawl into Ruby on Rails.
  </Card>
</CardGroup>

See the full list of quickstarts (Express, NestJS, Fastify, Hono, Bun, Remix, Nuxt, SvelteKit, Astro, Mastra, Django, Flask, Elixir, Java, Spring Boot, .NET, ASP.NET Core, and more) in the left sidebar.

## Agent Harnesses

Firecrawl works with the runtimes and frameworks agents actually live inside — coding agents, agent SDKs, and model aggregators. Most coding harnesses can auto-discover the Firecrawl skills via `npx -y firecrawl-cli@latest init --all --browser`; the rest call Firecrawl as a tool over MCP or the REST API.

<CardGroup>
  <Card title="Claude Code" icon="terminal" href="/quickstarts/claude-code">
    Anthropic's CLI — set up Firecrawl MCP in Claude Code.
  </Card>

  <Card title="Cursor" icon="arrow-pointer" href="/quickstarts/cursor">
    IDE agent — one-click install Firecrawl MCP in Cursor.
  </Card>

  <Card title="OpenCode" icon="code-branch" href="/quickstarts/opencode">
    Wire Firecrawl MCP into OpenCode.
  </Card>

  <Card title="Codex CLI" icon="code" href="/quickstarts/codex-cli">
    Wire Firecrawl MCP into OpenAI Codex CLI.
  </Card>

  <Card title="OpenRouter" icon="route" href="/quickstarts/openrouter">
    Pair any OpenRouter model with Firecrawl web tools.
  </Card>

  <Card title="Amp" icon="bolt" href="/quickstarts/amp">
    Wire Firecrawl MCP into Sourcegraph Amp.
  </Card>

  <Card title="Windsurf" icon="wind" href="/quickstarts/windsurf">
    Agentic IDE — set up Firecrawl MCP in Windsurf.
  </Card>

  <Card title="Antigravity" icon="rocket" href="/quickstarts/antigravity">
    Add Firecrawl MCP to Google's agentic IDE.
  </Card>

  <Card title="Gemini CLI" icon="gem" href="/quickstarts/gemini-cli">
    Wire Firecrawl MCP into Google Gemini CLI.
  </Card>

  <Card title="Nous Research" icon="brain" href="/quickstarts/nous-research">
    Use Firecrawl as a tool with Hermes models.
  </Card>

  <Card title="AutoGen" icon="robot" href="/quickstarts/autogen">
    Firecrawl tools inside Microsoft AutoGen multi-agent teams.
  </Card>
</CardGroup>

## SDKs

Official, typed SDKs covering the full Firecrawl API surface. Point your agent at the language matching your stack.

<CardGroup>
  <Card title="Python" icon="python" href="/sdks/python" />

  <Card title="Node" icon="node-js" href="/sdks/node" />

  <Card title="Go" icon="golang" href="/sdks/go" />

  <Card title="Java" icon="java" href="/sdks/java" />

  <Card title="Ruby" icon="gem" href="/sdks/ruby" />

  <Card title="Rust" icon="rust" href="/sdks/rust" />

  <Card title=".NET" icon="microsoft" href="/sdks/dotnet" />

  <Card title="PHP" icon="php" href="/sdks/php" />

  <Card title="Elixir" icon="droplet" href="/sdks/elixir" />

  <Card title="CLI" icon="terminal" href="/sdks/cli" />
</CardGroup>

Firecrawl also has first-class SDK bindings for the major LLM SDKs and agent frameworks — see [LLM SDKs and Frameworks](/developer-guides/llm-sdks-and-frameworks/openai) for OpenAI, Anthropic, Gemini, Google ADK, Vercel AI SDK, LangChain, LangGraph, LlamaIndex, Mastra, and ElevenAgents.


# Activity
Source: https://docs.firecrawl.dev/api-reference/endpoint/activity

/api-reference/v2-openapi.json GET /team/activity
Lists your team's recent API activity from the last 24 hours. Returns metadata about each job including the job ID, which can be used with the corresponding GET endpoint (e.g. GET /crawl/{id}) to retrieve full results. Supports cursor-based pagination and filtering by endpoint.

Lists your recent API activity from the last 24 hours. Use this to discover job IDs, then retrieve results with the corresponding GET endpoint.

| Endpoint       | Retrieval Endpoint          |
| -------------- | --------------------------- |
| `scrape`       | `GET /v2/scrape/{id}`       |
| `crawl`        | `GET /v2/crawl/{id}`        |
| `batch_scrape` | `GET /v2/batch/scrape/{id}` |
| `agent`        | `GET /v2/extract/{id}`      |


# Ask
Source: https://docs.firecrawl.dev/api-reference/endpoint/ask

/api-reference/v2-openapi.json POST /support/ask

The `/support/ask` endpoint is an AI support agent that diagnoses issues with your Firecrawl jobs, account, and API usage. Send a question and receive a verified answer with actionable fix parameters — typically in 15–30 seconds.

## Designed for AI agents

`/support/ask` is built for **agent-to-agent** communication. If you're building an AI agent that uses Firecrawl, wire this endpoint into your error-handling flow so your agent can self-diagnose scraping failures, crawl issues, and configuration problems without human intervention.

Pass a `rationale` field to give the support agent context about what your end user is trying to accomplish. This helps prioritize the evidence gathering.

## How it works

1. **You describe the problem** — a natural-language question describing the issue.
2. **The agent investigates** — it inspects job logs, account state, documentation, and source code.
3. **The agent validates** — when possible, the agent tests a fix against the live Firecrawl API (e.g., retrying a scrape with adjusted parameters).
4. **You get a verified answer** — the response includes a prose `answer`, machine-readable `fixParameters` you can apply directly, and `validation` results showing whether the fix was tested.

## Authentication

Uses your Firecrawl API key as the bearer token. The request is automatically scoped to your team — you can only query your own jobs and account data.

```bash theme={null}
curl -X POST https://api.firecrawl.dev/v2/support/ask \
  -H "Authorization: Bearer fc-YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "question": "my crawl returned 3 pages but I expected 50",
    "rationale": "user is on their third failed crawl attempt today"
  }'
```

## Response fields

| Field           | Type           | Description                                                                          |
| --------------- | -------------- | ------------------------------------------------------------------------------------ |
| `answer`        | string         | 2-4 sentence prose covering the diagnosis and fix                                    |
| `confidence`    | string         | `high`, `medium`, or `low`                                                           |
| `fixParameters` | object \| null | API parameters to apply the fix (e.g., `{"waitFor": 5000}`)                          |
| `validation`    | object \| null | Whether the fix was tested: `tested`, `result` (success/failure/skipped), `evidence` |
| `feedback`      | object \| null | Present when the agent gets stuck; `{ blockedBy, attempted }`. Null on success.      |
| `durationMs`    | integer        | Total execution time in milliseconds                                                 |

## Status codes

| Code  | Meaning                                         |
| ----- | ----------------------------------------------- |
| `200` | Answered or stuck (envelope always returned)    |
| `400` | Invalid JSON or schema violation                |
| `401` | Missing or invalid bearer token                 |
| `504` | Hit 60s hard budget — partial envelope returned |

For the feature guide with integration examples, see the [Ask feature documentation](/features/ask).

> Are you an AI agent that needs a Firecrawl API key? See [firecrawl.dev/agent-onboarding/SKILL.md](https://www.firecrawl.dev/agent-onboarding/SKILL.md) for automated onboarding instructions.


# Batch Scrape
Source: https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape

/api-reference/v2-openapi.json POST /batch/scrape

> Are you an AI agent that needs a Firecrawl API key? See [firecrawl.dev/agent-onboarding/SKILL.md](https://www.firecrawl.dev/agent-onboarding/SKILL.md) for automated onboarding instructions.


# Cancel Batch Scrape
Source: https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape-delete

/api-reference/v2-openapi.json DELETE /batch/scrape/{id}

> Are you an AI agent that needs a Firecrawl API key? See [firecrawl.dev/agent-onboarding/SKILL.md](https://www.firecrawl.dev/agent-onboarding/SKILL.md) for automated onboarding instructions.


# Get Batch Scrape Status
Source: https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape-get

/api-reference/v2-openapi.json GET /batch/scrape/{id}

> Are you an AI agent that needs a Firecrawl API key? See [firecrawl.dev/agent-onboarding/SKILL.md](https://www.firecrawl.dev/agent-onboarding/SKILL.md) for automated onboarding instructions.


# Get Batch Scrape Errors
Source: https://docs.firecrawl.dev/api-reference/endpoint/batch-scrape-get-errors

/api-reference/v2-openapi.json GET /batch/scrape/{id}/errors

> Are you an AI agent that needs a Firecrawl API key? See [firecrawl.dev/agent-onboarding/SKILL.md](https://www.firecrawl.dev/agent-onboarding/SKILL.md) for automated onboarding instructions.


# List Browser Sessions
Source: https://docs.firecrawl.dev/api-reference/endpoint/browser-list

/api-reference/v2-openapi.json GET /browser
Retrieve a list of all browser sessions, optionally filtered by status.

## Headers

| Header          | Value              |
| --------------- | ------------------ |
| `Authorization` | `Bearer <API_KEY>` |

## Query Parameters

| Parameter | Type   | Required | Description                                           |
| --------- | ------ | -------- | ----------------------------------------------------- |
| `status`  | string | No       | Filter by session status: `"active"` or `"destroyed"` |

## Response

| Field      | Type    | Description                   |
| ---------- | ------- | ----------------------------- |
| `success`  | boolean | Whether the request succeeded |
| `sessions` | array   | List of session objects       |

### Session Object

| Field                    | Type   | Description                                                         |
| ------------------------ | ------ | ------------------------------------------------------------------- |
| `id`                     | string | Unique session identifier                                           |
| `status`                 | string | Current session status (`"active"` or `"destroyed"`)                |
| `cdpUrl`                 | string | WebSocket URL for CDP connections                                   |
| `liveViewUrl`            | string | URL to watch the session in real time                               |
| `interactiveLiveViewUrl` | string | URL to interact with the session in real time (click, type, scroll) |
| `createdAt`              | string | ISO 8601 timestamp of session creation                              |
| `lastActivity`           | string | ISO 8601 timestamp of last activity                                 |

### Example Request

```bash theme={null}
curl -X GET "https://api.firecrawl.dev/v2/browser?status=active" \
  -H "Authorization: Bearer $FIRECRAWL_API_KEY"
```

### Example Response

```json theme={null}
{
  "success": true,
  "sessions": [
    {
      "id": "550e8400-e29b-41d4-a716-446655440000",
      "status": "active",
      "cdpUrl": "wss://cdp-proxy.firecrawl.dev/cdp/550e8400-e29b-41d4-a716-446655440000",
      "liveViewUrl": "https://liveview.firecrawl.dev/550e8400-e29b-41d4-a716-446655440000",
      "interactiveLiveViewUrl": "https://liveview.firecrawl.dev/550e8400-e29b-41d4-a716-446655440000?interactive=true",
      "createdAt": "2025-06-01T12:00:00Z",
      "lastActivity": "2025-06-01T12:05:30Z"
    }
  ]
}
```

> Are you an AI agent that needs a Firecrawl API key? See [firecrawl.dev/agent-onboarding/SKILL.md](https://www.firecrawl.dev/agent-onboarding/SKILL.md) for automated onboarding instructions.


# Get Active Crawls
Source: https://docs.firecrawl.dev/api-reference/endpoint/crawl-active

/api-reference/v2-openapi.json GET /crawl/active

> Are you an AI agent that needs a Firecrawl API key? See [firecrawl.dev/agent-onboarding/SKILL.md](https://www.firecrawl.dev/agent-onboarding/SKILL.md) for automated onboarding instructions.


# Cancel Crawl
Source: https://docs.firecrawl.dev/api-reference/endpoint/crawl-delete

/api-reference/v2-openapi.json DELETE /crawl/{id}

> Are you an AI agent that needs a Firecrawl API key? See [firecrawl.dev/agent-onboarding/SKILL.md](https://www.firecrawl.dev/agent-onboarding/SKILL.md) for automated onboarding instructions.


# Get Crawl Status
Source: https://docs.firecrawl.dev/api-reference/endpoint/crawl-get

/api-reference/v2-openapi.json GET /crawl/{id}

> Are you an AI agent that needs a Firecrawl API key? See [firecrawl.dev/agent-onboarding/SKILL.md](https://www.firecrawl.dev/agent-onboarding/SKILL.md) for automated onboarding instructions.


# Get Crawl Errors
Source: https://docs.firecrawl.dev/api-reference/endpoint/crawl-get-errors

/api-reference/v2-openapi.json GET /crawl/{id}/errors

> Are you an AI agent that needs a Firecrawl API key? See [firecrawl.dev/agent-onboarding/SKILL.md](https://www.firecrawl.dev/agent-onboarding/SKILL.md) for automated onboarding instructions.


# Crawl Params Preview
Source: https://docs.firecrawl.dev/api-reference/endpoint/crawl-params-preview

/api-reference/v2-openapi.json POST /crawl/params-preview



# Crawl
Source: https://docs.firecrawl.dev/api-reference/endpoint/crawl-post

/api-reference/v2-openapi.json POST /crawl

> Are you an AI agent that needs a Firecrawl API key? See [firecrawl.dev/agent-onboarding/SKILL.md](https://www.firecrawl.dev/agent-onboarding/SKILL.md) for automated onboarding instructions.


# Credit Usage
Source: https://docs.firecrawl.dev/api-reference/endpoint/credit-usage

/api-reference/v2-openapi.json GET /team/credit-usage

> Are you an AI agent that needs a Firecrawl API key? See [firecrawl.dev/agent-onboarding/SKILL.md](https://www.firecrawl.dev/agent-onboarding/SKILL.md) for automated onboarding instructions.


# Historical Credit Usage
Source: https://docs.firecrawl.dev/api-reference/endpoint/credit-usage-historical

/api-reference/v2-openapi.json GET /team/credit-usage/historical

Returns historical credit usage on a month-by-month basis. The endpoint can also breaks usage down by API key optionally.

> Are you an AI agent that needs a Firecrawl API key? See [firecrawl.dev/agent-onboarding/SKILL.md](https://www.firecrawl.dev/agent-onboarding/SKILL.md) for automated onboarding instructions.


# Docs Search
Source: https://docs.firecrawl.dev/api-reference/endpoint/docs-search

/api-reference/v2-openapi.json POST /support/docs-search

The `/support/docs-search` endpoint answers questions using Firecrawl's public documentation. Requires your Firecrawl API key. Returns a concise, docs-grounded answer with citations to the relevant documentation pages.

## Use cases

* **AI agents** that need to look up Firecrawl API usage, parameters, or best practices
* **Support bots** that answer customer questions from the docs
* **Developer tools** that surface relevant documentation inline

## Example

```bash theme={null}
curl -X POST https://api.firecrawl.dev/v2/support/docs-search \
  -H "Authorization: Bearer fc-YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "question": "what header carries the webhook signature, and how do I verify it?"
  }'
```

### Response

```json theme={null}
{
  "requestId": "req_...",
  "answer": "The signature is sent in the X-Firecrawl-Signature header...",
  "evidence": [
    {
      "pathOrUrl": "webhooks/security.mdx#L1-L52",
      "reason": "Documents webhook signature verification"
    }
  ],
  "usage": { "inputTokens": 4356, "outputTokens": 688, "totalTokens": 5044 },
  "durationMs": 11252
}
```

## Response fields

| Field        | Type    | Description                                                      |
| ------------ | ------- | ---------------------------------------------------------------- |
| `answer`     | string  | Concise answer grounded in Firecrawl documentation               |
| `evidence`   | array   | Documentation pages referenced, with `pathOrUrl` and `reason`    |
| `usage`      | object  | Token consumption (`inputTokens`, `outputTokens`, `totalTokens`) |
| `durationMs` | integer | Total execution time in milliseconds                             |

For the full feature guide, see the [Ask feature documentation](/features/ask).

> Are you an AI agent that needs a Firecrawl API key? See [firecrawl.dev/agent-onboarding/SKILL.md](https://www.firecrawl.dev/agent-onboarding/SKILL.md) for automated onboarding instructions.


# Map
Source: https://docs.firecrawl.dev/api-reference/endpoint/map

/api-reference/v2-openapi.json POST /map

> Are you an AI agent that needs a Firecrawl API key? See [firecrawl.dev/agent-onboarding/SKILL.md](https://www.firecrawl.dev/agent-onboarding/SKILL.md) for automated onboarding instructions.


# Parse
Source: https://docs.firecrawl.dev/api-reference/endpoint/parse

/api-reference/v2-openapi.json POST /parse

Upload a local or non-public document and convert it into clean, LLM-ready data. `/parse` accepts file bytes via `multipart/form-data` and returns Markdown, JSON, HTML, links, images, or a summary — with reading order and tables preserved.

* Turn PDF, DOCX, XLSX, HTML, and more into Markdown or structured JSON
* Up to **5x faster** parsing via a Rust-based engine
* Files up to **50 MB** per request
* Zero Data Retention support

## When to use `/parse`

Use `/parse` when the source document is **a local file** or **not publicly accessible by URL**. If you have a public URL that points to a document, prefer [`/scrape`](/api-reference/endpoint/scrape) — it auto-detects the file type from the extension or content type and parses it the same way.

| Source                                                           | Endpoint                                         |
| ---------------------------------------------------------------- | ------------------------------------------------ |
| Public URL to a document (e.g. `https://example.com/report.pdf`) | [`POST /scrape`](/api-reference/endpoint/scrape) |
| Local file or non-public bytes (PDF, DOCX, XLSX, HTML, …)        | `POST /parse` (this endpoint)                    |


# Queue Status
Source: https://docs.firecrawl.dev/api-reference/endpoint/queue-status

/api-reference/v2-openapi.json GET /team/queue-status

Metrics about your team's scrape queue.

> Are you an AI agent that needs a Firecrawl API key? See [firecrawl.dev/agent-onboarding/SKILL.md](https://www.firecrawl.dev/agent-onboarding/SKILL.md) for automated onboarding instructions.


# Scrape
Source: https://docs.firecrawl.dev/api-reference/endpoint/scrape

/api-reference/v2-openapi.json POST /scrape

## Interactions

For browser interactions (clicking, typing, navigating, extracting dynamic content), use the [Interact endpoint](/features/interact). Scrape a page first, then call `POST /v2/scrape/{scrapeId}/interact` with a natural-language prompt or Playwright code to take actions on the page.

See the [Interact documentation](/features/interact) for full details and examples.

Optionally you can also use the `actions` parameter, although it's not recommended to use it for complex interactions.

> Are you an AI agent that needs a Firecrawl API key? See [firecrawl.dev/agent-onboarding/SKILL.md](https://www.firecrawl.dev/agent-onboarding/SKILL.md) for automated onboarding instructions.


# Stop Interacting
Source: https://docs.firecrawl.dev/api-reference/endpoint/scrape-browser-delete

/api-reference/v2-openapi.json DELETE /scrape/{jobId}/interact
Stop the interactive browser session associated with a scrape job.

Use this endpoint to stop the interactive browser session for a scrape job. Stopping the session releases browser resources and finalizes billing.

Credits are billed based on session duration: **2 credits per browser minute**, prorated by the second.

## Path Parameters

| Parameter | Type          | Required | Description                                           |
| --------- | ------------- | -------- | ----------------------------------------------------- |
| `jobId`   | string (UUID) | Yes      | The scrape job ID associated with the browser session |

## Response

| Field               | Type    | Description                                    |
| ------------------- | ------- | ---------------------------------------------- |
| `success`           | boolean | Whether the session was successfully destroyed |
| `sessionDurationMs` | number  | Total session duration in milliseconds         |
| `creditsBilled`     | number  | Number of credits billed for the session       |
| `error`             | string  | Error message (only present on failure)        |

### Example Request

```bash theme={null}
curl -X DELETE "https://api.firecrawl.dev/v2/scrape/550e8400-e29b-41d4-a716-446655440000/interact" \
  -H "Authorization: Bearer $FIRECRAWL_API_KEY"
```

### Example Response

```json theme={null}
{
  "success": true
}
```

### Error Codes

| Status | Description                                  |
| ------ | -------------------------------------------- |
| `403`  | Session belongs to a different team          |
| `404`  | No browser session found for this scrape job |

For detailed usage with examples, see the [Interact feature guide](/features/interact).


# Interact with the page
Source: https://docs.firecrawl.dev/api-reference/endpoint/scrape-execute

/api-reference/v2-openapi.json POST /scrape/{jobId}/interact
Execute code or an AI prompt in the browser session bound to a scrape job.

Use this endpoint to continue interacting with the same browser state initialized from a previous scrape. Either `code` or `prompt` must be provided — not both.

`POST /v2/scrape/{jobId}/interact` handles the full lifecycle:

1. If no browser session exists for this scrape job yet, Firecrawl creates one at the same page state as the original scrape.
2. When `code` is provided, Firecrawl runs it in the browser sandbox. When `prompt` is provided, an AI agent automates the task using natural language.
3. Later `POST /interact` calls on the same `jobId` reuse the same live browser state.

When you are done, call `DELETE /v2/scrape/{jobId}/interact` to stop the session.

## Path Parameters

| Parameter | Type          | Required | Description                                                            |
| --------- | ------------- | -------- | ---------------------------------------------------------------------- |
| `jobId`   | string (UUID) | Yes      | The scrape job ID from `data.metadata.scrapeId` in the scrape response |

## Request Body

| Parameter  | Type   | Required | Default  | Description                                                                                |
| ---------- | ------ | -------- | -------- | ------------------------------------------------------------------------------------------ |
| `code`     | string | No       | —        | Code to execute in the browser sandbox (1–100,000 chars). Required if `prompt` is not set. |
| `prompt`   | string | No       | —        | Natural language task for the AI agent (1–10,000 chars). Required if `code` is not set.    |
| `language` | string | No       | `"node"` | One of `"python"`, `"node"`, or `"bash"`. Only used with `code`.                           |
| `timeout`  | number | No       | `30`     | Execution timeout in seconds (1–300).                                                      |
| `origin`   | string | No       | —        | Optional origin label used for telemetry.                                                  |

## Response

| Field                    | Type    | Description                                                                        |
| ------------------------ | ------- | ---------------------------------------------------------------------------------- |
| `success`                | boolean | Whether the execution completed without errors                                     |
| `liveViewUrl`            | string  | Read-only live view URL for the browser session                                    |
| `interactiveLiveViewUrl` | string  | Interactive live view URL (viewers can control the browser)                        |
| `output`                 | string  | AI agent's final response (only present when using `prompt`)                       |
| `stdout`                 | string  | Standard output from the code execution                                            |
| `result`                 | string  | Return value — last expression value for Node.js, final page snapshot for `prompt` |
| `stderr`                 | string  | Standard error output                                                              |
| `exitCode`               | number  | Exit code of the execution (`0` = success)                                         |
| `killed`                 | boolean | Whether the execution was terminated due to timeout                                |
| `error`                  | string  | Error message (only present on failure)                                            |

### Example Request (Code)

```bash theme={null}
curl -X POST "https://api.firecrawl.dev/v2/scrape/550e8400-e29b-41d4-a716-446655440000/interact" \
  -H "Authorization: Bearer $FIRECRAWL_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "code": "const title = await page.title(); JSON.stringify({ title });",
    "language": "node",
    "timeout": 30
  }'
```

### Example Response (Code)

```json theme={null}
{
  "success": true,
  "liveViewUrl": "https://liveview.firecrawl.dev/...",
  "interactiveLiveViewUrl": "https://liveview.firecrawl.dev/...",
  "stdout": "",
  "result": "{\"title\":\"Example Domain\"}",
  "stderr": "",
  "exitCode": 0,
  "killed": false
}
```

### Example Request (Prompt)

```bash theme={null}
curl -X POST "https://api.firecrawl.dev/v2/scrape/550e8400-e29b-41d4-a716-446655440000/interact" \
  -H "Authorization: Bearer $FIRECRAWL_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "prompt": "Find the pricing section and tell me the price of the Pro plan",
    "timeout": 60
  }'
```

### Example Response (Prompt)

```json theme={null}
{
  "success": true,
  "liveViewUrl": "https://liveview.firecrawl.dev/...",
  "interactiveLiveViewUrl": "https://liveview.firecrawl.dev/...",
  "output": "The Pro plan costs $49/month and includes unlimited scrapes, priority support, and custom integrations.",
  "stdout": "...",
  "result": "...",
  "stderr": "",
  "exitCode": 0,
  "killed": false
}
```

### Error Codes

| Status | Description                                                 |
| ------ | ----------------------------------------------------------- |
| `402`  | Insufficient credits for a browser session                  |
| `403`  | Scrape job belongs to a different team                      |
| `404`  | Scrape job not found                                        |
| `409`  | Replay context unavailable — rerun the scrape and try again |
| `410`  | Browser session has already been destroyed                  |
| `429`  | Maximum concurrent browser sessions reached                 |
| `502`  | Browser service or AI agent execution failed                |
| `503`  | Browser feature not configured (self-hosted only)           |

For detailed usage with examples, see the [Interact feature guide](/features/interact).


# Search
Source: https://docs.firecrawl.dev/api-reference/endpoint/search

/api-reference/v2-openapi.json POST /search

The search endpoint combines web search with Firecrawl's scraping capabilities to return full page content for any query.

Include `scrapeOptions` with `formats: [{"type": "markdown"}]` to get complete markdown content for each search result otherwise you will default to getting the results (url, title, description). You can also use other formats like `{"type": "summary"}` for condensed content.

## Supported query operators

We support a variety of query operators that allow you to filter your searches better.

| Operator      | Functionality                                                             | Examples                          |
| ------------- | ------------------------------------------------------------------------- | --------------------------------- |
| `""`          | Non-fuzzy matches a string of text                                        | `"Firecrawl"`                     |
| `-`           | Excludes certain keywords or negates other operators                      | `-bad`, `-site:firecrawl.dev`     |
| `site:`       | Only returns results from a specified website                             | `site:firecrawl.dev`              |
| `filetype:`   | Only returns results with a specific file extension                       | `filetype:pdf`, `-filetype:pdf`   |
| `inurl:`      | Only returns results that include a word in the URL                       | `inurl:firecrawl`                 |
| `allinurl:`   | Only returns results that include multiple words in the URL               | `allinurl:git firecrawl`          |
| `intitle:`    | Only returns results that include a word in the title of the page         | `intitle:Firecrawl`               |
| `allintitle:` | Only returns results that include multiple words in the title of the page | `allintitle:firecrawl playground` |
| `related:`    | Only returns results that are related to a specific domain                | `related:firecrawl.dev`           |
| `imagesize:`  | Only returns images with exact dimensions                                 | `imagesize:1920x1080`             |
| `larger:`     | Only returns images larger than specified dimensions                      | `larger:1920x1080`                |

## Location Parameter

Use the `location` parameter to get geo-targeted search results. Format: `"string"`. Examples: `"Germany"`, `"San Francisco,California,United States"`.

See the [complete list of supported locations](https://firecrawl.dev/search_locations.json) for all available countries and languages.

## Country Parameter

Use the `country` parameter to specify the country for search results using ISO country codes. Default: `"US"`.

Examples: `"US"`, `"DE"`, `"FR"`, `"JP"`, `"UK"`, `"CA"`.

```json theme={null}
{
  "query": "restaurants",
  "country": "DE"
}
```

## Categories Parameter

Filter search results by specific categories using the `categories` parameter:

* **`github`**: Search within GitHub repositories, code, issues, and documentation
* **`research`**: Search academic and research websites (arXiv, Nature, IEEE, PubMed, etc.)
* **`pdf`**: Search for PDFs

### Example Usage

```json theme={null}
{
  "query": "machine learning",
  "categories": ["github", "research"],
  "limit": 10
}
```

## Domain Filters

Use `includeDomains` to restrict results to specific domains, or `excludeDomains` to remove specific domains from the search. Domains should be hostnames only, without protocol or path.

`includeDomains` and `excludeDomains` are mutually exclusive.

### Include Domains Example

```json theme={null}
{
  "query": "web scraping",
  "includeDomains": ["firecrawl.dev", "docs.firecrawl.dev"],
  "limit": 10
}
```

### Exclude Domains Example

```json theme={null}
{
  "query": "web scraping tools",
  "excludeDomains": ["example.com"],
  "limit": 10
}
```

### Category Response

Each result includes a `category` field indicating its source:

```json theme={null}
{
  "success": true,
  "data": {
    "web": [
      {
        "url": "https://github.com/example/ml-project",
        "title": "Machine Learning Project",
        "description": "Implementation of ML algorithms",
        "category": "github"
      },
      {
        "url": "https://arxiv.org/abs/2024.12345",
        "title": "ML Research Paper",
        "description": "Latest advances in machine learning",
        "category": "research"
      }
    ]
  }
}
```

## Time-Based Search

Use the `tbs` parameter to filter results by time periods, including custom date ranges. See the [Search Feature documentation](https://docs.firecrawl.dev/features/search#time-based-search) for detailed examples and supported formats.

> Are you an AI agent that needs a Firecrawl API key? See [firecrawl.dev/agent-onboarding/SKILL.md](https://www.firecrawl.dev/agent-onboarding/SKILL.md) for automated onboarding instructions.


# Token Usage
Source: https://docs.firecrawl.dev/api-reference/endpoint/token-usage

/api-reference/v2-openapi.json GET /team/token-usage

<Info>We've simplified billing so that Extract now uses credits, just like all of the other endpoints. Each credit is worth 15 tokens. Reported token usage now includes usage from all endpoints.</Info>

> Are you an AI agent that needs a Firecrawl API key? See [firecrawl.dev/agent-onboarding/SKILL.md](https://www.firecrawl.dev/agent-onboarding/SKILL.md) for automated onboarding instructions.


# Historical Token Usage
Source: https://docs.firecrawl.dev/api-reference/endpoint/token-usage-historical

/api-reference/v2-openapi.json GET /team/token-usage/historical

Returns historical token usage on a month-by-month basis. The endpoint can also breaks usage down by API key optionally.

<Info>We've simplified billing so that Extract now uses credits, just like all of the other endpoints. Each credit is worth 15 tokens. Reported token usage now includes usage from all endpoints.</Info>

> Are you an AI agent that needs a Firecrawl API key? See [firecrawl.dev/agent-onboarding/SKILL.md](https://www.firecrawl.dev/agent-onboarding/SKILL.md) for automated onboarding instructions.


# Batch Scrape Completed
Source: https://docs.firecrawl.dev/api-reference/endpoint/webhook-batch-scrape-completed

/api-reference/webhooks-openapi.json webhook batchScrapeCompleted
Webhook event sent when all URLs in a batch scrape have been processed.

For payload examples, configuration, and retry behavior, see [Webhook Event Types](/webhooks/events#batch_scrapecompleted) and [Webhook Overview](/webhooks/overview).


# Batch Scrape Page
Source: https://docs.firecrawl.dev/api-reference/endpoint/webhook-batch-scrape-page

/api-reference/webhooks-openapi.json webhook batchScrapePage
Webhook event sent for each URL scraped during a batch scrape job.

For payload examples, configuration, and retry behavior, see [Webhook Event Types](/webhooks/events#batch_scrapepage) and [Webhook Overview](/webhooks/overview).


# Batch Scrape Started
Source: https://docs.firecrawl.dev/api-reference/endpoint/webhook-batch-scrape-started

/api-reference/webhooks-openapi.json webhook batchScrapeStarted
Webhook event sent when a batch scrape job begins processing.

For payload examples, configuration, and retry behavior, see [Webhook Event Types](/webhooks/events#batch_scrapestarted) and [Webhook Overview](/webhooks/overview).


# Crawl Completed
Source: https://docs.firecrawl.dev/api-reference/endpoint/webhook-crawl-completed

/api-reference/webhooks-openapi.json webhook crawlCompleted
Webhook event sent when a crawl job finishes and all pages have been processed.

For payload examples, configuration, and retry behavior, see [Webhook Event Types](/webhooks/events#crawlcompleted) and [Webhook Overview](/webhooks/overview).


# Crawl Page
Source: https://docs.firecrawl.dev/api-reference/endpoint/webhook-crawl-page

/api-reference/webhooks-openapi.json webhook crawlPage
Webhook event sent for each page scraped during a crawl job.

For payload examples, configuration, and retry behavior, see [Webhook Event Types](/webhooks/events#crawlpage) and [Webhook Overview](/webhooks/overview).


# Crawl Started
Source: https://docs.firecrawl.dev/api-reference/endpoint/webhook-crawl-started

/api-reference/webhooks-openapi.json webhook crawlStarted
Webhook event sent when a crawl job begins processing.

For payload examples, configuration, and retry behavior, see [Webhook Event Types](/webhooks/events#crawlstarted) and [Webhook Overview](/webhooks/overview).


# Errors
Source: https://docs.firecrawl.dev/api-reference/errors

Every API error code, what causes it, how to remedy it, and whether to retry.

Every Firecrawl error response uses the same JSON shape. Look up the `error` value (or HTTP status) in the table below to find the cause, remedy, and whether the request is safe to retry.

<Note>This catalog covers the errors most agents and clients will encounter. It is non-exhaustive — if you receive an error not listed here, please [open an issue](https://github.com/firecrawl/firecrawl/issues) so we can document it.</Note>

## Error response shape

All non-2xx responses return JSON with a top-level `success: false` and a string `error`. Some endpoints include additional fields (`details`, `code`) when more context is available.

```json theme={null}
{
  "success": false,
  "error": "Unauthorized: Invalid token",
  "details": "Optional structured details (only present on some errors)"
}
```

| Field     | Type      | Description                                                       |
| --------- | --------- | ----------------------------------------------------------------- |
| `success` | `boolean` | Always `false` on errors.                                         |
| `error`   | `string`  | Human-readable error message. Use this to look up the row below.  |
| `details` | `any`     | Optional. Structured per-field validation errors when applicable. |

## Errors

| HTTP | `error` (typical message)                        | Cause                                                                              | Remedy                                                                                                             | Retryable         |
| ---- | ------------------------------------------------ | ---------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------ | ----------------- |
| 400  | `Bad Request` / validation message               | Request body failed schema validation (missing or invalid fields).                 | Fix the request payload using the endpoint reference. Check `details` for fields.                                  | No                |
| 400  | `Invalid URL`                                    | The `url` field is missing, malformed, or uses an unsupported scheme.              | Pass an absolute `http(s)://` URL.                                                                                 | No                |
| 401  | `Unauthorized: Invalid token`                    | API key is missing, malformed, or revoked.                                         | Send `Authorization: Bearer fc-...` with a valid key from the [dashboard](https://www.firecrawl.dev/app/api-keys). | No                |
| 402  | `Payment Required: Insufficient credits`         | Plan credits are exhausted or billing is not configured.                           | Top up credits, enable auto-recharge, or upgrade your plan.                                                        | No                |
| 403  | `Forbidden`                                      | Key lacks permission for this endpoint or feature.                                 | Use a key with the required scope, or upgrade the plan that gates this feature.                                    | No                |
| 404  | `Not Found`                                      | The job ID, resource, or endpoint path does not exist.                             | Verify the resource ID and endpoint URL.                                                                           | No                |
| 408  | `Request Timeout`                                | The page took longer than the request `timeout` to load.                           | Increase `timeout`, simplify actions, or use `fastMode`.                                                           | Yes, with backoff |
| 409  | `Conflict`                                       | Resource is in a state that prevents the operation (e.g. already deleted).         | Re-fetch state and reconcile before retrying.                                                                      | No                |
| 413  | `Payload Too Large`                              | Request body exceeded the maximum allowed size.                                    | Reduce the payload (e.g. shorter schema, fewer URLs per batch).                                                    | No                |
| 422  | `Unprocessable Entity` / extraction schema error | Schema is invalid JSON Schema, or the model could not produce a conforming result. | Validate the schema; loosen required fields; try a different `model`.                                              | Sometimes         |
| 429  | `Rate limit exceeded`                            | Too many requests for your plan's per-minute limit.                                | Back off and retry after `Retry-After` seconds. See [Rate Limits](/rate-limits).                                   | Yes, with backoff |
| 429  | `Concurrency limit reached`                      | Concurrent browser limit for your plan reached.                                    | Wait for in-flight jobs to finish, lower concurrency, or upgrade your plan.                                        | Yes, with backoff |
| 500  | `Internal Server Error`                          | Unhandled server-side failure.                                                     | Retry with exponential backoff. If it persists, contact support with the request ID.                               | Yes, with backoff |
| 502  | `Bad Gateway`                                    | Upstream proxy or worker returned an invalid response.                             | Retry with backoff.                                                                                                | Yes, with backoff |
| 503  | `Service Unavailable`                            | Service temporarily unable to handle the request.                                  | Retry with backoff.                                                                                                | Yes, with backoff |
| 504  | `Gateway Timeout`                                | Request exceeded the gateway's timeout (typically long crawls).                    | Use the async crawl/batch endpoints and poll status instead.                                                       | Yes, with backoff |

For 429 responses, Firecrawl includes a `Retry-After` header (in seconds) when available — wait at least that long before retrying.

## Retry guidance

Treat the **Retryable** column as authoritative; do not infer from the HTTP status alone. The pattern below uses exponential backoff with jitter and respects `Retry-After` on 429.

<CodeGroup>
  ```python Python theme={null}
  import time
  import random
  import requests

  RETRYABLE_STATUSES = {408, 429, 500, 502, 503, 504}

  def request_with_retry(method, url, headers=None, json=None, max_attempts=5):
      for attempt in range(max_attempts):
          resp = requests.request(method, url, headers=headers, json=json)
          if resp.status_code < 400 or resp.status_code not in RETRYABLE_STATUSES:
              return resp
          # Honor Retry-After when present, otherwise exponential backoff with jitter.
          retry_after = resp.headers.get("Retry-After")
          delay = float(retry_after) if retry_after else min(2 ** attempt, 30) + random.random()
          time.sleep(delay)
      return resp
  ```

  ```js Node theme={null}
  const RETRYABLE_STATUSES = new Set([408, 429, 500, 502, 503, 504]);

  async function requestWithRetry(url, init = {}, maxAttempts = 5) {
    for (let attempt = 0; attempt < maxAttempts; attempt++) {
      const resp = await fetch(url, init);
      if (resp.ok || !RETRYABLE_STATUSES.has(resp.status)) return resp;
      // Honor Retry-After when present, otherwise exponential backoff with jitter.
      const retryAfter = resp.headers.get('Retry-After');
      const delayMs = retryAfter
        ? Number(retryAfter) * 1000
        : Math.min(2 ** attempt, 30) * 1000 + Math.random() * 1000;
      await new Promise((r) => setTimeout(r, delayMs));
    }
  }
  ```

  ```bash cURL theme={null}
  # Simple shell loop with exponential backoff for retryable statuses.
  attempt=0
  max=5
  until response=$(curl -sS -w "\n%{http_code}" -X POST "https://api.firecrawl.dev/v2/scrape" \
      -H "Authorization: Bearer $FIRECRAWL_API_KEY" \
      -H "Content-Type: application/json" \
      -d '{"url":"https://example.com"}'); do
    status=$(printf '%s' "$response" | tail -n1)
    case "$status" in
      408|429|500|502|503|504)
        attempt=$((attempt+1))
        [ "$attempt" -ge "$max" ] && break
        sleep $((2 ** attempt))
        ;;
      *) break ;;
    esac
  done
  echo "$response"
  ```
</CodeGroup>

## 429 responses

429 responses are the most common retryable error. Per-plan rate limits and concurrency limits are documented in [Rate Limits](/rate-limits). Always honor the `Retry-After` header when present rather than retrying immediately.


# Introduction
Source: https://docs.firecrawl.dev/api-reference/v2-introduction

Firecrawl API Reference (v2)

<Note>
  **For AI agents:** Use [llms.txt](/llms.txt) for a full index of all documentation.
</Note>

The Firecrawl API gives you programmatic access to web data. All endpoints share a common base URL, authentication scheme, and response format described on this page.

## Features

<CardGroup>
  <Card title="Scrape" icon="markdown" href="/api-reference/endpoint/scrape">
    Extract content from any webpage in markdown or json format.
  </Card>

  <Card title="Parse" icon="markdown" href="/api-reference/endpoint/parse">
    Upload files and parse them into markdown or other formats.
  </Card>

  <Card title="Crawl" icon="spider" href="/api-reference/endpoint/crawl-post">
    Crawl entire websites and get content from all pages.
  </Card>

  <Card title="Map" icon="map" href="/api-reference/endpoint/map">
    Get a complete list of URLs from any website quickly and reliably.
  </Card>

  <Card title="Search" icon="magnifying-glass" href="/api-reference/endpoint/search">
    Search the web and get full page content in any format.
  </Card>
</CardGroup>

## Agentic Features

<CardGroup>
  <Card title="Agent" icon="robot" href="/api-reference/endpoint/agent">
    Autonomous web data gathering powered by AI.
  </Card>

  <Card title="Browser" icon="browser" href="/api-reference/endpoint/browser-create">
    Create and control browser sessions for interactive web tasks.
  </Card>
</CardGroup>

## Base URL

All requests use the following base URL:

```bash theme={null}
https://api.firecrawl.dev
```

## Authentication

Every request requires an `Authorization` header with your API key:

```bash theme={null}
Authorization: Bearer fc-YOUR-API-KEY
```

Include this header in all API calls. You can find your API key in the [Firecrawl dashboard](https://www.firecrawl.dev/app/api-keys).

<CodeGroup>
  ```bash cURL theme={null}
  curl -X POST "https://api.firecrawl.dev/v2/scrape" \
    -H "Authorization: Bearer fc-YOUR-API-KEY" \
    -H "Content-Type: application/json" \
    -d '{"url": "https://example.com"}'
  ```

  ```python Python theme={null}
  from firecrawl import Firecrawl

  firecrawl = Firecrawl(api_key="fc-YOUR-API-KEY")

  result = firecrawl.scrape("https://example.com")
  ```

  ```js Node theme={null}
  import Firecrawl from '@mendable/firecrawl-js';

  const firecrawl = new Firecrawl({ apiKey: "fc-YOUR-API-KEY" });

  const result = await firecrawl.scrape('https://example.com');
  ```
</CodeGroup>

## Response codes

Firecrawl uses conventional HTTP status codes to indicate the outcome of your requests. Codes in the `2xx` range indicate success, `4xx` codes indicate client errors, and `5xx` codes indicate server errors.

See [Errors](/api-reference/errors) for the full reference, including the `error` string returned for each failure mode, retry guidance, and a copy-pasteable backoff snippet.

## 429 responses

When you exceed your plan's rate or concurrency limits, the API returns a `429` status code. See [Rate Limits](/rate-limits) for per-plan limits and [Errors](/api-reference/errors) for retry guidance.


# Building an AI Research Assistant with Firecrawl and AI SDK
Source: https://docs.firecrawl.dev/developer-guides/cookbooks/ai-research-assistant-cookbook

Build a complete AI-powered research assistant with web scraping and search capabilities

Build a complete AI-powered research assistant that can scrape websites and search the web to answer questions. The assistant automatically decides when to use web scraping or search tools to gather information, then provides comprehensive answers based on collected data.

<img alt="AI research assistant chatbot interface showing real-time web scraping with Firecrawl and conversational responses powered by OpenAI" />

## What You'll Build

An AI chat interface where users can ask questions about any topic. The AI assistant automatically decides when to use web scraping or search tools to gather information, then provides comprehensive answers based on the data it collects.

## Prerequisites

* Node.js 18 or later installed
* An OpenAI API key from [platform.openai.com](https://platform.openai.com)
* A Firecrawl API key from [firecrawl.dev](https://firecrawl.dev)
* Basic knowledge of React and Next.js

<Steps>
  <Step title="Create a New Next.js Project">
    Start by creating a fresh Next.js application and navigating into the project directory:

    ```bash theme={null}
    npx create-next-app@latest ai-sdk-firecrawl && cd ai-sdk-firecrawl
    ```

    When prompted, select the following options:

    * TypeScript: Yes
    * ESLint: Yes
    * Tailwind CSS: Yes
    * App Router: Yes
    * Use `src/` directory: No
    * Import alias: Yes (@/\*)
  </Step>

  <Step title="Install Dependencies">
    ### Install AI SDK Packages

    The AI SDK is a TypeScript toolkit that provides a unified API for working with different LLM providers:

    ```bash theme={null}
    npm i ai @ai-sdk/react zod
    ```

    These packages provide:

    * `ai`: Core SDK with streaming, tool calling, and response handling
    * `@ai-sdk/react`: React hooks like `useChat` for building chat interfaces
    * `zod`: Schema validation for tool inputs

    Learn more at [ai-sdk.dev/docs](https://ai-sdk.dev/docs).

    ### Install AI Elements

    AI Elements provides pre-built UI components for AI applications. Run the following command to scaffold all the necessary components:

    ```bash theme={null}
    npx ai-elements@latest
    ```

    This sets up AI Elements in your project, including conversation components, message displays, prompt inputs, and tool call visualizations.

    Documentation: [ai-sdk.dev/elements/overview](https://ai-sdk.dev/elements/overview).

    ### Install OpenAI Provider

    Install the OpenAI provider to connect with OpenAI's models:

    ```bash theme={null}
    npm install @ai-sdk/openai
    ```
  </Step>

  <Step title="Build the Frontend Chat Interface">
    Create the main page at `app/page.tsx` and copy the code from the Code tab below. This will be the chat interface where users interact with the AI assistant.

    <Tabs>
      <Tab title="Preview">
        <img alt="AI research assistant chatbot interface showing real-time web scraping with Firecrawl and conversational responses powered by OpenAI" />
      </Tab>

      <Tab title="Code">
        ```typescript app/page.tsx theme={null}
        "use client";

        import {
          Conversation,
          ConversationContent,
          ConversationScrollButton,
        } from "@/components/ai-elements/conversation";
        import {
          PromptInput,
          PromptInputActionAddAttachments,
          PromptInputActionMenu,
          PromptInputActionMenuContent,
          PromptInputActionMenuTrigger,
          PromptInputAttachment,
          PromptInputAttachments,
          PromptInputBody,
          PromptInputButton,
          PromptInputHeader,
          type PromptInputMessage,
          PromptInputSelect,
          PromptInputSelectContent,
          PromptInputSelectItem,
          PromptInputSelectTrigger,
          PromptInputSelectValue,
          PromptInputSubmit,
          PromptInputTextarea,
          PromptInputFooter,
          PromptInputTools,
        } from "@/components/ai-elements/prompt-input";
        import {
          MessageResponse,
          Message,
          MessageContent,
          MessageActions,
          MessageAction,
        } from "@/components/ai-elements/message";

        import { Fragment, useState } from "react";
        import { useChat } from "@ai-sdk/react";
        import type { ToolUIPart } from "ai";
        import {
          Tool,
          ToolContent,
          ToolHeader,
          ToolInput,
          ToolOutput,
        } from "@/components/ai-elements/tool";

        import { CopyIcon, GlobeIcon, RefreshCcwIcon } from "lucide-react";
        import {
          Source,
          Sources,
          SourcesContent,
          SourcesTrigger,
        } from "@/components/ai-elements/sources";
        import {
          Reasoning,
          ReasoningContent,
          ReasoningTrigger,
        } from "@/components/ai-elements/reasoning";
        import { Loader } from "@/components/ai-elements/loader";

        const models = [
          {
            name: "GPT 5 Mini (Thinking)",
            value: "gpt-5-mini",
          },
          {
            name: "GPT 4o Mini",
            value: "gpt-4o-mini",
          },
        ];

        const ChatBotDemo = () => {
          const [input, setInput] = useState("");
          const [model, setModel] = useState<string>(models[0].value);
          const [webSearch, setWebSearch] = useState(false);
          const { messages, sendMessage, status, regenerate } = useChat();

          const handleSubmit = (message: PromptInputMessage) => {
            const hasText = Boolean(message.text);
            const hasAttachments = Boolean(message.files?.length);

            if (!(hasText || hasAttachments)) {
              return;
            }

            sendMessage(
              {
                text: message.text || "Sent with attachments",
                files: message.files,
              },
              {
                body: {
                  model: model,
                  webSearch: webSearch,
                },
              }
            );
            setInput("");
          };

          return (
            <div className="max-w-4xl mx-auto p-6 relative size-full h-screen">
              <div className="flex flex-col h-full">
                <Conversation className="h-full">
                  <ConversationContent>
                    {messages.map((message) => (
                      <div key={message.id}>
                        {message.role === "assistant" &&
                          message.parts.filter((part) => part.type === "source-url")
                            .length > 0 && (
                            <Sources>
                              <SourcesTrigger
                                count={
                                  message.parts.filter(
                                    (part) => part.type === "source-url"
                                  ).length
                                }
                              />
                              {message.parts
                                .filter((part) => part.type === "source-url")
                                .map((part, i) => (
                                  <SourcesContent key={`${message.id}-${i}`}>
                                    <Source
                                      key={`${message.id}-${i}`}
                                      href={part.url}
                                      title={part.url}
                                    />
                                  </SourcesContent>
                                ))}
                            </Sources>
                          )}
                        {message.parts.map((part, i) => {
                          switch (part.type) {
                            case "text":
                              return (
                                <Fragment key={`${message.id}-${i}`}>
                                  <Message from={message.role}>
                                    <MessageContent>
                                      <MessageResponse>{part.text}</MessageResponse>
                                    </MessageContent>
                                  </Message>
                                  {message.role === "assistant" &&
                                    i === messages.length - 1 && (
                                      <MessageActions className="mt-2">
                                        <MessageAction
                                          onClick={() => regenerate()}
                                          label="Retry"
                                        >
                                          <RefreshCcwIcon className="size-3" />
                                        </MessageAction>
                                        <MessageAction
                                          onClick={() =>
                                            navigator.clipboard.writeText(part.text)
                                          }
                                          label="Copy"
                                        >
                                          <CopyIcon className="size-3" />
                                        </MessageAction>
                                      </MessageActions>
                                    )}
                                </Fragment>
                              );
                            case "reasoning":
                              return (
                                <Reasoning
                                  key={`${message.id}-${i}`}
                                  className="w-full"
                                  isStreaming={
                                    status === "streaming" &&
                                    i === message.parts.length - 1 &&
                                    message.id === messages.at(-1)?.id
                                  }
                                >
                                  <ReasoningTrigger />
                                  <ReasoningContent>{part.text}</ReasoningContent>
                                </Reasoning>
                              );
                            default: {
                              if (part.type.startsWith("tool-")) {
                                const toolPart = part as ToolUIPart;
                                return (
                                  <Tool
                                    key={`${message.id}-${i}`}
                                    defaultOpen={toolPart.state === "output-available"}
                                  >
                                    <ToolHeader
                                      type={toolPart.type}
                                      state={toolPart.state}
                                    />
                                    <ToolContent>
                                      <ToolInput input={toolPart.input} />
                                      <ToolOutput
                                        output={toolPart.output}
                                        errorText={toolPart.errorText}
                                      />
                                    </ToolContent>
                                  </Tool>
                                );
                              }
                              return null;
                            }
                          }
                        })}
                      </div>
                    ))}
                    {status === "submitted" && <Loader />}
                  </ConversationContent>
                  <ConversationScrollButton />
                </Conversation>

                <PromptInput
                  onSubmit={handleSubmit}
                  className="mt-4"
                  globalDrop
                  multiple
                >
                  <PromptInputHeader>
                    <PromptInputAttachments>
                      {(attachment) => <PromptInputAttachment data={attachment} />}
                    </PromptInputAttachments>
                  </PromptInputHeader>
                  <PromptInputBody>
                    <PromptInputTextarea
                      onChange={(e) => setInput(e.target.value)}
                      value={input}
                    />
                  </PromptInputBody>
                  <PromptInputFooter>
                    <PromptInputTools>
                      <PromptInputActionMenu>
                        <PromptInputActionMenuTrigger />
                        <PromptInputActionMenuContent>
                          <PromptInputActionAddAttachments />
                        </PromptInputActionMenuContent>
                      </PromptInputActionMenu>
                      <PromptInputButton
                        variant={webSearch ? "default" : "ghost"}
                        onClick={() => setWebSearch(!webSearch)}
                      >
                        <GlobeIcon size={16} />
                        <span>Search</span>
                      </PromptInputButton>
                      <PromptInputSelect
                        onValueChange={(value) => {
                          setModel(value);
                        }}
                        value={model}
                      >
                        <PromptInputSelectTrigger>
                          <PromptInputSelectValue />
                        </PromptInputSelectTrigger>
                        <PromptInputSelectContent>
                          {models.map((model) => (
                            <PromptInputSelectItem
                              key={model.value}
                              value={model.value}
                            >
                              {model.name}
                            </PromptInputSelectItem>
                          ))}
                        </PromptInputSelectContent>
                      </PromptInputSelect>
                    </PromptInputTools>
                    <PromptInputSubmit disabled={!input && !status} status={status} />
                  </PromptInputFooter>
                </PromptInput>
              </div>
            </div>
          );
        };

        export default ChatBotDemo;
        ```
      </Tab>
    </Tabs>

    ### Understanding the Frontend

    The frontend uses AI Elements components to provide a complete chat interface:

    **Key Features:**

    * **Conversation Display**: The `Conversation` component automatically handles message scrolling and display
    * **Message Rendering**: Each message part is rendered based on its type (text, reasoning, tool calls)
    * **Tool Visualization**: Tool calls are displayed with collapsible sections showing inputs and outputs
    * **Interactive Controls**: Users can toggle web search, select models, and attach files
    * **Message Actions**: Copy and retry actions for assistant messages
  </Step>

  <Step title="Add Markdown Rendering Support">
    To ensure the markdown from the LLM is correctly rendered, add the following import to your `app/globals.css` file:

    ```css theme={null}
    @source "../node_modules/streamdown/dist/index.js";
    ```

    This imports the necessary styles for rendering markdown content in the message responses.
  </Step>

  <Step title="Build the Basic API Route">
    Create the chat API endpoint at `app/api/chat/route.ts`. This route will handle incoming messages and stream responses from the AI.

    ```typescript theme={null}
    import { streamText, UIMessage, convertToModelMessages } from "ai";
    import { createOpenAI } from "@ai-sdk/openai";

    const openai = createOpenAI({
      apiKey: process.env.OPENAI_API_KEY!,
    });

    // Allow streaming responses up to 5 minutes
    export const maxDuration = 300;

    export async function POST(req: Request) {
      const {
        messages,
        model,
        webSearch,
      }: {
        messages: UIMessage[];
        model: string;
        webSearch: boolean;
      } = await req.json();

      const result = streamText({
        model: openai(model),
        messages: convertToModelMessages(messages),
        system:
          "You are a helpful assistant that can answer questions and help with tasks.",
      });

      // send sources and reasoning back to the client
      return result.toUIMessageStreamResponse({
        sendSources: true,
        sendReasoning: true,
      });
    }
    ```

    This basic route:

    * Receives messages from the frontend
    * Uses the OpenAI model selected by the user
    * Streams responses back to the client
    * Doesn't include tools yet - we'll add those next
  </Step>

  <Step title="Configure Environment Variables">
    Create a `.env.local` file in your project root:

    ```bash theme={null}
    touch .env.local
    ```

    Add your OpenAI API key:

    ```env theme={null}
    OPENAI_API_KEY=sk-your-openai-api-key
    ```

    The `OPENAI_API_KEY` is required for the AI model to function.
  </Step>

  <Step title="Test the Basic Chat">
    Now you can test the AI SDK chatbot without Firecrawl integration. Start the development server:

    ```bash theme={null}
    npm run dev
    ```

    Open [localhost:3000](http://localhost:3000) in your browser and test the basic chat functionality. The assistant should respond to messages, but won't have web scraping or search capabilities yet.

    <img alt="Basic AI chatbot without web scraping capabilities" />
  </Step>

  <Step title="Add Firecrawl Tools">
    Now let's enhance the assistant with web scraping and search capabilities using Firecrawl.

    ### Install Firecrawl SDK

    Firecrawl converts websites into LLM-ready formats with scraping and search capabilities:

    ```bash theme={null}
    npm i @mendable/firecrawl-js
    ```

    ### Create the Tools File

    Create a `lib` folder and add a `tools.ts` file inside it:

    ```bash theme={null}
    mkdir lib && touch lib/tools.ts
    ```

    Add the following code to define the web scraping and search tools:

    ```typescript lib/tools.ts theme={null}
    import FirecrawlApp from "@mendable/firecrawl-js";
    import { tool } from "ai";
    import { z } from "zod";

    const firecrawl = new FirecrawlApp({ apiKey: process.env.FIRECRAWL_API_KEY });

    export const scrapeWebsiteTool = tool({
      description: 'Scrape content from any website URL',
      inputSchema: z.object({
        url: z.string().url().describe('The URL to scrape')
      }),
      execute: async ({ url }) => {
        console.log('Scraping:', url);
        const result = await firecrawl.scrape(url, {
          formats: ['markdown'],
          onlyMainContent: true,
          timeout: 30000
        });
        console.log('Scraped content preview:', result.markdown?.slice(0, 200) + '...');
        return { content: result.markdown };
      }
    });

    export const searchWebTool = tool({
      description: 'Search the web using Firecrawl',
      inputSchema: z.object({
        query: z.string().describe('The search query'),
        limit: z.number().optional().describe('Number of results'),
        location: z.string().optional().describe('Location for localized results'),
        tbs: z.string().optional().describe('Time filter (qdr:h, qdr:d, qdr:w, qdr:m, qdr:y)'),
        sources: z.array(z.enum(['web', 'news', 'images'])).optional().describe('Result types'),
        categories: z.array(z.enum(['github', 'research', 'pdf'])).optional().describe('Filter categories'),
      }),
      execute: async ({ query, limit, location, tbs, sources, categories }) => {
        console.log('Searching:', query);
        const response = await firecrawl.search(query, {
          ...(limit && { limit }),
          ...(location && { location }),
          ...(tbs && { tbs }),
          ...(sources && { sources }),
          ...(categories && { categories }),
        }) as { web?: Array<{ title?: string; url?: string; description?: string }> };

        const results = (response.web || []).map((item) => ({
          title: item.title || item.url || 'Untitled',
          url: item.url || '',
          description: item.description || '',
        }));

        console.log('Search results:', results.length);
        return { results };
      },
    });
    ```

    ### Understanding the Tools

    **Scrape Website Tool:**

    * Accepts a URL as input (validated by Zod schema)
    * Uses Firecrawl's `scrape` method to fetch the page as markdown
    * Extracts only the main content to reduce token usage
    * Returns the scraped content for the AI to analyze

    **Search Web Tool:**

    * Accepts a search query with optional filters
    * Uses Firecrawl's `search` method to find relevant web pages
    * Supports advanced filters like location, time range, and content categories
    * Returns structured results with titles, URLs, and descriptions

    Learn more about tools: [ai-sdk.dev/docs/foundations/tools](https://ai-sdk.dev/docs/foundations/tools).
  </Step>

  <Step title="Update the API Route with Firecrawl Tools">
    Now update your `app/api/chat/route.ts` to include the Firecrawl tools we just created.

    <Accordion title="View complete app/api/chat/route.ts code">
      ```typescript theme={null}
      import { streamText, UIMessage, stepCountIs, convertToModelMessages } from "ai";
      import { createOpenAI } from "@ai-sdk/openai";
      import { scrapeWebsiteTool, searchWebTool } from "@/lib/tools";

      const openai = createOpenAI({
        apiKey: process.env.OPENAI_API_KEY!,
      });

      export const maxDuration = 300;

      export async function POST(req: Request) {
        const {
          messages,
          model,
          webSearch,
        }: {
          messages: UIMessage[];
          model: string;
          webSearch: boolean;
        } = await req.json();

        const result = streamText({
          model: openai(model),
          messages: convertToModelMessages(messages),
          system:
            "You are a helpful assistant that can answer questions and help with tasks.",
          // Add the Firecrawl tools here
          tools: {
            scrapeWebsite: scrapeWebsiteTool,
            searchWeb: searchWebTool,
          },
          stopWhen: stepCountIs(5),
          toolChoice: webSearch ? "auto" : "none",
        });

        return result.toUIMessageStreamResponse({
          sendSources: true,
          sendReasoning: true,
        });
      }
      ```
    </Accordion>

    The key changes from the basic route:

    * Import `stepCountIs` from the AI SDK
    * Import the Firecrawl tools from `@/lib/tools`
    * Add the `tools` object with both `scrapeWebsite` and `searchWeb` tools
    * Add `stopWhen: stepCountIs(5)` to limit execution steps
    * Set `toolChoice` to "auto" when web search is enabled, "none" otherwise

    Learn more about `streamText`: [ai-sdk.dev/docs/reference/ai-sdk-core/stream-text](https://ai-sdk.dev/docs/reference/ai-sdk-core/stream-text).
  </Step>

  <Step title="Add Your Firecrawl API Key">
    Update your `.env.local` file to include your Firecrawl API key:

    ```env theme={null}
    OPENAI_API_KEY=sk-your-openai-api-key
    FIRECRAWL_API_KEY=fc-your-firecrawl-api-key
    ```

    Get your Firecrawl API key from [firecrawl.dev](https://firecrawl.dev).
  </Step>

  <Step title="Test the Complete Application">
    Restart your development server:

    ```bash theme={null}
    npm run dev
    ```

    <img alt="AI chatbot with active Firecrawl tools" />

    Open [localhost:3000](http://localhost:3000) and test the enhanced assistant:

    1. Toggle the "Search" button to enable web search
    2. Ask: "What are the latest features from firecrawl.dev?"
    3. Watch as the AI calls the `searchWeb` or `scrapeWebsite` tool
    4. See the tool execution in the UI with inputs and outputs
    5. Read the AI's analysis based on the scraped data
  </Step>
</Steps>

## How It Works

### Message Flow

1. **User sends a message**: The user types a question and clicks submit
2. **Frontend sends request**: `useChat` sends the message to `/api/chat` with the selected model and web search setting
3. **Backend processes message**: The API route receives the message and calls `streamText`
4. **AI decides on tools**: The model analyzes the question and decides whether to use `scrapeWebsite` or `searchWeb` (only if web search is enabled)
5. **Tools execute**: If tools are called, Firecrawl scrapes or searches the web
6. **AI generates response**: The model analyzes tool results and generates a natural language response
7. **Frontend displays results**: The UI shows tool calls and the final response in real-time

### Tool Calling Process

The AI SDK's tool calling system ([ai-sdk.dev/docs/foundations/tools](https://ai-sdk.dev/docs/foundations/tools)) works as follows:

1. The model receives the user's message and available tool descriptions
2. If the model determines a tool is needed, it generates a tool call with parameters
3. The SDK executes the tool function with those parameters
4. The tool result is sent back to the model
5. The model uses the result to generate its final response

This all happens automatically within a single `streamText` call, with results streaming to the frontend in real-time.

## Key Features

### Model Selection

The application supports multiple OpenAI models:

* **GPT-5 Mini (Thinking)**: Recent OpenAI model with advanced reasoning capabilities
* **GPT-4o Mini**: Fast and cost-effective model

Users can switch between models using the dropdown selector.

### Web Search Toggle

The Search button controls whether the AI can use Firecrawl tools:

* **Enabled**: AI can call `scrapeWebsite` and `searchWeb` tools as needed
* **Disabled**: AI responds only with its training knowledge

This gives users control over when to use web data versus the model's built-in knowledge.

## Customization Ideas

### Add More Tools

Extend the assistant with additional tools:

* Database lookups for internal company data
* CRM integration to fetch customer information
* Email sending capabilities
* Document generation

Each tool follows the same pattern: define a schema with Zod, implement the execute function, and register it in the `tools` object.

### Change the AI Model

Swap OpenAI for another provider:

```typescript theme={null}
import { anthropic } from "@ai-sdk/anthropic";

const result = streamText({
  model: anthropic("claude-4.5-sonnet"),
  // ... rest of config
});
```

The AI SDK supports 20+ providers with the same API. Learn more: [ai-sdk.dev/docs/foundations/providers-and-models](https://ai-sdk.dev/docs/foundations/providers-and-models).

### Customize the UI

AI Elements components are built on shadcn/ui, so you can:

* Modify component styles in the component files
* Add new variants to existing components
* Create custom components that match the design system

## Best Practices

1. **Use appropriate tools**: Choose `searchWeb` to find relevant pages first, `scrapeWebsite` for single pages, or let the AI decide

2. **Monitor API usage**: Track your Firecrawl and OpenAI API usage to avoid unexpected costs

3. **Handle errors gracefully**: The tools include error handling, but consider adding user-facing error messages

4. **Optimize performance**: Use streaming to provide immediate feedback and consider caching frequently accessed content

5. **Set reasonable limits**: The `stopWhen: stepCountIs(5)` prevents excessive tool calls and runaway costs

***

## Related Resources

<CardGroup>
  <Card title="AI SDK Documentation" href="https://ai-sdk.dev/docs">
    Explore the AI SDK for building AI-powered applications with streaming, tool
    calling, and multi-provider support.
  </Card>

  <Card title="AI Elements Components" href="https://ai-sdk.dev/elements/overview">
    Pre-built UI components for AI applications built on shadcn/ui.
  </Card>
</CardGroup>


# Building a Brand Style Guide Generator with Firecrawl
Source: https://docs.firecrawl.dev/developer-guides/cookbooks/brand-style-guide-generator-cookbook

Generate professional PDF brand style guides by extracting design systems from any website using Firecrawl's branding format

Build a brand style guide generator that automatically extracts colors, typography, spacing, and visual identity from any website and compiles it into a professional PDF document.

<img alt="Brand style guide PDF generator using Firecrawl branding format to extract design system" />

## What You'll Build

A Node.js application that takes any website URL, extracts its complete brand identity using Firecrawl's branding format, and generates a polished PDF style guide with:

* Color palette with hex values
* Typography system (fonts, sizes, weights)
* Spacing and layout specifications
* Logo and brand imagery
* Theme information (light/dark mode)

<img alt="Example of generated brand style guide PDF with colors typography and spacing" />

## Prerequisites

* Node.js 18 or later installed
* A Firecrawl API key from [firecrawl.dev](https://firecrawl.dev)
* Basic knowledge of TypeScript and Node.js

<Steps>
  <Step title="Create a New Node.js Project">
    Start by creating a new directory for your project and initializing it:

    ```bash theme={null}
    mkdir brand-style-guide-generator && cd brand-style-guide-generator
    npm init -y
    ```

    Update your `package.json` to use ES modules:

    ```json package.json theme={null}
    {
      "name": "brand-style-guide-generator",
      "version": "1.0.0",
      "type": "module",
      "scripts": {
        "start": "npx tsx index.ts"
      }
    }
    ```
  </Step>

  <Step title="Install Dependencies">
    Install the required packages for web scraping and PDF generation:

    ```bash theme={null}
    npm i @mendable/firecrawl-js pdfkit
    npm i -D typescript tsx @types/node @types/pdfkit
    ```

    These packages provide:

    * `@mendable/firecrawl-js`: Firecrawl SDK for extracting brand identity from websites
    * `pdfkit`: PDF document generation library
    * `tsx`: TypeScript execution for Node.js
  </Step>

  <Step title="Build the Brand Style Guide Generator">
    Create the main application file at `index.ts`. This script extracts brand identity from a URL and generates a professional PDF style guide.

    ```typescript index.ts theme={null}
    import Firecrawl from "@mendable/firecrawl-js";
    import PDFDocument from "pdfkit";
    import fs from "fs";

    const API_KEY = "fc-YOUR-API-KEY";
    const URL = "https://firecrawl.dev";

    async function main() {
      const fc = new Firecrawl({ apiKey: API_KEY });
      const { branding: b } = (await fc.scrape(URL, { formats: ["branding"] })) as any;

      const doc = new PDFDocument({ size: "A4", margin: 50 });
      doc.pipe(fs.createWriteStream("brand-style-guide.pdf"));

      // Fetch logo (PNG/JPG only)
      let logoImg: Buffer | null = null;
      try {
        const logoUrl = b.images?.favicon || b.images?.ogImage;
        if (logoUrl?.match(/\.(png|jpg|jpeg)$/i)) {
          const res = await fetch(logoUrl);
          logoImg = Buffer.from(await res.arrayBuffer());
        }
      } catch {}

      // Header with logo
      doc.rect(0, 0, 595, 120).fill(b.colors?.primary || "#333");
      const titleX = logoImg ? 130 : 50;
      if (logoImg) doc.image(logoImg, 50, 30, { height: 60 });
      doc.fontSize(36).fillColor("#fff").text("Brand Style Guide", titleX, 38);
      doc.fontSize(14).text(URL, titleX, 80);

      // Colors
      doc.fontSize(18).fillColor("#333").text("Colors", 50, 160);
      const colors = Object.entries(b.colors || {}).filter(([, v]) => typeof v === "string" && (v as string).startsWith("#"));
      colors.forEach(([k, v], i) => {
        const x = 50 + i * 100;
        doc.rect(x, 195, 80, 80).fill(v as string);
        doc.fontSize(10).fillColor("#333").text(k, x, 282, { width: 80, align: "center" });
        doc.fontSize(9).fillColor("#888").text(v as string, x, 296, { width: 80, align: "center" });
      });

      // Typography
      doc.fontSize(18).fillColor("#333").text("Typography", 50, 340);
      doc.fontSize(13).fillColor("#444");
      doc.text(`Primary Font: ${b.typography?.fontFamilies?.primary || "—"}`, 50, 370);
      doc.text(`Heading Font: ${b.typography?.fontFamilies?.heading || "—"}`, 50, 392);
      doc.fontSize(12).fillColor("#666").text("Font Sizes:", 50, 422);
      Object.entries(b.typography?.fontSizes || {}).forEach(([k, v], i) => {
        doc.text(`${k.toUpperCase()}: ${v}`, 70, 445 + i * 22);
      });

      // Spacing & Theme
      doc.fontSize(18).fillColor("#333").text("Spacing & Theme", 320, 340);
      doc.fontSize(13).fillColor("#444");
      doc.text(`Base Unit: ${b.spacing?.baseUnit}px`, 320, 370);
      doc.text(`Border Radius: ${b.spacing?.borderRadius}`, 320, 392);
      doc.text(`Color Scheme: ${b.colorScheme}`, 320, 414);

      doc.end();
      console.log("Generated: brand-style-guide.pdf");
    }

    main();
    ```

    <Info>
      For this simple project, the API key is placed directly in the code. If you plan to push this to GitHub or share it with others, move the key to a `.env` file and use `process.env.FIRECRAWL_API_KEY` instead.
    </Info>

    Replace `fc-YOUR-API-KEY` with your Firecrawl API key from [firecrawl.dev](https://firecrawl.dev).

    ### Understanding the Code

    **Key Components:**

    * **Firecrawl Branding Format**: The `branding` format extracts comprehensive brand identity including colors, typography, spacing, and images
    * **PDFKit Document**: Creates a professional A4 PDF with proper margins and sections
    * **Color Swatches**: Renders visual color blocks with hex values and semantic names
    * **Typography Display**: Shows font families and sizes in an organized layout
    * **Spacing & Theme**: Documents the design system's spacing units and color scheme
  </Step>

  <Step title="Run the Generator">
    Run the script to generate a brand style guide:

    ```bash theme={null}
    npm start
    ```

    The script will:

    1. Extract the brand identity from the target URL using Firecrawl's branding format
    2. Generate a PDF named `brand-style-guide.pdf`
    3. Save it in your project directory

    To generate a style guide for a different website, simply change the `URL` constant in `index.ts`.
  </Step>
</Steps>

## How It Works

### Extraction Process

1. **URL Input**: The generator receives a target website URL
2. **Firecrawl Scrape**: Calls the `/scrape` endpoint with the `branding` format
3. **Brand Analysis**: Firecrawl analyzes the page's CSS, fonts, and visual elements
4. **Data Return**: Returns a structured `BrandingProfile` object with all design tokens

### PDF Generation Process

1. **Header Creation**: Generates a colored header using the primary brand color
2. **Logo Fetch**: Downloads and embeds the logo or favicon if available
3. **Color Palette**: Renders each color as a visual swatch with metadata
4. **Typography Section**: Documents font families, sizes, and weights
5. **Spacing Info**: Includes base units, border radius, and theme mode

### Branding Profile Structure

The [branding format](https://docs.firecrawl.dev/features/scrape#%2Fscrape-with-branding-endpoint) returns detailed brand information:

```typescript theme={null}
{
  colorScheme: "dark" | "light",
  logo: "https://example.com/logo.svg",
  colors: {
    primary: "#FF6B35",
    secondary: "#004E89",
    accent: "#F77F00",
    background: "#1A1A1A",
    textPrimary: "#FFFFFF",
    textSecondary: "#B0B0B0"
  },
  typography: {
    fontFamilies: { primary: "Inter", heading: "Inter", code: "Roboto Mono" },
    fontSizes: { h1: "48px", h2: "36px", body: "16px" },
    fontWeights: { regular: 400, medium: 500, bold: 700 }
  },
  spacing: {
    baseUnit: 8,
    borderRadius: "8px"
  },
  images: {
    logo: "https://example.com/logo.svg",
    favicon: "https://example.com/favicon.ico"
  }
}
```

## Key Features

### Automatic Color Extraction

The generator identifies and categorizes all brand colors:

* **Primary & Secondary**: Main brand colors
* **Accent**: Highlight and CTA colors
* **Background & Text**: UI foundation colors
* **Semantic Colors**: Success, warning, error states

### Typography Documentation

Captures the complete type system:

* **Font Families**: Primary, heading, and monospace fonts
* **Size Scale**: All heading and body text sizes
* **Font Weights**: Available weight variations

### Visual Output

The PDF includes:

* Color-coded header matching the brand
* Embedded logo when available
* Professional layout with clear hierarchy
* Metadata footer with generation date

<img alt="Side-by-side comparison of original website and generated brand style guide PDF" />

## Customization Ideas

### Add Component Documentation

Extend the generator to include UI component styles:

```typescript theme={null}
// Add after the Spacing & Theme section
if (b.components) {
  doc.addPage();
  doc.fontSize(20).fillColor("#333").text("UI Components", 50, 50);

  // Document button styles
  if (b.components.buttonPrimary) {
    const btn = b.components.buttonPrimary;
    doc.fontSize(14).text("Primary Button", 50, 90);
    doc.rect(50, 110, 120, 40).fill(btn.background);
    doc.fontSize(12).fillColor(btn.textColor).text("Button", 50, 120, { width: 120, align: "center" });
  }
}
```

### Export Multiple Formats

Add JSON export alongside the PDF:

```typescript theme={null}
// Add before doc.end()
fs.writeFileSync("brand-data.json", JSON.stringify(b, null, 2));
```

### Batch Processing

Generate guides for multiple websites:

```typescript theme={null}
const websites = [
  "https://stripe.com",
  "https://linear.app",
  "https://vercel.com"
];

for (const site of websites) {
  const { branding } = await fc.scrape(site, { formats: ["branding"] }) as any;
  // Generate PDF for each site...
}
```

### Custom PDF Themes

Create different PDF styles based on the extracted theme:

```typescript theme={null}
const isDarkMode = b.colorScheme === "dark";
const headerBg = isDarkMode ? b.colors?.background : b.colors?.primary;
const textColor = isDarkMode ? "#fff" : "#333";
```

## Best Practices

1. **Handle Missing Data**: Not all websites expose complete branding information. Always provide fallback values for missing properties.

2. **Respect Rate Limits**: When batch processing multiple sites, add delays between requests to respect Firecrawl's rate limits.

3. **Cache Results**: Store extracted branding data to avoid repeated API calls for the same site.

4. **Image Format Handling**: Some logos may be in formats PDFKit doesn't support (like SVG). Consider adding format conversion or graceful fallbacks.

5. **Error Handling**: Wrap the generation process in try-catch blocks and provide meaningful error messages.

***

## Related Resources

<CardGroup>
  <Card title="Branding Format Documentation" href="https://docs.firecrawl.dev/features/scrape#extract-brand-identity">
    Learn more about the branding format and all available properties you can extract.
  </Card>

  <Card title="Firecrawl Scrape API" href="https://docs.firecrawl.dev/api-reference/endpoint/scrape">
    Complete API reference for the scrape endpoint with all format options.
  </Card>

  <Card title="PDFKit Documentation" href="http://pdfkit.org/">
    Learn more about PDFKit for advanced PDF customization options.
  </Card>

  <Card title="Batch Scrape Guide" href="https://docs.firecrawl.dev/features/batch-scrape">
    Process multiple URLs efficiently with batch scraping.
  </Card>
</CardGroup>


# Anthropic
Source: https://docs.firecrawl.dev/developer-guides/llm-sdks-and-frameworks/anthropic

Use Firecrawl with Claude for web scraping + AI workflows

Integrate Firecrawl with Claude to build AI applications powered by web data.

## Setup

```bash theme={null}
npm install @mendable/firecrawl-js @anthropic-ai/sdk zod zod-to-json-schema
```

Create `.env` file:

```bash theme={null}
FIRECRAWL_API_KEY=your_firecrawl_key
ANTHROPIC_API_KEY=your_anthropic_key
```

> **Note:** If using Node \< 20, install `dotenv` and add `import 'dotenv/config'` to your code.

## Scrape + Summarize

This example demonstrates a simple workflow: scrape a website and summarize the content using Claude.

```typescript theme={null}
import FirecrawlApp from '@mendable/firecrawl-js';
import Anthropic from '@anthropic-ai/sdk';

const firecrawl = new FirecrawlApp({ apiKey: process.env.FIRECRAWL_API_KEY });
const anthropic = new Anthropic({ apiKey: process.env.ANTHROPIC_API_KEY });

const scrapeResult = await firecrawl.scrape('https://firecrawl.dev', {
    formats: ['markdown']
});

console.log('Scraped content length:', scrapeResult.markdown?.length);

const message = await anthropic.messages.create({
    model: 'claude-haiku-4-5',
    max_tokens: 1024,
    messages: [
        { role: 'user', content: `Summarize in 100 words: ${scrapeResult.markdown}` }
    ]
});

console.log('Response:', message);
```

## Tool Use

This example shows how to use Claude's tool use feature to let the model decide when to scrape websites based on user requests.

```typescript theme={null}
import FirecrawlApp from '@mendable/firecrawl-js';
import { Anthropic } from '@anthropic-ai/sdk';
import { z } from 'zod';
import { zodToJsonSchema } from 'zod-to-json-schema';

const firecrawl = new FirecrawlApp({
    apiKey: process.env.FIRECRAWL_API_KEY
});

const anthropic = new Anthropic({
    apiKey: process.env.ANTHROPIC_API_KEY
});

const ScrapeArgsSchema = z.object({
    url: z.string()
});

console.log("Sending user message to Claude and requesting tool use if necessary...");
const response = await anthropic.messages.create({
    model: 'claude-haiku-4-5',
    max_tokens: 1024,
    tools: [{
        name: 'scrape_website',
        description: 'Scrape and extract markdown content from a website URL',
        input_schema: zodToJsonSchema(ScrapeArgsSchema, 'ScrapeArgsSchema') as any
    }],
    messages: [{
        role: 'user',
        content: 'What is Firecrawl? Check firecrawl.dev'
    }]
});

const toolUse = response.content.find(block => block.type === 'tool_use');

if (toolUse && toolUse.type === 'tool_use') {
    const input = toolUse.input as { url: string };
    console.log(`Calling tool: ${toolUse.name} | URL: ${input.url}`);

    const result = await firecrawl.scrape(input.url, {
        formats: ['markdown']
    });

    console.log(`Scraped content preview: ${result.markdown?.substring(0, 300)}...`);
    // Continue with the conversation or process the scraped content as needed
}
```

## Structured Extraction

This example demonstrates how to use Claude to extract structured data from scraped website content.

```typescript theme={null}
import FirecrawlApp from '@mendable/firecrawl-js';
import Anthropic from '@anthropic-ai/sdk';
import { z } from 'zod';

const firecrawl = new FirecrawlApp({ apiKey: process.env.FIRECRAWL_API_KEY });
const anthropic = new Anthropic({ apiKey: process.env.ANTHROPIC_API_KEY });

const CompanyInfoSchema = z.object({
    name: z.string(),
    industry: z.string().optional(),
    description: z.string().optional()
});

const scrapeResult = await firecrawl.scrape('https://stripe.com', {
    formats: ['markdown'],
    onlyMainContent: true
});

const prompt = `Extract company information from this website content.

Output ONLY valid JSON in this exact format (no markdown, no explanation):

{
  "name": "Company Name",
  "industry": "Industry",
  "description": "One sentence description"
}

Website content:
${scrapeResult.markdown}`;

const message = await anthropic.messages.create({
    model: 'claude-haiku-4-5',
    max_tokens: 1024,
    messages: [
        { role: 'user', content: prompt },
        { role: 'assistant', content: '{' }
    ]
});

const textBlock = message.content.find(block => block.type === 'text');

if (textBlock && textBlock.type === 'text') {
    const jsonText = '{' + textBlock.text;
    const companyInfo = CompanyInfoSchema.parse(JSON.parse(jsonText));
  
    console.log(companyInfo);
}
```

For more examples, check the [Claude documentation](https://docs.anthropic.com/claude/docs).


# ElevenAgents
Source: https://docs.firecrawl.dev/developer-guides/llm-sdks-and-frameworks/elevenagents

Give ElevenLabs voice and chat agents real-time web access with Firecrawl

Give your [ElevenAgents](https://elevenlabs.io/agents) voice and chat agents the ability to scrape, search, and crawl the web in real time using Firecrawl. This guide covers two integration paths:

1. **MCP server** — connect the hosted Firecrawl MCP server for zero-code setup.
2. **Server webhook tool** — point a custom tool at Firecrawl's REST API for full control over requests.

## Prerequisites

* An [ElevenLabs](https://elevenlabs.io) account with access to ElevenAgents
* A Firecrawl API key from your [firecrawl.dev dashboard](https://firecrawl.dev)

## Option 1: Firecrawl MCP Server

The fastest way to give an agent web access. ElevenAgents supports remote MCP servers, and Firecrawl provides a hosted MCP endpoint.

### Add the MCP server

1. Open the [Integrations page](https://elevenlabs.io/app/agents/integrations) in ElevenLabs and click **+ Add integration**.
2. Select **Custom MCP Server** from the integration library.
3. Fill in the following fields:

| Field           | Value                                                        |
| --------------- | ------------------------------------------------------------ |
| **Name**        | Firecrawl                                                    |
| **Description** | Search, scrape, crawl, and extract content from any website. |
| **Server type** | Streamable HTTP                                              |
| **Server URL**  | `https://mcp.firecrawl.dev/YOUR_FIRECRAWL_API_KEY/v2/mcp`    |

Replace `YOUR_FIRECRAWL_API_KEY` with your actual key. Leave the **Type** dropdown set to **Value**. Treat this URL as a secret — it contains your API key.

<Note>
  You must select **Streamable HTTP** as the server type. The default SSE option does not work with the Firecrawl MCP endpoint.
</Note>

4. Under **Tool Approval Mode**, choose an approval level:
   * **No Approval** — the agent uses tools freely. Fine for read-only scraping.
   * **Fine-Grained Tool Approval** — lets you pre-select which tools can run automatically and which require approval. Good for controlling expensive crawl operations.
   * **Always Ask** (default) — the agent requests permission before every tool call.

5. Check **I trust this server**, then click **Add Server**.

ElevenLabs will connect to the server and list the available tools (scrape, search, crawl, map, and more).

### Attach it to an agent

1. Create or open an agent in the [ElevenAgents dashboard](https://elevenlabs.io/app/agents/agents).
2. Go to the **Tools** tab, then select the **MCP** sub-tab.
3. Click **Add server** and select the **Firecrawl** integration from the dropdown.

### Update the system prompt

In the **Agent** tab, add instructions to the **System prompt** so the agent knows when to use Firecrawl. For example:

```text theme={null}
You are a helpful research assistant. When the user asks about a website,
a company, or any topic that requires up-to-date information, use the
Firecrawl tools to search the web or scrape the relevant page, then
summarize the results.
```

### Test it

Click **Preview** in the top navigation bar. You can test using the text chat input or by starting a voice call. Try a prompt like:

> "What does firecrawl.dev do? Go to the site and summarize it for me."

The agent will call the Firecrawl MCP `scrape` tool, receive the page markdown, and respond with a summary.

***

## Option 2: Server Webhook Tool

Use this approach when you need precise control over request parameters (formats, headers, timeouts, etc.) or want to call a specific Firecrawl endpoint without exposing the full MCP tool set.

### Scrape tool

Create a tool that scrapes a single URL and returns its content as markdown.

1. Open your agent and go to the **Tools** tab.
2. Click **Add tool** and select **Webhook**.
3. Configure the tool:

| Field           | Value                                                      |
| --------------- | ---------------------------------------------------------- |
| **Name**        | scrape\_website                                            |
| **Description** | Scrape content from a URL and return it as clean markdown. |
| **Method**      | POST                                                       |
| **URL**         | `https://api.firecrawl.dev/v2/scrape`                      |

<Note>
  The **Method** field defaults to GET — make sure to change it to **POST**.
</Note>

4. Scroll to the **Headers** section and click **Add header** for authentication:

| Header          | Value                           |
| --------------- | ------------------------------- |
| `Authorization` | `Bearer YOUR_FIRECRAWL_API_KEY` |

Alternatively, if you have workspace auth connections configured, you can use the **Authentication** dropdown instead.

5. Add a **body parameter**:

| Parameter | Type   | Description       | Required |
| --------- | ------ | ----------------- | -------- |
| `url`     | string | The URL to scrape | Yes      |

6. Click **Add tool**.

The Firecrawl API returns the page content as markdown by default. The agent receives the JSON response and can use the `markdown` field to answer questions.

### Search tool

Create a tool that searches the web and returns results with scraped content.

1. Click **Add tool** → **Webhook** again and configure:

| Field           | Value                                                                     |
| --------------- | ------------------------------------------------------------------------- |
| **Name**        | search\_web                                                               |
| **Description** | Search the web for a query and return relevant results with page content. |
| **Method**      | POST                                                                      |
| **URL**         | `https://api.firecrawl.dev/v2/search`                                     |

2. Add the same `Authorization` header as above.

3. Add **body parameters**:

| Parameter | Type   | Description                                     | Required |
| --------- | ------ | ----------------------------------------------- | -------- |
| `query`   | string | The search query                                | Yes      |
| `limit`   | number | Maximum number of results to return (default 5) | No       |

4. Click **Add tool**.

### Update the system prompt

In the **Agent** tab, update the **System prompt**:

```text theme={null}
You are a knowledgeable assistant with access to web tools.

- Use `scrape_website` when the user gives you a specific URL to read.
- Use `search_web` when the user asks a general question that requires
  finding information online.

Always summarize the information concisely and cite the source URL.
```

### Test it

Click **Preview** and try asking:

> "Search for the latest Next.js features and give me a summary."

The agent will call `search_web`, receive results from Firecrawl, and respond with a summary of the findings.

***

## Tips

* **Model selection** — For reliable tool calling, use a high-intelligence model such as GPT-4o, Claude Sonnet 4.5 or later, or Gemini 2.5 Flash. Smaller models may struggle to extract the correct parameters.
* **Keep prompts specific** — Tell the agent exactly when to use each tool. Vague instructions lead to missed or incorrect tool calls.
* **Limit response size** — For voice agents, long scraped pages can overwhelm the LLM context. Use `onlyMainContent: true` in scrape options (or instruct the agent to summarize aggressively) to keep responses concise.
* **Tool call sounds** — In the webhook or MCP tool settings, you can configure a **Tool call sound** to play ambient audio while a tool runs. This signals to the user that the agent is working.

## Resources

* [ElevenAgents documentation](https://elevenlabs.io/docs/eleven-agents/overview)
* [ElevenAgents tools overview](https://elevenlabs.io/docs/eleven-agents/customization/tools)
* [ElevenAgents MCP integration](https://elevenlabs.io/docs/eleven-agents/customization/tools/mcp)
* [Firecrawl API reference](https://docs.firecrawl.dev/api-reference/v2-introduction)
* [Firecrawl MCP server](https://docs.firecrawl.dev/mcp-server)


# Gemini
Source: https://docs.firecrawl.dev/developer-guides/llm-sdks-and-frameworks/gemini

Use Firecrawl with Google's Gemini AI for web scraping + AI workflows

Integrate Firecrawl with Google's Gemini for AI applications powered by web data.

## Setup

```bash theme={null}
npm install @mendable/firecrawl-js @google/genai
```

Create `.env` file:

```bash theme={null}
FIRECRAWL_API_KEY=your_firecrawl_key
GEMINI_API_KEY=your_gemini_key
```

> **Note:** If using Node \< 20, install `dotenv` and add `import 'dotenv/config'` to your code.

## Scrape + Summarize

This example demonstrates a simple workflow: scrape a website and summarize the content using Gemini.

```typescript theme={null}
import FirecrawlApp from '@mendable/firecrawl-js';
import { GoogleGenAI } from '@google/genai';

const firecrawl = new FirecrawlApp({ apiKey: process.env.FIRECRAWL_API_KEY });
const ai = new GoogleGenAI({ apiKey: process.env.GEMINI_API_KEY });

const scrapeResult = await firecrawl.scrape('https://firecrawl.dev', {
    formats: ['markdown']
});

console.log('Scraped content length:', scrapeResult.markdown?.length);

const response = await ai.models.generateContent({
    model: 'gemini-2.5-flash',
    contents: `Summarize: ${scrapeResult.markdown}`,
});

console.log('Summary:', response.text);
```

## Content Analysis

This example shows how to analyze website content using Gemini's multi-turn conversation capabilities.

```typescript theme={null}
import FirecrawlApp from '@mendable/firecrawl-js';
import { GoogleGenAI } from '@google/genai';

const firecrawl = new FirecrawlApp({ apiKey: process.env.FIRECRAWL_API_KEY });
const ai = new GoogleGenAI({ apiKey: process.env.GEMINI_API_KEY });

const scrapeResult = await firecrawl.scrape('https://news.ycombinator.com/', {
    formats: ['markdown']
});

console.log('Scraped content length:', scrapeResult.markdown?.length);

const chat = ai.chats.create({
    model: 'gemini-2.5-flash'
});

// Ask for the top 3 stories on Hacker News
const result1 = await chat.sendMessage({
    message: `Based on this website content from Hacker News, what are the top 3 stories right now?\n\n${scrapeResult.markdown}`
});
console.log('Top 3 Stories:', result1.text);

// Ask for the 4th and 5th stories on Hacker News
const result2 = await chat.sendMessage({
    message: `Now, what are the 4th and 5th top stories on Hacker News from the same content?`
});
console.log('4th and 5th Stories:', result2.text);
```

## Structured Extraction

This example demonstrates how to extract structured data using Gemini's JSON mode from scraped website content.

```typescript theme={null}
import FirecrawlApp from '@mendable/firecrawl-js';
import { GoogleGenAI, Type } from '@google/genai';

const firecrawl = new FirecrawlApp({ apiKey: process.env.FIRECRAWL_API_KEY });
const ai = new GoogleGenAI({ apiKey: process.env.GEMINI_API_KEY });

const scrapeResult = await firecrawl.scrape('https://stripe.com', {
    formats: ['markdown']
});

console.log('Scraped content length:', scrapeResult.markdown?.length);

const response = await ai.models.generateContent({
    model: 'gemini-2.5-flash',
    contents: `Extract company information: ${scrapeResult.markdown}`,
    config: {
        responseMimeType: 'application/json',
        responseSchema: {
            type: Type.OBJECT,
            properties: {
                name: { type: Type.STRING },
                industry: { type: Type.STRING },
                description: { type: Type.STRING },
                products: {
                    type: Type.ARRAY,
                    items: { type: Type.STRING }
                }
            },
            propertyOrdering: ['name', 'industry', 'description', 'products']
        }
    }
});

console.log('Extracted company info:', response?.text);
```

For more examples, check the [Gemini documentation](https://ai.google.dev/docs).


# Agent Development Kit (ADK)
Source: https://docs.firecrawl.dev/developer-guides/llm-sdks-and-frameworks/google-adk

Integrate Firecrawl with Google's ADK using MCP for advanced agent workflows

Integrate Firecrawl with Google's Agent Development Kit (ADK) to build powerful AI agents with web scraping capabilities through the Model Context Protocol (MCP).

## Overview

Firecrawl provides an MCP server that seamlessly integrates with Google's ADK, enabling your agents to efficiently scrape, crawl, and extract structured data from any website. The integration supports both cloud-based and self-hosted Firecrawl instances with streamable HTTP for optimal performance.

## Features

* Efficient web scraping, crawling, and content discovery from any website
* Advanced search capabilities and intelligent content extraction
* Deep research and high-volume batch scraping
* Flexible deployment (cloud-based or self-hosted)
* Optimized for modern web environments with streamable HTTP support

## Prerequisites

* Obtain an API key for Firecrawl from [firecrawl.dev](https://firecrawl.dev)
* Install Google ADK

## Setup

<CodeGroup>
  ```python Remote MCP Server theme={null}
  from google.adk.agents.llm_agent import Agent
  from google.adk.tools.mcp_tool.mcp_session_manager import StreamableHTTPServerParams
  from google.adk.tools.mcp_tool.mcp_toolset import MCPToolset

  FIRECRAWL_API_KEY = "YOUR-API-KEY"

  root_agent = Agent(
      model="gemini-2.5-pro",
      name="firecrawl_agent",
      description='A helpful assistant for scraping websites with Firecrawl',
      instruction='Help the user search for website content',
      tools=[
          MCPToolset(
              connection_params=StreamableHTTPServerParams(
                  url=f"https://mcp.firecrawl.dev/{FIRECRAWL_API_KEY}/v2/mcp",
              ),
          )
      ],
  )
  ```

  ```python Local MCP Server theme={null}
  from google.adk.agents.llm_agent import Agent
  from google.adk.tools.mcp_tool.mcp_session_manager import StdioConnectionParams
  from google.adk.tools.mcp_tool.mcp_toolset import MCPToolset
  from mcp import StdioServerParameters

  root_agent = Agent(
      model='gemini-2.5-pro',
      name='firecrawl_agent',
      description='A helpful assistant for scraping websites with Firecrawl',
      instruction='Help the user search for website content',
      tools=[
          MCPToolset(
              connection_params=StdioConnectionParams(
                  server_params = StdioServerParameters(
                      command='npx',
                      args=[
                          "-y",
                          "firecrawl-mcp",
                      ],
                      env={
                          "FIRECRAWL_API_KEY": "YOUR-API-KEY",
                      }
                  ),
                  timeout=30,
              ),
          )
      ],
  )
  ```
</CodeGroup>

## Available Tools

| Tool               | Name                           | Description                                                                          |
| ------------------ | ------------------------------ | ------------------------------------------------------------------------------------ |
| Scrape Tool        | `firecrawl_scrape`             | Scrape content from a single URL with advanced options                               |
| Batch Scrape Tool  | `firecrawl_batch_scrape`       | Scrape multiple URLs efficiently with built-in rate limiting and parallel processing |
| Check Batch Status | `firecrawl_check_batch_status` | Check the status of a batch operation                                                |
| Map Tool           | `firecrawl_map`                | Map a website to discover all indexed URLs on the site                               |
| Search Tool        | `firecrawl_search`             | Search the web and optionally extract content from search results                    |
| Crawl Tool         | `firecrawl_crawl`              | Start an asynchronous crawl with advanced options                                    |
| Check Crawl Status | `firecrawl_check_crawl_status` | Check the status of a crawl job                                                      |
| Extract Tool       | `firecrawl_extract`            | Extract structured information from web pages using LLM capabilities                 |

## Configuration

### Required Configuration

**FIRECRAWL\_API\_KEY**: Your Firecrawl API key

* Required when using cloud API (default)
* Optional when using self-hosted instance with FIRECRAWL\_API\_URL

### Optional Configuration

**Firecrawl API URL (for self-hosted instances)**:

* `FIRECRAWL_API_URL`: Custom API endpoint
* Example: `https://firecrawl.your-domain.com`
* If not provided, the cloud API will be used

**Retry configuration**:

* `FIRECRAWL_RETRY_MAX_ATTEMPTS`: Maximum retry attempts (default: 3)
* `FIRECRAWL_RETRY_INITIAL_DELAY`: Initial delay in milliseconds (default: 1000)
* `FIRECRAWL_RETRY_MAX_DELAY`: Maximum delay in milliseconds (default: 10000)
* `FIRECRAWL_RETRY_BACKOFF_FACTOR`: Exponential backoff multiplier (default: 2)

**Credit usage monitoring**:

* `FIRECRAWL_CREDIT_WARNING_THRESHOLD`: Warning threshold (default: 1000)
* `FIRECRAWL_CREDIT_CRITICAL_THRESHOLD`: Critical threshold (default: 100)

## Example: Web Research Agent

```python theme={null}
from google.adk.agents.llm_agent import Agent
from google.adk.tools.mcp_tool.mcp_session_manager import StreamableHTTPServerParams
from google.adk.tools.mcp_tool.mcp_toolset import MCPToolset

FIRECRAWL_API_KEY = "YOUR-API-KEY"

# Create a research agent
research_agent = Agent(
    model="gemini-2.5-pro",
    name="research_agent",
    description='An AI agent that researches topics by scraping and analyzing web content',
    instruction='''You are a research assistant. When given a topic or question:
    1. Use the search tool to find relevant websites
    2. Scrape the most relevant pages for detailed information
    3. Extract structured data when needed
    4. Provide comprehensive, well-sourced answers''',
    tools=[
        MCPToolset(
            connection_params=StreamableHTTPServerParams(
                url=f"https://mcp.firecrawl.dev/{FIRECRAWL_API_KEY}/v2/mcp",
            ),
        )
    ],
)

# Use the agent
response = research_agent.run("What are the latest features in Python 3.13?")
print(response)
```

## Best Practices

1. **Use the right tool for the job**:
   * `firecrawl_search` when you need to find relevant pages first
   * `firecrawl_scrape` for single pages
   * `firecrawl_batch_scrape` for multiple known URLs
   * `firecrawl_crawl` for discovering and scraping entire sites

2. **Monitor your usage**: Configure credit thresholds to avoid unexpected usage

3. **Handle errors gracefully**: Configure retry settings based on your use case

4. **Optimize performance**: Use batch operations when scraping multiple URLs

***

## Related Resources

<CardGroup>
  <Card title="Comprehensive Guide to Building AI Agents Using Google Agent Development Kit (ADK) and Firecrawl" href="https://www.firecrawl.dev/blog/google-adk-multi-agent-tutorial">
    Learn how to build powerful multi-agent AI systems using Google's ADK framework with Firecrawl for web scraping capabilities.
  </Card>

  <Card title="MCP Server Documentation" href="https://docs.firecrawl.dev/mcp-server">
    Learn more about Firecrawl's Model Context Protocol (MCP) server integration and capabilities.
  </Card>

  <Card title="Google ADK Official Documentation" href="https://google.github.io/adk-docs/">
    Explore the official Google Agent Development Kit documentation for comprehensive guides and API references.
  </Card>
</CardGroup>


# LangChain
Source: https://docs.firecrawl.dev/developer-guides/llm-sdks-and-frameworks/langchain

Use Firecrawl with LangChain for web scraping + AI workflows

Integrate Firecrawl with LangChain to build AI applications powered by web data.

## Setup

```bash theme={null}
npm install @langchain/openai @mendable/firecrawl-js 
```

Create `.env` file:

```bash theme={null}
FIRECRAWL_API_KEY=your_firecrawl_key
OPENAI_API_KEY=your_openai_key
```

> **Note:** If using Node \< 20, install `dotenv` and add `import 'dotenv/config'` to your code.

## Scrape + Chat

This example demonstrates a simple workflow: scrape a website and process the content using LangChain.

```typescript theme={null}
import FirecrawlApp from '@mendable/firecrawl-js';
import { ChatOpenAI } from '@langchain/openai';
import { HumanMessage } from '@langchain/core/messages';

const firecrawl = new FirecrawlApp({ apiKey: process.env.FIRECRAWL_API_KEY });
const chat = new ChatOpenAI({
    model: 'gpt-5-nano',
    apiKey: process.env.OPENAI_API_KEY
});

const scrapeResult = await firecrawl.scrape('https://firecrawl.dev', {
    formats: ['markdown']
});

console.log('Scraped content length:', scrapeResult.markdown?.length);

const response = await chat.invoke([
    new HumanMessage(`Summarize: ${scrapeResult.markdown}`)
]);

console.log('Summary:', response.content);
```

## Chains

This example shows how to build a LangChain chain to process and analyze scraped content.

```typescript theme={null}
import FirecrawlApp from '@mendable/firecrawl-js';
import { ChatOpenAI } from '@langchain/openai';
import { ChatPromptTemplate } from '@langchain/core/prompts';
import { StringOutputParser } from '@langchain/core/output_parsers';

const firecrawl = new FirecrawlApp({ apiKey: process.env.FIRECRAWL_API_KEY });
const model = new ChatOpenAI({
    model: 'gpt-5-nano',
    apiKey: process.env.OPENAI_API_KEY
});

const scrapeResult = await firecrawl.scrape('https://stripe.com', {
    formats: ['markdown']
});

console.log('Scraped content length:', scrapeResult.markdown?.length);

// Create processing chain
const prompt = ChatPromptTemplate.fromMessages([
    ['system', 'You are an expert at analyzing company websites.'],
    ['user', 'Extract the company name and main products from: {content}']
]);

const chain = prompt.pipe(model).pipe(new StringOutputParser());

// Execute the chain
const result = await chain.invoke({
    content: scrapeResult.markdown
});

console.log('Chain result:', result);
```

## Tool Calling

This example demonstrates how to use LangChain's tool calling feature to let the model decide when to scrape websites.

```typescript theme={null}
import FirecrawlApp from '@mendable/firecrawl-js';
import { ChatOpenAI } from '@langchain/openai';
import { DynamicStructuredTool } from '@langchain/core/tools';
import { z } from 'zod';

const firecrawl = new FirecrawlApp({ apiKey: process.env.FIRECRAWL_API_KEY });

// Create the scraping tool
const scrapeWebsiteTool = new DynamicStructuredTool({
    name: 'scrape_website',
    description: 'Scrape content from any website URL',
    schema: z.object({
        url: z.string().url().describe('The URL to scrape')
    }),
    func: async ({ url }) => {
        console.log('Scraping:', url);
        const result = await firecrawl.scrape(url, {
            formats: ['markdown']
        });
        console.log('Scraped content preview:', result.markdown?.substring(0, 200) + '...');
        return result.markdown || 'No content scraped';
    }
});

const model = new ChatOpenAI({
    model: 'gpt-5-nano',
    apiKey: process.env.OPENAI_API_KEY
}).bindTools([scrapeWebsiteTool]);

const response = await model.invoke('What is Firecrawl? Visit firecrawl.dev and tell me about it.');

console.log('Response:', response.content);
console.log('Tool calls:', response.tool_calls);
```

## Structured Data Extraction

This example shows how to extract structured data using LangChain's structured output feature.

```typescript theme={null}
import FirecrawlApp from '@mendable/firecrawl-js';
import { ChatOpenAI } from '@langchain/openai';
import { z } from 'zod';

const firecrawl = new FirecrawlApp({ apiKey: process.env.FIRECRAWL_API_KEY });

const scrapeResult = await firecrawl.scrape('https://stripe.com', {
    formats: ['markdown']
});

console.log('Scraped content length:', scrapeResult.markdown?.length);

const CompanyInfoSchema = z.object({
    name: z.string(),
    industry: z.string(),
    description: z.string(),
    products: z.array(z.string())
});

const model = new ChatOpenAI({
    model: 'gpt-5-nano',
    apiKey: process.env.OPENAI_API_KEY
}).withStructuredOutput(CompanyInfoSchema);

const companyInfo = await model.invoke([
    {
        role: 'system',
        content: 'Extract company information from website content.'
    },
    {
        role: 'user',
        content: `Extract data: ${scrapeResult.markdown}`
    }
]);

console.log('Extracted company info:', companyInfo);
```

For more examples, check the [LangChain documentation](https://js.langchain.com/docs).


# LangGraph
Source: https://docs.firecrawl.dev/developer-guides/llm-sdks-and-frameworks/langgraph

Integrate Firecrawl with LangGraph for building agent workflows

This guide shows how to integrate Firecrawl with LangGraph to build AI agent workflows that can scrape and process web content.

## Setup

```bash theme={null}
npm install @langchain/langgraph @langchain/openai @mendable/firecrawl-js
```

Create `.env` file:

```bash theme={null}
FIRECRAWL_API_KEY=your_firecrawl_key
OPENAI_API_KEY=your_openai_key
```

> **Note:** If using Node \< 20, install `dotenv` and add `import 'dotenv/config'` to your code.

## Basic Workflow

This example demonstrates a basic LangGraph workflow that scrapes a website and analyzes the content.

```typescript theme={null}
import FirecrawlApp from '@mendable/firecrawl-js';
import { ChatOpenAI } from '@langchain/openai';
import { StateGraph, MessagesAnnotation, START, END } from '@langchain/langgraph';

// Initialize Firecrawl
const firecrawl = new FirecrawlApp({ apiKey: process.env.FIRECRAWL_API_KEY });

// Initialize LLM
const llm = new ChatOpenAI({
    model: "gpt-5-nano",
    apiKey: process.env.OPENAI_API_KEY
});

// Define the scrape node
async function scrapeNode(state: typeof MessagesAnnotation.State) {
    console.log('Scraping...');
    const result = await firecrawl.scrape('https://firecrawl.dev', { formats: ['markdown'] });
    return {
        messages: [{
            role: "system",
            content: `Scraped content: ${result.markdown}`
        }]
    };
}

// Define the analyze node
async function analyzeNode(state: typeof MessagesAnnotation.State) {
    console.log('Analyzing...');
    const response = await llm.invoke(state.messages);
    return { messages: [response] };
}

// Build the graph
const graph = new StateGraph(MessagesAnnotation)
    .addNode("scrape", scrapeNode)
    .addNode("analyze", analyzeNode)
    .addEdge(START, "scrape")
    .addEdge("scrape", "analyze")
    .addEdge("analyze", END);

// Compile the graph
const app = graph.compile();

// Run the workflow
const result = await app.invoke({
    messages: [{ role: "user", content: "Summarize the website" }]
});

console.log(JSON.stringify(result, null, 2));
```

## Multi-Step Workflow

This example demonstrates a more complex workflow that scrapes multiple URLs and processes them.

```typescript theme={null}
import FirecrawlApp from '@mendable/firecrawl-js';
import { ChatOpenAI } from '@langchain/openai';
import { StateGraph, Annotation, START, END } from '@langchain/langgraph';

const firecrawl = new FirecrawlApp({ apiKey: process.env.FIRECRAWL_API_KEY });
const llm = new ChatOpenAI({ model: "gpt-5-nano", apiKey: process.env.OPENAI_API_KEY });

// Define custom state
const WorkflowState = Annotation.Root({
    urls: Annotation<string[]>(),
    scrapedData: Annotation<Array<{ url: string; content: string }>>(),
    summary: Annotation<string>()
});

// Scrape multiple URLs
async function scrapeMultiple(state: typeof WorkflowState.State) {
    const scrapedData = [];
    for (const url of state.urls) {
        const result = await firecrawl.scrape(url, { formats: ['markdown'] });
        scrapedData.push({ url, content: result.markdown || '' });
    }
    return { scrapedData };
}

// Summarize all scraped content
async function summarizeAll(state: typeof WorkflowState.State) {
    const combinedContent = state.scrapedData
        .map(item => `Content from ${item.url}:\n${item.content}`)
        .join('\n\n');

    const response = await llm.invoke([
        { role: "user", content: `Summarize these websites:\n${combinedContent}` }
    ]);

    return { summary: response.content as string };
}

// Build the workflow graph
const workflow = new StateGraph(WorkflowState)
    .addNode("scrape", scrapeMultiple)
    .addNode("summarize", summarizeAll)
    .addEdge(START, "scrape")
    .addEdge("scrape", "summarize")
    .addEdge("summarize", END);

const app = workflow.compile();

// Execute workflow
const result = await app.invoke({
    urls: ["https://firecrawl.dev", "https://firecrawl.dev/pricing"]
});

console.log(result.summary);
```

For more examples, check the [LangGraph documentation](https://langchain-ai.github.io/langgraphjs/).


# LlamaIndex
Source: https://docs.firecrawl.dev/developer-guides/llm-sdks-and-frameworks/llamaindex

Use Firecrawl with LlamaIndex for RAG applications

Integrate Firecrawl with LlamaIndex to build AI applications with vector search and embeddings powered by web content.

## Setup

```bash theme={null}
npm install llamaindex @llamaindex/openai @mendable/firecrawl-js
```

Create `.env` file:

```bash theme={null}
FIRECRAWL_API_KEY=your_firecrawl_key
OPENAI_API_KEY=your_openai_key
```

> **Note:** If using Node \< 20, install `dotenv` and add `import 'dotenv/config'` to your code.

## RAG with Vector Search

This example demonstrates how to use LlamaIndex with Firecrawl to crawl a website, create embeddings, and query the content using RAG.

```typescript theme={null}
import Firecrawl from '@mendable/firecrawl-js';
import { Document, VectorStoreIndex, Settings } from 'llamaindex';
import { OpenAI, OpenAIEmbedding } from '@llamaindex/openai';

Settings.llm = new OpenAI({ model: "gpt-4o" });
Settings.embedModel = new OpenAIEmbedding({ model: "text-embedding-3-small" });

const firecrawl = new Firecrawl({ apiKey: process.env.FIRECRAWL_API_KEY });
const crawlResult = await firecrawl.crawl('https://firecrawl.dev', {
  limit: 10,
  scrapeOptions: { formats: ['markdown'] }
});
console.log(`Crawled ${crawlResult.data.length } pages`);

const documents = crawlResult.data.map((page: any, i: number) =>
  new Document({
    text: page.markdown,
    id_: `page-${i}`,
    metadata: { url: page.metadata?.sourceURL }
  })
);

const index = await VectorStoreIndex.fromDocuments(documents);
console.log('Vector index created with embeddings');

const queryEngine = index.asQueryEngine();
const response = await queryEngine.query({ query: 'What is Firecrawl and how does it work?' });

console.log('\nAnswer:', response.toString());
```

For more examples, check the [LlamaIndex documentation](https://ts.llamaindex.ai/).


# Mastra
Source: https://docs.firecrawl.dev/developer-guides/llm-sdks-and-frameworks/mastra

Use Firecrawl with Mastra for building AI workflows

Integrate Firecrawl with Mastra, the TypeScript framework for building AI agents and workflows.

## Setup

```bash theme={null}
npm install @mastra/core @mendable/firecrawl-js zod
```

Create `.env` file:

```bash theme={null}
FIRECRAWL_API_KEY=your_firecrawl_key
OPENAI_API_KEY=your_openai_key
```

> **Note:** If using Node \< 20, install `dotenv` and add `import 'dotenv/config'` to your code.

## Multi-Step Workflow

This example demonstrates a complete workflow that searches, scrapes, and summarizes documentation using Firecrawl and Mastra.

```typescript theme={null}
import { createWorkflow, createStep } from "@mastra/core/workflows";
import { z } from "zod";
import Firecrawl from "@mendable/firecrawl-js";
import { Agent } from "@mastra/core/agent";

const firecrawl = new Firecrawl({
  apiKey: process.env.FIRECRAWL_API_KEY || "fc-YOUR_API_KEY"
});

const agent = new Agent({
  name: "summarizer",
  instructions: "You are a helpful assistant that creates concise summaries of documentation.",
  model: "openai/gpt-5-nano",
});

// Step 1: Search with Firecrawl SDK
const searchStep = createStep({
  id: "search",
  inputSchema: z.object({
    query: z.string(),
  }),
  outputSchema: z.object({
    url: z.string(),
    title: z.string(),
  }),
  execute: async ({ inputData }: { inputData: { query: string } }) => {
    console.log(`Searching: ${inputData.query}`);
    const searchResults = await firecrawl.search(inputData.query, { limit: 1 });
    const webResults = (searchResults as any)?.web;

    if (!webResults || !Array.isArray(webResults) || webResults.length === 0) {
      throw new Error("No search results found");
    }

    const firstResult = webResults[0];
    console.log(`Found: ${firstResult.title}`);
    return {
      url: firstResult.url,
      title: firstResult.title,
    };
  },
});

// Step 2: Scrape the URL with Firecrawl SDK
const scrapeStep = createStep({
  id: "scrape",
  inputSchema: z.object({
    url: z.string(),
    title: z.string(),
  }),
  outputSchema: z.object({
    markdown: z.string(),
    title: z.string(),
  }),
  execute: async ({ inputData }: { inputData: { url: string; title: string } }) => {
    console.log(`Scraping: ${inputData.url}`);
    const scrapeResult = await firecrawl.scrape(inputData.url, {
      formats: ["markdown"],
    });

    console.log(`Scraped: ${scrapeResult.markdown?.length || 0} characters`);
    return {
      markdown: scrapeResult.markdown || "",
      title: inputData.title,
    };
  },
});

// Step 3: Summarize with Claude
const summarizeStep = createStep({
  id: "summarize",
  inputSchema: z.object({
    markdown: z.string(),
    title: z.string(),
  }),
  outputSchema: z.object({
    summary: z.string(),
  }),
  execute: async ({ inputData }: { inputData: { markdown: string; title: string } }) => {
    console.log(`Summarizing: ${inputData.title}`);

    const prompt = `Summarize the following documentation in 2-3 sentences:\n\nTitle: ${inputData.title}\n\n${inputData.markdown}`;
    const result = await agent.generate(prompt);

    console.log(`Summary generated`);
    return { summary: result.text };
  },
});

// Create workflow
export const workflow = createWorkflow({
  id: "firecrawl-workflow",
  inputSchema: z.object({
    query: z.string(),
  }),
  outputSchema: z.object({
    summary: z.string(),
  }),
  steps: [searchStep, scrapeStep, summarizeStep],
})
  .then(searchStep)
  .then(scrapeStep)
  .then(summarizeStep)
  .commit();

async function testWorkflow() {
  const run = await workflow.createRunAsync();
  const result = await run.start({
    inputData: { query: "Firecrawl documentation" }
  });

  if (result.status === "success") {
    const { summarize } = result.steps;

    if (summarize.status === "success") {
      console.log(`\n${summarize.output.summary}`);
    }
  } else {
    console.error("Workflow failed:", result.status);
  }
}

testWorkflow().catch(console.error);
```

For more examples, check the [Mastra documentation](https://mastra.ai/docs).


# OpenAI
Source: https://docs.firecrawl.dev/developer-guides/llm-sdks-and-frameworks/openai

Use Firecrawl with OpenAI for web scraping + AI workflows

Integrate Firecrawl with OpenAI to build AI applications powered by web data.

## Setup

```bash theme={null}
npm install @mendable/firecrawl-js openai zod
```

Create `.env` file:

```bash theme={null}
FIRECRAWL_API_KEY=your_firecrawl_key
OPENAI_API_KEY=your_openai_key
```

> **Note:** If using Node \< 20, install `dotenv` and add `import 'dotenv/config'` to your code.

## Scrape + Summarize

This example demonstrates a simple workflow: scrape a website and summarize the content using an OpenAI model.

```typescript theme={null}
import FirecrawlApp from '@mendable/firecrawl-js';
import OpenAI from 'openai';

const firecrawl = new FirecrawlApp({ apiKey: process.env.FIRECRAWL_API_KEY });
const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

// Scrape the website content
const scrapeResult = await firecrawl.scrape('https://firecrawl.dev', {
    formats: ['markdown']
});

console.log('Scraped content length:', scrapeResult.markdown?.length);

// Summarize with OpenAI model
const completion = await openai.chat.completions.create({
    model: 'gpt-5-nano',
    messages: [
        { role: 'user', content: `Summarize: ${scrapeResult.markdown}` }
    ]
});

console.log('Summary:', completion.choices[0]?.message.content);
```

## Function Calling

This example shows how to use OpenAI's function calling feature to let the model decide when to scrape websites based on user requests.

```typescript theme={null}
import FirecrawlApp from '@mendable/firecrawl-js';
import OpenAI from 'openai';
import { z } from 'zod';

const firecrawl = new FirecrawlApp({ apiKey: process.env.FIRECRAWL_API_KEY });
const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

const ScrapeArgsSchema = z.object({
    url: z.string().describe('The URL of the website to scrape')
});

const tools = [{
    type: 'function' as const,
    function: {
        name: 'scrape_website',
        description: 'Scrape content from any website URL',
        parameters: z.toJSONSchema(ScrapeArgsSchema)
    }
}];

const response = await openai.chat.completions.create({
    model: 'gpt-5-nano',
    messages: [{
        role: 'user',
        content: 'What is Firecrawl? Visit firecrawl.dev and tell me about it.'
    }],
    tools
});

const message = response.choices[0]?.message;

if (message?.tool_calls && message.tool_calls.length > 0) {
    for (const toolCall of message.tool_calls) {
        if (toolCall.type === 'function') {
            console.log('Tool called:', toolCall.function.name);

            const args = ScrapeArgsSchema.parse(JSON.parse(toolCall.function.arguments));
            const result = await firecrawl.scrape(args.url, {
                formats: ['markdown'] // Other formats: html, links, etc.
            });
            console.log('Scraped content:', result.markdown?.substring(0, 200) + '...');

            // Send the scraped content back to the model for final response
            const finalResponse = await openai.chat.completions.create({
                model: 'gpt-5-nano',
                messages: [
                    {
                        role: 'user',
                        content: 'What is Firecrawl? Visit firecrawl.dev and tell me about it.'
                    },
                    message,
                    {
                        role: 'tool',
                        tool_call_id: toolCall.id,
                        content: result.markdown || 'No content scraped'
                    }
                ],
                tools
            });

            console.log('Final response:', finalResponse.choices[0]?.message?.content);
        }
    }
} else {
    console.log('Direct response:', message?.content);
}
```

## Structured Data Extraction

This example demonstrates how to use OpenAI models with structured outputs to extract specific data from scraped content.

```typescript theme={null}
import FirecrawlApp from '@mendable/firecrawl-js';
import OpenAI from 'openai';
import { z } from 'zod';

const firecrawl = new FirecrawlApp({ apiKey: process.env.FIRECRAWL_API_KEY });
const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

const scrapeResult = await firecrawl.scrape('https://stripe.com', {
    formats: ['markdown']
});

console.log('Scraped content length:', scrapeResult.markdown?.length);

const CompanyInfoSchema = z.object({
    name: z.string(),
    industry: z.string(),
    description: z.string(),
    products: z.array(z.string())
});

const response = await openai.chat.completions.create({
    model: 'gpt-5-nano',
    messages: [
        {
            role: 'system',
            content: 'Extract company information from website content.'
        },
        {
            role: 'user',
            content: `Extract data: ${scrapeResult.markdown}`
        }
    ],
    response_format: {
        type: 'json_schema',
        json_schema: {
            name: 'company_info',
            schema: z.toJSONSchema(CompanyInfoSchema),
            strict: true
        }
    }
});

const content = response.choices[0]?.message?.content;
const companyInfo = content ? CompanyInfoSchema.parse(JSON.parse(content)) : null;
console.log('Validated company info:', companyInfo);
```

## Search + Analyze

This example combines Firecrawl's search capabilities with OpenAI model analysis to find and summarize information from multiple sources.

```typescript theme={null}
import FirecrawlApp from '@mendable/firecrawl-js';
import OpenAI from 'openai';

const firecrawl = new FirecrawlApp({ apiKey: process.env.FIRECRAWL_API_KEY });
const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

// Search for relevant information
const searchResult = await firecrawl.search('Next.js 16 new features', {
    limit: 3,
    sources: [{ type: 'web' }], // Other sources: { type: 'news' }, { type: 'images' }
    scrapeOptions: { formats: ['markdown'] }
});

console.log('Search results:', searchResult.web?.length, 'pages found');

// Analyze and summarize the key features
const analysis = await openai.chat.completions.create({
    model: 'gpt-5-nano',
    messages: [{
        role: 'user',
        content: `Summarize the key features: ${JSON.stringify(searchResult)}`
    }]
});

console.log('Analysis:', analysis.choices[0]?.message?.content);
```

## Responses API with MCP

This example shows how to use OpenAI's Responses API with Firecrawl configured as an MCP (Model Context Protocol) server.

```typescript theme={null}
import OpenAI from 'openai';

const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

const response = await openai.responses.create({
    model: 'gpt-5-nano',
    tools: [
        {
            type: 'mcp',
            server_label: 'firecrawl',
            server_description: 'A web search and scraping MCP server to scrape and extract content from websites.',
            server_url: `https://mcp.firecrawl.dev/${process.env.FIRECRAWL_API_KEY}/v2/mcp`,
            require_approval: 'never'
        }
    ],
    input: 'Find out what the top stories on Hacker News are and the latest blog post on OpenAI and summarize them in a bullet point format'
});

console.log('Response:', JSON.stringify(response.output, null, 2));
```

For more examples, check the [OpenAI documentation](https://platform.openai.com/docs).


# Vercel AI SDK
Source: https://docs.firecrawl.dev/developer-guides/llm-sdks-and-frameworks/vercel-ai-sdk

Firecrawl tools for Vercel AI SDK. Web scraping, search, interact, and crawling for AI applications.

Firecrawl tools for the Vercel AI SDK. Search, scrape, interact with pages, and crawl the web in your AI applications.

## Install

```bash theme={null}
npm install firecrawl-aisdk ai
```

Set environment variables:

```bash theme={null}
FIRECRAWL_API_KEY=fc-your-key       # https://firecrawl.dev
AI_GATEWAY_API_KEY=your-key         # https://vercel.com/ai-gateway
```

<Note>
  These examples use the [Vercel AI Gateway](https://vercel.com/ai-gateway) string model format, but Firecrawl tools work with any AI SDK provider. You can also use provider imports like `anthropic('claude-sonnet-4-5-20250514')` from `@ai-sdk/anthropic`.
</Note>

## Quick Start

`FirecrawlTools()` bundles `search`, `scrape`, and `interact` by default.

```typescript theme={null}
import { generateText, stepCountIs } from 'ai';
import { FirecrawlTools } from 'firecrawl-aisdk';

const { text } = await generateText({
  model: 'anthropic/claude-sonnet-4-5',
  tools: FirecrawlTools(),
  stopWhen: stepCountIs(30),
  prompt: `
    1. Use interact on Hacker News to identify the top story
    2. Search for other perspectives on the same topic
    3. Scrape the most relevant pages you found
    4. Summarize everything you found
  `,
});
```

## FirecrawlTools

`FirecrawlTools()` gives you the default tools plus an auto-generated `systemPrompt` you can pass to `generateText`.

```typescript theme={null}
import { generateText, stepCountIs } from 'ai';
import { FirecrawlTools } from 'firecrawl-aisdk';

const tools = FirecrawlTools();

const { text } = await generateText({
  model: 'anthropic/claude-sonnet-4-5',
  system: `${tools.systemPrompt}\n\nAnswer with citations when possible.`,
  tools,
  stopWhen: stepCountIs(20),
  prompt: 'Find the current Firecrawl pricing page and explain the available plans.',
});
```

You can customize defaults, opt into async tools, or disable individual tools:

```typescript theme={null}
const tools = FirecrawlTools({
  search: { limit: 5 },
  scrape: { formats: ['markdown'], onlyMainContent: true },
  interact: { profile: { name: 'my-session', saveChanges: true } },
  crawl: true,
  agent: true,
});
```

```typescript theme={null}
// Disable interact, keep search + scrape
FirecrawlTools({ interact: false });

// Opt into deprecated browser compatibility
FirecrawlTools({ browser: {} });

// Include every available tool
FirecrawlTools({ all: true });
```

When scraping to answer a question about a page, use the `question` format:

```typescript theme={null}
formats: [{ type: 'question', question: 'What does this page say about pricing and rate limits?' }]
```

Use `formats: ['markdown']` only when you need the full page content.

## Individual Tools

Every tool can be used directly or called with options:

```typescript theme={null}
import { generateText } from 'ai';
import { scrape, search } from 'firecrawl-aisdk';

const { text } = await generateText({
  model: 'anthropic/claude-sonnet-4-5',
  prompt: 'Search for Firecrawl, then scrape the most relevant result.',
  tools: { search, scrape },
});

const customScrape = scrape({ apiKey: 'fc-custom-key', apiUrl: 'https://api.firecrawl.dev' });
```

### Search + Scrape

```typescript theme={null}
import { generateText } from 'ai';
import { search, scrape } from 'firecrawl-aisdk';

const { text } = await generateText({
  model: 'anthropic/claude-sonnet-4-5',
  prompt: 'Search for Firecrawl, scrape the top official result, and explain what it does.',
  tools: { search, scrape },
});
```

### Map

```typescript theme={null}
import { generateText } from 'ai';
import { map } from 'firecrawl-aisdk';

const { text } = await generateText({
  model: 'anthropic/claude-sonnet-4-5',
  prompt: 'Map https://docs.firecrawl.dev and list the main sections.',
  tools: { map },
});
```

### Stream

```typescript theme={null}
import { streamText, stepCountIs } from 'ai';
import { scrape } from 'firecrawl-aisdk';

const result = streamText({
  model: 'anthropic/claude-sonnet-4-5',
  prompt: 'What is the first 100 words of firecrawl.dev?',
  tools: { scrape },
  stopWhen: stepCountIs(3),
});

for await (const chunk of result.textStream) {
  process.stdout.write(chunk);
}

await result.fullStream;
```

## Interact

`interact()` creates a scrape-backed interactive session. Call `start(url)` to bootstrap a session and get a live view URL, then let the model reuse that session through the `interact` tool.

```typescript theme={null}
import { generateText, stepCountIs } from 'ai';
import { interact, search } from 'firecrawl-aisdk';

const interactTool = interact();
console.log('Live view:', await interactTool.start('https://news.ycombinator.com'));

const { text } = await generateText({
  model: 'anthropic/claude-sonnet-4-5',
  tools: { interact: interactTool, search },
  stopWhen: stepCountIs(25),
  prompt: 'Use interact on the current Hacker News session, find the top story, then search for more context.',
});

await interactTool.close();
```

If you need the explicit live view URL after startup, use `interactTool.interactiveLiveViewUrl`.

Reuse browser state across sessions with profiles:

```typescript theme={null}
const interactTool = interact({
  profile: { name: 'my-session', saveChanges: true },
});
```

`browser()` is deprecated. Prefer `interact()`.

## Async Tools

Crawl, batch scrape, and agent return a job ID. Pair them with `poll`.

### Crawl

```typescript theme={null}
import { generateText } from 'ai';
import { crawl, poll } from 'firecrawl-aisdk';

const { text } = await generateText({
  model: 'anthropic/claude-sonnet-4-5',
  prompt: 'Crawl https://docs.firecrawl.dev (limit 3 pages) and summarize.',
  tools: { crawl, poll },
});
```

### Batch Scrape

```typescript theme={null}
import { generateText } from 'ai';
import { batchScrape, poll } from 'firecrawl-aisdk';

const { text } = await generateText({
  model: 'anthropic/claude-sonnet-4-5',
  prompt: 'Scrape https://firecrawl.dev and https://docs.firecrawl.dev, then compare them.',
  tools: { batchScrape, poll },
});
```

### Agent

Autonomous web data gathering that searches, navigates, and extracts on its own.

```typescript theme={null}
import { generateText, stepCountIs } from 'ai';
import { agent, poll } from 'firecrawl-aisdk';

const { text } = await generateText({
  model: 'anthropic/claude-sonnet-4-5',
  prompt: 'Find the founders of Firecrawl, their roles, and their backgrounds.',
  tools: { agent, poll },
  stopWhen: stepCountIs(10),
});
```

## Logging

```typescript theme={null}
import { generateText } from 'ai';
import { logStep, scrape, stepLogger } from 'firecrawl-aisdk';

const logger = stepLogger();

const { text, usage } = await generateText({
  model: 'anthropic/claude-sonnet-4-5',
  prompt: 'Scrape https://firecrawl.dev and summarize it.',
  tools: { scrape },
  onStepFinish: logger.onStep,
  experimental_onToolCallFinish: logger.onToolCallFinish,
});

logger.close();
logger.summary(usage);

await generateText({
  model: 'anthropic/claude-sonnet-4-5',
  prompt: 'Scrape https://firecrawl.dev and summarize it again.',
  tools: { scrape },
  onStepFinish: logStep,
});
```

## All Exports

```typescript theme={null}
import {
  // Core tools
  search,             // Search the web
  scrape,             // Scrape a single URL
  map,                // Discover URLs on a site
  crawl,              // Crawl multiple pages (async, use poll)
  batchScrape,        // Scrape multiple URLs (async, use poll)
  agent,              // Autonomous web research (async, use poll)

  // Job management
  poll,               // Poll async jobs for results
  status,             // Check job status
  cancel,             // Cancel running jobs

  // Browser/session tools
  interact,           // interact({ profile: { name: '...' } })
  browser,            // deprecated compatibility export

  // All-in-one bundle
  FirecrawlTools,     // FirecrawlTools({ search, scrape, interact, crawl, agent })

  // Helpers
  stepLogger,         // Token stats per tool call
  logStep,            // Simple one-liner logging
} from 'firecrawl-aisdk';
```


# Debug Firecrawl with Ask
Source: https://docs.firecrawl.dev/features/ask

Agentic debugging for your Firecrawl integration

Firecrawl `/support/ask` is an AI support agent exposed as an API. Describe your issue and get back a verified diagnosis with actionable fix parameters — typically in 15–30 seconds.

**Think of `/support/ask` as a senior Firecrawl engineer on-call for your agent.**

<Info>
  The Ask API is designed primarily for **AI agent callers**. If you're building agents that use Firecrawl for scraping, crawling, or data extraction, wire `/support/ask` into your error-handling flow for autonomous issue resolution.
</Info>

## Two endpoints

| Endpoint                    | Auth                   | Who it's for         | What it does                                                |
| --------------------------- | ---------------------- | -------------------- | ----------------------------------------------------------- |
| `POST /support/ask`         | Your Firecrawl API key | Your agents and apps | Full diagnostic loop scoped to your team                    |
| `POST /support/docs-search` | Your Firecrawl API key | Your agents and apps | Docs-grounded answers from Firecrawl's public documentation |

## Quick start

### Debug a failing crawl

```bash theme={null}
curl -X POST https://api.firecrawl.dev/v2/support/ask \
  -H "Authorization: Bearer fc-YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "question": "my crawl returned 3 pages but I expected 50"
  }'
```

### Search the docs

```bash theme={null}
curl -X POST https://api.firecrawl.dev/v2/support/docs-search \
  -H "Authorization: Bearer fc-YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "question": "how do I set up webhook signature verification?"
  }'
```

## How it works

When you call `/support/ask`, the AI agent:

1. **Gathers evidence** — inspects your job logs, account state, credit usage, and relevant documentation in parallel
2. **Diagnoses the issue** — reasons across all evidence to identify the root cause
3. **Proposes a fix** — generates machine-actionable `fixParameters` you can apply directly to your next API call
4. **Validates the fix** — when possible, tests the fix against the live Firecrawl API (e.g., retrying a scrape with adjusted parameters) and reports the result

## Using Ask in your agent

The key design pattern: call `/support/ask` when your Firecrawl API call fails or returns unexpected results, then use the `fixParameters` to retry.

### Python example

```python theme={null}
import requests

FIRECRAWL_API_KEY = "fc-YOUR_API_KEY"

def diagnose_firecrawl_issue(question, rationale=None):
    """Call the Firecrawl Ask API to debug an issue."""
    payload = {"question": question}
    if rationale:
        payload["rationale"] = rationale

    response = requests.post(
        "https://api.firecrawl.dev/v2/support/ask",
        headers={
            "Authorization": f"Bearer {FIRECRAWL_API_KEY}",
            "Content-Type": "application/json",
        },
        json=payload,
    )
    return response.json()


# Example: debug a scrape that returned empty content
result = diagnose_firecrawl_issue(
    question="scrape returned empty markdown for https://example.com",
    rationale="user needs product pricing data for competitive analysis",
)

print(result["answer"])
print(result["fixParameters"])  # e.g., {"waitFor": 5000, "actions": [...]}
print(result["confidence"])     # "high", "medium", or "low"
```

### Node.js example

```javascript theme={null}
async function diagnoseFirecrawlIssue(question, rationale) {
  const response = await fetch(
    "https://api.firecrawl.dev/v2/support/ask",
    {
      method: "POST",
      headers: {
        Authorization: `Bearer ${process.env.FIRECRAWL_API_KEY}`,
        "Content-Type": "application/json",
      },
      body: JSON.stringify({ question, rationale }),
    }
  );
  return response.json();
}

// Example: debug a crawl that stopped early
const result = await diagnoseFirecrawlIssue(
  "my crawl returned 3 pages but I expected 50",
  "user is on their third failed crawl attempt today"
);

console.log(result.answer);
console.log(result.fixParameters);
```

### Agent retry pattern

```python theme={null}
from firecrawl import Firecrawl

client = Firecrawl(api_key="fc-YOUR_API_KEY")

# Step 1: Try the scrape
doc = client.scrape("https://example.com/pricing", formats=["markdown"])

if not doc.markdown or len(doc.markdown) < 100:
    # Step 2: Ask for help debugging
    diagnosis = diagnose_firecrawl_issue(
        question=f"scrape returned only {len(doc.markdown or '')} chars of markdown for https://example.com/pricing",
    )

    # Step 3: Apply fix parameters and retry
    if diagnosis.get("fixParameters"):
        doc = client.scrape(
            "https://example.com/pricing",
            formats=["markdown"],
            **diagnosis["fixParameters"],
        )
```

## Parameters

### `/support/ask`

| Parameter   | Type   | Required | Description                                                                                                 |
| ----------- | ------ | -------- | ----------------------------------------------------------------------------------------------------------- |
| `question`  | string | Yes      | What to debug (1–8,000 characters)                                                                          |
| `rationale` | string | No       | Recommended for AI callers. What the end user is trying to accomplish. Helps prioritize evidence gathering. |
| `context`   | object | No       | Free-form metadata from your agent, included in the debugging prompt                                        |

### `/support/docs-search`

| Parameter  | Type   | Required | Description                                 |
| ---------- | ------ | -------- | ------------------------------------------- |
| `question` | string | Yes      | The question to answer (1–8,000 characters) |

## Response

### `/support/ask` response

```json theme={null}
{
  "requestId": "req_...",
  "answer": "<2-4 sentence prose diagnosis of the issue plus the recommended fix.>",
  "confidence": "high",
  "fixParameters": { "<param>": "<value>" },
  "validation": {
    "tested": true,
    "result": "success",
    "evidence": "<short summary of the validation tool call the agent ran to confirm the fix>"
  },
  "feedback": null,
  "durationMs": 18432
}
```

The actual `answer`, `fixParameters`, and `validation.evidence` are produced per request by the agent based on your specific run; the example above shows the response shape, not a real diagnosis.

### `/support/docs-search` response

```json theme={null}
{
  "requestId": "req_...",
  "answer": "The signature is sent in the X-Firecrawl-Signature header...",
  "evidence": [
    { "pathOrUrl": "webhooks/security.mdx#L1-L52", "reason": "..." }
  ],
  "usage": { "inputTokens": 4356, "outputTokens": 688, "totalTokens": 5044 },
  "durationMs": 11252
}
```

## Performance

| Metric  | Typical       | Maximum                   |
| ------- | ------------- | ------------------------- |
| Latency | 15–30 seconds | 60 seconds (hard ceiling) |

## API Reference

* [Ask endpoint API Reference](/api-reference/endpoint/ask)
* [Docs Search endpoint API Reference](/api-reference/endpoint/docs-search)

Have feedback or need help? Email [help@firecrawl.com](mailto:help@firecrawl.com).

> Are you an AI agent that needs a Firecrawl API key? See [firecrawl.dev/agent-onboarding/SKILL.md](https://www.firecrawl.dev/agent-onboarding/SKILL.md) for automated onboarding instructions.


# Partner Integration API
Source: https://docs.firecrawl.dev/partner-integration

API reference for approved Firecrawl partners to create and manage API keys for their users

## Overview

The Firecrawl Partner Integration API allows approved partners to create and manage Firecrawl API keys for their users directly from within their own platform. This enables a seamless onboarding experience where users can start using Firecrawl without leaving the partner's platform.

<Note>
  Partner API access is **only available to approved Firecrawl partners**. Partner keys are shared privately with approved partners. To apply for the partner program or request a partner API key, email [partnerships@firecrawl.dev](mailto:partnerships@firecrawl.dev).
</Note>

For details on what partner users receive, including credits, expiration, and plan limits, see [Partner Credits](/partner-credits).

## Base URL

```
https://integrations.firecrawl.dev
```

## Authentication

All Partner Integration API requests require an `Authorization` header with your partner key:

```bash theme={null}
Authorization: Bearer <partner key>
```

Partner keys are distinct from standard Firecrawl API keys and are provided directly by the Firecrawl team.

## Security Requirements

* **Server-side only**: Partner keys must only be used in server-side code. Never expose a partner key in frontend code, client-side JavaScript, or mobile applications.
* **Terms of Service**: Before calling `POST /partner/v1/accounts`, your platform must prompt the user to accept Firecrawl's [Terms of Service](https://www.firecrawl.dev/terms-of-service).

***

## Endpoints

### Create user

Creates a Firecrawl API key associated with the user's email address.

```
POST /partner/v1/accounts
```

#### Behavior

* If the user does not yet have a Firecrawl account, a new user and team will be created.
* If the user already has a Firecrawl account but does not have a team associated with your partner integration, a new partner-associated team will be created.
* If the user already has a Firecrawl account and already has a team associated with your partner integration, the existing team is returned and the promotional coupon is **not** re-applied.

#### Request

```bash cURL theme={null}
curl -X POST "https://integrations.firecrawl.dev/partner/v1/accounts" \
  -H "Authorization: Bearer <partner key>" \
  -H "Content-Type: application/json" \
  -d '{"email": "user@example.com"}'
```

**Body**

| Field   | Type   | Required | Description              |
| ------- | ------ | -------- | ------------------------ |
| `email` | string | Yes      | The user's email address |

#### Response

**`200 OK`**

```json theme={null}
{
  "apiKey": "fc-...",
  "alreadyExisted": false
}
```

| Field            | Type    | Description                                                                                                                        |
| ---------------- | ------- | ---------------------------------------------------------------------------------------------------------------------------------- |
| `apiKey`         | string  | The Firecrawl API key for this user's partner-associated team                                                                      |
| `alreadyExisted` | boolean | `true` if both the user and the partner-associated team already existed. When `true`, the promotional coupon is not applied again. |

#### Errors

| Status | Description                                                     |
| ------ | --------------------------------------------------------------- |
| `401`  | Unauthorized - the partner key is incorrect or invalid          |
| `500`  | Internal server error - these errors are monitored by Firecrawl |

***

### Validate API Key

Validates a Firecrawl API key and returns the associated team name and user email address. The API key will only return as valid if it was created through this partner integration.

```
POST /partner/v1/api-keys/validate
```

#### Important Notes

* Firecrawl API keys do not have permissions or an expiry date.
* API keys can be manually deleted by users at any time.
* Deleted API keys are not soft-deleted. Firecrawl cannot distinguish a deleted key from one that never existed.

#### Request

```bash cURL theme={null}
curl -X POST "https://integrations.firecrawl.dev/partner/v1/api-keys/validate" \
  -H "Authorization: Bearer <partner key>" \
  -H "Content-Type: application/json" \
  -d '{"apiKey": "fc-..."}'
```

**Body**

| Field    | Type   | Required | Description             |
| -------- | ------ | -------- | ----------------------- |
| `apiKey` | string | Yes      | The API key to validate |

#### Response

**`200 OK`**

```json theme={null}
{
  "teamName": "Example Team",
  "email": "user@example.com"
}
```

| Field      | Type   | Description                                                |
| ---------- | ------ | ---------------------------------------------------------- |
| `teamName` | string | The name of the team associated with this API key          |
| `email`    | string | The email address of the user associated with this API key |

#### Errors

| Status | Description                                                                                           |
| ------ | ----------------------------------------------------------------------------------------------------- |
| `401`  | Unauthorized - the partner key is incorrect or invalid                                                |
| `404`  | API key not identifiable - the key does not exist or was not created through this partner integration |
| `500`  | Internal server error - these errors are monitored by Firecrawl                                       |

***

### Rotate API Key

Deletes an existing Firecrawl API key and creates a new one for the same user and team.

```
POST /partner/v1/api-keys/rotate
```

#### Request

```bash cURL theme={null}
curl -X POST "https://integrations.firecrawl.dev/partner/v1/api-keys/rotate" \
  -H "Authorization: Bearer <partner key>" \
  -H "Content-Type: application/json" \
  -d '{"apiKey": "fc-..."}'
```

**Body**

| Field    | Type   | Required | Description                       |
| -------- | ------ | -------- | --------------------------------- |
| `apiKey` | string | Yes      | The API key to delete and replace |

#### Response

**`200 OK`**

```json theme={null}
{
  "apiKey": "fc-..."
}
```

| Field    | Type   | Description               |
| -------- | ------ | ------------------------- |
| `apiKey` | string | The newly created API key |

#### Errors

| Status | Description                                                                                           |
| ------ | ----------------------------------------------------------------------------------------------------- |
| `401`  | Unauthorized - the partner key is incorrect or invalid                                                |
| `404`  | API key not identifiable - the key does not exist or was not created through this partner integration |
| `500`  | Internal server error - these errors are monitored by Firecrawl                                       |

***

## Become a Partner

Firecrawl's partner program is available to approved platforms. If you're interested in integrating Firecrawl into your platform and offering credits to your users, contact us at [partnerships@firecrawl.dev](mailto:partnerships@firecrawl.dev) to learn more and request a partner API key.


# MCP Web Search & Scrape in Amp
Source: https://docs.firecrawl.dev/quickstarts/amp

Add Firecrawl web scraping and search to Sourcegraph Amp

Add Firecrawl's search, scrape, crawl, and browser tools to [Sourcegraph Amp](https://ampcode.com) via MCP.

## Quick Setup

### 1. Get Your API Key

Sign up at [firecrawl.dev/app](https://www.firecrawl.dev/app) and copy your API key.

### 2. Add Firecrawl to Amp

Open Amp settings and add an MCP server. Amp accepts standard MCP config:

```json theme={null}
{
  "amp.mcpServers": {
    "firecrawl": {
      "command": "npx",
      "args": ["-y", "firecrawl-mcp"],
      "env": {
        "FIRECRAWL_API_KEY": "fc-YOUR-API-KEY"
      }
    }
  }
}
```

Replace `fc-YOUR-API-KEY` with your Firecrawl API key.

### 3. Reload Amp

Reload the Amp window. Firecrawl tools are now available to the agent.

## Quick Demo

```
Search the web for "Sourcegraph Cody vs Amp" and summarize the differences.
```

```
Scrape https://docs.firecrawl.dev and list the core endpoints.
```

```
Crawl https://example.com and output a site map as JSON.
```

## Remote Hosted URL (no Node.js required)

```json theme={null}
{
  "amp.mcpServers": {
    "firecrawl": {
      "url": "https://mcp.firecrawl.dev/fc-YOUR-API-KEY/v2/mcp"
    }
  }
}
```

## Troubleshooting

* **Server fails to start** — check Amp's MCP log view for stderr output.
* **Missing `npx`** — install Node.js 18+ or use the remote hosted URL above.


# MCP Web Search & Scrape in Antigravity
Source: https://docs.firecrawl.dev/quickstarts/antigravity

Add Firecrawl web scraping and search to Google Antigravity

Add Firecrawl's search, scrape, crawl, and browser tools to [Google Antigravity](https://antigravity.google/) via MCP.

## Quick Setup

### 1. Get Your API Key

Sign up at [firecrawl.dev/app](https://www.firecrawl.dev/app) and copy your API key.

### 2. Add Firecrawl to Antigravity

Open Antigravity settings (`Cmd/Ctrl + ,`), search for **MCP Servers**, and add a new server with:

```json theme={null}
{
  "mcpServers": {
    "firecrawl": {
      "command": "npx",
      "args": ["-y", "firecrawl-mcp"],
      "env": {
        "FIRECRAWL_API_KEY": "fc-YOUR-API-KEY"
      }
    }
  }
}
```

Replace `fc-YOUR-API-KEY` with your Firecrawl API key.

### 3. Reload Antigravity

Reload the window (`Cmd/Ctrl + Shift + P` → `Reload Window`). The agent now has Firecrawl's web tools available.

## Quick Demo

In an Antigravity agent chat:

```
Search the web for "Vercel AI SDK v5 release notes" and summarize.
```

```
Scrape https://docs.firecrawl.dev/ai-onboarding and list every linked guide.
```

```
Crawl https://example.com and extract every page title.
```

Antigravity routes those tool calls through Firecrawl MCP automatically.

## Remote Hosted URL (no Node.js required)

```json theme={null}
{
  "mcpServers": {
    "firecrawl": {
      "url": "https://mcp.firecrawl.dev/fc-YOUR-API-KEY/v2/mcp"
    }
  }
}
```

## Troubleshooting

* **Server shows as "failed"** — check the MCP output panel for stderr. Most failures are a missing API key or `npx` not on `PATH`.
* **Tools not invoked** — explicitly mention Firecrawl (e.g., "Use Firecrawl to scrape…") in your first prompt so the agent picks the right tool.


# AutoGen
Source: https://docs.firecrawl.dev/quickstarts/autogen

Use Firecrawl as a tool inside Microsoft AutoGen multi-agent conversations.

Integrate Firecrawl with [Microsoft AutoGen](https://github.com/microsoft/autogen) to give multi-agent conversations live web search, scrape, and crawl tools.

## Setup

```bash theme={null}
pip install -U "autogen-agentchat" "autogen-ext[openai]" firecrawl-py
```

Set your keys:

```bash theme={null}
export FIRECRAWL_API_KEY=fc-YOUR-API-KEY
export OPENAI_API_KEY=sk-YOUR-OPENAI-KEY
```

## Firecrawl as an AutoGen Tool

This example wraps Firecrawl's `scrape` and `search` as AutoGen function tools, then lets a single `AssistantAgent` use them to answer a question.

```python theme={null}
import asyncio
import os
from firecrawl import FirecrawlApp
from autogen_agentchat.agents import AssistantAgent
from autogen_agentchat.ui import Console
from autogen_ext.models.openai import OpenAIChatCompletionClient

firecrawl = FirecrawlApp(api_key=os.environ["FIRECRAWL_API_KEY"])


def scrape_url(url: str) -> str:
    """Scrape a URL and return clean markdown."""
    result = firecrawl.scrape(url, formats=["markdown"])
    return result.markdown or ""


def web_search(query: str, limit: int = 5) -> list[dict]:
    """Search the web and return the top results."""
    result = firecrawl.search(query, limit=limit)
    return [
        {"title": r.title, "url": r.url, "snippet": r.description}
        for r in result.web or []
    ]


async def main() -> None:
    model = OpenAIChatCompletionClient(model="gpt-4o-mini")

    researcher = AssistantAgent(
        name="researcher",
        model_client=model,
        tools=[scrape_url, web_search],
        system_message=(
            "You are a web researcher. Use web_search to find candidate sources, "
            "then scrape_url to read the most relevant ones. Cite URLs in your answer."
        ),
    )

    await Console(
        researcher.run_stream(
            task="What does Firecrawl's /agent endpoint do? Cite the docs."
        )
    )


if __name__ == "__main__":
    asyncio.run(main())
```

Run it:

```bash theme={null}
python researcher.py
```

## Multi-Agent: Researcher + Writer

Hand Firecrawl output from a researcher agent to a writer agent in a round-robin team.

```python theme={null}
import asyncio
import os
from firecrawl import FirecrawlApp
from autogen_agentchat.agents import AssistantAgent
from autogen_agentchat.conditions import MaxMessageTermination
from autogen_agentchat.teams import RoundRobinGroupChat
from autogen_agentchat.ui import Console
from autogen_ext.models.openai import OpenAIChatCompletionClient

firecrawl = FirecrawlApp(api_key=os.environ["FIRECRAWL_API_KEY"])


def scrape_url(url: str) -> str:
    result = firecrawl.scrape(url, formats=["markdown"])
    return result.markdown or ""


def web_search(query: str, limit: int = 5) -> list[dict]:
    result = firecrawl.search(query, limit=limit)
    return [
        {"title": r.title, "url": r.url, "snippet": r.description}
        for r in result.web or []
    ]


async def main() -> None:
    model = OpenAIChatCompletionClient(model="gpt-4o-mini")

    researcher = AssistantAgent(
        name="researcher",
        model_client=model,
        tools=[scrape_url, web_search],
        system_message="Gather sources with web_search + scrape_url. Reply with bullet-point findings and URLs.",
    )

    writer = AssistantAgent(
        name="writer",
        model_client=model,
        system_message="Turn the researcher's findings into a 200-word briefing with inline citations.",
    )

    team = RoundRobinGroupChat(
        [researcher, writer],
        termination_condition=MaxMessageTermination(max_messages=6),
    )

    await Console(team.run_stream(task="Write a briefing on Firecrawl's crawl endpoint."))


if __name__ == "__main__":
    asyncio.run(main())
```

## Notes

* Firecrawl's Python SDK is synchronous; AutoGen will call your wrappers inside its event loop without issues for small workloads. For heavy concurrent scraping, move calls off the main thread or use [batch scrape](/features/batch-scrape).
* Replace `OpenAIChatCompletionClient` with any AutoGen-supported model client (Azure OpenAI, Anthropic via `autogen-ext`, Ollama, etc.). Firecrawl is model-agnostic.
* See the [AutoGen docs](https://microsoft.github.io/autogen/) for agent patterns beyond round-robin (selector, swarm, nested teams).


# MCP Web Search & Scrape in Claude Code
Source: https://docs.firecrawl.dev/quickstarts/claude-code

Add web scraping and search to Claude Code in 2 minutes

Add web scraping and search capabilities to Claude Code with Firecrawl MCP.

## Quick Setup

### 1. Get Your API Key

Sign up at [firecrawl.dev/app](https://firecrawl.dev/app) and copy your API key.

### 2. Add Firecrawl MCP Server

**Option A: Remote hosted URL (recommended)**

```bash theme={null}
claude mcp add firecrawl --url https://mcp.firecrawl.dev/your-api-key/v2/mcp
```

**Option B: Local (npx)**

```bash theme={null}
claude mcp add firecrawl -e FIRECRAWL_API_KEY=your-api-key -- npx -y firecrawl-mcp
```

Replace `your-api-key` with your actual Firecrawl API key.

Done! You can now search and scrape the web from Claude Code.

## Quick Demo

Try these in Claude Code:

**Search the web:**

```
Search for the latest Next.js 15 features
```

**Scrape a page:**

```
Scrape firecrawl.dev and tell me what it does
```

**Get documentation:**

```
Find and scrape the Stripe API docs for payment intents
```

Claude will automatically use Firecrawl's search and scrape tools to get the information.


# MCP Web Search & Scrape in Codex CLI
Source: https://docs.firecrawl.dev/quickstarts/codex-cli

Add Firecrawl web scraping and search to OpenAI Codex CLI

Add Firecrawl's search, scrape, crawl, and browser tools to [OpenAI Codex CLI](https://github.com/openai/codex) via MCP.

## Quick Setup

### 1. Get Your API Key

Sign up at [firecrawl.dev/app](https://www.firecrawl.dev/app) and copy your API key.

### 2. Add Firecrawl to Codex

Codex reads MCP server config from `~/.codex/config.toml`. Add the Firecrawl server:

```toml theme={null}
[mcp_servers.firecrawl]
command = "npx"
args = ["-y", "firecrawl-mcp"]

[mcp_servers.firecrawl.env]
FIRECRAWL_API_KEY = "fc-YOUR-API-KEY"
```

Replace `fc-YOUR-API-KEY` with your Firecrawl API key.

### 3. Start Codex

```bash theme={null}
codex
```

Codex discovers the Firecrawl tools on launch. Confirm they are loaded:

```bash theme={null}
/mcp
```

You should see `firecrawl` listed with tools like `firecrawl_search`, `firecrawl_scrape`, `firecrawl_crawl`, and `firecrawl_extract`.

## Quick Demo

Try these prompts:

```
Search the web for the latest Next.js App Router release notes and summarize.
```

```
Scrape https://docs.firecrawl.dev and list the top-level sections.
```

```
Crawl https://example.com and save the markdown for every page under /blog.
```

## Remote Hosted URL (no Node.js required)

If you prefer not to run `npx` locally:

```toml theme={null}
[mcp_servers.firecrawl]
url = "https://mcp.firecrawl.dev/fc-YOUR-API-KEY/v2/mcp"
```

## Troubleshooting

* **Codex doesn't see the tools** — run `codex --version` to confirm you're on a version with MCP support, then restart the CLI after editing `config.toml`.
* **`spawn npx ENOENT`** — install Node.js 18+ and ensure `npx` is on your `PATH`, or switch to the remote hosted URL above.
* **401 / invalid key** — regenerate an API key at [firecrawl.dev/app/api-keys](https://www.firecrawl.dev/app/api-keys).


# MCP Web Search & Scrape in Cursor
Source: https://docs.firecrawl.dev/quickstarts/cursor

Add web scraping and search to Cursor in 2 minutes

Add web scraping and search capabilities to Cursor with Firecrawl MCP.

## Quick Setup

### 1. Get Your API Key

Sign up at [firecrawl.dev/app](https://firecrawl.dev/app) and copy your API key.

### 2. Add to Cursor

Open Settings (`Cmd+,`), search for "MCP", and add:

```json theme={null}
{
  "mcpServers": {
    "firecrawl": {
      "command": "npx",
      "args": ["-y", "firecrawl-mcp"],
      "env": {
        "FIRECRAWL_API_KEY": "your_api_key_here"
      }
    }
  }
}
```

Replace `your_api_key_here` with your actual Firecrawl API key.

### 3. Restart Cursor

Done! You can now search and scrape the web from Cursor.

## Quick Demo

Try these in Cursor Chat (`Cmd+K`):

**Search:**

```
Search for TypeScript best practices 2025
```

**Scrape:**

```
Scrape firecrawl.dev and tell me what it does
```

**Get docs:**

```
Scrape the React hooks documentation and explain useEffect
```

Cursor will automatically use Firecrawl tools.

## Windows Troubleshooting

If you see a `spawn npx ENOENT` or "No server info found" error on Windows, Cursor cannot find `npx` in your PATH. Try one of these fixes:

**Option A: Use the full path to `npx.cmd`**

Run `where npx` in Command Prompt to get the full path, then update your config:

```json theme={null}
{
  "mcpServers": {
    "firecrawl": {
      "command": "C:\\Program Files\\nodejs\\npx.cmd",
      "args": ["-y", "firecrawl-mcp"],
      "env": {
        "FIRECRAWL_API_KEY": "your_api_key_here"
      }
    }
  }
}
```

Replace the `command` path with the output from `where npx`.

**Option B: Use the remote hosted URL (no Node.js required)**

```json theme={null}
{
  "mcpServers": {
    "firecrawl": {
      "url": "https://mcp.firecrawl.dev/YOUR-API-KEY/v2/mcp"
    }
  }
}
```

Replace `YOUR-API-KEY` with your Firecrawl API key.


# MCP Web Search & Scrape in Gemini CLI
Source: https://docs.firecrawl.dev/quickstarts/gemini-cli

Add Firecrawl web scraping and search to Google Gemini CLI

Add Firecrawl's search, scrape, crawl, and browser tools to [Google Gemini CLI](https://github.com/google-gemini/gemini-cli) via MCP.

## Quick Setup

### 1. Get Your API Key

Sign up at [firecrawl.dev/app](https://www.firecrawl.dev/app) and copy your API key.

### 2. Add Firecrawl to Gemini CLI

Gemini CLI reads MCP config from `~/.gemini/settings.json` (global) or `.gemini/settings.json` in your project. Add:

```json theme={null}
{
  "mcpServers": {
    "firecrawl": {
      "command": "npx",
      "args": ["-y", "firecrawl-mcp"],
      "env": {
        "FIRECRAWL_API_KEY": "fc-YOUR-API-KEY"
      }
    }
  }
}
```

Replace `fc-YOUR-API-KEY` with your Firecrawl API key.

### 3. Launch Gemini CLI

```bash theme={null}
gemini
```

Confirm the server is loaded:

```
/mcp list
```

You should see `firecrawl` and its tools.

## Quick Demo

```
Use firecrawl to search the web for "Gemini 2.5 context window" and summarize the top 5 results.
```

```
Scrape https://ai.google.dev/gemini-api/docs and outline the sections.
```

```
Crawl https://example.com and extract the product names from /products.
```

## Remote Hosted URL (no Node.js required)

```json theme={null}
{
  "mcpServers": {
    "firecrawl": {
      "url": "https://mcp.firecrawl.dev/fc-YOUR-API-KEY/v2/mcp"
    }
  }
}
```

## Troubleshooting

* **Tools don't show up** — restart Gemini CLI after editing `settings.json`; MCP servers are loaded at startup.
* **`spawn npx ENOENT`** — install Node.js 18+ or use the remote hosted URL.
* **Rate-limited** — upgrade your Firecrawl plan at [firecrawl.dev/pricing](https://www.firecrawl.dev/pricing).


# Nous Research
Source: https://docs.firecrawl.dev/quickstarts/nous-research

Use Firecrawl as a tool with Nous Research Hermes models.

Pair [Nous Research](https://nousresearch.com) Hermes models with Firecrawl to give Hermes live web search, scrape, and crawl.

Hermes models support OpenAI-compatible tool calls, so Firecrawl plugs in as a function the model can invoke. You can reach Hermes via the Nous Portal API, via [OpenRouter](/quickstarts/openrouter), or through Nous's Forge agent platform.

## Setup

```bash theme={null}
npm install @mendable/firecrawl-js openai zod
```

```bash theme={null}
export FIRECRAWL_API_KEY=fc-YOUR-API-KEY
export NOUS_API_KEY=YOUR-NOUS-PORTAL-KEY
```

## Hermes + Firecrawl Tool Call

```typescript theme={null}
import FirecrawlApp from '@mendable/firecrawl-js';
import OpenAI from 'openai';
import { z } from 'zod';

const firecrawl = new FirecrawlApp({ apiKey: process.env.FIRECRAWL_API_KEY });

// Nous Portal is OpenAI-compatible
const nous = new OpenAI({
  apiKey: process.env.NOUS_API_KEY,
  baseURL: 'https://inference-api.nousresearch.com/v1',
});

const SearchArgs = z.object({
  query: z.string().describe('The web search query'),
  limit: z.number().int().min(1).max(10).default(5),
});

const ScrapeArgs = z.object({
  url: z.string().describe('The URL to scrape'),
});

const tools = [
  {
    type: 'function' as const,
    function: {
      name: 'web_search',
      description: 'Search the web with Firecrawl and return top results.',
      parameters: z.toJSONSchema(SearchArgs),
    },
  },
  {
    type: 'function' as const,
    function: {
      name: 'scrape_website',
      description: 'Scrape the markdown content of a URL.',
      parameters: z.toJSONSchema(ScrapeArgs),
    },
  },
];

const response = await nous.chat.completions.create({
  model: 'Hermes-4-405B',
  tools,
  messages: [
    {
      role: 'user',
      content: 'Research Firecrawl\'s /agent endpoint and cite the docs.',
    },
  ],
});

for (const call of response.choices[0]?.message.tool_calls ?? []) {
  if (call.function.name === 'web_search') {
    const { query, limit } = SearchArgs.parse(JSON.parse(call.function.arguments));
    const results = await firecrawl.search(query, { limit });
    console.log(results.web);
  }
  if (call.function.name === 'scrape_website') {
    const { url } = ScrapeArgs.parse(JSON.parse(call.function.arguments));
    const page = await firecrawl.scrape(url, { formats: ['markdown'] });
    console.log(page.markdown);
  }
}
```

## Hermes via OpenRouter

Prefer a single gateway for multiple models? Route Hermes through OpenRouter:

```typescript theme={null}
const client = new OpenAI({
  apiKey: process.env.OPENROUTER_API_KEY,
  baseURL: 'https://openrouter.ai/api/v1',
});

await client.chat.completions.create({
  model: 'nousresearch/hermes-4-405b',
  // ...same tools, same messages
});
```

See the [OpenRouter guide](/quickstarts/openrouter) for the full pattern.

## Notes

* Hermes is strong at structured tool output — pair with Firecrawl's [JSON format](/features/llm-extract) to chain scrape → extract cleanly.
* For long-running agent loops, stream tool calls and use Firecrawl's [async crawl](/features/crawl) so the model isn't blocked on large scrapes.
* Verify the exact Nous Portal base URL and model slug at [portal.nousresearch.com](https://portal.nousresearch.com) — model names update as new Hermes generations ship.


# OpenClaw
Source: https://docs.firecrawl.dev/quickstarts/openclaw

Use Firecrawl with OpenClaw to give your agents web scraping, search, and browser automation capabilities.

Integrate Firecrawl with OpenClaw to give your agents the ability to scrape, search, crawl, extract, and interact with the web — all through the [Firecrawl CLI](/sdks/cli).

## Why Firecrawl + OpenClaw

* **No local browser needed** — every session runs in a remote, isolated sandbox. No Chromium installs, no driver conflicts, no RAM pressure on your machine.
* **Real parallelism** — run many browser sessions at once without local resource fights. Agents can browse in batches across multiple sites simultaneously.
* **Secure by default** — navigation, DOM evaluation, and extraction all happen inside disposable sandboxes, not on your workstation.
* **Better token economics** — agents get back clean artifacts (snapshots, extracted fields) instead of hauling giant DOMs and driver logs into the context window.
* **Full web toolkit** — scrape, search, and browser automation all through a single CLI that your agent already knows how to use.

## Setup

Tell your agent to install the Firecrawl CLI, authenticate and initialize the skill with this command:

```bash theme={null}
npx -y firecrawl-cli init --browser --all
```

* `--all` installs the Firecrawl skill to every detected AI coding agent
* `--browser` opens the browser for Firecrawl authentication automatically

or install everything seperately:

```bash theme={null}
npm install -g firecrawl-cli
firecrawl init skills
firecrawl login --browser
# Or, skip the browser and provide your API key directly:
export FIRECRAWL_API_KEY="fc-YOUR-API-KEY"
```

Verify everything is set up correctly:

```bash theme={null}
firecrawl --status
```

<Tip>
  Once the skill is installed, your OpenClaw agent automatically discovers and uses Firecrawl commands — no extra configuration needed.
</Tip>

## Scrape

Scrape a single page and extract its content:

```bash theme={null}
firecrawl https://example.com --only-main-content
```

Get specific formats:

```bash theme={null}
firecrawl https://example.com --format markdown,links --pretty
```

## Search

Search the web and optionally scrape the results:

```bash theme={null}
firecrawl search "latest AI funding rounds 2025" --limit 10

# Search and scrape the results
firecrawl search "OpenClaw documentation" --scrape --scrape-formats markdown
```

## Browser

Launch a remote browser session for interactive automation. Each session runs in an isolated sandbox — no local Chromium install required. `agent-browser` is pre-installed with 40+ commands.

```bash theme={null}
# Shorthand: auto-launches a session if none is active
firecrawl browser "open https://news.ycombinator.com"
firecrawl browser "snapshot"
firecrawl browser "scrape"
firecrawl browser close
```

Interact with page elements using refs from the snapshot:

```bash theme={null}
firecrawl browser "open https://example.com"
firecrawl browser "snapshot"
# snapshot returns @ref IDs — use them to interact
firecrawl browser "click @e5"
firecrawl browser "fill @e3 'search query'"
firecrawl browser "scrape"
firecrawl browser close
```

<Note>
  The shorthand form (`firecrawl browser "..."`) sends commands to `agent-browser` automatically and auto-launches a sandbox session if there isn't one active. Your agent issues intent-level actions (`open`, `click`, `fill`, `snapshot`, `scrape`) instead of writing Playwright code.
</Note>

## Example: tell your agent

Here are some prompts you can give your OpenClaw agent:

* *"Use Firecrawl to scrape [https://example.com](https://example.com) and summarize the main content."*
* *"Search for the latest OpenAI news and give me a summary of the top 5 results."*
* *"Use Firecrawl Browser to open Hacker News, get the top 5 stories, and the first 10 comments on each."*
* *"Crawl the docs at [https://docs.firecrawl.dev](https://docs.firecrawl.dev) and save them to a file."*

## Further reading

* [Firecrawl CLI reference](/sdks/cli)
* [Browser Sandbox docs](/features/browser)
* [Agent docs](/features/agent)


# MCP Web Search & Scrape in OpenCode
Source: https://docs.firecrawl.dev/quickstarts/opencode

Add Firecrawl web scraping and search to OpenCode

Add Firecrawl's search, scrape, crawl, and browser tools to [OpenCode](https://opencode.ai) via MCP.

## Quick Setup

### 1. Get Your API Key

Sign up at [firecrawl.dev/app](https://www.firecrawl.dev/app) and copy your API key.

### 2. Add Firecrawl to OpenCode

OpenCode reads config from `~/.config/opencode/config.json` (global) or `./opencode.json` in your project. Add:

```json theme={null}
{
  "$schema": "https://opencode.ai/config.json",
  "mcp": {
    "firecrawl": {
      "type": "local",
      "command": ["npx", "-y", "firecrawl-mcp"],
      "environment": {
        "FIRECRAWL_API_KEY": "fc-YOUR-API-KEY"
      }
    }
  }
}
```

Replace `fc-YOUR-API-KEY` with your Firecrawl API key.

### 3. Start OpenCode

```bash theme={null}
opencode
```

OpenCode loads MCP servers on startup. Confirm Firecrawl is attached:

```
/mcp
```

## Quick Demo

```
Search the web for "Bun 2.0 changelog" and summarize the top results.
```

```
Scrape https://docs.firecrawl.dev/introduction and list the code examples.
```

```
Crawl https://example.com/blog and save each post as markdown.
```

## Remote Hosted URL (no Node.js required)

```json theme={null}
{
  "mcp": {
    "firecrawl": {
      "type": "remote",
      "url": "https://mcp.firecrawl.dev/fc-YOUR-API-KEY/v2/mcp"
    }
  }
}
```

## Troubleshooting

* **Server not attached** — run `opencode doctor` to inspect MCP load errors.
* **Permission denied on `npx`** — install Node.js 18+ and ensure your shell picks up the install (`which npx`).


# OpenRouter
Source: https://docs.firecrawl.dev/quickstarts/openrouter

Use Firecrawl as a tool with any model served by OpenRouter.

Pair [OpenRouter](https://openrouter.ai) — a unified API for hundreds of LLMs — with Firecrawl to give any model live web search, scrape, and crawl.

OpenRouter's API is OpenAI-compatible, so you can use the OpenAI SDK pointed at OpenRouter's base URL plus Firecrawl's Python or JavaScript SDK as the tool.

## Setup

```bash theme={null}
npm install @mendable/firecrawl-js openai zod
```

```bash theme={null}
export FIRECRAWL_API_KEY=fc-YOUR-API-KEY
export OPENROUTER_API_KEY=sk-or-YOUR-OPENROUTER-KEY
```

## Scrape + Summarize with Any OpenRouter Model

This scrapes a page with Firecrawl and summarizes it with whatever model you pick from OpenRouter — here, Claude Haiku 4.5.

```typescript theme={null}
import FirecrawlApp from '@mendable/firecrawl-js';
import OpenAI from 'openai';

const firecrawl = new FirecrawlApp({ apiKey: process.env.FIRECRAWL_API_KEY });

const openrouter = new OpenAI({
  apiKey: process.env.OPENROUTER_API_KEY,
  baseURL: 'https://openrouter.ai/api/v1',
});

const scraped = await firecrawl.scrape('https://docs.firecrawl.dev', {
  formats: ['markdown'],
});

const completion = await openrouter.chat.completions.create({
  model: 'anthropic/claude-haiku-4.5',
  messages: [
    { role: 'user', content: `Summarize in 5 bullets: ${scraped.markdown}` },
  ],
});

console.log(completion.choices[0]?.message.content);
```

Switch the `model` string to any [OpenRouter-supported model](https://openrouter.ai/models) — `openai/gpt-5`, `google/gemini-2.5-pro`, `meta-llama/llama-4-maverick`, etc.

## Tool Calling: Model Decides When to Scrape

OpenRouter supports OpenAI-style tool calls, so Firecrawl plugs in as a function the model can invoke.

```typescript theme={null}
import FirecrawlApp from '@mendable/firecrawl-js';
import OpenAI from 'openai';
import { z } from 'zod';

const firecrawl = new FirecrawlApp({ apiKey: process.env.FIRECRAWL_API_KEY });
const openrouter = new OpenAI({
  apiKey: process.env.OPENROUTER_API_KEY,
  baseURL: 'https://openrouter.ai/api/v1',
});

const ScrapeArgs = z.object({
  url: z.string().describe('The URL to scrape'),
});

const tools = [
  {
    type: 'function' as const,
    function: {
      name: 'scrape_website',
      description: 'Scrape the markdown content of any URL via Firecrawl',
      parameters: z.toJSONSchema(ScrapeArgs),
    },
  },
];

const response = await openrouter.chat.completions.create({
  model: 'anthropic/claude-haiku-4.5',
  tools,
  messages: [
    {
      role: 'user',
      content: 'What is Firecrawl? Visit firecrawl.dev and tell me.',
    },
  ],
});

const call = response.choices[0]?.message.tool_calls?.[0];
if (call?.function.name === 'scrape_website') {
  const { url } = ScrapeArgs.parse(JSON.parse(call.function.arguments));
  const page = await firecrawl.scrape(url, { formats: ['markdown'] });
  console.log(page.markdown);
}
```

## Python

```python theme={null}
import os
from firecrawl import FirecrawlApp
from openai import OpenAI

firecrawl = FirecrawlApp(api_key=os.environ["FIRECRAWL_API_KEY"])

openrouter = OpenAI(
    api_key=os.environ["OPENROUTER_API_KEY"],
    base_url="https://openrouter.ai/api/v1",
)

page = firecrawl.scrape("https://docs.firecrawl.dev", formats=["markdown"])

completion = openrouter.chat.completions.create(
    model="anthropic/claude-haiku-4.5",
    messages=[{"role": "user", "content": f"Summarize: {page.markdown}"}],
)

print(completion.choices[0].message.content)
```

## Notes

* Firecrawl is fully model-agnostic — pick any OpenRouter model without changing the scrape code.
* Many top OpenRouter apps (Cline, Roo Code, Kilo, Cursor, Continue) are themselves agent harnesses that can use Firecrawl MCP — see [MCP Server](/mcp-server) to wire Firecrawl into those directly.
* For large jobs, use [batch scrape](/features/batch-scrape) to stay within LLM context budgets.


# MCP Web Search & Scrape in Windsurf
Source: https://docs.firecrawl.dev/quickstarts/windsurf

Add web scraping and search to Windsurf in 2 minutes

Add web scraping and search capabilities to Windsurf with Firecrawl MCP.

## Quick Setup

### 1. Get Your API Key

Sign up at [firecrawl.dev/app](https://firecrawl.dev/app) and copy your API key.

### 2. Add to Windsurf

Add this to your `./codeium/windsurf/model_config.json`:

```json theme={null}
{
  "mcpServers": {
    "firecrawl": {
      "command": "npx",
      "args": ["-y", "firecrawl-mcp"],
      "env": {
        "FIRECRAWL_API_KEY": "YOUR_API_KEY"
      }
    }
  }
}
```

Replace `YOUR_API_KEY` with your actual Firecrawl API key.

### 3. Restart Windsurf

Done! Windsurf can now search and scrape the web.

## Quick Demo

Try these in Windsurf:

**Search:**

```
Search for the latest Tailwind CSS features
```

**Scrape:**

```
Scrape firecrawl.dev and explain what it does
```

**Get docs:**

```
Find and scrape the Supabase authentication documentation
```

Windsurf's AI agents will automatically use Firecrawl tools.