|
| 1 | +--- |
| 2 | +navigation_title: "Firecrawl" |
| 3 | +applies_to: |
| 4 | + stack: preview 9.4 |
| 5 | + serverless: preview |
| 6 | +--- |
| 7 | + |
| 8 | +# Firecrawl connector [firecrawl-action-type] |
| 9 | + |
| 10 | +The Firecrawl connector uses the [Firecrawl REST API v2](https://docs.firecrawl.dev/api-reference/v2-introduction) to scrape web pages, search the web, map site URLs, and crawl websites. It supports Bearer token (API key) authentication. |
| 11 | + |
| 12 | +## Create connectors in {{kib}} [define-firecrawl-ui] |
| 13 | + |
| 14 | +You can create connectors in **{{stack-manage-app}} > {{connectors-ui}}**. For example: |
| 15 | + |
| 16 | +### Connector configuration [firecrawl-connector-configuration] |
| 17 | + |
| 18 | +Firecrawl connectors have the following configuration properties: |
| 19 | + |
| 20 | +API Key (Bearer token) |
| 21 | +: Your Firecrawl API key for authentication. Provide it as the Bearer token when creating or editing the connector. The connector sends it in the `Authorization: Bearer <key>` header to `https://api.firecrawl.dev`. |
| 22 | + |
| 23 | + |
| 24 | +## Test connectors [firecrawl-action-configuration] |
| 25 | + |
| 26 | +You can test connectors as you're creating or editing the connector in {{kib}}. |
| 27 | + |
| 28 | +The Firecrawl connector has the following actions: |
| 29 | + |
| 30 | +scrape |
| 31 | +: Scrape a single URL and extract content (for example, markdown or HTML). |
| 32 | + - **url** (required): The URL to scrape. |
| 33 | + - **formats** (optional): Output formats: `markdown`, `html`, `links`, `screenshot`, `extract`. Defaults to markdown. |
| 34 | + - **onlyMainContent** (optional): Return only main content, excluding nav/footer. Defaults to true. |
| 35 | + - **waitFor** (optional): Delay in milliseconds before fetching (useful for JS-rendered pages). |
| 36 | + |
| 37 | +search |
| 38 | +: Search the web and optionally get full content from results. |
| 39 | + - **query** (required): Search query string. |
| 40 | + - **limit** (optional): Maximum number of results (1–100). Defaults to 5. |
| 41 | + |
| 42 | +map |
| 43 | +: Map a website to discover indexed URLs. |
| 44 | + - **url** (required): Base URL to map. |
| 45 | + - **search** (optional): Search term to filter/order URLs by relevance. |
| 46 | + - **limit** (optional): Maximum URLs to return. Defaults to 5000. |
| 47 | + - **includeSubdomains** (optional): Include subdomains. Defaults to true. |
| 48 | + - **ignoreQueryParameters** (optional): Dedupe by path only. Defaults to true. |
| 49 | + |
| 50 | +crawl |
| 51 | +: Start an asynchronous crawl of a website. Returns a job ID; use **getCrawlStatus** to poll for results. |
| 52 | + - **url** (required): Base URL to start crawling from. |
| 53 | + - **limit** (optional): Maximum pages to crawl. Defaults to 100. |
| 54 | + - **maxDiscoveryDepth** (optional): Maximum discovery depth. |
| 55 | + - **allowExternalLinks** (optional): Follow external links. Defaults to false. |
| 56 | + |
| 57 | +getCrawlStatus |
| 58 | +: Get the status and results of a crawl job. |
| 59 | + - **id** (required): The crawl job ID (UUID) returned from the **crawl** action. |
| 60 | + |
| 61 | + |
| 62 | +## Get API credentials [firecrawl-api-credentials] |
| 63 | + |
| 64 | +To use the Firecrawl connector, you need to: |
| 65 | + |
| 66 | +1. Sign up at [Firecrawl](https://www.firecrawl.dev) or log in to your account. |
| 67 | +2. Open [API keys](https://www.firecrawl.dev/app/api-keys) in the Firecrawl dashboard. |
| 68 | +3. Create an API key or copy an existing one (for example, `fc-...`). |
| 69 | +4. In Kibana, when creating or editing the Firecrawl connector, provide this key as the Bearer token (API key / secret). |
| 70 | + |
| 71 | +Rate limits and billing depend on your Firecrawl plan. Check [Firecrawl billing](https://docs.firecrawl.dev/billing) for details. |
0 commit comments