MCP Integration
MCP Overview
Model Context Protocol server for AI agent integration
Essence includes a built-in Model Context Protocol (MCP) server that exposes scraping tools directly to AI agents like Claude, Cursor, and Windsurf.
Endpoint
GET /mcpThe MCP server uses Streamable HTTP transport and is available at the same address as the REST API.
Available Tools
The MCP server exposes four tools:
scrape
Scrape a single page to Markdown or other formats.
| Parameter | Type | Default | Description |
|---|---|---|---|
url | string | — | Required. URL to scrape |
formats | string[] | ["markdown"] | Output formats |
engine | string | "auto" | Engine selection |
timeoutMs | integer | 30000 | Timeout in milliseconds |
crawl
Crawl a website following links.
| Parameter | Type | Default | Description |
|---|---|---|---|
url | string | — | Required. Starting URL |
maxDepth | integer | 2 | Maximum link depth |
limit | integer | 100 | Maximum pages |
includePaths | string[] | — | Glob patterns to include |
excludePaths | string[] | — | Glob patterns to exclude |
allowBackwardLinks | boolean | false | Follow upward links |
allowExternalLinks | boolean | false | Follow external links |
map
Discover URLs via sitemaps and link extraction.
| Parameter | Type | Default | Description |
|---|---|---|---|
url | string | — | Required. Site URL |
search | string | — | Filter query |
ignoreSitemap | boolean | false | Skip sitemap.xml |
includeSubdomains | boolean | true | Include subdomains |
limit | integer | 5000 | Max URLs |
search
Search the web via DuckDuckGo, optionally scraping results.
| Parameter | Type | Default | Description |
|---|---|---|---|
query | string | — | Required. Search query |
limit | integer | 10 | Number of results |
scrapeResults | boolean | false | Scrape each result |
Next Steps
- MCP Setup Guide — configure Claude Desktop, Claude Code, Cursor, and more