# Request Scrape any URL using Serply servers with automatic captcha bypass. This endpoint is perfect for extracting content from websites for AI and LLM applications. ## Endpoint ``` POST /v1/request ``` ## Description The Request endpoint allows you to scrape content from any URL using Serply's servers. This endpoint automatically bypasses most captchas and is optimized for extracting content that can be used with AI and LLM applications. You can choose to receive the content in either full HTML format or as markdown. ## Authentication All requests require authentication using the `X-Api-Key` header. See the [Authentication guide](/docs/guides/authentication) for more details. ## Request Body Parameters The request body must be JSON with the following parameters: ### `url` (required) **Type:** `string` The URL to scrape. Provide the URL as a plain string in the JSON body. **Examples:** - `https://serply.io/` - `https://serply.io/pricing` - `https://news.ycombinator.com/item?id=12345678` ### `response_type` (required) **Type:** `string` The format of the response content. **Allowed values:** - `"full"` - Returns the full HTML content of the page - `"markdown"` - Returns the content converted to markdown format (ideal for AI/LLM processing) ## Request Examples ### Using cURL #### Full Response Type ```bash curl --request POST \ --url 'https://api.serply.io/v1/request' \ --header 'Content-Type: application/json' \ --header 'X-Api-Key: YOUR_API_KEY' \ --data '{ "url": "https://serply.io/", "response_type": "full" }' ``` #### Markdown Response Type ```bash curl --request POST \ --url 'https://api.serply.io/v1/request' \ --header 'Content-Type: application/json' \ --header 'X-Api-Key: YOUR_API_KEY' \ --data '{ "url": "https://serply.io/", "response_type": "markdown" }' ``` ### Using JavaScript/Node.js #### Full Response Type ```javascript const response = await fetch('https://api.serply.io/v1/request', { method: 'POST', headers: { 'Content-Type': 'application/json', 'X-Api-Key': 'YOUR_API_KEY' }, body: JSON.stringify({ url: 'https://serply.io/', response_type: 'full' }) }); const data = await response.json(); console.log(data.data); // the raw HTML ``` #### Markdown Response Type ```javascript const response = await fetch('https://api.serply.io/v1/request', { method: 'POST', headers: { 'Content-Type': 'application/json', 'X-Api-Key': 'YOUR_API_KEY' }, body: JSON.stringify({ url: 'https://serply.io/', response_type: 'markdown' }) }); const markdown = await response.text(); // plain markdown, not JSON console.log(markdown); ``` ### Using Python #### Full Response Type ```python import requests url = "https://api.serply.io/v1/request" headers = { "Content-Type": "application/json", "X-Api-Key": "YOUR_API_KEY" } payload = { "url": "https://serply.io/", "response_type": "full" } response = requests.post(url, headers=headers, json=payload) html = response.json()["data"] print(html) ``` #### Markdown Response Type ```python import requests url = "https://api.serply.io/v1/request" headers = { "Content-Type": "application/json", "X-Api-Key": "YOUR_API_KEY" } payload = { "url": "https://serply.io/", "response_type": "markdown" } response = requests.post(url, headers=headers, json=payload) markdown = response.text # plain markdown, not JSON print(markdown) ``` ## Response The two `response_type` values return different shapes -- `full` is JSON, `markdown` is plain text. ### Response Structure #### Full Response Type Returns a JSON object with the raw HTML in `data`, alongside the upstream response metadata: ```json { "data": "...", "status": 200, "headers": { "content-type": "text/html; charset=utf-8" }, "config": { "url": "https://serply.io/", "method": "GET" } } ``` #### Markdown Response Type Returns the markdown-converted content directly as the response body -- not wrapped in JSON: ``` Content-Type: text/html; charset=utf-8 # Article Title Article content in markdown format... ``` ### Response Fields - **Full**: a JSON object whose **`data`** field (string) holds the scraped page's raw HTML, plus **`status`** (the upstream HTTP status), **`headers`** (the upstream response headers), and **`config`** (the resolved request that was sent). - **Markdown**: the response body itself is the markdown text -- there is no `content`/`url`/`response_type` wrapper. Read it as plain text, not JSON. ### Example Response (Markdown) ``` Hacker News | [new](newest) | [past](front) | [comments](newcomments) 1. [Show HN: Example](https://example.com/) (example.com) 152 points by user 3 hours ago | 76 comments ``` ## Status Codes - **200 OK** - Successful response - **400 Bad Request** - Invalid parameters (missing or invalid URL, invalid response_type) - **401 Unauthorized** - Invalid or missing API key - **404 Not Found** - The requested URL could not be accessed - **405 Method Not Allowed** - The request used `GET`; this endpoint requires `POST` - **429 Too Many Requests** - Rate limit exceeded ## Error Responses See the [Errors guide](/docs/guides/errors) for information on error response formats.