Turn any web page into clean Markdown or plain text, ready for an LLM, a RAG pipeline or an AI agent.
Give it URLs. It fetches each page, strips menus, cookie banners, sidebars and footers, and returns the main content as tidy Markdown (headings, lists, links and tables kept) and/or plain text. Fast and lightweight: no browser, results in seconds.
Sample output
| url | title | wordCount | markdown (start) |
|---|---|---|---|
| https://en.wikipedia.org/wiki/Web_scraping | Web scraping | 4083 | **Web scraping**, **web harvesting**, or **web data extraction** is [data scraping](...) used for... |
| https://www.bbc.co.uk/news | Home - BBC News | 3136 | ## Top stories ... |
Each result also has status, httpStatus, description, language, truncated, depth, fetchedAt and, if you ask for it, links.
How to use
- Paste one or more URLs into URLs.
- Choose Markdown, Plain text or both.
- Optional: Follow links to also convert linked pages in the same site section (depth 1, 2...), Max pages, and Max characters per page to fit a model's context window.
- Run, then export as JSON, CSV or Excel, or read results via the API.
Pages that can't be converted are listed with status: "failed" or "skipped" and a reason, and are not charged.
Pricing
$1 per 1,000 pages converted ($0.001 per page). Failed and skipped pages are free. Platform usage is included.
For AI agents
Call it through the Apify API or the Apify MCP server with {"startUrls": [{"url": "https://example.com/article"}], "maxCharsPerPage": 20000} and read markdown from the result. Every result has the same fields, and errors come back as data rather than crashes, so agents can handle them.
Limits
This tool reads the HTML a site sends. Pages that only show content after JavaScript runs, or that need a login, will return little or no text. Sites whose robots.txt disallows automated access are skipped.
Data and responsibility
The tool fetches only public pages, identifies itself honestly as DatasetWorksBot, follows robots.txt and crawl delays, and never logs in or gets around captchas or blocks. You are responsible for how you use the content you convert, including copyright and any personal data on the pages you choose.
Support
Found a bug or need a feature? Open an issue on this page and we'll reply within one working day.
Built and maintained by Dataset Works.