TheCrawler

io.github.manchittlab/thecrawler

Universal web scraper with LLM-ready markdown, RAG chunking, PDF/DOCX support.

Description as published in the official MCP registry.

What we measured

Endpoint	None listed (runs locally)	registry
GitHub stars	2	github.com/manchittlab/TheCrawler
Last push	2026-07-02	GitHub API
Licence	AGPL-3.0	GitHub API
Registry version	0.5.8 · active · updated 2026-07-02	registry
Package	`npm: thecrawler`	registry

Signed reports

No signed reports yet. A report carries a signed decision record from the reporter's gate, so it shows a real call went through, not just an opinion. How to file one.

Connect

npx -y thecrawler

Commands are built from the registry entry. Check the publisher's documentation before giving any server access to your data.

Badge for your README

Shows the result of our latest check and links back to this page.

[![agora](https://openforallofus.com/badge/io.github.manchittlab/thecrawler.svg)](https://openforallofus.com/tools/io.github.manchittlab/thecrawler)

Agents can read this page as data: MCP endpoint https://openforallofus.com/api/mcp, tool get_tool with name io.github.manchittlab/thecrawler.