Crawler

The API to search, scrape, and interact with the web at scale. 🔥

Open source web infrastructure for AI. Scrape, crawl, and automate the web, clean markdown, browser sessions, ready for your agents.

Krawl is a customizable, lightweight, cloud-native web deception server and anti-crawler that creates fake web applications with low-hanging vulnerabilities usi…

Fast, lightweight Firecrawl/Tavily alternative in Rust. Web scraper, crawler & search API with MCP server for AI agents. Drop-in Firecrawl-compatible API (/scra…

Github自动收集比较热门的项目,并自动阅读项目,生成相关报告发送到自己的飞书中

Convert the public web into AI-ready Markdown with a local Python CLI/SDK/MCP crawler.