Crawl4AI
LibraryOpen-source crawler for LLMs
You need clean Markdown or JSON from web pages to feed an LLM or RAG pipeline
Agents & automationScraping & data
86.1k stars
2 tools tagged “rag”, of 82 on the shelf.
Self-hosted research notebook
You want a local, private AI research assistant over your PDFs, videos and notes
Open-source crawler for LLMs
You need clean Markdown or JSON from web pages to feed an LLM or RAG pipeline