Product Launch
anonymous
1 points
1 comments
Postedabout 1 month ago
Curlie
curlie.orgmaturelibrarydevelopers, data scientists
Key Features
large dataset of 2.9M websiteseditor-approved websitessuitable for AI training
dataArtificial Intelligence
Key Features
Now, the global community behind the Curlie.org internet directory offers the entire directory for download (open source, CC-BY).
The directory contains editorial descriptions of 2.9 million hand-picked websites, sorted into categories.
Vying to start your own search engine or LLM, but lacking the resources to find the quality training data? Just use the directory data, and crawl the included websites.
In fact, our scientific partner OpenWebIndex.eu already integrated the directory data into its open search index, so even the crawling part is already accomplished for you.
This is the contribution of the Curlie community to a knowledge infrastructure that is distributed, open source and under the users' control!
(Nitty gritty technical details in the download itself, or ask here.)
Not affiliated with Hacker News or Y Combinator. We simply enrich the public API with analytics.