[nixef85322]

I am figuring out how to create a leak database search engine ? Which Cloud services one can opt to store such large amount of data (billions of records) ? Which service one can opt to get results from that data within less than seconds. Has anyone any idea how one can build that or what can be the costing to store & fetch from such a large data ? I have noticed some telegram bots to check for leak data as well. How they are working ?

[gaard]

they're most likely using api or some other form of searching databases, because it'd be much easier to grab the info and pull it from the site rather than host billions of records somewhere, for spacial reasons. i know of telegram bots that have scraped searches on snusbase and intelx for their records, but their methods are unknown to me. sorry, but @netnsher could probably help.