Jul 24, 2026, 05:21 PM
There are only 50 parsers/scarpers in the list, distributed by purpose :
Universal parsers/scrapers (1–10)
1. Scrapy — https://github.com/scrapy/scrapy
2. BeautifulSoup — https://github.com/wention/BeautifulSoup4
3. Requests-HTML — https://github.com/psf/requests-html
4. MechanicalSoup — https://github.com/MechanicalSoup/MechanicalSoup
5. Playwright — https://github.com/microsoft/playwright
6. Selenium — https://github.com/SeleniumHQ/selenium
7. Colly (Go) — https://github.com/gocolly/colly
8. Puppeteer — https://github.com/puppeteer/puppeteer
9. Crawlee — https://github.com/apify/crawlee
10. Node-Crawler — https://github.com/bda-research/node-crawler
⸻
Websites / Content (11–20)
11. Photon — https://github.com/s0md3v/Photon
12. Hakrawler — https://github.com/hakluke/hakrawler
13. GoSpider — https://github.com/jaeles-project/gospider
14. Katana — https://github.com/projectdiscovery/katana
15. Waybackurls — https://github.com/tomnomnom/waybackurls
16. Gau — https://github.com/lc/gau
17. ParamSpider — https://github.com/devanshbatham/ParamSpider
18. LinkFinder — https://github.com/GerbenJavado/LinkFinder
19. JSParser — https://github.com/nahamsec/JSParser
20. Arjun — https://github.com/s0md3v/Arjun
⸻
Social networks/accounts (21–30)
21. TWINT — https://github.com/twintproject/twint
22. Instaloader — https://github.com/instaloader/instaloader
23. Facebook-scraper — https://github.com/kevinzg/facebook-scraper
24. TikTok-Api — https://github.com/davidteather/TikTok-Api
25. snscrape — https://github.com/JustAnotherArchivist/snscrape
26. Telegram-history-dump — https://github.com/LonamiWebs/Telethon
27. Reddit-scraper — https://github.com/praw-dev/praw
28. YouTube-dl — https://github.com/ytdl-org/youtube-dl
29. Gallery-dl — https://github.com/mikf/gallery-dl
30. Mastodon.py — https://github.com/halcy/Mastodon.py
⸻
Data Search / Leaks / paste (31–40)
31. PasteHunter — https://github.com/kevthehermit/PasteHunter
32. Scylla — https://github.com/imthaghost/scylla
33. h8mail — https://github.com/khast3x/h8mail
34. LeakLooker — https://github.com/woj-ciech/LeakLooker
35. Breach-Parse — https://github.com/hmaverickadams/breach-parse
36. Dehashed CLI — https://github.com/DeHashed/cli
37. EmailHarvester — https://github.com/maldevel/EmailHarvester
38. GHDB Scraper — https://github.com/IvanGlinkin/Fast-Google-Dorks-Scan
39. OnionSearch — https://github.com/megadose/OnionSearch
40. Darkdump — https://github.com/josh0xA/darkdump
⸻
Domains / DNS / infrastructure (41–50)
41. Amass — https://github.com/owasp-amass/amass
42. Subfinder — https://github.com/projectdiscovery/subfinder
43. Sublist3r — https://github.com/aboul3la/Sublist3r
44. MassDNS — https://github.com/blechschmidt/massdns
45. ShuffleDNS — https://github.com/projectdiscovery/shuffledns
46. dnsrecon — https://github.com/darkoperator/dnsrecon
47. crt.sh scraper — https://github.com/az7rb/crtsh
48. httpx — https://github.com/projectdiscovery/httpx
49. Shodan CLI — https://github.com/achillean/shodan-python
50. Censys CLI — https://github.com/censys/censys-python
Universal parsers/scrapers (1–10)
1. Scrapy — https://github.com/scrapy/scrapy
2. BeautifulSoup — https://github.com/wention/BeautifulSoup4
3. Requests-HTML — https://github.com/psf/requests-html
4. MechanicalSoup — https://github.com/MechanicalSoup/MechanicalSoup
5. Playwright — https://github.com/microsoft/playwright
6. Selenium — https://github.com/SeleniumHQ/selenium
7. Colly (Go) — https://github.com/gocolly/colly
8. Puppeteer — https://github.com/puppeteer/puppeteer
9. Crawlee — https://github.com/apify/crawlee
10. Node-Crawler — https://github.com/bda-research/node-crawler
⸻
Websites / Content (11–20)
11. Photon — https://github.com/s0md3v/Photon
12. Hakrawler — https://github.com/hakluke/hakrawler
13. GoSpider — https://github.com/jaeles-project/gospider
14. Katana — https://github.com/projectdiscovery/katana
15. Waybackurls — https://github.com/tomnomnom/waybackurls
16. Gau — https://github.com/lc/gau
17. ParamSpider — https://github.com/devanshbatham/ParamSpider
18. LinkFinder — https://github.com/GerbenJavado/LinkFinder
19. JSParser — https://github.com/nahamsec/JSParser
20. Arjun — https://github.com/s0md3v/Arjun
⸻
Social networks/accounts (21–30)
21. TWINT — https://github.com/twintproject/twint
22. Instaloader — https://github.com/instaloader/instaloader
23. Facebook-scraper — https://github.com/kevinzg/facebook-scraper
24. TikTok-Api — https://github.com/davidteather/TikTok-Api
25. snscrape — https://github.com/JustAnotherArchivist/snscrape
26. Telegram-history-dump — https://github.com/LonamiWebs/Telethon
27. Reddit-scraper — https://github.com/praw-dev/praw
28. YouTube-dl — https://github.com/ytdl-org/youtube-dl
29. Gallery-dl — https://github.com/mikf/gallery-dl
30. Mastodon.py — https://github.com/halcy/Mastodon.py
⸻
Data Search / Leaks / paste (31–40)
31. PasteHunter — https://github.com/kevthehermit/PasteHunter
32. Scylla — https://github.com/imthaghost/scylla
33. h8mail — https://github.com/khast3x/h8mail
34. LeakLooker — https://github.com/woj-ciech/LeakLooker
35. Breach-Parse — https://github.com/hmaverickadams/breach-parse
36. Dehashed CLI — https://github.com/DeHashed/cli
37. EmailHarvester — https://github.com/maldevel/EmailHarvester
38. GHDB Scraper — https://github.com/IvanGlinkin/Fast-Google-Dorks-Scan
39. OnionSearch — https://github.com/megadose/OnionSearch
40. Darkdump — https://github.com/josh0xA/darkdump
⸻
Domains / DNS / infrastructure (41–50)
41. Amass — https://github.com/owasp-amass/amass
42. Subfinder — https://github.com/projectdiscovery/subfinder
43. Sublist3r — https://github.com/aboul3la/Sublist3r
44. MassDNS — https://github.com/blechschmidt/massdns
45. ShuffleDNS — https://github.com/projectdiscovery/shuffledns
46. dnsrecon — https://github.com/darkoperator/dnsrecon
47. crt.sh scraper — https://github.com/az7rb/crtsh
48. httpx — https://github.com/projectdiscovery/httpx
49. Shodan CLI — https://github.com/achillean/shodan-python
50. Censys CLI — https://github.com/censys/censys-python
