爬虫工具

Fetch X/Twitter tweets, replies, timelines, and articles without login or API keys — field tool for AI agents.

A high performance web crawler / scraper in Elixir.

开箱即用的本地 JAV/视频 刮削、管理、播放软件，支持命令行一键安装和 Docker 部署。只需简单添加目录，即可打造你的私人 JAV/视频 媒体库，带给你顶级的浏览体验，懒人必备。| Your local JAV/video manager.

mmm lobster

HTTP API for Scrapy spiders

A Kotlin-based testing/scraping/parsing library providing the ability to analyze and extract data from HTML (server & client-side rendered). It places particular emphasis on ease of use and a high level of readability by providing an intuitive DSL. It aims to be a testing lib, but can also be used to scrape websites in a convenient fashion.

An open database of international sanctions data, persons of interest and politically exposed persons

Make a ZIM file from any Web site and surf offline!

Converts some webnovels to epub format

A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.

Java API For Chrome and Firefox

Free lead generation tool. Scrape Instagram, Twitch, TikTok, and LinkedIn profiles. Extract emails from bios, verify via SMTP, export to CSV.

Python package for scraping real estate property data

A Multi-User Selfhosted price tracker for Amazon, Aliexpress, ebay and many more along with custom stores. Get notified when price matches one or more criterias set by the user

A Scala library for scraping content from HTML pages

Twitter Internal API Document

Scrape Telegram group members (hidden members also) and add them to yours.

Telegram Members Adding Software/Script Using Termux.

Automated Tinder bot and scraper using selenium in python.

Declarative DOM extraction expression evaluator. 👨‍⚕️

Python script, which empowers people with no programming background to generate robust leads on a mass scale. This repo will be compiled of various versatile techniques used in lead generation.

A port of n0madic/twitter-scraper to Node.js.

Real-time detection of anti-bot systems, captchas & fingerprinting techniques. Identifies Cloudflare, Akamai, DataDome, reCAPTCHA, hCaptcha, Shape Security & more with confidence scoring and advanced capture tools.

Universal Reddit Scraper - Works on any Subreddit or User

A universal web-util for PHP.

Google Maps reviews scraping

Little tool in python to watch and download anime from the terminal (the better way to watch anime). Also applicable as an API

A YouTube scraper for scraping channels, playlists, and searching 🔎

Python quick start guides to get the most out of Oxylabs' Web Scraper API free trial.

A simple program for scraping Cloudflare clearance (cf_clearance) cookies from websites issuing Cloudflare challenges to visitors

SerpScrap retrieves structured search results for SEO, research, and automation. It is available as a Python package, a CLI, a Docker application, and an MCP-compatible server. Results are JSON-compatible and preserve the source engine, rank, URL, title, snippets, result type, and typed failures.