Collect and locally store comments from a Bilibili video or user for personal research.
Search collected comments and generate a word cloud from the results.
Share a collected comment dataset with a small team using JSONL export and import.
| doughnutbean/bilispider | aarav90-cpu/fetchit | adammartinez271828/x4-analysis | |
|---|---|---|---|
| Stars | 10 | 10 | 10 |
| Language | Python | Python | Python |
| Setup difficulty | easy | easy | easy |
| Complexity | 2/5 | 2/5 | 2/5 |
| Audience | developer | developer | general |
Figures from each repo's GitHub metadata at analysis time.
Running from source needs pip install plus a QR-code login step before collecting comments.
BiliSpider is a desktop tool for collecting and browsing comments from Bilibili, a Chinese video platform, meant for learning, personal research, and small team collaboration rather than large scale scraping. It has a simple graphical interface built with Tkinter, lets you log in by scanning a QR code, and stores everything it collects in a local SQLite database on your own computer. The README is explicit that people should keep their request rate and collection scope reasonable and not use the tool for high frequency bulk requests or to get around platform rules. Once logged in, you can look up an account by its user ID to see its profile and video list, then collect the top level and reply level comments on those videos. Collection can pause and resume where it left off, avoids collecting the same comment twice, and includes rate limiting, cooldowns, and protection against the anti scraping response the platform sometimes returns. Local searches combine what is already stored in the database with fresh results from the live API and mark which source each result came from. Search results can be turned into a word cloud that you can save as an image, zoom or resize, and filter by blocking specific high frequency words with one click. Both a built in stopword list and a personal custom list are supported, and the personal list stays only on your own machine. Data can be shared between collaborators through JSONL files rather than the raw database, with comments deduplicated automatically on import. Command line tools let you export all comments, export by user or video ID, split exports into separate files, and import datasets other people have contributed. The installed Windows version ships without any existing database, but can optionally check for and download small shared dataset updates on startup. To run it from source, you install the Python dependencies, then run a login script followed by the GUI script, or use command line collection directly with options for limiting by date range or number of videos. A prebuilt Windows installer is also available, built with PyInstaller and Inno Setup, which installs without needing administrator rights and keeps its data in the user's own application folder.
A desktop tool for collecting and searching Bilibili video comments locally, with a GUI, QR-code login, SQLite storage, and shared JSONL datasets for small teams.
Mainly Python. The stack also includes Python, Tkinter, SQLite.
Setup difficulty is rated easy, with roughly 30min to a first successful run.
Mainly developer.
This repo across BitVibe Labs
Don't trust strangers blindly. Verify against the repo.