shekhnoori552-hue/pylitestream- — explained in plain English
Analysis updated 2026-05-18
Process very large CSV files on an Android phone without the app crashing from memory limits.
Read a continuous live data feed, like a public API, over a weak mobile connection without memory buildup.
Analyze video frames on an edge device using an optional OpenCV based module.
Stream large binary files in fixed size chunks on hardware with limited RAM.
| shekhnoori552-hue/pylitestream- | 0xustaz/streamgate | a-bissell/unleash-lite | |
|---|---|---|---|
| Stars | 1 | 1 | 1 |
| Language | Python | Python | Python |
| Setup difficulty | easy | hard | hard |
| Complexity | 2/5 | 4/5 | 4/5 |
| Audience | developer | developer | researcher |
Figures from each repo's GitHub metadata at analysis time.
Installed via pip, optional OpenCV dependency needed only for video processing.
pylitestream is a Python library for reading large amounts of data, such as big CSV files, live web feeds, or video, without running out of memory. It was built specifically with Android phones and other small edge devices in mind, where memory is limited and a normal Python program that tries to load an entire large file at once will crash. According to the README, the project came out of the author's own experience developing on an Android phone using an app called Pydroid 3, without access to a regular laptop. The core idea is to process data in small chunks and immediately let Python's garbage collector free that memory once a chunk has been used, so memory use stays flat no matter how large the underlying file or stream is. The library is split into four main pieces. PyLiteStreamer is the base engine for reading large binary files in fixed size chunks. StreamCSV is built for tabular data like spreadsheets, and is designed to correctly handle rows that get split across chunk boundaries, so no data is lost or corrupted at chunk edges. StreamAPI is meant for continuous live data feeds over HTTP, such as stock prices or public data feeds, pulling data down in small pieces to avoid memory buildup on unreliable mobile connections. StreamVideo is an optional add-on for basic video frame processing, built on OpenCV, that is designed to keep the rest of the library working even if OpenCV itself cannot be installed on a given device. Installation is done through pip, either as the published package or directly from this GitHub repository for the newest changes. The README includes example code for each module and describes internal benchmarks run on ARM based devices tracking memory use over time. This tool is aimed at Python developers who need to process large files or streams on phones or other memory constrained hardware, rather than on a full desktop or server.
A Python library for processing large CSV files, live data feeds, and video in small memory efficient chunks, aimed at Android and edge devices.
Mainly Python. The stack also includes Python, OpenCV, PyPI.
Setup difficulty is rated easy, with roughly 5min to a first successful run.
Mainly developer.
This repo across BitVibe Labs
Don't trust strangers blindly. Verify against the repo.