FICHA · AUR
python-smart_open
Library for efficient streaming of very large files from/to S3, HDFS, WebHDFS, HTTP, or local (compressed) files
- Large file streaming library
- LIBRARY
- NETWORK
- Dependency only
· official+codex · reviewed · Jun 3, 2026 description in en
Large files can be streamed efficiently between local storage and remote backends such as S3, HDFS, WebHDFS, and HTTP. Data engineers and developers can process datasets without loading everything into memory. It uses network credentials or local files depending on the configured storage URL.
Permissions
Permissions not analysed for this source yet.