Description
SentencePiece tokenization can be used from Python through prebuilt bindings. It is developer support for NLP preprocessing and neural text generation workflows.
FICHA · AUR
python-sentencepiece-bin
Unsupervised text tokenizer for Neural Network-based text generation (Python bindings)
- Python tokenizer bindings
- LIBRARY
- PYTHON
- TOKENIZATION
- Dependency only
official+codex · reviewed · Jun 4, 2026 description in
en Permissions
Permissions not analysed for this source yet.