Description
Token counts for language-model text can be computed from LuaJIT through a prebuilt Rust tiktoken binding.
This binary package is useful for developers estimating prompt size, API costs, or chunking behavior in LuaJIT tools. It is a tokenization library, not an AI model.
Token estimates depend on the model encoding being used. Avoid sending private text to external services unless your workflow requires it.