Description
Wraps Python's standard tokenize module while preserving enough information to round-trip source code. It helps formatters, codemods, and lint tools modify tokens without unnecessarily changing unrelated text.
Developers use it as a source-code tooling library. Code rewriting should be tested with version control and representative files before applying broad automated changes.