TO
huggingface/tokenizers
💥 Fast State-of-the-Art Tokenizers optimized for Research and Production
10.6k 1.1k +3/wk
GitHub
bert gpt language-model natural-language-processing natural-language-understanding nlp transformers
Trend
3
Star & Fork Trend (40 data points)
Stars
Forks
Multi-Source Signals
Growth Velocity
huggingface/tokenizers has +3 stars this period . 7-day velocity: 0.1%.
Deep analysis is being generated for this repository.
Signal-backed technical analysis will be available soon.
| Metric | tokenizers | doccano | chatgpt_system_prompt | LEANN |
|---|---|---|---|---|
| Stars | 10.6k | 10.6k | 10.5k | 10.7k |
| Forks | 1.1k | 1.8k | 1.5k | 943 |
| Weekly Growth | +3 | +3 | +1 | +13 |
| Language | Rust | Python | HTML | Python |
| Sources | 1 | 1 | 1 | 1 |
| License | Apache-2.0 | MIT | MIT | MIT |
Capability Radar vs doccano
tokenizers
doccano
Maintenance Activity 100
Last code push 0 days ago.
Community Engagement 50
Fork-to-star ratio: 10.1%. Active community forking and contributing.
Issue Burden 70
Issue data not yet available.
Growth Momentum 42
+3 stars this period — 0.03% growth rate.
License Clarity 95
Licensed under Apache-2.0. Permissive — safe for commercial use.
Risk scores are computed from real-time repository data. Higher scores indicate healthier metrics.