Daniel AvilaCreated claude-code-templates, 32k+ GitHub stars349 installs on skills.shHuggingface Tokenizers
Fast tokenizers optimized for research and production.
3 static scans, 0 counted findings.
Edge checks it again each time your agent loads it.
What it does
- Fast tokenizers optimized for research and production.
- Rust-based implementation tokenizes 1GB in <20 seconds.
- Supports BPE, WordPiece, and Unigram algorithms.
- Train custom vocabularies, track alignments, handle padding/truncation.
- Integrates seamlessly with transformers.
- Use when you need high-performance tokenization or custom tokenizer training.
Why Daniel Avila
Co-founder of CodeGPT. Each fact links its public source.
- 32k+ GitHub stars on claude-code-templates
His CLI for configuring and monitoring Claude Code has over 32,000 stars.
GitHub - Co-founder of CodeGPT, 2M+ downloads
He co-founded CodeGPT, which has over 2 million downloads.
Personal site
How it works
- 1Copy
Try for free gives you one prompt.
- 2Paste
Paste it into Claude Code, Codex or Cursor.
- 3Ask
Ask for the outcome. Your agent loads Huggingface Tokenizers.
3 static scans, 0 counted findings.
Security state at 2026-09-30: reviewed. Static scans check the code, not how well the skill works.
State definitionsSelect the Files tab to load SKILL.md.
Skill details
Security scansReviewed
3 static scans, 0 counted findings. Security state at 2026-09-30: reviewed. Edge checks it again each time your agent loads it.
Edge static checks (Edge-run): edge-static/1.0.1. Scope: Static patterns for remote execution, credentials with network sends, obfuscation, prompt overrides, hidden Unicode, binaries and persistence. Raw findings: .
Cisco skill-scanner (Edge-run): cisco-skill-scanner/2.1.0. Scope: Local static, YARA, pipeline and behavioral analyzers; no LLM or AI Defense analysis. Raw findings: 3.
Semgrep Edge rules (Edge-run): semgrep/1.178.0+edge-rules.c15016a36fd3+offline-v1. Scope: Six Edge-authored static rules with Semgrep CE; offline execution and metrics disabled. Semgrep-maintained rules excluded. Raw findings: .
- Edge static checks
- Pass · 0 counted findings · 2026-09-27
- Cisco skill-scanner
- Pass · 0 counted findings · 2026-09-27
- Semgrep Edge rules
- Pass · 0 counted findings · 2026-09-27
Creator, source & licencedavila7
From the davila7 public repository. Edge did not write or edit this skill.
- Repository
- davila7/claude-code-templates
- Skill file
- View source at revision
- Revision
- 6af7cdf57b810aca5e544a0f695bf058f1c2a744
- Package hash
- sha256:893c0c8c0fa07650a33a0f0f45eca57a738edf769aaf217c7aa4568168f628e1
- Licence
- MIT
- Description
- Fast tokenizers optimized for research and production. Rust-based implementation tokenizes 1GB in <20 seconds. Supports BPE, WordPiece, and Unigram algorithms. Train custom vocabularies, track alignments, handle padding/truncation. Integrates seamlessly with transformers. Use when you need high-performance tokenization or custom tokenizer training.
Quoted from the skill’s own description
Read skill files5 files
5 files, 73.8 KB at revision 6af7cdf57b81.
Questions about Huggingface Tokenizers
What does Huggingface Tokenizers do?
Fast tokenizers optimized for research and production. Published by Daniel Avila (davila7/claude-code-templates).
Is Huggingface Tokenizers safe to use?
Edge security state: Reviewed (3 Edge-run static scans, scanned Sep 30, 2026). The Evidence tab shows the scan.
How do I use Huggingface Tokenizers in Claude Code, Codex or Cursor?
Click Try for free, pick your agent and follow its one step. Edge is free, with no Edge account.
Similar skills

Rigor Analyze / Rigor Audit read-only skill for deep learning research repositories. Use when the user wants to read and understand a reposi…

When the user wants to apply psychology to marketing, or asks why buyers behave or value things as they do, or whether a persuasion tactic i…

Comprehensive product research and opportunity analysis for Amazon sellers. Analyzes demand, competition, profit potential, market entry bar…

Produce an intensive, cited analytical report: executive summary, multi-angle findings, contrarian views, open questions, and full sources.…

Estimates TAM, SAM, and SOM for a market or sub-segment to ground sales capacity, territory, and quota planning - top-down, bottom-up on nam…

Organize research — manage references, notes, and collaboration.

Research a topic in depth with Tavily. Use this skill when a question needs many sources gathered and weighed together: analysing a market,…

Apply STRIDE methodology to systematically identify threats. Use when analyzing system security, conducting threat modeling sessions, or cre…