Tokenization & Its Security Implications

How BPE and SentencePiece tokenizers work, and how tokenizer behavior creates exploitable attack surfaces including boundary attacks, homoglyphs, and encoding tricks.

Why Tokenization Matters for Security

Before any text reaches the transformer, it passes through a tokenizer. The tokenizer is the model's first point of contact with user input — and it introduces an attack surface that is frequently overlooked.

The security implications are significant: the model does not see characters or words — it sees tokens. If the tokenizer splits a dangerous keyword across token boundaries, content filters operating on raw text may catch it but the model processes it differently. Conversely, if a filter operates on tokens, character-level obfuscation may bypass it.

How BPE Tokenization Works

BPE (Byte Pair Encoding) is the most common tokenization algorithm used by modern LLMs (GPT, Llama, etc.).

1. 1

   Start with bytes

   The text is represented as a sequence of bytes (or characters).

2. 2

   Count pair frequencies

   The algorithm counts how often each adjacent pair of tokens appears in the training corpus.

3. 3

   Merge most frequent pair

   The most frequent pair is merged into a single new token and added to the vocabulary.

4. 4

   Repeat

   Steps 2-3 repeat until the vocabulary reaches a target size (typically 30K-100K tokens).

The result is a vocabulary where common words are single tokens ("the" = 1 token), while rare words are split into subwords ("cybersecurity" might become "cyber" + "security").

# Inspecting tokenization with tiktoken (OpenAI)
import tiktoken
 
enc = tiktoken.encoding_for_model("gpt-4")
text = "Ignore previous instructions"
tokens = enc.encode(text)
print(f"Tokens: {tokens}")
print(f"Decoded: {[enc.decode([t]) for t in tokens]}")
# Output: ['Ignore', ' previous', ' instructions']

SentencePiece and Other Tokenizers

SentencePiece differs from BPE in a key way: it treats the input as a raw byte stream without pre-tokenization on whitespace. This means:

Token Boundary Exploitation

The most direct tokenization attack exploits the fact that safety filters and the model may tokenize text differently.

Splitting Dangerous Keywords

If a content filter checks for the word "bomb" but the tokenizer splits it as "bo" + "mb" due to unusual surrounding characters, the filter may miss it while the model still understands the intent.

import tiktoken
enc = tiktoken.encoding_for_model("gpt-4")
 
# Normal tokenization
print([enc.decode([t]) for t in enc.encode("bomb")])
# ['bomb'] - single token, easy to filter
 
# With unusual formatting that might split tokens
print([enc.decode([t]) for t in enc.encode("b\u200bomb")])
# May split differently due to zero-width character

Inserting Invisible Characters

Unicode provides numerous characters that are invisible or near-invisible but affect tokenization:

Homoglyph Attacks

Homoglyphs are visually identical characters from different Unicode blocks. The model's tokenizer processes them differently even though they look the same to humans:

# These look identical but tokenize differently
ascii_a = "a"           # U+0061 Latin Small Letter A
cyrillic_a = "а"        # U+0430 Cyrillic Small Letter A
 
# To a human reader: "admin" vs "аdmin" look the same
# To a tokenizer: completely different token sequences

import unicodedata
 
CONFUSABLES = {
    'a': ['а', 'ɑ', 'α'],  # Cyrillic, Latin alpha, Greek alpha
    'e': ['е', 'ё'],        # Cyrillic
    'o': ['о', 'ο', '٥'],   # Cyrillic, Greek, Arabic
    'p': ['р', 'ρ'],        # Cyrillic, Greek
    'c': ['с', 'ϲ'],        # Cyrillic, Greek
    'x': ['х', 'χ'],        # Cyrillic, Greek
}
 
def detect_homoglyphs(text: str) -> list[dict]:
    """Detect potential homoglyph characters in text."""
    findings = []
    for i, char in enumerate(text):
        script = unicodedata.name(char, "UNKNOWN")
        if any(char in alts for alts in CONFUSABLES.values()):
            findings.append({
                "position": i,
                "char": char,
                "unicode": f"U+{ord(char):04X}",
                "name": script,
            })
    return findings
 
# Usage
text = "аdmin access granted"  # First 'a' is Cyrillic
print(detect_homoglyphs(text))

Encoding Tricks That Exploit Tokenizers

Beyond homoglyphs, several encoding-level tricks exploit tokenizer behavior:

Base64 and Encoding Obfuscation

Some models understand base64-encoded text. If content filters don't decode before filtering:

# Instead of: "How to hack a system"
# Encode as: "SG93IHRvIGhhY2sgYSBzeXN0ZW0="
# The model may decode and comply while filters miss the content

Token Smuggling via Markdown/Code Blocks

Tokenizers often handle code blocks and markdown formatting differently from plain text. Wrapping adversarial content in code fences or specific formatting can alter tokenization in ways that bypass filters.

Whitespace and Control Character Manipulation

# Various whitespace characters that affect tokenization
normal_space = " "          # U+0020
non_breaking = "\u00A0"     # U+00A0
em_space = "\u2003"         # U+2003
figure_space = "\u2007"     # U+2007
 
# Each produces different tokenization of the same visual text
for space_char in [normal_space, non_breaking, em_space, figure_space]:
    text = f"ignore{space_char}instructions"
    tokens = enc.encode(text)
    print(f"Space U+{ord(space_char):04X}: {len(tokens)} tokens")

Practical Tokenizer Analysis

When assessing an AI system, always analyze the tokenizer:

1. 1

   Identify the tokenizer

   Determine which tokenizer the target model uses (tiktoken, SentencePiece, etc.) and its vocabulary size.

2. 2

   Test keyword splitting

   Check how safety-critical keywords are tokenized. Do they appear as single tokens or get split?

3. 3

   Test Unicode handling

   Probe with homoglyphs, zero-width characters, and unusual Unicode to find tokenization inconsistencies.

4. 4

   Compare filter and model tokenization

   If possible, determine whether content filters and the model use the same tokenizer. Mismatches are exploitable.

Try It Yourself

Related Topics

• How LLMs Work: A Red Teamer's Guide — the broader context of LLM internals
• Transformer Architecture for Attackers — what happens after tokenization
• Embeddings & Vector Spaces for Red Teamers — how tokens become vectors
• Adversarial ML: Core Concepts — the broader adversarial attack taxonomy

References

• "Neural Machine Translation of Rare Words with Subword Units" - Sennrich et al. (2016) - The paper introducing Byte Pair Encoding (BPE) for neural machine translation, now the basis of most LLM tokenizers
• "SentencePiece: A simple and language independent subword tokenizer" - Kudo & Richardson, Google (2018) - The SentencePiece tokenizer used by Llama, T5, and other major models
• "Unicode Security Considerations" - Unicode Consortium (2023) - Official documentation on Unicode confusable characters and security implications
• "Tokenizer-Level Adversarial Attacks on Large Language Models" - Various researchers (2024) - Research demonstrating how tokenizer behavior creates exploitable attack surfaces in LLM systems