Tokenization & Its 安全 Implications

How BPE and SentencePiece tokenizers work, and how tokenizer behavior creates exploitable attack surfaces including boundary attacks, homoglyphs, and encoding tricks.

Why Tokenization Matters for 安全

Before any text reaches the transformer, it passes through a 分詞器. The 分詞器 is 模型's first point of contact with 使用者輸入 — and it introduces an 攻擊面 that is frequently overlooked.

The 安全 implications are significant: 模型 does not see characters or words — it sees 符元. If the 分詞器 splits a dangerous keyword across 符元 boundaries, content filters operating on raw text may catch it but 模型 processes it differently. Conversely, if a filter operates on 符元, character-level obfuscation may bypass it.

How BPE Tokenization Works

BPE (Byte Pair Encoding) is the most common 分詞 algorithm used by modern LLMs (GPT, Llama, etc.).

1. 1

   Start with bytes

   The text is represented as a sequence of bytes (or characters).

2. 2

   Count pair frequencies

   The algorithm counts how often each adjacent pair of 符元 appears in the 訓練 corpus.

3. 3

   Merge most frequent pair

   The most frequent pair is merged into a single new 符元 and added to the vocabulary.

4. 4

   Repeat

   Steps 2-3 repeat until the vocabulary reaches a target size (typically 30K-100K 符元).

The result is a vocabulary where common words are single 符元 ("the" = 1 符元), while rare words are split into subwords ("cybersecurity" might become "cyber" + "安全").

# Inspecting 分詞 with tiktoken (OpenAI)
import tiktoken
 
enc = tiktoken.encoding_for_model("gpt-4")
text = "Ignore previous instructions"
符元 = enc.encode(text)
print(f"Tokens: {符元}")
print(f"Decoded: {[enc.decode([t]) for t in 符元]}")
# 輸出: ['Ignore', ' previous', ' instructions']

SentencePiece and Other Tokenizers

SentencePiece differs from BPE in a key way: it treats the 輸入 as a raw byte stream without pre-分詞 on whitespace. 這意味著:

Token Boundary 利用

The most direct 分詞 attack exploits the fact that 安全 filters and 模型 may tokenize text differently.

Splitting Dangerous Keywords

If a content filter checks for the word "bomb" but the 分詞器 splits it as "bo" + "mb" due to unusual surrounding characters, the filter may miss it while 模型 still understands the intent.

import tiktoken
enc = tiktoken.encoding_for_model("gpt-4")
 
# Normal 分詞
print([enc.decode([t]) for t in enc.encode("bomb")])
# ['bomb'] - single 符元, easy to filter
 
# With unusual formatting that might split 符元
print([enc.decode([t]) for t in enc.encode("b\u200bomb")])
# May split differently due to zero-width character

Inserting Invisible Characters

Unicode provides numerous characters that are invisible or near-invisible but affect 分詞:

Homoglyph 攻擊

Homoglyphs are visually identical characters from different Unicode blocks. 模型's 分詞器 processes them differently even though they look the same to humans:

# These look identical but tokenize differently
ascii_a = "a"           # U+0061 Latin Small Letter A
cyrillic_a = "а"        # U+0430 Cyrillic Small Letter A
 
# To a human reader: "admin" vs "аdmin" look the same
# To a 分詞器: completely different 符元 sequences

import unicodedata
 
CONFUSABLES = {
    'a': ['а', 'ɑ', 'α'],  # Cyrillic, Latin alpha, Greek alpha
    'e': ['е', 'ё'],        # Cyrillic
    'o': ['о', 'ο', '٥'],   # Cyrillic, Greek, Arabic
    'p': ['р', 'ρ'],        # Cyrillic, Greek
    'c': ['с', 'ϲ'],        # Cyrillic, Greek
    'x': ['х', 'χ'],        # Cyrillic, Greek
}
 
def detect_homoglyphs(text: str) -> list[dict]:
    """Detect potential homoglyph characters in text."""
    findings = []
    for i, char in enumerate(text):
        script = unicodedata.name(char, "UNKNOWN")
        if any(char in alts for alts in CONFUSABLES.values()):
            findings.append({
                "position": i,
                "char": char,
                "unicode": f"U+{ord(char):04X}",
                "name": script,
            })
    return findings
 
# Usage
text = "аdmin access granted"  # First 'a' is Cyrillic
print(detect_homoglyphs(text))

Encoding Tricks That 利用 Tokenizers

Beyond homoglyphs, several encoding-level tricks 利用 分詞器 behavior:

Base64 and Encoding Obfuscation

Some models 理解 base64-encoded text. If content filters don't decode before filtering:

# Instead of: "How to hack a system"
# Encode as: "SG93IHRvIGhhY2sgYSBzeXN0ZW0="
# 模型 may decode and comply while filters miss the content

Token Smuggling via Markdown/Code Blocks

Tokenizers often handle code blocks and markdown formatting differently from plain text. Wrapping 對抗性 content in code fences or specific formatting can alter 分詞 in ways that bypass filters.

Whitespace and Control Character Manipulation

# Various whitespace characters that affect 分詞
normal_space = " "          # U+0020
non_breaking = "\u00A0"     # U+00A0
em_space = "\u2003"         # U+2003
figure_space = "\u2007"     # U+2007
 
# Each produces different 分詞 of the same visual text
for space_char in [normal_space, non_breaking, em_space, figure_space]:
    text = f"ignore{space_char}instructions"
    符元 = enc.encode(text)
    print(f"Space U+{ord(space_char):04X}: {len(符元)} 符元")

Practical Tokenizer Analysis

When assessing an AI system, always analyze the 分詞器:

1. 1

   識別 the 分詞器

   Determine which 分詞器 the target model uses (tiktoken, SentencePiece, etc.) and its vocabulary size.

2. 2

   測試 keyword splitting

   Check how 安全-critical keywords are tokenized. Do they appear as single 符元 or get split?

3. 3

   測試 Unicode handling

   Probe with homoglyphs, zero-width characters, and unusual Unicode to find 分詞 inconsistencies.

4. 4

   Compare filter and model 分詞

   If possible, determine whether content filters and 模型 use the same 分詞器. Mismatches are exploitable.

Try It Yourself

相關主題

• How LLMs Work: A 紅隊員's Guide — the broader context of LLM internals
• Transformer Architecture for Attackers — what happens after 分詞
• Embeddings & Vector Spaces for Red Teamers — how 符元 become vectors
• 對抗性 ML: Core Concepts — the broader 對抗性 attack taxonomy

參考文獻

• "Neural Machine Translation of Rare Words with Subword Units" - Sennrich et al. (2016) - The paper introducing Byte Pair Encoding (BPE) for neural machine translation, now the basis of most LLM tokenizers
• "SentencePiece: A simple and language independent subword 分詞器" - Kudo & Richardson, Google (2018) - The SentencePiece 分詞器 used by Llama, T5, and other major models
• "Unicode 安全 Considerations" - Unicode Consortium (2023) - Official documentation on Unicode confusable characters and 安全 implications
• "Tokenizer-Level 對抗性 攻擊 on Large Language Models" - Various researchers (2024) - Research demonstrating how 分詞器 behavior creates exploitable attack surfaces in LLM systems