# The chunk-boundary bug: why streaming PII redaction is hard

> Scanning streaming chunks independently can miss a value split across them: "sarah@ac" followed by "me.com" is the same email after joining. A hold-back buffer gives detectors surrounding text to work with. Sether uses a configurable distance and bounded buffering safeguards; supported match lengths, long inputs and framing still need explicit tests.

Published 2026-07-09, updated 2026-10-17 by Godfrey Lebo.

## TL;DR

- A per-chunk regex can miss a value that only becomes recognisable after adjacent chunks are joined.
- A realistic test deliberately splits inside values, rather than relying on whichever boundaries a live demo happens to produce.
- Property-based partition tests exercise many split positions and retain failing fixtures; compare replaced spans and restoration rather than fresh random identifiers.
- Sether uses safeDistanceBytes (default 256) plus bounded buffering safeguards. A bounded implementation needs limits and tests for unusually long or custom matches.
- Test each enabled detector, actual transport framing, end-of-stream behavior and oversized inputs before relying on a streaming integration.

## How can you reproduce the chunk-boundary bug?

The simplest reproduction needs no live model. Start with text containing an email, split the address into two chunks, and compare a per-chunk detector with one that preserves context. The example isolates a transport-boundary failure from model behavior.

A naive streaming implementation takes each chunk, runs detectors on it, replaces matches and emits the result. It can pass ordinary happy-path fixtures whenever each sensitive value happens to arrive whole. Deliberately splitting the value exposes what those fixtures did not exercise.

Property-based partition tests can explore this systematically: generate many partitions of the same PII-bearing text and compare behavior. With random token identifiers, compare equivalent replaced spans and restoration rather than literal equality between fresh UUIDs.

In the email example, neither isolated half is a complete address. Joining the emitted chunks still reconstructs the original, so the user-visible result leaks the value even though each individual detector call returned no match.

## Why does per-chunk redaction fail?

Here is the naive version, simplified. It looks reasonable until an input deliberately splits in the middle of a value:

```ts
// Looks right. Is not.
async function* redactStream(chunks: AsyncIterable<string>) {
  for await (const chunk of chunks) {
    // Run detectors on this chunk in isolation
    yield redactText(chunk); // regex + validation on chunk only
  }
}
```

The problem is that the stream does not respect your detectors. Chunk boundaries land wherever the network and the provider's tokenizer put them. So "sarah@acme.com" can arrive as "sarah@ac" followed by "me.com".

Run an email regex on "sarah@ac" and it does not match, there is no dot-something after the domain start. Run it on "me.com" and it does not match either, there is no @ before it. Two innocent halves, one leaked email.

| Chunk 1 | Chunk 2 | Naive per-chunk result | Correct result |
| --- | --- | --- | --- |
| sarah@ac | me.com | sarah@acme.com leaks | <EMAIL_1> |
| call +234 80 | 31 234 5678 | phone number leaks | <PHONE_1> |
| 4111 1111 11 | 11 1111 | card number leaks | <CARD_1> |
| sk-proj-Ab | Cd3fGh... | API key leaks | <OPENAI_KEY_1> |

This is a design failure a per-chunk scanner can share, rather than a problem unique to one regex. Unless the implementation preserves context between chunks, the detector never gets a chance to inspect the joined value.

## Why can a demo miss it?

A short demo may deliver a whole email in one chunk. That is a useful happy-path check, but it exercises one partition of the text. The boundary case needs a deliberate fixture.

Chunk boundaries depend on token lengths, network buffering, and provider behavior, none of which you control. A redactor can run cleanly for a long time and then split a value at exactly the wrong byte. It is not flaky in a way you can see; it is correct-looking with a hole in it.

A live demonstration answers “did this request work?” A partition test asks a different question: “what happens when the same input arrives in smaller pieces?” Both are useful, but the second is what exposes the missing context.

## How does the hold-back window fix it?

The fix is to preserve context between chunks. In the current Node redactor, the option named `safeDistanceBytes` defaults to 256, but the implementation uses JavaScript string lengths and slices: UTF-16 code units, not a literal UTF-8 byte bound. Additional rules retain long non-whitespace runs, with a maximum-buffer guard of `max(safeDistanceBytes * 4, 8192)`. Values beyond that guard can retain residual risk; test encoding and input limits as well as the nominal setting.

So when "sarah@ac" arrives, the tail of it stays in the window instead of going out the door. When "me.com" arrives, the detector sees the joined text, matches the full email, and emits `<EMAIL_1>`. The reader downstream never knows the chunks were reassembled.

Buffering everything is an option when the application can wait for the full text and afford its memory cost. A bounded buffer preserves incremental output, but its safety depends on detector behavior and input limits. Test the longest values you permit, including custom secrets, rather than assuming every built-in or future detector fits inside the configured hold-back distance.

```ts
import { Sether } from '@raeven-co/sether';

const sether = new Sether();

// Node streams: redact on the way out, restore on the way back
prompt.pipe(sether.redact());       // bounded context; safeDistanceBytes defaults to 256
reply.pipe(sether.restore());       // restore recognised tokens with the matching vault

// SSE responses have their own helpers that respect event framing:
// createSSERedactStream / createSSERestoreStream
```

## How does a property-based test lock it down?

The test that caught the bug is now the test that guards the fix. The idea fits in a few lines of pseudocode:

```ts
// Property: redaction output must not depend on chunking.
// Token ids are random in production, so the test seeds them.
property('chunking-invariant', (text: PiiBearingText, seed: number) => {
  const expected = redactSync(text, { detectors, vault, uuid: counter() }); // whole text at once

  const chunks = randomPartition(text, seed); // split anywhere,
  // including mid-email, mid-card, mid-key

  const actual = collect(streamRedact(chunks, { uuid: counter() }));

  assert.equal(actual, expected);
});
```

Generate texts with known matches, partition them at varied positions, and compare detected/replaced spans with the whole-text path. When token identifiers are random, normalize identifiers or inject deterministic ones in a test fixture; also check that restoration with each run’s own vault recovers the input. Literal UUID equality is not the property being promised.

That last part matters. When the invariant fails, you do not get "something is wrong somewhere", you get "splitting at byte 14 inside this email leaks it". The bug that would have been invisible in production is a one-line reproduction in CI.

Add two layers of fixtures: raw text chunks and the HTTP/SSE envelopes that carry them. A text transform can pass its tests while an adapter drops a final fragment, joins events incorrectly or restores a nested JSON document as plain text. For a tool call, validate the assembled arguments and authorization separately from restoring its values.

## How do you test your own implementation?

If you have a streaming redactor in production, home-grown or vendored, here is the five-minute check I would run first, and the fuller list after it.

1. **The five-minute check.** Feed "my email is sarah@ac" then "me.com" as two chunks. Inspect the joined redacted output for the original address.
2. **Split every enabled detector type.** Include phone numbers, cards, API keys and long values, not only email.
3. **Vary byte boundaries and Unicode.** Exercise multibyte characters, one-byte chunks, line endings and stream completion with a possible match still held.
4. **Compare behavior, not fresh UUID strings.** Normalize generated token IDs and verify restoration with the corresponding vault. Keep each failing partition as a regression fixture.
5. **Exercise limits and failure paths.** Include long unbroken values beyond the buffer guard, custom detector widths, cancellation and upstream errors. Decide whether your application should buffer fully, reject oversized input or accept documented residual risk.

The repository’s [historical migration notes](https://github.com/raeven-co/sether/blob/main/CHANGELOG.md) document a chunk-boundary defect and a property-tested correction. They do not establish that every implementation or detector is safe for every partition. Keep the technical lesson reproducible: retain failing fixtures, exercise current limits and validate the adapter you actually deploy.

For the wider context on where streaming redaction fits in an LLM data-protection setup, start with the [complete guide to PII redaction for LLM apps](/blog/pii-redaction-for-llm-apps-complete-guide). And if you are wiring this into a TypeScript service today, the [step-by-step OpenAI walkthrough](/blog/redact-pii-before-openai-typescript) covers the full redact-then-restore round trip.

## Frequently asked questions

### What does the default safeDistanceBytes setting of 256 mean?

256 is the default safeDistanceBytes setting. Despite the name, the current Node implementation indexes JavaScript strings in UTF-16 code units rather than imposing a literal UTF-8 byte bound. Long-run buffering also has a maximum guard. Test Unicode, long values, custom detectors and the actual decoder/transport path.

### Does the hold-back window add noticeable latency?

It delays some emission and can affect time to first output. The effect depends on chunk sizes, input shape, detector work and configured distance. Measure end-to-end latency on your transport; there is no fixed “one small chunk” or user-perception guarantee.

### Does this apply to SSE streams too?

Yes, and SSE adds framing to the problem. Test the framing your application emits and the helper you use. Restoring a JSON document assembled inside streamed delta.content fragments requires collecting that document and using JSON-aware restore; see the [0.8.0 limits](/blog/sether-0-8-0-json-safe-restore).

### Could I just buffer the whole response and redact once?

You can scan a fully buffered response once when its size and latency are acceptable. That removes this particular chunk-context issue, but it does not make detection perfect. Bounded streaming trades memory and delay against documented match and buffer limits.


Canonical: https://sether.ai/blog/streaming-llm-redaction-chunk-boundary-bug