pii redaction · architecture · llm security · tokenization · concepts

Reversible PII redaction: how redact-then-restore works

How typed tokens and a retained vault support reply restoration, with practical checks for expired mappings, JSON output and tool authorization.

Godfrey Lebo Founder, Sether July 9, 2026 · updated October 17, 2026 · 8 min read

Sether™ / Field notes pii redaction A stand-in goes out. Originals return. Keep the mapping under application control.

Why does one-way masking break chat apps?

Classic redaction is one-way: find the email, replace it with [REDACTED], done. That is correct for logs and documents, where nobody needs the value back.

SetherThe idea, illustrated Your map. Your side.

A placeholder such as [REDACTED] removes detail, which is often the right goal. But a draft addressed to a customer may need an unambiguous reference to the value that your app will insert later. Typed, distinct tokens let the model refer to those fields without requiring the original strings.

Think of a coat check: the ticket refers to something held elsewhere. The analogy has a limit. A story about a distinctive coat may reveal its owner even without the ticket mapping. Likewise, tokenizing an email does not anonymize the rest of a customer’s message.

What makes a good redaction token?

Sether tokens look like <EMAIL_uuid>: a type prefix plus a unique identifier. Both parts do work.

• The type prefix tells the model what kind of value sits there. It can still say "I will send the invoice to <EMAIL_uuid>" and reason about it as an email address.
• The identifier is not an encoding of the original string. Resolving it uses the vault entry. Protect both the vault and the surrounding content; context can still reveal identity.
• A fresh token for every match means repeated appearances of sarah@acme.com can produce different tokens. Each retained entry can restore to the same address. The model cannot infer equality from those token strings alone, although the surrounding text may connect them.

Where do the real values live?

In the vault: a key-value store mapping tokens to original values. Sether ships MemoryVault, an in-process LRU that holds 10,000 entries with a 1-hour TTL.

The vault interface is pluggable. Different processes can redact and restore through a compatible mapping store you implement, provided authorized entries remain available. The TypeScript Vault lookup contract is synchronous: an ordinary asynchronous Redis client is not a drop-in implementation, because restore cannot await per-token lookups. Design the storage boundary explicitly, including isolation, availability and retention.

Why must both directions share the mapping?

The default instance owns a MemoryVault. Redaction writes a mapping there and restoration reads it back. Reusing that instance is the simplest way to keep both directions connected.

A second instance with its own default vault has no access to the first instance’s entries. By default, unresolved tokens stay in the output; strict restore can reject them in supported releases. A second instance configured with the same vault is different: it can resolve mappings that are still available and authorized for that conversation.

How does this survive a multi-turn conversation?

The redacted history carries the conversation. Turn one redacts sarah@acme.com to a token. If you keep the redacted messages as the history you resend, then on turn three, when the user says "actually send it to that same address", the history still holds that exact token, so the model refers back to it and the reply restores to the real address. If you re-redact the plain history on every turn instead, each pass mints new tokens, so one address can appear as several different tokens in the same prompt. Replies still restore correctly, but the model has less to go on.

Keeping redacted history avoids repeatedly sending originals from earlier turns, but new messages and unmatched context still need review. Set the vault lifetime for your actual session behavior. The one-hour default is a configuration choice, not a guarantee that a delayed reply or a resumed conversation will still have its mapping.

What are the security properties?

• Matched originals are withheld from that edited request. Unmatched values, attachments and identifying context are outside that statement.
• A token is a reference, not anonymization proof. Its random identifier does not encode the original, but context or other data may identify the person.
• The vault is sensitive application data. Restrict access, isolate tenants or conversations, and avoid copying mappings into ordinary logs.
• Retention must cover the intended round trip. Expiry and eviction limit stored mappings but can also prevent restoration; test both outcomes.

To be precise about the claim: this is a technical measure that helps with data minimisation. It shrinks what leaves your boundary; it does not remove the need for processor agreements or a lawful basis, which we walk through in the GDPR post.

Restoration is part of the full MIT SDK and remains a Free essential in the official tools. Optional Pro workflows do not sell your originals back to you: tool restoration still depends on having the matching session mapping. Pricing explains the separate tool pilot.

When is reversibility the wrong choice?

Reversibility exists to serve a live user who needs the real value back in the reply. If no user is waiting, keeping a path back to the data is a liability, not a feature.

Choose retention by destination and purpose. A log scrubber may never need a reverse mapping; a live customer reply may. Do not keep originals in a vault by default if the workflow has no reason to recover them.

The round trip in code

In TypeScript, redact and restore are stream transforms on one instance. The full tutorial, including the drop-in OpenAI wrapper and SSE streaming, is here.

import { Sether } from '@raeven-co/sether';
import { Readable } from 'node:stream';

const sether = new Sether(); // holds the vault

// Outgoing: values become tokens
const safeForLLM = Readable.from([userMessage]).pipe(sether.redact());

// Incoming: tokens become values again, same instance
const safeForUser = llmResponse.pipe(sether.restore());

In Python, the same design with sync calls (async and streaming also exist; the Python walkthrough covers them).

from sether import Sether

sether = Sether()  # holds the vault

safe = sether.redact_sync(user_message)   # values -> tokens
reply = call_llm(safe)
restored = sether.restore_sync(reply)     # tokens -> values

One subtlety worth knowing before you ship: restoring a streamed reply means tokens can split across chunks, and a naive restore misses them. Sether holds back a small window to stay boundary-safe; the details are in the chunk-boundary bug post.

Written by Godfrey Lebo

Founder of Sether. Ships production software for a living and reads every comment below. More about the team.