Integrating smoltext Into Your Write Path Without Breaking Reads
A practical migration guide: add compression to new writes while keeping legacy uncompressed records readable throughout the rollout.
The Migration Challenge
Adding compression to a live system is not just calling an API. You have existing uncompressed records that must stay readable while new records arrive compressed. Done carelessly, you break reads. Done well, the migration is invisible.
Step One: A Version Marker
The foundation is a one-byte version marker on every record. Reserve a byte (or a header field) to indicate the encoding:
0: legacy uncompressed1: smoltext compressed
Your read path inspects this byte and dispatches accordingly. This lets compressed and legacy records coexist indefinitely.
Step Two: Update the Read Path First
Always teach the system to read the new format before you start writing it. Deploy a read path that handles both markers:
record = fetch(key)
if record.marker == 1:
data = smoltext_decompress(record.body)
else:
data = record.body
Ship this everywhere and confirm it is stable before any compressed record exists.
Step Three: Flip Writes On
Once every reader understands the compressed format, enable compression on the write path:
compressed = POST https://api.smoltext.sprapp.com/v1/compress
store(key, marker=1, body=compressed)
New records are now compact. Old records remain readable via the legacy branch.
Step Four: Optional Backfill
You can leave legacy records as-is and let them age out, or backfill them by reading, recompressing, and rewriting. Backfilling captures storage savings on the existing corpus but is optional — the version marker means there is no hard deadline.
Verify the Round Trip
During rollout, assert decompress(compress(x)) == x on a sample of real records before trusting the write path. smoltext is lossless, but verifying on your actual data confirms the integration end-to-end.
Watch for the Bad-Fit Records
Some records will not compress well — high-entropy fields, already-tiny payloads. A robust write path checks the returned size and stores the record uncompressed (marker 0) when compression did not help. The API returns both sizes, so this check is trivial and prevents ever growing a record.
Rollback Safety
Because both formats coexist, rollback is safe at every step. If you need to disable compression, flip writes back to marker 0; existing compressed records still decompress fine on read. There is no irreversible step.
The End State
After rollout, your hot write paths emit compact records, your reads transparently handle both formats, and your per-byte costs drop in proportion to the compression ratio — all without a flag-day migration or a read outage.