Integrating smoltext Into Your Write Path Without Breaking Reads

The Migration Challenge

Adding compression to a live system is not just calling an API. You have existing uncompressed records that must stay readable while new records arrive compressed. Done carelessly, you break reads. Done well, the migration is invisible.

Step One: A Version Marker

The foundation is a one-byte version marker on every record. Reserve a byte (or a header field) to indicate the encoding:

• 0: legacy uncompressed
• 1: smoltext compressed

Your read path inspects this byte and dispatches accordingly. This lets compressed and legacy records coexist indefinitely.

Step Two: Update the Read Path First

Always teach the system to read the new format before you start writing it. Deploy a read path that handles both markers:

record = fetch(key)
if record.marker == 1:
    data = smoltext_decompress(record.body)
else:
    data = record.body

Ship this everywhere and confirm it is stable before any compressed record exists.

Step Three: Flip Writes On

Once every reader understands the compressed format, enable compression on the write path:

compressed = POST https://api.smoltext.sprapp.com/v1/compress
store(key, marker=1, body=compressed)

New records are now compact. Old records remain readable via the legacy branch.

Step Four: Optional Backfill

You can leave legacy records as-is and let them age out, or backfill them by reading, recompressing, and rewriting. Backfilling captures storage savings on the existing corpus but is optional — the version marker means there is no hard deadline.

Verify the Round Trip

During rollout, assert decompress(compress(x)) == x on a sample of real records before trusting the write path. smoltext is lossless, but verifying on your actual data confirms the integration end-to-end.

Watch for the Bad-Fit Records

Some records will not compress well — high-entropy fields, already-tiny payloads. A robust write path checks the returned size and stores the record uncompressed (marker 0) when compression did not help. The API returns both sizes, so this check is trivial and prevents ever growing a record.

Rollback Safety

Because both formats coexist, rollback is safe at every step. If you need to disable compression, flip writes back to marker 0; existing compressed records still decompress fine on read. There is no irreversible step.

The End State

After rollout, your hot write paths emit compact records, your reads transparently handle both formats, and your per-byte costs drop in proportion to the compression ratio — all without a flag-day migration or a read outage.