OneEnv
← All posts🧯 Part 2 of 21

Environment Variable Chaos at 40 Services

ShareLinkedInX

Take a workspace with 30 active services — not a hypothetical scale, just past the point where anyone can hold the dependency graph in their head. One of them, PAYMENT_API_VERSION, is read by every service that calls the payment provider directly: payment-service itself, plus order-service, billing-service, subscription-service, refund-service, invoice-service, and a dozen others that build request payloads against it. The platform team wants to move it from v1 to v2. This post walks that one change through, honestly, using the mechanics OneEnv actually runs — cross-referenced in The Env Management Engine and Branches & Merges.

The version without governance

This is the failure mode that motivated the whole product (see Why We Built This), played out at 40-repo scale instead of 22:

3:00pmSomeone edits PAYMENT_API_URL directly in three repos' .env files and opens PRs.
3:02pmpayment-service deploys clean.
3:05pmorder-service starts erroring — nobody grepped for the variable there before merging.
3:10pmSomeone finds a fourth repo, invoice-service, still hardcoded to v1 — a fifth is discovered an hour later.
3:20pmThe billing team says nobody told them this was happening.
3:40pmProduction gets rolled back. Nobody can say, from memory, which of the 30 services were ever actually confirmed compatible with v2.

None of this happened because the value was stored in the wrong place. It happened because nothing asked which services depend on the value before it changed, and nothing recorded who signed off. That's a governance gap, not a storage gap — which is why swapping .env files for a nicer key/value store doesn't close it on its own.

Why the usual answers don't fully close it

Each of these solves a real problem. None of them is a dependency-and-approval system, because none of them knows which services read a given value.

ToolWhat it actually solves
.env filesLocal dev convenience. No cross-repo visibility, no notion that the same key exists 30 times.
GitVersion history for one repository at a time. A shared variable's history is scattered across every repo that copied it in.
Vault / secrets managersSecure storage and access control for secret material. Doesn't know which services consume a given secret or whether they've adopted a new value.
Kubernetes ConfigMaps / SecretsRuntime delivery into a pod. Says nothing about who approved the value going in, or who else depends on it.

OneEnv isn't a replacement for any of these — it doesn't inject values into a running pod, and it isn't where you'd store a TLS private key. It sits one layer up: deciding, recording, and gating what the value should be and who has agreed to it, before that value ever reaches a .env file, a ConfigMap, or a Vault-backed secret. See Your Secrets Are Now Encrypted at Rest for how it handles values that are themselves secret.

The same change, staged and reviewed

Editing an EnvMaster's value is a branch_change, not a direct write — nothing touches the shared row until the branch merges (Branches & Merges covers the mechanics). Submitting creates one review notification, whole-branch, Owner/Admin, PR-shaped — not 30 separate approvals for one edit.

Stage on a branch
PAYMENT_API_VERSION: v1 → v2
→
Submit
one branch_review notification
→
Merge
value updates, fan-out begins
→
Scored & notified
every currently-inheriting service
→
Each service decides
approve / reject, independently

The merge doesn't guess at blast radius — it scores it

The notification that fans out at merge carries a computed impact score (0–100) and a risk level, weighted 40% how many services this reaches, 25% whether it's a genuinely breaking change versus a drift, 20% whether it touches a secret or a structural delete, and 15% how much of the reach has nobody able to act on it. For a variable read by roughly half the workspace, that's enough on its own to push the risk level to high before a single service has even opened the notification — detailed in Scores, History & Reports.

Not every service has to move at once

This is the part a naive "sync everything" approach gets wrong: the merge doesn't force every service onto v2 simultaneously. Export resolves each service's value from its own inheritance row, which only advances once that service approves. A service mid-review is still served v1 — not broken, not blocked, just not yet moved. A snapshot partway through this rollout looks like this:

api-gateway
approved → v2
auth-service
approved → v2
order-service
approved → v2
payment-service
approved → v2
cart-service
approved → v2
catalog-service
approved → v2
pricing-service
approved → v2
search-service
approved → v2
notification-service
approved → v2
shipping-service
approved → v2
review-service
approved → v2
recommendation-service
approved → v2
webhook-service
approved → v2
admin-service
approved → v2
billing-service
pending review
reconciliation-service
pending review
invoice-service
rejected — stays on v1
…and 22 more
27 of 30 active services approved

invoice-service rejecting isn't a stuck rollout — it's a legacy integration that isn't ready for v2 yet, explicitly recorded as such, still correctly serving v1 from its own last-synced value. The other 27 already moved. Nobody has to remember which repos still need a follow-up PR — the notification stays open against exactly the services that haven't actioned it, and the workspace's audit trail (History) has the full record of who approved what, and when, whenever someone asks.

What this doesn't claim

This is a walkthrough of real mechanics, not a customer story — there's no production incident behind it, and the 30-service topology above is illustrative, not a deployed workspace. OneEnv also isn't the thing that ends up injecting PAYMENT_API_VERSION into a running container; that's still whatever your export target already does — a .env file pulled at build time, a CI step calling the export API with a scoped token (Export & Publish, In Depth), or a ConfigMap generated from it. What changes is everything upstream of that: whether the value that lands there was reviewed, who it notified, and whether the service actually agreed to it before it started being served.

ShareLinkedInX