Case study
- Role
- Senior Software Developer - backend & infrastructure
- Organisation
- Greggs PLC
Kitchen Management System
A single kitchen-facing system that takes orders from every channel (kiosk, till, drive-through and the digital delivery platforms) and puts them in front of the people actually making the food. It runs on Linux and Kubernetes inside the shops, not only in the cloud, because a shop that loses its internet connection still has to serve customers.
01 / Overview
What I did.
I re-architected the backend and the infrastructure across both the cloud and the edge. The core of the work was re-assessing how orders move: replacing fire-and-forget cloud messaging with persistent data streaming, replacing continuous internet polling with WebSocket push and an edge sidecar, and making menu data available in-shop regardless of connection quality.
02 / Architecture
Layers listed outermost first.
How it's put together.
Two halves that have to keep working when the link between them does not. The cloud owns the record of what was ordered; the edge owns what the kitchen can see right now. Everything in between is designed around the assumption that shop connectivity is unreliable rather than treating an outage as an exception.
Ordering channels
Where orders originate.
In-store kiosk, till and drive-through alongside the digital delivery platforms. Each channel has its own shape and its own failure behaviour, so they are normalised on the way in rather than handled separately downstream.
Cloud services
The system of record.
The .NET services that accept, validate and persist orders, plus the trading backend that says whether a given shop is open and able to accept them. This layer is the authority; the edge is a cache of it.
Streaming backbone
Durable movement of order events.
Order events are published to a persistent stream rather than pushed once and forgotten. A consumer that was down when an order was placed can pick up where it left off instead of missing it, which is what took missing-order incidents down.
Edge runtime
The shop keeps working on its own.
Kubernetes on Linux inside each shop, with a sidecar holding the menu data the kitchen screens need. When the connection degrades, the screens keep rendering from local state instead of blanking.
Transport to the kitchen
Push, not poll.
WebSocket connections deliver order updates to the kitchen screens as they happen, replacing screens that repeatedly asked the cloud over the internet whether anything had changed.
Observability & test doubles
Knowing it works, before and after release.
Prometheus metrics from the edge and the cloud, and Wiremock standing in for upstream dependencies so behaviour can be exercised without a full environment.
03 / Decisions
Each of these had a credible alternative. The trade-off is stated, not hidden.
Choices, and what they cost.
Persistent streaming, not fire-and-forget messaging
- Chose
- An append-only event stream with retained offsets
- Instead of
- The existing fire-and-forget cloud messaging
- Direct HTTP calls between cloud and shop
- Why
- With fire-and-forget delivery, an order placed while a consumer was restarting or offline was simply gone, and in a kitchen a lost order is a customer standing at a counter. A retained stream lets a consumer resume from its last offset, so a restart or a network blip costs latency instead of data. This is what reduced missing orders by up to 98% across the estate.
- Trade-off
- Consumers now have to be idempotent and track their own position, and there is a broker to operate and monitor. Replay is a feature you have to design for, not one you get for free.
WebSocket push over continuous polling
- Chose
- A persistent WebSocket connection per kitchen screen
- Instead of
- Continuous HTTP polling over the internet
- Long-polling
- Why
- Every screen in every shop asking 'anything new?' on a timer produced constant internet traffic whose volume scaled with the estate and whose worst-case latency was the poll interval. Pushing on change removes the traffic that carries no news and gets the order to the kitchen as soon as it exists.
- Trade-off
- A long-lived connection is a thing that breaks. Reconnection, backoff and resynchronising missed state on reconnect are all now the client's problem, where a poll loop was trivially self-healing.
An edge sidecar holding menu data locally
- Chose
- A sidecar in the shop's cluster serving menu data from local state
- Instead of
- Reading menu data from the cloud on demand
- Baking the menu into the application image
- Why
- Menu availability was tied to connection quality, so a shop with poor internet had a degraded ordering experience for reasons that had nothing to do with that shop. Serving from local state decouples the two, and keeping it in a sidecar rather than in the image means the menu can be updated without redeploying the application.
- Trade-off
- A second copy of the menu is a second place it can be stale, so invalidation has to be explicit and observable rather than implicit.
Kubernetes on Linux at the edge
- Chose
- The same container orchestration in the shop as in the cloud
- Instead of
- Bespoke per-device deployment scripts
- Cloud-only deployment with thin in-shop clients
- Why
- One deployment model across hundreds of sites means a rollout and, more importantly, a rollback behave the same way everywhere, and health, restarts and resource limits are handled by the platform rather than by hand per shop.
- Trade-off
- Running a cluster in every shop is real operational surface: it has to be upgradable and diagnosable remotely by people who are not in the building.
04 / Stack
Grouped by the job they do rather than listed as a keyword run.
Technologies.
- Services
- Kitchen client
- Edge & infrastructure
- Delivery
- Operations
05 / Outcome
Where it landed.
Missing-order incidents fell by up to 98% across the Greggs estate, and menu data became available in-shop regardless of connection quality. The move from polling to push removed a constant stream of internet traffic that grew with every shop added.