Axoniq Conference 2026 · Utrecht

Rebuilding the Airplane in Flight

From browser CRUD to Event Sourcing — without landing the plane.

Richard Bouška · CTO, ASSIST

Ondřej Halata · Principal Engineer, ASSIST

CQRS · DDD · Event Sourcing

captain speakingLadies and gentlemen, this is your captain speaking. Welcome aboard. Please be advised that this aircraft is being rebuilt during the flight. Sit back and relax — our cabin crew is ready to ASSIST^™ you. Cabin crew, arm doors and cross-check.

Ladies and gentlemen, this is your captain speaking. Welcome aboard. Please be advised that this aircraft is being rebuilt during the flight. Sit back and relax — our cabin crew is ready to ASSIST you. Cabin crew, arm doors and cross-check.

This is the story of a business-critical system we rebuilt while real customers kept using it every day.

Our users were on board the whole time — and unaware that the airplane was being rebuilt around them.

And there was no budget for the rebuild, and no stop. It happened along the way — almost by the way — while a continuous stream of new business features kept coming in. The passengers didn’t notice, and they must not notice.

It is not a clean before-and-after story. It is about the in-between architectures — every one of which had to run in production.

The usual story

The terrifying part is getting there.

No maintenance window long enough. No big-bang rewrite. No moment to stop the world.

in production · every day · today

events / day

commands / day

queries / day

services

instances

processing groups

business users

customers a day

Twelve intermediate architectures. Every one of them had to fly.

The browser was the backend.

illustrative · the browser, orchestratinglist = await get( "/appointments") for (a of list) a.customer = await get( `/sessions/${a.id}`) await post("/session") await post("/appointment") // …and if the second one fails?

The browser wasn’t really a frontend. It had become the integration platform.

the NO_TRANSACTION dragon

One business action, several CRUD calls — nothing held them together.

Phase 0 · Control the chaos

Browser-as-Backend

Starting state · before the web BFF (2021)

captain speakingLadies and gentlemen, the CRUD swamp is now being left behind. Some turbulence is to be expected — please keep your seatbelts fastened while the seatbelt sign is illuminated. Should help be needed, the crew is standing by to ASSIST^™ you.

Ladies and gentlemen, the CRUD swamp is now being left behind. Some turbulence is to be expected — please keep your seatbelts fastened while the seatbelt sign is illuminated.

Phase zero. Our TypeScript frontend called many backend endpoints directly. Orchestration, state management and some business rules lived in the browser.

The browser wasn’t really a frontend any more — it had become our integration platform. [Next: a list of ten]

And it showed. A list of ten appointments was one call for the list and one more for every row — the user watched it fill in. [Next: one booking]

Worse: one business action, like booking an appointment, was several CRUD calls with no transaction around them. If the third call failed, the first two stayed. Half a booking. [Next: far away]

And every one of those calls went a long way. From the user's browser, across the internet, through APIGEE on GCP and the ALB on AWS, to the service — every hop a chance to stall, or to lose the call. Eleven of them for one list, three for one booking: the risk added up with every call.

Evidence gap: the frontend history is not in the backend repository. Bring an old screenshot, diagram or frontend commit.

Put a backend in the middle.

Mar 2022 · an entire service methodreturn accountsConnector .postPreRegister(…);

It wasn’t the new architecture yet. It was the place where the new architecture could start.

BFF: not “best friends forever”. Backend for Frontend — a protocol stereotype service.

the NO_TRANSACTION dragon

The same CRUD calls through the new door: still no transaction, and the third can still get lost.

Phase 1 · Control the chaos

Gateway BFF

Git · Nov 2021 BFF initial commit (14fe5c3de) · Mar 2022 “BFF for WebFE” (633555af7)

First move: put a backend in the middle — at first mostly a gateway. Many early BFF endpoints were thin forwarding layers — this method body is the whole method.

The value wasn’t what the BFF did, but where it stood: a controlled place where orchestration could move out of the browser, one endpoint at a time. [Next: one door]

Endpoint by endpoint, the browser’s calls moved behind it — one door, guarding everything that came in. Every one of them still crossed the internet — that never changes — but now through one service: one call per endpoint, where a single call to the BFF could carry the whole action. And behind the door, the BFF still went the long way round to the services: out through the proxy, APIGEE and back in through the ALB — more on that in phase 4. [Next: the same conversation]

But look at what came through the door: the same CRUD calls, still orchestrated by the browser. The same booking — three POSTs in, three POSTs out — and the third can still get lost. Robustness barely improved. What we gained was one place to change things.

Precision: some early endpoints already orchestrated (Dec 2021 appointment list). Say “many endpoints were thin”, not “pure proxy”.

CRUD starts producing events.

REST → service → JPA commit @PostPersist → CREATED @PostUpdate → MODIFIED @PostRemove → DELETED afterCommit() → SNS

First our events described what changed. Later they started describing why.

the OUT_OF_ORDER dragon

Ordering mattered within two weeks: FIFO message groups, May 2022.

Phase 2 · Control the chaos

CRUD + Change Events

Git · May 2022 · f4ada88e0 · “Send SNS notifications when an appointment is created/updated/deleted”

We built projections before we called them projections.

before fan-out at request time: config → resources → appointments → a customer session per appointment after one query: /projections/appointments/search

Read models before Event Sourcing. CQRS doesn’t require Event Sourcing — the write side was still CRUD.

the READ_MODEL dragon

Read models bring their own problems: projection deadlocks, May 2024.

Phase 3 · Local state & messaging

Read Models on the Side

Git · Dec 2022 – Apr 2023 · 0cea060ee, be704bc6e, 4b11db078

captain speakingLadies and gentlemen, the chaos has been left behind and cruising altitude is being reached. Local state and messaging can be seen on your left. Cabin crew, you are released to ASSIST^™ with the in-flight service.

Ladies and gentlemen, the chaos has been left behind and cruising altitude is being reached. Local state and messaging can be seen on your left. Cabin crew, you are released for the in-flight service.

Once changes were flowing over SNS/SQS, we used them to maintain local read models — data in the shape the screen needed, before we redesigned the write model at all. [Next: before]

Before: the appointment list was composed at request time with a network fan-out. Remember the user waiting on slide 03? Still waiting — just one floor further in. After: one query against a projection. [Next: after — and its dragon]

Strictly speaking that is already CQRS — with a CRUD write side. We built projections before we called them projections — or rather: we probably called it a projection already, though in reality it was more like a cache.

And read models bring their own dragons: later on, two updates to the appointment-list projection were waiting on each other — deadlocks, fixed in May 2024.

Precision: the fan-out and the projection search are related examples, not proof that this exact method was replaced. No measured latency numbers in the evidence.

Stop going outside to talk inside.

Apr 2023 · BFF configurationcustomer sessions APIGEE /commerce/v1 → /customer-sessions/api/v1

Still REST. Just no longer through the front door.

API management stays the external boundary — not the internal service bus.

the DETOUR dragon

Out and back in, just to talk to a neighbour: proxy, APIGEE, ALB — AWS to GCP and back.

Phase 4 · Local state & messaging

In-Cluster Services

Git · Jul 2022 d9819ada9 “switch connector from apigee…” · Apr 2023 63304cff4

A confession: the pictures so far were tidier than reality. APIGEE stood in front of every service. A call from one of our services to another left through the corporate HTTP proxy, went out to APIGEE, and came back in through the ALB, the ingress load balancer. And APIGEE ran on GCP, while we — and most of the other systems — ran on AWS: every one of those detours went from one cloud to the other and back. Four hops — out via the proxy, in via the ALB — to talk to the neighbour next door. [Next: the long way round]

So we sorted them. Internal: we own the source code and run the service. External: everything else. Internal calls now stay inside the cluster, through an internal ALB — two hops instead of four: better, not perfect. APIGEE stays the external boundary — finally just the bouncer at the door. In network speak: east–west traffic stays inside, only north–south goes through APIGEE — east–west as the network people say, not as drawn here. [Next: sorted]

Still REST at this point — just no longer through the front door. And not a big bang: one connection at a time, until it was mostly done.

Order: this started (Jul 2022) before the read models (Dec 2022).

We adopted Axon before we adopted Event Sourcing.

JPA afterCommit → Axon command → aggregate → event ↘ SNS, still running each path behind its own feature flag

Yes, there were aggregates. Their replay restored exactly one field: the ID.

the FILTER_CONFIG dragon

Every release shipped new filter config; every rollback had to undo it. We never found a forward/backward-compatible way.

Phase 5 · Local state & messaging

Axon Message Bus

Git · Jul – Aug 2023 · d45e981a1, fe6355276, d67519a83 “Replace SNS/SQS with Axon”

This is the surprising one: we adopted Axon before we adopted Event Sourcing.

So far we only spoke about three services: appointments, interventions and customer sessions. There were many more. [Next: the messaging view]

SNS/SQS was never one backbone. Every service had its own queue, with an SNS subscription filter policy: JSON files deciding who heard which change. Every release needed new filters, every rollback had to put the old ones back, and dev and devops had to coordinate each step. We never found a forward/backward-compatible way. [Next: the switchover]

And we did not switch in one go. Each event stream had its own feature flag: checklist events moved to Axon first, then appointments and resources. For a while part of the events travelled through SNS and part through Axon, and the services listened on both. [Next: with Axon]

So Axon Server came in as the messaging backbone. The same JPA listener now also sent an Axon command — in parallel with SNS, each behind its own feature flag. With Axon there is no filter configuration: each service subscribes to the events it handles in its own code, so a release or a rollback carries its own subscriptions.

Technically there were aggregates. Their event-sourcing handlers restored one field: the ID. The write model still lived in JPA.

From technical APIs to domain conversations.

before PATCH key-drop document { …fields… } worse PUT { the whole record } → diff it after SetParkingAccessCode UploadKeyDropMedia RemoveKeyDropMedia where no URL — the handlers register

We stopped making services talk HTTP to each other. We started making them speak the domain language.

the GUESS_THE_INTENT dragon

A PATCH says which fields changed. A PUT overwrites the whole record: diff old and new, then guess the intent.

Phase 6 · How the system speaks

Command & Query

Git · Mar – Nov 2024 · 8ce2a442d, 22d65d389, 96967f0de, d18d9db6a

captain speakingLadies and gentlemen, for the next stretch a new language will be spoken on board: domain conversations. Please refer to the safety card in the seat pocket in front of you; translations will be provided by ASSIST^™.

Ladies and gentlemen, for the next stretch a new language will be spoken on board: domain conversations. Please refer to the safety card in the seat pocket in front of you.

With Axon messaging in place, internal REST calls started turning into commands and queries.

The interesting part isn’t the transport. A PATCH on a key-drop document became SetParkingAccessCode, UploadKeyDropMedia, RemoveKeyDropMedia. Callers started asking for an action in the domain.

Not “REST is bad” — we moved from technical endpoint coupling to explicit domain intent. [Next: before]

Before: three different things a user does — change the parking code, add a photo, remove a photo — all left as the same PATCH of a key-drop document. The PATCH at least said which fields changed — not what happened. The worst were the PUTs: the whole record overwritten, so the receiver first had to diff old and new to find the changed entries, and then deduce the intent behind them. [Next: after]

After: three commands, each with its own handler. Nothing to guess. [Next: not every swap]

Honest about it: not every swap was a new meaning. GET dealer became GetDealer — the transport changed, the question didn’t. And PlanFollowUp was a proper command whose handler still called the existing service before emitting FollowUpPlanned — a real domain event, and the start of the next story. [Next: the bonus]

And a bonus on top: location transparency. With REST, the caller had to know the URL — who it was actually calling — and the routing lived in the ALB: explicit, complex configuration, maintained outside the code. With Axon Server it is the other way round: the handlers register, and the caller just sends the command or the query. The routing follows the handlers.

Every change, told twice.

POST / PUT / PATCH / DELETE → command → domain event + CDC event, emitted explicitly JPA afterCommit → retired, service by service the same change, twice · on purpose

Every event processor moves from CDC to domain events on its own — still today.

the TOLD_TWICE dragon

150+ processing groups to migrate, one by one — still in progress.

Phase 7 · How the system speaks

Dual Stream

Git · Aug 2023 – Nov 2024 · d67519a83, f9df230d2, d18d9db6a

Moving writes to commands had a side effect. HTTP POST, PUT, PATCH and DELETE became commands — and the commands emitted domain events: FollowUpPlanned, not just “something changed”.

But the JPA listener was still there, still publishing its CDC-style events after every commit. So the event stream told every change twice: once as a CDC event, once as a domain event.

And we kept it that way — on purpose. We have more than 150 processing groups reading the CDC events, and every one of them has to be migrated. So the pairs stay, and the processors move over one by one; that work is still going on today. Unpleasant, never a blocker. [Next: one change, two events]

One change: the command’s handler emits FollowUpPlanned; its JPA save commits, and afterCommit() emits a CDC-style MODIFIED. Two events for one change. [Next: the projection copes]

The projections coped with idempotency and a sequencing policy: one appointment’s events in order, and the twin changes nothing. Not an operational risk. The cost: double the storage — still true, but storage is not the biggest limitation these days — an event store that is harder to read, and hard to explain to anyone new. [Next: twice, on purpose]

Service by service, the JPA listeners went away — one was removed in November 2024 — for cleaner code, and to cut the tricky transaction juggling that was a potential source of deadlocks. The CDC event did not go: the command now emits it explicitly, right next to the domain event. The pairs are intentional, and they stay with us while the processors migrate.

Intent goes in. State changes come out.

Feb 2024 · the notification pathprojection → query update → Axon subscription query → WebSocket BFF → browser

Finally out of the 1:1 request–response paradigm.

Progressively, everything moves to WebSocket.

the LONG_POLLING dragon

Long polling for notifications: long requests hurt Apdex, and it was ugly.

Phase 8 · How the system speaks

Split Front Door

Git · Dec 2023 24d559197 (scaffold) · Feb 2024 9f12aa68f, 611d3e3bd (subscriptions)

We introduced a separate WebSocket BFF. It replaced long polling for notifications, then took over more. Its first, immediate effect was on the notification path: telling a user about a change that happened in the background.

A screen can subscribe to its read model through an Axon subscription query and receive the initial state plus every subsequent change. [Next: before]

Before: notifications went through long polling against the original BFF. A request held open until something happened or it timed out — then again. Those long requests dragged our Apdex down. And it was ugly. [Next: the WebSocket BFF]

With the WebSocket BFF there are three channels: one in, and two back — a topic that many subscribers share, and one for the responses of request–response. A background change goes out on the topic the moment it happens, to every subscriber. The notification path came first: no more long polling. [Next: beyond 1:1]

And now we are progressively moving everything to WebSocket. We have far better control over the asynchrony: incoming and outgoing messages are fully decoupled, and we are finally out of the 1:1 request–response paradigm — 1:0, 1:1, 1:n, 0:n. We can always pick the best communication pattern between frontend and BFF. And 1:n maps cleanly onto Axon subscription queries.

Every deployable gets an explicit job.

Feb 2024 · one image, two rolesspring profile: command spring profile: query

We stopped treating every deployment unit as a unique snowflake.

Spring profiles: not elegant — just easy.

the HALF_AND_HALF dragon

HTTP and Axon, both ways, in one service: deadlocks. Duplicated services: inconsistent code.

Phase 9 · Make it explicit

Building Blocks

Git · Feb 2024 5dc5bdc20 · Apr 2024 9aaab10f8 · Oct 2025 ace89af40 · Jul 2026 0a5a8fb82

captain speakingLadies and gentlemen, our approach to Event Sourcing has been initiated. Tray tables are to be stowed and seat backs returned to the upright position — the crew will ASSIST^™ you. Cabin crew, prepare for landing.

Ladies and gentlemen, our approach to Event Sourcing has been initiated. Tray tables are to be stowed and seat backs returned to the upright position. Cabin crew, prepare for landing.

The goal was never more microservices. It was a predictable topology: services split, merged and reshaped into recurring building blocks — handle commands, answer queries with their projections, expose the application over REST, adapt an external system. [Next: in the middle]

The move from HTTP to Axon was progressive, not a big bang — so for a long time we were in the middle, and partially we still are. One service was an HTTP server and client and an Axon server and client, for commands and queries. That led to deadlocks, or near-deadlocks. [Next: split by role]

In February 2024 the same image started running in two roles — command and query — selected by Spring profile. Not because profiles are elegant: because it was easy. The separation prevented the deadlocks. [Next: less code]

We also merged services. Some came from a different origin; during the takeover we saw they simply belonged together — with duplicated, inconsistent code. And progressively we moved to a monorepo, where a service contains only its own code: less code, simpler code. [Next: explicit jobs]

Today Customer sessions, Appointments and Interventions each have a command and a query role — query and projection live together, so subscription queries work correctly. The api service is a protocol service, a sibling of the BFFs: it exposes the whole application over REST. And many more services, each with an explicit job — which made the topology understandable and observable; Axon metrics were exposed in April 2024.

The event stream becomes the source of truth.

Apr 2025 · CustomViewDefinitionAggregate// decide from reconstructed state if (command.getName() != null && !Objects.equals(name, command.getName())) { apply(mapper.toRenamedEvent(command)); } @EventSourcingHandler public void on(CustomViewDefinitionRenamed event) { this.id = event.getId(); this.name = event.getName(); }

Not only in new aggregates: fully event-sourced flows inside existing hybrid ones.

1 command · 2 aggregate · 3 domain event · 4 Axon Server · 5 event processor · 6 projection

the HAND_PATCH dragon

Projections patched by hand over the years no longer match their events.

Phase 10 · Make it explicit

Event-Sourced Core

Git · Apr 2025 c5932fced · Aug 2025 69606607e

The event stream becomes the source of truth — not merely another integration channel. [Next: the aggregate]

Newer parts became genuinely event sourced — this is the event-sourcing pattern, by the book. The aggregate rebuilds its state from its own events and decides from it. The same name, nothing happens; a new name, Renamed — and that event goes into the store, where the aggregate's state lives. [Next: replay]

Projections are outputs of event processors, so they are, in principle, correct. What hurts: someone patched projections by hand over the years. So we rebuild them: a new version — appointment_projection_v2 — built from scratch next to the old one, by replaying every event; when the replay is done, the query handlers switch from v1 to v2, and v1 is retired. Some projections are already rebuilt from the events, more and more. And a replay visits every event, so the old, legacy ones get tested too. Imports and replay fixes were still landing in 2026. [Next: one slice at a time]

How did event sourcing get into a system that already existed? Custom views were a new aggregate, event sourced from scratch. But we didn’t want event sourcing only in new aggregates. So we cut a vertical slice through existing ones: one business flow at a time — the Payments flow in CustomerSessions. That flow is fully event sourced: its state is rebuilt from the CDC events of its history, then from domain events as soon as they exist. The aggregate around it stays hybrid. And where event sourcing wasn't ready yet — the Booking flow in Appointments — an external command handler keeps the state in the database, but already emits the events. Not whole aggregates at once — one flow at a time.

You can’t replay what was never written.

projection: rows from before the events event store: starts later import batch → …Imported events // ≈ 20 million of them replay → the whole story

First we wrote the history down. Then we could replay it.

the WIPE_AND_REPLAY dragon

A replay would have quietly dropped everything older than the first event.

Phase 11 · Make it explicit

Full-History Event Store

Git · Jan 2026 · a727e8da0 · import support for pre-Axon data

The payoff

The behaviour is already there. It is the event stream.

Once meaningful state changes are first-class events, a new capability is often just a new reader.

Every event processor has its own cassette — a copy of an Axon context, at its own place. Each one replays on its own.

R: real event names from the codebase · L: their CDC twins, illustrative · the herd: illustrative

captain speakingLadies and gentlemen, our destination has been reached — but no one is permitted to disembark. Please remain seated with your seatbelt fastened: the rebuild is being continued by ASSIST^™. Cabin crew, doors remain armed.

Ladies and gentlemen, our destination has been reached — but no one is permitted to disembark. Please remain seated with your seatbelt fastened: the rebuild is being continued. Cabin crew, doors remain armed.

Here is the payoff we didn’t fully plan for. Once important state transitions are events, a lot of new capability is just a new reader of facts the system already produces.

Business statistics, user behaviour, auditing, replay and rebuild — and the next projection nobody has asked for yet.

Look closer: it’s a stereo tape — two channels. Left, the old one: the CDC-style events. Right — the right one, as in Haskell — the domain events. Since phase 7 every change is on both, on purpose.

And we have more than 150 processing groups. Every one of them has to move from left to right, one by one; some already have, many still listen on the left. That work is still going on today. Yes, it takes twice the tape — but storage is not the biggest limitation these days. [Next: own cassettes]

And one tape for everyone? Not quite: every event processor has its own cassette — its own place on the stream. The cassette is an Axon context: processors on the same context play 1:1 copies of the same tape, each at its own place; user behaviour has its own context, its own tape. Rewind one and replay it from the first event, and the others just play on.

The behaviour is already there. It is the event stream.

Reality check

It was never a straight line.

Different parts of the system moved at different speeds. The phases overlapped — the order is a story, not a schedule.

Not a project, a process: driven by the business, changing as new needs arise — never an ivory tower.

Each dot is a selected commit anchoring a phase · Git author dates, not deployment dates · phase 0 predates this repository

Closing

None of these was the target architecture. Every one of them was a production architecture.

We never migrated the application to Event Sourcing. We continuously evolved it into an event-sourced system.

Thank you.

Questions — and war stories — welcome.

Richard Bouška · Ondřej Halata — ASSIST

links & slidesassist-links.web.appscan it before the dragon skates off with it

captain speakingLadies and gentlemen, thank you for flying with us. This aircraft is being kept in service. Should you need anything, our crew will be happy to ASSIST^™ you. Cabin crew, disarm doors and cross-check.

Rebuilding the Airplane in Flight

The terrifying part is getting there.

The browser was the backend.

Put a backend in the middle.

CRUD starts producing events.

We built projections before we called them projections.

Stop going outside to talk inside.

We adopted Axon before we adopted Event Sourcing.

From technical APIs to domain conversations.

Every change, told twice.

Intent goes in. State changes come out.

Every deployable gets an explicit job.

The event stream becomes the source of truth.

You can’t replay what was never written.

The behaviour is already there. It is the event stream.

It was never a straight line.

None of these was the target architecture. Every one of them was a production architecture.

Navigation

Restart the 30-minute clock?