Andrew Jones
Feb 11, 2022

--

Hey Rachid,

You're right, a lot of this does push responsibility to the data creators. However, we found our data creators wanted more autonomy over their data, and were willing to accept more responsibility in order to get that.

To answer your specific questions, backfilling is the responsibility of the creator. We have some documentation on how they can best do that, and may provide some tooling in future if we see a need.

On ensuring changes are captured and consistent, we're relying heavily on the outbox pattern for that. That ensures an event is only sent if the local transaction succeeds, and keeps both data stores in sync. It's a fairly well established pattern and should prevent any data loss/inconsistencies when generating the events.

--

--

Andrew Jones
Andrew Jones

Written by Andrew Jones

Principal Engineer. Instructor. Writer. Created data contracts and wrote the book on it.

No responses yet