Data is critical to the success of GoCardless, whether it’s driving the decisions we make, detecting fraud, or powering intelligent products such as Success+. However, we can only fully unlock the value of our data if it is of good quality. Users need to be able to discover the data…

Almost all data platforms start with a change data capture (CDC) service to extract data from an organisations transactional databases — the source of truth for their most valuable data. That data is then transformed, joined, and aggregated to drive analysis, modelling, and other downstream services. However, this data has…

I’m so proud to announce that my book on data contracts is out now! Available wherever you buy your books, including Amazon, Packt, ebooks.com, Kobo and through the O’Reilly learning platform. I’m excited to share with you everything I’ve learned about how data contracts solve some of the age-old problems…

When talking about data contracts, we naturally focus on the benefits for our data teams, and how by improving the reliability and quality of our data we can drive significant business value from the data. But at GoCardless there is more to it than that. The way we have implemented…

This is an interesting point! And actually what you expect from your data practitioners should inform if data contracts are valuable to your organisation. For example: Do you expect the data to be correct? Are you planning to use the data to drive important decisions? Do you need to trust the data? Is the data used to build any data-driven products, including ML-based products? Are the customers of those products internal or external?

It’s been 6 months since I introduced Data Contracts as our initiative to improve data quality at GoCardless. So, how are we getting on? What’s gone well, and what are the challenges we’ve faced? Starting with why

As someone who works in an enablement team, I’m trying to instil a best practice of answering with docs. This has a number of benefits to both us and our users: It ensures our documentation is constantly improving and being kept up to date It trains users to look in the docs first, before asking, allowing them to self-serve It reduces future toil This means when a question comes in via Slack or otherwise we try to answer by linking to the documentation. If there isn’t any, first write some up and then use that as the answer. This should apply to our internal documentation and our own runbooks.

Data is at the heart of every meaningful service, and it’s the effective use of that data which builds a product, and a business. It’s the data from one service that enables another, whether that’s a closely associated microservice, an anomaly detection pipeline, or any other data-driven service. A data…

We recently had an incident with our data pipeline, resulting in data being lost on route to our data platform. Of course, you never want an incident, but failures are a fact of life. …