A Data Platform is not just for analytics

Data is at the heart of every meaningful service, and it’s the effective use of that data which builds a product, and a business.

It’s the data from one service that enables another, whether that’s a closely associated microservice, an anomaly detection pipeline, or any other data-driven service.

A data platform has all the components we need to make effective use of this data, but so often we tie them to a specific use case or deployment, leaving the rest of the organisation to reimplement or go without.

By data platform I’m not talking about a data warehouse, or a data lake, or some other large and potentially foreign system. It could be an effective event streaming platform with excellent tooling around it, or a change data capture-like service that can help users implement an outbox, or highly reliable and resilient archiving and backups of data.

Data (Platform) Engineers are great at building these kind of things, but we keep them to ourselves. There’s no technical reason why we can’t implement a service that archives an event stream to a database that works not just with the data we’re responsible for ingesting, but any data that conforms to a known schema or contract, for example.

By opening the data platform up we can enable many more use cases across the organisation, increasing our impact and generating a greater return on investment for all the great tooling and services we are building.

Originally published at https://andrew-jones.com/blog/a-data-platform-is-not-just-for-analytics/.

Cover image from Unsplash.

--

--

--

Software engineer and tech lead. Enjoy solving problems and getting things done. Always looking to learn and improve.

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Moving Mina Forward — An Invitation

How To Create A Web Service Project In .NET Using Visual Studio

How to use custom icon fonts in flutter

How to create indexes in Mongodb

The Journey So Far

A strong partnership with Oracle(charli3)

Experience with our first Flutter project

2D Galaxy Shooter — New Enemy Type: Aggressive Enemy

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Andrew Jones

Andrew Jones

Software engineer and tech lead. Enjoy solving problems and getting things done. Always looking to learn and improve.

More from Medium

Are we ready for Data Mesh ?

4 keys to data quality on a data lake (or lake of lakes)

A Google Earth screenshot of a portion of Yathkyed Lake in Nunavut, with a yellow arrow pointing at the recursive island described in the photo caption

What is Data Ingestion? | The Definitive Guide

Too late to return or CDMP: Data Quality specialist exam