A Data Platform is not just for analytics

Data is at the heart of every meaningful service, and it’s the effective use of that data which builds a product, and a business.

It’s the data from one service that enables another, whether that’s a closely associated microservice, an anomaly detection pipeline, or any other data-driven service.

A data platform has all the components we need to make effective use of this data, but so often we tie them to a specific use case or deployment, leaving the rest of the organisation to reimplement or go without.

By data platform I’m not talking about a data warehouse, or a data lake, or some other large and potentially foreign system. It could be an effective event streaming platform with excellent tooling around it, or a change data capture-like service that can help users implement an outbox, or highly reliable and resilient archiving and backups of data.

Data (Platform) Engineers are great at building these kind of things, but we keep them to ourselves. There’s no technical reason why we can’t implement a service that archives an event stream to a database that works not just with the data we’re responsible for ingesting, but any data that conforms to a known schema or contract, for example.

By opening the data platform up we can enable many more use cases across the organisation, increasing our impact and generating a greater return on investment for all the great tooling and services we are building.

Originally published at https://andrew-jones.com/blog/a-data-platform-is-not-just-for-analytics/.

Cover image from Unsplash.

--

--

--

Software engineer and tech lead. Enjoy solving problems and getting things done. Always looking to learn and improve.

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

W I R E L E S S | P E R C E P T I O N

Season of Docs 2020: My experience halfway through the program

Contributing with Git

Nightfall AI DLP with Github Action

Email S3 Objects as Attachment

How I became an Android Developer Neophyte

Azure communication service to send message(.NET)

Learning Web Development, Programming Is Good For Your Career Advancement

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Andrew Jones

Andrew Jones

Software engineer and tech lead. Enjoy solving problems and getting things done. Always looking to learn and improve.

More from Medium

Clients want an easy life, for business that means Master Data Management (MDM)

If all you have is a hammer, everything looks like a nail

Why Do I Need Master Data Management?

Deploying data in Covid-19 response — What worked?

strategies that made data use for Covid-19 a lot more possible than we have seen before