Data Management at Scale: Modern Data Architecture with Data Mesh and Data Fabric 2nd Edition

This was great book on data management. I enjoyed the details and learning about data mesh and domain data and data projects. API was good too.

I bought this for educational purposes, whereas it's more of a biased intervention concept presented with academic contexts. It has a place, and is well written, but not what I needed!

In the second edition of Data Management at Scale, author Piethein Strengholt reacts to the privilege - and problem - of formulating one's thoughts at a certain point in time.In the preface of the second edition, Strengholt explains that the first edition of Data Management at Scale was published more or less at the same time as Zhamak Dehghani' s blog posts on Data Mesh. These posts - and the following book Data Mesh by Dehghani - successfully introduced a language for the kind of thinking that both Dehghani and Strengholt advocate - albeit with subtle differences.This is the privilege and problem for Strengholt - having precisely described a powerful way of thinking about data management at scale, but without a language for it, and now seeing this language emerge, complete with words he understands in a different way than Dehghani.Hence the need for a second edition.Strengholt' s thinking succinctly challenges a centralized data team, managing centralized data pipelines doing centralized data transformations on a centralized data platform. This is the core problem of companies that are trying to stay competitive in our era. They lack capacity to scale: One group of people, doing work for the rest of the company, cannot take on endlessly more work. The centralized architecture turns into a bottleneck. What is needed is a decentralized architecture and explaining how that is built is Strengholt' s intellectual endeavor.With the language of data mesh, Strengholt uses the second edition of his book to move away from an abstract but very thorough depiction of read-only data stores/batch jobs (RDS), application programming interface (API) and streaming to transport data between domains in a decentralized setup, with a metadata layer on top to ensure data discovery, that again ensures data governance and adjacent necessities. Instead, Strengholt smoothly moves into the data mesh language, expanding and detailing the various patterns and strategies to enable the transportation of data between domains. However, to Strengholt, an existing architecture cannot transform itself into the purist ideal of Dehghani' s mesh, it must decentralize pragmatically yet idealistic, taking into account RDS, API and stream in an IT landscape impacted by legacy and opaque data flows in daily operations.Strengholt persuasively argues for a reinterpretation of data products; their data and metadata are separated. With great clarity, Strengholt' s native Aalsmeer Flower Auction is used to argue that unlike actual products, data products are not naturally bound together with their metadata: Today, metadata sits in distinct, separate repositories such as a data catalog. This is merely an example of how Strengholt expands and details the language of data mesh.With the second edition, companies can now consider thrusting into data management at scale.