Skip to content

Building a data system when there is no time

More than once I have had to design a data-collection framework from scratch, mid-crisis, with no runway.

More than once I have had to design a data-collection framework from scratch, mid-crisis, with no runway and no patience for elegance. It taught me something that runs against every instinct a careful person has.

The instinct is to build the right schema. Map every field, anticipate every future need, get the structure correct before anyone touches it. In a calm quarter, that instinct is a virtue. In a crisis, it is a way to deliver the perfect system shortly after it stopped mattering.

What works instead is to capture the few fields that actually drive decisions, ship it, and improve it live while people use it. A rough framework that exists today and informs a real decision beats an elegant one that arrives next week, because in a crisis next week does not exist as a planning horizon.

There is a discipline to doing this without creating a mess you will pay for later. You capture the irreducible core, you document the known gaps honestly so nobody mistakes the rough version for the final one, and you accept that you will refactor under fire. That is the cost of moving at the speed of the problem.

The broader lesson followed me out of emergency work. A lot of good systems do not start as systems. They start as the smallest honest thing that helps, used by real people, improved in contact with reality. Waiting for the complete design is often just a respectable way of not starting.