Life on the Data River
In the early 1800’s, passengers and freight were transported on highly romantic but also highly flammable steamboats. These monstrous mayflies of the transportation world were navigating the ever-changing American rivers and waterways.
Mark Twain celebrated the ability to pilot these confection cakes of people, cotton and timber through snags, sand bars, changing currents, and an almost complete lack of navigation information that we now take for granted. The humblest houseboat can have an inexpensive sonar that would have provided life-saving information in that day and age. Modern dentistry is appreciated as well, but for the sake of this story, I’ll focus on data and technology.
The very core of “Life on the Mississippi” is the transfer of knowledge. As a “cub” pilot, he was trained somewhat methodically and alternately brutally and the ins and outs of navigation on a body of water that was unpredictable and deadly.
There were two solutions.
- Rote memorization - stressful and gap prone but thorough
- Channels for sharing information - the river passage formula
The second requires the first. A pilot has to know how to guide his steamboat. He (always back then) would know what it’s physically capable of and see how the river pushed back or aided the journey. He’d understand the ebb and flow of floods and droughts.
Once that knowledge was embedded, he’d have a subconscious framework that could be updated.
The riverboat pilots formed an association that required that the pilots complete a form after each trip. They’d note the route they took and particularly would remark on any changes.
The arriving pilots posted the results for other association members to see, and the receiving pilots would be that much more informed.
Pilots who weren’t in the know quickly became a liability, and the insurance companies quickly figured out that well-informed pilots didn’t make as many errors and mandated that all pilots join the association.
Modern World of Analytics
In this day and age, anyone can be an analyst. All they need is a little data, a little gumption, and with a bit of research, can become a local authority. What inevitably happens is the pilot … er… analyst does this more often than not on their own. As they start a job, they’ll have mentors and teachers. But as they grow into the role, they get better and faster at what they do. Some things will become rote memory without much effort; others will linger at the edges haunting them as they rarely get touched.
One beautiful day comes when they’ve made the big move to another challenging role, another team, another company, or perhaps they’re going to travel around the world with their fiancé. But all of their hard-won knowledge is locked tightly in their heads. They’re (always) tasked to hand over what they know, and no one gets hurt. One or two or four but never five in-depth dive sessions are held where the luckless fellow teammates or perhaps a new hire drink from the firehose of tips, tricks, traps, and half-realized dreams. In reality, the expert transfers 10% of their knowledge; 5 % is documented in poorly constructed pages that may or may not be readily available for the company.
My techniques are documented in my code!
Which is fine, if
- you’re as good a coder
- the code works
- there’s some semblance of comments
Likely, there aren’t any of these pillars of knowledge.
I’ve designed the Data Field Guide approach to gather some basic analytic techniques, automatically generated documentation and guidance that can be handed off and updated as the next pilot … err.. analyst takes the wheel and guides this costly business steamboat down the stream.
Credits
Images pulled from the digital Gutenberg copy of Mark Twain’s ‘Life on the Mississippi’
Book was memorized as a child so that part was just training.