The reason this thought about “data-darkness” (and inversely, “data-brightness”) came up for me was because I had wanted to find a kind of focus area to anchor any of my future projects in. To be intentional about building a niche. In doing so I wanted something that felt meaningful and, maybe most importantly, something that could stand the test of time and the speed of technology. I ran through a couple exercises to explore angles, but ultimately the most illuminating thing (see what I did there 🔦) was creating a list of all of my favourite apps and highlighting any common denominators. I figured drawing a thru-line might help me identify something that would be worth spending my time on. I think it did just that.

The list in question — Let’s call them ‘The Data Brighteners’ for now:

• - Google Maps
• - Uber
• - Transit App
• - Fotmob
• - Twitter
• - BikeShare
• - Amazon
• - Canadian Tire
• - Shazam

I thought about why these things peaked my interest and what they might share in common. In short order, it became clear that the commonality is real time data. Data leveraged and shown in a way that gives you confidence about things that you cannot see, use, touch or do in the moment. They quickly provide factual information where I otherwise would not have had it. In all of these instances I am being fed (normally) accurate, trustworthy data that simulates me being right there with the thing that I need information on at that time. They kind of subsidizes my lack of knowledge and connection to the thing that I’m attempting to do, use, buy etc.

Just a couple of examples to help fill out the point: Back in the day, when we were travelling somewhere new, we printed out directions. That wasn’t very long ago. If you got lost, you were on the phone asking for directions from someone already at the venue, or you parked your butt on the side of the road with a literal paper map and your crumpled list of step-by-step directions trying to find where you went wrong.

Realized you forgot something on the day of your holiday dinner? You’re either going straight to Rabba, or you’re calling around your local groceries to find out what their holiday hours are. No one answering the phone? Maybe they’re just busy. You’d slip your coat and boots on and make the trip and find out only once you got within sight of the parking lot whether you were in luck or not.

I actually kind of miss those adventures but, compared to today, those were the data dark ages. Data-bright is the new standard. Curious about holiday hours? Two taps of the thumb and you’re set. Want to know if your Canadian Tire location has that triple reinforced garden hose you’ve been meaning to get? Fifteen seconds later you’re hopping in the car with confidence. There is a certain level of incomparable value relative to effort that is the reason that these kinds of consumer tools appeal so much to me. It’s more than just saving time, it’s relieving stress and boosting quality of life. Those moments add up and they make a meaningful difference in my own life.

The Data Brighteners are all examples of software that creates data-brightness in various areas including transportation, news and commerce. It’s all well and good, but one question sticks out;

Where is this data coming from?

I have ideas, but i’m honestly not sure yet. If there’s one thing that’s been proven by these organizations it’s that the product doesn’t have to start that way. It can’t really. It’s almost a right of passage to start off with low lumens. There’s a certain level of software physics that has to be adhered to. If I had a magic genie, I’d ask for 1000 users who were willing to contribute regularly, maybe turn on their location sharing, excitedly leave comments about the state of our focus area or thing. That’s not the reality though, and that’s ok. Witnessing how smaller services like Fotmob, Transit, Bikeshare and even Shazam have grown from something simple and useful, to more robust real-time-capable things is proof enough for me that we can start simple and still work our way there — to data-brightness.

Ok, so we can start small, start simple. But what does that even look like? Where do we start?

Where do we start?

There is a metaphor that I’ve been using to aid with my own thinking behind this. It has helped to ground the concept of this data-brightness and the components that I believe it is comprised of. The metaphor is of the old explore and conquer game “Age of Empires”. It’s an open-world game with lots of things happening in the world, but you have no idea what until you actually go and explore the happening areas yourself. That is the basic concept that encapsulates the foundational thinking of data-brightness and data-darkness. The area that is around you and is visible is the "data-bright". Those areas in which you have not explored and where you are not nearby to is "data-dark". If you are not there, you are not aware… ™

We haven’t been in exactly that kind of position in a very long time. Even before cloud services, before email, before the internet, you could still find ways of remotely communicating to get factual information, albeit in a much more limited way (Ex. phone calling or sending letters). Fast forward to today and there are those areas in which the Data Brighteners provide perfect, near-instant, clarity on the state of the area in which they brighten (ex. Google Maps for business hours). But even today in this advanced world there are still areas, maybe sub or proto areas, in which they are lagging behind in the data-bright standard that most of us expect today. Think large shopping mall navigation, or store inventory, or within one of my favourite areas to spend time, amateur sports. There is much to be desired in terms of data brightness within those areas. To the question of ‘where do we start’ though, that’s where the metaphor comes in. Let’s take the example of amateur sports, like pickup soccer or beerleague baseball. I've listed what I believe to be the composition of true data-brightness below. We'll use the sports example to help clarify each piece.

The composition of data-brightness

The components of data brightness in my mind go like this:

• • Location (Where is it in relation to me?)
• • Speed (How quickly can I access info about this thing?)
• • Tempo (How often does info refresh?)
• • Spread (How many core data points are there? Size, hours, energy, contact etc.)
• • Brightness (How many signals do we have?)

Tempo, spread, and brightness all require a broad base of dynamic user information that is difficult to obtain, but location and speed are closer within grasp. The lowest lumen version of this kind of service is distance and location, using more static information, so let’s start there.

Location

Where is the thing in relation to where I am, and how far is it? In the sports example, this would simply function as a reliable source of where pitches and courts are located around you. This level of data doesn’t require much refreshing since the locations of these things don’t really change at all. That said it would benefit from the validation of others.

Speed

How quickly can you access the data you’re looking for? Is the service or product you’re using structured in a way that gets you where you’re going fast? In the sports example this would be the difference between quick flipping through various queries or results, and frustratedly waiting for the page to load as it tries to complete your request. Where software exists for that purpose, download speed is table stakes, but in the data-dark it provides a good measure of comparison.

Tempo

How often is this data refreshing? If the status of something changes from minute to minute, or hour to hour, day to day, is the data updating available at least that fast? I mentioned before that something like the location of a sports facility doesn’t really require that often of a refresh since it is not something that varies by any kind of metric consistently. How many people are at that facility or the availability of the space at all is something that would benefit from a higher refresh rate though — a higher tempo. Otherwise you risk using stale, inaccurate data to guide your decisions like when to go, what to bring etc.

Spread

How many varied pieces of factual information can you get about this thing? If it’s a tennis court, a low spread would be location and hours. A high spread would include those basics and more: facility size, number of courts, busy hours, surface quality, surface type, average wait time, reviews etc.

Brightness

How many signals can we get for each type of factual information? If we have a high number of people confirming that a field is free, or a game is looking for more players, or a court is booked for a private event then people can act with more certainty and comfort. The factual information that they’re getting has a higher brightness and therefore they can have more confidence in what they choose to do with it.

What does this accomplish anyway?

So what this means is that it gives me a north star. It gives me an anchor. It gives me a place to start and a place to shoot for. Ideally it takes us from data-dark to data-bright in some places. It provides the beginnings of a framework that aims at taking the best of what the Data Brighteners have to offer and attempting to emulate what they provide in the so called data-darkest of product places. If nothing else this breakdown accomplishes what my previous post had focused on which was taking the time to proactively consider a problem in advance of encountering it within the work place — Or at least in advance of needing to make any final decisions. We will see where it leads us.

Thank you for reading.

Nat Green