We walked past a record store recently, and, overcome by a fit of nostalgia, we stepped inside and had a look around. Those of you who are of at least a certain vintage may well have similar memories of flipping through crates or bins of new--or new to the store--vinyl, looking for new releases, or something from the back catalog of a favorite band, or maybe a discarded treasure from someone else's collection. This blog post will discuss that our history of consuming recorded music has some parallels to the way many of us now interact with data and data products at our organization.
As we were perusing the records, whose resurgence as a delivery method for recorded music has perhaps come as a surprise, given that music delivered digitally seemed destined to wipe the format from the earth, some of our muscle memory returned, and we flipped through with a little dexterity. Nothing like the old days, of course, when we could lay waste to a Tower Records or Newbury Comics in minutes, to say nothing of local independent stores with smaller stock that refreshed less frequently. But our actual memory came into play as well, and from this distance we couldn't help focusing on how inefficient the process was, and how much time and money it wasted.
Depending on the store you were in, and the city or town it was located in, inventory varied widely. And unless you had an encyclopedic memory, or maybe a homemade encyclopedia of records you were looking for, there was a pretty good chance that even the most well-stocked stores weren't going to have everything on your list, and certainly not in your budget.
Many stores would have a "new releases" section up front, or near checkout, but that generally reflected only a fraction of inventory. Stores that sold used product usually had a "newly acquired" section. Your odds of finding something you liked in this section might have been favorable, but your odds of finding exactly what you were looking for were pretty long.
One thing you were likely to find, especially in stores that carried used records, was a collection of records by bands you'd never heard of, or records by bands you knew that had songs you'd never heard before. Unless you wanted to judge the album by its cover (literally!), you had to do some research.
Research options included reading about a band or record in a magazine, hearing about it from a friend whose taste you trusted (maybe if you were lucky your friend had recorded a copy to cassette and you could preview the whole thing), or asking the record store clerk.
Each of these, in their own way, was a kind of curation. Certain magazines recommended certain types of music, so if you wanted jazz you might favor one magazine and if you wanted power pop you'd certainly rely on a different one. Some friends had different preferences, but given a diverse enough collection of them you might get consensus on a band or record, and then make your purchase somewhat more confidently. Store clerks were the ultimate wildcard: on the one hand, they listened to music all day long, and probably read all the music magazines in the store. In our experience, many of them played in bands! On the other hand, well, we will just note that this wide range of exposure didn't always translate into the best recommendations, and leave it at that. (For any whippersnappers who've made it this far, checking out the record store scenes in "High Fidelity" would illustrate the situation better than any more words here.)
Once compact discs became the standard product for record stores, many of these stores introduced listening stations, which allowed you to preview music before you bought. Some stores allowed something like this with vinyl and turntables, but that was a tedious and occasionally fraught process, given how easy it was and remains to scratch a record and render some of it unlistenable. Even previewing a record didn't always tell you how much or how little enjoyment you'd get from it. But in our experience these listening stations really helped out!
We are aware that not everyone had this experience, or cared to repeat it once they did have it. Plenty of people only listened to music on the radio, or when given to them, or in some circles, via mix tape (or cd compilation). But as we waxed nostalgic in the face of all that vinyl, it occurred to us that our history of consuming recorded music has some parallels to the way many of us now interact with data and data products.
Many of our clients come to us having developed libraries of thousands of canned reports, scripted integrations, semi-interactive dashboards, and other data products. Most have an extensive number of ways to access, view, and otherwise consume these products. In many cases, this inventory is largely uncatalogued, and almost certainly, curation efforts have been sporadic and, as in our record-buying days, all over the place. So finding out what data is available, and where, can be a major and unrewarding challenge. It's not uncommon to hear about users giving up and just accepting whatever data is easily available. To extend our analogy, these might be our radio listeners.
Almost all of our clients have data experts for certain systems or domains. If you have a question about whether data has been collected, where to get it, and how it should be understood, those experts are in a position to answer that question. These people are a little like our music journalists of old: if you wanted to know about a certain band or genre or subgenre, there were experts out there who could enlighten you. But beware: if you wanted a similar level of knowledge about something not in their musical wheelhouse, they were not a real resource.
We're sure you can see where this is going. Information technology professionals, particularly those who are involved in data architecture, modeling, or business intelligence, have been asked to be something like record store clerks. In some cases there may be a real overlap between what technical resources know and what data requesters need, but in many cases it's a matter of convenience and proximity. If you're in the record store and you have a question, you could ask anyone else in the store with you, but you know that the clerk has been there a while. Maybe they'll know?
We threw some subtle shade at clerks above, but we can only imagine the kinds of questions they'd get.
- Even the smallest store would have held thousands of records, perhaps across multiple formats, and yet these poor employees were expected to have an up-to-date inventory in their brains at all times.
- A shopper might say they looked in a certain section and couldn't find what they wanted. Well, the artist's first name starts with D but their last starts with E - where exactly had someone looked?
- What kind of music is it? rock? jazz? country? hip-hop? Plenty of stores organized by artist and genre.
- "It's not in the store now - when are you going to get it?" Well, if it's popular enough it might be part of our regular shipment from the label. Maybe someone will want to sell their copy soon. Or we can try to make a special order, but even then it's hard to say when something might arrive.
- Don't even get us started on the dance involved in buying and selling used records!
At this point, we've stretched our music analogy about as far as it can go.
- Nearly every bit of recorded music is now available on-line in some fashion and can be sampled in full before a purchase decision is made.
- Every music (or video) service has some kind of algorithm to say that if you like X, you might also like Y. (While we haven't found these to be foolproof, they're still leaps and bounds better than what you had with you when you walked into the record store.)
- Even in the physical world, it's usually easy to ascertain whether a given product is in stock at a given retail location, and to place a special order. Although your local independent record store may be an exception, or a deliberate holdout.
Having said that, what it's easy to do is stream the new Taylor Swift album, or to find out if it's in stock, or to request that it be delivered. Despite all the tools at our disposal, it may not be quite so easy to find out, for example, if vaguely country-inspired pop performed by a female singer is in stock, whatever that means, and whether, if it is in stock, that particular variety is what we're looking for.
And this still describes the state of data for many organizations. When it comes to data, too many organizations still face the twinned issues of inventory and curation. Users don't know what's available, they don't know how or where to get what is available, and they don't really know whether what's available meets their needs.
Our data catalog tool, the Data Cookbook, helps organizations address these issues. Have a couple thousand poorly-documented, confusingly-named, possibly duplicative reports in a legacy BI tool? Import what you do know into the Data Cookbook, which is searchable, which allows users to make comments, and which enables you to identify candidates for further documentation (curation). Use the curation process to expand on discoverability, by defining data elements using business terminology, by tracking lineage where appropriate, by relating quality standards and issues to key reporting deliverables.
You could look at every dashboard until you find the one that works for you, if it exists. You could ask your colleagues whether they've got something they can share with you, and you could ask your IT record store clerks if they've come across something you could use. Or, maybe, you could consult a tool designed to share knowledge, to unite functional and technical documentation, and to enable searching and revising.
It's not magic. The bin full of proverbial records is still in front of you. But a tool like the Data Cookbook helps you quickly decide which of those records you're most likely to want to listen to. At the end of the day, you still have to put the record on the turntable and listen to it to decide whether you like, how much you enjoy it, and if you're going to buy it and take it home. You still have to look at that dashboard, interpret its display, and decide what to do (or whether to do anything) in response to the information it contains.
We didn't spend too long sorting through records in that store a week ago. We remembered the thrill of discovery, but we had chosen to forget the long hours and frequently unsatisfying purchases that accompanied those occasional thrills.
When it comes to data, you could browse here and there, you could drop the needle on a few records, you could see what your colleagues are using and maybe adapt it to your needs. Or, your organization could get serious about cataloguing and curating data, so that the right data makes its way into the right products that actually meet your users' needs.
We've got to go--we just remembered there's a hole in our record collection that can be filled with only a few mouse clicks! But when you want to talk about data, or obscure vinyl, you know where to find us.
IData has a solution, the Data Cookbook, that can aid the employees and the organization in its data governance, data intelligence, data stewardship and data quality initiatives. IData also has experts that can assist with data governance, reporting, integration and other technology services on an as needed basis. Feel free to contact us and let us know how we can assist.
Photo Credit: StockSnap_5IWPD5X5XJ_RecordShopping_StacksofWax_BP #1300
