Archive for July 2015

Spotify Knows What I Like But Not What I Want


I’ve spent a fair amount of time looking at predicative analytics. Predicative analytics is made of software, data, and a model that should forecast, or predict, something. Consumer goods and retail companies use predicative analytics to try and determine what someone will want to buy. Media properties use predictive analytics to take a good guess at what you would like enough to consume. In other words, predictive analytics tries to determine what you want. This is tricky and sophisticated stuff. It requires lots of data about individuals and people like them, an understanding of the patterns that indicate taste, and software capable of processing all of that data incredibly fast. Which brings me to Spotify.

Full disclosure: I love Spotify. I love it enough that I’m willing to pay for it. On Spotify, I can find almost – almost – any kind of music I might want. And I like a lot of music. My taste is, to say the least, eclectic. Progressive rock from my early youth, punk and new wave from my teen and college years, electronic dance music (which everyone knows is the successor to 80’s techno), 90’s alternative, modern indie, blues, early music (as in 1500’s), reggae, rockabilly and so on. When it comes to music, I’m certain to give predictive analytics engines a fit. I recognize this upfront.

Spotify has been doing a lot with analytics of late. Recently they released something called Taste Rewind. Their software asks a user to pick a few bands from a group they display and then generates a playlist from each decade from the 1920’s onward. That’s a pretty straightforward analytics and big data task. That’s not to say it’s easy to do but at the heart of it, Taste Rewind is comparing the characteristics of the artists that you choose with other artists in their database for similar matches.

Last week they added something new to their service. Called Discover Weekly, Spotify puts together a weekly custom “mixtape of fresh music”. With Discover Weekly, Spotify is trying to predict music that I want to listen to. Based on my listening history, and comparing it to others with similar taste, they put together a playlist that should – and I mean should – give me music that I want and have not heard.

Alas, it’s close but not there yet. For the most part, the music is serves up is music I like a lot. That’s good. I don’t like all of their picks but I think that’s impossible. No forecast, whether human or software generated, is going to be 100% accurate. The bad is that little of it is particularly new to me. Of the 30 or so songs in a Discover Weekly playlist, on average I’ve heard probably 28. The worse is that, while it’s all good music, it’s not necessarily what I want to hear. Just because I like The Clash doesn’t mean I particularly want to hear them now. Instead of Discover Weekly, it should be called “Music that you probably like but might not be in the mood for right now” Weekly. That would be a terrible brand name though…

And this is the heart of the problem with predicative analytics. Humans are so complex that trying to determine what they want is too hard to do. It’s a gamble at best. Discover Weekly probably works for a great many people, especially those with limited taste. For listeners who have wide tastes and really get into their music it’s probably impossible to predict what they want to hear at any one moment. Unfortunately, those are likely to be Spotify’s best customers. They are the customers who really want to discover new and interesting music, and are willing to pay for it, whereas the teenage Beyoncé fans only want to know what’s popular.

Discover Weekly should get better over time as Spotify refines the data model and adds new sources of data. It will never be 100% accurate but hopefully will figure out how to server up really new tunes.

It will be at that moment when they kick Apple Music right to the curb.

Is there Open Source without Open Governance?

There are so many misconceptions about open source software. For example one fallacy is that open source means free software. A bigger misconception is that all open source is, in fact, really open. These misconceptions can drive poor decisions and expose companies to risks.

Open source software should mean software that is created and evolved by a community. The source code should be available for inspection, forking (the creation of another path based on the code at a point in time), modification, and use. There’s nothing wrong with open source that’s managed through a license. Like fences, license help set social boundaries. And as it is with a fence though the goal of the license should be the betterment of a community and not a single entity.

That brings us to the big problem with much of what passes for open source. It has become fashionable for companies to “open source” their code without really making it open. The company maintains full copyright and the future of the code base is governed by the company exclusively. The advantage is completely with the company releasing the code because they maintain the right to change the license and direct the evolution of the code. Meanwhile, legions of programmers test the code and develop features for free. Features that may not stay open if the license says so.

While technically open source, these type of projects are not open governance. Open governance requires a non-partisan controlling body such as not-for-profit foundation or industry group. This is the Apache model. The board of such as group, not a single company, makes decisions about the software. The board should, in theory, represent a community. It is entirely possible that a not-for-profit foundation or industry group may be heavily influenced by a few larger donors. We all know that money talks. The influence though has to be indirect. Otherwise, an overly controlling donor would open themselves to the derision of the IT community. That’s the same community that they want to sell products to…

I don’t object to closed software. If a company chooses to protect its intellectual property by using a closed copyright and trade secrets model, that’s their choice. Much of the time that is a good choice. What I object to is calling something “open” when it actually is not.

Without open governance – that is without the control of the license in the hands of a group that represents the community – then releasing the software as open source is just aform of a defensive disclosure. With the source in the wild, someone else will have a harder time filing a patent that might cover some aspect of the software. Meanwhile lots of developers are finding bugs and creating features while the company maintains complete control over the license. Ultimately, the community is not in the driver’s seat and is at risk to unilateral changes in the license. That’s not open

Open source is not really open unless there is open governance. The risk is much lower with open governance than many other open source situations. For an IT manager looking at open source software, consider who actually controls the code before making decisions about using the software.