Wanna get rich? Do this

posted by Rob on September 7, 2007 06:38 PM

bullion.jpg

When you want to buy a screwdriver, it's pretty easy to get one that will do the job (-> to tighten/untighten screws). They're standardised. But when you want to buy some fancy new slacks, it's only semi-easy to do. The sizes are standardised, but finding the right size is only half the job (-> to clothe you in style). When you want to buy a CD or a novel, it's not easy at all. Yes, they're standardised so you can be sure the CD will play in your stereo or that the novel is in the correct language, but will you like it? The surest way to find out is to buy it and read/listen to it. And risk wasting your money. If only someone would come up with a better way.

Popularity works to a point. If a squillion people like something, maybe you will too. But then again, if that was all there was to it, entertainment would be standardised like screwdrivers.

The next-step idea I've been talking to people about since the mid-Nineties is this: you get a hundred thousand people and ask them what their favourite twenty albums/novels are. Then you look for clumps in that data: people with eighteen or nineteen choices in common. Then you use the variations within a clump to make recommendations. If you and I have nineteen albums in common but you've never heard of my third favourite band, there's a good chance that you'll love them.

There are lots of alterations and tweaks you can make to that idea. Rating all your albums will provide better data than just covering the top twenty; mention the ones you hate too. Perhaps you'll need a bigger sample size. Maybe you'd have to try a few methods of making predictions to find one that worked.

But think of what you might discover. Maybe there'd be crossover between album and novel choices - and perhaps movies too. Maybe instead of infinite variety we'd find that, musically speaking, there are only forty-one types of humans. Or maybe no two people are identical in their tastes - but you can bet we'll find plenty of people who are very, very similar. Sooner or later someone has to make this work and I'm pretty convinced that, with a few trials and experiments, it's viable. There seem to be two main problems.

First, if it does work in, say, the music industry it will threaten most of the intermediate/middleman roles in that sector - and that very much includes all the people who are in charge right now. Most bands can learn to record their own MP3s. Most music fans can download them to their pseudoPods. All you need in the middle is a big website that collates preferences and makes good suggestions, plus a big music library/shop. Without all those middlemen, it becomes economically viable to charge one or two pounds for an album instead of ten or fifteen. That means music lovers get a lot more music for the same outlay. And hopefully they rate it and keep the process going. Music lovers do better, all but the biggest bands do better, and only a few suits lose out. But the suits are the ones with the capital and the lawyers and the friends in government. So that's problem one.

Problem two is that Amazon have a system that tries to do something like this already. Every time I mention this idea to people, that's what they tell me. But Amazon's system doesn't ever seem to work. Not for me, at any rate, or for anyone I've talked to. Every now and then I rate a bunch of books and albums on Amazon and let it update its recommendations accordingly. And I believe I've grumbled about the results before. Amazon's system delights in saying 'Hey, if you quite liked this book in paperback, we bet you'll love it in hardback. Or with a different cover. Or better still, have you thought of buying other books from the authors you already know you like?' Amazon's 'system' almost never makes a helpful recommendation. Most of us have already worked the brilliant notion of buying other books from our favourite authors. But what about authors we've never heard of? What about all those things we don't yet know we want? Well, twice now Amazon has recommended non-obvious albums and I've bought them. I didn't like them, but hey, at least they were imaginative suggestions. I had another look just now and of their 15 recommendations there are only two suggestions that go beyond just 'more by this author/band'. I'm almost tempted to waste some money on them. Though I won't be getting my hopes up.

Which brings me to a final point. Imagine what the system I've described would be like if it actually worked. With the Amazon version I'm fairly sure I'm throwing my money away. But what if they started getting it right? What if they began to inspire confidence? What if spending money on their recommended titles actually had a track record of producing treats instead of flops? Well, I think it's pretty clear what would happen. I'd start spending a lot more money with them. Or whoever else cracks this problem. So come on you garden-shed entrepreneurs, asssemble your database gurus and your stochastics experts and your bayesian-inference wizards and your hundred million in venture capital and make that most exciting of gadgets: a jukebox that plays my favourite song from a band I've never heard of.

spacer

Comments: 6


This does happen all over the net, you just simply dont see it.

This is a simple example: http://www.netflixprize.com/

Netflix have a team of monkeys working around the clock to make their algorithm better to recommend movies you would like to see based upon your history, and the ratings you gave the movies you saw, then comparing it with their database of millions of other peoples preferences and history.

They arent giving away a million dollars because they think it might be a nice concept - they are very serious about their algorithm!

Also check out http://www.pandora.com/ before you rush off to develop your million pound VC backed gadget music recommendation site.


I agree, there are already several websites that do this for music, & more in production. There was a rash of articles about applying the same idea to books (all of which mentioned the Amazon issues) about 6 months ago.


What? You're saying these systems you mention actually work? Interesting. Well, I'll admit I've been meaning to check out Pandora for a while. I'm fairly sure I recall Anna saying to me 'Pandora is awesome'. And it wasn't an ironic 'awesome'.


Pandora does actually work via the system you described. But I'm not sure I like that.

If you allow algorithms to choose what entertainment you're going to enjoy, you lose the system of trial and error that sometimes leads to wonderful discoveries. In the end, that leads to humans becoming incapable of making their own choices, or, less alarmist-ly, to humans becoming incapable of connecting dots on their own. This has already happened because of the internet; I barely remembered how to use a phone book when our internet went out last weekend. If I hadn't listened to loads of music that wasn't right for me, I wouldn't have found Bjork or Brian Eno or Moxy Fruvous, all favorites that have enriched my life. If I only saw movies that an algorithm determined that I would like, my enjoyment column would become so narrow that I wouldn't be able to determine the difference between "what I don't like" and "what sucks".

I feel like the only way to find hidden treasures is to wander around a dusty old bookstore, buying 10 books that are okay and one book that turns out to be incredible. Essentially, kissing a lot of frogs before you find a prince. Warts or not, I don't think I'd want that afternoon taken away by Amazon telling me what I'll probably like.


Surely you only need to worry about algorithms dictating your life if you intend to live in a hermetically sealed box, avoiding random acts of society for the rest of your life.

Actually, you could make a good argument for the notion that all of your book buying choices are based on a cosmic algorithm unique to you, based on your previous reading experience, the recommendations of your friends, styles of visual presentation, and historic susceptibility to suggestion.

At the very least, a site such as is posited here would be one of many, many factors influencing you - it might be more or less trusted, but it won't eliminate your free will, thankfully.


Katherine, I'm slightly reminded of that Simpsons episode where Professor Frink explains that using computers for dating is a bad idea because it would create matches so perfect it would remove all romance from the process. I can't believe it would go from rubbish to 'too good' so quickly we'd have a problem yet. All I'm after is a satisfactory hit rate. You're right, one doesn't want to become insular, but neither do you want to get frustrated in the hopes of finding new music that you like. I'm sure a happy medium is possible. And a good algorithm will have a setting for adventurousness that you can turn up.

spacer

Post a comment

We love hearing from our readers, but please stay relevant and pleasant. The comments are for responding to the specific blog post above. If you have any other queries, please contact Snowbooks via email. Off-topic or offensive comments will be removed without notice.

To screen out automated spam, please answer the following very easy question:

What colour is nice, new snow?

(please use all lower-case characters for your answer; no capitals)


Back to the blog »