Category: models

Survivor bias: an experiment

August 4, 2014 /

I write a simple computer program that randomly selects one of two outcomes. I send out letters telling people I can pick stocks and shares that will go up. About 50% of the people I write to think I know what I’m doing and think they might buy subscriptions to my stock picking service.

I do the same thing with the 50% that got the right answer, each time making an ever smaller group think I’m a stock picking genius.

Eventually I will run out of people, but right up to the last two it will appear to them that I really knew what I was doing, when I was effectively tossing a coin.

In the mean time quite a few of them may have subscribed to my stock picking service. Good for me, not so much for them.

The people at the end of this chain of probabilities will think that they are kings of the world, when they are only survivors of a simple process that could have picked anyone from the original group of stock buyers.

I recently attended a virtual course on complexity and discovered the fun simulation language Net Logo. For historical reasons the little actors displayed on the screen are called turtles. I could just as easily build a program that has a population that halves every turn. Is the lone turtle left blinking on the screen at the end a special turtle, did it somehow avoid the grim reaper against the odds? No, obviously not.

If that turtle was instead a person? The story it had to tell would be a story the rest of us would want to emulate because it was a survivor. This is why the biographies of the heroic entrepreneurs are often not that surprising. This is why their opinions quite often don’t differ a lot from those of others in the same cohort.

I used to work for Oracle and read the unauthorised biography of Larry Ellison – it’s an entertaining read and certainly not very complimentary. One of the things that comes through is luck, Oracle came close to going over several times, literally so close that a single large order from Japan saved it. I was talking about Richard Branson with someone recently. If I remember it right he originally had the idea of opening his record stores near tube stations late at night so people travelling home could buy records, he also had an aunt who lent him the money to open these stores. Great idea, but he didn’t have to go to a bank and get laughed at. Again, when Steve Jobs early Apple got some venture capital so they could make things happen – whoa, things started happening! (Thanks to Tim Spencer for this one).

There is an unknown population of other people and businesses that didn’t make the cut, or that stayed small service companies that are still around but not mega corporations. Probably 99.99% of them. Jobs himself acknowledged this with the famous analogy that the things joining the dots together are only apparent when you look back and make a story of them. In my last post I talked about pareidolia, which is the human tendency to invent patterns where none exist. We can all find stories up like this; if I hadn’t answered a job advert in the Independent I wouldn’t have met my wife, my kids wouldn’t exist and, like, wow man (sarcasm off). We make a causal chain of events but forget the massive part that chance plays in what happens to us. If the University had placed the ad in a different paper on a different day, who knows?

Survivor bias makes the winners’ stories compelling, but there are another 9999 (or many more) stories of other people we never hear. Remember this, next time someone tells you that you must emulate this or that hero of theirs. Success needs an element of luck, and the same person may not be lucky twice. Napoleon used to ask of a general is he lucky? He was no fool. We all love 37 signals’ (now Bascamps’) story, but there was luck there. Lots of people have tried their formula without getting what they have, or even losing everything.

This isn’t meant to sound like a doom and gloom principle, far from it, what it is saying is you need to find your own way that works for you. You also need to take that opportunity when it falls into your lap and do something with it. But you aren’t any more special than anyone else. It doesn’t matter. What matters is being clear about what you want and sticking to it.

Generic is the wrong answer

July 27, 2014 /

Dear reader, cast your mind back to the time when you first wanted to put some information on a website.

You had to trust the crusty programmer types, and if you wanted changes they wouldn’t necessarily be able to do them at any great speed. This was fine in the stone age, around the year 2000 or probably earlier. Then, one day, the technology started to catch up with the aspirations and everything had to be done at internet speed. So crusty developed less crusty tools that let things happen quite a bit quicker.

We suddenly had content that people created for themselves. Never to be outdone the constant search for simple generic solutions to be everything meant that the companies that made the very expensive PC based tools for document management had a go at creating something. Some of their original big corporate customers kept on with them. These systems don’t work for the internet, except when you are creating some kind of library, which is the metaphor they started with.

The more considered approach was to treat managing content like it was a solved problem. The content management system (CMS) was born (or at least became more mainstream) and allowed you to manage your catalogue in one place so you could, you know, sell your stuff to people that wanted it.

Then we fell victim to the illusion of best practice. Problems are in a space where they seem to be the same, so we (software engineers, academics and other people who should know better) develop generic solutions. I mean, content is just abstract stuff that we let those tricky user types create so they can pay us. We can just pull something open source or off the shelf and say off you go to them and get back to important stuff like arguing over which 20 year old text editor is best.

But here’s the thing. The pattern user -> catalogue -> put catalog on the interwebs in front of customers is the same, but the user / business / enterprise is most definitely not. Most (not all) open source CMS were created for a small community to share info – they do that really well. However, a commercial company that sells white goods does not have even remotely the same needs as one that sells online holidays where the stock is a multi lingual database of hotels they have a relationship with. They need content management, the thing, but not in the same way, and the OSS solution is probably completely wrong. Also – you don’t want your site to look exactly like your competitors or you are competing on price, which is suicide.

Human beings are subject to pareidolia. Seeing patterns that aren’t there in the data. This is also related to clustering, confirmation and congruence biases. When it meant running away from the wind in the grass because you thought it was a lion that was ok. Now we look for levers to fix and change things that often don’t exist. Eric Reiss calls this robots in the Lean Startup. He cites the case when the US manufacturers couldn’t work out how the Japanese were getting such massive productivity gains so they went to see and saw robots. After a robot installing frenzy the US was still far behind. The Japanese spent a lot of energy understanding how they did things first and put the automation in where it made sense, gradually, and always testing the results, for their specific needs. Good practice, but hard to emulate without a complete culture change. Pareidolia (and the other biases) put robots inside processes that were still broken. In knowledge work this is even easier because adopting the latest fad is only the cost of a couple of books and some (more) training, until your organisation fails catastrophically. Danger, Will Robinson.

Best practice only works in very simple environments where all of the problems are completely understood. Buying a CMS from someone assumes that they understand your needs properly. They probably don’t. It’s dangerous.

Good practice means the using the skills of analysing the problem properly and divining the principles you should follow. A company should control its own content; after all that’s what it’s selling. This means layout, search, everything. However you might use a markup language and a standard group of elements for layout to make it easy for non-web heads to create that content. Find levers for the easy stuff, for the things that do have generic solutions that work.

Startups don’t want to spend money on stuff like a CMS that is tailored to their needs and will tend to go for the OSS solution. I think that they need to start small and find out what those needs are, just like everything in the lean approach. So don’t buy one, and don’t slavishly pull in an OSS one that will bite you and turn out to be an expensive mistake. Create something as simple as possible that meets your needs, then you will know what the needs are and where the pain is. Then you can use good practice to get to where you want to be. Then you can find the metaphor that works for you, and let your competitors try and shoe horn someone else’s into places they don’t work.

Best practice implies generic – generic means cheapest and also (face it) BORING.

Good practice is searching for the right metaphors and being patient enough to wait for them to appear. And, of course, those metaphors are the ones that work for you, might not be right for someone else.

Coding Horror: Presentation: Be Vain

December 5, 2007 /

http://www.codinghorror.com/blog/archives/001007.html?r=21636#endcomments

I spent most of my spare time from age 8 to 16 learning to play the guitar and these instant gratification things just annoy me.

That said, I agree with the point that presentation is far more important that people think, and the example is a reasonable one. Please stop focusing on the specifics here – Jeff’s trying to make a very reasonable point.

Techies start from a data model and then wrap it with some half-assed forms from a generator – I know we do, I’ve done it myself lots. I learned that this does’t work because the logical view of the data isn’t the user’s view. Not listening to their pain and trying to help them get to where they want is arrogance. For example, on a system our users use (and I did’t write) they hate the way it takes forever to open the person details screen and all they ever want is the email address or phone number – a techie generated the user form out from the data model and joined it to the logically correct place. Imagine how much extra effort it would have been to put the phone number and email address on the summary? Hardly any, but it does’t fit the cod-head approach. I would lay money the coder has never talked to a user or watched one use the system.

The sexy unwritten thing is’t at all new. I worked on many projects in the pre-web days where the client had “bought the powerpoint”. Then they kicked us lots because it was all a lie.