Category: Uncategorized

Organisational muddiness

Big ball of mud

Way back in the dark mists of the early 2000s design patterns were a new-ish idea. People went crazy for them, no project planning meeting was completed without several links to various design pattern architectures and blah blah. Eventually people realised that the best way to write software was to write software, in an unconscious appeal to the original principles in the Agile Manifesto.

The tools we used that created bazillions of class diagrams that were then translated into incomprehensible Java were quietly put to one side by most people.

Some patterns persist and are useful, things like Factory, and sometimes refactoring to a known pattern can actively simplify how things work. There are also architectural patterns, Martin Fowler wrote an excellent book explaining them and DHH used a lot of them when designing Rails. I think this is why Rails worked so well, it was opinionated software and the opinions were the result of a lot of careful thought and deep experience.

The big ball of mud pattern is what you get when you don’t take steps to design your architecture and instead end up with something that works but is internally incoherent. You see this a lot with successful start up companies. They ran really fast, didn’t try to follow the long-term survival practices such as behaviour driven development. If they are successful, after about 18 months everything comes to an abrupt halt because it’s extremely hard to maintain and full of decisions which hindsight indicates are now poor and inflexible.

I contend that there is also an organisational equivalent, you end up with the structure that’s obvious when confronted with the next problem, and suddenly we have dev teams that work on the same thing but different, and devops because everyone else has one, and the sales teams are using squads because it’s hip and everyone’s in too much of a hurry to read Deming or Goldratt and properly think about what works, for them, now.

Conway’s law

This was originally meant as a joke but it’s been seen in the wild so many times that it isn’t one any longer. Put simply, the architecture of the software you build and the way your organisation is structured will mirror each other. in essence, if you have a pragmatic ball of mud with muddy communications then so will the software you produce.

Whirlpool of mud

So you end up with a self-reinforcing pragmatic organisation that will continue to create muddy solutions with muddy architectures for muddy conceptions of what your customers want. You need to be as deliberate about your organisation’s structure as you are about the software you need to run it.

Successful systems

A system that works is built from multiple smaller systems that also work. This sounds obvious but people often ignore broken or poorly performing sub systems and throw things together.

Poor process produces poor outcomes

It’s there in the section title, mud breeds mud, haste breeds waste, rushing breeds redoing, inconsistency breeds incoherence.

Wishing

Not addressing the mud means you are just wishing that it will go away. This doesn’t generally work that well, but you often find it if you’re working with people who have read too many self-help books that tell you how to build more confidence by affirmations confusing their desires with what causality makes possible.

Sales driven organisations can suffer really badly from this.

Solutions – a starter

Constancy of purpose

When I used to run the Lean Teams consultancy the very first thing I used to do when I ran a workshop for my clients was a session about constancy of purpose. In a nutshell, can you write down what your organisation does in a sentence or two? When you make decisions do they align with it or not? If not, don’t!

The other thing this helps to do is identify people’s needs. Customers and organisation.

Note it all down

  • Identify teams
  • Define responsibilities
  • Define info flows and transformations
  • Use the documentation to create consensus

Be mindful of Conway’s law, is the shape of the organisation correct for what you’re trying to do? Also, noting down means using pictures, long screeds of text are hard to write accurately and follow precisely. Interaction diagrams with notes where it works are by far the best way of doing this.

Double Looped learning

  • Single looped learning is being able to turn a handle and get a result.
  • Double looped is knowing what the objective is behind the turning of the handle and maybe changing it.

You need to get to single loop first. You will have a rough idea of what needs to be done and need to work out how to do it. Again, use pictures. Only attending to the single loop can make you very myopic and changes outside the loop can remove the need for it, causing you to waste your time.

Have a look here.

Good processes

Good processes have these qualities

  • Repeatability
  • Lightweight
  • Well defined interfaces with other processes
  • Learning
  • Less than ten steps

Lightweight means that a decent level of automation is also required – being able to repeat something that involves twenty manual steps which could be forgotten is not scaleable or useful.

Putting it together

After writing it all down draw your processes end to end.

  • Do they match the constancy of purpose?
  • Can they be done within one team?
  • Do they pass through many teams?
  • Have you documented any data transformations needed when passing between teams or roles?

Once you have this you need to start building the second loop. Constancy of purpose should be giving you an idea of the value each process and step adds, lots of data transforms means you’re involving lots of different teams or functional areas – is that a good idea?

The next thing you need to do is work on a consensus about how things are done.

Then it’s time to start thinking about the second loop. If you do some googling you will find that there are even more.

The end goal is not functionality

Many years ago I read an academic paper that talked about why the Apple Lisa (you won’t remember it) failed. The paper put it down to the three Fs: functionality, functionality and functionality. This is quite a witty point, and indeed the amount of new startup websites one sees that don’t actually say what the app is supposed to do because they’re trying to persuade their confused potential customers to tell them what they want is a testament to the idea.

There is a problem with this view, though. People who use your stuff don’t want functionality, they want capability. They have a problem they want to solve and you’re offering them a way to either get rid of it or manage it down to tolerability. The Lisa didn’t allow people to do things they needed to do. The Mac, later, was originally the only platform that did desktop publishing well and reasonably affordably (compared with the price of a conventional printing studio).

So, when you’re doing your interviews and looking for user pain you can meet you need to understand that pain isn’t functionality, it’s capability. So even making a hard thing a little easier to do is worthwhile. This changes your perspective on what you can and should deliver.

As with a lot of engineering type problems, take a step back, think about what the wider picture is. It’s always a valuable exercise.

Book Review: Black Box Thinking: Matthew Syed

Black Box Thinking: The Surprising Truth About SuccessBlack Box Thinking: The Surprising Truth About Success by Matthew Syed
My rating: 5 of 5 stars

I found the ideas in this book really resonated with my experience in a number of industries.

Syed’s thesis is that we live in a society that usually plays the blame game when things go wrong. He contrasts this with the ideas used by the aviation industry where mistakes and errors are seen as problems with the system, and not individuals. Because he draws from this industry it uses the thinking behind  black box from aeroplanes as the central metaphor.

If something goes catastrophically wrong it’s not the pilot that’s to blame but instead the system as a whole that allowed the mistake to happen. This systems approach is why there are so few aviation accidents.

In the early parts of the book he contrasts aviation errors with medical errors. Doctors are trained not to admit to failure and be “experts”. So instead of learning from failure there’s a culture of “accepting the inevitable” and not trying to stop things happening again. Indeed there’s a really silly idea that doctors’ training makes them infallible.

Syed gives an account of one awful medical accident where a young mother ended up with brain damage because her throat closed up under anasthetic and the doctors spent so long trying to insert a breathing tube they didn’t do a tracheotomy in time. It turns out that under stress people lose track of time, and can end up going past hard limits (like how long someone can survive when they can’t breathe) with bad consequences. In this case the poor woman’s husband was an airline pilot and he didn’t accept the “one of those things” arguments.

Eventually after a fight the husband managed to get to the truth of what had happened. Instead of being bitter he shared what he had found in medical journals, made sure that the simple things you can do to make sure your intense focus hasn’t made you blind was wider known. For example, you can make sure that all people involved can call for certain things, hierachies need to be questioned. Two people work on a problem, but one of them stays out of the decision making loop and the cognitive stress so they can see what’s happening and call time.

This information has saved a lot of lives. The woman’s husband has been written to by many surgeons all over the world. There are examples of how this telescoping time phenomenon caused crashes of aircraft, but that industry changed the way it did crisis management to make the problem far less likely to occur.

It sounds simple, doesn’t it? Learning from failure has the status of a cliché. But it turns out that, for reasons both prosaic and profound, a failure to learn from mistakes has been one of the single greatest obstacles to human progress. Healthcare is just one strand in a long, rich story of evasion. Confronting this could not only transform healthcare, but business, politics and much else besides. A progressive attitude to failure turns out to be a cornerstone of success for any institution.

Next he looks at experts. If there is no feedback loop after they qualify as experts then they do not improve. Without being able to measure your success you are stuck, probably making the same mistakes over and over again.

If we wish to improve the judgement of aspiring experts then, we shouldn’t just focus on conventional issues like motivation and commitment. In many cases, the only way to drive improvement is to find a way of ‘turning the lights on’. Without access to the ‘error signal’, one could spend years in training or in a profession without improving at all

And of course failure is necessary, as long as systems are in place to learn from it:

beneath the surface of success – outside our view, often outside our awareness – is a mountain of necessary failure

This goes much further than the old saw about learning from failure. Syed’s argument is that you must be systematic about it and not blame individuals for systematic failures. But also it is important that individuals take responsibility for what happens and own up when things go wrong. Without this there can be no learning.

Another extremely interesting thread later in the book is when he picks up on marginal gains. This is a way to find improvements and is used by teams like the British Cycling team who were so successful at the Rio olympics. In short: everything matters every detail that can hold back success or performance is important and when you address them all they compound together to create an unstoppable chain of improvements. Small gains, marginal gains become the root of great success.

Marginal gains is not about making small changes and hoping they fly. Rather, it is about breaking down a big problem into small parts in order to rigorously establish what works and what doesn’t.

They use the first test not to improve the strategy, but to create richer feedback. Only when they have a deeper understanding of all the relevant data do they start to iterate.

see weaknesses with a different set of eyes. Every error, every flaw, every failure, however small, is a marginal gain in disguise.

I heartily recommend this book, it’s easy to read and the stories make the examples stay in your head. I hadn’t heard of the marginal gains technique but will be using it myself.

View all my reviews

Can’t install postgis on PostgreSQL 9.4 after running apt-get postgis

I found this question on stack overflow and answered it – then the pedantic site moderators deleted it. So here is the answer again.

On a clean install of postgres and postgis you will see this error because postgis doesn’t have the dependency to pull in the shared library. When you go into psql the command

my_database=# CREATE EXTENSION postgis;

returns:

ERROR: could not access file “$libdir/postgis-2.1”: No such file or directory

found I also needed

sudo apt-get install postgresql-9.4-postgis-2.1

I found it by looking at

apt-cache search postgres | grep gis

Review of Antifragile: Things That Gain from Disorder

Antifragile: Things That Gain from DisorderAntifragile: Things That Gain from Disorder by Nassim Nicholas Taleb
My rating: 5 of 5 stars

I enjoyed this book but suspect others might find it a hard read.

Taleb points out that this is the last in the trilogy that includes Fooled by Randomness: The Hidden Role of Chance in Life and in the Markets and The Black Swan: The Impact of the Highly Improbable. He also says you could read the chapters from all 3 in any order if you want to. I got a lot out of the three books but reading them is a labour of love. If you want a quick textbook style explanation go looking for his other more technical works.

You first have to get to grips with his literary, raconteur (although he would prefer flâneur) style. It’s not a textbook and in fact the heavy number based, more academic, arguments are in other documents you can get from his website. Some readers find the style hard to get to grips with, but I like it.

He also makes words up, like Antifragile itself, sometimes for effect and sometimes because he doesn’t feel there is a word that works. I like this playing with words, it amuses me, I play with words a lot myself.

The core idea in Antifragile comes from the ones he explores in the other books. In essence we live in a world that isn’t dominated by the comforting shape of the normal distribution. There are events that are rare but will happen and they completely drown out the rest of the things you come into contact with the other 99% of the time. This is why the Black-Scholes equation is bunk, you can’t take a derivative of a catastrophic disconnect so the risk it gives is useless, without perfect knowledge of the future. It does work when things are stable, for example I’ve seen it used in estimating risks in queues in development processes, but as soon as you are open to catastrophic black swan events the figures it gives are meaningless, in fact dangerous.

If you have antifragility then you can take advantage of these sharp disconnects to make you richer, stronger or happier. He uses as examples where systems become stronger when challenged. Of course these are used mostly metaphorically to show that it does happen out there in the real world.

The bit that had me laughing out loud was his description of the “Soviet-Harvard illusion” where people assume that things that happen together have some connection in reality. He gives the example of a Harvard professor going and lecturing birds on how to fly because they wouldn’t be able to without the series of lectures and growing his or her own sense of importance because of it. This is his beef with academic theorists, none of their ideas have weight in the real world, and if you look at how we actually do things and what the real risks are when you take black swan events into account.

I also liked the Barbell concept, put most of your risk into very conservative places, and a small amount in very high risk (as in the risk is shape is a barbell, yes?). If the high risk pays off all is good, but you’ve not lost much if it doesn’t pan out. On the other hand, most of us go for “medium” risk, which is in fact not medium at all because of the propensity for the economy to have black swan events. This is in fact the riskiest long-term strategy and we’ve all bought into it because it’s been mis sold and feels safest when times are calm. It isn’t because long term times are not calm and never will be.

Similarly, take things like climate change, or fracking. The onus isn’t on the people who worry about it to prove there is a problem. Put simply if you start doing something novel or unusual you must prove it doesn’t change things for the worse – the onus is on the new to prove its safety. We already understand the old works fine. Again, this is about unknown black swans waiting for you.

So, if you want to meet Fat Tony and a host of interesting characters who live in this place, read the books. But remember – they aren’t text books, but a literary exploration of some interesting ideas and you have to be prepared to walk a while with Taleb while he tells you his stories.

View all my reviews

Towards a Lean Manifesto

Towards a lean manifesto

Tiny steps make things of beauty

Over a hundred years ago somebody thought it would be a good idea to make everything in tiny little steps. Each step could be done by an idiot pulling a handle. The hard part of the work was working out what the small steps were and how to put them in an order that would produce something as useful and complex as, say, a car. This somebody was Henry Ford.

If you want to make several things you need to reconfigure the more complex machines. Because this takes time and effort you will try and make sure you make lots of pieces with the complex machines before each change over. If you suddenly need to make something else because of a change in demand, the demand will have to wait.

If one of the parts of the long process to make something useful starts making things that are defective in some way the idiot will carry on pulling the handle, because that’s what they are paid to do. They aren’t paid to think or even notice that pulling the handle is making broken things.

You have to ask yourself: why do people think this is a good idea? and, who is the idiot? The person pulling the lever is at least getting paid to do it.

Before this components were made to approximate dimensions and a factory consisted of highly skilled workers putting these parts together and fettling them. Every car was in essence hand made and no two were identical, parts could not be swapped without refitting. This is where the job title of fitter comes from originally. It was a highly skilled craft job. Ford changed all this and made his cars essentially the same, and also made them so that they could be maintained by someone with a decent set of teeth and a hammer.

In essence the discovery that dividing labour up, and making the parts produced by that labour much more standard, and placing them on a moving conveyor where they are assembled in order was the beginnings of what we call mass production now. It didn’t need the highly skilled workers. That said, even Ford’s factory still had people who had the job of fettling cars that weren’t quite right, and they used to ship ones that had faults. It didn’t matter as long as they were being shipped. If you could read and were in possession of the kind of tools you would find in a farm you could get it running and the price was right.

By the way, the assertion that Ford said you can have any colour you want as long as it’s black is a myth. The early model T’s were dark green and navy blue.[^ModelT]

Beauty turns ugly

If you worked in one of these factories you were dehumanised. You were pulling a lever and didn’t need much education. Ford famously paid his workers well. He could afford to because he was producing so many cars at a relatively low cost. He still had a lot of trouble getting enough people to do this, and turnover was quite high.

The highly inflexible approach to building the cars also meant that producing new models was really difficult once the line got rolling. The entire assembly plant had to be reconfigured for every change. A new model could mean months of close down and disruption and then trying to reassemble the workforce you’d laid off.

Inflexibility had another cost, if sequences of operations were done in a way that was suboptimal changing them was nigh on impossible. If you multiply something that even adds a few seconds to a process by several million cars then this becomes a significant cost you can’t do anything about.

Workers could be added and removed depending on the demand, and were regarded as disposable and replaceable, which is how a lot of capitalists still operate.

Once upon a time in the East

After the second world war there was a small company called Toyota trying to make cars for the domestic market in Japan. They didn’t have a lot of capital to buy many different machines, and neither did they have lots of orders for the same kind of car. They also had to cope with a law that said they weren’t allowed to lay people off if they had no work for them. This meant they needed to be able to produce different types of car from the same set of equipment without a lot of ceremony and also that they needed to keep their workers happy and productive for 20 or 30 years.[^MachineThatChangedTheWorld]

Their productivity was about a third of that in Germany. This was in itself a third of the USA. So around 10% of the most productive people around.[^Bryant]

They tried many things and managed to get changing the complex machinery that stamps out body parts down to a few minutes when on paper it should take hours or even days. If things needed to be moved around to make the line run a little quicker they were. Workers were expected to become problem solvers, not beasts of burden. If pulling the handle meant creating defects you pulled a different handle that stopped everything until the problem was solved.

You got paid more depending on your seniority, on what you could bring to the organisation, not on your job title. The more senior you are, the better problem solver you have become. There’s also little point in jumping to another company, you will be paid a lot less and have to start again. The Western approach of having a narrow specialism and then progressing from junior to senior while hopping from company to company did not apply. Taiichi Ohno, one of the shepherds of these ideas, talked about creating work that’s fit for humans. This makes machines the slaves and humans the masters, which is the way it should be.

By the late 1970’s the productivity picture went entirely the other way.

Efficiency

Mass production is obsessed with efficiency, doing each step as fast as possible. Things that are relatively slow end up with piles of stock upstream of them. Things that are relatively fast end up with nothing to do. If each step is run as efficiently as possible you come to find that the system as a whole is anything but efficient. There’s a term for the efficient approach: local optimisation, as in each person optimises what they do with no regard for the system as a whole.

Instead Ohno traded efficiency for effectiveness. Not just doing but doing the right thing. Only making things that need to be made. Effectiveness is making enough to meet the needs of the business with a cadence that minimises the amount of defects, of stopping when defects are found and finding the root cause of them.

Piecework, which is where workers are rewarded for making a number of pieces, is an early form of trying to use targets to make things work properly. But if you have to make 15 widgets to get paid a living wage by the close of the working day you will make them any old how, and maybe try to make some more if you will get paid for them. So the human being will keep pulling the lever because that’s what they’re paid to do by the idiot who owns the lever.

Quality

After something has been made it’s a bit late to see if it works properly. If you have thousands of them and they’re all broken it is also a very expensive mistake. If you optimise each step for its own sake you this is where you end up. Everybody has been doing their best by staring at their feet instead of looking at how what they do fits with everyone else.

If your culture is a problem solving culture, where everyone is listened to, then good quality will just become something you are, rather than something you do after the fact.

Silos

It’s easy to point at manufacturers of stuff and say they should eschew local optimisation in their production processes. It’s also easy say that there are benefits in pulling groups of similar products together and creating systems that can switch between them easily.

In knowledge work we can say that this doesn’t affect us because we don’t make a physical product. We can make sure that our department is as efficient as possible. We can incentivise with targets and arbitrary bonuses.

If you take a few steps back from this and think of the departments that make up an enterprise of any size you will find that you have processes that pass information between them. You will also find that there are places where this information (or stock, in fact) is gathered and starts decaying and becoming a problem. It’s the same issue of local optimisation but hidden away in the rough and tumble of operating a company; you have the departments because that’s what everybody else does. What you know, of course, is that everybody else is also struggling with quality and getting things done but just accepting it as part of the status quo.

In Lean Thinking Womak and Jones talk through a case study of a manufacturer of stretch wrapping machines. They attack the production problems and get a 40% (and more later) increase in capacity without employing any new people, they shrink their lead time from months to less than a week. Because they no longer needed to hold onto lots of unusable stock they did this without needing any more space. They no longer have to ask people to expedite orders because all orders are being delivered on time or early, the average number of defects falls from 6 per machine to less than one. They pay their workers well above the industry average because they are so much more productive so everybody wins. All of this is amazing. Then they realise that their sales process takes up to six weeks.

You need to turn the searchlight on the whole enterprise. Just because you make something intangible doesn’t mean you shouldn’t question your thinking and how you do things. In some ways it’s much easier for you because you haven’t got the expense of moving heavy machines about to make your production lines more effective, in others it’s bad because you don’t see the pain. A pile of physical things costing you money and decaying into worthlessness is easier to see.

The silo metaphor is used to describe the separation of concerns into their own departments. People work for departments, not the enterprise, and their objectives align with that narrow view. Then of course we can start playing the blame game. A siloed organisation is maladaptive and usually blind to its own faults. This is one of the reasons small companies are much more nimble – they have less silos, less accretion of bony substance to choke on.

For those readers who work in the IT function, this is why you are always the blame monkey: you have too many masters. More enlightened organisations that are now breaking their offerings down into product groups or using product focussed multi functional teams that have the job of meeting specific customer needs directly without having to go and beg for resources from several different departments. This usually means the middle management tier are no longer required. Their job was originally to co-ordinate between the silos. When you have no silos you don’t need to negotiate and there are no empires to defend. These companies are winning, they actually manage do do the more with less mantra without it being another corporate self delusion.

Value

The thing to pursue in your quest for effectiveness is value. Value is that thing that your customer has in their hand and wants to pay you for. Whatever you do, this is what matters. If you look at what you do and can’t see value then you know that you should stop doing it.

One of the first things that you need to be able to do is see what your constancy of purpose is – whose problems do we solve, how, and what makes us special? This gives you your value proposition. Then you can use this to examine what you do and throw out all of the things that don’t fit. It’s very liberating.

The British Olympic rowing team spent 4 years with the mantra does it make the boat go faster? They took this single minded approach after losing by a few hundredths of a second. It meant that every decision they made suddenly became easier, there was no deed for debate. If it goes faster, keep doing it, if not stop doing it. Very simple.

Flow

A customer expresses a need. The steps you follow and information passed between people to meet that need are the flow. The flow is different for each organisation, even in the same industry each will have its own history and steps it follows when the people working there worked out how to do things. This is why best practice is often a myth. Unless of course that best practice is a way of thinking about customer needs.

You can go on waste walks, where you follow a customer need for value all the way through and look for where you are doing things you shouldn’t. Ohno had three things he used to say you need to do when working to improve things.

  • Go see
  • Ask why
  • Show respect

In the West we are awful at showing respect and we often stop at the first answer when asking why. Ohno developed a technique called the 5 whys – you keep asking why until the primary reason is uncovered. It may well be a lot more than 5 times, or only two or three. One of the main purposes of this is to get past the blame game and off to the root causes of problems.

Showing respect means that you should never treat the people doing the work like idiots or fools. They are doing what they were told to do, and if there is anything they perhaps shouldn’t be doing it’s roots lie in how the work was originally organised. Quite often they know a better way and can tell you what it is once you remove the earplugs of arrogance and compromise.

Dirty flow

If the next step in a process can’t happen because the information or thing passed to it is wrong or incomplete then you have to go back and fix it. The flow is dirty. This is often a place where you find hidden costs, and is often why silos are so bad – the problems inside each department aren’t obvious because that department works well in its own terms. It’s only when you look at the interfaces you start to see where the pain is.

This is why the purchasing department buys cheaper tools that make the creation of defective products so much easier. This is why competing on costs is an insanely bad idea.

Visualise the work

One of the most useful and easy things you can pick up from this is the Kanban board. Kanban just means something like sign in Japanese. The boards are used so that everybody working can see where the work pieces are. They also limit the work in progress (WIP), if you hit a WIP limit you know there’s a problem before it causes you a lot of pain.

You can create Kanban boards for every endeavour, I use Personal Kanban, which in essence consists of the sentences visualise the work and limit the WIP for most of my work. I have to switch between projects so I have many boards.

It doesn’t have to be a board, people have used mind maps. When I ran a workshop recently someone just drew a picture with some illustrations of that things looked like and some arrows. The important thing is that everyone can see what’s going on.

Compromise is bad

Good enough isn’t good enough. If you only expect 80% from a process or person then you will be lucky to get 50%. Always striving for 100% means you can uncover ways to improve. Again this means leaving egos at the door and questioning everything.

Goldratt’s book The Choice talks about his approach to problem solving. He was regarded as a genius by many of the people he worked with because he could see problems with great clarity.

Goldratt would look at very complex seeming systems in manufacturing, retail or project management and seem to be able to get to the kernel of what they are about. In the book he says that it doesn’t matter what the veneer of complexity is (one company profiled makes hundreds of products with a bewildering variety of products and variations) you can find a couple of simple levers that drive the whole business. He insists that it’s not genius, it’s taking the time to see clearly and not compromising on good enough.

He said was that we become habituated to difficult problems and start living with them. This cognitive blindness means that usually takes an external consultant or new manager to point out there is a better way. In order to reduce conflict we compromise with whatever’s causing the pain and then act like it can never change. You should never listen to or be taken in by compromises like this. Just as there is always a simple lever, and there is always a way to address problems where everybody wins, you just have to push really hard at it.

Typically his consultancy would find huge savings by applying their knowledge of constraints for the client, and then find at least the same again by helping the client work out problems with suppliers and customers, and then find even more by going around the loop again. Never believe you’ve finished in the process of improvement. Today’s great idea can work for a while and then become a bad idea. Things change – it’s the only certain thing in life.

Ohno would strive to improve product lines that were being phased out until the very last day, you can still learn from the improvements and why not keep your money in your pocket?

Slack

If you work for an efficient organisation and dare to be stood around you will be told to find something to do. Effective organisations have slack, where people can think about things and also where they are available to meet short term needs or swarm on problems. In finance they talk about liquidity, which means that not all of the resources are committed. Some of them are kept back so that they can be used to overcome short term problems in the market. You can do the same thing with people. But, please, stop calling them resources. Staplers and pencils don’t have initiative or drive, they are very bad at problem solving without the help of a friendly human.

Experimentation

Not all of the changes you try will have a positive effect, sometimes they will make things worse. This is part of discovering what works. You need to have experiments that can fail without destroying the company. You also need to say what success and failure look like before you make any changes. You may not be able to say anything quantitative, or only guess at an improvement. This is fine.

We have been involved in a lot of software changes where all we were supposed to do was ratify a pet idea coming from the board. So there wasn’t any point in calling it an experiment or even collecting any data. If you are doing this just be honest about it or it devalues the process.

Small batches

An accidental product of the Toyota way of organising work was they used small batches to make relatively small runs of specific cars. Then they realised that this meant they were only making what they needed to without having to create a lot of stock and the machines were running at the right cadence to deliver.

Jim Benson, one of the originators of Personal Kanban, says that big batches can kill and I agree with him. They take up space, they decay over time, and if you have introduced defects you end up with lots of them instead of a few you can throw away or recycle.

This also means that you probably don’t run into constraints in the way Goldratt talks about, where they control the whole process. Small batches and reasonable buffers stop constraints from … constraining you. That said, understanding Goldratt’s ideas gives you some more tools if you become stuck.

Buffers

If there are a lot of steps to deliver value you can have one of them stop working for a while because stuff happens. You need to be able to keep moving, so having a small buffer upstream of any complex or expensive processes means you can keep running while you sort it out. These processes are your constraints.

Projects are evil

If you want a can of beans you can go buy one. If you want to do something new or different with your customer you can’t go buy that capability. Projects start from the false assumption that you can buy things like market share or better quality. What you can do is experiment and overcome your ignorance, then you can see how the knowledge you have gained can be used to improve or avoid mistakes.

Budgets are evil

Budgets buy projects that buy beans, not capability. Instead a pool of money for experiments is needed. This mitigates risk and makes sure you aren’t throwing your money at white elephant projects that won’t add value.

So a budget becomes how much you are willing to spend on experiments discovering new markets or enhancing what you’ve already got. You can stop spending when you realise it’s not worth it any more and do something else. This is also very liberating.

It does mean that the simpleminded invest a couple of bucks and get four bucks back model doesn’t work. It used to in the days of massive brands and lack of choice, which ended in the 1980’s.

A Lean Manifesto

We acknowledge that lean has become a brand. Despite this we feel it is less compromised than agile has sadly become, and also has a much broader remit that software alone.

In brief:

  • Preferring work that’s fit for humans over humans fitting work
  • Preferring shared visualisation over unread documentation
  • Preferring rational continuous improvement over slogans
  • Preferring testing over guessing and waste
  • Preferring 100% over good enough
  • Preferring collaboration over winning
  • Preferring feedback over fire fighting
  • Preferring value over bureaucracy

Everything we do is contingent on discovering a better way

Theory comes from doing and seeing

Theory is contingent on discovering better theory

The only constant is change

If we learn we do not fail

Deming and the 95%

A few people have commented on twitter about Deming’s contention that 95% of what you see is down to the system and only 5% is down to the individual. They keep asking where he got the 95% from.

He used to run a course, which is still available as a book: Four Days with Doctor Deming. At the very beginning of the course he invited the participants to play a game with red and white beads. There’s a youtube video describing it here. The you pick up beads with paddles, the workers are rewarded for picking up white beads and punished for red, even ‘fired’.

When you look at the numbers for the game it turns out that the workers have no control over the physical process of selecting the beads. In one trial someone may do really well, and in another badly and get ‘fired’.

In statistics we use confidence limits. When we take a new measurement you look at it and can perform some analysis to say how likely it is that the measurement falls inside the existing set of measurements. Typically we use the 95% confidence limit to say that there is a one in twenty chance that the measurement doesn’t fit what we were expecting. If it falls inside then we have what’s known as the null hypothesis, that there isn’t any noticeable effect for that particular measurement.

Deming’s point was that in most processes any effect that the individual may have is swamped by the system their are part of, in fact the variability they cause is just part of that system overall. So if measurements are falling within the confidence limits you can’t point to one person over another and say they’re better or worse.

So changing things, for better or worse, mean you have to change the entire system. It’s an iron rule, the system always wins. No amount of arbitrary targets, shouting, or wishing can change this. I cover this in more detail in the video talks on my consultancy website.

A life without the standards police

In any complicated endeavour there are some things that can really cause you to get a little stuck and working on stuff that’s just not important. When working with teams it’s pretty important to ensure that each team has a similar approach to things like how they realise architecture, lay out applications and code, which of the excellent JavaScript libraries they use and so on. Otherwise it starts to create silos and mean that there is waste when you try to manage the flow across teams.

One way of doing this is to have some kind of gatekeeper who inspects everything and makes sure it complies. This is the standard backwards command and control way of assuring quality: do the inspection after you can perform any intervention that might resolve the problem. It also means you have guaranteed that there will be a bottleneck. Of course, you can have it in front of any work, where there is some kind of oversight. This just means the delay is upstream of any value instead of down.

Instead, there’s an old trick, which is to have swat teams. These are teams made up of experienced individuals who know each other well and work together. Their job is to start new projects, but not finish them. They also work with teams that are stuck or lagging to help them get past whatever problems they have.

The swat team hands off to the less experienced team that will finish the job, it means that there is already a body of knowledge of how we approach things, what the architecture is, etc. built into the project. Of course, this will require pairing and patience. But it means that work gets started well, in a standard way, with all of the levers for continuous improvement in place, by people who know how to do this. Then it’s picked up by others who learn what the standard approach is, without a gatekeeper and a lot of bureaucracy.

This is a team version of staff liquidity, which is discussed in the Commitment book. Staff liquidity uses experts’ knowledge and skills in such a way as to make sure there is enough slack to deal with emergencies and still get the work done.

Next release of Unicorns in the Mist, speaking tour of the UK

I’ve just released another version of Unicorns in the Mist and working on building my consultancy portfolio. Have a look at FJFDIDM to see what I’ve been doing. All of the video content is here.

Speaking at Lean Agile Scotland http://www.leanagilescotland.com/speakers#francisfish – this is September 19-20 2013 in Edinburgh. The talk is based on the new material presented in the September 2013 release of the book.

Speaking at Agile Cambridge http://agilecambridge.net/ac2013/programme/ this is September 24-27th, 2013 at Churchill College, Cambridge UK. This is an updated reprise of the “it’s not your fault” talk on the consultancy site. I believe that InfoQ will be filming the session and I’m proud to follow in the footsteps of many people I admire.

I’m attending, but not speaking at, Design it, Build it http://www.dibiconference.com/ which is 7-8th October at Gateshead, UK (the 8th is my birthday).

I’m also going to Lean Conf 2013 http://www.leanconf.co.uk/ which is 26-27th October in Manchester. So, if any readers want to talk to me please seek me out if you are at any of these events, they’re all worth going to.

Indistinguishable From Magic: Shaming, Social Media, and the Impact of Online Power

Indistinguishable From Magic: Shaming, Social Media, and the Impact of Online Power