This is the first in a series of essays about what it means to be an Agile Practitioner, at least from where I stand. Agile has a venerable history and started from a sense of dissatisfaction with existing practices and methods. I’ve heard lots of people say that they’ve “done Agile” and it didn’t work. I overheard a project manager saying that it doesn’t work, or only works in certain situations, to the sage nodding of the guys around him.

I wanted to shout oh yes it does!

But there are a lot of things it isn’t:

  • A set of rules
  • Diagramming conventions
  • No documentation vs too much documentation

What I’m reaching for here is that it’s a state of mind. This is hard to explain to people. In essence you can do what Burger King once said to Tom Peters Even when we did it right the first time it was still pretty average.1 If all you do is follow procedures, treating people (particularly that stubborn breed of cat called a Developer) like they can be programmed then you get something that is really average, the sum of mediocrity. Sure, it will probably work, but be buggy and poor – so what? 

Agile started as a movement when a group of practitioners got together and wrote the  Agile Manifesto2. In essence, put the customer at the centre of what you do, and have an approach that lets you change your mind easily without lots of documentation and other things that create inertia:

  • Individuals and interactions over processes and tools
  • Working software over comprehensive documentation
  • Customer collaboration over contract negotiation
  • Responding to change over following a plan

That is, while there is value in the items on the right, we value the items on the left more.

This has practical implications for anyone who wants to write software, or do anything involving thinking and planning with other people. One of the eye openers for me was where the principles section of the site says Welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage. This meant that change is not the enemy, a feeling that had been driving me on for a long time. It also means that change requests are not an excuse to charge exorbitant sums of cash. I had worked on projects where you walked into a room the size of a football pitch and saw a number of people in a dozen teams (one project I worked on had 200 people in “the room” and didn’t include another 50 or so in portakabins outside and dotted all over Europe). You knew you were doomed when you walked through the door. The enemies of productivity, of getting things done, are (in no particular order):

  • Bureaucracy – documentation for its own sake
  • Large teams with no focus – documentation as a poor communication medium
  • Process replacing understanding – documentation, anyone?
  • Meetings “explaining” (i.e. mandating often inappropriate) architecture – pictures and discussion about fit, anyone?
  • Meetings about meetings (this does happen)
  • Unfocussed lone consultants on site saying “yes” to the client but not feeding anything back3
  • Progress reports when the project is less than a day old – “it’s procedure” (oh, F-off)
  • Whole team meetings stopping coding
  • Meetings that could be summarised in a two paragraph email
  • Control freak meetings (as in you have to explain things to someone senior who won’t understand what you’re doing over and over again) – these are also sometimes massaging the Project Manager’s ego meetings.

If you want a refreshing take on meetings, read Rework4

You can get things done with a traditional large team (or in practice set of teams mediated through yards of documentation and meetings) but it’s hard to do well, and expensive too. I used to work for the database company Oracle and we were very good at delivery (then, no idea if it’s still true). On site we would always do our best to make sure that we left the customer with something they could use. A common complaint from the on site team was that you’d spend the first four months of a six month project writing about what you were going to do in an incomprehensibly boring pile of document templates generated from Microsoft Word macros (the macro suite was called Oracle Method, funnily enough). Piles of words on top of each other, with the occasional code fragment – sometimes there were so many code fragments you could  have pasted them in and probably had some working software. I remember starting a rebellion and putting some before and after pictures in the documents and losing several yards of verbose technical text that people who didn’t have English as a first language would really have struggled with (and there were lots of them). It followed the utterly mad waterfall (don’t forget that each of these phrases has the words and sign off at the end – usually four signatories):

  • Functional specification
  • Technical specification
  • Design document
  • Implementation document
  • Some code actually gets written
  • Unit test documentation
  • Functional test documentation
  • User acceptance test documentation
  • System test documentation
  • Fluffy cloud documentation
  • Finally we can go home and cry documentation (ok, I’m being silly now)

If something has been signed off then you have to go into Change Control – oh, the pain, the meetings about nothing. The cost!

We rebelled and merged functional and technical on one project. This saved us over a month of elapsed time. Consultants (or at least their management and sales force) and governments (tasty bureaucracy, yum) love this approach because you can bill for it and it looks like you’re doing something. But you aren’t. It’s bullshit business (as in busy-ness) and doesn’t get much useful done. One of my colleagues said: I didn’t join Oracle to spend all day using Microsoft products. This was terribly funny, because Oracle corporately saw Microsoft as the enemy, the devil incarnate, and yet they’d invested in this sub-product written in Word Basic that generated all these templates and employed people as auditors running around making sure they were being used. I’m finding it hard not to write swear words here, it was unbelievable, it was insane and we did it because everybody else did it. Did I say I joined Oracle from working at KPMG Management Consultants? This is not a critique of Oracle, it’s a critique of everybody around at the time. When all you have is the Waterfall approach then everything turns into one.

So I read the Agile Manifesto as a breath of fresh air against all this crazy shit. And yes, shit is the right word, honestly.

There was some confusion in my mind because I was switching to using Object Oriented techniques, e.g. learning to write use cases and follow slightly less crazy methodologies like RUP5 – which is still mind-bogglingly bureaucratic and was bought by IBM (surprised???). RUP has some good things in it, like the artefacts (i.e. documents of one form or another) are not mandatory and you pick the ones that work for your project. Also you do a risk analysis on project inception and do the risky things first, this means you can make sure you deliver something useful or, worst case, discover that you can’t deliver anything within your budget before spending a ton of cash on failure. So it’s definitely better than pure consultancy-driven waterfall. I never really got into UML (oh, the UML, sorry) because I was already a hard-core user of Oracle Designer and, if you start from the database like I was taught UML is an inferior way to describe relationships (disagree if you want to, like I care, honestly). UML works well in those insanely large projects with bazillions of dependencies and abstraction layers, but of course that’s where it came from.

Thing is, in 2002-3 North West England, as an Oracle PL/SQL specialist learning and getting certified in Java, it was impossible to find people who were using Agile. Whether it was XP, Scrum, Kanban, or roll your own. So I made an agile approach of my own work, questioning documents where code will do and drawing pictures instead of using lots of words. Wherever possible delivering working software. Testing software in the crucible of real people actually using it. 

I switched to using Ruby on Rails (RoR) after falling out of love with Java. I think Java is a language tailor-made for bureaucracy: curly braces everywhere, endless cruft from limitations of the language, no syntax for common stuff, use yet another class in the standard library, lots of useless layers in J2EE, “agnostic” approaches that mean configuration options coming out of your arse, 20-minute Ant builds, strong typing, everything verb has to be wrapped in a noun – no wonder corporations and consultancies love it, the bloat is built in right from the start. Again, Ruby set my mind free to worry about delivering working software. The other thing from RoR is I caught the test-driven bug. I tend not to write software without tests anymore, at least for production code. I’m not sure how purist Agile this is, except that it allows you to change your mind without breaking things which is a key need for Agile thinking. I recently put one of my own projects on pause while I got the tests working after migrating to Rails 3. Yes, it cost me a couple of weeks part-time, and no, I wasn’t going to compromise.

I also like RoR because it starts from the database. Code is built using generators, and some tests are also built in. But you start from a data-centric place. Not some pretty design done by a designer, but from the data. I believe that working software comes from data. As in, understanding the data flow means you understand the business you’re working with. But on the other hand, I’ve also worked with beautiful data models from Oracle Designer Generator that created software you couldn’t use because it was technically correct but very hostile. As usual the middle way (which appeals to me as a committed Buddhist) is the most sensible one. Rails makes it easy to find the middle way between data and design because it doesn’t sweat the simple stuff, and lets you be very strict about how your model is used.

So, I will move on and start the next essay in this series in a few days. Talking about the Agile heart, as against the Bureaucratic Tendency – you can measure bureaucracy so it’s popular (ISO 9001 certified shops take note). You have to do agility, and think, and care about what you do and how your customers feel, so it’s as much a feeling as anything else. A feeling that leaves your competitors behind, though.

1 Tom Peters Circle of Innovation, 1997

2http://agilemanifesto.org

3 Anyone else worked with Earnst and Young back in the day?

4DHH and Jason Fried, Rework, 2010

5 Rational Unified Process. Methodology used to sell the Rational Rose modelling tool. I think the Rational was dropped when IBM bought it.

Imported Comments:

Ralf Westphal

Being agile as a state of mind. Yes, I guess that´s it. Even though I call it a lifestyle I guess we mean the same: agility is not something you switch on while you´re in a software project meeting.

I see a lot of people claiming to be agile – and when they´re buying a coffee machine they browse the internet for hours, read reviews on end, check every store in tone to know exactly what´s the best offer. To me that´s a contradiction and I always wonder how agile their projects really are.

If someone claims to be agile and it does not show every minute of his life… then something is missing.

As for “As in, understanding the data flow means you understand the business you’re working with.”: That´s a great observation. But I´d like to ask: And then? What do you do once you see the data flow. How do you get from there to code? By translating data flows into domain object models?

-Ralf Westphal
http://geekswithblogs.com/architectsnapkin

Francis

Thanks for the comment Ralph!

I agree the data flow comment isn’t very detailed – it was more of a
reaction to when I see designers go mad with half understood
requirements and we end up with a mess. The data flow, for me, always
tells the truth. But then I’m an old database hacker 🙂