"meat & potatoes" agile talk (now with built-in tangents!)

I recently gave this tech talk, basically "meat & potatoes" agile, for the team I'd just started working with.  There was a bit of extra emphasis on continuous integration, as at the time I was persuading on of the projects to set up a build box to generate their deliverables (as opposed to crafting on somone's desktop).

During talks, I sometimes worry about going of on tangents and diluting the message. So this time experimented with baking two tangents into the talk itself.  And it kind of worked!

The two tangents were:

  • IT projects are (relative to other projects) often very expensive, and IT people well compensated (relative to other people, who do _real_ work, like teachers, journalists, firefighters, etc.).  We need to respect this.  One way to respect this is to work hard to deliver actual value to the customer as quickly as possible.  This is a lot of what agile is about.
  • I make a mild joke, contrasting the points of view where new stuff is considered hip & innovative, and old stuff antiquated and out-moded, and the point of view where new stuff is crazy and untested, and old stuff is classic and reliable.  I try to use the joke to suggest that keep perspective, avoiding the hype cycle, as well as excessive caution, to choose your practices with pragmatism in mind.  With agile crossing its first decade, there's no shortage of dogmatism, so this is important as well.

--


Capistrano, Puppet, and Chef

Our team gets together at the head office regularly for a "Tech Night", with pizza, beer, and casual presentations.  I put together this brief talk.
As I was writing up the slides, I realized I was explaining more where the area of app deployment and config management has come from, and where it is going. Its all still changing, but in a good way!

Studies Show Agile Teams Need a Little Structure Too

Discussing "hierarchy" in relation to agile teams is generally pretty difficult.  Agile teams tend to pride themselves on being flat, self-organizing, and via all-in retrospectives democratic, though perhaps with a "benevolent dictator" or two to move things along.  But within any group, there are implicit hierarchies, and often, power mist.

But when roles and responsibilities get unclear, does it really matter?  I've always thought it was important, but something you could work out over time.  I also thought that as the team became more familiar with each other, things would get better.  I'm not so sure about this now.

A recent working paper from Harvard Business School, Disagreement about the Team's Status Hierarchy: An Insidious Obstacle to Coordination and Performance, "used survey data from a longitudinal field study of 89 consulting and accounting teams from a Big Four firm to test a model of how teams experience status disagreement over time", indicates that not only does this disparity a significant impact on performance, team familiarity actually makes the problem worse.

Keep in mind, we're not talking about massive bunfights for control, boardroom battles, or territorial pissing matches here:

"... [team members] sometimes arrive at different assessments of each other’s relative rankings. In a cross-functional team, for instance, designers might bestow status according to the creativity of members’ contributions whereas engineers might bestow status to those who demonstrate quantitative excellence; these team members’ perceptions of the status hierarchy might therefore differ significantly. Although members might generally behave as if there were a consensus on status hierarchy, especially in situations in which social norms are highly regarded, they might actually disagree about it."

As the bulk of the study's data is survey-based, they found that even if the team just felt unclear on hierarchy, that was enough to cause significant issues:

"One novel finding here is that difference in team members’ perceptions of the status hierarchy is itself enough to hamper team coordination and generate task conflict; the difference in perception doesn’t have to generate blatant status rivalries —or even be recognized at all—in order to cause trouble. While prior research has examined the effects of overt contests ... team members in our study were not necessarily even aware that disagreement existed. Our qualitative evidence suggests, in fact, that most team members assumed that everyone on their team did agree on 'where they stood on the totem pole,' even in teams where status disagreement was actually high. After hearing his team’s actual status disagreement score, one AuditCo manager initially expressed deep surprise; two weeks later, however, he telephoned us to say that he’d been doing a 'mental post-mortem' on the team’s processes. 'I can’t stop thinking about this disagreement [that existed within our team] – it was like an insidious obstacle to performance. It was the unknown unknown that pulled us down.'"

Surprisingly, the damage apparently gets worse with teams that know each other well, or "high familiarity" teams.  These teams showed status disagreement worsened coordination and task conflict problems.  So you can't rely on it simply working itself out over time as the team gets to know one another, it just gets worse!

I think its often a cherished myth that development teams are flat and structure free.  It is important to identify the hierarchy your team has, even if its a shallow one, and work out everyones role and responsibilities within it in an open manner.  This might lead to some disagreement but, like continuous integration, better to get that sorted out early in the process than to find yourself with an "insidious obstacle" further down the road!


Where are the Capistrano alternatives?

Is Capistrano the new CruiseControl?

I used to love CruiseControl, back when setting up a continuous integration build was a wonderful new thing, and you tolerated all its quirks and shortcomings and just made it work to get that build goodness going.  But time moved on, and the rate of updates seemed to stagnate, and well, other cool stuff happened along.  Now, despite Thoughtwork's rejuvenation efforts, its hard to sell CruiseControl to a dev group. We currently group use Hudson, but there are other more and less ornate options out there.

Has the same stagnation happened to Capistrano?

We're trying to pool some automated deployment goodness between a few teams. One team is just setting up, one has a Capistrano based setup, and the other started out with cap, but became frustrated and crafted their own ruby solution. Hmmm, what to do?

The Operations team we're all working with would prefer one flavour, their friendliness towards automated deployment is relatively new, and hard won. The dev teams should agree on a common set of tools and work with them.  I'm not so hot on developing our own as configuration management and deployment setups can get surprisingly complex and cumbersome to refactor.  It isn't as easy as it looks.

I'm inclined to push for Capistrano, regardless.  Sure, it hasn't been updated in almost a year, but then again, how much has the problem changed? (aargh, I remember saying the same thing about CruiseControl!).  Perhaps our frustrations with it are simply through not understanding it thoroughly enough.  We need to look deeper.

Or look elsewhere.

Cosimo Streppone has a good roundup of Capistrano alternatives making a distinction between configuration management and "last mile" deployment, which is fair enough.

configuration tools:
deployment tools:

In the end, he went with Puppet for config, and Fabric for the deployment.  His main reluctance was that Fabric does not support parallel processing.  I'm not sure if that is an option for us, as Fabric is Python.  We're having enough issues supporting Java + Ruby bilingualism without adding another dialect (if you say "Erlang" I'm going to have to strike you again, my friend).

This as good a list of options as I've been able to find, but there is a lot of overhead to adopting many of the configuration setups.  We may end up sticking with Capistrano just because of its simplicity, like a happy old car.


In praise of efficient production

My favorite place for lunch is Don Don's in Melbourne.  Not just because its yummy, but also because it is blindingly efficient.  Through a small menu, efficient process, and massive energy you can cross the street, get a beautiful Sashi Don (picture below) take away, and be out on the curb in time for the next walk signal.  I am not making that up.

They could probably be given the same analytical treatment given Starbucks Barista Heuristics, which I have to admit, also shares some beauty in efficiently producing happy snacks.

(Don Don images via Bunrab Daily Feed, thanks!)


Project Management Zeitgeist

Liquid Planner has put out a list of the top thinkers in project management today which makes pretty good reading.

Granted, they've got Steve McConnell at the top of the list (with disclaimers, he's on their board), but I was happy to see a mixture of familiar and unfamiliar names.  Which means I'm feeling the awesome because I've read a bunch, and I'm feeling the tired because my reading list just blew out. Again.


SLA's, the last resort

Entp makes a spirited case against SLA's and I like it.  I wish I could use it to eradicate SLA culture.  But I can't.

There's something confrontational right out of the box about SLA's.  Negotiating them is a painful process, getting to signoff has all the feelgood moments of negotiating a pre-nuptual agreement.  And in practice, when you have to go to the SLA, its never a shining moment for inter-departmental harmony.

What the case highlights is how poisonous it is to assume that the SLA's describe an acceptable support procedure.  The SLA should be considered the absolute worst case support, and actual support practice should aspire to be much better.  Problem is, the SLA is almost always encoded as the acceptable process.


very handy tool: Rubular

Thanks to James for showing me Rubular, a Ruby regular expression editor.  It is extremely handy if you're trying to fashion truly impenetrable, er, awesome regular expressions.


W. Edward Deming