Tuesday, August 28, 2007

A "Joel Test" for infrastructure/operations groups

So - Joel Spolsky has famously (well, famously among a certain type of techies) spelled out a simple test to evaluate how good at software development a shop is.

It's really simple:

The Joel Test
1. Do you use source control?
2. Can you make a build in one step?
3. Do you make daily builds?
4. Do you have a bug database?
5. Do you fix bugs before writing new code?
6. Do you have an up-to-date schedule?
7. Do you have a spec?
8. Do programmers have quiet working conditions?
9. Do you use the best tools money can buy?
10. Do you have testers?
11. Do new candidates write code during their interview?
12. Do you do hallway usability testing?

I highly recommend you pop over and read his blog post about this, it's just a page or so.

I'll wait.

Ok, so the question is - can we come up with a similar list for infrastructure/operations groups? I'm particularly coming at this from the POV of an enterprise client group - desktop/laptop/mobile systems - but I suspect similar issues will apply across many types of infrastructure.

Something that is to the Joel Test what ITIL is to SEMA.

Although there is a defined scoring criteria for SEMA - CMMI - there is not such a scorecard for ITIL that I'm aware of. (This in no way proves that no white crows exist.) Hmm, could there actually be an ISO standard in the works? Oh my.

If an ISO standard is approaching, I think a Joel Test for infrastructure is very badly needed!

So, here's a late night, staying up late to catch the eclipse, first draft, unconsidered stab at it.

1. Do you use source control?

Basically the same, but instead of source-code oriented, oriented towards saving, organizing, and tracking the artificats needed for running infrastrucutre: config files, manifests, scripts, documentation, install packages and system images?

2. Can you make a build in one step?

This translates easily. Can you deploy a system in one step, fully automated, the same way every time?

3. Do you make daily builds?

A bit trickier.

4. Do you have a bug database?

Same. But more of a problem/incident database than strictly speaking a bug database.

Don't worry Joel, you can use FogBugz for this, too. :-)


5. Do you fix bugs before writing new code?

Ah.... might have to punt on this for the moment, while giving a quick nod to the canard about choosing between delivering new systems vs. refactoring creaky systems that are held together with duct tape. (It's a false dilemma. More on that in a future draft?)

6. Do you have an up-to-date schedule?

Should be adaptable. "Do you know who is working on what tasks, and have an idea when they should be done?"

7. Do you have a spec?

Yeah!

8. Do programmers have quiet working conditions?

Yeah! (s/programmers/sysadmins/)

9. Do you use the best tools money can buy?

Yeah!

10. Do you have testers?

Yeah!


11. Do new candidates write code during their interview?

Ummm... ? Do new candidates fix broken systems during their interview? Do new candidates review system requirements specs written by BA's and see the obvious problems? ("Ah, you are specifying that the system uses MS Java Virtual Machine - that is end of life, and won't be available on Vista, never mind Linux or Mac. Suggest you specify Sun Java instead.")


12. Do you do hallway usability testing?

Do you have a pilot group of real users? Do you use them while it's still early enough to make a difference? Do you really try hard to get their feedback? Do you listen to it?

So, to make this Joel-Test-ish, let's summarize:

1. Do you use version control for documentation, scripts, binaries, and other artifacts needed to build and run your systems?
2. Can you deploy a system in one step?
3. Do you monitor your environment for compliance with its desired state and to see deviations (unusual events) when they happen?
4. Do you have a bug (issue/problem) database?
5. (not sure this works, but:) Do you think you have to choose between fixing problems and deploying new systems?
6. Do you have an up-to-date schedule?
7. Do you have a spec?
8. Do sysadmins have quiet working conditions?
9. Do you use the best tools money can buy?
10. Do you have testers?
11. (needs work - suggestions?)
12. Do you pilot to real users?

3 comments:

  1. Hey man that's a pretty damn good list. Did you come back with update as to the false delema question?

    ReplyDelete
  2. Looks like the Joel Test is still alive and kicking, but needs an update for modern times:

    http://geekswithblogs.net/btudor/archive/2009/06/16/132842.aspx

    ReplyDelete