The importance of being measurable.

“But I’m not on the X-BOX!”

How many parents have heard those magic words before, I wonder? My son is fifteen years of age and like most of his peers with a games console, they seem to be surgically attached to these infernal devices twitching involuntarily every so often and grunting into a headset to their friends after some “brilliant or ACE manoeuvre”.

“But you are” I retort. “You have the hand thing in your hands, the headphone thing in your ears and something very horrid on the screen!”

He detaches himself and shows me a screen that logs his time on X-BOX Live! It backs up his story that he wasn’t on the X-BOX. X-BOX Live! Is what the console is about, so I am told. I leave the room – I have no will to argue and more importantly only subjective assertions which his “totally objective X-BOX Live!” log will refute. I go and tell my wife – out of concern for lordship but also I will not be beaten!

“This just like that project you took over last year” she says. This was not the sort of support I was looking for. The project she was referring to was a nightmare. It was to deliver several e-learning modules for a council extranet. I took over from a PM who was leaving, there was a new customer project manager coming in, the budget was gone and the delivery dates were no longer achievable. Add in to the mix the fact that the documentation was “thin” in places – it really wasn’t a nice place to be.

Both parties clearly wanted out of the project. It hadn’t been well defined or executed. Such projects do exist!

The only stumbling block that I had to getting closure – and more importantly the client to sign off the last invoice – was a statement that was made in the initial brief about the courses being “more effective than their classroom equivalent”. That was not an unreasonable request. Why would you commission something less effective that what you already had? Would that make for a viable business case? Hardly. Both I and the customer project manager were Prince2 practitioners. The thought going through both our minds, we admitted to each other later was, “How can/do we demonstrate effectiveness?”

Prince2 makes a clear statement in this regard when assembling the Project Brief (2005) and the Project Product Description (2009). You should capture the Customer Quality Expectations (CQE) – which is what “more effective than their classroom equivalent” represented. However there were no Acceptance Criteria (AC) detailed – those measurable definitions of quality that are used and combined to indicate customer acceptance. When I teach Prince2 courses I always differentiate ACs as being the “black and white” aspects of quality – they are or are not proven. CQE I talk of as being “grey” or aspirational aspects of quality which are significant for the customer and that they need clear definition as part of the AC.

Both of us – the customer PM and myself – agreed that a pre and post survey of a new group using the updated materials would be the best approach. We drafted a questionnaire document which assessed an employee’s ability to perform certain tasks before using the training materials. We then interviewed a sample group and conducted a paper based survey of the remainder of the pilot group asking closed questions in each instance which created an overall weighted index. We needed something measurable which we could use to prove our agreed definition of “effective”.

The results were better than I expected to be honest. The ability of employees to use certain systems was measurably faster, their awareness of certain key information stores was enhanced and overall they felt pretty comfortable with the content. There were areas of weakness too, but the overall conclusion – backed up by empirical data – was that the course materials were more effective than their classroom equivalent.

We both had what we needed to close the project amicably – yet not profitably. We both also felt that if the key Prince2 message of understanding and proving quality from the customer perspective as well as the supplier’s had been effectively documented and agreed from the beginning, the project could have yielded a more positive outcome.

Posted in SkillSolve | Comments (0)