|
|
|
|
|
|
|
June Forum “The Regression Test: an absolute gauge
June Forum 'The Regression Test: an absolute gauge
John opened by stating that conventional tests of “success” for projects are confused and flawed whereas the regression test, an approach emerging from research being done by himself and Ofer Zwikael at the ANU, might provide the answer. Background and definitionsProject performance measurement must be assessed at the beginning of a project to enable decisions to be taken re funding (funder) and accepting a role (project owner & PM) and at the end to make judgments about the resulting investment when there are two key players—the project owner and the project
In expanding on this, John defined two key terms: Outcomes: these are the measureable end effects (both desirable and undesirable), e.g. “the reduced waiting time for elective surgery”; and the project owner is accountable for securing these. Outputs: are deliverable artifacts, e.g. “re-engineered hospital
John then proceeded to define the concept of project worth by the formula Worth = Fn (Benefits, Disbenefits, Costs), where Benefits are driven by desirable outcomes, Disbenefits are driven by undesirable outcomes, and Costs are driven by the project’s outputs. He noted that Worth is simply an index of value
With these definitions and concepts in mind, John then suggested a project assessment framework involved assessment processes (2 of); assessment targets (3 of); assessment tests (4 components) and triggers (3 of).
As indicated above, the assessment process is both before (“appraisal”) and after (“evaluation”) and this talk was about the evaluation. The three investment targets were: the investment itself; the project; and the project management which indicates there could be three, separate, judgments about success! The 4 components of the test were: A specific set of performance variables; measurements of those variables; a criterion (reference value) for each; and a rule showing how to use the resulting measures to make a judgment. The triggers involve the Appraisal (the tabling of the business case) and the Evaluation, of which there are two: the Outputs closeout report (when outputs are delivered); and the Outcomes closeout report (when target outcomes are secured). Concept of regression testIf effect, the question is posed to the original decision maker, “What would you have decided had you been given the achieved business case?”i.e. all the original parameters are replaced with the actual values. If the response is to reject the project, then it is declared a failure, but if it was to accept, then it is declared a success. It was noted that the regression test allows for trade-offs amongst parameters in the business case.
John suggested that the (notorious) “iron triangle” of scope/quality, time and cost, as a test of project management success does not account for detrimental outcomes, “Undesirable outcomes that are unexpected, unacceptable and avoidable,” e.g. staff loss. Rather, John suggested a “steel tetrahedron” which had a 4th ‘corner’, namely detrimental outcomes.
So, when measuring a project’s success, John showed a graph of worth against riskiness with two contours on it: the project investment frontier; and the business case value contour. Any project above this second contour would be deemed a success.
When judging investment success, the second contour is removed and provided the project is above the project investment frontier, it is a success.
However, when judging the overall project, there are three Success/Failure assessments to be made regarding the project management, the project itself, and the investment.
Four examples were used to help clarify this point: Apollo 11 was a success on all three criteria when getting the stricken spacecraft back to earth; the Sydney Cross-City tunnel might have been a success from a project management perspective, but was a failure on the other two criteria; the Hubble Space Telescope was a failure from a project management and overall project perspective, but a success from an investment criteria; and finally, FAA’s Advanced Automation System failed on all three counts.
To conclude, John reaffirmed his opening remark that conventional treatments of project success fail whereas an absolute test of success for projects:
Thanks go to Cordelta for sponsoring the May General forum |
