Quantifying Project Portfolios
One of the greatest challenges in managing IT projects
within a portfolio is gathering the metrics that will
aid in understanding just how well the projects are
performing and how much these projects are contributing
to the overall corporate strategy. Understanding how IT
investments benefit the corporate goals are critical in
times of growth and especially in times of budget
constraints.
However, the vast majority of companies do not have the
disciplines or infrastructure to be able to capture
information about IT projects, much less gather these
projects within a portfolio. More than 75 percent of
companies operate projects at a Software Engineering
Institute Capability Maturity Model Level of 1, meaning
they have chaotic processes and have no repeatable methods
for building and maintaining software.
Christian Verhoef of the Free University of Amsterdam
attempted to address this problem in his article entitled
"Quantitative IT Portfolio Management," published in the
2002 journal Science of Computer Programming. Verhoef
states that his approach is to provide "a set of mathematical
formulas based on public benchmark information to
quantitatively manage IT portfolios." The intent is to
provide a set of formulas that could be added to a
spreadsheet or a statistical calculator to capture and
analyze portfolio data.
The Key Metrics
In Verhoef's initial assessment, he found that identifying
common metrics was critical to portfolio analysis. However,
with most companies operating at CMM Level 1, identifying
key metrics can be a problem. While some companies capture
a variety of data such as schedule, costs, risks, return
on investment and project size in databases, most companies
only have basic information about cost and delivery dates
with no correlating information.
Verhoef's analysis determined that the key metric to be
used for quantifying IT portfolio management is the number
of function points for each application in the IT portfolio.
Function points represent a widely accepted metric for
measuring size and productivity of applications. Because
function point counts can be made as platform-independent,
and because there are years of historical data regarding
estimates and costs of similar systems, they represent
a key measure to use to compare multiple applications
across the organization.
Following function points as the key metric, Verhoef goes
on to indicate that three other metrics can be brought
together to create the IT portfolio. He states that,
along with the function points for the system, the
portfolio should also capture for each project the start
date, the delivery date and the total project costs.
These three metrics provide basic statistical data on
the general scope, resource level and time required to
execute the project and delivery the product.
Performing the Analysis
Using these four metrics, Verhoef begins to create
organizational relations between the size of the project,
its duration and overall costs. He does emphasize that
these analyses are not meant to determine or estimate
costs for individual projects but rather to compare
projects within a portfolio and to provide a view of the
overall portfolio itself.
After gathering information for the database, Verhoef
recommends cleaning the information to begin real analysis.
This cleansing of the data may mean providing assumptions
regarding project durations for older efforts, estimating
costs and performing function point estimates on deployed
systems.
Once the analysis and cleansing is performed, a normalized
database should appear. From this database, you will then
be able to compare and contrast the portfolio against
such comparisons as:
Costs of projects by Function Points
Costs of projects by Duration
Duration of projects by Function Points
These analyses can be graphed to show general trends in
the database. With these trends, projects appearing
above the trend line may represent those that are
outperforming the portfolio, while those appearing under
the trend line might represent projects that are
underperforming the portfolio.
Dealing With Hidden Costs
Verhoef does warn that many executives believe that in
assessing the overall merit of a project, the use of
total cost of ownership is not a fair or valid measure.
Many executives view IT as a black hole of costs without
any more understanding of how much systems truly cost to
operate and maintain. Verhoef advises that in order to
truly control your IT budget in the portfolio, you must
have a clearer picture of the total costs of ownership
for the system.
Verhoef starts this assessment by creating a formula for
calculating just how long a system is likely to be in
service. From this formula, a second calculation is made
to assume, based on benchmarks, just how much the
application will cost to operate during that calculated
period. From this, he creates a minimal cost of operating
the system.
With the cost of operations in line, Verhoef completes
the other side of the assessment by generating a calculation
of the total costs for development and combines the two
into a minimum total cost of ownership. Using this
estimate, the portfolio can understand the true costs
over time of each project.
Greater Details
The remainder of Verhoef's paper addresses how, with this
portfolio data in place, portfolio managers can begin
performing predictive analyses against there projects
using existing baseline data and the provided formulas.
Such calculations that Verhoef presents include:
Projecting potential failure rates of projects
Projecting likelihood of delay in delivery
Projecting cost overruns in projects
Supporting make or buy decisions for projects
Ultimately, Verhoef provides a comprehensive set of
formulas to use to aid in the understanding and assessment
of the IT portfolio.
Required Reading
In understanding how project portfolios come together,
you have to understand what metrics are the most useful
to use in comparing projects. Christian Verhoef has done
a wonderful job outlining and using basic project metrics
to create a solid portfolio analysis model that any
company can deploy. For further information, check out
the 96-page analysis in Issue 45 of the Science of Computer
Programming.
Paul X. Harder
Meer weten over de wondere wereld van ICT
in Jip en Janneke taal? Ga dan naar de
knipselkrant van Chris Verhoef