Quantifying Project Portfolios

PDF Versie

One of the greatest challenges in managing IT projects
within a portfolio is gathering the metrics that will
aid in understanding just how well the projects are
performing and how much these projects are contributing
to the overall corporate strategy. Understanding how IT
investments benefit the corporate goals are critical in
times of growth and especially in times of budget
constraints.

However, the vast majority of companies do not have the
disciplines or infrastructure to be able to capture
information about IT projects, much less gather these
projects within a portfolio. More than 75 percent of
companies operate projects at a Software Engineering
Institute Capability Maturity Model Level of 1, meaning
they have chaotic processes and have no repeatable methods
for building and maintaining software.

Christian Verhoef of the Free University of Amsterdam
attempted to address this problem in his article entitled
"Quantitative IT Portfolio Management," published in the
2002 journal Science of Computer Programming. Verhoef
states that his approach is to provide "a set of mathematical
formulas based on public benchmark information to
quantitatively manage IT portfolios." The intent is to
provide a set of formulas that could be added to a
spreadsheet or a statistical calculator to capture and
analyze portfolio data.

The Key Metrics

In Verhoef's initial assessment, he found that identifying
common metrics was critical to portfolio analysis. However,
with most companies operating at CMM Level 1, identifying
key metrics can be a problem. While some companies capture
a variety of data such as schedule, costs, risks, return
on investment and project size in databases, most companies
only have basic information about cost and delivery dates
with no correlating information.

Verhoef's analysis determined that the key metric to be
used for quantifying IT portfolio management is the number
of function points for each application in the IT portfolio.
Function points represent a widely accepted metric for
measuring size and productivity of applications. Because
function point counts can be made as platform-independent,
and because there are years of historical data regarding
estimates and costs of similar systems, they represent
a key measure to use to compare multiple applications
across the organization.

Following function points as the key metric, Verhoef goes
on to indicate that three other metrics can be brought
together to create the IT portfolio. He states that,
along with the function points for the system, the
portfolio should also capture for each project the start
date, the delivery date and the total project costs.
These three metrics provide basic statistical data on
the general scope, resource level and time required to
execute the project and delivery the product.

Performing the Analysis

Using these four metrics, Verhoef begins to create
organizational relations between the size of the project,
its duration and overall costs. He does emphasize that
these analyses are not meant to determine or estimate
costs for individual projects but rather to compare
projects within a portfolio and to provide a view of the
overall portfolio itself.

After gathering information for the database, Verhoef
recommends cleaning the information to begin real analysis.
This cleansing of the data may mean providing assumptions
regarding project durations for older efforts, estimating
costs and performing function point estimates on deployed
systems.

Once the analysis and cleansing is performed, a normalized
database should appear. From this database, you will then
be able to compare and contrast the portfolio against
such comparisons as:

 
  • Costs of projects by Function Points
  • Costs of projects by Duration
  • Duration of projects by Function Points
  • These analyses can be graphed to show general trends in the database. With these trends, projects appearing above the trend line may represent those that are outperforming the portfolio, while those appearing under the trend line might represent projects that are underperforming the portfolio. Dealing With Hidden Costs Verhoef does warn that many executives believe that in assessing the overall merit of a project, the use of total cost of ownership is not a fair or valid measure. Many executives view IT as a black hole of costs without any more understanding of how much systems truly cost to operate and maintain. Verhoef advises that in order to truly control your IT budget in the portfolio, you must have a clearer picture of the total costs of ownership for the system. Verhoef starts this assessment by creating a formula for calculating just how long a system is likely to be in service. From this formula, a second calculation is made to assume, based on benchmarks, just how much the application will cost to operate during that calculated period. From this, he creates a minimal cost of operating the system. With the cost of operations in line, Verhoef completes the other side of the assessment by generating a calculation of the total costs for development and combines the two into a minimum total cost of ownership. Using this estimate, the portfolio can understand the true costs over time of each project. Greater Details The remainder of Verhoef's paper addresses how, with this portfolio data in place, portfolio managers can begin performing predictive analyses against there projects using existing baseline data and the provided formulas. Such calculations that Verhoef presents include:
  • Projecting potential failure rates of projects
  • Projecting likelihood of delay in delivery
  • Projecting cost overruns in projects
  • Supporting make or buy decisions for projects
  • Ultimately, Verhoef provides a comprehensive set of formulas to use to aid in the understanding and assessment of the IT portfolio. Required Reading In understanding how project portfolios come together, you have to understand what metrics are the most useful to use in comparing projects. Christian Verhoef has done a wonderful job outlining and using basic project metrics to create a solid portfolio analysis model that any company can deploy. For further information, check out the 96-page analysis in Issue 45 of the Science of Computer Programming. Paul X. Harder Meer weten over de wondere wereld van ICT in Jip en Janneke taal? Ga dan naar de knipselkrant van Chris Verhoef