EQUITY 2007 REPORT

Need for scientific research on IT value

The goal of this conference is to set up a new research field to study IT value and its components, says co-chair Elliot Chikofsky, chief consultant of Engineering Management and Integration, an IT consultancy firm, in his opening statement. Looking at the high costs of IT made him wonder whether the research effort was maybe too late. Chikofsky predicts that concepts and methods need to be combined to discover how much is known about IT value and what elements are lacking. "This is a 'casa blanca' conference to round up the usual suspects: who are the people doing the research in this area?"

Valueing IT requires new methods

New cost estimation methods are needed to consider reuse, COTS (commercial off-the-shelf), and IT value. According to Barry Boehm, IT researcher at the University of Southern California, business case analysis needs to incorporate 'real' value, the tracking of market share and ROI.

In the opening keynote, Boehm said IT value annalists should not study 'earned' value, based on schedules and budget. Boehm, author of the well-known book 'Software engineering economics', dealing with cost estimation, business case analysis and risk management, noted how few of the ideas in this study were being picked up. "Only a few of all newly written books on this topic contain the words IT costs and IT value." According to him, one of the most valuable figures in the book can help prevent impossible deadlines. It is the graph giving the relation between total effort in man months and minimum minimum development time (27 to 8) which shows the maximum schedule compression that is possible.

IT portfolio management linked to company strategies.

Thorough IT Portfolio management within any company is linked to budget management and acquisition management, says Elliot Chikofsky, chief consultant at Engineering Management and Integration, a software development consultancy.

Chikofsky studied data from the U.S. Department of Defence to develop a method for employing proper IT Portfolio Management.

The follow-up session illustrated that it is more harmful replacing an IT system too soon rather than too late. This is show by a mathematical analysis of the development, operational and maintenance costs at Cordares, a Dutch company offering pension management services and income protection insurance.

According to Cordares' board member Adri van der Wurff, the optimal moment to kill a system was revealed, as well as Cordares' results using this method, demonstrating that it is more harmful to replace a system too soon rather than too late.

Van der Wurff showed how Cordares dealt with IT-systems legacy and longevity. He stressed the importance of the power to say no to new and ambitious IT-projects, in search of the right moment to replace a running system with a new one.

IT road map enables growth in IT yield

Having a systematic road map enables companies to increase their IT yields. According to Jim Kenneally, an IT researcher at Intel Corporation, this computer technology company captures the business value of IT by incorporating value metrics inside the Capability Maturity Framework.

Kenneally said business value should be sought in various aspects of IT, such as revenue, cost reduction, productivity and risk. CMFs view on value enabled Intel to gain insight in the IT numbers. In an ideal situation CMF could help to increase a company's monetary profits. The framework is good at evaluating certain business values. Profits cannot however be easily derived from these. According to Kenneally, the majority of companies do not calculate the Return On Investment of their IT. Citing a survey done among CIOs in 2002, he said 73 per cent of companies do not compute it. Of the companies that do calculate the ROI, 70 per cent finds it difficult.

In the next session, Carnegie Mellon University computer scientist Mary Shaw, showed how she is using design information, development method and software characteristics predictors, to develope a model that she says will determine at the early stages of development whether a product meets end-user expectations.

"We need better ways to analyse software design", Shaw said. She also thinks software developers should be able to predict the value projects' implementation will offer to their customer or the producer. Her model for predictive analysis of design gives an estimation of software design value to a client with certain preferences.

Approximation of complexity improves Function Point Analysis

Methodological approximation helps reduce the estimation effort involved in ranking the complexity of the components of an Function Point Analysis.

Stefano Fabrizi, a IT system analyst at Banca d'Italia, discussed an approximation function. For this he combined historical data on FPA for conducted IT projects with the distribution of complexity of the FPA components. The approximated method proved to be statistically sound and was successfully applied in practise, Fabrizi showed.

Improving FPA is imported, he argued. "The major dilemma in software size estimation using FPA is, that in some cases it consumes 15 to 20 per cent of the entire software maintenance activity."

'IT should use CAPM finance instrument to determine performance'

IT managers and CIO should use Capital Asset Pricing Model, a finance theory to calculate rates of return, to determine IT performance, says Geert Jan Beekman, of Atos Consulting, an consultancy firm.

He says in many companies the cost structure and performance of IT remain hidden. The IT managers and CIOs need tools to improve the performance of IT, but first of all need to have an idea of the costs. Beekman proposes to adapt CAPM to make it useful for these projects. He adapts the finance calculations, adding the chance of failure, cost overruns, requirements creep and time compression. Using CAPM, he says, IT projects can be restructured to increase their economic benefit. Beekman said he used the adapted CAPM to help one company get an idea of the costs of underestimating the risk of failure in some of their IT projects. It lead to an increase in hurdle rate, and to a decrease in the estimated return of these projects. By not managing that risk their IT portfolio of 250 million euro had its returns drop by 12.5 million per year. "Given that most applications last for at least eight years, that is as serious amount of money."

IT departments should learn to use tools such as CAPM. Without them, they are on a collision course with economic reality, says Beekman. "Some 300 billion dollars are spent annually on IT investments that fail. The world spends as much on beer in a year."

80-20 rule better at ranking priorities

Using the 80--20 rule to prioritise various assessment techniques is better than relying on prioritisation done by experts themselves. This is shown in a case study done by Rick Kazman of the University of Hawaii.

Kazman studied test prioritisation at the NASA Ames research centre. According to him, this method is now used in Japan at JAXA/JAMMS to strategically plan activities. By using the strategic method, the 80-20 rule is used in advance.

Experts now use the strategic method to prioritise different testing methods in such a way that problems are detected as soon as possible. By estimating probabilities of failures, cost of those failures and detection probabilities, an optimal prioritisation of techniques is made beforehand.

Company characteristics tell on its IT value

The value of information within any company depends greatly on its characteristics for that specific company. These characteristics can be used to assess this value to the company.

Paraskevi Mentzelou, a computer science professor at Alexander Technological Institute of Thessaloniki is developing a method to measure the relation between information and its value to a company.

Mentzelou gave an insight into the low level of IT usage in Greek companies, prompting the Greek government to promote the usage of computers.

Andrea Trentini, from the University of Milan, was the second speaker in this session. According to him, a company's network value can be determined by measuring the total of data flow through a company. This is done by calculating and adding up the values of all channels independently.

Trentini studies the valuation of the transfer of information specifically. He is developing a model to value information channels. For this he assigns values to content type, content format, frequency of transmission and channel features, in terms of speeds and costs.

FP Lite an alternative method for sizing software

FP Lite is a good alternative for sizing compared to FPA. The accuracy of this method to determine the size of software project when a detailed requirements specification is missing, increases with the size of the project.

FP Lite takes less effort than a detailed Function Point Analysis, says Sheila Dennis, of David Consulting Group, an IT consultancy firm. She conducted experiments with both methods to discover the differences such as the required effort.

Her results show that more than 70 per cent of the size estimates derived using FP Lite were within +/- 20 per cent of the detailed FP count.

FPA is a suitable method for determining software size in industrial context, agreed Dennis and Banca d'Italia's Fabrizi. FPA can also give a measure of software value and costs.

Crowdsourcing produces real value

Involving end users, also known as crowdsourcing, can help a business produce real value, says Menno van Doorn, Director Research Institute for the Analysis of New Technology, an IT services company. "Crowdsourcing is about milking the masses."

Van Doorn says crowdsourcing tries to involve the outside world in the organisation in order to innovate or solve problems. Examples of crowdsourcing are toy company Lego and the Toronto traffic governance board.

Van Doorn is studying the use of crowdsourcing. He wants to know the boundaries within which this mechanism helps value creation. He also aims to establish what competences are needed to find crowds, how or if to reward it, whether a company needs courage to trust the masses and who owns the new created value. Crowdsourcing is possible even within a company, it appears. Peter Kapitein, a program manager at the central bank of the Netherlands, described in the next session how a high amount of applications in use at this bank, made it hard to tally the total of functionality. This in turn frustrated decisions on new IT.

Recently, the bank sifted through all 3400 applications, and in a first round found it could immediately do away with 1200 of them, described Kapitein.

Ranking all applications, the bank found another 1200 applications that could potentially be scrapped, depending on whether they offer redundant functionality.

The bank is using the results to draw up strict rules on implementing new applications. Kapitein estimates that these rules and the further weeding of redundancy will lead to a 38 per cent reduction of applications and significant cost savings.

Infrastructure indicators key for IT security evaluations

Indicators for IT infrastructure security and survivability are key to IT security evaluations, says Philadelphia University computer scientist Khalil Abuosba.

In this research methods to quantify security attributes that enable a comprehensive performance evaluation of a software system are improved. Such evaluations generally focus on configuration management, vulnerability management, risk management, change management and networks management. These evaluations should be updated.

University of Ontario Institute of Technology IT security specialist Clemens Martin followed up on Abuosba, showing that security metrics can be collected in a holistic manner if integratedin a Total Quality Management approach.

Martin is using the TQM to establish security quantification. According to him organisations generally have problems getting such data. He says the quantification scores derived from TQM can be applied to each security domain and provide an overall result. He is working on basic scorecards and on a prototype to implement his method.

Blind IT investments do not lead to profits

There is no direct relation between IT investments and company profits, says Paul Strassmann, computer science professor at George Mason University. Therefore, one cannot blindly invest in IT to increase company profits.

Strassmann, who once was the Chief Information Officer at the Department of Defence, never found a direct relation between investing in IT and profits. IT by itself cannot provide justification; it depends on how the technology is used. He established a benchmark that helps companies decide whether they should spend less or more on IT. He also provided a method to calculate the Information Productivity. These methods can be downloaded from his website, http://www.strassmann.com/.

Quantitative choices in software engineering

Exploring the quantitative yields of Software Engineering Methods in the Software Life Cycle is a complex problem that can be solved in a number of steps, says Anca-Juliana Stoica, professor at the Royal Institute of Technology, school of IT in Stockholm, Sweden.

Her research on Software Engineering Methods shows the possibilities of applying measurements, metrics and models to the Software Life-cycle in order to make a quantitative choice between several different Software Engineering Methods and its possibilities for designing new SEMs.

Barry Boehm, co-director of the Centre for Systems and Software Engineering at the University of Southern California, next debated that an analysis of the overall time spent on an IT-project as a function of the percentage of time added for architecture and risk resolution, shows there is an ideal amount of system engineering.

At this 'sweet spot' there is a minimum of total costs and schedule time, says Boehm. In his research on ROI on system engineering he addressed the question of how much system engineering is enough. It showed the problems caused by too little System Engineering in IT-investments, such as cost and schedule overrun. The sweet spot, Boehm found, tends to shift to the right for larger projects, indicating a greater need for big design up front for the larger projects.

Agile development speeds-up software development

Combining the right type project and developers, the Agile develpoment management for software development will get applications out sooner than other project management styles, such as Waterfall or Rapid Application Development.

According to David Garmus of David Consulting Group, and IT consultancy, Agile development is best used for projects where customer and software developers work in a team, increments are short, software is released often and with programmers that "thrive on chaos". Among the requirements for using Agile development is that developers have to be able to get along with one another. "They have to like working together, they like playing together and working around conflicts is just not possible. Such developers are motivated by the team dynamics." Agile development is most effective in projects developing applications with between a fifty and two hundred function points. It is also good for using in exploratory or research and development projects, where choices can be made as the project progresses. The development team should consist of six to ten programmers.

Agile development should not be used for outsourced projects, projects with a set deadline or projects that have high financial risks, as these require many more test phases. "Don't use this for e-business where more security is needed, but Agile development can significantly increase the development of web-applications."

"Choose the right development style will maximise software delivery. Agile development's approach is like that of cowboys, it does not work well for all projects."

Re-appreciation for Agile development

There is a growing appreciation of Agile development and lean software development practises, says Jaap Bloem, senior analist at the Institute for the Analysis of New Technology, part of Sogeti, a consulting firm.

Referring to Agile development as a more human way of software development, Bloem says it provides an alternative to what he calls heavyweight bean-counter development methods. "It can deliver decent productivity levels. I expect it to be embraced and integrated in key IT projects."

Bloem says using the Capability Maturity Model Integration, a model to improve business processes, ignores human aspects in software development projects such as learning and collaboration. Its thorough and systematic approach of technology and relationships however, provides enough flexibility to allow Agile development teams.

According to Bloem researchers and consultants involved with Agile development and Waterfall development, an other method for software development, are discussing a merger between the two.

Making IT user-centric makes its value measurable

IT should not be defined in terms of cost, but in terms of value for the users. David Sward, User Experience Researcher at Intel Corporation say only then IT's business value can be measured. "By itself, IT is nothing."

Sward, author of the book 'Measuring the Business Value of Information Technology', is working on a framework for Intel to manage the value of IT in the company. Defining IT in user centric terms, he says, is a way to put the technology in value dials, such as 'days of inventory', or 'employee productivity'.

Sward's session was followed by a business case from Japan. Japanese companies that invested in their IT infrastructure saw more improvements of their profits than those that improved their plant equipments did.

Rules on IT design help reduce costs

Creating flexible design rules for IT development, helps reduce development costs, says Daniel Cabrero, IT security specialist at Spain's ministry for Labour and Social Affairs.

According to him, numerous IT application designs can meet the clients requirements. Some designs however create higher maintenance costs than others. Finding the best design method, therefor, is worthwhile.

Cabrero says adding flexibility to the IT designs reduces the overall costs. The design should choose between stability, changeability and analysability. Such rules help estimate whether changes to an IT application's design will be profitable.

The second session on this topic was presented by Warner Sijsling from Sogeti, a Dutch IT consultancy firm. Optimistic estimates driven by competition make it difficult to calculate the costs of software development. More data on project histories will improve such cost estimates, he says.

For Sogeti getting the development costs correct is vital, for 25 per cent of its software projects are fixed price. Most of these fixed price projects have a small profit margin: asking too much will make the project go to the competition and asking less will create little profit.

Sogeti created their own tool to aid them with cost estimation. It uses historical data and combines these with best practises. "The issue is how to deal with variance in the estimates", says Sijsling. "People do not want to hear about uncertainty." It helps he noted, to make them aware of the uncertainty.

first presentation was cancelled
for second presentation see session Portfolio Analysis.

Legacy: replace it or cherish it?

The so-called real option model is a useful tool to decide whether to replace or keep a legacy IT system. The model values IT assets in the same way as financials options are valued. Masafumi Kotani a researcher at IBM Japan, used this model to time the replacement of legacy systems.

He used a four step model to do this. The first three steps used project appraisal, portfolio generation and value-based optimisation. These three generated a Pareto front of solutions. The given solutions are optimal when seen from multiple perspectives, like Net Present Value, cost and speed. In the final step, the new portfolio was manually selected based on the presented solutions.

Following Kotani's presentation, Karl Reed, computer science professor at LaTrobe University in Australia, discussed how business process modelling and metrics help prevent organisational disruptions that can arise when new IT projects are implemented. Software engineers should take such disruptions into account when building new applications, Reed says.

In order to prevent such implementation failures, care should be taken to re-use resources when designing the new business process. Reed noted how on the continent some systems were recently completed to specifications. The systems were all accepted. Their adoption, however, seriously disrupted the organisations, described Reed. "These traumas can not be classified as a requirements failure."

Reed found that currently most software researchers only focus on the period before the delivery of a new system.

'Outsourcing likely to be less efficient'

Outsourcing software development makes the development process less or equally as efficient as developing in-house, says Todd Little, software developer at Landmark Graphics, part of multinational Halliburton Energy Services.

Little compared software projects done in the US and in Landmarks' outsourcing countries, Canada, India, Norway and Pakistan. Outsourcing increased management overhead by 10 to 25 per cent, he says. Depending on exchange rates, inflation and currency risks, the cost ratio between projects varied between 0.25 in Pakistan and 1.0 in the US. The specialised skills needed to develop Landmarks oil-exploration software, people with C, C++, Unix, Motif and Fortran skills that also understand the Oil exploration business, resulted in extreme staff turnover rates. Average turn over in India is 25 percent and in Pakistan it is about 10 per cent, says Little. Landmark had to deal with 75 per cent staff turn over. "Because of the contract with a previous employer, a programmers would be required to wait three months before he could start with us. While waiting, he would get a better offer."

Time difference

Finding talent creates risks, such as value lost because of delays, decreases in quality, lowered security and because of missed opportunities. For example, early development projects in Pakistan resulted in saving of no more than 18 per cent. "Overall, we are doing much better now."

"This kind of turn over is terrible", Little experienced, as it has a lot of influence on the management of the software development. Another "horrific" influence is the time difference. "A twelve hour time difference means no rich conversations. Everything is done by email, and there is limited video face-to-face communication. This hampers creating relationship with the developers and that is a real challenge." Another problem is IP theft and IP pollution. "We have to check for thing brought into the code. American laws and regulation further complicate this. Little: "Every time someone who is not a US citizen touches the code, we need to make sure it is still compliant."

Useful simulation of outsourcing

Simulation of outsourcing processes can be very useful. It helps determine the conditions that makes programmers most productive, shows Peter Kampstra, an IT researcher at the Vrije Universiteit in Amsterdam

Kampstra is trying to improve computer simulations of outsourcing processes. One of his aims is to assess the impact an outsourcing process has on development productivity. The simulations so far allowed him to find a link between efficient programing and code reviews. Tweaking the approval rates, he found, has an obvious impact on the time needed to finish the development. The simulation also showed that this stricter approving of code has a very big impact on productivity. "Productive programmers are in a certain state of mind, called flow", he explains. "Tweak approval rates too much, and you make them doodle." According to Kampstra, simulating outsourcing has other advantages. It helps identify bottlenecks in the outsourcing process, and allows to run several 'what if' scenarios. "You'll discover if your process has too many steps."

Outsourcing companies sometimes simply expect too much of the results, says Kampstra. "They outsource their laundry and expect to receive new clothes in return."

first presentation was cancelled

Integrating stakeholder value in software development

Stakeholder value considerations can be integrated into the full range of systems and software development principles and practises.

Apurva Jain, a Phd-student at the university of Southern California is working on a method for value-based software engineering (VBSE), using the stakeholder win-win Theory W.

Jain and his professor Barry Boehm are also making progress in the area of the little understood Dependency Theory.

Risk assessments help estimate costs

Risk-assessment can be used to compare the expected costs, says Ton Tijdink from Cibit, an IT consultancy. On deciding on using a risk-mitigating solution it is important to determine the value this creates in terms of decrease in expected costs due to that specific risk.

Tijdink studied the process of patient queries and accessing a pathology report database to explain the concept of return on security investments (ROSI). According to him risks can be assessed through the notion that risk is the product of a risk-probability and a certain impact, the Annual Rate of Occurrence and the Single Loss Expectancy, and we can use this notion to determine the return on certain security measures.

In the second discussion on the topic of understanding IT-Value, Hans van Herwaarden from Quint, an IT consultancy, discussed that companies should either focus on efficiency or effectiveness. Theu should also align their business and IT goals.

According to Van Herwaarden the business value of IT can be increased in one of three ways, either increasing revenue at the same cost level, decreasing costs at the same revenue level or by increasing the return on the IT assets.

A third issue was raised by Todd Little, senior development manager at Landmark Graphics, provider of software and consulting services for the upstream oil and gas industry. He said companies should maximise the value gained from development activities.

Little found the amount of value creation depends on several indicators during the software development process, such as team size, pair programming and the total of working days. Using data from the software development process he plotted development time in months versus value created for the company. He says it proved to be very useful in bridging the gap between the perspective of a business person and an IT person.

"Cost estimation is relatively easy, whereas value creation estimation is still hard to do."

2007 ©

Gijs Hillenius <gijs@hillenius.net> (editor),
Laurenz Eveleens <laurenz@few.vu.nl>,
Peter Kampstra <pkampst@few.vu.nl>,
Łukasz Kwiatkowski <lukasz@few.vu.nl> and
Sander Sman <asn270@few.vu.nl>.
all rights reserved