SOME MYTHS AND FACTS ABOUT MEASUREMENT

When someone does not want to do something, it does not really matter why it isn’t done as much as it matters that it isn’t done. Since myths usually contradict or exaggerate facts, perpetuating a generally accepted myth can often be an effective means to prevent measurement from moving forward.

Myth: Measurement Is Expensive

Fact: Measurement is a cost. In a business, the concepts of expensive and cheap do not apply. Outlays have a cost—the amount of cash or other resources committed. Benefits are always associated with costs. Cost-benefit analyses are used to support decisions about which outlays should be made. For example, successful software engineering process measurement programs require an outlay of 1 percent to 3 percent of the total cost of software engineering. Many industry leaders invest more than 5 percent of total costs in measurement. Typical benefits realized by these companies as an indirect result are continuously increased product and service quality, as well as annual improvements in productivity of 10 percent to 15 percent.

Myth: Measurements Can Be Used against the Persons Being Measured

Fact: The myth is true, in the same sense that matches can be used to incinerate civilization. It is possible to say that they can be, but only once. After that, what is done is no longer measurement, but is more akin to creative writing and risk management. Dr. W. Edwards Deming observed that the system dictates its own performance; the performance of people working in the system is largely a product of the system. In a system fraught with fear, one compensates by producing measures that mitigate the results of what the people in the system fear. One of Deming’s famous 14 points is to drive out fear. Measurements are used correctly to assess the performance of the system and to identify system improvements.

Myth: Measuring Can Improve Processes

Fact: Processes can be improved only by changing the processes. Measurement helps identify beneficial changes that must be implemented for improvements to occur. However, processes can be changed independent of measurement. The advantage of measurement is leveraging the change investment. Good measurement identifies those changes with the highest return. For example, in software engineering, if processes are routinely changed by investing in all the newest tools and technologies, an annual improvement of about 3 percent in productivity can be realized. Most of what is acquired ends up as “shelfware” and considerable effort is spent trying out these new tools that are never used. By using measurements to identify the best improvements and focusing training on implementing those improvements, the same investment can consistently yield 10 percent to 15 percent productivity improvements.