Software Metrics

General Metrics | Function Points | Benchmarking

General Metrics

Extreme Chaos
In 1995 the Standish Group set out to identify the scope of software project failures, the major factors that cause software projects to fail, and the key ingredients that can reduce software project failures. This article, published in 2001, summarizes their results and findings after 7 years of research into these questions. (12 pages)
Click Here to Read "Extreme Chaos"!

A Software Metrics Primer
Software measurement lets you quantify your software schedule, work effort, product size, project status, and software quality performance. Ultimately, software measurement helps you better control your software projects while learning more about the way your software organization works. This article by Dr. Karl E. Wiegers of Process Impact provides an excellent summary of basic software measurement principles and suggests a few software metrics that can help you understand and improve the way your software organization operates right away. (5 pages)
Click Here to Read "A Software Metrics Primer"!

Software By the Numbers: An Aerial View of the Software Metrics Landscape
For software organizations still in a state of software metrics paralysis, this article by Michael C. Mah and Lawrence H. Putnam, Sr. provides an excellent place to start. In addition to outlining key objectives for any software metrics program, the authors also seek to answer two basic questions: "What software metrics apply?" and "What does one do with the software metrics?" (14 pages)
Click Here to Read "Software By the Numbers: An Aerial View of the Software Metrics Landscape "!

Usefulness of Metrics and Models in Software Maintenance and Evolution
Over the years, more than two hundred different software metrics and dozens of different models have been proposed for application to software systems. Faced with such diversity, which software metrics and models should be applied across software maintenance releases, software projects, and software organizations? This position paper by Ned Chapin of InfoSci, Inc. explores that question and summarizes seven points that can improve the usefulness and value of using software metrics and models. (4 pages)
Click Here to Read "Usefulness of Metrics and Models in Software Maintenance and Evolution"!

Eight Secrets of Software Measurement
An effective software measurement program is the keystone for both effective software estimation and effective software process improvement. In this article by Dr. Elizabeth Clark, you will find eight "secrets" of successful software measurement, i.e. insights gleaned only after a long career in the software metrics field. (3 pages)
Click Here to Read "Eight Secrets of Software Measurement"!

A CAI State of the Practice Interview with Dr. Stephen Kan, IBM Senior Technical Staff
Dr. Stephen H. Kan is a Senior Technical Staff Member (STSM) and a technical manager in software programming at IBM in Rochester, Minnesota. He is responsible for the Quality Management Process in software development for IBM's eServer iSeries. Dr. Kan is the author of the book Metrics and Models in Software Quality Engineering, numerous technical reports, and articles and chapters in professional journals. He established the Capability Maturity Model (CMM) strategy for the iSeries software organization and led the software process improvement effort with a core team in achieving a Capability Maturity Model (CMM) Level 5 assessment in 2004. Our interview between Stephen Kan and Michael Milutis, Executive Director of the IT Metrics and Productivity Institute (ITMPI), took place in August of 2005.
Click Here to Read the Interview with Dr. Stephen Kan!

Take Measurement, Reliability, and Validity Seriously
This article by Dr. Stephen Kan of IBM provides an introduction to the concepts of software measurement reliability and validity. According to Dr. Kan, reliability and validity are the two most important issues in software measurement quality. They should be well thought through before a software metric is proposed and implemented. " (3 pages)
Click Here to Read "Take Measurement, Reliability, and Validity Seriously"!

Don't Take Anything For Granted When Using LOC (Lines of Code) Related Metrics
Lines of code (LOC) related software metrics are anything but simple. The major problem stems from the ambiguity of the actual counting methods, both within and across different software programming languages. This article by Dr. Stephen Kan of IBM highlights some of the factors that render lines of code (LOC) software metrics risky and misleading while offering a simple, practical approach for extracting meaningful value from them and mitigating these risks. (4 pages)
Click Here to Read "Don't Take Anything For Granted When Using LOC (Lines of Code) Related Metrics "!

A CAI State of the Practice Interview with Joseph McMakin, former CIO of Air Products and Chemicals
In 2002, as CIO of Air Products and Chemicals, Joseph McMakin was awarded Gartner's CIO Choice Award for Excellence in Information Technology. During his tenure as CIO, Air Products rose to a top 30 position in e-business rankings, and won the Gartner award in 2001 for continuous improvement. With a staff of approximately 1000 professionals and global operations extending to over 30 countries, McMakin formulated and directed significant changes in infrastructure and application approaches, as well as in governance of IT in the enterprise. Our interview with Joseph McMakin and Michael Milutis, Executive Director of the IT Metrics and Productivity Institute (ITMPI), took place in August of 2005.
Click Here for the Interview with Joseph McMakin!

CIO Metrics and Decision Making: Survey Results
CIO's and other senior IT executives from mid- to large-sized organizations make significant business-altering decisions every day of their professional lives. Nevertheless, little research has examined the processes that IT executives engage in to gather information for making these decisions, or the metrics they employ. This article by CXO Systems produces the results of a study that surveyed 52 senior IT executives with the aim of answering these questions. (10 pages)
Click Here to Read "CIO Metrics and Decision Making: Survey Results"!

A CAI State of the Practice Interview with Dr. Karl E. Wiegers of Process Impact
Dr. Karl E. Wiegers is Principal Consultant with Process Impact, a software process consulting and education company in Portland, Oregon. His interests include software requirements engineering, software peer reviews, software process improvement, software project management, software risk management, and software metrics. Karl is the author of More About Software Requirements which will be available through Microsoft Press in January of 2006. Karl is also the author of Software Requirements, 2nd Edition (Microsoft Press, 2003), Peer Reviews in Software: A Practical Guide (Addison-Wesley, 2002), and Creating a Software Engineering Culture (Dorset House, 1996). Karl has written more than 160 articles on software development, chemistry, and military history. He has served on the Editorial Board for IEEE Software magazine and also as a contributing editor for Software Development magazine. He is a frequent speaker at software conferences and professional software society meetings. Our interview between Karl Wiegers and Michael Milutis, Executive Director of the IT Metrics and Productivity Institute (ITMPI), took place in September of 2005.
Click Here to Read the Interview with Karl Wiegers!

A CAI State of the Practice Interview with Linda Westfall, Expert Software Metrics Practitioner
Linda Westfall is an expert software metrics practitioner and President of The Westfall Team. Her specialties include software quality engineering, software metrics, software project management, software risk management, software requirements engineering and management, software peer reviews, software testing, software process definition, and software process improvement. Linda has more than twenty years of experience in real time software engineering, quality and metrics. She is the past Chair of the American Society for Quality (ASQ) Software Division. Our interview between Linda Westfall and Michael Milutis, Executive Director of the IT Metrics and Productivity Institute (ITMPI), took place in November of 2005.
Click Here to Read the Interview with Linda Westfall!

12 Steps to Useful Software Metrics
Linda Westfall's "12 Steps to Useful Software Metrics" introduces the reader to a practical process for establishing and tailoring a software metrics program that focuses on goals and information needs. The process provides a practical, systematic, start-to-finish method of selecting, designing and implementing software metrics. It outlines a cookbook method that the reader can use to simplify the journey from software metrics in concept to delivered information. (13 pages)
Click Here to Read "12 Steps to Useful Software Metrics"!

Are We Doing Well or Are We Doing Poorly?
Software metrics don't solve problems - people solve problems. What software metrics can do is provide information so you can make informed decisions and better choices. According to the new ISO/IEC 15939 Software Measurement Process standard, decision criteria are the "thresholds, targets, or patterns used to determine the need for action or further investigation, or to describe the level of confidence in a given result." In other words, you need decision criteria to obtain guidance that will help you interpret the software measurement results. This paper by Linda Westfall shows you how to establish useful decision criteria for different types of software metrics. (13 pages)
Click Here to Read "Are We Doing Well or Are We Doing Poorly?"!

Establishing a Software Measurement Process
This report by Donald R. McAndrews of the Software Process Measurement Project presents guidelines for establishing a software measurement process as part of an organization's overall software process. Methods are suggested that can be used to design a repeatable software measurement process. This report also suggests some steps for starting a software measurement program. (46 pages)
Click Here to Read "Establishing a Software Measurement Process"!

Seven Steps to Designing a Software Metric
If software metrics are to provide useful information, everyone involved in designing, implementing, collecting data for and utilizing software metrics must understand their definition and purpose. This paper by Linda L. Westfall of Software Measurement Services outlines seven steps for documenting the design of software metrics in order to insure this understanding. Areas covered include software measurement objectives, software measurement definitions, software measurement models, counting criteria, software measurement success criteria, software measurement reporting, and the human factor. (6 pages)
Click Here to Read "Seven Steps to Designing a Software Metric"!

Software Metrics: Ten Traps to Avoid
This article by Dr. Karl E. Wiegers of Process Impact identifies ten software metrics traps that can sabotage unsuspecting software metrics practitioners. Several symptoms of each software metrics trap are described along with some possible solutions. By being aware of these common risks, software metrics practitioners can chart a course toward successful measurement of their organization's software development activities. (9 pages)
Click Here to Read "Software Metrics: Ten Traps to Avoid"!

Metrics and the Immature Software Process
There has been much discussion in recent years about the role of software metrics in helping software organizations improve software productivity and software quality. In our never-ending search for simple solutions to complex problems, software measurement is often seen as a panacea for our software ills. This article by Fred Koch presents a software measurement strategy for software organizations that are in the early stages of establishing a software process. (2 pages)
Click Here to Read "Metrics and the Immature Software Process"!

What to Measure?
At a recent visit to an IT client, the new software QA manager took me aside and said: "JR, here are the software metrics I want to measure on a monthly basis. What do you think?" I asked him what the reactions to these software measurements were. None of the managers seemed to want to measure things; no one wanted to see his software measurements. He had no idea why no one was interested. I asked, "What do you want to accomplish with these software measurements?" He was stumped. He had no idea what he wanted to do with the software measurements, he just knew he wanted to collect them. (3 pages)
Click Here to Read "What to Measure?"!

A CAI State of the Practice Interview with Dr. Norman Fenton, Author of Software Metrics
Dr. Norman Fenton is Professor of Computing at Queen Mary (London University) and is also Chief Executive Officer of Agena, a company that specializes in risk management for critical systems. Between 1989 and March 2000 he was Professor of Computing Science at the Centre for Software Reliability, City University. He has been project manager and principal researcher in many major collaborative projects in the areas of: software metrics; formal methods; empirical software engineering; software standards, and safety critical systems. His recent research projects, however, have focused on the use of Bayesian Nets (BNs) for risk assessment. Our interview between Dr. Norman Fenton and Michael Milutis, Executive Director of the IT Metrics and Productivity Institute, took place in March of 2006.
Click Here to Read "An Interview with Dr. Norman Fenton"!

Measuring your Risks: Numbers that would make sense to Bruce Willis
By destroying the meteor in the film Armageddon, Bruce Willis saved the world. The probability of the meteor strike was so large, and the consequences so great, that nothing much else mattered except trying to prevent the strike. Combining the probability and impact of a risk in order to define its size is standard practice. But in most cases it is irrational, and it certainly would not have explained to Bruce Willis and his crew why their mission made sense. In this article by Norman Fenton and Martin Neil, the authors explain that to get rational measures of risk you need a causal model ('risk map') that links triggers, controls, events, mitigants and consequences. Once you do this, measuring risk starts to make sense. (6 pages)
Click Here to Read "Measuring your Risks: Numbers that would make sense to Bruce Willis"!

A CAI State of the Practice Interview with Dr. Barbara Kitchenham
Dr. Barbara Kitchenham is Professor of Quantitative Software Engineering at Keele University in the UK; she is also a Senior Principal Research at National ICT Australia. She has worked in software engineering for nearly 30 years both in industry and academia. Her main research interest is software metrics and its application to project management, quality control, risk management and evaluation of software technologies. She is particularly interested in the limitations of technology and the practical problems associated with applying measurement technologies and experimental methods to software engineering. She is a Chartered Mathematician and Fellow of the Institute of Mathematics and Its Applications. She is also a Fellow of the Royal Statistical Society. Our interview between Barbara Kitchenham and Michael Milutis, Executive Director of the IT Metrics and Productivity Institute, was conducted in June of 2006.
Click Here to Read the Interview with Dr. Barbara Kitchenham!

An Interview with Lawrence Putnam, Software Pioneer
Lawrence (Larry) Putnam is a renowned authority on software estimation and measurement. Larry was the first recipient of "Freiman" award for sustained superior performance in parametric estimation covering a ten-year period. He is the founder and has been President of Quantitative Software Management Inc. since 1978. Before establishing QSM, Larry had over 26 years of experience in software and hardware resource planning, estimating and allocation. Over the past 20 years, Larry has conducted extensive research into software cost estimating techniques using operations research modelling techniques to determine significant cost, schedule and reliability drivers. He has extensive experience in data collection and analysis techniques. Besides publishing over 30 papers on the subject of software estimation and measurement, Mr. Putnam has authored/co-authored many books, including Five Core Metrics: The Intelligence Behind Successful Software Management. Our interview between Larry Putnam and Michael Milutis, Executive Director of the IT Metrics and Productivity Institute, took place in May of 2006.
Click Here to Read the Interview with Lawrence Putnam!

End Your Frustration With Software Development 
The constant litany we hear from high-level managers about software development projects is that projects take too long and cost too much. Then they complain that they don't find out that projects are in trouble until it is too late to do anything about them. And finally they ask, "Am I getting good value for the investments I am authorizing for process improvement?" Well, you can't tell much just by looking. That is the nature of knowledge work. But still there is work going on. How have managers of earlier forms of project work solved this problem? Find out more in this article by Larry Putnam and Ware Myers. (12 pages)
Click Here to Read "End Your Frustration With Software Development"!

An Interview with Grant Rule, Master Software Metrics Practitioner
Grant Rule is a founder of Software Measurement Services Ltd, specialising in project and process appraisal, performance measurement, estimating, benchmarking and continuous improvement. Grant has some 34 years experience in IT. He is a recognised authority in using quantitative methods to continuously improve the quality of the software process and its products. Grant worked with Ken Dymond to introduce the Software Engineering Institute's 'Capability Maturity Model' into the UK, and helped bring to Europe the first public 'Introduction to the CMMI®' training. He has contributed to structured methods and to ISO standards, and helped improve the inter-counter consistency of counting practices for IFPUG and MkII Function Point Analysis. Our interview between Grant Rule and Michael Milutis, Executive Director of the IT Metrics and Productivity Institute, took place in August of 2006.
Click Here to Read "An Interview with Grant Rule, Master Software Metrics Practitioner"!

Making Measurement Work 
A successful measurement process becomes a way of doing business. Measurement is embedded in the organization, and performance improves because people are making fact-based decisions. This article by Cheryl Jones of the US Army describes characteristics of successful measurement programs using the Practical Software and Systems Measurement Initiative guidance. (11 pages)
Click Here to Read "Making Measurement Work"! 

Measurement for Process Improvement 
Specific software process improvement measures need to be established that best suit an organization. This article from 2005 by Joyce Statz discusses and highlights some of the guidance on measurent provided by the Practical Software and Systems Measurement community since 1998. (59 pages)
Click Here to Read "Measurement for Process Improvement"!

POWER POINT: Measures for Success 
Excellent presentation produced by the US Army on lessons learned from software measurement and software process improvement initiatives. (22 pages)
Click Here to View the Presentation!

An Interview with Dr. Christof Ebert, Director of Process Improvement at Alcatel 
Dr. Christof Ebert is Director of Software Coordination and Process Improvement for Alcatel. His current responsibilities include establishing shared software platforms and leading Alcatel's global CMM/CMMI programs. A senior member of IEEE, Dr. Ebert lectures at the University of Stuttgart and serves as a keynote speaker and on program comittees of various software engineering conferences. Since the end of the 1980s, he has been an educator, researcher and consultant in software measurement. He is a member of the editorial board of the Journal of Systems and Software and is IEEE Software associate editor-in-chief. He serves on the board of the German Interest Group on software metrics within the German Informatics Society (GI). He is also the co-author of Best Practices in Software Measurement. Our interview between Christof Ebert and Michael Milutis, Executive Director of the IT Metrics and Productivity Institute, took place in June of 2006.
Click Here to Read "An Interview with Dr. Christof Ebert"!

POWER POINT: Measuring and Improving Productivity 
Useful and informative slide presentation created by Dr. Christof Ebert that 1) defines productivity; 2) outlines how to measure it; and 3) explains how to improve it.
Click Here to Read "Measuring and Improving Productivity"!

Planning the Measurement Process 
This excerpt from Dr. Christof Ebert's book Best Practices in Software Measurement considers the aspects of planning a software measurement process, including the different models, approaches and standards. The first part describes the usefulness of current measurement paradigms based on the literature in this area. The discussion of measurement planning is supported through an industrial example of the evaluation of customer satisfaction, which was investigated in a project at Deutsche Telekom. (11 pages)
Click Here to Read "Planning the Measurement Process"!

Saving the World One Project at a Time: Planning by Numbers 
Planning by Numbers is the concept of using historical project level metrics to aid the project manager in discovering facts about software projects.  This information can be used for project planning, validation, and risk assessment.  Software metrics provide many benefits for such things as managing outsourcing contracts, process improvement, productivity benchmarking, and balanced scorecards; however, you would be missing one of the greatest benefits, if historical software metrics were not used for making decisions and planning projects. Find out more in this article by Jim Mayes. (15 pages)
Click Here to Read "Saving the World One Project at a Time: Planning by Numbers"!

An Interview with Michael Mah 
Michael Mah is a Senior Consultant with Cutter Consortium's Business Technology Trends & Impacts, Measurement and Benchmarking, Agile Software Development & Project Management, and Sourcing & Vendor Relationships Practices. He is also owner/partner at QSM Associates Inc. Mr. Mah is a recognized expert on practical applications of software metrics, project estimation/control, and IT productivity benchmarking. Over the past 10 years, he has published numerous articles on these and other management topics. His recent work merges concepts in software measurement and benchmarking with negotiation and dispute resolution techniques for IT outsourcing and relationship management. Mr. Mah's particular interest is in people dynamics, such as the complex interactions between people, groups, divisions, and partnered companies working on the technology revolution at "Internet speed." He is also focused on the latest research and theory on negotiation, including the use of game theory, role playing, and training to increase corporate and personal effectiveness. Our interview between Michael Milutis, Executive Director of the IT Metrics and Productivity Institute, and Michael Mah took place in August of 2006.
Click Here to Read "An Interview with Michael Mah"!

Back to the Basics
Measurements and metrics are key tools to understanding the behaviors, successes, and failures of our programs and projects. This article by Tim Perkins, Roald Peterson, and Larry Smith highlights the basic principles of measures and metrics and encourages the reader to improve his or her use of these tools. (9 pages) 
Click Here to Read "Back to the Basics"!
 
A Software Metrics Primer
Software measurement is a challenging but essential component of a healthy and highly capable software engineering culture. In this article by Karl Wiegers, the author describes some basic software measurement principles and suggest some metrics that can help you understand and improve the way your organization operates. (5 pages) 
Click Here to Read "A Software Metrics Primer"!

An Interview with Dr. Hans Sassenburg 
Dr. Hans Sassenburg received a Master of Science degree in electrical engineering from the Eindhoven University of Technology (Netherlands) in 1986 and a PhD degree in economics from the University of Groningen (Netherlands) in 2006. He worked as an independent consultant until 1996, when he co-founded a consulting and training firm. In 2001 he moved to Switzerland, where he founded the new consulting firm SE-CURE AG. In addition, he has been a visiting scientist at the Software Engineering Institute since January of 2005. In 2006 he co-founded the joint venture Software Improvement Group AG, offering a portfolio to create management transparency with respect to the quality of software applications. The Software Improvement Group supports its services and products with automated source code analysis. Our interview between Hans Sassenburg and Michael Milutis, Executive Director of the IT Metrics and Productivity Institute, took place in November of 2006.
Click Here to Read "An Interview with Dr. Hans Sassenburg"!

A CAI State of the Practice Interview with Dr. David Zubrow, Director of Measurement and Analysis, SEI
Dr. David Zubrow manages the software measurement and analysis program at Carnegie Mellon's Software Engineering Institute (SEI). Since his arrival at the SEI in 1992, Dr. Zubrow has been a member of the CMMI Product Development Team and the lead developer of the Software Process Maturity Questionnaire. He is authorized by the SEI as an instructor for Implementing Goal-Driven Software Measurement, Introduction to the CMM, and Lead Assessor Training courses. He is also an authorized Lead Assessor. Dr. Zubrow serves on the Technical Steering Group for the DoD Practical Software Measurement Project and the Executive Steering Committee for the Data Analysis Center for Software (DACS).  Our interview between David Zubrow and Michael Milutis, Executive Director of the IT Metrics and Productivity Institute, took place in November of 2006.
Click Here to Read "An Interview with Dr. David Zubrow"!

Measurement with a Focus: Goal Driven Software Measurement
The benefit and value of software measurement comes from the decisions and actions taken in response to analysis of the data, not from the collection of the data. In this article by David Zubrow, Director of Measurement and Analysis at the SEI, Dr. Zubrow explains how, through analysis, measurement can be used to anticipate the future rather than to merely record the past. This is the perspective promoted by the Software Engineering Institute's (SEI) Goal-Driven Software Measurement process as well as the DoD's initiative for Practical Software Measurement. (4 pages) 
Click Here to Read "Measurement with a Focus: Goal Driven Software Measurement"!

Can You Tell Me the Industry Average?
In this article by Dr. David Zubrow, the author attempts to stimulate skepticism and inquisitiveness on the part of those seeking numbers to meet their needs. Unfortunately, there are little high-quality data easily available to answer the kinds of questions that need to be asked. Nevertheless, we must try to be as informed as possible about the quality and appropriateness of the data we are using to build and evaluate our proposals. The questions outlined in this article should be able to provide a starting point for you in evaluating your own sources of data. (2 pages)
Click Here to Read "Can You Tell Me the Industry Average?"!

A CAI State of the Practice Interview with Beth Layman of Borland Software
Beth Layman is a Senior Consultant at Borland Software with over 25 years of experience in software and systems development. Her consulting experience encompasses commercial IT, government, aerospace, and product software organizations. At Borland, Beth is currently performing internal management consulting to establish IT management and governance and Enterprise PMO processes and tools. Prior to joining Borland, Beth was a Senior Consultant at Lockheed Martin providing software measurement and process improvement consulting support to commercial, government, and Lockheed organizations.Beth is a co-author of "Practical Software Measurement: Objective Information for Decision Makers." She is an SEI-authorized SCAMPI Lead Appraiser for CMMI. Our interview between Beth Layman and Michael Milutis, Executive Director of the IT Metrics and Productivity Institute, as conducted in December of 2006.
Click Here to Read the Interview with Beth Layman!

Strengths and Weaknesses of Software Metrics
From a distance, the software metrics domain is fragmented, incomplete, and gives the appearance of being more influenced by "metrics politics" than by technical considerations. In this report, Capers Jones analyzes key software size metrics and explores the underlying technical problems associated with software measurement in general. He also summarizes the current state of software metrics research. (17 pages)
Click Here to Read "Strengths and Weaknesses of Software Metrics"!

When Good Numbers Go Bad
Whether you are a function point specialist, project manager or metrics guru one of your roles is to act as a steward of the numbers and a high priest of information. This article by Thomas M. Cagely Jr. explores the stark realities of how measures can go wrong while suggesting how to address such realities.(21 pages) 
Click Here to Read "When Good Numbers Go Bad"!

The Challenge of Productivity Measurement
In an era of tight budgets and increased outsourcing, getting a good measure of an organization's productivity is a persistent management concern. In an article from the Pacific Northwest Software Quality Conference, David M. Card discusses the key considerations for defining an effective productivity measure while exploring the relationship between quality and productivity. (10 pages) 
Click Here to Read "The Challenge of Productivity Measurement"!

Software Size Uncertainty
Business decisions about a software project are made at the time of minimum knowledge and maximum uncertaintly about a software product's effective size. Mike Ross proposes definitions for and the relationship between two key contributors to software size uncertainty: growth and estimation process variability. (8 pages) 
Click Here to Read "Software Size Uncertainty"!

The Practical Application of Software Measurement
IT has become an important part of organizations' strategy, competitive advantage and profitability. There is management pressure to build systems faster, better and at minimum cost. The return on investment that an organization can get from the money it spends on IT has come under increased scrutiny from senior business executives and directors. Consequently, IT now has to operate like other parts of the organization, being aware of its performance and its contribution to the organization's success and opportunities for improvement. How can IT executives achieve this without performance data? They can't. So what is it that managers need to know? Find out in this article by Peter Hill, Executive Director of the ISBSG. (6 pages)
Click Here to Read "The Practical Application of Software Measurement"!

Strengths and Weaknesses of Software Metrics
The software industry lacks standard metric and measurement practices. Almost every software metric has multiple definitions and ambiguous counting rules. There are also key topics with no metrics at all, such as quantifying the volume or quality levels of data bases, data warehouses, and web sites. The result of metrics problems is a lack of solid empirical data on software costs, effort, schedules, quality, and other tangible matters. This report by Capers Jones analyzes some of the key software size metrics and the underlying technical problems associated with software measurement. (17 pages)
Click Here to Read "Strengths and Weaknesses of Software Metrics"!

Creating a Metrics Program: Step #1 - Document the Software Process
A software development process is simply the procedures that are followed to transform specified requirements into a software product. The objective of this step is to document these procedures so they can be applied consistently across projects. One of the benefits of a metrics program is that it forces a company to focus on how it develops software. There are always areas of the software development process that can be improved. However, before improvements can be made to the development process, the process has to be identified. (3 pages)
Click Here to Read "Creating a Metrics Program: Step #1 - Document the Software Process"!

Creating a Metrics Program: Step #2 - State the Goals
Every company will have different reasons for establishing a metrics program. If specific goals have already been determined, document them. If goals have not been determined, you should refer to the table in Figure 3.1. This table lists some of the goals that commonly drive metrics programs. The goals presented are all closely related and rely on much of the same data. The differences lie in which metrics will be constructed from the data, and how these metrics will be used. (3 pages)
Click Here to Read "Creating a Metrics Program: Step #2 - State the Goals"!

Creating a Metrics Program: Step #3 - Define Metrics Required to Meet Goals
The goals for a metrics program determine the specific metrics on which the program will be based. This section analyzes each of the eight goals outlined in Step 2. The goals will be converted into a set of metrics by applying the Basili GQM (Goals/Questions/Metrics) model (23 pages)
Click Here to Read "Creating a Metrics Program: Step #3 - Define Metrics Required to Meet Goals"!

A CAI State of the Practice Interview with Roberto Meli
Roberto Meli is an expert in project management, software development methodologies, and software metrics. He attends to and teaches training courses in Italy and foreign countries and is a coordinator of GUFPI - ISMA CPC (Counting Practices Committee, Technical Committee for the Rules of function point counting). Since 1990 he has been the Managing Director and CIO for DPO. Our interview between Roberto Meli and Michael Milutis, Executive Director of the IT Metrics and Productivity Institute, took place in August of 2008.
Click Here to Read the Interview with Roberto Meli!

100 IT Performance Metrics
The mission of an IT organization is to provide an information processing capability that benefits the business.  In order to fulfill this mission IT must provide the following services while managing costs and prioritizing requests to optimize value: Operate and support the infrastructure required to process, store, secure, and communicate information; Operate and support the business applications that process information; Provide technology consulting, training, and planning services; Employ, train, and deploy staff required to provide these services; Plan, develop/purchase, test, and implement new infrastructure or software to fix problems or provide enhanced information processing capabilities to the business.This document identifies data elements for measuring IT performance by related categories.  Some of the metrics represent averages while others are reported in the form of a graph.  By reporting these metrics on a regular basis (monthly is the minimum recommended reporting period), trends can be observed across the reporting periods.  In many cases the trends are more important than the actual value.   
Click Here to Read "100 IT Performance Metrics"!

Counting Lines of Code:  Virtually Worthless for Estimating and Software Sizing
In 2005 CrossTalk published an article that examined the use of lines of code (LOC) as the basis for the size of a software application. [1]  That paper was based on the data collected from Personal Software Process (PSPSM) students who developed the same programs as part of the PSP course.  The variation in the resulting products (software programs) was reduced in part by the classes being taught by the same SEI-authorized PSP instructor, using the same program descriptions, and using the same acceptance criteria reviewed by the same instructor.  Thematically, that paper warned of the unpredictable nature of lines of code when used for software measurement or estimating.  Questions regarding human performance were only briefly introduced in that article.  This article provides the background, data, and preliminary conclusions for the programs developed in those classes with insights around human performance.  While potentially unpopular, the use of quantitative data on human performance in the IT world is not unlike personal performance measures common in other industries. (3 pages)   
Click Here to Read "Counting Lines of Code:  Virtually Worthless for Estimating and Software Sizing"!

Is There Value to using Lines of Code for Measuring People After All?
In Counting Lines of Code: Virtually Worthless for Estimating and Software Sizing, an argument was made that it was time to measure individual performance for software development much as is done in other industries. Statistically analyzed data was used to discover unpredictable swings in software size. Is There a Weakest Link After All? examined programmer data to finds patterns of repeated lengthy products. Eliminating under performing programmers resulted in statistical improvement for the classes, as to be expected. In this last article, all the class groups are combined into a single population and the same criteria applied to discover under performers. Finally, reasons for why more code is not good, are offered. (2 pages)   
Click Here to Read "Is There Value to using Lines of Code for Measuring People After All?"!

Is There a Weakest Link After All?
In the first part of this "trilogy" data that was analyzed previously, evidenced statistically significant variation in the size of software programs developed under semi-controlled (but more controlled than other published data) conditions. The intent of the re-examination was to determine if patterns in the data indicated a "weak link" in the programming staff (in this case represented by a class of programmers). (3 pages)   
Click Here to Read "Is There a Weakest Link After All?"!

Don't Generate Countless Measurement Specifications
New project, process, product, and organizational information needs arise all the time. In this article by Pat Otoole, Pat explains why we should focus our attention on establishing the measurement infrastructure to address these new needs - rather than investing a lot of time backfilling measurement specifications (in an attempt to solve problems that don't exist). (2 pages)   
Click Here to Read "Don't Generate Countless Measurement Specifications"!

Do Define What Counts - and What Doesn't
This article by Pat Otoole examines how to create operational definitions for metrics without falling into the trap of saying too much. (2 pages)  
Click Here to Read "Do Define What Counts - and What Doesn't"!

Do Weigh These Thoughts Carefully
If you weigh yourself on a regular basis, you've probably considered, at least implicitly, that consistency across measuring sessions increases integrity. Applying such guidelines to software leads to capturing data with greater integrity, while breaching - or overlooking such guidelines - reduces that integrity, as well as the corresponding accuracy of any subsequent analysis based on the tainted data. (2 pages)   
Click Here to Read "Do Weigh These Thoughts Carefully"!

Seven (+/- Two) Software Measurmenet Conundrums
Since computer science has traditionally considered itself more an application of discrete logic than of continuous functions, measurement has not played as important a role in software as it has in other areas of engineering. Since few computer science programs teach courses in measurement theory, statistics, experimental design, or related areas, software engineering has not benefited from rigorous development as a quantitative science. This article by Dr. Bill Curtis discusses several current problems facing the quantitative study of software engineering that present challenges to the software measurement community. (11 pages) 
Click Here to Read "Seven (+/- Two) Software Measurmenet Conundrums"!

From MCC to CMM: Technology Transfers Bright and Dim
This paper describes lessons learned during the author's five lives in technology transfer. The author, Dr. Bill Curtis, proposes a career model of technology transfer. (11 pages)   
Click Here to Read "From MCC to CMM: Technology Transfers Bright and Dim"!

Measuring Up to Metrics: How I Became a Missourian
The use of metrics in software project management, often referred to as management by data, was the subject of a survey we conducted with the Cutter Consortium. The goal of the survey was to learn how software development organizations collect, analyze, and use metrics. In this
report on the survey, E.M. Bennatan will look at the software development scorecard. He will examine the problems encountered by organizations in collecting and analyzing metrics and will review some of the most common methods and practices.  Finally, Elli will attempt to conclude whether software developers have earned the right to be called Missourians (at least in an honorary capacity). (7 pages)   
Click Here to Read " Measuring Up to Metrics: How I Became a Missourian"!

Top

Function Points

A CAI State of the Practice Interview with David Herron, Author of "Function Point Analysis"
David Herron is a Principal in The David Consulting Group. Mr. Herron is an authority in areas such as Functional Measurement and Software Process Improvement. He has over 25 years of experience in software development. During the past ten years he has served as a consultant to Fortune 1000 companies in the areas of software metrics, software process improvement and applications outsourcing management. He is an acknowledged authority in the measurement and estimation of software productivity and quality, specializing in the determination of software project size, effort and cost. His engagements have supported clients on the use of metrics to monitor the impact of IT on the business, on the advancement of IT organizations to higher levels on the Software Engineering Institute's Capability Maturity Model and on the governance of offshore outsourcing arrangements. Mr. Herron is also the author, along with David Garmus, of Function Point Analysis: Measurement Practices for Successful Software Projects. Our interview with David Herron and Michael Milutis, Executive Director of the IT Metrics and Productivity Institute, took place in April of 2006.
Click Here to Read the Interview with David Herron!