# Software Metrics Courses Summary

[TOC]

## Lecture 1 : Introduction :

### Definition of measurement :
"Measurement is the **process** by which **numbers** or **symbols** are assigned to **attributes** of **entities** in the **real world**in such a way as to describe them according to **clearly defined rules**"

- Entity is an object (a system for example)
- Attributes are features or property of an entity (size, understandability, ...)
- Numbers and symbol are abstraction that we use to reflet our perceptions of the real world

**It is important to differentiate entity and attribute, as measure an entity is ambiguous, when measure a attribute of an entity is correct.**

### Measurement in real world of Software Engineering :
Has it has been considered a luxury for most project, **we fail to** :
- Set measurable target for our project,
- Understand and quantity components cost,
- Make serious study to determine if a technology is efficient or not.
- Do measurements frequently, consistently, completely.

### Objectives for software measurement :

Assesing the status of our product, motivated by a particular goal or need :
- For managers :
- Process cost,
- Staff productivity,
- Code quality,
- User satisfaction,
- Improvement of processus.
- For engineers :
- Requirement strict enough,
- All fault find,
- Met the process/product goals.

Measurement is for :
- Understanding,
- Control,
- Improvement.
But for every previous step, the user of the data should always be aware of **limited accuracy of prediction**, and **the margin of error of measurements.**

## Lecture 2 : Basics of measurements :

### Representational theory of measurement
In any measurment activity there are **rules** to be followed for **consistent** measurement, and basis for **interpretation** :
- Empirical relation :
- The representational theory of measurement seeks to formalize our intuition about the way the world work
- Manipulation of collected data should preserve the relationship that we observe among entities
- To sum up : we have to preserve **intuition and observation**.
- Measurement is mapping from empirical to formal :
- Real world is domain, mathematical world is the range,
- A measure is the **number of symbol** assigned to an entity by this mapping, in order to **characterize** an **attribute**

### Direct and indirect measures :
Direct measure of an attribute of an entity involves no other attributes or entity.
For example :
- **Distance travelled and time** take are **direct measures**, when **speed** using the two previous measure, and hence is **indirect**.
- Fault is a direct measures. Faults per kloc is a indirect measure.

### Scale type :
There are different types of measurements scales for metrics :
- **Nominal**,
- Most primitive for of measurement
- Define class or categories and place entities in particular class based on value of attributes.
- Characteristics :
- Entities are only organized in different classes,
- No notion of ordering among the classes,
- Any numbering or symbolic representation of the classes are acceptable, but there is no notion of magnitude associated between the number or symbole (no magnitude of steps)
- Example : Classify people per gender.
- **Ordinal**,
- Augment nominal scale with ordering relation between class or categories,
- Characteristics :
- Entities are ordered with respect to the attributes,
- Any mapping that preserve the order is acceptable,
- The numbers represent ranking only, so addition/substraction/mean would be meaningless.
- Complexity of modules with fives classes : "trivial","simple","moderate","complex","incomprehensible"
- **Interval**,
- Augment nominal and ordinal scale with the size of the jump from one class to another.
- Characteristics :
- An interval scale preserves order,
- An interval scale preserves differences but not ratios,
- Addition and substraction are acceptable but not multiplication and divsion.
- Very few example in SE :
- Ordinal scale example but difference in complexity between trivial and simple is the same as between simple and moderate system etc.
- Air temperature
- **Ratio**,
- Characteristics :
- Preserve ordering, size of intervals between entities, and ratios betweend entities.
- There is a zero element representing total lack of attribute.
- Measurement mapping must start at Zero and increase at equals intervals, know as "unit"
- All arithmetic can be meaningfully applied
- The zero element can exist as a limit, as heigh can have no attributes, but thing get smaller and smaller.
- Example : Length of software Code :
- Entity : code,
- Attribute : length,
- Measure : LOC
- **Absolute**, is the most restrictive of all,
- Characteristics :
- For an absolute scale is made simply by counting the number of element of an entity set,
- Take the forms of **Numbers of occurences of x** in the entity,
- There is only one possible measurement mapping, the acutal **count**,
- **All arithmec analysis** of the resulting count is meaningfull.
- Examples : Number of bugs.

Considering the scale help us to select the suitable one for measuring an attribute, and pick the right way to analyze/interpret the measures.

### Meaningfullness of measurement

For example based on the scale, we can know which method to apply to measure central tendancy and dispersion (gaussian test)

- **Nominal**,
- Relations : Equivalence
- Statistic : Mode, Frequency.
- **Ordinal**,
- Relations : Equivalence, Greater Than,
- Statistic : Median, Percentile, Spearman, Kendall.
- **Interval**,
- Relations : Equivalence, Greater Than, Step of any interval
- Statistic : Mean, Standart deviation, Pearson correlation.
- **Ratio**,
- Relations : Equivalence, Greatern Than, Step of any interval, Step of any two steps value,
- Statistic : Geometric mean, coefficient variation

- Objectives measure : make sure that different people produce the same measures results
- Subjectives measure : depend on environment and can vary with the person measuring. Still be usefull if we can understand their imprecision.
- Derived measure :
- Measuring complex attribute in terms of simpler sub-attributes,
- Many SE attributes are not directly measurables such as software quality that we split into (High cohesion, low coupling)
- Same rules of measurement are applicable.

## Lecture 3 : Goal Question Metric (GQM) framework :

### Goal based measurement
In goal based measurement, we don't focus on measurements that we should use, but **what we want to improve**. It not about how much data, but rather having the **exact information** you need to **understand, manage and improve** your processes, products, and business.
There is two point to focus on :
- What to measure :
- Entities to be examined (Process,Product,Ressource)
- Relevant goals
- How to measure :
- Relevant metrics to used.

In software Engineering, there is three main entities :
- **Process**,
- Software related activities (maybe with timescale) such as
- development,
- maintenance,
- testing
- reuse,
- management
- **Product**,
- Any artifact, deliverable or document that result from a process activities
- **Ressource**,
- Entities required by a process activity

### Attributes types :

There is two type of attributes :
- Internal attributes :
- Measurable entirely in terms of the process, product, or resource itself,
- They can be measured by examining the process, product, ressource ** on its own, seperate from its behavior**
- External attributes
- Attributes that can be measured only with respect to how the proces, product or ressource related to its environement,
- Here the behavior of the process, product, ressource is important rather than the entity itself.

We mostly want to measure external attributes such as cost effectiveness,
but they are more difficult to measure, and came late in development, so using internal attributes we want to make **inferences/judgment** about externals. If we manage to find **relationship between internal and external attributes** we can do so.

But internal attributes are also important as SE methods give structure to software products. The structure make the product easier to understand, analyze and test. The structure involve two aspects :
- The development process,
- The products themselves.
The product structure can be asses by levels of internal attributes such as modularity, coupling, or cohesivness.

### GQM approach
The goal-question-metrics approach to process metrics provides a framework for deriving measures from organization or business goals.

#### Basisili GQM Process
- [ ] Developing a set of corporate, division, and project goals.
- [ ] generating questions that define those goals as completely as possible **in a quantifiaable way**
- [ ] specify metrics needed to be collected to answer those questions
- [ ] Developping mechanism for data collection
- [ ] Collecting, validating, and analyzing the data in real time to provide feedback to project for corrective action, to asses conformance, and make recommandation for future improvements.

There is three main components :
- Goal : List major goals of development or maintenance
- Question : derive from each goal the question that mus be answered to determine if the goal are being met
- Metrics Decide what must be measured in order to be able to answer the question adequately

For goal template, see lecture 3, slide 16 [here](https://bth.instructure.com/courses/2510/files/282986?module_item_id=50235).

### Measurement and process improvement
Measurement offers visibility in which the processes, products, ressources, methods, and technologies of software development relate to one another.
It is useful for :
- understand,
- etablishing a baseline,
- assesing and predicting.
but only in the **larger context** of assesment and improvement.

There is 5 level of process maturity which process and management can upgrade from one level to another :
- Level 1 : **Initial**, Baseline measurement are needed, transition from input to ouput undefined and uncontrolled
- Process discipline
- Project management
- Level 2 : **Repeatable**, indetified inputs, outputs, constraint and ressources. A subroutine is repeatable.
- Process definition
- Engineering management
- Level 3 : **Defined**, provied visibility into the creation of process. Intermediate activites are defined, and in-ouput are known, which mean examined, measured and assesed.
- Process Control,
- Quantitative management
- Level 4 : **Managed**, add management oversight to a defined process, and the use of feedback to set priorities for current and later-on activities.
- Continuous process improvement
- Change management
- Level 5 : **Optimizing**, measures are used to improves the process, by adding are removing activities, and changing the process structure dynamically in response to measurement feedback. It affect **the organization**, **project** and the **process**

Combining Process maturity and GQM approach help to identify the metrics which will be the most useful for the organisation at a particular moment in time.

## Lecture 4 : Empirical Studies

### Principles of empirical studies

#### Study type and control on variables

- **Survey**,
- It is a retrospective study of a situation to try to document relationship and outcomes
- Scale : research in the large
- **Case Study**,
- Not retrospective, but can be made retrospectively.
- Scale : research in the typical
- **Experiment**
- Not retrospective, and **involve** testing of well defined hypotheses concerning postulated **effect of independant variables** on **dependent variables** in a setting that **minimizes other factors** that might affect outcomes
- Scale :
- research in the small

Empirical studies that involve **observation** where potential **confounding factors** can not be controlled are called observational studies / quasi experiments.
A case study is a quasi experiment where you dientify key factors that may affect the outcome of an activity and then document the activity.

#### Study goals and hypothesis
- **Goals** help you to decide which type of research method is the most appropriate for your situation. They can be formulated in form of hypothesis.
- **Hypothesis** is the **tentative idea** that you think explain the behavior you want to explore

#### Threat to validity

there is multiple source of threat to an empirical studies such as :
- **Conclusion validity**
- Threats to conclusion validity include using **wrong statistical tests**, having too **small a sample**, and searching for relationships between too many variables
- **Construct validity**
- Threats to construct validity include **using measures** that are **not relevant to the study** and/or are not meaningful.
- **Internal validity**
- It relates to the **cause-effect relationship** between independent and dependent variables. A study has internal validity if the treatment actually caused the effect shown in the dependent variables.
- **External validity**
- It refers to how well can you **generalize** from the results of one study to the wider world

### Relevant and meaningful studies
- **Confirming theories and conventional wisdom**
- Confirm a common notion that pair programming increases code quality
- **Exploring relationships**
- How does the design structure affect the maintainability in the code
- **Evaluating accuracy of models**
- **Validating measures**
- A measure is said to be valid if it reflect the characteristics of an attribute under different conditions.

### Define data :
- [ ] Process/Product/Ressource -> Data collection
- [ ] Raw data -> Extraction
- [ ] Refined data -> Analysis (Limit between direct and indirect measurement)
- [ ] Derived attribute value

### How to collect data :
- Manual data collection,
- Automated data collection,
- Guidelines :
- KISS (keep it stupidly simple)
- Avoid unnecessary recording
- Train staff in the need to record data
- Provide results of the data collection and analysis to original providers promptly and in a useful form.
- Validate
- Data collections forms
- There should be clearly defined to minimize biases and increase reliability and consistency
- Interview protocol/guide
- What are the meaningful question and in which order they can be asked
- Data collection tools
- There are many tools that support the recoding and tracking of software fault and attributes

### Decision tree for analysis techniques :
See the lecture 4 slide 17, [here](https://bth.instructure.com/courses/2510/files/287225?module_item_id=50681)

## Lecture 5 : Internal Product Attributes : Size and Structure
TODO
## Lecture 6 : Measuring External Product Attributes

### Software Quality

#### Modeling Sofware quality



#### Quality standards
- ISO 9126-1 :
- Functionnality,
- Reliability,
- Efficiency,
- Usability,
- Maintainability,
- Portability.
- Software Quality equirement and Evaluation (SQuaRE) :
- Functional **suitability**,
- **Performance** efficiency,
- **Compatibility**,
- Usability,
- Reliability,
- **Security**,
- Maintainability,
- Portability.

### Defects based quality measures
- Defect density measures
- Defect density = N° of defect / product size
- System spoilage
- Time to fix post release defects / total system development time

#### Limitation of defect density measures
A defect can be either a fault or a failure
- Fault : A condition that causes the software to fail to perform its required function
- Failure : It is the inability of a system or component to perform required function accorsding to its specification

There is multiple (as definition is ambiguous) metrics for measures of defect :
- Post release failures,
- Residuals faults,
- All know faults,
- The set of faults discover after some fixed point in SDLC (software developement life circle)

It is a product measures or process measure or both ?
A lot of problem is around this notion, and company still used them, arguing that they have they own formal definition and aplly it consistently in their environments. (So not scalable, etc.)

### Usability and Maintainability

#### Usability definition

Usability is the degree to which a product or system can be used by specified **users to achieve specified goals** with **effectiveness, efficiency and satisfaction** in a specified context of use

Of an external view :
- **Effectiveness** measures indicate the degree to which users can correctly complete tasks
- **Efficiency** measures generally involve time required to complete tasks.
- **Satisfaction** measures indicate subjective notions of the quality of interaction with a system

Of an internal view :
- Collecting early data on usability is difficult, before system is developped,
- A way to bypass this is looking to internal attributes that lead to good usability
- Well structured manuals,
- Good use of menus and graphics,
- Informative error messages,
- Help functions,
- Consistent interface.
- We can measure the previous attributes, however we cannot define them to be measure as they have to be well written and we have nothing to prove this.

#### Maintainability defintion

There is 4 major type of maintenance :
- Corrective,
- Adaptive,
- Preventive,
- Perfective.

Of an external view :
- Mean Time To Repaire **(MTTR)**
- Problem recognition time,
- Administrative delay time,
- Problem analysis time,
- Change specification time,
- Change time (with teest and review)
- Ratio of total change implementation time to total numbers of changees implemented
- Number of unresolved problems
- Time spent on unresolved problem
- Percentage of changes that introduce new faults,
- Number of modules modified to implement a change

**All of theses measures paint a picture of the degree of maintenance acitivity, and the effectiveness of the maintenance process.**

Of an internal view :
- A number of the **complexity measures**, discussed in the previous lectures that correlate to levels of maintenance effort.
- We need to remember that correlation with a characteristic doe not make something a measure of characteristic
- Some of the structural measures can be used for "risk avoidance", and we may for example , identify a particular module having measurably poor structure.

## Lecture 7 : Software effort/cost estimation
TODO
## Lecture 8 : Software process improvement and measurement program
TODO