Evidence-Based Software Portfolio Management Hennie Huijgens Delft University of Technology, Delft, The Netherlands PDF Delft University of Technology Evidence Based Software Portfolio pure tudelft nl ws files 37694615 Thesis Hennie Huijgens final pdf PDF Evidence Based Software Portfolio Management CEUR Workshop ceur ws Vol 1469 paper4 pdf PDF Evidence Based Software Engineering And Systematic
PDF Evidence for adaptive changes in egg laying in Semantic Scholar pdf s semanticscholar 8b20e6f99e826ed7185fec2a37aab899e988 pdf PDF Parasitic manipulation Mivegec IRD mivegec ird images stories PDF files 0219
7 Sep 2010 syrup in food for seasoning and they used no salt for preserving food first recorded evidence is from 1623 when Étienne Brulé, who was cruel and deadly lesson of the Kinmount and Muskoka catastrophe Þorvaldsson
ótica da evidenciação voluntária dos elementos do Capital Intelectual nos Relatórios da Administração disponibilizados pelas companhias, utilizando como Resumo O Capital Intelectual (CI), nos últimos anos, tornou se mais relevante para as organizações É notório que sua evidenciação interfira como nova fonte Evidenciação do Capital
PDF Condiciones del Contexto para el Emprendimiento Negocios UDDnegocios udd cl files 2011 05 GEM 5 AÑOS completo FINAL pdf PDF 2 principales aportes teóricos sobre emprendimiento Municipiosmunicipios unq edu ar modules MonografiaVersionFinal pdf PDF Emprendimiento e
La medicina actual está cambiando drásticamente, en los últimos años, durante exposiciones científicas se escucha con frecuencia la expresión “la evidencia evidencia y la importancia que ha tenido sobre la medicina conductual habilidades para la búsqueda y aplicación de la evidencia en el caso en turno,
7 Sep 2007 Medicina Basada en la Evidencia o en Pruebas • Uso consciente, explícito Los resúmenes de la “web temática de la espalda” de la Fundación Kovacs Guías clínicas en Fisterra, fisterra guias2
4 Feb 2010 Evidencias en el tratamiento con hormona del crecimiento Nuevas indicaciones Emilio García García Unidad de Endocrinología Pediátrica La Hormona de crecimiento (GH) es la hormona de la glándula pituitaria En efecto hay evidencia de que la terapia GH
PDF Evidenzbasierte Leitlinie der Deutschen Diabetes Gesellschaft deutsche diabetes gesellschaft de Leitlinien Evidenzbasierte Leitlinien EBL Alter 2004 pdf PDF S3 Leitlinie Therapie des Typ 1 Diabetes, 2 Auflage Deutsche deutsche
PDF Evidenzbasierte Medizin und systematische Übersichtsarbeiten cochrane de Blumle EbM Cochrane DerMKG Chirurg pdf PDF Systematische Übersichtsarbeiten zu Fragen der Therapie und ebm netzwerk de was ist ebm timmer ebm teil 1
Evidence-Based Software Portfolio Management Hennie Huijgens Delft University of Technology,
combination with an approach to measure and analyze stakeholder satisfaction and perceived value of software projects.
ABSTRACT In this paper,
we describe the research proposal for an approach for Evidence-Based Software Portfolio Management
a new way to help software companies in steering their software portfolio’s based on cost,
defects found on the one hand and stakeholder satisfaction and perceived value on the other.
the identification of success and failure factors for software projects,
and the collection of data on finalized software projects from portfolios of different companies in a research repository.
In this paper,
we describe the research proposal for the development of evidence-based software portfolio management as a practical approach on organizing and decision-making with regard to large portfolios of software projects in information-intensive companies.
the main contributions in the current state of the research are: 1.
We propose a Cost / Duration Matrix as an instrument for analysis of good practice and bad practice in large,
companywide portfolios of software projects.
We identify success and failure factors for software projects,
based on analysis of a large subset of data of finalized software projects from three different companies.
We analyze series of software releases in order to identify additional factors that contributes to projects being best-inclass.
We propose a light-weight value measurement technique based on quantitative analysis and post-project interviews.
We contrast these core metrics with collected data on stakeholder satisfaction and perceived value,
and look for links between them.
Categories and Subject Descriptors D.2.8 [Software Engineering]: Metrics – Process Metrics,
INTRODUCTION The goal of data-driven software portfolio management is to use project data collected from the past to predict and monitor the success of other software projects,
In such a portfolio management perspective,
project duration and post-release defects is a common practice.
these core metrics only tell a part of the story,
and as such companies should be careful in steering their software project portfolios on these data points alone.
The remainder of this paper is organized in the following way: In Section 2 we outline relevant prior work.
In Section 3 we describe our research objectives and questions.
Section 5 is about the most important metrics with regard to our research and Section 6 is about the data analysis methods and techniques that we apply.
In Section 7 we evaluate validity threats.
after all be possible that a specific project costing twice as much as typical for its size would still be highly valuable to the organization.
but especially in environments that use agile approaches additional goals enter the arena,
such as early delivery of valuable software and an increased focus on stakeholder satisfaction.
BACKGROUND AND RELATED WORK In this section we describe a brief survey of the background of our research area and related work with regard to our research subject.
Where many other studies use either a quantitative approach (e.g.
analyze core metrics) or a qualitative approach (e.g.
perform surveys or interviews) to analyze software projects,
we combine both ways and look at a company’s software project portfolio from a holistic point of view.
data-driven approach on analysis of finalized software project portfolios with a qualitative,
survey-based approach in order to identify factors related to project success and failure,
A common idea of many research performed in the former millennium is that success and failure of software projects are interconnected with process-based activities: in other words,
follow the process and success will come       .
More recent work emphasizes the success and failure factors of shorter iterations due to an agile way of working   .
From the millennium onwards concepts such as agile and added value become important factors in software engineering .
And one of the effects that can be seen in industry nowadays is that the instruments of the “old” world,
such as algorithm based estimation,
and measurement and analysis seem
not to go together with “new” tools such as story points,
and less focus on control and documentation.
RQ6.1: Is process quality,
measured in number of defects found during a project,
an early indicator for stakeholder satisfaction
Shepperd argues that “the primary goal of more accurate cost prediction systems remains largely unachieved” .
as is for example showed in recent research that undermines the longlasting application of algorithmic cost models .
Nevertheless the need for good economic models will grow rather than diminish as software becomes increasingly ubiquitous .
Besides that combinations of effort estimation methods in many cases show better results than single effort estimation methods .
RQ7: In what way are project size,
and process quality (measured in number of defects) in a software project portfolio correlated with perceived value of finalized software projects
measured in function points (FPs),
an early indicator for perceived value of finalized software projects
In practice many software companies perform benchmarking of their software activities,
often based on measurement of the functional size of projects and software applications .
a growing variety of available benchmarks,
including large differences in outcomes of analyses on different benchmark sources,
does not always help to make life easier for decision makers involved in software development .
PROPOSED APPROACH In this section we describe the approach that we use in order to answer the research questions as stated above.
Because we perform our research in close cooperation with software companies (to be read as information-intensive companies,
we set-up the research in a way that fits with practice.
we perform case studies  .
The case studies that we perform are mixed studies: we perform both quantitative and qualitative research on the subject projects within a portfolio as a whole of a company or organization.
Our focus is not to study single software projects,
but instead look at the effects of all software projects performed over a period of time in a portfolio as a whole
by doing so we assume to analyze both good practice projects and bad practice projects.
a clear link with existing approaches that focus on measurements such as project size,
and number of defects is not to be found.
A challenge in industrial practice is that usually several approaches for estimation,
and benchmarking of software projects are in place and that,
replacing an organizational process at once is not feasible due to technical and social issues.
as for example is found in .
supplemented with nonstructured interviews as techniques to challenge findings from the quantitative analysis.
although it is difficult to quantify,
motivation is considered to an important factor in software developer productivity.
There are also suggestions that low motivation is an important factor in software development project failure .
A precondition that limits our research approach is the fact that we perform research in real,
live organizational environments.
Therefore the approach must not interfere with the daily operation of the studied software projects.
and analysis is usually to be useful for improvement purposes in daily operations.
RESEARCH OBJECTIVES We define as our research objective: “an integrated approach that links existing measurements based on project size,
and defects of software projects with a limited set of relatively easy to collect additional measures on stakeholder satisfaction and perceived value,
and additional qualitative research on the backgrounds of project success and failure”.
IMPORTANT METRICS For our research we make use of an existing data set of 352 finalized software projects from three different organizations.
the research repository will mature during the development of the research,
both in number of projects and in applied metrics.
Based on the above we define the following research questions: RQ1: In what way are project size,
and process quality (measured in number of defects) in a software project portfolio correlated
? RQ2: What factors can be found that influence a company’s software project portfolio in a positive or negative way
Based on the collected metrics as inventoried in Table 1 we calculate three key performance indicators.
? RQ4: Are function points (FPs) compatible with story points (SPs) on agile projects
evidence-based pricing approach for software engineering,
be used as a single instrument (without a connection with expert judgment),
in distributed environments to create cost transparency and performance management of software project portfolios
Cost per FP
For all three indicators we use project size (FPs) as the weighting factor (instead of number of projects).
In order to measure stakeholder satisfaction and perceived value we ask all stakeholders of a finalized software project (e.g.
and tester) to rate scores for both metrics on a 5-point scale.
Stakeholder satisfaction is measured for both satisfaction with regard to the projects process and the deliverables of the
RQ6: In what way are project size,
and process quality (measured in number of defects) in a software project portfolio correlated with stakeholder satisfaction of finalized software projects
the number of projects in the repository will grow over time.
Year of Go Life
Project Keyword (KW)
Definition of Project Factors Identification code of the company where a project was performed
three companies were applicable (nr.
of occurrence between brackets): B1 (206),
Year when a project was finalized
the following years Go Live were applicable: 2008 (32),
the following BD were applicable: Finance & Risk (54),
Client & Account Management (incl.
Savings & Loans (40),
Data warehouse & BI (18),
the following PPL were applicable: JAVA (154),
unknown was what specific languages were applicable here),
unknown what specific language was applicable) were referred at as Other.
Classification of the used delivery model
two DM were applicable: Structured (e.g.
Classification of the development
the following DC were applicable: New development (173),
Major enhancement (25-75% new) (124),
Minor enhancement (5-25% new) (27),
on one project no keyword was mapped)
the following keywords were applicable: Singleapplication (270),
Phased project (part of program) (65),
Technology driven (58),
Multi-application release (21),
Package off-the-shelf (1).
Size of a project in Function Points (FPs).
Duration of a project in Months
measured from the start of Project Initiation to (technical) Go Live.
measured from the start of Project Initiation to (technical) Go Live.
Effort Ratio 352 Effort spent in a project in Person Hours (PHRs)
measured from the start of Project Initiation to (technical) Go Live.
Defects Ratio 172 The number of errors or faults found in a project from System Integration Test to (technical) Go Live.
for 172 projects defects info was recorded in the repository.
due to the fact that these are different for every measured project.
from blue to red indicates the process quality (number of defects per FP) of each project,
where blue stand for a good process quality and red for a bad quality (meaning more than average defects per FP for a specific project).
The Cost / Duration Matrix (see Figure 1) is used as a model to visualize and assess the performance in terms of cost,
duration and quality of software projects based on four quadrants that describe specific characterizations  :
DATA ANALYSIS TECHNIQUES In our research we make use of different data analysis techniques,
as described in the following paragraphs.
This matrix is a model based on power regression of project cost (Euros) versus project size (FPs) and project duration (months) versus project size (FPs).
For both regressions the percentage deviation from the mean is calculated for each software project.
depending on the size in FPs of a specific project.
Good Practice (upper right): This quadrant shows software projects that scored better than average of the total repository (or a specific subset of the repository) for both cost and duration.
Cost over Time (bottom right): In this quadrant software projects are reported that scored better than the average of the total repository (or a specific subset of the repository) for cost,
yet worse than average for duration.
Bad Practice (bottom left): This quadrant holds software projects that scored worse than average of the total repository
The Cost/Duration Matrix,
representing a subset of finalized software projects.
(or a specific subset of the repository) for bot cost and duration.
as an approach for software companies to prioritize software activities within their software project portfolio,
based on quantification of project size,
and perceived value of finalized software projects.
We test this approach as a whole in a different information-intensive organization to examine whether the outcomes correlate with those companies that are already available in our research repository,
and to analyze whether the approach can be used in practice as a valuable addition to tools already in place for a software project portfolio management capability in organizations.
Cost over Time (upper left): In this quadrant software projects are plotted that scored better than average of the total repository (or a specific subset of the repository) for duration,
and worse than average for project cost.
Keep in mind that the underlying nominator for all software projects in the Cost / Duration Quadrant is functional size (measured in FPs).
and value of projects with different sizes with each other.
We develop a Software Project Benchmark Tool that enables the functionality for practitioners in industry to benchmark a subset of finalized software projects against our research repository.
The tool is based on the Cost / Duration Matrix and makes it possible for measurement practitioners in industry to upload a subset of finalized software projects from their own organization and to benchmark the performance in terms of cost,
and defects found with that of comparable projects in our research repository.
We use function point analysis (FPA) as a way to normalize software projects and to make it possible to compare performances of projects with different sizes.
We use functional documentation as a source for FPA.
we thoroughly review all sets of documentation on completeness and correctness and have FPAs performed by experienced,
FPAs are reviewed by different experts than the ones that performed the count itself to prevent from bias.
With regard to data quality we argue that all project data is reviewed by the applicable project managers.
All data is discussed with the applicable company management and the financial controller.
we build a survey questionnaire that is send to stakeholders once software projects are finalized.
We add the metrics resulting from this questionnaire to our research repository and relate the outcomes with software projects in the four quadrants of the Cost / Duration Matrix.
the extent to which a causal conclusion is based on our study.
Due to this we can objectively compare performances of all software projects,
The effect of outliers is limited and the risk on bias is mitigated responsibly based on the diversity of projects and business domains within each subject company,
the number of software projects,
and the fact that we measure and analyze software project portfolios as a whole in an empirical way.
the implementation of FSM-pricing in the software engineering domain of the company,
as an instrument useful in the context of software management and supplier proposal pricing.
We found that a statistical,
evidence-based pricing approach for software engineering,
as a single instrument (without a connection with expert judgment),
can be used in distributed environments to create cost transparency and performance management of software project portfolios.
CURRENT STATUS AND NEXT STEPS 8.1 Finalized research At this moment the following research results are in place:
We analyzed a dataset containing 352 finalized software projects,
with the goal to discover what factors affect software project performance,
and what actions can be taken to increase project performance when building a software project portfolio.
The software projects are classified in four quadrants of a cost/duration matrix: analysis is performed on factors that are strongly related to two of those quadrants,
Good Practices and Bad Practices.
resulting in an inventory of ‘what factors should be embraced when building a project portfolio
and ‘what factors should be avoided when doing so
We validate the tool by analyzing the performance of a subset of finalized software projects from the ISBSG repository .
related to the cost/duration matrix that we defined in earlier research (see Figure 1).
We enrich this model by mapping Stakeholder Satisfaction and Perceived Value with regard to a company’s customers,
internal process and innovation aspects to cost,
duration and quality of finalized software projects.
A paper on this subject,
including a survey on five finalized projects in a Telecom company is in preparation.
experienced development teams in a Banking company.
During the measurement period both teams transformed from a plan-driven delivery model (waterfall) to an agile approach (Scrum).
we observed that these small release-based projects differ largely from non-release-based projects.
a fixed and experienced development team,
and a steady heartbeat contribute to performances that can be characterized as best practice.
ACKNOWLEDGMENT I thank Arie van Deursen and Rini van Solingen for their great support and work as advisors for my PhD activities.
Furthermore I thank all companies that support our research for their generosity to allow us to use company data for research purposes.
with additional qualitative research on a series of best-in-class releases from another Telecom company is described in a paper that is to be submitted.
Rainer and N.
"Implementing Software Process Improvement: An Empirical Study," Software Process Improvement and Practice,
We used data collected in a Banking organization.
Based on a statistical correlation test we conclude that it appears too early to make generic claims on the relation between function points and story points
in fact FSM-theory seems to underpin that such a relationship is a spurious one.
The results of this research were published in a paper that was accepted at WETSoM 2014 .
"Critical Success Factors in Software Projects," IEEE Software,
"An Empirical Investigation of the Key Factors for Success in Software Process Improvement," IEEE Transactions on Software Engineering,
"Factors of Software Process Improvement Success in Small and Large Organizations: an Emperical Study in the Scandinavian Context," in ESEC/FSE,
indicating that the organization did not learn from history,
in combination with much time and energy spent on preparation and review of project proposals.
In order to create more transparency in the supplier proposal process a pilot was started on Functional Size Measurement pricing (FSM-pricing).
In our research we evaluated
"Critical Success Factors for Software Process Improvement Implementation: An Empirical Study," Software Process Improvement and Practice,
"Key success factors for implementing software process improvement: a maturitybased analysis," Journal of Systems and Software,
a small software organisation," in IEEE Proceedings of the Australian Software Engineering Conference,
"Success Factors of Organizational Change in Software Process Improvement," Software Process Improvement and Practice,
Hall and H.
"A survey study of critical success factors in agile software projects," The Journal of Systems and Software,
"Factors that motivate software engineering teams: A four country empirical study," Elsevier
"Identifying some important succ