A common question we get is: How do we know if a project is a good candidate for an agile project approach.  To answer this question, we need to look at both the project as well as the organization and environment in which we plan to complete the project.

The self-assessment survey based approach presented below is, perhaps, an overly academic approach to determining if a particular project is a good fit for an agile methodology, but we think that it provides at least one systematic and rational way to evaluate your specific situation and help you make a decision.  Even if you don’t want to actually build and distribute the survey to your project team and stakeholders – just thinking through these questions and how they apply to the project and environmental factors in your organization will help you make a more informed decision.

Agile approaches may not be for every project

Even with more than a decade of experience and several studies showing higher success rates and greater satisfaction with agile approaches over more traditional (waterfall) style methodologies, this does not mean that agile is the right choice for every project or every organization.

To determine if a project is a good candidate for an agile approach we need to answer these questions:

Q: When looking at a new project, how do we know if we should use an agile approach or some other project management approach?

  • Do the project’s characteristics allow for and benefit from an agile approach?

  • Does the environment support and facilitate the use of an agile approach, or can it be made to support and facilitate an agile approach?

Q: How do we put rigor around our selection of a project methodology so we don’t let our biases drive the selection process?

Agile Methodology

Agile project development mythologies developed out of the experiences of hundreds of software projects that did not deliver the promised value using traditional (waterfall) development methodologies.  Agile methodologies have been found to be particularly useful for software development projects and can provide valuable benefits including:

  • Focus on developing and deploying working software to the customer sooner driving a higher return on investment and faster delivery of value to the customer.

  • Better alignment of the software features and performance with the business goals of the organization.

  • Better visibility into the progress and value delivered by the software throughout the project.

  • Reduction in overall project risk.

  • Better working experience for the product management and development teams.

Project Evaluation Approach

To come up with a way to objectively determine if a proposed project is a good fit for an agile methodology, we turn to a rigorous self-assessment approach using a questionnaire that surveys the proposed project team and stakeholders to determine if the project’s characteristics and environment are a good fit for an agile approach.  This assessment methodology was originally proposed by  Barry Boehm and Richard Turner in Balancing Agility with Discipline.

We modify their approach to better suite the types of projects that we typically see. Our modifications are described below.

Background

The Boehm and Turner model looks at 5 aspects of a proposed project to determine if it may be good fit with agile methods.  These include:

Factor

Description

Scale

Criticality

The extent of loss due to defects in the product.

Loss of comfort to loss of many lives.

Personnel

The capabilities of the personnel on the project, assessed as fitting into three levels of capability.

Level 1, Level 2, Level 3 – Defined in more detail later in this document.

Dynamisim

The amount of change in requirements expected over the course of the project.

Expected percent change in requirements per month.

Culture

The organizations cultural tolerance of dis-order on a project.

Percent of the organization that thrives on order verses those that thrive on dynamic change

Size

Number of people on the project.

From 3 people to 300 people in size.

Each of the five aspects: Criticality, Personnel, Dynamism, Culture, and Size are evaluated on its own scale and plotted using a radar chart, as shown below.  The closer the five values are to the center of a set of concentric circles, the more likely that project is to work with agile methods.

 

Modifying the approach for our purposes

The Boehm & Turner model provides a good approach for assessing the suitability of both the characteristics of the project and the attributes of the environment needed for success with an agile approach.

The approach, as presented, takes a broader view than is needed to assess most business system projects.  To better suite our assessment needs, we tailor the Boehm & Turner approach as described below.

Our Approach

We build on the Boehm & Turner approach by assuming that the Criticality of the business system lies somewhere between discretionary funds and essential funds and remove that axis from our analysis.

We add a new concept: Integration Points – the number of other systems and / or projects the system must interact and coordinate with – to the project characteristics and break down the Culture into more specific variables plotted on the four axis’s:

  • Agile Leadership

  • Customer Involvement

  • Agile Infrastructure

  • Management support.

The modified radar chart below shows our changes with eight axis’s and grouping of what is consider a pure-project characteristic (those things that don’t change no matter what project delivery approach is used) and what is considered an environmental characteristic (those things that may vary based on the type of project delivery approach used).

Figure 1 Assessment Grouping

Conducting the Assessment

We assess the project along each axis using known values – such as the number of people dedicated to the project or the number of integration points expected in the software – and computed values – gathered as a result of the agile self assessment survey.

The following table describes the assessment approach used for each dimension:

Axis

Assessment Approach

Dynamism

Percent requirements change per month on the project. (From Bohem)

Size

Number of personnel on the project (From Bohem)

Agile Leadership

This is the experience and commitment level of the agile leadership assigned to the project.  Agile leadership includes the Scrum Master, Product Owner, and possibly an agile coach. This is a calculated value based on the results of the agile self-assessment.

Customer Involvement

This is the level of commitment, engagement, and interaction with the Product Owner, or customer who is responsible for specifying and the prioritizing the work that makes up the product.  This is a calculated value based on the results of the agile self-assessment.

Agile Infrastructure

This indicates the existing agile infrastructure that exists and techniques used by the team to support agile development. This includes continuous integration, coding standards, test driven development, automated unit testing, automated functional testing, agile estimating, planning, and tracking, etc.  This is a calculated value based on the results of the agile self-assessment.

Management Support

This indicates the level of management support provided for adapting agile methods in the project.  This includes the managers understanding of a self-organizing team, agile metrics, and the adaptive nature of agile planning.  This is a calculated value based on the results of the agile self-assessment.

Personnel

Measures  the staff’s capabilities using a subjective scale of level 1, level 2, or level 3.  Level 1 are very inexperienced or lack basic competence in software development, level 2 are competent developers, and level 3 are experienced and competent developers, tech leads, or architects. The Personnel axis does not specifically address agile experience for these team members. (From Bohem)

 

Conducting the self-assessment

The questionnaire is prepared for and distributed to each member of the the team.  We typically set up each targeted group of questions using an online survey tool like Survey Monkey (www.surveymonkey.com) and distribute the required links to team members based on their role.

We create a different version of the self-assessment questionnaire for each of the three groups that we consider for the project team: Managers, Customers, and Builders.  The assessment contains the questions are listed below for each group.

Group

Roles in Group

Managers

Project Manager, Scrum Master, Functional Management

Customers

Product Owner, Business Representative, Customer – requestor of the software product.

Builders

Developer, Tester, Automation Tester, Technical writer, Business Analysts, UX designers, and other technical team members.

 

The responses to the questions are evaluated and the results are plotted on the radar chart.  The questions groups are shown below along with the axis of the radar chart were the results are plotted.

Self-assessment questions

Note: These questions come from a variety of sources including the Sidky Agile Assessment, Boehm & Turner and the DSDM project filter.

Questions for Managers

ID

Question

Axis

AL_M1

You and your team have experience with agile projects.

Agile

AL_M2

You are comfortable estimating, planning, and reporting on an agile project.

Agile

AL_M3

Your team has as experienced scrum master.

Agile

AL_M4

Your team has an experienced agile coach.

Agile

AL_M5

The team sees value in agile methods and is willing to try various methods and focus on continuous improvement throughout the project.

Agile

AL_M6

An agile approach is a disciplined approach.

Agile

APN_M1

The plan for upcoming iteration may change based on customer feedback from the previous or current iteration.

Mgmt

APN_M2

You agree with developing the detailed plan for an iteration only after the conclusion of the previous iteration.

Mgmt

BLG_M1

You are willing to keep an up-to-date list of all the work that remains to be done for the project.

Mgmt

BLG_M2

When working on a project, you keep an up-to-date list of all the work that remains to be done.

Mgmt

CCF_M1

The customer should have the opportunity to give his/her feedback about the product throughout the development process by means of interacting with a working piece of software.

Mgmt

CCF_M2

The team has a method by which it gathers continuous feedback/criticism from the customer during the development process.

Mgmt

CCF_M3

Customers should be encouraged to regularly change their expectations for the product being developed to ensure that the product satisfies their business priorities.

Mgmt

CCF_M4

As the perception of what they need changes, customers are expected to articulate those changes and so affect the product being built.

Mgmt

CCF_M5

The customer should give his/her feedback throughout the development process even if it means that requirements must be changed.

Mgmt

CDL_M1

There is a clear and known software development process in place for this team; software development isn’t ad hoc or haphazard.

Mgmt

CDL_M2

The software-development process consists of a clear set of activities. Each of these activities has clear, standardized deliverables.

Mgmt

CDL_M4

It is a common practice for you to divide the system into mini-projects or phases. The system is seldom developed as one large project.

Mgmt

CDL_M5

The incremental-iterative approach has more benefits than the waterfall approach.

Mgmt

CDL_M6

You are willing to use the incremental-iterative approach to develop software.

Mgmt

CDL_M7

Delivering a working increment every 1–4 weeks will not cause you any additional stress.

Mgmt

CDL_M8

No big up-front requirements-gathering and analysis should be conducted when using the incremental-iterative approach. In other words, you don’t need to gather all the requirements before you start developing software in an incremental-iterative approach.

Mgmt

CDL_M9

You fully understand the principles of the incremental-iterative development approach.

Mgmt

CDI_M1

As the perception of what they need changes, customers are expected to articulate those changes by prioritizing the features they would like to see in the next iteration.

Mgmt

CDI_M2

Customers should be encouraged to regularly change their expectations for the product being developed, to ensure that the product satisfies their business priorities.

Mgmt

CDI_M3

The customer should be given the authority to determine which features need to be developed in the upcoming iteration.

Mgmt

DPT_M1

You are willing to meet daily for the progress update of a project.

Mgmt

DPT_M2

Indicate how often you meet with the rest of the team to discuss and update each other on the progress of the project.

Mgmt

EVR_M1

The team is familiar with the procedures to gather requirements from clients.

Mgmt

EVR_M2

In any project, requirements are always gathered from the customer in a structured manner and not haphazardly.

Mgmt

EVR_M3

Indicate how often you manage a project in which not all the requirements are known up front and an evolutionary requirements approach is used.

Mgmt

EVR_M4

You can start development of a project without knowing the exact requirements of the whole project.

Mgmt

EVR_M5

If circumstances dictate that not all the details are available before you start a project, you do not mind the uncertainty.

Mgmt

EVR_M6

You do not mind starting a project knowing that its requirements will change in the future.

Mgmt

EVR_M7

You can tell the difference between requirements that will influence the architecture and design of a project and requirements that will not influence it.

Mgmt

EVR_M8

In a project, you can recognize the high-level features that most probably will not change versus detailed requirements that might change.

Mgmt

EVR_M9

Throughout the project, the client has full right to change the requirements in order to meet his/her business needs.

Mgmt

EVR_A1

After observation or review of documents or other information, it is evident that the team has a process it uses to gather requirements from its clients.

Mgmt

EMT_M1

You usually seek your subordinates’ opinions before making a decision.

Mgmt

EMT_M2

You frequently seek the input of your subordinates on technical issues.

Mgmt

EMT_M3

If needed, you do not mind granting your subordinates unregulated access to the customer.

Mgmt

EMT_M4

You allow your subordinates to choose their own tasks for a project.

Mgmt

PPG_M3

Productivity is about how much customer value you can create per dollar spent, not about how many lines of code or classes coded per dollar spent.

Mgmt

PPG_M4

An atmosphere of assistance exists in the team.

Mgmt

PPG_M5

Common ownership of the code is important.

Mgmt

RTP_M1

You are willing to dedicate time after each iteration/release to review how the process could be improved.

Mgmt

RTP_M2

You are willing to undergo a process change even if it requires some extra work from the team.

Mgmt

RTP_M3

If there is a need for process change, that change should not be considered a burden on the team even if significant process changes have been made previously during the project.

Mgmt

RTP_M4

Process change in the middle of the project isn’t considered a disruption because the process change is worth the benefit it will bring.

Mgmt

SOT_M1

You agree that it is very important for the employees to work in teams where they can divide the team tasks among themselves.

Mgmt

SOT_M2

You trust that your employees can determine the best way to accomplish tasks by themselves without your (management’s) interference.

Mgmt

SOT_M3

You are willing to allow a self-organizing team to grow and not micromanage it.

Mgmt

SOT_M4

Employees are competent and disciplined enough to work in self-organizing teams.

Mgmt

SOT_M5

The team is an entity that has its knowledge, perspective, motivation, and expertise and should be treated as a partner with management and the customer.

Mgmt

TDD_M1

Test Driven Development will produce better software with fewer bugs.

Mgmt

TDD_M2

You are willing to tolerate the learning curve of the development team while they transition to Test Driven Development.

Mgmt

TDD_M3

The organization will be willing to provide software tools for creating and maintaining automated test suites.

Mgmt

TSV_M1

You allow your subordinates to choose their own tasks for a project.

Mgmt

TSV_M2

You believe that subordinates would perform better and be more effective if they were to choose their own tasks.

Mgmt

UT_M1

The developers are competent enough to write good unit tests for the methods and functions in the code.

Mgmt

UT_M2

It is important for developers to write unit tests for their methods and functions while they code, even if that will take additional time from them.

Mgmt

UT_M3

Writing unit tests for code is as important as writing new code for more functionality.

Mgmt

 

Questions for Customer / Product Owner

ID

Question

Axis

APN_C1

The plan for upcoming iteration may change based on feedback from the previous or current iteration.

Cust

APN_C2

You are comfortable developing the detailed plan for an iteration only after the conclusion of the previous iteration.

Cust

APN_C3

You are willing to set the priority of the features and re-evaluate the priority throughout the project.

Cust

CCF_C1

You are willing to give your feedback about the product throughout the development process by means of interacting with a working piece of software.

Cust

CCF_C2

You have a clear understanding of your business priorities and can  ensure the product being developed meets those needs.

Cust

CCF_C3

As the perception of what you need changes, you are willing and able to articulate those changes and so affect the product being built.

Cust

CCF_C4

You should give your feedback throughout the development process even if it means that requirements must be changed.

Cust

CDL_C1

Delivering a working increment every 1–4 weeks will not cause you any additional stress.

Cust

CDI_C1

As the perception of what you need changes, you will articulate those changes by prioritizing the features you would like to see in the next iteration.

Cust

CDI_C2

The customer should be given the authority to determine which features need to be developed in the upcoming iteration.

Cust

DPT_C2

Indicate how often you meet with the rest of the team to discuss and update each other on the progress of the project.

Cust

EVR_C1

If circumstances dictate that not all the details are available before you start a project, you do not mind the uncertainty.

Cust

EVR_C2

You do not mind starting a project knowing that its requirements will change in the future.

Cust

EVR_C3

Throughout the project, the client has full right to change the requirements in order to meet his/her business needs.

Cust

PPG_C1

An atmosphere of assistance exists in the team.

Cust

RTP_C1

You are willing to dedicate time after each iteration/release to review how the process could be improved.

Cust

RTP_C2

You are willing to undergo a process change even if it requires some extra work from the team.

Cust

 

Questions for Team Members (Dev / QA / BA, etc.)

ID

Question

Axis

AL_D1

You have experience working with agile projects.

Agile

AL_D2

You are comfortable estimating, planning, and reporting on an agile project.

Agile

AL_D3

Your team has as experienced scrum master.

Agile

AL_D4

Your team has an experienced agile coach.

Agile

AL_D5

The team sees value in agile methods and is willing to try various methods and focus on continuous improvement throughout the project.

Agile

AL_D6

An agile approach is a disciplined approach.

Agile

CCF_D1

Customers should be encouraged to regularly change their expectations for the product being developed to ensure that the product satisfies their business priorities.

Agile

CCF_D2

The team has a method by which it gathers continuous feedback/criticism from the customer during the development process.

Agile

CCF_D3

The customer should give his/her feedback throughout the development process even if it means that requirements must be changed.

Agile

CDL_D5

Delivering a working increment every 1–4 weeks will not cause you any additional stress.

Agile

CDL_D6

The incremental-iterative approach has more benefits than the waterfall approach.

Agile

CDL_D7

You are willing to do more integration (integrate after each iteration) in order to accommodate the incremental-iterative development approach.

Agile

CDL_D8

No big up-front requirements-gathering and analysis should be conducted when using the incremental-iterative approach. In other words, you don’t need to gather all the requirements before you start developing software in an incremental-iterative approach.

Agile

CDL_D9

You fully understand the principles of the incremental-iterative development approach.

Agile

DPT_D1

You are willing to meet daily to check in and synchronize efforts with your team members.

Agile

EVR_D2

You are willing start development of a project without knowing the exact requirements of the whole project.

Agile

EVR_D3

If circumstances dictate that not all the details are available before you start a project, you do not mind the uncertainty.

Agile

EVR_D4

You do not mind starting a project knowing that its requirements will evolve or change in the future.

Agile

EVR_D5

You can tell the difference between requirements that will influence the architecture and design of a project and requirements that will not influence it.

Agile

EVR_D6

In a project, you can recognize the high-level features that most probably will not

change versus detailed requirements that might change.

Agile

EVR_D7

Throughout the project, the client has full right to change the requirements in order to meet his/her business needs.

Agile

EVR_D8

In order to deliver valuable software to clients, change should be welcomed and not constrained.

Agile

EMT_D1

Your manager gives you the authority to make decisions without referring back to him/her.

Agile

EMT_D2

Your manager seeks your input on technical issues.

Agile

EMT_D3

You usually participate in the planning process of the project you are working on.

Agile

EMT_D4

When in a group, you feel that your participation is important.

Agile

EMT_D5

The team values you and your expertise.

Agile

EMT_D6

Your manager has high expectations of you.

Agile

EMT_D7

You are motivated by your job.

Agile

PPG_D4

Helping other team members with their work is a waste of my time.

Agile

PPG_D5

Whenever you need help, people are willing to help you.

Agile

RTP_D1

You are willing to dedicate time after each iteration/release to review how the process could be improved.

Agile

RTP_D2

You are willing to undergo a process change even if it requires some extra work from the team.

Agile

RTP_D3

If there is a need for process change, that change should not be considered a burden on the team even if significant process changes have been made previously during the project.

Agile

RTP_D4

Process change in the middle of the project isn’t considered a disruption because the process change is worth the benefit it will bring.

Agile

SOT_D1

You like to work on a team that management regards as one entity: addressing not individual team members in rewards or tasks but one team.

Agile

SOT_D2

You do not mind working without direct managerial supervision as long as you are on a team that is treated as a partner with management.

Agile

SOT_D3

You consider yourself competent and disciplined enough to work in a self-organizing team.

Agile

TSV_D1

You would do a better job choosing your own task on a project instead of being assigned one by your manager.

Agile

UT_D1

It is important to write unit tests for methods and functions while coding them even if that will take additional time.

Agile

UT_D2

Writing unit tests for code is as important as writing new code for more functionality.

Agile

UT_D3

You are willing to commit to writing unit tests while you code for every method or function in your code.

Agile

UT_D5

You consider yourself competent enough to write good and comprehensive unit tests for the methods and functions in your code.

Agile

CDL_D1

Software development in this team isn’t ad hoc or haphazard; there is a clear and known process in place.

Infrastructure

CDL_D2

Every project involves a clear set of activities. Each of these activities has clear, standardized deliverables.

Infrastructure

CNI_D1

The usual time it takes to create a build of the system is

Infrastructure

CNI_D2

Instead of integrating the system at the end of the development effort, it is better to regularly integrate the system throughout the whole development process.

Infrastructure

CNI_D3

You are willing to integrate your software throughout the development process, even if it means more work for you.

Infrastructure

CNI_D4

You are comfortable and competent using a Software Configuration Management (SCM) tool.

Infrastructure

CNI_D5

You are comfortable and competent using a continuous-integration tool.

Infrastructure

CGS_D1

There should be a coding standard for development.

Infrastructure

CGS_D2

If the team has a coding standard, then developers should use it when coding, even in crunch time.

Infrastructure

PPG_D1

Pair programming increases productivity, contrary to what others may say about pair programming (that it decreases productivity by half).

Infrastructure

PPG_D3

You are willing to program in pairs.

Infrastructure

TDD_D1

Indicate how often you write unit tests for every function/method/class.

Infrastructure

TDD_D2

You have no problems or challenges writing unit tests for functions/methods/classes.

Infrastructure

TDD_D3

The suite of unit tests that you write is comprehensive and usually encompasses all possible test scenarios.

Infrastructure

TDD_D4

You are willing to employ a test-driven approach to development.

Infrastructure

TDD_D5

You think Test Driven Development is easy.

Infrastructure

 

Calculating the Results

To make the calculations simple, we assume the following:

  1. We ignore common biases for this type of survey.

  2. We do not use any negative (inverted) questions.

  3. All questions / responses have equal weight.

  4. We treat ordinal data as interval data when computing average.

  5. We work with single values and don’t map these values to a weighted interval values.


We tabulate the survey results for each participant by the question and project characteristic (axis).  For each group of questions associated with a project characteristic (axis), we average the respondents score for each question and then average all respondent’s scores for the characteristic.

 

Customer Involvement Axis

Question
ID

Question
ID

Question
ID

Question
ID

Question
ID

APN_C1

APN_C2

APN_C3

CCF_C1

CCF_C2

Participant 1

3

4

5

2

1

Participant 2

3

4

3

2

3

Participant 3

2

4

4

3

2

Participant 4

4

4

1

4

5

Mean

3

4

3.25

2.75

2.75

Mode

3

4

n/a

2

n/a

Axis Mean

3.15

 

With the data tabulated, we then plot the computed value on our radar chart as shown below:

 

Interpreting the results

Once the survey results are tabulated and plotted on our radar chart, we have a visual representation of where our project characteristics are located and if there is a good fit  for an agile approach.  The closer the points are to the center of the radar chart, the more agile these elements are considered to be.

One possible use of this assessment is to trigger the use of an agile approach to a selected project, or as an indicator that an agile approach may not be a good fit for the project in the current environment.  To do this your organization needs to establish a threshold for the use of an agile methodology.  For example, that 80% of the points lie within the center circle of the graph and no point are in the 3rd circle of the graph to be considered a strong candidate for an agile approach.

Another use of this assessment is to simply provide a rigorous method of gathering information on the perceived project and environmental characteristics to allow you to make a more informed and rational decision on which project approach to used in a given situation.

Recommendations

We recommend that you use this information as a guide to your decision and not as a threshold metric.  It’s important to consider that this assessment is better at communicating the current state of things – some of the things you can change – than it does providing the end all be all of the decision making criteria.

We agree that not all projects and project environments are a good fit for agile methodologies, but for pure software development projects we and the software development industry in general have seen more success using agile methods than other ‘traditional’ approaches and we think it’s worth looking for ways to leverage agile principles on your project.

Look at any items in the in the Environmental Characteristics group – those things that you can change – and see if there are ways to enhance these within your organization so you can get the bring the benefits of an agile approach to your projects.

Focus your early project efforts on those things that you can change to make your environment better suited to utilize and gain the benefits of an agile approach.

For those things that you cannot change – the inherent project characteristics – you want to think carefully about how you approach the project.  If you have a project with characteristics that are less suited to an agile approach, consider how another project approach may or may not work better in your situation.