EssayBark -Excellent Writing Services

     3.3 Contracting Issues and Solutions for Agile

One of our government reviewers said, “This is the single biggest barrier to the government

operating like a commercial entity. We take 5 to 10 times longer to execute routine contract

actions.”

A particular management concern for Agile methods in the DoD, especially from the acquisition

side, but also from the execution side, is the selection and implementation of appropriate

contracting vehicles to support the types of practices that successful Agile projects exhibit. Due to

the iterative nature of Agile and its propensity to accept (even welcome) change, many

contracting vehicles present unique challenges for employing Agile methods. A particular issue is

the reporting and milestone requirements often levied against DoD contracts. These may be

especially difficult to meet using Agile methods. For example, one of the interviewed programs

was entering PDR. The biggest issue was to ensure the government review team understood what

was included in PDR and what was not. Requirements assigned to iterations in the future would

not have any design detail available, as opposed to a traditional PDR where everything would

have some type of design available at PDR. Section 4 discusses technical milestones in detail.

Another challenge is to construct any contract type so that it rewards working software and

meaningful milestone review compliance rather than just traditional artifact production. Several of

the interviewees stated that achieving rewards for working software that is balanced with

sufficient documentation was a challenge.

This section is not meant to be an all-inclusive and exhaustive discussion of the contracting issues

associated with employing Agile methods. Rather, it is only a brief introduction to the types of

contracts and some associated issues. Four of the most common contracting vehicles are discussed

in this section, along with comments about what we found in our interviews in relation to the use

of these contracting vehicles. One interviewee specifically stated that any contract type could be

used; some were just easier to work with than others.

3.3.1 Cost Plus Incentive Fee (CPIF)20

CPIF contracts are used for a wide variety of purposes in the DoD. In terms of Agile

development, a CPIF vehicle is a reasonable contracting vehicle to use when (as is often the case

with Agile projects) requirements will be quickly evolving and redirection of the tactical details of

the software is expected. The incentive fees can be constructed around measures and motivators

that support Agile methods, if the acquisition manager is cognizant of the kinds of management

and measurement aspects, as we have described above.

The management team of one of our interviewees credits its CPIF contract structure as a key

success factor. The program delivers multiple releases per year of ever-improving functional

requirements despite having zero software requirements specified in the enabling contracts.

 

20 For information on contracting vehicles discussed here, as well as other contract types typically used in the

DoD, please refer to http://www.dtc.dla.mil/dsbusiness/Info/contracts1.htm or

http://guidebook.dcma.mil/18/ContRecRevconttypes.htm.

CMU/SEI-2011-TN-002 | 34

This is possible because the contract is a services type contract for software engineering support.

This allows the development team to avoid situations where a contractor is obligated to satisfy a

requirement that has become irrelevant or obsolete. The team can respond to changing needs and

priorities without onerous contracting actions.

3.3.2 Fixed Price (FP)

There are a variety of fixed price contracts: firm fixed price, fixed price incentive, fixed price with

economic price adjustment, fixed price redeterminable, and fixed price level of effort.21 The

government prefers this type of contract because it encourages the contractor to contain costs.

Fixed price contracts are used with Agile projects in industry, but usually after the product

backlog and critical elements for each release are defined. In industry, the contracts are generally

short in time span in comparison to what is typical in the DoD.

3.3.3 Indefinite Delivery Indefinite Quantity(IDIQ))/Delivery Orders

IDIQ/Delivery orders, or task orders, comprise the largest percentage of contract type in terms of

the programs that we interviewed. Some of the benefits were that IDIQ vehicles provided the

most flexibility for adjusting to changing operational needs and provided more opportunities for

close working relationships between the developers and the end-user community they are serving.

Although none of our interviewed programs expressed it exactly this way, IDIQ types of vehicles

can be managed more like a service contract, where the service being provided is on-demand

software evolution, rather than like a product contract. Some of the author team sees this framing

of software development as a productive service for Agile projects in the DoD, one that is

supported by the prevalent use of the IDIQ contract type.

However, just because IDIQ provides flexibility, the government still needs to require an estimate

in cases where a release is mapped to (or the subject of) a task order. The releases closer to being

performed should have better estimates than those further out in the schedule. That is, the

immediate releases will have deliberate estimates and those further out will have planning

estimates. If the estimates change, the contractor needs to provide a reason. Otherwise, the

contractor may not look ahead further than a single increment.

One potential frailty of the IDIQ contracting model, particularly for a large overall program, is the

management of multiple delivery order contracts running in parallel. Even if the art of the

iteration has been mastered by the team(s) involved, there is a potential for the focus of program

management to be limited to each of the delivery order contracts in isolation, particularly if metric

reporting is limited to the scope of individual delivery orders. Make sure that oversight is across

the entire program rather than focused on individual tasks, and keep a continuing focus on quality

and integrity within product management rather than just looking at the project management

detail. While this may seem obvious to the seasoned program manager, remember that your team

is learning how to work with Agile and may need to be reminded of the basics.

 

21 http://guidebook.dcma.mil/18/ContRecRevconttypes.htm

CMU/SEI-2011-TN-002 | 35

3.3.4 Time and Material (T&M)

T&M contracts are used when it is not possible to estimate accurately the extent or duration of the

work or to anticipate costs with any reasonable degree of confidence.22 From a government

perspective, this type of contract requires significant oversight to assure that the contractor is

performing efficiently and using effective cost control measures. From a contractor perspective, it

fits rather well with the typical Agile planning and estimation process. T&M contracts may be the

most successful of the contracting types discussed here for use with Agile. In fact, many of the

interviewees, both contractor and government, commented that T&M contracts were the easiest to

use.

3.3.5 Contracting Vehicles Summary

Different contracting vehicles can work for Agile methods. Acquirers must be savvy about the

interpretation limits of different contracting regulations that affect the type of contract they are

contemplating. In cases where we saw awareness of differences in the cultural norms of Agile and

traditional acquisition teams, the contract vehicle chosen was not an impediment to getting the

desired results using Agile. In other words, the contracting personnel were aware of how Agile

methods worked: change is part of the norm, not an exception, they knew what would be

delivered when using those methods, and they adjusted their expectations accordingly within the

limits of the contracting rules and regulations.

One of the key management and contracting aspects in the DoD is the structure and requirements

for technical milestones. The next section addresses technical milestones explicitly, within the

context of DoD management and contracting.

 

22 http://guidebook.dcma.mil/18/ContRecRevconttypes.htm#Time and Material

CMU/SEI-2011-TN-002 | 36

CMU/SEI-2011-TN-002 | 37

4 Technical Milestone Reviews

The DoD 5000 series provides the framework for acquiring systems. This framework includes a

series of technical reviews including but not limited to System Requirements Review (SRR),

System Design Review (SDR), Software Specification Review (SSR), Preliminary Design

Review (PDR), and Critical Design Review (CDR) among others. Historically, documents such as

MIL-STD-1521, Technical Reviews and Audits for Systems, Equipment, and Computer Software,

or the United States Air Force Weapon Systems Software Management Guidebook are used as

guidance for conducting these reviews.23 Some smaller programs do not require this level of

review; however, programs that do require them expect a certain level of documentation and rigor

regardless of the development methodology employed.

These milestone reviews are system reviews. From a systems point of view, some would say these

reviews are not part of software development methodology and that this is an incorrect use of the

terms. For purposes of this report and in the view of the authors, software intensive systems

typically are subjected to PDRs, CDRs, and other reviews. While the overall system may be a

plane, tank, ship, or satellite, the software still must pass the PDR, CDR, and other milestones. If

the system in question is an IT system, then the PDR, CDR, and other reviews apply directly, as

software is the main component of the system (along with hardware to run it on). This section is

aimed at readers who are contemplating using Agile methods and who are subject to the typical

review activities prevalent in DoD acquisitions. The authors hope that this section will provide

some useful guidance on how to approach Agile methods when following traditional technical

milestones.

4.1 Milestone Review Issue

In the SEI report Considerations for Using Agile in DoD Acquisition, the authors stated: “A very

specific acquisition issue and sticking point is that Agile methodology does not accommodate

large capstone events such as Critical Design Review (CDR), which is usually a major, multi-day

event with many smaller technical meetings leading up to it. This approach requires a great deal

of documentation and many technical reviews by the contractor” [Lapham 2010]. The types of

documentation expected at these milestone events are considered high ritual and are not typically

produced when using Agile.24 If the PMO intends to embrace Agile methods then it will need to

determine how to meet the standard milestone criteria for the major milestones reviews,

particularly SRR, SDR, PDR, and CDR. However, Agile can be adapted within many different

types of systems. One of our reviewers said that “the Atlas V heavy launch guidance system is

produced using eXtreme Programming and has all the DoD procurement program events.” The

 

23 Note that this report does not address the major milestone decision points, i.e., Milestone A, B, or C. Only

technical milestones are addressed here.

24 “High ritual” is a term used within the Agile community often interchangeably with “high ceremony.” Alistair

Cockburn defined ceremony as the amount of precision and the tightness of tolerance in the methodology

[Cockburn 2007]. For instance, plan-driven methods such as waterfall are considered high ritual or ceremony as

they require an extensive amount of documentation and control. Agile methods are generally considered low

ritual as the amount of documentation and control should be “just enough” for the situation.

CMU/SEI-2011-TN-002 | 38

key here is knowing how long or how much effort is required to review the program in question

(an hour or a week).

In addition, according to one of our reviewers who has monitored Agile-based projects from a

government perspective:

[I] found that in projects using an Agile methodology, technical reviews are around delivery

of major capabilities and/or leadership is invited to certain sprint reviews. [I] also found

that technical reviews are not as needed as much because stakeholders buy-in to

stories/development/schedule at the end of every sprint, which we found is more productive

than large reviews every so often.

In order to determine the type of criteria needed for any review, the intent of the review or

purpose must be known. This raises a question: What is the purpose of any technical milestone

review?

4.2 Intent of Technical Milestone Reviews

The Defense Acquisition Guidebook (DAG) says,

Technical Assessment activities measure technical progress and assess both program plans

and requirements. Activities within Technical Assessment include …the conduct of Technical

Reviews (including Preliminary Design Review/Critical Design Review Reports)…A

structured technical review process should demonstrate and confirm completion of required

accomplishments and exit criteria as defined in program and system planning…. Technical

reviews are an important oversight tool that the program manager can use to review and

evaluate the state of the system and the program, redirecting activity if necessary. [Defense

Acquisition University 2011a]

From a software perspective, each of these reviews is used to evaluate progress on and/or review

specific aspects of the proposed technical software solution. Thus, expectations and criteria need

to be created that reflect the level and type of documentation that would be acceptable for those

milestones and yet work within an Agile environment.

4.3 Challenges

The intent of any technical milestone review is for evaluation of progress and/or technical

solution. For PMOs trained and experienced in the traditional acquisition methods, evaluating

program progress and technical solutions follows well established guidelines and regulations.

Very specific documentation is produced to provide the data required to meet the intent of the

technical review as called out in the program specific Contract Data Requirements List (CDRL).

The content of these documents and the entry and exit criteria for each review is well

documented. However, even in traditional acquisitions (using traditional methods), these

documents, exit and entry criteria can be and usually are tailored for the specific program. Since

the documentation output from Agile methods appears to be “light” in comparison to traditional

programs, the tailoring aspects take on additional aspects. Some of the specific challenges for

Agile adoption that we observed during our interviews that must be addressed are

• incentives to collaborate (see Section 4.4.1)
• shared understanding of definitions/key concepts (see Section 4.4.2)

CMU/SEI-2011-TN-002 | 39

• document content—the look and feel may be different but the intent is the same (see Section

4.4.3)

• regulatory language (see Section 4.4.4)

Many of these challenges are not specific to adopting Agile in DoD acquisitions but are also

common to other incremental development approaches such as Rational Unified Process (RUP).

In fact, RUP does address this level of milestone review. The reviews are called Lifecycle

Architecture (LCA) Review and Lifecycle Objectives (LCO) Review. Some potential solutions to

these challenges for Agile adoption are discussed in the following sections.

4.4 Agile Success Depends on Tackling Challenges

The PMO that is adopting or thinking of adopting Agile for an acquisition must set the stage to

allow for the necessary collaboration required between the PMO and other government

stakeholders and between the PMO and the contractor(s). The introduction of Agile into DoD

acquisition can be considered yet another type of acquisition reform. The Defense Science Board

has provided some cautionary words on any novel approaches for acquiring IT:

With so many prior acquisition reform efforts to leverage, any novel approach for acquiring

IT is unlikely to have meaningful impact unless it addresses the barriers that prevented prior

reform efforts from taking root. Perhaps the two most important barriers to address are

experienced proven leadership and incentives (or lack thereof) to alter the behavior of

individuals and organizations. According to the Defense Acquisition Performance

Assessment Panel, ‘… current governance structure does not promote program success—

actually, programs advance in spite of the oversight process rather than because of it.’ This

sentiment was echoed by a defense agency director in characterizing IT acquisition as

hampered by the oversight organizations with little “skin in the Game.” [Defense Science

Board 2009]

4.4.1 Incentives for Acquirers and Contractors to Collaborate

While this topic is relevant to all contracting issues, it is particularly important if the “world of

traditional milestone review” collides with the “world of agile development.” The authors have

observed programs where collaboration was not yet optimal, which resulted in major avoidable

issues when milestone review occurred. Thus, we emphasize them here.

We take the comment “skin in the game” to mean that the parties involved must collaborate.

Thus, some form of incentive to do so must be created. (As the Defense Science Board noted, “the

two most important barriers are leadership and incentives or lack thereof.”). It is key that

leadership convey clear goals, objectives, and vision for people to create internal group

collaboration and move toward success. One of our interviewees likened an Agile program to a

group of technical climbers going up a mountain: you are all roped together and if one of you

falls, you all fall. If the Agile project is thought of in these terms, it is quite easy to have an

incentive to collaborate. The key for DoD programs is to determine some incentive for both

contractor and government personnel that is as imperative as the life and death one is for technical

climbers.

Incentives need to be team based, not individual based. Agile incentives need to cross government

and contractor boundaries as much as is practical, given regulations that must be followed. Each

individual program has a specific environment and corresponding constraints. Even with the

CMU/SEI-2011-TN-002 | 40

typical constraints, the successful programs we interviewed had a program identity, which crossed

the contractor–government boundary. The incentives to collaborate were ingrained in the team

culture with rewards being simple things like lunches or group gatherings for peer recognition.

It is incumbent upon the PMO and the contractor to negotiate the appropriate incentives to

encourage their teams to collaborate. This negotiation needs to consider all the stakeholders that

must participate during the product’s lifecycle, including external stakeholders. The external

stakeholders (i.e., special review panels, other programs that interface with the program, senior

leadership in the Command and OSD, etc.) need to be trained so they understand the methods

being used, including, but not limited to, types of artifacts that are produced and how they relate

to typical program artifacts. Without specific training, serious roadblocks to program progress can

occur. Buy-in occurs with understanding and knowledge of Agile and the value it can bring.

Defense Acquisition University (DAU) is beginning to provide some training on Agile methods

and the associated acquisition processes.

Incentives should not be limited to just the contractor team but also encouraged for the

government team as well. Government teams are often incented to grow their programs and

follow the regulations. However, these actions should not be incented in an Agile environment

unless the users’ needs are met first.

The type of contract could also play into the incentive issue. One of the interviewees using a

T&M contract stated that they had a limited structure for rewards mainly due to the structure of

the contract, which provided limited motivation for the contracting company to provide rewards

as the contract personnel were relegated to the role of body shop. With this said, we observed a

very collaborative environment on this program.

In order to collaborate, all the parties must have a common understanding of terms and key

concepts. According to Alan Shalloway, Agile “has to be cooperative [in] nature and [people] are

not going to be convinced if they don’t want to [be]” [Shalloway 2009]. We have addressed a set

of Agile and DoD terms for which common understanding is required in Section 2.3.1. Some

additional key concepts specific to supporting technical reviews in an Agile setting are discussed

in the following sections.