One of our government reviewers said, “This is the single biggest barrier to the government operating like a commercial entity. We take 5 to 10 times longer to execute routine contract actions.” A particular management concern for Agile methods in the DoD, especially from the acquisition side, but also from the execution side, is the selection and implementation of appropriate contracting vehicles to support the types of practices that successful Agile projects exhibit. Due to the iterative nature of Agile and its propensity to accept (even welcome) change, many contracting vehicles present unique challenges for employing Agile methods. A particular issue is the reporting and milestone requirements often levied against DoD contracts. These may be especially difficult to meet using Agile methods. For example, one of the interviewed programs was entering PDR. The biggest issue was to ensure the government review team understood what was included in PDR and what was not. Requirements assigned to iterations in the future would not have any design detail available, as opposed to a traditional PDR where everything would have some type of design available at PDR. Section 4 discusses technical milestones in detail. Another challenge is to construct any contract type so that it rewards working software and meaningful milestone review compliance rather than just traditional artifact production. Several of the interviewees stated that achieving rewards for working software that is balanced with sufficient documentation was a challenge. This section is not meant to be an all-inclusive and exhaustive discussion of the contracting issues associated with employing Agile methods. Rather, it is only a brief introduction to the types of contracts and some associated issues. Four of the most common contracting vehicles are discussed in this section, along with comments about what we found in our interviews in relation to the use of these contracting vehicles. One interviewee specifically stated that any contract type could be used; some were just easier to work with than others. 3.3.1 Cost Plus Incentive Fee (CPIF)20 CPIF contracts are used for a wide variety of purposes in the DoD. In terms of Agile development, a CPIF vehicle is a reasonable contracting vehicle to use when (as is often the case with Agile projects) requirements will be quickly evolving and redirection of the tactical details of the software is expected. The incentive fees can be constructed around measures and motivators that support Agile methods, if the acquisition manager is cognizant of the kinds of management and measurement aspects, as we have described above. The management team of one of our interviewees credits its CPIF contract structure as a key success factor. The program delivers multiple releases per year of ever-improving functional requirements despite having zero software requirements specified in the enabling contracts. 20 For information on contracting vehicles discussed here, as well as other contract types typically used in the DoD, please refer to http://www.dtc.dla.mil/dsbusiness/Info/contracts1.htm or http://guidebook.dcma.mil/18/ContRecRevconttypes.htm. CMU/SEI-2011-TN-002 | 34 This is possible because the contract is a services type contract for software engineering support. This allows the development team to avoid situations where a contractor is obligated to satisfy a requirement that has become irrelevant or obsolete. The team can respond to changing needs and priorities without onerous contracting actions. 3.3.2 Fixed Price (FP) There are a variety of fixed price contracts: firm fixed price, fixed price incentive, fixed price with economic price adjustment, fixed price redeterminable, and fixed price level of effort.21 The government prefers this type of contract because it encourages the contractor to contain costs. Fixed price contracts are used with Agile projects in industry, but usually after the product backlog and critical elements for each release are defined. In industry, the contracts are generally short in time span in comparison to what is typical in the DoD. 3.3.3 Indefinite Delivery Indefinite Quantity(IDIQ))/Delivery Orders IDIQ/Delivery orders, or task orders, comprise the largest percentage of contract type in terms of the programs that we interviewed. Some of the benefits were that IDIQ vehicles provided the most flexibility for adjusting to changing operational needs and provided more opportunities for close working relationships between the developers and the end-user community they are serving. Although none of our interviewed programs expressed it exactly this way, IDIQ types of vehicles can be managed more like a service contract, where the service being provided is on-demand software evolution, rather than like a product contract. Some of the author team sees this framing of software development as a productive service for Agile projects in the DoD, one that is supported by the prevalent use of the IDIQ contract type. However, just because IDIQ provides flexibility, the government still needs to require an estimate in cases where a release is mapped to (or the subject of) a task order. The releases closer to being performed should have better estimates than those further out in the schedule. That is, the immediate releases will have deliberate estimates and those further out will have planning estimates. If the estimates change, the contractor needs to provide a reason. Otherwise, the contractor may not look ahead further than a single increment. One potential frailty of the IDIQ contracting model, particularly for a large overall program, is the management of multiple delivery order contracts running in parallel. Even if the art of the iteration has been mastered by the team(s) involved, there is a potential for the focus of program management to be limited to each of the delivery order contracts in isolation, particularly if metric reporting is limited to the scope of individual delivery orders. Make sure that oversight is across the entire program rather than focused on individual tasks, and keep a continuing focus on quality and integrity within product management rather than just looking at the project management detail. While this may seem obvious to the seasoned program manager, remember that your team is learning how to work with Agile and may need to be reminded of the basics. 21 http://guidebook.dcma.mil/18/ContRecRevconttypes.htm CMU/SEI-2011-TN-002 | 35 3.3.4 Time and Material (T&M) T&M contracts are used when it is not possible to estimate accurately the extent or duration of the work or to anticipate costs with any reasonable degree of confidence.22 From a government perspective, this type of contract requires significant oversight to assure that the contractor is performing efficiently and using effective cost control measures. From a contractor perspective, it fits rather well with the typical Agile planning and estimation process. T&M contracts may be the most successful of the contracting types discussed here for use with Agile. In fact, many of the interviewees, both contractor and government, commented that T&M contracts were the easiest to use. 3.3.5 Contracting Vehicles Summary Different contracting vehicles can work for Agile methods. Acquirers must be savvy about the interpretation limits of different contracting regulations that affect the type of contract they are contemplating. In cases where we saw awareness of differences in the cultural norms of Agile and traditional acquisition teams, the contract vehicle chosen was not an impediment to getting the desired results using Agile. In other words, the contracting personnel were aware of how Agile methods worked: change is part of the norm, not an exception, they knew what would be delivered when using those methods, and they adjusted their expectations accordingly within the limits of the contracting rules and regulations. One of the key management and contracting aspects in the DoD is the structure and requirements for technical milestones. The next section addresses technical milestones explicitly, within the context of DoD management and contracting. 22 http://guidebook.dcma.mil/18/ContRecRevconttypes.htm#Time and Material CMU/SEI-2011-TN-002 | 36 CMU/SEI-2011-TN-002 | 37 4 Technical Milestone Reviews The DoD 5000 series provides the framework for acquiring systems. This framework includes a series of technical reviews including but not limited to System Requirements Review (SRR), System Design Review (SDR), Software Specification Review (SSR), Preliminary Design Review (PDR), and Critical Design Review (CDR) among others. Historically, documents such as MIL-STD-1521, Technical Reviews and Audits for Systems, Equipment, and Computer Software, or the United States Air Force Weapon Systems Software Management Guidebook are used as guidance for conducting these reviews.23 Some smaller programs do not require this level of review; however, programs that do require them expect a certain level of documentation and rigor regardless of the development methodology employed. These milestone reviews are system reviews. From a systems point of view, some would say these reviews are not part of software development methodology and that this is an incorrect use of the terms. For purposes of this report and in the view of the authors, software intensive systems typically are subjected to PDRs, CDRs, and other reviews. While the overall system may be a plane, tank, ship, or satellite, the software still must pass the PDR, CDR, and other milestones. If the system in question is an IT system, then the PDR, CDR, and other reviews apply directly, as software is the main component of the system (along with hardware to run it on). This section is aimed at readers who are contemplating using Agile methods and who are subject to the typical review activities prevalent in DoD acquisitions. The authors hope that this section will provide some useful guidance on how to approach Agile methods when following traditional technical milestones. 4.1 Milestone Review Issue In the SEI report Considerations for Using Agile in DoD Acquisition, the authors stated: “A very specific acquisition issue and sticking point is that Agile methodology does not accommodate large capstone events such as Critical Design Review (CDR), which is usually a major, multi-day event with many smaller technical meetings leading up to it. This approach requires a great deal of documentation and many technical reviews by the contractor” [Lapham 2010]. The types of documentation expected at these milestone events are considered high ritual and are not typically produced when using Agile.24 If the PMO intends to embrace Agile methods then it will need to determine how to meet the standard milestone criteria for the major milestones reviews, particularly SRR, SDR, PDR, and CDR. However, Agile can be adapted within many different types of systems. One of our reviewers said that “the Atlas V heavy launch guidance system is produced using eXtreme Programming and has all the DoD procurement program events.” The 23 Note that this report does not address the major milestone decision points, i.e., Milestone A, B, or C. Only technical milestones are addressed here. 24 “High ritual” is a term used within the Agile community often interchangeably with “high ceremony.” Alistair Cockburn defined ceremony as the amount of precision and the tightness of tolerance in the methodology [Cockburn 2007]. For instance, plan-driven methods such as waterfall are considered high ritual or ceremony as they require an extensive amount of documentation and control. Agile methods are generally considered low ritual as the amount of documentation and control should be “just enough” for the situation. CMU/SEI-2011-TN-002 | 38 key here is knowing how long or how much effort is required to review the program in question (an hour or a week). In addition, according to one of our reviewers who has monitored Agile-based projects from a government perspective: [I] found that in projects using an Agile methodology, technical reviews are around delivery of major capabilities and/or leadership is invited to certain sprint reviews. [I] also found that technical reviews are not as needed as much because stakeholders buy-in to stories/development/schedule at the end of every sprint, which we found is more productive than large reviews every so often. In order to determine the type of criteria needed for any review, the intent of the review or purpose must be known. This raises a question: What is the purpose of any technical milestone review? 4.2 Intent of Technical Milestone Reviews The Defense Acquisition Guidebook (DAG) says, Technical Assessment activities measure technical progress and assess both program plans and requirements. Activities within Technical Assessment include …the conduct of Technical Reviews (including Preliminary Design Review/Critical Design Review Reports)…A structured technical review process should demonstrate and confirm completion of required accomplishments and exit criteria as defined in program and system planning…. Technical reviews are an important oversight tool that the program manager can use to review and evaluate the state of the system and the program, redirecting activity if necessary. [Defense Acquisition University 2011a] From a software perspective, each of these reviews is used to evaluate progress on and/or review specific aspects of the proposed technical software solution. Thus, expectations and criteria need to be created that reflect the level and type of documentation that would be acceptable for those milestones and yet work within an Agile environment. 4.3 Challenges The intent of any technical milestone review is for evaluation of progress and/or technical solution. For PMOs trained and experienced in the traditional acquisition methods, evaluating program progress and technical solutions follows well established guidelines and regulations. Very specific documentation is produced to provide the data required to meet the intent of the technical review as called out in the program specific Contract Data Requirements List (CDRL). The content of these documents and the entry and exit criteria for each review is well documented. However, even in traditional acquisitions (using traditional methods), these documents, exit and entry criteria can be and usually are tailored for the specific program. Since the documentation output from Agile methods appears to be “light” in comparison to traditional programs, the tailoring aspects take on additional aspects. Some of the specific challenges for Agile adoption that we observed during our interviews that must be addressed are CMU/SEI-2011-TN-002 | 39 4.4.3) Many of these challenges are not specific to adopting Agile in DoD acquisitions but are also common to other incremental development approaches such as Rational Unified Process (RUP). In fact, RUP does address this level of milestone review. The reviews are called Lifecycle Architecture (LCA) Review and Lifecycle Objectives (LCO) Review. Some potential solutions to these challenges for Agile adoption are discussed in the following sections. 4.4 Agile Success Depends on Tackling Challenges The PMO that is adopting or thinking of adopting Agile for an acquisition must set the stage to allow for the necessary collaboration required between the PMO and other government stakeholders and between the PMO and the contractor(s). The introduction of Agile into DoD acquisition can be considered yet another type of acquisition reform. The Defense Science Board has provided some cautionary words on any novel approaches for acquiring IT: With so many prior acquisition reform efforts to leverage, any novel approach for acquiring IT is unlikely to have meaningful impact unless it addresses the barriers that prevented prior reform efforts from taking root. Perhaps the two most important barriers to address are experienced proven leadership and incentives (or lack thereof) to alter the behavior of individuals and organizations. According to the Defense Acquisition Performance Assessment Panel, ‘… current governance structure does not promote program success— actually, programs advance in spite of the oversight process rather than because of it.’ This sentiment was echoed by a defense agency director in characterizing IT acquisition as hampered by the oversight organizations with little “skin in the Game.” [Defense Science Board 2009] 4.4.1 Incentives for Acquirers and Contractors to Collaborate While this topic is relevant to all contracting issues, it is particularly important if the “world of traditional milestone review” collides with the “world of agile development.” The authors have observed programs where collaboration was not yet optimal, which resulted in major avoidable issues when milestone review occurred. Thus, we emphasize them here. We take the comment “skin in the game” to mean that the parties involved must collaborate. Thus, some form of incentive to do so must be created. (As the Defense Science Board noted, “the two most important barriers are leadership and incentives or lack thereof.”). It is key that leadership convey clear goals, objectives, and vision for people to create internal group collaboration and move toward success. One of our interviewees likened an Agile program to a group of technical climbers going up a mountain: you are all roped together and if one of you falls, you all fall. If the Agile project is thought of in these terms, it is quite easy to have an incentive to collaborate. The key for DoD programs is to determine some incentive for both contractor and government personnel that is as imperative as the life and death one is for technical climbers. Incentives need to be team based, not individual based. Agile incentives need to cross government and contractor boundaries as much as is practical, given regulations that must be followed. Each individual program has a specific environment and corresponding constraints. Even with the CMU/SEI-2011-TN-002 | 40 typical constraints, the successful programs we interviewed had a program identity, which crossed the contractor–government boundary. The incentives to collaborate were ingrained in the team culture with rewards being simple things like lunches or group gatherings for peer recognition. It is incumbent upon the PMO and the contractor to negotiate the appropriate incentives to encourage their teams to collaborate. This negotiation needs to consider all the stakeholders that must participate during the product’s lifecycle, including external stakeholders. The external stakeholders (i.e., special review panels, other programs that interface with the program, senior leadership in the Command and OSD, etc.) need to be trained so they understand the methods being used, including, but not limited to, types of artifacts that are produced and how they relate to typical program artifacts. Without specific training, serious roadblocks to program progress can occur. Buy-in occurs with understanding and knowledge of Agile and the value it can bring. Defense Acquisition University (DAU) is beginning to provide some training on Agile methods and the associated acquisition processes. Incentives should not be limited to just the contractor team but also encouraged for the government team as well. Government teams are often incented to grow their programs and follow the regulations. However, these actions should not be incented in an Agile environment unless the users’ needs are met first. The type of contract could also play into the incentive issue. One of the interviewees using a T&M contract stated that they had a limited structure for rewards mainly due to the structure of the contract, which provided limited motivation for the contracting company to provide rewards as the contract personnel were relegated to the role of body shop. With this said, we observed a very collaborative environment on this program. In order to collaborate, all the parties must have a common understanding of terms and key concepts. According to Alan Shalloway, Agile “has to be cooperative [in] nature and [people] are not going to be convinced if they don’t want to [be]” [Shalloway 2009]. We have addressed a set of Agile and DoD terms for which common understanding is required in Section 2.3.1. Some additional key concepts specific to supporting technical reviews in an Agile setting are discussed in the following sections.
[supsystic-tables id=4]
I have been affected severally with late delivery of assignment but since I hired an EssayBark writer, my assignment reaches me before the expiry of the agreed deadline. Some tasks proved to be difficult since they required me to express my original ideas after carrying out an extensive research. This used to give me sleepless nights and occasionally used to turn down my friends invites to parties. However, EssayBark.com came at a time I needed them most and their services proved to be of high quality. If it were not for EssayBark.com, I don't think I would be where I am today. 90 percent of my success came from this company and I can say without any fear of contradiction that they are the best essay writing company in the whole world. Nothing seemed to work my way until I hired EssayBark.com. I can now confidently say that I enjoy my Environmental Science Course because the ideas provided by the writers have simplified all the technicalities of the subject. It seems that your writer understood the subject well. I believe she has a doctorate in the subject. Thanks to all of them for treating me personally. Since I started hiring EssayBark.com I learned that investing in your future is more important than any monetary investment. The knowledge provided in my political science essays by your writers helped me defend my thesis professionally in front of my teachers. I am highly obliged to your writers. I sometimes lack words to describe how happy I am since I met you guys. You are a gift to many as the essays provided by your writers are original, accurate, and timely. Meeting EssayBark.com, to me is fate. I had never dreamt of getting an A in my English literature course but you made my dream come true. Thank you so much. Since I started working with Essaybark.com life has never been the same again. I can comfortably say that my grades have greatly improved and I no longer have to worry about failing. My experience with Essaybark.com was one of a kind. They completely erased all my worries and as per now, I feel like am a master of all difficult topics which used to give me sleepless nights. Thanks a lot for opening my eyes. When I decided to join MBA, I wasn’t sure how I was going to handle those essays which required creative writing skills because I was literary very poor in it. However, my problem was solved by Essaybark.com. Apart from helping me with my assignments, they gave me free advice on how to polish my writing skills. My future is now bright all thanks to Essaybark.com. My worry with most companies offering writing services had been originality but since I started engaging Essaybark.com, all I get is quality and original work, delivered within the specified time frame. If you are looking for quality and non-plagiarized work, I recommend this site for you.
[wpadm-chat]