Extreme Programming (XP) is a discipline of software development based on values of simplicity, communication, feedback, and courage. It is intended to improve software quality and responsiveness to changing customer requirements. As a type of agile software development, it advocates frequent “releases” in short development cycles, which is intended to improve productivity and introduce checkpoints where new customer requirements can be adopted.
Extreme Programming was first introduced by Kent Beck in the mid-1990s when he was the project leader on a large, long-term project to rewrite Chrysler Corporation’s payroll system. Although extreme programming itself is relatively new, many of its practices have been around for some time; the methodology takes “best practices” to extreme levels. For example, the “practice of test-first development, planning and writing tests before each micro-increment” was used as early as NASA’s Project Mercury, in the early 1960s. Refactoring, modularity, bottom-up and incremental design were described by Leo Brodie in his book published in 1984.
In traditional system development methods the requirements for the system are determined at the beginning of the development project and often fixed from that point on. This means that the cost of changing the requirements at a later stage (a common feature of software engineering projects) will be high. Like other agile software development methods, XP attempts to reduce the cost of change by having multiple short development cycles, rather than one long one. In this doctrine changes are a natural, inescapable and desirable aspect of software development projects, and should be planned for instead of attempting to define a stable set of requirements.
Extreme Programming emphasizes teamwork and works by bringing the whole team together in the presence of simple practices, with enough feedback to enable the team to see where they are and to tune the practices to their unique situation. This team must include a business representative — the “Customer” — who provides the requirements, sets the priorities, and steers the project. It’s best if the Customer or one an aide is a real end user who knows the domain and what is needed. The team will of course have programmers. The team may include testers, who help the Customer define the customer acceptance tests. Analysts may serve as helpers to the Customer, helping to define the requirements. There is commonly a coach, who helps the team keep on track, and facilitates the process. There may be a manager, providing resources, handling external communication, coordinating activities.
Once project managers (and customers) accept their roles, customers begin writing user stories, which describe discrete features of the project. The XP methodology is to describe each story in a simple sentence or two, such as “The system should check the spelling of all words entered in the comments field,” written out on an index card. User stories should be independent of each other, short enough to be coded and delivered in a few weeks, and testable using automated test suites. On a regular basis, the project manager reviews the stories with the programmers, recasts if necessary longer stories into multiple short stories, and develops estimates as to the cost and time required to build each story’s functionality in the project. The customer then sets priorities based on business goals as well as technical dependencies, and the project manager hands out the highest-priority story cards to the project’s programmer teams. When the programmers are finished, the process repeats itself.
The user-story writing, review and coding cycle is XP’s strength. Because nobody is asked to commit to a complete specification of an entire project up front, and customers can change or add new stories at any time, XP is inherently flexible and adaptable to rapidly changing business needs. But that flexibility must be tempered with reality—if requirements are always changing, then the project never gets completed. The solution to that is the release plan, which specifies a series of dates, perhaps a few months apart, on which the team will stop and deploy what’s written to the customer. Even if a few user-story estimates prove to be too optimistic, the release dates should be inviolate: Nothing erodes confidence faster than missing deadlines.
XP planning addresses two key questions in software development: predicting what will be accomplished by the due date, and determining what to do next. The emphasis is on steering the project — which is quite straightforward — rather than on exact prediction of what will be needed and how long it will take — which is quite difficult. There are two key planning steps in XP, addressing these two questions, release planning and iteration planning.
Release Planning is a practice where the Customer presents the desired features to the programmers, and the programmers estimate their difficulty. With the cost estimates in hand, and with knowledge of the importance of the features, the Customer lays out a plan for the project. Initial release plans are necessarily imprecise: neither the priorities nor the estimates are truly solid, and until the team begins to work, we won’t know just how fast they will go. Even the first release plan is accurate enough for decision making, however, and XP teams revise the release plan regularly.
Iteration Planning is the practice whereby the team is given direction every couple of weeks. XP teams build software in two-week “iterations”, delivering running useful software at the end of each iteration. During Iteration Planning, the Customer presents the features desired for the next two weeks. The programmers break them down into tasks, and estimate their cost (at a finer level of detail than in Release Planning). Based on the amount of work accomplished in the previous iteration, the team signs up for what will be undertaken in the current iteration.
These planning steps are very simple, yet they provide very good information and excellent steering control in the hands of the Customer. Every couple of weeks, the amount of progress is entirely visible. There is no “ninety percent done” in XP: a feature story was completed, or it was not. This focus on visibility results in a nice little paradox: on the one hand, with so much visibility, the Customer is in a position to cancel the project if progress is not sufficient. On the other hand, progress is so visible, and the ability to decide what will be done next is so complete, that XP projects tend to deliver more of what is needed, with less pressure and stress.
Extreme Programming focuses on delivering business value in every iteration. To accomplish this over the course of the whole project, the software must be well-designed. The alternative would be to slow down and ultimately get stuck. So XP uses a process of continuous design improvement called Refactoring, from the title of Martin Fowler’s book, “Refactoring: Improving the Design of Existing Code“.
The refactoring process focuses on removal of duplication, and on increasing the “cohesion” of the code, while lowering the “coupling”. High cohesion and low coupling have been recognized as the hallmarks of well-designed code for at least thirty years. The result is that XP teams start with a good, simple design, and always have a good, simple design for the software. This lets them sustain their development speed, and in fact generally increase speed as the project goes forward.
Refactoring is, of course, strongly supported by comprehensive testing to be sure that as the design evolves, nothing is broken. Thus the customer tests and programmer tests are a critical enabling factor. The XP practices support each other: they are stronger together than separately.
In the arena of project management, XP gets all the attention but it is not the only agile method in play. According to Alistair Cockburn, originator of the Crystal method, “all (methods) are less explicit, less prescriptive than XP. They focus heavily on project management and communication practices (as does XP) but shy away from specifying actual engineering practices, such as testing code as it’s written, a key practice in XP.”
Jim Highsmith, originator of the Adaptive method, and a senior consultant and director of the agile project management advisory service at Cutter Consortium, in Arlington, Mass. States that producing software was easy when programs were small and people sat in a cubicle and wrote code. Today much more complex programs are being written, and no one person has the breadth of knowledge to do it themselves. The Adaptive method assigns a minimum of four people to a project. “You put people with different skills together and let them think through the issues, then one or two people code it,” Highsmith further states.
Per Kroll, manager of methods for IBM Rational states that there are more similarities than differences among the agile methodologies, including XP. He goes on to say that “they all propose iterative development. They all focus on working software, rather than on planning, design and requirements.” All are committed to a common set of values, expressed in the Manifesto for Agile Software Development (agilemanifesto.org), which says that “people and their interactions are the core of success, more than process and tools,” and that “the true measure of success is working software,” said Highsmith.
In 2003, Matt Stephens and Doug Rosenberg published Extreme Programming Refactored: The Case Against XP which questioned the value of the XP process and suggested ways in which it could be improved. This triggered a lengthy debate in articles, internet newsgroups, and web-site chat areas. The core argument of the book is that XP’s practices are interdependent but that few practical organizations are willing/able to adopt all the practices; therefore the entire process fails. Some of the criticisms of XP laid out by the book include:
- A methodology is only as effective as the people involved, Agile does not solve this
- Often used as a means to bleed money from customers through lack of defining a deliverable
- Lack of structure and necessary documentation
- Only works with senior-level developers
- Incorporates insufficient software design
- Requires meetings at frequent intervals at enormous expense to customers
- Requires too much cultural change to adopt
- Can lead to more difficult contractual negotiations
- Can be very inefficient—if the requirements for one area of code change through various iterations, the same programming may need to be done several times over. Whereas if a plan were there to be followed, a single area of code is expected to be written once.
- Impossible to develop realistic estimates of work effort needed to provide a quote, because at the beginning of the project no one knows the entire scope/requirements
The advocates of XP argue that the only truly important product of the system development process is code – software instructions a computer can interpret. Without code, there is no working product.
Coding can also be used to figure out the most suitable solution. For instance, XP would advocate that faced with several alternatives for a programming problem, one should simply code all solutions and determine with automated tests which solution is most suitable. Coding can also help to communicate thoughts about programming problems. A programmer dealing with a complex programming problem and finding it hard to explain the solution to fellow programmers might code it and use the code to demonstrate what he or she means. Code, say the proponents of this position, is always clear and concise and cannot be interpreted in more than one way. Other programmers can give feedback on this code by also coding their thoughts.
In conclusion, extreme programming is a discipline of software development based on values of simplicity, communication, feedback, and courage. It works by bringing the whole team together in the presence of simple practices, with enough feedback to enable the team to see where they are and to tune the practices to their unique situation. Extreme Programming Explained describes Extreme Programming as a software development discipline that organizes people to produce higher quality software more productively. Extreme Programming certainly lends itself better to some types of projects than others. An e-commerce Web site, end-user application or collection of components? Yes, incremental feature-by-feature evolution is a good way to go. The firmware for the Space Shuttle, an operating system or a standards-based compiler? Probably not. For big projects with firm specifications, traditional requirement-gathering, estimation and management techniques are better, and XP need not apply.
- Beck, Kent. (2001, March 23). “Interview with Kent Beck and Martin Fowler.” Retrieved November 15, 2010 from http://www.informit.com/articles/article.aspx?p=20972.
- deJong, Jennifer. (2005, February 1). Taking the Extreme out of Extreme Programming. Retrieved from http://www.sdtimes.com/link/28409
- Extreme Programming: A Gentle Introduction, Retrieved November 15, 2010 from http://www.extremeprogramming.org/
- Hoffer, Jeffrey A., George, Joey F., Valacich, Joseph S. (2007, March 12). Modern Systems Analysis and Design. Prentice Hall
- Kendall, Kenneth E. and Kendall, Julie E. (2008, February 3). Systems Analysis and Design. Prentice Hall
- Marchewka, Jack T. (2009). Information Technology Project Management. Wiley
- Stephens, Matt and Rosenberg, Doug. (2003, August 5). Extreme Programming Refactored: The Case Against XP. Apress.
- Zeichick, Alan. January 15, 2001. Extreme Planning. Retrieved from http://www.gantthead.com/content/books/16443.cfm on November 15, 2010