The Sprint

The sprint (or monthly iteration) is the most visible aspect of the SCRUM process, in particular to developers, since all development effort takes place within the sprint.

The sprint starts with a sprint planning meeting at the beginning of the month where all stakeholders, including customers, product managers, marketing personnel, developers, testers, technical writers and others are present.

At this meeting, the engineering team presents a list of tasks from the product backlog that they believe they are in a position to complete in the sprint being planned. All other stakeholders participate in the presentation by giving their inputs. At the end of this meeting, everyone has a clear understanding of the work to be done in the sprint, and the end-deliverables.

Likewise, the sprint ends with a similar meeting at the end of the month, where the engineering team gives a demonstration of the features that have been added to the product during the just-concluded sprint.

Once the sprint planning is done, the engineering team becomes largely self-organizing, and is left alone to figure out how best to achieve their sprint goals. However, there are two important mechanisms to track progress and provide visibility for management during the course of the sprint. These are the daily stand up meetings, and the burndown charts.

The daily stand-up meetings.
The engineering team holds a very short internal meeting every day, known as the daily scrum. This meeting is also known as the daily stand-up meeting because it is suggested that all participants be standing through the meeting, as a mechanism to keep it focussed and brief. This meeting is called for and conducted by a designated leader, known as the scrummaster. The designated scrummaster typically is the technical lead, project manager or any of the developers in the team.

During this meeting, all participants are required to explain what they have accomplished in the past working day, and what they intend to complete in the upcoming day. They are also expected to raise any blocks they are facing (also known as “impediments”) which are preventing them, or has prevented them from meeting their goals. It is then the team lead's (scrum master's) responsibility to follow up on and resolve the impediments. The scrum master ensures that this meeting is focussed only on status updates and identification of impediments, and does not digress into an actual discussion of the impediments.

Once this meeting is over, all engineers go back to their desks to begin or continue work on their stated goals, while the scrum master works towards resolution of impediments.

The burndown chart.
Immediately after the sprint planning meeting at the beginning of the month, all teams do a detailed analysis of the tasks to be completed in the sprint, and prepare a list of subtasks they need to perform for every task. These subtasks would include all development activities, testing activities, documentation updates, etc.

Along with the subtasks, all team members also provide an estimate, specifically in hours, of the amount of time they expect to take to complete each subtask. It is recommended that the subtasks be as granular as possible, such that each subtask is between 4-8 hours.

Every day, all team members updates the estimate for the subtask they are currently working on, and replaces it with a value that represents the amount of time now left on it. This is important: at all stages, the estimates, whether original or updated, must represent as accurately as possible, the amount of time left to work on it.

A snapshot of this data (the estimates) is collected every day, and plotted on a graph with date on the X axis. Ideally, this graph should show a line going downwards from left to right, something like the one shown below (copyrighted image from www.scrumalliance.org):



As can be seen, this graph provides a very clear picture of the state of the sprint to all stakeholders. If the graph is moving horizontally, or upwards, it could be an early indication that all is not well, and that the team might not meet their sprint goals.

In my experience, a very effective usage of the burndown chart is for an automated system to generate them once a day, and and immediately email them to all parties.

Estimation and Velocity

An iterative development model enables the engineering team to provide management with more accurate estimates for upcoming tasks, and consequently help management make more accurate plans and projections. This entry describes how.

When creating a project plan following traditional waterfall techniques, the engineering team or analysts typically do so by evaluating the scope of the tasks and the capabilities of the team to arrive at a number, usually in man-days or man-months, for each task. Next, the dependencies between tasks are considered and serialized accordingly to arrive at start and end dates for each task and the entire project.

This approach has two key disadvantages. The first disadvantage is that it makes the project plan inflexible and limits the management's options for speeding up the pace of work.

The second disadvantage is that the team's ability to perform at a certain rate can change due to unforeseen circumstances, such as an unexpected resignation or vacation by a key member, and this will affect the entire project plan. The productivity level of a team can change during the course of a project, specifically, an unexpected loss of a key team member can slow down the project, or perhaps an unforeseen technology requirement comes up which involves a learning curve and slows down the team, and so on.

When following the waterfall model, the options available to deal with these situations are expensive and/or detrimental to the quality of the end-product. For example, management may choose to drop or more affected features, which will require rework at almost all levels: planning, design, development, test plans, customer expectations, etc. Alternatively, one may choose to find a workaround to the unexpected technical obstacle, which might compromise the product quality, and result in increased post-release costs.

The first disadvantage: An inflexible project plan.
On a side note to start with, it is often said that “you cannot put together a group of 9 women to produce a baby in 1 month”. It is no coincidence that this analogy is usually stated by people following the waterfall model, where the project goes through a series of steps, and each step is completely dependent on the previous one, and where the final release can be made only at the end of all these steps.

A careful study of any waterfall project plan will reveal that dependencies are inevitable; a result of phase-wise planning. Clearly, you cannot start the design phase until the requirements phase is complete. You cannot start the development phase until the design is complete, and you cannot start testing until development is complete.

However, the woman-baby analogy does not hold good with the iterative model which requires that the project be split into independent tasks. Very rarely do we experience dependencies among individual features of a project. So by planning the project around features instead of phases, the organization has eliminated a good percentage of the dependencies right there, and makes the project more amenable to being executed in parallel.

If the iterative model has to respond to the woman-baby analogy, it would be “Yes, it is possible to have 9 women work for 9 months to produce one baby each, thus achieving an effective rate of 1 baby a month”.

The iterative model provides a very effective and flexible mechanism to increase productivity rates by increasing team strength.

The second disadvantage: Changing productivity levels.
An iterative process makes it very easy to detect whenever the productivity level of a team changes (due to loss of team member, or unexpected learning curve), and to rework the projections.

The key is, the team does not estimate the tasks directly in man-days or man-months. The team estimates the complexity of the tasks, instead of approximate time required. The exact scale against which complexity is estimated can vary from organization to organization. Some organizations may use a linear scale, like 1 to 10, while others may use exponentially increasing discrete values, like 1, 2, 5, 10, 20, 50, 100 etc. The important point to remember is that the scale and the estimates must be consistent all through the lifetime of the project. For the sake of discussion, we can call these values “complexity points”.

Once this estimation of complexity is complete, it becomes easy to track the number of hours required per point. This figure is called the velocity of the team.

As an example, if we consider a team of 5 people, we can assume 800 man-hours available per month (160 hours x 5 people). If this team is confident that they can consume tasks whose complexity points add up to 100, we have an intial velocity of 8 hours per point. As more and more iterations are completed, the velocity keeps getting revised for greater accuracy.

This approach makes it easy to monitor and optimize the performance of development teams. Management can introspect on questions such as:

• “Can we decrease the number of hours per point?” - For increasing performance.
• “Can we increase the points consumed per month by adding more people?” - For increasing production rate.
• “How many tasks can we complete for the next quarterly release?” - For making projections.
• "By when do we expect to make feature X available?" - For responding to market queries.
  

This approach also ensures that any change in the productivity of the team does not require the engineering team to revisit the estimates for all the tasks.

Benefits of Iterative Development

The iterative approach of SCRUM provides 3 key benefits to the organization.

1. Sets a Rhythm: It helps the organization to quickly establish an internal development rhythm, and all cross-functional teams begin to work by this rhythm.
   
   The development teams gets into the mode of planning and delivering a release every month with new features, the business analysts and product owner gets into the habit of ensuring that the highest priority backlog items are completely well-defined in time for the next sprint, and the customer learns to expect a new release every month, with added functionality that can be objectively seen, commented upon and accepted.
   
   Month after month, the rhythm is maintained, and soon management is able to identify key metrics for which data can be collected, monitored and fine-tuned.
   
   
2. Focuses on tangible deliverables: The waterfall model is also broken into multiple phases, but the deliverables at the end of each phase are intangible artifacts, such as requirement specifications, design documentation, project plans, test plans, untested code, etc.
   
   The only tangible delivery ever made in the project is when the product is released to the customer. no doubt, there is value in all the various artifacts generated at the end of each phase in the waterfall model, and indeed, many of these artifacts are also created and used in iterative processes.
   
   However, they are merely an aid to the development effort and nothing more. A customer has no use for a design document, nor will the company's share price go up because a project plan is complete. On the other hand, being able go to market with the first version of a product ahead of the competition, or being able to achieve faster turnaround times for bug fixes and new features will definitely go a long way towards keeping customers and shareholders happy.
   
   An iterative development model keeps everyone focussed on the software as the ultimate deliverable.
   
   
3. Emphasis on timebound delivery: The 3 main factors that impact the success of a product is features, quality and timeliness, and planning a project is typically an exercise in achieving the right balance between these 3 factors; one factor cannot be changed without having an impact on at least one of the others.
   
   The key difference between a waterfall-based project and an iterative project is that the former has a greater interest in keeping the feature set intact, while the latter is time-bound. That is, an iterative project places the emphasis on making a release at the specified time, even if it means shipping with one less feature.
   
   A company might, for example, have a policy to release a new version of their product every quarter. With such a policy, the organization can confidently make an announcement to their customers that the next release will be made on a particular date, with a certain set of features. All cross-functional teams such as development, QA, business analysis, technical writing, etc are in tune with this policy, and work towards maximizing the number of features and bug-fixes they can complete and integrate in time for the next quarterly release.
   
   Now, if there is a delay in one of the features, or something changed to make a feature redundant, the deadline is not affected. The organization can still make a release on the announced deadline, perhaps with a feature or two less. The delayed features are subsequently released in the next quarter.
   
   

Iterative Development

An iterative development model such as SCRUM is a very powerful mechanism for change control. In any project managed using typical waterfall models, every change proposal had to go through an evaluation process to estimate it's impact on the project deadline. Sometimes, even if a proposed change is beneficial to the product, it gets rejected because of the amount of rework to be done by the development team and/or its impact on the deadline.

An iterative process, on the other hand, deals with this problem very elegantly by maintaining a narrow time window within which specifications gets frozen, and development work is done only within this window. This window is an iteration, known in SCRUM as a Sprint, and is typically 1 month in duration. The other teams may modify any task on the backlog which has not yet been taken up by the development team.

At the start of an iteration, the development team looks at the Product Backlog (which has the highest priority tasks at the top) and identifies a certain number of tasks from the top which they believe they can complete within the one month sprint. A more detailed explanation of this estimation technique is given later.

Once the tasks to be undertaken in the current sprint (or iteration) is identified, they are moved into the sprint backlog and the team starts working on them. At this stage, none of the other teams and stakeholders may subsequently alter any of these tasks.

The work done on a task within the sprint is a mini-project in itself, and typically covers all stages of development, from design, development, testing, documentation and integration.

The Product Backlog

Managers who have worked with products that are well-established in the market would be familiar with the maintenance pattern, where all work becomes a series of new feature additions, change requests or bug fixes on that product.

They would also know that it is in fact much easier to manage a project which is in this stage, because all work is defined as a series of discrete tasks that can be individually assigned, tracked, developed and released. Progress and performance can now be measured numerically, based on the number of tasks completed and the complexity of each task, as in any other industry.

In contrast, a project for developing the first version of a product tends to be more chaotic, more so when developed using one of the traditional waterfall models.

• Multiple features are specified, defined and estimated by the project management team.
• The engineering team tries to design and develop all these features in one big step
• QA tries to test them all in another big step, and
• the customer expects to see only the end result which consists of all these features.

A delay in any of these steps affects the final deadline that has been committed to the customer.

Using SCRUM, one aims to bring the project to the “maintenance state” described above as early as possible, so that all subsequent work can be defined as a series of tasks in a pipeline.

This is done by identifying the bare minimum functionality required to demonstrate a working application. Having reached this state, all subsequent work is defined as a series of tasks that adds more functionality to the existing application.

Consider the example of a scientific calculator software. One can define the baseline product as a simple calculator, capable of performing basic arithmetic operations such as add, subtract, multiply and divide. Clearly, this baseline product is too immature to release to the general market, but it can definitely be released to various internal stakeholders.

For example:

• The QA team (and perhaps beta testers) can start testing for obvious bugs and edge cases.
• The business analysts can either verify that the product is in line with their expectations, or make changes to the original requirements they had defined.
• The user interface and usability experts now has a concrete application to provide recommendations against, and so on.

The significant benefit here is that all stakeholders can start providing inputs, and the company has the flexibility to make changes to the original requirements of the product based on these inputs.

It is important to note that all requirements are defined as a series of tasks. Each task can, at a minimum, be either a new feature to be added, or a change to an existing feature, or a bug fix. The job of the project manager now is to prioritize and manage this task pipeline efficiently.

In the calculator example above, subsequent feature requests could be for implementing square and square root functionality, sin and cos functions, etc, such that each of these functions become a separate task in the pipeline.

In SCRUM terminology, this pipeline is called the Product Backlog. Despite "Backlog" being a word with a slightly negative connotation, it is apt. The Backlog is the list of tasks that should be completed for the next deliverable version of the Product, and becomes the central axis around which all development efforts revolve.