Description

Despite being a philosophy first and foremost Agile still has practical and tactical elements to its delivery. The phases used in Agile have some specific objectives that can lend themselves to a process as does the challenge of balancing planning with releasing. The methodology element becomes most important as we scale up Agile into larger projects and larger organisations.

Examine the practical aspects of the core phases of Agile projects including tactics on how to achieve the right outcomes to the right standards. We will explore release planning and some of the actions and decisions needed for a smooth transition from build to usage, and this will be considered from a large scale perspective as well.

Consider the following questions as a guide to understanding this topic. Ask yourself how well you understand these principles and how satisfied you are that you can explain them.

* Why is alignment so important when defining the vision of a product?
* Why is feedback so important for us to adapt?
* What is the difference between iteration planning and wave planning?
* How does agile combat wish-based planning?
* What is the difference between coordination and collaboration in large scale agile?

As we explore the methods of Agile take these considerations into account when forming your own opinions.

* There are important differences between speculating and planning and ways in which we make use of this in agile projects.
* Agile projects have a start and finish that has many things in common with waterfall projects.
* The apparent paradox of planning over flexibility is what makes agile challenging
* Complexity and uncertainty are different challenges met by different solutions.
* Large scale empirical projects benefit from Agile just as much as small ones do.

Lecture 4A Agile Phases

Phases are : Envision, Speculate, Explore, Adapt, and Close. Speculate, Explore, and Adapt are cyclic and iterative in the project. Envision is a concept phase and interfaces with the larger organisation, links with return on investment. Product owner is a major sponsor, usually run by the project management. Aligned with the conceptual awareness with the objective. What is the purpose of the outcome of the project?

Three goals in the vision; (i) understand the concept of the value, (ii) to what standard is it to built, (iii) what constraints and limitations. After goals look at the architecture, divided into two components; support and interface (fixing and moving components). Both components are leveraged and can be adapted over time, with larger of effects in the fixed support architecture. Hence key people leaded in the envision stage. Quality includes grade of a product and how this is to be established; differs from description in Waterfall and plan-based projects, in Agile more about absence of defects.

A plan is a speculation based on predictions; hence the Speculation phase replaces the Planning phase in other projects. Agile gives respects to the results, not to the plan. Speculate phase creates the product backlog, the work not yet completed. When mapped out it creates a (speculative) release plan, knowing full well it will probably change - it is an iterative process. In each iteration the speculative release plan is revised and is story-driven with a story (customer) feature (builder) interaction.

Explore phase is where the immediacy is leveraged. Leadership is crucial in this phase to maximise productivity and innovation. All done with the customer story perspective. Three main activities; iteration planning (what we will do), workload (who will do it, self-organised teams will determine among themselves), and progress monitoring and review (daily stand-up meetings, what the Scrum master needs to do). Agile assumes the plan is wrong and the workers is doing the best to their ability. Technical practises are localised, project specific. Agile in not a substitution for excellence. Technical debt (failures, weaknesses) can threaten a project. Need to be dealt with in Explore phase.

Adapt phase is the feedback loop of what we've done and what can be changed. Adapting is change with a purpose, which allows the creation of the value. Test the difference between what was requested, what was delivered. Customer feedback, quality testing and technical review, method and performance review, vision status. Uses adaptive action, rather than traditional 'corrective action'. Based on the information we have now.

Close phase requires a final review for on-going improvement for future projects, looking at a larger strategic issue.

Lecture 4B Agile Planning

Release planning is the decision when particular features or stories are started and expected to be finished; it's a roadmap of the entire project to meet the vision. The release plan is the justification or reason that the original objectives are worth engaging in. Agile needs to answer business questions such as 'when will it be finished?', 'how much will it cost?', 'what is providing for us?'. Agile implementations cannot be entirely reactionary. Overcoming the contradiction between the business requirements and the flexibility of Agile is a balancing act. Just because strict plans don't work, it doesn't mean that no plan is the answer.

Main components of an Agile plan needs a reasonable prediction of the value versus the return of investment and a way to check this. It needs an assessment and mitigation strategy of risk. Guidelines to shape smaller decisions to add value. A means to prioritise activity. Provide understanding to the project team and to management. Avoid 'wish-based' planning, as desires cannot be implemented leading to distress and frustration, typically occurs when management does not listen to technical expertise. To avoid this use "adequate estimations" - a plan with what, when, and how much? The Release Plan is used to identify unrealistic demands and poor estimations.

Multilevel planning is used to balance in the conflict between flexibility and predictability. This is a top-down macro process which breaks down the iterative process into subsets ("waves"), to avoid the need for twenty-plus story-based iterations. "Waves" looks at completed stories from the iterative process, "Releases" look at completed product. Iteratives have a feedback loop, but they don't have completion, a "wave", with improved functionality, allows for partial releases. Iteration plans focus on the feedback, wave plans focuses on milestones of major components. A roadmap may include multiple releases. The term 'epic' is used with its relationship with 'stories'. The degree of interconnectiveness between stories which constitutes an epic. The just-in-time we only plan far ahead as we can see. The large scale epic plan gives general terms, including general product backlog. The self-organising teams choose and select what can be put in each iteration for the wave.

Lecture 4C Agile Scaling

There is a myth that Agile does not scale. How big does a project have to be until value doesn't matter? Or that a project doesn't need flexibility? Or that a plan is entirely predictable? Smaller projects are usually more successful than larger projects regardless of methodology. Complexity and scale differs. Complexity benefits from structure. The challenge is to use Agile principles within that more complex structure; they are used to solve different problems. Recognise the differences between agility and structure. Structure is a system for handling activity, agility is a method. To increase certainty, increase feedback as fast as possible to make use of it, therefore we need iteration.

The traditional mindset prefers process over people and compliance takes over from adaptability and assumes that better processes can replace weak people. Agile says that where there is uncertainty, make use of the people, relying on how teams communicate and work together. Using waves and epics appropriate teams can be allocated to ensure that the right people are working on the right tasks in small enough teams but allows for bridges between the groups to prevent silos.

The factors for a larger Agile structure; (i) organisational design (team creation and grouping) (ii) decision making (how do the right people contribute to the right decisions) (iii) collaboration design (how can emergence still thrive?) (iv) the agile principles (keep the principles in place). If one area is hard to break up, then use another (e.g., if teams cannot be broken, break up decision making). Distinguish between collaboration and coordination; the former is the shared creation of understanding (the input, the decision-making), the latter is the shared understanding (the output). Agile emphasises the former, but in larger teams does not require input from everybody all the time on all things.

Structure deals with complexity; Agile deals with uncertainty. With a larger group we need more leaders - often we get more managers instead. Good leaders are often a bit scary to traditional managers who are typically making decisions in significant resource allocations. Increasing leaders is not about vertical and linear decision-making structures; it's a horizontal and dynamic collaborative design. In a decision-making design we need a collaboration teams that sits across different project teams that communicates and shares knowledges between the teams.

Readings :Chapter 6: The Envision Phase

"The purpose of the Envisioning phase is to clearly identify what is to be done and how the work is to be accomplished... For small projects, much if not most of the work of the Envision and Speculate phases can be accomplished in a single 'kickoff' chartering session. For larger projects, chartering, requirements-gathering, additional training, resource procurement, and architectural work may take longer and can be included in Iteration 0 ... After the Envision phase it needs to be reviewed periodically for any changes and to ensure that the team continues to understand the vision." (p94-95)

"A product vision (defined by a product vision box and elevator test statement) galvinizes members of the product team into focussing their often disparate views of the product into a concise, visual, and short textual form. These two project artifacts, and the product roadmap, provide a 'high concept' of the product for marketers, developers, and managers" (p96)

"Innovation, creating emergent results that we cannot predict, requires an evolutionary process that can accommodate exploration and mistakes. A good product vision remains relatively constant, whereas the path to implementing the vision needs room to wander." (p96)

"The elevator test statement... takes the following format:
* For (target customer)
* Who (statement of the need or opportunity)
* The (product name) is a (product category)
* That (key benefit, compelling reason to buy)
* Unlike (primary competitive alternative)
* Our product (statement of primary differentation)" (Moore, 1991)

"The objective of product architecture is to depict the internal plumbing of the project - a design that facilitates and guides ongoing product development... In agile development, architecture is a guide, not a straightjacket" (p101)

"A feature breakdown structure (FBS) can be used to depict a product architecture in either software or hardware projects... Many architectural representations are useful for technical teams, but the FBS serves to communicate between customer and development teams and acts as a bridge between the Envision and Speculate phases" (p102)

"One of the reasons project leaders need technical domain experience as well as project management skills is that they need to understand the interactions between technical architecture, project organization, and planning. Poor technical structures can cause organizational problems, just as poor organizatioon structured can sometimes result in bad technical decisions. Poor organizational structures make release planning and roadmap development difficult... A better model [is] for the human organization and the technical architecture to co-evolve over the early phases of the project" (p103)

"A project data sheet (PDS) is a single-page summary of key business and quality objectives, product capabilities, and project management information. It is a simple document with a powerful impact whose condensed format appeals to the entire project community and constantly reminds them of the strategic aspects of the project" (p105)

"The tradeoff matrix helps the development team, the product team, and the executive stakeholders manage change during a project. The trade-off matrix informs all participants that changes have consequences and acts as a basis for decision making. The trade-off matrix indicates the relative importance of the three constraints (scope, schedule, cost) identified on the agile triangle (value, quality, constraints)." (p108)

"An exploration factor acts as a barometer of the uncertainty and risk of a project...The exploration factor is derived from a combination of the volatility of a product's requirements (ends) and the newness - and thus uncertainty - of a technology platform (means)" p109-110.

"A key project community vision can be summed up in four words: Get the Right People... 'Right' consists both having the appropriate technical ability (or domain expertise) and exhibiting the right self-disciplined behavior" (p112)

"Most people are coming to the understanding that process isn't a substitute for skill" (p113)

"Project participants comprise any individuals or groups that are part of the project community: from customers of the product, who can influence the project and determine requirements to executives who provide funding and assume some managerial oversight of the project, to the core team members who would to deliver the product.. There are three broad categories of participants: customers... development team members, and stakeholders.. From a broad brush perspective, customers provide requirements, team members build the product, and stakeholders contribute oversight nd constraints." (p115)

"One of the key reasons internal IT projects fail, or underperform, is misunderstanding the project leader and product manager roles. The product manager must come from outside the IT department" (p120)

"Project managers often perpetuate a hierarchical, command-control management philosophy in their project organizations by focussing on organizational charts, detailed processes and activity identification, and documentation requirements. Thinking about a self-organizing strategy attempts to break this mindset by having the team focus first on interactions... The strategy establishes the team's approach to communications, coordination, collaboration, decision making, and other individual-to-individual and team-to-team interactions" (p123)

Readings : Chapter 7: The Speculate Phase

"Planning is an important activity, but speculating is a more appropriate term as uncertainity increases. Speculating establishes a target and a direction, but at the same time, it indicates that we we expect much to change over the life of a project.. Plans are always conjectures about the future... The Speculate phase spotlights product and project - creating understanding the product structure, the backlog of capabilities and stories, and the release plan... The product of the Speculate phase is a release plan based on capabilities or stories to be delivered rather than activities (as in traditional project plans)" (p129-130)

"The first goal of feature-based planning and development is to make the process visible, and understandable to the customer team.." (p130)

"The objective of creating a product backlog is to expand the product vision, through an evolutionary requirements definition process, into a product feature list, or backlog.. Because software is more malleable than nearly any other product, an evolutionary specification process will serve it best.. Companies need to use this characteristic to their competitive advantage, and sticking to traditional waterfall development negates this advantage." (p133)

".. a feature or story is defined as a piece of a product that delivers some useful and valuable functionality to a customer. The basic difference in a story and a feature is that a story is a small piece that delivers useful functionality, but may not deliver a complete function... several stories [may] be needed to deliver [a] complete feature... capabilities, features, and stories are primarily used as a hierarchical structure to manage increasing product size... Traditional project plans focus on .. technical task areas. Stories, on the other hand, are customer oriented." (p134-135)

"Story cards ... are index cards on which the project and customer team members record the information gathered in their discussions... They provide a mobile, tactile medium that team members can write on, shuffle about on the table, and have a conversation around... The information on the cards becomes the product of the team's collaborative effort and a focal point for mutual understanding of the product at a detailed level." (p1367-138)

"A backlog is a list of capabilities, features, and stories that the product team has identified... The backlog data is used for both release and iteration planning." (p140)

"A release plan presents a roadmap of how the team intends to achieve the product vision within the project objectives and constrains identified in the project data sheet... A feature breakdown structure (FBS) may strike some project leaders as being difficult to administer, but it offers the significant advantage of increasing interaction between the team and customers... phase-based WBSs (requirements, design, built, test, etc.) may be easier to administer, but they don't match how engineers actually work." p143

"A product's scope should be driven by customer value, technical feasibility, cost, and critical business schedule needs. It should not be held hostage to a plan developed when the product and product knowledge was still in its infancy" (p146)

Iteration 0; a balance between too much planning and diving straight in. Doesn't deliver anything useful to the customer, therefore is kept short. (p147)

"For each iteration the team should develop a guiding theme. This is critical for focussing and ensuring the team balances between breadth and depth of features. Theme development often evolves from the process of assigning stories to iterations, by they can be predetermined. These themes .. help focus the teams in ways that a list of individual stories may not." (p151)

"Because there are significant benefits to early product deployment, a product team should determine a first feasible deployment (FFD) strategy.. Iterative development creates done-done [development done, product acceptance done] deployable pieces of a product. Those deployable pieces may or may not be actually deployed." (p152)

Readings : Chapter 8: Advanced Release Planning

"Management approves a project because they want a business problem solved. They want to know how the problem will be solved, how much it will cost, how long it will take, and how much risk they're incurring. When an agile team, or any other team for that matter, doesn't respond to this request, they are viewed as lacking commitment. The issue is one of flexibility versus stability... Neither forced schedules or no schedules is the right answer." (p158)

"Most companies do a lousy job of capacity planning, that is, balancing the demands for work to be done with the actual capacity of the organization. Too often the planning is wish-based, not capacity-based. Balancing capacity and demand is complex because it involves quantitative issues such as estimation, and qualitative issues such as escalating customer demands and staff motivation... In a wish-base culture, impeccable estimates will always be overridden by wishes, so the accuracy of the estimates makes little difference." (p159-160)

"Multi-level planning helps address a critical dilemma: the conflicting needs for predictability and flexibility... think of capacity level as reasonably fixed and story level as flexibility" (p153)

Four layers; Roadmap (longest), Release, Wave, Iteration (shortest). (p153)

"As a company's ability to release new version quickly increases, their horizon for detail planning can shrink, greatly increasing their ability to respond to customer needs." (p166)

Capability aka "epic", because it's made up of several stories (p166)

"Stories... can be measured in either relative (story points) or absolute terms (hours)... The total story points really represent the total cost of the project... Value points are a ways of showing we are serious about value... Cost and value should be equally emphasized. If the team doesn't have time to estimate value points then it doesn't have time to estimate costs." (p171-173)

"Many organizations do not want to go through the step of converting relative value points to monetary value points....One benefit is that monetary value points and monetary story points can be used together to provide value-earned versus cost progress reports.... monetary capability value points are calculated by allocating the net present value (NPV) of the revenue stream" (p176)

"There are two ways to improve productivity: Do better, or do less. Doing better, improving the flow of output per input, can clearly be accomplished by agile methods... The second way to improve productivity is harder to measure, but very significant: Identify features we either don’t do or do less of." (p181)

Until a final product emerges, the product development process is fundamentally one of gathering information and collaborating. Inherent schedule flaws occur when the size of the product to be developed is either grossly misestimated or the engineering team is given a date based on fantasy rather than reality. For a highly uncertain product, failing to meet a schedule plan may not be a flaw but merely the impossibility of scheduling the unknown." (p182-183)

"Requirements creep must be differentiated from requirements evolution. Requirements evolution is inevitable in exploration projects; in fact it is desirable, because the cost of change remains low in agile projects, and therefore requirements evolution remains cost effective. Requirements evolution is a rational process in which the development and customer teams are constantly evolving the requirements of the product while considering other constraints. Requirements evolution is a joint effort in which all parties participate in deciding on features. Requirements creep occurs when there isn’t a joint effort, when customers or developers add features indiscriminately. Requirements creep has acquired a negative connotation because of the high cost of change in many development efforts." (p183)

"Agile teams can place too much emphasis on adaptation or evolution and too little on anticipation (in planning, architecture, design, requirements definition). Failure to take advantage of knowable information leads to sloppy planning, reactive thinking, excessive rework, and delay. Remember: Agility is the art of balancing." (p186)

"Capacity" problems in organizations, as opposed to projects, arise from ignoring throughput issues...Agile teams combat this trend because of the agile emphasis on full-time team participation, but the problem persists. Capacity planning based on wishes rather than facts and this sub-optimal utilization of staff are both issues caused by a basic failure to prioritize...Many companies find it very difficult to focus on throughput because they must sell customers on delayed start times for projects." (p195)

The term Kanban came from the Japanese process for continuous improvement and the name has been applied to an approach to software development... Kanban systems begin to solve the high WIP problem and are increasing in popularity in the software arena... Kanban introduces a signaling system into the scheduling process in which a limit is set on WIP (depending on resources), and new work (stories or other work items) is introduced as in-process work completes. Work is “pulled” through the system (as an item is completed it pulls the next item off the backlog)." (p197)

Readings :Chapter 11: Scaling Agile Projects

"The best way to approach large, uncertain, complex projects is to reduce the project’s size, use small teams and increase staff only as absolutely necessary, hire only highly talented and experienced people, collocate the team, and use agile methods." (p271)

"Agile teams balance flexibility and structure. So as project size increases, structure—of necessity—increases also. But they don’t have to revert to an authoritarian, hierarchical structure. Large organizations can be adaptive, flexible, and exploratory—they just have to expand their structures in concert with agile principles, not abandon them." (p273)

".. the major components of a scaling model: business goals, agile values, organization, product (product backlog), and process. Organization, product backlog, and process are shown at three levels (the actual number of levels in practice depends on overall team size)—levels that are needed to manage larger projects. This model illustrates that agile development reflects a product lifecycle approach (continuous delivery of value), rather than a project approach (begin-end)... The product backlog structure in the figure consists of three levels—capability, feature, and story. The process/practice structure consists of product roadmap, release plan, and iteration plan. The product backlog results from practices such as product visioning, architectural planning, and defining product requirements. " (p276-277)

".. a large team may appear to be a daunting and time consuming task, but the design process should be based on a risk assessment and some standard guidelines. From the preceding discussion, the higher risk areas would be product components that have high degrees of coupling or integration (and even higher risk if teams are distributed), teams that need a higher level of relationship building (cultural or trust issues are apparent), and activities in which either critical and or many decisions must be made (release planning, for example)." (p283)

"The primary problem with documentation is the difference between context and content. Documentation can provide content, but understanding the context requires domain expertise.... Understanding comes from a combination of documentation and interaction, or conversation—the conversations among people who have domain knowledge. Furthermore, as the complexity of the knowledge to be transferred increases, the ability of documentation alone to convey that knowledge decreases and the need for interaction with knowledgeable people increases dramatically. Documentation provides content (partially), but conversations are necessary to provide the necessary context." (p287-288)

"An agile team consists of individuals—quasi-independent agents—who interact within a structure of self-organized and self-disciplined rules of engagement. Individuals have a degree of autonomy within this loose structure, and they, in turn, exercise self-discipline to be accountable for results and behave as responsible and thoughtful members of the team. Larger agile teams, those consisting of multiple feature and speciality teams, operate the same way—individuals are the agents in teams, whereas feature teams are the agents in a larger project. In building large agile teams, a network replaces the common hierarchical structure, decision making and collaboration must be carefully designed, and discipline reflects rules of engagement across teams." (p291)

Slides and Lecture

Envision Activities
Vision : What will it look like when finished?
Project Data Sheet : Project boundaries on a single page
Community : The right people, and only them
Approach : Practices, processes and guidelines

Iteration
- Speculate
Decisions we are not ready for.
Creation of stories
What will we do?
- Explore
Make the decisions we need to
Create the value
Do it.
- Adapt
Review, revise, respond
Constant change for perfection
What did we do?

Closing
* The vision defines the closure
* Closure is personal as well as functional

Capability
* Release plan delivers capability
* Capability is what we pay for
* Story is value to the user, capability is value to the organisation or business
Epics often relate to capability
Capability is managed by the project

Readings are from Agile Project Management 2nd edition (Jim Highsmith, Addison-Wesley, 2010)