Mapping the Project Management Body of Knowledge (PMBoK) quality management to the Agile process within TFS

In our last blog entry we listed and described how to use the various artifacts within TFS to facilitate continuous integration, help manage quality with Agile work items and speed up work.  As a result of having all artifacts in one place and linked together in TFS, team members spend less time looking for information, freeing up time to do more work.  Discovering issues early in the development lifecycle through continuous integration drastically reduces time to troubleshoot problems if they are found later in the project lifecycle.  In this blog entry we map PMBoK quality areas to the quality components and process areas for Agile TFS.  As Agile software development spreads to larger organizations employing methods that reflect or are based on the PMBoK, this mapping can assist with the expansion or modification of the out of the box Agile process within TFS.  The primary focus of the mapping will be on product quality (all software/deliverable related testing and fix activities).  A secondary focus will be on project quality mapping through the data and reports that are available in TFS.

Table 1 below maps the PMBoK quality processes descriptions to Agile quality processes in TFS

PMBoK MSF Agile in TFS
Plan Quality Management – The process of identifying quality requirements and/or standards for the project and its deliverables and documenting how the project will demonstrate compliance with quality requirements or standards. Planning software (deliverable) quality management with Agile in TFS maps into the PMBoK planning process – determining what to test begins with an analysis of requirements in user stories – deciding on what to focus on for testing e.g. areas that need the most coverage, how to test e.g. mix of automated and manual testing, when to test e.g. will the project use continuous integration and test as early in the SDLC as possible.  The Test Approach document that is part of the Agile process in TFS provides guidance on how the project can demonstrate that quality requirements are met by the software being developed.Agile in TFS provides no guidance on planning project quality management, however, the process template is flexible and can accommodate plans for the management of overall project quality.
Perform Quality Assurance – The process of auditing the quality requirements and the results from quality control measurements to ensure that appropriate quality standards and operational definitions are used. Auditing and review to ensure that what is stated in the quality management plan is actually being done is not explicit in the TFS Agile process but can nevertheless be practiced by reviewing quality metrics available through TFS reports and comparing them to the Test Approach document that is part of the Agile process in TFS.
Control Quality – The process of monitoring and recording results of executing the quality activities to assess performance and recommend necessary changes. Controlling quality in the TFS Agile process is conducted through the execution of test plans containing test suites and test cases, bug generation and repair.   Controlling quality in TFS Agile can also include looking at patterns in bug reports to improve the product e.g. code churn, code coverage, components that have a high number of bugs, looking for the patterns where number of bugs discovered and re-opened is falling in relation to them being closed (x pattern in bug report).

 

Plan Quality Management

PMBoK describes the inputs required, tools and techniques and the outputs for quality management plan development.  Tables 3, 4 and 5 below map PMBoK inputs, tools and techniques and the outputs for quality management to TFS Agile below.

Table 3. Plan Quality Management Inputs mapping

PMBoK MSF Agile in TFS
Project Management Plan

  • Scope
  • Schedule
  • Cost
  • Vision document
  • Iteration length (set with the iteration backlog workbook in TFS 2010 and through the web interface for TFS 2012 and TFS 2013).  The entire project schedule can be set by planning out all iterations envisioned.
  • Hours estimated in user stories and tasks multiplied by rate
Stakeholder register Stakeholder matrix document
Risk Register There is no native artifact in TFS Agile template – using a SharePoint team site list (which is included as part of the TFS Agile project) to manage a risk register for the project is possible.
Requirements User stories

 

Table 4. Plan Quality Management Tools and Techniques mapping

PMBoK MSF Agile in TFS
Tools and Techniques are meant to facilitate the creation of the Quality Management Plan There is nothing in the TFS Agile process providing guidance on the creation of a quality management plan.  How to create the plan is left to the team participating in the project allowing them to leverage the PMBoK for guidance.

 

Table 5. Plan Quality Management Outputs mapping

PMBoK MSF Agile in TFS
Quality management plan The Test Approach document contains items covered by the quality management plan in PMBoK.  The Test Approach document focuses on deliverable quality and contains:

  • Reference to user stories (requirements) to be tested
  • Scope including:
    • Functional tests
    • Integration tests
    • Security tests
    • Load and performance tests
    • Stress tests
  • Release criteria and the thresholds (which are part of the quality metrics)
    • Code coverage minimum
    • Kind of bugs and severity
    • Performance levels that need to be met
    • Level of test automation
  • Also defined in the test approach are:
    • Roles and responsibilities
    • How bugs will be tracked (in TFS)
    • Definitions of severity and priority levels for bugs
    • When triage meetings will be held
    • Test tools
    • Schedules and milestones (at a high level – but it would be advisable to leverage the iterations in TFS for schedule).

Other artifacts known as configurations, test suites and test cases in TFS and Lab Management are combined into test plans.  See Figure 1 below for an illustration of a test plan and its component artifacts.

Process improvement plan There is no template in TFS for a process improvement plan.
Quality metrics (Defect frequency, failure rate, availability, reliability and test coverage) The Test Approach document contains some guidance on quality metrics:

  • Release criteria and the thresholds
    • Code coverage minimum
    • Kind of bugs and severity
    • Performance levels that need to be met
    • Level of test automation
Quality checklists There is no template in TFS for quality checklists

 

Figure 1 – from the MSDN developer network shows the test plan in TFS and its configuration, test suites and test case components.

 

Planning guidance from Microsoft on the Agile process states “test points are created in your test plan based on your test cases and test configurations for each test suite. A test point is a pairing of a test case with a test configuration.”

See http://msdn.microsoft.com/en-us/library/dd286682.aspx for additional information and planning guidance.

 

Perform Quality Assurance

Quality assurance in PMBoK is an auditing or review process that is applied throughout the project life cycle.  Auditing and review to ensure that what is stated in the quality management plan is being performed is not explicit in the TFS Agile process but can nevertheless be practiced.  Project managers or team leads can elect to regularly review the Test Approach document on a weekly basis.  They can compare reports in TFS on code coverage, bug severity, performance levels and the level of test automation with the quality metrics in the test approach document to determine if they are meeting quality minimums.  Figures 2 and 3 below, illustrate examples of reports available in TFS, once testing and bug fixing begin in the project lifecycle.

Figure 2: Bug Trends

 

Figure 3: Burndown Chart and Burn Rate

 

Control Quality

Once the planning outputs are complete the team is ready to control quality.  In PMBoK control quality is defined as identifying the causes of poor process or product quality and taking action to eliminate them and validating that project deliverables meet requirements and will receive final acceptance by stakeholders.  Control quality in PMBoK maps to the execution of test plans and their component configurations and test cases that are grouped into test suites.  Control quality also encompasses the execution of test cases on their own when continuous integration is incorporated into the Agile project process.  Tests are run every time changes are made to the code, as part of continuous integration to ensure that nothing is broken and quality remains high.

PMBoK describes the inputs required, tools and techniques and the outputs for controlling quality.  Tables 6, 7 and 8 below map PMBoK inputs, tools and techniques and the outputs for quality control to TFS Agile.

Table 6 Quality Control Inputs

PMBoK MSF Agile in Team Foundation Server
Project management plan There are a few documents that can come together to help form an integrated project management plan:

  • Vision
  • Iteration length (schedule)
  • Cost
  • Project Structure
  • Test Approach
  • Test Plans (comprised of configurations, test suites and test cases)
Quality metrics The Test Approach document contains some guidance on quality metrics:

  • Release criteria and the thresholds
  • Code coverage minimum
  • Kind of bugs and severity
  • Performance levels that need to be met
  • Level of test automation
Quality checklists TFS Agile does not contain any quality checklist templates.
Work performance data TFS Agile reports such as code coverage, bug severity, performance levels and the level of test automation can be used to collect work performance data.
Approved change requests Changes are not handled through formal requests, which are either approved or rejected.  The product backlog for each iteration acts as a record and can be monitored for changes.
Deliverables The final deliverable after each iteration using Agile in TFS will be a usable application.
Project documents The project documents that can be used for the control of quality are the persona definition (describes actions of the users with the application being built) and the Test Approach.  These documents are used in combination with TFS artifacts called user stories, test plans and their component test suites and test cases.  The personas and user stories are the baseline against which the application is measured for completeness and quality.  The test plans and their component artifacts are used to test and their results are the quality measurements recorded in TFS.
Organizational process assets – issue and defect reporting procedures and communication The Agile process template makes use of the bug artifact to record and track defects.  Reports such as bug status, bug trends and reactivations can be used to communicate progress on bug discovery and fixes.  Issues can be recorded and communicated through a spreadsheet that is provided with the TFS Agile template.

 

Table 7 Quality Control Techniques

PMBoK MSF Agile in Team Foundation Server
Seven basic quality tools There is no equivalent in the TFS Agile process for the seven basic quality tools.  However, team members can elect to use the tools outside of TFS.
Statistical sampling There is no equivalent in the TFS Agile process for statistical sampling.  However, the team can select user stories and their corresponding test cases for additional inspection at their discretion.  If a user story and the corresponding test cases are part of the iteration the team is working on, it is assumed that they will go through the full inspection process described below.
Inspection Testing through the execution of test plans and their component test suites and test cases is the equivalent of inspection.  User acceptance testing of the application (deliverable) is the final step for inspection.
Approved change request review The product backlog for each iteration acts as a record and can be monitored for changes.  There is no formal approval of changes.  However, a suggested practice is to review the product backlog with stakeholders at least weekly and highlight any changes to the order of the backlog (priorities) or items that have moved from the current iteration to future iterations.

 

Table 8 Quality Control Outputs mapping

PMBoK MSF Agile in TFS
Quality control measurements Preparing for and conducting testing generates reports within TFS that can be used as measurements of quality control.  Reports include:

  • Test case readiness
  • Test plan progress
  • Bug status
  • Bug trends
  • Reactivations
Validated changes Changes are validated with tests – if application functionality is deferred from the current iteration to future iterations, the corresponding user stories and test cases are shifted out to the next iteration.  If new functionality is added, new user stories and corresponding test cases are created and added to the iteration backlog.  Passed tests serve as a record of validated changes.
Verified deliverables A usable application that has gone through successful testing and UAT.
Work performance information Reports in TFS provide useful work performance information from both a quality and general perspective.  All of the TFS reports employ the build, user story, task, test case and bug artifacts.   In addition to the reports listed for quality control measures above, work performance information can be found in reports for:

  • Build quality indicators
  • Build success over time
  • Build summary
  • Burn rate
  • Burndown
  • Stories progress
  • Remaining work
  • Unplanned work
  • Status on all iterations
  • Stories overview
Change requests Changes are not handled through formal requests.  The product backlog for each iteration acts as a record and can be monitored for changes.  Individual user stories can be examined for change by reviewing the history tab to trace any modifications made e.g. updates to the description or when a user story was moved to a future iteration/made obsolete.
Project management plan updates Project management plan updates
Project documents updates Project documents updates
Organizational process assets – Completed checklists and lessons learned documents Iteration Retrospective document – contains a section on lessons learned describing what went well and what could be improved in the next iteration plus concrete actions for improvement.

 

A team using TFS and the Agile process has a good basis for quality management of a software project.  The absence of guidance on project quality management (as opposed to deliverable/software quality management), quality assurance and formal change request management in the Agile process are the biggest differences with PMBoK in the area of quality management.  Guidance gaps in Agile can be covered by team members who are familiar with PMBoK through the creation of a broader project quality management plan, and leveraging artifacts and reports that are available in the TFS Agile process.

The artifacts in TFS including user stories, test plans, test suites, test cases and bugs provide significant benefits to quality management when compared to processes driven by Word documents.  TFS contains all artifacts which can be linked together making it easier for team members to find what they need and frees up more time for work and generally speeds up project progress.  Empowering the project team members to use these artifacts in TFS results in the creation of powerful records, facilitates quality auditing, and makes overall management of the project more transparent. In addition, all of these artifacts feed into reports that can be used by any team member to measure project progress and look for issues that need to be resolved.

Incoming search terms:

  • maping of agile artefacts to waterfall artefacts
  • content
This entry was posted in Lab Management, TFS 2013 and tagged , , , , , , , , , , . Bookmark the permalink.

Comments are closed.