In our previous 2 posts we covered the basics of the Agile process in TFS by reviewing the artifacts, how they can be used to manage quality, increase speed of delivery and how they map into PMBoK quality management. This blog will review some of our experiences assisting customers with their project quality management process implementation in TFS. It will also review our approach including an analysis of what the customer is currently doing to manage quality for their software products, planning and executing a proof of concept (POC), or testing in TFS before any processes or systems integrations are moved into production. This post will focus on moving items from Hewlett Packard’s Quality Center to TFS, plus the implementation of continuous integration in TFS.
Most companies who we have worked with have some processes for quality management in place, often documented and built leveraging other processes or the PMBoK. A walkthrough of the customer’s process and the supporting systems is often a good place to start. Once we have collected all of the information on the quality management processes, supporting systems and issues that the customer has experienced with either/both their processes and systems, we analyze the data and compare it against the out of the box Agile process template in TFS. The output of the analysis is often a spreadsheet mapping of data from the company’s quality management suite of choice to TFS and a Visio diagram of system functions and workflows, which is reviewed with the customer. Ideas on how processes and data can be moved to TFS are often presented at the same time as the mapping review and decisions are made to help plan for migration.
We have worked with Hewlett Packard’s Quality Centre (QC) to move items into TFS permanently and to decommission the use of QC. Some of the challenges driving migration to TFS for customers who use both QC and TFS are:
- QA and development teams are disconnected and don’t collaborate because they are using different tools for their work.
- No direct linkages exist between issues discovered by the testing team and the source code owned by developers. This causes traceability issues and can lead to the development team not being able to reproduce an issue discovered by the testing team.
- No aggregate reporting on work since development tasks are managed in TFS and QA is managed in QC.
The customer work we did to move QC items into TFS started with an overview of how QC was used by the testing team. During the review, a detailed mapping of data from QC to TFS was performed at the field level and documented in a spreadsheet. This mapping drove the modification of the Agile process template bug work item in TFS. Figures 1 and 2 below demonstrate the difference between the out of the box bug work item and the customized work item.
Figure 1: Out of the box Bug Work Item
Figure 2: Customized Bug Work Item
The workflow in QC was also documented and moved into TFS. This required some custom development to allow for Bugs to be automatically assigned to developers or other team members. In addition to the custom development work, we were able to make use of native TFS functionality to modify the workflow that manages a Work Item’s lifecycle (in this case a Bug).
The customized bug work item and workflow was reviewed with the QA team and modifications were made through 2-3 iterations. The review was conducted through the QA team’s use of the TFS customized bug work item – team members created new bugs and added data, plus moved the bugs through the workflow in order to understand how their process had been translated into TFS and provided feedback for changes.
In parallel with customization of the bug work item and the implementation of workflow and testing/iterating with the QA team, an investigation of tools and manual processes was conducted in preparation for the move of existing bugs from QC to TFS. Several options, including Scrat from Sela Group, were considered, but ultimately deemed unworkable. The high volume bugs in QC built up over the several years of its use, compounded with the loose organization of functional areas in QC and the number of customizations made to TFS led us to conclude that no commercially available tools were up to the migration task. The customer needed a proven method to move bugs from QC to TFS in a short period of time, because no bugs could be logged or modified during the migration from QC to TFS effectively, stopping QA work.
A reliable manual process was developed to move the bugs from QC to TFS – a group of project managers exported all QC Defects that were destined for TFS to several excel worksheets; they then ensured that the data in all mapped fields passed the TFS validation tests. This process was tested through several dry runs and provided the team with an indication of the bug creation/edit blackout period required for the migration.
After the QA team’s acceptance of the bug work item, the workflow implemented in TFS and the finalization of the manual process to move bugs from QC to TFS, the migration of items was executed. The team using QC was told to halt their work on defects for one day, during which over 1000 Defects were exported to TFS. The QA team began using TFS the following day. The modifications to the Bug template in TFS displayed earlier in figure 2, helped ease the transition to TFS by emulating the familiar QC environment.
Today our customer uses the customized TFS bug work item and benefits from the QA and development teams using the same tools in TFS. Linking bugs to source code allows the team to trace issues and report on testing and development activities in aggregate.
The implementation of continuous integration for another customer began through their use of JetBrains TeamCity. The customer’s objective was to add testing early in the product lifecycle; at the point of check-in by developers. By adding testing early in the project lifecycle, subsequent issues in the product development lifecycle were reduced. These issues were taking up an inordinate amount of time and resources to resolve. The customer demonstrated the build definitions in TFS that were to be used with TeamCity and we determined that a more seamless environment, which included Lab Management could meet the objectives of our customer for continuous integration.
We implemented a Proof of Concept with Lab Management, utilizing the creation of golden templates to deploy test machines and environments. The POC allowed the customer to run builds, deploy built code to test VMs and run tests, prior to checking the code into TFS. The golden template underwent several revisions, as we refined the required testing machines’ configurations. The POC build included scripts to set up the required databases from scratch in the test environments. For users of Visual Studio, the POC was a success. However, the database team was not using Visual Studio but rather a RedGate source control plug-in for SQL Server Management Studio. The gated check-in policy, requiring developers to build, deploy and test code with Lab Management before final check-in to TFS, did not work for the database team using RedGate. The logic for the builds, deployments and tests was moved into a new build definition that would be triggered upon check-in – so rather than front end load the tests prior to formal check-in, test are run immediately after check-in by either the development or database team using Visual Studio or RedGate respectively. This approach continues to meet the objective of the customer to test early in the development process, and identify issues that arise immediately after check-in.
Through our past 3 posts we have outlined the use of the Agile process for quality management, mapped PMBoK to Agile in TFS and provided a sample of our own customer experiences implementing quality processes and continuous integration in TFS. As a result of having all artifacts in one place and linked together in TFS, team members spend less time looking for information, freeing up time to do more work. Discovering issues early in the development lifecycle through continuous integration drastically reduces time to troubleshoot problems if they are found later in the project lifecycle. The use of the Agile process in TFS to facilitate the management of quality coupled with continuous integration has helped our customers to speed up development and ensure the high quality of software deliverables.