Page - 10 - in The Future of Software Quality Assurance

10 S. Amann and E. Jürgens Fig. 6 Feature requestTS-15717: enable deleting account credentials eachoftheroughly6.5ktests(includingunittests, integrationtests,systemtests,and UItests) inourtestsuite.Toobtainthisdata,weaugmentedtheCIenvironmentwith profilers that record test-wise coverage in both the JavaScript code of the frontend andtheJavacodeof thebackendandran theentire test suiteonce,whichtookabout 45min. Based on this data, TIA now determines a list of six tests impacted by the changes: Five regression tests covering parts of the changed code and the new test that camewith the changes.TheentireCI runwith these impacted tests takesabout 1.5min, savingusover96%runtime. Thanks to TIA, only1.5min after committingher changes,our developer learns that the new test (ranked second by TIA) fails. With her changes still fresh in her mind,she investigates theproblemandfixes it inabout10min,committing thenew changesas TS-15717: Fix deletion of external credentials. Since the fix is very local, TIA selects only a single impacted test, namely, the test that previously failed. Therefore, the second CI run takes only about 45s in total.This time all tests pass. Overall, two consecutive CI runs using TIA plus correcting the mistake in between took less time than one executionof our full test suite. Figure 7 illustrates this improvement. 4.2 TestingwithTGA To ensure that all her changes are properly tested, our developer next looks at the test-gaptreemapforherchanges in thecontextofTS-15717.Figure8showshow TGA displays the test coverage recorded in the two previousCI runs. The treemap shows what we call the ticket coverage ofTS-15717: most of the code changes