Page - 31 - in The Future of Software Quality Assurance

Testing in DevOps 31 definitiona customerhas.Business processescancovermultipleservices, andend- to-end testing will test the services of multiple teams. The responsibility of these end-to-endtestscancauseaproblembecause theycanbeownedbymultiple teams. One way to dealwith thisproblemis touse contract-based testing [11].DevOps lets you look at software as a service and it is common to create a contract for a service. Contract-based testing is verifying if the service upholds that contract. The tests should imitate a customer (or another service) and make sure there is no contract breach. If a company consists of multiple services and all teams have successfully implemented contract-based testing, you could even argue that there is no need for end-to-end testing anymore. The functionality should be covered in all contract and therefore in the tests. This, however, is only possible when the contractmatcheswiththecustomer’sneedsandnomistakesaremadeinthechainof contracts.Thereshouldbesomethingtocoveranydiscrepanciesbetweencontracts. End-to-end testing could be used to mitigate the risk in these discrepancies. In a microservices architecture [12] contract-based testing is a more applicable test strategy, because the architecture consists of multiple services. In a monolithic architectureitcanbemoredifficult toapplycontract-basetesting,becauseit ismore difficult to identifyservices that canbe individually tested. End-to-end testing that cover multiple teams could be made from small tests connectedtoeachother ina frameworkuntil theycover theentirebusinessprocess. Teamswillbe responsible fora smallpartof the tests andshouldofferhelp toother teams tomakesure the test chainwillwork.With thisapproachit is still possible to performend-to-endtestingwithDevOps teams. It requiresa lotofcoordinationand a testing frameworkthat supports the inputofmultiple teams. Incertainsituations,itcanandwillbemorefruitfultomakeoneteamresponsible for end-to-end testing when it covers multiple services. This team can deliver an end-to-endtestingserviceformultiple teams.DevOpsteamsshouldprovidecriteria for the tests for the parts that cover their service and should act on negative results from the tests. The end-to-end testing team should be responsible for making sure the tests will be executed, but the DevOps teams are still responsible for their own servicein these tests.Whateveryouchoose, it isbest toensurethata teamiscreated with theprincipleofcustomer-centricactionfirst. 3.2 Customer-CentricTesting The Agile Manifesto was the start of changing software development. It focusses highly on the process. In his book Specification by Example, Gojko Adzic writes: “building the product right and building the right product are two different things” [13].Hemakes thedistinctionbetween theprocessand theoutcomeof theprocess. While your process can be as you need it to be, there is no guarantee that the outcome of your process – your product/service– has the right quality. To achieve quality, a direct focus on the customer is needed, because they determine what the requirementsof your service should be. In this context it makes sense that the first