Quality for DevOps teams

Foreword by Patrick Debois 3
Foreword by the authors 4
Acknowledgements by the product owner 6

Part 1 Introduction 15
1 What value will you find in this book? 16
1.1 The DevOps IT delivery model 17
1.2 Continuous quality engineering 18
1.3 README.TXT (reading guide) 21
1.4 TMAP evolution 23
2 Successful high-performance IT delivery depends on people 26
3 The VOICE model 34
4 Examples of indicators in the VOICE model 39
5 Introduction to quality and testing 42
6 CI/CD pipelines and tooling 53

Part 2 IT delivery models 61
7 Overview of IT delivery models 62
8 Sequential IT delivery models 66
8.1 Waterfall 66
8.2 V model 68
9 High-performance IT delivery models 70
9.1 Scrum 71
9.2 DevOps 73
9.3 Other high-performance IT delivery “models” 78
10 Hybrid IT delivery models 85
10.1 Demand/supply model 86
10.2 SAFe model 89
10 Quality for DevOps teams

Part 3 QA & testing topics 93
11 Introduction to QA & testing topics 94
12 Introduction to organizing QA & testing topics 98
13 Introduction to performing QA & testing topics 101
14 Topics plotted on the IT delivery models 103
14.1 Sequential IT delivery models 103
14.2 High-performance IT delivery models (especially DevOps) 104
14.3 Hybrid IT delivery models (especially SAFe) 107

Part 4 Organizing topics explained for DevOps 111
15 Quality & test policy 112
16 Responsibilities & roles 118
17 Monitoring & control 124
18 Anomaly management in DevOps 130
19 Reporting & alerting 135
20 Estimating the effort 145
21 Planning the delivery 155
22 Infrastructure 165
23 Tooling 171
24 Metrics 181
25 Continuous improvement 193

Part 5 Performing topics explained for DevOps 205
26 Quality risk analysis & test strategy 206
27 Acceptance criteria 216
28 Quality measures 219
29 Reviewing (static testing) 222
30 Test design 229
31 Test data management (TDM) 231
32 Test automation 238
33 Test execution 249
34 Investigate and assess the outcome of testing 252

Part 6 Quality measures and skills 257
35 Description of quality measures 258
35.1 Root cause analysis 258
35.2 Specification and Example (SaE) 259
35.3 Test-driven development 265
35.4 Pair programming 267
35.5 Pairing 268
35.6 Reviewing 269
35.7 Test design techniques 270
35.8 Feature toggles 271
35.9 Monitoring of product quality 273
35.10 Parallel testing 274
36 Personal, interpersonal and team skills 276

Part 7 Test varieties 285
37 Make sure there is variety in your testing 286
38 Performance testing 294
39 Usability testing 306
40 Security testing 314
41 Maintainability testing 321
42 Mutation testing tests the test 325

Part 8 Test design 331
43 The many aspects of creating tests and exploring a test object 332
43.1 Two distinct ways of preparing and performing tests 333
43.2 Experience-based testing 333
43.3 Coverage-based testing 334
43.4 Always combine experience- and coverage-based testing 337
44 Test design entities 338
45 Test approaches applied 343
45.1 Coverage-based testing 343
45.2 Process-oriented test design 345
45.3 Condition-oriented test design 345
45.4 Data-oriented test design 346
45.5 Appearance-oriented test design 347
45.6 Selecting and combining approaches and techniques 347
46 There are many techniques, which one to use? 350
46.1 No risk – no test – no development 352
46.2 Example case: “The magic boat ride in TestLand” 352
46.3 Process-oriented testing: path coverage 353
46.4 Condition-oriented test design 358
46.5 Data-oriented testing with EP and BVA 362
46.6 Data combination test 366
46.7 Syntactic testing 370
46.8 Code coverage in many different intensities 371
47 Experience-based testing 376
47.1 Experience-based testing approaches 376
47.2 Checklist 379
47.3 Error guessing 381
47.4 Exploratory testing 383
47.5 Crowd testing 390
47.6 What are heuristics and when to use them? 394
48 Is there any value in unstructured testing? 398

Appendix: Quality characteristics and non-functional testing 404

References 415
Index 423