WEBVTT

1
00:00:17.735 --> 00:00:19.885
<00:00:17.735>Thank <00:00:17.935>you <00:00:18.134>so <00:00:18.335>much <00:00:18.535>Stefan, <00:00:18.735>for <00:00:18.935>this, <00:00:19.134>uh,

2
00:00:20.565 --> 00:00:21.565
<00:00:20.565>terrific <00:00:21.465>introduction

3
00:00:21.625 --> 00:00:23.125
<00:00:21.625>and <00:00:21.817>of <00:00:22.010>course, <00:00:22.203>uh, <00:00:22.396>many <00:00:22.589>thanks <00:00:22.782>to <00:00:22.975>all

4
00:00:23.125 --> 00:00:24.485
<00:00:23.125>of <00:00:23.431>you, <00:00:23.737>uh, <00:00:24.043>for <00:00:24.349>joining.

5
00:00:25.745 --> 00:00:27.005
<00:00:25.745>Uh, <00:00:26.123>I'm <00:00:26.501>Ingo <00:00:26.878>Ed,

6
00:00:27.025 --> 00:00:30.365
<00:00:27.025>and, <00:00:27.180>uh, <00:00:27.336>today's <00:00:27.491>session <00:00:27.647>without <00:00:27.802>a <00:00:27.958>big <00:00:28.113>surprise <00:00:28.269>is <00:00:28.424>all

7
00:00:30.365 --> 00:00:31.645
<00:00:30.365>about <00:00:30.941>software <00:00:31.517>testing.

8
00:00:32.265 --> 00:00:34.485
<00:00:32.265>And, <00:00:32.439>uh, <00:00:32.615>this <00:00:32.790>talk <00:00:32.965>comes <00:00:33.140>in, <00:00:33.315>uh, <00:00:33.490>two <00:00:33.665>parts.

9
00:00:34.705 --> 00:00:35.725
<00:00:34.705>In, <00:00:35.010>uh, <00:00:35.317>part <00:00:35.623>one.

10
00:00:36.485 --> 00:00:39.525
<00:00:36.485>I <00:00:36.640>wanna <00:00:36.796>share, <00:00:36.951>you <00:00:37.107>know, <00:00:37.262>my <00:00:37.418>notion <00:00:37.573>of <00:00:37.729>software <00:00:37.885>testing.

11
00:00:39.685 --> 00:00:42.605
<00:00:39.685>I <00:00:39.825>wanna <00:00:39.965>let <00:00:40.105>you <00:00:40.245>know <00:00:40.385>what <00:00:40.525>software <00:00:40.665>testing <00:00:40.805>means <00:00:40.945>to <00:00:41.085>me

12
00:00:42.715 --> 00:00:43.965
<00:00:42.715>that <00:00:42.996>will <00:00:43.277>be <00:00:43.558>part <00:00:43.840>one.

13
00:00:44.585 --> 00:00:48.005
<00:00:44.585>And <00:00:44.760>in <00:00:44.935>part <00:00:45.110>two <00:00:45.285>I <00:00:45.460>wanna <00:00:45.635>share <00:00:45.810>some <00:00:45.985>characteristics

14
00:00:48.435 --> 00:00:49.925
<00:00:48.435>that <00:00:48.882>make <00:00:49.329>software <00:00:49.775>testing

15
00:00:50.565 --> 00:00:52.245
<00:00:50.565>excellent <00:00:50.915>from <00:00:51.265>my <00:00:51.614>naive <00:00:51.964>perspective.

16
00:00:52.825 --> 00:00:56.365
<00:00:52.825>So <00:00:53.000>that's <00:00:53.175>the <00:00:53.350>simple <00:00:53.525>logic <00:00:53.700>behind, <00:00:53.875>uh, <00:00:54.050>this <00:00:54.225>talk.

17
00:00:56.445 --> 00:00:59.245
<00:00:56.445>That's <00:00:56.600>the <00:00:56.756>master <00:00:56.911>plan. <00:00:57.067>I <00:00:57.222>hope <00:00:57.378>that <00:00:57.533>works <00:00:57.689>for <00:00:57.845>you.

18
00:00:59.665 --> 00:01:00.805
<00:00:59.665>So <00:00:59.921>let's <00:01:00.178>dive <00:01:00.434>right <00:01:00.691>in.

19
00:01:01.225 --> 00:01:04.845
<00:01:01.225>You <00:01:01.365>know, <00:01:01.505>the <00:01:01.645>more <00:01:01.785>and <00:01:01.925>more <00:01:02.065>I, <00:01:02.205>I'm <00:01:02.345>studying <00:01:02.485>the <00:01:02.625>field

20
00:01:04.845 --> 00:01:06.765
<00:01:04.845>of <00:01:05.078>software <00:01:05.311>testing, <00:01:05.545>the <00:01:05.778>more <00:01:06.011>I <00:01:06.245>learn

21
00:01:07.195 --> 00:01:09.645
<00:01:07.195>that <00:01:07.369>software <00:01:07.544>testing <00:01:07.720>is <00:01:07.895>like <00:01:08.070>a <00:01:08.244>the <00:01:08.419>other <00:01:08.595>end.

22
00:01:09.945 --> 00:01:11.765
<00:01:09.945>And <00:01:10.145>the <00:01:10.345>reason <00:01:10.544>for <00:01:10.744>this <00:01:10.945>is <00:01:11.145>fairly <00:01:11.345>simple

23
00:01:12.115 --> 00:01:15.405
<00:01:12.115>because <00:01:12.270>the <00:01:12.426>people <00:01:12.581>who <00:01:12.737>really <00:01:12.892>need <00:01:13.048>it <00:01:13.203>never <00:01:13.359>use <00:01:13.515>it.

24
00:01:15.985 --> 00:01:18.125
<00:01:15.985>And <00:01:16.160>that's <00:01:16.335>exactly <00:01:16.510>the <00:01:16.685>reason <00:01:16.860>why <00:01:17.035>I'm <00:01:17.210>here <00:01:17.385>today

25
00:01:18.125 --> 00:01:19.365
<00:01:18.125>because <00:01:18.348>I <00:01:18.571>want <00:01:18.794>to <00:01:19.017>change <00:01:19.241>that.

26
00:01:20.115 --> 00:01:23.295
<00:01:20.115>And <00:01:20.289>to <00:01:20.464>do <00:01:20.640>that, <00:01:20.815>I <00:01:20.990>wanna <00:01:21.164>say <00:01:21.339>some <00:01:21.515>words

27
00:01:23.305 --> 00:01:24.495
<00:01:23.305>about <00:01:23.662>my <00:01:24.019>person <00:01:24.376>first.

28
00:01:25.195 --> 00:01:28.335
<00:01:25.195>You <00:01:25.335>know, <00:01:25.475>in <00:01:25.615>the, <00:01:25.755>in <00:01:25.895>the <00:01:26.035>last <00:01:26.175>10 <00:01:26.315>years <00:01:26.455>I've <00:01:26.595>worked

29
00:01:28.395 --> 00:01:29.455
<00:01:28.395>as <00:01:28.713>a <00:01:29.030>product <00:01:29.349>manager

30
00:01:29.635 --> 00:01:34.335
<00:01:29.635>for <00:01:29.790>several <00:01:29.946>tool <00:01:30.101>vendors <00:01:30.257>in <00:01:30.412>the <00:01:30.568>automation <00:01:30.723>space <00:01:30.879>in <00:01:31.035>RPA,

31
00:01:34.335 --> 00:01:36.055
<00:01:34.335>that <00:01:34.685>means <00:01:35.035>robotic <00:01:35.385>process <00:01:35.735>automation

32
00:01:36.195 --> 00:01:39.295
<00:01:36.195>and <00:01:36.369>in <00:01:36.544>automation <00:01:36.720>for <00:01:36.895>the <00:01:37.070>purpose <00:01:37.244>of <00:01:37.419>software <00:01:37.595>testing.

33
00:01:39.915 --> 00:01:42.495
<00:01:39.914>So <00:01:40.195>I <00:01:40.475>lift <00:01:40.755>and <00:01:41.035>briefed <00:01:41.315>automation.

34
00:01:42.665 --> 00:01:44.295
<00:01:42.664>Automation <00:01:43.014>was <00:01:43.365>my <00:01:43.714>daily <00:01:44.065>bread.

35
00:01:44.315 --> 00:01:47.455
<00:01:44.315>It <00:01:44.490>was <00:01:44.664>at <00:01:44.840>the <00:01:45.015>epicenter <00:01:45.190>of <00:01:45.365>my <00:01:45.539>professional <00:01:45.715>life.

36
00:01:47.475 --> 00:01:51.375
<00:01:47.475>So <00:01:47.615>to <00:01:47.755>say <00:01:47.895>it, <00:01:48.035>it <00:01:48.175>truly <00:01:48.315>dominated <00:01:48.455>my, <00:01:48.595>my <00:01:48.735>daily <00:01:48.875>thinking.

37
00:01:51.995 --> 00:01:56.015
<00:01:51.995>And <00:01:52.150>all <00:01:52.306>this <00:01:52.461>told <00:01:52.617>me <00:01:52.772>one <00:01:52.928>important <00:01:53.083>fact <00:01:53.239>about <00:01:53.395>software

38
00:01:56.015 --> 00:02:00.255
<00:01:56.015>testing <00:01:56.142>and <00:01:56.269>that <00:01:56.396>is <00:01:56.524>that <00:01:56.651>I <00:01:56.778>should <00:01:56.905>rather <00:01:57.033>see <00:01:57.160>tools <00:01:57.287>as <00:01:57.415>gases.

39
00:02:00.915 --> 00:02:03.975
<00:02:00.915>So <00:02:01.115>by <00:02:01.315>designing <00:02:01.515>tools <00:02:01.715>for <00:02:01.915>yeah, <00:02:02.115>the <00:02:02.315>purpose

40
00:02:04.635 --> 00:02:06.335
<00:02:04.635>of <00:02:05.101>automation <00:02:05.568>in <00:02:06.035>testing

41
00:02:07.035 --> 00:02:11.055
<00:02:07.035>and <00:02:07.235>by <00:02:07.435>observing <00:02:07.634>what <00:02:07.835>software <00:02:08.035>tests <00:02:08.234>actually <00:02:08.435>do,

42
00:02:11.775 --> 00:02:15.495
<00:02:11.775>I <00:02:11.930>learned <00:02:12.086>that <00:02:12.241>tools <00:02:12.397>do <00:02:12.552>not <00:02:12.708>test <00:02:12.863>just <00:02:13.019>as <00:02:13.175>gases.

43
00:02:15.675 --> 00:02:19.255
<00:02:15.675>Do <00:02:15.815>not <00:02:15.955>see. <00:02:16.095>So <00:02:16.235>tools <00:02:16.375>only <00:02:16.515>help <00:02:16.655>us <00:02:16.795>to <00:02:16.935>test <00:02:17.075>just

44
00:02:19.255 --> 00:02:21.015
<00:02:19.255>as <00:02:19.488>gases <00:02:19.721>only <00:02:19.954>help <00:02:20.188>us <00:02:20.421>to <00:02:20.655>see.

45
00:02:21.235 --> 00:02:22.895
<00:02:21.235>Or <00:02:21.515>in <00:02:21.795>other <00:02:22.075>words, <00:02:22.355>you <00:02:22.635>know,

46
00:02:22.995 --> 00:02:25.655
<00:02:22.995>the <00:02:23.228>gases <00:02:23.461>don't <00:02:23.695>do <00:02:23.928>the <00:02:24.161>seeing <00:02:24.395>just

47
00:02:25.675 --> 00:02:27.935
<00:02:25.675>as <00:02:25.908>the <00:02:26.141>tools <00:02:26.375>don't <00:02:26.608>do <00:02:26.841>the <00:02:27.075>testing.

48
00:02:29.195 --> 00:02:32.735
<00:02:29.195>So <00:02:29.370>the <00:02:29.545>bottom <00:02:29.720>line <00:02:29.894>is <00:02:30.070>that <00:02:30.245>testing <00:02:30.420>is <00:02:30.595>something

49
00:02:32.805 --> 00:02:35.415
<00:02:32.805>that <00:02:33.005>doesn't <00:02:33.205>happen <00:02:33.405>in <00:02:33.605>our <00:02:33.805>tools, <00:02:34.005>it's <00:02:34.205>something

50
00:02:35.415 --> 00:02:36.735
<00:02:35.415>that <00:02:35.712>happens <00:02:36.009>in <00:02:36.306>our <00:02:36.603>heads.

51
00:02:37.205 --> 00:02:38.455
<00:02:37.204>It's <00:02:37.579>a <00:02:37.954>thinking <00:02:38.329>activity.

52
00:02:38.995 --> 00:02:42.295
<00:02:38.995>And <00:02:39.150>for <00:02:39.306>that <00:02:39.461>simple <00:02:39.617>reason, <00:02:39.772>I <00:02:39.928>do <00:02:40.083>not <00:02:40.239>reduce <00:02:40.395>testing

53
00:02:42.435 --> 00:02:46.095
<00:02:42.435>to <00:02:42.590>an <00:02:42.746>act <00:02:42.901>of <00:02:43.057>artifact <00:02:43.212>management <00:02:43.368>where <00:02:43.523>you <00:02:43.679>use <00:02:43.835>tools

54
00:02:46.095 --> 00:02:48.775
<00:02:46.095>to <00:02:46.375>create, <00:02:46.655>to <00:02:46.935>automate, <00:02:47.215>to <00:02:47.495>execute

55
00:02:48.835 --> 00:02:50.735
<00:02:48.835>and <00:02:49.185>to <00:02:49.535>maintain <00:02:49.885>test <00:02:50.235>cases.

56
00:02:50.875 --> 00:02:53.495
<00:02:50.875>For <00:02:51.015>example, <00:02:51.155>testing <00:02:51.295>to <00:02:51.435>me <00:02:51.575>is <00:02:51.715>so <00:02:51.855>much <00:02:51.995>more <00:02:52.135>than <00:02:52.275>that.

57
00:02:53.795 --> 00:02:56.295
<00:02:53.795>And <00:02:53.995>therefore <00:02:54.195>I <00:02:54.395>like <00:02:54.595>to <00:02:54.795>say <00:02:54.995>that <00:02:55.195>uh,

58
00:02:56.295 --> 00:02:58.735
<00:02:56.295>testing <00:02:56.528>is <00:02:56.761>no <00:02:56.995>more <00:02:57.228>about <00:02:57.461>tools <00:02:57.695>than

59
00:02:59.245 --> 00:03:01.125
<00:02:59.245>astronomy <00:02:59.711>is <00:03:00.178>about <00:03:00.645>telescopes.

60
00:03:01.945 --> 00:03:06.925
<00:03:01.945>And <00:03:02.144>if, <00:03:02.345>if <00:03:02.545>we <00:03:02.745>would <00:03:02.945>reduce <00:03:03.144>the <00:03:03.345>act

61
00:03:06.925 --> 00:03:09.485
<00:03:06.925>of <00:03:07.100>testing <00:03:07.275>to <00:03:07.450>an <00:03:07.625>act <00:03:07.800>of <00:03:07.975>test <00:03:08.150>case <00:03:08.325>creation

62
00:03:09.705 --> 00:03:11.965
<00:03:09.705>and <00:03:10.055>especially <00:03:10.405>test <00:03:10.755>case <00:03:11.105>automation,

63
00:03:12.155 --> 00:03:14.605
<00:03:12.155>then <00:03:12.295>I <00:03:12.435>think <00:03:12.575>we <00:03:12.715>also <00:03:12.855>would <00:03:12.995>need <00:03:13.135>to <00:03:13.275>accept <00:03:13.415>the <00:03:13.555>fact

64
00:03:14.675 --> 00:03:16.405
<00:03:14.675>that <00:03:14.955>when <00:03:15.235>we <00:03:15.515>are <00:03:15.795>testing <00:03:16.075>badly,

65
00:03:19.045 --> 00:03:20.875
<00:03:19.045>which <00:03:19.220>just <00:03:19.395>help <00:03:19.570>us <00:03:19.745>to <00:03:19.920>test <00:03:20.095>badly <00:03:20.270>at <00:03:20.445>an

66
00:03:20.875 --> 00:03:22.035
<00:03:20.875>accelerated <00:03:21.397>rate, <00:03:21.918>right?

67
00:03:22.375 --> 00:03:25.235
<00:03:22.375>So <00:03:22.515>this <00:03:22.655>means <00:03:22.795>that <00:03:22.935>we <00:03:23.075>need <00:03:23.215>to <00:03:23.355>avoid <00:03:23.495>to <00:03:23.635>automate <00:03:23.775>chaos

68
00:03:25.345 --> 00:03:28.875
<00:03:25.345>because <00:03:25.484>if <00:03:25.625>you <00:03:25.765>automate <00:03:25.905>chaos, <00:03:26.045>all <00:03:26.185>you <00:03:26.325>get <00:03:26.465>is <00:03:26.605>faster <00:03:26.745>chaos.

69
00:03:29.535 --> 00:03:31.035
<00:03:29.535>And <00:03:29.805>this <00:03:30.075>usually <00:03:30.345>looks <00:03:30.615>like <00:03:30.885>that.

70
00:03:31.035 --> 00:03:33.515
<00:03:31.035>So <00:03:31.234>here <00:03:31.435>is <00:03:31.635>a <00:03:31.835>typically <00:03:32.035>software <00:03:32.234>testing <00:03:32.435>timeline

71
00:03:33.895 --> 00:03:36.715
<00:03:33.895>and <00:03:34.128>this <00:03:34.361>software <00:03:34.595>testing <00:03:34.828>timeline <00:03:35.061>is <00:03:35.295>usually

72
00:03:37.195 --> 00:03:38.355
<00:03:37.195>composed <00:03:37.543>of <00:03:37.891>four <00:03:38.238>phases.

73
00:03:38.975 --> 00:03:41.155
<00:03:38.975>Now, <00:03:39.150>in <00:03:39.325>the <00:03:39.500>first <00:03:39.674>phase <00:03:39.850>I <00:03:40.025>hear <00:03:40.200>people <00:03:40.375>saying

74
00:03:41.155 --> 00:03:42.555
<00:03:41.155>testing, <00:03:41.470>never <00:03:41.785>heard <00:03:42.100>of <00:03:42.415>that.

75
00:03:42.685 --> 00:03:46.115
<00:03:42.685>There <00:03:42.918>is <00:03:43.151>only <00:03:43.385>automation, <00:03:43.618>we <00:03:43.851>automate <00:03:44.085>everything.

76
00:03:46.775 --> 00:03:48.475
<00:03:46.775>Now <00:03:47.055>it <00:03:47.335>usually <00:03:47.615>takes <00:03:47.895>some <00:03:48.175>time

77
00:03:48.845 --> 00:03:51.075
<00:03:48.845>until <00:03:49.078>those <00:03:49.311>people <00:03:49.545>then <00:03:49.778>start <00:03:50.011>falling <00:03:50.245>down.

78
00:03:51.545 --> 00:03:54.595
<00:03:51.545>What <00:03:51.685>I <00:03:51.825>like <00:03:51.965>to <00:03:52.105>call <00:03:52.245>the <00:03:52.385>mount <00:03:52.525>stupid <00:03:52.665>of <00:03:52.805>software <00:03:52.945>testing

79
00:03:55.015 --> 00:03:57.155
<00:03:55.015>and <00:03:55.248>it's <00:03:55.481>a <00:03:55.714>huge <00:03:55.948>mountain, <00:03:56.181>trust <00:03:56.415>me.

80
00:03:57.575 --> 00:03:59.755
<00:03:57.575>But <00:03:57.774>when <00:03:57.975>those <00:03:58.174>people <00:03:58.375>then <00:03:58.575>start <00:03:58.774>falling <00:03:58.975>down

81
00:03:59.755 --> 00:04:04.365
<00:03:59.755>that <00:03:59.930>mount <00:04:00.105>stupid, <00:04:00.280>they <00:04:00.454>immerse <00:04:00.630>into <00:04:00.805>what <00:04:00.980>I <00:04:01.155>like

82
00:04:04.365 --> 00:04:06.005
<00:04:04.365>to <00:04:04.598>call <00:04:04.831>the <00:04:05.065>value <00:04:05.298>of <00:04:05.531>des <00:04:05.765>pair.

83
00:04:06.385 --> 00:04:09.245
<00:04:06.385>And <00:04:06.512>in <00:04:06.639>that <00:04:06.766>value <00:04:06.894>of <00:04:07.021>dep <00:04:07.148>pair, <00:04:07.275>I <00:04:07.403>hear <00:04:07.530>those <00:04:07.657>people <00:04:07.785>saying,

84
00:04:09.555 --> 00:04:10.925
<00:04:09.555>okay, <00:04:09.966>there <00:04:10.377>is <00:04:10.788>testing

85
00:04:11.825 --> 00:04:14.365
<00:04:11.825>and <00:04:12.024>testing <00:04:12.225>is <00:04:12.424>probably <00:04:12.625>more <00:04:12.825>than <00:04:13.024>just <00:04:13.225>automation,

86
00:04:14.545 --> 00:04:17.205
<00:04:14.545>but <00:04:14.684>we <00:04:14.825>are <00:04:14.964>just <00:04:15.105>not <00:04:15.244>convinced <00:04:15.385>that <00:04:15.524>we <00:04:15.665>really <00:04:15.804>need <00:04:15.945>it.

87
00:04:17.665 --> 00:04:19.685
<00:04:17.665>And <00:04:17.840>you <00:04:18.015>know <00:04:18.190>what? <00:04:18.365>The <00:04:18.540>last <00:04:18.715>two <00:04:18.890>phases <00:04:19.065>then

88
00:04:19.685 --> 00:04:20.805
<00:04:19.685>often <00:04:20.021>look <00:04:20.357>like <00:04:20.692>this.

89
00:04:21.965 --> 00:04:24.985
<00:04:21.964>So <00:04:22.120>now <00:04:22.276>let's <00:04:22.431>zoom <00:04:22.587>into <00:04:22.742>those <00:04:22.898>last <00:04:23.053>two <00:04:23.209>phases, <00:04:23.364>uh,

90
00:04:25.205 --> 00:04:26.825
<00:04:25.205>to <00:04:25.438>see <00:04:25.671>what's <00:04:25.905>really <00:04:26.138>going <00:04:26.371>on <00:04:26.604>there.

91
00:04:27.405 --> 00:04:28.905
<00:04:27.405>And <00:04:27.674>now <00:04:27.945>I <00:04:28.215>need <00:04:28.484>your <00:04:28.755>imagination.

92
00:04:29.295 --> 00:04:32.465
<00:04:29.295>Imagine <00:04:29.435>that <00:04:29.575>all <00:04:29.715>these <00:04:29.855>dots <00:04:29.995>you <00:04:30.135>see <00:04:30.275>now <00:04:30.415>on <00:04:30.555>that <00:04:30.695>slide

93
00:04:33.425 --> 00:04:36.745
<00:04:33.425>represent <00:04:33.600>all <00:04:33.775>the <00:04:33.950>test <00:04:34.125>cases <00:04:34.300>you <00:04:34.475>have, <00:04:34.650>for <00:04:34.825>example,

94
00:04:36.805 --> 00:04:38.425
<00:04:36.805>in <00:04:37.155>your <00:04:37.505>regression <00:04:37.855>test <00:04:38.205>portfolio.

95
00:04:39.045 --> 00:04:43.345
<00:04:39.045>And <00:04:39.185>now <00:04:39.325>imagine <00:04:39.465>that <00:04:39.605>all <00:04:39.745>these <00:04:39.885>test <00:04:40.025>cases <00:04:40.165>pass <00:04:40.305>in, <00:04:40.445>yeah,

96
00:04:43.365 --> 00:04:44.905
<00:04:43.365>the <00:04:43.711>last <00:04:44.058>regression <00:04:44.404>test <00:04:44.751>run

97
00:04:44.905 --> 00:04:47.865
<00:04:44.905>before <00:04:45.080>you <00:04:45.255>decide <00:04:45.429>to <00:04:45.604>ship <00:04:45.780>your <00:04:45.955>product <00:04:46.130>into <00:04:46.304>production.

98
00:04:48.325 --> 00:04:50.705
<00:04:48.325>And <00:04:48.465>then <00:04:48.604>you <00:04:48.745>might <00:04:48.885>con <00:04:49.025>Google, <00:04:49.164>well <00:04:49.305>that <00:04:49.445>looks <00:04:49.585>good, <00:04:49.724>right?

99
00:04:50.725 --> 00:04:52.905
<00:04:50.725>So <00:04:50.900>let's <00:04:51.075>ship <00:04:51.250>it, <00:04:51.425>let's <00:04:51.600>ship <00:04:51.775>our <00:04:51.950>software <00:04:52.125>product.

100
00:04:53.325 --> 00:04:56.185
<00:04:53.325>And <00:04:53.480>that <00:04:53.636>is <00:04:53.791>what <00:04:53.947>you <00:04:54.102>then <00:04:54.258>most <00:04:54.413>probably <00:04:54.569>will <00:04:54.724>do.

101
00:04:56.885 --> 00:04:59.425
<00:04:56.885>But <00:04:57.025>just <00:04:57.164>a <00:04:57.305>second <00:04:57.445>later <00:04:57.585>you <00:04:57.724>will <00:04:57.865>see <00:04:58.005>something <00:04:58.145>like <00:04:58.284>this.

102
00:04:59.565 --> 00:05:02.545
<00:04:59.565>You <00:04:59.765>will <00:04:59.965>see <00:05:00.165>that <00:05:00.365>repetitive <00:05:00.565>test <00:05:00.765>case <00:05:00.965>execution,

103
00:05:03.005 --> 00:05:05.305
<00:05:03.005>no <00:05:03.238>matter <00:05:03.471>if <00:05:03.705>it's <00:05:03.938>done <00:05:04.171>by <00:05:04.405>machines

104
00:05:05.305 --> 00:05:07.945
<00:05:05.305>or <00:05:05.505>by <00:05:05.705>humans, <00:05:05.905>is <00:05:06.105>by <00:05:06.305>far <00:05:06.505>not <00:05:06.705>enough

105
00:05:08.685 --> 00:05:10.265
<00:05:08.685>to <00:05:09.035>avoid <00:05:09.385>scenarios <00:05:09.735>like <00:05:10.085>that.

106
00:05:10.885 --> 00:05:12.625
<00:05:10.885>Now <00:05:11.085>this <00:05:11.284>is <00:05:11.485>usually <00:05:11.685>the <00:05:11.885>point <00:05:12.085>in <00:05:12.284>time

107
00:05:12.625 --> 00:05:14.185
<00:05:12.625>where <00:05:13.091>people <00:05:13.558>start <00:05:14.025>realizing

108
00:05:14.415 --> 00:05:17.345
<00:05:14.415>that <00:05:14.615>testing <00:05:14.815>shouldn't <00:05:15.015>be <00:05:15.215>confused <00:05:15.415>with <00:05:15.615>test <00:05:15.815>cases,

109
00:05:17.915 --> 00:05:19.505
<00:05:17.915>these <00:05:18.381>people <00:05:18.848>start <00:05:19.315>realizing

110
00:05:19.505 --> 00:05:22.545
<00:05:19.505>that <00:05:19.660>testing <00:05:19.816>shouldn't <00:05:19.971>be <00:05:20.127>reduced <00:05:20.282>to <00:05:20.438>the <00:05:20.593>act <00:05:20.749>of <00:05:20.905>creating,

111
00:05:22.555 --> 00:05:24.465
<00:05:22.555>automating, <00:05:22.905>and <00:05:23.255>executing <00:05:23.605>test <00:05:23.955>cases.

112
00:05:24.735 --> 00:05:26.625
<00:05:24.735>Testing <00:05:24.968>is <00:05:25.201>so <00:05:25.435>much <00:05:25.668>more <00:05:25.901>than <00:05:26.135>that.

113
00:05:27.245 --> 00:05:31.105
<00:05:27.245>So <00:05:27.385>now <00:05:27.525>let's <00:05:27.665>dive <00:05:27.805>a <00:05:27.945>little <00:05:28.085>bit <00:05:28.225>deeper <00:05:28.365>into <00:05:28.505>the <00:05:28.645>subject

114
00:05:31.245 --> 00:05:34.545
<00:05:31.245>of <00:05:31.400>software <00:05:31.556>testing <00:05:31.711>to <00:05:31.867>understand <00:05:32.022>that <00:05:32.178>a <00:05:32.333>little <00:05:32.489>bit <00:05:32.645>better.

115
00:05:34.965 --> 00:05:37.465
<00:05:34.965>And <00:05:35.140>to <00:05:35.315>do <00:05:35.490>that <00:05:35.665>I <00:05:35.840>need <00:05:36.015>your <00:05:36.190>imagination <00:05:36.365>again.

116
00:05:38.045 --> 00:05:42.065
<00:05:38.045>So <00:05:38.185>now <00:05:38.325>imagine <00:05:38.465>that <00:05:38.605>this <00:05:38.745>black <00:05:38.885>rectangle <00:05:39.025>shown <00:05:39.165>on <00:05:39.305>the <00:05:39.445>right

117
00:05:42.065 --> 00:05:45.985
<00:05:42.065>hand <00:05:42.298>side <00:05:42.531>now <00:05:42.765>represents <00:05:42.998>your <00:05:43.231>software <00:05:43.465>product

118
00:05:46.885 --> 00:05:48.985
<00:05:46.885>and <00:05:47.164>it <00:05:47.445>represents <00:05:47.724>the <00:05:48.005>entire <00:05:48.284>functional

119
00:05:48.985 --> 00:05:51.265
<00:05:48.985>and <00:05:49.265>nonfunctional <00:05:49.545>spectrum <00:05:49.825>of <00:05:50.105>your <00:05:50.385>software.

120
00:05:51.445 --> 00:05:54.705
<00:05:51.445>And <00:05:51.620>it <00:05:51.795>also <00:05:51.969>represents <00:05:52.145>all <00:05:52.320>the <00:05:52.495>environments <00:05:52.670>your <00:05:52.844>software

121
00:05:54.965 --> 00:05:58.045
<00:05:54.965>can <00:05:55.105>be <00:05:55.245>situated <00:05:55.385>in <00:05:55.525>and <00:05:55.665>it <00:05:55.805>also <00:05:55.945>represents <00:05:56.085>all <00:05:56.225>the <00:05:56.365>people

122
00:05:58.275 --> 00:05:59.725
<00:05:58.275>that <00:05:58.601>might <00:05:58.927>use <00:05:59.253>your <00:05:59.580>software.

123
00:06:00.305 --> 00:06:01.645
<00:06:00.305>So <00:06:00.606>that's <00:06:00.908>all <00:06:01.209>there <00:06:01.510>is.

124
00:06:01.745 --> 00:06:05.685
<00:06:01.745>That's <00:06:01.920>all <00:06:02.095>you <00:06:02.270>could <00:06:02.445>possibly <00:06:02.620>know <00:06:02.795>about <00:06:02.970>your <00:06:03.145>software.

125
00:06:05.865 --> 00:06:10.445
<00:06:05.865>Try <00:06:06.020>to <00:06:06.176>imagine <00:06:06.331>that. <00:06:06.487>Now <00:06:06.642>then <00:06:06.798>the <00:06:06.953>green <00:06:07.109>rectangle <00:06:07.265>represents

126
00:06:11.265 --> 00:06:13.725
<00:06:11.265>all <00:06:11.465>what <00:06:11.664>you <00:06:11.865>currently <00:06:12.065>know <00:06:12.265>about <00:06:12.465>your <00:06:12.664>software.

127
00:06:14.025 --> 00:06:17.805
<00:06:14.025>And <00:06:14.180>this <00:06:14.336>rectangle <00:06:14.491>is <00:06:14.647>much <00:06:14.802>smaller <00:06:14.958>than <00:06:15.113>the <00:06:15.269>previous <00:06:15.424>one

128
00:06:17.805 --> 00:06:19.525
<00:06:17.805>because <00:06:18.271>it's <00:06:18.738>simply <00:06:19.205>impossible

129
00:06:19.545 --> 00:06:22.205
<00:06:19.545>to <00:06:19.745>always <00:06:19.945>know <00:06:20.145>everything <00:06:20.345>about <00:06:20.545>your <00:06:20.745>software <00:06:20.945>down

130
00:06:22.225 --> 00:06:23.765
<00:06:22.225>to <00:06:22.502>the <00:06:22.779>very <00:06:23.056>last <00:06:23.333>detail, <00:06:23.611>right?

131
00:06:24.705 --> 00:06:28.245
<00:06:24.705>And <00:06:24.905>the <00:06:25.104>red <00:06:25.305>rectangle <00:06:25.505>represents <00:06:25.705>everything <00:06:25.905>you <00:06:26.104>repeatedly

132
00:06:28.535 --> 00:06:30.685
<00:06:28.535>check <00:06:28.815>in <00:06:29.095>your <00:06:29.375>software, <00:06:29.655>for <00:06:29.935>example,

133
00:06:30.785 --> 00:06:32.965
<00:06:30.785>by <00:06:31.135>automated <00:06:31.485>repetitive <00:06:31.835>regression <00:06:32.185>checking.

134
00:06:33.625 --> 00:06:35.125
<00:06:33.625>And <00:06:33.850>again, <00:06:34.075>you <00:06:34.300>will <00:06:34.525>never <00:06:34.750>be <00:06:34.975>able

135
00:06:35.125 --> 00:06:37.085
<00:06:35.125>to <00:06:35.325>check <00:06:35.525>everything <00:06:35.725>you <00:06:35.925>know <00:06:36.125>at <00:06:36.325>any <00:06:36.525>time

136
00:06:37.395 --> 00:06:39.405
<00:06:37.395>because <00:06:37.674>of <00:06:37.955>the <00:06:38.234>time <00:06:38.515>the <00:06:38.794>resource

137
00:06:39.425 --> 00:06:43.085
<00:06:39.425>and <00:06:39.600>the <00:06:39.775>budget <00:06:39.950>constraints <00:06:40.125>you <00:06:40.300>are <00:06:40.475>confronted <00:06:40.650>with, <00:06:40.825>yeah,

138
00:06:43.255 --> 00:06:46.165
<00:06:43.255>every <00:06:43.455>single <00:06:43.655>moment <00:06:43.855>in <00:06:44.055>your <00:06:44.255>daily <00:06:44.455>testing <00:06:44.655>business.

139
00:06:46.985 --> 00:06:50.685
<00:06:46.985>And <00:06:47.101>so <00:06:47.218>the <00:06:47.335>question <00:06:47.451>here <00:06:47.568>is, <00:06:47.685>well <00:06:47.801>would <00:06:47.918>it <00:06:48.035>make <00:06:48.151>sense <00:06:48.268>to <00:06:48.385>try

140
00:06:50.865 --> 00:06:53.765
<00:06:50.865>to <00:06:51.098>automate, <00:06:51.331>to <00:06:51.565>codify <00:06:51.798>all <00:06:52.031>your <00:06:52.265>knowledge?

141
00:06:54.635 --> 00:06:56.445
<00:06:54.635>Well, <00:06:54.835>most <00:06:55.034>people <00:06:55.235>would <00:06:55.435>say <00:06:55.635>yes, <00:06:55.835>of <00:06:56.034>course

142
00:06:56.445 --> 00:06:58.405
<00:06:56.445>that's <00:06:56.678>the <00:06:56.911>ultimate <00:06:57.145>goal <00:06:57.378>of <00:06:57.611>software <00:06:57.844>testing.

143
00:06:58.905 --> 00:07:00.045
<00:06:58.905>But <00:06:59.246>I'm <00:06:59.589>saying <00:06:59.931>no

144
00:07:00.785 --> 00:07:03.355
<00:07:00.785>because <00:07:01.065>when <00:07:01.345>you <00:07:01.625>would <00:07:01.905>only <00:07:02.185>aim

145
00:07:03.495 --> 00:07:06.315
<00:07:03.495>to <00:07:03.650>automate <00:07:03.806>all <00:07:03.961>your <00:07:04.117>knowledge, <00:07:04.272>then <00:07:04.428>the <00:07:04.583>question <00:07:04.739>is, <00:07:04.895>well

146
00:07:06.315 --> 00:07:08.595
<00:07:06.315>how <00:07:06.515>on <00:07:06.715>earth <00:07:06.915>would <00:07:07.115>you <00:07:07.315>then <00:07:07.515>be <00:07:07.715>able

147
00:07:08.595 --> 00:07:10.795
<00:07:08.595>to <00:07:08.828>create <00:07:09.061>new <00:07:09.295>knowledge <00:07:09.528>about <00:07:09.761>the <00:07:09.995>software?

148
00:07:11.695 --> 00:07:14.395
<00:07:11.695>And <00:07:11.850>you <00:07:12.006>have <00:07:12.161>to <00:07:12.317>create <00:07:12.472>new <00:07:12.628>knowledge <00:07:12.783>about <00:07:12.939>your <00:07:13.094>software

149
00:07:14.395 --> 00:07:16.435
<00:07:14.395>because <00:07:14.745>your <00:07:15.094>software <00:07:15.445>constantly <00:07:15.794>grows.

150
00:07:16.855 --> 00:07:18.395
<00:07:16.855>And <00:07:17.201>so <00:07:17.548>you <00:07:17.894>constantly <00:07:18.241>need

151
00:07:18.415 --> 00:07:20.835
<00:07:18.415>to <00:07:18.648>create <00:07:18.881>new <00:07:19.115>knowledge <00:07:19.348>about <00:07:19.581>your <00:07:19.815>software

152
00:07:21.055 --> 00:07:24.595
<00:07:21.055>to <00:07:21.195>reveal <00:07:21.335>all <00:07:21.475>the <00:07:21.615>problems <00:07:21.755>in <00:07:21.895>your <00:07:22.035>software <00:07:22.175>that <00:07:22.315>are <00:07:22.455>far,

153
00:07:24.815 --> 00:07:26.635
<00:07:24.815>far <00:07:25.165>outside <00:07:25.515>your <00:07:25.865>checking <00:07:26.215>regime.

154
00:07:27.535 --> 00:07:30.515
<00:07:27.535>Now, <00:07:27.768>that <00:07:28.001>is <00:07:28.235>what <00:07:28.468>software <00:07:28.701>testing <00:07:28.935>is

155
00:07:30.515 --> 00:07:31.635
<00:07:30.515>all <00:07:30.716>about, <00:07:30.918>at <00:07:31.119>least <00:07:31.321>to <00:07:31.522>me.

156
00:07:31.635 --> 00:07:34.675
<00:07:31.635>Testing <00:07:31.835>is <00:07:32.034>all <00:07:32.235>about <00:07:32.435>closing <00:07:32.635>this <00:07:32.835>knowledge <00:07:33.034>gap.

157
00:07:34.985 --> 00:07:38.195
<00:07:34.985>It's <00:07:35.140>all <00:07:35.296>about <00:07:35.451>closing <00:07:35.607>the <00:07:35.762>gap <00:07:35.918>between <00:07:36.073>what <00:07:36.229>we <00:07:36.385>know

158
00:07:38.195 --> 00:07:40.435
<00:07:38.195>and <00:07:38.395>what <00:07:38.594>we <00:07:38.795>don't <00:07:38.995>know <00:07:39.195>about <00:07:39.395>our <00:07:39.594>software.

159
00:07:40.935 --> 00:07:42.795
<00:07:40.935>So <00:07:41.135>the <00:07:41.335>goal <00:07:41.535>of <00:07:41.735>testing <00:07:41.935>is <00:07:42.135>rather <00:07:42.335>yeah,

160
00:07:42.865 --> 00:07:44.875
<00:07:42.865>information <00:07:43.145>than <00:07:43.425>automation, <00:07:43.705>I <00:07:43.985>would <00:07:44.265>say.

161
00:07:45.135 --> 00:07:48.035
<00:07:45.135>And <00:07:45.335>the <00:07:45.534>reason <00:07:45.735>for <00:07:45.935>this <00:07:46.135>is <00:07:46.335>fairly <00:07:46.534>simple.

162
00:07:48.755 --> 00:07:51.595
<00:07:48.755>A <00:07:49.034>test <00:07:49.315>itself <00:07:49.594>is <00:07:49.875>just <00:07:50.155>yeah,

163
00:07:51.835 --> 00:07:53.555
<00:07:51.835>a <00:07:52.114>question <00:07:52.395>you <00:07:52.674>ask <00:07:52.955>your <00:07:53.234>software.

164
00:07:53.695 --> 00:07:56.235
<00:07:53.695>And <00:07:53.895>the <00:07:54.094>purpose <00:07:54.295>of <00:07:54.495>performing <00:07:54.695>the <00:07:54.895>test <00:07:55.094>is

165
00:07:56.235 --> 00:07:58.795
<00:07:56.235>to <00:07:56.515>gain <00:07:56.795>information <00:07:57.075>about <00:07:57.355>your <00:07:57.635>software

166
00:07:58.815 --> 00:08:00.675
<00:07:58.815>to <00:07:59.048>reveal <00:07:59.281>those <00:07:59.515>problems <00:07:59.748>you've <00:07:59.981>seen <00:08:00.215>before.

167
00:08:01.415 --> 00:08:03.555
<00:08:01.415>So <00:08:01.695>testing <00:08:01.975>is <00:08:02.255>always <00:08:02.535>has <00:08:02.815>been

168
00:08:03.615 --> 00:08:07.915
<00:08:03.615>and <00:08:03.790>hopefully <00:08:03.965>always <00:08:04.140>will <00:08:04.315>be <00:08:04.490>a <00:08:04.665>search <00:08:04.840>for <00:08:05.015>information.

169
00:08:08.295 --> 00:08:10.355
<00:08:08.295>And <00:08:08.528>therefore <00:08:08.761>I <00:08:08.995>like <00:08:09.228>to <00:08:09.461>see <00:08:09.695>testing

170
00:08:10.495 --> 00:08:14.675
<00:08:10.495>as <00:08:10.670>an <00:08:10.845>information <00:08:11.020>service, <00:08:11.195>not <00:08:11.370>as <00:08:11.545>an <00:08:11.720>automation <00:08:11.895>service.

171
00:08:15.925 --> 00:08:19.065
<00:08:15.925>So <00:08:16.100>in <00:08:16.275>its <00:08:16.450>shortest <00:08:16.625>possible <00:08:16.800>form, <00:08:16.975>I <00:08:17.150>would <00:08:17.325>say

172
00:08:19.065 --> 00:08:21.105
<00:08:19.065>that <00:08:19.344>the <00:08:19.625>purpose <00:08:19.905>of <00:08:20.185>testing <00:08:20.465>is

173
00:08:21.105 --> 00:08:24.865
<00:08:21.105>to <00:08:21.305>collect <00:08:21.505>quality <00:08:21.705>related <00:08:21.905>information <00:08:22.105>about <00:08:22.305>our <00:08:22.505>software

174
00:08:25.285 --> 00:08:28.745
<00:08:25.285>to <00:08:25.460>enable <00:08:25.635>other <00:08:25.810>people <00:08:25.985>to <00:08:26.160>make <00:08:26.335>better <00:08:26.510>informed <00:08:26.685>decisions.

175
00:08:28.845 --> 00:08:32.425
<00:08:28.845>That's <00:08:29.020>it. <00:08:29.195>So <00:08:29.370>testing <00:08:29.545>at <00:08:29.720>its <00:08:29.895>core <00:08:30.070>is <00:08:30.245>just

176
00:08:32.425 --> 00:08:33.665
<00:08:32.424>about <00:08:32.982>three <00:08:33.540>things.

177
00:08:34.095 --> 00:08:36.985
<00:08:34.095>It's <00:08:34.375>about <00:08:34.655>information, <00:08:34.935>people <00:08:35.215>and <00:08:35.495>decisions.

178
00:08:37.525 --> 00:08:40.105
<00:08:37.525>So <00:08:37.699>first <00:08:37.875>we <00:08:38.049>collect <00:08:38.225>quality <00:08:38.400>related <00:08:38.574>information <00:08:38.750>about <00:08:38.924>our

179
00:08:40.345 --> 00:08:42.145
<00:08:40.345>software, <00:08:40.625>and <00:08:40.905>that's <00:08:41.185>information, <00:08:41.465>for <00:08:41.745>example,

180
00:08:42.155 --> 00:08:45.265
<00:08:42.155>about <00:08:42.310>the <00:08:42.466>actual <00:08:42.621>or <00:08:42.777>the <00:08:42.932>potential <00:08:43.088>problems <00:08:43.243>in <00:08:43.399>our <00:08:43.554>software.

181
00:08:45.885 --> 00:08:48.705
<00:08:45.885>And <00:08:46.060>then <00:08:46.235>we <00:08:46.410>provide <00:08:46.585>this <00:08:46.760>information <00:08:46.935>to <00:08:47.110>other <00:08:47.285>people,

182
00:08:48.925 --> 00:08:52.785
<00:08:48.925>people <00:08:49.099>such <00:08:49.275>as <00:08:49.449>developers, <00:08:49.625>people <00:08:49.800>such <00:08:49.974>as <00:08:50.150>product <00:08:50.324>owners,

183
00:08:52.785 --> 00:08:55.565
<00:08:52.785>product <00:08:53.064>managers, <00:08:53.344>support <00:08:53.625>engineers, <00:08:53.905>release <00:08:54.185>engineers

184
00:08:55.945 --> 00:08:59.045
<00:08:55.945>to <00:08:56.120>enable <00:08:56.295>these <00:08:56.470>people <00:08:56.645>to <00:08:56.820>make <00:08:56.995>better <00:08:57.170>informed <00:08:57.345>decisions

185
00:08:59.575 --> 00:09:01.445
<00:08:59.575>based <00:08:59.855>on <00:09:00.135>the <00:09:00.415>information <00:09:00.695>we <00:09:00.975>provide.

186
00:09:01.985 --> 00:09:04.445
<00:09:01.985>And <00:09:02.218>these <00:09:02.451>decisions <00:09:02.685>might <00:09:02.918>be <00:09:03.151>shipping <00:09:03.385>decisions

187
00:09:04.445 --> 00:09:06.325
<00:09:04.445>or <00:09:04.795>fixing <00:09:05.145>decisions, <00:09:05.495>for <00:09:05.845>example.

188
00:09:06.785 --> 00:09:09.205
<00:09:06.785>So <00:09:06.985>the <00:09:07.185>bottom <00:09:07.385>line <00:09:07.584>is <00:09:07.785>helping <00:09:07.985>other <00:09:08.185>people

189
00:09:09.465 --> 00:09:11.205
<00:09:09.465>to <00:09:09.815>make <00:09:10.165>better <00:09:10.515>informed <00:09:10.865>decisions.

190
00:09:11.465 --> 00:09:12.525
<00:09:11.465>So <00:09:11.703>that's <00:09:11.942>the <00:09:12.180>key <00:09:12.419>here

191
00:09:12.665 --> 00:09:15.165
<00:09:12.665>and <00:09:12.898>that <00:09:13.131>is <00:09:13.365>how <00:09:13.598>we <00:09:13.831>create <00:09:14.064>value

192
00:09:15.195 --> 00:09:17.805
<00:09:15.195>through <00:09:15.545>testing <00:09:15.895>from <00:09:16.245>my <00:09:16.595>perspective.

193
00:09:18.985 --> 00:09:21.085
<00:09:18.985>Now <00:09:19.265>this <00:09:19.545>backs <00:09:19.825>the <00:09:20.105>question, <00:09:20.385>well

194
00:09:21.085 --> 00:09:24.245
<00:09:21.085>how <00:09:21.225>do <00:09:21.365>we <00:09:21.505>collect <00:09:21.645>this <00:09:21.785>information <00:09:21.925>in <00:09:22.065>the <00:09:22.205>best <00:09:22.345>possible <00:09:22.485>way?

195
00:09:24.465 --> 00:09:26.725
<00:09:24.465>Or <00:09:24.665>in <00:09:24.865>other <00:09:25.065>words, <00:09:25.265>what <00:09:25.465>does <00:09:25.665>it <00:09:25.865>take

196
00:09:26.785 --> 00:09:27.925
<00:09:26.785>to <00:09:27.127>do <00:09:27.468>excellent <00:09:27.810>testing?

197
00:09:28.665 --> 00:09:30.845
<00:09:28.665>And <00:09:28.820>I <00:09:28.976>wanna <00:09:29.131>give <00:09:29.287>you <00:09:29.442>an <00:09:29.598>answer <00:09:29.753>to <00:09:29.909>this <00:09:30.064>question

198
00:09:31.625 --> 00:09:33.485
<00:09:31.625>by <00:09:31.905>telling <00:09:32.185>you <00:09:32.465>a <00:09:32.745>short <00:09:33.025>story.

199
00:09:33.825 --> 00:09:35.525
<00:09:33.825>And <00:09:34.105>this <00:09:34.385>story <00:09:34.665>is <00:09:34.945>about <00:09:35.225>Alis.

200
00:09:36.755 --> 00:09:40.095
<00:09:36.755>Now, <00:09:36.930>ALIS <00:09:37.105>not <00:09:37.280>only <00:09:37.455>is <00:09:37.630>a, <00:09:37.805>a <00:09:37.980>software <00:09:38.155>tester,

201
00:09:40.575 --> 00:09:42.415
<00:09:40.575>ALIS <00:09:40.855>is <00:09:41.135>an <00:09:41.415>excellent <00:09:41.695>software <00:09:41.975>tester.

202
00:09:42.955 --> 00:09:45.455
<00:09:42.955>Now, <00:09:43.130>excellent <00:09:43.305>software <00:09:43.480>testing <00:09:43.655>is <00:09:43.830>really <00:09:44.005>hard <00:09:44.180>to <00:09:44.355>describe

203
00:09:45.455 --> 00:09:47.055
<00:09:45.455>because <00:09:45.805>you <00:09:46.155>only <00:09:46.505>find <00:09:46.855>out

204
00:09:47.365 --> 00:09:49.975
<00:09:47.365>what <00:09:47.598>excellent <00:09:47.831>testing <00:09:48.065>is <00:09:48.298>when <00:09:48.531>you <00:09:48.765>see.

205
00:09:50.555 --> 00:09:55.295
<00:09:50.555>So <00:09:50.694>here <00:09:50.834>is <00:09:50.974>what <00:09:51.114>I've <00:09:51.255>seen <00:09:51.395>here <00:09:51.535>are <00:09:51.675>18 <00:09:51.814>characteristics <00:09:51.954>of,

206
00:09:55.315 --> 00:09:57.175
<00:09:55.315>uh, <00:09:55.781>SSIS <00:09:56.248>testing <00:09:56.715>excellence.

207
00:09:58.035 --> 00:10:01.295
<00:09:58.035>Now <00:09:58.268>before <00:09:58.501>I <00:09:58.735>dive <00:09:58.968>into <00:09:59.201>these <00:09:59.435>characteristics,

208
00:10:01.615 --> 00:10:03.095
<00:10:01.615>I <00:10:01.837>want <00:10:02.059>you <00:10:02.281>to <00:10:02.503>do <00:10:02.725>one <00:10:02.947>thing.

209
00:10:03.755 --> 00:10:06.695
<00:10:03.755>So <00:10:03.895>I <00:10:04.035>want <00:10:04.175>you <00:10:04.314>to <00:10:04.455>just <00:10:04.595>lean <00:10:04.735>back <00:10:04.875>now <00:10:05.015>to <00:10:05.155>relax

210
00:10:06.875 --> 00:10:09.055
<00:10:06.875>and <00:10:07.075>also <00:10:07.275>feel <00:10:07.475>free <00:10:07.675>to <00:10:07.875>close <00:10:08.075>your <00:10:08.275>eyes

211
00:10:09.055 --> 00:10:10.815
<00:10:09.055>for <00:10:09.288>the <00:10:09.521>next <00:10:09.755>15 <00:10:09.988>minutes <00:10:10.221>or <00:10:10.454>so,

212
00:10:10.885 --> 00:10:14.375
<00:10:10.885>because <00:10:11.060>I <00:10:11.235>have <00:10:11.410>to <00:10:11.585>admit <00:10:11.760>that <00:10:11.935>the <00:10:12.110>next <00:10:12.285>uh,

213
00:10:14.515 --> 00:10:16.575
<00:10:14.515>slides <00:10:14.981>aren't <00:10:15.448>super <00:10:15.915>exciting.

214
00:10:17.035 --> 00:10:19.375
<00:10:17.035>The <00:10:17.268>story <00:10:17.501>is <00:10:17.735>what <00:10:17.968>truly <00:10:18.201>matters <00:10:18.435>now.

215
00:10:20.035 --> 00:10:23.575
<00:10:20.035>So <00:10:20.175>I <00:10:20.314>will <00:10:20.454>now <00:10:20.594>turn <00:10:20.735>this, <00:10:20.875>uh, <00:10:21.015>presentation <00:10:21.155>into <00:10:21.295>a <00:10:21.435>kind

216
00:10:23.575 --> 00:10:25.335
<00:10:23.575>of <00:10:23.808>a <00:10:24.041>play <00:10:24.275>reading, <00:10:24.508>so <00:10:24.741>to <00:10:24.975>say.

217
00:10:25.595 --> 00:10:28.095
<00:10:25.595>And, <00:10:25.750>and <00:10:25.906>I <00:10:26.061>really <00:10:26.217>heard, <00:10:26.372>uh, <00:10:26.528>hope <00:10:26.683>that <00:10:26.839>works <00:10:26.995>out,

218
00:10:28.955 --> 00:10:29.975
<00:10:28.955>na <00:10:29.184>then <00:10:29.414>let's <00:10:29.643>get <00:10:29.873>started.

219
00:10:30.075 --> 00:10:34.215
<00:10:30.075>Number <00:10:30.355>one, <00:10:30.635>Alice <00:10:30.915>knows <00:10:31.195>that <00:10:31.475>testing

220
00:10:34.755 --> 00:10:36.575
<00:10:34.755>is <00:10:35.221>a <00:10:35.688>social <00:10:36.155>science.

221
00:10:37.235 --> 00:10:38.535
<00:10:37.235>She <00:10:37.527>knows <00:10:37.819>that <00:10:38.112>the <00:10:38.405>quality

222
00:10:38.595 --> 00:10:41.055
<00:10:38.595>of <00:10:38.945>software <00:10:39.295>is <00:10:39.645>inherently <00:10:39.995>subjective.

223
00:10:41.395 --> 00:10:44.015
<00:10:41.395>So <00:10:41.550>she <00:10:41.706>knows <00:10:41.861>that <00:10:42.017>quality <00:10:42.172>is <00:10:42.328>value <00:10:42.483>to <00:10:42.639>some <00:10:42.795>people

224
00:10:44.075 --> 00:10:45.295
<00:10:44.075>who <00:10:44.349>matter <00:10:44.624>at <00:10:44.898>some <00:10:45.173>time.

225
00:10:45.915 --> 00:10:49.615
<00:10:45.915>And <00:10:46.089>she <00:10:46.265>understands <00:10:46.439>that <00:10:46.615>a <00:10:46.790>bug <00:10:46.964>is <00:10:47.140>just <00:10:47.314>something

226
00:10:49.615 --> 00:10:50.935
<00:10:49.615>that <00:10:50.011>threatens <00:10:50.406>this <00:10:50.803>value.

227
00:10:51.515 --> 00:10:53.895
<00:10:51.515>So <00:10:51.689>she <00:10:51.865>realizes <00:10:52.040>that <00:10:52.215>a <00:10:52.390>bug <00:10:52.564>is <00:10:52.740>simply <00:10:52.915>something

228
00:10:54.005 --> 00:10:55.415
<00:10:54.005>that <00:10:54.639>bugs <00:10:55.274>somebody

229
00:10:55.645 --> 00:10:56.695
<00:10:55.645>because <00:10:56.117>she's <00:10:56.590>learned

230
00:10:56.695 --> 00:10:59.255
<00:10:56.695>that <00:10:56.895>a <00:10:57.095>bug <00:10:57.295>isn't <00:10:57.495>necessarily <00:10:57.695>in <00:10:57.895>the <00:10:58.095>software,

231
00:10:59.435 --> 00:11:02.735
<00:10:59.435>but <00:10:59.635>rather <00:10:59.834>represents <00:11:00.035>a <00:11:00.234>relationship <00:11:00.435>between <00:11:00.635>a <00:11:00.834>person

232
00:11:02.915 --> 00:11:03.975
<00:11:02.915>and <00:11:03.391>the <00:11:03.869>software.

233
00:11:04.435 --> 00:11:06.095
<00:11:04.435>And <00:11:04.901>therefore <00:11:05.368>she <00:11:05.834>accepts

234
00:11:06.405 --> 00:11:10.415
<00:11:06.405>that <00:11:06.579>quality <00:11:06.755>cannot <00:11:06.930>really <00:11:07.105>be <00:11:07.280>measured <00:11:07.454>but <00:11:07.630>only <00:11:07.805>assessed.

235
00:11:11.515 --> 00:11:14.855
<00:11:11.515>So <00:11:11.715>Alice <00:11:11.915>acts <00:11:12.115>like <00:11:12.314>a <00:11:12.515>sociologist <00:11:12.715>who <00:11:12.915>looks

236
00:11:14.875 --> 00:11:18.095
<00:11:14.875>for <00:11:15.075>different <00:11:15.275>problems <00:11:15.475>that <00:11:15.675>matter <00:11:15.875>to <00:11:16.075>different <00:11:16.275>people

237
00:11:18.435 --> 00:11:19.975
<00:11:18.435>who <00:11:18.781>matter <00:11:19.127>because <00:11:19.474>she <00:11:19.821>knows

238
00:11:20.205 --> 00:11:23.135
<00:11:20.205>that <00:11:20.405>different <00:11:20.605>people <00:11:20.805>will <00:11:21.005>look <00:11:21.205>at, <00:11:21.405>you <00:11:21.605>know,

239
00:11:23.155 --> 00:11:25.815
<00:11:23.155>one <00:11:23.388>in <00:11:23.621>the <00:11:23.855>same <00:11:24.088>software <00:11:24.321>accepting <00:11:24.555>different

240
00:11:25.815 --> 00:11:27.135
<00:11:25.815>levels <00:11:26.409>of <00:11:27.003>quality.

241
00:11:28.305 --> 00:11:30.055
<00:11:28.305>Alice <00:11:29.005>also <00:11:29.704>understands

242
00:11:30.055 --> 00:11:34.215
<00:11:30.055>that <00:11:30.210>our <00:11:30.366>thinking <00:11:30.521>errors <00:11:30.677>can <00:11:30.832>become <00:11:30.988>software <00:11:31.143>errors <00:11:31.299>when <00:11:31.454>we

243
00:11:34.215 --> 00:11:36.975
<00:11:34.215>aren't <00:11:34.448>aware <00:11:34.681>of <00:11:34.915>our <00:11:35.148>cognitive <00:11:35.381>biases <00:11:35.615>and

244
00:11:36.975 --> 00:11:39.975
<00:11:36.975>therefore <00:11:37.115>she <00:11:37.255>looks <00:11:37.395>for <00:11:37.535>the <00:11:37.675>bug <00:11:37.815>in <00:11:37.955>the <00:11:38.095>mind <00:11:38.235>of <00:11:38.375>people.

245
00:11:40.675 --> 00:11:43.615
<00:11:40.675>So <00:11:40.849>she <00:11:41.025>acts <00:11:41.199>like <00:11:41.375>a <00:11:41.550>psychologist, <00:11:41.724>so <00:11:41.900>to <00:11:42.074>say,

246
00:11:43.995 --> 00:11:47.775
<00:11:43.995>who <00:11:44.135>dwells <00:11:44.275>into <00:11:44.415>the <00:11:44.555>mind <00:11:44.695>of <00:11:44.835>people <00:11:44.975>to <00:11:45.115>analyze <00:11:45.255>their <00:11:45.395>thinking

247
00:11:47.885 --> 00:11:51.135
<00:11:47.885>because <00:11:48.085>she <00:11:48.285>knows <00:11:48.485>that <00:11:48.685>most <00:11:48.885>of <00:11:49.085>people's <00:11:49.285>knowledge,

248
00:11:51.325 --> 00:11:53.740
<00:11:51.325>including <00:11:51.675>their <00:11:52.025>opinions, <00:11:52.375>their <00:11:52.725>assumptions

249
00:11:53.745 --> 00:11:57.445
<00:11:53.745>and <00:11:53.900>their <00:11:54.056>expectations <00:11:54.211>about <00:11:54.367>the <00:11:54.522>software <00:11:54.678>is <00:11:54.833>in <00:11:54.989>their <00:11:55.145>heads.

250
00:11:58.885 --> 00:12:03.675
<00:11:58.885>Alice <00:11:59.085>is <00:11:59.285>also <00:11:59.485>aware <00:11:59.685>that <00:11:59.885>automated <00:12:00.085>repetitive <00:12:00.285>checking

251
00:12:04.295 --> 00:12:06.515
<00:12:04.295>is <00:12:04.645>important <00:12:04.995>because <00:12:05.344>she <00:12:05.694>knows

252
00:12:06.515 --> 00:12:08.755
<00:12:06.515>that <00:12:06.748>she <00:12:06.981>cannot <00:12:07.215>trust <00:12:07.448>the <00:12:07.681>software <00:12:07.915>simply

253
00:12:08.755 --> 00:12:11.515
<00:12:08.755>because <00:12:08.988>the <00:12:09.221>software <00:12:09.455>continuously <00:12:09.688>changes <00:12:09.921>its <00:12:10.155>mind,

254
00:12:12.095 --> 00:12:13.795
<00:12:12.095>the <00:12:12.328>software <00:12:12.561>and <00:12:12.795>the <00:12:13.028>business <00:12:13.261>as <00:12:13.495>well

255
00:12:13.795 --> 00:12:16.875
<00:12:13.795>as <00:12:13.950>the <00:12:14.106>technical <00:12:14.261>domain <00:12:14.417>in <00:12:14.572>which <00:12:14.728>the <00:12:14.883>software <00:12:15.039>is <00:12:15.194>situated

256
00:12:17.245 --> 00:12:18.915
<00:12:17.245>constantly <00:12:17.711>and <00:12:18.178>rapidly <00:12:18.645>evolve.

257
00:12:19.615 --> 00:12:21.675
<00:12:19.615>Now <00:12:19.848>for <00:12:20.081>that <00:12:20.315>reason, <00:12:20.548>she <00:12:20.781>automates <00:12:21.015>checks

258
00:12:21.735 --> 00:12:24.675
<00:12:21.735>to <00:12:21.910>make <00:12:22.085>the <00:12:22.260>process <00:12:22.435>of <00:12:22.610>comparing <00:12:22.785>the <00:12:22.960>current <00:12:23.135>state

259
00:12:24.675 --> 00:12:25.715
<00:12:24.675>with <00:12:24.987>some <00:12:25.299>past <00:12:25.611>state

260
00:12:25.715 --> 00:12:28.115
<00:12:25.715>of <00:12:25.948>the <00:12:26.181>software <00:12:26.415>faster <00:12:26.648>endless <00:12:26.881>error <00:12:27.115>pro.

261
00:12:28.855 --> 00:12:32.675
<00:12:28.855>So <00:12:29.010>she's <00:12:29.166>also <00:12:29.321>learned <00:12:29.477>that <00:12:29.632>there <00:12:29.788>is <00:12:29.943>no <00:12:30.099>such <00:12:30.255>thing

262
00:12:33.515 --> 00:12:35.295
<00:12:33.515>as <00:12:33.715>free <00:12:33.915>lunch <00:12:34.115>when <00:12:34.314>it <00:12:34.515>comes <00:12:34.715>to <00:12:34.915>automation

263
00:12:35.295 --> 00:12:38.935
<00:12:35.295>because <00:12:35.528>she <00:12:35.761>knows <00:12:35.995>that <00:12:36.228>automation <00:12:36.461>requires <00:12:36.694>maintenance

264
00:12:40.495 --> 00:12:43.195
<00:12:40.495>and <00:12:40.635>therefore <00:12:40.775>she <00:12:40.915>treats <00:12:41.055>automation <00:12:41.195>like <00:12:41.335>a <00:12:41.475>cake, <00:12:41.615>I <00:12:41.755>would <00:12:41.895>say

265
00:12:43.555 --> 00:12:47.915
<00:12:43.555>because <00:12:43.710>she <00:12:43.866>knows <00:12:44.021>that <00:12:44.177>the <00:12:44.332>small <00:12:44.488>amount <00:12:44.643>yeah, <00:12:44.799>of <00:12:44.954>cake.

266
00:12:47.975 --> 00:12:50.755
<00:12:47.975>And <00:12:48.255>therefore <00:12:48.535>automation <00:12:48.815>can <00:12:49.095>taste <00:12:49.375>great,

267
00:12:51.015 --> 00:12:53.115
<00:12:51.015>but <00:12:51.248>too <00:12:51.481>much <00:12:51.715>usually <00:12:51.948>makes <00:12:52.181>her <00:12:52.415>sick.

268
00:12:53.775 --> 00:12:56.075
<00:12:53.775>So <00:12:54.055>she <00:12:54.334>doesn't <00:12:54.615>just <00:12:54.895>automate <00:12:55.175>checks,

269
00:12:56.615 --> 00:12:58.355
<00:12:56.615>she <00:12:56.965>automates <00:12:57.315>checks <00:12:57.665>that <00:12:58.015>matter.

270
00:12:58.655 --> 00:12:59.675
<00:12:58.655>She <00:12:59.113>automates <00:12:59.573>checks

271
00:12:59.675 --> 00:13:02.235
<00:12:59.675>that <00:13:00.025>repeatedly <00:13:00.375>reveal <00:13:00.724>useful <00:13:01.074>information

272
00:13:02.325 --> 00:13:03.395
<00:13:02.325>about <00:13:02.806>the <00:13:03.288>software.

273
00:13:03.855 --> 00:13:07.475
<00:13:03.855>So <00:13:03.982>the <00:13:04.109>bottom <00:13:04.236>line <00:13:04.364>here <00:13:04.491>is <00:13:04.618>that <00:13:04.745>Alice <00:13:04.873>acts <00:13:05.000>like <00:13:05.127>a <00:13:05.255>controller

274
00:13:07.975 --> 00:13:10.755
<00:13:07.975>who <00:13:08.175>carefully <00:13:08.375>controls <00:13:08.575>the <00:13:08.775>benefit <00:13:08.975>to <00:13:09.175>cost <00:13:09.375>ratio

275
00:13:11.055 --> 00:13:12.195
<00:13:11.055>of <00:13:11.568>automated <00:13:12.081>checks

276
00:13:12.575 --> 00:13:15.115
<00:13:12.575>to <00:13:12.808>not <00:13:13.041>get <00:13:13.275>drowned <00:13:13.508>in <00:13:13.741>maintenance <00:13:13.975>efforts.

277
00:13:16.295 --> 00:13:17.515
<00:13:16.295>In <00:13:16.661>addition <00:13:17.026>to <00:13:17.393>that,

278
00:13:19.695 --> 00:13:22.835
<00:13:19.695>at <00:13:19.850>least <00:13:20.006>clearly <00:13:20.161>understands <00:13:20.317>that <00:13:20.472>software <00:13:20.628>testing <00:13:20.783>is <00:13:20.939>not <00:13:21.095>a

279
00:13:23.395 --> 00:13:25.435
<00:13:23.395>mechanic <00:13:23.861>one <00:13:24.328>dimensional <00:13:24.795>activity,

280
00:13:25.735 --> 00:13:29.795
<00:13:25.735>but <00:13:25.968>a <00:13:26.201>multidimensional <00:13:26.435>creative <00:13:26.668>and <00:13:26.901>human-centric <00:13:27.135>activity.

281
00:13:30.055 --> 00:13:32.115
<00:13:30.055>And <00:13:30.334>therefore <00:13:30.614>she <00:13:30.895>sees <00:13:31.175>human <00:13:31.454>testers

282
00:13:32.455 --> 00:13:35.555
<00:13:32.455>and <00:13:32.610>their <00:13:32.766>skills <00:13:32.921>at <00:13:33.077>the <00:13:33.232>center <00:13:33.388>of <00:13:33.543>testing <00:13:33.699>not <00:13:33.855>machines.

283
00:13:36.295 --> 00:13:39.395
<00:13:36.295>Now <00:13:36.422>Ellis <00:13:36.549>is <00:13:36.676>also <00:13:36.804>a <00:13:36.931>master <00:13:37.058>in <00:13:37.185>programming, <00:13:37.313>I <00:13:37.440>have <00:13:37.567>to <00:13:37.694>say,

284
00:13:39.575 --> 00:13:40.915
<00:13:39.575>but <00:13:39.816>she <00:13:40.057>doesn't <00:13:40.298>feel <00:13:40.539>the <00:13:40.781>need

285
00:13:40.915 --> 00:13:42.875
<00:13:40.915>to <00:13:41.265>artificially <00:13:41.615>elevate <00:13:41.964>her <00:13:42.314>role

286
00:13:43.255 --> 00:13:45.235
<00:13:43.255>by <00:13:43.535>inventing <00:13:43.814>new <00:13:44.095>names <00:13:44.375>for <00:13:44.655>it.

287
00:13:45.615 --> 00:13:48.115
<00:13:45.615>She <00:13:45.848>knows <00:13:46.081>that <00:13:46.315>saying <00:13:46.548>tester <00:13:46.781>is <00:13:47.015>enough

288
00:13:48.505 --> 00:13:51.315
<00:13:48.505>because <00:13:48.705>she <00:13:48.905>understands <00:13:49.105>that <00:13:49.305>she's <00:13:49.505>actually <00:13:49.705>being <00:13:49.905>paid

289
00:13:51.415 --> 00:13:53.155
<00:13:51.415>to <00:13:51.765>test, <00:13:52.115>not <00:13:52.464>to <00:13:52.814>code.

290
00:13:53.695 --> 00:13:56.635
<00:13:53.695>And <00:13:53.870>for <00:13:54.045>that <00:13:54.220>reason <00:13:54.395>Alice <00:13:54.570>sees <00:13:54.745>tools <00:13:54.920>as <00:13:55.095>something

291
00:13:56.635 --> 00:14:00.715
<00:13:56.635>that <00:13:56.810>can <00:13:56.985>accelerate, <00:13:57.160>that <00:13:57.335>can <00:13:57.510>extend <00:13:57.685>and <00:13:57.860>simplify <00:13:58.035>testing.

292
00:14:00.815 --> 00:14:02.715
<00:14:00.815>So <00:14:01.015>she <00:14:01.215>doesn't <00:14:01.415>consider <00:14:01.615>tools <00:14:01.815>as <00:14:02.015>something <00:14:02.215>that

293
00:14:03.405 --> 00:14:06.555
<00:14:03.405>eliminates <00:14:03.605>the <00:14:03.805>need <00:14:04.005>for <00:14:04.204>human <00:14:04.405>thinking <00:14:04.605>in <00:14:04.805>testing.

294
00:14:07.535 --> 00:14:09.435
<00:14:07.535>She <00:14:07.709>knows <00:14:07.885>that <00:14:08.060>it <00:14:08.235>is <00:14:08.410>silly <00:14:08.584>to <00:14:08.760>blame <00:14:08.935>gravity

295
00:14:09.575 --> 00:14:11.395
<00:14:09.575>for <00:14:09.808>falling <00:14:10.041>flat <00:14:10.275>on <00:14:10.508>the <00:14:10.741>face <00:14:10.975>and

296
00:14:11.395 --> 00:14:12.955
<00:14:11.395>therefore <00:14:11.745>she <00:14:12.095>doesn't <00:14:12.444>blame <00:14:12.795>tools

297
00:14:13.255 --> 00:14:15.275
<00:14:13.255>for <00:14:13.535>the <00:14:13.814>poor <00:14:14.095>tests <00:14:14.375>she <00:14:14.655>performs.

298
00:14:16.215 --> 00:14:18.995
<00:14:16.215>She <00:14:16.415>knows <00:14:16.615>that <00:14:16.815>only <00:14:17.015>poor <00:14:17.215>testers <00:14:17.415>blame <00:14:17.615>tools

299
00:14:19.015 --> 00:14:20.755
<00:14:19.015>for <00:14:19.295>the <00:14:19.574>poor <00:14:19.855>test <00:14:20.135>they <00:14:20.415>perform.

300
00:14:21.495 --> 00:14:25.195
<00:14:21.495>So <00:14:21.635>Alice <00:14:21.775>acts <00:14:21.915>like <00:14:22.055>a <00:14:22.195>leader <00:14:22.335>who <00:14:22.475>is <00:14:22.615>leading <00:14:22.755>by <00:14:22.895>example,

301
00:14:25.655 --> 00:14:28.995
<00:14:25.655>by <00:14:25.888>holding <00:14:26.121>herself <00:14:26.355>responsible <00:14:26.588>for <00:14:26.821>her <00:14:27.055>actions

302
00:14:29.095 --> 00:14:32.395
<00:14:29.095>and <00:14:29.270>accountable <00:14:29.445>for <00:14:29.620>the <00:14:29.795>results <00:14:29.970>of <00:14:30.145>her <00:14:30.320>testing <00:14:30.495>actions.

303
00:14:33.515 --> 00:14:35.415
<00:14:33.515>Now <00:14:33.715>the <00:14:33.915>list <00:14:34.115>goes <00:14:34.314>on, <00:14:34.515>ladies <00:14:34.715>and <00:14:34.915>gentlemen.

304
00:14:35.465 --> 00:14:38.615
<00:14:35.465>Alice <00:14:35.698>knows <00:14:35.931>that <00:14:36.165>testing <00:14:36.398>is <00:14:36.631>context <00:14:36.865>driven.

305
00:14:39.305 --> 00:14:41.695
<00:14:39.305>She's <00:14:39.505>aware <00:14:39.704>that <00:14:39.905>there <00:14:40.104>aren't <00:14:40.305>any <00:14:40.505>best <00:14:40.704>test

306
00:14:42.495 --> 00:14:45.815
<00:14:42.495>practices <00:14:42.670>out <00:14:42.845>there, <00:14:43.020>only <00:14:43.195>good <00:14:43.370>testing <00:14:43.545>practices <00:14:43.720>in <00:14:43.895>context

307
00:14:46.165 --> 00:14:47.575
<00:14:46.165>because <00:14:46.418>she <00:14:46.672>knows <00:14:46.926>that <00:14:47.180>the <00:14:47.434>value

308
00:14:47.595 --> 00:14:50.415
<00:14:47.595>of <00:14:47.828>any <00:14:48.061>practice <00:14:48.295>depends <00:14:48.528>on <00:14:48.761>its <00:14:48.995>context.

309
00:14:51.425 --> 00:14:53.205
<00:14:51.425>For <00:14:51.704>example, <00:14:51.984>the <00:14:52.265>project, <00:14:52.545>the <00:14:52.824>product,

310
00:14:53.425 --> 00:14:54.805
<00:14:53.425>the <00:14:53.735>people <00:14:54.045>and <00:14:54.356>the <00:14:54.666>environment.

311
00:14:56.335 --> 00:14:58.995
<00:14:56.335>She <00:14:56.535>knows <00:14:56.735>that <00:14:56.935>she <00:14:57.135>wouldn't <00:14:57.335>pass <00:14:57.535>an <00:14:57.735>autopilot

312
00:14:59.495 --> 00:15:01.755
<00:14:59.495>of <00:14:59.650>an <00:14:59.806>airline <00:14:59.961>in <00:15:00.117>the <00:15:00.272>same <00:15:00.428>way <00:15:00.583>as <00:15:00.739>a <00:15:00.895>website.

313
00:15:01.895 --> 00:15:04.275
<00:15:01.895>And <00:15:02.069>of <00:15:02.245>course <00:15:02.420>sometimes <00:15:02.595>instead <00:15:02.770>of, <00:15:02.944>you <00:15:03.120>know, <00:15:03.295>finding

314
00:15:05.375 --> 00:15:09.275
<00:15:05.375>and <00:15:05.515>learning <00:15:05.655>how <00:15:05.795>to <00:15:05.935>use <00:15:06.075>a <00:15:06.215>shizzle, <00:15:06.355>she <00:15:06.495>all <00:15:06.635>sometimes <00:15:06.775>turns

315
00:15:09.275 --> 00:15:10.795
<00:15:09.275>to <00:15:09.548>the <00:15:09.822>hammer <00:15:10.095>she's <00:15:10.369>already <00:15:10.642>holding.

316
00:15:11.055 --> 00:15:14.315
<00:15:11.055>But <00:15:11.229>in <00:15:11.405>hindsight, <00:15:11.579>she <00:15:11.755>becomes <00:15:11.930>aware <00:15:12.104>of <00:15:12.280>that <00:15:12.454>and

317
00:15:14.315 --> 00:15:16.835
<00:15:14.315>therefore <00:15:14.548>she <00:15:14.781>equips <00:15:15.015>herself <00:15:15.248>with <00:15:15.481>a <00:15:15.715>variety

318
00:15:16.975 --> 00:15:19.835
<00:15:16.975>of <00:15:17.255>testing <00:15:17.535>techniques <00:15:17.815>such <00:15:18.095>as <00:15:18.375>heuristics.

319
00:15:20.295 --> 00:15:22.915
<00:15:20.295>And <00:15:20.469>she <00:15:20.645>has <00:15:20.819>these <00:15:20.995>techniques <00:15:21.170>ready <00:15:21.344>in <00:15:21.520>different <00:15:21.694>forms,

320
00:15:23.055 --> 00:15:26.395
<00:15:23.055>for <00:15:23.210>example, <00:15:23.366>in <00:15:23.521>the <00:15:23.677>form <00:15:23.832>of <00:15:23.988>checklists, <00:15:24.143>in <00:15:24.299>the <00:15:24.454>form

321
00:15:26.455 --> 00:15:28.995
<00:15:26.455>of <00:15:26.630>cheat <00:15:26.805>sheets, <00:15:26.980>in <00:15:27.155>the <00:15:27.330>form <00:15:27.505>of <00:15:27.680>mind <00:15:27.855>maps.

322
00:15:29.855 --> 00:15:33.595
<00:15:29.855>Now <00:15:29.995>she <00:15:30.135>does <00:15:30.275>also <00:15:30.415>to <00:15:30.555>enable <00:15:30.695>herself <00:15:30.835>to <00:15:30.975>design, <00:15:31.115>to <00:15:31.255>perform,

323
00:15:33.735 --> 00:15:35.475
<00:15:33.735>to <00:15:34.085>prioritize <00:15:34.435>and <00:15:34.785>to <00:15:35.135>analyze

324
00:15:36.255 --> 00:15:38.355
<00:15:36.255>her <00:15:36.605>tests <00:15:36.955>in <00:15:37.305>different <00:15:37.655>contexts.

325
00:15:39.135 --> 00:15:43.355
<00:15:39.135>So <00:15:39.262>in <00:15:39.389>short, <00:15:39.516>Alice <00:15:39.644>knows <00:15:39.771>that <00:15:39.898>testing <00:15:40.025>is <00:15:40.153>not <00:15:40.280>just <00:15:40.407>a <00:15:40.535>talent,

326
00:15:43.815 --> 00:15:46.155
<00:15:43.815>but <00:15:43.970>rather <00:15:44.126>a <00:15:44.281>set <00:15:44.437>of <00:15:44.592>skills <00:15:44.748>that <00:15:44.903>can <00:15:45.059>be <00:15:45.215>learned.

327
00:15:46.815 --> 00:15:48.835
<00:15:46.815>She <00:15:47.048>also <00:15:47.281>relates <00:15:47.515>testing <00:15:47.748>to <00:15:47.981>other <00:15:48.215>disciplines,

328
00:15:49.065 --> 00:15:51.995
<00:15:49.065>disciplines <00:15:49.345>such <00:15:49.625>as <00:15:49.905>physics, <00:15:50.185>mathematics, <00:15:50.465>history,

329
00:15:52.405 --> 00:15:54.475
<00:15:52.405>philosophy, <00:15:52.685>and <00:15:52.964>social <00:15:53.245>science <00:15:53.525>for <00:15:53.805>example.

330
00:15:54.975 --> 00:15:56.515
<00:15:54.975>And <00:15:55.252>she <00:15:55.529>does <00:15:55.806>that <00:15:56.083>to <00:15:56.361>broaden

331
00:15:56.695 --> 00:16:00.195
<00:15:56.695>and <00:15:56.870>strengthen <00:15:57.045>her <00:15:57.220>skills, <00:15:57.395>her <00:15:57.570>skills <00:15:57.745>in <00:15:57.920>problem <00:15:58.095>solving,

332
00:16:00.295 --> 00:16:02.075
<00:16:00.295>her <00:16:00.645>skills <00:16:00.995>in <00:16:01.344>modeling <00:16:01.694>speaking,

333
00:16:02.415 --> 00:16:04.755
<00:16:02.415>and <00:16:02.765>her <00:16:03.115>skills <00:16:03.464>in <00:16:03.814>critically

334
00:16:04.855 --> 00:16:06.595
<00:16:04.855>and <00:16:05.205>lateral <00:16:05.555>thinking, <00:16:05.905>for <00:16:06.255>example.

335
00:16:07.215 --> 00:16:10.035
<00:16:07.215>So <00:16:07.390>Alice <00:16:07.565>acts <00:16:07.740>like <00:16:07.915>a <00:16:08.090>chameleon, <00:16:08.265>so <00:16:08.440>to <00:16:08.615>say,

336
00:16:10.225 --> 00:16:14.115
<00:16:10.225>that <00:16:10.400>has <00:16:10.575>the <00:16:10.750>ability <00:16:10.925>to <00:16:11.100>adapt <00:16:11.275>to <00:16:11.450>new <00:16:11.625>unknown

337
00:16:14.255 --> 00:16:16.675
<00:16:14.255>or <00:16:14.721>unfamiliar <00:16:15.188>environments <00:16:15.655>fast.

338
00:16:18.175 --> 00:16:19.405
<00:16:18.175>Alice <00:16:18.451>knows <00:16:18.728>that <00:16:19.005>the <00:16:19.281>set

339
00:16:19.405 --> 00:16:22.045
<00:16:19.405>of <00:16:19.560>models <00:16:19.716>she's <00:16:19.871>using <00:16:20.027>in <00:16:20.182>testing <00:16:20.338>are <00:16:20.493>just <00:16:20.649>a <00:16:20.805>set

340
00:16:22.265 --> 00:16:24.685
<00:16:22.265>of <00:16:22.615>ideas <00:16:22.965>about <00:16:23.314>the <00:16:23.665>software.

341
00:16:25.465 --> 00:16:28.085
<00:16:25.465>So <00:16:25.640>she's <00:16:25.815>aware <00:16:25.990>that <00:16:26.165>these <00:16:26.340>models <00:16:26.515>aren't <00:16:26.690>the <00:16:26.865>software.

342
00:16:28.085 --> 00:16:31.405
<00:16:28.085>These <00:16:28.285>models <00:16:28.485>are <00:16:28.685>abstract <00:16:28.885>representations <00:16:29.085>of <00:16:29.285>the <00:16:29.485>software

343
00:16:31.585 --> 00:16:34.365
<00:16:31.585>and <00:16:31.760>she's <00:16:31.935>using <00:16:32.110>these <00:16:32.285>models <00:16:32.460>to <00:16:32.635>understand <00:16:32.810>certain <00:16:32.985>aspects

344
00:16:34.365 --> 00:16:35.845
<00:16:34.365>of <00:16:35.031>complex <00:16:35.697>software.

345
00:16:36.105 --> 00:16:38.685
<00:16:36.105>In <00:16:36.338>simple <00:16:36.571>terms, <00:16:36.805>she's <00:16:37.038>using <00:16:37.271>these <00:16:37.505>models

346
00:16:38.745 --> 00:16:42.365
<00:16:38.745>to <00:16:38.920>form <00:16:39.095>conjectures, <00:16:39.270>to <00:16:39.445>direct <00:16:39.620>her <00:16:39.795>testing, <00:16:39.970>to <00:16:40.145>make

347
00:16:42.885 --> 00:16:45.205
<00:16:42.885>decisions <00:16:43.165>and <00:16:43.444>to <00:16:43.725>communicate <00:16:44.005>her <00:16:44.285>conclusions.

348
00:16:46.025 --> 00:16:49.725
<00:16:46.025>Now <00:16:46.225>she's <00:16:46.425>aware <00:16:46.625>that <00:16:46.824>these <00:16:47.025>models <00:16:47.225>are <00:16:47.425>simplifications

349
00:16:49.725 --> 00:16:51.445
<00:16:49.725>of <00:16:49.958>the <00:16:50.191>software <00:16:50.425>and <00:16:50.658>she <00:16:50.891>understands <00:16:51.125>that

350
00:16:52.015 --> 00:16:56.165
<00:16:52.015>simplicity <00:16:52.248>doesn't <00:16:52.481>precede <00:16:52.715>complexity <00:16:52.948>but <00:16:53.181>follows <00:16:53.415>it.

351
00:16:56.745 --> 00:16:59.805
<00:16:56.745>So <00:16:56.920>she <00:16:57.095>reflects <00:16:57.270>on <00:16:57.445>her <00:16:57.620>conjectures, <00:16:57.795>on <00:16:57.970>her <00:16:58.145>decisions

352
00:16:59.805 --> 00:17:03.005
<00:16:59.805>and <00:17:00.084>conclusions <00:17:00.364>by <00:17:00.645>considering <00:17:00.925>alternative <00:17:01.204>models

353
00:17:03.195 --> 00:17:06.005
<00:17:03.195>because <00:17:03.370>she <00:17:03.545>knows <00:17:03.720>that <00:17:03.895>these <00:17:04.070>models <00:17:04.245>are <00:17:04.420>probably, <00:17:04.595>yeah,

354
00:17:06.305 --> 00:17:08.885
<00:17:06.305>flawed <00:17:06.655>representations <00:17:07.005>of <00:17:07.355>the <00:17:07.705>software

355
00:17:09.075 --> 00:17:12.525
<00:17:09.075>that <00:17:09.215>are <00:17:09.355>based <00:17:09.495>on <00:17:09.635>her <00:17:09.775>flawed <00:17:09.915>mental <00:17:10.055>model <00:17:10.194>of <00:17:10.335>the <00:17:10.475>software.

356
00:17:13.145 --> 00:17:17.685
<00:17:13.145>So <00:17:13.272>she <00:17:13.399>acts <00:17:13.526>like, <00:17:13.654>uh, <00:17:13.781>like, <00:17:13.908>like <00:17:14.035>her <00:17:14.163>own <00:17:14.290>best <00:17:14.417>critique, <00:17:14.545>so

357
00:17:17.685 --> 00:17:20.565
<00:17:17.685>to <00:17:17.859>say, <00:17:18.034>who <00:17:18.210>questions <00:17:18.385>her <00:17:18.560>own <00:17:18.734>models <00:17:18.909>in <00:17:19.085>testing

358
00:17:21.065 --> 00:17:24.045
<00:17:21.065>to <00:17:21.220>not <00:17:21.376>get <00:17:21.531>fooled <00:17:21.687>by <00:17:21.842>them <00:17:21.998>over <00:17:22.153>and <00:17:22.309>over <00:17:22.465>again.

359
00:17:25.125 --> 00:17:27.505
<00:17:25.125>Now <00:17:25.358>Alice <00:17:25.591>also <00:17:25.825>acts <00:17:26.058>like <00:17:26.291>an <00:17:26.525>astronomer.

360
00:17:27.645 --> 00:17:29.065
<00:17:27.645>And <00:17:27.858>the <00:17:28.070>reason <00:17:28.284>for <00:17:28.497>this <00:17:28.710>is <00:17:28.923>simple.

361
00:17:29.475 --> 00:17:30.865
<00:17:29.475>Alice <00:17:29.891>knows <00:17:30.309>that <00:17:30.726>testing

362
00:17:31.445 --> 00:17:33.825
<00:17:31.445>the <00:17:31.645>software <00:17:31.845>is <00:17:32.044>more <00:17:32.245>than <00:17:32.445>just <00:17:32.645>checking <00:17:32.845>the

363
00:17:34.105 --> 00:17:35.505
<00:17:34.105>specification <00:17:34.525>of <00:17:34.945>the <00:17:35.365>software.

364
00:17:36.125 --> 00:17:38.865
<00:17:36.125>So <00:17:36.325>she <00:17:36.525>doesn't <00:17:36.725>just <00:17:36.925>compare <00:17:37.125>the <00:17:37.325>actual <00:17:37.525>software

365
00:17:38.865 --> 00:17:39.905
<00:17:38.865>to <00:17:39.333>it <00:17:39.801>specification.

366
00:17:40.245 --> 00:17:42.745
<00:17:40.245>She <00:17:40.445>also <00:17:40.645>compares <00:17:40.844>the <00:17:41.044>idea <00:17:41.245>of <00:17:41.445>the <00:17:41.645>software

367
00:17:42.925 --> 00:17:43.985
<00:17:42.925>to <00:17:43.243>the <00:17:43.561>actual <00:17:43.879>software.

368
00:17:44.285 --> 00:17:48.355
<00:17:44.285>And <00:17:44.751>now <00:17:45.218>she <00:17:45.685>know

369
00:17:49.225 --> 00:17:52.805
<00:17:49.225>of <00:17:49.399>the <00:17:49.574>software <00:17:49.750>is <00:17:49.925>to <00:17:50.100>solve <00:17:50.274>someone's <00:17:50.449>problem <00:17:50.625>and

370
00:17:52.805 --> 00:17:55.405
<00:17:52.805>therefore <00:17:52.980>she <00:17:53.155>evaluates <00:17:53.330>the <00:17:53.505>purpose <00:17:53.680>of <00:17:53.855>the <00:17:54.030>software <00:17:54.205>too,

371
00:17:55.405 --> 00:17:57.565
<00:17:55.405>because <00:17:55.638>she <00:17:55.871>knows <00:17:56.105>that <00:17:56.338>testing <00:17:56.571>the <00:17:56.805>purpose

372
00:17:57.585 --> 00:18:00.085
<00:17:57.585>of <00:17:57.760>the <00:17:57.935>software <00:17:58.110>is <00:17:58.285>one <00:17:58.460>purpose <00:17:58.635>of <00:17:58.810>software <00:17:58.985>testing.

373
00:18:01.145 --> 00:18:05.085
<00:18:01.145>Now <00:18:01.300>she <00:18:01.456>does <00:18:01.611>so <00:18:01.767>because <00:18:01.922>she <00:18:02.078>understands <00:18:02.233>that <00:18:02.389>the <00:18:02.545>conformance

374
00:18:05.435 --> 00:18:08.325
<00:18:05.435>with <00:18:05.784>the <00:18:06.135>specification <00:18:06.485>has <00:18:06.835>little

375
00:18:08.425 --> 00:18:11.845
<00:18:08.425>to <00:18:08.580>no <00:18:08.736>value <00:18:08.891>if <00:18:09.047>the <00:18:09.202>software <00:18:09.358>solves <00:18:09.513>the <00:18:09.669>wrong <00:18:09.825>problem.

376
00:18:13.125 --> 00:18:15.705
<00:18:13.125>So <00:18:13.358>therefore, <00:18:13.591>Ellis <00:18:13.825>acts <00:18:14.058>like <00:18:14.291>an <00:18:14.525>astronomer

377
00:18:16.405 --> 00:18:19.865
<00:18:16.405>who <00:18:16.580>not <00:18:16.754>only <00:18:16.930>uses <00:18:17.105>a <00:18:17.280>microscope <00:18:17.455>to <00:18:17.629>evaluate, <00:18:17.805>uh,

378
00:18:19.865 --> 00:18:21.305
<00:18:19.865>the <00:18:20.189>software <00:18:20.513>from <00:18:20.837>a <00:18:21.161>yeah,

379
00:18:21.305 --> 00:18:23.745
<00:18:21.305>rather <00:18:21.655>small <00:18:22.005>scale <00:18:22.355>technical <00:18:22.705>perspective,

380
00:18:24.285 --> 00:18:27.545
<00:18:24.285>but <00:18:24.440>she <00:18:24.596>also <00:18:24.751>uses <00:18:24.907>a <00:18:25.062>telescope <00:18:25.218>to <00:18:25.373>judge <00:18:25.529>the <00:18:25.685>value

381
00:18:27.545 --> 00:18:31.465
<00:18:27.545>of <00:18:27.700>the <00:18:27.856>software <00:18:28.011>from <00:18:28.167>a <00:18:28.322>a <00:18:28.478>large <00:18:28.633>scale <00:18:28.789>conceptual <00:18:28.945>perspective.

382
00:18:32.475 --> 00:18:34.305
<00:18:32.475>Alice <00:18:32.824>also <00:18:33.175>understands <00:18:33.524>that <00:18:33.875>those

383
00:18:34.325 --> 00:18:37.265
<00:18:34.325>who <00:18:34.480>cannot <00:18:34.636>learn <00:18:34.791>from <00:18:34.947>history <00:18:35.102>are <00:18:35.258>doomed <00:18:35.413>to <00:18:35.569>repeat <00:18:35.725>it.

384
00:18:37.765 --> 00:18:40.465
<00:18:37.765>And <00:18:37.905>therefore <00:18:38.045>she <00:18:38.185>looks <00:18:38.325>back <00:18:38.465>to <00:18:38.605>the <00:18:38.745>mistakes <00:18:38.885>she <00:18:39.025>has <00:18:39.165>made.

385
00:18:40.845 --> 00:18:43.825
<00:18:40.845>She <00:18:41.125>identifies, <00:18:41.405>yeah, <00:18:41.685>patterns <00:18:41.965>of <00:18:42.245>failures

386
00:18:44.045 --> 00:18:46.385
<00:18:44.045>and <00:18:44.245>she <00:18:44.445>extracts <00:18:44.645>the <00:18:44.845>lesson <00:18:45.045>she <00:18:45.245>has <00:18:45.445>learned.

387
00:18:46.845 --> 00:18:50.345
<00:18:46.845>Now, <00:18:46.985>in <00:18:47.125>doing <00:18:47.265>so, <00:18:47.405>you <00:18:47.545>know, <00:18:47.685>she <00:18:47.825>neither <00:18:47.965>denies <00:18:48.105>her <00:18:48.245>mistakes,

388
00:18:50.405 --> 00:18:52.105
<00:18:50.405>nor <00:18:50.685>she, <00:18:50.965>she <00:18:51.245>doesn't <00:18:51.524>hide <00:18:51.805>them.

389
00:18:52.445 --> 00:18:54.065
<00:18:52.445>She, <00:18:52.794>she <00:18:53.145>documents <00:18:53.495>these <00:18:53.845>mistakes

390
00:18:54.085 --> 00:18:56.985
<00:18:54.085>and <00:18:54.260>then <00:18:54.435>she <00:18:54.610>shares <00:18:54.785>these <00:18:54.960>mistakes <00:18:55.135>with <00:18:55.310>other <00:18:55.485>people,

391
00:18:57.805 --> 00:19:00.105
<00:18:57.805>people <00:18:58.038>such <00:18:58.271>as <00:18:58.505>developers <00:18:58.738>and <00:18:58.971>product <00:18:59.205>owners

392
00:19:00.105 --> 00:19:01.905
<00:19:00.105>and <00:19:00.385>her <00:19:00.665>fellow <00:19:00.945>tests <00:19:01.225>for <00:19:01.505>example.

393
00:19:03.225 --> 00:19:05.445
<00:19:03.225>And <00:19:03.458>she <00:19:03.691>does <00:19:03.925>that <00:19:04.158>by <00:19:04.391>contacting <00:19:04.625>regular

394
00:19:05.445 --> 00:19:06.805
<00:19:05.445>retrospective <00:19:06.669>sessions.

395
00:19:07.265 --> 00:19:10.365
<00:19:07.265>And <00:19:07.405>in <00:19:07.545>doing <00:19:07.685>so, <00:19:07.825>she <00:19:07.965>not <00:19:08.105>only <00:19:08.245>learns <00:19:08.385>from <00:19:08.525>the <00:19:08.665>mistakes

396
00:19:10.365 --> 00:19:13.205
<00:19:10.365>of <00:19:10.565>other <00:19:10.765>people, <00:19:10.965>she <00:19:11.165>also <00:19:11.365>enables <00:19:11.565>other <00:19:11.765>people

397
00:19:13.345 --> 00:19:16.005
<00:19:13.345>to <00:19:13.545>benefit <00:19:13.745>from <00:19:13.945>the <00:19:14.145>lessons <00:19:14.345>she <00:19:14.545>has <00:19:14.745>learned.

398
00:19:16.505 --> 00:19:18.205
<00:19:16.505>So <00:19:16.785>she <00:19:17.065>reflects <00:19:17.345>on <00:19:17.625>her <00:19:17.905>mistakes

399
00:19:18.205 --> 00:19:20.885
<00:19:18.205>because <00:19:18.360>she <00:19:18.516>takes <00:19:18.671>on <00:19:18.827>when <00:19:18.982>she <00:19:19.138>takes <00:19:19.293>on <00:19:19.449>new <00:19:19.605>challenges,

400
00:19:21.445 --> 00:19:24.805
<00:19:21.445>challenges <00:19:21.620>such <00:19:21.794>as <00:19:21.970>new <00:19:22.145>projects, <00:19:22.320>for <00:19:22.495>example, <00:19:22.669>new <00:19:22.845>releases

401
00:19:24.825 --> 00:19:27.485
<00:19:24.825>or <00:19:25.105>new <00:19:25.385>sprints, <00:19:25.665>to <00:19:25.945>prepare <00:19:26.225>herself

402
00:19:27.585 --> 00:19:31.165
<00:19:27.585>to <00:19:27.760>not <00:19:27.935>repeat <00:19:28.110>these <00:19:28.285>mistakes <00:19:28.460>over <00:19:28.635>and <00:19:28.810>over <00:19:28.985>again

403
00:19:31.555 --> 00:19:35.165
<00:19:31.555>because <00:19:31.710>she <00:19:31.866>knows <00:19:32.021>that <00:19:32.177>prevention <00:19:32.332>is <00:19:32.488>just <00:19:32.643>better <00:19:32.799>than <00:19:32.955>cure.

404
00:19:36.105 --> 00:19:37.845
<00:19:36.105>So <00:19:36.385>the <00:19:36.665>bottom <00:19:36.945>line <00:19:37.225>here <00:19:37.505>is

405
00:19:37.845 --> 00:19:40.165
<00:19:37.845>that <00:19:38.125>Alice <00:19:38.405>acts <00:19:38.685>like <00:19:38.965>any <00:19:39.245>historian

406
00:19:40.985 --> 00:19:42.485
<00:19:40.985>who <00:19:41.322>builds <00:19:41.659>a <00:19:41.997>learning <00:19:42.334>culture

407
00:19:43.065 --> 00:19:45.845
<00:19:43.065>by <00:19:43.345>analyzing <00:19:43.625>her <00:19:43.905>past <00:19:44.185>testing <00:19:44.465>activities

408
00:19:46.265 --> 00:19:49.965
<00:19:46.265>to <00:19:46.498>constantly <00:19:46.731>improve <00:19:46.965>her <00:19:47.198>future <00:19:47.431>testing <00:19:47.665>activities.

409
00:19:51.185 --> 00:19:54.165
<00:19:51.185>Now, <00:19:51.385>on <00:19:51.585>top <00:19:51.784>of <00:19:51.985>that, <00:19:52.185>Alice <00:19:52.385>also <00:19:52.585>understands

410
00:19:54.355 --> 00:19:57.845
<00:19:54.355>that <00:19:54.510>having <00:19:54.666>a <00:19:54.821>testing <00:19:54.977>specialist <00:19:55.132>on <00:19:55.288>the <00:19:55.443>team <00:19:55.599>doesn't <00:19:55.755>mean

411
00:19:58.225 --> 00:20:01.685
<00:19:58.225>to <00:19:58.504>restrict <00:19:58.784>the <00:19:59.064>responsibility <00:19:59.344>of <00:19:59.625>testing

412
00:20:01.945 --> 00:20:03.045
<00:20:01.945>to <00:20:02.275>that <00:20:02.605>single <00:20:02.935>person.

413
00:20:03.705 --> 00:20:06.725
<00:20:03.705>So <00:20:03.879>she's <00:20:04.054>aware <00:20:04.230>that <00:20:04.405>excellent <00:20:04.580>testing <00:20:04.754>is <00:20:04.929>never <00:20:05.105>done

414
00:20:06.745 --> 00:20:08.885
<00:20:06.745>by <00:20:07.024>one <00:20:07.304>single <00:20:07.584>person <00:20:07.864>in <00:20:08.145>isolation.

415
00:20:09.145 --> 00:20:11.885
<00:20:09.145>She <00:20:09.378>understands <00:20:09.611>that <00:20:09.845>excellent <00:20:10.078>testing <00:20:10.311>requires <00:20:10.545>the

416
00:20:11.885 --> 00:20:14.205
<00:20:11.885>collective <00:20:12.085>brain <00:20:12.285>power <00:20:12.485>of <00:20:12.685>a <00:20:12.885>group <00:20:13.085>of <00:20:13.285>people

417
00:20:14.595 --> 00:20:18.325
<00:20:14.595>because <00:20:14.770>she <00:20:14.945>understands <00:20:15.120>that <00:20:15.295>great <00:20:15.470>testing <00:20:15.645>teams <00:20:15.820>are <00:20:15.995>teams

418
00:20:18.425 --> 00:20:20.605
<00:20:18.425>of <00:20:18.891>diverse <00:20:19.358>talents <00:20:19.825>cooperating.

419
00:20:21.265 --> 00:20:25.365
<00:20:21.265>In <00:20:21.440>short, <00:20:21.615>she <00:20:21.790>knows <00:20:21.965>that <00:20:22.140>diversity <00:20:22.315>makes <00:20:22.490>testing <00:20:22.665>powerful.

420
00:20:26.025 --> 00:20:27.645
<00:20:26.025>So <00:20:26.305>she <00:20:26.585>knows <00:20:26.865>that <00:20:27.145>the <00:20:27.425>idea

421
00:20:27.745 --> 00:20:31.125
<00:20:27.745>of <00:20:27.900>one <00:20:28.056>person <00:20:28.211>having <00:20:28.367>all <00:20:28.522>the <00:20:28.678>answers <00:20:28.833>is <00:20:28.989>deeply <00:20:29.145>flawed.

422
00:20:31.705 --> 00:20:33.685
<00:20:31.705>So <00:20:31.938>she <00:20:32.171>understands <00:20:32.405>that <00:20:32.638>knowledge <00:20:32.871>comes <00:20:33.105>in

423
00:20:33.685 --> 00:20:34.765
<00:20:33.685>many <00:20:34.171>different <00:20:34.657>flavors.

424
00:20:34.945 --> 00:20:38.405
<00:20:34.945>And <00:20:35.085>so <00:20:35.225>she <00:20:35.365>looks <00:20:35.504>for, <00:20:35.645>you <00:20:35.784>know, <00:20:35.925>sources <00:20:36.064>of <00:20:36.205>potential <00:20:36.345>support

425
00:20:38.745 --> 00:20:40.365
<00:20:38.745>and <00:20:39.024>she <00:20:39.304>does <00:20:39.584>that <00:20:39.864>through <00:20:40.145>various

426
00:20:40.825 --> 00:20:44.165
<00:20:40.825>yet <00:20:41.025>different <00:20:41.225>collaborative <00:20:41.425>activities <00:20:41.625>which <00:20:41.825>we <00:20:42.025>call <00:20:42.225>payer

427
00:20:44.165 --> 00:20:46.045
<00:20:44.165>testing <00:20:44.445>or <00:20:44.725>mob <00:20:45.004>testing <00:20:45.284>for <00:20:45.565>example.

428
00:20:46.865 --> 00:20:51.045
<00:20:46.865>So <00:20:47.020>she <00:20:47.176>acts <00:20:47.331>like <00:20:47.487>a <00:20:47.642>guide <00:20:47.798>who <00:20:47.953>actively <00:20:48.109>involves <00:20:48.265>guides

429
00:20:51.065 --> 00:20:53.405
<00:20:51.065>and <00:20:51.345>advises <00:20:51.625>other <00:20:51.905>people <00:20:52.185>in <00:20:52.465>testing

430
00:20:54.145 --> 00:20:55.685
<00:20:54.145>by <00:20:54.607>stimulating <00:20:55.069>these <00:20:55.531>people

431
00:20:55.785 --> 00:20:59.805
<00:20:55.785>to <00:20:55.985>continuously <00:20:56.185>think <00:20:56.385>critically <00:20:56.585>about <00:20:56.785>the <00:20:56.985>software <00:20:57.185>they

432
00:21:00.085 --> 00:21:01.805
<00:21:00.085>envision <00:21:00.551>design <00:21:01.018>and <00:21:01.485>build.

433
00:21:03.065 --> 00:21:05.495
<00:21:03.065>Alice <00:21:03.298>also <00:21:03.531>knows <00:21:03.765>that <00:21:03.998>the <00:21:04.231>time <00:21:04.465>needed

434
00:21:05.515 --> 00:21:07.575
<00:21:05.515>for <00:21:05.865>testing <00:21:06.215>is <00:21:06.565>always <00:21:06.915>infinitely

435
00:21:07.595 --> 00:21:09.255
<00:21:07.595>larger <00:21:07.945>than <00:21:08.295>the <00:21:08.645>time <00:21:08.995>available.

436
00:21:09.875 --> 00:21:12.295
<00:21:09.875>So <00:21:10.155>she <00:21:10.435>prioritizes <00:21:10.715>her <00:21:10.995>testing <00:21:11.275>activities

437
00:21:12.295 --> 00:21:15.615
<00:21:12.295>because <00:21:12.495>she <00:21:12.695>knows <00:21:12.895>that <00:21:13.095>testing <00:21:13.295>has <00:21:13.495>to <00:21:13.695>stop

438
00:21:16.495 --> 00:21:18.135
<00:21:16.495>although <00:21:16.844>it <00:21:17.195>actually <00:21:17.544>never <00:21:17.895>ends.

439
00:21:18.715 --> 00:21:19.975
<00:21:18.715>She <00:21:18.941>is <00:21:19.168>aware <00:21:19.395>of <00:21:19.622>the <00:21:19.849>fact

440
00:21:19.975 --> 00:21:23.535
<00:21:19.975>that <00:21:20.130>she <00:21:20.286>cannot <00:21:20.441>prove <00:21:20.597>the <00:21:20.752>software <00:21:20.908>right, <00:21:21.063>but <00:21:21.219>only <00:21:21.375>wrong.

441
00:21:23.795 --> 00:21:27.615
<00:21:23.795>She <00:21:23.950>knows <00:21:24.106>that <00:21:24.261>she <00:21:24.417>has <00:21:24.572>to <00:21:24.728>think <00:21:24.883>of <00:21:25.039>falsification <00:21:25.195>instead

442
00:21:27.615 --> 00:21:28.855
<00:21:27.615>of <00:21:28.731>verification,

443
00:21:29.315 --> 00:21:30.375
<00:21:29.315>and <00:21:29.553>she <00:21:29.792>accepts <00:21:30.030>the <00:21:30.269>fact

444
00:21:30.375 --> 00:21:32.015
<00:21:30.375>that <00:21:30.655>she <00:21:30.935>cannot <00:21:31.215>answer <00:21:31.495>the <00:21:31.775>question.

445
00:21:32.395 --> 00:21:33.815
<00:21:32.395>Did <00:21:32.714>we <00:21:33.034>find <00:21:33.353>all <00:21:33.673>bugs?

446
00:21:34.315 --> 00:21:37.175
<00:21:34.315>She <00:21:34.515>understands <00:21:34.715>that <00:21:34.915>she <00:21:35.115>cannot <00:21:35.315>quantify <00:21:35.515>the <00:21:35.715>number

447
00:21:37.435 --> 00:21:38.535
<00:21:37.435>of <00:21:37.930>absent <00:21:38.425>bugs

448
00:21:38.565 --> 00:21:42.135
<00:21:38.565>because <00:21:38.720>she <00:21:38.876>knows <00:21:39.031>that <00:21:39.187>testing <00:21:39.342>can <00:21:39.498>only <00:21:39.653>prove <00:21:39.809>the <00:21:39.965>presence

449
00:21:42.135 --> 00:21:44.735
<00:21:42.135>of <00:21:42.415>bugs <00:21:42.695>but <00:21:42.975>not <00:21:43.254>their <00:21:43.535>absence.

450
00:21:46.465 --> 00:21:50.845
<00:21:46.465>So <00:21:46.639>Alice <00:21:46.814>acts <00:21:46.990>like <00:21:47.165>a <00:21:47.340>realist <00:21:47.514>who <00:21:47.689>clearly <00:21:47.865>understands

451
00:21:51.235 --> 00:21:55.325
<00:21:51.235>that <00:21:51.435>software <00:21:51.635>testing <00:21:51.834>at <00:21:52.034>its <00:21:52.235>core <00:21:52.435>is <00:21:52.635>undecidable

452
00:21:55.325 --> 00:21:56.965
<00:21:55.325>because <00:21:55.605>she <00:21:55.885>understands <00:21:56.165>that <00:21:56.445>the <00:21:56.725>problem

453
00:21:56.985 --> 00:22:00.725
<00:21:56.985>of <00:21:57.140>finding <00:21:57.296>all <00:21:57.451>the <00:21:57.607>bug <00:21:57.762>in <00:21:57.918>the <00:21:58.073>software <00:21:58.229>is <00:21:58.385>undecidable.

454
00:22:01.705 --> 00:22:04.365
<00:22:01.705>In <00:22:01.860>addition <00:22:02.016>to <00:22:02.171>that, <00:22:02.327>Alice <00:22:02.482>knows <00:22:02.638>that <00:22:02.793>testing <00:22:02.949>is <00:22:03.105>learning.

455
00:22:04.985 --> 00:22:08.765
<00:22:04.985>So <00:22:05.140>whenever <00:22:05.296>she <00:22:05.451>runs <00:22:05.607>out <00:22:05.762>of <00:22:05.918>past <00:22:06.073>ideas, <00:22:06.229>she <00:22:06.385>suspects

456
00:22:08.765 --> 00:22:11.405
<00:22:08.765>that <00:22:08.940>she <00:22:09.115>probably <00:22:09.290>hasn't <00:22:09.465>learned <00:22:09.640>enough <00:22:09.815>about <00:22:09.990>the <00:22:10.165>software

457
00:22:12.575 --> 00:22:14.035
<00:22:12.575>and <00:22:12.903>therefore <00:22:13.232>she <00:22:13.560>keeps <00:22:13.889>exploring.

458
00:22:14.055 --> 00:22:15.195
<00:22:14.055>She <00:22:14.568>keeps <00:22:15.081>investigating

459
00:22:15.215 --> 00:22:18.595
<00:22:15.215>and <00:22:15.448>experimenting <00:22:15.681>to <00:22:15.915>constantly <00:22:16.148>evaluate <00:22:16.381>this <00:22:16.615>conjecture

460
00:22:19.095 --> 00:22:21.155
<00:22:19.095>and <00:22:19.295>she <00:22:19.495>never <00:22:19.695>stops <00:22:19.895>learning <00:22:20.095>about <00:22:20.295>the <00:22:20.495>software

461
00:22:21.415 --> 00:22:24.355
<00:22:21.415>and <00:22:21.590>the <00:22:21.764>domain <00:22:21.940>in <00:22:22.115>which <00:22:22.290>the <00:22:22.465>software <00:22:22.639>is <00:22:22.815>situated.

462
00:22:25.015 --> 00:22:27.995
<00:22:25.015>She <00:22:25.248>knows <00:22:25.481>that <00:22:25.715>testing <00:22:25.948>is <00:22:26.181>experimental <00:22:26.415>science

463
00:22:28.535 --> 00:22:31.795
<00:22:28.535>and <00:22:28.710>so <00:22:28.885>she <00:22:29.060>sees <00:22:29.235>testing <00:22:29.410>as <00:22:29.585>an <00:22:29.760>adaptive <00:22:29.935>investigation

464
00:22:32.375 --> 00:22:35.715
<00:22:32.375>rather <00:22:32.575>than <00:22:32.775>a <00:22:32.975>factory <00:22:33.175>process <00:22:33.375>of <00:22:33.575>creating, <00:22:33.775>automating

465
00:22:35.715 --> 00:22:37.275
<00:22:35.715>and <00:22:36.181>executing <00:22:36.648>test <00:22:37.115>cases.

466
00:22:38.455 --> 00:22:42.635
<00:22:38.455>So <00:22:38.610>she <00:22:38.766>knows <00:22:38.921>that <00:22:39.077>her <00:22:39.232>past <00:22:39.388>success <00:22:39.543>is <00:22:39.699>not <00:22:39.855>defined

467
00:22:42.935 --> 00:22:44.275
<00:22:42.935>by <00:22:43.136>the <00:22:43.337>number <00:22:43.538>of <00:22:43.739>her <00:22:43.940>past <00:22:44.141>cases.

468
00:22:45.615 --> 00:22:48.755
<00:22:45.615>She <00:22:45.755>knows <00:22:45.895>that <00:22:46.035>her <00:22:46.175>past <00:22:46.315>success <00:22:46.455>is <00:22:46.595>defined <00:22:46.735>by <00:22:46.875>the <00:22:47.015>quality

469
00:22:48.935 --> 00:22:50.115
<00:22:48.935>of <00:22:49.289>her <00:22:49.643>past <00:22:49.997>ideas.

470
00:22:50.935 --> 00:22:53.915
<00:22:50.935>So <00:22:51.110>Alice <00:22:51.284>acts <00:22:51.460>like <00:22:51.635>an <00:22:51.810>experimentalist, <00:22:51.985>so <00:22:52.159>to <00:22:52.335>say,

471
00:22:54.175 --> 00:22:58.595
<00:22:54.175>who <00:22:54.375>does <00:22:54.575>not <00:22:54.774>confuse <00:22:54.975>testing <00:22:55.175>with <00:22:55.375>test <00:22:55.575>cases

472
00:22:59.345 --> 00:23:03.955
<00:22:59.345>because <00:22:59.545>she <00:22:59.745>considers <00:22:59.945>her <00:23:00.145>tests <00:23:00.345>as <00:23:00.545>experiments <00:23:00.745>to

473
00:23:03.975 --> 00:23:07.155
<00:23:03.975>to <00:23:04.208>challenge <00:23:04.441>hypothesis, <00:23:04.675>to <00:23:04.908>challenge <00:23:05.141>assumptions, <00:23:05.375>opinions

474
00:23:07.155 --> 00:23:10.635
<00:23:07.155>and <00:23:07.388>various <00:23:07.621>different <00:23:07.855>perspectives <00:23:08.088>about <00:23:08.321>the <00:23:08.555>software.

475
00:23:11.595 --> 00:23:13.465
<00:23:11.595>Alice <00:23:11.875>also <00:23:12.155>acts <00:23:12.435>like <00:23:12.715>a <00:23:12.995>stakeholder

476
00:23:13.695 --> 00:23:16.385
<00:23:13.695>because <00:23:13.895>she <00:23:14.095>knows <00:23:14.294>the <00:23:14.495>testing <00:23:14.695>means <00:23:14.895>comparing <00:23:15.095>the

477
00:23:16.385 --> 00:23:17.945
<00:23:16.385>invisible <00:23:16.851>to <00:23:17.318>the <00:23:17.785>ambiguous.

478
00:23:18.485 --> 00:23:21.065
<00:23:18.485>She <00:23:18.718>knows <00:23:18.951>that <00:23:19.185>the <00:23:19.418>stakeholders <00:23:19.651>knowledge, <00:23:19.885>goals,

479
00:23:21.125 --> 00:23:23.945
<00:23:21.125>and <00:23:21.358>experiences <00:23:21.591>are <00:23:21.825>not <00:23:22.058>identical <00:23:22.291>to <00:23:22.525>hers.

480
00:23:24.685 --> 00:23:28.025
<00:23:24.685>She <00:23:24.840>knows <00:23:24.996>that <00:23:25.151>the <00:23:25.307>universe <00:23:25.462>of <00:23:25.618>stakeholders <00:23:25.773>is <00:23:25.929>simply <00:23:26.085>vast,

481
00:23:28.445 --> 00:23:32.545
<00:23:28.445>but <00:23:28.585>she <00:23:28.725>doesn't <00:23:28.865>get <00:23:29.004>tired <00:23:29.145>to <00:23:29.284>examine <00:23:29.425>risks <00:23:29.564>for <00:23:29.705>both <00:23:29.845>internal

482
00:23:32.685 --> 00:23:34.265
<00:23:32.685>and <00:23:33.385>external <00:23:34.085>stakeholders.

483
00:23:35.565 --> 00:23:37.705
<00:23:35.565>She <00:23:35.845>considers <00:23:36.125>how <00:23:36.405>internal <00:23:36.685>stakeholders <00:23:36.965>such

484
00:23:37.705 --> 00:23:39.545
<00:23:37.705>as <00:23:38.054>product <00:23:38.405>owners, <00:23:38.754>developers, <00:23:39.105>sales

485
00:23:39.565 --> 00:23:43.305
<00:23:39.565>and <00:23:39.740>marketers <00:23:39.915>would <00:23:40.090>look <00:23:40.265>at <00:23:40.440>the <00:23:40.615>risks <00:23:40.790>she's <00:23:40.965>uncovering

486
00:23:43.565 --> 00:23:46.245
<00:23:43.565>and <00:23:43.765>she <00:23:43.965>ties <00:23:44.165>these <00:23:44.365>risks <00:23:44.565>to <00:23:44.765>their <00:23:44.965>objectives,

487
00:23:46.245 --> 00:23:47.565
<00:23:46.245>to <00:23:46.443>their <00:23:46.640>needs <00:23:46.839>and <00:23:47.037>to <00:23:47.235>their <00:23:47.433>goals.

488
00:23:48.145 --> 00:23:50.245
<00:23:48.145>She <00:23:48.378>also <00:23:48.611>examines <00:23:48.845>risks <00:23:49.078>from <00:23:49.311>the <00:23:49.545>perspective

489
00:23:50.305 --> 00:23:53.445
<00:23:50.305>of <00:23:50.538>external <00:23:50.771>stakeholders <00:23:51.005>such <00:23:51.238>as <00:23:51.471>the <00:23:51.705>customers

490
00:23:53.445 --> 00:23:54.565
<00:23:53.445>and <00:23:53.781>partners <00:23:54.117>for <00:23:54.453>example.

491
00:23:55.105 --> 00:23:58.565
<00:23:55.105>And <00:23:55.338>in <00:23:55.571>doing <00:23:55.805>so, <00:23:56.038>she <00:23:56.271>avoids <00:23:56.505>saying

492
00:23:58.745 --> 00:24:02.245
<00:23:58.745>and <00:23:58.872>uh, <00:23:58.999>even <00:23:59.126>thinking <00:23:59.254>things <00:23:59.381>like <00:23:59.508>no <00:23:59.635>one <00:23:59.763>would <00:23:59.890>ever <00:24:00.017>do <00:24:00.145>that

493
00:24:02.275 --> 00:24:06.405
<00:24:02.275>because <00:24:02.430>she <00:24:02.586>knows <00:24:02.741>the <00:24:02.897>more <00:24:03.052>advanced <00:24:03.208>she <00:24:03.363>gets <00:24:03.519>in <00:24:03.675>the

494
00:24:06.645 --> 00:24:10.105
<00:24:06.645>software, <00:24:06.878>the <00:24:07.111>harder <00:24:07.345>it <00:24:07.578>becomes <00:24:07.811>to <00:24:08.045>relate

495
00:24:10.125 --> 00:24:12.585
<00:24:10.125>to <00:24:10.358>drew <00:24:10.591>beginners <00:24:10.825>to <00:24:11.058>the <00:24:11.291>end <00:24:11.525>users.

496
00:24:13.565 --> 00:24:17.265
<00:24:13.565>So <00:24:13.692>she's <00:24:13.819>trying <00:24:13.946>to <00:24:14.074>wear <00:24:14.201>the <00:24:14.328>end <00:24:14.455>user's <00:24:14.583>shoes <00:24:14.710>to <00:24:14.837>not <00:24:14.965>forget

497
00:24:17.265 --> 00:24:19.345
<00:24:17.265>what <00:24:17.498>it's <00:24:17.731>like <00:24:17.965>to <00:24:18.198>be <00:24:18.431>one, <00:24:18.665>and

498
00:24:19.345 --> 00:24:22.705
<00:24:19.345>therefore <00:24:19.500>she <00:24:19.656>always <00:24:19.811>tries <00:24:19.967>to <00:24:20.122>step <00:24:20.278>out <00:24:20.433>of <00:24:20.589>her <00:24:20.745>role

499
00:24:22.765 --> 00:24:26.985
<00:24:22.765>as <00:24:22.905>a <00:24:23.045>software <00:24:23.185>tester <00:24:23.325>by <00:24:23.465>taking <00:24:23.605>on <00:24:23.745>the <00:24:23.885>perspective <00:24:24.025>of <00:24:24.165>yeah,

500
00:24:26.985 --> 00:24:29.865
<00:24:26.985>different <00:24:27.185>stakeholders <00:24:27.385>by <00:24:27.584>looking <00:24:27.784>at <00:24:27.985>the <00:24:28.185>risks <00:24:28.385>from

501
00:24:29.865 --> 00:24:30.945
<00:24:29.865>their <00:24:30.351>vantage <00:24:30.837>points.

502
00:24:31.765 --> 00:24:33.625
<00:24:31.765>And <00:24:31.998>she <00:24:32.231>does <00:24:32.465>so <00:24:32.698>because <00:24:32.931>she <00:24:33.165>knows

503
00:24:33.625 --> 00:24:37.035
<00:24:33.625>that <00:24:33.905>these <00:24:34.185>different <00:24:34.465>viewpoints <00:24:34.745>allow <00:24:35.025>her

504
00:24:37.035 --> 00:24:38.395
<00:24:37.035>to <00:24:37.209>look <00:24:37.384>at <00:24:37.559>the <00:24:37.734>value <00:24:37.909>of <00:24:38.084>the <00:24:38.259>software

505
00:24:38.735 --> 00:24:39.835
<00:24:38.735>and <00:24:39.230>the <00:24:39.725>impact

506
00:24:40.695 --> 00:24:43.835
<00:24:40.695>of <00:24:40.835>a <00:24:40.975>threat <00:24:41.115>against <00:24:41.254>the <00:24:41.395>software <00:24:41.534>in <00:24:41.675>a <00:24:41.814>variety <00:24:41.955>of <00:24:42.095>ways.

507
00:24:44.375 --> 00:24:46.995
<00:24:44.375>So <00:24:44.550>she <00:24:44.725>thinks <00:24:44.900>like <00:24:45.075>a <00:24:45.250>stakeholder, <00:24:45.425>so <00:24:45.600>to <00:24:45.775>say,

508
00:24:47.295 --> 00:24:51.675
<00:24:47.295>to <00:24:47.495>enrich <00:24:47.695>the <00:24:47.895>way <00:24:48.095>she <00:24:48.295>assesses, <00:24:48.495>she <00:24:48.695>prioritizes

509
00:24:52.095 --> 00:24:56.275
<00:24:52.095>and <00:24:52.235>she <00:24:52.375>analyzes <00:24:52.515>risk <00:24:52.655>to <00:24:52.795>get <00:24:52.935>a <00:24:53.075>much <00:24:53.215>broader, <00:24:53.355>uh, <00:24:53.495>picture

510
00:24:56.335 --> 00:24:57.915
<00:24:56.335>of <00:24:56.685>the <00:24:57.035>overall <00:24:57.385>risk <00:24:57.735>profile.

511
00:24:59.235 --> 00:25:01.295
<00:24:59.235>Now, <00:24:59.584>besides <00:24:59.935>that, <00:25:00.284>Alice <00:25:00.635>knows

512
00:25:01.525 --> 00:25:06.055
<00:25:01.525>that <00:25:01.680>she <00:25:01.836>cannot <00:25:01.991>expect <00:25:02.147>answers <00:25:02.302>to <00:25:02.458>questions <00:25:02.613>she <00:25:02.769>didn't <00:25:02.925>ask.

513
00:25:06.995 --> 00:25:08.775
<00:25:06.995>She <00:25:07.274>isn't <00:25:07.554>afraid <00:25:07.834>of <00:25:08.114>looking <00:25:08.395>stupid

514
00:25:09.475 --> 00:25:11.095
<00:25:09.475>by <00:25:09.941>asking <00:25:10.408>simple <00:25:10.875>questions.

515
00:25:11.475 --> 00:25:13.775
<00:25:11.475>She <00:25:11.754>understands <00:25:12.034>that <00:25:12.314>tests <00:25:12.594>are <00:25:12.875>questions.

516
00:25:14.355 --> 00:25:15.415
<00:25:14.355>She <00:25:14.673>asks <00:25:14.991>the <00:25:15.309>software

517
00:25:15.415 --> 00:25:17.975
<00:25:15.415>to <00:25:15.648>reveal <00:25:15.881>useful <00:25:16.115>information <00:25:16.348>about <00:25:16.581>the <00:25:16.815>software

518
00:25:18.125 --> 00:25:20.535
<00:25:18.125>because <00:25:18.475>she <00:25:18.825>understands <00:25:19.175>that <00:25:19.525>assumptions

519
00:25:20.675 --> 00:25:23.855
<00:25:20.675>or <00:25:20.955>unexamined <00:25:21.235>beliefs <00:25:21.514>are <00:25:21.794>simply <00:25:22.075>the

520
00:25:23.975 --> 00:25:25.335
<00:25:23.975>termites <00:25:24.383>in <00:25:24.791>software <00:25:25.199>testing.

521
00:25:26.195 --> 00:25:28.615
<00:25:26.195>And <00:25:26.395>she <00:25:26.595>suspects <00:25:26.794>that <00:25:26.995>she <00:25:27.195>probably <00:25:27.395>has <00:25:27.595>answers

522
00:25:28.615 --> 00:25:31.135
<00:25:28.615>that <00:25:28.755>might <00:25:28.895>be <00:25:29.035>wrong <00:25:29.175>and <00:25:29.315>beliefs <00:25:29.455>that <00:25:29.595>might <00:25:29.735>not <00:25:29.875>be <00:25:30.015>true,

523
00:25:31.475 --> 00:25:33.735
<00:25:31.475>but <00:25:31.675>she <00:25:31.875>isn't <00:25:32.074>afraid <00:25:32.274>of <00:25:32.475>not <00:25:32.675>knowing <00:25:32.875>things.

524
00:25:34.395 --> 00:25:35.815
<00:25:34.395>So <00:25:34.714>she <00:25:35.034>doesn't <00:25:35.353>get <00:25:35.673>tired

525
00:25:35.835 --> 00:25:38.575
<00:25:35.835>of <00:25:36.068>applying <00:25:36.301>this <00:25:36.535>crazy <00:25:36.768>method <00:25:37.001>called <00:25:37.235>asking

526
00:25:39.035 --> 00:25:41.135
<00:25:39.035>before <00:25:39.501>she <00:25:39.968>assumes <00:25:40.435>something.

527
00:25:41.755 --> 00:25:44.535
<00:25:41.755>And <00:25:41.930>for <00:25:42.105>the <00:25:42.280>reason <00:25:42.455>she <00:25:42.630>touches <00:25:42.805>herself, <00:25:42.980>not <00:25:43.155>only

528
00:25:44.795 --> 00:25:49.095
<00:25:44.795>by <00:25:44.922>the <00:25:45.049>answers <00:25:45.176>she <00:25:45.304>gives, <00:25:45.431>but <00:25:45.558>also <00:25:45.685>by <00:25:45.813>the <00:25:45.940>questions <00:25:46.067>she <00:25:46.195>asks

529
00:25:49.365 --> 00:25:51.895
<00:25:49.365>because <00:25:49.645>she <00:25:49.925>knows <00:25:50.205>that <00:25:50.485>curiosity <00:25:50.765>is

530
00:25:52.445 --> 00:25:54.975
<00:25:52.445>what <00:25:52.725>fuels <00:25:53.004>excellence <00:25:53.284>of <00:25:53.564>the <00:25:53.845>testing.

531
00:25:56.155 --> 00:25:59.455
<00:25:56.155>So <00:25:56.330>Alice <00:25:56.504>acts <00:25:56.680>like <00:25:56.855>a <00:25:57.030>curious <00:25:57.205>little <00:25:57.379>kid <00:25:57.555>who,

532
00:25:59.595 --> 00:26:01.015
<00:25:59.595>who <00:26:00.021>never <00:26:00.447>stops <00:26:00.873>asking

533
00:26:01.015 --> 00:26:02.015
<00:26:01.015>because <00:26:01.465>she <00:26:01.915>knows

534
00:26:02.245 --> 00:26:05.215
<00:26:02.245>that <00:26:02.478>asking <00:26:02.711>great <00:26:02.945>questions <00:26:03.178>is <00:26:03.411>just <00:26:03.645>the

535
00:26:05.215 --> 00:26:06.415
<00:26:05.215>source <00:26:05.574>of <00:26:05.935>great <00:26:06.295>testing.

536
00:26:07.515 --> 00:26:08.855
<00:26:07.515>Now <00:26:07.816>Alice <00:26:08.118>is <00:26:08.419>also <00:26:08.721>aware

537
00:26:09.005 --> 00:26:11.095
<00:26:09.005>that <00:26:09.238>if <00:26:09.471>she <00:26:09.705>doesn't <00:26:09.938>have <00:26:10.171>any <00:26:10.405>questions

538
00:26:11.985 --> 00:26:13.615
<00:26:11.985>about <00:26:12.264>the <00:26:12.544>risks <00:26:12.824>in <00:26:13.104>the <00:26:13.385>software,

539
00:26:13.615 --> 00:26:15.375
<00:26:13.615>then <00:26:13.790>there <00:26:13.965>is <00:26:14.140>no <00:26:14.315>reason <00:26:14.490>to <00:26:14.665>test <00:26:14.840>at <00:26:15.015>all.

540
00:26:16.115 --> 00:26:18.695
<00:26:16.115>And <00:26:16.290>if <00:26:16.465>she <00:26:16.640>has <00:26:16.815>at <00:26:16.990>least <00:26:17.165>one <00:26:17.340>such <00:26:17.515>question,

541
00:26:19.005 --> 00:26:20.535
<00:26:19.005>well <00:26:19.280>then <00:26:19.555>she <00:26:19.831>will <00:26:20.106>ask, <00:26:20.382>well,

542
00:26:20.605 --> 00:26:23.015
<00:26:20.605>will <00:26:20.838>these <00:26:21.071>tests <00:26:21.305>cost <00:26:21.538>more <00:26:21.771>to <00:26:22.005>perform?

543
00:26:23.515 --> 00:26:25.015
<00:26:23.515>And <00:26:23.785>the <00:26:24.055>answers <00:26:24.325>will <00:26:24.595>be <00:26:24.865>worth.

544
00:26:25.795 --> 00:26:28.615
<00:26:25.795>So <00:26:25.950>she <00:26:26.106>acts <00:26:26.261>like <00:26:26.417>an <00:26:26.572>accountant <00:26:26.728>in, <00:26:26.883>in <00:26:27.039>a <00:26:27.195>sense

545
00:26:28.615 --> 00:26:30.055
<00:26:28.615>because <00:26:29.263>she <00:26:29.911>understands

546
00:26:30.055 --> 00:26:33.695
<00:26:30.055>that <00:26:30.288>good <00:26:30.521>testing <00:26:30.755>involves <00:26:30.988>balancing <00:26:31.221>the <00:26:31.455>need

547
00:26:34.075 --> 00:26:37.775
<00:26:34.075>to <00:26:34.275>mitigate <00:26:34.475>risk <00:26:34.675>against <00:26:34.875>the <00:26:35.075>risk <00:26:35.275>of <00:26:35.475>trying

548
00:26:37.775 --> 00:26:40.935
<00:26:37.775>to <00:26:37.975>gather <00:26:38.175>too <00:26:38.375>much <00:26:38.575>information <00:26:38.775>about <00:26:38.975>the <00:26:39.175>software.

549
00:26:41.725 --> 00:26:43.645
<00:26:41.725>Now <00:26:41.958>Ellis <00:26:42.191>also <00:26:42.425>acts <00:26:42.658>like <00:26:42.891>a <00:26:43.125>skeptic

550
00:26:43.915 --> 00:26:47.485
<00:26:43.915>because <00:26:44.115>she <00:26:44.315>knows <00:26:44.514>that <00:26:44.715>she <00:26:44.915>shouldn't <00:26:45.115>believe <00:26:45.315>everything.

551
00:26:47.715 --> 00:26:50.725
<00:26:47.715>Yeah, <00:26:47.870>she <00:26:48.026>hears <00:26:48.181>and <00:26:48.337>only <00:26:48.492>half <00:26:48.648>of <00:26:48.803>what <00:26:48.959>she <00:26:49.115>sees.

552
00:26:51.345 --> 00:26:53.645
<00:26:51.345>She <00:26:51.520>knows <00:26:51.695>that <00:26:51.870>there <00:26:52.045>are <00:26:52.220>no <00:26:52.395>facts, <00:26:52.570>only <00:26:52.745>interpretation.

553
00:26:53.785 --> 00:26:57.085
<00:26:53.785>She <00:26:53.960>knows <00:26:54.135>that <00:26:54.310>absurdity <00:26:54.485>or <00:26:54.660>that <00:26:54.835>certainty <00:26:55.010>is <00:26:55.185>absurd.

554
00:26:57.905 --> 00:27:00.325
<00:26:57.905>So <00:26:58.254>she <00:26:58.605>thinks <00:26:58.955>ly <00:26:59.305>about

555
00:27:00.355 --> 00:27:02.685
<00:27:00.355>what <00:27:00.555>she <00:27:00.755>has <00:27:00.955>been <00:27:01.155>told <00:27:01.355>about <00:27:01.555>the <00:27:01.755>software

556
00:27:02.945 --> 00:27:04.205
<00:27:02.945>and <00:27:03.322>she's <00:27:03.701>skeptically <00:27:04.079>about

557
00:27:04.275 --> 00:27:05.925
<00:27:04.275>what <00:27:04.555>she <00:27:04.835>knows <00:27:05.115>about <00:27:05.395>the <00:27:05.675>software

558
00:27:06.225 --> 00:27:07.565
<00:27:06.225>and <00:27:06.397>she <00:27:06.569>knows <00:27:06.741>that <00:27:06.914>you <00:27:07.086>will <00:27:07.258>never <00:27:07.431>know

559
00:27:07.565 --> 00:27:08.805
<00:27:07.565>everything <00:27:07.937>about <00:27:08.309>the <00:27:08.681>software.

560
00:27:09.345 --> 00:27:12.685
<00:27:09.345>So <00:27:09.472>she's <00:27:09.599>aware <00:27:09.726>of <00:27:09.854>the <00:27:09.981>fact <00:27:10.108>that <00:27:10.235>she <00:27:10.363>isn't <00:27:10.490>la <00:27:10.617>plus's <00:27:10.745>demon,

561
00:27:12.905 --> 00:27:14.005
<00:27:12.905>so <00:27:13.235>to <00:27:13.565>say, <00:27:13.895>and

562
00:27:14.005 --> 00:27:15.845
<00:27:14.005>therefore <00:27:14.471>she <00:27:14.938>focuses <00:27:15.405>on

563
00:27:16.595 --> 00:27:18.955
<00:27:16.595>what <00:27:16.828>she <00:27:17.061>doesn't <00:27:17.295>know <00:27:17.528>about <00:27:17.761>the <00:27:17.995>software

564
00:27:18.955 --> 00:27:22.755
<00:27:18.955>and <00:27:19.110>she <00:27:19.266>doesn't <00:27:19.421>just <00:27:19.577>settle <00:27:19.732>with <00:27:19.888>what <00:27:20.043>she <00:27:20.199>currently <00:27:20.355>knows.

565
00:27:23.175 --> 00:27:26.555
<00:27:23.175>So <00:27:23.350>in <00:27:23.524>short, <00:27:23.700>she <00:27:23.875>keeps <00:27:24.050>thinking <00:27:24.225>despite <00:27:24.399>already <00:27:24.575>knowing

566
00:27:26.555 --> 00:27:29.835
<00:27:26.555>because <00:27:26.755>she <00:27:26.955>sees <00:27:27.155>testing <00:27:27.355>as <00:27:27.555>a <00:27:27.755>thinking <00:27:27.955>activity

567
00:27:29.975 --> 00:27:31.555
<00:27:29.975>and <00:27:30.254>she <00:27:30.534>treats <00:27:30.814>it <00:27:31.094>like <00:27:31.375>one.

568
00:27:32.625 --> 00:27:34.685
<00:27:32.625>She <00:27:32.800>knows <00:27:32.975>that <00:27:33.150>she <00:27:33.325>has <00:27:33.500>to <00:27:33.675>change <00:27:33.850>her <00:27:34.025>thinking

569
00:27:34.685 --> 00:27:37.045
<00:27:34.685>to <00:27:34.918>change <00:27:35.151>her <00:27:35.385>testing <00:27:35.618>because <00:27:35.851>she's <00:27:36.085>aware

570
00:27:37.195 --> 00:27:40.965
<00:27:37.195>that <00:27:37.335>the <00:27:37.475>software <00:27:37.615>is <00:27:37.754>a <00:27:37.895>canning <00:27:38.034>disease <00:27:38.175>that <00:27:38.314>is, <00:27:38.455>uh, <00:27:38.595>yeah,

571
00:27:41.745 --> 00:27:46.005
<00:27:41.745>always <00:27:41.885>constantly <00:27:42.024>trying <00:27:42.165>to <00:27:42.304>fool <00:27:42.445>her <00:27:42.584>in <00:27:42.725>one <00:27:42.864>way <00:27:43.004>or <00:27:43.145>another.

572
00:27:46.105 --> 00:27:49.485
<00:27:46.105>And <00:27:46.338>for <00:27:46.571>that <00:27:46.805>reason, <00:27:47.038>she <00:27:47.271>remains <00:27:47.505>cautious,

573
00:27:49.545 --> 00:27:50.605
<00:27:49.545>she <00:27:50.022>remains <00:27:50.499>curious

574
00:27:50.745 --> 00:27:52.245
<00:27:50.745>and <00:27:51.195>she <00:27:51.645>remains <00:27:52.094>critical

575
00:27:52.425 --> 00:27:54.125
<00:27:52.425>to <00:27:52.625>not <00:27:52.825>get <00:27:53.024>fooled <00:27:53.225>over <00:27:53.425>and <00:27:53.625>over <00:27:53.825>again.

576
00:27:54.785 --> 00:27:58.565
<00:27:54.785>So <00:27:54.925>the <00:27:55.065>bottom <00:27:55.205>line <00:27:55.345>is <00:27:55.485>Alice <00:27:55.625>acts <00:27:55.765>like <00:27:55.905>a <00:27:56.045>cosmic <00:27:56.185>skeptic

577
00:27:58.795 --> 00:27:59.885
<00:27:58.795>because <00:27:59.285>she <00:27:59.776>knows

578
00:27:59.885 --> 00:28:03.005
<00:27:59.885>that <00:28:00.118>excellent <00:28:00.351>testing <00:28:00.585>is <00:28:00.818>just <00:28:01.051>the <00:28:01.285>byproduct

579
00:28:03.305 --> 00:28:05.165
<00:28:03.305>of <00:28:03.771>continuous <00:28:04.238>critical <00:28:04.705>thinking.

580
00:28:06.025 --> 00:28:08.565
<00:28:06.025>Now, <00:28:06.225>Elli <00:28:06.425>also <00:28:06.625>understands <00:28:06.825>that <00:28:07.025>the <00:28:07.225>primary <00:28:07.425>goal

581
00:28:08.565 --> 00:28:11.525
<00:28:08.565>of <00:28:08.765>testing <00:28:08.965>is <00:28:09.165>to <00:28:09.365>make <00:28:09.565>other <00:28:09.765>people <00:28:09.965>aware

582
00:28:11.525 --> 00:28:12.965
<00:28:11.525>of <00:28:11.849>problems <00:28:12.173>in <00:28:12.497>the <00:28:12.821>software.

583
00:28:13.465 --> 00:28:15.165
<00:28:13.465>She <00:28:13.745>knows <00:28:14.024>that <00:28:14.304>making <00:28:14.584>people <00:28:14.865>aware

584
00:28:15.165 --> 00:28:17.645
<00:28:15.165>of <00:28:15.398>problems <00:28:15.631>is <00:28:15.865>a <00:28:16.098>socially <00:28:16.331>difficult <00:28:16.565>task

585
00:28:17.755 --> 00:28:21.045
<00:28:17.755>because <00:28:17.930>of <00:28:18.105>the <00:28:18.280>people's <00:28:18.455>natural <00:28:18.630>desire <00:28:18.805>to <00:28:18.980>avoid <00:28:19.155>trouble

586
00:28:21.865 --> 00:28:23.445
<00:28:21.865>and <00:28:22.145>she <00:28:22.425>understands <00:28:22.705>that <00:28:22.985>problems <00:28:23.265>are

587
00:28:23.445 --> 00:28:24.645
<00:28:23.445>something <00:28:23.715>negative <00:28:23.985>at <00:28:24.254>a <00:28:24.525>chance.

588
00:28:25.425 --> 00:28:28.205
<00:28:25.425>Now <00:28:25.580>she, <00:28:25.736>she <00:28:25.891>considers <00:28:26.047>the <00:28:26.202>problem <00:28:26.358>as <00:28:26.513>a <00:28:26.669>deviation <00:28:26.825>from

589
00:28:28.395 --> 00:28:30.565
<00:28:28.395>what <00:28:28.628>is <00:28:28.861>perceived <00:28:29.095>and <00:28:29.328>what <00:28:29.561>is <00:28:29.795>desired.

590
00:28:30.905 --> 00:28:33.925
<00:28:30.905>And <00:28:31.080>the <00:28:31.254>bigger <00:28:31.430>this <00:28:31.605>deviation, <00:28:31.780>the <00:28:31.955>bigger <00:28:32.129>this <00:28:32.305>gap,

591
00:28:34.465 --> 00:28:36.565
<00:28:34.465>the <00:28:34.639>bigger <00:28:34.814>the <00:28:34.990>problem <00:28:35.165>and <00:28:35.340>the <00:28:35.514>bigger <00:28:35.689>the <00:28:35.865>problem,

592
00:28:36.825 --> 00:28:39.885
<00:28:36.825>the <00:28:36.980>more <00:28:37.136>it <00:28:37.291>might <00:28:37.447>threaten <00:28:37.602>the <00:28:37.758>value <00:28:37.913>of <00:28:38.069>the <00:28:38.225>software.

593
00:28:41.745 --> 00:28:43.725
<00:28:41.745>And <00:28:41.978>she's <00:28:42.211>also <00:28:42.445>rational <00:28:42.678>enough <00:28:42.911>to <00:28:43.145>know

594
00:28:43.725 --> 00:28:46.605
<00:28:43.725>that <00:28:44.004>different <00:28:44.284>people <00:28:44.564>will <00:28:44.844>react <00:28:45.125>differently

595
00:28:46.665 --> 00:28:49.765
<00:28:46.665>and <00:28:46.945>sometimes <00:28:47.225>irrationally <00:28:47.504>to <00:28:47.784>potential <00:28:48.065>threats.

596
00:28:50.265 --> 00:28:52.645
<00:28:50.265>She <00:28:50.420>knows <00:28:50.576>that <00:28:50.731>there <00:28:50.887>are <00:28:51.042>people <00:28:51.198>you <00:28:51.353>know, <00:28:51.509>who <00:28:51.665>welcome

597
00:28:53.205 --> 00:28:55.485
<00:28:53.205>critical <00:28:53.405>feedback <00:28:53.605>and <00:28:53.804>there <00:28:54.004>are <00:28:54.205>people <00:28:54.405>who <00:28:54.605>don't.

598
00:28:55.905 --> 00:28:58.045
<00:28:55.905>She <00:28:56.060>knows <00:28:56.216>that <00:28:56.371>there <00:28:56.527>are <00:28:56.682>people <00:28:56.838>who <00:28:56.993>value <00:28:57.149>the <00:28:57.305>perspective

599
00:28:58.065 --> 00:29:00.565
<00:28:58.065>of <00:28:58.265>others, <00:28:58.465>and <00:28:58.665>there <00:28:58.865>are <00:28:59.065>people <00:28:59.265>who <00:28:59.465>don't.

600
00:29:00.985 --> 00:29:02.005
<00:29:00.985>She <00:29:01.168>knows <00:29:01.352>that <00:29:01.535>there <00:29:01.719>are <00:29:01.903>people

601
00:29:02.005 --> 00:29:03.645
<00:29:02.005>who <00:29:02.471>hold <00:29:02.938>themselves <00:29:03.405>responsible

602
00:29:04.145 --> 00:29:07.525
<00:29:04.145>and <00:29:04.495>accountable <00:29:04.845>for <00:29:05.195>their <00:29:05.545>mistakes,

603
00:29:07.665 --> 00:29:09.405
<00:29:07.665>and <00:29:07.945>there <00:29:08.225>are <00:29:08.504>people <00:29:08.784>who <00:29:09.065>don't.

604
00:29:11.335 --> 00:29:13.475
<00:29:11.335>So <00:29:11.568>what <00:29:11.801>she <00:29:12.035>does <00:29:12.268>is <00:29:12.501>very <00:29:12.735>simple.

605
00:29:14.135 --> 00:29:16.315
<00:29:14.135>She <00:29:14.335>first <00:29:14.535>of <00:29:14.735>all <00:29:14.935>realizes <00:29:15.135>that <00:29:15.335>the <00:29:15.535>nature

606
00:29:16.415 --> 00:29:17.755
<00:29:16.415>of <00:29:16.817>humans <00:29:17.219>is <00:29:17.621>diverse.

607
00:29:19.025 --> 00:29:21.085
<00:29:19.025>So <00:29:19.305>she <00:29:19.585>studies <00:29:19.865>the <00:29:20.145>people's <00:29:20.425>personalities

608
00:29:21.225 --> 00:29:24.605
<00:29:21.225>and <00:29:21.380>the <00:29:21.536>relationships <00:29:21.691>among <00:29:21.847>each <00:29:22.002>other <00:29:22.158>to <00:29:22.313>make <00:29:22.469>them <00:29:22.625>yeah,

609
00:29:24.655 --> 00:29:28.085
<00:29:24.655>aware <00:29:24.855>of <00:29:25.055>problems <00:29:25.254>in <00:29:25.455>a, <00:29:25.655>uh, <00:29:25.855>dispassionate, <00:29:26.055>deliberate

610
00:29:28.085 --> 00:29:29.485
<00:29:28.085>and <00:29:28.505>then <00:29:28.925>personal <00:29:29.344>way.

611
00:29:30.145 --> 00:29:33.765
<00:29:30.145>She <00:29:30.320>understands <00:29:30.495>that <00:29:30.670>if <00:29:30.845>she <00:29:31.020>isn't <00:29:31.195>making <00:29:31.370>people <00:29:31.545>aware

612
00:29:33.765 --> 00:29:35.805
<00:29:33.765>of <00:29:34.231>problems <00:29:34.698>right <00:29:35.165>away,

613
00:29:37.125 --> 00:29:39.815
<00:29:37.125>then <00:29:37.358>these <00:29:37.591>problems <00:29:37.825>will <00:29:38.058>eventually <00:29:38.291>find <00:29:38.525>them.

614
00:29:40.155 --> 00:29:44.005
<00:29:40.155>So <00:29:40.330>she <00:29:40.504>communicates <00:29:40.680>this <00:29:40.855>problem <00:29:41.030>with <00:29:41.205>modesty <00:29:41.379>and <00:29:41.555>empathy.

615
00:29:44.455 --> 00:29:47.885
<00:29:44.455>She's <00:29:44.655>direct <00:29:44.855>critically, <00:29:45.054>but <00:29:45.254>she's <00:29:45.455>not <00:29:45.655>judgmental <00:29:45.855>and

616
00:29:47.885 --> 00:29:51.085
<00:29:47.885>therefore <00:29:48.040>she <00:29:48.196>talks <00:29:48.351>about <00:29:48.507>negative <00:29:48.662>things <00:29:48.818>in <00:29:48.973>a <00:29:49.129>positive <00:29:49.285>way

617
00:29:51.085 --> 00:29:54.325
<00:29:51.085>because <00:29:51.260>she <00:29:51.435>sees <00:29:51.610>problems <00:29:51.785>as <00:29:51.960>an <00:29:52.135>opportunity <00:29:52.310>to <00:29:52.485>improve.

618
00:29:55.305 --> 00:29:58.285
<00:29:55.305>She <00:29:55.538>knows <00:29:55.771>that <00:29:56.005>the <00:29:56.238>software <00:29:56.471>doesn't <00:29:56.705>have

619
00:29:58.305 --> 00:30:01.525
<00:29:58.305>to <00:29:58.480>behave <00:29:58.655>the <00:29:58.830>way <00:29:59.005>she <00:29:59.180>wants <00:29:59.355>it <00:29:59.530>to <00:29:59.705>behave.

620
00:30:01.895 --> 00:30:04.965
<00:30:01.895>She's <00:30:02.128>aware <00:30:02.361>that <00:30:02.595>she <00:30:02.828>doesn't <00:30:03.061>design <00:30:03.295>code

621
00:30:04.965 --> 00:30:06.365
<00:30:04.965>or <00:30:05.280>sell <00:30:05.595>the <00:30:05.910>software, <00:30:06.225>right?

622
00:30:06.665 --> 00:30:08.125
<00:30:06.665>She <00:30:06.927>tries <00:30:07.190>to <00:30:07.453>help <00:30:07.716>other <00:30:07.979>people

623
00:30:08.145 --> 00:30:11.405
<00:30:08.145>to <00:30:08.345>understand <00:30:08.545>the <00:30:08.745>software <00:30:08.945>they <00:30:09.145>have <00:30:09.345>got <00:30:09.545>so

624
00:30:11.405 --> 00:30:14.285
<00:30:11.405>that <00:30:11.560>they <00:30:11.716>can <00:30:11.871>decide <00:30:12.027>whether <00:30:12.182>it's <00:30:12.338>the <00:30:12.493>software <00:30:12.649>they <00:30:12.805>want.

625
00:30:15.145 --> 00:30:19.845
<00:30:15.145>So <00:30:15.300>she <00:30:15.456>acts <00:30:15.611>like <00:30:15.767>a <00:30:15.922>friend <00:30:16.078>who <00:30:16.233>reduces <00:30:16.389>people's <00:30:16.545>ignorance

626
00:30:20.575 --> 00:30:24.125
<00:30:20.575>about <00:30:20.855>the <00:30:21.135>software <00:30:21.415>without <00:30:21.695>embracing <00:30:21.975>arrogance.

627
00:30:26.305 --> 00:30:27.815
<00:30:26.305>Alice <00:30:26.984>also <00:30:27.664>understands

628
00:30:28.005 --> 00:30:31.295
<00:30:28.005>that <00:30:28.180>software <00:30:28.355>testing <00:30:28.530>is <00:30:28.705>the <00:30:28.880>headlights <00:30:29.055>quality <00:30:29.230>is <00:30:29.405>the

629
00:30:31.295 --> 00:30:35.335
<00:30:31.295>journey, <00:30:31.495>and <00:30:31.695>the <00:30:31.895>business <00:30:32.095>outcomes <00:30:32.295>are <00:30:32.495>the <00:30:32.695>destination.

630
00:30:35.875 --> 00:30:38.375
<00:30:35.875>So <00:30:36.075>she <00:30:36.275>lights <00:30:36.475>the <00:30:36.675>way <00:30:36.875>for <00:30:37.075>other <00:30:37.275>people

631
00:30:38.515 --> 00:30:41.495
<00:30:38.515>by <00:30:38.670>describing <00:30:38.826>the <00:30:38.981>status <00:30:39.137>of <00:30:39.292>the <00:30:39.448>software <00:30:39.603>in <00:30:39.759>terms <00:30:39.915>of

632
00:30:41.645 --> 00:30:44.335
<00:30:41.645>what <00:30:41.772>it <00:30:41.899>does, <00:30:42.026>how <00:30:42.154>it <00:30:42.281>works, <00:30:42.408>and <00:30:42.535>how <00:30:42.663>it <00:30:42.790>might <00:30:42.917>not <00:30:43.045>work.

633
00:30:44.835 --> 00:30:47.295
<00:30:44.835>She <00:30:45.035>describes <00:30:45.235>her <00:30:45.435>testing <00:30:45.635>activity <00:30:45.835>in <00:30:46.035>terms <00:30:46.235>of

634
00:30:47.485 --> 00:30:50.655
<00:30:47.485>what <00:30:47.640>has <00:30:47.796>been <00:30:47.951>tested, <00:30:48.107>what <00:30:48.262>has <00:30:48.418>not <00:30:48.573>been <00:30:48.729>tested <00:30:48.885>yet,

635
00:30:50.875 --> 00:30:53.255
<00:30:50.875>and <00:30:51.050>what <00:30:51.225>probably <00:30:51.400>won't <00:30:51.575>be <00:30:51.750>tested <00:30:51.925>at <00:30:52.100>all <00:30:52.275>due

636
00:30:53.255 --> 00:30:56.135
<00:30:53.255>to <00:30:53.488>several <00:30:53.721>constraints <00:30:53.955>such <00:30:54.188>as <00:30:54.421>time, <00:30:54.655>resource,

637
00:30:56.135 --> 00:30:57.295
<00:30:56.135>and <00:30:56.657>budget <00:30:57.179>constraints.

638
00:30:58.035 --> 00:31:01.135
<00:30:58.035>She <00:30:58.210>also <00:30:58.385>describes <00:30:58.560>the <00:30:58.735>quality <00:30:58.910>of <00:30:59.085>testing <00:30:59.260>in <00:30:59.435>terms

639
00:31:01.155 --> 00:31:03.615
<00:31:01.155>of <00:31:01.504>the <00:31:01.855>successes <00:31:02.205>she <00:31:02.555>achieved

640
00:31:03.995 --> 00:31:06.375
<00:31:03.995>and <00:31:04.228>the <00:31:04.461>obstacles <00:31:04.695>that <00:31:04.928>impeded <00:31:05.161>her <00:31:05.395>testing.

641
00:31:06.875 --> 00:31:09.455
<00:31:06.875>So <00:31:07.075>she <00:31:07.275>not <00:31:07.475>only, <00:31:07.675>yeah, <00:31:07.875>looks <00:31:08.075>for <00:31:08.275>problems

642
00:31:09.455 --> 00:31:12.415
<00:31:09.455>that <00:31:09.610>threaten <00:31:09.766>the <00:31:09.921>value <00:31:10.077>of <00:31:10.232>the <00:31:10.388>software, <00:31:10.543>she <00:31:10.699>also <00:31:10.855>looks

643
00:31:12.475 --> 00:31:15.215
<00:31:12.475>for <00:31:12.824>problems <00:31:13.175>that <00:31:13.524>threaten <00:31:13.875>testing

644
00:31:15.605 --> 00:31:18.455
<00:31:15.605>because <00:31:15.805>she <00:31:16.005>understands <00:31:16.205>that <00:31:16.405>problems <00:31:16.605>in <00:31:16.805>the <00:31:17.005>process

645
00:31:18.595 --> 00:31:20.735
<00:31:18.595>of <00:31:18.945>looking <00:31:19.295>for <00:31:19.645>problems <00:31:19.995>testing

646
00:31:22.435 --> 00:31:24.655
<00:31:22.435>can <00:31:22.610>prevent <00:31:22.784>her <00:31:22.960>from <00:31:23.135>finding <00:31:23.310>problems <00:31:23.485>in <00:31:23.659>the <00:31:23.835>software.

647
00:31:24.875 --> 00:31:29.015
<00:31:24.875>So <00:31:25.015>in <00:31:25.155>short, <00:31:25.295>she <00:31:25.435>provides <00:31:25.575>a <00:31:25.715>free <00:31:25.855>60 <00:31:25.995>degree <00:31:26.135>narrative <00:31:26.275>on

648
00:31:29.015 --> 00:31:30.775
<00:31:29.015>testing <00:31:29.481>packed <00:31:29.948>with <00:31:30.415>useful

649
00:31:30.795 --> 00:31:33.335
<00:31:30.795>and <00:31:31.261>actionable <00:31:31.728>information <00:31:32.195>instead

650
00:31:33.455 --> 00:31:37.095
<00:31:33.455>of <00:31:33.655>just <00:31:33.855>bombarding <00:31:34.054>her <00:31:34.254>stakeholders <00:31:34.455>with <00:31:34.655>ran <00:31:34.855>metrics

651
00:31:37.445 --> 00:31:38.895
<00:31:37.445>that <00:31:37.771>are <00:31:38.097>packed <00:31:38.423>into <00:31:38.750>shiny,

652
00:31:39.195 --> 00:31:40.975
<00:31:39.195>but <00:31:39.544>you <00:31:39.895>know, <00:31:40.245>meaningless <00:31:40.595>dashboards.

653
00:31:41.595 --> 00:31:44.375
<00:31:41.595>So <00:31:41.750>she <00:31:41.906>acts <00:31:42.061>like <00:31:42.217>a, <00:31:42.372>a <00:31:42.528>storyteller, <00:31:42.683>so <00:31:42.839>to <00:31:42.995>say,

654
00:31:44.595 --> 00:31:47.575
<00:31:44.595>who <00:31:44.828>understands <00:31:45.061>that <00:31:45.295>testing <00:31:45.528>requires <00:31:45.761>doing <00:31:45.995>testing

655
00:31:48.155 --> 00:31:49.575
<00:31:48.155>and <00:31:48.794>describing <00:31:49.433>testing.

656
00:31:50.895 --> 00:31:52.955
<00:31:50.895>And <00:31:51.245>last, <00:31:51.595>Alice <00:31:51.945>also <00:31:52.295>knows

657
00:31:52.955 --> 00:31:56.595
<00:31:52.955>that <00:31:53.095>she <00:31:53.235>cannot <00:31:53.375>assure <00:31:53.514>quality <00:31:53.655>in <00:31:53.794>the <00:31:53.935>same <00:31:54.074>way <00:31:54.215>a <00:31:54.355>doctor

658
00:31:56.855 --> 00:31:58.115
<00:31:56.855>cannot <00:31:57.422>assure <00:31:57.989>health.

659
00:31:58.655 --> 00:32:02.555
<00:31:58.655>She <00:31:58.855>understands <00:31:59.055>that <00:31:59.254>her <00:31:59.455>primary <00:31:59.655>objective, <00:31:59.855>again <00:32:00.055>is

660
00:32:02.555 --> 00:32:04.555
<00:32:02.555>to <00:32:02.905>collect <00:32:03.255>quality <00:32:03.605>related <00:32:03.955>information

661
00:32:04.555 --> 00:32:08.435
<00:32:04.555>to <00:32:04.730>enable <00:32:04.905>other <00:32:05.080>people <00:32:05.255>to <00:32:05.430>make <00:32:05.605>better <00:32:05.780>informed <00:32:05.955>decisions.

662
00:32:09.215 --> 00:32:13.955
<00:32:09.215>So <00:32:09.355>in <00:32:09.495>short, <00:32:09.635>she's <00:32:09.774>aware <00:32:09.915>that <00:32:10.054>she <00:32:10.195>tests, <00:32:10.334>but <00:32:10.475>others <00:32:10.615>decide.

663
00:32:14.895 --> 00:32:18.155
<00:32:14.895>So <00:32:15.070>she <00:32:15.245>sees <00:32:15.420>testing <00:32:15.595>as <00:32:15.770>an <00:32:15.945>information <00:32:16.120>based <00:32:16.295>science

664
00:32:18.455 --> 00:32:22.355
<00:32:18.455>and <00:32:18.629>she <00:32:18.804>understands <00:32:18.980>that <00:32:19.155>she <00:32:19.330>provides <00:32:19.504>value <00:32:19.679>through <00:32:19.855>testing

665
00:32:22.535 --> 00:32:25.715
<00:32:22.535>by <00:32:22.735>helping <00:32:22.935>other <00:32:23.135>people <00:32:23.335>to <00:32:23.535>provide <00:32:23.735>value <00:32:23.935>and

666
00:32:25.715 --> 00:32:28.835
<00:32:25.715>therefore <00:32:25.948>she <00:32:26.181>considers <00:32:26.415>herself <00:32:26.648>as <00:32:26.881>an <00:32:27.115>assistant

667
00:32:29.585 --> 00:32:34.195
<00:32:29.585>that <00:32:29.785>neither <00:32:29.985>does <00:32:30.185>quality <00:32:30.385>assurance, <00:32:30.585>lower <00:32:30.785>quality <00:32:30.985>control,

668
00:32:34.415 --> 00:32:36.435
<00:32:34.415>but <00:32:34.881>rather <00:32:35.348>quality <00:32:35.815>assistance.

669
00:32:37.775 --> 00:32:40.245
<00:32:37.775>Now <00:32:37.950>ladies <00:32:38.125>and <00:32:38.300>gentlemen, <00:32:38.475>this <00:32:38.650>list <00:32:38.825>could <00:32:39.000>go <00:32:39.175>on

670
00:32:40.345 --> 00:32:41.525
<00:32:40.345>and <00:32:40.699>on <00:32:41.053>and <00:32:41.407>on.

671
00:32:41.755 --> 00:32:44.245
<00:32:41.755>It's <00:32:41.988>not <00:32:42.221>complete <00:32:42.455>and <00:32:42.688>it'll <00:32:42.921>never <00:32:43.155>be,

672
00:32:44.745 --> 00:32:46.205
<00:32:44.745>but <00:32:44.964>I <00:32:45.183>hope <00:32:45.401>you <00:32:45.620>get <00:32:45.840>the <00:32:46.059>point.

673
00:32:46.845 --> 00:32:49.725
<00:32:46.845>A <00:32:47.045>list <00:32:47.245>is <00:32:47.445>a <00:32:47.645>cocktail <00:32:47.845>of <00:32:48.045>testing <00:32:48.245>excellence

674
00:32:50.035 --> 00:32:53.205
<00:32:50.035>because <00:32:50.315>her <00:32:50.595>testing <00:32:50.875>is <00:32:51.155>fast, <00:32:51.435>inexpensive,

675
00:32:53.685 --> 00:32:54.925
<00:32:53.685>credible <00:32:54.243>and <00:32:54.801>accountable.

676
00:32:56.165 --> 00:32:59.825
<00:32:56.165>Now, <00:32:56.281>if <00:32:56.398>that <00:32:56.514>was <00:32:56.631>a <00:32:56.748>little <00:32:56.865>bit <00:32:56.981>too <00:32:57.098>fast <00:32:57.215>for <00:32:57.331>you, <00:32:57.448>no <00:32:57.565>worries,

677
00:33:00.285 --> 00:33:02.145
<00:33:00.285>you <00:33:00.518>can <00:33:00.751>find <00:33:00.985>all <00:33:01.218>the <00:33:01.451>details <00:33:01.685>of

678
00:33:02.145 --> 00:33:05.065
<00:33:02.145>what <00:33:02.320>I've <00:33:02.495>just <00:33:02.670>said <00:33:02.845>in <00:33:03.020>my <00:33:03.195>recent <00:33:03.370>article <00:33:03.545>on

679
00:33:05.065 --> 00:33:06.105
<00:33:05.065>my <00:33:05.533>LinkedIn <00:33:06.001>profile.

680
00:33:06.525 --> 00:33:07.945
<00:33:06.525>And <00:33:06.780>this <00:33:07.036>article <00:33:07.291>is <00:33:07.547>simply <00:33:07.803>called

681
00:33:07.945 --> 00:33:09.105
<00:33:07.945>Excellence <00:33:08.466>software <00:33:08.989>testing.

682
00:33:09.645 --> 00:33:11.985
<00:33:09.645>And <00:33:09.800>feel <00:33:09.956>free <00:33:10.111>to <00:33:10.267>use <00:33:10.422>this <00:33:10.578>article <00:33:10.733>as <00:33:10.889>a <00:33:11.045>basis,

683
00:33:12.045 --> 00:33:16.145
<00:33:12.045>as <00:33:12.220>an <00:33:12.395>inspiration, <00:33:12.570>um, <00:33:12.745>to <00:33:12.920>work <00:33:13.095>out <00:33:13.270>your <00:33:13.445>notion

684
00:33:16.205 --> 00:33:17.425
<00:33:16.205>of <00:33:16.754>testing <00:33:17.303>excellence.

685
00:33:18.125 --> 00:33:19.865
<00:33:18.125>Now, <00:33:18.280>here's <00:33:18.436>a <00:33:18.591>little <00:33:18.747>secret <00:33:18.902>I <00:33:19.058>want <00:33:19.213>to <00:33:19.369>tell <00:33:19.525>you,

686
00:33:19.865 --> 00:33:21.465
<00:33:19.865>and <00:33:20.065>you <00:33:20.265>might <00:33:20.465>have <00:33:20.665>already <00:33:20.865>guessed <00:33:21.065>it, <00:33:21.265>right?

687
00:33:21.885 --> 00:33:25.225
<00:33:21.885>The <00:33:22.025>picture <00:33:22.165>of <00:33:22.305>Alice, <00:33:22.445>uh, <00:33:22.585>you <00:33:22.725>are <00:33:22.865>currently <00:33:23.004>looking <00:33:23.145>at, <00:33:23.285>uh,

688
00:33:25.235 --> 00:33:28.785
<00:33:25.235>shows <00:33:25.409>my <00:33:25.584>sister <00:33:25.760>and <00:33:25.935>my <00:33:26.110>sister <00:33:26.284>has <00:33:26.459>absolutely <00:33:26.635>nothing

689
00:33:28.925 --> 00:33:30.185
<00:33:28.925>to <00:33:29.208>do <00:33:29.492>with <00:33:29.775>software <00:33:30.059>testing.

690
00:33:31.165 --> 00:33:34.425
<00:33:31.165>So <00:33:31.292>what <00:33:31.419>I <00:33:31.546>wanna <00:33:31.674>say <00:33:31.801>by <00:33:31.928>that <00:33:32.055>is <00:33:32.183>that <00:33:32.310>Alice <00:33:32.437>is <00:33:32.565>real,

691
00:33:35.045 --> 00:33:36.905
<00:33:35.045>but <00:33:35.325>only <00:33:35.605>in <00:33:35.885>a <00:33:36.165>methodically <00:33:36.445>sense.

692
00:33:37.365 --> 00:33:39.785
<00:33:37.365>So <00:33:37.645>Alice <00:33:37.925>is <00:33:38.205>a <00:33:38.485>fictional <00:33:38.765>character

693
00:33:40.015 --> 00:33:44.145
<00:33:40.015>that <00:33:40.248>symbolizes <00:33:40.481>the <00:33:40.715>collective, <00:33:40.948>uh, <00:33:41.181>testing <00:33:41.415>excellence

694
00:33:44.445 --> 00:33:47.465
<00:33:44.445>of <00:33:44.794>a <00:33:45.145>large, <00:33:45.495>inspiring <00:33:45.845>and,

695
00:33:47.465 --> 00:33:50.745
<00:33:47.465>and <00:33:47.665>diverse <00:33:47.865>group <00:33:48.064>of <00:33:48.264>excellence <00:33:48.465>software <00:33:48.665>testers <00:33:48.865>with

696
00:33:51.335 --> 00:33:53.865
<00:33:51.335>whom <00:33:51.568>I <00:33:51.801>had <00:33:52.035>the <00:33:52.268>pleasure, <00:33:52.501>the <00:33:52.735>honor

697
00:33:54.405 --> 00:33:57.825
<00:33:54.405>and, <00:33:54.532>uh, <00:33:54.659>the <00:33:54.786>privilege <00:33:54.914>to <00:33:55.041>work <00:33:55.168>with <00:33:55.295>in <00:33:55.423>the <00:33:55.550>last <00:33:55.677>10 <00:33:55.805>years.

698
00:33:59.105 --> 00:34:01.285
<00:33:59.105>Now <00:33:59.385>finally, <00:33:59.665>let <00:33:59.945>me <00:34:00.225>highlight <00:34:00.505>that

699
00:34:01.795 --> 00:34:04.765
<00:34:01.795>this <00:34:02.028>short <00:34:02.261>story <00:34:02.495>stands <00:34:02.728>on <00:34:02.961>the <00:34:03.195>shoulders

700
00:34:04.945 --> 00:34:07.165
<00:34:04.945>of <00:34:05.178>these <00:34:05.411>giants <00:34:05.645>listed <00:34:05.878>on <00:34:06.111>that <00:34:06.345>slide.

701
00:34:07.665 --> 00:34:10.565
<00:34:07.665>So <00:34:07.898>ladies <00:34:08.131>and <00:34:08.364>gentlemen, <00:34:08.598>don't <00:34:08.831>follow <00:34:09.065>me.

702
00:34:10.985 --> 00:34:12.405
<00:34:10.985>Follow <00:34:11.624>these <00:34:12.263>people

703
00:34:12.405 --> 00:34:15.565
<00:34:12.405>because <00:34:12.580>they <00:34:12.755>know <00:34:12.930>the <00:34:13.105>way <00:34:13.280>they <00:34:13.455>go <00:34:13.630>the <00:34:13.805>way

704
00:34:16.145 --> 00:34:19.605
<00:34:16.145>and <00:34:16.320>they <00:34:16.495>show <00:34:16.670>the <00:34:16.844>way <00:34:17.020>of <00:34:17.195>excellent <00:34:17.370>software <00:34:17.545>testers.

705
00:34:20.745 --> 00:34:21.925
<00:34:20.745>And <00:34:21.099>that's <00:34:21.453>it. <00:34:21.807>That's,

706
00:34:22.025 --> 00:34:24.365
<00:34:22.025>that's <00:34:22.225>all <00:34:22.425>I <00:34:22.625>wanted <00:34:22.825>to <00:34:23.025>share <00:34:23.225>with <00:34:23.425>you.

707
00:34:24.465 --> 00:34:26.845
<00:34:24.465>So <00:34:24.698>thank <00:34:24.931>you <00:34:25.165>again <00:34:25.398>for, <00:34:25.631>for <00:34:25.865>joining.

708
00:34:27.545 --> 00:34:29.365
<00:34:27.545>Um, <00:34:27.778>I <00:34:28.011>hope <00:34:28.245>you <00:34:28.478>enjoyed <00:34:28.711>the <00:34:28.945>session.

709
00:34:29.705 --> 00:34:33.085
<00:34:29.705>I'm <00:34:29.880>now <00:34:30.054>handing <00:34:30.230>a <00:34:30.404>pack <00:34:30.580>to <00:34:30.755>my <00:34:30.929>friend <00:34:31.105>Stefan

710
00:34:33.145 --> 00:34:34.565
<00:34:33.145>to <00:34:33.400>open <00:34:33.656>the <00:34:33.911>q <00:34:34.167>and <00:34:34.423>a.

711
00:34:34.985 --> 00:34:36.645
<00:34:34.985>Stefan, <00:34:35.335>are <00:34:35.685>you <00:34:36.035>ready <00:34:36.385>to.