WEBVTT

1
00:00:02.975 --> 00:00:05.115
<00:00:02.975>Uh, <00:00:03.150>well, <00:00:03.325>uh, <00:00:03.500>good <00:00:03.675>morning <00:00:03.850>also <00:00:04.025>from <00:00:04.200>my <00:00:04.375>side,

2
00:00:05.115 --> 00:00:07.515
<00:00:05.115>and <00:00:05.290>welcome <00:00:05.465>to <00:00:05.640>this <00:00:05.815>talk <00:00:05.990>about <00:00:06.165>effective <00:00:06.340>machine <00:00:06.515>learning,

3
00:00:08.435 --> 00:00:10.115
<00:00:08.435>managing <00:00:08.901>complexity <00:00:09.368>with, <00:00:09.835>uh,

4
00:00:10.115 --> 00:00:12.195
<00:00:10.115>machine <00:00:10.465>learning <00:00:10.815>operations, <00:00:11.165>ML <00:00:11.515>ops.

5
00:00:12.975 --> 00:00:15.515
<00:00:12.975>My <00:00:13.150>name, <00:00:13.325>as <00:00:13.500>you've <00:00:13.674>already <00:00:13.850>heard, <00:00:14.024>is <00:00:14.200>Simon <00:00:14.375>Steena.

6
00:00:15.775 --> 00:00:17.715
<00:00:15.775>Uh, <00:00:15.975>I <00:00:16.175>do <00:00:16.375>must <00:00:16.575>correct <00:00:16.775>Stefan <00:00:16.975>a <00:00:17.175>bit.

7
00:00:17.915 --> 00:00:20.915
<00:00:17.915>I <00:00:18.090>am, <00:00:18.265>uh, <00:00:18.439>now <00:00:18.615>at <00:00:18.790>the <00:00:18.965>Transaction <00:00:19.140>Monitor <00:00:19.314>Netherlands

8
00:00:20.915 --> 00:00:23.115
<00:00:20.915>and <00:00:21.148>leading <00:00:21.381>the <00:00:21.615>ML <00:00:21.848>ops <00:00:22.081>team <00:00:22.314>there.

9
00:00:23.215 --> 00:00:26.195
<00:00:23.215>So <00:00:23.370>I'm <00:00:23.526>lead <00:00:23.681>machine <00:00:23.837>learning <00:00:23.992>engineer <00:00:24.148>at <00:00:24.303>a <00:00:24.459>fantastic <00:00:24.615>company

10
00:00:26.195 --> 00:00:28.875
<00:00:26.195>that's <00:00:26.428>devoted <00:00:26.661>to <00:00:26.895>battling <00:00:27.128>money <00:00:27.361>laundering <00:00:27.595>at

11
00:00:28.875 --> 00:00:29.955
<00:00:28.875>the <00:00:29.361>global <00:00:29.846>scale.

12
00:00:29.975 --> 00:00:32.795
<00:00:29.975>And <00:00:30.115>next <00:00:30.255>to <00:00:30.395>that, <00:00:30.535>while <00:00:30.675>I'm <00:00:30.815>still <00:00:30.955>teaching <00:00:31.095>at <00:00:31.235>the <00:00:31.375>University

13
00:00:32.795 --> 00:00:34.835
<00:00:32.795>of <00:00:33.261>Applied <00:00:33.728>Sciences, <00:00:34.195>Vienna.

14
00:00:35.435 --> 00:00:37.535
<00:00:35.435>Anyway, <00:00:35.668>in <00:00:35.901>the <00:00:36.135>next <00:00:36.368>45 <00:00:36.601>minutes, <00:00:36.835>roughly,

15
00:00:37.995 --> 00:00:39.735
<00:00:37.995>we <00:00:38.228>will <00:00:38.461>dive <00:00:38.695>deep <00:00:38.928>into <00:00:39.161>the <00:00:39.394>topic

16
00:00:40.315 --> 00:00:42.535
<00:00:40.315>of <00:00:40.595>machine <00:00:40.875>learning <00:00:41.155>operations, <00:00:41.434>ML <00:00:41.714>ops.

17
00:00:42.535 --> 00:00:45.895
<00:00:42.535>We <00:00:42.690>will <00:00:42.846>start <00:00:43.001>off, <00:00:43.157>but <00:00:43.312>first <00:00:43.468>building <00:00:43.623>an <00:00:43.779>understanding <00:00:43.934>why

18
00:00:45.895 --> 00:00:47.775
<00:00:45.895>machine <00:00:46.245>learning <00:00:46.595>operations <00:00:46.945>is <00:00:47.295>actually

19
00:00:47.775 --> 00:00:48.975
<00:00:47.775>important <00:00:48.045>and <00:00:48.315>what <00:00:48.585>it <00:00:48.855>is.

20
00:00:49.455 --> 00:00:52.135
<00:00:49.455>Subsequently, <00:00:49.655>we'll <00:00:49.855>dive <00:00:50.055>deep <00:00:50.254>into <00:00:50.455>the <00:00:50.655>field, <00:00:50.855>uh,

21
00:00:52.135 --> 00:00:53.135
<00:00:52.135>using <00:00:52.434>a <00:00:52.735>case <00:00:53.035>study,

22
00:00:53.155 --> 00:00:56.695
<00:00:53.155>and <00:00:53.282>in <00:00:53.409>that <00:00:53.536>case <00:00:53.664>study, <00:00:53.791>we <00:00:53.918>will <00:00:54.045>learn <00:00:54.173>how <00:00:54.300>we <00:00:54.427>can <00:00:54.555>lift

23
00:00:57.715 --> 00:01:00.255
<00:00:57.715>our <00:00:57.995>machine <00:00:58.275>learning <00:00:58.555>development <00:00:58.835>process, <00:00:59.115>um,

24
00:01:00.435 --> 00:01:03.415
<00:01:00.435>to <00:01:00.668>the <00:01:00.901>next <00:01:01.135>level <00:01:01.368>by <00:01:01.601>introducing <00:01:01.835>machine,

25
00:01:03.415 --> 00:01:06.895
<00:01:03.415>machine <00:01:03.695>learning <00:01:03.975>operations, <00:01:04.254>principles <00:01:04.535>and <00:01:04.815>tooling.

26
00:01:07.195 --> 00:01:08.535
<00:01:07.195>But <00:01:07.395>before <00:01:07.597>we <00:01:07.798>do <00:01:07.999>that, <00:01:08.199>we <00:01:08.401>need

27
00:01:08.535 --> 00:01:11.655
<00:01:08.535>to <00:01:08.675>travel <00:01:08.815>back <00:01:08.955>in <00:01:09.095>time <00:01:09.235>a <00:01:09.375>bit, <00:01:09.515>uh, <00:01:09.655>to <00:01:09.795>be <00:01:09.935>precise.

28
00:01:11.885 --> 00:01:14.815
<00:01:11.885>Well, <00:01:12.118>nine <00:01:12.351>years, <00:01:12.585>almost <00:01:12.818>a <00:01:13.051>decade <00:01:13.285>ago,

29
00:01:15.245 --> 00:01:18.335
<00:01:15.245>Harvard <00:01:15.445>Business <00:01:15.645>Review <00:01:15.845>published <00:01:16.045>an <00:01:16.245>article <00:01:16.445>titling <00:01:16.645>data

30
00:01:18.615 --> 00:01:20.415
<00:01:18.615>scientist, <00:01:18.815>the <00:01:19.015>sexiest <00:01:19.214>job <00:01:19.414>of <00:01:19.615>the <00:01:19.815>21st <00:01:20.015>century.

31
00:01:20.785 --> 00:01:24.175
<00:01:20.785>Right? <00:01:20.940>And <00:01:21.096>I'm <00:01:21.251>quite <00:01:21.407>sure <00:01:21.562>that <00:01:21.718>you <00:01:21.873>have <00:01:22.029>heard <00:01:22.185>that,

32
00:01:24.675 --> 00:01:25.855
<00:01:24.675>uh, <00:01:25.206>that <00:01:25.737>sentence.

33
00:01:26.645 --> 00:01:29.815
<00:01:26.645>Well, <00:01:26.800>what's <00:01:26.956>fascinating <00:01:27.111>about <00:01:27.267>this <00:01:27.422>is <00:01:27.578>looking <00:01:27.733>at <00:01:27.889>it <00:01:28.045>from,

34
00:01:30.085 --> 00:01:33.895
<00:01:30.085>from <00:01:30.225>the <00:01:30.365>year <00:01:30.505>2021, <00:01:30.645>is <00:01:30.785>that <00:01:30.925>a <00:01:31.065>10 <00:01:31.205>years <00:01:31.345>have <00:01:31.485>passed,

35
00:01:33.895 --> 00:01:35.295
<00:01:33.895>that <00:01:34.315>decade <00:01:34.735>has <00:01:35.155>passed,

36
00:01:35.435 --> 00:01:38.415
<00:01:35.435>and, <00:01:35.635>you <00:01:35.835>know, <00:01:36.035>in <00:01:36.235>tech <00:01:36.435>a <00:01:36.635>decade <00:01:36.835>that's,

37
00:01:38.415 --> 00:01:39.535
<00:01:38.414>that's <00:01:38.919>almost <00:01:39.423>eternity.

38
00:01:39.775 --> 00:01:40.975
<00:01:39.775>A <00:01:40.135>lot <00:01:40.495>happens <00:01:40.854>here.

39
00:01:40.995 --> 00:01:43.135
<00:01:40.995>And <00:01:41.170>this <00:01:41.345>really <00:01:41.520>shows <00:01:41.695>that <00:01:41.870>data <00:01:42.045>science, <00:01:42.220>machine <00:01:42.395>learning,

40
00:01:43.155 --> 00:01:46.055
<00:01:43.155>you <00:01:43.282>know, <00:01:43.409>they <00:01:43.536>are <00:01:43.664>not <00:01:43.791>the <00:01:43.918>new <00:01:44.045>kid <00:01:44.173>on <00:01:44.300>the <00:01:44.427>block <00:01:44.555>anymore.

41
00:01:46.085 --> 00:01:48.375
<00:01:46.085>They <00:01:46.285>have <00:01:46.485>been <00:01:46.685>around <00:01:46.885>for <00:01:47.085>quite <00:01:47.285>a <00:01:47.485>while.

42
00:01:48.845 --> 00:01:51.415
<00:01:48.845>Fortunately, <00:01:49.000>also, <00:01:49.156>a <00:01:49.311>lot <00:01:49.467>has <00:01:49.622>happened <00:01:49.778>in <00:01:49.933>these <00:01:50.089>10 <00:01:50.245>years,

43
00:01:52.195 --> 00:01:54.175
<00:01:52.195>for <00:01:52.544>example, <00:01:52.895>looking <00:01:53.244>at <00:01:53.595>education.

44
00:01:54.715 --> 00:01:57.455
<00:01:54.715>Um, <00:01:54.890>universities <00:01:55.065>have <00:01:55.240>been <00:01:55.415>ramping <00:01:55.590>up <00:01:55.765>their <00:01:55.940>study <00:01:56.115>program,

45
00:01:57.955 --> 00:02:00.455
<00:01:57.955>um, <00:01:58.304>offering, <00:01:58.655>um, <00:01:59.005>really <00:01:59.355>bachelor

46
00:02:00.455 --> 00:02:02.175
<00:02:00.455>and <00:02:00.735>master <00:02:01.015>programs <00:02:01.295>in <00:02:01.575>machine <00:02:01.855>learning

47
00:02:02.175 --> 00:02:04.735
<00:02:02.175>and <00:02:02.350>data <00:02:02.524>science, <00:02:02.700>so <00:02:02.875>they <00:02:03.050>can <00:02:03.225>really <00:02:03.399>output <00:02:03.575>them,

48
00:02:04.735 --> 00:02:07.695
<00:02:04.735>highly <00:02:04.910>skilled <00:02:05.085>people <00:02:05.260>that <00:02:05.435>know <00:02:05.610>how <00:02:05.785>the <00:02:05.960>field <00:02:06.135>works.

49
00:02:08.405 --> 00:02:10.895
<00:02:08.405>Also, <00:02:08.560>on <00:02:08.716>top <00:02:08.871>of <00:02:09.027>that, <00:02:09.182>Python <00:02:09.338>has <00:02:09.493>become <00:02:09.649>the <00:02:09.805>lingo

50
00:02:10.915 --> 00:02:12.135
<00:02:10.914>franca <00:02:11.281>of <00:02:11.647>data <00:02:12.012>science.

51
00:02:12.475 --> 00:02:14.415
<00:02:12.475>And <00:02:12.755>within <00:02:13.035>Python, <00:02:13.315>an <00:02:13.595>incredible <00:02:13.875>tooling

52
00:02:14.415 --> 00:02:15.535
<00:02:14.415>ecosystem <00:02:14.918>has <00:02:15.423>emerged.

53
00:02:15.535 --> 00:02:16.855
<00:02:15.535>Yeah, <00:02:15.733>naming <00:02:15.930>just <00:02:16.129>a <00:02:16.327>few <00:02:16.524>of <00:02:16.722>them,

54
00:02:17.195 --> 00:02:19.375
<00:02:17.195>I'm <00:02:17.370>sure <00:02:17.545>you're <00:02:17.720>quite <00:02:17.894>familiar <00:02:18.070>with, <00:02:18.245>with <00:02:18.420>quite <00:02:18.595>some

55
00:02:19.375 --> 00:02:21.695
<00:02:19.375>of <00:02:19.608>these <00:02:19.841>packages <00:02:20.075>such <00:02:20.308>as <00:02:20.541>TensorFlow, <00:02:20.775>psychic,

56
00:02:21.695 --> 00:02:23.095
<00:02:21.695>learn, <00:02:22.010>PyTorch <00:02:22.325>and <00:02:22.640>so <00:02:22.954>on.

57
00:02:24.125 --> 00:02:26.655
<00:02:24.125>Last, <00:02:24.405>but <00:02:24.685>definitely <00:02:24.965>not <00:02:25.245>least, <00:02:25.525>also,

58
00:02:27.055 --> 00:02:28.735
<00:02:27.055>business <00:02:27.405>adoption <00:02:27.755>has <00:02:28.105>grown <00:02:28.455>tremendously.

59
00:02:28.965 --> 00:02:32.575
<00:02:28.965>According <00:02:29.120>to <00:02:29.276>a <00:02:29.431>McKinsey <00:02:29.587>survey <00:02:29.742>from <00:02:29.898>last <00:02:30.053>year, <00:02:30.209>one <00:02:30.365>out

60
00:02:32.575 --> 00:02:34.855
<00:02:32.575>of <00:02:32.855>two <00:02:33.135>organizations <00:02:33.415>they <00:02:33.695>surveyed <00:02:33.975>said

61
00:02:34.855 --> 00:02:36.495
<00:02:34.855>that <00:02:35.204>they <00:02:35.554>implemented <00:02:35.905>machine <00:02:36.255>learning

62
00:02:36.495 --> 00:02:38.495
<00:02:36.495>and <00:02:36.670>data <00:02:36.845>science <00:02:37.020>in <00:02:37.195>at <00:02:37.370>least <00:02:37.545>one <00:02:37.720>business <00:02:37.895>function.

63
00:02:38.795 --> 00:02:40.415
<00:02:38.795>And <00:02:39.145>that's <00:02:39.495>really <00:02:39.845>a <00:02:40.195>lot.

64
00:02:40.435 --> 00:02:43.535
<00:02:40.435>That's <00:02:40.635>a <00:02:40.835>tremendous <00:02:41.035>increase <00:02:41.235>over <00:02:41.435>the <00:02:41.635>last <00:02:41.835>years.

65
00:02:44.995 --> 00:02:47.495
<00:02:44.995>So <00:02:45.228>looking <00:02:45.461>at <00:02:45.695>these <00:02:45.928>three <00:02:46.161>points, <00:02:46.395>right,

66
00:02:47.495 --> 00:02:48.655
<00:02:47.495>we <00:02:47.843>have <00:02:48.191>skilled <00:02:48.539>professionals,

67
00:02:48.755 --> 00:02:51.735
<00:02:48.755>we <00:02:48.894>have <00:02:49.035>a <00:02:49.174>mature <00:02:49.315>tooling <00:02:49.454>ecosystem, <00:02:49.595>at <00:02:49.734>least <00:02:49.875>when <00:02:50.015>it <00:02:50.155>comes

68
00:02:51.735 --> 00:02:56.215
<00:02:51.735>to <00:02:51.910>data <00:02:52.085>modeling <00:02:52.260>and <00:02:52.435>data <00:02:52.610>processing <00:02:52.785>and <00:02:52.960>machine <00:02:53.135>learning

69
00:02:56.215 --> 00:02:58.055
<00:02:56.215>and <00:02:56.448>data <00:02:56.681>sciences <00:02:56.915>also <00:02:57.148>applied <00:02:57.381>in <00:02:57.615>practice.

70
00:02:58.055 --> 00:02:59.605
<00:02:58.055>You <00:02:58.287>know, <00:02:58.520>you <00:02:58.752>could <00:02:58.984>really <00:02:59.217>think <00:02:59.450>that

71
00:03:00.515 --> 00:03:02.165
<00:03:00.515>this <00:03:00.795>field <00:03:01.075>has <00:03:01.355>grown <00:03:01.635>up, <00:03:01.915>yeah,

72
00:03:02.265 --> 00:03:03.765
<00:03:02.265>in <00:03:02.433>the <00:03:02.602>last <00:03:02.771>10 <00:03:02.940>years, <00:03:03.108>so <00:03:03.277>much <00:03:03.446>has <00:03:03.614>happened.

73
00:03:03.765 --> 00:03:08.605
<00:03:03.765>It's <00:03:03.904>a <00:03:04.045>grownup <00:03:04.184>field <00:03:04.325>Now, <00:03:04.464>however, <00:03:04.605>on <00:03:04.744>the <00:03:04.885>other <00:03:05.024>hand, <00:03:05.165>still,

74
00:03:08.705 --> 00:03:10.805
<00:03:08.705>and <00:03:08.832>you <00:03:08.959>have <00:03:09.086>probably <00:03:09.214>heard <00:03:09.341>that <00:03:09.468>a <00:03:09.595>lot <00:03:09.723>of <00:03:09.850>times <00:03:09.977>as <00:03:10.105>well,

75
00:03:10.805 --> 00:03:12.245
<00:03:10.805>or <00:03:11.021>perhaps <00:03:11.237>even <00:03:11.453>your, <00:03:11.669>you <00:03:11.885>can <00:03:12.101>see

76
00:03:12.245 --> 00:03:15.845
<00:03:12.245>that <00:03:12.420>in <00:03:12.595>your <00:03:12.770>own <00:03:12.945>organization <00:03:13.120>still <00:03:13.295>an <00:03:13.470>incredibly <00:03:13.645>large

77
00:03:15.895 --> 00:03:20.165
<00:03:15.895>share <00:03:16.070>of <00:03:16.245>organizations, <00:03:16.420>um, <00:03:16.595>fail <00:03:16.770>to <00:03:16.945>get <00:03:17.120>value <00:03:17.295>out

78
00:03:20.165 --> 00:03:21.485
<00:03:20.165>of <00:03:20.462>the <00:03:20.759>machine <00:03:21.056>learning <00:03:21.353>projects

79
00:03:21.785 --> 00:03:25.085
<00:03:21.785>or <00:03:21.940>even <00:03:22.096>a <00:03:22.251>lot <00:03:22.407>of <00:03:22.562>machine <00:03:22.718>learning <00:03:22.873>projects <00:03:23.029>just <00:03:23.185>fail

80
00:03:25.345 --> 00:03:27.565
<00:03:25.345>before <00:03:25.520>they <00:03:25.695>even <00:03:25.870>get <00:03:26.045>the <00:03:26.220>chance <00:03:26.395>to <00:03:26.570>deliver <00:03:26.745>value.

81
00:03:28.545 --> 00:03:30.955
<00:03:28.545>Yeah. <00:03:28.778>And <00:03:29.011>you <00:03:29.245>know, <00:03:29.478>looking <00:03:29.711>at <00:03:29.945>the,

82
00:03:30.955 --> 00:03:32.275
<00:03:30.954>looking <00:03:31.153>at <00:03:31.351>this <00:03:31.549>slide, <00:03:31.747>that's <00:03:31.945>quite <00:03:32.143>weird.

83
00:03:32.335 --> 00:03:33.555
<00:03:32.335>On <00:03:32.518>the <00:03:32.701>left <00:03:32.884>hand <00:03:33.067>side, <00:03:33.250>we <00:03:33.433>seem

84
00:03:33.555 --> 00:03:35.515
<00:03:33.555>to <00:03:33.835>have <00:03:34.115>a <00:03:34.395>very <00:03:34.675>solid <00:03:34.955>foundation

85
00:03:36.255 --> 00:03:38.715
<00:03:36.255>for <00:03:36.454>actually <00:03:36.655>getting <00:03:36.855>value <00:03:37.055>out <00:03:37.255>of <00:03:37.454>machine <00:03:37.655>learning.

86
00:03:39.135 --> 00:03:41.915
<00:03:39.135>But <00:03:39.274>then <00:03:39.415>on, <00:03:39.554>on <00:03:39.695>the <00:03:39.834>right <00:03:39.975>hand <00:03:40.114>side, <00:03:40.255>um, <00:03:40.394>we <00:03:40.535>see

87
00:03:41.915 --> 00:03:43.915
<00:03:41.915>that <00:03:42.381>somehow <00:03:42.848>organizations <00:03:43.315>do,

88
00:03:44.015 --> 00:03:46.515
<00:03:44.015>do <00:03:44.214>often <00:03:44.415>do <00:03:44.614>not <00:03:44.815>manage <00:03:45.015>to <00:03:45.214>do <00:03:45.415>so.

89
00:03:47.055 --> 00:03:48.555
<00:03:47.055>So <00:03:47.325>of <00:03:47.595>course <00:03:47.865>the <00:03:48.135>question <00:03:48.405>arises,

90
00:03:48.665 --> 00:03:50.355
<00:03:48.665>what <00:03:49.015>is <00:03:49.364>the <00:03:49.715>problem <00:03:50.065>actually?

91
00:03:50.385 --> 00:03:52.755
<00:03:50.385>What <00:03:50.584>is <00:03:50.785>the <00:03:50.984>issue <00:03:51.185>we <00:03:51.385>are <00:03:51.584>facing <00:03:51.785>here?

92
00:03:53.925 --> 00:03:56.945
<00:03:53.925>And <00:03:54.065>to <00:03:54.205>be, <00:03:54.345>to <00:03:54.485>be <00:03:54.625>fair, <00:03:54.765>well, <00:03:54.905>there <00:03:55.045>is <00:03:55.185>not <00:03:55.325>only,

93
00:03:57.115 --> 00:03:59.705
<00:03:57.115>there <00:03:57.315>is <00:03:57.515>not <00:03:57.715>only <00:03:57.915>one <00:03:58.115>issue, <00:03:58.315>right <00:03:58.515>there,

94
00:03:59.705 --> 00:04:01.185
<00:03:59.704>there <00:04:00.149>are <00:04:00.593>multiple, <00:04:01.037>uh,

95
00:04:01.185 --> 00:04:04.105
<00:04:01.185>but <00:04:01.301>one <00:04:01.418>of <00:04:01.535>the <00:04:01.651>big <00:04:01.768>ones, <00:04:01.885>one <00:04:02.001>of <00:04:02.118>the <00:04:02.235>really <00:04:02.351>big, <00:04:02.468>big <00:04:02.585>problems

96
00:04:04.135 --> 00:04:06.865
<00:04:04.135>that, <00:04:04.334>that <00:04:04.535>holds <00:04:04.734>organizations <00:04:04.935>from <00:04:05.135>gaining <00:04:05.334>value <00:04:05.535>out

97
00:04:06.865 --> 00:04:07.905
<00:04:06.865>of <00:04:07.177>machine <00:04:07.489>learning <00:04:07.801>is

98
00:04:07.975 --> 00:04:10.745
<00:04:07.975>that <00:04:08.208>when <00:04:08.441>organizations <00:04:08.674>think <00:04:08.908>about <00:04:09.141>machine <00:04:09.375>learning,

99
00:04:10.745 --> 00:04:14.305
<00:04:10.745>what <00:04:10.920>they <00:04:11.095>usually <00:04:11.270>think <00:04:11.445>about <00:04:11.620>looks <00:04:11.795>like <00:04:11.970>this, <00:04:12.145>right?

100
00:04:14.375 --> 00:04:16.545
<00:04:14.375>It's, <00:04:14.550>they, <00:04:14.725>they, <00:04:14.900>they <00:04:15.075>think <00:04:15.250>they <00:04:15.425>hire <00:04:15.600>data <00:04:15.775>scientist

101
00:04:16.805 --> 00:04:17.865
<00:04:16.805>and <00:04:17.043>they <00:04:17.281>build <00:04:17.520>a <00:04:17.759>model.

102
00:04:18.245 --> 00:04:20.505
<00:04:18.245>The <00:04:18.445>data <00:04:18.644>scientist, <00:04:18.845>you <00:04:19.045>know, <00:04:19.245>does <00:04:19.445>some <00:04:19.644>experimentation.

103
00:04:20.525 --> 00:04:22.345
<00:04:20.524>He <00:04:20.875>evaluates <00:04:21.224>the <00:04:21.575>models <00:04:21.924>well,

104
00:04:22.645 --> 00:04:25.465
<00:04:22.645>and <00:04:22.878>then <00:04:23.111>the <00:04:23.344>problem <00:04:23.578>is <00:04:23.811>solved <00:04:24.044>once

105
00:04:25.465 --> 00:04:27.305
<00:04:25.465>that <00:04:25.744>model <00:04:26.025>hits <00:04:26.304>some <00:04:26.585>accuracy <00:04:26.864>goal.

106
00:04:28.845 --> 00:04:32.255
<00:04:28.845>Well, <00:04:28.985>to <00:04:29.125>be <00:04:29.265>fair, <00:04:29.405>there <00:04:29.545>is <00:04:29.685>at <00:04:29.825>least <00:04:29.965>one <00:04:30.105>problem <00:04:30.245>solved,

107
00:04:32.265 --> 00:04:34.255
<00:04:32.265>which <00:04:32.498>could <00:04:32.731>be <00:04:32.965>the <00:04:33.198>modeling <00:04:33.431>problem, <00:04:33.664>right?

108
00:04:34.275 --> 00:04:37.255
<00:04:34.275>If <00:04:34.414>you <00:04:34.554>have <00:04:34.695>a <00:04:34.835>model <00:04:34.974>that's <00:04:35.114>that's <00:04:35.255>efficiently <00:04:35.395>good, <00:04:35.534>you <00:04:35.674>can,

109
00:04:37.255 --> 00:04:38.415
<00:04:37.255>you <00:04:37.516>can <00:04:37.777>consider <00:04:38.038>that <00:04:38.299>solved.

110
00:04:38.445 --> 00:04:42.455
<00:04:38.445>However, <00:04:38.620>definitely <00:04:38.795>your <00:04:38.969>business <00:04:39.145>problem <00:04:39.320>is <00:04:39.495>not <00:04:39.670>solved <00:04:39.844>in

111
00:04:42.455 --> 00:04:45.055
<00:04:42.455>order <00:04:42.655>to <00:04:42.854>really <00:04:43.055>solve <00:04:43.255>your <00:04:43.455>business <00:04:43.655>problem <00:04:43.854>and

112
00:04:45.055 --> 00:04:47.175
<00:04:45.055>therefore <00:04:45.405>get <00:04:45.755>business <00:04:46.105>value <00:04:46.455>out

113
00:04:47.175 --> 00:04:48.375
<00:04:47.175>of <00:04:47.445>your <00:04:47.715>machine <00:04:47.985>learning <00:04:48.255>model.

114
00:04:48.605 --> 00:04:51.575
<00:04:48.605>Well, <00:04:48.885>you <00:04:49.165>need <00:04:49.445>to <00:04:49.725>go <00:04:50.005>further.

115
00:04:51.795 --> 00:04:53.575
<00:04:51.795>You <00:04:51.995>need <00:04:52.195>to <00:04:52.395>take <00:04:52.595>that <00:04:52.795>model <00:04:52.995>to <00:04:53.195>production,

116
00:04:54.395 --> 00:04:55.855
<00:04:54.395>and <00:04:54.614>that <00:04:54.832>involves <00:04:55.052>a <00:04:55.271>few <00:04:55.490>more <00:04:55.709>steps

117
00:04:56.035 --> 00:04:57.175
<00:04:56.034>to <00:04:56.376>productionize <00:04:56.719>your <00:04:57.061>model.

118
00:04:57.595 --> 00:05:00.175
<00:04:57.595>You <00:04:57.770>need <00:04:57.945>to <00:04:58.120>test <00:04:58.295>your <00:04:58.470>entire <00:04:58.645>machine <00:04:58.820>learning <00:04:58.995>systems.

119
00:05:00.285 --> 00:05:02.655
<00:05:00.285>That <00:05:00.518>includes <00:05:00.751>code <00:05:00.985>data <00:05:01.218>and <00:05:01.451>your <00:05:01.685>model.

120
00:05:02.885 --> 00:05:04.455
<00:05:02.885>Then, <00:05:03.118>then <00:05:03.351>you <00:05:03.585>need <00:05:03.818>to <00:05:04.051>deploy <00:05:04.284>it

121
00:05:04.875 --> 00:05:08.215
<00:05:04.875>and <00:05:05.075>actually <00:05:05.275>keep <00:05:05.475>it <00:05:05.675>running <00:05:05.875>and <00:05:06.075>continuously <00:05:06.275>monitor

122
00:05:08.755 --> 00:05:10.375
<00:05:08.755>and <00:05:09.105>observe <00:05:09.455>what <00:05:09.805>is <00:05:10.155>happening.

123
00:05:10.435 --> 00:05:12.495
<00:05:10.435>And <00:05:10.635>at <00:05:10.835>some <00:05:11.035>point, <00:05:11.235>you <00:05:11.435>will <00:05:11.635>discover, <00:05:11.835>well,

124
00:05:12.495 --> 00:05:14.815
<00:05:12.495>your <00:05:12.695>model <00:05:12.895>just <00:05:13.095>doesn't <00:05:13.295>work <00:05:13.495>anymore <00:05:13.695>as <00:05:13.895>expected.

125
00:05:15.195 --> 00:05:17.215
<00:05:15.195>And <00:05:15.370>this <00:05:15.545>is <00:05:15.719>the <00:05:15.895>point <00:05:16.070>where <00:05:16.245>you <00:05:16.420>probably <00:05:16.594>need

126
00:05:17.215 --> 00:05:20.495
<00:05:17.215>to <00:05:17.342>go <00:05:17.469>back <00:05:17.596>all <00:05:17.724>this <00:05:17.851>way, <00:05:17.978>tweak <00:05:18.105>the <00:05:18.233>model, <00:05:18.360>do <00:05:18.487>some <00:05:18.614>changes,

127
00:05:21.005 --> 00:05:23.255
<00:05:21.005>perhaps <00:05:21.355>retrain <00:05:21.705>or <00:05:22.055>only <00:05:22.405>retrain,

128
00:05:23.475 --> 00:05:27.015
<00:05:23.475>and <00:05:23.675>then <00:05:23.875>move <00:05:24.075>through <00:05:24.275>all <00:05:24.475>these <00:05:24.675>stages <00:05:24.875>again.

129
00:05:28.075 --> 00:05:30.895
<00:05:28.075>So <00:05:28.250>we <00:05:28.425>see, <00:05:28.599>in <00:05:28.775>order <00:05:28.950>to <00:05:29.125>really <00:05:29.300>produce <00:05:29.474>value

130
00:05:30.895 --> 00:05:32.855
<00:05:30.895>with <00:05:31.128>machine <00:05:31.361>learning, <00:05:31.594>you <00:05:31.828>need <00:05:32.061>to <00:05:32.294>go

131
00:05:33.165 --> 00:05:34.975
<00:05:33.165>through <00:05:33.515>this <00:05:33.865>entire <00:05:34.215>long <00:05:34.565>process.

132
00:05:35.195 --> 00:05:37.055
<00:05:35.195>You <00:05:35.428>actually <00:05:35.661>need <00:05:35.895>to <00:05:36.128>ship <00:05:36.361>to <00:05:36.594>production,

133
00:05:37.055 --> 00:05:41.415
<00:05:37.055>and <00:05:37.182>you <00:05:37.309>need <00:05:37.436>to, <00:05:37.564>to, <00:05:37.691>to <00:05:37.818>monitor <00:05:37.945>and <00:05:38.073>be <00:05:38.200>able <00:05:38.327>to <00:05:38.455>retrain

134
00:05:41.475 --> 00:05:42.535
<00:05:41.475>and <00:05:41.952>iterate <00:05:42.429>fast.

135
00:05:43.805 --> 00:05:46.135
<00:05:43.805>Well, <00:05:43.945>it <00:05:44.085>turns <00:05:44.225>out <00:05:44.365>this <00:05:44.505>process <00:05:44.645>is <00:05:44.785>not <00:05:44.925>only <00:05:45.065>longer <00:05:45.205>than

136
00:05:46.135 --> 00:05:47.895
<00:05:46.135>what <00:05:46.485>organizations <00:05:46.835>typically <00:05:47.185>think <00:05:47.534>about,

137
00:05:48.675 --> 00:05:50.855
<00:05:48.675>but <00:05:49.025>it's <00:05:49.375>also <00:05:49.725>significantly <00:05:50.075>deeper.

138
00:05:51.245 --> 00:05:53.695
<00:05:51.245>Yeah. <00:05:51.445>That <00:05:51.645>means <00:05:51.845>at <00:05:52.045>each <00:05:52.245>of <00:05:52.445>these <00:05:52.645>steps

139
00:05:53.755 --> 00:05:56.615
<00:05:53.755>of <00:05:53.988>this <00:05:54.221>long <00:05:54.455>circular <00:05:54.688>process, <00:05:54.921>we <00:05:55.155>need

140
00:05:56.615 --> 00:05:58.045
<00:05:56.615>to <00:05:57.044>deal <00:05:57.473>with <00:05:57.902>artifacts.

141
00:05:58.225 --> 00:06:01.485
<00:05:58.225>And <00:05:58.400>that's, <00:05:58.575>that <00:05:58.750>artifact <00:05:58.925>is <00:05:59.100>not <00:05:59.275>only <00:05:59.450>code, <00:05:59.625>right?

142
00:06:01.485 --> 00:06:02.925
<00:06:01.485>We, <00:06:01.670>we <00:06:01.855>know <00:06:02.040>how <00:06:02.225>to <00:06:02.410>deal <00:06:02.595>with <00:06:02.781>code.

143
00:06:02.925 --> 00:06:05.005
<00:06:02.925>We've <00:06:03.158>learned <00:06:03.391>that <00:06:03.625>from <00:06:03.858>traditional <00:06:04.091>software <00:06:04.325>engineering.

144
00:06:05.265 --> 00:06:06.645
<00:06:05.265>That's <00:06:05.575>not <00:06:05.885>the <00:06:06.196>big <00:06:06.507>topic.

145
00:06:07.235 --> 00:06:10.125
<00:06:07.235>However, <00:06:07.468>each <00:06:07.701>of <00:06:07.935>these <00:06:08.168>steps <00:06:08.401>also <00:06:08.635>produces

146
00:06:10.305 --> 00:06:11.645
<00:06:10.305>or <00:06:10.908>consumes <00:06:11.510>models.

147
00:06:12.475 --> 00:06:13.525
<00:06:12.475>That <00:06:12.711>could <00:06:12.947>be <00:06:13.183>one <00:06:13.419>model,

148
00:06:13.545 --> 00:06:16.445
<00:06:13.545>but <00:06:13.700>that <00:06:13.856>could <00:06:14.011>also <00:06:14.167>be <00:06:14.322>hundreds <00:06:14.478>of <00:06:14.633>models <00:06:14.789>and <00:06:14.945>data.

149
00:06:16.625 --> 00:06:18.405
<00:06:16.625>And <00:06:16.780>the <00:06:16.936>data <00:06:17.091>can <00:06:17.247>be <00:06:17.402>big, <00:06:17.558>that <00:06:17.713>can <00:06:17.869>be <00:06:18.025>small,

150
00:06:18.405 --> 00:06:20.805
<00:06:18.405>that <00:06:18.560>can <00:06:18.716>be <00:06:18.871>changing <00:06:19.027>and <00:06:19.182>come <00:06:19.338>in <00:06:19.493>a <00:06:19.649>vari <00:06:19.804>in,

151
00:06:20.805 --> 00:06:22.445
<00:06:20.805>in <00:06:21.085>a <00:06:21.365>variety <00:06:21.645>of <00:06:21.925>different <00:06:22.205>formats.

152
00:06:23.105 --> 00:06:25.285
<00:06:23.105>And <00:06:23.245>we <00:06:23.385>need <00:06:23.525>to <00:06:23.665>make <00:06:23.805>sure, <00:06:23.945>if <00:06:24.085>you <00:06:24.225>want <00:06:24.365>to <00:06:24.505>go

153
00:06:25.285 --> 00:06:27.165
<00:06:25.285>through <00:06:25.518>this <00:06:25.751>process, <00:06:25.985>and <00:06:26.218>if <00:06:26.451>you <00:06:26.685>want

154
00:06:27.165 --> 00:06:28.405
<00:06:27.165>to <00:06:27.444>deploy <00:06:27.723>machine <00:06:28.002>learning <00:06:28.280>models

155
00:06:28.425 --> 00:06:30.565
<00:06:28.425>to <00:06:28.625>production <00:06:28.825>successfully, <00:06:29.025>we <00:06:29.225>need <00:06:29.425>to <00:06:29.625>make <00:06:29.825>sure

156
00:06:30.675 --> 00:06:35.125
<00:06:30.675>that <00:06:30.955>these <00:06:31.235>artifacts <00:06:31.515>are <00:06:31.795>created, <00:06:32.075>stored,

157
00:06:35.475 --> 00:06:36.685
<00:06:35.475>managed, <00:06:36.564>versioned,

158
00:06:36.945 --> 00:06:38.885
<00:06:36.945>and <00:06:37.145>consumed <00:06:37.344>at <00:06:37.545>the <00:06:37.745>right <00:06:37.945>point <00:06:38.145>in <00:06:38.344>time

159
00:06:38.885 --> 00:06:39.925
<00:06:38.885>throughout <00:06:39.353>this <00:06:39.821>process.

160
00:06:41.425 --> 00:06:45.405
<00:06:41.425>So <00:06:41.580>we <00:06:41.736>can <00:06:41.891>see <00:06:42.047>well, <00:06:42.202>taking <00:06:42.358>a <00:06:42.513>machine <00:06:42.669>learning <00:06:42.825>model

161
00:06:45.405 --> 00:06:48.285
<00:06:45.405>to <00:06:45.544>production <00:06:45.684>and <00:06:45.825>keeping <00:06:45.965>it <00:06:46.104>running <00:06:46.244>at <00:06:46.385>the <00:06:46.525>high <00:06:46.664>quality <00:06:46.804>and

162
00:06:48.285 --> 00:06:50.645
<00:06:48.284>therefore <00:06:48.564>enabling <00:06:48.844>to <00:06:49.124>create <00:06:49.405>business <00:06:49.684>value.

163
00:06:51.195 --> 00:06:53.805
<00:06:51.195>Well, <00:06:51.474>that <00:06:51.755>is <00:06:52.034>a <00:06:52.315>complex <00:06:52.594>process.

164
00:06:54.265 --> 00:06:55.965
<00:06:54.265>That's <00:06:54.544>by <00:06:54.825>far <00:06:55.104>not <00:06:55.385>as <00:06:55.664>easy

165
00:06:56.425 --> 00:06:58.845
<00:06:56.425>as <00:06:56.775>most <00:06:57.125>organiz <00:06:57.475>organizations <00:06:57.825>think.

166
00:06:59.875 --> 00:07:02.255
<00:06:59.875>And <00:07:00.108>out <00:07:00.341>of <00:07:00.575>this <00:07:00.808>complexity, <00:07:01.041>typical <00:07:01.275>problems,

167
00:07:02.255 --> 00:07:03.335
<00:07:02.255>arise <00:07:03.227>problems.

168
00:07:03.795 --> 00:07:05.455
<00:07:03.795>I'm <00:07:03.995>quite <00:07:04.195>sure <00:07:04.395>you've <00:07:04.595>read <00:07:04.795>about <00:07:04.995>the <00:07:05.195>problem.

169
00:07:05.455 --> 00:07:06.895
<00:07:05.455>You've <00:07:05.671>seen <00:07:05.887>them <00:07:06.103>in <00:07:06.318>your <00:07:06.534>own <00:07:06.751>organizations.

170
00:07:06.905 --> 00:07:09.095
<00:07:06.905>These <00:07:07.255>problems <00:07:07.604>are <00:07:07.955>classic <00:07:08.304>reproducibility.

171
00:07:09.555 --> 00:07:12.375
<00:07:09.555>How <00:07:09.695>can <00:07:09.835>I <00:07:09.975>make <00:07:10.115>sure <00:07:10.255>that <00:07:10.395>it <00:07:10.535>can <00:07:10.675>actually <00:07:10.815>reproduce <00:07:10.955>the

172
00:07:12.375 --> 00:07:13.895
<00:07:12.375>experiments <00:07:12.831>that <00:07:13.287>I've <00:07:13.743>conducted?

173
00:07:14.125 --> 00:07:16.455
<00:07:14.125>Yeah. <00:07:14.825>Or <00:07:15.525>consistency.

174
00:07:16.915 --> 00:07:19.095
<00:07:16.915>Um, <00:07:17.070>how <00:07:17.226>do <00:07:17.381>I <00:07:17.537>make <00:07:17.692>sure <00:07:17.848>that <00:07:18.003>my <00:07:18.159>training <00:07:18.315>pipeline

175
00:07:19.115 --> 00:07:20.855
<00:07:19.115>and <00:07:19.395>my <00:07:19.675>inference <00:07:19.955>pipeline, <00:07:20.235>for <00:07:20.515>example,

176
00:07:20.995 --> 00:07:22.535
<00:07:20.995>use <00:07:21.341>the <00:07:21.688>same <00:07:22.034>pre-processing <00:07:22.381>steps,

177
00:07:22.965 --> 00:07:25.535
<00:07:22.965>even <00:07:23.198>if <00:07:23.431>I <00:07:23.664>frequently <00:07:23.898>change <00:07:24.131>certain <00:07:24.364>parts?

178
00:07:27.115 --> 00:07:29.405
<00:07:27.115>Then <00:07:27.290>well, <00:07:27.465>comprehensive <00:07:27.640>testing, <00:07:27.815>you <00:07:27.990>do <00:07:28.165>not <00:07:28.340>only <00:07:28.515>need

179
00:07:29.405 --> 00:07:31.605
<00:07:29.405>to <00:07:29.544>test <00:07:29.684>code, <00:07:29.825>but <00:07:29.965>you <00:07:30.104>also <00:07:30.244>need <00:07:30.385>to <00:07:30.525>test <00:07:30.664>your <00:07:30.804>model,

180
00:07:31.625 --> 00:07:32.725
<00:07:31.625>for <00:07:31.955>example, <00:07:32.285>for <00:07:32.615>bias.

181
00:07:32.825 --> 00:07:34.725
<00:07:32.825>And <00:07:33.058>ideally, <00:07:33.291>you <00:07:33.525>also <00:07:33.758>test <00:07:33.991>your <00:07:34.224>data.

182
00:07:35.985 --> 00:07:37.645
<00:07:35.985>Of <00:07:36.265>course, <00:07:36.545>once <00:07:36.825>things <00:07:37.105>are <00:07:37.385>tested

183
00:07:37.785 --> 00:07:40.245
<00:07:37.784>and <00:07:37.960>you <00:07:38.135>have <00:07:38.309>your <00:07:38.484>model <00:07:38.659>productionized, <00:07:38.835>you <00:07:39.010>also <00:07:39.184>need

184
00:07:40.245 --> 00:07:41.325
<00:07:40.245>to <00:07:40.569>think <00:07:40.893>about <00:07:41.217>deployment.

185
00:07:41.625 --> 00:07:44.125
<00:07:41.625>And <00:07:41.825>of <00:07:42.025>course, <00:07:42.225>one <00:07:42.425>thing <00:07:42.625>are <00:07:42.825>deployment <00:07:43.025>strategies.

186
00:07:44.275 --> 00:07:47.005
<00:07:44.275>Sometimes <00:07:44.450>in <00:07:44.625>machine <00:07:44.799>learning, <00:07:44.974>you <00:07:45.150>would <00:07:45.325>favor <00:07:45.500>going <00:07:45.674>perhaps

187
00:07:47.005 --> 00:07:49.525
<00:07:47.005>for <00:07:47.205>a <00:07:47.405>bit <00:07:47.605>more <00:07:47.805>exotic <00:07:48.005>deployment <00:07:48.205>strategies <00:07:48.405>such

188
00:07:49.525 --> 00:07:51.005
<00:07:49.525>as <00:07:49.969>multi-arm <00:07:50.413>banded <00:07:50.857>deployments,

189
00:07:51.025 --> 00:07:53.125
<00:07:51.025>and <00:07:51.180>you <00:07:51.336>should, <00:07:51.491>you <00:07:51.647>should <00:07:51.802>be <00:07:51.958>able <00:07:52.113>to <00:07:52.269>support <00:07:52.424>those.

190
00:07:53.385 --> 00:07:54.645
<00:07:53.385>But <00:07:53.574>you <00:07:53.763>also <00:07:53.952>need <00:07:54.140>to <00:07:54.330>think <00:07:54.519>about

191
00:07:55.075 --> 00:07:56.885
<00:07:55.075>what <00:07:55.354>hardware <00:07:55.635>do <00:07:55.914>I <00:07:56.195>actually <00:07:56.474>need?

192
00:07:56.905 --> 00:07:57.965
<00:07:56.905>At <00:07:57.143>what <00:07:57.382>point <00:07:57.620>in <00:07:57.859>time

193
00:07:58.225 --> 00:08:00.485
<00:07:58.225>to <00:07:58.425>deploy <00:07:58.625>my <00:07:58.825>model <00:07:59.025>and <00:07:59.225>keep <00:07:59.425>it <00:07:59.625>running?

194
00:08:00.675 --> 00:08:03.285
<00:08:00.675>That <00:08:00.875>can <00:08:01.075>range <00:08:01.275>right <00:08:01.475>from <00:08:01.675>theoretically <00:08:01.875>a <00:08:02.075>cluster

195
00:08:03.305 --> 00:08:06.445
<00:08:03.305>of <00:08:03.460>GPUs <00:08:03.616>all <00:08:03.771>the <00:08:03.927>way <00:08:04.082>to <00:08:04.238>an <00:08:04.393>edge <00:08:04.549>device, <00:08:04.705>so

196
00:08:06.445 --> 00:08:09.005
<00:08:06.445>that <00:08:06.600>the <00:08:06.756>bandwidth <00:08:06.911>is, <00:08:07.067>is <00:08:07.222>very <00:08:07.378>broad <00:08:07.533>there <00:08:07.689>as <00:08:07.844>well.

197
00:08:09.995 --> 00:08:12.205
<00:08:09.995>Once <00:08:10.195>things <00:08:10.395>are <00:08:10.595>deployed, <00:08:10.795>it <00:08:10.995>doesn't <00:08:11.195>stop <00:08:11.395>there.

198
00:08:12.555 --> 00:08:13.605
<00:08:12.555>Then <00:08:12.870>the <00:08:13.185>big <00:08:13.500>challenge

199
00:08:13.665 --> 00:08:16.885
<00:08:13.665>of <00:08:13.945>monitoring <00:08:14.225>your <00:08:14.505>model <00:08:14.785>appropriately <00:08:15.065>arises.

200
00:08:17.315 --> 00:08:19.925
<00:08:17.315>That <00:08:17.455>means, <00:08:17.594>well, <00:08:17.735>you <00:08:17.875>do <00:08:18.015>not <00:08:18.155>only <00:08:18.295>need <00:08:18.435>to <00:08:18.575>monitor <00:08:18.715>uptime,

201
00:08:19.925 --> 00:08:21.045
<00:08:19.925>downtime <00:08:20.429>response <00:08:20.933>time,

202
00:08:21.185 --> 00:08:23.445
<00:08:21.185>but <00:08:21.418>you <00:08:21.651>also <00:08:21.885>need <00:08:22.118>to <00:08:22.351>monitor <00:08:22.585>data

203
00:08:23.475 --> 00:08:25.725
<00:08:23.475>that <00:08:23.708>goes <00:08:23.941>into <00:08:24.175>your, <00:08:24.408>into <00:08:24.641>your <00:08:24.875>model.

204
00:08:25.905 --> 00:08:27.645
<00:08:25.905>For <00:08:26.371>example, <00:08:26.838>feature <00:08:27.304>distributions

205
00:08:27.985 --> 00:08:30.965
<00:08:27.985>and <00:08:28.185>also <00:08:28.385>the <00:08:28.585>output <00:08:28.785>distribution <00:08:28.985>your <00:08:29.185>model <00:08:29.385>produces,

206
00:08:30.965 --> 00:08:32.645
<00:08:30.965>or <00:08:31.198>perhaps <00:08:31.431>if <00:08:31.664>you <00:08:31.898>can <00:08:32.131>even <00:08:32.365>prediction

207
00:08:32.845 --> 00:08:34.925
<00:08:32.845>accuracy <00:08:33.311>at <00:08:33.778>some <00:08:34.245>point.

208
00:08:34.925 --> 00:08:37.645
<00:08:34.924>Inevitably, <00:08:35.158>you <00:08:35.391>will <00:08:35.625>see <00:08:35.858>that <00:08:36.091>accuracy <00:08:36.324>drop

209
00:08:37.705 --> 00:08:40.485
<00:08:37.705>or <00:08:37.860>you <00:08:38.016>will <00:08:38.171>see <00:08:38.327>things, <00:08:38.482>uh, <00:08:38.638>in <00:08:38.793>the <00:08:38.949>distributions <00:08:39.105>changing.

210
00:08:40.505 --> 00:08:42.325
<00:08:40.505>And <00:08:40.679>at <00:08:40.855>that <00:08:41.030>point, <00:08:41.205>you <00:08:41.380>need <00:08:41.554>to <00:08:41.730>consider <00:08:41.905>retraining.

211
00:08:42.475 --> 00:08:44.125
<00:08:42.475>That <00:08:42.825>means <00:08:43.175>you <00:08:43.525>probably <00:08:43.875>need

212
00:08:44.125 --> 00:08:46.005
<00:08:44.125>to <00:08:44.325>go <00:08:44.525>all <00:08:44.725>the <00:08:44.925>way <00:08:45.125>back <00:08:45.325>this <00:08:45.525>circle

213
00:08:46.425 --> 00:08:47.885
<00:08:46.425>and <00:08:46.687>go <00:08:46.950>through <00:08:47.213>this <00:08:47.476>circle <00:08:47.739>again,

214
00:08:47.985 --> 00:08:49.405
<00:08:47.985>or <00:08:48.240>you <00:08:48.496>just <00:08:48.751>retrain <00:08:49.007>the <00:08:49.262>model.

215
00:08:50.155 --> 00:08:53.325
<00:08:50.155>Well, <00:08:50.310>ideally, <00:08:50.466>uh, <00:08:50.621>that <00:08:50.777>would <00:08:50.932>happen <00:08:51.088>in <00:08:51.243>an <00:08:51.399>automated <00:08:51.555>way

216
00:08:53.325 --> 00:08:54.525
<00:08:53.325>because <00:08:53.865>otherwise, <00:08:54.405>uh,

217
00:08:54.525 --> 00:08:56.285
<00:08:54.525>your <00:08:54.725>data <00:08:54.925>scientists <00:08:55.125>might <00:08:55.324>get <00:08:55.525>a <00:08:55.725>bit <00:08:55.925>bored.

218
00:08:58.025 --> 00:08:59.205
<00:08:58.025>So <00:08:58.290>we <00:08:58.556>can <00:08:58.821>see, <00:08:59.087>right?

219
00:08:59.265 --> 00:09:01.365
<00:08:59.265>Uh, <00:08:59.439>this <00:08:59.615>is <00:08:59.790>a, <00:08:59.965>this <00:09:00.140>is <00:09:00.314>a <00:09:00.490>complex <00:09:00.665>process,

220
00:09:01.585 --> 00:09:02.885
<00:09:01.585>taking <00:09:01.877>a <00:09:02.170>machine <00:09:02.462>learning <00:09:02.755>model

221
00:09:02.885 --> 00:09:04.205
<00:09:02.885>to <00:09:03.182>production, <00:09:03.479>keeping <00:09:03.776>it <00:09:04.073>there.

222
00:09:04.585 --> 00:09:07.885
<00:09:04.585>And <00:09:04.760>frequently <00:09:04.935>there <00:09:05.110>were, <00:09:05.285>there <00:09:05.460>were <00:09:05.635>very <00:09:05.810>typical <00:09:05.985>problems

223
00:09:07.885 --> 00:09:10.245
<00:09:07.885>that <00:09:08.235>arise <00:09:08.585>from <00:09:08.935>that <00:09:09.285>challenges.

224
00:09:11.345 --> 00:09:16.165
<00:09:11.345>So <00:09:11.520>the <00:09:11.695>big <00:09:11.870>question <00:09:12.045>is <00:09:12.220>how <00:09:12.395>do <00:09:12.570>organizations <00:09:12.745>handle

225
00:09:16.755 --> 00:09:18.165
<00:09:16.755>this <00:09:18.024>complexity?

226
00:09:18.465 --> 00:09:19.805
<00:09:18.465>How <00:09:18.706>do <00:09:18.947>they <00:09:19.188>actually <00:09:19.429>do <00:09:19.670>that?

227
00:09:20.915 --> 00:09:24.245
<00:09:20.915>Well, <00:09:21.194>um, <00:09:21.474>in <00:09:21.755>short, <00:09:22.035>most <00:09:22.314>organizations,

228
00:09:24.755 --> 00:09:26.445
<00:09:24.755>they <00:09:25.105>don't <00:09:25.455>handle <00:09:25.805>this <00:09:26.155>complexity.

229
00:09:27.115 --> 00:09:28.765
<00:09:27.115>Yeah. <00:09:27.395>They, <00:09:27.675>they <00:09:27.955>don't <00:09:28.235>handle <00:09:28.515>it

230
00:09:28.765 --> 00:09:31.205
<00:09:28.765>because <00:09:28.965>some <00:09:29.165>are <00:09:29.365>not <00:09:29.564>aware <00:09:29.765>of <00:09:29.965>it, <00:09:30.165>right?

231
00:09:31.275 --> 00:09:33.245
<00:09:31.275>They, <00:09:31.449>they <00:09:31.625>might <00:09:31.800>have <00:09:31.975>started <00:09:32.150>a <00:09:32.324>year <00:09:32.500>or <00:09:32.675>two,

232
00:09:33.245 --> 00:09:35.725
<00:09:33.245>or <00:09:33.420>maybe <00:09:33.595>three, <00:09:33.770>three <00:09:33.945>years <00:09:34.120>ago <00:09:34.295>with <00:09:34.470>machine <00:09:34.645>learning.

233
00:09:36.225 --> 00:09:38.885
<00:09:36.225>And, <00:09:36.380>uh, <00:09:36.536>they, <00:09:36.691>they <00:09:36.847>have <00:09:37.002>just <00:09:37.158>not <00:09:37.313>built <00:09:37.469>the <00:09:37.625>understanding

234
00:09:39.385 --> 00:09:41.965
<00:09:39.385>to, <00:09:39.585>to <00:09:39.785>recognize <00:09:39.985>the <00:09:40.185>complexity <00:09:40.385>of <00:09:40.585>this <00:09:40.785>process

235
00:09:42.065 --> 00:09:43.605
<00:09:42.065>and <00:09:42.296>what <00:09:42.527>it <00:09:42.758>requires <00:09:42.989>to <00:09:43.220>handle <00:09:43.451>it.

236
00:09:43.935 --> 00:09:45.885
<00:09:43.935>Other <00:09:44.285>companies <00:09:44.635>might <00:09:44.984>recognize <00:09:45.334>it,

237
00:09:46.185 --> 00:09:48.045
<00:09:46.185>but, <00:09:46.418>uh, <00:09:46.651>they <00:09:46.885>like <00:09:47.118>the <00:09:47.351>technical <00:09:47.584>abilities

238
00:09:48.145 --> 00:09:49.245
<00:09:48.145>to <00:09:48.475>actually <00:09:48.805>handle <00:09:49.135>it.

239
00:09:50.385 --> 00:09:53.445
<00:09:50.385>The <00:09:50.560>bad <00:09:50.735>news <00:09:50.910>is, <00:09:51.085>well, <00:09:51.260>not <00:09:51.435>handling <00:09:51.610>this <00:09:51.785>complexity

240
00:09:53.985 --> 00:09:56.285
<00:09:53.985>is <00:09:54.335>simply <00:09:54.685>not <00:09:55.035>an <00:09:55.385>option.

241
00:09:57.515 --> 00:10:00.605
<00:09:57.515>It's <00:09:57.655>not <00:09:57.795>an <00:09:57.935>option <00:09:58.074>if <00:09:58.215>you <00:09:58.355>want <00:09:58.495>to <00:09:58.635>create <00:09:58.775>business <00:09:58.915>value

242
00:10:01.075 --> 00:10:03.045
<00:10:01.075>with <00:10:01.425>your <00:10:01.775>machine <00:10:02.125>learning <00:10:02.475>endeavors,

243
00:10:03.235 --> 00:10:05.845
<00:10:03.235>because, <00:10:03.410>well, <00:10:03.585>if <00:10:03.760>you <00:10:03.935>do <00:10:04.110>not <00:10:04.285>handle <00:10:04.460>this <00:10:04.635>complexity,

244
00:10:05.845 --> 00:10:08.445
<00:10:05.845>what <00:10:06.000>you're <00:10:06.156>gonna <00:10:06.311>face <00:10:06.467>are <00:10:06.622>things <00:10:06.778>like <00:10:06.933>the <00:10:07.089>deployment <00:10:07.245>gap.

245
00:10:09.115 --> 00:10:12.565
<00:10:09.115>That <00:10:09.270>means <00:10:09.426>you're <00:10:09.581>gonna <00:10:09.737>have <00:10:09.892>20 <00:10:10.048>POCs, <00:10:10.203>um, <00:10:10.359>lined <00:10:10.515>up,

246
00:10:12.865 --> 00:10:14.485
<00:10:12.865>but <00:10:13.098>not <00:10:13.331>a <00:10:13.565>single <00:10:13.798>model <00:10:14.031>in <00:10:14.265>production.

247
00:10:15.555 --> 00:10:17.205
<00:10:15.555>Then <00:10:15.905>your <00:10:16.255>model, <00:10:16.604>your <00:10:16.954>model,

248
00:10:17.265 --> 00:10:19.885
<00:10:17.265>and <00:10:17.405>the, <00:10:17.545>the <00:10:17.685>process <00:10:17.824>you <00:10:17.965>need <00:10:18.105>to <00:10:18.245>go <00:10:18.385>through <00:10:18.525>in <00:10:18.665>order

249
00:10:19.885 --> 00:10:21.045
<00:10:19.885>to <00:10:20.233>productionize <00:10:20.581>your <00:10:20.929>model

250
00:10:21.045 --> 00:10:23.125
<00:10:21.045>and <00:10:21.278>keep <00:10:21.511>it <00:10:21.745>running <00:10:21.978>is <00:10:22.211>highly <00:10:22.444>inefficient.

251
00:10:23.435 --> 00:10:27.325
<00:10:23.435>It's <00:10:23.668>highly, <00:10:23.901>it's <00:10:24.135>highly <00:10:24.368>laborers <00:10:24.601>manual <00:10:24.834>work.

252
00:10:27.865 --> 00:10:30.885
<00:10:27.865>That's <00:10:28.020>not <00:10:28.176>only <00:10:28.331>gonna <00:10:28.487>make <00:10:28.642>your <00:10:28.798>data <00:10:28.953>scientist <00:10:29.109>very <00:10:29.265>unhappy,

253
00:10:31.345 --> 00:10:32.805
<00:10:31.345>but <00:10:31.607>also <00:10:31.870>this <00:10:32.133>means <00:10:32.396>that <00:10:32.659>your,

254
00:10:32.875 --> 00:10:35.825
<00:10:32.875>your <00:10:33.108>process <00:10:33.341>is <00:10:33.575>quite <00:10:33.808>error <00:10:34.041>prone, <00:10:34.275>right?

255
00:10:35.845 --> 00:10:37.425
<00:10:35.845>You, <00:10:36.078>you're <00:10:36.311>dealing <00:10:36.545>with <00:10:36.778>this <00:10:37.011>complex <00:10:37.245>process,

256
00:10:37.565 --> 00:10:40.345
<00:10:37.565>and <00:10:37.705>then <00:10:37.845>you <00:10:37.985>have <00:10:38.125>lots <00:10:38.265>of <00:10:38.405>human <00:10:38.545>steps <00:10:38.685>in <00:10:38.825>the <00:10:38.965>middle,

257
00:10:40.705 --> 00:10:41.705
<00:10:40.705>probably <00:10:41.005>a <00:10:41.305>few <00:10:41.605>handovers.

258
00:10:41.975 --> 00:10:43.985
<00:10:41.975>It's <00:10:42.441>inevitable <00:10:42.908>that <00:10:43.375>quite,

259
00:10:43.985 --> 00:10:46.345
<00:10:43.985>that <00:10:44.218>you <00:10:44.451>will <00:10:44.685>face <00:10:44.918>quite <00:10:45.151>some <00:10:45.385>errors

260
00:10:46.475 --> 00:10:47.585
<00:10:46.475>throughout <00:10:46.974>this <00:10:47.474>process.

261
00:10:48.515 --> 00:10:52.145
<00:10:48.515>Eventually, <00:10:48.748>all <00:10:48.981>this <00:10:49.215>means <00:10:49.448>that <00:10:49.681>your <00:10:49.915>iterations,

262
00:10:52.145 --> 00:10:54.745
<00:10:52.145>your <00:10:52.345>development <00:10:52.545>iterations <00:10:52.745>will <00:10:52.944>be <00:10:53.145>very, <00:10:53.345>very <00:10:53.545>slow.

263
00:10:55.035 --> 00:10:58.365
<00:10:55.035>It'll <00:10:55.314>take <00:10:55.594>you <00:10:55.875>months <00:10:56.155>and <00:10:56.435>perhaps,

264
00:10:58.425 --> 00:11:01.685
<00:10:58.425>and <00:10:58.580>unfortunately <00:10:58.736>that's <00:10:58.891>no <00:10:59.047>joke, <00:10:59.202>even <00:10:59.358>a <00:10:59.513>year <00:10:59.669>to <00:10:59.824>put,

265
00:11:01.785 --> 00:11:04.525
<00:11:01.785>to <00:11:01.959>take <00:11:02.135>a <00:11:02.310>model <00:11:02.485>from <00:11:02.660>experimentation <00:11:02.834>to <00:11:03.010>put <00:11:03.185>action.

266
00:11:05.305 --> 00:11:07.565
<00:11:05.305>And <00:11:05.584>this <00:11:05.864>really <00:11:06.145>is <00:11:06.425>what <00:11:06.704>kills

267
00:11:08.235 --> 00:11:09.925
<00:11:08.235>machine <00:11:08.701>learning, <00:11:09.168>delivering <00:11:09.635>value.

268
00:11:10.835 --> 00:11:13.885
<00:11:10.835>Machine <00:11:11.068>learning <00:11:11.301>is <00:11:11.535>a <00:11:11.768>highly <00:11:12.001>iterative <00:11:12.235>process.

269
00:11:14.075 --> 00:11:15.325
<00:11:14.075>It's <00:11:14.450>an <00:11:14.825>experimental <00:11:15.200>process.

270
00:11:15.945 --> 00:11:19.205
<00:11:15.945>You <00:11:16.085>need <00:11:16.225>to <00:11:16.365>be <00:11:16.505>able <00:11:16.645>to <00:11:16.785>take <00:11:16.925>models <00:11:17.065>to <00:11:17.205>production <00:11:17.345>quickly

271
00:11:19.665 --> 00:11:23.965
<00:11:19.665>to <00:11:19.839>really <00:11:20.015>learn <00:11:20.189>how <00:11:20.365>your <00:11:20.540>model <00:11:20.714>works <00:11:20.890>in <00:11:21.064>production.

272
00:11:24.515 --> 00:11:27.965
<00:11:24.515>Yeah, <00:11:24.670>every <00:11:24.826>data <00:11:24.981>scientist <00:11:25.137>who <00:11:25.292>has <00:11:25.448>done <00:11:25.603>that <00:11:25.759>will <00:11:25.915>confirm

273
00:11:27.995 --> 00:11:31.125
<00:11:27.995>that <00:11:28.170>in <00:11:28.345>production <00:11:28.520>data <00:11:28.695>might <00:11:28.870>look <00:11:29.045>a <00:11:29.220>bit <00:11:29.395>different.

274
00:11:31.125 --> 00:11:33.285
<00:11:31.125>You <00:11:31.280>might <00:11:31.436>discover, <00:11:31.591>oh, <00:11:31.747>well, <00:11:31.902>that <00:11:32.058>feature <00:11:32.213>I've <00:11:32.369>been <00:11:32.525>using,

275
00:11:33.965 --> 00:11:35.205
<00:11:33.965>suddenly <00:11:34.244>it's <00:11:34.523>not <00:11:34.802>really <00:11:35.081>available

276
00:11:35.305 --> 00:11:36.405
<00:11:35.305>all <00:11:35.552>the <00:11:35.800>time <00:11:36.047>in <00:11:36.295>production.

277
00:11:36.585 --> 00:11:39.805
<00:11:36.585>And <00:11:36.712>all <00:11:36.839>these <00:11:36.966>are <00:11:37.094>things <00:11:37.221>you <00:11:37.348>only <00:11:37.475>learn <00:11:37.603>when <00:11:37.730>you <00:11:37.857>actually <00:11:37.985>go

278
00:11:39.825 --> 00:11:43.165
<00:11:39.825>to <00:11:40.105>production, <00:11:40.385>and <00:11:40.665>therefore, <00:11:40.945>fast <00:11:41.225>iterations.

279
00:11:43.255 --> 00:11:46.085
<00:11:43.255>Being <00:11:43.455>able <00:11:43.655>to <00:11:43.855>go <00:11:44.055>from <00:11:44.255>experiment <00:11:44.455>to <00:11:44.655>production

280
00:11:46.105 --> 00:11:47.325
<00:11:46.105>and <00:11:46.471>back <00:11:46.837>to <00:11:47.203>experiment

281
00:11:47.505 --> 00:11:51.765
<00:11:47.505>and <00:11:47.660>again <00:11:47.816>to <00:11:47.971>production <00:11:48.127>is <00:11:48.282>really <00:11:48.438>key <00:11:48.593>to <00:11:48.749>creating <00:11:48.905>models

282
00:11:52.075 --> 00:11:54.725
<00:11:52.075>that <00:11:52.355>actually <00:11:52.635>deliver <00:11:52.915>value <00:11:53.195>in <00:11:53.475>production.

283
00:11:57.115 --> 00:11:59.965
<00:11:57.115>Well, <00:11:57.315>fortunately, <00:11:57.515>in <00:11:57.715>the <00:11:57.915>last <00:11:58.115>few <00:11:58.315>years, <00:11:58.515>uh,

284
00:12:00.685 --> 00:12:02.485
<00:12:00.685>a <00:12:00.964>relatively <00:12:01.244>new <00:12:01.525>field <00:12:01.805>has <00:12:02.084>emerged,

285
00:12:02.545 --> 00:12:03.685
<00:12:02.545>and <00:12:02.750>that's, <00:12:02.955>this <00:12:03.160>is <00:12:03.365>the <00:12:03.570>field

286
00:12:03.685 --> 00:12:05.805
<00:12:03.685>of <00:12:03.964>machine <00:12:04.244>learning <00:12:04.525>operations, <00:12:04.805>ML <00:12:05.084>ops,

287
00:12:05.895 --> 00:12:07.685
<00:12:05.895>which <00:12:06.245>is <00:12:06.595>essentially <00:12:06.944>the <00:12:07.295>application

288
00:12:07.705 --> 00:12:11.045
<00:12:07.705>and <00:12:07.845>a <00:12:07.985>bit <00:12:08.125>of <00:12:08.265>adaption <00:12:08.405>of <00:12:08.545>DevOps <00:12:08.685>principles <00:12:08.825>to <00:12:08.965>the <00:12:09.105>field

289
00:12:11.165 --> 00:12:12.365
<00:12:11.165>of <00:12:11.525>machine <00:12:11.885>learning <00:12:12.245>development.

290
00:12:13.385 --> 00:12:16.685
<00:12:13.385>And <00:12:13.560>ML <00:12:13.735>ops <00:12:13.910>really <00:12:14.085>has <00:12:14.260>the, <00:12:14.435>has <00:12:14.610>the <00:12:14.785>objective

291
00:12:16.825 --> 00:12:19.885
<00:12:16.825>to <00:12:17.058>make <00:12:17.291>this <00:12:17.525>process <00:12:17.758>going <00:12:17.991>from <00:12:18.225>experimentation

292
00:12:20.505 --> 00:12:25.445
<00:12:20.505>to <00:12:20.660>production <00:12:20.816>as <00:12:20.971>easy, <00:12:21.127>as <00:12:21.282>quickly, <00:12:21.438>as <00:12:21.593>robust <00:12:21.749>as <00:12:21.905>possible.

293
00:12:26.995 --> 00:12:31.155
<00:12:26.995>And <00:12:27.170>it <00:12:27.345>does <00:12:27.520>so <00:12:27.695>by <00:12:27.870>drastically <00:12:28.045>increasing <00:12:28.220>the <00:12:28.395>degree

294
00:12:31.155 --> 00:12:34.755
<00:12:31.155>of <00:12:31.329>automation <00:12:31.505>you <00:12:31.680>see <00:12:31.855>throughout <00:12:32.030>this <00:12:32.204>complex <00:12:32.380>process, <00:12:32.555>and

295
00:12:34.755 --> 00:12:37.675
<00:12:34.755>therefore <00:12:34.988>enabling <00:12:35.221>you <00:12:35.455>to <00:12:35.688>actually <00:12:35.921>iterate <00:12:36.155>faster

296
00:12:38.775 --> 00:12:41.475
<00:12:38.775>and <00:12:39.008>also <00:12:39.241>ship <00:12:39.475>with <00:12:39.708>improved <00:12:39.941>quality <00:12:40.175>due

297
00:12:41.475 --> 00:12:43.875
<00:12:41.475>to <00:12:41.755>the <00:12:42.035>increased <00:12:42.315>standardization <00:12:42.595>you <00:12:42.875>introduce

298
00:12:44.095 --> 00:12:46.435
<00:12:44.095>by <00:12:44.328>applying <00:12:44.561>wide <00:12:44.795>automation <00:12:45.028>to <00:12:45.261>your <00:12:45.495>process.

299
00:12:47.625 --> 00:12:48.675
<00:12:47.625>That <00:12:47.939>sounds <00:12:48.255>great, <00:12:48.569>right?

300
00:12:48.695 --> 00:12:51.075
<00:12:48.695>Um, <00:12:48.870>at <00:12:49.045>that <00:12:49.220>point, <00:12:49.395>you <00:12:49.570>probably <00:12:49.745>think <00:12:49.920>yes, <00:12:50.095>um,

301
00:12:51.755 --> 00:12:52.765
<00:12:51.755>that, <00:12:52.058>that <00:12:52.361>sounds <00:12:52.664>awesome.

302
00:12:53.045 --> 00:12:54.605
<00:12:53.045>I <00:12:53.219>wanna <00:12:53.395>buy <00:12:53.569>that. <00:12:53.745>How <00:12:53.920>much <00:12:54.094>does <00:12:54.270>it <00:12:54.444>cost?

303
00:12:55.195 --> 00:12:56.965
<00:12:55.195>Well, <00:12:55.370>the <00:12:55.545>bad <00:12:55.720>thing <00:12:55.895>is <00:12:56.070>it's <00:12:56.245>not <00:12:56.420>that <00:12:56.595>easy,

304
00:12:57.825 --> 00:12:59.565
<00:12:57.825>but <00:12:58.058>let's <00:12:58.291>jump <00:12:58.525>into <00:12:58.758>a <00:12:58.991>case <00:12:59.225>study

305
00:12:59.865 --> 00:13:02.645
<00:12:59.865>and <00:13:00.065>actually <00:13:00.265>see, <00:13:00.465>um, <00:13:00.665>how <00:13:00.865>that <00:13:01.065>can <00:13:01.265>work.

306
00:13:02.985 --> 00:13:04.765
<00:13:02.985>We <00:13:03.185>are <00:13:03.385>gonna <00:13:03.585>follow <00:13:03.785>a <00:13:03.985>fictitious <00:13:04.185>company <00:13:04.385>here.

307
00:13:05.145 --> 00:13:07.325
<00:13:05.145>Uh, <00:13:05.378>that <00:13:05.611>company <00:13:05.845>is <00:13:06.078>called <00:13:06.311>Acme <00:13:06.545>Inc.

308
00:13:07.635 --> 00:13:09.405
<00:13:07.635>It's, <00:13:07.915>you <00:13:08.194>know, <00:13:08.475>an <00:13:08.755>industrial <00:13:09.035>company.

309
00:13:09.555 --> 00:13:11.645
<00:13:09.555>They <00:13:09.755>produce, <00:13:09.954>in <00:13:10.155>our <00:13:10.354>case, <00:13:10.555>high <00:13:10.755>Performance <00:13:10.954>Tire.

310
00:13:11.645 --> 00:13:12.685
<00:13:11.645>Doesn't <00:13:11.957>really <00:13:12.269>matter <00:13:12.580>though.

311
00:13:13.185 --> 00:13:15.125
<00:13:13.185>And <00:13:13.418>we <00:13:13.651>are <00:13:13.885>gonna <00:13:14.118>follow <00:13:14.351>that <00:13:14.584>company

312
00:13:15.425 --> 00:13:19.245
<00:13:15.425>and <00:13:15.599>observe <00:13:15.775>how <00:13:15.949>they <00:13:16.125>lift <00:13:16.300>their <00:13:16.474>machine <00:13:16.650>learning <00:13:16.824>development

313
00:13:19.245 --> 00:13:21.365
<00:13:19.245>process <00:13:19.525>to <00:13:19.805>the <00:13:20.085>next <00:13:20.365>level, <00:13:20.645>uh,

314
00:13:21.435 --> 00:13:24.085
<00:13:21.435>with <00:13:21.901>machine <00:13:22.368>learning <00:13:22.834>operations.

315
00:13:25.025 --> 00:13:26.965
<00:13:25.025>And <00:13:25.180>we <00:13:25.336>are <00:13:25.491>going <00:13:25.647>to <00:13:25.802>do <00:13:25.958>this <00:13:26.113>in <00:13:26.269>four <00:13:26.425>steps.

316
00:13:27.615 --> 00:13:29.925
<00:13:27.615>First, <00:13:27.755>we <00:13:27.895>are <00:13:28.035>gonna <00:13:28.175>sit <00:13:28.315>down <00:13:28.455>with <00:13:28.595>the <00:13:28.735>data <00:13:28.875>science <00:13:29.015>team

317
00:13:30.265 --> 00:13:33.005
<00:13:30.265>and <00:13:30.498>analyze <00:13:30.731>how <00:13:30.965>they <00:13:31.198>actually <00:13:31.431>currently <00:13:31.665>develop

318
00:13:33.005 --> 00:13:34.125
<00:13:33.005>machine <00:13:33.509>learning <00:13:34.013>solutions.

319
00:13:34.945 --> 00:13:38.125
<00:13:34.945>And <00:13:35.100>probably <00:13:35.256>you <00:13:35.411>are <00:13:35.567>gonna <00:13:35.722>see <00:13:35.878>some, <00:13:36.033>some <00:13:36.189>similarities <00:13:36.345>to,

320
00:13:38.145 --> 00:13:39.845
<00:13:38.145>to <00:13:38.345>the <00:13:38.545>organization <00:13:38.745>that <00:13:38.944>you <00:13:39.145>work <00:13:39.345>in <00:13:39.545>here.

321
00:13:40.345 --> 00:13:43.085
<00:13:40.345>And <00:13:40.545>subsequently, <00:13:40.745>we <00:13:40.945>are <00:13:41.145>gonna <00:13:41.345>sketch <00:13:41.545>out, <00:13:41.745>um,

322
00:13:43.395 --> 00:13:46.485
<00:13:43.395>what <00:13:43.595>could <00:13:43.795>a <00:13:43.995>ideal <00:13:44.194>development <00:13:44.395>process <00:13:44.595>look <00:13:44.795>like?

323
00:13:46.825 --> 00:13:49.685
<00:13:46.825>You <00:13:47.058>know, <00:13:47.291>an <00:13:47.525>ML <00:13:47.758>ops <00:13:47.991>enabled <00:13:48.225>process,

324
00:13:50.545 --> 00:13:53.205
<00:13:50.545>and <00:13:50.700>of <00:13:50.856>course, <00:13:51.011>how <00:13:51.167>can <00:13:51.322>we <00:13:51.478>define <00:13:51.633>a <00:13:51.789>tech <00:13:51.944>stack

325
00:13:53.475 --> 00:13:54.765
<00:13:53.475>that <00:13:53.862>supports <00:13:54.249>this <00:13:54.636>process?

326
00:13:55.875 --> 00:13:57.925
<00:13:55.875>Finally, <00:13:56.225>well, <00:13:56.575>we <00:13:56.925>still <00:13:57.275>need

327
00:13:57.925 --> 00:13:59.845
<00:13:57.925>to <00:13:58.275>define <00:13:58.625>a <00:13:58.974>transition <00:13:59.324>strategy.

328
00:13:59.905 --> 00:14:04.285
<00:13:59.905>So <00:14:00.032>how <00:14:00.159>are <00:14:00.286>we <00:14:00.414>actually <00:14:00.541>going <00:14:00.668>to <00:14:00.795>execute <00:14:00.923>our, <00:14:01.050>our, <00:14:01.177>our <00:14:01.305>plan?

329
00:14:04.345 --> 00:14:07.005
<00:14:04.345>How <00:14:04.545>are <00:14:04.745>we <00:14:04.945>gonna <00:14:05.145>move <00:14:05.345>from <00:14:05.545>the <00:14:05.745>current,

330
00:14:07.155 --> 00:14:10.005
<00:14:07.155>what <00:14:07.310>we <00:14:07.466>currently <00:14:07.621>have, <00:14:07.777>the <00:14:07.932>current <00:14:08.088>state, <00:14:08.243>all <00:14:08.399>the <00:14:08.555>way

331
00:14:10.005 --> 00:14:11.125
<00:14:10.005>to <00:14:10.341>our <00:14:10.677>objective <00:14:11.013>state

332
00:14:11.185 --> 00:14:12.285
<00:14:11.185>or <00:14:11.680>objective <00:14:12.175>process

333
00:14:12.635 --> 00:14:15.765
<00:14:12.635>that <00:14:12.810>should <00:14:12.985>enable <00:14:13.160>us <00:14:13.335>shipping <00:14:13.510>faster, <00:14:13.685>shipping <00:14:13.860>more <00:14:14.035>reliably.

334
00:14:17.225 --> 00:14:19.685
<00:14:17.225>Before <00:14:17.458>we <00:14:17.691>dive <00:14:17.925>into <00:14:18.158>these <00:14:18.391>four <00:14:18.625>steps,

335
00:14:20.175 --> 00:14:21.765
<00:14:20.175>let's <00:14:20.454>take <00:14:20.734>a <00:14:21.015>small <00:14:21.295>step <00:14:21.574>back

336
00:14:21.765 --> 00:14:24.005
<00:14:21.765>and <00:14:21.965>first <00:14:22.165>learn <00:14:22.365>a <00:14:22.564>bit <00:14:22.765>more <00:14:22.965>about <00:14:23.165>acme,

337
00:14:24.065 --> 00:14:26.365
<00:14:24.065>get <00:14:24.265>a <00:14:24.465>bit <00:14:24.665>more <00:14:24.865>technical <00:14:25.065>context <00:14:25.265>actually <00:14:25.465>to,

338
00:14:26.545 --> 00:14:28.245
<00:14:26.545>to <00:14:26.895>enable <00:14:27.245>us <00:14:27.594>understand <00:14:27.944>the

339
00:14:28.365 --> 00:14:29.645
<00:14:28.365>challenges <00:14:28.595>at <00:14:28.825>least <00:14:29.056>a <00:14:29.286>bit <00:14:29.516>better.

340
00:14:30.225 --> 00:14:31.765
<00:14:30.225>So <00:14:30.502>from <00:14:30.779>a <00:14:31.056>tech <00:14:31.333>perspective, <00:14:31.611>um,

341
00:14:32.075 --> 00:14:34.005
<00:14:32.075>Acme <00:14:32.355>manages <00:14:32.635>their <00:14:32.915>own <00:14:33.195>data <00:14:33.475>center

342
00:14:34.905 --> 00:14:36.845
<00:14:34.905>and <00:14:35.060>they <00:14:35.216>run <00:14:35.371>a <00:14:35.527>Kubernetes <00:14:35.682>cluster <00:14:35.838>in <00:14:35.993>the <00:14:36.149>data <00:14:36.305>center.

343
00:14:37.195 --> 00:14:41.165
<00:14:37.195>They <00:14:37.370>develop <00:14:37.545>all <00:14:37.720>their <00:14:37.895>company <00:14:38.070>internal <00:14:38.245>software <00:14:38.420>in <00:14:38.595>with,

344
00:14:41.425 --> 00:14:44.365
<00:14:41.425>um, <00:14:41.564>using <00:14:41.704>Java <00:14:41.844>on <00:14:41.984>the <00:14:42.125>backend, <00:14:42.265>react <00:14:42.405>on <00:14:42.545>the <00:14:42.685>front <00:14:42.824>end,

345
00:14:44.545 --> 00:14:48.325
<00:14:44.545>and <00:14:44.719>they <00:14:44.895>deploy <00:14:45.069>them, <00:14:45.245>uh, <00:14:45.420>they <00:14:45.594>deploy <00:14:45.770>the <00:14:45.944>software

346
00:14:48.325 --> 00:14:52.645
<00:14:48.325>as <00:14:48.500>microservices <00:14:48.675>in <00:14:48.850>their <00:14:49.025>Kubernetes <00:14:49.200>cluster <00:14:49.375>from, <00:14:49.550>uh, <00:14:49.725>well,

347
00:14:52.645 --> 00:14:56.605
<00:14:52.645>of <00:14:52.800>course <00:14:52.956>they <00:14:53.111>know <00:14:53.267>that, <00:14:53.422>you <00:14:53.578>know, <00:14:53.733>ACME <00:14:53.889>produces <00:14:54.045>tires

348
00:14:57.105 --> 00:14:58.725
<00:14:57.105>and <00:14:57.385>managing <00:14:57.665>their <00:14:57.945>own <00:14:58.225>Kubernetes <00:14:58.505>cluster,

349
00:14:59.125 --> 00:15:00.125
<00:14:59.125>managing <00:14:59.350>their <00:14:59.575>own <00:14:59.800>data <00:15:00.025>center.

350
00:15:00.425 --> 00:15:02.965
<00:15:00.425>That's <00:15:00.625>something <00:15:00.824>that's <00:15:01.025>just <00:15:01.224>not <00:15:01.425>really <00:15:01.625>their <00:15:01.824>core

351
00:15:02.965 --> 00:15:05.965
<00:15:02.965>capabilities <00:15:03.165>or <00:15:03.365>they <00:15:03.565>know, <00:15:03.765>all <00:15:03.965>right, <00:15:04.165>we <00:15:04.365>want

352
00:15:05.965 --> 00:15:07.045
<00:15:05.965>to <00:15:06.208>move <00:15:06.451>to <00:15:06.694>the <00:15:06.937>cloud.

353
00:15:07.425 --> 00:15:09.925
<00:15:07.425>Our <00:15:07.580>objective <00:15:07.736>is <00:15:07.891>the <00:15:08.047>next <00:15:08.202>two <00:15:08.358>years <00:15:08.513>we <00:15:08.669>wanna <00:15:08.824>migrate

354
00:15:09.925 --> 00:15:11.765
<00:15:09.925>everything <00:15:10.158>to <00:15:10.391>the <00:15:10.625>cloud <00:15:10.858>and <00:15:11.091>actually <00:15:11.324>become

355
00:15:11.765 --> 00:15:12.805
<00:15:11.765>a <00:15:12.077>cloud <00:15:12.389>native <00:15:12.700>company.

356
00:15:13.145 --> 00:15:15.565
<00:15:13.145>In <00:15:13.378>our <00:15:13.611>case <00:15:13.845>here, <00:15:14.078>um, <00:15:14.311>on <00:15:14.545>AWS,

357
00:15:15.665 --> 00:15:17.045
<00:15:15.665>and <00:15:15.842>I <00:15:16.019>think <00:15:16.197>here <00:15:16.374>it's <00:15:16.552>important <00:15:16.729>to <00:15:16.906>note

358
00:15:17.045 --> 00:15:19.845
<00:15:17.045>that <00:15:17.172>throughout <00:15:17.299>this <00:15:17.426>case <00:15:17.554>study <00:15:17.681>we <00:15:17.808>are <00:15:17.935>gonna <00:15:18.063>see <00:15:18.190>a <00:15:18.317>WSA <00:15:18.444>lot,

359
00:15:20.345 --> 00:15:22.045
<00:15:20.345>but <00:15:20.695>of <00:15:21.045>course, <00:15:21.395>um, <00:15:21.745>essentially

360
00:15:22.045 --> 00:15:26.245
<00:15:22.045>what <00:15:22.185>we <00:15:22.324>are <00:15:22.464>gonna <00:15:22.604>see <00:15:22.745>is <00:15:22.885>basically <00:15:23.025>also <00:15:23.165>able, <00:15:23.305>um, <00:15:23.444>possible

361
00:15:26.245 --> 00:15:28.965
<00:15:26.245>to <00:15:26.420>do <00:15:26.595>with <00:15:26.770>any <00:15:26.945>other <00:15:27.120>major <00:15:27.295>cloud <00:15:27.470>provider, <00:15:27.645>right?

362
00:15:28.965 --> 00:15:30.085
<00:15:28.965>So <00:15:29.217>it's <00:15:29.469>just, <00:15:29.721>it's <00:15:29.973>not,

363
00:15:30.085 --> 00:15:32.125
<00:15:30.085>it's <00:15:30.365>not <00:15:30.645>a <00:15:30.925>specific <00:15:31.205>AWS <00:15:31.485>advertisement

364
00:15:32.875 --> 00:15:34.285
<00:15:32.875>from <00:15:33.192>a <00:15:33.509>data <00:15:33.826>science <00:15:34.144>perspective.

365
00:15:34.875 --> 00:15:36.845
<00:15:34.875>Well, <00:15:35.075>they <00:15:35.275>have <00:15:35.475>ramped <00:15:35.675>up <00:15:35.875>the <00:15:36.075>data <00:15:36.275>science

366
00:15:36.845 --> 00:15:38.205
<00:15:36.845>effort <00:15:37.089>over <00:15:37.334>the <00:15:37.579>last <00:15:37.824>few <00:15:38.069>years.

367
00:15:38.235 --> 00:15:40.565
<00:15:38.235>They've <00:15:38.410>hired <00:15:38.585>a <00:15:38.760>data <00:15:38.935>science <00:15:39.110>team <00:15:39.285>to <00:15:39.460>be <00:15:39.635>precise,

368
00:15:40.625 --> 00:15:41.725
<00:15:40.625>six <00:15:41.120>data <00:15:41.615>scientists

369
00:15:42.065 --> 00:15:44.165
<00:15:42.065>and <00:15:42.298>these <00:15:42.531>data <00:15:42.765>scientists, <00:15:42.998>they <00:15:43.231>use <00:15:43.465>Python

370
00:15:44.425 --> 00:15:47.285
<00:15:44.425>for <00:15:44.580>the <00:15:44.736>data <00:15:44.891>science <00:15:45.047>and <00:15:45.202>machine <00:15:45.358>learning <00:15:45.513>work <00:15:45.669>so <00:15:45.824>far,

371
00:15:47.755 --> 00:15:49.205
<00:15:47.755>they <00:15:48.081>have <00:15:48.407>deployed <00:15:48.733>one <00:15:49.060>model,

372
00:15:49.465 --> 00:15:50.565
<00:15:49.465>and <00:15:49.712>that's <00:15:49.960>running <00:15:50.207>in <00:15:50.455>production.

373
00:15:50.565 --> 00:15:52.765
<00:15:50.565>It's <00:15:50.740>running <00:15:50.915>in <00:15:51.090>a <00:15:51.265>microservice <00:15:51.440>on <00:15:51.615>the <00:15:51.790>Kubernetes <00:15:51.965>cluster

374
00:15:53.065 --> 00:15:54.845
<00:15:53.065>that's <00:15:53.415>doing <00:15:53.765>some <00:15:54.115>LY <00:15:54.465>detection.

375
00:15:56.315 --> 00:15:59.045
<00:15:56.315>Okay, <00:15:56.490>so <00:15:56.665>much <00:15:56.840>on <00:15:57.015>the <00:15:57.190>tech <00:15:57.365>side <00:15:57.540>of <00:15:57.715>Acme.

376
00:15:59.065 --> 00:16:01.325
<00:15:59.065>So <00:15:59.205>you <00:15:59.345>probably <00:15:59.485>got <00:15:59.625>a <00:15:59.765>bit <00:15:59.905>of <00:16:00.045>an <00:16:00.185>idea <00:16:00.325>of <00:16:00.465>what's,

377
00:16:01.325 --> 00:16:03.045
<00:16:01.325>what's <00:16:01.558>happening <00:16:01.791>in <00:16:02.025>that <00:16:02.258>company <00:16:02.491>right <00:16:02.725>now?

378
00:16:04.475 --> 00:16:07.285
<00:16:04.475>Well, <00:16:04.630>let's <00:16:04.786>now <00:16:04.941>sit <00:16:05.097>together <00:16:05.252>with <00:16:05.408>the <00:16:05.563>data <00:16:05.719>science <00:16:05.875>team

379
00:16:07.745 --> 00:16:09.085
<00:16:07.745>to <00:16:08.046>understand <00:16:08.348>a <00:16:08.649>bit <00:16:08.951>better

380
00:16:09.905 --> 00:16:12.285
<00:16:09.905>how <00:16:10.185>they <00:16:10.464>currently <00:16:10.745>actually <00:16:11.025>develop <00:16:11.305>their

381
00:16:12.285 --> 00:16:13.445
<00:16:12.285>machine <00:16:12.807>learning <00:16:13.329>solutions.

382
00:16:14.225 --> 00:16:17.535
<00:16:14.225>So <00:16:14.352>we <00:16:14.479>meet <00:16:14.606>up <00:16:14.734>with <00:16:14.861>the <00:16:14.988>data <00:16:15.115>science <00:16:15.243>team, <00:16:15.370>you <00:16:15.497>know, <00:16:15.625>it's,

383
00:16:17.535 --> 00:16:18.735
<00:16:17.535>uh, <00:16:17.805>assume <00:16:18.075>it's <00:16:18.345>post <00:16:18.615>COVID.

384
00:16:18.835 --> 00:16:20.775
<00:16:18.835>So <00:16:19.010>we <00:16:19.185>meet <00:16:19.360>up <00:16:19.535>with <00:16:19.710>the <00:16:19.885>data <00:16:20.060>science <00:16:20.235>team

385
00:16:21.235 --> 00:16:23.455
<00:16:21.235>and <00:16:21.362>we <00:16:21.489>get <00:16:21.616>into <00:16:21.744>a, <00:16:21.871>we <00:16:21.998>get <00:16:22.125>into <00:16:22.253>a <00:16:22.380>large <00:16:22.507>meeting <00:16:22.635>room

386
00:16:23.615 --> 00:16:25.015
<00:16:23.615>and <00:16:23.867>there <00:16:24.119>is <00:16:24.371>this <00:16:24.623>massive <00:16:24.875>whiteboard

387
00:16:25.355 --> 00:16:26.495
<00:16:25.355>and <00:16:25.611>the <00:16:25.868>data <00:16:26.124>science <00:16:26.381>team,

388
00:16:26.805 --> 00:16:28.535
<00:16:26.805>they <00:16:27.084>start <00:16:27.364>drawing <00:16:27.645>on <00:16:27.925>that <00:16:28.204>whiteboard.

389
00:16:28.535 --> 00:16:30.695
<00:16:28.535>They <00:16:28.768>really <00:16:29.001>start <00:16:29.235>sketching <00:16:29.468>out <00:16:29.701>their <00:16:29.935>process,

390
00:16:31.005 --> 00:16:32.855
<00:16:31.005>what <00:16:31.238>they're <00:16:31.471>currently <00:16:31.705>doing, <00:16:31.938>how <00:16:32.171>they're <00:16:32.405>working,

391
00:16:33.155 --> 00:16:35.295
<00:16:33.155>and <00:16:33.310>what <00:16:33.466>they <00:16:33.621>sketch <00:16:33.777>out <00:16:33.932>kind <00:16:34.088>of <00:16:34.243>looks <00:16:34.399>like <00:16:34.555>this.

392
00:16:36.315 --> 00:16:37.855
<00:16:36.315>Um, <00:16:36.546>so <00:16:36.777>we <00:16:37.008>can, <00:16:37.239>we <00:16:37.470>can <00:16:37.701>see

393
00:16:37.855 --> 00:16:39.975
<00:16:37.855>that <00:16:38.088>they're <00:16:38.321>basically <00:16:38.555>having <00:16:38.788>these <00:16:39.021>three <00:16:39.255>phases.

394
00:16:39.975 --> 00:16:42.055
<00:16:39.975>They're <00:16:40.325>having <00:16:40.675>an <00:16:41.025>experimentation <00:16:41.375>phase

395
00:16:42.055 --> 00:16:43.575
<00:16:42.055>that <00:16:42.511>then <00:16:42.967>they <00:16:43.423>productionize

396
00:16:43.835 --> 00:16:45.855
<00:16:43.835>and <00:16:44.035>then, <00:16:44.235>uh, <00:16:44.435>they <00:16:44.635>deploy <00:16:44.835>to <00:16:45.035>the <00:16:45.235>cluster

397
00:16:46.075 --> 00:16:47.695
<00:16:46.075>and <00:16:46.425>the <00:16:46.775>serving <00:16:47.125>happens <00:16:47.475>there.

398
00:16:47.995 --> 00:16:50.095
<00:16:47.995>That's <00:16:48.228>kind <00:16:48.461>of <00:16:48.695>the <00:16:48.928>big <00:16:49.161>three <00:16:49.395>phases.

399
00:16:50.995 --> 00:16:53.335
<00:16:50.995>Diving <00:16:51.170>in <00:16:51.345>into <00:16:51.520>that <00:16:51.695>in <00:16:51.870>a <00:16:52.045>bit <00:16:52.220>more <00:16:52.395>detail,

400
00:16:53.395 --> 00:16:56.335
<00:16:53.395>we <00:16:53.535>can <00:16:53.675>see <00:16:53.814>on <00:16:53.954>the <00:16:54.095>very <00:16:54.235>left <00:16:54.375>their <00:16:54.515>machine <00:16:54.655>data, <00:16:54.795>they,

401
00:16:56.335 --> 00:16:58.855
<00:16:56.335>they <00:16:56.568>host <00:16:56.801>that <00:16:57.035>on, <00:16:57.268>on <00:16:57.501>Amazon <00:16:57.735>Redshift,

402
00:16:58.855 --> 00:17:01.455
<00:16:58.855>which <00:16:59.135>is <00:16:59.415>AWS <00:16:59.695>data <00:16:59.975>warehouse <00:17:00.255>essentially,

403
00:17:02.395 --> 00:17:05.175
<00:17:02.395>and <00:17:02.550>data <00:17:02.706>scientists <00:17:02.861>get <00:17:03.017>the <00:17:03.172>data <00:17:03.328>from <00:17:03.483>there <00:17:03.639>using <00:17:03.795>sql,

404
00:17:05.365 --> 00:17:07.415
<00:17:05.365>pulling <00:17:05.598>that <00:17:05.831>data <00:17:06.065>onto <00:17:06.298>the <00:17:06.531>local <00:17:06.765>computer

405
00:17:07.425 --> 00:17:09.535
<00:17:07.425>where <00:17:07.625>they <00:17:07.825>have <00:17:08.024>a <00:17:08.224>local <00:17:08.425>Jupyter <00:17:08.625>Notebook <00:17:08.825>instance

406
00:17:09.535 --> 00:17:10.775
<00:17:09.535>running <00:17:10.651>there.

407
00:17:10.805 --> 00:17:13.295
<00:17:10.805>They <00:17:11.038>do <00:17:11.271>the, <00:17:11.505>you <00:17:11.738>know, <00:17:11.971>exploratory <00:17:12.205>analysis,

408
00:17:13.295 --> 00:17:15.615
<00:17:13.295>some <00:17:13.575>statistics, <00:17:13.855>uh, <00:17:14.135>some <00:17:14.415>model <00:17:14.695>training,

409
00:17:15.615 --> 00:17:17.375
<00:17:15.615>some <00:17:15.895>experimentation, <00:17:16.175>and <00:17:16.454>at <00:17:16.734>some <00:17:17.015>point

410
00:17:18.085 --> 00:17:19.655
<00:17:18.085>they <00:17:18.365>are <00:17:18.645>happy <00:17:18.925>with <00:17:19.204>that <00:17:19.485>model.

411
00:17:20.195 --> 00:17:22.055
<00:17:20.194>You <00:17:20.428>know, <00:17:20.661>they, <00:17:20.895>they <00:17:21.128>have <00:17:21.361>reached <00:17:21.595>their,

412
00:17:22.055 --> 00:17:23.335
<00:17:22.055>reached <00:17:22.439>their <00:17:22.823>accuracy <00:17:23.207>goal

413
00:17:23.915 --> 00:17:27.495
<00:17:23.915>and <00:17:24.042>they <00:17:24.169>push <00:17:24.296>it <00:17:24.424>then <00:17:24.551>to <00:17:24.678>Amazon <00:17:24.805>S3 <00:17:24.933>to <00:17:25.060>some <00:17:25.187>storage <00:17:25.315>bucket,

414
00:17:27.955 --> 00:17:31.015
<00:17:27.954>you <00:17:28.129>know, <00:17:28.304>where <00:17:28.480>basically <00:17:28.655>the <00:17:28.830>production <00:17:29.004>ready <00:17:29.179>models <00:17:29.355>live.

415
00:17:32.045 --> 00:17:35.335
<00:17:32.045>That <00:17:32.245>means <00:17:32.445>they <00:17:32.645>want <00:17:32.845>to <00:17:33.045>continue <00:17:33.245>towards <00:17:33.445>production.

416
00:17:36.035 --> 00:17:37.535
<00:17:36.035>So <00:17:36.227>first <00:17:36.420>thing <00:17:36.613>they're <00:17:36.806>gonna <00:17:36.999>do <00:17:37.192>is <00:17:37.385>actually,

417
00:17:37.955 --> 00:17:39.535
<00:17:37.955>uh, <00:17:38.235>a <00:17:38.514>lot <00:17:38.794>of <00:17:39.074>copy <00:17:39.355>pasting.

418
00:17:39.725 --> 00:17:42.775
<00:17:39.725>They <00:17:39.852>copy <00:17:39.979>paste <00:17:40.106>a <00:17:40.234>lot <00:17:40.361>of <00:17:40.488>their <00:17:40.615>code <00:17:40.743>out <00:17:40.870>the <00:17:40.997>Jupyter <00:17:41.125>Notebook

419
00:17:42.775 --> 00:17:45.815
<00:17:42.775>to <00:17:42.950>create <00:17:43.125>the <00:17:43.300>Python <00:17:43.475>module, <00:17:43.650>uh, <00:17:43.825>rewrite <00:17:44.000>some <00:17:44.175>code

420
00:17:45.995 --> 00:17:48.415
<00:17:45.995>to <00:17:46.344>establish <00:17:46.695>their <00:17:47.044>inference <00:17:47.395>pipeline.

421
00:17:48.435 --> 00:17:50.965
<00:17:48.435>So <00:17:48.590>actually <00:17:48.746>the <00:17:48.901>thing <00:17:49.057>that <00:17:49.212>should <00:17:49.368>be <00:17:49.523>running <00:17:49.679>in <00:17:49.835>their

422
00:17:50.965 --> 00:17:54.375
<00:17:50.965>microservice <00:17:51.245>in <00:17:51.524>the <00:17:51.804>Kubernetes <00:17:52.084>cluster, <00:17:52.365>they

423
00:17:54.375 --> 00:17:55.535
<00:17:54.375>of <00:17:54.583>course <00:17:54.792>also <00:17:55.001>add <00:17:55.210>some <00:17:55.419>tests.

424
00:17:55.675 --> 00:17:56.815
<00:17:55.675>You <00:17:55.880>know, <00:17:56.085>in <00:17:56.290>their <00:17:56.495>case <00:17:56.701>they,

425
00:17:56.925 --> 00:17:59.215
<00:17:56.925>it's <00:17:57.158>mainly <00:17:57.391>unit <00:17:57.625>tests, <00:17:57.858>um, <00:17:58.091>on <00:17:58.325>code.

426
00:17:59.555 --> 00:18:02.055
<00:17:59.555>And <00:17:59.695>once <00:17:59.835>that's <00:17:59.975>done, <00:18:00.115>you <00:18:00.255>know, <00:18:00.395>they <00:18:00.535>have <00:18:00.675>this <00:18:00.815>Python <00:18:00.955>module

427
00:18:02.565 --> 00:18:05.095
<00:18:02.565>test <00:18:02.705>it, <00:18:02.845>at <00:18:02.985>least <00:18:03.125>to <00:18:03.265>some <00:18:03.405>extent, <00:18:03.545>uh, <00:18:03.685>they <00:18:03.825>take <00:18:03.965>that

428
00:18:05.155 --> 00:18:06.615
<00:18:05.155>and <00:18:05.417>go <00:18:05.680>to <00:18:05.943>the <00:18:06.206>DevOps <00:18:06.469>team

429
00:18:06.615 --> 00:18:08.015
<00:18:06.615>and <00:18:06.930>say, <00:18:07.245>Hey <00:18:07.560>guys, <00:18:07.875>please,

430
00:18:08.425 --> 00:18:11.295
<00:18:08.425>could <00:18:08.625>you <00:18:08.825>please <00:18:09.024>deploy <00:18:09.225>this <00:18:09.425>in <00:18:09.625>our <00:18:09.825>cluster?

431
00:18:12.495 --> 00:18:15.955
<00:18:12.495>And <00:18:12.669>the <00:18:12.844>DevOps <00:18:13.020>team <00:18:13.195>says, <00:18:13.370>yeah, <00:18:13.544>sure, <00:18:13.719>no <00:18:13.895>problem.

432
00:18:16.335 --> 00:18:17.875
<00:18:16.335>And <00:18:16.566>they <00:18:16.797>add <00:18:17.028>it <00:18:17.259>to <00:18:17.490>the <00:18:17.721>backlog.

433
00:18:18.705 --> 00:18:21.885
<00:18:18.705>So <00:18:18.845>that's <00:18:18.985>the <00:18:19.125>current <00:18:19.264>process, <00:18:19.405>how <00:18:19.544>that <00:18:19.685>works <00:18:19.824>at <00:18:19.965>some <00:18:20.105>time.

434
00:18:21.885 --> 00:18:25.285
<00:18:21.885>Then <00:18:22.060>in <00:18:22.235>future, <00:18:22.410>that <00:18:22.585>model <00:18:22.760>is <00:18:22.935>gonna <00:18:23.110>run <00:18:23.285>in

435
00:18:25.285 --> 00:18:26.405
<00:18:25.285>production <00:18:25.621>on <00:18:25.957>the <00:18:26.293>cluster.

436
00:18:27.945 --> 00:18:31.325
<00:18:27.945>Of <00:18:28.100>course, <00:18:28.256>the <00:18:28.411>data <00:18:28.567>science <00:18:28.722>team <00:18:28.878>also <00:18:29.033>says, <00:18:29.189>you <00:18:29.345>know,

437
00:18:31.325 --> 00:18:32.645
<00:18:31.325>there <00:18:31.523>are <00:18:31.721>a <00:18:31.919>few <00:18:32.117>things <00:18:32.315>going <00:18:32.513>wrong.

438
00:18:33.025 --> 00:18:35.965
<00:18:33.025>So <00:18:33.180>to <00:18:33.336>be <00:18:33.491>very <00:18:33.647>honest, <00:18:33.802>one <00:18:33.958>thing <00:18:34.113>is <00:18:34.269>definitely <00:18:34.425>right.

439
00:18:35.985 --> 00:18:37.205
<00:18:35.985>We <00:18:36.259>can't <00:18:36.533>do <00:18:36.808>scalable <00:18:37.082>training.

440
00:18:37.385 --> 00:18:39.605
<00:18:37.385>We <00:18:37.585>train <00:18:37.785>on <00:18:37.985>our <00:18:38.185>local <00:18:38.385>computers, <00:18:38.585>you <00:18:38.785>know,

441
00:18:39.605 --> 00:18:41.845
<00:18:39.605>machine <00:18:39.838>data <00:18:40.071>can <00:18:40.305>be <00:18:40.538>quite <00:18:40.771>large, <00:18:41.005>so,

442
00:18:42.705 --> 00:18:44.405
<00:18:42.705>and <00:18:42.938>we <00:18:43.171>just <00:18:43.405>can't <00:18:43.638>live <00:18:43.871>leverage <00:18:44.105>that.

443
00:18:44.745 --> 00:18:46.445
<00:18:44.745>So <00:18:44.978>that's <00:18:45.211>a <00:18:45.445>big <00:18:45.678>problem <00:18:45.911>they're <00:18:46.145>seeing.

444
00:18:46.555 --> 00:18:48.525
<00:18:46.555>Then <00:18:46.905>also <00:18:47.255>reproducibility, <00:18:47.605>you <00:18:47.955>know,

445
00:18:48.525 --> 00:18:49.885
<00:18:48.525>they <00:18:48.729>just <00:18:48.933>do <00:18:49.137>not <00:18:49.341>have <00:18:49.545>any <00:18:49.749>process

446
00:18:50.225 --> 00:18:55.165
<00:18:50.225>or <00:18:50.425>tool <00:18:50.625>support <00:18:50.824>to <00:18:51.024>ensure <00:18:51.225>reproducibility <00:18:51.425>across <00:18:51.625>experiments

447
00:18:55.165 --> 00:18:56.685
<00:18:55.165>and <00:18:55.507>also <00:18:55.849>across <00:18:56.191>data <00:18:56.533>scientists.

448
00:18:56.985 --> 00:19:00.245
<00:18:56.985>Of <00:18:57.125>course, <00:18:57.264>they <00:18:57.405>say <00:18:57.544>some <00:18:57.685>data <00:18:57.824>scientists <00:18:57.965>take <00:18:58.104>some <00:18:58.245>notes <00:18:58.385>in

449
00:19:00.245 --> 00:19:03.165
<00:19:00.245>the <00:19:00.445>Jupyter <00:19:00.645>Notebooks, <00:19:00.844>some <00:19:01.044>markdown <00:19:01.245>to <00:19:01.445>actually <00:19:01.645>remember

450
00:19:03.195 --> 00:19:05.445
<00:19:03.195>what <00:19:03.370>experiments <00:19:03.544>they <00:19:03.720>did <00:19:03.895>and <00:19:04.070>how <00:19:04.245>they <00:19:04.419>turned <00:19:04.595>out.

451
00:19:05.785 --> 00:19:08.605
<00:19:05.785>But <00:19:05.985>to <00:19:06.185>be <00:19:06.385>honest, <00:19:06.585>that's <00:19:06.785>really <00:19:06.985>not <00:19:07.185>comprehensive

452
00:19:08.825 --> 00:19:11.285
<00:19:08.825>and <00:19:08.980>it's <00:19:09.136>also <00:19:09.291>hard <00:19:09.447>for <00:19:09.602>other <00:19:09.758>data <00:19:09.913>scientists <00:19:10.069>to <00:19:10.225>read.

453
00:19:13.635 --> 00:19:15.405
<00:19:13.635>Consistency <00:19:13.915>is <00:19:14.195>also <00:19:14.475>a <00:19:14.754>frequent <00:19:15.035>issue.

454
00:19:16.425 --> 00:19:18.645
<00:19:16.425>So <00:19:16.580>they <00:19:16.736>write <00:19:16.891>the <00:19:17.047>training <00:19:17.202>pipelines <00:19:17.358>usually <00:19:17.513>in <00:19:17.669>the <00:19:17.825>Jupyter

455
00:19:18.805 --> 00:19:21.765
<00:19:18.805>Notebook, <00:19:18.960>and <00:19:19.116>then <00:19:19.271>they, <00:19:19.427>they <00:19:19.582>copy <00:19:19.738>paste <00:19:19.893>a <00:19:20.049>few <00:19:20.205>snippets

456
00:19:21.765 --> 00:19:25.285
<00:19:21.765>of <00:19:21.872>code <00:19:21.980>and <00:19:22.088>then <00:19:22.195>rewrite <00:19:22.303>a <00:19:22.411>bit <00:19:22.518>of <00:19:22.626>a <00:19:22.734>code <00:19:22.841>to <00:19:22.949>get <00:19:23.057>to <00:19:23.165>get

457
00:19:25.285 --> 00:19:27.485
<00:19:25.285>to <00:19:25.460>the <00:19:25.635>inference <00:19:25.810>pipeline, <00:19:25.985>which <00:19:26.160>is <00:19:26.335>then <00:19:26.510>deployed <00:19:26.685>inside,

458
00:19:27.745 --> 00:19:30.565
<00:19:27.745>uh, <00:19:27.919>the, <00:19:28.094>in <00:19:28.270>inside <00:19:28.445>the <00:19:28.620>microservice <00:19:28.794>on <00:19:28.969>the <00:19:29.145>cluster.

459
00:19:31.025 --> 00:19:32.085
<00:19:31.025>And <00:19:31.263>of <00:19:31.502>course, <00:19:31.740>you <00:19:31.979>know,

460
00:19:32.085 --> 00:19:34.725
<00:19:32.085>if <00:19:32.260>they <00:19:32.435>change <00:19:32.610>small <00:19:32.785>things <00:19:32.960>in <00:19:33.135>the <00:19:33.310>training <00:19:33.485>pipeline

461
00:19:35.065 --> 00:19:37.085
<00:19:35.065>and <00:19:35.415>they <00:19:35.765>forget <00:19:36.115>about <00:19:36.465>implementing

462
00:19:37.085 --> 00:19:39.765
<00:19:37.085>that <00:19:37.260>also <00:19:37.435>in <00:19:37.610>the <00:19:37.785>inference <00:19:37.960>pipeline, <00:19:38.135>um, <00:19:38.310>you <00:19:38.485>know,

463
00:19:39.765 --> 00:19:41.925
<00:19:39.765>that <00:19:40.045>really <00:19:40.325>results <00:19:40.605>in <00:19:40.885>the <00:19:41.165>models

464
00:19:41.925 --> 00:19:43.165
<00:19:41.925>predicting <00:19:42.483>absolute <00:19:43.041>garbage.

465
00:19:43.545 --> 00:19:46.845
<00:19:43.545>That's <00:19:43.778>a <00:19:44.011>big <00:19:44.245>issue. <00:19:44.478>They're <00:19:44.711>also <00:19:44.945>aware

466
00:19:46.845 --> 00:19:48.285
<00:19:46.845>that <00:19:47.104>code <00:19:47.363>testing <00:19:47.622>is <00:19:47.881>not <00:19:48.141>everything.

467
00:19:48.675 --> 00:19:52.405
<00:19:48.675>Yeah. <00:19:48.850>You, <00:19:49.024>in <00:19:49.200>the <00:19:49.375>machine <00:19:49.550>learning <00:19:49.725>system, <00:19:49.899>you're <00:19:50.075>dealing

468
00:19:52.405 --> 00:19:53.725
<00:19:52.405>with <00:19:52.702>a <00:19:52.999>variety <00:19:53.295>of <00:19:53.592>artifacts.

469
00:19:54.225 --> 00:19:57.845
<00:19:54.225>So <00:19:54.399>only <00:19:54.574>test, <00:19:54.750>well, <00:19:54.925>code <00:19:55.100>testing <00:19:55.274>is <00:19:55.449>important, <00:19:55.625>right?

470
00:19:57.945 --> 00:19:59.045
<00:19:57.945>But <00:19:58.275>it's <00:19:58.605>not <00:19:58.935>everything.

471
00:19:59.065 --> 00:20:01.885
<00:19:59.065>You <00:19:59.220>also <00:19:59.376>need <00:19:59.531>to <00:19:59.687>think <00:19:59.842>about <00:19:59.998>testing <00:20:00.153>a <00:20:00.309>model, <00:20:00.465>right?

472
00:20:01.895 --> 00:20:03.565
<00:20:01.895>Their <00:20:02.175>models <00:20:02.455>could <00:20:02.735>be, <00:20:03.014>for <00:20:03.295>example,

473
00:20:03.585 --> 00:20:05.765
<00:20:03.585>biased <00:20:04.051>towards <00:20:04.518>certain <00:20:04.985>machines.

474
00:20:05.825 --> 00:20:08.165
<00:20:05.825>You <00:20:06.000>mentioned <00:20:06.175>they <00:20:06.350>have <00:20:06.525>some <00:20:06.700>animal <00:20:06.875>e <00:20:07.050>detection <00:20:07.225>running

475
00:20:08.225 --> 00:20:11.365
<00:20:08.225>across <00:20:08.425>a <00:20:08.625>variety <00:20:08.824>of <00:20:09.024>different, <00:20:09.225>very <00:20:09.425>similar <00:20:09.625>machines,

476
00:20:11.705 --> 00:20:13.205
<00:20:11.705>but <00:20:11.929>perhaps <00:20:12.155>they <00:20:12.379>have <00:20:12.605>a <00:20:12.830>bias <00:20:13.054>towards

477
00:20:13.205 --> 00:20:14.245
<00:20:13.205>certain <00:20:13.516>machines <00:20:13.829>in <00:20:14.140>there.

478
00:20:15.105 --> 00:20:17.525
<00:20:15.105>Um, <00:20:15.305>or <00:20:15.505>also <00:20:15.705>they <00:20:15.905>should <00:20:16.105>consider <00:20:16.305>data <00:20:16.505>testing.

479
00:20:17.865 --> 00:20:20.805
<00:20:17.865>So <00:20:18.005>actually <00:20:18.145>checking <00:20:18.285>what <00:20:18.425>are <00:20:18.565>the <00:20:18.705>data <00:20:18.845>that <00:20:18.985>comes <00:20:19.125>in <00:20:19.265>does,

480
00:20:21.185 --> 00:20:25.925
<00:20:21.185>um, <00:20:21.325>does <00:20:21.465>look <00:20:21.605>like <00:20:21.745>what <00:20:21.885>they <00:20:22.024>expected <00:20:22.165>to <00:20:22.304>see <00:20:22.445>so <00:20:22.585>far?

481
00:20:25.925 --> 00:20:27.525
<00:20:25.925>They <00:20:26.100>don't <00:20:26.274>do <00:20:26.450>that. <00:20:26.625>They <00:20:26.800>only <00:20:26.975>do <00:20:27.149>code <00:20:27.325>tests

482
00:20:27.665 --> 00:20:30.405
<00:20:27.665>and <00:20:27.820>they <00:20:27.976>do <00:20:28.131>not <00:20:28.287>have <00:20:28.442>any <00:20:28.598>standardized <00:20:28.753>procedure <00:20:28.909>of, <00:20:29.065>and,

483
00:20:30.505 --> 00:20:32.085
<00:20:30.505>and, <00:20:30.785>um, <00:20:31.065>a <00:20:31.345>code <00:20:31.625>of <00:20:31.905>conduct

484
00:20:32.315 --> 00:20:33.885
<00:20:32.315>what <00:20:32.595>they <00:20:32.875>should <00:20:33.155>actually <00:20:33.435>be <00:20:33.715>testing.

485
00:20:35.675 --> 00:20:37.885
<00:20:35.675>Well, <00:20:35.875>uh, <00:20:36.075>the <00:20:36.274>handover, <00:20:36.475>this <00:20:36.675>is <00:20:36.875>really <00:20:37.075>something

486
00:20:37.885 --> 00:20:40.805
<00:20:37.885>that <00:20:38.085>hurts <00:20:38.285>them <00:20:38.485>a <00:20:38.685>lot <00:20:38.885>because <00:20:39.085>for <00:20:39.285>the,

487
00:20:40.835 --> 00:20:43.445
<00:20:40.835>when <00:20:40.975>they <00:20:41.115>hand <00:20:41.255>over <00:20:41.395>the <00:20:41.535>Python <00:20:41.675>module <00:20:41.815>to <00:20:41.955>the <00:20:42.095>DevOps <00:20:42.235>team,

488
00:20:43.915 --> 00:20:46.365
<00:20:43.915>well, <00:20:44.055>for <00:20:44.195>the <00:20:44.335>DevOps <00:20:44.475>team, <00:20:44.615>it's <00:20:44.754>not <00:20:44.895>really <00:20:45.034>a <00:20:45.175>priority <00:20:45.315>to,

489
00:20:46.385 --> 00:20:48.245
<00:20:46.385>to <00:20:46.540>get <00:20:46.696>that <00:20:46.851>model <00:20:47.007>up <00:20:47.162>and <00:20:47.318>running <00:20:47.473>on <00:20:47.629>the <00:20:47.785>cluster.

490
00:20:48.425 --> 00:20:49.685
<00:20:48.425>You <00:20:48.614>know, <00:20:48.803>they <00:20:48.992>added <00:20:49.181>to <00:20:49.370>the <00:20:49.559>backlog

491
00:20:49.685 --> 00:20:50.965
<00:20:49.685>and <00:20:49.849>they, <00:20:50.014>they <00:20:50.178>do <00:20:50.343>it <00:20:50.507>at <00:20:50.672>some <00:20:50.837>point,

492
00:20:51.025 --> 00:20:53.045
<00:20:51.025>but <00:20:51.200>to <00:20:51.375>be <00:20:51.550>honest, <00:20:51.725>it <00:20:51.900>takes <00:20:52.075>them <00:20:52.250>usually <00:20:52.425>weeks

493
00:20:53.065 --> 00:20:55.485
<00:20:53.065>and <00:20:53.265>perhaps <00:20:53.465>even <00:20:53.665>a <00:20:53.865>few <00:20:54.065>months <00:20:54.265>to <00:20:54.465>get

494
00:20:55.755 --> 00:20:58.045
<00:20:55.755>that <00:20:56.035>model <00:20:56.315>running <00:20:56.595>on <00:20:56.875>the <00:20:57.155>cluster.

495
00:20:58.865 --> 00:21:01.605
<00:20:58.865>And, <00:20:59.040>uh, <00:20:59.215>yeah, <00:20:59.390>that's, <00:20:59.565>that's <00:20:59.740>a <00:20:59.915>big <00:21:00.090>problem, <00:21:00.265>right?

496
00:21:01.795 --> 00:21:03.165
<00:21:01.795>Because <00:21:02.103>if <00:21:02.411>you <00:21:02.719>wait <00:21:03.028>months

497
00:21:03.655 --> 00:21:06.085
<00:21:03.655>until <00:21:03.830>you <00:21:04.004>actually <00:21:04.180>see <00:21:04.355>your <00:21:04.530>model <00:21:04.705>running <00:21:04.879>in <00:21:05.055>production,

498
00:21:06.515 --> 00:21:08.245
<00:21:06.515>well <00:21:06.865>that, <00:21:07.215>that's <00:21:07.565>really <00:21:07.915>frustrating

499
00:21:08.245 --> 00:21:09.365
<00:21:08.245>from <00:21:08.496>a <00:21:08.749>data <00:21:09.001>science <00:21:09.253>perspective.

500
00:21:09.385 --> 00:21:11.885
<00:21:09.385>And <00:21:09.585>of <00:21:09.785>course <00:21:09.985>also <00:21:10.185>kills <00:21:10.385>your <00:21:10.585>iterations, <00:21:10.785>right?

501
00:21:12.105 --> 00:21:13.725
<00:21:12.105>And <00:21:12.455>your <00:21:12.805>iterations <00:21:13.155>from <00:21:13.505>experiment

502
00:21:13.725 --> 00:21:14.885
<00:21:13.725>to <00:21:14.072>production <00:21:14.421>are <00:21:14.769>essentially,

503
00:21:15.105 --> 00:21:16.605
<00:21:15.105>are <00:21:15.375>essentially <00:21:15.645>dead <00:21:15.915>at <00:21:16.185>this <00:21:16.455>point.

504
00:21:16.945 --> 00:21:18.885
<00:21:16.945>If <00:21:17.100>you, <00:21:17.256>if <00:21:17.411>you <00:21:17.567>notice <00:21:17.722>that <00:21:17.878>something <00:21:18.033>went <00:21:18.189>wrong <00:21:18.345>in

505
00:21:18.885 --> 00:21:21.805
<00:21:18.885>production, <00:21:19.060>you <00:21:19.235>need <00:21:19.410>to <00:21:19.585>go <00:21:19.760>back, <00:21:19.935>um, <00:21:20.110>change <00:21:20.285>something,

506
00:21:22.065 --> 00:21:24.405
<00:21:22.065>and <00:21:22.192>then <00:21:22.319>you <00:21:22.446>need <00:21:22.574>to <00:21:22.701>wait <00:21:22.828>another <00:21:22.955>a <00:21:23.083>few <00:21:23.210>weeks <00:21:23.337>to <00:21:23.465>redeploy

507
00:21:24.405 --> 00:21:26.365
<00:21:24.405>that <00:21:24.754>this <00:21:25.105>is <00:21:25.455>an <00:21:25.805>issue.

508
00:21:27.415 --> 00:21:31.065
<00:21:27.415>Once <00:21:27.615>things <00:21:27.815>are <00:21:28.014>deployed, <00:21:28.215>well, <00:21:28.415>you <00:21:28.615>still <00:21:28.815>need

509
00:21:31.065 --> 00:21:33.145
<00:21:31.065>to <00:21:31.415>think <00:21:31.765>about <00:21:32.115>monitoring <00:21:32.465>acme.

510
00:21:33.285 --> 00:21:34.985
<00:21:33.285>Of <00:21:33.565>course, <00:21:33.845>they <00:21:34.125>do <00:21:34.405>some <00:21:34.685>monitoring,

511
00:21:35.005 --> 00:21:37.505
<00:21:35.005>but <00:21:35.160>you <00:21:35.316>know, <00:21:35.471>the <00:21:35.627>DevOps <00:21:35.782>team <00:21:35.938>is <00:21:36.093>really <00:21:36.249>only <00:21:36.405>familiar

512
00:21:37.505 --> 00:21:39.065
<00:21:37.505>with <00:21:37.785>the <00:21:38.065>classic <00:21:38.345>software <00:21:38.625>engineering <00:21:38.905>side,

513
00:21:39.245 --> 00:21:41.225
<00:21:39.245>and <00:21:39.419>the <00:21:39.594>data <00:21:39.770>scientists <00:21:39.945>are <00:21:40.120>not <00:21:40.294>very <00:21:40.469>familiar <00:21:40.645>with

514
00:21:41.225 --> 00:21:43.385
<00:21:41.225>how <00:21:41.380>they <00:21:41.536>should <00:21:41.691>set <00:21:41.847>up <00:21:42.002>the <00:21:42.158>monitoring <00:21:42.313>in <00:21:42.469>the <00:21:42.625>microservice.

515
00:21:43.725 --> 00:21:46.745
<00:21:43.725>So <00:21:43.899>what <00:21:44.074>they <00:21:44.250>do <00:21:44.425>is <00:21:44.600>just <00:21:44.774>classic <00:21:44.949>software <00:21:45.125>monitoring,

516
00:21:47.205 --> 00:21:48.505
<00:21:47.205>you <00:21:47.399>know, <00:21:47.595>if <00:21:47.790>you <00:21:47.985>could <00:21:48.180>say <00:21:48.375>that.

517
00:21:48.525 --> 00:21:51.305
<00:21:48.525>So <00:21:48.680>that's <00:21:48.836>the <00:21:48.991>classic <00:21:49.147>thing, <00:21:49.302>such <00:21:49.458>as <00:21:49.613>response <00:21:49.769>time <00:21:49.925>and,

518
00:21:51.485 --> 00:21:52.705
<00:21:51.485>and <00:21:51.851>uptime, <00:21:52.216>for <00:21:52.582>example.

519
00:21:53.255 --> 00:21:55.945
<00:21:53.255>They <00:21:53.410>do <00:21:53.566>not <00:21:53.721>actually <00:21:53.877>check <00:21:54.032>whether <00:21:54.188>the <00:21:54.343>input <00:21:54.499>data <00:21:54.655>changes,

520
00:21:55.945 --> 00:21:59.945
<00:21:55.945>whether, <00:21:56.120>you <00:21:56.294>know, <00:21:56.470>some <00:21:56.645>machines <00:21:56.820>produce <00:21:56.995>data, <00:21:57.169>um, <00:21:57.345>that,

521
00:22:00.005 --> 00:22:02.745
<00:22:00.005>that's <00:22:00.160>some <00:22:00.316>that <00:22:00.471>doesn't <00:22:00.627>look <00:22:00.782>like <00:22:00.938>what <00:22:01.093>we've <00:22:01.249>seen <00:22:01.405>before.

522
00:22:04.005 --> 00:22:08.905
<00:22:04.005>So <00:22:04.160>obviously <00:22:04.316>there <00:22:04.471>are <00:22:04.627>quite <00:22:04.782>some <00:22:04.938>pain <00:22:05.093>points <00:22:05.249>in <00:22:05.405>this

523
00:22:08.905 --> 00:22:11.745
<00:22:08.905>process, <00:22:09.060>but <00:22:09.216>at <00:22:09.371>the <00:22:09.527>same <00:22:09.682>time, <00:22:09.838>right, <00:22:09.993>this <00:22:10.149>is, <00:22:10.305>no,

524
00:22:11.745 --> 00:22:15.065
<00:22:11.745>this <00:22:11.919>is <00:22:12.094>not <00:22:12.270>an <00:22:12.445>especially <00:22:12.620>bad <00:22:12.794>machine <00:22:12.969>learning <00:22:13.145>development

525
00:22:15.065 --> 00:22:16.105
<00:22:15.065>process, <00:22:15.377>to <00:22:15.689>be <00:22:16.001>honest.

526
00:22:16.295 --> 00:22:17.345
<00:22:16.295>This <00:22:16.610>is <00:22:16.925>fairly <00:22:17.240>standard.

527
00:22:17.345 --> 00:22:20.745
<00:22:17.345>This <00:22:17.500>is <00:22:17.656>something <00:22:17.811>you <00:22:17.967>see <00:22:18.122>probably <00:22:18.278>in <00:22:18.433>75% <00:22:18.589>of <00:22:18.745>organizations.

528
00:22:21.215 --> 00:22:23.825
<00:22:21.215>Yeah, <00:22:21.389>some <00:22:21.564>organizations <00:22:21.740>not <00:22:21.915>even <00:22:22.090>making <00:22:22.264>it <00:22:22.439>to <00:22:22.615>the,

529
00:22:23.825 --> 00:22:25.465
<00:22:23.825>to <00:22:24.025>the <00:22:24.225>deployment <00:22:24.425>stage <00:22:24.625>at <00:22:24.825>the <00:22:25.025>very <00:22:25.225>right.

530
00:22:26.485 --> 00:22:29.305
<00:22:26.485>So <00:22:26.625>this <00:22:26.764>is <00:22:26.905>actually, <00:22:27.044>I <00:22:27.185>would <00:22:27.324>say <00:22:27.465>quite <00:22:27.604>a <00:22:27.745>standard <00:22:27.885>process,

531
00:22:29.415 --> 00:22:31.505
<00:22:29.415>unfortunately <00:22:29.648>with <00:22:29.881>a <00:22:30.115>lot <00:22:30.348>of <00:22:30.581>pain <00:22:30.815>points,

532
00:22:32.525 --> 00:22:35.865
<00:22:32.525>but <00:22:32.680>this <00:22:32.836>is <00:22:32.991>what <00:22:33.147>it <00:22:33.302>is <00:22:33.458>that's <00:22:33.613>Acme's <00:22:33.769>current <00:22:33.925>development

533
00:22:35.865 --> 00:22:37.025
<00:22:35.865>process <00:22:36.213>and <00:22:36.561>tech <00:22:36.909>stack.

534
00:22:38.005 --> 00:22:41.985
<00:22:38.005>So <00:22:38.180>what <00:22:38.355>would <00:22:38.530>be <00:22:38.705>a <00:22:38.880>process <00:22:39.055>that <00:22:39.230>would <00:22:39.405>actually

535
00:22:42.805 --> 00:22:43.865
<00:22:42.805>really <00:22:43.282>fit <00:22:43.759>them

536
00:22:44.085 --> 00:22:46.065
<00:22:44.085>or <00:22:44.285>a <00:22:44.485>process <00:22:44.685>where <00:22:44.885>they <00:22:45.085>would <00:22:45.285>think, <00:22:45.485>yes,

537
00:22:46.065 --> 00:22:48.625
<00:22:46.065>this <00:22:46.265>would <00:22:46.465>really <00:22:46.665>enable <00:22:46.865>us <00:22:47.065>to <00:22:47.265>ship <00:22:47.465>fast

538
00:22:48.645 --> 00:22:50.385
<00:22:48.645>and <00:22:48.925>ship <00:22:49.205>often <00:22:49.485>and <00:22:49.764>ship <00:22:50.045>safely?

539
00:22:51.885 --> 00:22:56.095
<00:22:51.885>Well, <00:22:52.118>fortunately, <00:22:52.351>fortunately, <00:22:52.585>Acme's <00:22:52.818>management, <00:22:53.051>you <00:22:53.285>know,

540
00:22:56.095 --> 00:22:58.255
<00:22:56.095>they're <00:22:56.375>really <00:22:56.655>convinced <00:22:56.935>of <00:22:57.215>the <00:22:57.495>value,

541
00:22:58.275 --> 00:22:59.855
<00:22:58.275>the <00:22:58.555>potential <00:22:58.835>value <00:22:59.115>of <00:22:59.395>machine <00:22:59.675>learning.

542
00:22:59.965 --> 00:23:02.455
<00:22:59.965>They <00:23:00.165>say, <00:23:00.365>yeah, <00:23:00.564>you <00:23:00.764>know, <00:23:00.965>we <00:23:01.165>really <00:23:01.365>recognize

543
00:23:02.455 --> 00:23:04.935
<00:23:02.455>that <00:23:02.610>there <00:23:02.766>are <00:23:02.921>some, <00:23:03.077>there <00:23:03.232>are <00:23:03.388>some <00:23:03.543>really <00:23:03.699>important <00:23:03.855>use

544
00:23:04.935 --> 00:23:07.295
<00:23:04.935>cases <00:23:05.090>for <00:23:05.246>us, <00:23:05.401>especially <00:23:05.557>when <00:23:05.712>we <00:23:05.868>look <00:23:06.023>at <00:23:06.179>cost <00:23:06.335>reduction,

545
00:23:07.295 --> 00:23:09.255
<00:23:07.295>predictive <00:23:07.495>maintenance <00:23:07.695>and <00:23:07.895>all <00:23:08.095>that, <00:23:08.295>all <00:23:08.495>that <00:23:08.695>stuff

546
00:23:09.685 --> 00:23:12.335
<00:23:09.685>that <00:23:09.840>could <00:23:09.996>really, <00:23:10.151>uh, <00:23:10.307>save <00:23:10.462>us <00:23:10.618>a <00:23:10.773>lot <00:23:10.929>of <00:23:11.085>money.

547
00:23:12.475 --> 00:23:14.495
<00:23:12.475>So <00:23:12.649>definitely <00:23:12.824>we <00:23:13.000>want <00:23:13.175>to <00:23:13.350>invest <00:23:13.524>in <00:23:13.699>machine <00:23:13.875>learning.

548
00:23:14.795 --> 00:23:19.655
<00:23:14.795>We <00:23:14.950>want <00:23:15.106>machine <00:23:15.261>learning <00:23:15.417>to <00:23:15.572>become <00:23:15.728>a <00:23:15.883>first <00:23:16.039>class <00:23:16.195>citizen

549
00:23:20.195 --> 00:23:21.815
<00:23:20.195>in <00:23:20.661>our <00:23:21.128>tech <00:23:21.595>landscape.

550
00:23:23.045 --> 00:23:25.135
<00:23:23.045>That <00:23:23.395>means <00:23:23.745>to <00:23:24.095>be <00:23:24.445>precise,

551
00:23:25.445 --> 00:23:27.295
<00:23:25.445>that <00:23:25.794>developing <00:23:26.145>machine <00:23:26.495>learning <00:23:26.845>systems,

552
00:23:28.115 --> 00:23:30.655
<00:23:28.115>at <00:23:28.348>least <00:23:28.581>from <00:23:28.815>a <00:23:29.048>tech <00:23:29.281>support <00:23:29.515>perspective,

553
00:23:31.195 --> 00:23:34.175
<00:23:31.195>should <00:23:31.428>not <00:23:31.661>be <00:23:31.895>any <00:23:32.128>harder <00:23:32.361>than <00:23:32.595>developing

554
00:23:34.895 --> 00:23:37.175
<00:23:34.895>a <00:23:35.245>classic <00:23:35.595>soft <00:23:35.945>engineering <00:23:36.295>solution.

555
00:23:37.235 --> 00:23:39.175
<00:23:37.235>You <00:23:37.468>know, <00:23:37.701>some <00:23:37.935>backend <00:23:38.168>service, <00:23:38.401>for <00:23:38.635>example.

556
00:23:40.115 --> 00:23:41.935
<00:23:40.115>And <00:23:40.348>that's <00:23:40.581>actually, <00:23:40.815>that's <00:23:41.048>a <00:23:41.281>big <00:23:41.515>thing

557
00:23:42.125 --> 00:23:46.525
<00:23:42.125>because <00:23:42.300>what <00:23:42.475>this <00:23:42.650>means <00:23:42.825>is <00:23:43.000>that <00:23:43.175>translates <00:23:43.350>concretely <00:23:43.525>that

558
00:23:46.585 --> 00:23:49.085
<00:23:46.585>to <00:23:46.725>that <00:23:46.865>the <00:23:47.005>data <00:23:47.145>science <00:23:47.285>team <00:23:47.425>in <00:23:47.565>the <00:23:47.705>next <00:23:47.845>two <00:23:47.985>years.

559
00:23:49.185 --> 00:23:51.045
<00:23:49.185>So <00:23:49.385>we <00:23:49.585>have <00:23:49.784>some <00:23:49.985>time <00:23:50.185>should <00:23:50.385>come <00:23:50.585>up

560
00:23:51.045 --> 00:23:53.965
<00:23:51.045>with <00:23:51.245>a <00:23:51.445>best <00:23:51.645>practice <00:23:51.845>machine <00:23:52.045>learning <00:23:52.245>development <00:23:52.445>process

561
00:23:54.355 --> 00:23:56.885
<00:23:54.355>with <00:23:54.705>end-to-end <00:23:55.055>ML <00:23:55.405>ops <00:23:55.755>support.

562
00:23:57.425 --> 00:24:00.005
<00:23:57.425>That's <00:23:57.705>the <00:23:57.985>objective. <00:23:58.264>Well, <00:23:58.544>of <00:23:58.825>course,

563
00:24:00.005 --> 00:24:01.165
<00:24:00.005>that's <00:24:00.266>for <00:24:00.527>data <00:24:00.788>science <00:24:01.049>team.

564
00:24:01.165 --> 00:24:02.445
<00:24:01.165>That's <00:24:01.395>great <00:24:01.625>news, <00:24:01.856>right? <00:24:02.086>They <00:24:02.317>are,

565
00:24:02.755 --> 00:24:04.325
<00:24:02.755>they <00:24:02.988>are <00:24:03.221>very, <00:24:03.455>very <00:24:03.688>happy <00:24:03.921>with <00:24:04.155>that.

566
00:24:04.435 --> 00:24:07.365
<00:24:04.435>They <00:24:04.635>see <00:24:04.835>big <00:24:05.034>investment, <00:24:05.235>actually <00:24:05.435>a <00:24:05.635>big <00:24:05.835>commitment

567
00:24:07.425 --> 00:24:08.805
<00:24:07.425>to <00:24:07.673>machine <00:24:07.921>learning <00:24:08.170>in <00:24:08.418>the <00:24:08.667>company,

568
00:24:08.945 --> 00:24:11.645
<00:24:08.945>and <00:24:09.100>they <00:24:09.256>also <00:24:09.411>get <00:24:09.567>some <00:24:09.722>budget <00:24:09.878>to <00:24:10.033>hire <00:24:10.189>some <00:24:10.345>consultants

569
00:24:11.905 --> 00:24:14.205
<00:24:11.905>and <00:24:12.138>to, <00:24:12.371>together <00:24:12.605>with <00:24:12.838>the <00:24:13.071>consultants, <00:24:13.305>they,

570
00:24:14.205 --> 00:24:16.805
<00:24:14.205>they <00:24:14.360>sketch <00:24:14.516>out <00:24:14.671>the <00:24:14.827>process <00:24:14.982>that <00:24:15.138>could <00:24:15.293>work <00:24:15.449>for <00:24:15.605>them,

571
00:24:16.965 --> 00:24:19.805
<00:24:16.965>a <00:24:17.120>best <00:24:17.276>practice <00:24:17.431>process, <00:24:17.587>which <00:24:17.742>kind <00:24:17.898>of <00:24:18.053>looks <00:24:18.209>like <00:24:18.365>this.

572
00:24:20.785 --> 00:24:24.645
<00:24:20.785>Uh, <00:24:20.901>let's <00:24:21.018>zoom <00:24:21.135>in <00:24:21.251>a <00:24:21.368>bit, <00:24:21.485>a <00:24:21.601>bit <00:24:21.718>more <00:24:21.835>on <00:24:21.951>that <00:24:22.068>process <00:24:22.185>and,

573
00:24:25.745 --> 00:24:28.445
<00:24:25.745>and <00:24:25.885>go <00:24:26.024>over <00:24:26.165>it <00:24:26.304>step <00:24:26.445>by <00:24:26.584>step <00:24:26.725>to <00:24:26.864>actually, <00:24:27.004>you <00:24:27.145>know,

574
00:24:28.445 --> 00:24:32.125
<00:24:28.445>understand <00:24:28.620>a <00:24:28.794>bit <00:24:28.970>what <00:24:29.145>characterizes <00:24:29.320>such <00:24:29.495>a, <00:24:29.669>you <00:24:29.845>know,

575
00:24:32.195 --> 00:24:34.365
<00:24:32.195>best <00:24:32.475>practice <00:24:32.754>machine <00:24:33.034>learning <00:24:33.314>development <00:24:33.595>process

576
00:24:34.835 --> 00:24:36.245
<00:24:34.835>supported <00:24:35.258>by <00:24:35.681>ML <00:24:36.103>ops.

577
00:24:37.625 --> 00:24:41.085
<00:24:37.625>So <00:24:37.752>let's <00:24:37.879>start <00:24:38.006>at <00:24:38.134>the <00:24:38.261>very <00:24:38.388>left <00:24:38.515>where <00:24:38.643>it <00:24:38.770>says <00:24:38.897>feature <00:24:39.025>store.

578
00:24:41.275 --> 00:24:44.205
<00:24:41.275>This <00:24:41.508>green, <00:24:41.741>this <00:24:41.975>green <00:24:42.208>data <00:24:42.441>store <00:24:42.675>symbol.

579
00:24:45.325 --> 00:24:46.445
<00:24:45.325>A <00:24:45.577>feature <00:24:45.829>store <00:24:46.081>is <00:24:46.332>something

580
00:24:46.875 --> 00:24:50.405
<00:24:46.875>that <00:24:47.050>is <00:24:47.225>a <00:24:47.400>quite <00:24:47.575>common <00:24:47.750>element <00:24:47.925>in <00:24:48.100>machine <00:24:48.275>learning

581
00:24:50.405 --> 00:24:53.445
<00:24:50.405>development <00:24:50.685>processes <00:24:50.965>supported <00:24:51.245>by <00:24:51.524>ML <00:24:51.805>ops.

582
00:24:53.865 --> 00:24:56.525
<00:24:53.865>And <00:24:54.005>I <00:24:54.145>think <00:24:54.285>feature <00:24:54.425>store <00:24:54.565>is <00:24:54.705>actually, <00:24:54.845>it's <00:24:54.985>a <00:24:55.125>fancy <00:24:55.265>word.

583
00:24:56.715 --> 00:25:00.125
<00:24:56.715>It's <00:24:56.889>a <00:24:57.064>fancy <00:24:57.240>word <00:24:57.415>for <00:24:57.590>something <00:24:57.764>that, <00:24:57.939>uh, <00:24:58.115>yeah,

584
00:25:00.125 --> 00:25:03.485
<00:25:00.125>for <00:25:00.280>something <00:25:00.436>relatively, <00:25:00.591>at <00:25:00.747>least <00:25:00.902>at <00:25:01.058>the <00:25:01.213>very <00:25:01.369>core, <00:25:01.525>simple,

585
00:25:03.825 --> 00:25:06.605
<00:25:03.825>uh, <00:25:03.965>it <00:25:04.105>just <00:25:04.245>means <00:25:04.385>we <00:25:04.525>want <00:25:04.665>to <00:25:04.805>have <00:25:04.945>a <00:25:05.085>central <00:25:05.225>storage

586
00:25:07.105 --> 00:25:09.645
<00:25:07.105>for <00:25:07.260>all <00:25:07.416>the <00:25:07.571>features <00:25:07.727>for <00:25:07.882>the <00:25:08.038>data <00:25:08.193>that <00:25:08.349>we <00:25:08.505>use

587
00:25:09.985 --> 00:25:11.205
<00:25:09.985>for <00:25:10.533>machine <00:25:11.082>learning.

588
00:25:11.385 --> 00:25:14.045
<00:25:11.385>So <00:25:11.540>basically <00:25:11.696>both <00:25:11.851>a <00:25:12.007>train <00:25:12.162>time <00:25:12.318>and <00:25:12.473>at <00:25:12.629>inference <00:25:12.785>time,

589
00:25:14.045 --> 00:25:15.085
<00:25:14.045>and <00:25:14.357>for <00:25:14.669>data <00:25:14.981>exploration,

590
00:25:15.595 --> 00:25:19.125
<00:25:15.595>data <00:25:15.770>scientists <00:25:15.945>can <00:25:16.120>always <00:25:16.295>tap <00:25:16.470>into <00:25:16.645>this <00:25:16.820>feature <00:25:16.995>store.

591
00:25:20.115 --> 00:25:21.485
<00:25:20.115>Typically, <00:25:20.526>these <00:25:20.937>feature <00:25:21.348>stores

592
00:25:21.715 --> 00:25:23.005
<00:25:21.715>have <00:25:22.295>certain <00:25:22.876>characteristics.

593
00:25:23.705 --> 00:25:26.005
<00:25:23.705>Uh, <00:25:23.879>for <00:25:24.054>example, <00:25:24.230>um, <00:25:24.405>usually <00:25:24.580>there <00:25:24.754>is <00:25:24.929>an <00:25:25.105>online

594
00:25:26.005 --> 00:25:28.285
<00:25:26.005>and <00:25:26.180>an <00:25:26.355>offline <00:25:26.530>element <00:25:26.705>to <00:25:26.880>it, <00:25:27.055>the <00:25:27.230>online <00:25:27.405>element,

595
00:25:28.305 --> 00:25:30.885
<00:25:28.305>making <00:25:28.505>sure <00:25:28.705>that <00:25:28.905>we <00:25:29.105>can <00:25:29.305>serve <00:25:29.505>features, <00:25:29.705>uh,

596
00:25:30.885 --> 00:25:33.285
<00:25:30.885>with <00:25:31.085>low <00:25:31.285>latency, <00:25:31.485>which <00:25:31.685>is <00:25:31.885>especially <00:25:32.085>important <00:25:32.285>inference

597
00:25:33.285 --> 00:25:34.485
<00:25:33.285>time <00:25:33.645>for <00:25:34.004>some <00:25:34.365>applications.

598
00:25:35.025 --> 00:25:38.245
<00:25:35.025>And <00:25:35.165>typically <00:25:35.305>they <00:25:35.445>also <00:25:35.585>have <00:25:35.725>an <00:25:35.865>offline <00:25:36.005>part <00:25:36.145>that <00:25:36.285>means, <00:25:36.425>uh,

599
00:25:38.245 --> 00:25:41.285
<00:25:38.245>this <00:25:38.400>is <00:25:38.556>the <00:25:38.711>part <00:25:38.867>that's <00:25:39.022>optimized <00:25:39.178>for <00:25:39.333>big <00:25:39.489>batch <00:25:39.645>loading,

600
00:25:41.455 --> 00:25:44.205
<00:25:41.455>which, <00:25:41.610>uh, <00:25:41.766>could <00:25:41.921>be <00:25:42.077>relevant <00:25:42.232>for <00:25:42.388>train <00:25:42.543>time, <00:25:42.699>for <00:25:42.855>instance.

601
00:25:44.635 --> 00:25:47.645
<00:25:44.635>Also, <00:25:44.810>some <00:25:44.985>feature <00:25:45.160>stores, <00:25:45.335>um, <00:25:45.510>have <00:25:45.685>the <00:25:45.860>nice <00:25:46.035>characteristic

602
00:25:47.645 --> 00:25:49.365
<00:25:47.645>that <00:25:47.845>they <00:25:48.045>let <00:25:48.245>you <00:25:48.445>travel <00:25:48.645>back <00:25:48.845>in <00:25:49.045>time.

603
00:25:49.875 --> 00:25:52.565
<00:25:49.875>That <00:25:50.050>means, <00:25:50.225>for <00:25:50.400>example, <00:25:50.575>if <00:25:50.750>you <00:25:50.925>had <00:25:51.100>customer <00:25:51.275>data

604
00:25:53.065 --> 00:25:56.845
<00:25:53.065>and <00:25:53.220>you <00:25:53.376>wanna <00:25:53.531>know <00:25:53.687>what <00:25:53.842>the <00:25:53.998>customer <00:25:54.153>properties <00:25:54.309>were <00:25:54.465>two

605
00:25:56.845 --> 00:26:00.285
<00:25:56.845>weeks <00:25:57.045>ago, <00:25:57.245>you <00:25:57.445>can <00:25:57.645>query <00:25:57.845>that <00:25:58.045>specifically, <00:25:58.245>right?

606
00:26:00.505 --> 00:26:02.805
<00:26:00.505>So <00:26:00.705>you <00:26:00.905>basically, <00:26:01.105>you <00:26:01.305>could <00:26:01.505>have <00:26:01.705>a <00:26:01.905>timeline

607
00:26:02.825 --> 00:26:04.765
<00:26:02.825>for <00:26:03.058>each <00:26:03.291>entity <00:26:03.525>in <00:26:03.758>your <00:26:03.991>feature <00:26:04.225>store.

608
00:26:06.675 --> 00:26:08.565
<00:26:06.675>Yeah, <00:26:06.850>this <00:26:07.024>is <00:26:07.200>essentially <00:26:07.375>what <00:26:07.550>a <00:26:07.725>feature <00:26:07.899>store <00:26:08.075>is,

609
00:26:08.565 --> 00:26:10.845
<00:26:08.565>just <00:26:08.915>a <00:26:09.265>central <00:26:09.615>data <00:26:09.965>repository

610
00:26:10.845 --> 00:26:12.125
<00:26:10.845>where <00:26:11.075>we <00:26:11.305>store <00:26:11.536>all <00:26:11.766>our <00:26:11.997>features

611
00:26:13.265 --> 00:26:14.325
<00:26:13.265>and <00:26:13.503>we <00:26:13.742>can <00:26:13.980>use <00:26:14.219>them,

612
00:26:14.355 --> 00:26:15.725
<00:26:14.355>different <00:26:14.663>teams <00:26:14.971>can <00:26:15.279>use <00:26:15.588>them

613
00:26:15.725 --> 00:26:16.925
<00:26:15.725>throughout <00:26:15.995>the <00:26:16.264>machine <00:26:16.534>learning <00:26:16.804>process.

614
00:26:17.395 --> 00:26:21.445
<00:26:17.395>Typically, <00:26:17.535>a <00:26:17.675>data <00:26:17.815>scientist <00:26:17.955>now <00:26:18.095>at <00:26:18.235>the <00:26:18.375>very <00:26:18.514>top <00:26:18.655>left <00:26:18.795>starts

615
00:26:21.945 --> 00:26:25.165
<00:26:21.945>his <00:26:22.120>or <00:26:22.294>her <00:26:22.470>data <00:26:22.645>analysis <00:26:22.820>exactly <00:26:22.995>there, <00:26:23.169>you <00:26:23.345>know,

616
00:26:25.165 --> 00:26:26.765
<00:26:25.165>they <00:26:25.445>open <00:26:25.725>a <00:26:26.004>Jupyter <00:26:26.284>Notebook, <00:26:26.565>um,

617
00:26:27.015 --> 00:26:29.885
<00:26:27.015>query <00:26:27.170>some <00:26:27.326>data <00:26:27.481>from <00:26:27.637>the <00:26:27.792>feature <00:26:27.948>store, <00:26:28.103>do <00:26:28.259>some <00:26:28.415>plots,

618
00:26:30.065 --> 00:26:32.965
<00:26:30.065>do <00:26:30.205>some <00:26:30.345>statistics, <00:26:30.485>so <00:26:30.625>they <00:26:30.765>get <00:26:30.905>some <00:26:31.045>idea <00:26:31.185>of <00:26:31.325>the <00:26:31.465>data.

619
00:26:33.305 --> 00:26:35.525
<00:26:33.305>At <00:26:33.480>some <00:26:33.655>point <00:26:33.830>they <00:26:34.005>say, <00:26:34.180>all <00:26:34.355>right, <00:26:34.530>let's <00:26:34.705>start

620
00:26:35.635 --> 00:26:37.005
<00:26:35.635>with <00:26:36.251>some <00:26:36.868>experiments.

621
00:26:37.705 --> 00:26:40.285
<00:26:37.705>And <00:26:37.845>an <00:26:37.985>experiment, <00:26:38.125>you <00:26:38.264>know, <00:26:38.405>is <00:26:38.544>a, <00:26:38.685>is <00:26:38.824>a <00:26:38.965>word <00:26:39.105>for,

622
00:26:40.395 --> 00:26:41.845
<00:26:40.395>this <00:26:40.656>is <00:26:40.917>a <00:26:41.178>word <00:26:41.439>that <00:26:41.700>describes,

623
00:26:42.125 --> 00:26:44.125
<00:26:42.125>I <00:26:42.325>wanna <00:26:42.525>do <00:26:42.725>some <00:26:42.925>pre-processing, <00:26:43.125>some <00:26:43.325>model <00:26:43.525>training,

624
00:26:44.125 --> 00:26:46.685
<00:26:44.125>some <00:26:44.325>model <00:26:44.525>evaluation <00:26:44.725>essentially <00:26:44.925>to <00:26:45.125>see, <00:26:45.325>you <00:26:45.525>know,

625
00:26:46.785 --> 00:26:48.925
<00:26:46.785>how <00:26:47.065>good <00:26:47.345>are <00:26:47.625>my <00:26:47.905>models <00:26:48.185>actually?

626
00:26:49.875 --> 00:26:52.725
<00:26:49.875>Well, <00:26:50.030>um, <00:26:50.186>in <00:26:50.341>that <00:26:50.497>process, <00:26:50.652>the, <00:26:50.808>the <00:26:50.963>small <00:26:51.119>difference <00:26:51.275>is

627
00:26:52.725 --> 00:26:55.125
<00:26:52.725>that <00:26:52.880>we <00:26:53.036>are <00:26:53.191>doing <00:26:53.347>these <00:26:53.502>experiments <00:26:53.658>in <00:26:53.813>an <00:26:53.969>orchestrated <00:26:54.125>way,

628
00:26:55.885 --> 00:26:57.665
<00:26:55.885>and <00:26:56.235>that <00:26:56.585>means <00:26:56.935>concretely, <00:26:57.285>um,

629
00:26:57.665 --> 00:27:00.785
<00:26:57.665>typically <00:26:57.840>you <00:26:58.014>would <00:26:58.190>chop <00:26:58.365>up <00:26:58.540>your <00:26:58.715>experiments <00:26:58.889>in <00:26:59.065>different

630
00:27:00.785 --> 00:27:02.265
<00:27:00.785>stages <00:27:01.229>and <00:27:01.673>different <00:27:02.117>steps,

631
00:27:02.525 --> 00:27:06.025
<00:27:02.525>and <00:27:02.665>these <00:27:02.805>steps <00:27:02.945>are <00:27:03.085>wired <00:27:03.225>together, <00:27:03.365>um, <00:27:03.505>in <00:27:03.645>a <00:27:03.785>dag <00:27:03.925>yeah.

632
00:27:06.165 --> 00:27:08.905
<00:27:06.165>So <00:27:06.365>you <00:27:06.565>have <00:27:06.764>a <00:27:06.965>sequence <00:27:07.165>of <00:27:07.365>steps, <00:27:07.565>um,

633
00:27:08.905 --> 00:27:11.385
<00:27:08.905>that <00:27:09.185>characterize <00:27:09.465>your <00:27:09.745>experiment, <00:27:10.024>typically <00:27:10.305>starting

634
00:27:11.385 --> 00:27:14.465
<00:27:11.385>with <00:27:11.560>a <00:27:11.735>data <00:27:11.910>extraction <00:27:12.085>step, <00:27:12.260>then <00:27:12.435>some <00:27:12.610>data <00:27:12.785>validation,

635
00:27:14.465 --> 00:27:17.825
<00:27:14.465>perhaps <00:27:14.665>model <00:27:14.865>training, <00:27:15.064>model <00:27:15.264>evaluation, <00:27:15.465>and <00:27:15.665>so <00:27:15.865>on.

636
00:27:19.345 --> 00:27:21.155
<00:27:19.345>Also, <00:27:19.811>uh, <00:27:20.278>usually <00:27:20.745>part

637
00:27:21.155 --> 00:27:23.355
<00:27:21.155>of <00:27:21.388>such <00:27:21.621>an <00:27:21.855>orchestrated <00:27:22.088>experiment <00:27:22.321>is <00:27:22.555>quite

638
00:27:23.365 --> 00:27:24.435
<00:27:23.365>meticulous <00:27:24.328>tracking.

639
00:27:24.865 --> 00:27:28.515
<00:27:24.865>This <00:27:25.020>means <00:27:25.176>we <00:27:25.331>track <00:27:25.487>exactly <00:27:25.642>what <00:27:25.798>goes <00:27:25.953>into <00:27:26.109>each <00:27:26.265>step

640
00:27:28.615 --> 00:27:30.035
<00:27:28.615>and <00:27:28.828>what <00:27:29.041>goes <00:27:29.254>out <00:27:29.467>of <00:27:29.680>each <00:27:29.893>step,

641
00:27:30.055 --> 00:27:33.235
<00:27:30.055>and <00:27:30.255>which <00:27:30.455>parameters <00:27:30.655>are <00:27:30.855>used <00:27:31.055>within <00:27:31.255>each <00:27:31.455>step

642
00:27:33.775 --> 00:27:35.915
<00:27:33.775>to <00:27:33.975>really <00:27:34.175>enable <00:27:34.375>us <00:27:34.575>to <00:27:34.775>get <00:27:34.975>a <00:27:35.175>very,

643
00:27:35.915 --> 00:27:37.515
<00:27:35.915>very <00:27:36.381>comprehensive <00:27:36.848>picture <00:27:37.315>of

644
00:27:37.515 --> 00:27:39.835
<00:27:37.515>what <00:27:37.795>this <00:27:38.075>experiment <00:27:38.355>exactly <00:27:38.635>looked <00:27:38.915>like.

645
00:27:40.615 --> 00:27:43.035
<00:27:40.615>Uh, <00:27:40.848>this <00:27:41.081>is <00:27:41.315>important <00:27:41.548>for <00:27:41.781>ensuring <00:27:42.015>reproducibility.

646
00:27:43.465 --> 00:27:45.635
<00:27:43.465>Even <00:27:43.620>if <00:27:43.776>you <00:27:43.931>did <00:27:44.087>an <00:27:44.242>experiment, <00:27:44.398>uh, <00:27:44.553>two <00:27:44.709>weeks <00:27:44.865>ago,

647
00:27:45.815 --> 00:27:48.475
<00:27:45.815>you <00:27:45.970>can <00:27:46.126>always <00:27:46.281>hop <00:27:46.437>back <00:27:46.592>and <00:27:46.748>see <00:27:46.903>what <00:27:47.059>exactly <00:27:47.215>happened

648
00:27:48.475 --> 00:27:49.955
<00:27:48.475>and <00:27:48.665>what <00:27:48.855>results <00:27:49.045>you <00:27:49.236>got <00:27:49.426>out <00:27:49.616>of <00:27:49.807>it.

649
00:27:50.335 --> 00:27:52.635
<00:27:50.335>And <00:27:50.568>that's <00:27:50.801>super <00:27:51.035>important <00:27:51.268>in <00:27:51.501>experimentation <00:27:51.735>phases.

650
00:27:54.065 --> 00:27:56.355
<00:27:54.065>Well, <00:27:54.298>so <00:27:54.531>our <00:27:54.765>experiment, <00:27:54.998>you <00:27:55.231>know, <00:27:55.465>uh,

651
00:27:56.375 --> 00:27:57.995
<00:27:56.375>our <00:27:56.725>data <00:27:57.075>scientist <00:27:57.425>does <00:27:57.775>experiments.

652
00:27:57.995 --> 00:27:59.875
<00:27:57.995>He <00:27:58.228>does, <00:27:58.461>let's <00:27:58.695>see, <00:27:58.928>10, <00:27:59.161>20 <00:27:59.395>experiments,

653
00:27:59.875 --> 00:28:02.435
<00:27:59.875>and <00:28:00.002>at <00:28:00.129>some <00:28:00.256>point <00:28:00.384>he <00:28:00.511>thinks, <00:28:00.638>yes, <00:28:00.765>now <00:28:00.893>I'm <00:28:01.020>happy <00:28:01.147>with <00:28:01.275>that.

654
00:28:02.495 --> 00:28:05.755
<00:28:02.495>You <00:28:02.611>know, <00:28:02.728>my, <00:28:02.844>my <00:28:02.961>model <00:28:03.078>seems <00:28:03.195>to <00:28:03.311>be <00:28:03.428>good, <00:28:03.544>seems <00:28:03.661>to <00:28:03.778>be <00:28:03.895>robust.

655
00:28:05.995 --> 00:28:07.715
<00:28:05.995>I <00:28:06.228>wanna <00:28:06.461>go <00:28:06.695>to <00:28:06.928>production <00:28:07.161>with <00:28:07.395>that.

656
00:28:08.295 --> 00:28:12.235
<00:28:08.295>So <00:28:08.411>he <00:28:08.528>let <00:28:08.645>for <00:28:08.761>example, <00:28:08.878>checks <00:28:08.995>in, <00:28:09.111>uh, <00:28:09.228>the, <00:28:09.345>the <00:28:09.461>code <00:28:09.578>for <00:28:09.695>the,

657
00:28:12.775 --> 00:28:14.995
<00:28:12.775>for <00:28:12.930>his <00:28:13.086>best <00:28:13.241>experiments <00:28:13.397>he <00:28:13.552>wants <00:28:13.708>to <00:28:13.863>take <00:28:14.019>to <00:28:14.175>production,

658
00:28:14.995 --> 00:28:16.035
<00:28:14.995>to <00:28:15.307>the <00:28:15.619>master <00:28:15.931>branch,

659
00:28:16.695 --> 00:28:19.075
<00:28:16.695>and <00:28:16.870>then <00:28:17.044>an <00:28:17.220>automatic <00:28:17.395>build <00:28:17.570>process <00:28:17.745>is <00:28:17.919>kicked <00:28:18.095>off.

660
00:28:19.145 --> 00:28:20.675
<00:28:19.145>Then <00:28:19.374>we, <00:28:19.604>we <00:28:19.833>are <00:28:20.062>building <00:28:20.292>that <00:28:20.522>pipeline.

661
00:28:20.675 --> 00:28:22.155
<00:28:20.675>We <00:28:20.897>are <00:28:21.119>testing <00:28:21.341>it, <00:28:21.562>we're <00:28:21.784>packaging <00:28:22.007>it,

662
00:28:22.455 --> 00:28:23.755
<00:28:22.455>and <00:28:22.747>we <00:28:23.040>are <00:28:23.332>deploying <00:28:23.625>it.

663
00:28:24.685 --> 00:28:27.115
<00:28:24.685>We're <00:28:24.918>deploying <00:28:25.151>it <00:28:25.385>to <00:28:25.618>some <00:28:25.851>computational <00:28:26.085>infrastructure.

664
00:28:27.265 --> 00:28:30.315
<00:28:27.265>That <00:28:27.420>could <00:28:27.576>be <00:28:27.731>a <00:28:27.887>Kubernetes <00:28:28.042>cluster, <00:28:28.198>could <00:28:28.353>be, <00:28:28.509>you <00:28:28.665>know,

665
00:28:30.385 --> 00:28:33.595
<00:28:30.385>some <00:28:30.585>computation <00:28:30.785>server <00:28:30.985>with <00:28:31.185>a <00:28:31.385>few <00:28:31.585>GPUs <00:28:31.785>inside,

666
00:28:34.155 --> 00:28:35.235
<00:28:34.155>whatever, <00:28:35.127>right?

667
00:28:35.535 --> 00:28:38.155
<00:28:35.535>And <00:28:35.885>there, <00:28:36.235>this <00:28:36.585>automated <00:28:36.935>pipeline,

668
00:28:38.225 --> 00:28:41.435
<00:28:38.225>this <00:28:38.399>big <00:28:38.574>blue <00:28:38.750>block <00:28:38.925>in <00:28:39.100>the <00:28:39.274>middle <00:28:39.449>then <00:28:39.625>lifts.

669
00:28:41.825 --> 00:28:44.515
<00:28:41.825>Yeah, <00:28:42.000>it's <00:28:42.175>essentially <00:28:42.350>a <00:28:42.525>job <00:28:42.700>that <00:28:42.875>can <00:28:43.050>be <00:28:43.225>triggered.

670
00:28:45.295 --> 00:28:46.915
<00:28:45.295>Uh, <00:28:45.575>inside <00:28:45.855>the <00:28:46.135>job <00:28:46.415>is <00:28:46.695>exactly

671
00:28:46.945 --> 00:28:48.395
<00:28:46.945>what <00:28:47.271>the <00:28:47.597>data <00:28:47.923>scientists <00:28:48.250>define.

672
00:28:48.395 --> 00:28:52.735
<00:28:48.395>It's <00:28:48.595>exactly <00:28:48.795>these <00:28:48.995>steps <00:28:49.195>he <00:28:49.395>or <00:28:49.595>she <00:28:49.795>defined.

673
00:28:53.805 --> 00:28:55.215
<00:28:53.805>Such <00:28:54.058>a <00:28:54.312>pipeline <00:28:54.566>can <00:28:54.820>be <00:28:55.073>triggered.

674
00:28:55.215 --> 00:28:57.135
<00:28:55.215>Then <00:28:55.415>it <00:28:55.615>can <00:28:55.814>be <00:28:56.014>triggered <00:28:56.215>by <00:28:56.415>multiple <00:28:56.615>things.

675
00:28:57.235 --> 00:28:58.455
<00:28:57.235>It <00:28:57.454>could <00:28:57.674>be <00:28:57.893>triggered <00:28:58.113>by <00:28:58.332>time.

676
00:28:58.555 --> 00:28:59.815
<00:28:58.555>So <00:28:58.838>let's <00:28:59.122>say <00:28:59.405>every <00:28:59.689>Monday,

677
00:28:59.995 --> 00:29:01.655
<00:28:59.995>6:00 <00:29:00.195>AM <00:29:00.395>we <00:29:00.594>are <00:29:00.794>gonna <00:29:00.995>run <00:29:01.195>a <00:29:01.395>training,

678
00:29:02.235 --> 00:29:04.695
<00:29:02.235>or <00:29:02.409>it <00:29:02.584>could <00:29:02.760>also <00:29:02.935>be <00:29:03.110>triggered <00:29:03.284>by <00:29:03.459>performance <00:29:03.635>metrics,

679
00:29:04.795 --> 00:29:07.575
<00:29:04.795>you <00:29:05.075>know, <00:29:05.355>some <00:29:05.635>accuracy <00:29:05.915>metric, <00:29:06.195>dropping

680
00:29:07.625 --> 00:29:08.775
<00:29:07.625>below <00:29:07.970>a <00:29:08.315>certain <00:29:08.660>threshold.

681
00:29:08.775 --> 00:29:10.735
<00:29:08.775>Then <00:29:08.975>we're <00:29:09.175>gonna <00:29:09.375>retrain <00:29:09.575>on <00:29:09.775>the <00:29:09.975>newest <00:29:10.175>data,

682
00:29:11.155 --> 00:29:13.815
<00:29:11.155>or <00:29:11.282>it <00:29:11.409>could <00:29:11.536>also <00:29:11.664>be <00:29:11.791>triggered <00:29:11.918>by <00:29:12.045>the <00:29:12.173>push <00:29:12.300>of <00:29:12.427>a <00:29:12.555>button.

683
00:29:13.965 --> 00:29:15.095
<00:29:13.965>That <00:29:14.219>could <00:29:14.473>also <00:29:14.727>be <00:29:14.982>possible.

684
00:29:16.125 --> 00:29:17.375
<00:29:16.125>When <00:29:16.406>the <00:29:16.687>pipeline <00:29:16.968>is <00:29:17.250>triggered

685
00:29:17.375 --> 00:29:19.295
<00:29:17.375>and <00:29:17.575>it <00:29:17.775>runs, <00:29:17.975>it's <00:29:18.175>gonna <00:29:18.375>output <00:29:18.575>two <00:29:18.775>things.

686
00:29:19.625 --> 00:29:23.615
<00:29:19.625>First <00:29:19.780>of <00:29:19.936>all, <00:29:20.091>it's <00:29:20.247>gonna <00:29:20.402>store <00:29:20.558>metadata <00:29:20.713>into <00:29:20.869>this <00:29:21.025>metadata

687
00:29:23.615 --> 00:29:27.575
<00:29:23.615>store, <00:29:23.815>the <00:29:24.015>big <00:29:24.215>gray <00:29:24.415>storage <00:29:24.615>symbol, <00:29:24.815>uh, <00:29:25.015>bit

688
00:29:27.585 --> 00:29:29.295
<00:29:27.585>below <00:29:28.051>this <00:29:28.518>automated <00:29:28.985>pipeline.

689
00:29:30.395 --> 00:29:32.175
<00:29:30.395>So <00:29:30.745>such <00:29:31.095>a <00:29:31.445>metadata <00:29:31.795>store.

690
00:29:33.015 --> 00:29:35.855
<00:29:33.015>I <00:29:33.155>think <00:29:33.295>it's <00:29:33.435>also <00:29:33.575>a <00:29:33.715>fancy <00:29:33.855>word <00:29:33.995>for, <00:29:34.135>um, <00:29:34.275>some <00:29:34.415>kind

691
00:29:35.855 --> 00:29:36.895
<00:29:35.855>of <00:29:36.323>database, <00:29:36.791>essentially.

692
00:29:36.915 --> 00:29:39.295
<00:29:36.915>You <00:29:37.070>know, <00:29:37.226>you <00:29:37.381>could <00:29:37.537>use <00:29:37.692>a <00:29:37.848>Postgres <00:29:38.003>database <00:29:38.159>here, <00:29:38.315>whatever.

693
00:29:39.295 --> 00:29:41.135
<00:29:39.295>There <00:29:39.495>are <00:29:39.695>also <00:29:39.895>a <00:29:40.095>few <00:29:40.295>specialized <00:29:40.495>feature <00:29:40.695>stores,

694
00:29:41.180 --> 00:29:42.405
<00:29:41.180>though <00:29:42.282>typically

695
00:29:42.405 --> 00:29:44.925
<00:29:42.405>what <00:29:42.545>we <00:29:42.685>store <00:29:42.825>in <00:29:42.965>there <00:29:43.105>is <00:29:43.245>any <00:29:43.385>inf <00:29:43.524>any <00:29:43.665>media <00:29:43.805>data,

696
00:29:45.025 --> 00:29:47.165
<00:29:45.025>any <00:29:45.225>information <00:29:45.425>we <00:29:45.625>get <00:29:45.825>from <00:29:46.025>that <00:29:46.225>pipeline <00:29:46.425>run.

697
00:29:47.345 --> 00:29:50.685
<00:29:47.345>So <00:29:47.500>when <00:29:47.656>did <00:29:47.811>that <00:29:47.967>pipeline <00:29:48.122>run? <00:29:48.278>Why <00:29:48.433>was <00:29:48.589>it <00:29:48.745>triggered?

698
00:29:50.685 --> 00:29:53.125
<00:29:50.685>Which <00:29:50.885>steps <00:29:51.085>include, <00:29:51.284>were <00:29:51.485>included <00:29:51.685>in <00:29:51.885>that <00:29:52.085>pipeline?

699
00:29:53.125 --> 00:29:55.405
<00:29:53.125>Then <00:29:53.405>perhaps <00:29:53.685>also <00:29:53.965>evaluation <00:29:54.245>metrics, <00:29:54.525>uh,

700
00:29:55.405 --> 00:29:57.085
<00:29:55.405>what <00:29:55.580>the <00:29:55.754>model <00:29:55.930>was <00:29:56.105>called <00:29:56.280>that <00:29:56.455>came <00:29:56.629>out <00:29:56.805>of

701
00:29:57.085 --> 00:29:58.285
<00:29:57.085>that <00:29:57.445>end, <00:29:57.805>end, <00:29:58.165>end.

702
00:29:58.465 --> 00:30:01.005
<00:29:58.465>So <00:29:58.639>eventually <00:29:58.814>everything <00:29:58.990>that <00:29:59.165>could, <00:29:59.340>could <00:29:59.514>be <00:29:59.689>of <00:29:59.865>interest

703
00:30:01.025 --> 00:30:05.485
<00:30:01.025>to <00:30:01.200>us <00:30:01.375>about <00:30:01.550>this <00:30:01.725>pipeline, <00:30:01.900>again, <00:30:02.075>to, <00:30:02.250>to <00:30:02.425>ensure

704
00:30:05.485 --> 00:30:07.085
<00:30:05.485>that <00:30:05.834>from <00:30:06.185>an <00:30:06.534>outside <00:30:06.885>perspective,

705
00:30:07.185 --> 00:30:09.125
<00:30:07.185>we <00:30:07.465>knew <00:30:07.745>exactly <00:30:08.024>what <00:30:08.304>is <00:30:08.585>happening.

706
00:30:09.825 --> 00:30:12.605
<00:30:09.825>The <00:30:09.980>second <00:30:10.136>big <00:30:10.291>output <00:30:10.447>that <00:30:10.602>this <00:30:10.758>pipeline <00:30:10.913>produces <00:30:11.069>is, <00:30:11.225>well,

707
00:30:12.605 --> 00:30:14.125
<00:30:12.605>of <00:30:12.833>course, <00:30:13.061>a <00:30:13.289>model, <00:30:13.517>a <00:30:13.745>trained <00:30:13.973>model,

708
00:30:14.665 --> 00:30:18.045
<00:30:14.665>and <00:30:14.820>that <00:30:14.976>model <00:30:15.131>is <00:30:15.287>being <00:30:15.442>checked <00:30:15.598>into <00:30:15.753>a <00:30:15.909>model <00:30:16.065>registry.

709
00:30:18.525 --> 00:30:21.805
<00:30:18.525>A <00:30:18.680>model <00:30:18.836>registry, <00:30:18.991>again, <00:30:19.147>is <00:30:19.302>some <00:30:19.458>kind <00:30:19.613>of <00:30:19.769>data <00:30:19.925>storage

710
00:30:22.195 --> 00:30:23.765
<00:30:22.195>that <00:30:22.661>takes <00:30:23.128>models, <00:30:23.595>right?

711
00:30:23.765 --> 00:30:26.245
<00:30:23.765>That <00:30:23.940>stores <00:30:24.115>models <00:30:24.290>at <00:30:24.465>the, <00:30:24.640>you <00:30:24.815>know, <00:30:24.990>at <00:30:25.165>the

712
00:30:26.765 --> 00:30:30.005
<00:30:26.765>simplest <00:30:26.920>form <00:30:27.076>this <00:30:27.231>is, <00:30:27.387>uh, <00:30:27.542>some, <00:30:27.698>some <00:30:27.853>blob <00:30:28.009>storage, <00:30:28.165>some,

713
00:30:30.305 --> 00:30:32.525
<00:30:30.305>um, <00:30:30.771>Amazon <00:30:31.238>S3 <00:30:31.705>bucket.

714
00:30:33.185 --> 00:30:35.805
<00:30:33.185>You <00:30:33.340>could <00:30:33.496>call <00:30:33.651>that <00:30:33.807>a <00:30:33.962>model <00:30:34.118>registry, <00:30:34.273>even <00:30:34.429>if <00:30:34.585>it,

715
00:30:35.915 --> 00:30:38.085
<00:30:35.915>even <00:30:36.090>if <00:30:36.264>it's, <00:30:36.440>you <00:30:36.615>know, <00:30:36.790>doesn't <00:30:36.965>really <00:30:37.139>have <00:30:37.315>much,

716
00:30:38.115 --> 00:30:40.245
<00:30:38.115>much <00:30:38.348>support <00:30:38.581>for <00:30:38.815>storing <00:30:39.048>anything <00:30:39.281>but <00:30:39.515>your,

717
00:30:40.345 --> 00:30:41.685
<00:30:40.345>but <00:30:40.586>your <00:30:40.827>models <00:30:41.068>and <00:30:41.309>other <00:30:41.551>files.

718
00:30:42.185 --> 00:30:43.805
<00:30:42.185>Uh, <00:30:42.360>but <00:30:42.534>there <00:30:42.710>are <00:30:42.885>also <00:30:43.060>a <00:30:43.235>bit <00:30:43.409>more, <00:30:43.585>um,

719
00:30:43.805 --> 00:30:48.045
<00:30:43.805>elaborate <00:30:44.038>modern <00:30:44.271>registries <00:30:44.505>that <00:30:44.738>are <00:30:44.971>specifically <00:30:45.205>optimized

720
00:30:48.345 --> 00:30:51.005
<00:30:48.345>to, <00:30:48.500>to <00:30:48.656>store <00:30:48.811>your <00:30:48.967>models, <00:30:49.122>to <00:30:49.278>help <00:30:49.433>you <00:30:49.589>version <00:30:49.745>them,

721
00:30:51.005 --> 00:30:54.005
<00:30:51.005>help <00:30:51.160>you <00:30:51.316>manage <00:30:51.471>them, <00:30:51.627>and <00:30:51.782>also <00:30:51.938>perhaps <00:30:52.093>store <00:30:52.249>certain <00:30:52.405>types

722
00:30:54.005 --> 00:30:57.005
<00:30:54.005>of <00:30:54.238>evaluation <00:30:54.471>metrics <00:30:54.705>together <00:30:54.938>with <00:30:55.171>these <00:30:55.405>models.

723
00:30:57.725 --> 00:30:59.495
<00:30:57.725>Yeah, <00:30:59.125>well,

724
00:30:59.495 --> 00:31:02.295
<00:30:59.495>once <00:30:59.695>your <00:30:59.895>automated <00:31:00.094>pipeline <00:31:00.294>training <00:31:00.495>run <00:31:00.695>is <00:31:00.895>finished,

725
00:31:02.395 --> 00:31:04.695
<00:31:02.395>it <00:31:02.628>also <00:31:02.861>kicks <00:31:03.095>off <00:31:03.328>model <00:31:03.561>serving <00:31:03.795>the

726
00:31:04.695 --> 00:31:05.975
<00:31:04.695>deployment <00:31:05.079>of <00:31:05.463>model <00:31:05.847>serving.

727
00:31:06.075 --> 00:31:08.935
<00:31:06.075>It <00:31:06.275>basically <00:31:06.475>tells <00:31:06.675>the, <00:31:06.875>the, <00:31:07.075>the <00:31:07.275>deployment <00:31:07.475>part

728
00:31:08.935 --> 00:31:10.895
<00:31:08.935>of <00:31:09.090>the <00:31:09.246>model <00:31:09.401>serving, <00:31:09.557>Hey, <00:31:09.712>there <00:31:09.868>is <00:31:10.023>a <00:31:10.179>new <00:31:10.335>model,

729
00:31:10.895 --> 00:31:12.735
<00:31:10.895>please <00:31:11.245>redeploy <00:31:11.595>my <00:31:11.945>prediction <00:31:12.295>service

730
00:31:13.235 --> 00:31:16.255
<00:31:13.235>and <00:31:13.362>get <00:31:13.489>that <00:31:13.616>new <00:31:13.744>model, <00:31:13.871>by <00:31:13.998>the <00:31:14.125>way, <00:31:14.253>from <00:31:14.380>our <00:31:14.507>model <00:31:14.635>registry.

731
00:31:17.315 --> 00:31:19.775
<00:31:17.315>So <00:31:17.490>that's <00:31:17.665>what <00:31:17.840>happens. <00:31:18.015>Our <00:31:18.190>prediction <00:31:18.365>service <00:31:18.540>is <00:31:18.715>redeployed

732
00:31:20.065 --> 00:31:21.895
<00:31:20.065>loads, <00:31:20.265>the <00:31:20.465>newest <00:31:20.665>model, <00:31:20.865>and <00:31:21.065>then <00:31:21.265>it's <00:31:21.465>working,

733
00:31:22.075 --> 00:31:23.855
<00:31:22.075>and <00:31:22.355>it's <00:31:22.635>of <00:31:22.915>course <00:31:23.195>outputting <00:31:23.475>predictions.

734
00:31:24.675 --> 00:31:27.175
<00:31:24.675>But <00:31:24.875>next <00:31:25.075>to <00:31:25.274>that <00:31:25.475>it's <00:31:25.675>also <00:31:25.875>providing <00:31:26.075>metrics.

735
00:31:27.475 --> 00:31:29.575
<00:31:27.475>So <00:31:27.630>what's <00:31:27.786>going <00:31:27.941>on? <00:31:28.097>What <00:31:28.252>does <00:31:28.408>the <00:31:28.563>data <00:31:28.719>look <00:31:28.875>like

736
00:31:29.575 --> 00:31:30.935
<00:31:29.575>that's <00:31:29.881>going <00:31:30.187>into <00:31:30.493>that <00:31:30.799>service?

737
00:31:31.325 --> 00:31:33.295
<00:31:31.325>What <00:31:31.525>does, <00:31:31.725>what <00:31:31.925>do <00:31:32.125>my <00:31:32.325>predictions <00:31:32.525>look <00:31:32.725>like?

738
00:31:33.525 --> 00:31:34.615
<00:31:33.525>Perhaps, <00:31:33.852>if <00:31:34.179>I <00:31:34.506>can,

739
00:31:34.955 --> 00:31:37.455
<00:31:34.955>how <00:31:35.155>does <00:31:35.355>my <00:31:35.554>prediction <00:31:35.754>accuracy <00:31:35.955>develop <00:31:36.155>over <00:31:36.355>time?

740
00:31:38.165 --> 00:31:42.135
<00:31:38.165>Yeah, <00:31:38.340>so <00:31:38.514>this <00:31:38.690>was <00:31:38.865>a <00:31:39.040>lot <00:31:39.215>of <00:31:39.389>information <00:31:39.565>now,

741
00:31:42.475 --> 00:31:45.815
<00:31:42.475>and <00:31:42.602>it's <00:31:42.729>a <00:31:42.856>quite <00:31:42.984>a, <00:31:43.111>yeah, <00:31:43.238>it <00:31:43.365>is <00:31:43.493>a, <00:31:43.620>it <00:31:43.747>is <00:31:43.875>a,

742
00:31:45.815 --> 00:31:46.975
<00:31:45.815>it <00:31:46.023>is <00:31:46.232>quite <00:31:46.441>a <00:31:46.650>process, <00:31:46.859>right?

743
00:31:47.045 --> 00:31:50.335
<00:31:47.045>Such <00:31:47.220>a, <00:31:47.395>such <00:31:47.570>a <00:31:47.745>best <00:31:47.920>practice <00:31:48.095>machine <00:31:48.270>learning <00:31:48.445>development

744
00:31:50.335 --> 00:31:51.895
<00:31:50.335>process <00:31:50.685>powered <00:31:51.035>by <00:31:51.385>ML <00:31:51.735>ops.

745
00:31:52.635 --> 00:31:54.455
<00:31:52.635>So <00:31:52.835>what, <00:31:53.035>what <00:31:53.235>are <00:31:53.435>the <00:31:53.635>key <00:31:53.835>takeaways <00:31:54.035>here?

746
00:31:54.975 --> 00:31:57.855
<00:31:54.975>I <00:31:55.149>think <00:31:55.324>it's <00:31:55.500>first <00:31:55.675>super <00:31:55.850>important <00:31:56.024>to <00:31:56.199>highlight <00:31:56.375>that

747
00:31:57.965 --> 00:31:59.935
<00:31:57.965>what <00:31:58.165>we've <00:31:58.365>seen <00:31:58.564>now, <00:31:58.764>this <00:31:58.965>process <00:31:59.165>is <00:31:59.365>complex,

748
00:32:00.555 --> 00:32:04.535
<00:32:00.555>but <00:32:00.730>the <00:32:00.905>big <00:32:01.080>difference <00:32:01.255>to <00:32:01.430>the <00:32:01.605>complexity, <00:32:01.780>um, <00:32:01.955>to

749
00:32:04.535 --> 00:32:08.095
<00:32:04.535>what <00:32:04.690>we've <00:32:04.846>seen <00:32:05.001>before <00:32:05.157>is <00:32:05.312>that <00:32:05.468>from <00:32:05.623>a <00:32:05.779>user <00:32:05.935>perspective,

750
00:32:08.245 --> 00:32:10.295
<00:32:08.245>from <00:32:08.594>a <00:32:08.945>data <00:32:09.294>scientist <00:32:09.645>perspective,

751
00:32:10.925 --> 00:32:13.135
<00:32:10.925>this <00:32:11.125>process <00:32:11.325>is <00:32:11.524>actually <00:32:11.725>not <00:32:11.925>complex <00:32:12.125>at <00:32:12.325>all.

752
00:32:13.645 --> 00:32:15.615
<00:32:13.645>Because <00:32:13.925>what <00:32:14.205>the <00:32:14.485>data <00:32:14.764>scientist <00:32:15.045>touches

753
00:32:15.725 --> 00:32:16.735
<00:32:15.725>here <00:32:16.027>is <00:32:16.331>very <00:32:16.634>limited.

754
00:32:17.035 --> 00:32:19.295
<00:32:17.035>The <00:32:17.190>only <00:32:17.346>thing <00:32:17.501>that <00:32:17.657>the <00:32:17.812>data <00:32:17.968>scientist <00:32:18.123>really <00:32:18.279>cares <00:32:18.435>about,

755
00:32:19.435 --> 00:32:21.815
<00:32:19.435>or <00:32:19.575>that <00:32:19.715>he <00:32:19.855>needs <00:32:19.995>to <00:32:20.135>take <00:32:20.274>care <00:32:20.415>about <00:32:20.554>in <00:32:20.695>this <00:32:20.835>process

756
00:32:22.475 --> 00:32:24.375
<00:32:22.475>is <00:32:22.941>designing <00:32:23.408>the <00:32:23.875>experiment.

757
00:32:25.325 --> 00:32:26.785
<00:32:25.325>That's <00:32:25.587>the <00:32:25.850>only <00:32:26.113>thing <00:32:26.376>he <00:32:26.639>touches.

758
00:32:27.405 --> 00:32:30.385
<00:32:27.405>And <00:32:27.560>typically <00:32:27.716>that's <00:32:27.871>also <00:32:28.027>what <00:32:28.182>data <00:32:28.338>scientists <00:32:28.493>like <00:32:28.649>to <00:32:28.805>do,

759
00:32:31.005 --> 00:32:33.025
<00:32:31.005>and <00:32:31.205>usually <00:32:31.405>that's <00:32:31.605>also <00:32:31.805>what <00:32:32.005>they're <00:32:32.205>best <00:32:32.405>at,

760
00:32:33.025 --> 00:32:35.865
<00:32:33.025>what <00:32:33.225>they're <00:32:33.425>trained <00:32:33.625>as, <00:32:33.825>you <00:32:34.025>know, <00:32:34.225>designing <00:32:34.425>experiments

761
00:32:36.135 --> 00:32:39.885
<00:32:36.135>that, <00:32:36.335>that <00:32:36.535>deliver <00:32:36.735>models <00:32:36.935>that <00:32:37.135>work, <00:32:37.335>you <00:32:37.535>know,

762
00:32:39.885 --> 00:32:43.205
<00:32:39.885>so <00:32:40.040>basically <00:32:40.196>a <00:32:40.351>data <00:32:40.507>scientist <00:32:40.662>only <00:32:40.818>touches <00:32:40.973>this <00:32:41.129>left <00:32:41.285>top

763
00:32:43.675 --> 00:32:44.805
<00:32:43.675>part <00:32:44.014>of <00:32:44.353>this <00:32:44.692>process.

764
00:32:45.385 --> 00:32:48.325
<00:32:45.385>All <00:32:45.618>the <00:32:45.851>rest <00:32:46.085>is <00:32:46.318>actually <00:32:46.551>fully <00:32:46.785>automated

765
00:32:48.625 --> 00:32:50.485
<00:32:48.625>by <00:32:48.975>your <00:32:49.325>ML <00:32:49.675>ops <00:32:50.025>platform.

766
00:32:51.095 --> 00:32:52.905
<00:32:51.095>Yeah, <00:32:51.445>and <00:32:51.795>this <00:32:52.145>is <00:32:52.495>really

767
00:32:52.905 --> 00:32:55.265
<00:32:52.905>where <00:32:53.080>we <00:32:53.254>can <00:32:53.430>see <00:32:53.605>the <00:32:53.780>strength <00:32:53.955>of <00:32:54.129>ML <00:32:54.305>ops.

768
00:32:55.265 --> 00:32:56.905
<00:32:55.265>We <00:32:55.545>have <00:32:55.825>this <00:32:56.105>complex <00:32:56.385>process, <00:32:56.665>right?

769
00:32:56.925 --> 00:32:58.585
<00:32:56.925>We <00:32:57.391>recognize <00:32:57.858>the <00:32:58.325>complexity

770
00:32:59.125 --> 00:33:02.145
<00:32:59.125>and <00:32:59.280>we, <00:32:59.436>we <00:32:59.591>try <00:32:59.747>to <00:32:59.902>abstract <00:33:00.058>away <00:33:00.213>from <00:33:00.369>this <00:33:00.525>complexity

771
00:33:02.285 --> 00:33:04.905
<00:33:02.285>for <00:33:02.565>the <00:33:02.845>user <00:33:03.125>by <00:33:03.405>really <00:33:03.685>rigorous

772
00:33:05.195 --> 00:33:06.825
<00:33:05.195>automation <00:33:05.661>throughout <00:33:06.128>this <00:33:06.595>process.

773
00:33:08.175 --> 00:33:11.905
<00:33:08.175>This <00:33:08.375>is <00:33:08.575>really <00:33:08.774>the <00:33:08.975>key <00:33:09.175>behind <00:33:09.375>such <00:33:09.575>a,

774
00:33:12.505 --> 00:33:14.195
<00:33:12.505>such <00:33:12.785>a <00:33:13.065>best <00:33:13.345>practice <00:33:13.625>ML <00:33:13.905>ops

775
00:33:14.405 --> 00:33:17.715
<00:33:14.405>setup, <00:33:14.685>you <00:33:14.965>know, <00:33:15.245>that <00:33:15.524>sounds <00:33:15.805>great.

776
00:33:17.715 --> 00:33:19.995
<00:33:17.715>So <00:33:17.915>we <00:33:18.115>have <00:33:18.314>our <00:33:18.514>target <00:33:18.715>process <00:33:18.915>now, <00:33:19.115>uh,

777
00:33:19.995 --> 00:33:21.515
<00:33:19.995>that's <00:33:20.268>a <00:33:20.542>big, <00:33:20.815>big <00:33:21.089>step <00:33:21.363>done.

778
00:33:21.655 --> 00:33:24.155
<00:33:21.655>And <00:33:21.810>also <00:33:21.966>I <00:33:22.121>think <00:33:22.277>here <00:33:22.432>it's <00:33:22.588>important <00:33:22.743>to <00:33:22.899>note, <00:33:23.055>right?

779
00:33:24.155 --> 00:33:26.275
<00:33:24.155>Such <00:33:24.355>a <00:33:24.555>target <00:33:24.754>process, <00:33:24.955>such <00:33:25.155>an <00:33:25.355>objective <00:33:25.555>process

780
00:33:26.345 --> 00:33:29.035
<00:33:26.345>that <00:33:26.500>could <00:33:26.656>look <00:33:26.811>a <00:33:26.967>bit <00:33:27.122>different <00:33:27.278>for, <00:33:27.433>for <00:33:27.589>any <00:33:27.745>organization.

781
00:33:29.035 --> 00:33:31.795
<00:33:29.035>It <00:33:29.175>always <00:33:29.315>depends <00:33:29.455>a <00:33:29.595>bit <00:33:29.735>what <00:33:29.875>exactly <00:33:30.015>are <00:33:30.155>your <00:33:30.295>use <00:33:30.435>cases?

782
00:33:31.865 --> 00:33:33.195
<00:33:31.865>What <00:33:32.064>exactly <00:33:32.264>do <00:33:32.463>you <00:33:32.663>want <00:33:32.862>to <00:33:33.062>achieve?

783
00:33:33.625 --> 00:33:35.315
<00:33:33.625>Also, <00:33:33.858>what <00:33:34.091>is <00:33:34.325>your <00:33:34.558>maturity <00:33:34.791>from <00:33:35.025>an

784
00:33:35.315 --> 00:33:36.355
<00:33:35.315>organizational <00:33:36.251>perspective?

785
00:33:36.505 --> 00:33:38.955
<00:33:36.505>What <00:33:36.680>we've <00:33:36.855>just <00:33:37.030>seen, <00:33:37.205>what <00:33:37.380>Acme <00:33:37.555>wants <00:33:37.730>to <00:33:37.905>implement.

786
00:33:39.295 --> 00:33:43.675
<00:33:39.295>That's <00:33:39.450>really, <00:33:39.606>that's <00:33:39.761>a <00:33:39.917>big <00:33:40.072>step <00:33:40.228>that's <00:33:40.383>really, <00:33:40.539>um, <00:33:40.695>um,

787
00:33:43.915 --> 00:33:46.795
<00:33:43.915>a <00:33:44.090>very, <00:33:44.264>a <00:33:44.440>best <00:33:44.615>practice, <00:33:44.790>a <00:33:44.965>best <00:33:45.139>practice <00:33:45.315>process.

788
00:33:48.535 --> 00:33:50.635
<00:33:48.535>So, <00:33:48.675>right, <00:33:48.815>uh, <00:33:48.955>such <00:33:49.095>a, <00:33:49.235>having <00:33:49.375>such <00:33:49.515>a <00:33:49.655>process <00:33:49.795>is <00:33:49.935>nice,

789
00:33:50.695 --> 00:33:53.515
<00:33:50.695>but <00:33:50.895>eventually <00:33:51.095>you <00:33:51.294>always <00:33:51.495>need <00:33:51.695>a <00:33:51.895>tech <00:33:52.095>stack

790
00:33:53.515 --> 00:33:55.515
<00:33:53.515>that <00:33:53.865>actually <00:33:54.215>powers <00:33:54.565>that <00:33:54.915>process.

791
00:33:56.255 --> 00:33:59.715
<00:33:56.255>Uh, <00:33:56.430>let's <00:33:56.605>quickly <00:33:56.780>hop <00:33:56.955>back <00:33:57.130>into <00:33:57.305>this <00:33:57.480>diagram <00:33:57.655>again.

792
00:34:00.105 --> 00:34:03.275
<00:34:00.105>When <00:34:00.232>we <00:34:00.359>went <00:34:00.486>over <00:34:00.614>this <00:34:00.741>diagram, <00:34:00.868>we <00:34:00.995>saw <00:34:01.123>a <00:34:01.250>few <00:34:01.377>things, <00:34:01.505>right?

793
00:34:03.295 --> 00:34:06.115
<00:34:03.295>We <00:34:03.450>saw <00:34:03.606>a <00:34:03.761>few <00:34:03.917>tools <00:34:04.072>that <00:34:04.228>we <00:34:04.383>need <00:34:04.539>to <00:34:04.695>choose.

794
00:34:06.735 --> 00:34:08.635
<00:34:06.735>For <00:34:06.909>example, <00:34:07.084>we <00:34:07.260>need <00:34:07.435>to <00:34:07.610>choose <00:34:07.784>a <00:34:07.959>feature <00:34:08.134>store.

795
00:34:09.255 --> 00:34:11.235
<00:34:09.255>We <00:34:09.455>have <00:34:09.655>a <00:34:09.855>variety <00:34:10.055>of <00:34:10.255>different <00:34:10.455>options <00:34:10.655>there.

796
00:34:11.495 --> 00:34:13.635
<00:34:11.495>We <00:34:11.670>need <00:34:11.844>to <00:34:12.020>choose <00:34:12.194>how <00:34:12.370>we're <00:34:12.545>gonna <00:34:12.719>orchestrate <00:34:12.895>our

797
00:34:13.635 --> 00:34:16.075
<00:34:13.635>experiments <00:34:13.810>and <00:34:13.985>also <00:34:14.160>how <00:34:14.335>we're <00:34:14.510>gonna <00:34:14.685>track <00:34:14.860>our <00:34:15.035>experiments.

798
00:34:16.655 --> 00:34:18.435
<00:34:16.655>Of <00:34:16.830>course, <00:34:17.005>we <00:34:17.180>need <00:34:17.355>to <00:34:17.530>choose <00:34:17.705>a <00:34:17.880>source <00:34:18.055>repository.

799
00:34:18.435 --> 00:34:19.715
<00:34:18.435>That's <00:34:18.723>probably <00:34:19.011>an <00:34:19.299>easier <00:34:19.587>choice.

800
00:34:20.705 --> 00:34:24.635
<00:34:20.705>Then <00:34:20.832>we <00:34:20.959>need <00:34:21.086>to <00:34:21.214>choose <00:34:21.341>how <00:34:21.468>we <00:34:21.595>will <00:34:21.723>do <00:34:21.850>our <00:34:21.977>CICD <00:34:22.105>pipeline.

801
00:34:25.985 --> 00:34:28.555
<00:34:25.985>Also, <00:34:26.125>what <00:34:26.265>are <00:34:26.405>we <00:34:26.545>gonna <00:34:26.685>use <00:34:26.825>as <00:34:26.965>a <00:34:27.105>model <00:34:27.245>registry, <00:34:27.385>right?

802
00:34:28.605 --> 00:34:30.955
<00:34:28.605>We've <00:34:28.804>heard <00:34:29.005>they're <00:34:29.205>very, <00:34:29.405>they're <00:34:29.605>very <00:34:29.804>simple <00:34:30.005>options,

803
00:34:31.055 --> 00:34:32.355
<00:34:31.054>but <00:34:31.347>then <00:34:31.639>they're <00:34:31.932>also <00:34:32.225>quite,

804
00:34:32.485 --> 00:34:35.075
<00:34:32.485>quite <00:34:32.951>sophisticated <00:34:33.418>model <00:34:33.885>registries.

805
00:34:36.135 --> 00:34:38.595
<00:34:36.135>We <00:34:36.310>need <00:34:36.485>to <00:34:36.660>think <00:34:36.835>about <00:34:37.010>what <00:34:37.185>should <00:34:37.360>support <00:34:37.535>us

806
00:34:38.775 --> 00:34:39.915
<00:34:38.775>to <00:34:39.117>serve <00:34:39.458>our <00:34:39.801>models?

807
00:34:40.445 --> 00:34:42.355
<00:34:40.445>Where <00:34:40.645>do <00:34:40.845>we <00:34:41.045>want <00:34:41.245>to <00:34:41.445>store <00:34:41.645>our <00:34:41.845>metadata?

808
00:34:42.695 --> 00:34:44.555
<00:34:42.695>And <00:34:42.870>also <00:34:43.045>what <00:34:43.220>tooling <00:34:43.395>do <00:34:43.570>we <00:34:43.745>want <00:34:43.920>to <00:34:44.095>use

809
00:34:44.555 --> 00:34:45.555
<00:34:44.554>to <00:34:44.855>track <00:34:45.154>our <00:34:45.455>performance?

810
00:34:45.855 --> 00:34:47.755
<00:34:45.855>And, <00:34:45.995>you <00:34:46.135>know, <00:34:46.275>these <00:34:46.415>are <00:34:46.554>by <00:34:46.695>far <00:34:46.835>not <00:34:46.975>all <00:34:47.115>the <00:34:47.255>choices

811
00:34:47.785 --> 00:34:51.395
<00:34:47.784>that <00:34:47.901>we <00:34:48.018>need <00:34:48.134>to <00:34:48.251>make <00:34:48.368>in <00:34:48.484>such <00:34:48.601>to, <00:34:48.718>to <00:34:48.835>build <00:34:48.951>a <00:34:49.068>tech <00:34:49.185>stack

812
00:34:51.395 --> 00:34:52.475
<00:34:51.395>that <00:34:51.638>supports <00:34:51.880>such <00:34:52.123>a <00:34:52.366>process.

813
00:34:53.255 --> 00:34:57.835
<00:34:53.255>So <00:34:53.410>from <00:34:53.566>a <00:34:53.721>probably, <00:34:53.877>perhaps, <00:34:54.032>you <00:34:54.188>might <00:34:54.343>think, <00:34:54.499>um, <00:34:54.655>well,

814
00:34:58.165 --> 00:35:00.795
<00:34:58.165>there <00:34:58.398>must <00:34:58.631>be <00:34:58.865>eight <00:34:59.098>dominant <00:34:59.331>stack, <00:34:59.565>right?

815
00:35:01.045 --> 00:35:02.555
<00:35:01.045>Can't <00:35:01.239>be <00:35:01.433>so <00:35:01.627>hard <00:35:01.821>to <00:35:02.015>make <00:35:02.209>these <00:35:02.404>choices.

816
00:35:03.205 --> 00:35:05.955
<00:35:03.205>There <00:35:03.360>must <00:35:03.516>be <00:35:03.671>a <00:35:03.827>stack <00:35:03.982>that <00:35:04.138>90% <00:35:04.293>of <00:35:04.449>companies <00:35:04.605>use

817
00:35:06.335 --> 00:35:07.915
<00:35:06.335>and <00:35:06.510>you <00:35:06.685>know, <00:35:06.860>we <00:35:07.035>can <00:35:07.210>just <00:35:07.385>go <00:35:07.560>with <00:35:07.735>that.

818
00:35:08.865 --> 00:35:12.995
<00:35:08.865>Well, <00:35:08.992>the <00:35:09.119>unfortunate <00:35:09.246>truth <00:35:09.374>is <00:35:09.501>that <00:35:09.628>when <00:35:09.755>it <00:35:09.883>comes <00:35:10.010>to <00:35:10.137>ML <00:35:10.265>ops,

819
00:35:13.255 --> 00:35:14.915
<00:35:13.255>that's <00:35:13.721>not <00:35:14.188>the <00:35:14.655>case.

820
00:35:15.245 --> 00:35:18.395
<00:35:15.245>There <00:35:15.595>is <00:35:15.944>no <00:35:16.295>dominant <00:35:16.645>stack.

821
00:35:18.395 --> 00:35:20.435
<00:35:18.395>There <00:35:18.628>is <00:35:18.861>a, <00:35:19.095>the <00:35:19.328>ML <00:35:19.561>ops <00:35:19.795>stack

822
00:35:20.905 --> 00:35:23.715
<00:35:20.905>because, <00:35:21.105>well, <00:35:21.305>this <00:35:21.505>field <00:35:21.705>is <00:35:21.905>very <00:35:22.105>new, <00:35:22.305>um,

823
00:35:23.815 --> 00:35:26.675
<00:35:23.815>and <00:35:24.015>it, <00:35:24.215>it <00:35:24.415>has <00:35:24.615>been <00:35:24.815>under <00:35:25.015>incredible <00:35:25.215>development

824
00:35:26.695 --> 00:35:30.275
<00:35:26.695>and <00:35:26.895>the <00:35:27.095>incredible <00:35:27.295>flux <00:35:27.495>over <00:35:27.695>the <00:35:27.895>past <00:35:28.095>years.

825
00:35:30.465 --> 00:35:33.795
<00:35:30.465>Yeah. <00:35:30.620>So <00:35:30.776>there <00:35:30.931>are, <00:35:31.087>of <00:35:31.242>course, <00:35:31.398>there <00:35:31.553>are <00:35:31.709>some <00:35:31.865>players

826
00:35:33.795 --> 00:35:35.155
<00:35:33.795>that <00:35:33.969>are <00:35:34.144>a <00:35:34.319>bit <00:35:34.494>more <00:35:34.669>established <00:35:34.844>than <00:35:35.019>others,

827
00:35:35.615 --> 00:35:37.365
<00:35:35.615>but <00:35:35.895>it <00:35:36.174>is <00:35:36.455>important <00:35:36.734>to <00:35:37.015>understand

828
00:35:37.365 --> 00:35:39.525
<00:35:37.365>that <00:35:37.564>this <00:35:37.765>ecosystem <00:35:37.964>is <00:35:38.165>under <00:35:38.365>such <00:35:38.564>a <00:35:38.765>flux

829
00:35:39.635 --> 00:35:42.725
<00:35:39.635>that <00:35:39.790>essentially <00:35:39.946>every <00:35:40.101>week <00:35:40.257>you <00:35:40.412>have <00:35:40.568>new <00:35:40.723>packages <00:35:40.879>pop <00:35:41.035>up,

830
00:35:42.785 --> 00:35:45.965
<00:35:42.785>new <00:35:42.960>tools <00:35:43.134>pop <00:35:43.310>up, <00:35:43.484>gaining, <00:35:43.660>gaining <00:35:43.835>support <00:35:44.009>very <00:35:44.185>fast,

831
00:35:46.055 --> 00:35:48.925
<00:35:46.055>other <00:35:46.254>tools, <00:35:46.455>getting <00:35:46.654>deprecated, <00:35:46.855>merged <00:35:47.055>with <00:35:47.254>other <00:35:47.455>tools,

832
00:35:49.065 --> 00:35:50.565
<00:35:49.065>big <00:35:49.335>players <00:35:49.605>stepping <00:35:49.875>into <00:35:50.145>the <00:35:50.415>game.

833
00:35:51.505 --> 00:35:55.045
<00:35:51.505>And, <00:35:51.632>and, <00:35:51.759>and, <00:35:51.886>and <00:35:52.014>then <00:35:52.141>you <00:35:52.268>do <00:35:52.395>not <00:35:52.523>only <00:35:52.650>have <00:35:52.777>this <00:35:52.905>plethora

834
00:35:55.045 --> 00:35:57.565
<00:35:55.045>of <00:35:55.245>tools, <00:35:55.445>but <00:35:55.645>what <00:35:55.845>you <00:35:56.045>also <00:35:56.245>have, <00:35:56.445>uh,

835
00:35:57.825 --> 00:36:00.925
<00:35:57.825>but <00:35:57.952>what <00:35:58.079>you <00:35:58.206>also <00:35:58.334>face <00:35:58.461>is <00:35:58.588>that <00:35:58.715>these <00:35:58.843>tools, <00:35:58.970>in <00:35:59.097>some <00:35:59.225>aspects,

836
00:36:00.925 --> 00:36:03.205
<00:36:00.925>they're <00:36:01.125>very <00:36:01.325>similar, <00:36:01.525>but <00:36:01.725>then <00:36:01.925>in <00:36:02.125>other <00:36:02.325>aspects,

837
00:36:03.205 --> 00:36:04.365
<00:36:03.205>they're <00:36:03.727>incredibly <00:36:04.249>different.

838
00:36:04.545 --> 00:36:06.765
<00:36:04.545>You <00:36:04.745>have <00:36:04.945>tools <00:36:05.145>that, <00:36:05.345>that <00:36:05.545>you <00:36:05.745>can <00:36:05.945>use

839
00:36:06.765 --> 00:36:09.285
<00:36:06.765>to <00:36:06.964>essentially <00:36:07.165>build <00:36:07.365>your <00:36:07.565>entire <00:36:07.765>platform <00:36:07.964>around, <00:36:08.165>right?

840
00:36:09.315 --> 00:36:10.965
<00:36:09.315>They <00:36:09.665>basically <00:36:10.015>deliver <00:36:10.365>everything <00:36:10.715>you

841
00:36:10.965 --> 00:36:12.045
<00:36:10.965>need <00:36:11.289>or <00:36:11.613>almost <00:36:11.937>everything.

842
00:36:12.585 --> 00:36:15.645
<00:36:12.585>And <00:36:12.712>then <00:36:12.839>you <00:36:12.966>have <00:36:13.094>tools <00:36:13.221>that <00:36:13.348>only <00:36:13.475>do <00:36:13.603>one <00:36:13.730>thing <00:36:13.857>very <00:36:13.985>well,

843
00:36:16.065 --> 00:36:18.565
<00:36:16.065>but <00:36:16.240>that <00:36:16.415>really, <00:36:16.590>very, <00:36:16.765>very <00:36:16.940>well <00:36:17.115>and <00:36:17.290>very <00:36:17.465>deep.

844
00:36:19.435 --> 00:36:21.085
<00:36:19.435>Then <00:36:19.590>you <00:36:19.746>have <00:36:19.901>tools <00:36:20.057>that <00:36:20.212>are <00:36:20.368>made <00:36:20.523>to <00:36:20.679>be <00:36:20.835>used

845
00:36:21.085 --> 00:36:23.845
<00:36:21.085>by <00:36:21.260>data <00:36:21.435>scientists, <00:36:21.610>others, <00:36:21.785>for <00:36:21.960>others, <00:36:22.135>you <00:36:22.310>need, <00:36:22.485>uh,

846
00:36:24.395 --> 00:36:26.045
<00:36:24.395>deep <00:36:24.861>ML <00:36:25.328>ops <00:36:25.795>knowledge.

847
00:36:26.905 --> 00:36:27.965
<00:36:26.905>Um, <00:36:27.382>and, <00:36:27.859>and,

848
00:36:28.025 --> 00:36:30.685
<00:36:28.025>and <00:36:28.225>so <00:36:28.425>these <00:36:28.625>tools <00:36:28.825>really <00:36:29.025>did <00:36:29.225>differ <00:36:29.425>in

849
00:36:30.865 --> 00:36:34.485
<00:36:30.865>so <00:36:31.020>many <00:36:31.176>different <00:36:31.331>dimensions <00:36:31.487>that <00:36:31.642>it <00:36:31.798>makes <00:36:31.953>it <00:36:32.109>really <00:36:32.265>hard

850
00:36:35.185 --> 00:36:37.045
<00:36:35.185>to <00:36:35.535>make <00:36:35.884>appropriate <00:36:36.235>tooling <00:36:36.585>choices.

851
00:36:38.325 --> 00:36:39.705
<00:36:38.325>So <00:36:38.635>what <00:36:38.946>ACME <00:36:39.256>does, <00:36:39.567>ac

852
00:36:39.855 --> 00:36:42.025
<00:36:39.855>what <00:36:40.055>Acme <00:36:40.255>does <00:36:40.455>is <00:36:40.655>they <00:36:40.855>recognize <00:36:41.055>this <00:36:41.255>right?

853
00:36:42.195 --> 00:36:45.545
<00:36:42.195>After <00:36:42.335>some <00:36:42.475>research, <00:36:42.615>they <00:36:42.755>see, <00:36:42.895>wow, <00:36:43.035>okay, <00:36:43.175>we <00:36:43.315>have <00:36:43.455>found, <00:36:43.595>um,

854
00:36:46.035 --> 00:36:48.385
<00:36:46.035>50 <00:36:46.234>block <00:36:46.435>posts, <00:36:46.634>everybody <00:36:46.835>recommending <00:36:47.035>a <00:36:47.234>different <00:36:47.435>tool

855
00:36:48.385 --> 00:36:51.745
<00:36:48.385>for <00:36:48.585>this, <00:36:48.785>and <00:36:48.985>that, <00:36:49.185>that <00:36:49.385>sounds <00:36:49.585>really <00:36:49.785>complex.

856
00:36:51.965 --> 00:36:53.785
<00:36:51.965>So <00:36:52.120>what <00:36:52.276>they <00:36:52.431>do <00:36:52.587>is <00:36:52.742>exactly <00:36:52.898>what <00:36:53.053>you <00:36:53.209>should <00:36:53.365>do.

857
00:36:53.935 --> 00:36:57.185
<00:36:53.935>They <00:36:54.134>start <00:36:54.335>structuring <00:36:54.535>information, <00:36:54.735>they <00:36:54.935>start <00:36:55.134>ask, <00:36:55.335>trying

858
00:36:57.185 --> 00:36:59.145
<00:36:57.185>to <00:36:57.418>ask <00:36:57.651>the <00:36:57.884>right <00:36:58.118>questions <00:36:58.351>they <00:36:58.585>need

859
00:36:59.165 --> 00:37:00.705
<00:36:59.165>to <00:36:59.442>actually <00:36:59.719>make <00:36:59.996>the <00:37:00.273>right <00:37:00.551>decisions.

860
00:37:00.885 --> 00:37:03.025
<00:37:00.885>And <00:37:01.085>for <00:37:01.285>that, <00:37:01.485>they <00:37:01.685>use <00:37:01.885>something <00:37:02.085>very <00:37:02.285>pragmatic.

861
00:37:03.285 --> 00:37:06.345
<00:37:03.285>Um, <00:37:03.460>and <00:37:03.634>ML <00:37:03.810>ops <00:37:03.984>Canvas, <00:37:04.160>such <00:37:04.335>a <00:37:04.509>canvas, <00:37:04.685>you,

862
00:37:06.345 --> 00:37:07.945
<00:37:06.345>you've <00:37:06.811>probably <00:37:07.278>seen <00:37:07.745>different

863
00:37:08.145 --> 00:37:09.705
<00:37:08.145>canvases, <00:37:08.495>uh, <00:37:08.845>throughout <00:37:09.195>your <00:37:09.545>career.

864
00:37:10.215 --> 00:37:12.865
<00:37:10.215>This <00:37:10.342>one <00:37:10.469>is <00:37:10.596>one <00:37:10.724>that's <00:37:10.851>a <00:37:10.978>bit <00:37:11.105>more <00:37:11.233>tailored <00:37:11.360>towards <00:37:11.487>ML <00:37:11.615>ops,

865
00:37:12.865 --> 00:37:14.185
<00:37:12.865>and <00:37:13.102>it <00:37:13.340>helps <00:37:13.577>you <00:37:13.815>ask <00:37:14.053>questions

866
00:37:14.725 --> 00:37:17.945
<00:37:14.725>and <00:37:14.924>structure <00:37:15.125>information <00:37:15.325>questions <00:37:15.525>such <00:37:15.725>as, <00:37:15.924>okay, <00:37:16.125>um,

867
00:37:18.205 --> 00:37:19.185
<00:37:18.205>how <00:37:18.381>do, <00:37:18.557>how <00:37:18.734>does <00:37:18.910>the <00:37:19.087>tool

868
00:37:19.385 --> 00:37:20.425
<00:37:19.385>actually <00:37:19.572>add <00:37:19.759>value <00:37:19.946>to <00:37:20.133>your <00:37:20.321>process?

869
00:37:21.335 --> 00:37:23.265
<00:37:21.335>Does <00:37:21.490>it <00:37:21.646>save <00:37:21.801>you <00:37:21.957>time? <00:37:22.112>Does <00:37:22.268>it <00:37:22.423>improve <00:37:22.579>your <00:37:22.735>quality?

870
00:37:23.325 --> 00:37:24.345
<00:37:23.325>Why <00:37:23.508>do <00:37:23.692>you <00:37:23.875>actually <00:37:24.059>need <00:37:24.243>it?

871
00:37:25.855 --> 00:37:28.705
<00:37:25.855>Also, <00:37:26.055>how <00:37:26.255>does <00:37:26.455>it <00:37:26.655>integrate <00:37:26.855>into <00:37:27.055>your <00:37:27.255>infrastructure?

872
00:37:29.005 --> 00:37:32.225
<00:37:29.005>That's <00:37:29.180>one <00:37:29.355>thing. <00:37:29.530>But <00:37:29.705>also, <00:37:29.880>how <00:37:30.055>does <00:37:30.230>it <00:37:30.405>integrate

873
00:37:32.255 --> 00:37:35.225
<00:37:32.255>with <00:37:32.455>other <00:37:32.655>steps <00:37:32.855>in <00:37:33.055>your <00:37:33.255>machine <00:37:33.455>learning <00:37:33.655>development

874
00:37:35.225 --> 00:37:36.585
<00:37:35.225>process <00:37:35.531>that <00:37:35.837>you've <00:37:36.143>sketched <00:37:36.449>out?

875
00:37:36.605 --> 00:37:39.265
<00:37:36.605>That's <00:37:36.780>a <00:37:36.955>tricky <00:37:37.130>one <00:37:37.305>because <00:37:37.480>some <00:37:37.655>tools, <00:37:37.830>they're <00:37:38.005>made

876
00:37:39.285 --> 00:37:40.585
<00:37:39.285>to <00:37:39.674>work <00:37:40.065>together <00:37:40.455>essentially.

877
00:37:40.845 --> 00:37:43.825
<00:37:40.845>And <00:37:41.000>then <00:37:41.156>there <00:37:41.311>are <00:37:41.467>tools, <00:37:41.622>uh, <00:37:41.778>that <00:37:41.933>will <00:37:42.089>require <00:37:42.245>you

878
00:37:43.825 --> 00:37:45.065
<00:37:43.825>to <00:37:44.104>write <00:37:44.383>tons <00:37:44.662>of <00:37:44.941>code

879
00:37:45.085 --> 00:37:47.065
<00:37:45.085>to <00:37:45.285>actually <00:37:45.485>MER <00:37:45.685>work, <00:37:45.885>make <00:37:46.085>them <00:37:46.285>work <00:37:46.485>together.

880
00:37:48.335 --> 00:37:49.785
<00:37:48.335>Also, <00:37:48.552>you <00:37:48.770>need, <00:37:48.987>you <00:37:49.205>should <00:37:49.422>be <00:37:49.640>aware

881
00:37:49.805 --> 00:37:51.665
<00:37:49.805>of <00:37:50.154>your <00:37:50.504>diversity <00:37:50.855>of <00:37:51.205>stakeholders.

882
00:37:52.045 --> 00:37:53.305
<00:37:52.045>You <00:37:52.328>might <00:37:52.612>think <00:37:52.895>that, <00:37:53.179>uh,

883
00:37:53.305 --> 00:37:55.505
<00:37:53.305>the <00:37:53.480>choices <00:37:53.654>you <00:37:53.830>make <00:37:54.004>while <00:37:54.180>they <00:37:54.355>impact, <00:37:54.529>of <00:37:54.705>course,

884
00:37:55.505 --> 00:37:58.505
<00:37:55.505>your <00:37:55.680>ML <00:37:55.855>ops <00:37:56.030>team <00:37:56.205>and <00:37:56.380>perhaps <00:37:56.555>your <00:37:56.730>data <00:37:56.905>scientists,

885
00:37:58.505 --> 00:37:59.505
<00:37:58.505>but <00:37:58.685>that's <00:37:58.865>not <00:37:59.045>quite <00:37:59.225>the <00:37:59.405>case.

886
00:37:59.765 --> 00:38:03.065
<00:37:59.765>Uh, <00:37:59.892>the <00:38:00.019>changes, <00:38:00.146>the <00:38:00.274>tools <00:38:00.401>you <00:38:00.528>pick <00:38:00.655>might <00:38:00.783>have <00:38:00.910>a <00:38:01.037>lot <00:38:01.165>wider

887
00:38:03.085 --> 00:38:05.225
<00:38:03.085>impact, <00:38:03.318>might <00:38:03.551>have <00:38:03.785>impact <00:38:04.018>on <00:38:04.251>the <00:38:04.485>ability

888
00:38:05.365 --> 00:38:07.065
<00:38:05.365>to <00:38:05.714>actually <00:38:06.064>explain <00:38:06.415>your <00:38:06.765>predictions.

889
00:38:07.255 --> 00:38:10.745
<00:38:07.255>They <00:38:07.410>might <00:38:07.566>have <00:38:07.721>impact <00:38:07.877>on <00:38:08.032>how <00:38:08.188>you <00:38:08.343>can <00:38:08.499>actually <00:38:08.655>deploy

890
00:38:11.165 --> 00:38:15.265
<00:38:11.165>and <00:38:11.365>which <00:38:11.565>infrastructure <00:38:11.765>is <00:38:11.965>the <00:38:12.165>right <00:38:12.365>one <00:38:12.565>also.

891
00:38:15.445 --> 00:38:17.505
<00:38:15.445>Um, <00:38:15.678>it's, <00:38:15.911>it's <00:38:16.145>important <00:38:16.378>to, <00:38:16.611>to <00:38:16.845>recognize

892
00:38:17.505 --> 00:38:20.785
<00:38:17.505>that <00:38:17.705>different <00:38:17.905>packages <00:38:18.105>have <00:38:18.305>different <00:38:18.505>needs <00:38:18.705>for <00:38:18.905>skills.

893
00:38:21.245 --> 00:38:23.985
<00:38:21.245>So <00:38:21.444>depending <00:38:21.645>on <00:38:21.845>the <00:38:22.045>tool, <00:38:22.245>you're <00:38:22.444>gonna <00:38:22.645>need

894
00:38:24.575 --> 00:38:27.065
<00:38:24.575>very <00:38:24.730>different <00:38:24.886>skills <00:38:25.041>for <00:38:25.197>setting <00:38:25.352>it <00:38:25.508>up, <00:38:25.663>for <00:38:25.819>maintaining <00:38:25.975>it,

895
00:38:27.065 --> 00:38:28.105
<00:38:27.065>and <00:38:27.299>also <00:38:27.533>for <00:38:27.767>using <00:38:28.001>it.

896
00:38:28.275 --> 00:38:30.625
<00:38:28.275>There <00:38:28.402>are <00:38:28.529>tools <00:38:28.656>that <00:38:28.784>are <00:38:28.911>made <00:38:29.038>to <00:38:29.165>be <00:38:29.293>used <00:38:29.420>by <00:38:29.547>data <00:38:29.675>scientists,

897
00:38:30.765 --> 00:38:33.585
<00:38:30.765>and <00:38:30.920>then <00:38:31.076>there <00:38:31.231>are <00:38:31.387>tools <00:38:31.542>that <00:38:31.698>require <00:38:31.853>very <00:38:32.009>deep <00:38:32.165>knowledge

898
00:38:33.605 --> 00:38:35.965
<00:38:33.605>of <00:38:33.838>DevOps <00:38:34.071>or <00:38:34.305>ML <00:38:34.538>ops, <00:38:34.771>for <00:38:35.005>example.

899
00:38:37.665 --> 00:38:40.205
<00:38:37.665>So <00:38:37.820>Acme, <00:38:37.976>they <00:38:38.131>do <00:38:38.287>their <00:38:38.442>homework, <00:38:38.598>they <00:38:38.753>do <00:38:38.909>that, <00:38:39.065>right?

900
00:38:40.205 --> 00:38:41.845
<00:38:40.205>They <00:38:40.438>fill <00:38:40.671>out <00:38:40.904>20 <00:38:41.138>of <00:38:41.371>these <00:38:41.605>canvases,

901
00:38:41.845 --> 00:38:42.965
<00:38:41.845>they <00:38:42.096>do <00:38:42.349>the <00:38:42.601>tooling <00:38:42.853>research,

902
00:38:43.105 --> 00:38:46.205
<00:38:43.105>and, <00:38:43.280>uh, <00:38:43.455>they <00:38:43.630>really <00:38:43.805>spend <00:38:43.980>a <00:38:44.155>week <00:38:44.330>doing <00:38:44.505>this.

903
00:38:46.225 --> 00:38:47.685
<00:38:46.225>So <00:38:46.444>they <00:38:46.663>have <00:38:46.882>the <00:38:47.101>feeling, <00:38:47.319>yes, <00:38:47.538>uh,

904
00:38:47.785 --> 00:38:50.565
<00:38:47.785>we <00:38:48.018>can <00:38:48.251>now <00:38:48.484>make <00:38:48.718>our <00:38:48.951>tech <00:38:49.185>decisions.

905
00:38:50.625 --> 00:38:54.845
<00:38:50.625>We <00:38:50.780>can <00:38:50.936>essentially <00:38:51.091>put <00:38:51.247>some <00:38:51.402>fancy <00:38:51.558>logos <00:38:51.713>on <00:38:51.869>the <00:38:52.025>process

906
00:38:55.065 --> 00:38:56.725
<00:38:55.065>and <00:38:55.298>we <00:38:55.531>can <00:38:55.765>start <00:38:55.998>building <00:38:56.231>that <00:38:56.465>platform.

907
00:38:57.555 --> 00:39:02.405
<00:38:57.555>However, <00:38:57.710>also, <00:38:57.866>they, <00:38:58.021>they <00:38:58.177>rec, <00:38:58.332>they <00:38:58.488>see <00:38:58.643>that, <00:38:58.799>um, <00:38:58.955>well,

908
00:39:02.405 --> 00:39:05.005
<00:39:02.405>there <00:39:02.605>is <00:39:02.805>an <00:39:03.005>elephant <00:39:03.205>in <00:39:03.405>the <00:39:03.605>room <00:39:03.805>specifically.

909
00:39:05.795 --> 00:39:08.445
<00:39:05.795>They <00:39:05.950>have <00:39:06.106>the <00:39:06.261>feeling <00:39:06.417>that <00:39:06.572>they <00:39:06.728>keep <00:39:06.883>stumbling <00:39:07.039>upon <00:39:07.195>this

910
00:39:08.445 --> 00:39:09.645
<00:39:08.445>big <00:39:08.805>strategic <00:39:09.165>question <00:39:09.525>here.

911
00:39:10.065 --> 00:39:12.525
<00:39:10.065>You <00:39:10.192>know, <00:39:10.319>are <00:39:10.446>we <00:39:10.574>gonna <00:39:10.701>build <00:39:10.828>our <00:39:10.955>platform <00:39:11.083>in <00:39:11.210>a <00:39:11.337>custom <00:39:11.465>way

912
00:39:12.525 --> 00:39:14.485
<00:39:12.525>or <00:39:12.725>are <00:39:12.925>we <00:39:13.125>gonna <00:39:13.325>build <00:39:13.525>a <00:39:13.725>managed <00:39:13.925>platform

913
00:39:14.865 --> 00:39:16.725
<00:39:14.865>and <00:39:15.064>are <00:39:15.265>we <00:39:15.464>gonna <00:39:15.665>use <00:39:15.865>a <00:39:16.064>managed <00:39:16.265>platform?

914
00:39:16.725 --> 00:39:18.565
<00:39:16.725>And <00:39:16.958>that's <00:39:17.191>an <00:39:17.424>important <00:39:17.658>question <00:39:17.891>to <00:39:18.125>ask.

915
00:39:19.035 --> 00:39:20.445
<00:39:19.035>Yeah, <00:39:19.352>that's <00:39:19.669>a <00:39:19.986>difficult <00:39:20.304>one

916
00:39:20.445 --> 00:39:22.565
<00:39:20.445>that's <00:39:20.620>gonna <00:39:20.795>tie <00:39:20.970>you <00:39:21.145>down <00:39:21.320>for <00:39:21.495>years <00:39:21.670>to <00:39:21.845>come.

917
00:39:22.865 --> 00:39:24.965
<00:39:22.865>So <00:39:23.020>what <00:39:23.176>they <00:39:23.331>do <00:39:23.487>is <00:39:23.642>they <00:39:23.798>sketch <00:39:23.953>out <00:39:24.109>two <00:39:24.265>scenarios.

918
00:39:25.175 --> 00:39:27.765
<00:39:25.175>First <00:39:25.375>they <00:39:25.575>sketch <00:39:25.775>out <00:39:25.975>the <00:39:26.175>custom <00:39:26.375>scenario, <00:39:26.575>right?

919
00:39:27.765 --> 00:39:29.525
<00:39:27.765>What, <00:39:27.940>what <00:39:28.115>would <00:39:28.290>happen <00:39:28.464>if <00:39:28.640>you <00:39:28.815>build <00:39:28.990>the <00:39:29.165>platform

920
00:39:29.525 --> 00:39:30.925
<00:39:29.525>by <00:39:29.840>wiring <00:39:30.155>together <00:39:30.470>different <00:39:30.785>tools?

921
00:39:31.345 --> 00:39:32.565
<00:39:31.345>And <00:39:31.501>that's <00:39:31.658>what <00:39:31.815>it <00:39:31.972>kind <00:39:32.129>of <00:39:32.286>looks <00:39:32.443>like.

922
00:39:32.665 --> 00:39:35.285
<00:39:32.665>You <00:39:32.805>can <00:39:32.945>see <00:39:33.085>a <00:39:33.225>few <00:39:33.365>fancy <00:39:33.505>words <00:39:33.645>here. <00:39:33.785>ML <00:39:33.925>flow <00:39:34.065>probably.

923
00:39:35.425 --> 00:39:36.805
<00:39:35.425>Uh, <00:39:35.602>that's <00:39:35.779>a <00:39:35.957>very, <00:39:36.134>that's <00:39:36.312>a <00:39:36.489>very <00:39:36.667>popular

924
00:39:36.805 --> 00:39:38.285
<00:39:36.805>package <00:39:37.249>for <00:39:37.692>experiment <00:39:38.136>tracking.

925
00:39:38.305 --> 00:39:40.885
<00:39:38.305>And <00:39:38.460>also <00:39:38.616>as <00:39:38.771>a <00:39:38.927>model <00:39:39.082>registry, <00:39:39.238>for <00:39:39.393>example, <00:39:39.549>cube <00:39:39.705>flow

926
00:39:40.885 --> 00:39:43.125
<00:39:40.885>for <00:39:41.235>pipeline <00:39:41.585>orchestration, <00:39:41.935>hops <00:39:42.285>works

927
00:39:43.145 --> 00:39:44.645
<00:39:43.145>as <00:39:43.370>a <00:39:43.595>feature <00:39:43.820>store <00:39:44.045>end, <00:39:44.270>end <00:39:44.495>end.

928
00:39:44.985 --> 00:39:46.605
<00:39:44.985>You <00:39:45.160>can <00:39:45.335>see <00:39:45.510>you're <00:39:45.685>gonna <00:39:45.860>be, <00:39:46.035>you're <00:39:46.210>gonna <00:39:46.385>have

929
00:39:46.605 --> 00:39:47.765
<00:39:46.605>to <00:39:46.779>wire <00:39:46.953>together <00:39:47.127>a <00:39:47.301>lot <00:39:47.475>of <00:39:47.649>things.

930
00:39:48.665 --> 00:39:50.085
<00:39:48.665>On <00:39:48.920>the <00:39:49.176>other <00:39:49.431>hand, <00:39:49.687>a <00:39:49.943>managed

931
00:39:50.365 --> 00:39:51.445
<00:39:50.365>platform <00:39:50.607>is <00:39:50.851>quite <00:39:51.094>different <00:39:51.337>here.

932
00:39:52.595 --> 00:39:54.285
<00:39:52.595>It's, <00:39:53.061>um, <00:39:53.528>basically <00:39:53.995>they,

933
00:39:54.315 --> 00:39:57.085
<00:39:54.315>they <00:39:54.515>say <00:39:54.715>Amazon <00:39:54.915>SageMaker <00:39:55.115>could <00:39:55.315>be <00:39:55.515>reasonable <00:39:55.715>choice.

934
00:39:57.145 --> 00:39:59.605
<00:39:57.145>So <00:39:57.272>let's <00:39:57.399>go <00:39:57.526>for <00:39:57.654>that. <00:39:57.781>Let's, <00:39:57.908>let's <00:39:58.035>have <00:39:58.163>a <00:39:58.290>look <00:39:58.417>at <00:39:58.545>this.

935
00:39:59.825 --> 00:40:02.045
<00:39:59.825>And <00:40:00.024>you <00:40:00.225>can <00:40:00.424>see <00:40:00.625>SageMaker <00:40:00.825>essentially <00:40:01.024>delivers <00:40:01.225>you

936
00:40:02.315 --> 00:40:04.805
<00:40:02.315>what <00:40:02.595>you <00:40:02.875>need <00:40:03.155>throughout <00:40:03.435>your <00:40:03.715>process

937
00:40:05.065 --> 00:40:08.725
<00:40:05.065>and <00:40:05.205>even <00:40:05.345>what <00:40:05.485>you <00:40:05.625>can't <00:40:05.765>get <00:40:05.905>from, <00:40:06.045>from, <00:40:06.185>from <00:40:06.325>SageMaker, <00:40:06.465>uh,

938
00:40:08.785 --> 00:40:11.805
<00:40:08.785>you <00:40:08.960>find <00:40:09.134>in <00:40:09.310>the <00:40:09.484>AWS <00:40:09.660>ecosystem <00:40:09.835>somewhere <00:40:10.009>else, <00:40:10.185>right?

939
00:40:11.865 --> 00:40:14.525
<00:40:11.865>That's <00:40:12.040>a <00:40:12.214>very <00:40:12.390>classic <00:40:12.564>example <00:40:12.740>of <00:40:12.915>a <00:40:13.089>managed <00:40:13.265>platform.

940
00:40:15.505 --> 00:40:17.645
<00:40:15.505>The, <00:40:15.738>the <00:40:15.971>real <00:40:16.205>choice <00:40:16.438>then <00:40:16.671>burns <00:40:16.905>down

941
00:40:17.645 --> 00:40:19.205
<00:40:17.645>to <00:40:17.995>a <00:40:18.345>few <00:40:18.695>dimensions <00:40:19.045>dimension

942
00:40:19.205 --> 00:40:20.565
<00:40:19.205>that <00:40:19.449>you <00:40:19.694>should <00:40:19.939>pay <00:40:20.184>attention <00:40:20.429>to.

943
00:40:20.825 --> 00:40:21.885
<00:40:20.825>Not <00:40:20.984>all <00:40:21.143>of <00:40:21.302>them <00:40:21.461>are <00:40:21.620>listed <00:40:21.779>here,

944
00:40:21.885 --> 00:40:23.965
<00:40:21.885>but <00:40:22.085>some <00:40:22.285>important <00:40:22.485>ones <00:40:22.685>are <00:40:22.885>listed <00:40:23.085>here, <00:40:23.285>such

945
00:40:23.965 --> 00:40:25.405
<00:40:23.965>as <00:40:24.613>process <00:40:25.261>integration.

946
00:40:26.235 --> 00:40:28.445
<00:40:26.235>With <00:40:26.390>a <00:40:26.546>managed <00:40:26.701>platform, <00:40:26.857>getting <00:40:27.012>everything <00:40:27.168>out <00:40:27.323>of <00:40:27.479>one <00:40:27.635>hand,

947
00:40:28.445 --> 00:40:29.965
<00:40:28.445>you <00:40:28.673>will <00:40:28.901>have <00:40:29.129>a <00:40:29.357>very <00:40:29.585>smooth <00:40:29.813>experience.

948
00:40:29.965 --> 00:40:32.125
<00:40:29.965>On <00:40:30.165>the <00:40:30.365>other <00:40:30.565>hand, <00:40:30.765>building <00:40:30.965>a <00:40:31.165>custom <00:40:31.365>platform,

949
00:40:32.125 --> 00:40:34.525
<00:40:32.125>that's <00:40:32.280>never <00:40:32.436>gonna <00:40:32.591>be <00:40:32.747>as <00:40:32.902>smooth <00:40:33.058>probably <00:40:33.213>from <00:40:33.369>a <00:40:33.525>cost

950
00:40:34.525 --> 00:40:36.285
<00:40:34.525>perspective, <00:40:34.758>well, <00:40:34.991>that <00:40:35.225>could <00:40:35.458>be <00:40:35.691>very <00:40:35.925>different.

951
00:40:36.995 --> 00:40:39.325
<00:40:36.995>Managed <00:40:37.170>platform <00:40:37.345>sometimes <00:40:37.520>have <00:40:37.694>a <00:40:37.870>pay <00:40:38.045>per <00:40:38.220>use <00:40:38.395>model.

952
00:40:39.435 --> 00:40:42.245
<00:40:39.435>They <00:40:39.590>often <00:40:39.746>do <00:40:39.901>not <00:40:40.057>require <00:40:40.212>a <00:40:40.368>lot <00:40:40.523>of <00:40:40.679>upfront <00:40:40.835>investment.

953
00:40:42.385 --> 00:40:45.245
<00:40:42.385>Not <00:40:42.560>always <00:40:42.735>though, <00:40:42.910>building <00:40:43.085>a <00:40:43.260>custom <00:40:43.435>platform, <00:40:43.610>you <00:40:43.785>need

954
00:40:45.245 --> 00:40:46.285
<00:40:45.245>to <00:40:45.479>hire <00:40:45.712>a <00:40:45.946>dedicated <00:40:46.181>team.

955
00:40:46.405 --> 00:40:47.445
<00:40:46.405>A <00:40:46.639>lot <00:40:46.873>of <00:40:47.107>upfront <00:40:47.341>investment,

956
00:40:47.705 --> 00:40:49.125
<00:40:47.705>and <00:40:47.960>that's <00:40:48.216>gonna <00:40:48.471>be <00:40:48.727>quite <00:40:48.983>costly.

957
00:40:49.355 --> 00:40:50.485
<00:40:49.355>Also, <00:40:49.609>the <00:40:49.863>skills <00:40:50.117>you <00:40:50.372>require

958
00:40:50.485 --> 00:40:54.205
<00:40:50.485>for <00:40:50.640>building <00:40:50.796>a <00:40:50.951>custom <00:40:51.107>platform <00:40:51.262>are <00:40:51.418>very, <00:40:51.573>very, <00:40:51.729>uh, <00:40:51.885>rare.

959
00:40:54.475 --> 00:40:56.115
<00:40:54.475>Yeah. <00:40:55.875>And

960
00:40:56.425 --> 00:41:00.075
<00:40:56.425>however, <00:40:56.600>the <00:40:56.775>flexibility <00:40:56.950>you <00:40:57.125>get <00:40:57.300>from <00:40:57.475>building <00:40:57.650>a <00:40:57.825>custom

961
00:41:00.435 --> 00:41:04.235
<00:41:00.435>platform <00:41:00.590>is, <00:41:00.746>is <00:41:00.901>really, <00:41:01.057>um, <00:41:01.212>that <00:41:01.368>you <00:41:01.523>get <00:41:01.679>full <00:41:01.835>flexibility.

962
00:41:04.235 --> 00:41:05.835
<00:41:04.235>You <00:41:04.410>can <00:41:04.585>really <00:41:04.760>tailor <00:41:04.935>it <00:41:05.110>to <00:41:05.285>your <00:41:05.460>use <00:41:05.635>cases.

963
00:41:06.025 --> 00:41:09.475
<00:41:06.025>That <00:41:06.165>could <00:41:06.305>make <00:41:06.445>sense <00:41:06.585>if <00:41:06.725>you <00:41:06.865>have <00:41:07.005>very <00:41:07.145>specific <00:41:07.285>use <00:41:07.425>cases

964
00:41:09.625 --> 00:41:12.315
<00:41:09.625>that <00:41:09.825>a <00:41:10.025>managed <00:41:10.225>platform <00:41:10.425>does <00:41:10.625>not <00:41:10.825>accommodate <00:41:11.025>to.

965
00:41:13.465 --> 00:41:15.395
<00:41:13.465>Also, <00:41:13.640>you're <00:41:13.815>not <00:41:13.990>gonna <00:41:14.165>face <00:41:14.340>a <00:41:14.515>lock <00:41:14.690>in, <00:41:14.865>right?

966
00:41:15.825 --> 00:41:17.315
<00:41:15.825>Your <00:41:16.016>platform <00:41:16.208>is <00:41:16.399>not <00:41:16.591>gonna <00:41:16.782>lock <00:41:16.974>you <00:41:17.166>in.

967
00:41:17.315 --> 00:41:19.075
<00:41:17.315>Some, <00:41:17.595>some <00:41:17.875>specific <00:41:18.155>packages <00:41:18.435>might <00:41:18.715>do,

968
00:41:19.075 --> 00:41:22.315
<00:41:19.075>but <00:41:19.230>it's <00:41:19.386>always <00:41:19.541>easy <00:41:19.697>to <00:41:19.852>swap <00:41:20.008>out <00:41:20.163>a <00:41:20.319>specific <00:41:20.475>package.

969
00:41:22.385 --> 00:41:23.875
<00:41:22.385>Then <00:41:22.832>actually <00:41:23.279>change <00:41:23.726>platforms.

970
00:41:23.875 --> 00:41:25.155
<00:41:23.875>If <00:41:24.105>you <00:41:24.335>wanna <00:41:24.566>change <00:41:24.796>from <00:41:25.027>SageMaker

971
00:41:25.295 --> 00:41:27.035
<00:41:25.295>to <00:41:25.575>let's <00:41:25.855>say <00:41:26.135>Google's <00:41:26.415>Vertex <00:41:26.695>ai,

972
00:41:27.415 --> 00:41:31.255
<00:41:27.415>that's <00:41:27.590>gonna <00:41:27.765>definitely <00:41:27.940>be <00:41:28.115>quite <00:41:28.290>a <00:41:28.465>challenge <00:41:28.640>for <00:41:28.815>Acme,

973
00:41:31.685 --> 00:41:33.095
<00:41:31.685>this <00:41:32.002>choice <00:41:32.319>is <00:41:32.636>relatively <00:41:32.953>clear.

974
00:41:33.275 --> 00:41:36.205
<00:41:33.275>They <00:41:33.402>say, <00:41:33.529>well, <00:41:33.656>we <00:41:33.784>really <00:41:33.911>want <00:41:34.038>to <00:41:34.165>go <00:41:34.293>for <00:41:34.420>a <00:41:34.547>managed <00:41:34.675>platform

975
00:41:36.205 --> 00:41:37.405
<00:41:36.205>for <00:41:36.475>SageMaker, <00:41:36.745>to <00:41:37.015>be <00:41:37.285>honest,

976
00:41:37.635 --> 00:41:39.165
<00:41:37.635>because <00:41:38.094>they <00:41:38.553>say, <00:41:39.012>well,

977
00:41:39.585 --> 00:41:42.125
<00:41:39.585>our <00:41:39.760>long-term <00:41:39.935>IT <00:41:40.110>strategy <00:41:40.285>is <00:41:40.460>based <00:41:40.635>in <00:41:40.810>AWS <00:41:40.985>anyway.

978
00:41:42.125 --> 00:41:44.245
<00:41:42.125>We <00:41:42.358>wanna <00:41:42.591>become <00:41:42.825>cloud <00:41:43.058>native <00:41:43.291>on <00:41:43.525>AWS.

979
00:41:44.505 --> 00:41:46.085
<00:41:44.505>So <00:41:44.738>the <00:41:44.971>lockin <00:41:45.205>doesn't <00:41:45.438>really <00:41:45.671>worry <00:41:45.905>us.

980
00:41:46.155 --> 00:41:50.245
<00:41:46.155>Also, <00:41:46.330>we <00:41:46.505>believe <00:41:46.680>that <00:41:46.855>SageMaker <00:41:47.030>really, <00:41:47.205>uh, <00:41:47.380>is <00:41:47.555>topnotch

981
00:41:50.265 --> 00:41:52.725
<00:41:50.265>and <00:41:50.464>Amazon <00:41:50.665>has <00:41:50.865>big, <00:41:51.065>big <00:41:51.265>stake <00:41:51.464>in <00:41:51.665>it.

982
00:41:52.745 --> 00:41:56.005
<00:41:52.745>So <00:41:52.900>they're <00:41:53.056>gonna <00:41:53.211>stay <00:41:53.367>topnotch <00:41:53.522>for <00:41:53.678>the <00:41:53.833>next <00:41:53.989>few <00:41:54.145>years.

983
00:41:56.555 --> 00:41:58.765
<00:41:56.555>They <00:41:56.754>also <00:41:56.955>do <00:41:57.154>not <00:41:57.355>really <00:41:57.555>require <00:41:57.754>much <00:41:57.955>flexibility.

984
00:41:59.105 --> 00:42:00.405
<00:41:59.105>You <00:41:59.300>know, <00:41:59.495>they <00:41:59.690>have <00:41:59.885>their <00:42:00.080>use <00:42:00.275>cases,

985
00:42:00.585 --> 00:42:01.685
<00:42:00.585>but <00:42:00.915>they're <00:42:01.245>fairly <00:42:01.575>standard.

986
00:42:02.025 --> 00:42:03.885
<00:42:02.025>And <00:42:02.305>eventually <00:42:02.585>they <00:42:02.865>also <00:42:03.145>say, <00:42:03.425>well,

987
00:42:03.885 --> 00:42:05.085
<00:42:03.885>we <00:42:04.155>have <00:42:04.425>six <00:42:04.695>data <00:42:04.965>scientists

988
00:42:05.085 --> 00:42:06.685
<00:42:05.085>and <00:42:05.260>in <00:42:05.435>two <00:42:05.610>years <00:42:05.785>it's <00:42:05.960>perhaps <00:42:06.135>gonna <00:42:06.310>be <00:42:06.485>12,

989
00:42:06.865 --> 00:42:09.125
<00:42:06.865>but <00:42:07.020>we <00:42:07.176>don't <00:42:07.331>wanna <00:42:07.487>hire <00:42:07.642>a <00:42:07.798>team <00:42:07.953>of <00:42:08.109>five <00:42:08.265>people

990
00:42:09.225 --> 00:42:11.485
<00:42:09.225>to <00:42:09.365>build <00:42:09.505>a <00:42:09.645>platform <00:42:09.785>that's <00:42:09.924>gonna <00:42:10.065>be <00:42:10.205>used <00:42:10.345>by <00:42:10.485>10 <00:42:10.625>people.

991
00:42:11.905 --> 00:42:13.805
<00:42:11.905>You <00:42:12.080>know, <00:42:12.255>that <00:42:12.430>just <00:42:12.605>doesn't <00:42:12.780>pay <00:42:12.955>off <00:42:13.130>for <00:42:13.305>them.

992
00:42:14.545 --> 00:42:15.925
<00:42:14.545>So <00:42:14.793>the <00:42:15.041>choice <00:42:15.290>is <00:42:15.538>fairly <00:42:15.787>clear,

993
00:42:15.945 --> 00:42:19.485
<00:42:15.945>and <00:42:16.061>this <00:42:16.178>is <00:42:16.295>also <00:42:16.411>how <00:42:16.528>we <00:42:16.645>get <00:42:16.761>our <00:42:16.878>tech <00:42:16.995>stack <00:42:17.111>in <00:42:17.228>that <00:42:17.345>case.

994
00:42:20.025 --> 00:42:21.485
<00:42:20.025>So <00:42:20.287>we <00:42:20.550>have <00:42:20.813>our <00:42:21.076>process <00:42:21.339>now

995
00:42:21.825 --> 00:42:24.845
<00:42:21.825>and <00:42:21.980>we <00:42:22.136>have <00:42:22.291>mapped <00:42:22.447>our <00:42:22.602>tech <00:42:22.758>stack <00:42:22.913>onto <00:42:23.069>that <00:42:23.225>process.

996
00:42:24.905 --> 00:42:26.845
<00:42:24.905>So <00:42:25.080>we <00:42:25.255>roughly <00:42:25.430>know <00:42:25.605>what <00:42:25.780>tools <00:42:25.955>we <00:42:26.130>now <00:42:26.305>need

997
00:42:27.265 --> 00:42:29.645
<00:42:27.265>to <00:42:27.615>support <00:42:27.964>this <00:42:28.315>development <00:42:28.665>process.

998
00:42:31.505 --> 00:42:33.925
<00:42:31.505>Lastly, <00:42:31.680>right, <00:42:31.855>we <00:42:32.030>need <00:42:32.205>to <00:42:32.380>define <00:42:32.555>a <00:42:32.730>transition <00:42:32.905>strategy.

999
00:42:34.025 --> 00:42:36.445
<00:42:34.025>So <00:42:34.180>how <00:42:34.336>we <00:42:34.491>actually <00:42:34.647>go <00:42:34.802>from <00:42:34.958>what <00:42:35.113>we <00:42:35.269>currently <00:42:35.425>have,

1000
00:42:36.445 --> 00:42:39.485
<00:42:36.445>this <00:42:36.620>inefficient <00:42:36.795>manual <00:42:36.970>process <00:42:37.145>to <00:42:37.320>a <00:42:37.495>best <00:42:37.670>practice <00:42:37.845>process

1001
00:42:39.795 --> 00:42:40.805
<00:42:39.795>with <00:42:40.249>ML <00:42:40.703>ops.

1002
00:42:41.795 --> 00:42:43.845
<00:42:41.795>Acme <00:42:41.950>says, <00:42:42.106>well, <00:42:42.261>we <00:42:42.417>really <00:42:42.572>wanna <00:42:42.728>do <00:42:42.883>a <00:42:43.039>stepwise <00:42:43.195>transition.

1003
00:42:43.845 --> 00:42:48.005
<00:42:43.845>So <00:42:44.020>we <00:42:44.194>wanna <00:42:44.370>stepwise <00:42:44.544>incremental <00:42:44.720>layer <00:42:44.895>our <00:42:45.069>tool <00:42:45.245>stack,

1004
00:42:48.525 --> 00:42:51.285
<00:42:48.525>starting <00:42:48.725>with <00:42:48.925>the <00:42:49.125>starting <00:42:49.325>by <00:42:49.525>changing <00:42:49.725>the <00:42:49.925>aspect

1005
00:42:51.285 --> 00:42:54.205
<00:42:51.285>of <00:42:51.460>our <00:42:51.634>current <00:42:51.810>process <00:42:51.984>that <00:42:52.160>really <00:42:52.335>hurt <00:42:52.509>us <00:42:52.685>most,

1006
00:42:54.275 --> 00:42:56.605
<00:42:54.275>that <00:42:54.625>go <00:42:54.975>wrong <00:42:55.325>terribly, <00:42:55.675>perhaps.

1007
00:42:57.025 --> 00:42:59.925
<00:42:57.025>And <00:42:57.180>by, <00:42:57.336>by <00:42:57.491>doing <00:42:57.647>this, <00:42:57.802>by <00:42:57.958>first <00:42:58.113>tackling <00:42:58.269>the <00:42:58.425>things

1008
00:42:59.925 --> 00:43:02.045
<00:42:59.925>that <00:43:00.100>go <00:43:00.275>wrong <00:43:00.450>a <00:43:00.625>lot <00:43:00.800>in <00:43:00.975>our <00:43:01.150>current <00:43:01.325>process,

1009
00:43:02.105 --> 00:43:03.685
<00:43:02.105>we <00:43:02.338>can <00:43:02.571>relieve <00:43:02.805>these <00:43:03.038>pain <00:43:03.271>points <00:43:03.505>fast

1010
00:43:03.685 --> 00:43:05.805
<00:43:03.685>and <00:43:03.840>quickly <00:43:03.996>get <00:43:04.151>the <00:43:04.307>value <00:43:04.462>out <00:43:04.618>of <00:43:04.773>it <00:43:04.929>as <00:43:05.085>well.

1011
00:43:06.795 --> 00:43:09.125
<00:43:06.795>Also, <00:43:07.028>this <00:43:07.261>stepwise <00:43:07.495>transition, <00:43:07.728>step <00:43:07.961>by <00:43:08.195>step,

1012
00:43:09.325 --> 00:43:11.085
<00:43:09.325>changing <00:43:09.558>our <00:43:09.791>process, <00:43:10.024>introducing <00:43:10.258>a <00:43:10.491>new <00:43:10.725>tool.

1013
00:43:11.085 --> 00:43:14.165
<00:43:11.085>We <00:43:11.212>give <00:43:11.339>our <00:43:11.466>team <00:43:11.594>time <00:43:11.721>to <00:43:11.848>accommodate <00:43:11.975>to <00:43:12.103>it, <00:43:12.230>to <00:43:12.357>get <00:43:12.485>familiar

1014
00:43:14.165 --> 00:43:17.245
<00:43:14.165>with <00:43:14.340>it, <00:43:14.515>and <00:43:14.690>we <00:43:14.865>also <00:43:15.040>minimize <00:43:15.215>business <00:43:15.390>disruption, <00:43:15.565>right?

1015
00:43:17.305 --> 00:43:18.885
<00:43:17.305>Our <00:43:17.538>data <00:43:17.771>scientists, <00:43:18.004>they <00:43:18.238>can <00:43:18.471>keep <00:43:18.705>working

1016
00:43:18.885 --> 00:43:20.645
<00:43:18.885>essentially <00:43:20.285>concretely.

1017
00:43:21.065 --> 00:43:23.725
<00:43:21.065>For, <00:43:21.240>for <00:43:21.415>Acme, <00:43:21.590>this <00:43:21.765>means <00:43:21.940>that <00:43:22.115>they <00:43:22.290>will <00:43:22.465>start

1018
00:43:23.725 --> 00:43:25.125
<00:43:23.725>with <00:43:24.040>the <00:43:24.355>big <00:43:24.670>pain <00:43:24.985>point

1019
00:43:25.425 --> 00:43:27.605
<00:43:25.425>of <00:43:25.705>fixing <00:43:25.985>this <00:43:26.265>very <00:43:26.545>expensive <00:43:26.825>handover,

1020
00:43:27.865 --> 00:43:30.205
<00:43:27.865>the <00:43:28.040>handover <00:43:28.214>from <00:43:28.390>data <00:43:28.564>scientists <00:43:28.740>to <00:43:28.915>the <00:43:29.089>DevOps <00:43:29.265>team

1021
00:43:30.275 --> 00:43:31.805
<00:43:30.275>that <00:43:30.550>cost <00:43:30.825>them <00:43:31.101>weeks <00:43:31.376>and <00:43:31.652>months.

1022
00:43:32.355 --> 00:43:34.045
<00:43:32.355>What <00:43:32.530>they're <00:43:32.705>gonna <00:43:32.880>do <00:43:33.055>is <00:43:33.230>they're <00:43:33.405>gonna <00:43:33.580>switch <00:43:33.755>that.

1023
00:43:34.045 --> 00:43:36.085
<00:43:34.045>First, <00:43:34.278>they're <00:43:34.511>gonna <00:43:34.745>switch <00:43:34.978>from <00:43:35.211>deploying <00:43:35.445>at,

1024
00:43:36.225 --> 00:43:39.525
<00:43:36.225>at <00:43:36.400>the <00:43:36.575>local <00:43:36.750>Kubernetes <00:43:36.924>cluster <00:43:37.100>to <00:43:37.275>deploying <00:43:37.450>on <00:43:37.625>SageMaker.

1025
00:43:39.525 --> 00:43:41.245
<00:43:39.525>And <00:43:39.758>this <00:43:39.991>is <00:43:40.225>fairly <00:43:40.458>easy. <00:43:40.691>This <00:43:40.925>means

1026
00:43:41.245 --> 00:43:42.605
<00:43:41.245>that <00:43:41.551>the <00:43:41.857>data <00:43:42.163>scientists <00:43:42.469>now

1027
00:43:43.165 --> 00:43:45.045
<00:43:43.165>actually <00:43:43.340>have <00:43:43.515>the <00:43:43.690>deployment <00:43:43.865>in <00:43:44.040>serving <00:43:44.215>in <00:43:44.390>their <00:43:44.565>hands

1028
00:43:45.265 --> 00:43:47.965
<00:43:45.265>and <00:43:45.440>they <00:43:45.615>can <00:43:45.790>speed <00:43:45.964>up, <00:43:46.140>um, <00:43:46.315>going <00:43:46.490>from <00:43:46.665>experiments

1029
00:43:48.025 --> 00:43:49.245
<00:43:48.025>to <00:43:48.391>production <00:43:48.757>big <00:43:49.123>time.

1030
00:43:51.625 --> 00:43:54.085
<00:43:51.625>So <00:43:51.825>that's <00:43:52.025>our <00:43:52.225>transition <00:43:52.425>strategy <00:43:52.625>step <00:43:52.825>by <00:43:53.025>step.

1031
00:43:54.085 --> 00:43:55.565
<00:43:54.085>We <00:43:54.307>are <00:43:54.529>gonna <00:43:54.751>layer <00:43:54.973>our <00:43:55.195>tool <00:43:55.417>stack,

1032
00:43:55.565 --> 00:43:58.005
<00:43:55.565>we're <00:43:55.740>gonna <00:43:55.915>change <00:43:56.090>our <00:43:56.265>process <00:43:56.440>to <00:43:56.615>eventually <00:43:56.790>end <00:43:56.965>up

1033
00:43:58.005 --> 00:44:00.285
<00:43:58.005>with <00:43:58.180>our <00:43:58.355>target <00:43:58.530>process <00:43:58.705>over <00:43:58.880>the <00:43:59.055>course <00:43:59.230>of <00:43:59.405>one

1034
00:44:00.285 --> 00:44:01.685
<00:44:00.285>to <00:44:00.705>two <00:44:01.125>years, <00:44:01.545>right?

1035
00:44:01.745 --> 00:44:03.725
<00:44:01.745>And <00:44:01.944>by <00:44:02.145>taking <00:44:02.345>this <00:44:02.545>first <00:44:02.745>step <00:44:02.944>of <00:44:03.145>this,

1036
00:44:03.945 --> 00:44:06.045
<00:44:03.945>of <00:44:04.225>this <00:44:04.505>transition <00:44:04.785>strategy, <00:44:05.065>we <00:44:05.345>already

1037
00:44:06.785 --> 00:44:08.125
<00:44:06.785>do <00:44:07.086>a <00:44:07.388>highly <00:44:07.689>important <00:44:07.991>thing.

1038
00:44:08.125 --> 00:44:09.605
<00:44:08.125>We <00:44:08.458>bridge <00:44:08.791>our <00:44:09.124>deployment <00:44:09.457>gap.

1039
00:44:09.705 --> 00:44:12.365
<00:44:09.705>We <00:44:09.880>can <00:44:10.055>now <00:44:10.230>quickly <00:44:10.404>go <00:44:10.580>from <00:44:10.755>experiment <00:44:10.930>to <00:44:11.105>production,

1040
00:44:12.705 --> 00:44:14.565
<00:44:12.705>or <00:44:13.055>at <00:44:13.404>least <00:44:13.755>the <00:44:14.105>deployment

1041
00:44:15.195 --> 00:44:17.125
<00:44:15.195>barrier <00:44:15.428>does <00:44:15.661>not <00:44:15.895>hold <00:44:16.128>us <00:44:16.361>off <00:44:16.595>anymore.

1042
00:44:17.835 --> 00:44:20.165
<00:44:17.835>With <00:44:18.035>every <00:44:18.235>additional <00:44:18.435>step <00:44:18.635>they're <00:44:18.835>gonna <00:44:19.035>take, <00:44:19.235>uh,

1043
00:44:20.445 --> 00:44:22.765
<00:44:20.445>process <00:44:20.620>will <00:44:20.795>become <00:44:20.970>more <00:44:21.145>efficient <00:44:21.320>and <00:44:21.495>more <00:44:21.670>automated, <00:44:21.845>and

1044
00:44:22.765 --> 00:44:26.285
<00:44:22.765>therefore <00:44:22.998>already <00:44:23.231>also <00:44:23.464>limiting <00:44:23.698>human <00:44:23.931>error, <00:44:24.165>right?

1045
00:44:26.425 --> 00:44:30.205
<00:44:26.425>And <00:44:26.658>eventually <00:44:26.891>they <00:44:27.125>will <00:44:27.358>get <00:44:27.591>incredibly <00:44:27.825>fast

1046
00:44:30.205 --> 00:44:31.210
<00:44:30.205>iterations <00:44:30.431>out <00:44:30.657>of, <00:44:30.883>of <00:44:31.109>it.

1047
00:44:31.210 --> 00:44:34.565
<00:44:31.210>They <00:44:31.365>will <00:44:31.521>be <00:44:31.676>able <00:44:31.832>to <00:44:31.987>ship <00:44:32.143>machine <00:44:32.298>learning <00:44:32.454>systems <00:44:32.610>within

1048
00:44:35.765 --> 00:44:38.555
<00:44:35.765>hours <00:44:35.998>or <00:44:36.231>days <00:44:36.464>instead <00:44:36.698>of <00:44:36.931>weeks, <00:44:37.165>months,

1049
00:44:38.615 --> 00:44:40.075
<00:44:38.615>or <00:44:38.877>perhaps <00:44:39.140>even <00:44:39.403>a <00:44:39.666>year <00:44:39.928>before.

1050
00:44:40.295 --> 00:44:41.715
<00:44:40.295>And <00:44:40.508>this <00:44:40.721>is <00:44:40.934>exactly <00:44:41.147>what <00:44:41.360>you <00:44:41.573>need.

1051
00:44:41.815 --> 00:44:43.875
<00:44:41.815>You <00:44:42.048>need <00:44:42.281>to <00:44:42.515>ship <00:44:42.748>fast, <00:44:42.981>ship <00:44:43.215>often,

1052
00:44:44.255 --> 00:44:47.675
<00:44:44.255>and <00:44:44.430>ship <00:44:44.605>safely <00:44:44.780>for <00:44:44.955>machine <00:44:45.130>learning <00:44:45.305>systems <00:44:45.480>to <00:44:45.655>really,

1053
00:44:48.255 --> 00:44:50.195
<00:44:48.255>to <00:44:48.605>really <00:44:48.955>contribute <00:44:49.305>business <00:44:49.655>value.

1054
00:44:51.955 --> 00:44:52.995
<00:44:51.955>I <00:44:52.142>hope <00:44:52.329>you <00:44:52.516>enjoyed <00:44:52.703>the <00:44:52.891>talk

1055
00:44:52.995 --> 00:44:54.795
<00:44:52.995>and <00:44:53.170>you <00:44:53.345>learned <00:44:53.520>a <00:44:53.694>bit <00:44:53.870>about <00:44:54.045>machine <00:44:54.220>learning <00:44:54.395>operations

1056
00:44:54.795 --> 00:44:58.395
<00:44:54.795>and <00:44:54.935>why <00:44:55.075>it's <00:44:55.215>so <00:44:55.355>critical <00:44:55.495>to <00:44:55.635>really <00:44:55.775>get <00:44:55.915>the <00:44:56.055>value <00:44:56.195>out

1057
00:44:58.395 --> 00:45:00.355
<00:44:58.395>of <00:44:58.570>your <00:44:58.745>machine <00:44:58.920>learning <00:44:59.095>systems <00:44:59.270>that <00:44:59.445>you <00:44:59.620>would <00:44:59.795>like.

1058
00:45:01.095 --> 00:45:04.315
<00:45:01.095>Um, <00:45:01.294>if <00:45:01.495>you, <00:45:01.694>if <00:45:01.895>that <00:45:02.095>topic <00:45:02.294>interests <00:45:02.495>you,

1059
00:45:04.315 --> 00:45:05.835
<00:45:04.315>and <00:45:04.588>perhaps <00:45:04.862>not <00:45:05.135>only <00:45:05.409>this <00:45:05.683>topic,

1060
00:45:05.855 --> 00:45:08.155
<00:45:05.855>but <00:45:06.055>in <00:45:06.255>general, <00:45:06.455>if <00:45:06.655>you're <00:45:06.855>a <00:45:07.055>curious <00:45:07.255>engineer

1061
00:45:08.695 --> 00:45:11.235
<00:45:08.695>and <00:45:08.928>you're <00:45:09.161>interested <00:45:09.395>putting <00:45:09.628>your <00:45:09.861>work, <00:45:10.095>uh,

1062
00:45:11.385 --> 00:45:14.475
<00:45:11.385>putting <00:45:11.525>your <00:45:11.665>work <00:45:11.805>to <00:45:11.945>play <00:45:12.085>to <00:45:12.225>create <00:45:12.365>the <00:45:12.505>world <00:45:12.645>a <00:45:12.785>bit

1063
00:45:14.475 --> 00:45:16.915
<00:45:14.475>of <00:45:14.650>a <00:45:14.825>better <00:45:15.000>place, <00:45:15.174>uh, <00:45:15.350>then <00:45:15.525>also <00:45:15.700>check <00:45:15.875>out

1064
00:45:16.985 --> 00:45:19.355
<00:45:16.985>what <00:45:17.160>we <00:45:17.335>are <00:45:17.510>doing <00:45:17.685>at <00:45:17.860>transaction <00:45:18.035>monitoring <00:45:18.210>Netherlands, <00:45:18.385>TM

1065
00:45:19.355 --> 00:45:22.155
<00:45:19.355>and <00:45:19.482>L <00:45:19.609>10 <00:45:19.736>L, <00:45:19.864>we <00:45:19.991>are <00:45:20.118>hiring <00:45:20.245>big <00:45:20.373>time <00:45:20.500>to <00:45:20.627>build, <00:45:20.755>uh,

1066
00:45:22.275 --> 00:45:25.155
<00:45:22.275>a <00:45:22.555>worldwide, <00:45:22.835>world <00:45:23.115>worldwide <00:45:23.395>unique <00:45:23.675>transaction

1067
00:45:25.605 --> 00:45:27.115
<00:45:25.605>monitoring <00:45:26.964>platform.

1068
00:45:27.825 --> 00:45:29.195
<00:45:27.825>Also, <00:45:28.001>of <00:45:28.177>course, <00:45:28.353>if <00:45:28.529>you <00:45:28.705>have <00:45:28.881>any <00:45:29.058>questions

1069
00:45:29.225 --> 00:45:32.235
<00:45:29.225>regarding <00:45:29.365>the <00:45:29.505>talk, <00:45:29.645>feel <00:45:29.785>free <00:45:29.924>to <00:45:30.065>to <00:45:30.205>send <00:45:30.345>me <00:45:30.485>an <00:45:30.625>email.

1070
00:45:32.235 --> 00:45:35.275
<00:45:32.235>Always <00:45:32.390>happy <00:45:32.546>to <00:45:32.701>get <00:45:32.857>in <00:45:33.012>touch, <00:45:33.168>um, <00:45:33.323>perhaps <00:45:33.479>via <00:45:33.635>LinkedIn

1071
00:45:35.275 --> 00:45:36.675
<00:45:35.275>as <00:45:35.455>well, <00:45:35.635>and <00:45:35.815>we <00:45:35.995>can <00:45:36.175>exchange <00:45:36.355>a <00:45:36.535>few

1072
00:45:36.675 --> 00:45:38.075
<00:45:36.675>messages <00:45:36.990>ago <00:45:37.305>for <00:45:37.620>virtual <00:45:37.935>coffee.

1073
00:45:38.285 --> 00:45:39.035
<00:45:38.285>Thank <00:45:38.509>you <00:45:38.734>very <00:45:38.960>much.