WEBVTT

1
00:00:05.780 --> 00:00:09.410
<00:00:05.780>Hello, <00:00:06.060>hello, <00:00:06.340>hello. <00:00:06.620>Good <00:00:06.900>afternoon, <00:00:07.180>everybody.

2
00:00:10.280 --> 00:00:14.290
<00:00:10.280>Welcome <00:00:10.420>back. <00:00:10.559>We <00:00:10.700>are <00:00:10.840>gonna <00:00:10.979>head <00:00:11.120>now <00:00:11.260>to <00:00:11.399>our <00:00:11.540>next <00:00:11.680>talk.

3
00:00:15.150 --> 00:00:19.010
<00:00:15.150>And, <00:00:15.305>uh, <00:00:15.461>before, <00:00:15.616>let <00:00:15.772>me <00:00:15.927>again <00:00:16.083>introduce <00:00:16.238>myself. <00:00:16.394>I'm <00:00:16.550>jp,

4
00:00:19.610 --> 00:00:23.250
<00:00:19.610>I <00:00:19.717>am <00:00:19.825>moderator. <00:00:19.933>I'm <00:00:20.040>a <00:00:20.148>web <00:00:20.256>engineer <00:00:20.363>myself, <00:00:20.471>working <00:00:20.579>as <00:00:20.686>an <00:00:20.794>artist <00:00:20.902>right <00:00:21.009>now.

5
00:00:23.990 --> 00:00:27.410
<00:00:23.990>But <00:00:24.270>besides <00:00:24.549>that, <00:00:24.830>a <00:00:25.110>brief, <00:00:25.389>uh,

6
00:00:27.660 --> 00:00:32.450
<00:00:27.660>thank <00:00:27.767>you <00:00:27.875>to <00:00:27.983>all <00:00:28.090>of <00:00:28.198>our <00:00:28.306>partner <00:00:28.413>that <00:00:28.521>is <00:00:28.629>allowing <00:00:28.736>us <00:00:28.844>this <00:00:28.952>event <00:00:29.060>possible.

7
00:00:32.910 --> 00:00:37.370
<00:00:32.910>And <00:00:33.037>a <00:00:33.164>small <00:00:33.291>question <00:00:33.419>for <00:00:33.546>everybody, <00:00:33.673>have <00:00:33.800>you <00:00:33.928>all <00:00:34.055>downloaded <00:00:34.182>the, <00:00:34.309>uh,

8
00:00:37.390 --> 00:00:42.000
<00:00:37.390>we <00:00:37.740>are <00:00:38.090>developer <00:00:38.440>app <00:00:38.790>Answer?

9
00:00:42.660 --> 00:00:45.160
<00:00:42.660>Yes, <00:00:42.834>thank <00:00:43.010>you. <00:00:43.184>That's <00:00:43.360>the <00:00:43.535>spirit <00:00:43.709>that <00:00:43.885>we <00:00:44.059>want.

10
00:00:45.540 --> 00:00:48.560
<00:00:45.540>And <00:00:45.695>I <00:00:45.851>am <00:00:46.006>asking <00:00:46.162>you <00:00:46.317>that <00:00:46.473>because <00:00:46.628>through <00:00:46.784>the <00:00:46.940>app,

11
00:00:48.590 --> 00:00:52.600
<00:00:48.590>besides <00:00:48.730>checking <00:00:48.870>all <00:00:49.010>of <00:00:49.150>the <00:00:49.290>amazing <00:00:49.430>events <00:00:49.570>happening, <00:00:49.710>workshops <00:00:49.850>and <00:00:49.990>everything,

12
00:00:53.060 --> 00:00:54.200
<00:00:53.060>you <00:00:53.402>can <00:00:53.744>ask <00:00:54.086>questions.

13
00:00:55.100 --> 00:00:59.160
<00:00:55.100>So <00:00:55.200>if <00:00:55.300>you <00:00:55.400>find <00:00:55.500>something <00:00:55.600>very <00:00:55.700>interesting <00:00:55.800>in <00:00:55.900>the <00:00:56.000>talk <00:00:56.100>and <00:00:56.200>you <00:00:56.300>wanna <00:00:56.400>ask <00:00:56.500>a

14
00:00:59.280 --> 00:01:01.440
<00:00:59.280>question, <00:00:59.396>you <00:00:59.513>can <00:00:59.630>ask <00:00:59.746>it <00:00:59.863>in <00:00:59.980>the <00:01:00.096>app, <00:01:00.213>and <00:01:00.330>then <00:01:00.446>at <00:01:00.563>the <00:01:00.680>end,

15
00:01:01.860 --> 00:01:06.280
<00:01:01.860>we <00:01:01.960>can <00:01:02.060>have <00:01:02.160>some <00:01:02.260>times <00:01:02.360>and <00:01:02.460>then <00:01:02.560>read <00:01:02.660>some <00:01:02.760>q <00:01:02.860>and <00:01:02.960>a. <00:01:03.060>But <00:01:03.160>beside <00:01:03.260>that,

16
00:01:06.460 --> 00:01:10.600
<00:01:06.460>we <00:01:06.600>are <00:01:06.740>devi <00:01:06.880>diving <00:01:07.020>a <00:01:07.160>lot <00:01:07.300>into <00:01:07.440>AI <00:01:07.580>and <00:01:07.720>machine <00:01:07.860>learning.

17
00:01:10.940 --> 00:01:15.920
<00:01:10.940>We <00:01:11.067>started <00:01:11.194>from <00:01:11.321>the <00:01:11.449>morning <00:01:11.576>talking <00:01:11.703>about <00:01:11.830>how <00:01:11.958>is <00:01:12.085>machine <00:01:12.212>learning <00:01:12.340>affecting

18
00:01:15.920 --> 00:01:18.400
<00:01:15.920>our <00:01:16.270>everyday <00:01:16.620>life? <00:01:16.970>How, <00:01:17.320>uh,

19
00:01:18.400 --> 00:01:22.120
<00:01:18.400>what <00:01:18.600>is <00:01:18.800>the <00:01:19.000>dangerous <00:01:19.200>about <00:01:19.400>it? <00:01:19.600>How, <00:01:19.800>uh,

20
00:01:22.660 --> 00:01:26.720
<00:01:22.660>AI <00:01:22.815>is <00:01:22.971>getting <00:01:23.126>into <00:01:23.282>the <00:01:23.437>automotive <00:01:23.593>and <00:01:23.748>how <00:01:23.904>it <00:01:24.060>is,

21
00:01:27.460 --> 00:01:31.400
<00:01:27.460>uh, <00:01:27.615>helping <00:01:27.771>the <00:01:27.926>auto <00:01:28.082>industry <00:01:28.237>and <00:01:28.393>changing <00:01:28.548>the <00:01:28.704>auto <00:01:28.860>industry.

22
00:01:32.020 --> 00:01:36.320
<00:01:32.020>But <00:01:32.136>I <00:01:32.253>think <00:01:32.370>the <00:01:32.486>next <00:01:32.603>talk <00:01:32.720>is <00:01:32.836>amazing <00:01:32.953>because <00:01:33.070>we <00:01:33.186>are <00:01:33.303>gonna <00:01:33.420>have,

23
00:01:36.820 --> 00:01:39.800
<00:01:36.820>um, <00:01:37.100>a <00:01:37.380>data <00:01:37.660>scientist <00:01:37.940>that, <00:01:38.220>uh,

24
00:01:39.940 --> 00:01:42.920
<00:01:39.940>is <00:01:40.140>gonna <00:01:40.340>share <00:01:40.539>with <00:01:40.740>us <00:01:40.940>some <00:01:41.140>trends, <00:01:41.340>challenges,

25
00:01:42.920 --> 00:01:46.440
<00:01:42.920>and <00:01:43.120>best <00:01:43.320>practices <00:01:43.520>for <00:01:43.720>AI <00:01:43.920>at <00:01:44.120>the <00:01:44.320>edge.

26
00:01:46.900 --> 00:01:51.480
<00:01:46.900>And <00:01:47.040>now <00:01:47.180>I <00:01:47.320>would <00:01:47.460>like <00:01:47.600>to <00:01:47.740>give <00:01:47.880>the <00:01:48.020>stage <00:01:48.160>to <00:01:48.300>Irina

27
00:01:51.840 --> 00:01:55.440
<00:01:51.840>C <00:01:52.120>Nova. <00:01:52.400>Round <00:01:52.680>of <00:01:52.960>applause, <00:01:53.240>everybody.

28
00:02:10.570 --> 00:02:12.200
<00:02:10.570>Thank <00:02:10.850>you <00:02:11.130>for <00:02:11.410>this <00:02:11.690>lovely <00:02:11.970>introduction,

29
00:02:12.260 --> 00:02:15.840
<00:02:12.260>and <00:02:12.399>I'm <00:02:12.540>so <00:02:12.679>happy <00:02:12.820>to <00:02:12.959>be <00:02:13.100>back <00:02:13.239>here <00:02:13.380>at <00:02:13.519>Via <00:02:13.660>Developers.

30
00:02:16.080 --> 00:02:19.800
<00:02:16.079>I <00:02:16.207>was <00:02:16.334>presenting <00:02:16.461>here <00:02:16.589>already <00:02:16.716>last <00:02:16.843>year, <00:02:16.970>and <00:02:17.098>it <00:02:17.225>was <00:02:17.352>amazing. <00:02:17.480>Um,

31
00:02:20.500 --> 00:02:24.840
<00:02:20.500>the <00:02:20.655>topic <00:02:20.811>of <00:02:20.966>my <00:02:21.122>today's <00:02:21.277>talk <00:02:21.433>is, <00:02:21.588>uh, <00:02:21.744>trends, <00:02:21.900>challenges,

32
00:02:24.900 --> 00:02:27.760
<00:02:24.900>and <00:02:25.100>best <00:02:25.300>practices <00:02:25.500>for <00:02:25.700>AI <00:02:25.900>at <00:02:26.100>the <00:02:26.300>Edge.

33
00:02:27.980 --> 00:02:32.000
<00:02:27.980>So <00:02:28.096>what <00:02:28.213>do <00:02:28.329>we <00:02:28.446>understand <00:02:28.563>as <00:02:28.679>AI <00:02:28.796>at <00:02:28.913>the <00:02:29.030>edge? <00:02:29.146>Raise <00:02:29.263>your <00:02:29.380>hands.

34
00:02:32.020 --> 00:02:35.920
<00:02:32.020>Who <00:02:32.195>knows <00:02:32.370>what <00:02:32.545>AI <00:02:32.720>at <00:02:32.895>the <00:02:33.070>edge <00:02:33.245>means? <00:02:33.420>Okay,

35
00:02:36.350 --> 00:02:39.760
<00:02:36.350>half <00:02:36.489>of <00:02:36.630>you <00:02:36.769>probably <00:02:36.910>know. <00:02:37.049>So <00:02:37.190>by <00:02:37.329>AI <00:02:37.470>at <00:02:37.609>the <00:02:37.750>edge,

36
00:02:40.140 --> 00:02:45.120
<00:02:40.140>we <00:02:40.315>understand <00:02:40.489>AI <00:02:40.665>computations <00:02:40.839>happening <00:02:41.015>near <00:02:41.190>to <00:02:41.364>where <00:02:41.540>the

37
00:02:45.120 --> 00:02:48.400
<00:02:45.120>data <00:02:45.275>is <00:02:45.431>being <00:02:45.586>collected <00:02:45.742>at <00:02:45.897>the <00:02:46.053>edge <00:02:46.208>of <00:02:46.364>the <00:02:46.520>network.

38
00:02:48.980 --> 00:02:53.920
<00:02:48.980>So <00:02:49.135>imagine <00:02:49.291>a <00:02:49.446>fleet <00:02:49.602>of <00:02:49.757>embedded <00:02:49.913>devices <00:02:50.068>and <00:02:50.224>robots <00:02:50.380>operating

39
00:02:53.940 --> 00:02:54.773
<00:02:53.940>in <00:02:54.314>the <00:02:54.689>field,

40
00:02:54.820 --> 00:02:59.720
<00:02:54.820>and <00:02:54.920>they <00:02:55.019>don't <00:02:55.120>necessarily <00:02:55.220>have <00:02:55.320>a <00:02:55.420>network <00:02:55.519>connection <00:02:55.620>to <00:02:55.720>a <00:02:55.820>data <00:02:55.920>center <00:02:56.019>or <00:02:56.120>to <00:02:56.220>the

41
00:02:59.720 --> 00:03:00.553
cloud.

42
00:03:00.780 --> 00:03:05.080
<00:03:00.780>So <00:03:00.920>you <00:03:01.060>need <00:03:01.200>to <00:03:01.340>run <00:03:01.480>these <00:03:01.620>computations <00:03:01.760>in <00:03:01.900>this <00:03:02.040>constrained <00:03:02.180>environments.

43
00:03:05.940 --> 00:03:06.773
<00:03:05.940>And, <00:03:06.689>uh,

44
00:03:07.030 --> 00:03:11.520
<00:03:07.030>this <00:03:07.146>is <00:03:07.263>usually <00:03:07.380>a <00:03:07.496>challenging <00:03:07.613>process <00:03:07.730>because <00:03:07.846>these <00:03:07.963>devices <00:03:08.080>are <00:03:08.196>smaller <00:03:08.313>in <00:03:08.430>sizes

45
00:03:11.820 --> 00:03:13.440
<00:03:11.820>and <00:03:12.170>have <00:03:12.519>a <00:03:12.870>smaller <00:03:13.220>compute.

46
00:03:14.140 --> 00:03:18.760
<00:03:14.140>So <00:03:14.279>how <00:03:14.420>do <00:03:14.559>we <00:03:14.700>actually <00:03:14.839>optimize <00:03:14.980>our <00:03:15.119>AI <00:03:15.260>models <00:03:15.399>and <00:03:15.540>AI

47
00:03:18.760 --> 00:03:23.120
<00:03:18.760>applications <00:03:18.915>to <00:03:19.071>run <00:03:19.226>on <00:03:19.382>the <00:03:19.537>small <00:03:19.693>devices? <00:03:19.848>And <00:03:20.004>with <00:03:20.160>Nvidia,

48
00:03:23.180 --> 00:03:26.920
<00:03:23.180>we <00:03:23.355>have <00:03:23.530>collected, <00:03:23.705>uh, <00:03:23.880>some <00:03:24.055>best <00:03:24.230>practices <00:03:24.405>and <00:03:24.580>techniques.

49
00:03:27.540 --> 00:03:28.920
<00:03:27.540>And <00:03:27.850>in <00:03:28.161>my <00:03:28.471>today's <00:03:28.782>talk,

50
00:03:29.100 --> 00:03:33.800
<00:03:29.100>I'm <00:03:29.216>going <00:03:29.333>to <00:03:29.450>share <00:03:29.566>with <00:03:29.683>you <00:03:29.799>this <00:03:29.916>best <00:03:30.033>practices <00:03:30.150>and <00:03:30.266>techniques. <00:03:30.383>And, <00:03:30.500>uh,

51
00:03:34.550 --> 00:03:39.440
<00:03:34.550>well, <00:03:34.690>let <00:03:34.830>me <00:03:34.970>start <00:03:35.110>telling <00:03:35.250>you <00:03:35.390>that <00:03:35.530>actually <00:03:35.670>h <00:03:35.810>ai <00:03:35.950>transforms

52
00:03:39.700 --> 00:03:42.160
<00:03:39.700>almost <00:03:40.166>every <00:03:40.633>industry. <00:03:41.100>Um,

53
00:03:42.580 --> 00:03:46.200
<00:03:42.579>you <00:03:42.735>may <00:03:42.891>even <00:03:43.046>not <00:03:43.202>be <00:03:43.357>aware <00:03:43.513>of <00:03:43.668>that, <00:03:43.824>but, <00:03:43.980>uh,

54
00:03:46.590 --> 00:03:51.160
<00:03:46.590>it's <00:03:46.717>everywhere <00:03:46.844>in <00:03:46.971>our <00:03:47.099>life. <00:03:47.226>So <00:03:47.353>in <00:03:47.480>manufacturing, <00:03:47.608>we <00:03:47.735>rely <00:03:47.862>on, <00:03:47.990>uh,

55
00:03:51.160 --> 00:03:55.920
<00:03:51.160>distributed <00:03:51.359>edge <00:03:51.560>systems <00:03:51.760>for <00:03:51.960>applications <00:03:52.160>like <00:03:52.359>smart <00:03:52.560>factory

56
00:03:56.020 --> 00:03:58.920
<00:03:56.020>and <00:03:56.370>automation, <00:03:56.720>quality <00:03:57.070>control, <00:03:57.420>uh,

57
00:03:59.020 --> 00:04:03.320
<00:03:59.020>defect <00:03:59.220>detection <00:03:59.420>on <00:03:59.620>assembly <00:03:59.820>lines <00:04:00.020>in <00:04:00.220>retail, <00:04:00.420>uh,

58
00:04:03.340 --> 00:04:07.920
<00:04:03.340>you <00:04:03.480>may <00:04:03.620>know <00:04:03.760>about, <00:04:03.900>uh, <00:04:04.040>applications <00:04:04.180>like <00:04:04.320>automatic <00:04:04.460>checked <00:04:04.600>out <00:04:04.740>or,

59
00:04:08.260 --> 00:04:08.880
uh,

60
00:04:08.880 --> 00:04:13.800
<00:04:08.880>frictionless <00:04:08.987>shopping <00:04:09.095>where <00:04:09.203>you <00:04:09.310>walk <00:04:09.418>into <00:04:09.526>the <00:04:09.633>store <00:04:09.741>and <00:04:09.849>you <00:04:09.956>grab <00:04:10.064>some <00:04:10.172>goods <00:04:10.280>and,

61
00:04:14.020 --> 00:04:16.520
<00:04:14.020>uh, <00:04:14.220>you're <00:04:14.420>being <00:04:14.620>charged <00:04:14.820>without <00:04:15.020>even <00:04:15.220>going <00:04:15.420>to.

62
00:04:16.520 --> 00:04:21.480
<00:04:16.519>A <00:04:16.647>cashier <00:04:16.774>like <00:04:16.901>Amazon <00:04:17.029>goal <00:04:17.156>is <00:04:17.283>implementing <00:04:17.410>such <00:04:17.538>solutions <00:04:17.665>in <00:04:17.792>smart <00:04:17.919>cities

63
00:04:21.580 --> 00:04:22.440
<00:04:21.579>and <00:04:21.966>smart <00:04:22.354>spaces.

64
00:04:22.820 --> 00:04:27.200
<00:04:22.820>We <00:04:22.960>have <00:04:23.099>distributed <00:04:23.240>h <00:04:23.380>ai <00:04:23.520>and <00:04:23.659>I <00:04:23.800>o <00:04:23.940>OT <00:04:24.080>devices <00:04:24.219>everywhere.

65
00:04:27.530 --> 00:04:32.120
<00:04:27.530>Think <00:04:27.685>of <00:04:27.841>applications <00:04:27.996>like <00:04:28.152>smart <00:04:28.307>park, <00:04:28.463>smart <00:04:28.618>parking, <00:04:28.774>smart, <00:04:28.929>uh,

66
00:04:32.270 --> 00:04:33.840
<00:04:32.270>traffic <00:04:32.969>lights. <00:04:33.669>Uh,

67
00:04:34.020 --> 00:04:38.600
<00:04:34.020>so <00:04:34.147>this <00:04:34.274>is <00:04:34.401>all <00:04:34.529>being <00:04:34.656>powered <00:04:34.783>by <00:04:34.910>AI <00:04:35.038>at <00:04:35.165>the <00:04:35.292>edge. <00:04:35.419>Uh,

68
00:04:38.700 --> 00:04:43.120
<00:04:38.700>in <00:04:38.840>healthcare, <00:04:38.979>we <00:04:39.120>are <00:04:39.260>working <00:04:39.400>with, <00:04:39.539>uh, <00:04:39.680>uh, <00:04:39.820>surgical <00:04:39.960>robots. <00:04:40.099>Uh,

69
00:04:43.120 --> 00:04:47.960
<00:04:43.120>these <00:04:43.247>are <00:04:43.374>handheld <00:04:43.501>devices <00:04:43.629>which <00:04:43.756>apply <00:04:43.883>AI <00:04:44.010>in <00:04:44.138>order <00:04:44.265>to <00:04:44.392>assist <00:04:44.520>doctors

70
00:04:48.130 --> 00:04:52.120
<00:04:48.130>performing <00:04:48.363>surgeries. <00:04:48.596>Uh, <00:04:48.830>then <00:04:49.063>we <00:04:49.296>have, <00:04:49.530>uh,

71
00:04:52.520 --> 00:04:57.160
<00:04:52.520>logistics <00:04:52.659>where <00:04:52.799>we <00:04:52.940>have <00:04:53.080>smart <00:04:53.219>picking <00:04:53.359>and <00:04:53.500>placer <00:04:53.640>robots. <00:04:53.780>And, <00:04:53.919>uh,

72
00:04:57.820 --> 00:05:02.400
<00:04:57.820>in <00:04:57.960>agriculture <00:04:58.099>we <00:04:58.240>can <00:04:58.380>put <00:04:58.520>our <00:04:58.659>embedded <00:04:58.800>devices <00:04:58.940>on <00:04:59.080>a <00:04:59.219>tractor,

73
00:05:02.940 --> 00:05:06.600
<00:05:02.940>and <00:05:03.056>then <00:05:03.173>we <00:05:03.290>can <00:05:03.406>recognize <00:05:03.523>different <00:05:03.640>crops. <00:05:03.756>Is <00:05:03.873>it <00:05:03.990>a <00:05:04.106>crop <00:05:04.223>or <00:05:04.340>wheat?

74
00:05:06.620 --> 00:05:10.680
<00:05:06.620>And <00:05:06.760>then <00:05:06.900>we <00:05:07.040>can <00:05:07.180>treat <00:05:07.320>it <00:05:07.460>accordingly. <00:05:07.600>Uh, <00:05:07.740>at <00:05:07.880>home, <00:05:08.020>uh,

75
00:05:10.860 --> 00:05:14.240
<00:05:10.860>you <00:05:11.060>can <00:05:11.260>think <00:05:11.460>of, <00:05:11.660>uh, <00:05:11.860>robots, <00:05:12.060>which, <00:05:12.260>uh,

76
00:05:14.250 --> 00:05:18.640
<00:05:14.250>serve <00:05:14.405>you <00:05:14.561>as <00:05:14.716>entertainment <00:05:14.872>and <00:05:15.027>care <00:05:15.183>systems. <00:05:15.338>And <00:05:15.494>at <00:05:15.650>last,

77
00:05:18.660 --> 00:05:23.280
<00:05:18.660>but <00:05:18.767>not <00:05:18.875>least, <00:05:18.983>in <00:05:19.090>metals <00:05:19.198>and <00:05:19.306>mining, <00:05:19.413>uh, <00:05:19.521>we <00:05:19.629>can <00:05:19.736>have <00:05:19.844>unmanned <00:05:19.952>ground <00:05:20.060>vehicles,

78
00:05:23.280 --> 00:05:26.760
<00:05:23.280>which <00:05:23.559>autonomously <00:05:23.840>perform <00:05:24.119>hard <00:05:24.400>tasks, <00:05:24.679>uh,

79
00:05:27.430 --> 00:05:32.320
<00:05:27.430>replacing <00:05:27.663>humans. <00:05:27.896>So, <00:05:28.130>uh, <00:05:28.363>the, <00:05:28.596>the <00:05:28.830>applications,

80
00:05:32.540 --> 00:05:34.760
<00:05:32.540>uh, <00:05:32.740>there <00:05:32.940>are <00:05:33.140>many <00:05:33.340>applications <00:05:33.540>in <00:05:33.740>the <00:05:33.940>area,

81
00:05:35.100 --> 00:05:40.040
<00:05:35.100>and <00:05:35.227>there <00:05:35.354>is <00:05:35.481>a <00:05:35.609>huge <00:05:35.736>potential <00:05:35.863>for <00:05:35.990>businesses <00:05:36.118>in <00:05:36.245>this <00:05:36.372>domain. <00:05:36.500>Um,

82
00:05:40.460 --> 00:05:44.480
<00:05:40.460>and <00:05:40.576>the, <00:05:40.693>the, <00:05:40.810>there's <00:05:40.926>a <00:05:41.043>major <00:05:41.159>macro <00:05:41.276>trend, <00:05:41.393>which <00:05:41.510>we <00:05:41.626>observe <00:05:41.743>in, <00:05:41.859>uh,

83
00:05:44.540 --> 00:05:47.200
<00:05:44.540>the <00:05:44.773>past <00:05:45.006>couple <00:05:45.240>of <00:05:45.473>years. <00:05:45.706>So, <00:05:45.940>um,

84
00:05:47.700 --> 00:05:52.640
<00:05:47.700>we <00:05:47.827>started <00:05:47.954>with <00:05:48.081>the <00:05:48.209>fixed <00:05:48.336>devices <00:05:48.463>like <00:05:48.590>smart <00:05:48.718>cameras <00:05:48.845>mount <00:05:48.972>on <00:05:49.099>the

85
00:05:52.640 --> 00:05:53.473
wall,

86
00:05:53.610 --> 00:05:58.600
<00:05:53.610>which <00:05:53.785>then <00:05:53.960>started <00:05:54.135>gaining <00:05:54.310>some <00:05:54.485>mobility <00:05:54.660>and <00:05:54.835>evolved <00:05:55.010>into

87
00:05:58.710 --> 00:06:02.680
<00:05:58.710>complex <00:05:58.943>systems. <00:05:59.176>And <00:05:59.409>as <00:05:59.643>a <00:05:59.876>consequence, <00:06:00.109>uh,

88
00:06:03.220 --> 00:06:07.960
<00:06:03.220>AI <00:06:03.336>compute <00:06:03.453>power <00:06:03.570>demand <00:06:03.686>also <00:06:03.803>grows. <00:06:03.920>So <00:06:04.036>in <00:06:04.153>the <00:06:04.270>past <00:06:04.386>couple <00:06:04.503>of <00:06:04.620>years,

89
00:06:08.140 --> 00:06:12.520
<00:06:08.140>we <00:06:08.280>observed <00:06:08.419>the <00:06:08.560>trend <00:06:08.700>of <00:06:08.840>these <00:06:08.979>devices <00:06:09.120>being <00:06:09.260>more <00:06:09.400>complex <00:06:09.539>and

90
00:06:12.750 --> 00:06:17.400
<00:06:12.750>demanding <00:06:12.890>more <00:06:13.030>power. <00:06:13.170>Um, <00:06:13.310>think <00:06:13.450>of <00:06:13.590>the <00:06:13.730>applications <00:06:13.870>like <00:06:14.010>Evan <00:06:14.150>mentioned,

91
00:06:17.520 --> 00:06:21.920
<00:06:17.520>handheld <00:06:17.753>medical <00:06:17.986>instruments, <00:06:18.219>surgical <00:06:18.453>robots, <00:06:18.686>delivery <00:06:18.919>robots,

92
00:06:22.390 --> 00:06:25.160
<00:06:22.390>warehouse <00:06:22.590>and <00:06:22.789>logistics <00:06:22.990>systems <00:06:23.190>where <00:06:23.390>we <00:06:23.590>have, <00:06:23.789>uh,

93
00:06:25.640 --> 00:06:29.200
<00:06:25.640>a <00:06:25.873>fleet <00:06:26.106>of <00:06:26.340>interconnected <00:06:26.573>devices <00:06:26.806>monitoring, <00:06:27.039>uh,

94
00:06:29.200 --> 00:06:33.880
<00:06:29.200>different <00:06:29.355>things <00:06:29.511>in <00:06:29.666>the <00:06:29.822>factory <00:06:29.977>and <00:06:30.133>autonomously <00:06:30.288>moving <00:06:30.444>robots <00:06:30.599>and

95
00:06:33.880 --> 00:06:36.760
<00:06:33.880>vehicles. <00:06:34.580>And, <00:06:35.280>uh,

96
00:06:37.030 --> 00:06:39.600
<00:06:37.030>it's <00:06:37.263>important <00:06:37.496>to <00:06:37.729>say <00:06:37.963>that <00:06:38.196>edge <00:06:38.429>devices,

97
00:06:40.130 --> 00:06:45.040
<00:06:40.130>there <00:06:40.246>is <00:06:40.363>a <00:06:40.480>big <00:06:40.596>difference <00:06:40.713>between <00:06:40.830>edge <00:06:40.946>devices <00:06:41.063>and, <00:06:41.180>uh, <00:06:41.296>physical <00:06:41.413>data <00:06:41.530>center.

98
00:06:45.820 --> 00:06:50.120
<00:06:45.820>Uh, <00:06:45.960>first <00:06:46.099>big <00:06:46.240>difference <00:06:46.380>is <00:06:46.520>lower <00:06:46.659>compute. <00:06:46.800>So <00:06:46.940>this <00:06:47.080>embedded <00:06:47.219>devices,

99
00:06:50.350 --> 00:06:54.600
<00:06:50.350>they <00:06:50.490>usually <00:06:50.630>have, <00:06:50.770>they're <00:06:50.910>usually <00:06:51.050>small <00:06:51.190>in <00:06:51.330>size, <00:06:51.470>like <00:06:51.610>for <00:06:51.750>example,

100
00:06:54.660 --> 00:06:56.160
<00:06:54.659>our <00:06:55.109>Nvidia <00:06:55.559>Jetson <00:06:56.010>devices,

101
00:06:56.310 --> 00:07:01.080
<00:06:56.310>they <00:06:56.403>can <00:06:56.496>be <00:06:56.590>of <00:06:56.683>like <00:06:56.776>credit <00:06:56.870>card <00:06:56.963>size <00:06:57.056>as <00:06:57.150>a <00:06:57.243>consequence, <00:06:57.336>they <00:06:57.430>have <00:06:57.523>a <00:06:57.616>smaller <00:06:57.710>chip,

102
00:07:01.380 --> 00:07:06.240
<00:07:01.380>and <00:07:01.496>it <00:07:01.613>means <00:07:01.730>that <00:07:01.846>they <00:07:01.963>can <00:07:02.080>put, <00:07:02.196>uh, <00:07:02.313>less <00:07:02.430>computations <00:07:02.546>than, <00:07:02.663>for <00:07:02.780>example,

103
00:07:06.380 --> 00:07:10.960
<00:07:06.380>on <00:07:06.480>a <00:07:06.580>powerful <00:07:06.680>H <00:07:06.780>100 <00:07:06.880>or <00:07:06.980>a <00:07:07.080>100 <00:07:07.180>G <00:07:07.280>P <00:07:07.380>U <00:07:07.480>in <00:07:07.580>the <00:07:07.680>data <00:07:07.780>center.

104
00:07:11.580 --> 00:07:16.040
<00:07:11.580>So <00:07:11.687>the <00:07:11.795>compute <00:07:11.903>is <00:07:12.010>limited. <00:07:12.118>Um, <00:07:12.226>the <00:07:12.333>storage <00:07:12.441>of <00:07:12.549>these <00:07:12.656>devices <00:07:12.764>is <00:07:12.872>also <00:07:12.979>limited.

105
00:07:16.420 --> 00:07:19.960
<00:07:16.420>So <00:07:16.575>sometimes <00:07:16.731>these <00:07:16.886>devices <00:07:17.042>are <00:07:17.197>not <00:07:17.353>connected <00:07:17.508>to <00:07:17.664>the <00:07:17.820>network,

106
00:07:20.490 --> 00:07:24.680
<00:07:20.490>which <00:07:20.597>means <00:07:20.705>that <00:07:20.813>they <00:07:20.920>need <00:07:21.028>to <00:07:21.136>collect <00:07:21.243>all <00:07:21.351>the <00:07:21.459>data <00:07:21.566>and <00:07:21.674>store <00:07:21.782>it, <00:07:21.890>uh,

107
00:07:24.940 --> 00:07:29.400
<00:07:24.940>on <00:07:25.095>their <00:07:25.251>integrated <00:07:25.406>hard <00:07:25.562>drives. <00:07:25.717>And, <00:07:25.873>uh, <00:07:26.028>as <00:07:26.184>a <00:07:26.340>developer,

108
00:07:29.400 --> 00:07:30.760
<00:07:29.400>you <00:07:29.604>need <00:07:29.808>to <00:07:30.012>think <00:07:30.216>of <00:07:30.419>things <00:07:30.623>like,

109
00:07:30.820 --> 00:07:34.800
<00:07:30.820>do <00:07:30.936>I <00:07:31.053>store <00:07:31.170>all <00:07:31.286>the <00:07:31.403>data <00:07:31.520>collected <00:07:31.636>or <00:07:31.753>maybe <00:07:31.870>just <00:07:31.986>some <00:07:32.103>selective <00:07:32.219>data,

110
00:07:34.910 --> 00:07:36.640
<00:07:34.909>like <00:07:35.143>if <00:07:35.376>I <00:07:35.609>have <00:07:35.843>a <00:07:36.076>video <00:07:36.309>feed,

111
00:07:36.780 --> 00:07:39.960
<00:07:36.780>do <00:07:36.919>I <00:07:37.059>store <00:07:37.200>all <00:07:37.340>the <00:07:37.479>frames <00:07:37.619>or <00:07:37.760>maybe <00:07:37.900>some <00:07:38.039>selected <00:07:38.179>frames?

112
00:07:40.780 --> 00:07:43.120
<00:07:40.780>And <00:07:41.013>they <00:07:41.246>also <00:07:41.479>have <00:07:41.713>restricted <00:07:41.946>power <00:07:42.179>consumption.

113
00:07:43.460 --> 00:07:47.560
<00:07:43.460>So <00:07:43.587>think <00:07:43.714>of <00:07:43.841>devices <00:07:43.969>like <00:07:44.096>autonomous <00:07:44.223>drones. <00:07:44.350>They're <00:07:44.478>not <00:07:44.605>collected <00:07:44.732>to <00:07:44.859>power,

114
00:07:47.590 --> 00:07:50.640
<00:07:47.590>they're <00:07:47.790>not <00:07:47.990>connected <00:07:48.190>to <00:07:48.390>power <00:07:48.590>outlets <00:07:48.790>24 <00:07:48.990>7.

115
00:07:51.340 --> 00:07:54.800
<00:07:51.340>So <00:07:51.515>when <00:07:51.690>developing <00:07:51.865>applications <00:07:52.040>for <00:07:52.215>AI <00:07:52.390>at <00:07:52.565>the <00:07:52.740>edge,

116
00:07:55.220 --> 00:07:57.640
<00:07:55.220>you <00:07:55.395>need <00:07:55.570>to <00:07:55.745>be <00:07:55.920>mindful <00:07:56.095>of <00:07:56.270>their <00:07:56.445>power <00:07:56.620>consumption.

117
00:07:58.820 --> 00:08:02.680
<00:07:58.820>And <00:07:58.975>when <00:07:59.131>starting <00:07:59.286>developing <00:07:59.442>AI <00:07:59.597>applications <00:07:59.753>for <00:07:59.908>the <00:08:00.064>edge, <00:08:00.219>uh,

118
00:08:02.700 --> 00:08:04.800
<00:08:02.700>you <00:08:02.979>will <00:08:03.260>definitely <00:08:03.539>face <00:08:03.820>many <00:08:04.099>challenges.

119
00:08:05.620 --> 00:08:08.760
<00:08:05.620>And <00:08:05.820>the <00:08:06.020>major <00:08:06.220>challenges <00:08:06.420>are <00:08:06.620>the <00:08:06.820>following. <00:08:07.020>Uh,

120
00:08:08.860 --> 00:08:13.480
<00:08:08.860>so <00:08:09.000>the <00:08:09.140>first <00:08:09.280>one <00:08:09.420>is <00:08:09.560>accuracy. <00:08:09.700>So <00:08:09.840>you <00:08:09.980>need <00:08:10.120>your, <00:08:10.260>uh,

121
00:08:13.700 --> 00:08:18.240
<00:08:13.700>AI <00:08:13.840>networks <00:08:13.979>to <00:08:14.120>be <00:08:14.260>accurate <00:08:14.400>enough <00:08:14.539>to <00:08:14.680>provide, <00:08:14.820>uh, <00:08:14.960>meaningful <00:08:15.099>insights.

122
00:08:18.980 --> 00:08:23.440
<00:08:18.980>So <00:08:19.080>first <00:08:19.180>of <00:08:19.280>all, <00:08:19.380>you <00:08:19.480>need <00:08:19.580>to <00:08:19.680>find <00:08:19.780>a <00:08:19.880>good <00:08:19.980>architecture, <00:08:20.080>which <00:08:20.180>will <00:08:20.280>reflect, <00:08:20.380>uh,

123
00:08:23.460 --> 00:08:27.240
<00:08:23.460>the, <00:08:23.659>uh, <00:08:23.859>um, <00:08:24.060>the, <00:08:24.260>the, <00:08:24.460>the <00:08:24.659>goal, <00:08:24.859>uh,

124
00:08:27.340 --> 00:08:28.520
<00:08:27.340>of <00:08:27.871>your <00:08:28.402>application.

125
00:08:28.980 --> 00:08:33.080
<00:08:28.980>And <00:08:29.120>it <00:08:29.260>has <00:08:29.400>to <00:08:29.540>have <00:08:29.680>a <00:08:29.820>small <00:08:29.960>number <00:08:30.100>of <00:08:30.240>false <00:08:30.380>positive.

126
00:08:33.180 --> 00:08:34.840
<00:08:33.179>And <00:08:33.413>the <00:08:33.646>accuracy <00:08:33.880>needs <00:08:34.113>to <00:08:34.346>be <00:08:34.579>high.

127
00:08:35.460 --> 00:08:39.640
<00:08:35.460>You <00:08:35.547>also <00:08:35.635>need <00:08:35.722>to <00:08:35.810>collect <00:08:35.897>a <00:08:35.985>lot <00:08:36.072>of <00:08:36.160>data <00:08:36.247>in <00:08:36.335>order <00:08:36.422>to <00:08:36.510>make <00:08:36.597>it <00:08:36.685>happen, <00:08:36.772>uh, <00:08:36.860>because,

128
00:08:39.860 --> 00:08:41.960
<00:08:39.860>uh, <00:08:40.034>you <00:08:40.210>know <00:08:40.385>that <00:08:40.560>AI <00:08:40.735>is <00:08:40.909>powered <00:08:41.085>by <00:08:41.260>data.

129
00:08:43.190 --> 00:08:47.680
<00:08:43.190>Once <00:08:43.297>you <00:08:43.405>have <00:08:43.513>solved <00:08:43.620>the <00:08:43.728>accuracy <00:08:43.836>challenge, <00:08:43.943>um, <00:08:44.051>you <00:08:44.159>need <00:08:44.266>to <00:08:44.374>think <00:08:44.482>of, <00:08:44.590>uh,

130
00:08:47.680 --> 00:08:51.600
<00:08:47.680>increasing <00:08:47.880>the <00:08:48.079>throughput <00:08:48.280>of <00:08:48.479>your <00:08:48.680>application. <00:08:48.880>And <00:08:49.079>well,

131
00:08:51.600 --> 00:08:54.600
<00:08:51.600>for <00:08:51.755>some <00:08:51.911>applications <00:08:52.066>like <00:08:52.222>chat <00:08:52.377>G <00:08:52.533>P <00:08:52.688>T <00:08:52.844>for <00:08:53.000>example,

132
00:08:54.780 --> 00:08:58.880
<00:08:54.780>you <00:08:54.887>can <00:08:54.995>allow <00:08:55.103>waiting <00:08:55.210>for <00:08:55.318>a <00:08:55.426>couple <00:08:55.533>of, <00:08:55.641>uh, <00:08:55.749>seconds <00:08:55.856>to <00:08:55.964>get <00:08:56.072>the <00:08:56.180>response.

133
00:08:59.780 --> 00:09:02.400
<00:08:59.780>Uh, <00:09:00.060>AI <00:09:00.339>at <00:09:00.620>the <00:09:00.900>edge <00:09:01.180>requires,

134
00:09:02.450 --> 00:09:05.760
<00:09:02.450>often <00:09:02.730>requires <00:09:03.010>immediate <00:09:03.290>and <00:09:03.570>realtime <00:09:03.850>response.

135
00:09:06.170 --> 00:09:09.200
<00:09:06.170>Think <00:09:06.403>of <00:09:06.636>applications <00:09:06.870>like <00:09:07.103>autonomous <00:09:07.336>driving <00:09:07.569>car,

136
00:09:09.340 --> 00:09:13.560
<00:09:09.340>and <00:09:09.456>a <00:09:09.573>situation <00:09:09.690>when <00:09:09.806>a <00:09:09.923>child, <00:09:10.040>uh, <00:09:10.156>jumps, <00:09:10.273>uh, <00:09:10.390>next <00:09:10.506>to <00:09:10.623>your <00:09:10.740>car,

137
00:09:14.020 --> 00:09:18.720
<00:09:14.020>you <00:09:14.175>need <00:09:14.331>to <00:09:14.486>break <00:09:14.642>immediately, <00:09:14.797>or, <00:09:14.953>uh, <00:09:15.108>a <00:09:15.264>surgery <00:09:15.420>device,

138
00:09:18.940 --> 00:09:21.760
<00:09:18.940>uh, <00:09:19.173>you <00:09:19.406>need <00:09:19.640>to <00:09:19.873>react <00:09:20.106>immediately <00:09:20.340>there.

139
00:09:21.940 --> 00:09:25.600
<00:09:21.940>So <00:09:22.115>real <00:09:22.290>time <00:09:22.465>performance <00:09:22.640>is <00:09:22.815>crucial <00:09:22.990>for <00:09:23.165>edge <00:09:23.340>devices.

140
00:09:26.460 --> 00:09:30.520
<00:09:26.460>And <00:09:26.635>as <00:09:26.810>developers, <00:09:26.985>as <00:09:27.160>developers <00:09:27.335>of <00:09:27.510>edge <00:09:27.685>AI <00:09:27.860>applications,

141
00:09:30.780 --> 00:09:35.760
<00:09:30.780>we <00:09:30.887>really <00:09:30.995>need <00:09:31.103>to <00:09:31.210>focus <00:09:31.318>on <00:09:31.426>making <00:09:31.533>these <00:09:31.641>applications <00:09:31.749>working <00:09:31.856>in <00:09:31.964>real <00:09:32.072>time <00:09:32.180>and

142
00:09:35.760 --> 00:09:37.400
<00:09:35.760>decreasing, <00:09:36.226>uh, <00:09:36.693>the <00:09:37.160>latency.

143
00:09:38.220 --> 00:09:41.400
<00:09:38.220>And <00:09:38.420>the <00:09:38.620>final <00:09:38.820>challenge <00:09:39.020>is <00:09:39.220>deployment <00:09:39.420>at <00:09:39.620>scale.

144
00:09:42.020 --> 00:09:46.360
<00:09:42.020>So <00:09:42.136>think <00:09:42.253>of <00:09:42.370>the <00:09:42.486>situation <00:09:42.603>you've <00:09:42.720>developed <00:09:42.836>one <00:09:42.953>single <00:09:43.069>good <00:09:43.186>application <00:09:43.303>with <00:09:43.420>good

145
00:09:46.640 --> 00:09:47.840
<00:09:46.640>accuracy <00:09:47.000>and <00:09:47.360>good <00:09:47.720>throughput.

146
00:09:48.420 --> 00:09:53.320
<00:09:48.420>Now <00:09:48.520>you <00:09:48.620>need <00:09:48.719>to <00:09:48.819>solve <00:09:48.920>the <00:09:49.020>problem <00:09:49.120>of <00:09:49.219>how <00:09:49.319>do <00:09:49.420>I <00:09:49.520>deploy <00:09:49.620>this <00:09:49.719>application <00:09:49.819>across

147
00:09:53.760 --> 00:09:55.800
<00:09:53.760>multiple <00:09:54.040>geographies <00:09:54.319>all <00:09:54.600>around <00:09:54.880>the <00:09:55.160>world?

148
00:09:56.300 --> 00:10:00.000
<00:09:56.300>And <00:09:56.416>how <00:09:56.533>do <00:09:56.650>I <00:09:56.766>maintain <00:09:56.883>these <00:09:57.000>applications? <00:09:57.116>How <00:09:57.233>do <00:09:57.349>I <00:09:57.466>update <00:09:57.583>my <00:09:57.699>models?

149
00:10:00.740 --> 00:10:05.400
<00:10:00.740>How <00:10:00.833>do <00:10:00.926>I, <00:10:01.020>uh, <00:10:01.113>well <00:10:01.206>take <00:10:01.300>the, <00:10:01.393>the <00:10:01.486>application <00:10:01.580>down <00:10:01.673>and <00:10:01.766>so <00:10:01.860>on. <00:10:01.953>So <00:10:02.046>you <00:10:02.140>need,

150
00:10:05.400 --> 00:10:10.120
<00:10:05.400>you <00:10:05.540>need <00:10:05.680>to <00:10:05.819>take <00:10:05.959>care <00:10:06.100>about <00:10:06.240>these <00:10:06.380>things. <00:10:06.520>And, <00:10:06.660>uh, <00:10:06.800>well,

151
00:10:10.120 --> 00:10:12.000
<00:10:10.120>at <00:10:10.353>Nvidia <00:10:10.586>we <00:10:10.820>have, <00:10:11.053>as <00:10:11.286>I <00:10:11.520>mentioned,

152
00:10:12.120 --> 00:10:14.840
<00:10:12.120>a <00:10:12.295>set <00:10:12.470>of <00:10:12.645>best <00:10:12.820>practices <00:10:12.995>and <00:10:13.170>solutions <00:10:13.345>for <00:10:13.520>you.

153
00:10:15.020 --> 00:10:16.680
<00:10:15.020>And <00:10:15.220>let <00:10:15.420>me <00:10:15.620>share <00:10:15.819>now <00:10:16.020>them <00:10:16.220>with <00:10:16.420>you.

154
00:10:17.260 --> 00:10:20.560
<00:10:17.260>And <00:10:17.460>the <00:10:17.660>first <00:10:17.860>solution <00:10:18.060>is <00:10:18.260>to <00:10:18.460>increase <00:10:18.660>accuracy.

155
00:10:20.980 --> 00:10:25.520
<00:10:20.980>And <00:10:21.067>it's <00:10:21.155>a <00:10:21.242>big <00:10:21.330>trend <00:10:21.417>also <00:10:21.505>in <00:10:21.592>AI <00:10:21.680>in <00:10:21.767>general, <00:10:21.855>not <00:10:21.942>only <00:10:22.030>in <00:10:22.117>ai, <00:10:22.205>the <00:10:22.292>edge, <00:10:22.380>and,

156
00:10:25.940 --> 00:10:28.600
<00:10:25.940>uh, <00:10:26.173>the <00:10:26.406>first, <00:10:26.640>the <00:10:26.873>solution <00:10:27.106>is, <00:10:27.340>uh,

157
00:10:28.650 --> 00:10:31.560
<00:10:28.650>using <00:10:28.850>of <00:10:29.050>synthetic <00:10:29.250>data <00:10:29.449>to <00:10:29.650>train <00:10:29.850>AI <00:10:30.050>models.

158
00:10:32.380 --> 00:10:35.760
<00:10:32.380>So <00:10:32.580>instead <00:10:32.780>of <00:10:32.980>relying <00:10:33.180>purely <00:10:33.380>on <00:10:33.580>real <00:10:33.780>data,

159
00:10:36.460 --> 00:10:41.360
<00:10:36.460>you <00:10:36.567>can <00:10:36.675>assist, <00:10:36.783>uh, <00:10:36.890>your <00:10:36.998>network <00:10:37.106>and <00:10:37.213>you <00:10:37.321>can <00:10:37.429>augment <00:10:37.536>your <00:10:37.644>data <00:10:37.752>sets, <00:10:37.860>uh,

160
00:10:41.360 --> 00:10:42.520
<00:10:41.360>with <00:10:41.708>some <00:10:42.056>synthetic <00:10:42.404>data.

161
00:10:43.760 --> 00:10:48.600
<00:10:43.760>Synthetic <00:10:43.867>data <00:10:43.975>offers <00:10:44.083>you <00:10:44.190>many <00:10:44.298>benefits <00:10:44.406>in <00:10:44.513>the <00:10:44.621>sense <00:10:44.729>like <00:10:44.836>you <00:10:44.944>can <00:10:45.052>add, <00:10:45.160>uh,

162
00:10:48.820 --> 00:10:49.653
<00:10:48.820>Corona <00:10:49.569>cases.

163
00:10:50.270 --> 00:10:54.880
<00:10:50.270>Imagine <00:10:50.377>a <00:10:50.485>situation <00:10:50.593>which <00:10:50.700>rarely <00:10:50.808>happens <00:10:50.916>in <00:10:51.023>real <00:10:51.131>time <00:10:51.239>and <00:10:51.346>is <00:10:51.454>really <00:10:51.562>hard <00:10:51.670>to

164
00:10:54.880 --> 00:10:57.240
<00:10:54.880>correct, <00:10:55.113>collect <00:10:55.346>real <00:10:55.580>data <00:10:55.813>for <00:10:56.046>the <00:10:56.280>situation.

165
00:10:57.740 --> 00:11:02.560
<00:10:57.740>You <00:10:57.840>can <00:10:57.940>easily <00:10:58.040>model <00:10:58.140>it <00:10:58.240>in <00:10:58.340>your <00:10:58.440>sys <00:10:58.540>in <00:10:58.640>your <00:10:58.740>synthetic <00:10:58.840>data <00:10:58.940>systems <00:10:59.040>like <00:10:59.140>using

166
00:11:02.790 --> 00:11:06.120
<00:11:02.790>relying <00:11:02.990>on <00:11:03.189>c <00:11:03.390>G <00:11:03.589>I, <00:11:03.790>for <00:11:03.990>example. <00:11:04.189>Um,

167
00:11:06.920 --> 00:11:11.240
<00:11:06.920>synthetic <00:11:07.060>data <00:11:07.199>also <00:11:07.339>solves <00:11:07.479>the <00:11:07.620>problem <00:11:07.760>of <00:11:07.900>expensive <00:11:08.040>labeling. <00:11:08.180>So, <00:11:08.319>uh,

168
00:11:11.340 --> 00:11:14.440
<00:11:11.340>by <00:11:11.573>using <00:11:11.806>synthetic <00:11:12.040>data <00:11:12.273>and <00:11:12.506>automatic, <00:11:12.740>uh,

169
00:11:14.690 --> 00:11:18.880
<00:11:14.690>algorithms <00:11:14.865>to <00:11:15.040>label <00:11:15.215>your, <00:11:15.390>uh, <00:11:15.565>data <00:11:15.740>objects <00:11:15.915>of <00:11:16.090>interest,

170
00:11:19.500 --> 00:11:23.240
<00:11:19.500>uh, <00:11:19.675>you <00:11:19.850>eliminate <00:11:20.025>the <00:11:20.200>need <00:11:20.375>of <00:11:20.550>hiring <00:11:20.725>many, <00:11:20.900>uh,

171
00:11:23.410 --> 00:11:26.000
<00:11:23.410>human <00:11:23.876>labelers <00:11:24.343>and, <00:11:24.810>uh,

172
00:11:26.280 --> 00:11:30.920
<00:11:26.280>synthetic <00:11:26.420>data <00:11:26.560>offers <00:11:26.699>you <00:11:26.839>also <00:11:26.980>an <00:11:27.120>environment <00:11:27.260>environmental <00:11:27.400>diversity <00:11:27.540>where <00:11:27.680>you

173
00:11:30.920 --> 00:11:34.320
<00:11:30.920>can <00:11:31.153>easily <00:11:31.386>swap <00:11:31.620>textures, <00:11:31.853>objects, <00:11:32.086>and, <00:11:32.319>uh,

174
00:11:35.180 --> 00:11:39.310
<00:11:35.180>you <00:11:35.354>are <00:11:35.530>the <00:11:35.704>creator <00:11:35.880>of <00:11:36.055>your <00:11:36.229>data <00:11:36.405>and, <00:11:36.579>uh,

175
00:11:39.650 --> 00:11:40.830
<00:11:39.650>not <00:11:40.004>talking <00:11:40.358>to, <00:11:40.712>uh,

176
00:11:40.830 --> 00:11:44.870
<00:11:40.830>this <00:11:40.957>throughput <00:11:41.084>and <00:11:41.211>the <00:11:41.339>techniques <00:11:41.466>which <00:11:41.593>help <00:11:41.720>you <00:11:41.848>to <00:11:41.975>increase <00:11:42.102>your <00:11:42.230>throughput.

177
00:11:45.530 --> 00:11:50.190
<00:11:45.530>So <00:11:45.670>one, <00:11:45.810>one <00:11:45.949>of <00:11:46.089>my <00:11:46.230>favorite <00:11:46.370>techniques <00:11:46.510>is <00:11:46.650>the <00:11:46.790>so-called <00:11:46.930>quantization.

178
00:11:50.930 --> 00:11:54.920
<00:11:50.930>And <00:11:51.104>quantization <00:11:51.280>basically <00:11:51.454>means <00:11:51.630>reducing, <00:11:51.805>uh, <00:11:51.979>beats <00:11:52.155>per <00:11:52.329>weight.

179
00:11:55.420 --> 00:11:59.600
<00:11:55.420>So <00:11:55.560>typically <00:11:55.699>your <00:11:55.839>networks <00:11:55.979>are <00:11:56.120>being <00:11:56.260>trained <00:11:56.400>in <00:11:56.540>powerful <00:11:56.680>data <00:11:56.819>centers

180
00:12:00.220 --> 00:12:03.840
<00:12:00.220>in <00:12:00.453>the <00:12:00.686>floating <00:12:00.920>point <00:12:01.153>32. <00:12:01.386>And, <00:12:01.620>uh,

181
00:12:04.160 --> 00:12:09.080
<00:12:04.160>a <00:12:04.276>clever <00:12:04.393>idea <00:12:04.510>was <00:12:04.626>to <00:12:04.743>reduce <00:12:04.860>this, <00:12:04.976>uh, <00:12:05.093>precision <00:12:05.209>to <00:12:05.326>floating <00:12:05.443>point <00:12:05.560>16.

182
00:12:10.100 --> 00:12:11.360
<00:12:10.100>And <00:12:10.326>this <00:12:10.553>is <00:12:10.780>easy <00:12:11.007>to <00:12:11.234>do.

183
00:12:11.500 --> 00:12:15.600
<00:12:11.500>You <00:12:11.616>basically <00:12:11.733>just <00:12:11.850>round <00:12:11.966>up <00:12:12.083>your <00:12:12.200>network <00:12:12.316>weights <00:12:12.433>to <00:12:12.550>flow <00:12:12.666>in <00:12:12.783>point <00:12:12.900>16,

184
00:12:15.620 --> 00:12:20.560
<00:12:15.620>and <00:12:15.736>then <00:12:15.853>it <00:12:15.970>works. <00:12:16.086>But <00:12:16.203>then <00:12:16.320>the, <00:12:16.436>another <00:12:16.553>clever <00:12:16.670>idea <00:12:16.786>was <00:12:16.903>to <00:12:17.020>reduce,

185
00:12:20.740 --> 00:12:23.560
<00:12:20.740>uh, <00:12:20.915>the <00:12:21.090>bits <00:12:21.265>even <00:12:21.440>further <00:12:21.615>to <00:12:21.790>the <00:12:21.965>integer <00:12:22.140>eight.

186
00:12:23.980 --> 00:12:28.400
<00:12:23.980>And <00:12:24.073>you <00:12:24.166>can <00:12:24.260>imagine <00:12:24.353>that <00:12:24.446>we <00:12:24.540>cannot <00:12:24.633>just <00:12:24.726>simply <00:12:24.820>round <00:12:24.913>it <00:12:25.006>up <00:12:25.100>because <00:12:25.193>it'll <00:12:25.286>lose <00:12:25.380>a

187
00:12:28.400 --> 00:12:32.000
<00:12:28.400>lot <00:12:28.555>of <00:12:28.711>accuracy. <00:12:28.866>So <00:12:29.022>the <00:12:29.177>clear <00:12:29.333>way <00:12:29.488>to, <00:12:29.644>uh, <00:12:29.800>the,

188
00:12:32.000 --> 00:12:36.760
<00:12:32.000>the <00:12:32.127>clever <00:12:32.254>way <00:12:32.381>to <00:12:32.509>address <00:12:32.636>it <00:12:32.763>was <00:12:32.890>to <00:12:33.018>provide <00:12:33.145>a <00:12:33.272>mapping <00:12:33.400>from

189
00:12:37.180 --> 00:12:41.040
<00:12:37.180>the, <00:12:37.354>uh, <00:12:37.530>the <00:12:37.704>interval, <00:12:37.880>uh, <00:12:38.055>in <00:12:38.229>floating <00:12:38.405>point <00:12:38.579>32,

190
00:12:41.190 --> 00:12:45.240
<00:12:41.190>minimum <00:12:41.330>to <00:12:41.470>maximum <00:12:41.610>to <00:12:41.750>the <00:12:41.890>integer <00:12:42.030>interval, <00:12:42.170>which <00:12:42.310>is <00:12:42.450>between, <00:12:42.590>uh,

191
00:12:45.240 --> 00:12:48.640
<00:12:45.240>minus <00:12:45.590>127 <00:12:45.940>to <00:12:46.290>plus <00:12:46.640>127.

192
00:12:49.220 --> 00:12:52.800
<00:12:49.220>And <00:12:49.336>there <00:12:49.453>are <00:12:49.570>basically <00:12:49.686>two <00:12:49.803>ways <00:12:49.920>in <00:12:50.036>doing <00:12:50.153>that. <00:12:50.270>The <00:12:50.386>first <00:12:50.503>way, <00:12:50.620>uh,

193
00:12:52.860 --> 00:12:55.160
<00:12:52.860>is <00:12:53.210>the <00:12:53.560>so-called <00:12:53.910>post-training <00:12:54.260>quantization,

194
00:12:55.450 --> 00:13:00.280
<00:12:55.450>where <00:12:55.550>you <00:12:55.650>train <00:12:55.750>your <00:12:55.850>model <00:12:55.950>just <00:12:56.050>in <00:12:56.150>a <00:12:56.250>normal <00:12:56.350>way, <00:12:56.450>uh, <00:12:56.550>in <00:12:56.650>floating <00:12:56.750>point <00:12:56.850>32.

195
00:13:00.700 --> 00:13:05.160
<00:13:00.700>And <00:13:00.816>then <00:13:00.933>you <00:13:01.050>use <00:13:01.166>a <00:13:01.283>separate <00:13:01.400>data <00:13:01.516>set <00:13:01.633>as <00:13:01.750>a <00:13:01.866>calibration <00:13:01.983>data <00:13:02.100>set

196
00:13:05.580 --> 00:13:09.480
<00:13:05.580>to <00:13:05.720>find <00:13:05.860>this <00:13:06.000>minimum <00:13:06.140>and <00:13:06.280>maximum, <00:13:06.420>uh, <00:13:06.560>values <00:13:06.700>in <00:13:06.840>your <00:13:06.980>network.

197
00:13:10.260 --> 00:13:13.280
<00:13:10.260>And <00:13:10.415>then <00:13:10.571>you <00:13:10.726>derive, <00:13:10.882>and <00:13:11.037>then <00:13:11.193>you <00:13:11.348>interpolate <00:13:11.504>it, <00:13:11.660>uh,

198
00:13:13.280 --> 00:13:14.840
<00:13:13.280>to <00:13:13.560>the <00:13:13.839>mapping <00:13:14.120>to <00:13:14.400>integer <00:13:14.680>eight.

199
00:13:15.430 --> 00:13:19.720
<00:13:15.430>Another <00:13:15.546>way <00:13:15.663>to <00:13:15.780>do <00:13:15.896>it <00:13:16.013>is <00:13:16.130>by <00:13:16.246>running <00:13:16.363>the <00:13:16.479>so-called <00:13:16.596>quantization <00:13:16.713>aware <00:13:16.829>training.

200
00:13:19.980 --> 00:13:22.560
<00:13:19.980>And <00:13:20.180>some <00:13:20.380>of <00:13:20.580>the <00:13:20.780>frameworks <00:13:20.980>allowed <00:13:21.180>that, <00:13:21.380>uh,

201
00:13:22.560 --> 00:13:25.760
<00:13:22.560>essentially <00:13:22.760>in <00:13:22.959>quantization <00:13:23.160>aware <00:13:23.359>training, <00:13:23.560>what <00:13:23.760>you <00:13:23.959>do,

202
00:13:25.820 --> 00:13:28.200
<00:13:25.820>you <00:13:26.053>have <00:13:26.286>an <00:13:26.520>extra <00:13:26.753>loss <00:13:26.986>function, <00:13:27.220>uh,

203
00:13:28.210 --> 00:13:32.760
<00:13:28.210>which <00:13:28.350>you <00:13:28.490>use <00:13:28.630>then <00:13:28.770>to <00:13:28.910>estimate <00:13:29.050>this, <00:13:29.190>to <00:13:29.330>learn <00:13:29.470>this, <00:13:29.610>uh,

204
00:13:32.840 --> 00:13:37.760
<00:13:32.840>mappings <00:13:32.967>from <00:13:33.094>the <00:13:33.221>Flo <00:13:33.349>point <00:13:33.476>32 <00:13:33.603>to <00:13:33.730>integer <00:13:33.858>eight. <00:13:33.985>As <00:13:34.112>a <00:13:34.240>consequence,

205
00:13:37.820 --> 00:13:41.720
<00:13:37.820>you <00:13:37.995>get <00:13:38.170>a <00:13:38.345>more <00:13:38.520>accurate <00:13:38.695>mapping <00:13:38.870>and, <00:13:39.045>uh, <00:13:39.220>uh,

206
00:13:41.790 --> 00:13:46.080
<00:13:41.790>your <00:13:41.930>model <00:13:42.069>is <00:13:42.209>easily <00:13:42.349>convertible <00:13:42.490>in <00:13:42.630>floating <00:13:42.770>point <00:13:42.910>in <00:13:43.050>J <00:13:43.189>eight.

207
00:13:47.100 --> 00:13:50.960
<00:13:47.100>Uh, <00:13:47.255>the <00:13:47.411>next <00:13:47.566>powerful <00:13:47.722>technique <00:13:47.877>is <00:13:48.033>the <00:13:48.188>so-called <00:13:48.344>network <00:13:48.500>pruning.

208
00:13:51.300 --> 00:13:56.000
<00:13:51.300>In <00:13:51.439>network <00:13:51.579>pruning, <00:13:51.719>you <00:13:51.859>basically <00:13:52.000>remove <00:13:52.140>some <00:13:52.280>neurons <00:13:52.420>from <00:13:52.560>the <00:13:52.699>network,

209
00:13:56.500 --> 00:13:59.560
<00:13:56.500>and <00:13:56.640>as <00:13:56.780>a <00:13:56.920>consequence, <00:13:57.060>you <00:13:57.200>also <00:13:57.340>remove <00:13:57.480>some <00:13:57.620>connections <00:13:57.760>between <00:13:57.900>them,

210
00:13:59.850 --> 00:14:03.920
<00:13:59.850>which <00:13:59.990>makes <00:14:00.130>the <00:14:00.270>network <00:14:00.410>small <00:14:00.550>in <00:14:00.690>size <00:14:00.830>and <00:14:00.970>faster <00:14:01.110>in <00:14:01.250>execution.

211
00:14:04.660 --> 00:14:06.800
<00:14:04.660>How <00:14:04.893>do <00:14:05.126>you <00:14:05.360>remove <00:14:05.593>these <00:14:05.826>networks? <00:14:06.060>Basically,

212
00:14:07.060 --> 00:14:11.880
<00:14:07.060>you <00:14:07.187>run <00:14:07.314>your <00:14:07.441>network <00:14:07.569>and <00:14:07.696>then <00:14:07.823>you <00:14:07.950>observe <00:14:08.078>which <00:14:08.205>n <00:14:08.332>which <00:14:08.459>neurons

213
00:14:11.880 --> 00:14:15.640
<00:14:11.880>in <00:14:11.996>the <00:14:12.113>network <00:14:12.230>do <00:14:12.346>not <00:14:12.463>participate <00:14:12.580>in <00:14:12.696>the <00:14:12.813>final <00:14:12.930>outcome <00:14:13.046>of <00:14:13.163>the <00:14:13.280>model.

214
00:14:16.340 --> 00:14:19.560
<00:14:16.340>And <00:14:16.495>then <00:14:16.651>you <00:14:16.806>compare <00:14:16.962>this <00:14:17.117>value <00:14:17.273>to <00:14:17.428>some <00:14:17.584>certain <00:14:17.740>threshold,

215
00:14:20.020 --> 00:14:24.560
<00:14:20.020>and <00:14:20.127>then <00:14:20.235>you <00:14:20.343>can <00:14:20.450>eliminate <00:14:20.558>them <00:14:20.666>from <00:14:20.773>the <00:14:20.881>network. <00:14:20.989>Uh, <00:14:21.096>it's <00:14:21.204>a <00:14:21.312>loss <00:14:21.420>process,

216
00:14:24.590 --> 00:14:29.280
<00:14:24.590>unfortunately, <00:14:24.765>and <00:14:24.940>you <00:14:25.115>will, <00:14:25.290>uh, <00:14:25.465>lose <00:14:25.640>some <00:14:25.815>accuracy. <00:14:25.990>However,

217
00:14:29.500 --> 00:14:34.120
<00:14:29.500>it <00:14:29.616>is <00:14:29.733>easy <00:14:29.850>to <00:14:29.966>restore <00:14:30.083>by <00:14:30.200>running <00:14:30.316>a <00:14:30.433>training <00:14:30.550>for <00:14:30.666>a <00:14:30.783>couple <00:14:30.900>of

218
00:14:34.210 --> 00:14:35.160
<00:14:34.210>extra <00:14:35.064>iterations.

219
00:14:35.620 --> 00:14:39.960
<00:14:35.620>So <00:14:35.727>it's <00:14:35.835>another <00:14:35.943>powerful <00:14:36.050>technique <00:14:36.158>which <00:14:36.266>can <00:14:36.373>help <00:14:36.481>to <00:14:36.589>shrink <00:14:36.696>your <00:14:36.804>network <00:14:36.912>in <00:14:37.020>size

220
00:14:40.780 --> 00:14:43.080
<00:14:40.780>and, <00:14:40.980>uh, <00:14:41.180>to <00:14:41.380>make <00:14:41.579>it <00:14:41.780>faster <00:14:41.980>in <00:14:42.180>execution.

221
00:14:44.150 --> 00:14:48.720
<00:14:44.150>Another <00:14:44.305>technique <00:14:44.461>is <00:14:44.616>the <00:14:44.772>so-called <00:14:44.927>network <00:14:45.083>graph <00:14:45.238>optimizations. <00:14:45.394>And, <00:14:45.550>uh,

222
00:14:48.980 --> 00:14:49.813
<00:14:48.980>in <00:14:49.354>this <00:14:49.729>approach,

223
00:14:50.060 --> 00:14:54.840
<00:14:50.060>you <00:14:50.187>parse <00:14:50.314>your <00:14:50.441>network <00:14:50.569>graph <00:14:50.696>and <00:14:50.823>try <00:14:50.950>to <00:14:51.078>fuse <00:14:51.205>some <00:14:51.332>of <00:14:51.459>its

224
00:14:54.840 --> 00:14:59.560
<00:14:54.840>layers, <00:14:55.040>uh, <00:14:55.240>horizontally <00:14:55.440>but <00:14:55.640>also <00:14:55.840>vertically. <00:14:56.040>And, <00:14:56.240>uh,

225
00:15:00.580 --> 00:15:03.960
<00:15:00.580>the <00:15:00.720>reason <00:15:00.860>why <00:15:01.000>it <00:15:01.140>works <00:15:01.280>is <00:15:01.420>basically <00:15:01.560>all <00:15:01.700>these <00:15:01.840>kernels, <00:15:01.980>uh,

226
00:15:04.070 --> 00:15:08.720
<00:15:04.070>they <00:15:04.210>rely <00:15:04.350>on <00:15:04.490>protocols, <00:15:04.630>uh, <00:15:04.770>to <00:15:04.910>your <00:15:05.050>G <00:15:05.190>P <00:15:05.330>U <00:15:05.470>and,

227
00:15:08.980 --> 00:15:12.560
<00:15:08.980>uh, <00:15:09.087>these <00:15:09.195>kernels <00:15:09.303>are <00:15:09.410>super <00:15:09.518>fast <00:15:09.626>in <00:15:09.733>execution. <00:15:09.841>And <00:15:09.949>the <00:15:10.056>G <00:15:10.164>P <00:15:10.272>U, <00:15:10.380>however,

228
00:15:13.180 --> 00:15:16.680
<00:15:13.180>the <00:15:13.354>transitions <00:15:13.530>between <00:15:13.704>these <00:15:13.880>kernels <00:15:14.055>may <00:15:14.229>slowly <00:15:14.405>process <00:15:14.579>down.

229
00:15:17.260 --> 00:15:18.800
<00:15:17.260>So <00:15:17.491>in <00:15:17.722>order <00:15:17.953>to <00:15:18.184>make <00:15:18.415>it <00:15:18.646>faster,

230
00:15:19.260 --> 00:15:23.400
<00:15:19.260>you <00:15:19.387>try <00:15:19.514>to <00:15:19.641>combine <00:15:19.769>as <00:15:19.896>much <00:15:20.023>as <00:15:20.150>possible <00:15:20.278>without <00:15:20.405>altering <00:15:20.532>the <00:15:20.660>mathematical

231
00:15:23.400 --> 00:15:28.080
<00:15:23.400>computations <00:15:23.527>in <00:15:23.654>the <00:15:23.781>background. <00:15:23.909>Uh, <00:15:24.036>by <00:15:24.163>doing <00:15:24.290>that, <00:15:24.418>you <00:15:24.545>can <00:15:24.672>achieve, <00:15:24.800>uh,

232
00:15:28.300 --> 00:15:30.280
<00:15:28.300>the <00:15:28.579>really <00:15:28.859>highest <00:15:29.140>throughput <00:15:29.420>as <00:15:29.699>well.

233
00:15:31.100 --> 00:15:35.280
<00:15:31.100>The <00:15:31.240>next <00:15:31.380>technique <00:15:31.520>is <00:15:31.660>the <00:15:31.800>so-called <00:15:31.940>kernel <00:15:32.080>tit <00:15:32.220>tuning. <00:15:32.360>And, <00:15:32.500>uh,

234
00:15:35.410 --> 00:15:39.000
<00:15:35.410>those <00:15:35.565>of <00:15:35.721>you <00:15:35.876>who <00:15:36.032>actively <00:15:36.187>work <00:15:36.343>on, <00:15:36.498>uh, <00:15:36.654>developing, <00:15:36.810>uh,

235
00:15:39.020 --> 00:15:40.600
<00:15:39.020>neural <00:15:39.370>networks <00:15:39.720>and <00:15:40.069>deploying <00:15:40.420>them,

236
00:15:40.660 --> 00:15:43.760
<00:15:40.660>you <00:15:40.834>know <00:15:41.010>that <00:15:41.185>we <00:15:41.360>have <00:15:41.535>the <00:15:41.709>multitude <00:15:41.885>of <00:15:42.060>configurations.

237
00:15:43.940 --> 00:15:47.800
<00:15:43.940>We <00:15:44.080>can <00:15:44.220>have <00:15:44.360>different <00:15:44.500>batches, <00:15:44.640>we <00:15:44.780>can <00:15:44.920>have <00:15:45.060>different <00:15:45.200>tenor <00:15:45.340>layouts,

238
00:15:48.500 --> 00:15:51.240
<00:15:48.500>and <00:15:48.675>you <00:15:48.850>also <00:15:49.025>deploy <00:15:49.200>your <00:15:49.375>networks <00:15:49.550>on <00:15:49.725>different <00:15:49.900>devices.

239
00:15:51.860 --> 00:15:56.640
<00:15:51.860>We <00:15:52.015>have <00:15:52.171>devices <00:15:52.326>like <00:15:52.482>Embedded <00:15:52.637>Jetsons <00:15:52.793>or <00:15:52.948>X <00:15:53.104>86 <00:15:53.260>architectures.

240
00:15:57.070 --> 00:16:01.520
<00:15:57.070>They <00:15:57.197>are <00:15:57.324>all <00:15:57.451>different, <00:15:57.579>and <00:15:57.706>our <00:15:57.833>networks <00:15:57.960>perform <00:15:58.088>differently <00:15:58.215>on <00:15:58.342>these <00:15:58.470>devices.

241
00:16:02.340 --> 00:16:06.560
<00:16:02.340>You <00:16:02.447>may <00:16:02.555>also <00:16:02.663>know <00:16:02.770>that <00:16:02.878>there <00:16:02.986>are <00:16:03.093>multiple <00:16:03.201>ways <00:16:03.309>to <00:16:03.416>implement <00:16:03.524>a <00:16:03.632>single <00:16:03.740>operation

242
00:16:06.560 --> 00:16:08.360
<00:16:06.560>like <00:16:07.026>convolution, <00:16:07.493>for <00:16:07.959>example,

243
00:16:08.780 --> 00:16:12.520
<00:16:08.780>and <00:16:08.935>also <00:16:09.091>different <00:16:09.246>implementations <00:16:09.402>can <00:16:09.557>work <00:16:09.713>differently <00:16:09.868>on <00:16:10.024>different <00:16:10.180>devices.

244
00:16:13.260 --> 00:16:18.200
<00:16:13.260>So <00:16:13.376>the <00:16:13.493>idea <00:16:13.610>here <00:16:13.726>is <00:16:13.843>just <00:16:13.960>to <00:16:14.076>brute <00:16:14.193>force <00:16:14.310>your <00:16:14.426>current <00:16:14.543>setup <00:16:14.660>on

245
00:16:18.220 --> 00:16:22.880
<00:16:18.220>the <00:16:18.360>target <00:16:18.500>device <00:16:18.640>and <00:16:18.780>see <00:16:18.920>which <00:16:19.060>combination <00:16:19.200>of <00:16:19.340>the <00:16:19.480>algorithms <00:16:19.620>and

246
00:16:23.440 --> 00:16:27.280
<00:16:23.440>configurations <00:16:23.595>gives <00:16:23.751>you <00:16:23.906>the <00:16:24.062>best <00:16:24.217>performance <00:16:24.373>on <00:16:24.528>the <00:16:24.684>given <00:16:24.840>device.

247
00:16:27.820 --> 00:16:29.680
<00:16:27.820>So <00:16:28.053>this <00:16:28.286>is <00:16:28.520>called <00:16:28.753>kernel <00:16:28.986>out <00:16:29.220>tuning.

248
00:16:30.510 --> 00:16:34.960
<00:16:30.510>Another <00:16:30.665>technique <00:16:30.821>is <00:16:30.976>the <00:16:31.132>so-called <00:16:31.287>dynamic <00:16:31.443>tenson <00:16:31.598>memory <00:16:31.754>upon <00:16:31.910>inference.

249
00:16:35.700 --> 00:16:36.390
<00:16:35.700>And, <00:16:36.321>uh,

250
00:16:36.390 --> 00:16:40.760
<00:16:36.390>this <00:16:36.545>technique <00:16:36.701>has <00:16:36.856>been <00:16:37.012>inspired <00:16:37.167>by <00:16:37.323>register <00:16:37.478>allocation <00:16:37.634>in <00:16:37.790>compiling.

251
00:16:41.500 --> 00:16:46.480
<00:16:41.500>So <00:16:41.675>here <00:16:41.850>we <00:16:42.025>combine <00:16:42.200>tensors <00:16:42.375>in <00:16:42.550>our <00:16:42.725>network <00:16:42.900>into

252
00:16:46.480 --> 00:16:50.560
<00:16:46.480>some <00:16:46.655>regions, <00:16:46.830>and <00:16:47.005>these <00:16:47.180>regions <00:16:47.355>have <00:16:47.530>disjoint <00:16:47.705>execution <00:16:47.880>time.

253
00:16:51.300 --> 00:16:55.480
<00:16:51.300>And, <00:16:51.439>uh, <00:16:51.579>we <00:16:51.719>then <00:16:51.859>assign <00:16:52.000>these <00:16:52.140>regions <00:16:52.280>to <00:16:52.420>blocks. <00:16:52.560>And, <00:16:52.699>uh,

254
00:16:56.190 --> 00:16:58.920
<00:16:56.190>well <00:16:56.423>basically <00:16:56.656>also <00:16:56.890>regions. <00:16:57.123>Uh, <00:16:57.356>the, <00:16:57.590>the,

255
00:16:59.190 --> 00:17:02.920
<00:16:59.190>this <00:16:59.365>regions <00:16:59.540>assigned <00:16:59.715>to <00:16:59.890>blocks <00:17:00.065>also <00:17:00.240>have <00:17:00.415>disjoint <00:17:00.590>lifetimes.

256
00:17:03.340 --> 00:17:07.040
<00:17:03.340>It <00:17:03.480>also <00:17:03.620>helps <00:17:03.760>you <00:17:03.900>to <00:17:04.040>speed <00:17:04.180>up <00:17:04.319>your <00:17:04.460>training <00:17:04.600>and <00:17:04.740>increase,

257
00:17:07.560 --> 00:17:08.800
<00:17:07.560>decrease <00:17:07.932>the <00:17:08.303>memory <00:17:08.675>footprint.

258
00:17:10.220 --> 00:17:14.680
<00:17:10.220>And <00:17:10.375>another <00:17:10.531>technique <00:17:10.686>is <00:17:10.842>the <00:17:10.997>so-called <00:17:11.153>multistream <00:17:11.308>concurrent <00:17:11.464>execution. <00:17:11.620>Uh,

259
00:17:14.780 --> 00:17:19.560
<00:17:14.780>in <00:17:14.907>kuda <00:17:15.034>we <00:17:15.161>have <00:17:15.289>the, <00:17:15.416>uh, <00:17:15.543>the <00:17:15.670>so-called <00:17:15.798>paradigm <00:17:15.925>called <00:17:16.052>kuda <00:17:16.180>streams.

260
00:17:20.380 --> 00:17:25.360
<00:17:20.380>And <00:17:20.496>basically <00:17:20.613>it <00:17:20.730>means <00:17:20.846>that <00:17:20.963>we <00:17:21.080>can <00:17:21.196>run <00:17:21.313>multiple <00:17:21.430>processes <00:17:21.546>at <00:17:21.663>the <00:17:21.780>same

261
00:17:25.360 --> 00:17:30.040
<00:17:25.359>time <00:17:25.487>on <00:17:25.614>the <00:17:25.741>same <00:17:25.869>physical <00:17:25.996>device <00:17:26.123>in <00:17:26.250>parallel <00:17:26.378>without <00:17:26.505>interfering <00:17:26.632>with <00:17:26.760>each

262
00:17:30.040 --> 00:17:32.600
<00:17:30.040>other. <00:17:30.320>And <00:17:30.600>when <00:17:30.879>executing <00:17:31.159>our <00:17:31.440>network,

263
00:17:32.780 --> 00:17:37.040
<00:17:32.780>we <00:17:32.907>can <00:17:33.034>actually <00:17:33.161>power <00:17:33.289>paralyze <00:17:33.416>some <00:17:33.543>processes, <00:17:33.670>uh, <00:17:33.798>inside <00:17:33.925>of <00:17:34.052>the <00:17:34.180>network.

264
00:17:37.980 --> 00:17:42.040
<00:17:37.980>So <00:17:38.135>for <00:17:38.291>example, <00:17:38.446>we <00:17:38.602>can <00:17:38.757>paralyze <00:17:38.913>host <00:17:39.068>device <00:17:39.224>memory <00:17:39.380>copy.

265
00:17:42.390 --> 00:17:47.200
<00:17:42.390>This <00:17:42.472>is <00:17:42.554>a <00:17:42.637>special <00:17:42.719>terminology <00:17:42.801>which <00:17:42.884>we <00:17:42.966>use <00:17:43.048>for <00:17:43.131>copying <00:17:43.213>between <00:17:43.295>the <00:17:43.378>C <00:17:43.460>P <00:17:43.542>U <00:17:43.625>and <00:17:43.707>G <00:17:43.790>P

266
00:17:47.240 --> 00:17:49.480
<00:17:47.240>U, <00:17:47.706>uh, <00:17:48.173>retrieving, <00:17:48.640>uh,

267
00:17:49.500 --> 00:17:54.440
<00:17:49.500>device <00:17:49.587>to <00:17:49.675>host <00:17:49.762>from <00:17:49.850>the <00:17:49.937>G <00:17:50.025>P <00:17:50.112>U <00:17:50.200>to <00:17:50.287>the <00:17:50.375>C <00:17:50.462>P <00:17:50.550>U <00:17:50.637>and <00:17:50.725>the <00:17:50.812>kernel <00:17:50.900>execution.

268
00:17:54.540 --> 00:17:58.840
<00:17:54.540>And <00:17:54.656>by <00:17:54.773>kernel <00:17:54.889>we <00:17:55.006>mean <00:17:55.123>we <00:17:55.240>understand <00:17:55.356>the <00:17:55.473>actual <00:17:55.590>computations <00:17:55.706>running <00:17:55.823>in <00:17:55.940>your

269
00:17:58.840 --> 00:18:03.320
<00:17:58.840>network. <00:17:58.956>So <00:17:59.073>by <00:17:59.189>doing <00:17:59.306>that, <00:17:59.423>we <00:17:59.540>can <00:17:59.656>also <00:17:59.773>shrink <00:17:59.889>our <00:18:00.006>network <00:18:00.123>in, <00:18:00.240>uh,

270
00:18:03.500 --> 00:18:07.960
<00:18:03.500>the <00:18:03.675>execution <00:18:03.850>time. <00:18:04.025>So <00:18:04.200>all <00:18:04.375>of <00:18:04.550>these <00:18:04.725>techniques, <00:18:04.900>uh,

271
00:18:08.630 --> 00:18:11.840
<00:18:08.630>yeah, <00:18:08.785>all <00:18:08.941>of <00:18:09.096>these <00:18:09.252>techniques <00:18:09.407>are <00:18:09.563>very <00:18:09.718>powerful <00:18:09.874>and, <00:18:10.030>uh,

272
00:18:11.910 --> 00:18:16.560
<00:18:11.910>they <00:18:12.065>can <00:18:12.221>lead <00:18:12.376>to <00:18:12.532>your <00:18:12.687>performance <00:18:12.843>improvement <00:18:12.998>a <00:18:13.154>lot. <00:18:13.310>And,

273
00:18:16.700 --> 00:18:20.000
<00:18:16.700>uh, <00:18:16.827>to <00:18:16.954>make <00:18:17.081>your <00:18:17.209>model <00:18:17.336>small <00:18:17.463>in <00:18:17.590>size <00:18:17.718>and <00:18:17.845>faster <00:18:17.972>in <00:18:18.100>execution.

274
00:18:20.820 --> 00:18:21.920
<00:18:20.820>And <00:18:21.315>with <00:18:21.810>Nvidia,

275
00:18:21.980 --> 00:18:26.880
<00:18:21.980>we <00:18:22.107>actually <00:18:22.234>aim <00:18:22.361>at <00:18:22.489>supporting <00:18:22.616>you <00:18:22.743>through <00:18:22.870>the <00:18:22.998>whole <00:18:23.125>AI <00:18:23.252>productization <00:18:23.380>cycle.

276
00:18:27.620 --> 00:18:32.440
<00:18:27.620>So <00:18:27.775>from <00:18:27.931>data <00:18:28.086>collection <00:18:28.242>to <00:18:28.397>training <00:18:28.553>of <00:18:28.708>your <00:18:28.864>networks <00:18:29.020>to

277
00:18:32.440 --> 00:18:36.160
<00:18:32.440>network <00:18:32.673>optimization, <00:18:32.906>and <00:18:33.140>finally <00:18:33.373>to <00:18:33.606>network <00:18:33.840>deployment.

278
00:18:36.700 --> 00:18:40.480
<00:18:36.700>And <00:18:36.800>many <00:18:36.900>people <00:18:37.000>know <00:18:37.100>us <00:18:37.200>as <00:18:37.300>a <00:18:37.400>G <00:18:37.500>P <00:18:37.600>U <00:18:37.700>company <00:18:37.800>or <00:18:37.900>a <00:18:38.000>gaming <00:18:38.100>company,

279
00:18:40.940 --> 00:18:42.800
<00:18:40.940>but <00:18:41.173>we <00:18:41.406>are <00:18:41.640>actually <00:18:41.873>a <00:18:42.106>platform <00:18:42.340>company.

280
00:18:43.100 --> 00:18:45.920
<00:18:43.100>So <00:18:43.274>we've <00:18:43.449>been <00:18:43.625>developing <00:18:43.800>cuda <00:18:43.975>for <00:18:44.149>many <00:18:44.324>years <00:18:44.500>already,

281
00:18:46.540 --> 00:18:50.880
<00:18:46.540>and <00:18:46.667>we <00:18:46.794>actually <00:18:46.921>have <00:18:47.049>more <00:18:47.176>software <00:18:47.303>engineers <00:18:47.430>than <00:18:47.558>hardware <00:18:47.685>engineers <00:18:47.812>in <00:18:47.940>our

282
00:18:50.880 --> 00:18:53.320
<00:18:50.880>company. <00:18:51.230>Uh, <00:18:51.580>meaning <00:18:51.930>that <00:18:52.280>our,

283
00:18:53.860 --> 00:18:58.520
<00:18:53.860>our <00:18:53.987>focus <00:18:54.114>on <00:18:54.241>software <00:18:54.369>is <00:18:54.496>large <00:18:54.623>and <00:18:54.750>we <00:18:54.878>also <00:18:55.005>develop <00:18:55.132>many <00:18:55.260>software

284
00:18:58.520 --> 00:19:03.320
<00:18:58.520>products <00:18:58.660>which <00:18:58.800>have <00:18:58.940>free <00:18:59.080>for <00:18:59.220>commercial <00:18:59.360>use <00:18:59.500>and <00:18:59.639>which <00:18:59.780>can <00:18:59.920>assist

285
00:19:03.340 --> 00:19:07.720
<00:19:03.340>you <00:19:03.514>in, <00:19:03.689>uh, <00:19:03.865>developing <00:19:04.040>and <00:19:04.215>deploying <00:19:04.389>your <00:19:04.564>networks, <00:19:04.740>uh,

286
00:19:07.950 --> 00:19:12.640
<00:19:07.950>also <00:19:08.057>on <00:19:08.165>embedded <00:19:08.273>in <00:19:08.380>edge <00:19:08.488>devices. <00:19:08.596>So <00:19:08.703>let <00:19:08.811>me <00:19:08.919>introduce <00:19:09.026>a <00:19:09.134>couple <00:19:09.242>of, <00:19:09.350>uh,

287
00:19:12.700 --> 00:19:14.520
<00:19:12.700>our <00:19:13.166>software, <00:19:13.633>software <00:19:14.100>products,

288
00:19:14.520 --> 00:19:18.920
<00:19:14.520>which <00:19:14.636>you <00:19:14.753>can <00:19:14.870>use <00:19:14.986>for <00:19:15.103>free <00:19:15.220>in <00:19:15.336>order <00:19:15.453>to <00:19:15.570>make <00:19:15.686>your <00:19:15.803>development <00:19:15.920>and

289
00:19:19.050 --> 00:19:24.040
<00:19:19.050>deployment <00:19:19.190>more <00:19:19.330>efficient. <00:19:19.470>And <00:19:19.610>the <00:19:19.750>first <00:19:19.889>such <00:19:20.030>product, <00:19:20.169>uh, <00:19:20.310>is, <00:19:20.450>uh,

290
00:19:24.140 --> 00:19:25.880
<00:19:24.140>the <00:19:24.840>Nvidia <00:19:25.540>replicator,

291
00:19:26.010 --> 00:19:29.320
<00:19:26.010>which <00:19:26.210>is <00:19:26.410>an <00:19:26.610>extension <00:19:26.810>of <00:19:27.010>our <00:19:27.210>omniverse <00:19:27.410>platform.

292
00:19:29.470 --> 00:19:34.320
<00:19:29.470>Omniverse <00:19:29.597>platform <00:19:29.724>is <00:19:29.851>a <00:19:29.979>platform <00:19:30.106>for <00:19:30.233>creation <00:19:30.360>of <00:19:30.488>metaverse <00:19:30.615>and <00:19:30.742>virtual <00:19:30.870>words,

293
00:19:35.140 --> 00:19:37.840
<00:19:35.140>and <00:19:35.280>you <00:19:35.420>can <00:19:35.560>also <00:19:35.700>program <00:19:35.840>some <00:19:35.980>extensions <00:19:36.120>on <00:19:36.260>top <00:19:36.400>of <00:19:36.540>it.

294
00:19:38.500 --> 00:19:42.160
<00:19:38.500>And <00:19:38.675>Nvidia <00:19:38.850>replicator <00:19:39.025>is <00:19:39.200>used <00:19:39.375>to <00:19:39.550>create <00:19:39.725>synthetic <00:19:39.900>data.

295
00:19:42.580 --> 00:19:46.920
<00:19:42.580>So <00:19:42.707>underneath <00:19:42.834>it <00:19:42.961>runs <00:19:43.089>a <00:19:43.216>set <00:19:43.343>of <00:19:43.470>the <00:19:43.598>main <00:19:43.725>randomization <00:19:43.852>techniques, <00:19:43.980>uh,

296
00:19:46.970 --> 00:19:51.720
<00:19:46.970>which <00:19:47.086>using <00:19:47.203>which <00:19:47.320>you <00:19:47.436>can <00:19:47.553>then <00:19:47.670>create <00:19:47.786>a <00:19:47.903>huge <00:19:48.020>variety <00:19:48.136>of <00:19:48.253>different <00:19:48.370>data

297
00:19:51.830 --> 00:19:54.680
<00:19:51.830>sets. <00:19:52.110>Uh, <00:19:52.389>it <00:19:52.669>also <00:19:52.949>has, <00:19:53.230>uh,

298
00:19:54.820 --> 00:19:59.800
<00:19:54.820>an <00:19:54.960>extension <00:19:55.100>for <00:19:55.240>robotic <00:19:55.379>specifically, <00:19:55.520>which <00:19:55.659>is <00:19:55.800>called <00:19:55.939>omniverse <00:19:56.080>Isaac <00:19:56.220>Sim.

299
00:20:00.460 --> 00:20:05.160
<00:20:00.460>And <00:20:00.553>the <00:20:00.646>QR <00:20:00.740>code <00:20:00.833>I'm <00:20:00.926>sharing <00:20:01.020>with <00:20:01.113>you <00:20:01.206>is <00:20:01.300>basically <00:20:01.393>a <00:20:01.486>free <00:20:01.580>d <00:20:01.673>l <00:20:01.766>i <00:20:01.860>training,

300
00:20:05.480 --> 00:20:08.120
<00:20:05.480>d <00:20:05.655>l <00:20:05.830>i <00:20:06.005>is <00:20:06.180>our <00:20:06.355>deep <00:20:06.530>learning <00:20:06.705>institute, <00:20:06.880>uh,

301
00:20:08.330 --> 00:20:12.480
<00:20:08.330>which <00:20:08.446>you <00:20:08.563>can <00:20:08.679>use <00:20:08.796>and <00:20:08.913>you <00:20:09.030>can <00:20:09.146>learn <00:20:09.263>how <00:20:09.379>to <00:20:09.496>use <00:20:09.613>this <00:20:09.730>software.

302
00:20:15.060 --> 00:20:19.920
<00:20:15.060>Um, <00:20:15.235>another, <00:20:15.409>well, <00:20:15.585>another <00:20:15.760>contribution <00:20:15.935>of <00:20:16.110>Nvidia <00:20:16.284>in, <00:20:16.460>uh,

303
00:20:20.020 --> 00:20:22.680
<00:20:20.020>the <00:20:20.253>space <00:20:20.486>are <00:20:20.720>our <00:20:20.953>Nvidia <00:20:21.186>pre-trained <00:20:21.420>models.

304
00:20:23.180 --> 00:20:27.800
<00:20:23.180>So <00:20:23.335>we <00:20:23.491>collect <00:20:23.646>and <00:20:23.802>create <00:20:23.957>huge <00:20:24.113>data <00:20:24.268>sets <00:20:24.424>for <00:20:24.580>many

305
00:20:28.120 --> 00:20:32.360
<00:20:28.120>specific <00:20:28.247>tasks. <00:20:28.374>And <00:20:28.501>then <00:20:28.629>we <00:20:28.756>train <00:20:28.883>our <00:20:29.010>models <00:20:29.138>on <00:20:29.265>these <00:20:29.392>data <00:20:29.520>sets,

306
00:20:32.810 --> 00:20:37.400
<00:20:32.810>which <00:20:32.950>you <00:20:33.090>can <00:20:33.230>use <00:20:33.370>them <00:20:33.510>for <00:20:33.649>production, <00:20:33.790>also <00:20:33.929>for <00:20:34.070>commercial <00:20:34.210>use.

307
00:20:37.860 --> 00:20:40.240
<00:20:37.860>And <00:20:38.060>they <00:20:38.260>basically <00:20:38.459>have <00:20:38.659>two <00:20:38.860>categories <00:20:39.060>of <00:20:39.260>models.

308
00:20:40.500 --> 00:20:45.000
<00:20:40.500>The <00:20:40.655>first <00:20:40.811>one <00:20:40.966>are <00:20:41.122>the <00:20:41.277>so-called <00:20:41.433>pre-trained, <00:20:41.588>uh, <00:20:41.744>task, <00:20:41.900>uh,

309
00:20:45.000 --> 00:20:49.040
<00:20:45.000>task-based <00:20:45.200>models, <00:20:45.400>which <00:20:45.600>include <00:20:45.800>model <00:20:46.000>for <00:20:46.200>people <00:20:46.400>detection,

310
00:20:49.150 --> 00:20:53.920
<00:20:49.150>license <00:20:49.383>plate <00:20:49.616>recognition, <00:20:49.850>gesture <00:20:50.083>classification, <00:20:50.316>and <00:20:50.550>many,

311
00:20:53.920 --> 00:20:54.800
<00:20:53.920>many <00:20:54.316>other <00:20:54.712>models.

312
00:20:55.290 --> 00:21:00.200
<00:20:55.290>These <00:20:55.406>models <00:20:55.523>you <00:20:55.639>can <00:20:55.756>download <00:20:55.873>from <00:20:55.990>our <00:20:56.106>Nvidia <00:20:56.223>N <00:20:56.340>G <00:20:56.456>C <00:20:56.573>catalog <00:20:56.690>and

313
00:21:00.200 --> 00:21:04.080
<00:21:00.200>you <00:21:00.307>can <00:21:00.415>integrate <00:21:00.523>them <00:21:00.630>in <00:21:00.738>your <00:21:00.846>application <00:21:00.953>and <00:21:01.061>use <00:21:01.169>just <00:21:01.276>out <00:21:01.384>of <00:21:01.492>the <00:21:01.600>box.

314
00:21:05.300 --> 00:21:06.720
<00:21:05.300>Um, <00:21:05.619>in <00:21:05.939>addition <00:21:06.258>to <00:21:06.578>that,

315
00:21:06.900 --> 00:21:11.160
<00:21:06.900>we <00:21:07.075>also <00:21:07.250>have <00:21:07.425>some <00:21:07.600>generic <00:21:07.775>applications <00:21:07.950>for <00:21:08.125>image <00:21:08.300>classification,

316
00:21:11.250 --> 00:21:13.200
<00:21:11.250>image <00:21:11.716>detection <00:21:12.183>for <00:21:12.650>segmentation.

317
00:21:13.660 --> 00:21:18.640
<00:21:13.660>We <00:21:13.800>also <00:21:13.940>have <00:21:14.080>some <00:21:14.220>transformer <00:21:14.360>architectures <00:21:14.500>there, <00:21:14.640>like <00:21:14.780>vision <00:21:14.920>Transformer, <00:21:15.060>uh,

318
00:21:18.810 --> 00:21:22.240
<00:21:18.810>deformable <00:21:19.090>data <00:21:19.370>for <00:21:19.649>detection. <00:21:19.929>So, <00:21:20.210>uh,

319
00:21:22.350 --> 00:21:26.200
<00:21:22.350>it's <00:21:22.477>a <00:21:22.604>powerful <00:21:22.731>set <00:21:22.859>of <00:21:22.986>models <00:21:23.113>and <00:21:23.240>you <00:21:23.368>can <00:21:23.495>use <00:21:23.622>them <00:21:23.750>rely,

320
00:21:26.500 --> 00:21:31.120
<00:21:26.500>you <00:21:26.616>can <00:21:26.733>use <00:21:26.850>them <00:21:26.966>using <00:21:27.083>transfer <00:21:27.200>learning <00:21:27.316>in <00:21:27.433>order <00:21:27.550>to <00:21:27.666>learn <00:21:27.783>some <00:21:27.900>new

321
00:21:31.120 --> 00:21:35.640
<00:21:31.120>capabilities. <00:21:31.294>And <00:21:31.469>in <00:21:31.645>order <00:21:31.820>to <00:21:31.995>use <00:21:32.169>transfer <00:21:32.344>learning, <00:21:32.520>uh,

322
00:21:35.740 --> 00:21:39.360
<00:21:35.740>we <00:21:35.915>also <00:21:36.090>offer <00:21:36.265>our <00:21:36.440>software <00:21:36.615>called <00:21:36.790>NVIDIA <00:21:36.965>Tower <00:21:37.140>Toolkit.

323
00:21:40.380 --> 00:21:45.120
<00:21:40.380>Nvidia <00:21:40.507>Tower <00:21:40.634>Toolkit <00:21:40.761>is <00:21:40.889>a <00:21:41.016>zero <00:21:41.143>coding <00:21:41.270>framework <00:21:41.398>to <00:21:41.525>train <00:21:41.652>artificial <00:21:41.780>neural

324
00:21:45.120 --> 00:21:49.200
<00:21:45.120>networks. <00:21:45.353>It <00:21:45.586>includes <00:21:45.820>well <00:21:46.053>scripts <00:21:46.286>for <00:21:46.520>training,

325
00:21:49.820 --> 00:21:54.080
<00:21:49.820>but <00:21:49.960>it <00:21:50.100>also <00:21:50.240>includes <00:21:50.379>such <00:21:50.520>optimizations <00:21:50.659>like <00:21:50.800>network <00:21:50.939>pruning <00:21:51.080>and <00:21:51.220>quantization

326
00:21:54.080 --> 00:21:54.913
<00:21:54.080>aware <00:21:54.829>training.

327
00:21:55.500 --> 00:21:59.800
<00:21:55.500>And <00:21:55.627>it <00:21:55.754>can <00:21:55.881>of <00:21:56.009>course <00:21:56.136>run <00:21:56.263>on <00:21:56.390>any <00:21:56.518>NVIDIA <00:21:56.645>cloud <00:21:56.772>devices <00:21:56.900>and

328
00:21:59.800 --> 00:22:03.400
<00:21:59.800>workstations <00:22:00.033>stations. <00:22:00.266>And <00:22:00.500>it <00:22:00.733>also <00:22:00.966>includes, <00:22:01.200>uh,

329
00:22:03.400 --> 00:22:07.520
<00:22:03.400>data <00:22:03.575>augmentation <00:22:03.750>in <00:22:03.925>order <00:22:04.100>to <00:22:04.275>further <00:22:04.450>extend <00:22:04.625>your <00:22:04.800>dataset.

330
00:22:08.660 --> 00:22:09.380
Uh,

331
00:22:09.380 --> 00:22:14.080
<00:22:09.380>one <00:22:09.507>also <00:22:09.634>important <00:22:09.761>feature <00:22:09.889>of <00:22:10.016>AU <00:22:10.143>toolkit <00:22:10.270>is <00:22:10.398>that <00:22:10.525>it <00:22:10.652>supports <00:22:10.780>multi

332
00:22:14.280 --> 00:22:18.560
<00:22:14.280>G <00:22:14.387>P <00:22:14.495>U <00:22:14.603>and <00:22:14.710>multi-node <00:22:14.818>training. <00:22:14.926>You <00:22:15.033>know <00:22:15.141>that <00:22:15.249>modern <00:22:15.356>trainings, <00:22:15.464>they <00:22:15.572>can <00:22:15.680>be,

333
00:22:19.140 --> 00:22:19.460
uh,

334
00:22:19.460 --> 00:22:24.160
<00:22:19.460>really <00:22:19.567>heavy <00:22:19.675>in <00:22:19.783>compute <00:22:19.890>and <00:22:19.998>sometimes <00:22:20.106>require <00:22:20.213>more <00:22:20.321>than <00:22:20.429>a <00:22:20.536>single <00:22:20.644>G <00:22:20.752>P <00:22:20.860>U.

335
00:22:24.580 --> 00:22:27.560
<00:22:24.580>And <00:22:24.720>those <00:22:24.860>of <00:22:25.000>you <00:22:25.139>who <00:22:25.280>have <00:22:25.419>trained <00:22:25.560>distributed <00:22:25.699>models, <00:22:25.840>you <00:22:25.980>know,

336
00:22:27.660 --> 00:22:32.160
<00:22:27.660>that's <00:22:27.815>usually <00:22:27.971>a <00:22:28.126>hustle <00:22:28.282>to <00:22:28.437>distribute <00:22:28.593>your <00:22:28.748>training <00:22:28.904>across, <00:22:29.060>uh,

337
00:22:32.720 --> 00:22:34.880
<00:22:32.720>multiple <00:22:32.920>gpu. <00:22:33.120>So <00:22:33.320>in <00:22:33.520>total, <00:22:33.720>it's <00:22:33.920>very <00:22:34.120>simple,

338
00:22:34.910 --> 00:22:37.640
<00:22:34.910>it's <00:22:35.143>just <00:22:35.376>passing <00:22:35.610>a <00:22:35.843>single <00:22:36.076>parameter. <00:22:36.310>Uh,

339
00:22:37.700 --> 00:22:40.840
<00:22:37.700>it <00:22:37.900>also <00:22:38.100>offers <00:22:38.300>the <00:22:38.500>so-called <00:22:38.700>automatic <00:22:38.900>mixed <00:22:39.100>precision,

340
00:22:40.840 --> 00:22:45.040
<00:22:40.840>which <00:22:40.967>can <00:22:41.094>further <00:22:41.221>increase, <00:22:41.349>uh, <00:22:41.476>which <00:22:41.603>can <00:22:41.730>further <00:22:41.858>optimize <00:22:41.985>your <00:22:42.112>training <00:22:42.240>time.

341
00:22:45.740 --> 00:22:48.280
<00:22:45.740>And <00:22:45.895>uh, <00:22:46.051>it <00:22:46.206>can <00:22:46.362>also <00:22:46.517>allow <00:22:46.673>you <00:22:46.828>to <00:22:46.984>use, <00:22:47.140>uh,

342
00:22:48.390 --> 00:22:51.920
<00:22:48.390>batches <00:22:48.565>big <00:22:48.740>in <00:22:48.915>size <00:22:49.090>and <00:22:49.265>as <00:22:49.440>I <00:22:49.615>already <00:22:49.790>mentioned,

343
00:22:51.920 --> 00:22:56.640
<00:22:51.920>it <00:22:52.075>includes <00:22:52.231>pruning <00:22:52.386>and <00:22:52.542>quantization. <00:22:52.697>And, <00:22:52.853>uh, <00:22:53.008>in <00:22:53.164>the <00:22:53.320>nutshell,

344
00:22:57.020 --> 00:23:01.800
<00:22:57.020>the <00:22:57.160>process <00:22:57.300>of <00:22:57.440>development, <00:22:57.580>uh, <00:22:57.720>with <00:22:57.860>our <00:22:58.000>technologist <00:22:58.139>looks <00:22:58.280>like <00:22:58.420>follows.

345
00:23:01.800 --> 00:23:05.720
<00:23:01.800>So <00:23:01.907>you <00:23:02.015>go <00:23:02.123>to <00:23:02.230>the <00:23:02.338>N <00:23:02.446>G <00:23:02.553>C, <00:23:02.661>you <00:23:02.769>download <00:23:02.876>the <00:23:02.984>pre-trained <00:23:03.092>models, <00:23:03.200>um,

346
00:23:06.020 --> 00:23:09.960
<00:23:06.020>you <00:23:06.160>can <00:23:06.300>use <00:23:06.440>synthetic <00:23:06.580>data <00:23:06.720>created <00:23:06.860>in <00:23:07.000>our <00:23:07.139>omniverse, <00:23:07.280>and <00:23:07.420>uh,

347
00:23:10.100 --> 00:23:12.840
<00:23:10.100>you <00:23:10.274>train <00:23:10.449>it <00:23:10.625>then <00:23:10.800>in <00:23:10.975>the <00:23:11.149>NVIDIA <00:23:11.324>Tower <00:23:11.500>toolkit.

348
00:23:13.740 --> 00:23:17.520
<00:23:13.740>And <00:23:13.880>then <00:23:14.020>we <00:23:14.160>have <00:23:14.300>a <00:23:14.440>set <00:23:14.580>of, <00:23:14.720>uh, <00:23:14.860>deployment <00:23:15.000>tools, <00:23:15.140>uh,

349
00:23:17.620 --> 00:23:21.600
<00:23:17.620>or <00:23:17.747>our <00:23:17.874>deployment <00:23:18.001>as <00:23:18.129>the <00:23:18.256>case <00:23:18.383>which <00:23:18.510>you <00:23:18.638>can <00:23:18.765>use <00:23:18.892>for <00:23:19.020>inference.

350
00:23:23.020 --> 00:23:23.853
<00:23:23.020>And, <00:23:23.769>uh,

351
00:23:24.460 --> 00:23:29.280
<00:23:24.460>one <00:23:24.587>such <00:23:24.714>powerful <00:23:24.841>tool <00:23:24.969>which <00:23:25.096>help <00:23:25.223>you <00:23:25.350>in <00:23:25.478>your <00:23:25.605>inference <00:23:25.732>is, <00:23:25.860>uh,

352
00:23:29.300 --> 00:23:30.200
<00:23:29.300>our <00:23:29.705>Nvidia <00:23:30.110>10.

353
00:23:30.980 --> 00:23:35.640
<00:23:30.980>And <00:23:31.087>this <00:23:31.195>is <00:23:31.303>really <00:23:31.410>my <00:23:31.518>favorite <00:23:31.626>piece <00:23:31.733>of <00:23:31.841>software <00:23:31.949>because <00:23:32.056>it <00:23:32.164>includes <00:23:32.272>all <00:23:32.380>the

354
00:23:35.640 --> 00:23:40.640
<00:23:35.640>optimization <00:23:35.780>techniques, <00:23:35.920>which <00:23:36.060>I <00:23:36.200>have <00:23:36.340>mentioned <00:23:36.480>before. <00:23:36.620>So <00:23:36.760>it <00:23:36.900>has, <00:23:37.040>uh,

355
00:23:40.640 --> 00:23:44.360
<00:23:40.640>layer <00:23:40.920>10 <00:23:41.200>fusion <00:23:41.480>quantization <00:23:41.760>post-training, <00:23:42.040>quantization,

356
00:23:44.710 --> 00:23:48.000
<00:23:44.710>dynamic <00:23:44.990>Tanza <00:23:45.270>memory <00:23:45.550>and <00:23:45.830>Multistream <00:23:46.110>execution.

357
00:23:48.630 --> 00:23:52.840
<00:23:48.630>It's <00:23:48.830>an <00:23:49.030>optimizer <00:23:49.230>optimizing <00:23:49.430>compiler <00:23:49.630>and <00:23:49.830>a <00:23:50.030>runtime.

358
00:23:53.140 --> 00:23:55.320
<00:23:53.140>So <00:23:53.340>once <00:23:53.540>you <00:23:53.740>have <00:23:53.940>optimized <00:23:54.140>it <00:23:54.340>with <00:23:54.540>tanza,

359
00:23:55.420 --> 00:23:58.840
<00:23:55.420>and <00:23:55.547>it's <00:23:55.674>basically <00:23:55.801>as <00:23:55.929>simple <00:23:56.056>as <00:23:56.183>pressing <00:23:56.310>a <00:23:56.438>button <00:23:56.565>saying <00:23:56.692>like, <00:23:56.820>Hey,

360
00:23:59.240 --> 00:24:03.600
<00:23:59.240>optimize <00:23:59.380>me <00:23:59.520>this <00:23:59.660>network. <00:23:59.800>And <00:23:59.940>uh, <00:24:00.080>once <00:24:00.220>you <00:24:00.360>have <00:24:00.500>optimized <00:24:00.640>it,

361
00:24:03.660 --> 00:24:08.480
<00:24:03.660>you <00:24:03.787>can <00:24:03.914>use <00:24:04.041>tenzer <00:24:04.169>to <00:24:04.296>runtime <00:24:04.423>to <00:24:04.550>integrate <00:24:04.678>your <00:24:04.805>optimized <00:24:04.932>architecture <00:24:05.060>into

362
00:24:08.480 --> 00:24:12.440
<00:24:08.480>your <00:24:08.620>c <00:24:08.760>plus <00:24:08.900>plus <00:24:09.040>or <00:24:09.180>Python <00:24:09.320>application. <00:24:09.460>So <00:24:09.600>it's <00:24:09.740>very <00:24:09.880>powerful.

363
00:24:13.540 --> 00:24:17.720
<00:24:13.540>Uh, <00:24:13.639>another <00:24:13.740>great <00:24:13.840>piece <00:24:13.940>of <00:24:14.040>software, <00:24:14.139>which <00:24:14.240>is <00:24:14.340>by <00:24:14.440>the <00:24:14.540>way, <00:24:14.639>open <00:24:14.740>source <00:24:14.840>and <00:24:14.940>uh,

364
00:24:17.720 --> 00:24:22.280
<00:24:17.720>we <00:24:17.860>hosted <00:24:18.000>on <00:24:18.140>GitHub, <00:24:18.280>uh, <00:24:18.420>it's <00:24:18.560>called <00:24:18.700>Nvidia <00:24:18.840>Trian <00:24:18.980>Inference <00:24:19.120>Server.

365
00:24:23.020 --> 00:24:27.960
<00:24:23.020>And <00:24:23.175>this <00:24:23.331>is <00:24:23.486>our <00:24:23.642>versatile <00:24:23.797>and <00:24:23.953>flexible <00:24:24.108>serving <00:24:24.264>solution <00:24:24.420>for

366
00:24:28.220 --> 00:24:29.800
<00:24:28.220>AI <00:24:28.920>models. <00:24:29.620>Uh,

367
00:24:29.860 --> 00:24:34.800
<00:24:29.860>you <00:24:29.967>can <00:24:30.075>create <00:24:30.183>your <00:24:30.290>models <00:24:30.398>in <00:24:30.506>no <00:24:30.613>matter <00:24:30.721>which <00:24:30.829>framework. <00:24:30.936>Is <00:24:31.044>it <00:24:31.152>a <00:24:31.260>PyTorch,

368
00:24:34.800 --> 00:24:37.720
<00:24:34.800>is <00:24:34.955>it <00:24:35.111>tensile <00:24:35.266>flow, <00:24:35.422>is <00:24:35.577>it <00:24:35.733>10 <00:24:35.888>30 <00:24:36.044>optimized <00:24:36.200>model?

369
00:24:38.180 --> 00:24:42.600
<00:24:38.180>You <00:24:38.307>can <00:24:38.434>even <00:24:38.561>implement <00:24:38.689>your <00:24:38.816>own <00:24:38.943>custom <00:24:39.070>frameworks <00:24:39.198>like <00:24:39.325>for <00:24:39.452>tasks <00:24:39.580>like

370
00:24:42.620 --> 00:24:44.680
<00:24:42.620>pre-processing <00:24:43.320>and <00:24:44.020>post-processing.

371
00:24:45.020 --> 00:24:49.440
<00:24:45.020>So <00:24:45.127>you <00:24:45.235>can <00:24:45.343>run <00:24:45.450>this <00:24:45.558>all <00:24:45.666>in <00:24:45.773>the <00:24:45.881>backend <00:24:45.989>in <00:24:46.096>the <00:24:46.204>optimized <00:24:46.312>way. <00:24:46.420>Um,

372
00:24:49.500 --> 00:24:53.840
<00:24:49.500>the <00:24:49.607>cool <00:24:49.715>feature <00:24:49.823>of <00:24:49.930>it <00:24:50.038>is, <00:24:50.146>uh, <00:24:50.253>dynamic <00:24:50.361>tons <00:24:50.469>of <00:24:50.576>memory, <00:24:50.684>uh, <00:24:50.792>dynamic, <00:24:50.900>uh,

373
00:24:53.840 --> 00:24:54.350
batching,

374
00:24:54.350 --> 00:24:59.280
<00:24:54.350>meaning <00:24:54.490>that <00:24:54.629>once <00:24:54.770>you <00:24:54.909>get <00:24:55.050>multiple <00:24:55.189>client <00:24:55.330>requests <00:24:55.469>from <00:24:55.610>different <00:24:55.750>sources,

375
00:24:59.860 --> 00:25:04.600
<00:24:59.860>you <00:25:00.000>can <00:25:00.139>actually <00:25:00.280>batch <00:25:00.419>them <00:25:00.560>together <00:25:00.699>for <00:25:00.840>faster <00:25:00.979>execution <00:25:01.120>for <00:25:01.260>to

376
00:25:04.760 --> 00:25:08.480
<00:25:04.760>increase <00:25:04.900>the <00:25:05.040>throughput <00:25:05.180>of <00:25:05.320>your <00:25:05.460>application. <00:25:05.600>And <00:25:05.740>another <00:25:05.879>cool <00:25:06.020>thing, <00:25:06.160>uh,

377
00:25:08.580 --> 00:25:12.480
<00:25:08.580>is <00:25:08.735>also <00:25:08.891>concurrent <00:25:09.046>model <00:25:09.202>execution. <00:25:09.357>So <00:25:09.513>relying <00:25:09.668>on <00:25:09.824>kudos <00:25:09.980>streams,

378
00:25:12.780 --> 00:25:17.760
<00:25:12.780>you <00:25:12.887>can <00:25:12.995>host <00:25:13.103>several <00:25:13.210>models <00:25:13.318>at <00:25:13.426>the <00:25:13.533>same <00:25:13.641>time <00:25:13.749>on <00:25:13.856>the <00:25:13.964>same <00:25:14.072>physical <00:25:14.180>device.

379
00:25:19.340 --> 00:25:21.520
<00:25:19.340>Uh, <00:25:19.689>it <00:25:20.040>also <00:25:20.389>supports <00:25:20.740>ensembles.

380
00:25:21.580 --> 00:25:25.880
<00:25:21.580>So <00:25:21.687>if <00:25:21.795>you <00:25:21.903>have <00:25:22.010>the <00:25:22.118>pre <00:25:22.226>pre-processing <00:25:22.333>and <00:25:22.441>post-processing <00:25:22.549>and <00:25:22.656>then <00:25:22.764>a <00:25:22.872>couple <00:25:22.980>of

381
00:25:25.880 --> 00:25:27.000
<00:25:25.880>models <00:25:26.384>chained <00:25:26.888>together,

382
00:25:27.540 --> 00:25:32.440
<00:25:27.540>it <00:25:27.656>all <00:25:27.773>can <00:25:27.889>work <00:25:28.006>seamlessly <00:25:28.123>in <00:25:28.240>the <00:25:28.356>so-called <00:25:28.473>zero <00:25:28.590>memory, <00:25:28.706>uh, <00:25:28.823>copy <00:25:28.940>fashion.

383
00:25:33.220 --> 00:25:35.880
<00:25:33.220>So <00:25:33.420>any, <00:25:33.620>it <00:25:33.820>even <00:25:34.020>can <00:25:34.220>run <00:25:34.420>on <00:25:34.620>CPUs.

384
00:25:36.180 --> 00:25:39.440
<00:25:36.180>So <00:25:36.320>when <00:25:36.460>we <00:25:36.600>say <00:25:36.740>it's <00:25:36.880>very <00:25:37.020>flexible, <00:25:37.160>we <00:25:37.300>really <00:25:37.440>mean <00:25:37.580>that.

385
00:25:40.540 --> 00:25:41.600
<00:25:40.540>The <00:25:41.016>next <00:25:41.494>technology,

386
00:25:41.850 --> 00:25:46.640
<00:25:41.850>which <00:25:42.005>is <00:25:42.161>specifically <00:25:42.316>developed <00:25:42.472>for <00:25:42.627>video <00:25:42.783>analytic <00:25:42.938>applications <00:25:43.094>is <00:25:43.250>our

387
00:25:46.780 --> 00:25:49.000
<00:25:46.780>Nvidia <00:25:46.980>deep <00:25:47.180>stream. <00:25:47.379>S <00:25:47.580>d <00:25:47.780>k <00:25:47.980>deep <00:25:48.180>stream.

388
00:25:49.200 --> 00:25:52.440
<00:25:49.200>S <00:25:49.355>D <00:25:49.511>K <00:25:49.666>is <00:25:49.822>a <00:25:49.977>software <00:25:50.133>based <00:25:50.288>on <00:25:50.444>GStreamer <00:25:50.600>framework.

389
00:25:52.730 --> 00:25:57.400
<00:25:52.730>Those <00:25:52.837>of <00:25:52.945>you <00:25:53.053>who <00:25:53.160>work <00:25:53.268>with <00:25:53.376>uh, <00:25:53.483>video, <00:25:53.591>you <00:25:53.699>may <00:25:53.806>have <00:25:53.914>tried <00:25:54.022>GStreamer <00:25:54.130>before,

390
00:25:58.020 --> 00:26:02.920
<00:25:58.020>and <00:25:58.147>we <00:25:58.274>basically <00:25:58.401>just <00:25:58.529>enhanced <00:25:58.656>GStreamer <00:25:58.783>with <00:25:58.910>G <00:25:59.038>P <00:25:59.165>U <00:25:59.292>accelerated <00:25:59.420>plugins.

391
00:26:03.900 --> 00:26:04.460
<00:26:03.900>And, <00:26:04.404>uh,

392
00:26:04.460 --> 00:26:08.320
<00:26:04.460>the <00:26:04.576>great <00:26:04.693>thing <00:26:04.810>about <00:26:04.926>Deep <00:26:05.043>Stream <00:26:05.160>that <00:26:05.276>it <00:26:05.393>relies <00:26:05.510>on <00:26:05.626>the <00:26:05.743>so-called <00:26:05.860>pipeline

393
00:26:08.410 --> 00:26:10.680
<00:26:08.410>efficiency <00:26:08.760>with <00:26:09.110>zero <00:26:09.460>memory <00:26:09.810>copies.

394
00:26:11.150 --> 00:26:15.200
<00:26:11.150>Once <00:26:11.243>you <00:26:11.336>data <00:26:11.430>is <00:26:11.523>bivian <00:26:11.616>moved <00:26:11.710>from <00:26:11.803>the <00:26:11.896>C <00:26:11.990>P <00:26:12.083>U <00:26:12.176>to <00:26:12.270>the <00:26:12.363>G <00:26:12.456>P <00:26:12.550>U,

395
00:26:15.740 --> 00:26:20.720
<00:26:15.740>it <00:26:15.840>stays <00:26:15.940>there <00:26:16.040>until <00:26:16.140>the <00:26:16.240>end <00:26:16.340>of <00:26:16.440>the <00:26:16.540>pipeline <00:26:16.640>being <00:26:16.740>passed <00:26:16.840>as <00:26:16.940>pointers <00:26:17.040>within <00:26:17.140>the

396
00:26:20.840 --> 00:26:23.880
<00:26:20.840>G <00:26:21.014>P <00:26:21.189>U <00:26:21.365>that <00:26:21.540>makes <00:26:21.715>your <00:26:21.889>applications <00:26:22.064>very <00:26:22.240>efficient.

397
00:26:24.620 --> 00:26:27.840
<00:26:24.620>And <00:26:24.736>here's <00:26:24.853>a <00:26:24.969>small <00:26:25.086>demo, <00:26:25.203>which <00:26:25.320>I've <00:26:25.436>developed <00:26:25.553>a <00:26:25.669>couple <00:26:25.786>of <00:26:25.903>years <00:26:26.020>ago.

398
00:26:28.180 --> 00:26:32.880
<00:26:28.180>So <00:26:28.355>using <00:26:28.530>all <00:26:28.705>these <00:26:28.880>technologies, <00:26:29.055>I've <00:26:29.230>basically <00:26:29.405>downloaded, <00:26:29.580>uh,

399
00:26:33.060 --> 00:26:37.160
<00:26:33.060>the <00:26:33.167>model <00:26:33.275>trained <00:26:33.383>for <00:26:33.490>to <00:26:33.598>detect <00:26:33.706>people <00:26:33.813>and <00:26:33.921>I've <00:26:34.029>trained <00:26:34.136>it <00:26:34.244>on <00:26:34.352>some <00:26:34.460>public

400
00:26:37.310 --> 00:26:38.920
<00:26:37.310>dataset <00:26:37.776>for <00:26:38.243>hand <00:26:38.710>detection.

401
00:26:39.420 --> 00:26:43.800
<00:26:39.420>And <00:26:39.527>then <00:26:39.635>I <00:26:39.743>changed <00:26:39.850>this <00:26:39.958>model <00:26:40.066>for <00:26:40.173>a <00:26:40.281>gesture <00:26:40.389>classification <00:26:40.496>model <00:26:40.604>and <00:26:40.712>I <00:26:40.820>deployed

402
00:26:43.800 --> 00:26:47.320
<00:26:43.800>it <00:26:43.916>in <00:26:44.033>the <00:26:44.149>deep <00:26:44.266>stream, <00:26:44.383>uh, <00:26:44.500>using <00:26:44.616>deep <00:26:44.733>stream <00:26:44.850>as <00:26:44.966>d <00:26:45.083>k. <00:26:45.200>Uh,

403
00:26:48.010 --> 00:26:51.520
<00:26:48.010>we've <00:26:48.126>hosted <00:26:48.243>it <00:26:48.360>on <00:26:48.476>the <00:26:48.593>GitHub, <00:26:48.710>so <00:26:48.826>it <00:26:48.943>works <00:26:49.060>quite <00:26:49.176>well. <00:26:49.293>And, <00:26:49.410>uh,

404
00:26:51.520 --> 00:26:54.280
<00:26:51.520>important <00:26:51.675>to <00:26:51.831>say <00:26:51.986>that <00:26:52.142>this, <00:26:52.297>this <00:26:52.453>is <00:26:52.608>a <00:26:52.764>prerecorded <00:26:52.920>video,

405
00:26:54.620 --> 00:26:58.200
<00:26:54.620>but <00:26:54.747>it <00:26:54.874>runs <00:26:55.001>real <00:26:55.129>time <00:26:55.256>on <00:26:55.383>my <00:26:55.510>Jetson <00:26:55.638>a <00:26:55.765>G <00:26:55.892>X <00:26:56.020>device,

406
00:26:58.200 --> 00:27:02.160
<00:26:58.200>which <00:26:58.355>is <00:26:58.511>like <00:26:58.666>a <00:26:58.822>small <00:26:58.977>embedded <00:26:59.133>device <00:26:59.288>to <00:26:59.444>models, <00:26:59.600>um,

407
00:27:02.700 --> 00:27:04.840
<00:27:02.700>but <00:27:02.900>works <00:27:03.100>really <00:27:03.300>smooth <00:27:03.500>and <00:27:03.700>in <00:27:03.900>real <00:27:04.100>time.

408
00:27:05.700 --> 00:27:08.800
<00:27:05.700>That's <00:27:05.840>another <00:27:05.980>kind <00:27:06.120>of <00:27:06.260>example <00:27:06.400>you <00:27:06.540>can <00:27:06.680>do <00:27:06.820>is <00:27:06.960>deep <00:27:07.100>stream.

409
00:27:08.900 --> 00:27:12.840
<00:27:08.900>So <00:27:09.007>you <00:27:09.115>see <00:27:09.223>like <00:27:09.330>there <00:27:09.438>are <00:27:09.546>IDs <00:27:09.653>and <00:27:09.761>you <00:27:09.869>can <00:27:09.976>track <00:27:10.084>people <00:27:10.192>there. <00:27:10.300>Um,

410
00:27:13.220 --> 00:27:17.320
<00:27:13.220>you <00:27:13.327>can <00:27:13.435>chain <00:27:13.543>multiple <00:27:13.650>models <00:27:13.758>as <00:27:13.866>I <00:27:13.973>mentioned. <00:27:14.081>So <00:27:14.189>this <00:27:14.296>model <00:27:14.404>detects, <00:27:14.512>uh, <00:27:14.620>uh,

411
00:27:17.560 --> 00:27:21.600
<00:27:17.560>vehicles <00:27:17.676>and <00:27:17.793>road <00:27:17.909>signs <00:27:18.026>and <00:27:18.143>if <00:27:18.260>it <00:27:18.376>detects <00:27:18.493>a <00:27:18.610>vehicle, <00:27:18.726>it <00:27:18.843>can <00:27:18.960>say,

412
00:27:21.600 --> 00:27:25.880
<00:27:21.600>is <00:27:21.716>it <00:27:21.833>a <00:27:21.949>van <00:27:22.066>or <00:27:22.183>a <00:27:22.300>sedan <00:27:22.416>being <00:27:22.533>changed <00:27:22.649>with <00:27:22.766>a <00:27:22.883>second <00:27:23.000>classifier?

413
00:27:27.020 --> 00:27:28.680
<00:27:27.020>And <00:27:27.370>to <00:27:27.720>sum <00:27:28.070>up, <00:27:28.420>uh,

414
00:27:29.050 --> 00:27:32.840
<00:27:29.050>there <00:27:29.166>are <00:27:29.283>many <00:27:29.399>ways <00:27:29.516>in <00:27:29.633>order <00:27:29.750>to <00:27:29.866>improve <00:27:29.983>the <00:27:30.100>performance <00:27:30.216>of <00:27:30.333>your <00:27:30.450>model,

415
00:27:32.930 --> 00:27:37.800
<00:27:32.930>which <00:27:33.070>is <00:27:33.210>specifically <00:27:33.350>useful <00:27:33.490>for <00:27:33.630>deploying <00:27:33.770>AI <00:27:33.910>applications <00:27:34.050>at <00:27:34.190>the <00:27:34.330>edge.

416
00:27:38.540 --> 00:27:42.640
<00:27:38.540>And <00:27:38.647>you <00:27:38.755>need <00:27:38.863>to <00:27:38.970>take <00:27:39.078>care <00:27:39.186>of <00:27:39.293>the <00:27:39.401>following <00:27:39.509>things. <00:27:39.616>So <00:27:39.724>first <00:27:39.832>of <00:27:39.940>all,

417
00:27:42.640 --> 00:27:44.280
<00:27:42.640>in <00:27:42.920>order <00:27:43.200>to <00:27:43.480>increase <00:27:43.760>your <00:27:44.040>accuracy,

418
00:27:44.340 --> 00:27:47.240
<00:27:44.340>you <00:27:44.540>can <00:27:44.740>rely <00:27:44.939>on <00:27:45.139>simulation <00:27:45.340>and <00:27:45.540>synthetic <00:27:45.740>data.

419
00:27:47.260 --> 00:27:52.040
<00:27:47.260>And <00:27:47.387>for <00:27:47.514>that <00:27:47.641>we <00:27:47.769>have <00:27:47.896>tools <00:27:48.023>like <00:27:48.150>Replicators <00:27:48.278>and <00:27:48.405>Isaac <00:27:48.532>Sea. <00:27:48.660>Um,

420
00:27:52.380 --> 00:27:55.680
<00:27:52.380>in <00:27:52.555>order <00:27:52.730>to <00:27:52.905>further <00:27:53.080>boost <00:27:53.255>your <00:27:53.430>accuracy <00:27:53.605>and <00:27:53.780>performance,

421
00:27:55.740 --> 00:28:00.080
<00:27:55.740>you <00:27:55.895>can <00:27:56.051>rely <00:27:56.206>on <00:27:56.362>transfer <00:27:56.517>learning, <00:27:56.673>uh, <00:27:56.828>data <00:27:56.984>pruning, <00:27:57.140>uh,

422
00:28:00.170 --> 00:28:04.160
<00:28:00.170>model <00:28:00.325>pruning, <00:28:00.481>quantization <00:28:00.636>aware <00:28:00.792>training, <00:28:00.947>and <00:28:01.103>we <00:28:01.258>integrate <00:28:01.414>this, <00:28:01.570>uh,

423
00:28:04.160 --> 00:28:08.400
<00:28:04.160>technologies <00:28:04.300>in <00:28:04.440>our <00:28:04.580>tower <00:28:04.720>tool <00:28:04.860>kit <00:28:05.000>at <00:28:05.140>last, <00:28:05.280>but <00:28:05.420>not <00:28:05.560>least,

424
00:28:08.400 --> 00:28:12.280
<00:28:08.400>you <00:28:08.500>need <00:28:08.600>to <00:28:08.700>optimize <00:28:08.800>your <00:28:08.900>models <00:28:09.000>and <00:28:09.100>for <00:28:09.200>that <00:28:09.300>you <00:28:09.400>can <00:28:09.500>rely <00:28:09.600>on <00:28:09.700>the <00:28:09.800>software

425
00:28:12.280 --> 00:28:17.040
<00:28:12.280>product <00:28:12.396>like <00:28:12.513>10 <00:28:12.629>30 <00:28:12.746>and <00:28:12.863>then <00:28:12.980>you <00:28:13.096>deploy <00:28:13.213>it <00:28:13.330>at <00:28:13.446>the <00:28:13.563>edge <00:28:13.680>using

426
00:28:17.100 --> 00:28:21.440
<00:28:17.100>our <00:28:17.227>inference <00:28:17.354>and <00:28:17.481>SDKs <00:28:17.609>like <00:28:17.736>Triton <00:28:17.863>and <00:28:17.990>Deep <00:28:18.118>Stream <00:28:18.245>s <00:28:18.372>D <00:28:18.500>K.

427
00:28:22.060 --> 00:28:26.760
<00:28:22.060>So <00:28:22.159>while <00:28:22.260>as <00:28:22.360>models <00:28:22.460>are <00:28:22.560>growing, <00:28:22.659>uh, <00:28:22.760>there <00:28:22.860>is <00:28:22.960>a <00:28:23.060>huge <00:28:23.159>demand <00:28:23.260>in <00:28:23.360>optimizing <00:28:23.460>them.

428
00:28:27.020 --> 00:28:29.760
<00:28:27.020>So <00:28:27.195>let's <00:28:27.370>all <00:28:27.545>together <00:28:27.720>make <00:28:27.895>our <00:28:28.070>models <00:28:28.245>more <00:28:28.420>efficient.

429
00:28:38.010 --> 00:28:41.440
<00:28:38.010>Thank <00:28:38.165>you <00:28:38.321>very <00:28:38.476>much. <00:28:38.632>Catina. <00:28:38.787>The <00:28:38.943>time <00:28:39.098>is <00:28:39.254>almost <00:28:39.410>over,

430
00:28:41.500 --> 00:28:43.200
<00:28:41.500>but <00:28:41.733>I'm <00:28:41.966>gonna <00:28:42.200>ask <00:28:42.433>you <00:28:42.666>one <00:28:42.900>question.

431
00:28:43.200 --> 00:28:46.640
<00:28:43.200>There <00:28:43.282>are <00:28:43.364>a <00:28:43.447>lot <00:28:43.529>of <00:28:43.611>question <00:28:43.694>that <00:28:43.776>you <00:28:43.858>can <00:28:43.941>find <00:28:44.023>in <00:28:44.105>the <00:28:44.188>app <00:28:44.270>and <00:28:44.352>then <00:28:44.435>you <00:28:44.517>can <00:28:44.600>answer

432
00:28:47.110 --> 00:28:47.430
like,

433
00:28:47.430 --> 00:28:52.200
<00:28:47.430>does <00:28:47.537>synthetic <00:28:47.645>data <00:28:47.753>behave <00:28:47.860>the <00:28:47.968>same <00:28:48.076>as <00:28:48.183>real <00:28:48.291>data <00:28:48.399>or <00:28:48.506>could <00:28:48.614>it <00:28:48.722>also <00:28:48.830>distort

434
00:28:52.260 --> 00:28:53.093
<00:28:52.260>the <00:28:53.009>model?

435
00:28:53.220 --> 00:28:56.480
<00:28:53.220>Um, <00:28:53.375>synthetic <00:28:53.531>data <00:28:53.686>has <00:28:53.842>a <00:28:53.997>big <00:28:54.153>difference <00:28:54.308>from <00:28:54.464>real <00:28:54.620>data.

436
00:28:56.700 --> 00:29:00.080
<00:28:56.700>And <00:28:56.875>of <00:28:57.050>course <00:28:57.225>when <00:28:57.400>developing <00:28:57.575>applications <00:28:57.750>using <00:28:57.925>synthetic <00:28:58.100>data,

437
00:29:00.140 --> 00:29:02.560
<00:29:00.140>we <00:29:00.295>need <00:29:00.451>to <00:29:00.606>take <00:29:00.762>care <00:29:00.917>of <00:29:01.073>the <00:29:01.228>so-called <00:29:01.384>domain <00:29:01.540>gap.

438
00:29:03.180 --> 00:29:06.040
<00:29:03.180>And <00:29:03.355>the <00:29:03.530>main <00:29:03.705>gap <00:29:03.880>can <00:29:04.055>be <00:29:04.230>of <00:29:04.405>various <00:29:04.580>types.

439
00:29:06.140 --> 00:29:08.960
<00:29:06.140>So <00:29:06.373>you <00:29:06.606>can <00:29:06.840>have <00:29:07.073>different <00:29:07.306>object <00:29:07.540>geometry,

440
00:29:09.100 --> 00:29:11.080
<00:29:09.100>you <00:29:09.379>can <00:29:09.659>have <00:29:09.939>different <00:29:10.219>object <00:29:10.500>texture.

441
00:29:11.900 --> 00:29:16.880
<00:29:11.900>So <00:29:12.000>it's <00:29:12.100>definitely <00:29:12.200>a <00:29:12.300>trial <00:29:12.400>and <00:29:12.500>error <00:29:12.600>process <00:29:12.700>and <00:29:12.800>you <00:29:12.900>also <00:29:13.000>need <00:29:13.100>to <00:29:13.200>tweak <00:29:13.300>your

442
00:29:16.880 --> 00:29:20.960
<00:29:16.880>data <00:29:17.007>sets. <00:29:17.134>It's <00:29:17.261>a <00:29:17.389>huge <00:29:17.516>challenge <00:29:17.643>on <00:29:17.770>its <00:29:17.898>own, <00:29:18.025>but <00:29:18.152>it's <00:29:18.280>solvable.

443
00:29:21.700 --> 00:29:22.920
<00:29:21.700>And <00:29:21.974>one <00:29:22.249>very <00:29:22.523>fast <00:29:22.798>question,

444
00:29:22.940 --> 00:29:26.880
<00:29:22.940>do <00:29:23.056>all <00:29:23.173>of <00:29:23.290>these <00:29:23.406>tools <00:29:23.523>and <00:29:23.640>model <00:29:23.756>require <00:29:23.873>Nvidia <00:29:23.990>G <00:29:24.106>P <00:29:24.223>U <00:29:24.340>hardware?

445
00:29:27.380 --> 00:29:31.760
<00:29:27.380>Um, <00:29:27.487>most <00:29:27.595>of <00:29:27.703>them, <00:29:27.810>yes. <00:29:27.918>But <00:29:28.026>for <00:29:28.133>example, <00:29:28.241>Trian <00:29:28.349>Inference <00:29:28.456>server, <00:29:28.564>as <00:29:28.672>I <00:29:28.780>mentioned,

446
00:29:31.860 --> 00:29:35.360
<00:29:31.860>it <00:29:31.987>can <00:29:32.114>run <00:29:32.241>on <00:29:32.369>CPUs <00:29:32.496>or <00:29:32.623>also <00:29:32.750>on <00:29:32.878>GPUs <00:29:33.005>of <00:29:33.132>other <00:29:33.260>vendors.

447
00:29:36.910 --> 00:29:40.680
<00:29:36.910>Well, <00:29:37.003>the <00:29:37.096>rest <00:29:37.190>of <00:29:37.283>the <00:29:37.376>question, <00:29:37.470>as <00:29:37.563>I <00:29:37.656>said, <00:29:37.750>you <00:29:37.843>can <00:29:37.936>find <00:29:38.030>it <00:29:38.123>in <00:29:38.216>the <00:29:38.310>app.

448
00:29:41.010 --> 00:29:42.600
<00:29:41.010>Thank <00:29:41.360>you <00:29:41.710>very <00:29:42.060>much, <00:29:42.410>Catina.

449
00:29:43.690 --> 00:29:44.523
<00:29:43.690>Thank <00:29:44.439>you.

450
00:29:47.380 --> 00:29:48.440
<00:29:47.380>We <00:29:47.570>are <00:29:47.761>going <00:29:47.952>to <00:29:48.143>have <00:29:48.334>now.