WEBVTT

1
00:00:02.035 --> 00:00:04.485
<00:00:02.035>Well, <00:00:02.190>this <00:00:02.346>is <00:00:02.501>when <00:00:02.657>I <00:00:02.812>need <00:00:02.968>to <00:00:03.123>do <00:00:03.279>some <00:00:03.435>feelings.

2
00:00:04.505 --> 00:00:05.725
<00:00:04.505>So, <00:00:05.054>yeah. <00:00:05.603>Okay.

3
00:00:06.025 --> 00:00:07.165
<00:00:06.025>So <00:00:06.281>chat <00:00:06.538>bots <00:00:06.794>are <00:00:07.051>going

4
00:00:07.165 --> 00:00:09.325
<00:00:07.165>to <00:00:07.398>destroy <00:00:07.631>infrastructures <00:00:07.865>and <00:00:08.098>new <00:00:08.331>cloud <00:00:08.565>build.

5
00:00:09.345 --> 00:00:12.765
<00:00:09.345>That's <00:00:09.485>kind <00:00:09.625>of, <00:00:09.765>you <00:00:09.905>know, <00:00:10.045>a <00:00:10.185>big <00:00:10.325>topic <00:00:10.465>to <00:00:10.605>talk <00:00:10.745>about.

6
00:00:13.345 --> 00:00:17.765
<00:00:13.345>So <00:00:13.461>a <00:00:13.578>bit <00:00:13.695>about <00:00:13.811>me. <00:00:13.928>Uh, <00:00:14.045>I'm <00:00:14.161>the <00:00:14.278>CEO <00:00:14.395>and <00:00:14.511>CTO <00:00:14.628>of <00:00:14.745>quiver.

7
00:00:18.145 --> 00:00:21.325
<00:00:18.145>And <00:00:18.300>before <00:00:18.456>that <00:00:18.611>I <00:00:18.767>used <00:00:18.922>to <00:00:19.078>be <00:00:19.233>a <00:00:19.389>DevOps, <00:00:19.544>meaning

8
00:00:21.325 --> 00:00:23.525
<00:00:21.325>that <00:00:21.480>this <00:00:21.636>is <00:00:21.791>going <00:00:21.947>to <00:00:22.102>be <00:00:22.258>focused <00:00:22.413>on <00:00:22.569>the <00:00:22.724>how

9
00:00:24.225 --> 00:00:27.685
<00:00:24.225>and <00:00:24.365>what <00:00:24.505>you <00:00:24.645>should <00:00:24.785>not <00:00:24.925>do <00:00:25.065>when <00:00:25.205>you're <00:00:25.345>building <00:00:25.485>a <00:00:25.625>chatbot.

10
00:00:28.505 --> 00:00:31.405
<00:00:28.505>Um, <00:00:28.705>so <00:00:28.904>just <00:00:29.105>to <00:00:29.305>let <00:00:29.505>you <00:00:29.705>know, <00:00:29.904>um,

11
00:00:31.885 --> 00:00:34.485
<00:00:31.885>I <00:00:32.060>created <00:00:32.235>an <00:00:32.410>open <00:00:32.585>source <00:00:32.760>project <00:00:32.935>a <00:00:33.110>year <00:00:33.285>ago,

12
00:00:34.905 --> 00:00:37.285
<00:00:34.905>and <00:00:35.105>it <00:00:35.305>started <00:00:35.505>with <00:00:35.705>200 <00:00:35.905>lines <00:00:36.105>of <00:00:36.305>calls

13
00:00:37.385 --> 00:00:40.525
<00:00:37.385>and <00:00:37.525>it <00:00:37.665>became <00:00:37.805>viral, <00:00:37.945>meaning <00:00:38.085>that <00:00:38.224>I <00:00:38.364>didn't <00:00:38.504>have <00:00:38.644>any <00:00:38.785>time

14
00:00:41.065 --> 00:00:43.725
<00:00:41.065>to <00:00:41.205>think <00:00:41.345>about <00:00:41.485>what <00:00:41.625>I <00:00:41.765>was <00:00:41.905>doing, <00:00:42.044>I <00:00:42.184>was <00:00:42.324>just <00:00:42.464>building.

15
00:00:44.475 --> 00:00:48.485
<00:00:44.475>That <00:00:44.602>means <00:00:44.729>that <00:00:44.856>today <00:00:44.984>we <00:00:45.111>have <00:00:45.238>almost <00:00:45.365>one <00:00:45.493>year <00:00:45.620>of <00:00:45.747>legacy <00:00:45.875>code

16
00:00:48.905 --> 00:00:50.245
<00:00:48.905>and <00:00:49.146>things <00:00:49.387>that <00:00:49.628>we <00:00:49.869>did <00:00:50.111>wrong.

17
00:00:50.865 --> 00:00:53.245
<00:00:50.865>So <00:00:51.020>if <00:00:51.176>you <00:00:51.331>can <00:00:51.487>take <00:00:51.642>one <00:00:51.798>thing <00:00:51.953>from <00:00:52.109>me <00:00:52.265>today

18
00:00:53.945 --> 00:00:57.285
<00:00:53.945>is <00:00:54.072>don't <00:00:54.199>make <00:00:54.326>the <00:00:54.454>same <00:00:54.581>mistakes <00:00:54.708>as <00:00:54.835>I <00:00:54.963>did, <00:00:55.090>and <00:00:55.217>that's <00:00:55.345>it.

19
00:01:00.435 --> 00:01:02.975
<00:01:00.435>So <00:01:00.635>before <00:01:00.835>Gene <00:01:01.035>ai, <00:01:01.235>we <00:01:01.435>had <00:01:01.635>two <00:01:01.835>kinds

20
00:01:02.975 --> 00:01:04.415
<00:01:02.975>of <00:01:03.191>people, <00:01:03.407>well, <00:01:03.623>a <00:01:03.839>little <00:01:04.055>bit <00:01:04.271>more.

21
00:01:04.755 --> 00:01:07.375
<00:01:04.754>We <00:01:04.955>had <00:01:05.155>the <00:01:05.354>web <00:01:05.554>developers, <00:01:05.755>so <00:01:05.955>these <00:01:06.155>kind

22
00:01:07.375 --> 00:01:08.895
<00:01:07.375>of <00:01:07.603>people <00:01:07.831>they <00:01:08.059>used <00:01:08.286>to <00:01:08.515>work <00:01:08.743>with.

23
00:01:08.895 --> 00:01:11.615
<00:01:08.895>Limited <00:01:09.128>resources, <00:01:09.361>meaning <00:01:09.595>you <00:01:09.828>have, <00:01:10.061>you <00:01:10.295>know,

24
00:01:11.615 --> 00:01:13.175
<00:01:11.615>everything <00:01:11.848>needs <00:01:12.081>to <00:01:12.315>run <00:01:12.548>on <00:01:12.781>your <00:01:13.015>computer,

25
00:01:13.555 --> 00:01:14.855
<00:01:13.555>so <00:01:13.847>you <00:01:14.140>don't <00:01:14.432>need <00:01:14.725>GPUs.

26
00:01:15.235 --> 00:01:17.255
<00:01:15.235>You <00:01:15.515>can <00:01:15.795>simply, <00:01:16.075>I <00:01:16.355>don't <00:01:16.635>know,

27
00:01:17.355 --> 00:01:20.135
<00:01:17.355>pip <00:01:17.588>install <00:01:17.821>NPM, <00:01:18.055>all <00:01:18.288>of <00:01:18.521>these <00:01:18.755>things.

28
00:01:20.675 --> 00:01:23.015
<00:01:20.675>And <00:01:20.908>they <00:01:21.141>are <00:01:21.375>usually <00:01:21.608>CPU <00:01:21.841>bound <00:01:22.075>task.

29
00:01:23.715 --> 00:01:26.695
<00:01:23.715>And <00:01:23.890>you <00:01:24.065>can <00:01:24.240>horizontally <00:01:24.415>scale <00:01:24.590>all <00:01:24.765>of <00:01:24.940>your <00:01:25.115>applications,

30
00:01:26.695 --> 00:01:28.215
<00:01:26.695>meaning <00:01:26.923>that <00:01:27.151>if <00:01:27.378>you <00:01:27.607>have <00:01:27.835>more <00:01:28.063>users,

31
00:01:28.435 --> 00:01:31.295
<00:01:28.435>you <00:01:28.610>can <00:01:28.785>simply <00:01:28.960>have <00:01:29.135>more <00:01:29.310>replicas <00:01:29.485>of <00:01:29.660>your <00:01:29.835>app.

32
00:01:31.955 --> 00:01:36.135
<00:01:31.955>And <00:01:32.130>the, <00:01:32.304>the <00:01:32.480>web <00:01:32.655>developers <00:01:32.830>usually <00:01:33.005>work <00:01:33.179>with <00:01:33.355>databases,

33
00:01:36.725 --> 00:01:38.415
<00:01:36.725>APIs, <00:01:37.191>and <00:01:37.658>background <00:01:38.125>workers

34
00:01:38.485 --> 00:01:40.295
<00:01:38.485>that <00:01:38.718>are <00:01:38.951>very <00:01:39.185>slow, <00:01:39.418>uh, <00:01:39.651>very <00:01:39.885>fast.

35
00:01:40.395 --> 00:01:42.375
<00:01:40.395>For <00:01:40.675>example, <00:01:40.955>sending <00:01:41.235>notification <00:01:41.515>to <00:01:41.795>users.

36
00:01:43.115 --> 00:01:45.535
<00:01:43.115>And <00:01:43.242>then <00:01:43.369>we <00:01:43.496>have <00:01:43.624>the <00:01:43.751>second <00:01:43.878>kind <00:01:44.005>of <00:01:44.133>people <00:01:44.260>that <00:01:44.387>I <00:01:44.515>like

37
00:01:45.535 --> 00:01:49.335
<00:01:45.535>to <00:01:45.690>talk <00:01:45.846>about <00:01:46.001>is <00:01:46.157>the <00:01:46.312>data <00:01:46.468>scientists <00:01:46.623>and <00:01:46.779>data <00:01:46.935>engineers.

38
00:01:49.675 --> 00:01:54.135
<00:01:49.675>So <00:01:49.875>these <00:01:50.075>people <00:01:50.274>usually <00:01:50.475>have <00:01:50.675>unlimited <00:01:50.875>resources, <00:01:51.075>meaning

39
00:01:54.135 --> 00:01:56.415
<00:01:54.134>that <00:01:54.335>you <00:01:54.535>can <00:01:54.734>have, <00:01:54.934>I <00:01:55.134>don't <00:01:55.335>know, <00:01:55.535>clusters

40
00:01:56.415 --> 00:01:57.855
<00:01:56.414>of <00:01:57.062>computing <00:01:57.710>power.

41
00:01:57.995 --> 00:02:00.975
<00:01:57.995>You <00:01:58.228>have <00:01:58.461>GPUs, <00:01:58.695>like <00:01:58.928>thousands <00:01:59.161>of <00:01:59.395>them.

42
00:02:01.445 --> 00:02:03.575
<00:02:01.445>Well, <00:02:01.572>if <00:02:01.699>you're <00:02:01.826>in <00:02:01.954>a <00:02:02.081>big <00:02:02.208>company, <00:02:02.335>if <00:02:02.463>you're <00:02:02.590>a <00:02:02.717>small <00:02:02.845>company,

43
00:02:03.765 --> 00:02:04.855
<00:02:03.765>GPUs <00:02:04.010>are <00:02:04.255>hard <00:02:04.500>to <00:02:04.746>find.

44
00:02:05.435 --> 00:02:08.335
<00:02:05.435>But <00:02:05.635>they <00:02:05.835>usually <00:02:06.035>work <00:02:06.235>with <00:02:06.435>GPU <00:02:06.635>Bound <00:02:06.835>task

45
00:02:08.835 --> 00:02:10.895
<00:02:08.835>and <00:02:09.068>they <00:02:09.301>can <00:02:09.535>work <00:02:09.768>with <00:02:10.001>horizonal <00:02:10.235>scaling,

46
00:02:10.895 --> 00:02:13.815
<00:02:10.895>meaning <00:02:11.095>they <00:02:11.295>can <00:02:11.495>have <00:02:11.695>more <00:02:11.895>GPUs, <00:02:12.095>more <00:02:12.295>replicas,

47
00:02:13.955 --> 00:02:17.015
<00:02:13.955>or <00:02:14.110>have <00:02:14.266>bigger <00:02:14.421>GPUs, <00:02:14.577>bigger <00:02:14.732>CPUs <00:02:14.888>and <00:02:15.043>all <00:02:15.199>these <00:02:15.355>things.

48
00:02:17.475 --> 00:02:19.575
<00:02:17.475>And <00:02:17.650>they <00:02:17.825>work <00:02:18.000>with <00:02:18.174>many <00:02:18.350>more <00:02:18.525>different <00:02:18.700>things, <00:02:18.875>such

49
00:02:19.575 --> 00:02:24.455
<00:02:19.575>as <00:02:19.730>databases <00:02:19.886>where <00:02:20.041>they <00:02:20.197>work <00:02:20.352>with <00:02:20.508>bronze, <00:02:20.663>gold <00:02:20.819>data, <00:02:20.975>ETL,

50
00:02:24.555 --> 00:02:26.775
<00:02:24.555>and <00:02:24.788>they <00:02:25.021>have <00:02:25.255>long <00:02:25.488>background <00:02:25.721>working <00:02:25.955>burners.

51
00:02:27.315 --> 00:02:30.455
<00:02:27.315>So <00:02:27.442>just <00:02:27.569>to <00:02:27.696>show <00:02:27.824>of <00:02:27.951>hand, <00:02:28.078>who <00:02:28.205>is <00:02:28.333>a <00:02:28.460>software <00:02:28.587>developer <00:02:28.715>here?

52
00:02:32.085 --> 00:02:34.775
<00:02:32.085>Okay. <00:02:32.260>And <00:02:32.435>who <00:02:32.610>is <00:02:32.785>the <00:02:32.960>data <00:02:33.135>scientist? <00:02:33.310>Data <00:02:33.485>engineer.

53
00:02:36.195 --> 00:02:37.575
<00:02:36.195>Ooh, <00:02:36.608>just <00:02:37.023>one. <00:02:37.436>Okay.

54
00:02:38.425 --> 00:02:41.845
<00:02:38.425>So <00:02:38.565>what <00:02:38.705>happened <00:02:38.845>with <00:02:38.985>Gene <00:02:39.125>AI <00:02:39.265>is <00:02:39.405>a <00:02:39.545>new <00:02:39.685>breed <00:02:39.825>of

55
00:02:42.485 --> 00:02:44.925
<00:02:42.485>engineers <00:02:42.685>came <00:02:42.885>out <00:02:43.085>and <00:02:43.285>it's <00:02:43.485>called <00:02:43.685>AI <00:02:43.885>Engineers.

56
00:02:45.385 --> 00:02:50.205
<00:02:45.385>So <00:02:45.501>this <00:02:45.618>is <00:02:45.734>genetic, <00:02:45.851>but <00:02:45.968>uh, <00:02:46.084>Chad, <00:02:46.201>GPT, <00:02:46.318>but <00:02:46.435>you <00:02:46.551>have <00:02:46.668>the <00:02:46.785>big

57
00:02:50.305 --> 00:02:53.245
<00:02:50.305>and <00:02:50.771>tough <00:02:51.238>data <00:02:51.705>scientists.

58
00:02:53.565 --> 00:02:55.685
<00:02:53.565>Engineers <00:02:53.765>will <00:02:53.965>have <00:02:54.165>a <00:02:54.365>lot <00:02:54.565>of <00:02:54.765>computing <00:02:54.965>power.

59
00:02:56.155 --> 00:02:58.725
<00:02:56.155>Then <00:02:56.295>you <00:02:56.435>have <00:02:56.575>the <00:02:56.715>web <00:02:56.855>developer <00:02:56.995>that <00:02:57.135>now <00:02:57.275>can <00:02:57.415>do <00:02:57.555>things

60
00:02:59.235 --> 00:03:00.685
<00:02:59.235>that <00:02:59.670>weren't <00:03:00.105>possible <00:03:00.540>before.

61
00:03:01.105 --> 00:03:03.525
<00:03:01.105>And <00:03:01.338>so <00:03:01.571>now <00:03:01.804>you <00:03:02.038>have <00:03:02.271>AI <00:03:02.505>engineers

62
00:03:03.665 --> 00:03:05.245
<00:03:03.665>and <00:03:03.945>you <00:03:04.225>work <00:03:04.505>with <00:03:04.785>different <00:03:05.065>things.

63
00:03:05.385 --> 00:03:07.765
<00:03:05.385>So <00:03:05.665>you <00:03:05.945>have <00:03:06.225>now <00:03:06.505>limited <00:03:06.785>resources,

64
00:03:08.425 --> 00:03:11.005
<00:03:08.425>but <00:03:08.580>the <00:03:08.736>cloud <00:03:08.891>providers <00:03:09.047>give <00:03:09.202>you <00:03:09.358>a <00:03:09.513>lot <00:03:09.669>of <00:03:09.825>money.

65
00:03:11.345 --> 00:03:13.205
<00:03:11.345>So <00:03:11.520>if <00:03:11.695>you <00:03:11.870>start <00:03:12.045>a <00:03:12.220>company, <00:03:12.395>you <00:03:12.570>can <00:03:12.745>kind

66
00:03:13.205 --> 00:03:14.645
<00:03:13.205>of <00:03:13.464>have <00:03:13.723>a <00:03:13.982>hundred <00:03:14.241>thousand <00:03:14.501>dollars

67
00:03:14.865 --> 00:03:19.285
<00:03:14.865>of <00:03:15.098>computing <00:03:15.331>power <00:03:15.565>from <00:03:15.798>A-W-S-G-C-P, <00:03:16.031>uh, <00:03:16.265>Asia.

68
00:03:19.345 --> 00:03:23.005
<00:03:19.345>So <00:03:19.472>you <00:03:19.599>don't <00:03:19.726>mind <00:03:19.854>wasting <00:03:19.981>a <00:03:20.108>lot <00:03:20.235>of <00:03:20.363>computing <00:03:20.490>power <00:03:20.617>for <00:03:20.745>now.

69
00:03:23.985 --> 00:03:25.525
<00:03:23.985>Now <00:03:24.331>you're <00:03:24.678>working <00:03:25.024>with <00:03:25.371>CPU

70
00:03:25.525 --> 00:03:29.525
<00:03:25.525>and <00:03:25.725>GPU <00:03:25.925>bound <00:03:26.125>things, <00:03:26.325>horizontal <00:03:26.525>scaling, <00:03:26.725>vertical <00:03:26.925>scaling

71
00:03:30.205 --> 00:03:32.645
<00:03:30.204>databases, <00:03:30.484>vector <00:03:30.765>stores, <00:03:31.045>data <00:03:31.325>lake <00:03:31.605>APIs.

72
00:03:32.645 --> 00:03:34.725
<00:03:32.645>You <00:03:32.845>understand <00:03:33.045>what <00:03:33.245>I <00:03:33.445>mean? <00:03:33.645>It <00:03:33.845>makes <00:03:34.045>it

73
00:03:35.405 --> 00:03:36.965
<00:03:35.405>a <00:03:35.755>new <00:03:36.105>breed <00:03:36.455>of <00:03:36.805>engineers.

74
00:03:37.345 --> 00:03:39.445
<00:03:37.345>And <00:03:37.520>these <00:03:37.695>engineers, <00:03:37.870>if <00:03:38.045>you <00:03:38.220>come <00:03:38.395>from <00:03:38.570>g <00:03:38.745>uh,

75
00:03:39.445 --> 00:03:41.885
<00:03:39.445>from <00:03:39.678>data <00:03:39.911>science, <00:03:40.144>data <00:03:40.378>engineers <00:03:40.611>like <00:03:40.845>Mr.

76
00:03:42.745 --> 00:03:46.645
<00:03:42.745>You <00:03:42.872>have, <00:03:42.999>you <00:03:43.126>know <00:03:43.254>how <00:03:43.381>to, <00:03:43.508>to <00:03:43.635>have <00:03:43.763>a <00:03:43.890>lot <00:03:44.017>of <00:03:44.145>resources.

77
00:03:47.065 --> 00:03:48.605
<00:03:47.065>But <00:03:47.296>if <00:03:47.527>you <00:03:47.757>come <00:03:47.989>from <00:03:48.220>web <00:03:48.451>development,

78
00:03:48.665 --> 00:03:50.125
<00:03:48.665>you <00:03:48.993>are <00:03:49.322>here <00:03:49.650>to <00:03:49.978>optimize.

79
00:03:50.545 --> 00:03:53.045
<00:03:50.545>And <00:03:50.745>so <00:03:50.945>making <00:03:51.145>the <00:03:51.345>merge <00:03:51.545>between <00:03:51.745>the <00:03:51.945>two

80
00:03:53.045 --> 00:03:54.765
<00:03:53.045>of <00:03:53.395>them <00:03:53.745>is <00:03:54.095>very <00:03:54.445>hard.

81
00:03:55.305 --> 00:03:57.245
<00:03:55.305>And <00:03:55.505>so <00:03:55.705>if <00:03:55.905>you're <00:03:56.105>a <00:03:56.305>data <00:03:56.505>data <00:03:56.705>scientist

82
00:03:57.245 --> 00:04:01.725
<00:03:57.245>or <00:03:57.400>web <00:03:57.556>developers, <00:03:57.711>you <00:03:57.867>don't <00:03:58.022>have <00:03:58.178>the <00:03:58.333>same, <00:03:58.489>um, <00:03:58.645>um,

83
00:04:01.895 --> 00:04:04.085
<00:04:01.895>experience, <00:04:02.095>meaning <00:04:02.295>that <00:04:02.495>you're <00:04:02.695>going <00:04:02.895>to <00:04:03.095>make <00:04:03.295>mistakes

84
00:04:04.085 --> 00:04:05.285
<00:04:04.085>that <00:04:04.265>the <00:04:04.445>other <00:04:04.625>one <00:04:04.805>does <00:04:04.985>not <00:04:05.165>make.

85
00:04:06.085 --> 00:04:08.745
<00:04:06.085>So <00:04:06.212>this <00:04:06.339>is <00:04:06.466>the <00:04:06.594>story <00:04:06.721>of <00:04:06.848>all <00:04:06.975>of <00:04:07.103>the <00:04:07.230>mistakes <00:04:07.357>I've <00:04:07.485>made.

86
00:04:09.665 --> 00:04:12.605
<00:04:09.665>So <00:04:09.840>let's <00:04:10.015>start <00:04:10.190>with <00:04:10.364>a <00:04:10.540>simple, <00:04:10.715>the <00:04:10.890>simplest <00:04:11.065>brag.

87
00:04:12.785 --> 00:04:15.485
<00:04:12.785>So <00:04:12.960>you <00:04:13.135>have <00:04:13.310>probably <00:04:13.484>heard <00:04:13.660>about <00:04:13.835>brag <00:04:14.010>in <00:04:14.185>the

88
00:04:15.485 --> 00:04:16.985
<00:04:15.485>last <00:04:15.822>year <00:04:16.160>and <00:04:16.497>a <00:04:16.835>half.

89
00:04:17.415 --> 00:04:20.985
<00:04:17.415>It's <00:04:17.542>a <00:04:17.669>way <00:04:17.796>to <00:04:17.924>add <00:04:18.051>context, <00:04:18.178>but <00:04:18.305>the <00:04:18.433>LLM <00:04:18.560>wasn't <00:04:18.687>trained <00:04:18.815>on.

90
00:04:21.445 --> 00:04:23.985
<00:04:21.445>So <00:04:21.678>it <00:04:21.911>starts <00:04:22.145>with <00:04:22.378>reading <00:04:22.611>the <00:04:22.844>document.

91
00:04:24.895 --> 00:04:26.875
<00:04:24.895>So <00:04:25.070>what <00:04:25.245>you <00:04:25.419>do <00:04:25.594>is <00:04:25.770>you <00:04:25.945>take <00:04:26.120>a <00:04:26.294>PDF

92
00:04:27.095 --> 00:04:28.635
<00:04:27.095>and <00:04:27.268>you <00:04:27.441>read <00:04:27.614>it, <00:04:27.788>you <00:04:27.961>load <00:04:28.134>it <00:04:28.307>into <00:04:28.481>memo.

93
00:04:28.785 --> 00:04:31.235
<00:04:28.785>This <00:04:28.940>is <00:04:29.096>usually <00:04:29.251>what <00:04:29.407>I <00:04:29.562>like <00:04:29.718>to <00:04:29.873>call <00:04:30.029>CPO <00:04:30.185>bound.

94
00:04:31.985 --> 00:04:33.555
<00:04:31.985>Then <00:04:32.265>you <00:04:32.545>have <00:04:32.825>the <00:04:33.105>chunking, <00:04:33.385>meaning

95
00:04:33.555 --> 00:04:34.755
<00:04:33.555>that <00:04:33.825>you <00:04:34.095>take <00:04:34.365>your <00:04:34.635>document

96
00:04:35.135 --> 00:04:38.115
<00:04:35.135>and <00:04:35.368>you <00:04:35.601>chunk <00:04:35.835>it <00:04:36.068>into, <00:04:36.301>uh, <00:04:36.534>paragraphs

97
00:04:38.415 --> 00:04:40.155
<00:04:38.415>or <00:04:38.648>why <00:04:38.881>it's <00:04:39.115>the <00:04:39.348>best <00:04:39.581>to <00:04:39.815>split.

98
00:04:40.685 --> 00:04:43.215
<00:04:40.685>Then <00:04:40.965>we <00:04:41.245>use <00:04:41.525>an <00:04:41.805>embedding, <00:04:42.085>meaning

99
00:04:43.215 --> 00:04:46.015
<00:04:43.215>that <00:04:43.390>it <00:04:43.565>creates <00:04:43.740>a <00:04:43.915>vector <00:04:44.090>representation <00:04:44.265>of <00:04:44.440>the <00:04:44.615>meaning

100
00:04:46.435 --> 00:04:50.915
<00:04:46.435>of <00:04:46.575>your <00:04:46.715>paragraph, <00:04:46.855>and <00:04:46.995>you <00:04:47.135>store <00:04:47.275>that <00:04:47.415>into <00:04:47.555>a <00:04:47.695>vector <00:04:47.835>store.

101
00:04:51.975 --> 00:04:55.045
<00:04:51.975>Here <00:04:52.150>I <00:04:52.325>put <00:04:52.500>the <00:04:52.675>things <00:04:52.850>that <00:04:53.025>I <00:04:53.200>kind <00:04:53.375>of

102
00:04:55.755 --> 00:04:59.275
<00:04:55.755>a <00:04:55.910>web <00:04:56.066>developer <00:04:56.221>has <00:04:56.377>experience <00:04:56.532>with <00:04:56.688>CPUs <00:04:56.843>at <00:04:56.999>the <00:04:57.155>bottom.

103
00:04:59.415 --> 00:05:01.635
<00:04:59.415>And <00:04:59.615>then <00:04:59.815>here <00:05:00.015>for <00:05:00.215>the <00:05:00.415>embedding <00:05:00.615>is <00:05:00.815>something

104
00:05:01.635 --> 00:05:02.755
<00:05:01.635>that <00:05:01.971>is <00:05:02.307>GPU <00:05:02.643>bound.

105
00:05:02.895 --> 00:05:06.035
<00:05:02.895>It <00:05:03.070>needs <00:05:03.245>a <00:05:03.419>model <00:05:03.594>that <00:05:03.770>runs <00:05:03.945>on <00:05:04.120>A <00:05:04.294>GPU.

106
00:05:06.035 --> 00:05:07.355
<00:05:06.035>You <00:05:06.204>can <00:05:06.374>have <00:05:06.544>some <00:05:06.713>much <00:05:06.883>one <00:05:07.053>on <00:05:07.223>CPU,

107
00:05:07.355 --> 00:05:09.195
<00:05:07.355>but <00:05:07.555>still <00:05:07.755>it <00:05:07.955>needs <00:05:08.155>a <00:05:08.355>lot <00:05:08.555>of <00:05:08.755>resources.

108
00:05:09.735 --> 00:05:11.935
<00:05:09.735>So <00:05:09.935>that's <00:05:10.135>the <00:05:10.335>first <00:05:10.535>step <00:05:10.735>of <00:05:10.935>a <00:05:11.135>bag.

109
00:05:12.155 --> 00:05:16.855
<00:05:12.155>You <00:05:12.310>ingest <00:05:12.466>the <00:05:12.621>data, <00:05:12.777>then <00:05:12.932>the <00:05:13.088>user <00:05:13.243>asks <00:05:13.399>a <00:05:13.554>question.

110
00:05:17.075 --> 00:05:19.335
<00:05:17.075>So <00:05:17.215>what <00:05:17.354>happens <00:05:17.495>when <00:05:17.635>the <00:05:17.775>user <00:05:17.914>asks <00:05:18.055>a <00:05:18.195>question <00:05:18.335>on <00:05:18.474>the

111
00:05:19.535 --> 00:05:21.095
<00:05:19.535>platform <00:05:19.768>to <00:05:20.001>answer <00:05:20.235>questions <00:05:20.468>on <00:05:20.701>a <00:05:20.935>document?

112
00:05:21.715 --> 00:05:24.255
<00:05:21.715>The <00:05:21.948>first <00:05:22.181>thing <00:05:22.414>is, <00:05:22.648>uh, <00:05:22.881>we <00:05:23.114>create

113
00:05:24.255 --> 00:05:25.535
<00:05:24.255>what <00:05:24.485>we <00:05:24.715>call <00:05:24.946>a <00:05:25.176>standard <00:05:25.407>question.

114
00:05:25.715 --> 00:05:28.295
<00:05:25.715>We <00:05:25.948>use <00:05:26.181>the <00:05:26.414>history <00:05:26.648>of <00:05:26.881>the <00:05:27.114>question

115
00:05:28.835 --> 00:05:31.735
<00:05:28.835>to <00:05:29.010>generate <00:05:29.185>a <00:05:29.359>new <00:05:29.534>question <00:05:29.710>that <00:05:29.885>is <00:05:30.060>more <00:05:30.234>relevant.

116
00:05:32.285 --> 00:05:34.015
<00:05:32.284>That <00:05:32.518>is <00:05:32.751>a <00:05:32.984>call <00:05:33.218>to <00:05:33.451>an <00:05:33.684>LLM.

117
00:05:34.785 --> 00:05:37.035
<00:05:34.784>Then <00:05:35.064>we <00:05:35.344>call <00:05:35.624>the <00:05:35.905>vector <00:05:36.184>store

118
00:05:37.495 --> 00:05:39.755
<00:05:37.495>and <00:05:37.695>we <00:05:37.895>retrieve <00:05:38.095>the <00:05:38.295>paragraphs <00:05:38.495>of <00:05:38.695>the <00:05:38.895>document

119
00:05:39.755 --> 00:05:41.515
<00:05:39.755>that <00:05:39.988>are <00:05:40.221>most <00:05:40.455>relevant <00:05:40.688>to <00:05:40.921>the <00:05:41.155>question.

120
00:05:42.555 --> 00:05:46.645
<00:05:42.555>And <00:05:42.695>then <00:05:42.835>we <00:05:42.975>send <00:05:43.115>these <00:05:43.255>paragraphs <00:05:43.395>to <00:05:43.535>the <00:05:43.675>LLM, <00:05:43.815>meaning <00:05:43.955>that

121
00:05:47.225 --> 00:05:50.005
<00:05:47.225>in <00:05:47.400>this <00:05:47.575>time <00:05:47.750>you <00:05:47.925>made <00:05:48.100>two <00:05:48.275>calls <00:05:48.450>to <00:05:48.625>LLS

122
00:05:50.385 --> 00:05:52.605
<00:05:50.385>and <00:05:50.585>one <00:05:50.784>to <00:05:50.985>something <00:05:51.185>that <00:05:51.385>is <00:05:51.585>CPU <00:05:51.784>bound.

123
00:05:53.665 --> 00:05:56.655
<00:05:53.665>There <00:05:53.820>are <00:05:53.976>many <00:05:54.131>issues <00:05:54.287>with <00:05:54.442>what <00:05:54.598>I've <00:05:54.753>showed <00:05:54.909>you <00:05:55.065>today.

124
00:05:57.685 --> 00:05:59.945
<00:05:57.685>The <00:05:57.885>first <00:05:58.085>one <00:05:58.285>is <00:05:58.485>you <00:05:58.685>are <00:05:58.885>vertically <00:05:59.085>bound

125
00:06:00.165 --> 00:06:04.905
<00:06:00.165>and <00:06:00.340>isly <00:06:00.515>bound, <00:06:00.690>uh, <00:06:00.865>steps <00:06:01.040>meaning <00:06:01.215>that, <00:06:01.390>for <00:06:01.565>example,

126
00:06:05.245 --> 00:06:08.505
<00:06:05.245>um, <00:06:05.400>things <00:06:05.556>that <00:06:05.711>are <00:06:05.867>CPU <00:06:06.022>bound, <00:06:06.178>you <00:06:06.333>could <00:06:06.489>scale <00:06:06.645>horizontally,

127
00:06:09.005 --> 00:06:11.065
<00:06:09.005>but <00:06:09.205>then <00:06:09.405>for <00:06:09.605>example, <00:06:09.805>the <00:06:10.005>embedding, <00:06:10.205>you're <00:06:10.405>limiting

128
00:06:11.085 --> 00:06:12.465
<00:06:11.085>to <00:06:11.333>the <00:06:11.581>size <00:06:11.830>of <00:06:12.078>your <00:06:12.327>embedding.

129
00:06:12.695 --> 00:06:14.585
<00:06:12.695>Like <00:06:12.870>if <00:06:13.045>you <00:06:13.219>have <00:06:13.395>a <00:06:13.570>one <00:06:13.745>GPU, <00:06:13.920>you <00:06:14.094>need

130
00:06:14.585 --> 00:06:16.465
<00:06:14.585>to <00:06:14.818>scale <00:06:15.051>another <00:06:15.284>one <00:06:15.518>or <00:06:15.751>bigger <00:06:15.984>one.

131
00:06:17.125 --> 00:06:20.595
<00:06:17.125>There <00:06:17.280>are <00:06:17.436>GPUs <00:06:17.591>that <00:06:17.747>are <00:06:17.902>required <00:06:18.058>in <00:06:18.213>user <00:06:18.369>facing <00:06:18.525>steps.

132
00:06:21.295 --> 00:06:24.115
<00:06:21.295>And <00:06:21.450>this <00:06:21.606>is <00:06:21.761>for <00:06:21.917>me, <00:06:22.072>coming <00:06:22.228>as <00:06:22.383>a <00:06:22.539>DevOps, <00:06:22.695>uh,

133
00:06:24.225 --> 00:06:28.315
<00:06:24.225>part <00:06:24.365>is <00:06:24.505>very <00:06:24.645>hard <00:06:24.785>to <00:06:24.925>do <00:06:25.065>'cause <00:06:25.205>GPUs <00:06:25.345>are <00:06:25.485>more <00:06:25.625>expensive.

134
00:06:29.065 --> 00:06:32.155
<00:06:29.065>It's <00:06:29.240>harder <00:06:29.415>to <00:06:29.590>maintain <00:06:29.765>and <00:06:29.940>scale <00:06:30.115>at <00:06:30.290>in <00:06:30.465>production.

135
00:06:32.895 --> 00:06:36.645
<00:06:32.895>The <00:06:33.022>first, <00:06:33.149>the <00:06:33.276>third <00:06:33.404>one <00:06:33.531>is <00:06:33.658>you <00:06:33.785>have <00:06:33.913>three <00:06:34.040>to <00:06:34.167>four <00:06:34.294>seconds

136
00:06:36.785 --> 00:06:38.805
<00:06:36.784>to <00:06:36.984>answer <00:06:37.184>a <00:06:37.385>single <00:06:37.585>question <00:06:37.784>from <00:06:37.984>a <00:06:38.184>user.

137
00:06:39.475 --> 00:06:42.445
<00:06:39.475>Usually <00:06:39.755>from <00:06:40.035>a <00:06:40.315>web <00:06:40.595>developer <00:06:40.875>perspective,

138
00:06:43.315 --> 00:06:46.455
<00:06:43.315>an <00:06:43.515>answer <00:06:43.715>takes <00:06:43.915>like <00:06:44.115>a <00:06:44.315>hundred, <00:06:44.515>150 <00:06:44.715>milliseconds.

139
00:06:46.525 --> 00:06:48.375
<00:06:46.525>Here <00:06:46.724>you're <00:06:46.924>talking <00:06:47.125>about <00:06:47.325>three <00:06:47.525>to <00:06:47.724>four <00:06:47.924>seconds.

140
00:06:49.335 --> 00:06:51.905
<00:06:49.335>Then <00:06:49.474>you <00:06:49.614>have <00:06:49.755>what <00:06:49.895>we <00:06:50.034>call <00:06:50.174>streaming <00:06:50.315>and <00:06:50.455>non <00:06:50.594>swimming <00:06:50.734>steps.

141
00:06:53.085 --> 00:06:54.945
<00:06:53.085>For, <00:06:53.284>for <00:06:53.484>example, <00:06:53.685>if <00:06:53.885>I <00:06:54.085>come <00:06:54.284>back <00:06:54.484>here,

142
00:06:55.375 --> 00:06:59.105
<00:06:55.375>this <00:06:55.491>can <00:06:55.608>be <00:06:55.725>shown <00:06:55.841>token <00:06:55.958>by <00:06:56.075>token <00:06:56.191>word <00:06:56.308>by <00:06:56.425>word <00:06:56.541>to <00:06:56.658>the <00:06:56.775>user.

143
00:06:59.805 --> 00:07:03.025
<00:06:59.805>But <00:07:00.005>the <00:07:00.205>step, <00:07:00.405>the, <00:07:00.605>the, <00:07:00.805>the <00:07:01.005>standalone <00:07:01.205>question,

144
00:07:03.025 --> 00:07:05.745
<00:07:03.025>the <00:07:03.200>generation <00:07:03.375>is <00:07:03.549>not <00:07:03.724>something <00:07:03.900>that <00:07:04.075>is <00:07:04.250>user <00:07:04.424>facing.

145
00:07:06.405 --> 00:07:10.425
<00:07:06.405>So <00:07:06.544>everything <00:07:06.684>before <00:07:06.825>that <00:07:06.965>needs <00:07:07.104>to <00:07:07.244>be <00:07:07.385>as <00:07:07.525>fast <00:07:07.664>as <00:07:07.804>possible

146
00:07:10.445 --> 00:07:12.905
<00:07:10.445>to <00:07:10.724>have <00:07:11.005>the <00:07:11.284>best <00:07:11.565>user <00:07:11.844>experience.

147
00:07:14.435 --> 00:07:18.225
<00:07:14.435>And <00:07:14.635>the <00:07:14.835>first, <00:07:15.035>yeah, <00:07:15.235>okay, <00:07:15.435>it's <00:07:15.635>one <00:07:15.835>everywhere.

148
00:07:18.285 --> 00:07:21.305
<00:07:18.285>But <00:07:18.518>the <00:07:18.751>fifth <00:07:18.985>part <00:07:19.218>is <00:07:19.451>sequential <00:07:19.685>steps.

149
00:07:21.765 --> 00:07:24.305
<00:07:21.765>You <00:07:21.998>can't <00:07:22.231>do <00:07:22.465>thing <00:07:22.698>in <00:07:22.931>parallel, <00:07:23.164>meaning

150
00:07:24.305 --> 00:07:26.705
<00:07:24.305>that <00:07:24.480>if <00:07:24.655>one <00:07:24.830>step <00:07:25.005>takes <00:07:25.180>four <00:07:25.355>seconds, <00:07:25.530>five <00:07:25.705>seconds,

151
00:07:27.725 --> 00:07:30.375
<00:07:27.725>your <00:07:27.852>app <00:07:27.979>is <00:07:28.106>going <00:07:28.234>to <00:07:28.361>take <00:07:28.488>at <00:07:28.615>least <00:07:28.743>five <00:07:28.870>seconds <00:07:28.997>to <00:07:29.125>answer.

152
00:07:32.095 --> 00:07:35.455
<00:07:32.095>So <00:07:32.250>now <00:07:32.406>that <00:07:32.561>you <00:07:32.717>know <00:07:32.872>all <00:07:33.028>of <00:07:33.183>that, <00:07:33.339>let's <00:07:33.495>start

153
00:07:35.455 --> 00:07:36.775
<00:07:35.455>with <00:07:35.692>a <00:07:35.930>real <00:07:36.167>world <00:07:36.405>use <00:07:36.643>case.

154
00:07:36.875 --> 00:07:38.415
<00:07:36.875>And <00:07:37.106>this <00:07:37.337>is <00:07:37.568>the <00:07:37.799>story <00:07:38.030>of <00:07:38.261>quiver

155
00:07:38.415 --> 00:07:39.615
<00:07:38.415>and <00:07:38.631>how <00:07:38.847>I've <00:07:39.063>started <00:07:39.279>the <00:07:39.495>project.

156
00:07:40.505 --> 00:07:42.605
<00:07:40.505>And <00:07:40.680>please <00:07:40.855>don't <00:07:41.030>make <00:07:41.205>the <00:07:41.380>same <00:07:41.555>mistake <00:07:41.730>as <00:07:41.905>me.

157
00:07:43.275 --> 00:07:45.575
<00:07:43.275>But <00:07:43.450>if <00:07:43.625>you <00:07:43.799>want <00:07:43.974>to <00:07:44.150>start <00:07:44.325>a <00:07:44.500>bag <00:07:44.674>application,

158
00:07:45.755 --> 00:07:47.175
<00:07:45.755>you <00:07:46.074>can <00:07:46.394>use <00:07:46.713>a <00:07:47.033>ui.

159
00:07:47.395 --> 00:07:50.175
<00:07:47.395>So <00:07:47.534>you're <00:07:47.674>going <00:07:47.815>to <00:07:47.955>use <00:07:48.094>stream, <00:07:48.234>lead <00:07:48.375>or <00:07:48.515>radio <00:07:48.655>for <00:07:48.794>Python

160
00:07:50.635 --> 00:07:54.255
<00:07:50.635>or <00:07:50.775>next <00:07:50.914>GS <00:07:51.055>for, <00:07:51.195>you <00:07:51.335>know, <00:07:51.474>if <00:07:51.615>your <00:07:51.755>TypeScript <00:07:51.895>user, <00:07:52.034>then

161
00:07:54.255 --> 00:07:57.055
<00:07:54.255>for <00:07:54.430>the <00:07:54.605>database, <00:07:54.780>you're <00:07:54.955>probably <00:07:55.130>going <00:07:55.305>to <00:07:55.480>use <00:07:55.655>PostgreSQL

162
00:07:57.055 --> 00:07:58.455
<00:07:57.055>with <00:07:57.370>superb <00:07:57.685>base <00:07:58.000>that <00:07:58.315>manages

163
00:07:58.455 --> 00:07:59.895
<00:07:58.455>authentication <00:07:58.779>and <00:07:59.103>all <00:07:59.426>these <00:07:59.751>things.

164
00:08:00.355 --> 00:08:03.045
<00:08:00.355>And <00:08:00.510>then <00:08:00.666>you're <00:08:00.821>going <00:08:00.977>to <00:08:01.132>choose <00:08:01.288>a <00:08:01.443>framework, <00:08:01.599>for <00:08:01.755>example,

165
00:08:03.155 --> 00:08:06.845
<00:08:03.155>long <00:08:03.271>chain <00:08:03.388>or <00:08:03.505>LAMA <00:08:03.621>index <00:08:03.738>that <00:08:03.854>allow <00:08:03.971>you <00:08:04.088>to <00:08:04.205>do, <00:08:04.321>uh, <00:08:04.438>to <00:08:04.554>tool

166
00:08:06.905 --> 00:08:09.165
<00:08:06.905>or <00:08:07.184>to <00:08:07.465>create <00:08:07.744>generative <00:08:08.025>AI <00:08:08.304>apps.

167
00:08:09.165 --> 00:08:10.805
<00:08:09.165>Simply <00:08:09.631>that's <00:08:10.098>the <00:08:10.565>beginning.

168
00:08:11.545 --> 00:08:14.605
<00:08:11.545>And <00:08:11.720>here <00:08:11.895>you <00:08:12.070>have <00:08:12.245>your <00:08:12.420>first <00:08:12.595>chat <00:08:12.770>bot. <00:08:12.945>Congrats.

169
00:08:15.475 --> 00:08:18.975
<00:08:15.475>So <00:08:15.615>it's <00:08:15.755>very <00:08:15.895>efficient, <00:08:16.035>it <00:08:16.175>doesn't <00:08:16.315>have <00:08:16.455>a <00:08:16.595>lot <00:08:16.735>of <00:08:16.875>packages.

170
00:08:19.595 --> 00:08:23.495
<00:08:19.595>And <00:08:19.735>as <00:08:19.875>you <00:08:20.015>can <00:08:20.155>see <00:08:20.295>here, <00:08:20.435>it <00:08:20.575>only <00:08:20.715>use <00:08:20.855>0.2 <00:08:20.995>CPUs.

171
00:08:24.695 --> 00:08:26.345
<00:08:24.695>Well, <00:08:24.974>you <00:08:25.255>can't <00:08:25.534>split <00:08:25.815>A <00:08:26.094>CPU,

172
00:08:26.445 --> 00:08:29.265
<00:08:26.445>but <00:08:26.620>what <00:08:26.795>happens <00:08:26.969>is <00:08:27.145>in <00:08:27.320>the <00:08:27.495>cloud <00:08:27.670>providers, <00:08:27.844>if

173
00:08:29.905 --> 00:08:33.025
<00:08:29.905>I <00:08:30.032>make <00:08:30.159>a <00:08:30.286>generic <00:08:30.414>a <00:08:30.541>simplification, <00:08:30.668>it <00:08:30.795>means <00:08:30.923>that <00:08:31.050>if <00:08:31.177>you <00:08:31.304>have

174
00:08:33.535 --> 00:08:37.265
<00:08:33.534>five <00:08:33.690>of <00:08:33.846>this <00:08:34.001>running <00:08:34.157>on <00:08:34.312>a <00:08:34.468>simple <00:08:34.623>on <00:08:34.779>one <00:08:34.934>CPU,

175
00:08:37.965 --> 00:08:41.105
<00:08:37.965>12 <00:08:38.165>seconds <00:08:38.365>out <00:08:38.565>of <00:08:38.765>every <00:08:38.965>minute, <00:08:39.165>you're <00:08:39.365>allowed

176
00:08:41.105 --> 00:08:42.265
<00:08:41.105>to <00:08:41.453>use <00:08:41.801>the <00:08:42.149>CPU.

177
00:08:42.905 --> 00:08:45.635
<00:08:42.905>Okay? <00:08:43.060>So <00:08:43.216>that <00:08:43.371>means <00:08:43.527>that <00:08:43.682>I <00:08:43.838>can <00:08:43.993>have <00:08:44.149>five <00:08:44.305>instances

178
00:08:45.695 --> 00:08:48.635
<00:08:45.695>of <00:08:45.850>my <00:08:46.006>app <00:08:46.161>running <00:08:46.317>on <00:08:46.472>one <00:08:46.628>single <00:08:46.783>CPU, <00:08:46.939>very <00:08:47.095>simple.

179
00:08:50.025 --> 00:08:51.565
<00:08:50.025>And <00:08:50.487>the <00:08:50.949>Docker <00:08:51.411>image,

180
00:08:52.065 --> 00:08:54.245
<00:08:52.065>who <00:08:52.265>knows <00:08:52.465>Docker <00:08:52.665>here <00:08:52.865>or <00:08:53.065>who <00:08:53.265>doesn't <00:08:53.465>maybe.

181
00:08:54.635 --> 00:08:57.265
<00:08:54.635>Okay. <00:08:54.915>Okay, <00:08:55.194>so <00:08:55.475>everybody, <00:08:55.755>oh, <00:08:56.035>okay.

182
00:08:57.365 --> 00:08:59.825
<00:08:57.365>So <00:08:57.520>Docker <00:08:57.676>is <00:08:57.831>a <00:08:57.987>way <00:08:58.142>to <00:08:58.298>put <00:08:58.453>everything <00:08:58.609>you <00:08:58.765>need

183
00:09:00.205 --> 00:09:01.745
<00:09:00.205>inside <00:09:00.667>a <00:09:01.129>single <00:09:01.591>container.

184
00:09:02.325 --> 00:09:04.705
<00:09:02.325>And <00:09:02.525>so <00:09:02.725>here <00:09:02.925>it's <00:09:03.125>only <00:09:03.325>half <00:09:03.525>a <00:09:03.725>gigabytes.

185
00:09:04.705 --> 00:09:06.065
<00:09:04.705>So <00:09:05.011>it's <00:09:05.317>very <00:09:05.623>fairly <00:09:05.929>manageable.

186
00:09:06.125 --> 00:09:07.505
<00:09:06.125>You <00:09:06.373>can <00:09:06.621>be <00:09:06.870>installed <00:09:07.118>fairly <00:09:07.367>quickly,

187
00:09:07.825 --> 00:09:11.265
<00:09:07.825>download <00:09:08.000>it <00:09:08.175>very <00:09:08.350>quickly, <00:09:08.525>but <00:09:08.700>then <00:09:08.875>you <00:09:09.050>showcase <00:09:09.225>this

188
00:09:11.325 --> 00:09:14.465
<00:09:11.325>to <00:09:11.441>your <00:09:11.558>boss <00:09:11.675>or <00:09:11.791>you <00:09:11.908>know, <00:09:12.025>to <00:09:12.141>the <00:09:12.258>users <00:09:12.375>and <00:09:12.491>they <00:09:12.608>love <00:09:12.725>it.

189
00:09:14.495 --> 00:09:17.625
<00:09:14.495>They <00:09:14.650>want <00:09:14.806>more. <00:09:14.961>Now <00:09:15.117>they <00:09:15.272>want <00:09:15.428>you <00:09:15.583>to <00:09:15.739>ingest <00:09:15.895>documents.

190
00:09:18.345 --> 00:09:19.675
<00:09:18.345>This <00:09:18.584>is <00:09:18.823>where <00:09:19.063>the <00:09:19.302>film <00:09:19.541>begins.

191
00:09:20.885 --> 00:09:22.865
<00:09:20.885>So <00:09:21.085>in <00:09:21.285>order <00:09:21.485>to <00:09:21.685>ingest <00:09:21.885>documents, <00:09:22.085>you <00:09:22.285>have

192
00:09:22.865 --> 00:09:24.105
<00:09:22.865>to <00:09:23.237>add <00:09:23.609>more <00:09:23.981>things,

193
00:09:24.645 --> 00:09:29.225
<00:09:24.645>but <00:09:24.772>are <00:09:24.899>GPU <00:09:25.026>bound, <00:09:25.154>for <00:09:25.281>example, <00:09:25.408>the <00:09:25.535>OCR, <00:09:25.663>the <00:09:25.790>ability <00:09:25.917>to <00:09:26.045>read

194
00:09:29.765 --> 00:09:32.985
<00:09:29.765>images <00:09:29.920>and <00:09:30.076>extract <00:09:30.231>text <00:09:30.387>from <00:09:30.542>it <00:09:30.698>or <00:09:30.853>information <00:09:31.009>from <00:09:31.165>it.

195
00:09:33.525 --> 00:09:36.865
<00:09:33.525>And <00:09:33.699>this <00:09:33.875>is <00:09:34.050>requires <00:09:34.225>a <00:09:34.400>lot <00:09:34.574>of <00:09:34.750>computing <00:09:34.925>power.

196
00:09:37.245 --> 00:09:38.745
<00:09:37.245>And <00:09:37.437>if <00:09:37.630>you <00:09:37.823>want <00:09:38.016>it <00:09:38.209>to <00:09:38.402>be <00:09:38.595>efficient

197
00:09:38.765 --> 00:09:41.115
<00:09:38.765>and <00:09:38.998>good, <00:09:39.231>you <00:09:39.465>need <00:09:39.698>to <00:09:39.931>have <00:09:40.165>GPUs.

198
00:09:42.675 --> 00:09:46.935
<00:09:42.675>You, <00:09:42.802>uh, <00:09:42.929>also <00:09:43.056>had <00:09:43.184>a, <00:09:43.311>it's <00:09:43.438>a <00:09:43.565>way <00:09:43.693>to <00:09:43.820>improve <00:09:43.947>the <00:09:44.074>quality

199
00:09:47.315 --> 00:09:50.335
<00:09:47.315>of <00:09:47.455>the <00:09:47.595>chunks <00:09:47.735>that <00:09:47.875>I've <00:09:48.015>retrieved, <00:09:48.155>uh, <00:09:48.295>from <00:09:48.435>the <00:09:48.575>Vector <00:09:48.715>store

200
00:09:50.915 --> 00:09:53.655
<00:09:50.915>and <00:09:51.148>then <00:09:51.381>many <00:09:51.615>more <00:09:51.848>components, <00:09:52.081>the <00:09:52.314>embedding,

201
00:09:53.715 --> 00:09:55.015
<00:09:53.715>the, <00:09:54.105>the, <00:09:54.495>the <00:09:54.885>passer.

202
00:09:55.195 --> 00:10:00.175
<00:09:55.195>But <00:09:55.335>what <00:09:55.475>happens <00:09:55.615>here <00:09:55.755>is <00:09:55.895>your <00:09:56.035>Docker <00:09:56.175>image <00:09:56.315>increases <00:09:56.455>in <00:09:56.595>size

203
00:10:01.145 --> 00:10:04.515
<00:10:01.145>because <00:10:01.285>you <00:10:01.425>have <00:10:01.564>to <00:10:01.704>download <00:10:01.845>some <00:10:01.985>models <00:10:02.125>for <00:10:02.265>DOCR, <00:10:02.405>you <00:10:02.545>have

204
00:10:04.515 --> 00:10:05.995
<00:10:04.515>to <00:10:04.781>add <00:10:05.047>a <00:10:05.314>bit <00:10:05.580>more <00:10:05.847>packages.

205
00:10:06.735 --> 00:10:07.875
<00:10:06.735>And <00:10:07.761>also

206
00:10:07.875 --> 00:10:09.115
<00:10:07.875>because <00:10:08.154>you <00:10:08.433>are <00:10:08.712>doing <00:10:08.991>things

207
00:10:09.115 --> 00:10:11.275
<00:10:09.115>that <00:10:09.395>need <00:10:09.675>much <00:10:09.955>more <00:10:10.235>computing <00:10:10.515>power.

208
00:10:11.815 --> 00:10:14.955
<00:10:11.815>Now <00:10:11.970>you <00:10:12.126>need <00:10:12.281>one <00:10:12.437>CPU <00:10:12.592>for <00:10:12.748>your <00:10:12.903>simple <00:10:13.059>chatbot, <00:10:13.215>meaning

209
00:10:14.955 --> 00:10:17.115
<00:10:14.955>that <00:10:15.130>if <00:10:15.305>you <00:10:15.480>want <00:10:15.655>to <00:10:15.830>scale <00:10:16.005>to <00:10:16.180>five <00:10:16.355>instances,

210
00:10:17.855 --> 00:10:19.195
<00:10:17.855>now <00:10:18.156>you <00:10:18.458>need <00:10:18.759>five <00:10:19.061>CPUs

211
00:10:19.195 --> 00:10:21.475
<00:10:19.195>and <00:10:19.395>not <00:10:19.595>one <00:10:19.795>CPU <00:10:19.995>as <00:10:20.195>I <00:10:20.395>mentioned <00:10:20.595>before.

212
00:10:21.985 --> 00:10:23.395
<00:10:21.985>This <00:10:22.238>is <00:10:22.492>where <00:10:22.746>the <00:10:23.000>trouble <00:10:23.254>starts.

213
00:10:25.215 --> 00:10:27.395
<00:10:25.215>Now <00:10:25.415>your <00:10:25.615>boss <00:10:25.815>says, <00:10:26.015>okay, <00:10:26.215>that's <00:10:26.415>very <00:10:26.615>nice,

214
00:10:27.415 --> 00:10:28.835
<00:10:27.415>but <00:10:27.597>now <00:10:27.780>I <00:10:27.962>want <00:10:28.145>to <00:10:28.327>build <00:10:28.510>a <00:10:28.693>chatbot

215
00:10:28.835 --> 00:10:33.635
<00:10:28.835>but <00:10:28.962>can <00:10:29.089>talk <00:10:29.216>to <00:10:29.344>my <00:10:29.471>Gmail <00:10:29.598>account <00:10:29.725>and <00:10:29.853>create <00:10:29.980>meetings <00:10:30.107>for <00:10:30.235>me.

216
00:10:33.975 --> 00:10:35.235
<00:10:33.975>So <00:10:34.164>you <00:10:34.353>add <00:10:34.542>a <00:10:34.731>bit <00:10:34.920>more <00:10:35.109>things,

217
00:10:35.295 --> 00:10:37.035
<00:10:35.295>you <00:10:35.574>add <00:10:35.854>now <00:10:36.135>an <00:10:36.415>orchestration <00:10:36.694>tool.

218
00:10:37.575 --> 00:10:41.555
<00:10:37.575>You <00:10:37.682>also, <00:10:37.790>uh, <00:10:37.898>if <00:10:38.005>you <00:10:38.113>want <00:10:38.221>to <00:10:38.328>chatbot <00:10:38.436>to <00:10:38.544>be <00:10:38.651>able <00:10:38.759>to, <00:10:38.867>to <00:10:38.975>talk

219
00:10:41.625 --> 00:10:45.395
<00:10:41.625>with <00:10:41.765>web <00:10:41.905>pages, <00:10:42.045>meaning <00:10:42.185>that <00:10:42.325>it <00:10:42.465>can <00:10:42.605>do <00:10:42.745>some <00:10:42.885>GS <00:10:43.025>rendering,

220
00:10:45.655 --> 00:10:47.995
<00:10:45.655>you <00:10:45.855>insert <00:10:46.055>permium, <00:10:46.255>that's <00:10:46.454>another <00:10:46.655>gigabytes <00:10:46.855>of <00:10:47.055>data,

221
00:10:48.055 --> 00:10:50.685
<00:10:48.055>but <00:10:48.229>you <00:10:48.405>have <00:10:48.579>to <00:10:48.755>add <00:10:48.930>in <00:10:49.104>your <00:10:49.280>Docker <00:10:49.454>so

222
00:10:51.225 --> 00:10:54.085
<00:10:51.225>you <00:10:51.458>don't <00:10:51.691>add <00:10:51.925>any <00:10:52.158>more <00:10:52.391>computing <00:10:52.625>requirements,

223
00:10:54.145 --> 00:10:56.885
<00:10:54.145>but <00:10:54.319>you <00:10:54.495>add <00:10:54.670>some <00:10:54.845>size <00:10:55.020>to <00:10:55.194>your <00:10:55.370>Docker <00:10:55.545>image

224
00:10:57.105 --> 00:10:58.645
<00:10:57.105>and <00:10:57.382>now <00:10:57.659>you <00:10:57.936>add <00:10:58.213>six <00:10:58.491>gigabytes.

225
00:10:58.985 --> 00:11:00.885
<00:10:58.985>And <00:10:59.140>so <00:10:59.296>at <00:10:59.451>this <00:10:59.607>time <00:10:59.762>you <00:10:59.918>tell <00:11:00.073>your <00:11:00.229>boss, <00:11:00.385>okay,

226
00:11:00.945 --> 00:11:02.405
<00:11:00.945>now <00:11:01.207>it's <00:11:01.470>time <00:11:01.733>to <00:11:01.996>do <00:11:02.259>refactor.

227
00:11:03.755 --> 00:11:06.495
<00:11:03.755>And <00:11:03.895>so <00:11:04.035>what <00:11:04.175>you <00:11:04.314>do <00:11:04.455>is <00:11:04.595>you <00:11:04.735>split <00:11:04.875>your <00:11:05.015>code <00:11:05.155>base.

228
00:11:07.725 --> 00:11:11.705
<00:11:07.725>And <00:11:07.865>what <00:11:08.005>we <00:11:08.145>did <00:11:08.285>at <00:11:08.425>Quiver, <00:11:08.565>for <00:11:08.705>example, <00:11:08.845>is <00:11:08.985>we, <00:11:09.125>everything

229
00:11:11.705 --> 00:11:14.945
<00:11:11.705>that <00:11:11.860>is <00:11:12.016>user <00:11:12.171>facing, <00:11:12.327>that <00:11:12.482>is, <00:11:12.638>you <00:11:12.793>know, <00:11:12.949>have <00:11:13.105>um,

230
00:11:15.665 --> 00:11:16.665
<00:11:15.665>business <00:11:15.964>logic <00:11:16.265>into <00:11:16.564>it.

231
00:11:16.685 --> 00:11:19.505
<00:11:16.685>You <00:11:16.885>put <00:11:17.084>it <00:11:17.285>on <00:11:17.484>a <00:11:17.685>very <00:11:17.885>simple <00:11:18.084>app

232
00:11:19.775 --> 00:11:22.905
<00:11:19.775>with <00:11:19.975>no <00:11:20.175>LLM <00:11:20.375>embedding <00:11:20.574>all <00:11:20.775>of <00:11:20.975>these <00:11:21.175>things

233
00:11:23.325 --> 00:11:27.585
<00:11:23.325>and <00:11:23.465>you <00:11:23.605>put <00:11:23.745>all <00:11:23.885>of <00:11:24.025>the <00:11:24.165>other <00:11:24.305>things <00:11:24.445>into <00:11:24.585>another <00:11:24.725>component

234
00:11:27.585 --> 00:11:30.555
<00:11:27.585>that <00:11:27.785>is <00:11:27.985>a, <00:11:28.185>it's <00:11:28.385>a <00:11:28.585>worker, <00:11:28.785>it's <00:11:28.985>something

235
00:11:30.555 --> 00:11:32.755
<00:11:30.555>that <00:11:30.682>runs <00:11:30.809>in <00:11:30.936>the <00:11:31.064>background <00:11:31.191>and <00:11:31.318>that <00:11:31.445>takes <00:11:31.573>a <00:11:31.700>bit <00:11:31.827>more <00:11:31.954>time.

236
00:11:32.815 --> 00:11:34.395
<00:11:32.815>But <00:11:32.990>that <00:11:33.165>means <00:11:33.340>that <00:11:33.515>now <00:11:33.690>you <00:11:33.865>have <00:11:34.040>to <00:11:34.215>have

237
00:11:34.415 --> 00:11:35.795
<00:11:34.415>to <00:11:35.036>implement <00:11:35.656>notifications.

238
00:11:36.295 --> 00:11:37.555
<00:11:36.295>You <00:11:36.483>have <00:11:36.673>to <00:11:36.862>tell <00:11:37.050>the <00:11:37.240>user, <00:11:37.429>okay,

239
00:11:37.555 --> 00:11:40.395
<00:11:37.555>your <00:11:37.710>document <00:11:37.866>is <00:11:38.021>being <00:11:38.177>passed, <00:11:38.332>it's <00:11:38.488>taking <00:11:38.643>a <00:11:38.799>few <00:11:38.954>minutes,

240
00:11:41.035 --> 00:11:42.195
<00:11:41.035>I <00:11:41.209>can, <00:11:41.383>you'll <00:11:41.557>come <00:11:41.731>back <00:11:41.905>to <00:11:42.079>me.

241
00:11:42.415 --> 00:11:44.035
<00:11:42.415>It <00:11:42.881>complexifies <00:11:43.348>the <00:11:43.814>project.

242
00:11:45.355 --> 00:11:49.095
<00:11:45.355>The <00:11:45.530>fun <00:11:45.705>thing <00:11:45.880>about <00:11:46.055>that <00:11:46.230>is <00:11:46.405>you <00:11:46.580>can <00:11:46.755>manage

243
00:11:49.115 --> 00:11:52.775
<00:11:49.115>to <00:11:49.242>be <00:11:49.369>just <00:11:49.496>the <00:11:49.624>size <00:11:49.751>here <00:11:49.878>so <00:11:50.005>you <00:11:50.133>can <00:11:50.260>answer <00:11:50.387>more <00:11:50.515>questions.

244
00:11:53.385 --> 00:11:56.085
<00:11:53.385>Uh, <00:11:53.560>like, <00:11:53.735>uh, <00:11:53.910>I <00:11:54.085>dunno, <00:11:54.260>what <00:11:54.435>is <00:11:54.610>the <00:11:54.785>username?

245
00:11:56.135 --> 00:11:58.525
<00:11:56.135>Everything <00:11:56.335>that <00:11:56.535>is <00:11:56.735>not <00:11:56.935>code <00:11:57.135>for <00:11:57.335>the <00:11:57.535>LLM

246
00:11:58.865 --> 00:12:02.095
<00:11:58.865>and <00:11:59.005>only <00:11:59.145>this <00:11:59.285>part <00:11:59.425>you <00:11:59.565>can <00:11:59.705>do <00:11:59.845>the <00:11:59.985>long <00:12:00.125>running <00:12:00.265>task.

247
00:12:03.035 --> 00:12:05.735
<00:12:03.035>We <00:12:03.190>did <00:12:03.346>not <00:12:03.501>manage <00:12:03.657>to <00:12:03.812>do <00:12:03.968>that <00:12:04.123>here <00:12:04.279>at <00:12:04.435>Quiver.

248
00:12:05.955 --> 00:12:08.865
<00:12:05.955>We <00:12:06.155>still <00:12:06.355>have <00:12:06.555>a <00:12:06.755>six <00:12:06.955>gigabytes <00:12:07.155>docker <00:12:07.355>container

249
00:12:08.865 --> 00:12:10.945
<00:12:08.865>because <00:12:09.145>the <00:12:09.425>code <00:12:09.705>was <00:12:09.985>so <00:12:10.265>entangled,

250
00:12:11.975 --> 00:12:14.955
<00:12:11.975>but <00:12:12.102>now <00:12:12.229>it <00:12:12.356>makes <00:12:12.484>it <00:12:12.611>for <00:12:12.738>us <00:12:12.865>very <00:12:12.993>hard <00:12:13.120>to <00:12:13.247>to <00:12:13.375>split.

251
00:12:15.675 --> 00:12:17.295
<00:12:15.675>So <00:12:16.025>be <00:12:16.375>careful <00:12:16.724>with <00:12:17.074>that.

252
00:12:18.155 --> 00:12:21.125
<00:12:18.155>Okay, <00:12:18.310>so <00:12:18.466>now <00:12:18.621>your <00:12:18.777>boss, <00:12:18.932>because <00:12:19.088>we <00:12:19.243>are <00:12:19.399>in <00:12:19.555>Europe

253
00:12:21.465 --> 00:12:23.045
<00:12:21.465>and <00:12:21.745>we <00:12:22.025>don't <00:12:22.305>trust <00:12:22.585>the <00:12:22.865>Americans.

254
00:12:23.045 --> 00:12:24.565
<00:12:23.045>Well, <00:12:23.318>the <00:12:23.592>big <00:12:23.865>companies <00:12:24.139>don't <00:12:24.413>trust

255
00:12:24.745 --> 00:12:26.085
<00:12:24.745>OpenAI, <00:12:24.986>for <00:12:25.227>example, <00:12:25.468>for <00:12:25.709>the <00:12:25.951>data.

256
00:12:26.825 --> 00:12:30.775
<00:12:26.825>Now <00:12:27.058>we <00:12:27.291>want <00:12:27.525>to <00:12:27.758>do <00:12:27.991>on-premise, <00:12:28.225>okay,

257
00:12:31.035 --> 00:12:32.375
<00:12:31.035>so <00:12:31.437>on-premise, <00:12:31.839>this <00:12:32.241>is

258
00:12:32.375 --> 00:12:34.815
<00:12:32.375>where <00:12:32.575>the <00:12:32.775>firm <00:12:32.975>begins <00:12:33.175>on-premise <00:12:33.375>is <00:12:33.575>very <00:12:33.775>hard

259
00:12:34.815 --> 00:12:37.335
<00:12:34.815>to <00:12:35.015>have <00:12:35.215>multiple <00:12:35.415>things <00:12:35.615>running <00:12:35.815>on <00:12:36.015>different <00:12:36.215>servers.

260
00:12:37.715 --> 00:12:39.895
<00:12:37.715>So <00:12:37.870>what <00:12:38.026>you <00:12:38.181>do <00:12:38.337>is <00:12:38.492>you <00:12:38.648>create <00:12:38.803>one <00:12:38.959>big <00:12:39.115>app

261
00:12:40.755 --> 00:12:42.505
<00:12:40.755>where <00:12:41.035>there <00:12:41.314>is <00:12:41.595>everything <00:12:41.875>in <00:12:42.155>it,

262
00:12:43.085 --> 00:12:44.825
<00:12:43.085>and <00:12:43.318>that <00:12:43.551>is <00:12:43.785>what <00:12:44.018>the <00:12:44.251>firm <00:12:44.485>begins

263
00:12:44.825 --> 00:12:46.985
<00:12:44.825>because <00:12:44.980>you <00:12:45.136>have <00:12:45.291>to <00:12:45.447>have <00:12:45.602>the <00:12:45.758>LM <00:12:45.913>running, <00:12:46.069>you <00:12:46.225>have

264
00:12:46.985 --> 00:12:49.105
<00:12:46.985>to <00:12:47.160>have <00:12:47.335>the <00:12:47.510>embedding <00:12:47.685>running, <00:12:47.860>you <00:12:48.035>have <00:12:48.210>the <00:12:48.385>women,

265
00:12:49.485 --> 00:12:51.585
<00:12:49.485>the <00:12:49.685>vector <00:12:49.885>database <00:12:50.085>and <00:12:50.285>all <00:12:50.485>of <00:12:50.685>these <00:12:50.885>things.

266
00:12:51.645 --> 00:12:54.265
<00:12:51.645>And <00:12:51.845>so <00:12:52.045>now <00:12:52.245>your <00:12:52.444>app <00:12:52.645>is <00:12:52.845>very <00:12:53.045>big

267
00:12:54.845 --> 00:12:56.755
<00:12:54.845>and <00:12:55.311>this <00:12:55.778>to <00:12:56.245>burn

268
00:12:57.235 --> 00:13:00.915
<00:12:57.235>'cause <00:12:57.410>you <00:12:57.585>have <00:12:57.760>probably <00:12:57.935>$10 <00:12:58.110>an <00:12:58.285>hour, <00:12:58.460>uh, <00:12:58.635>just

269
00:13:00.915 --> 00:13:02.235
<00:13:00.915>for <00:13:01.212>the <00:13:01.509>GPUs <00:13:01.806>for <00:13:02.103>example.

270
00:13:03.825 --> 00:13:07.665
<00:13:03.825>And <00:13:04.058>um, <00:13:04.291>so <00:13:04.525>yeah, <00:13:04.758>so <00:13:04.991>this <00:13:05.225>is

271
00:13:07.665 --> 00:13:10.195
<00:13:07.665>what <00:13:07.944>you <00:13:08.224>should <00:13:08.505>not <00:13:08.785>do, <00:13:09.064>okay?

272
00:13:10.655 --> 00:13:13.955
<00:13:10.655>And <00:13:10.782>so <00:13:10.909>at <00:13:11.036>this <00:13:11.164>time, <00:13:11.291>you, <00:13:11.418>you <00:13:11.545>show, <00:13:11.673>you <00:13:11.800>try, <00:13:11.927>you <00:13:12.055>implement

273
00:13:13.955 --> 00:13:15.275
<00:13:13.955>that, <00:13:14.153>it <00:13:14.351>takes <00:13:14.549>you <00:13:14.747>a <00:13:14.944>few <00:13:15.143>weeks

274
00:13:15.815 --> 00:13:18.195
<00:13:15.815>and <00:13:15.942>now <00:13:16.069>you <00:13:16.196>tell <00:13:16.324>your <00:13:16.451>boss, <00:13:16.578>okay, <00:13:16.705>it <00:13:16.833>doesn't <00:13:16.960>work <00:13:17.087>at <00:13:17.215>all.

275
00:13:18.215 --> 00:13:19.235
<00:13:18.215>We <00:13:18.444>need <00:13:18.674>to <00:13:18.903>split <00:13:19.133>again.

276
00:13:19.655 --> 00:13:22.955
<00:13:19.655>So <00:13:19.771>what <00:13:19.888>you <00:13:20.005>do, <00:13:20.121>and <00:13:20.238>this <00:13:20.355>is <00:13:20.471>what <00:13:20.588>we <00:13:20.704>are <00:13:20.821>doing <00:13:20.938>now, <00:13:21.055>again,

277
00:13:23.425 --> 00:13:26.705
<00:13:23.425>this <00:13:23.580>is <00:13:23.736>real <00:13:23.891>life <00:13:24.047>story, <00:13:24.202>so <00:13:24.358>I'm <00:13:24.513>not <00:13:24.669>inventing <00:13:24.824>anything,

278
00:13:27.325 --> 00:13:31.945
<00:13:27.325>is <00:13:27.525>we <00:13:27.725>splitted <00:13:27.925>the <00:13:28.125>components <00:13:28.325>into <00:13:28.525>multiple <00:13:28.725>services.

279
00:13:32.655 --> 00:13:35.275
<00:13:32.655>So <00:13:32.810>we <00:13:32.966>have <00:13:33.121>what <00:13:33.277>I <00:13:33.432>like <00:13:33.588>to <00:13:33.743>call <00:13:33.899>the <00:13:34.055>things.

280
00:13:35.785 --> 00:13:38.955
<00:13:35.785>It's <00:13:35.925>the <00:13:36.064>things <00:13:36.204>that <00:13:36.344>you <00:13:36.485>know, <00:13:36.625>they <00:13:36.765>can <00:13:36.905>do <00:13:37.045>fairly <00:13:37.185>quickly.

281
00:13:38.955 --> 00:13:41.555
<00:13:38.955>They're <00:13:39.188>just <00:13:39.421>bound <00:13:39.655>to <00:13:39.888>a <00:13:40.121>Postgre <00:13:40.355>database.

282
00:13:42.155 --> 00:13:43.775
<00:13:42.155>And <00:13:42.355>so <00:13:42.555>this <00:13:42.755>is <00:13:42.954>everything <00:13:43.155>on <00:13:43.355>the <00:13:43.555>left.

283
00:13:43.925 --> 00:13:46.895
<00:13:43.925>It's <00:13:44.099>my <00:13:44.275>also <00:13:44.449>chat <00:13:44.625>bots. <00:13:44.800>That's <00:13:44.974>a <00:13:45.150>simple <00:13:45.324>database.

284
00:13:47.395 --> 00:13:50.745
<00:13:47.395>The <00:13:47.675>authentication, <00:13:47.954>then <00:13:48.235>everything <00:13:48.515>that <00:13:48.795>is,

285
00:13:51.485 --> 00:13:53.985
<00:13:51.485>and <00:13:51.685>the <00:13:51.885>word <00:13:52.085>syn <00:13:52.285>is <00:13:52.485>maybe <00:13:52.685>another <00:13:52.885>statement,

286
00:13:54.005 --> 00:13:57.385
<00:13:54.005>but <00:13:54.205>everything <00:13:54.405>that <00:13:54.605>is <00:13:54.805>very <00:13:55.005>slow, <00:13:55.205>for <00:13:55.405>example,

287
00:13:57.405 --> 00:13:59.665
<00:13:57.405>the <00:13:57.685>embedding <00:13:57.964>the, <00:13:58.245>the <00:13:58.525>classifier, <00:13:58.805>the,

288
00:14:00.935 --> 00:14:03.035
<00:14:00.935>you <00:14:01.135>put <00:14:01.334>them <00:14:01.535>in <00:14:01.734>the <00:14:01.935>own <00:14:02.135>service <00:14:02.334>in

289
00:14:03.035 --> 00:14:05.795
<00:14:03.035>that <00:14:03.235>you <00:14:03.435>can <00:14:03.635>scale <00:14:03.834>them <00:14:04.035>more <00:14:04.235>easily <00:14:04.435>alone.

290
00:14:06.415 --> 00:14:08.755
<00:14:06.415>And <00:14:06.615>the <00:14:06.814>same <00:14:07.015>thing <00:14:07.214>for <00:14:07.415>the <00:14:07.615>vector <00:14:07.814>database.

291
00:14:09.495 --> 00:14:14.115
<00:14:09.495>Uh, <00:14:09.635>and <00:14:09.775>for <00:14:09.915>the <00:14:10.055>broker <00:14:10.195>and <00:14:10.335>your <00:14:10.475>passer <00:14:10.615>really <00:14:10.755>this <00:14:10.895>place

292
00:14:14.695 --> 00:14:17.515
<00:14:14.695>and <00:14:14.870>this <00:14:15.045>place <00:14:15.220>are <00:14:15.395>the <00:14:15.570>two <00:14:15.745>things <00:14:15.920>you <00:14:16.095>need

293
00:14:17.515 --> 00:14:18.675
<00:14:17.515>to <00:14:17.776>be <00:14:18.036>very <00:14:18.298>careful <00:14:18.559>about.

294
00:14:19.145 --> 00:14:21.675
<00:14:19.145>It's <00:14:19.319>where <00:14:19.495>most <00:14:19.670>of <00:14:19.845>your <00:14:20.020>computing <00:14:20.194>power <00:14:20.370>will <00:14:20.545>go.

295
00:14:22.645 --> 00:14:26.185
<00:14:22.645>And <00:14:22.772>if <00:14:22.899>you <00:14:23.026>don't <00:14:23.154>optimize <00:14:23.281>while <00:14:23.408>you're <00:14:23.535>up, <00:14:23.663>you <00:14:23.790>won't <00:14:23.917>be <00:14:24.045>able

296
00:14:26.345 --> 00:14:30.385
<00:14:26.345>to <00:14:26.485>scale <00:14:26.625>it <00:14:26.765>easily <00:14:26.905>because <00:14:27.045>everything <00:14:27.185>will <00:14:27.325>be <00:14:27.465>bound <00:14:27.605>in <00:14:27.745>a

297
00:14:30.385 --> 00:14:31.745
<00:14:30.385>single <00:14:31.609>container.

298
00:14:34.055 --> 00:14:36.995
<00:14:34.055>So <00:14:34.229>for <00:14:34.405>me, <00:14:34.579>I <00:14:34.755>think <00:14:34.930>there <00:14:35.104>are <00:14:35.280>three <00:14:35.454>kinds.

299
00:14:37.145 --> 00:14:39.155
<00:14:37.145>Well, <00:14:37.319>there <00:14:37.495>is <00:14:37.670>a <00:14:37.845>new <00:14:38.020>kind <00:14:38.194>of <00:14:38.370>web <00:14:38.545>apps.

300
00:14:39.415 --> 00:14:42.355
<00:14:39.415>If <00:14:39.555>you <00:14:39.694>have <00:14:39.834>a <00:14:39.974>low <00:14:40.115>load, <00:14:40.255>if <00:14:40.395>you <00:14:40.535>have <00:14:40.675>five <00:14:40.814>users,

301
00:14:42.605 --> 00:14:44.435
<00:14:42.605>don't <00:14:42.885>bother <00:14:43.165>about <00:14:43.445>anything <00:14:43.725>I've <00:14:44.005>mentioned.

302
00:14:44.945 --> 00:14:49.675
<00:14:44.945>Just <00:14:45.072>do <00:14:45.199>a <00:14:45.326>simple <00:14:45.454>app, <00:14:45.581>make <00:14:45.708>all <00:14:45.835>of <00:14:45.963>the <00:14:46.090>mistakes <00:14:46.217>I've <00:14:46.345>made

303
00:14:50.135 --> 00:14:52.085
<00:14:50.135>and <00:14:50.275>it's <00:14:50.415>okay, <00:14:50.555>you're <00:14:50.694>going <00:14:50.835>to <00:14:50.975>save <00:14:51.115>a <00:14:51.255>lot <00:14:51.395>of <00:14:51.535>time.

304
00:14:52.855 --> 00:14:54.315
<00:14:52.855>If <00:14:53.117>you <00:14:53.380>have <00:14:53.643>a <00:14:53.906>continuous <00:14:54.169>load

305
00:14:54.415 --> 00:14:56.155
<00:14:54.415>and <00:14:54.615>you <00:14:54.814>don't <00:14:55.015>care <00:14:55.214>so <00:14:55.415>much <00:14:55.615>about <00:14:55.814>money,

306
00:14:57.015 --> 00:14:58.315
<00:14:57.015>simply <00:14:57.307>do <00:14:57.600>the <00:14:57.892>same <00:14:58.185>thing.

307
00:14:58.335 --> 00:15:00.355
<00:14:58.335>As <00:14:58.568>I <00:14:58.801>mentioned, <00:14:59.035>just <00:14:59.268>put <00:14:59.501>everything <00:14:59.735>together.

308
00:15:01.305 --> 00:15:02.925
<00:15:01.305>But <00:15:01.655>if <00:15:02.005>you <00:15:02.354>have <00:15:02.704>high

309
00:15:03.305 --> 00:15:05.725
<00:15:03.305>or <00:15:03.505>despite <00:15:03.704>load, <00:15:03.905>meaning <00:15:04.104>that <00:15:04.305>during <00:15:04.505>the <00:15:04.704>day,

310
00:15:05.725 --> 00:15:06.965
<00:15:05.725>during <00:15:05.948>the <00:15:06.171>night <00:15:06.394>or <00:15:06.617>different <00:15:06.841>times

311
00:15:06.965 --> 00:15:10.245
<00:15:06.965>of <00:15:07.081>day <00:15:07.198>you <00:15:07.315>have <00:15:07.431>a <00:15:07.548>higher <00:15:07.665>load, <00:15:07.781>you <00:15:07.898>will <00:15:08.015>need <00:15:08.131>to <00:15:08.248>be <00:15:08.365>able

312
00:15:10.245 --> 00:15:12.445
<00:15:10.245>to <00:15:10.595>scale <00:15:10.945>different <00:15:11.295>components <00:15:11.645>individually.

313
00:15:13.065 --> 00:15:15.525
<00:15:13.065>And <00:15:13.220>if <00:15:13.376>you <00:15:13.531>did <00:15:13.687>not <00:15:13.842>speak <00:15:13.998>to <00:15:14.153>infras <00:15:14.309>your <00:15:14.465>components

314
00:15:16.145 --> 00:15:19.525
<00:15:16.145>in <00:15:16.285>different <00:15:16.425>services, <00:15:16.564>you're <00:15:16.704>going <00:15:16.845>to <00:15:16.985>have <00:15:17.125>a <00:15:17.265>lot <00:15:17.405>of <00:15:17.545>issues

315
00:15:19.585 --> 00:15:22.525
<00:15:19.585>or <00:15:19.725>it's <00:15:19.865>going <00:15:20.005>to <00:15:20.145>cost <00:15:20.285>you <00:15:20.425>a <00:15:20.565>lot <00:15:20.705>of <00:15:20.845>computing <00:15:20.985>power

316
00:15:22.985 --> 00:15:26.885
<00:15:22.985>and <00:15:23.218>AWS <00:15:23.451>credits <00:15:23.685>or <00:15:23.918>Asia <00:15:24.151>or <00:15:24.385>GCP.

317
00:15:28.405 --> 00:15:30.745
<00:15:28.405>So <00:15:28.755>my <00:15:29.105>recommendation <00:15:29.454>for <00:15:29.805>you

318
00:15:31.725 --> 00:15:33.945
<00:15:31.725>is <00:15:31.958>gene <00:15:32.191>AI <00:15:32.425>is <00:15:32.658>still <00:15:32.891>very <00:15:33.125>new.

319
00:15:34.735 --> 00:15:37.065
<00:15:34.735>Like <00:15:35.015>this <00:15:35.295>thing <00:15:35.575>changing <00:15:35.855>every <00:15:36.135>day.

320
00:15:37.955 --> 00:15:42.035
<00:15:37.955>Don't <00:15:38.095>sell <00:15:38.235>force <00:15:38.375>on <00:15:38.515>premise. <00:15:38.655>Okay? <00:15:38.795>We <00:15:38.935>are <00:15:39.075>not <00:15:39.215>there <00:15:39.355>yet.

321
00:15:42.185 --> 00:15:45.235
<00:15:42.185>It's <00:15:42.324>very <00:15:42.464>hard <00:15:42.604>to <00:15:42.744>do <00:15:42.885>the <00:15:43.025>infrastructures. <00:15:43.165>I'm <00:15:43.305>not <00:15:43.444>here <00:15:43.584>yet.

322
00:15:45.335 --> 00:15:47.755
<00:15:45.335>And <00:15:45.510>what <00:15:45.685>I <00:15:45.860>mean <00:15:46.035>don't <00:15:46.210>sell <00:15:46.385>force <00:15:46.560>on <00:15:46.735>premise.

323
00:15:48.255 --> 00:15:52.115
<00:15:48.255>I'm <00:15:48.488>mostly <00:15:48.721>talking <00:15:48.955>about <00:15:49.188>use <00:15:49.421>hosted <00:15:49.655>models.

324
00:15:53.435 --> 00:15:55.345
<00:15:53.435>Don't <00:15:53.785>burn <00:15:54.135>your <00:15:54.484>models <00:15:54.834>yourself.

325
00:15:55.855 --> 00:15:57.145
<00:15:55.855>It's <00:15:56.048>going <00:15:56.242>to <00:15:56.435>cost <00:15:56.629>you <00:15:56.822>a <00:15:57.016>lot.

326
00:15:57.215 --> 00:16:00.025
<00:15:57.215>It's <00:15:57.370>very <00:15:57.526>hard <00:15:57.681>to <00:15:57.837>scale <00:15:57.992>if <00:15:58.148>you <00:15:58.303>have <00:15:58.459>multiple <00:15:58.615>models

327
00:16:00.025 --> 00:16:01.145
<00:16:00.025>running <00:16:00.276>at <00:16:00.529>the <00:16:00.781>same <00:16:01.033>time.

328
00:16:01.565 --> 00:16:03.665
<00:16:01.565>For <00:16:01.740>example, <00:16:01.915>you <00:16:02.090>have <00:16:02.265>multiple <00:16:02.440>version <00:16:02.615>of <00:16:02.790>one <00:16:02.965>model

329
00:16:03.665 --> 00:16:04.865
<00:16:03.665>because <00:16:03.845>you <00:16:04.025>have <00:16:04.204>fine <00:16:04.385>tuning <00:16:04.565>your <00:16:04.745>model

330
00:16:04.865 --> 00:16:06.105
<00:16:04.865>and <00:16:05.144>you <00:16:05.423>want <00:16:05.702>to <00:16:05.981>evaluate.

331
00:16:06.415 --> 00:16:08.265
<00:16:06.415>That <00:16:06.615>means <00:16:06.814>that <00:16:07.015>you <00:16:07.214>have <00:16:07.415>to <00:16:07.615>have <00:16:07.814>five

332
00:16:08.825 --> 00:16:10.505
<00:16:08.825>LLM <00:16:09.058>models <00:16:09.291>running <00:16:09.525>at <00:16:09.758>the <00:16:09.991>same <00:16:10.225>time,

333
00:16:10.565 --> 00:16:11.905
<00:16:10.565>for <00:16:10.866>example, <00:16:11.168>in <00:16:11.469>your <00:16:11.771>infrastructure.

334
00:16:12.295 --> 00:16:16.785
<00:16:12.295>That <00:16:12.402>cost <00:16:12.510>a <00:16:12.618>lot. <00:16:12.725>So <00:16:12.833>use <00:16:12.941>APIs <00:16:13.048>as <00:16:13.156>much <00:16:13.264>as <00:16:13.371>you <00:16:13.479>can <00:16:13.587>for <00:16:13.694>all

335
00:16:16.785 --> 00:16:18.825
<00:16:16.785>of <00:16:16.985>the <00:16:17.185>things <00:16:17.385>I've <00:16:17.584>said <00:16:17.785>are <00:16:17.985>GPU <00:16:18.185>bound.

336
00:16:19.825 --> 00:16:23.755
<00:16:19.825>But <00:16:19.980>if <00:16:20.136>you <00:16:20.291>have <00:16:20.447>to <00:16:20.602>do <00:16:20.758>a <00:16:20.913>service <00:16:21.069>architecture, <00:16:21.225>not

337
00:16:23.755 --> 00:16:24.755
<00:16:23.755>to <00:16:24.655>microservices,

338
00:16:24.755 --> 00:16:26.515
<00:16:24.755>because <00:16:25.221>microservices <00:16:25.688>need <00:16:26.155>you

339
00:16:26.515 --> 00:16:28.275
<00:16:26.515>to <00:16:26.865>have <00:16:27.215>very <00:16:27.564>small <00:16:27.915>containers,

340
00:16:28.945 --> 00:16:33.195
<00:16:28.945>but <00:16:29.085>try <00:16:29.225>to <00:16:29.365>separate <00:16:29.505>the <00:16:29.645>components <00:16:29.785>of <00:16:29.925>your <00:16:30.065>app <00:16:30.205>into <00:16:30.345>services

341
00:16:33.705 --> 00:16:37.595
<00:16:33.705>like <00:16:33.860>I-F-D-L-L-M <00:16:34.016>service, <00:16:34.171>but <00:16:34.327>is <00:16:34.482>outside <00:16:34.638>of <00:16:34.793>the <00:16:34.949>code <00:16:35.105>base.

342
00:16:37.595 --> 00:16:41.005
<00:16:37.595>There <00:16:37.750>is <00:16:37.906>no <00:16:38.061>dependency <00:16:38.217>between <00:16:38.372>them <00:16:38.528>and <00:16:38.683>all <00:16:38.839>of <00:16:38.995>that.

343
00:16:41.275 --> 00:16:43.525
<00:16:41.275>It's <00:16:41.415>too <00:16:41.555>small <00:16:41.694>to <00:16:41.834>call <00:16:41.975>it <00:16:42.115>to <00:16:42.255>mi, <00:16:42.395>it's <00:16:42.535>too <00:16:42.675>big

344
00:16:43.525 --> 00:16:44.605
<00:16:43.525>to <00:16:43.768>call <00:16:44.011>it <00:16:44.254>to <00:16:44.497>microservice.

345
00:16:45.145 --> 00:16:47.525
<00:16:45.145>But <00:16:45.378>if <00:16:45.611>you <00:16:45.845>know <00:16:46.078>about <00:16:46.311>microservices, <00:16:46.545>try

346
00:16:47.525 --> 00:16:49.085
<00:16:47.525>to <00:16:47.758>apply <00:16:47.991>the <00:16:48.225>same <00:16:48.458>concept <00:16:48.691>to <00:16:48.925>it.

347
00:16:50.465 --> 00:16:52.285
<00:16:50.465>And <00:16:50.815>mutualize <00:16:51.165>with <00:16:51.515>other <00:16:51.865>apps.

348
00:16:52.875 --> 00:16:56.255
<00:16:52.875>If <00:16:53.030>you <00:16:53.186>have, <00:16:53.341>for <00:16:53.497>example, <00:16:53.652>a <00:16:53.808>LAMA <00:16:53.963>running <00:16:54.119>locally, <00:16:54.275>try

349
00:16:56.255 --> 00:16:58.815
<00:16:56.255>to <00:16:56.410>mutualize <00:16:56.566>it <00:16:56.721>with <00:16:56.877>many <00:16:57.032>more <00:16:57.188>apps <00:16:57.343>in <00:16:57.499>your <00:16:57.655>company,

350
00:16:59.315 --> 00:17:01.095
<00:16:59.315>you're <00:16:59.490>not <00:16:59.665>going <00:16:59.840>to <00:17:00.015>use <00:17:00.190>it <00:17:00.365>all <00:17:00.540>the <00:17:00.715>time.

351
00:17:03.285 --> 00:17:08.065
<00:17:03.285>And <00:17:03.440>because <00:17:03.596>at <00:17:03.751>this <00:17:03.907>time <00:17:04.062>you <00:17:04.218>are <00:17:04.373>going <00:17:04.529>to <00:17:04.685>probably

352
00:17:08.575 --> 00:17:11.905
<00:17:08.575>have <00:17:08.750>optimized <00:17:08.925>your <00:17:09.100>app, <00:17:09.275>there's <00:17:09.450>going <00:17:09.625>to <00:17:09.800>be <00:17:09.975>somebody

353
00:17:11.905 --> 00:17:14.025
<00:17:11.905>that <00:17:12.185>made <00:17:12.464>something <00:17:12.744>with <00:17:13.024>your <00:17:13.305>app.

354
00:17:14.685 --> 00:17:18.465
<00:17:14.685>And <00:17:14.801>I <00:17:14.918>just <00:17:15.034>want <00:17:15.151>to <00:17:15.268>showcase <00:17:15.385>you <00:17:15.501>that <00:17:15.618>you <00:17:15.734>can <00:17:15.851>do <00:17:15.968>a <00:17:16.085>lot

355
00:17:18.465 --> 00:17:19.705
<00:17:18.464>of <00:17:18.743>things <00:17:19.022>with <00:17:19.301>gen <00:17:19.580>ai

356
00:17:20.165 --> 00:17:22.745
<00:17:20.165>and <00:17:20.305>at <00:17:20.444>one <00:17:20.585>time <00:17:20.724>you <00:17:20.865>probably <00:17:21.004>have <00:17:21.145>somebody <00:17:21.284>that <00:17:21.425>is <00:17:21.565>going

357
00:17:22.745 --> 00:17:25.385
<00:17:22.744>to <00:17:22.900>be <00:17:23.056>something <00:17:23.211>and <00:17:23.367>break <00:17:23.522>you <00:17:23.678>up <00:17:23.833>in <00:17:23.989>a <00:17:24.145>matter

358
00:17:25.385 --> 00:17:28.305
<00:17:25.385>and <00:17:25.512>a <00:17:25.639>half <00:17:25.766>an <00:17:25.894>hour <00:17:26.021>because <00:17:26.148>it's <00:17:26.275>going <00:17:26.403>to <00:17:26.530>dedos <00:17:26.657>your <00:17:26.785>app.

359
00:17:28.645 --> 00:17:30.505
<00:17:28.645>And <00:17:28.845>so <00:17:29.045>this <00:17:29.245>is <00:17:29.445>something, <00:17:29.645>something <00:17:29.845>I <00:17:30.045>made

360
00:17:30.505 --> 00:17:31.505
<00:17:30.505>during <00:17:30.955>your <00:17:31.405>hackathon

361
00:17:32.245 --> 00:17:35.545
<00:17:32.245>and <00:17:32.445>it's <00:17:32.645>an <00:17:32.844>a <00:17:33.044>way <00:17:33.245>to <00:17:33.445>evaluate <00:17:33.645>s

362
00:17:35.645 --> 00:17:38.905
<00:17:35.645>and <00:17:35.752>just <00:17:35.860>to <00:17:35.968>show <00:17:36.075>you <00:17:36.183>what <00:17:36.291>you <00:17:36.398>can <00:17:36.506>do <00:17:36.614>with <00:17:36.721>gene <00:17:36.829>ai, <00:17:36.937>uh, <00:17:37.045>no,

363
00:17:38.905 --> 00:17:41.205
<00:17:38.905>sorry, <00:17:39.605>wrong <00:17:40.305>slide.

364
00:17:41.365 --> 00:17:44.755
<00:17:41.365>I <00:17:41.565>I'll <00:17:41.765>talk <00:17:41.965>about <00:17:42.165>it <00:17:42.365>later. <00:17:42.565>Uh, <00:17:42.765>yeah,

365
00:17:44.795 --> 00:17:46.635
<00:17:44.795>there <00:17:44.995>are <00:17:45.195>things <00:17:45.395>I <00:17:45.595>did <00:17:45.795>not <00:17:45.995>talk <00:17:46.195>about.

366
00:17:46.935 --> 00:17:48.995
<00:17:46.935>The <00:17:47.168>first <00:17:47.401>thing <00:17:47.635>is <00:17:47.868>GPU <00:17:48.101>on <00:17:48.335>Kubernetes.

367
00:17:49.685 --> 00:17:50.985
<00:17:49.685>Having <00:17:50.074>GPUs <00:17:50.465>on <00:17:50.855>the,

368
00:17:51.165 --> 00:17:53.505
<00:17:51.165>on <00:17:51.340>the <00:17:51.514>Kubernetes <00:17:51.690>cluster <00:17:51.865>is <00:17:52.040>very <00:17:52.215>hard <00:17:52.389>to <00:17:52.565>maintain.

369
00:17:53.505 --> 00:17:54.705
<00:17:53.505>It's <00:17:53.775>very <00:17:54.045>hard <00:17:54.315>to <00:17:54.585>deploy.

370
00:17:55.375 --> 00:17:58.265
<00:17:55.375>It's, <00:17:55.515>you <00:17:55.655>need <00:17:55.795>to <00:17:55.935>have <00:17:56.075>people <00:17:56.215>that <00:17:56.355>are <00:17:56.495>working <00:17:56.635>full <00:17:56.775>time.

371
00:17:58.425 --> 00:17:59.425
<00:17:58.425>I <00:17:58.605>used <00:17:58.784>to <00:17:58.965>be <00:17:59.145>a <00:17:59.325>DevOps.

372
00:17:59.935 --> 00:18:03.265
<00:17:59.935>I've <00:18:00.135>only <00:18:00.335>once <00:18:00.534>deployed, <00:18:00.735>uh, <00:18:00.935>Kubernetes <00:18:01.135>with <00:18:01.335>GPUs.

373
00:18:03.885 --> 00:18:07.045
<00:18:03.885>And <00:18:04.351>yeah, <00:18:04.818>it's, <00:18:05.285>it's,

374
00:18:07.345 --> 00:18:10.485
<00:18:07.345>you <00:18:07.485>can't <00:18:07.625>have <00:18:07.765>multiple <00:18:07.905>apps <00:18:08.045>on <00:18:08.185>the <00:18:08.325>same <00:18:08.465>GPU, <00:18:08.605>meaning <00:18:08.745>that,

375
00:18:10.585 --> 00:18:13.445
<00:18:10.585>the <00:18:10.740>thing <00:18:10.896>that <00:18:11.051>I <00:18:11.207>mentioned <00:18:11.362>with <00:18:11.518>the, <00:18:11.673>the <00:18:11.829>0.2 <00:18:11.985>CPU,

376
00:18:13.625 --> 00:18:15.045
<00:18:13.625>you <00:18:13.880>can't <00:18:14.136>have <00:18:14.391>that <00:18:14.647>with <00:18:14.903>GPUs.

377
00:18:15.905 --> 00:18:17.845
<00:18:15.905>So <00:18:16.185>that <00:18:16.465>makes <00:18:16.745>it <00:18:17.024>very <00:18:17.305>expensive

378
00:18:17.845 --> 00:18:19.485
<00:18:17.845>to <00:18:18.195>scale <00:18:18.545>the <00:18:18.895>model <00:18:19.245>versioning.

379
00:18:19.765 --> 00:18:23.045
<00:18:19.765>I <00:18:19.920>mentioned <00:18:20.076>it. <00:18:20.231>Hey, <00:18:20.387>I'm <00:18:20.542>not <00:18:20.698>even <00:18:20.853>talking <00:18:21.009>about <00:18:21.165>multimodal.

380
00:18:23.305 --> 00:18:26.005
<00:18:23.305>So <00:18:23.480>having <00:18:23.655>multiple <00:18:23.830>like <00:18:24.005>models <00:18:24.180>that <00:18:24.355>can <00:18:24.530>ingest <00:18:24.705>video

381
00:18:26.065 --> 00:18:29.085
<00:18:26.065>and <00:18:26.240>billing <00:18:26.415>and <00:18:26.590>same <00:18:26.765>thing, <00:18:26.940>very <00:18:27.115>hard <00:18:27.290>traffic <00:18:27.465>spikes,

382
00:18:29.345 --> 00:18:30.885
<00:18:29.345>vector <00:18:29.807>databases <00:18:30.269>at <00:18:30.731>scale.

383
00:18:31.145 --> 00:18:35.965
<00:18:31.145>But <00:18:31.300>if <00:18:31.456>you <00:18:31.611>have <00:18:31.767>questions, <00:18:31.922>there's <00:18:32.078>a <00:18:32.233>guy <00:18:32.389>from <00:18:32.545>uh,

384
00:18:36.155 --> 00:18:39.085
<00:18:36.155>backups <00:18:36.330>and <00:18:36.504>GDPR <00:18:36.680>because <00:18:36.855>you're <00:18:37.030>ingesting <00:18:37.205>data <00:18:37.379>from <00:18:37.555>people.

385
00:18:39.145 --> 00:18:41.165
<00:18:39.145>But <00:18:39.425>private <00:18:39.705>data, <00:18:39.985>you <00:18:40.264>also <00:18:40.545>have

386
00:18:41.165 --> 00:18:42.605
<00:18:41.165>to <00:18:41.350>take <00:18:41.535>care <00:18:41.720>of <00:18:41.905>this, <00:18:42.090>uh, <00:18:42.275>the <00:18:42.461>backup.

387
00:18:43.145 --> 00:18:45.685
<00:18:43.145>You <00:18:43.378>have <00:18:43.611>to <00:18:43.845>save <00:18:44.078>terabytes <00:18:44.311>of <00:18:44.545>data

388
00:18:46.185 --> 00:18:47.605
<00:18:46.185>and <00:18:46.367>make <00:18:46.550>sure <00:18:46.732>that <00:18:46.915>there <00:18:47.097>is <00:18:47.280>a <00:18:47.463>cycle

389
00:18:47.825 --> 00:18:49.045
<00:18:47.825>for <00:18:48.099>deleting <00:18:48.374>all <00:18:48.648>of <00:18:48.923>this.

390
00:18:49.465 --> 00:18:52.165
<00:18:49.465>And <00:18:49.620>then <00:18:49.776>testing. <00:18:49.931>How <00:18:50.087>do <00:18:50.242>you <00:18:50.398>test <00:18:50.553>an <00:18:50.709>LLM <00:18:50.865>app?

391
00:18:52.515 --> 00:18:54.325
<00:18:52.515>This <00:18:52.690>is <00:18:52.865>something <00:18:53.040>that <00:18:53.215>is <00:18:53.390>very <00:18:53.565>hard <00:18:53.740>to <00:18:53.915>do

392
00:18:54.325 --> 00:18:57.445
<00:18:54.325>because <00:18:54.480>it's <00:18:54.636>not <00:18:54.791>deterministic, <00:18:54.947>meaning <00:18:55.102>that <00:18:55.258>if <00:18:55.413>you <00:18:55.569>put <00:18:55.725>one

393
00:18:57.445 --> 00:19:00.285
<00:18:57.445>as <00:18:57.600>input, <00:18:57.756>sometimes <00:18:57.911>you <00:18:58.067>have <00:18:58.222>one, <00:18:58.378>sometimes <00:18:58.533>you <00:18:58.689>have <00:18:58.845>zero,

394
00:19:00.285 --> 00:19:01.845
<00:19:00.285>sometimes <00:19:00.518>you <00:19:00.751>have <00:19:00.985>15, <00:19:01.218>you <00:19:01.451>don't <00:19:01.685>know.

395
00:19:01.845 --> 00:19:02.925
<00:19:01.845>So <00:19:02.039>it's <00:19:02.233>very <00:19:02.428>hard <00:19:02.622>to <00:19:02.817>test.

396
00:19:03.425 --> 00:19:06.805
<00:19:03.425>And <00:19:03.600>so <00:19:03.774>the <00:19:03.950>testing <00:19:04.125>infrastructure, <00:19:04.300>usually <00:19:04.475>if <00:19:04.649>you <00:19:04.825>want

397
00:19:06.805 --> 00:19:10.095
<00:19:06.805>to <00:19:06.945>pay <00:19:07.085>open <00:19:07.225>AI <00:19:07.365>hundreds <00:19:07.505>of <00:19:07.645>dollars <00:19:07.785>a <00:19:07.925>day <00:19:08.065>for <00:19:08.205>testing,

398
00:19:10.595 --> 00:19:11.735
<00:19:10.595>you <00:19:10.851>deploy <00:19:11.108>your <00:19:11.364>own <00:19:11.620>model.

399
00:19:11.995 --> 00:19:13.615
<00:19:11.995>But <00:19:12.274>then <00:19:12.554>it's <00:19:12.834>the <00:19:13.114>same <00:19:13.395>issues.

400
00:19:14.885 --> 00:19:17.945
<00:19:14.885>And <00:19:15.012>now <00:19:15.139>it's <00:19:15.266>the <00:19:15.394>thing <00:19:15.521>that <00:19:15.648>I <00:19:15.775>built <00:19:15.903>in <00:19:16.030>the <00:19:16.157>weekend <00:19:16.285>to

401
00:19:18.465 --> 00:19:20.465
<00:19:18.465>evaluate <00:19:18.698>lms, <00:19:18.931>it <00:19:19.165>just <00:19:19.398>to <00:19:19.631>showcase <00:19:19.865>you.

402
00:19:21.295 --> 00:19:23.915
<00:19:21.295>But <00:19:21.528>anything <00:19:21.761>is <00:19:21.995>possible <00:19:22.228>with <00:19:22.461>LLM <00:19:22.695>and,

403
00:19:23.915 --> 00:19:25.755
<00:19:23.915>but <00:19:24.090>the <00:19:24.264>goal <00:19:24.440>here <00:19:24.615>is <00:19:24.790>really <00:19:24.965>to <00:19:25.139>have <00:19:25.315>fun

404
00:19:25.755 --> 00:19:27.675
<00:19:25.755>because <00:19:25.988>things <00:19:26.221>are <00:19:26.455>evolving <00:19:26.688>all <00:19:26.921>the <00:19:27.155>time.

405
00:19:33.295 --> 00:19:35.845
<00:19:33.295>Oh, <00:19:33.761>I <00:19:34.228>don't <00:19:34.695>know.

406
00:19:38.135 --> 00:19:41.145
<00:19:38.135>Yeah, <00:19:38.601>it's <00:19:39.068>a <00:19:39.535>video.

407
00:19:45.885 --> 00:19:48.015
<00:19:45.885>Yeah, <00:19:46.085>you <00:19:46.285>need <00:19:46.485>to <00:19:46.685>press <00:19:46.885>play, <00:19:47.085>I <00:19:47.285>think.

408
00:19:50.875 --> 00:19:55.625
<00:19:50.875>Sorry. <00:19:51.030>So <00:19:51.186>here <00:19:51.341>what <00:19:51.497>happens <00:19:51.652>is, <00:19:51.808>uh, <00:19:51.963>during <00:19:52.119>ral <00:19:52.275>ACA

409
00:19:55.775 --> 00:19:58.705
<00:19:55.775>what <00:19:55.915>we <00:19:56.055>did <00:19:56.195>is <00:19:56.335>we <00:19:56.475>realized <00:19:56.615>that <00:19:56.755>there's <00:19:56.895>no <00:19:57.035>real <00:19:57.175>way

410
00:19:58.705 --> 00:20:01.665
<00:19:58.705>to <00:19:58.938>evaluate <00:19:59.171>the, <00:19:59.405>the <00:19:59.638>capabilities <00:19:59.871>of <00:20:00.105>LLMs.

411
00:20:02.145 --> 00:20:05.365
<00:20:02.145>So <00:20:02.320>what <00:20:02.495>we <00:20:02.670>did, <00:20:02.845>instead <00:20:03.020>of <00:20:03.195>using, <00:20:03.370>you <00:20:03.545>know,

412
00:20:05.585 --> 00:20:07.805
<00:20:05.585>how <00:20:05.785>good <00:20:05.985>an <00:20:06.185>LLM <00:20:06.385>is <00:20:06.585>at <00:20:06.785>answering <00:20:06.985>questions,

413
00:20:08.435 --> 00:20:11.845
<00:20:08.435>what <00:20:08.551>we <00:20:08.668>did <00:20:08.784>is <00:20:08.901>we <00:20:09.018>made <00:20:09.135>the <00:20:09.251>LLM <00:20:09.368>fight <00:20:09.485>on <00:20:09.601>Street <00:20:09.718>fighter <00:20:09.835>in

414
00:20:11.845 --> 00:20:15.665
<00:20:11.845>real <00:20:11.985>time, <00:20:12.125>meaning <00:20:12.265>that <00:20:12.405>we <00:20:12.545>took <00:20:12.685>the <00:20:12.825>image <00:20:12.965>of <00:20:13.105>the <00:20:13.245>LLM

415
00:20:16.085 --> 00:20:17.625
<00:20:16.085>and <00:20:16.362>like <00:20:16.639>we <00:20:16.916>took <00:20:17.193>the <00:20:17.471>image

416
00:20:17.625 --> 00:20:22.625
<00:20:17.625>of <00:20:18.325>Sweet <00:20:19.025>Fighter

417
00:20:23.705 --> 00:20:25.685
<00:20:23.705>So <00:20:23.905>we <00:20:24.105>took <00:20:24.304>the, <00:20:24.504>we <00:20:24.705>took <00:20:24.905>the <00:20:25.105>image

418
00:20:26.305 --> 00:20:30.445
<00:20:26.305>and <00:20:26.421>we <00:20:26.538>analyzed <00:20:26.655>all <00:20:26.771>of <00:20:26.888>the, <00:20:27.005>the <00:20:27.121>things <00:20:27.238>that <00:20:27.355>are <00:20:27.471>in <00:20:27.588>the <00:20:27.705>image

419
00:20:30.445 --> 00:20:31.925
<00:20:30.445>and <00:20:30.635>we <00:20:30.825>sent <00:20:31.015>that <00:20:31.206>to <00:20:31.396>the <00:20:31.586>LM <00:20:31.777>saying,

420
00:20:32.155 --> 00:20:34.005
<00:20:32.155>your <00:20:32.435>advi <00:20:32.715>is <00:20:32.995>on <00:20:33.274>your <00:20:33.555>left.

421
00:20:34.305 --> 00:20:36.205
<00:20:34.305>The <00:20:34.480>last <00:20:34.655>move <00:20:34.830>they <00:20:35.005>did <00:20:35.180>is <00:20:35.355>right <00:20:35.530>on <00:20:35.705>that,

422
00:20:36.665 --> 00:20:41.165
<00:20:36.665>but <00:20:36.820>allowed <00:20:36.976>us <00:20:37.131>to <00:20:37.287>measure <00:20:37.442>the <00:20:37.598>capabilities <00:20:37.753>of <00:20:37.909>lms, <00:20:38.065>meaning

423
00:20:41.195 --> 00:20:44.165
<00:20:41.195>that <00:20:41.395>for <00:20:41.595>most <00:20:41.794>use <00:20:41.995>cases <00:20:42.195>we <00:20:42.395>found <00:20:42.595>out

424
00:20:44.315 --> 00:20:46.885
<00:20:44.315>that <00:20:44.515>small <00:20:44.715>models <00:20:44.915>are <00:20:45.115>better <00:20:45.315>than <00:20:45.515>big <00:20:45.715>models

425
00:20:47.235 --> 00:20:50.245
<00:20:47.235>Because <00:20:47.409>big <00:20:47.584>models <00:20:47.760>take <00:20:47.935>a <00:20:48.110>longer <00:20:48.284>time <00:20:48.459>to <00:20:48.635>answer

426
00:20:50.665 --> 00:20:52.925
<00:20:50.665>and <00:20:51.014>they're <00:20:51.365>not <00:20:51.715>particularly <00:20:52.065>waste

427
00:20:52.925 --> 00:20:54.325
<00:20:52.925>matter <00:20:53.240>for <00:20:53.555>most <00:20:53.870>use <00:20:54.185>cases.

428
00:20:55.025 --> 00:20:57.775
<00:20:55.025>So <00:20:55.225>last <00:20:55.425>advice <00:20:55.625>I <00:20:55.825>can <00:20:56.025>give <00:20:56.225>you <00:20:56.425>not

429
00:20:57.775 --> 00:20:59.575
<00:20:57.775>to <00:20:57.975>make <00:20:58.175>the <00:20:58.375>same <00:20:58.575>mistakes <00:20:58.775>as <00:20:58.975>I <00:20:59.175>did.

430
00:21:00.355 --> 00:21:03.815
<00:21:00.355>Use <00:21:00.530>small <00:21:00.705>models <00:21:00.880>first, <00:21:01.055>especially <00:21:01.230>if <00:21:01.405>you <00:21:01.580>self <00:21:01.755>them

431
00:21:04.035 --> 00:21:05.575
<00:21:04.035>and <00:21:04.381>then <00:21:04.728>use <00:21:05.074>bigger <00:21:05.421>models

432
00:21:05.925 --> 00:21:08.855
<00:21:05.925>because <00:21:06.274>the <00:21:06.625>quality <00:21:06.975>doesn't <00:21:07.325>increase

433
00:21:08.855 --> 00:21:10.655
<00:21:08.855>with <00:21:09.135>the <00:21:09.415>complexity <00:21:09.695>of <00:21:09.975>hosting <00:21:10.255>them.

434
00:21:14.505 --> 00:21:16.565
<00:21:14.505>And <00:21:14.738>that's <00:21:14.971>the <00:21:15.205>end <00:21:15.438>of <00:21:15.671>my <00:21:15.905>presentation.

435
00:21:16.585 --> 00:21:19.665
<00:21:16.585>If <00:21:16.725>you <00:21:16.865>have <00:21:17.005>any <00:21:17.145>questions, <00:21:17.285>please <00:21:17.425>free <00:21:17.565>to <00:21:17.705>ask. <00:21:17.845>And <00:21:17.985>yeah.