WEBVTT

1
00:00:05.595 --> 00:00:08.055
<00:00:05.595>Now <00:00:05.770>it <00:00:05.944>seems <00:00:06.119>like <00:00:06.295>everyone <00:00:06.470>is <00:00:06.645>running <00:00:06.819>AI <00:00:06.994>models

2
00:00:08.115 --> 00:00:09.135
<00:00:08.115>in <00:00:08.344>one <00:00:08.574>form <00:00:08.803>or <00:00:09.033>another.

3
00:00:09.275 --> 00:00:10.335
<00:00:09.275>As <00:00:09.752>developers <00:00:10.229>here,

4
00:00:10.465 --> 00:00:13.455
<00:00:10.465>we're <00:00:10.665>constantly <00:00:10.865>using <00:00:11.065>AI <00:00:11.265>models <00:00:11.465>in <00:00:11.665>our <00:00:11.865>co-pilots,

5
00:00:13.455 --> 00:00:15.335
<00:00:13.455>in <00:00:13.688>our <00:00:13.921>ides, <00:00:14.155>right <00:00:14.388>to <00:00:14.621>write <00:00:14.855>code

6
00:00:15.335 --> 00:00:16.935
<00:00:15.335>and <00:00:15.685>ship <00:00:16.035>code <00:00:16.385>faster, <00:00:16.735>right?

7
00:00:17.235 --> 00:00:20.295
<00:00:17.235>But <00:00:17.390>maybe <00:00:17.546>it's <00:00:17.701>in <00:00:17.857>your <00:00:18.012>own <00:00:18.168>internal <00:00:18.323>organization <00:00:18.479>through <00:00:18.634>rag

8
00:00:20.515 --> 00:00:22.535
<00:00:20.515>or <00:00:20.748>through <00:00:20.981>question <00:00:21.215>and <00:00:21.448>answering <00:00:21.681>over <00:00:21.915>documents

9
00:00:22.535 --> 00:00:23.895
<00:00:22.535>that <00:00:22.779>you're <00:00:23.024>using <00:00:23.269>these <00:00:23.514>AI <00:00:23.759>models.

10
00:00:24.075 --> 00:00:25.255
<00:00:24.075>But <00:00:24.340>there's <00:00:24.605>a <00:00:24.871>new <00:00:25.137>trend

11
00:00:25.255 --> 00:00:27.255
<00:00:25.255>that <00:00:25.430>we're <00:00:25.605>gonna <00:00:25.779>be <00:00:25.955>talking <00:00:26.130>about <00:00:26.305>in <00:00:26.480>today's <00:00:26.654>session,

12
00:00:27.265 --> 00:00:30.335
<00:00:27.265>which <00:00:27.498>is <00:00:27.731>self-hosting <00:00:27.965>AI <00:00:28.198>models <00:00:28.431>and <00:00:28.665>LLMs.

13
00:00:30.755 --> 00:00:32.335
<00:00:30.755>And <00:00:30.988>this <00:00:31.221>is <00:00:31.455>a <00:00:31.688>really <00:00:31.921>important <00:00:32.155>topic

14
00:00:32.335 --> 00:00:34.735
<00:00:32.335>that <00:00:32.475>we're <00:00:32.615>gonna <00:00:32.755>cover <00:00:32.895>today <00:00:33.035>and <00:00:33.175>show <00:00:33.315>you <00:00:33.455>with <00:00:33.595>some <00:00:33.735>demos

15
00:00:35.045 --> 00:00:38.295
<00:00:35.045>because <00:00:35.220>there's <00:00:35.395>no, <00:00:35.570>you <00:00:35.745>know, <00:00:35.920>why <00:00:36.095>send <00:00:36.270>your <00:00:36.445>data

16
00:00:38.295 --> 00:00:39.415
<00:00:38.295>to <00:00:38.547>a <00:00:38.799>third <00:00:39.051>party <00:00:39.303>service

17
00:00:39.515 --> 00:00:42.175
<00:00:39.515>or <00:00:39.631>pay <00:00:39.748>for <00:00:39.865>someone <00:00:39.981>else <00:00:40.098>to <00:00:40.215>run <00:00:40.331>an <00:00:40.448>LLM <00:00:40.565>when <00:00:40.681>you <00:00:40.798>could <00:00:40.915>do

18
00:00:42.175 --> 00:00:43.815
<00:00:42.175>so <00:00:42.455>on <00:00:42.735>your <00:00:43.015>own <00:00:43.294>hardware, <00:00:43.574>right?

19
00:00:44.155 --> 00:00:47.175
<00:00:44.155>And <00:00:44.310>so <00:00:44.466>me <00:00:44.621>and <00:00:44.777>Roberto <00:00:44.932>all <00:00:45.088>day, <00:00:45.243>we <00:00:45.399>help <00:00:45.555>developers

20
00:00:47.195 --> 00:00:49.135
<00:00:47.195>and <00:00:47.475>organizations <00:00:47.755>host <00:00:48.035>their <00:00:48.315>own <00:00:48.595>models

21
00:00:49.235 --> 00:00:51.695
<00:00:49.235>and <00:00:49.435>integrate <00:00:49.635>that <00:00:49.835>into, <00:00:50.035>uh, <00:00:50.235>you <00:00:50.435>know, <00:00:50.635>Linux.

22
00:00:51.695 --> 00:00:53.815
<00:00:51.695>Whether <00:00:51.870>it's <00:00:52.045>uh, <00:00:52.220>you <00:00:52.395>know, <00:00:52.570>whether <00:00:52.745>it's <00:00:52.920>a <00:00:53.095>VM

23
00:00:53.835 --> 00:00:55.495
<00:00:53.835>or <00:00:54.068>say <00:00:54.301>scale <00:00:54.535>it <00:00:54.768>up <00:00:55.001>with <00:00:55.235>Kubernetes.

24
00:00:55.675 --> 00:00:58.015
<00:00:55.675>And <00:00:55.830>we're <00:00:55.986>excited <00:00:56.141>to <00:00:56.297>show <00:00:56.452>you <00:00:56.608>with, <00:00:56.763>uh, <00:00:56.919>a <00:00:57.074>lot

25
00:00:58.015 --> 00:00:59.495
<00:00:58.015>of <00:00:58.281>fun <00:00:58.547>live <00:00:58.814>demos, <00:00:59.080>right? <00:00:59.346>And

26
00:00:59.495 --> 00:01:00.495
<00:00:59.495>Live <00:00:59.720>demos <00:00:59.945>As <00:01:00.169>well. <00:01:00.394>Live

27
00:01:00.495 --> 00:01:02.815
<00:01:00.495>demos, <00:01:00.728>how <00:01:00.961>we, <00:01:01.195>uh, <00:01:01.428>what <00:01:01.661>we've <00:01:01.894>learned

28
00:01:02.875 --> 00:01:05.215
<00:01:02.875>and <00:01:03.050>how <00:01:03.225>we <00:01:03.400>do <00:01:03.575>so <00:01:03.750>using <00:01:03.925>open <00:01:04.099>source <00:01:04.275>technology.

29
00:01:05.275 --> 00:01:06.495
<00:01:05.275>So <00:01:05.494>we're <00:01:05.714>gonna <00:01:05.933>give <00:01:06.153>you <00:01:06.373>resources

30
00:01:06.675 --> 00:01:09.735
<00:01:06.675>and <00:01:06.815>repos <00:01:06.955>that <00:01:07.095>you <00:01:07.235>can <00:01:07.375>use <00:01:07.515>today <00:01:07.655>to <00:01:07.795>self-host <00:01:07.935>your <00:01:08.075>models

31
00:01:09.755 --> 00:01:11.415
<00:01:09.755>and <00:01:10.035>go <00:01:10.315>from <00:01:10.595>zero <00:01:10.875>to <00:01:11.155>inference.

32
00:01:11.955 --> 00:01:13.375
<00:01:11.955>So <00:01:12.210>let's <00:01:12.466>kick <00:01:12.721>it <00:01:12.977>off. <00:01:13.233>Alright,

33
00:01:13.635 --> 00:01:16.095
<00:01:13.635>So <00:01:13.762>we <00:01:13.889>have <00:01:14.016>a <00:01:14.144>lot <00:01:14.271>of <00:01:14.398>ground <00:01:14.525>to <00:01:14.653>go, <00:01:14.780>isn't <00:01:14.907>it? <00:01:15.035>We

34
00:01:16.095 --> 00:01:17.215
<00:01:16.095>Do <00:01:16.296>have <00:01:16.498>a <00:01:16.699>lot <00:01:16.901>to <00:01:17.103>cover.

35
00:01:17.515 --> 00:01:20.335
<00:01:17.515>Um, <00:01:17.655>so <00:01:17.795>the <00:01:17.935>new <00:01:18.075>trend, <00:01:18.215>right, <00:01:18.355>as <00:01:18.495>I <00:01:18.635>talked <00:01:18.775>about, <00:01:18.915>is

36
00:01:20.865 --> 00:01:23.575
<00:01:20.865>being <00:01:21.020>able <00:01:21.176>to <00:01:21.331>run <00:01:21.487>your <00:01:21.642>own <00:01:21.798>open <00:01:21.953>source <00:01:22.109>models <00:01:22.265>locally.

37
00:01:23.675 --> 00:01:25.655
<00:01:23.675>And <00:01:23.850>this <00:01:24.024>is <00:01:24.200>really <00:01:24.375>important <00:01:24.550>because <00:01:24.725>instead <00:01:24.899>of <00:01:25.075>having

38
00:01:25.655 --> 00:01:28.775
<00:01:25.655>to <00:01:25.771>send <00:01:25.888>your <00:01:26.005>data <00:01:26.121>to <00:01:26.238>a <00:01:26.355>third <00:01:26.471>party, <00:01:26.588>API, <00:01:26.705>you <00:01:26.821>can <00:01:26.938>send <00:01:27.055>that

39
00:01:28.775 --> 00:01:29.855
<00:01:28.775>to <00:01:29.018>local <00:01:29.261>host <00:01:29.504>and <00:01:29.747>get

40
00:01:29.855 --> 00:01:31.215
<00:01:29.855>that <00:01:30.059>answer <00:01:30.263>back, <00:01:30.467>which <00:01:30.671>is <00:01:30.875>really <00:01:31.079>nice.

41
00:01:31.555 --> 00:01:33.375
<00:01:31.555>And <00:01:31.755>there's <00:01:31.955>a <00:01:32.155>lot <00:01:32.355>of <00:01:32.555>open <00:01:32.755>source <00:01:32.955>projects

42
00:01:33.375 --> 00:01:35.095
<00:01:33.375>that <00:01:33.575>have <00:01:33.775>been <00:01:33.975>developed <00:01:34.175>in <00:01:34.375>the <00:01:34.575>past <00:01:34.775>year

43
00:01:35.195 --> 00:01:37.135
<00:01:35.195>and <00:01:35.395>few <00:01:35.595>years <00:01:35.794>to <00:01:35.994>help <00:01:36.195>you <00:01:36.395>do <00:01:36.595>so.

44
00:01:37.325 --> 00:01:39.455
<00:01:37.325>Some <00:01:37.500>logos <00:01:37.675>that <00:01:37.850>you <00:01:38.025>might <00:01:38.200>see <00:01:38.375>that <00:01:38.550>are <00:01:38.725>familiar.

45
00:01:40.115 --> 00:01:42.615
<00:01:40.115>The <00:01:40.270>thing <00:01:40.426>is, <00:01:40.581>Roberto, <00:01:40.737>what <00:01:40.892>are <00:01:41.048>we <00:01:41.203>gonna <00:01:41.359>pick, <00:01:41.515>right?

46
00:01:42.615 --> 00:01:44.335
<00:01:42.615>Yeah, <00:01:42.965>there's <00:01:43.315>over <00:01:43.665>a <00:01:44.015>million

47
00:01:44.435 --> 00:01:47.135
<00:01:44.435>and <00:01:44.562>a <00:01:44.689>half <00:01:44.816>models <00:01:44.944>now <00:01:45.071>on <00:01:45.198>hugging <00:01:45.325>face <00:01:45.453>that <00:01:45.580>are <00:01:45.707>open <00:01:45.835>sourced

48
00:01:47.365 --> 00:01:48.895
<00:01:47.365>that <00:01:47.594>you <00:01:47.824>can <00:01:48.053>choose <00:01:48.283>and <00:01:48.512>run <00:01:48.742>locally.

49
00:01:49.395 --> 00:01:50.975
<00:01:49.395>But <00:01:49.595>which <00:01:49.795>one <00:01:49.995>should <00:01:50.195>you <00:01:50.395>choose <00:01:50.595>and <00:01:50.795>why?

50
00:01:50.975 --> 00:01:53.255
<00:01:50.975>Because <00:01:51.255>let's <00:01:51.535>say <00:01:51.815>you <00:01:52.095>have <00:01:52.375>PDFs

51
00:01:53.255 --> 00:01:54.735
<00:01:53.255>or <00:01:53.445>you <00:01:53.635>have <00:01:53.825>a <00:01:54.016>code <00:01:54.206>base, <00:01:54.396>you <00:01:54.587>need

52
00:01:54.735 --> 00:01:55.895
<00:01:54.735>to <00:01:54.943>integrate <00:01:55.152>that <00:01:55.361>into <00:01:55.570>the <00:01:55.779>model.

53
00:01:56.395 --> 00:01:58.695
<00:01:56.395>And <00:01:56.595>with <00:01:56.795>the <00:01:56.995>demo <00:01:57.195>and <00:01:57.395>the <00:01:57.595>session <00:01:57.795>today,

54
00:01:58.695 --> 00:02:00.015
<00:01:58.695>we're <00:01:58.843>going <00:01:58.991>to <00:01:59.140>show <00:01:59.289>you <00:01:59.437>how <00:01:59.586>to <00:01:59.734>do <00:01:59.883>that.

55
00:02:00.235 --> 00:02:02.975
<00:02:00.235>So, <00:02:00.390>uh, <00:02:00.546>feel <00:02:00.701>free <00:02:00.857>to <00:02:01.012>check <00:02:01.168>out <00:02:01.323>the <00:02:01.479>slides <00:02:01.635>here

56
00:02:02.975 --> 00:02:05.855
<00:02:02.975>where <00:02:03.091>we're <00:02:03.208>gonna <00:02:03.324>talk <00:02:03.441>about <00:02:03.558>how <00:02:03.675>to <00:02:03.791>run <00:02:03.908>your <00:02:04.024>own <00:02:04.141>l <00:02:04.258>lms, <00:02:04.375>um,

57
00:02:06.035 --> 00:02:07.655
<00:02:06.035>how <00:02:06.268>to <00:02:06.501>choose <00:02:06.735>that <00:02:06.968>model <00:02:07.201>and <00:02:07.435>integrate

58
00:02:07.655 --> 00:02:08.855
<00:02:07.655>that <00:02:07.925>into <00:02:08.195>your <00:02:08.465>existing <00:02:08.734>data.

59
00:02:09.275 --> 00:02:11.735
<00:02:09.275>And <00:02:09.475>we're <00:02:09.675>gonna <00:02:09.875>show <00:02:10.075>three, <00:02:10.275>three <00:02:10.475>demos, <00:02:10.675>three

60
00:02:11.735 --> 00:02:13.335
<00:02:11.735>Demos, <00:02:12.085>live <00:02:12.435>demos, <00:02:12.785>fingers <00:02:13.135>crossed,

61
00:02:13.545 --> 00:02:14.545
<00:02:13.545>Knock <00:02:13.845>on <00:02:14.144>wood. <00:02:14.445>And

62
00:02:14.545 --> 00:02:15.775
<00:02:14.545>we're <00:02:14.729>gonna <00:02:14.914>have <00:02:15.098>some <00:02:15.283>fun <00:02:15.467>with <00:02:15.652>it.

63
00:02:16.455 --> 00:02:17.535
<00:02:16.454>Absolutely. <00:02:17.427>Cool.

64
00:02:18.315 --> 00:02:19.935
<00:02:18.315>Oh, <00:02:18.470>we <00:02:18.626>can <00:02:18.781>keep <00:02:18.937>it <00:02:19.092>up <00:02:19.248>here <00:02:19.403>for <00:02:19.559>a <00:02:19.715>second.

65
00:02:19.995 --> 00:02:22.695
<00:02:19.995>If, <00:02:20.195>if <00:02:20.395>you <00:02:20.595>didn't <00:02:20.795>get <00:02:20.995>it. <00:02:21.195>Sorry. <00:02:21.395>Sorry.

66
00:02:23.595 --> 00:02:25.535
<00:02:23.595>Um, <00:02:23.734>and <00:02:23.875>yeah, <00:02:24.015>the <00:02:24.155>last <00:02:24.295>one <00:02:24.435>is <00:02:24.575>gonna <00:02:24.715>be <00:02:24.855>really <00:02:24.995>fun

67
00:02:25.535 --> 00:02:27.655
<00:02:25.535>because <00:02:25.734>we're <00:02:25.935>gonna <00:02:26.135>actually <00:02:26.335>show, <00:02:26.535>you <00:02:26.734>know, <00:02:26.935>how

68
00:02:27.655 --> 00:02:29.655
<00:02:27.655>to <00:02:27.935>start <00:02:28.215>building <00:02:28.495>a <00:02:28.775>agent <00:02:29.055>capabilities

69
00:02:29.725 --> 00:02:30.935
<00:02:29.725>with <00:02:29.942>a <00:02:30.160>local <00:02:30.378>model <00:02:30.596>as <00:02:30.814>well.

70
00:02:31.065 --> 00:02:33.535
<00:02:31.065>Every <00:02:31.345>Yeah, <00:02:31.625>yeah, <00:02:31.905>everything, <00:02:32.185>everything <00:02:32.465>local.

71
00:02:33.605 --> 00:02:36.255
<00:02:33.605>Yeah, <00:02:33.804>a <00:02:34.005>hundred <00:02:34.204>percent <00:02:34.405>live. <00:02:34.605>All <00:02:34.804>right. <00:02:35.005>Alright,

72
00:02:36.265 --> 00:02:37.265
<00:02:36.265>Let's <00:02:36.565>kick <00:02:36.864>it <00:02:37.165>off.

73
00:02:37.485 --> 00:02:38.775
<00:02:37.485>Cool. <00:02:37.872>So <00:02:38.259>one <00:02:38.646>question.

74
00:02:39.035 --> 00:02:41.535
<00:02:39.035>Why <00:02:39.268>everyone <00:02:39.501>is <00:02:39.734>running <00:02:39.968>their <00:02:40.201>own <00:02:40.435>A

75
00:02:41.535 --> 00:02:42.975
<00:02:41.535>models, <00:02:41.859>why <00:02:42.183>should <00:02:42.507>they <00:02:42.831>care?

76
00:02:43.165 --> 00:02:46.655
<00:02:43.165>Yeah, <00:02:43.340>that's <00:02:43.515>a <00:02:43.690>great <00:02:43.864>question. <00:02:44.040>Um, <00:02:44.215>oh, <00:02:44.390>oh, <00:02:44.565>sorry.

77
00:02:47.845 --> 00:02:50.815
<00:02:47.845>There's <00:02:48.045>a <00:02:48.245>couple <00:02:48.445>reasons, <00:02:48.645>especially <00:02:48.845>as <00:02:49.045>developers <00:02:49.245>here,

78
00:02:50.815 --> 00:02:52.415
<00:02:50.815>that <00:02:50.990>you <00:02:51.165>would <00:02:51.340>wanna <00:02:51.515>run <00:02:51.690>your <00:02:51.865>own <00:02:52.040>model, <00:02:52.215>right?

79
00:02:52.675 --> 00:02:55.775
<00:02:52.675>The <00:02:52.815>biggest <00:02:52.955>thing <00:02:53.095>is <00:02:53.235>that <00:02:53.375>kind <00:02:53.515>of <00:02:53.655>inner <00:02:53.795>loop <00:02:53.935>of <00:02:54.075>development,

80
00:02:55.965 --> 00:02:59.965
<00:02:55.965>similar <00:02:56.092>to <00:02:56.219>how, <00:02:56.346>for <00:02:56.474>example, <00:02:56.601>I <00:02:56.728>can, <00:02:56.855>you <00:02:56.983>know, <00:02:57.110>run <00:02:57.237>a <00:02:57.365>MongoDB

81
00:02:59.965 --> 00:03:02.245
<00:02:59.965>or <00:03:00.198>Postgres <00:03:00.431>database <00:03:00.665>locally <00:03:00.898>in <00:03:01.131>a <00:03:01.365>container.

82
00:03:02.245 --> 00:03:04.405
<00:03:02.245>Well, <00:03:02.400>we <00:03:02.556>can <00:03:02.711>do <00:03:02.867>the <00:03:03.022>same <00:03:03.178>thing <00:03:03.333>with <00:03:03.489>a <00:03:03.645>container,

83
00:03:04.435 --> 00:03:07.325
<00:03:04.435>with <00:03:04.562>an <00:03:04.689>AI <00:03:04.816>model <00:03:04.944>with <00:03:05.071>an <00:03:05.198>LLM <00:03:05.325>from <00:03:05.453>hugging <00:03:05.580>face, <00:03:05.707>for <00:03:05.835>example.

84
00:03:07.705 --> 00:03:09.405
<00:03:07.705>And <00:03:07.905>this <00:03:08.105>is, <00:03:08.305>you <00:03:08.505>know, <00:03:08.705>a <00:03:08.905>lot <00:03:09.105>easier

85
00:03:09.465 --> 00:03:10.685
<00:03:09.465>to <00:03:09.831>integrate <00:03:10.197>my <00:03:10.563>applications

86
00:03:10.945 --> 00:03:12.365
<00:03:10.945>or <00:03:11.200>connect <00:03:11.456>to <00:03:11.711>my <00:03:11.967>local <00:03:12.223>resources

87
00:03:12.365 --> 00:03:14.965
<00:03:12.365>because <00:03:12.520>I'm <00:03:12.676>not <00:03:12.831>dealing <00:03:12.987>with <00:03:13.142>an <00:03:13.298>API <00:03:13.453>or <00:03:13.609>authentication <00:03:13.765>token.

88
00:03:15.385 --> 00:03:17.805
<00:03:15.385>And <00:03:15.540>it <00:03:15.696>allows <00:03:15.851>me <00:03:16.007>a <00:03:16.162>lot <00:03:16.318>of <00:03:16.473>customization <00:03:16.629>and <00:03:16.785>control.

89
00:03:18.225 --> 00:03:21.205
<00:03:18.225>And <00:03:18.364>for <00:03:18.505>organizations <00:03:18.644>that <00:03:18.785>want <00:03:18.924>to <00:03:19.065>run <00:03:19.204>kind <00:03:19.345>of <00:03:19.484>a <00:03:19.625>model

90
00:03:21.265 --> 00:03:24.765
<00:03:21.265>as <00:03:21.420>a <00:03:21.576>service <00:03:21.731>internally <00:03:21.887>for <00:03:22.042>their <00:03:22.198>users, <00:03:22.353>well <00:03:22.509>they <00:03:22.665>have

91
00:03:24.765 --> 00:03:26.205
<00:03:24.765>that <00:03:25.196>complete <00:03:25.629>data <00:03:26.060>privacy

92
00:03:26.425 --> 00:03:28.365
<00:03:26.425>and <00:03:26.658>governance <00:03:26.891>over <00:03:27.125>the <00:03:27.358>models <00:03:27.591>they <00:03:27.825>choose

93
00:03:28.625 --> 00:03:30.005
<00:03:28.625>and <00:03:28.873>they're <00:03:29.121>not <00:03:29.370>forced <00:03:29.618>to <00:03:29.867>upgrade.

94
00:03:30.105 --> 00:03:32.165
<00:03:30.105>Say <00:03:30.304>for <00:03:30.505>example, <00:03:30.704>when <00:03:30.905>Chad <00:03:31.105>GBT <00:03:31.304>moves <00:03:31.505>from

95
00:03:32.165 --> 00:03:33.325
<00:03:32.165>four <00:03:32.512>to <00:03:32.861>five, <00:03:33.209>right?

96
00:03:33.325 --> 00:03:37.805
<00:03:33.325>They <00:03:33.464>can <00:03:33.605>keep <00:03:33.744>what <00:03:33.885>they <00:03:34.024>have <00:03:34.165>running, <00:03:34.304>uh, <00:03:34.445>locally <00:03:34.584>thing <00:03:34.725>is

97
00:03:38.385 --> 00:03:39.385
<00:03:38.385>We <00:03:38.685>have <00:03:38.984>a <00:03:39.285>lot.

98
00:03:39.435 --> 00:03:41.405
<00:03:39.435>There's <00:03:39.715>a <00:03:39.995>lot <00:03:40.275>isn't <00:03:40.555>it? <00:03:40.835>Right?

99
00:03:41.505 --> 00:03:43.845
<00:03:41.505>And <00:03:41.704>actually <00:03:41.905>this <00:03:42.105>is <00:03:42.305>last <00:03:42.505>year's <00:03:42.704>machine <00:03:42.905>learning

100
00:03:43.845 --> 00:03:47.165
<00:03:43.845>and <00:03:44.020>AI <00:03:44.195>data <00:03:44.370>landscape <00:03:44.545>for <00:03:44.720>everything <00:03:44.895>from <00:03:45.070>inferencing <00:03:45.245>models

101
00:03:47.305 --> 00:03:49.165
<00:03:47.305>to <00:03:47.585>training <00:03:47.865>models <00:03:48.145>to <00:03:48.425>monitoring <00:03:48.705>models.

102
00:03:49.275 --> 00:03:51.605
<00:03:49.275>Yeah. <00:03:49.450>And <00:03:49.625>I <00:03:49.800>mean, <00:03:49.975>I <00:03:50.150>don't <00:03:50.325>know <00:03:50.500>about <00:03:50.675>you,

103
00:03:52.045 --> 00:03:54.245
<00:03:52.045>I <00:03:52.245>dunno, <00:03:52.445>I <00:03:52.645>mean <00:03:52.845>our <00:03:53.045>boss <00:03:53.245>wants <00:03:53.445>everything.

104
00:03:54.785 --> 00:03:56.085
<00:03:54.785>He <00:03:55.077>wants <00:03:55.370>everything. <00:03:55.662>I <00:03:55.955>kind

105
00:03:56.085 --> 00:03:57.325
<00:03:56.085>of <00:03:56.271>feel <00:03:56.457>like <00:03:56.643>this <00:03:56.829>some <00:03:57.015>days, <00:03:57.201>right?

106
00:03:57.355 --> 00:03:59.165
<00:03:57.355>Yeah. <00:03:57.704>Well <00:03:58.054>the <00:03:58.405>thing <00:03:58.755>is,

107
00:03:59.585 --> 00:04:02.045
<00:03:59.585>you <00:03:59.725>don't <00:03:59.865>wanna <00:04:00.005>end <00:04:00.145>up <00:04:00.285>like <00:04:00.425>Roberto <00:04:00.565>here <00:04:00.705>who <00:04:00.845>is <00:04:00.985>trying

108
00:04:02.045 --> 00:04:05.085
<00:04:02.045>to <00:04:02.184>figure <00:04:02.325>out <00:04:02.464>man, <00:04:02.605>how <00:04:02.744>to <00:04:02.885>work <00:04:03.024>with <00:04:03.165>the <00:04:03.304>prompt <00:04:03.445>templates,

109
00:04:05.145 --> 00:04:07.645
<00:04:05.145>how <00:04:05.378>to, <00:04:05.611>uh, <00:04:05.845>manage <00:04:06.078>his <00:04:06.311>models, <00:04:06.545>how

110
00:04:07.645 --> 00:04:09.685
<00:04:07.645>to <00:04:07.820>compress <00:04:07.995>l <00:04:08.170>LMS <00:04:08.345>so <00:04:08.520>they <00:04:08.695>can <00:04:08.870>run <00:04:09.045>efficiently.

111
00:04:09.945 --> 00:04:11.765
<00:04:09.945>So <00:04:10.120>that's <00:04:10.295>why <00:04:10.470>we're <00:04:10.644>gonna <00:04:10.820>give <00:04:10.995>you <00:04:11.170>the <00:04:11.345>resources

112
00:04:11.765 --> 00:04:15.285
<00:04:11.765>that <00:04:11.881>you <00:04:11.998>need <00:04:12.114>to <00:04:12.231>today <00:04:12.348>in <00:04:12.464>order <00:04:12.581>to, <00:04:12.698>uh, <00:04:12.815>run <00:04:12.931>and <00:04:13.048>manage <00:04:13.165>LLMs

113
00:04:15.285 --> 00:04:16.405
<00:04:15.285>and <00:04:15.486>put <00:04:15.688>them <00:04:15.889>in <00:04:16.091>production. <00:04:16.292>Yeah,

114
00:04:16.445 --> 00:04:18.045
<00:04:16.445>I <00:04:16.644>just <00:04:16.844>as <00:04:17.045>random <00:04:17.245>developer <00:04:17.445>that <00:04:17.644>I <00:04:17.844>want

115
00:04:18.045 --> 00:04:20.605
<00:04:18.045>to <00:04:18.185>use <00:04:18.325>AI <00:04:18.465>without <00:04:18.605>my <00:04:18.745>applications, <00:04:18.885>how <00:04:19.025>can <00:04:19.165>I <00:04:19.305>do <00:04:19.445>it?

116
00:04:20.935 --> 00:04:23.805
<00:04:20.935>Let's <00:04:21.090>see. <00:04:21.246>So, <00:04:21.401>you <00:04:21.557>know, <00:04:21.712>we <00:04:21.868>work <00:04:22.023>at <00:04:22.179>Red <00:04:22.335>Hat,

117
00:04:23.865 --> 00:04:27.045
<00:04:23.865>red <00:04:24.040>Hat's <00:04:24.215>completely <00:04:24.390>open <00:04:24.565>source, <00:04:24.740>um, <00:04:24.915>and, <00:04:25.090>and <00:04:25.265>upstream.

118
00:04:27.145 --> 00:04:28.925
<00:04:27.145>So <00:04:27.320>we <00:04:27.495>work <00:04:27.669>on <00:04:27.844>a <00:04:28.020>lot <00:04:28.195>of <00:04:28.370>different <00:04:28.544>projects

119
00:04:28.925 --> 00:04:30.085
<00:04:28.925>to <00:04:29.133>help <00:04:29.342>us <00:04:29.551>run <00:04:29.760>these <00:04:29.969>models.

120
00:04:30.345 --> 00:04:32.885
<00:04:30.345>Um, <00:04:30.625>who <00:04:30.905>here <00:04:31.185>identifies <00:04:31.465>this <00:04:31.745>game?

121
00:04:32.985 --> 00:04:34.165
<00:04:32.985>Who <00:04:33.197>knows <00:04:33.409>what <00:04:33.622>game <00:04:33.834>this <00:04:34.047>is?

122
00:04:34.785 --> 00:04:36.885
<00:04:34.784>All <00:04:34.984>right, <00:04:35.184>my <00:04:35.385>people, <00:04:35.585>my <00:04:35.784>people, <00:04:35.984>all <00:04:36.184>right.

123
00:04:36.885 --> 00:04:38.085
<00:04:36.885>This <00:04:37.155>is <00:04:37.425>Super <00:04:37.695>Smash <00:04:37.965>Bros.

124
00:04:38.265 --> 00:04:39.885
<00:04:38.265>And <00:04:38.544>my <00:04:38.825>favorite <00:04:39.104>game <00:04:39.385>for <00:04:39.664>sure

125
00:04:39.885 --> 00:04:41.765
<00:04:39.885>because <00:04:40.040>you <00:04:40.196>can <00:04:40.351>choose <00:04:40.507>to <00:04:40.662>be <00:04:40.818>any <00:04:40.973>of <00:04:41.129>these <00:04:41.284>characters

126
00:04:42.065 --> 00:04:43.725
<00:04:42.065>and <00:04:42.298>they <00:04:42.531>all <00:04:42.765>have <00:04:42.998>a <00:04:43.231>different <00:04:43.465>superpower,

127
00:04:44.485 --> 00:04:46.285
<00:04:44.485>a <00:04:44.685>different <00:04:44.885>ability <00:04:45.085>when <00:04:45.285>you're <00:04:45.485>playing <00:04:45.685>the <00:04:45.885>game.

128
00:04:46.545 --> 00:04:49.165
<00:04:46.545>And <00:04:46.672>I <00:04:46.799>feel <00:04:46.926>like <00:04:47.054>this <00:04:47.181>is <00:04:47.308>kind <00:04:47.435>of <00:04:47.563>similar <00:04:47.690>to, <00:04:47.817>for <00:04:47.945>example,

129
00:04:49.585 --> 00:04:52.405
<00:04:49.585>the <00:04:49.724>world <00:04:49.864>of <00:04:50.005>open <00:04:50.145>source <00:04:50.284>and <00:04:50.424>open <00:04:50.565>source <00:04:50.705>AI <00:04:50.844>right <00:04:50.984>now

130
00:04:52.405 --> 00:04:54.005
<00:04:52.405>because <00:04:52.755>you <00:04:53.104>have <00:04:53.455>different <00:04:53.804>projects

131
00:04:54.195 --> 00:04:55.965
<00:04:54.195>that <00:04:54.395>will <00:04:54.594>help <00:04:54.795>you <00:04:54.995>for <00:04:55.195>different <00:04:55.395>use <00:04:55.594>cases.

132
00:04:56.185 --> 00:04:59.005
<00:04:56.185>And <00:04:56.385>let's <00:04:56.585>overview <00:04:56.785>them <00:04:56.985>really <00:04:57.185>quickly, <00:04:57.385>right? <00:04:57.585>Yeah.

133
00:04:59.865 --> 00:05:02.245
<00:04:59.865>So <00:05:00.005>a <00:05:00.145>lot <00:05:00.285>of <00:05:00.425>us <00:05:00.565>might <00:05:00.705>have <00:05:00.845>already <00:05:00.985>used <00:05:01.125>this <00:05:01.265>tool

134
00:05:02.245 --> 00:05:03.365
<00:05:02.245>before, <00:05:02.497>which <00:05:02.749>is <00:05:03.001>O <00:05:03.253>lama,

135
00:05:03.365 --> 00:05:06.365
<00:05:03.365>but <00:05:03.505>it <00:05:03.645>was <00:05:03.785>created <00:05:03.925>by <00:05:04.065>a <00:05:04.205>Docker <00:05:04.345>pm <00:05:04.485>essentially <00:05:04.625>to <00:05:04.765>have

136
00:05:06.365 --> 00:05:07.805
<00:05:06.365>that <00:05:06.689>same <00:05:07.013>type <00:05:07.337>of <00:05:07.661>confi

137
00:05:07.865 --> 00:05:10.925
<00:05:07.865>or <00:05:08.040>same <00:05:08.215>type <00:05:08.390>of <00:05:08.565>experience <00:05:08.740>of <00:05:08.915>a <00:05:09.090>docker <00:05:09.265>pool

138
00:05:10.925 --> 00:05:13.205
<00:05:10.925>and <00:05:11.100>a <00:05:11.275>docker <00:05:11.450>run <00:05:11.625>by <00:05:11.800>container <00:05:11.975>for <00:05:12.150>a <00:05:12.325>model.

139
00:05:13.345 --> 00:05:15.205
<00:05:13.345>So, <00:05:13.545>uh, <00:05:13.745>you <00:05:13.945>know, <00:05:14.145>you'd <00:05:14.345>owe <00:05:14.545>Lama <00:05:14.745>pool

140
00:05:15.205 --> 00:05:17.765
<00:05:15.205>and <00:05:15.405>o <00:05:15.604>lama <00:05:15.805>run, <00:05:16.005>for <00:05:16.205>example, <00:05:16.405>llama <00:05:16.604>3.1.

141
00:05:18.065 --> 00:05:20.765
<00:05:18.065>And <00:05:18.181>you <00:05:18.298>also <00:05:18.415>have <00:05:18.531>what's <00:05:18.648>known <00:05:18.765>as <00:05:18.881>the <00:05:18.998>model <00:05:19.115>file <00:05:19.231>as <00:05:19.348>a <00:05:19.465>way

142
00:05:20.765 --> 00:05:21.885
<00:05:20.765>to <00:05:21.101>customize <00:05:21.437>a <00:05:21.772>model

143
00:05:21.915 --> 00:05:24.245
<00:05:21.915>with <00:05:22.195>specific <00:05:22.475>configuration <00:05:22.755>and <00:05:23.035>prompt <00:05:23.315>templates.

144
00:05:24.495 --> 00:05:26.005
<00:05:24.495>We're <00:05:24.646>gonna <00:05:24.797>talk <00:05:24.948>about <00:05:25.099>why <00:05:25.250>is <00:05:25.401>this <00:05:25.552>a <00:05:25.703>good <00:05:25.854>tool,

145
00:05:26.005 --> 00:05:28.605
<00:05:26.005>but <00:05:26.238>also <00:05:26.471>if <00:05:26.705>you're <00:05:26.938>a <00:05:27.171>developer, <00:05:27.405>why?

146
00:05:28.605 --> 00:05:30.165
<00:05:28.605>There's <00:05:28.780>other <00:05:28.955>tools <00:05:29.130>that <00:05:29.305>you <00:05:29.480>can <00:05:29.655>use <00:05:29.830>as <00:05:30.005>well,

147
00:05:30.165 --> 00:05:31.445
<00:05:30.165>because <00:05:30.741>let's <00:05:31.317>say

148
00:05:31.445 --> 00:05:34.525
<00:05:31.445>that <00:05:31.561>you <00:05:31.678>wanna <00:05:31.795>scale <00:05:31.911>things <00:05:32.028>up <00:05:32.145>while <00:05:32.261>O <00:05:32.378>lama <00:05:32.495>is <00:05:32.611>good <00:05:32.728>on <00:05:32.844>a

149
00:05:34.525 --> 00:05:37.205
<00:05:34.525>single <00:05:34.724>machine, <00:05:34.924>uh, <00:05:35.125>if <00:05:35.325>you <00:05:35.525>want <00:05:35.724>concurrent <00:05:35.924>requests

150
00:05:37.205 --> 00:05:39.805
<00:05:37.205>and <00:05:37.360>be <00:05:37.516>able <00:05:37.671>to <00:05:37.827>batch <00:05:37.982>your <00:05:38.138>requests <00:05:38.293>to <00:05:38.449>different <00:05:38.604>GPUs,

151
00:05:40.265 --> 00:05:42.445
<00:05:40.265>for <00:05:40.544>example, <00:05:40.825>with <00:05:41.104>Kubernetes, <00:05:41.385>yeah, <00:05:41.664>that's

152
00:05:42.445 --> 00:05:44.285
<00:05:42.445>where <00:05:42.620>VLM <00:05:42.795>is <00:05:42.969>really <00:05:43.145>going <00:05:43.320>to <00:05:43.495>kick <00:05:43.670>in. <00:05:43.844>Design

153
00:05:44.285 --> 00:05:45.405
<00:05:44.284>It <00:05:44.537>for <00:05:44.789>that. <00:05:45.041>For <00:05:45.292>scaling.

154
00:05:45.785 --> 00:05:47.685
<00:05:45.784>For <00:05:46.064>scaling, <00:05:46.344>right? <00:05:46.624>So <00:05:46.905>it <00:05:47.184>comes

155
00:05:47.685 --> 00:05:49.445
<00:05:47.685>with <00:05:47.965>an <00:05:48.245>algorithm <00:05:48.525>called <00:05:48.805>page <00:05:49.085>detention,

156
00:05:49.655 --> 00:05:51.965
<00:05:49.655>which <00:05:49.854>is <00:05:50.054>more <00:05:50.255>efficient <00:05:50.455>on <00:05:50.655>your <00:05:50.854>GPU <00:05:51.054>resources.

157
00:05:51.985 --> 00:05:54.365
<00:05:51.985>So <00:05:52.185>if <00:05:52.385>you <00:05:52.585>have <00:05:52.785>Nvidia <00:05:52.985>a <00:05:53.185>MD <00:05:53.385>hardware,

158
00:05:54.605 --> 00:05:55.725
<00:05:54.605>whatever <00:05:54.941>accelerator <00:05:55.277>you're <00:05:55.613>using,

159
00:05:55.955 --> 00:05:57.765
<00:05:55.955>it's <00:05:56.155>gonna <00:05:56.354>be <00:05:56.555>more <00:05:56.755>efficient <00:05:56.955>when <00:05:57.155>you're <00:05:57.354>loading

160
00:05:58.025 --> 00:05:59.125
<00:05:58.025>and <00:05:58.354>inferencing <00:05:58.685>a <00:05:59.015>model

161
00:05:59.125 --> 00:06:00.965
<00:05:59.125>because <00:05:59.358>of <00:05:59.591>the <00:05:59.825>processing <00:06:00.058>of <00:06:00.291>KV <00:06:00.525>cash,

162
00:06:01.175 --> 00:06:03.325
<00:06:01.175>which <00:06:01.375>is <00:06:01.575>like <00:06:01.775>virtual <00:06:01.975>memory <00:06:02.175>for <00:06:02.375>an <00:06:02.575>LLM.

163
00:06:03.665 --> 00:06:06.525
<00:06:03.665>So <00:06:03.820>this <00:06:03.976>is <00:06:04.131>more <00:06:04.287>of <00:06:04.442>a <00:06:04.598>production <00:06:04.753>ready, <00:06:04.909>uh, <00:06:05.065>runtime

164
00:06:06.945 --> 00:06:09.645
<00:06:06.945>for <00:06:07.100>developers <00:06:07.256>though <00:06:07.411>what <00:06:07.567>we <00:06:07.722>work <00:06:07.878>with <00:06:08.033>is <00:06:08.189>Ramal <00:06:08.344>lama.

165
00:06:09.745 --> 00:06:13.125
<00:06:09.745>So <00:06:09.885>this <00:06:10.025>is <00:06:10.165>a <00:06:10.305>project <00:06:10.445>that <00:06:10.585>contributes <00:06:10.725>to <00:06:10.865>Lacy <00:06:11.005>plus <00:06:11.145>plus,

166
00:06:13.125 --> 00:06:16.205
<00:06:13.125>which <00:06:13.300>is <00:06:13.475>a, <00:06:13.650>um, <00:06:13.825>kind <00:06:14.000>of <00:06:14.175>the <00:06:14.350>upstream <00:06:14.525>project

167
00:06:16.205 --> 00:06:18.565
<00:06:16.205>that <00:06:16.438>powers <00:06:16.671>both <00:06:16.905>llama <00:06:17.138>and <00:06:17.371>this <00:06:17.604>one.

168
00:06:18.865 --> 00:06:20.085
<00:06:18.865>But <00:06:19.139>instead <00:06:19.414>of, <00:06:19.688>you <00:06:19.962>know,

169
00:06:20.115 --> 00:06:22.205
<00:06:20.115>running <00:06:20.395>containers <00:06:20.675>locally <00:06:20.955>on <00:06:21.235>your <00:06:21.515>machine,

170
00:06:22.485 --> 00:06:24.925
<00:06:22.485>directly <00:06:22.625>on <00:06:22.765>the <00:06:22.905>hardware, <00:06:23.045>this <00:06:23.185>runs <00:06:23.325>it <00:06:23.465>in <00:06:23.605>a <00:06:23.745>container <00:06:23.885>so

171
00:06:24.925 --> 00:06:27.005
<00:06:24.925>that <00:06:25.100>you <00:06:25.275>could <00:06:25.450>pull <00:06:25.625>from <00:06:25.800>llama <00:06:25.975>or <00:06:26.150>hugging <00:06:26.325>face

172
00:06:27.025 --> 00:06:28.445
<00:06:27.025>or <00:06:27.450>whatever <00:06:27.877>OCI <00:06:28.303>registry.

173
00:06:28.745 --> 00:06:31.045
<00:06:28.745>And <00:06:28.900>when <00:06:29.056>you <00:06:29.211>have <00:06:29.367>your <00:06:29.522>model <00:06:29.678>running <00:06:29.833>in <00:06:29.989>a <00:06:30.145>container,

174
00:06:31.045 --> 00:06:33.405
<00:06:31.045>then <00:06:31.220>you <00:06:31.395>can <00:06:31.570>take <00:06:31.745>that <00:06:31.920>and <00:06:32.095>scale <00:06:32.270>that, <00:06:32.445>say

175
00:06:33.405 --> 00:06:34.485
<00:06:33.405>for <00:06:33.729>example, <00:06:34.053>on <00:06:34.377>Kubernetes.

176
00:06:34.485 --> 00:06:36.085
<00:06:34.485>Well, <00:06:34.765>it's <00:06:35.045>simple, <00:06:35.325>isn't <00:06:35.605>it? <00:06:35.885>Right?

177
00:06:36.085 --> 00:06:37.325
<00:06:36.085>You <00:06:36.457>have <00:06:36.829>that <00:06:37.200>consistency.

178
00:06:37.985 --> 00:06:39.005
<00:06:37.985>And <00:06:38.214>then <00:06:38.444>one <00:06:38.673>more <00:06:38.903>tool

179
00:06:39.005 --> 00:06:40.605
<00:06:39.005>that <00:06:39.160>we're <00:06:39.316>gonna <00:06:39.471>take <00:06:39.627>a <00:06:39.782>look <00:06:39.938>at <00:06:40.093>today <00:06:40.249>is <00:06:40.405>called

180
00:06:40.745 --> 00:06:41.805
<00:06:40.745>Pod <00:06:41.063>Man <00:06:41.381>AI <00:06:41.699>lab.

181
00:06:41.945 --> 00:06:44.685
<00:06:41.945>So <00:06:42.072>a <00:06:42.199>lot <00:06:42.326>of <00:06:42.454>developers <00:06:42.581>use <00:06:42.708>Pod <00:06:42.835>Man <00:06:42.963>as <00:06:43.090>a <00:06:43.217>container <00:06:43.344>engine,

182
00:06:44.865 --> 00:06:46.725
<00:06:44.865>but <00:06:45.145>pod <00:06:45.425>man <00:06:45.705>also <00:06:45.985>supports <00:06:46.265>uh,

183
00:06:46.955 --> 00:06:49.445
<00:06:46.955>this <00:06:47.130>extension <00:06:47.305>called <00:06:47.479>the <00:06:47.655>AI <00:06:47.830>lab, <00:06:48.005>which <00:06:48.180>allows <00:06:48.354>you

184
00:06:49.445 --> 00:06:50.805
<00:06:49.445>to <00:06:49.751>run <00:06:50.057>models <00:06:50.363>and <00:06:50.669>containers

185
00:06:51.145 --> 00:06:53.445
<00:06:51.145>and <00:06:51.378>to <00:06:51.611>build <00:06:51.844>applications <00:06:52.078>with <00:06:52.311>generative <00:06:52.544>ai.

186
00:06:53.545 --> 00:06:57.005
<00:06:53.545>So <00:06:53.672>you <00:06:53.799>run <00:06:53.926>a <00:06:54.054>model, <00:06:54.181>you <00:06:54.308>get <00:06:54.435>that <00:06:54.563>swagger <00:06:54.690>interface <00:06:54.817>in <00:06:54.945>order

187
00:06:57.005 --> 00:06:58.165
<00:06:57.005>to <00:06:57.353>test <00:06:57.701>the <00:06:58.049>APIs

188
00:06:58.385 --> 00:07:00.685
<00:06:58.385>and <00:06:58.540>then <00:06:58.696>you <00:06:58.851>can <00:06:59.007>easily <00:06:59.162>integrate <00:06:59.318>that <00:06:59.473>with <00:06:59.629>code <00:06:59.784>samples

189
00:07:00.945 --> 00:07:04.405
<00:07:00.945>and <00:07:01.120>templates <00:07:01.295>for <00:07:01.469>gen <00:07:01.645>ai, <00:07:01.820>for <00:07:01.995>rag <00:07:02.170>for <00:07:02.344>Agentic.

190
00:07:04.425 --> 00:07:05.765
<00:07:04.425>So <00:07:04.626>right, <00:07:04.827>we're <00:07:05.028>gonna <00:07:05.229>have <00:07:05.430>fun <00:07:05.631>there.

191
00:07:06.705 --> 00:07:08.325
<00:07:06.705>Now <00:07:06.905>I <00:07:07.104>have <00:07:07.305>a <00:07:07.505>question <00:07:07.705>for <00:07:07.905>you <00:07:08.104>Roberto.

192
00:07:08.325 --> 00:07:10.365
<00:07:08.325>Alright, <00:07:08.500>I <00:07:08.675>have <00:07:08.849>all <00:07:09.025>these <00:07:09.200>tools <00:07:09.375>to <00:07:09.550>run <00:07:09.724>models.

193
00:07:10.995 --> 00:07:12.045
<00:07:10.995>What <00:07:11.231>model <00:07:11.467>do <00:07:11.703>I <00:07:11.940>use?

194
00:07:12.235 --> 00:07:15.165
<00:07:12.235>Well, <00:07:12.410>we <00:07:12.585>have <00:07:12.760>a <00:07:12.935>lot <00:07:13.110>of <00:07:13.285>choices <00:07:13.460>in <00:07:13.635>it

195
00:07:15.225 --> 00:07:18.685
<00:07:15.225>and <00:07:15.380>we <00:07:15.536>have <00:07:15.691>plenty <00:07:15.847>of <00:07:16.002>models <00:07:16.158>both <00:07:16.313>open <00:07:16.469>and <00:07:16.625>close.

196
00:07:18.785 --> 00:07:19.885
<00:07:18.785>Uh, <00:07:19.032>model <00:07:19.280>choice <00:07:19.527>as <00:07:19.775>well.

197
00:07:19.915 --> 00:07:23.845
<00:07:19.915>This <00:07:20.042>is <00:07:20.169>just <00:07:20.296>a <00:07:20.424>charge <00:07:20.551>of <00:07:20.678>how <00:07:20.805>many <00:07:20.933>models <00:07:21.060>we <00:07:21.187>have <00:07:21.315>up

198
00:07:24.025 --> 00:07:25.325
<00:07:24.025>to <00:07:25.195>2025

199
00:07:25.465 --> 00:07:28.565
<00:07:25.465>and <00:07:25.640>we <00:07:25.815>have <00:07:25.989>more <00:07:26.164>than <00:07:26.340>1 <00:07:26.515>million <00:07:26.690>over, <00:07:26.864>um,

200
00:07:29.385 --> 00:07:32.445
<00:07:29.385>the <00:07:29.540>different <00:07:29.696>models <00:07:29.851>that <00:07:30.007>we <00:07:30.162>have <00:07:30.318>available <00:07:30.473>in <00:07:30.629>hack <00:07:30.784>face

201
00:07:32.445 --> 00:07:35.205
<00:07:32.445>that <00:07:32.620>it's <00:07:32.795>like <00:07:32.969>GitHub, <00:07:33.145>but <00:07:33.320>for <00:07:33.495>the <00:07:33.670>different <00:07:33.844>models.

202
00:07:35.345 --> 00:07:38.125
<00:07:35.345>So <00:07:35.520>what <00:07:35.695>will <00:07:35.870>be <00:07:36.045>the <00:07:36.220>proper <00:07:36.395>model <00:07:36.570>to <00:07:36.745>run

203
00:07:38.345 --> 00:07:39.965
<00:07:38.345>for <00:07:38.625>our, <00:07:38.905>uh, <00:07:39.185>own <00:07:39.465>use <00:07:39.745>cases?

204
00:07:41.145 --> 00:07:43.085
<00:07:41.145>And <00:07:41.378>we <00:07:41.611>are <00:07:41.844>back <00:07:42.078>again <00:07:42.311>in <00:07:42.544>here.

205
00:07:43.345 --> 00:07:44.685
<00:07:43.345>We <00:07:43.546>have <00:07:43.747>a <00:07:43.948>lot <00:07:44.149>of <00:07:44.350>different <00:07:44.551>choices

206
00:07:44.825 --> 00:07:48.565
<00:07:44.825>and <00:07:44.952>we <00:07:45.079>don't <00:07:45.206>know <00:07:45.334>which <00:07:45.461>is <00:07:45.588>the <00:07:45.715>best <00:07:45.843>model <00:07:45.970>to <00:07:46.097>run <00:07:46.224>it

207
00:07:48.985 --> 00:07:51.645
<00:07:48.985>to <00:07:49.265>our <00:07:49.545>uh, <00:07:49.825>use <00:07:50.105>case. <00:07:50.385>Alright,

208
00:07:52.625 --> 00:07:53.845
<00:07:52.625>Uh, <00:07:52.899>alright, <00:07:53.174>it's <00:07:53.448>the <00:07:53.723>speaker.

209
00:07:54.865 --> 00:07:59.325
<00:07:54.865>But <00:07:55.005>again, <00:07:55.145>um, <00:07:55.285>we, <00:07:55.425>we <00:07:55.565>will <00:07:55.705>use <00:07:55.845>our <00:07:55.985>um, <00:07:56.125>smash, <00:07:56.265>um,

210
00:08:00.735 --> 00:08:03.485
<00:08:00.735>again, <00:08:00.910>we <00:08:01.085>will <00:08:01.260>have <00:08:01.435>different <00:08:01.610>uh, <00:08:01.785>type <00:08:01.960>of <00:08:02.135>models

211
00:08:03.945 --> 00:08:05.525
<00:08:03.945>for <00:08:04.178>the <00:08:04.411>different <00:08:04.645>use <00:08:04.878>cases <00:08:05.111>as <00:08:05.344>well.

212
00:08:06.185 --> 00:08:09.725
<00:08:06.185>One <00:08:06.385>of <00:08:06.585>them <00:08:06.785>could <00:08:06.985>be <00:08:07.185>granites, <00:08:07.385>sstl, <00:08:07.585>dipsy,

213
00:08:10.325 --> 00:08:11.365
<00:08:10.325>llama, <00:08:10.637>you <00:08:10.949>name <00:08:11.261>it.

214
00:08:11.365 --> 00:08:15.045
<00:08:11.365>Because <00:08:11.505>we <00:08:11.645>have <00:08:11.785>a <00:08:11.925>lot <00:08:12.065>of <00:08:12.205>uh, <00:08:12.345>different <00:08:12.485>models <00:08:12.625>that <00:08:12.765>uh,

215
00:08:15.185 --> 00:08:17.525
<00:08:15.185>are <00:08:15.385>very, <00:08:15.585>very <00:08:15.785>good <00:08:15.985>at <00:08:16.185>different <00:08:16.385>use <00:08:16.585>cases.

216
00:08:17.665 --> 00:08:19.605
<00:08:17.665>For <00:08:17.898>example, <00:08:18.131>if <00:08:18.365>we <00:08:18.598>wanted <00:08:18.831>to <00:08:19.065>um,

217
00:08:19.865 --> 00:08:22.645
<00:08:19.865>use <00:08:20.065>different <00:08:20.265>reasoning <00:08:20.465>task, <00:08:20.665>we <00:08:20.865>can <00:08:21.065>use <00:08:21.265>dipsy,

218
00:08:23.105 --> 00:08:25.565
<00:08:23.105>but <00:08:23.280>if <00:08:23.455>we <00:08:23.630>wanted <00:08:23.805>to <00:08:23.980>use <00:08:24.155>small <00:08:24.330>language <00:08:24.505>models

219
00:08:25.565 --> 00:08:29.405
<00:08:25.565>that <00:08:25.740>perform <00:08:25.915>fare <00:08:26.090>well <00:08:26.265>with <00:08:26.440>different <00:08:26.615>uh, <00:08:26.790>MPL <00:08:26.965>task

220
00:08:29.505 --> 00:08:31.965
<00:08:29.505>and <00:08:29.660>multimodal <00:08:29.816>applications, <00:08:29.971>we <00:08:30.127>can <00:08:30.282>use <00:08:30.438>run <00:08:30.593>it <00:08:30.749>as <00:08:30.905>well.

221
00:08:32.825 --> 00:08:37.685
<00:08:32.825>And <00:08:32.980>Mr. <00:08:33.136>Lama <00:08:33.291>are <00:08:33.447>very <00:08:33.602>good <00:08:33.758>and <00:08:33.913>strong <00:08:34.069>and <00:08:34.225>summarization

222
00:08:37.745 --> 00:08:39.045
<00:08:37.745>and <00:08:38.329>same <00:08:38.914>analysis

223
00:08:39.145 --> 00:08:42.045
<00:08:39.145>and <00:08:39.272>we <00:08:39.399>will <00:08:39.526>use <00:08:39.654>it <00:08:39.781>as <00:08:39.908>well <00:08:40.035>for <00:08:40.163>doing <00:08:40.290>some <00:08:40.417>agent <00:08:40.544>stuff

224
00:08:42.395 --> 00:08:43.445
<00:08:42.395>with <00:08:43.340>MCP.

225
00:08:43.705 --> 00:08:48.045
<00:08:43.705>Um, <00:08:43.821>as <00:08:43.938>you <00:08:44.055>can <00:08:44.171>see, <00:08:44.288>but <00:08:44.405>not <00:08:44.521>all <00:08:44.638>the <00:08:44.755>models <00:08:44.871>are <00:08:44.988>the <00:08:45.105>same.

226
00:08:48.545 --> 00:08:50.805
<00:08:48.545>We <00:08:48.778>have <00:08:49.011>uni <00:08:49.245>models <00:08:49.478>and <00:08:49.711>multi <00:08:49.944>models.

227
00:08:51.485 --> 00:08:55.565
<00:08:51.485>Uni <00:08:51.625>models <00:08:51.765>is <00:08:51.905>just <00:08:52.045>using <00:08:52.185>text <00:08:52.325>to <00:08:52.465>image, <00:08:52.605>text <00:08:52.745>to <00:08:52.885>text

228
00:08:55.985 --> 00:08:58.605
<00:08:55.985>but <00:08:56.218>are <00:08:56.451>not <00:08:56.685>ringing <00:08:56.918>both <00:08:57.151>walls <00:08:57.385>together.

229
00:08:59.065 --> 00:09:02.845
<00:08:59.065>And <00:08:59.192>we <00:08:59.319>have <00:08:59.446>also, <00:08:59.574>the <00:08:59.701>other <00:08:59.828>way <00:08:59.955>that <00:09:00.083>is <00:09:00.210>multimodel, <00:09:00.337>that <00:09:00.465>is

230
00:09:02.845 --> 00:09:06.325
<00:09:02.845>where <00:09:03.020>we <00:09:03.195>are <00:09:03.370>using <00:09:03.545>stable <00:09:03.720>diffusion <00:09:03.895>for <00:09:04.070>example, <00:09:04.245>we,

231
00:09:06.385 --> 00:09:10.085
<00:09:06.385>we <00:09:06.560>are <00:09:06.735>asking <00:09:06.910>generate <00:09:07.085>one <00:09:07.260>very <00:09:07.435>cute <00:09:07.610>uh, <00:09:07.785>joke

232
00:09:10.225 --> 00:09:13.725
<00:09:10.225>and <00:09:10.380>it's <00:09:10.536>generating <00:09:10.691>one <00:09:10.847>image <00:09:11.002>so <00:09:11.158>it's <00:09:11.313>entering <00:09:11.469>the <00:09:11.625>text

233
00:09:13.945 --> 00:09:16.765
<00:09:13.945>and <00:09:14.145>afterwards <00:09:14.345>uh, <00:09:14.545>it's <00:09:14.745>uh, <00:09:14.945>generating <00:09:15.145>one <00:09:15.345>image

234
00:09:17.785 --> 00:09:21.085
<00:09:17.785>and <00:09:17.925>also <00:09:18.064>we <00:09:18.204>have <00:09:18.344>the <00:09:18.485>this <00:09:18.625>naming <00:09:18.765>convention <00:09:18.905>when <00:09:19.045>you <00:09:19.185>are

235
00:09:21.085 --> 00:09:23.405
<00:09:21.085>going <00:09:21.285>to <00:09:21.485>hack <00:09:21.685>face, <00:09:21.885>that <00:09:22.085>is <00:09:22.285>the <00:09:22.485>GitHub.

236
00:09:23.985 --> 00:09:25.965
<00:09:23.985>We <00:09:24.265>have <00:09:24.545>the <00:09:24.825>different <00:09:25.105>family <00:09:25.385>names

237
00:09:26.625 --> 00:09:30.205
<00:09:26.625>and <00:09:26.800>also <00:09:26.975>this <00:09:27.150>model <00:09:27.325>architecture <00:09:27.500>that <00:09:27.675>it's <00:09:27.850>super <00:09:28.025>important

238
00:09:30.205 --> 00:09:33.805
<00:09:30.205>because <00:09:30.438>finds <00:09:30.671>the <00:09:30.905>different <00:09:31.138>number <00:09:31.371>of <00:09:31.605>parameters

239
00:09:33.985 --> 00:09:35.445
<00:09:33.985>and <00:09:34.313>these <00:09:34.642>effects <00:09:34.970>as <00:09:35.299>well.

240
00:09:35.905 --> 00:09:40.765
<00:09:35.905>And <00:09:36.045>finally <00:09:36.185>we <00:09:36.324>have <00:09:36.464>this <00:09:36.605>base, <00:09:36.745>all <00:09:36.885>the <00:09:37.025>different <00:09:37.165>um, <00:09:37.305>model,

241
00:09:41.235 --> 00:09:44.045
<00:09:41.235>fine <00:09:41.390>tune <00:09:41.546>because <00:09:41.701>depending <00:09:41.857>on <00:09:42.012>the <00:09:42.168>use <00:09:42.323>case, <00:09:42.479>for <00:09:42.635>example,

242
00:09:44.065 --> 00:09:48.165
<00:09:44.065>if <00:09:44.240>you're <00:09:44.415>using <00:09:44.590>vision, <00:09:44.765>this <00:09:44.940>vision <00:09:45.115>models <00:09:45.290>are <00:09:45.465>designed

243
00:09:48.345 --> 00:09:50.965
<00:09:48.345>and <00:09:48.520>uh, <00:09:48.695>fine <00:09:48.870>tuned <00:09:49.045>to <00:09:49.220>use <00:09:49.395>it <00:09:49.570>with <00:09:49.745>images.

244
00:09:51.665 --> 00:09:54.805
<00:09:51.665>So <00:09:51.805>depending <00:09:51.944>on <00:09:52.084>the <00:09:52.224>different <00:09:52.365>use <00:09:52.505>cases <00:09:52.645>when <00:09:52.785>you <00:09:52.925>are <00:09:53.064>trying

245
00:09:54.825 --> 00:09:56.885
<00:09:54.825>to <00:09:55.058>look <00:09:55.291>to <00:09:55.525>the <00:09:55.758>different <00:09:55.991>uh, <00:09:56.225>models,

246
00:09:57.345 --> 00:10:01.925
<00:09:57.345>you <00:09:57.500>can <00:09:57.656>identify <00:09:57.811>this <00:09:57.967>in <00:09:58.122>order <00:09:58.278>to <00:09:58.433>um, <00:09:58.589>perform <00:09:58.745>better

247
00:10:02.155 --> 00:10:03.685
<00:10:02.155>with <00:10:02.351>the <00:10:02.548>different <00:10:02.745>use <00:10:02.941>cases <00:10:03.138>that <00:10:03.335>you <00:10:03.531>have

248
00:10:04.705 --> 00:10:07.465
<00:10:04.705>and <00:10:04.938>how <00:10:05.171>to <00:10:05.405>deploy <00:10:05.638>these <00:10:05.871>larger <00:10:06.105>models

249
00:10:07.465 --> 00:10:09.345
<00:10:07.465>because <00:10:07.745>we <00:10:08.025>wanted <00:10:08.305>to <00:10:08.585>use <00:10:08.865>always

250
00:10:09.925 --> 00:10:11.465
<00:10:09.925>the <00:10:10.202>bigger <00:10:10.479>models, <00:10:10.756>isn't <00:10:11.033>it? <00:10:11.311>I

251
00:10:11.465 --> 00:10:13.465
<00:10:11.465>Want <00:10:11.698>the <00:10:11.931>best <00:10:12.165>benchmarks <00:10:12.398>for <00:10:12.631>my <00:10:12.865>application.

252
00:10:13.615 --> 00:10:17.185
<00:10:13.615>Alright, <00:10:13.848>so <00:10:14.081>what <00:10:14.315>about <00:10:14.548>the <00:10:14.781>model <00:10:15.015>size?

253
00:10:17.185 --> 00:10:19.145
<00:10:17.185>Bigger <00:10:17.418>is <00:10:17.651>always, <00:10:17.885>uh, <00:10:18.118>bad, <00:10:18.351>isn't <00:10:18.584>it?

254
00:10:19.695 --> 00:10:21.905
<00:10:19.695>Well <00:10:19.975>we <00:10:20.255>will <00:10:20.535>end <00:10:20.815>like <00:10:21.095>this.

255
00:10:22.405 --> 00:10:26.585
<00:10:22.405>So <00:10:22.532>if <00:10:22.659>you <00:10:22.786>wanted <00:10:22.914>to <00:10:23.041>run <00:10:23.168>these <00:10:23.295>huge <00:10:23.423>models <00:10:23.550>in <00:10:23.677>your <00:10:23.805>laptop,

256
00:10:26.605 --> 00:10:28.705
<00:10:26.605>you <00:10:26.805>will <00:10:27.005>end <00:10:27.205>put <00:10:27.405>in <00:10:27.605>your <00:10:27.805>on <00:10:28.005>fire.

257
00:10:29.085 --> 00:10:30.745
<00:10:29.085>We, <00:10:29.285>we <00:10:29.485>don't <00:10:29.685>want <00:10:29.885>it <00:10:30.085>to <00:10:30.285>do <00:10:30.485>that

258
00:10:30.745 --> 00:10:32.305
<00:10:30.745>because <00:10:30.978>we <00:10:31.211>need <00:10:31.445>it <00:10:31.678>for <00:10:31.911>our <00:10:32.145>demo

259
00:10:32.885 --> 00:10:36.785
<00:10:32.885>or <00:10:33.085>having <00:10:33.285>this <00:10:33.485>extra <00:10:33.685>large <00:10:33.885>bfi <00:10:34.085>uh, <00:10:34.285>laptop

260
00:10:36.785 --> 00:10:38.305
<00:10:36.785>that <00:10:37.127>is <00:10:37.468>portable <00:10:37.810>kind <00:10:38.152>of,

261
00:10:38.565 --> 00:10:40.665
<00:10:38.565>but <00:10:38.740>we <00:10:38.915>don't <00:10:39.090>want <00:10:39.265>it <00:10:39.440>to <00:10:39.615>do <00:10:39.790>that <00:10:39.965>to.

262
00:10:40.725 --> 00:10:43.705
<00:10:40.725>So <00:10:40.925>to <00:10:41.125>solve <00:10:41.325>that <00:10:41.525>problem <00:10:41.725>we <00:10:41.925>have <00:10:42.125>quantization

263
00:10:44.245 --> 00:10:46.785
<00:10:44.245>and <00:10:44.445>it's <00:10:44.645>a <00:10:44.845>way <00:10:45.045>of <00:10:45.245>compressing <00:10:45.445>the <00:10:45.645>model

264
00:10:47.285 --> 00:10:50.745
<00:10:47.285>but <00:10:47.485>maintaining <00:10:47.685>also <00:10:47.885>these <00:10:48.084>accuracy <00:10:48.285>to <00:10:48.485>try <00:10:48.685>to

265
00:10:51.325 --> 00:10:53.745
<00:10:51.325>reduce <00:10:51.605>the <00:10:51.885>consumption <00:10:52.165>of <00:10:52.445>our <00:10:52.725>model.

266
00:10:54.445 --> 00:10:57.945
<00:10:54.445>But <00:10:54.645>without <00:10:54.845>make <00:10:55.045>it <00:10:55.245>at <00:10:55.445>number, <00:10:55.645>it's <00:10:55.845>trying

267
00:10:57.965 --> 00:10:59.745
<00:10:57.965>to <00:10:58.315>also <00:10:58.665>maintain <00:10:59.015>these <00:10:59.365>accuracy

268
00:10:59.745 --> 00:11:02.105
<00:10:59.745>and <00:10:59.978>maintain <00:11:00.211>this <00:11:00.445>ratio <00:11:00.678>of <00:11:00.911>compressing <00:11:01.145>it.

269
00:11:02.445 --> 00:11:06.265
<00:11:02.445>And <00:11:02.600>we <00:11:02.756>are <00:11:02.911>putting <00:11:03.067>all <00:11:03.222>of <00:11:03.378>these <00:11:03.533>weights <00:11:03.689>that <00:11:03.845>it's

270
00:11:06.265 --> 00:11:08.545
<00:11:06.265>where <00:11:06.498>the <00:11:06.731>knowledge <00:11:06.965>um, <00:11:07.198>has <00:11:07.431>been <00:11:07.665>stored

271
00:11:09.055 --> 00:11:10.305
<00:11:09.055>into <00:11:09.430>a <00:11:09.805>lower <00:11:10.180>position

272
00:11:10.845 --> 00:11:14.065
<00:11:10.845>and <00:11:11.045>reducing <00:11:11.245>the <00:11:11.445>memory <00:11:11.645>footprint <00:11:11.845>to <00:11:12.045>be <00:11:12.245>capable

273
00:11:14.245 --> 00:11:18.185
<00:11:14.245>to <00:11:14.420>run <00:11:14.595>on <00:11:14.770>your <00:11:14.945>laptop <00:11:15.120>on <00:11:15.295>both <00:11:15.470>of <00:11:15.645>GPU

274
00:11:18.205 --> 00:11:19.345
<00:11:18.205>but <00:11:18.547>also <00:11:18.889>on <00:11:19.231>CPU

275
00:11:19.345 --> 00:11:21.145
<00:11:19.345>because <00:11:19.578>we <00:11:19.811>have <00:11:20.045>a <00:11:20.278>lot <00:11:20.511>of <00:11:20.745>situations

276
00:11:21.145 --> 00:11:25.705
<00:11:21.145>that <00:11:21.319>we <00:11:21.495>have <00:11:21.670>some <00:11:21.845>developers <00:11:22.020>that <00:11:22.194>have <00:11:22.370>notch, <00:11:22.545>uh,

277
00:11:25.905 --> 00:11:28.785
<00:11:25.905>a <00:11:26.105>laptop <00:11:26.305>with <00:11:26.505>our <00:11:26.704>um, <00:11:26.905>GPU <00:11:27.105>included, <00:11:27.305>right?

278
00:11:29.245 --> 00:11:32.665
<00:11:29.245>And <00:11:29.400>that's <00:11:29.556>where <00:11:29.711>the <00:11:29.867>quantization <00:11:30.022>can <00:11:30.178>help <00:11:30.333>uh, <00:11:30.489>US <00:11:30.645>out

279
00:11:32.765 --> 00:11:36.425
<00:11:32.765>as <00:11:32.892>well <00:11:33.019>because <00:11:33.146>the <00:11:33.274>benefit <00:11:33.401>is <00:11:33.528>that <00:11:33.655>we <00:11:33.783>can <00:11:33.910>run <00:11:34.037>these <00:11:34.165>LMS

280
00:11:37.005 --> 00:11:39.265
<00:11:37.005>on <00:11:37.205>any <00:11:37.405>device, <00:11:37.605>not <00:11:37.805>only <00:11:38.005>on <00:11:38.205>this <00:11:38.405>laptop,

281
00:11:39.275 --> 00:11:42.665
<00:11:39.275>could <00:11:39.415>be <00:11:39.555>also <00:11:39.694>on <00:11:39.834>the <00:11:39.975>edge <00:11:40.115>on <00:11:40.255>iot, <00:11:40.395>you <00:11:40.535>name <00:11:40.675>it.

282
00:11:42.685 --> 00:11:44.905
<00:11:42.685>We <00:11:42.885>have <00:11:43.084>a <00:11:43.285>a <00:11:43.484>bunch <00:11:43.685>of <00:11:43.885>different <00:11:44.084>situations

283
00:11:45.525 --> 00:11:48.025
<00:11:45.525>and <00:11:45.758>with <00:11:45.991>this <00:11:46.225>conversation <00:11:46.458>we <00:11:46.691>have <00:11:46.925>faster

284
00:11:48.445 --> 00:11:52.065
<00:11:48.445>and <00:11:48.645>lighter <00:11:48.845>models <00:11:49.045>that <00:11:49.245>maintains <00:11:49.445>a <00:11:49.645>reasonable <00:11:49.845>accuracy.

285
00:11:52.405 --> 00:11:54.605
<00:11:52.405>So <00:11:52.755>this <00:11:53.105>knowledge, <00:11:53.454>it's <00:11:53.805>available

286
00:11:54.825 --> 00:11:58.125
<00:11:54.825>and <00:11:54.980>we <00:11:55.136>will <00:11:55.291>compress <00:11:55.447>the <00:11:55.602>model <00:11:55.758>but <00:11:55.913>maintain <00:11:56.069>this <00:11:56.225>ratio.

287
00:11:58.505 --> 00:12:02.645
<00:11:58.505>And <00:11:58.632>one <00:11:58.759>important <00:11:58.886>thing <00:11:59.014>is <00:11:59.141>that <00:11:59.268>we <00:11:59.395>will <00:11:59.523>speed <00:11:59.650>up <00:11:59.777>the <00:11:59.905>um,

288
00:12:03.595 --> 00:12:06.845
<00:12:03.595>performance <00:12:03.828>and <00:12:04.061>we <00:12:04.295>have <00:12:04.528>this, <00:12:04.761>um, <00:12:04.995>possibility

289
00:12:06.845 --> 00:12:10.085
<00:12:06.845>of <00:12:07.020>running <00:12:07.195>both <00:12:07.370>on <00:12:07.545>GBU <00:12:07.720>but <00:12:07.895>also <00:12:08.070>on <00:12:08.245>CBU

290
00:12:10.085 --> 00:12:12.205
<00:12:10.085>because <00:12:10.365>if <00:12:10.645>you <00:12:10.925>have <00:12:11.205>one <00:12:11.485>laptop

291
00:12:12.275 --> 00:12:14.365
<00:12:12.275>that <00:12:12.430>runs <00:12:12.586>on <00:12:12.741>CPU, <00:12:12.897>you <00:12:13.052>can <00:12:13.208>run <00:12:13.363>it <00:12:13.519>as <00:12:13.675>well.

292
00:12:15.385 --> 00:12:17.205
<00:12:15.385>And <00:12:15.665>we <00:12:15.944>have <00:12:16.225>an <00:12:16.505>open <00:12:16.785>repository.

293
00:12:17.665 --> 00:12:21.845
<00:12:17.665>If <00:12:17.792>you <00:12:17.919>scan <00:12:18.046>this <00:12:18.174>Q <00:12:18.301>code, <00:12:18.428>you <00:12:18.555>will <00:12:18.683>land <00:12:18.810>into <00:12:18.937>one <00:12:19.064>page

294
00:12:22.115 --> 00:12:25.605
<00:12:22.115>that <00:12:22.242>is <00:12:22.369>red <00:12:22.496>AI <00:12:22.624>as <00:12:22.751>well <00:12:22.878>with <00:12:23.005>a <00:12:23.133>broad <00:12:23.260>collections <00:12:23.387>of <00:12:23.515>quantized

295
00:12:25.865 --> 00:12:28.045
<00:12:25.865>and <00:12:26.145>we <00:12:26.425>have <00:12:26.705>different <00:12:26.985>hard <00:12:27.265>value.

296
00:12:28.185 --> 00:12:31.845
<00:12:28.185>So <00:12:28.292>you <00:12:28.400>can <00:12:28.508>run <00:12:28.615>it <00:12:28.723>in <00:12:28.831>Nvidia <00:12:28.938>A <00:12:29.046>MD <00:12:29.154>in <00:12:29.261>the <00:12:29.369>cloud <00:12:29.477>as <00:12:29.584>well

297
00:12:31.985 --> 00:12:35.725
<00:12:31.985>and <00:12:32.185>Intel <00:12:32.385>and <00:12:32.585>the <00:12:32.785>different <00:12:32.985>formats <00:12:33.185>of <00:12:33.385>quantization.

298
00:12:36.105 --> 00:12:38.645
<00:12:36.105>And <00:12:36.385>with <00:12:36.665>that <00:12:36.945>you <00:12:37.225>have <00:12:37.505>available

299
00:12:39.185 --> 00:12:44.005
<00:12:39.185>and <00:12:39.340>you <00:12:39.496>can <00:12:39.651>download <00:12:39.807>this <00:12:39.962>that <00:12:40.118>will <00:12:40.273>have <00:12:40.429>the <00:12:40.584>LLMs

300
00:12:44.005 --> 00:12:45.925
<00:12:44.005>that <00:12:44.285>cut <00:12:44.564>the <00:12:44.845>GPO <00:12:45.125>custom <00:12:45.405>half

301
00:12:46.225 --> 00:12:48.125
<00:12:46.225>and <00:12:46.425>ready <00:12:46.625>to <00:12:46.825>deploy <00:12:47.025>this <00:12:47.225>inference <00:12:47.425>as <00:12:47.625>well.

302
00:12:48.185 --> 00:12:49.485
<00:12:48.185>So <00:12:48.477>you <00:12:48.770>can <00:12:49.062>run <00:12:49.355>it

303
00:12:49.485 --> 00:12:50.485
<00:12:49.485>and <00:12:49.710>we <00:12:49.935>will <00:12:50.160>use <00:12:50.385>it

304
00:12:50.485 --> 00:12:52.565
<00:12:50.485>as <00:12:50.640>well <00:12:50.796>in <00:12:50.951>this <00:12:51.107>email <00:12:51.262>in <00:12:51.418>a <00:12:51.573>couple <00:12:51.729>of <00:12:51.885>minutes.

305
00:12:53.385 --> 00:12:58.085
<00:12:53.385>So <00:12:53.540>Cedric, <00:12:53.696>we <00:12:53.851>have <00:12:54.007>AI <00:12:54.162>engine <00:12:54.318>check, <00:12:54.473>a <00:12:54.629>model <00:12:54.785>check,

306
00:12:58.505 --> 00:13:01.805
<00:12:58.505>but <00:12:58.705>how <00:12:58.905>about <00:12:59.105>my <00:12:59.305>data? <00:12:59.505>It's <00:12:59.705>important, <00:12:59.905>right?

307
00:13:01.805 --> 00:13:02.885
<00:13:01.805>That's <00:13:02.048>a <00:13:02.290>good <00:13:02.534>question, <00:13:02.777>right?

308
00:13:03.005 --> 00:13:05.285
<00:13:03.005>I <00:13:03.145>think <00:13:03.285>the <00:13:03.425>LLM <00:13:03.564>without <00:13:03.705>your <00:13:03.845>data <00:13:03.985>is <00:13:04.125>pretty <00:13:04.265>useless, <00:13:04.405>right?

309
00:13:05.305 --> 00:13:06.965
<00:13:05.305>It <00:13:05.479>doesn't <00:13:05.655>know <00:13:05.829>you <00:13:06.005>and <00:13:06.180>it <00:13:06.354>can't <00:13:06.530>help <00:13:06.704>you.

310
00:13:07.265 --> 00:13:10.165
<00:13:07.265>So <00:13:07.420>three <00:13:07.576>demos <00:13:07.731>we're <00:13:07.887>gonna <00:13:08.042>show <00:13:08.198>today <00:13:08.353>are <00:13:08.509>about <00:13:08.665>integrating

311
00:13:10.165 --> 00:13:12.485
<00:13:10.165>these <00:13:10.339>LLMs <00:13:10.515>that <00:13:10.689>you're <00:13:10.865>running <00:13:11.040>locally <00:13:11.214>with <00:13:11.390>your <00:13:11.564>data.

312
00:13:12.825 --> 00:13:15.045
<00:13:12.825>The <00:13:13.000>first <00:13:13.175>thing <00:13:13.350>is <00:13:13.525>going <00:13:13.700>to <00:13:13.875>be <00:13:14.050>through <00:13:14.225>question

313
00:13:15.045 --> 00:13:16.445
<00:13:15.045>and <00:13:15.360>answering <00:13:15.675>on <00:13:15.990>your <00:13:16.305>documents.

314
00:13:16.445 --> 00:13:18.645
<00:13:16.445>So <00:13:16.600>PDFs, <00:13:16.756>and <00:13:16.911>this <00:13:17.067>is <00:13:17.222>going <00:13:17.378>to <00:13:17.533>be <00:13:17.689>through <00:13:17.845>rag,

315
00:13:18.865 --> 00:13:21.205
<00:13:18.865>so <00:13:19.005>we're <00:13:19.145>gonna <00:13:19.285>use <00:13:19.425>some <00:13:19.565>open <00:13:19.705>source <00:13:19.845>tools <00:13:19.985>like <00:13:20.125>anything <00:13:20.265>LLM.

316
00:13:21.625 --> 00:13:24.045
<00:13:21.625>The <00:13:21.780>second <00:13:21.936>demo <00:13:22.091>you're <00:13:22.247>gonna <00:13:22.402>see <00:13:22.558>today <00:13:22.713>is <00:13:22.869>code <00:13:23.025>assistance.

317
00:13:24.145 --> 00:13:26.365
<00:13:24.145>Uh, <00:13:24.319>a <00:13:24.495>lot <00:13:24.670>of <00:13:24.845>developers <00:13:25.020>wanna <00:13:25.194>use <00:13:25.370>code <00:13:25.545>assistance

318
00:13:26.365 --> 00:13:29.325
<00:13:26.365>but <00:13:26.520>their <00:13:26.676>organizations <00:13:26.831>say <00:13:26.987>no, <00:13:27.142>you <00:13:27.298>can't <00:13:27.453>use <00:13:27.609>Chad <00:13:27.765>GBT.

319
00:13:29.465 --> 00:13:31.365
<00:13:29.465>So <00:13:29.620>we're <00:13:29.776>gonna <00:13:29.931>show <00:13:30.087>how <00:13:30.242>to <00:13:30.398>run <00:13:30.553>a <00:13:30.709>local <00:13:30.865>model

320
00:13:31.435 --> 00:13:32.765
<00:13:31.435>that <00:13:31.634>will <00:13:31.834>be <00:13:32.033>able <00:13:32.233>to <00:13:32.432>help <00:13:32.632>you

321
00:13:32.765 --> 00:13:35.365
<00:13:32.765>as <00:13:32.939>a <00:13:33.115>pair <00:13:33.290>programmer <00:13:33.465>using <00:13:33.640>a <00:13:33.814>tool <00:13:33.990>called <00:13:34.165>continue.

322
00:13:35.905 --> 00:13:38.605
<00:13:35.905>And <00:13:36.045>the <00:13:36.185>last <00:13:36.324>part <00:13:36.464>is <00:13:36.605>going <00:13:36.745>to <00:13:36.885>be <00:13:37.025>about <00:13:37.165>building <00:13:37.305>applications

323
00:13:38.605 --> 00:13:39.765
<00:13:38.605>that <00:13:38.953>use <00:13:39.301>generative <00:13:39.649>ai.

324
00:13:39.785 --> 00:13:42.645
<00:13:39.785>So <00:13:39.940>we're <00:13:40.096>going <00:13:40.251>to <00:13:40.407>use <00:13:40.562>uh, <00:13:40.718>the <00:13:40.873>AI <00:13:41.029>lab, <00:13:41.185>uh,

325
00:13:42.645 --> 00:13:44.565
<00:13:42.645>there's <00:13:42.878>also <00:13:43.111>a <00:13:43.345>docker <00:13:43.578>gen <00:13:43.811>AI <00:13:44.045>stack

326
00:13:44.565 --> 00:13:46.045
<00:13:44.565>that <00:13:44.831>can <00:13:45.097>help <00:13:45.364>you <00:13:45.630>build <00:13:45.896>applications

327
00:13:46.115 --> 00:13:49.325
<00:13:46.115>with <00:13:46.270>embedding <00:13:46.426>databases <00:13:46.581>and <00:13:46.737>run <00:13:46.892>all <00:13:47.048>of <00:13:47.203>this <00:13:47.359>in <00:13:47.515>containers.

328
00:13:49.545 --> 00:13:50.845
<00:13:49.545>So <00:13:49.740>we're <00:13:49.935>gonna <00:13:50.130>see <00:13:50.325>that <00:13:50.520>as <00:13:50.715>well

329
00:13:51.465 --> 00:13:54.165
<00:13:51.465>and <00:13:51.581>uh, <00:13:51.698>let's, <00:13:51.815>let's <00:13:51.931>have <00:13:52.048>a <00:13:52.165>fun <00:13:52.281>time. <00:13:52.398>Yeah, <00:13:52.515>so <00:13:52.631>let's <00:13:52.748>do <00:13:52.865>it.

330
00:13:54.165 --> 00:13:57.125
<00:13:54.165>Yeah, <00:13:54.339>let's <00:13:54.515>do <00:13:54.689>it. <00:13:54.865>So <00:13:55.040>first <00:13:55.214>demo, <00:13:55.390>first <00:13:55.564>thing

331
00:13:57.125 --> 00:14:01.205
<00:13:57.125>that <00:13:57.280>we <00:13:57.436>need <00:13:57.591>is <00:13:57.747>to <00:13:57.902>run <00:13:58.058>our <00:13:58.213>local <00:13:58.369>item, <00:13:58.525>right?

332
00:14:01.295 --> 00:14:04.765
<00:14:01.295>Right. <00:14:01.450>So <00:14:01.606>how <00:14:01.761>we <00:14:01.917>can <00:14:02.072>do <00:14:02.228>it, <00:14:02.383>you <00:14:02.539>have <00:14:02.694>this

333
00:14:05.985 --> 00:14:08.125
<00:14:05.985>key <00:14:06.265>code <00:14:06.545>there <00:14:06.825>also <00:14:07.105>this <00:14:07.385>link,

334
00:14:08.215 --> 00:14:10.125
<00:14:08.215>everything <00:14:08.448>is <00:14:08.681>open <00:14:08.915>source <00:14:09.148>from <00:14:09.381>day <00:14:09.615>one.

335
00:14:10.715 --> 00:14:15.005
<00:14:10.715>Also, <00:14:10.855>um, <00:14:10.995>everything <00:14:11.135>it's <00:14:11.275>uh, <00:14:11.415>if <00:14:11.555>you <00:14:11.695>go <00:14:11.835>to <00:14:11.975>the <00:14:12.115>repository,

336
00:14:15.305 --> 00:14:17.485
<00:14:15.305>you <00:14:15.444>will <00:14:15.584>have <00:14:15.724>all <00:14:15.864>of <00:14:16.005>the <00:14:16.145>demos <00:14:16.285>that <00:14:16.425>we <00:14:16.564>have <00:14:16.704>available

337
00:14:17.745 --> 00:14:18.885
<00:14:17.745>and <00:14:17.950>also <00:14:18.155>the <00:14:18.360>slide <00:14:18.565>as <00:14:18.771>well.

338
00:14:19.305 --> 00:14:22.005
<00:14:19.305>But <00:14:19.505>the <00:14:19.704>first <00:14:19.905>thing <00:14:20.104>first, <00:14:20.305>so <00:14:20.505>let's <00:14:20.704>switch

339
00:14:22.065 --> 00:14:23.765
<00:14:22.065>to <00:14:22.531>Portman <00:14:22.998>desktop, <00:14:23.465>right?

340
00:14:23.785 --> 00:14:26.405
<00:14:23.785>And <00:14:23.925>inside <00:14:24.064>of <00:14:24.204>Portman <00:14:24.344>desktop <00:14:24.485>we <00:14:24.625>have <00:14:24.765>an <00:14:24.905>extension <00:14:25.045>as <00:14:25.185>well

341
00:14:26.795 --> 00:14:28.005
<00:14:26.795>that <00:14:27.067>you <00:14:27.339>can <00:14:27.611>install <00:14:27.884>it.

342
00:14:28.005 --> 00:14:30.445
<00:14:28.005>And <00:14:28.205>this <00:14:28.405>extension <00:14:28.605>that <00:14:28.805>we <00:14:29.005>already <00:14:29.205>install <00:14:29.405>it

343
00:14:30.445 --> 00:14:32.445
<00:14:30.445>as <00:14:30.725>well <00:14:31.005>is <00:14:31.285>called <00:14:31.565>AI <00:14:31.845>lab.

344
00:14:33.025 --> 00:14:36.005
<00:14:33.025>So <00:14:33.141>with <00:14:33.258>this <00:14:33.375>AI <00:14:33.491>lab <00:14:33.608>you <00:14:33.725>can <00:14:33.841>run <00:14:33.958>your <00:14:34.074>own <00:14:34.191>models <00:14:34.308>as <00:14:34.425>well,

345
00:14:36.425 --> 00:14:38.565
<00:14:36.425>but <00:14:36.658>which <00:14:36.891>one <00:14:37.125>we <00:14:37.358>wanted <00:14:37.591>to <00:14:37.824>run?

346
00:14:38.635 --> 00:14:41.125
<00:14:38.635>It's <00:14:38.775>a <00:14:38.915>good <00:14:39.055>point. <00:14:39.194>Well <00:14:39.335>we <00:14:39.475>can <00:14:39.615>select <00:14:39.755>the <00:14:39.895>different <00:14:40.035>models

347
00:14:41.125 --> 00:14:46.045
<00:14:41.125>that <00:14:41.218>we <00:14:41.311>have <00:14:41.405>and <00:14:41.498>in <00:14:41.591>this <00:14:41.685>case <00:14:41.778>we <00:14:41.871>are <00:14:41.965>using <00:14:42.058>ONEI <00:14:42.151>3 <00:14:42.245>0 <00:14:42.338>3 <00:14:42.431>8 <00:14:42.525>B

348
00:14:46.345 --> 00:14:48.285
<00:14:46.345>and <00:14:46.625>this <00:14:46.905>is <00:14:47.185>the <00:14:47.465>instruct <00:14:47.745>part.

349
00:14:48.305 --> 00:14:51.645
<00:14:48.305>So <00:14:48.505>it <00:14:48.704>instructed <00:14:48.905>to <00:14:49.104>be <00:14:49.305>like <00:14:49.505>a <00:14:49.704>chatbot.

350
00:14:51.665 --> 00:14:53.445
<00:14:51.665>So <00:14:52.131>it's <00:14:52.598>well <00:14:53.064>designed

351
00:14:53.465 --> 00:14:55.885
<00:14:53.465>and <00:14:53.640>it's <00:14:53.815>fine <00:14:53.990>tuned <00:14:54.165>to <00:14:54.340>be <00:14:54.515>able <00:14:54.690>to <00:14:54.865>respond

352
00:14:56.185 --> 00:14:59.805
<00:14:56.185>and <00:14:56.324>alter <00:14:56.464>to <00:14:56.604>help <00:14:56.744>us <00:14:56.885>to <00:14:57.025>do <00:14:57.165>code <00:14:57.305>assistance <00:14:57.444>to <00:14:57.584>uh,

353
00:14:59.825 --> 00:15:02.125
<00:14:59.825>do <00:15:00.000>these <00:15:00.175>agent <00:15:00.350>frameworks <00:15:00.525>and <00:15:00.700>these <00:15:00.875>type <00:15:01.050>of <00:15:01.225>things.

354
00:15:02.785 --> 00:15:05.565
<00:15:02.785>But <00:15:03.064>also <00:15:03.344>we <00:15:03.625>have <00:15:03.905>other <00:15:04.185>availables

355
00:15:05.665 --> 00:15:06.885
<00:15:05.665>and <00:15:05.848>you <00:15:06.031>can <00:15:06.214>import <00:15:06.396>it <00:15:06.579>as <00:15:06.763>well.

356
00:15:07.065 --> 00:15:10.845
<00:15:07.065>So <00:15:07.192>you <00:15:07.319>can <00:15:07.446>just <00:15:07.574>download <00:15:07.701>it <00:15:07.828>from <00:15:07.955>hanging <00:15:08.083>face <00:15:08.210>if <00:15:08.337>it's <00:15:08.465>not

357
00:15:10.845 --> 00:15:12.405
<00:15:10.845>here <00:15:11.195>and <00:15:11.545>just <00:15:11.895>use <00:15:12.245>it.

358
00:15:13.065 --> 00:15:15.805
<00:15:13.065>And <00:15:13.265>finally <00:15:13.465>we <00:15:13.665>need <00:15:13.865>to <00:15:14.065>run <00:15:14.265>it, <00:15:14.465>right?

359
00:15:16.185 --> 00:15:17.605
<00:15:16.185>So <00:15:16.504>we <00:15:16.824>already <00:15:17.143>run <00:15:17.463>it

360
00:15:17.865 --> 00:15:21.325
<00:15:17.865>and <00:15:18.040>we <00:15:18.215>have <00:15:18.390>our <00:15:18.565>model <00:15:18.740>in <00:15:18.915>here <00:15:19.090>running <00:15:19.265>locally

361
00:15:21.745 --> 00:15:23.125
<00:15:21.745>and <00:15:22.055>I'm <00:15:22.366>using <00:15:22.676>A <00:15:22.987>GPU,

362
00:15:23.185 --> 00:15:25.845
<00:15:23.185>but <00:15:23.359>also <00:15:23.535>you <00:15:23.709>can <00:15:23.885>run <00:15:24.060>the <00:15:24.234>exact <00:15:24.410>same <00:15:24.584>model

363
00:15:26.515 --> 00:15:28.325
<00:15:26.515>with <00:15:26.865>the <00:15:27.215>CPU <00:15:27.564>as <00:15:27.915>well.

364
00:15:28.425 --> 00:15:30.285
<00:15:28.425>So <00:15:28.658>we <00:15:28.891>have <00:15:29.125>both <00:15:29.358>sides <00:15:29.591>covered, <00:15:29.824>right?

365
00:15:30.465 --> 00:15:32.805
<00:15:30.465>We <00:15:30.698>have <00:15:30.931>CPU <00:15:31.165>and <00:15:31.398>we <00:15:31.631>have <00:15:31.865>GPU

366
00:15:33.305 --> 00:15:35.365
<00:15:33.305>and <00:15:33.479>the <00:15:33.655>name <00:15:33.829>of <00:15:34.005>the <00:15:34.180>model <00:15:34.354>is <00:15:34.530>in <00:15:34.704>here.

367
00:15:35.905 --> 00:15:39.725
<00:15:35.905>So <00:15:36.032>you <00:15:36.159>can <00:15:36.286>use <00:15:36.414>this <00:15:36.541>playground, <00:15:36.668>you <00:15:36.795>know <00:15:36.923>that <00:15:37.050>you <00:15:37.177>just <00:15:37.305>go

368
00:15:40.265 --> 00:15:44.925
<00:15:40.265>and <00:15:40.405>say <00:15:40.545>we <00:15:40.685>are <00:15:40.824>developers <00:15:40.965>and <00:15:41.105>we <00:15:41.245>will <00:15:41.385>create <00:15:41.525>the <00:15:41.665>background

369
00:15:45.465 --> 00:15:47.845
<00:15:45.465>and <00:15:45.698>inside <00:15:45.931>of <00:15:46.165>this <00:15:46.398>we <00:15:46.631>will <00:15:46.865>say

370
00:15:49.555 --> 00:15:51.045
<00:15:49.555>talk <00:15:50.002>like <00:15:50.449>a <00:15:50.896>pirate.

371
00:15:51.585 --> 00:15:56.205
<00:15:51.585>Mm. <00:15:51.760>And <00:15:51.935>we'll <00:15:52.110>say <00:15:52.285>hi, <00:15:52.460>we <00:15:52.635>are <00:15:52.810>in <00:15:52.985>the

372
00:15:57.145 --> 00:15:59.485
<00:15:57.145>uh, <00:15:57.845>what <00:15:58.545>conference

373
00:16:00.705 --> 00:16:02.725
<00:16:00.705>and <00:16:00.985>let's <00:16:01.265>see <00:16:01.545>what's <00:16:01.825>going <00:16:02.105>on.

374
00:16:03.185 --> 00:16:05.565
<00:16:03.185>The <00:16:03.385>Met, <00:16:03.584>we <00:16:03.785>are <00:16:03.984>in <00:16:04.185>the <00:16:04.385>what <00:16:04.584>conference?

375
00:16:05.685 --> 00:16:07.765
<00:16:05.685>A <00:16:06.035>Keep <00:16:06.385>your <00:16:06.734>catalyst <00:16:07.084>short.

376
00:16:07.765 --> 00:16:09.805
<00:16:07.765>Alright, <00:16:08.045>brilliant. <00:16:08.324>So <00:16:08.605>good <00:16:08.885>to <00:16:09.165>go.

377
00:16:10.075 --> 00:16:12.485
<00:16:10.075>Model's <00:16:10.308>running. <00:16:10.541>That's <00:16:10.775>the <00:16:11.008>model <00:16:11.241>running. <00:16:11.475>Brilliant.

378
00:16:12.825 --> 00:16:17.765
<00:16:12.825>So <00:16:12.952>we <00:16:13.079>will <00:16:13.206>need <00:16:13.334>to <00:16:13.461>check <00:16:13.588>also <00:16:13.715>other <00:16:13.843>things <00:16:13.970>that <00:16:14.097>we <00:16:14.225>want

379
00:16:17.875 --> 00:16:19.365
<00:16:17.875>that <00:16:18.210>it's <00:16:18.545>doing <00:16:18.880>rock, <00:16:19.216>right?

380
00:16:19.455 --> 00:16:21.925
<00:16:19.455>Right. <00:16:19.735>Well, <00:16:20.015>but <00:16:20.295>I'm <00:16:20.575>a <00:16:20.855>developer

381
00:16:21.945 --> 00:16:25.765
<00:16:21.945>and <00:16:22.100>sometimes <00:16:22.256>I <00:16:22.411>need <00:16:22.567>to <00:16:22.722>do <00:16:22.878>this <00:16:23.033>RAC <00:16:23.189>very <00:16:23.345>easily.

382
00:16:25.905 --> 00:16:28.485
<00:16:25.905>So <00:16:26.045>I <00:16:26.185>wanted <00:16:26.324>to <00:16:26.464>check <00:16:26.605>and <00:16:26.745>talk <00:16:26.885>with <00:16:27.025>my <00:16:27.165>uh, <00:16:27.305>documents.

383
00:16:28.485 --> 00:16:29.965
<00:16:28.485>And <00:16:28.707>sometimes <00:16:28.929>it's <00:16:29.151>a <00:16:29.373>little <00:16:29.595>bit <00:16:29.817>tricky

384
00:16:29.965 --> 00:16:32.085
<00:16:29.965>because <00:16:30.315>RAC <00:16:30.665>implies <00:16:31.015>a <00:16:31.365>lot

385
00:16:32.085 --> 00:16:34.365
<00:16:32.085>of <00:16:32.318>different <00:16:32.551>things <00:16:32.785>like <00:16:33.018>embedding <00:16:33.251>Vector <00:16:33.485>db

386
00:16:34.545 --> 00:16:36.045
<00:16:34.545>and <00:16:34.882>I'm <00:16:35.219>a <00:16:35.557>single <00:16:35.895>developer.

387
00:16:36.125 --> 00:16:38.205
<00:16:36.125>I <00:16:36.300>just <00:16:36.475>wanted <00:16:36.650>to <00:16:36.825>start <00:16:37.000>talking <00:16:37.175>with <00:16:37.350>my, <00:16:37.525>uh,

388
00:16:38.865 --> 00:16:40.165
<00:16:38.865>my <00:16:39.450>um, <00:16:40.035>application.

389
00:16:40.225 --> 00:16:44.125
<00:16:40.225>So <00:16:40.380>first <00:16:40.536>of <00:16:40.691>all, <00:16:40.847>this <00:16:41.002>is <00:16:41.158>anything <00:16:41.313>LLM <00:16:41.469>totally <00:16:41.625>free.

390
00:16:44.675 --> 00:16:48.765
<00:16:44.675>Also <00:16:44.802>you <00:16:44.929>can <00:16:45.056>download <00:16:45.184>it <00:16:45.311>and <00:16:45.438>use <00:16:45.565>uh, <00:16:45.693>in <00:16:45.820>your <00:16:45.947>own <00:16:46.074>laptop.

391
00:16:48.825 --> 00:16:52.165
<00:16:48.825>You <00:16:49.025>can <00:16:49.225>connect <00:16:49.425>it <00:16:49.625>to <00:16:49.825>your <00:16:50.025>own <00:16:50.225>LLM

392
00:16:52.165 --> 00:16:55.205
<00:16:52.165>because <00:16:52.365>in <00:16:52.564>here <00:16:52.765>we <00:16:52.964>have <00:16:53.165>open <00:16:53.365>ai, <00:16:53.564>API

393
00:16:55.465 --> 00:16:58.765
<00:16:55.465>and <00:16:55.745>we <00:16:56.025>are <00:16:56.305>using <00:16:56.585>these <00:16:56.865>services.

394
00:16:59.065 --> 00:17:01.605
<00:16:59.065>So <00:16:59.240>you <00:16:59.415>can <00:16:59.590>run <00:16:59.765>it <00:16:59.940>here, <00:17:00.115>you <00:17:00.290>just <00:17:00.465>copy

395
00:17:02.305 --> 00:17:05.405
<00:17:02.305>and <00:17:02.432>afterwards <00:17:02.559>you <00:17:02.686>can <00:17:02.814>put <00:17:02.941>it <00:17:03.068>here, <00:17:03.195>the <00:17:03.323>name <00:17:03.450>of <00:17:03.577>the <00:17:03.704>model

396
00:17:05.465 --> 00:17:06.765
<00:17:05.464>and <00:17:05.698>the <00:17:05.933>AP <00:17:06.166>key <00:17:06.401>is <00:17:06.635>none.

397
00:17:07.185 --> 00:17:08.365
<00:17:07.185>So <00:17:07.397>we <00:17:07.609>are <00:17:07.822>ready <00:17:08.034>to <00:17:08.247>go.

398
00:17:08.745 --> 00:17:11.725
<00:17:08.744>We <00:17:08.944>also <00:17:09.145>have <00:17:09.344>built <00:17:09.544>in <00:17:09.744>a <00:17:09.944>vector <00:17:10.145>database

399
00:17:11.725 --> 00:17:13.245
<00:17:11.724>that <00:17:11.998>we <00:17:12.272>will <00:17:12.545>put <00:17:12.819>our <00:17:13.092>document

400
00:17:13.785 --> 00:17:18.725
<00:17:13.785>and <00:17:13.960>afterwards <00:17:14.135>we <00:17:14.310>just <00:17:14.485>need <00:17:14.660>to <00:17:14.835>say <00:17:15.010>another <00:17:15.185>thread

401
00:17:18.785 --> 00:17:22.715
<00:17:18.785>and <00:17:18.985>say <00:17:19.185>hi, <00:17:19.385>let's <00:17:19.585>see <00:17:19.785>if <00:17:19.985>it's <00:17:20.185>responding.

402
00:17:23.665 --> 00:17:25.115
<00:17:23.665>Hopefully <00:17:24.317>it's <00:17:24.970>responding.

403
00:17:25.115 --> 00:17:29.315
<00:17:25.115>Alright, <00:17:25.270>so <00:17:25.426>we'll <00:17:25.581>have <00:17:25.737>hallucinations. <00:17:25.892>But <00:17:26.048>no <00:17:26.203>is <00:17:26.359>a <00:17:26.515>tool

404
00:17:29.945 --> 00:17:32.755
<00:17:29.945>That, <00:17:30.072>so <00:17:30.199>I <00:17:30.326>think <00:17:30.454>this <00:17:30.581>is <00:17:30.708>a <00:17:30.835>good <00:17:30.963>point, <00:17:31.090>uh, <00:17:31.217>to <00:17:31.345>understand

405
00:17:33.055 --> 00:17:34.475
<00:17:33.055>how <00:17:33.310>running <00:17:33.566>these <00:17:33.821>models <00:17:34.077>works, <00:17:34.332>right?

406
00:17:34.775 --> 00:17:36.115
<00:17:34.775>So <00:17:34.976>let's <00:17:35.177>say <00:17:35.378>you're <00:17:35.579>on <00:17:35.780>a <00:17:35.981>MacBook

407
00:17:36.115 --> 00:17:37.395
<00:17:36.115>or <00:17:36.499>a <00:17:36.883>Windows <00:17:37.267>computer

408
00:17:37.395 --> 00:17:39.115
<00:17:37.395>where <00:17:37.570>you're <00:17:37.745>gonna <00:17:37.920>be <00:17:38.095>trying <00:17:38.270>to <00:17:38.445>run <00:17:38.620>this <00:17:38.795>LLM.

409
00:17:40.295 --> 00:17:44.355
<00:17:40.295>So <00:17:40.450>you're <00:17:40.606>both <00:17:40.761>loading <00:17:40.917>the <00:17:41.072>LLM <00:17:41.228>into <00:17:41.383>your <00:17:41.539>virtual <00:17:41.695>memory,

410
00:17:44.535 --> 00:17:48.075
<00:17:44.535>but <00:17:44.690>you're <00:17:44.846>also <00:17:45.001>keeping <00:17:45.157>some <00:17:45.312>space <00:17:45.468>available <00:17:45.623>to <00:17:45.779>be <00:17:45.935>able

411
00:17:48.075 --> 00:17:51.325
<00:17:48.075>to <00:17:48.275>use <00:17:48.475>as <00:17:48.675>cash <00:17:48.875>in <00:17:49.075>order <00:17:49.275>to <00:17:49.475>encode

412
00:17:51.325 --> 00:17:53.165
<00:17:51.325>and <00:17:51.558>decode <00:17:51.791>the <00:17:52.025>responses <00:17:52.258>from <00:17:52.491>the <00:17:52.725>LLM.

413
00:17:53.505 --> 00:17:55.765
<00:17:53.505>And <00:17:53.645>so <00:17:53.785>when <00:17:53.925>we're <00:17:54.065>running <00:17:54.205>these <00:17:54.345>in <00:17:54.485>a <00:17:54.625>container, <00:17:54.765>let's <00:17:54.905>say

414
00:17:55.765 --> 00:17:57.565
<00:17:55.765>that <00:17:55.998>there's <00:17:56.231>an <00:17:56.465>issue <00:17:56.698>with <00:17:56.931>the <00:17:57.165>application

415
00:17:57.905 --> 00:17:59.125
<00:17:57.905>or <00:17:58.124>the <00:17:58.344>model <00:17:58.563>that <00:17:58.783>you're <00:17:59.003>using.

416
00:17:59.125 --> 00:18:01.605
<00:17:59.125>Yeah, <00:17:59.265>what <00:17:59.405>you're <00:17:59.545>able <00:17:59.685>to <00:17:59.825>do <00:17:59.965>is <00:18:00.105>just <00:18:00.245>restart <00:18:00.385>that <00:18:00.525>container

417
00:18:02.025 --> 00:18:03.365
<00:18:02.025>and <00:18:02.326>then, <00:18:02.628>uh, <00:18:02.929>essentially, <00:18:03.231>uh,

418
00:18:03.365 --> 00:18:05.005
<00:18:03.365>containers <00:18:03.715>are <00:18:04.065>stateless, <00:18:04.415>they're <00:18:04.765>ephemeral.

419
00:18:05.185 --> 00:18:07.165
<00:18:05.185>So <00:18:05.465>we're <00:18:05.745>restarting <00:18:06.024>that <00:18:06.304>completely <00:18:06.585>up

420
00:18:07.905 --> 00:18:10.045
<00:18:07.905>and <00:18:08.105>we've <00:18:08.305>kind <00:18:08.504>of <00:18:08.705>got <00:18:08.905>that <00:18:09.105>same <00:18:09.305>consistent

421
00:18:10.045 --> 00:18:11.405
<00:18:10.045>environment <00:18:10.249>that <00:18:10.453>we <00:18:10.657>had <00:18:10.861>the <00:18:11.065>first <00:18:11.269>time.

422
00:18:11.625 --> 00:18:14.205
<00:18:11.625>And <00:18:11.780>that's <00:18:11.936>why <00:18:12.091>it's <00:18:12.247>easy <00:18:12.402>once <00:18:12.558>you're <00:18:12.713>developing <00:18:12.869>these <00:18:13.025>in

423
00:18:14.205 --> 00:18:17.285
<00:18:14.205>containers, <00:18:14.345>you're <00:18:14.485>able <00:18:14.625>to <00:18:14.764>scale <00:18:14.905>this <00:18:15.044>and <00:18:15.185>move <00:18:15.324>this <00:18:15.465>to <00:18:15.605>GKE

424
00:18:17.285 --> 00:18:18.885
<00:18:17.285>or <00:18:17.635>Kubernetes <00:18:17.985>or <00:18:18.335>wherever <00:18:18.685>you

425
00:18:19.085 --> 00:18:20.165
<00:18:19.085>actually <00:18:19.409>deploy <00:18:19.733>your <00:18:20.057>applications.

426
00:18:20.555 --> 00:18:23.045
<00:18:20.555>Yeah, <00:18:20.788>so <00:18:21.021>let's <00:18:21.255>see <00:18:21.488>if <00:18:21.721>it's <00:18:21.955>responding

427
00:18:23.045 --> 00:18:27.245
<00:18:23.045>because <00:18:23.161>we <00:18:23.278>just, <00:18:23.395>we <00:18:23.511>started <00:18:23.628>as <00:18:23.745>is <00:18:23.861>and <00:18:23.978>we <00:18:24.095>are <00:18:24.211>back <00:18:24.328>on <00:18:24.445>track.

428
00:18:27.245 --> 00:18:28.805
<00:18:27.245>There <00:18:27.478>we <00:18:27.711>go. <00:18:27.945>And <00:18:28.178>we <00:18:28.411>also <00:18:28.645>demonstrate

429
00:18:28.805 --> 00:18:31.085
<00:18:28.805>that <00:18:29.155>demo <00:18:29.505>guys <00:18:29.855>Demo <00:18:30.205>guide.

430
00:18:31.385 --> 00:18:34.485
<00:18:31.385>So <00:18:31.585>if <00:18:31.785>we <00:18:31.985>ask <00:18:32.185>the, <00:18:32.385>give <00:18:32.585>some <00:18:32.785>sessions

431
00:18:35.265 --> 00:18:38.325
<00:18:35.265>around <00:18:35.420>the <00:18:35.576>we <00:18:35.731>are <00:18:35.887>developers, <00:18:36.042>if <00:18:36.198>we <00:18:36.353>are <00:18:36.509>asking <00:18:36.665>that

432
00:18:39.315 --> 00:18:40.405
<00:18:39.315>this <00:18:39.805>will <00:18:40.296>say

433
00:18:40.465 --> 00:18:42.525
<00:18:40.465>and <00:18:40.698>will <00:18:40.931>assassinate <00:18:41.165>just <00:18:41.398>gimme <00:18:41.631>a <00:18:41.865>sec

434
00:18:42.525 --> 00:18:43.525
<00:18:42.525>that <00:18:42.705>I <00:18:42.885>will <00:18:43.065>increase <00:18:43.245>the <00:18:43.425>funds

435
00:18:43.595 --> 00:18:47.085
<00:18:43.595>because <00:18:43.828>we <00:18:44.061>cannot <00:18:44.295>provide <00:18:44.528>specific <00:18:44.761>details <00:18:44.995>about

436
00:18:47.635 --> 00:18:49.725
<00:18:47.635>what <00:18:47.985>are <00:18:48.335>this <00:18:48.685>specific <00:18:49.035>session

437
00:18:49.725 --> 00:18:54.605
<00:18:49.725>because <00:18:49.958>the <00:18:50.191>LLM <00:18:50.425>was <00:18:50.658>trained <00:18:50.891>until <00:18:51.125>2021,

438
00:18:54.625 --> 00:18:56.445
<00:18:54.625>2022 <00:18:55.091>as <00:18:55.558>well, <00:18:56.025>right?

439
00:18:56.445 --> 00:18:58.365
<00:18:56.445>Alright, <00:18:56.794>so <00:18:57.145>let's <00:18:57.495>do <00:18:57.845>this

440
00:18:58.825 --> 00:19:02.965
<00:18:58.825>but <00:18:59.058>with <00:18:59.291>some <00:18:59.525>special, <00:18:59.758>um, <00:18:59.991>um, <00:19:00.225>information

441
00:19:03.005 --> 00:19:06.085
<00:19:03.005>because <00:19:03.160>the <00:19:03.316>thing <00:19:03.471>that <00:19:03.627>we <00:19:03.782>did <00:19:03.938>is <00:19:04.093>that <00:19:04.249>we <00:19:04.405>use

442
00:19:06.105 --> 00:19:09.765
<00:19:06.105>and <00:19:06.338>we <00:19:06.571>fit <00:19:06.805>with <00:19:07.038>this <00:19:07.271>agent, <00:19:07.505>um,

443
00:19:10.595 --> 00:19:13.245
<00:19:10.595>with <00:19:10.770>this <00:19:10.945>um, <00:19:11.120>agenda <00:19:11.295>that <00:19:11.470>is <00:19:11.645>we <00:19:11.820>are <00:19:11.995>developers.

444
00:19:13.465 --> 00:19:15.245
<00:19:13.465>So <00:19:13.665>the <00:19:13.865>thing <00:19:14.064>that <00:19:14.264>we <00:19:14.465>did, <00:19:14.665>it's <00:19:14.865>to

445
00:19:15.785 --> 00:19:19.565
<00:19:15.785>fit <00:19:16.018>this <00:19:16.251>conference <00:19:16.485>program <00:19:16.718>into <00:19:16.951>the <00:19:17.185>application

446
00:19:19.955 --> 00:19:22.525
<00:19:19.955>that <00:19:20.235>contains <00:19:20.514>all <00:19:20.794>of <00:19:21.074>the <00:19:21.355>information.

447
00:19:22.945 --> 00:19:26.205
<00:19:22.945>And <00:19:23.072>as <00:19:23.199>you <00:19:23.326>can <00:19:23.454>see, <00:19:23.581>if <00:19:23.708>we <00:19:23.835>ask <00:19:23.963>the <00:19:24.090>exact <00:19:24.217>same <00:19:24.345>question,

448
00:19:26.515 --> 00:19:30.445
<00:19:26.515>give <00:19:26.670>me <00:19:26.826>some <00:19:26.981>tops <00:19:27.137>around <00:19:27.292>the <00:19:27.448>we <00:19:27.603>are <00:19:27.759>developers <00:19:27.915>conference.

449
00:19:30.985 --> 00:19:33.445
<00:19:30.985>We <00:19:31.218>will <00:19:31.451>get <00:19:31.685>all <00:19:31.918>of <00:19:32.151>the <00:19:32.385>information

450
00:19:33.445 --> 00:19:37.645
<00:19:33.445>because <00:19:33.600>the <00:19:33.756>thing <00:19:33.911>that <00:19:34.067>it's <00:19:34.222>doing, <00:19:34.378>it's <00:19:34.533>retrieving <00:19:34.689>this, <00:19:34.845>uh,

451
00:19:38.085 --> 00:19:39.445
<00:19:38.085>document, <00:19:38.697>it's <00:19:39.309>splitting

452
00:19:39.445 --> 00:19:41.845
<00:19:39.445>and <00:19:39.725>then <00:19:40.004>putting <00:19:40.284>into <00:19:40.564>embeddings <00:19:40.845>automatically

453
00:19:42.065 --> 00:19:44.285
<00:19:42.065>and <00:19:42.345>is <00:19:42.625>receiving <00:19:42.905>all <00:19:43.185>of <00:19:43.465>this

454
00:19:44.385 --> 00:19:48.325
<00:19:44.385>and <00:19:44.525>querying <00:19:44.665>two <00:19:44.805>the <00:19:44.945>document <00:19:45.085>as <00:19:45.225>well <00:19:45.365>with <00:19:45.504>all <00:19:45.645>of <00:19:45.785>this

455
00:19:48.785 --> 00:19:52.405
<00:19:48.785>and <00:19:48.985>enhancing <00:19:49.185>the <00:19:49.385>answer, <00:19:49.585>um, <00:19:49.785>using <00:19:49.985>this, <00:19:50.185>uh,

456
00:19:52.685 --> 00:19:53.765
<00:19:52.685>material <00:19:53.171>augmented <00:19:53.657>generation.

457
00:19:54.065 --> 00:19:57.685
<00:19:54.065>So <00:19:54.298>that's <00:19:54.531>the <00:19:54.765>difference <00:19:54.998>between <00:19:55.231>not <00:19:55.465>having

458
00:19:58.655 --> 00:20:00.645
<00:19:58.655>these <00:19:58.935>embeddings <00:19:59.215>and <00:19:59.495>this, <00:19:59.774>uh, <00:20:00.055>information

459
00:20:00.785 --> 00:20:02.645
<00:20:00.785>and <00:20:01.135>having <00:20:01.485>also <00:20:01.835>these <00:20:02.185>embeddings

460
00:20:02.645 --> 00:20:03.725
<00:20:02.645>and <00:20:02.888>this <00:20:03.130>information <00:20:03.374>as <00:20:03.617>well.

461
00:20:04.235 --> 00:20:08.525
<00:20:04.235>Very, <00:20:04.375>very <00:20:04.514>cool. <00:20:04.655>So <00:20:04.794>we <00:20:04.935>covered <00:20:05.074>the <00:20:05.215>model <00:20:05.354>serving <00:20:05.495>and <00:20:05.635>rac.

462
00:20:08.835 --> 00:20:13.655
<00:20:08.835>Alright, <00:20:08.990>so <00:20:09.146>next <00:20:09.301>thing <00:20:09.457>that <00:20:09.612>will <00:20:09.768>be <00:20:09.923>code <00:20:10.079>assistance, <00:20:10.235>right?

463
00:20:13.655 --> 00:20:14.975
<00:20:13.655>Because <00:20:13.853>we <00:20:14.051>need <00:20:14.249>code <00:20:14.447>assistant <00:20:14.645>as <00:20:14.842>well.

464
00:20:15.075 --> 00:20:16.255
<00:20:15.075>I'm <00:20:15.429>a <00:20:15.783>developer, <00:20:16.137>right?

465
00:20:16.255 --> 00:20:18.895
<00:20:16.255>And <00:20:16.455>I <00:20:16.655>need <00:20:16.855>to <00:20:17.055>be <00:20:17.255>able <00:20:17.455>to <00:20:17.655>go

466
00:20:19.195 --> 00:20:24.015
<00:20:19.195>and <00:20:19.335>help <00:20:19.475>me <00:20:19.615>to <00:20:19.754>for <00:20:19.895>example, <00:20:20.034>develop <00:20:20.175>uh, <00:20:20.314>my <00:20:20.455>own <00:20:20.595>obligation.

467
00:20:24.155 --> 00:20:25.575
<00:20:24.155>I'm <00:20:24.581>a <00:20:25.007>Python <00:20:25.433>developer,

468
00:20:25.755 --> 00:20:28.815
<00:20:25.755>but <00:20:25.895>also <00:20:26.035>you <00:20:26.175>can, <00:20:26.315>uh, <00:20:26.455>do <00:20:26.595>whatever <00:20:26.735>you <00:20:26.875>want <00:20:27.015>like <00:20:27.155>Java,

469
00:20:29.165 --> 00:20:31.135
<00:20:29.165>like <00:20:29.340>uh, <00:20:29.514>TypeScript <00:20:29.690>and <00:20:29.865>so <00:20:30.040>on <00:20:30.215>and <00:20:30.389>so <00:20:30.565>forth.

470
00:20:31.915 --> 00:20:35.895
<00:20:31.915>So <00:20:32.055>first <00:20:32.195>things <00:20:32.335>first, <00:20:32.475>we <00:20:32.615>need <00:20:32.754>to <00:20:32.895>go <00:20:33.034>to <00:20:33.175>our <00:20:33.315>obligation

471
00:20:35.955 --> 00:20:38.695
<00:20:35.955>and <00:20:36.235>we <00:20:36.514>will <00:20:36.794>use <00:20:37.074>continue, <00:20:37.355>continue.

472
00:20:38.725 --> 00:20:41.575
<00:20:38.725>It's <00:20:38.925>a <00:20:39.125>extension <00:20:39.324>also <00:20:39.524>that <00:20:39.725>you <00:20:39.925>can <00:20:40.125>go

473
00:20:41.675 --> 00:20:44.775
<00:20:41.675>and <00:20:41.830>you <00:20:41.986>can <00:20:42.141>say, <00:20:42.297>um, <00:20:42.452>that <00:20:42.608>I <00:20:42.763>want <00:20:42.919>to <00:20:43.075>go

474
00:20:45.195 --> 00:20:46.855
<00:20:45.195>and <00:20:45.544>install <00:20:45.895>this, <00:20:46.245>uh, <00:20:46.595>extension

475
00:20:47.115 --> 00:20:49.885
<00:20:47.115>that <00:20:47.315>is <00:20:47.515>an <00:20:47.715>open <00:20:47.915>source <00:20:48.115>AI <00:20:48.315>code <00:20:48.515>assistant

476
00:20:50.305 --> 00:20:53.325
<00:20:50.305>and <00:20:50.480>we <00:20:50.655>will <00:20:50.830>power <00:20:51.005>this <00:20:51.180>local <00:20:51.355>assistant <00:20:51.530>as <00:20:51.705>well.

477
00:20:53.985 --> 00:20:56.165
<00:20:53.985>As <00:20:54.185>you <00:20:54.385>can <00:20:54.584>see, <00:20:54.784>I <00:20:54.985>have <00:20:55.185>different <00:20:55.385>options

478
00:20:56.625 --> 00:21:01.405
<00:20:56.625>and <00:20:56.765>we <00:20:56.905>are <00:20:57.045>using <00:20:57.185>our <00:20:57.325>local <00:20:57.465>model <00:20:57.605>with <00:20:57.745>this <00:20:57.885>local <00:20:58.025>host

479
00:21:01.425 --> 00:21:03.325
<00:21:01.425>as <00:21:01.705>well <00:21:01.985>with <00:21:02.264>uh, <00:21:02.544>the <00:21:02.825>title.

480
00:21:03.625 --> 00:21:05.525
<00:21:03.625>And <00:21:03.905>um, <00:21:04.185>as <00:21:04.465>you <00:21:04.745>can <00:21:05.025>see,

481
00:21:05.545 --> 00:21:08.925
<00:21:05.545>we <00:21:05.720>can <00:21:05.895>also <00:21:06.070>put <00:21:06.245>different <00:21:06.420>options <00:21:06.595>like <00:21:06.770>the <00:21:06.945>temperature

482
00:21:09.425 --> 00:21:11.325
<00:21:09.425>and <00:21:09.625>um, <00:21:09.825>different <00:21:10.024>things <00:21:10.225>that <00:21:10.425>we <00:21:10.625>will <00:21:10.825>see.

483
00:21:12.105 --> 00:21:15.805
<00:21:12.105>But <00:21:12.245>let's <00:21:12.385>see, <00:21:12.525>first <00:21:12.665>of <00:21:12.805>all, <00:21:12.945>if <00:21:13.085>I <00:21:13.225>can <00:21:13.365>do <00:21:13.505>this,

484
00:21:17.085 --> 00:21:21.565
<00:21:17.085>I <00:21:17.285>have <00:21:17.485>my, <00:21:17.685>um, <00:21:17.885>just <00:21:18.085>gimme <00:21:18.285>a <00:21:18.485>sec.

485
00:21:22.125 --> 00:21:26.445
<00:21:22.125>I <00:21:22.405>have <00:21:22.685>my, <00:21:22.965>um, <00:21:23.245>here <00:21:23.525>everything,

486
00:21:26.635 --> 00:21:28.445
<00:21:26.635>it's <00:21:26.915>open <00:21:27.195>source <00:21:27.475>from <00:21:27.754>day <00:21:28.035>one

487
00:21:29.265 --> 00:21:32.725
<00:21:29.265>and <00:21:29.405>the <00:21:29.545>first <00:21:29.685>thing <00:21:29.825>that <00:21:29.965>we <00:21:30.105>will <00:21:30.245>do <00:21:30.385>is <00:21:30.525>ask <00:21:30.665>continue

488
00:21:33.415 --> 00:21:35.445
<00:21:33.415>using <00:21:33.764>my <00:21:34.115>local <00:21:34.465>granite <00:21:34.815>coach.

489
00:21:36.025 --> 00:21:38.685
<00:21:36.025>One <00:21:36.200>thing <00:21:36.375>that <00:21:36.550>it's <00:21:36.725>called <00:21:36.900>write <00:21:37.075>up <00:21:37.250>python <00:21:37.425>fast,

490
00:21:38.805 --> 00:21:40.365
<00:21:38.805>API <00:21:39.155>example <00:21:39.505>with <00:21:39.855>two <00:21:40.205>roots.

491
00:21:40.865 --> 00:21:44.125
<00:21:40.865>And <00:21:41.145>hopefully <00:21:41.425>if <00:21:41.705>it's <00:21:41.985>not <00:21:42.265>hallucinating

492
00:21:44.425 --> 00:21:47.765
<00:21:44.425>and <00:21:44.580>everything, <00:21:44.736>it's <00:21:44.891>running <00:21:45.047>okay, <00:21:45.202>we <00:21:45.358>will <00:21:45.513>get <00:21:45.669>a <00:21:45.825>response

493
00:21:47.765 --> 00:21:50.085
<00:21:47.765>because <00:21:47.965>the <00:21:48.165>thing <00:21:48.365>it's, <00:21:48.565>uh, <00:21:48.765>that <00:21:48.965>it's <00:21:49.165>doing,

494
00:21:50.275 --> 00:21:55.245
<00:21:50.275>it's <00:21:50.430>using <00:21:50.586>my <00:21:50.741>local <00:21:50.897>LLM <00:21:51.052>in <00:21:51.208>order <00:21:51.363>to <00:21:51.519>ask <00:21:51.675>query

495
00:21:55.705 --> 00:21:57.325
<00:21:55.705>and <00:21:56.054>power <00:21:56.405>everything <00:21:56.754>using <00:21:57.105>this.

496
00:21:57.705 --> 00:22:01.445
<00:21:57.705>So <00:21:57.879>as <00:21:58.054>you <00:21:58.230>can <00:21:58.405>see <00:21:58.580>it's <00:21:58.754>generating <00:21:58.929>the <00:21:59.105>code

497
00:22:01.465 --> 00:22:04.085
<00:22:01.465>and <00:22:01.665>it's <00:22:01.865>pretty <00:22:02.064>good <00:22:02.264>because <00:22:02.465>it's <00:22:02.665>understanding <00:22:02.865>also

498
00:22:04.085 --> 00:22:05.605
<00:22:04.085>that <00:22:04.427>I'm <00:22:04.769>using <00:22:05.111>fast <00:22:05.453>API,

499
00:22:05.955 --> 00:22:08.005
<00:22:05.955>it's <00:22:06.235>giving <00:22:06.514>me <00:22:06.794>the <00:22:07.074>different <00:22:07.355>inputs,

500
00:22:08.115 --> 00:22:09.565
<00:22:08.115>it's <00:22:08.376>giving <00:22:08.637>me <00:22:08.898>the <00:22:09.159>different <00:22:09.420>things

501
00:22:10.105 --> 00:22:13.325
<00:22:10.105>and <00:22:10.305>it's <00:22:10.505>also <00:22:10.705>ready <00:22:10.905>to <00:22:11.105>just <00:22:11.305>copy <00:22:11.505>paste.

502
00:22:13.825 --> 00:22:18.605
<00:22:13.825>And <00:22:13.941>it's <00:22:14.058>one <00:22:14.175>by <00:22:14.291>one <00:22:14.408>giving <00:22:14.525>me <00:22:14.641>the, <00:22:14.758>uh, <00:22:14.875>steps <00:22:14.991>that <00:22:15.108>I <00:22:15.225>need

503
00:22:18.605 --> 00:22:21.645
<00:22:18.605>to <00:22:18.805>do <00:22:19.005>in <00:22:19.205>order <00:22:19.405>to <00:22:19.605>install <00:22:19.805>my <00:22:20.005>application.

504
00:22:21.725 --> 00:22:26.685
<00:22:21.725>I <00:22:21.852>test <00:22:21.979>it. <00:22:22.106>Alright, <00:22:22.234>so, <00:22:22.361>uh, <00:22:22.488>another <00:22:22.615>thing <00:22:22.743>that <00:22:22.870>uh, <00:22:22.997>I <00:22:23.125>need

505
00:22:26.685 --> 00:22:31.005
<00:22:26.685>to <00:22:26.825>do, <00:22:26.965>it's <00:22:27.105>um, <00:22:27.245>you <00:22:27.385>for <00:22:27.524>example, <00:22:27.665>you <00:22:27.804>grab <00:22:27.945>this <00:22:28.085>code

506
00:22:31.105 --> 00:22:32.285
<00:22:31.105>and <00:22:31.370>you <00:22:31.636>don't <00:22:31.901>understand <00:22:32.167>it

507
00:22:32.285 --> 00:22:35.005
<00:22:32.285>because <00:22:32.460>you <00:22:32.635>are <00:22:32.810>not, <00:22:32.985>uh, <00:22:33.160>familiar <00:22:33.335>with <00:22:33.510>my <00:22:33.685>application.

508
00:22:35.065 --> 00:22:37.805
<00:22:35.065>So <00:22:35.220>you <00:22:35.376>need <00:22:35.531>to <00:22:35.687>be <00:22:35.842>able <00:22:35.998>to <00:22:36.153>explain <00:22:36.309>it, <00:22:36.465>right?

509
00:22:37.845 --> 00:22:41.045
<00:22:37.845>Alright, <00:22:38.045>brilliant. <00:22:38.245>The <00:22:38.445>same <00:22:38.645>way <00:22:38.845>that <00:22:39.045>we <00:22:39.245>have

510
00:22:41.395 --> 00:22:46.045
<00:22:41.395>with <00:22:41.550>these <00:22:41.706>copilots, <00:22:41.861>we <00:22:42.017>have <00:22:42.172>also <00:22:42.328>our, <00:22:42.483>um, <00:22:42.639>code <00:22:42.795>system.

511
00:22:46.415 --> 00:22:47.965
<00:22:46.415>Everything <00:22:46.694>that <00:22:46.973>you <00:22:47.252>can <00:22:47.531>see <00:22:47.810>here,

512
00:22:48.015 --> 00:22:51.245
<00:22:48.015>everything <00:22:48.215>is <00:22:48.415>running <00:22:48.615>locally <00:22:48.815>with <00:22:49.015>my <00:22:49.215>own <00:22:49.415>LLM.

513
00:22:51.545 --> 00:22:56.525
<00:22:51.545>And <00:22:51.685>I'm <00:22:51.825>able <00:22:51.965>also <00:22:52.105>should <00:22:52.245>you <00:22:52.385>things <00:22:52.525>like, <00:22:52.665>hey, <00:22:52.805>explain <00:22:52.945>me

514
00:22:57.035 --> 00:23:00.245
<00:22:57.035>this <00:22:57.268>code <00:22:57.501>will <00:22:57.735>analyze <00:22:57.968>the <00:22:58.201>different <00:22:58.435>code

515
00:23:00.545 --> 00:23:03.325
<00:23:00.545>and <00:23:00.745>will <00:23:00.945>explain <00:23:01.145>also <00:23:01.345>part <00:23:01.545>of <00:23:01.745>this <00:23:01.945>code.

516
00:23:03.325 --> 00:23:06.725
<00:23:03.325>Because <00:23:03.525>if <00:23:03.725>you <00:23:03.925>see <00:23:04.125>here, <00:23:04.325>it's <00:23:04.525>just <00:23:04.725>checking

517
00:23:07.185 --> 00:23:08.845
<00:23:07.185>and <00:23:07.651>it's <00:23:08.118>enabling <00:23:08.585>also

518
00:23:08.865 --> 00:23:11.925
<00:23:08.865>and <00:23:09.020>explaining <00:23:09.176>me <00:23:09.331>the <00:23:09.487>different <00:23:09.642>code <00:23:09.798>that <00:23:09.953>uh, <00:23:10.109>it's <00:23:10.265>doing

519
00:23:12.185 --> 00:23:13.245
<00:23:12.185>and <00:23:12.503>it's <00:23:12.821>capable <00:23:13.139>to

520
00:23:13.665 --> 00:23:18.445
<00:23:13.665>and <00:23:13.840>understand <00:23:14.014>that <00:23:14.190>it's <00:23:14.365>um, <00:23:14.540>missing <00:23:14.715>one <00:23:14.889>specific <00:23:15.065>library

521
00:23:18.635 --> 00:23:20.765
<00:23:18.635>because <00:23:18.915>this <00:23:19.195>code <00:23:19.475>was <00:23:19.754>not <00:23:20.035>accurate.

522
00:23:21.225 --> 00:23:23.165
<00:23:21.225>So <00:23:21.425>with <00:23:21.625>this <00:23:21.824>I <00:23:22.024>can <00:23:22.225>accept <00:23:22.425>as <00:23:22.625>well.

523
00:23:23.505 --> 00:23:26.685
<00:23:23.505>And <00:23:23.680>this <00:23:23.855>is <00:23:24.030>also <00:23:24.205>producing <00:23:24.380>me <00:23:24.555>the <00:23:24.730>different <00:23:24.905>endpoints.

524
00:23:26.905 --> 00:23:31.165
<00:23:26.905>So <00:23:27.045>it's <00:23:27.185>able <00:23:27.325>to <00:23:27.465>first, <00:23:27.605>uh, <00:23:27.745>be <00:23:27.885>also <00:23:28.024>a <00:23:28.165>chat, <00:23:28.305>uh,

525
00:23:31.515 --> 00:23:34.165
<00:23:31.515>code <00:23:31.748>template <00:23:31.981>but <00:23:32.215>can <00:23:32.448>analyze <00:23:32.681>my <00:23:32.915>code

526
00:23:34.425 --> 00:23:38.485
<00:23:34.425>and <00:23:34.552>things <00:23:34.679>that <00:23:34.806>are <00:23:34.934>pretty <00:23:35.061>cool <00:23:35.188>as <00:23:35.315>well <00:23:35.443>that <00:23:35.570>um, <00:23:35.697>it's <00:23:35.825>able

527
00:23:38.545 --> 00:23:43.165
<00:23:38.545>to <00:23:38.685>do <00:23:38.825>things <00:23:38.965>like, <00:23:39.105>um, <00:23:39.245>point <00:23:39.385>out <00:23:39.525>to <00:23:39.665>the <00:23:39.805>um, <00:23:39.945>different

528
00:23:43.685 --> 00:23:47.005
<00:23:43.685>MCP <00:23:43.860>servers, <00:23:44.034>uh, <00:23:44.210>can <00:23:44.385>also <00:23:44.560>put <00:23:44.735>um, <00:23:44.909>different <00:23:45.085>files.

529
00:23:47.105 --> 00:23:48.925
<00:23:47.105>So <00:23:47.455>I <00:23:47.805>can <00:23:48.155>um, <00:23:48.505>see

530
00:23:48.985 --> 00:23:52.165
<00:23:48.985>and <00:23:49.218>I <00:23:49.451>can <00:23:49.685>uh, <00:23:49.918>ask, <00:23:50.151>hey, <00:23:50.385>just

531
00:23:53.375 --> 00:23:57.725
<00:23:53.375>check <00:23:53.530>this <00:23:53.686>and <00:23:53.841>uh, <00:23:53.997>try <00:23:54.152>to <00:23:54.308>um, <00:23:54.463>add <00:23:54.619>another <00:23:54.775>endpoint,

532
00:23:58.985 --> 00:24:00.205
<00:23:58.985>um, <00:23:59.533>with <00:24:00.082>hello.

533
00:24:00.905 --> 00:24:03.885
<00:24:00.905>And <00:24:01.138>with <00:24:01.371>this <00:24:01.605>we <00:24:01.838>know <00:24:02.071>the <00:24:02.305>context

534
00:24:03.915 --> 00:24:06.485
<00:24:03.915>because <00:24:04.090>I'm <00:24:04.264>providing <00:24:04.440>me, <00:24:04.615>uh, <00:24:04.790>I'm <00:24:04.965>providing <00:24:05.139>the <00:24:05.315>context

535
00:24:06.555 --> 00:24:09.205
<00:24:06.555>that <00:24:06.755>um, <00:24:06.955>I <00:24:07.155>need <00:24:07.355>to <00:24:07.555>uh, <00:24:07.755>add <00:24:07.955>this

536
00:24:09.465 --> 00:24:11.605
<00:24:09.465>and <00:24:09.745>will <00:24:10.024>use <00:24:10.304>the <00:24:10.584>local <00:24:10.865>LLM.

537
00:24:11.855 --> 00:24:14.165
<00:24:11.855>Again, <00:24:12.055>everything <00:24:12.255>that <00:24:12.455>you <00:24:12.655>can <00:24:12.855>see <00:24:13.055>it's <00:24:13.255>local

538
00:24:14.465 --> 00:24:18.245
<00:24:14.465>and <00:24:14.639>will <00:24:14.814>produce <00:24:14.990>uh, <00:24:15.165>everything <00:24:15.340>um, <00:24:15.514>locally <00:24:15.689>in <00:24:15.865>order

539
00:24:18.245 --> 00:24:19.405
<00:24:18.245>to <00:24:18.592>change <00:24:18.941>the <00:24:19.289>code

540
00:24:19.705 --> 00:24:23.605
<00:24:19.705>and <00:24:19.845>um, <00:24:19.985>will <00:24:20.125>also <00:24:20.264>analyze <00:24:20.405>if <00:24:20.544>there <00:24:20.685>is <00:24:20.824>anything, <00:24:20.965>uh, <00:24:21.105>missing.

541
00:24:23.945 --> 00:24:25.445
<00:24:23.945>So <00:24:24.282>it's <00:24:24.620>producing <00:24:24.957>this, <00:24:25.294>this

542
00:24:26.065 --> 00:24:29.085
<00:24:26.065>and <00:24:26.220>it's <00:24:26.376>also <00:24:26.531>analyzing <00:24:26.687>the <00:24:26.842>different <00:24:26.998>code <00:24:27.153>that <00:24:27.309>I <00:24:27.465>have.

543
00:24:29.585 --> 00:24:32.005
<00:24:29.585>And <00:24:29.785>in <00:24:29.985>fact <00:24:30.185>afterwards <00:24:30.385>it's <00:24:30.585>producing <00:24:30.785>this <00:24:30.985>hello

544
00:24:32.265 --> 00:24:33.765
<00:24:32.265>and <00:24:32.602>it's <00:24:32.940>asking <00:24:33.277>me, <00:24:33.615>Hey,

545
00:24:33.915 --> 00:24:36.605
<00:24:33.915>I've <00:24:34.055>inserted <00:24:34.195>this <00:24:34.335>in <00:24:34.475>the <00:24:34.615>code <00:24:34.754>you <00:24:34.895>wanted <00:24:35.034>to <00:24:35.175>run <00:24:35.315>it.

546
00:24:37.165 --> 00:24:40.245
<00:24:37.165>Brilliant. <00:24:37.398>Everything <00:24:37.631>that <00:24:37.865>you <00:24:38.098>can <00:24:38.331>see <00:24:38.565>here,

547
00:24:40.955 --> 00:24:43.285
<00:24:40.955>obviously <00:24:41.235>you <00:24:41.514>can <00:24:41.794>run <00:24:42.074>different <00:24:42.355>models

548
00:24:43.505 --> 00:24:45.365
<00:24:43.505>so <00:24:43.705>you <00:24:43.905>can <00:24:44.105>switch <00:24:44.305>your <00:24:44.505>models <00:24:44.705>in <00:24:44.905>order

549
00:24:45.425 --> 00:24:46.885
<00:24:45.425>to <00:24:45.753>use <00:24:46.081>different <00:24:46.410>use <00:24:46.739>cases.

550
00:24:47.025 --> 00:24:49.285
<00:24:47.025>So <00:24:47.258>if <00:24:47.491>you're <00:24:47.725>using <00:24:47.958>for <00:24:48.191>example, <00:24:48.425>multi-model

551
00:24:49.285 --> 00:24:51.485
<00:24:49.285>with <00:24:49.460>different <00:24:49.635>images, <00:24:49.810>you <00:24:49.985>can <00:24:50.160>run <00:24:50.335>it <00:24:50.510>as <00:24:50.685>well.

552
00:24:52.315 --> 00:24:57.165
<00:24:52.315>Alright, <00:24:52.470>last <00:24:52.626>but <00:24:52.781>not <00:24:52.937>least, <00:24:53.092>Ooh, <00:24:53.248>the <00:24:53.403>identical <00:24:53.559>ai. <00:24:53.715>Nice.

553
00:24:57.305 --> 00:25:00.125
<00:24:57.305>So <00:24:57.480>we <00:24:57.655>decide, <00:24:57.830>uh, <00:24:58.005>we <00:24:58.180>designed <00:24:58.355>it <00:24:58.530>one <00:24:58.705>thing

554
00:25:00.395 --> 00:25:02.365
<00:25:00.395>that <00:25:00.628>it's <00:25:00.861>called <00:25:01.095>a <00:25:01.328>very <00:25:01.561>simple <00:25:01.795>app,

555
00:25:02.865 --> 00:25:06.005
<00:25:02.865>but <00:25:03.098>this <00:25:03.331>application, <00:25:03.565>it's <00:25:03.798>one <00:25:04.031>front <00:25:04.265>end,

556
00:25:06.745 --> 00:25:08.885
<00:25:06.745>one <00:25:07.094>backend, <00:25:07.445>and <00:25:07.794>one <00:25:08.145>LLM.

557
00:25:09.055 --> 00:25:11.765
<00:25:09.055>These <00:25:09.255>LLM <00:25:09.455>we <00:25:09.655>are <00:25:09.855>using <00:25:10.055>our <00:25:10.255>local <00:25:10.455>lm.

558
00:25:12.425 --> 00:25:15.485
<00:25:12.425>We <00:25:12.625>will <00:25:12.825>use <00:25:13.024>different <00:25:13.225>tools, <00:25:13.425>different <00:25:13.625>MCP <00:25:13.825>tools

559
00:25:16.185 --> 00:25:18.765
<00:25:16.185>and <00:25:16.418>also <00:25:16.651>we <00:25:16.885>will <00:25:17.118>use <00:25:17.351>uh, <00:25:17.585>backend

560
00:25:18.905 --> 00:25:21.745
<00:25:18.905>and <00:25:19.080>the <00:25:19.254>front <00:25:19.430>end, <00:25:19.605>everything <00:25:19.780>that <00:25:19.955>you <00:25:20.129>can <00:25:20.305>see,

561
00:25:22.295 --> 00:25:23.425
<00:25:22.295>it's <00:25:22.803>running <00:25:23.312>locally.

562
00:25:24.205 --> 00:25:26.785
<00:25:24.205>And <00:25:24.360>the <00:25:24.516>first <00:25:24.671>thing <00:25:24.827>that <00:25:24.982>we <00:25:25.138>need <00:25:25.293>is <00:25:25.449>to <00:25:25.605>go

563
00:25:27.005 --> 00:25:31.145
<00:25:27.005>and <00:25:27.180>deploy <00:25:27.355>our <00:25:27.530>application <00:25:27.705>that <00:25:27.880>uh, <00:25:28.055>we <00:25:28.230>need <00:25:28.405>it.

564
00:25:31.685 --> 00:25:34.945
<00:25:31.685>So <00:25:31.965>let <00:25:32.245>me <00:25:32.524>just <00:25:32.804>pull <00:25:33.085>this

565
00:25:35.725 --> 00:25:39.065
<00:25:35.725>and <00:25:35.958>I <00:25:36.191>need <00:25:36.425>to <00:25:36.658>run <00:25:36.891>the <00:25:37.125>backend.

566
00:25:40.205 --> 00:25:43.335
<00:25:40.205>Alright, <00:25:40.438>we <00:25:40.671>have <00:25:40.905>our <00:25:41.138>fast <00:25:41.371>API <00:25:41.605>backend

567
00:25:43.845 --> 00:25:46.295
<00:25:43.845>that <00:25:44.045>is <00:25:44.245>calling <00:25:44.445>also <00:25:44.645>my, <00:25:44.845>uh, <00:25:45.045>my <00:25:45.245>LLM.

568
00:25:46.295 --> 00:25:51.255
<00:25:46.295>Mm-hmm. <00:25:46.435>And <00:25:46.575>I <00:25:46.715>have <00:25:46.855>my <00:25:46.995>front <00:25:47.135>end <00:25:47.275>that <00:25:47.415>will <00:25:47.555>hopefully <00:25:47.695>run

569
00:25:52.555 --> 00:25:53.975
<00:25:52.555>my <00:25:53.194>obligation <00:25:53.832>brilliance.

570
00:25:54.555 --> 00:25:58.695
<00:25:54.555>So <00:25:54.730>with <00:25:54.905>this, <00:25:55.080>let's <00:25:55.255>do, <00:25:55.430>we <00:25:55.605>are <00:25:55.780>in <00:25:55.955>Germany.

571
00:25:59.075 --> 00:26:01.415
<00:25:59.075>We <00:25:59.275>want <00:25:59.475>to <00:25:59.675>know <00:25:59.875>what's <00:26:00.075>the <00:26:00.275>stock <00:26:00.475>price

572
00:26:01.595 --> 00:26:03.415
<00:26:01.595>of <00:26:02.061>Bosch <00:26:02.528>today, <00:26:02.995>right?

573
00:26:03.965 --> 00:26:06.935
<00:26:03.965>Alright, <00:26:04.431>let's <00:26:04.898>use <00:26:05.365>this.

574
00:26:06.955 --> 00:26:10.855
<00:26:06.955>But <00:26:07.095>before <00:26:07.235>of <00:26:07.375>that, <00:26:07.514>let <00:26:07.655>us <00:26:07.794>explain <00:26:07.935>one <00:26:08.074>thing <00:26:08.215>that <00:26:08.355>it's,

575
00:26:10.915 --> 00:26:12.335
<00:26:10.915>we <00:26:11.341>will <00:26:11.767>use <00:26:12.193>tools

576
00:26:12.925 --> 00:26:14.895
<00:26:12.925>because <00:26:13.391>an <00:26:13.858>iGen <00:26:14.325>application

577
00:26:15.565 --> 00:26:19.615
<00:26:15.565>will <00:26:15.705>not <00:26:15.845>only <00:26:15.985>use <00:26:16.125>the <00:26:16.265>own <00:26:16.405>LLM <00:26:16.545>itself, <00:26:16.685>the <00:26:16.825>own <00:26:16.965>knowledge

578
00:26:19.615 --> 00:26:21.375
<00:26:19.615>of <00:26:19.895>the <00:26:20.175>LM <00:26:20.455>because <00:26:20.735>this <00:26:21.015>LLM,

579
00:26:21.645 --> 00:26:23.535
<00:26:21.645>it's <00:26:21.995>just <00:26:22.345>having <00:26:22.695>static <00:26:23.045>knowledge.

580
00:26:23.795 --> 00:26:26.215
<00:26:23.795>We <00:26:24.145>need <00:26:24.495>update, <00:26:24.845>uh, <00:26:25.195>information

581
00:26:26.235 --> 00:26:30.615
<00:26:26.235>and <00:26:26.390>for <00:26:26.546>that <00:26:26.701>we <00:26:26.857>will <00:26:27.012>enable <00:26:27.168>some <00:26:27.323>tools <00:26:27.479>using <00:26:27.635>MCB.

582
00:26:30.615 --> 00:26:31.975
<00:26:30.615>That <00:26:30.859>is <00:26:31.104>a <00:26:31.349>model <00:26:31.594>context <00:26:31.839>protocol.

583
00:26:32.005 --> 00:26:34.175
<00:26:32.005>That <00:26:32.285>is <00:26:32.565>a <00:26:32.845>protocol <00:26:33.125>on <00:26:33.405>standardization.

584
00:26:34.635 --> 00:26:39.615
<00:26:34.635>And <00:26:34.790>if <00:26:34.946>I <00:26:35.101>ask <00:26:35.257>this, <00:26:35.412>hopefully <00:26:35.568>we'll <00:26:35.723>run, <00:26:35.879>we'll <00:26:36.035>ask

585
00:26:39.725 --> 00:26:40.935
<00:26:39.725>what <00:26:39.997>is <00:26:40.269>the <00:26:40.541>stock <00:26:40.813>price

586
00:26:41.395 --> 00:26:46.245
<00:26:41.395>and <00:26:41.535>if <00:26:41.675>we <00:26:41.815>see <00:26:41.955>the <00:26:42.095>backend, <00:26:42.235>alright, <00:26:42.375>the <00:26:42.514>latest <00:26:42.655>close, <00:26:42.795>uh,

587
00:26:46.245 --> 00:26:50.165
<00:26:46.245>closing <00:26:46.524>price <00:26:46.804>of <00:26:47.084>Bosch <00:26:47.364>is <00:26:47.645>$19.

588
00:26:50.725 --> 00:26:54.165
<00:26:50.725>Brilliant. <00:26:51.004>And <00:26:51.284>alright, <00:26:51.564>so <00:26:51.844>let's <00:26:52.125>imagine

589
00:26:54.235 --> 00:26:55.925
<00:26:54.235>that <00:26:54.584>I <00:26:54.935>walk <00:26:55.284>in <00:26:55.635>Starbucks

590
00:26:56.025 --> 00:26:58.565
<00:26:56.025>and <00:26:56.225>I <00:26:56.425>have <00:26:56.625>hundred <00:26:56.825>shares <00:26:57.025>of, <00:26:57.225>uh, <00:26:57.425>Starbucks

591
00:26:58.785 --> 00:27:03.725
<00:26:58.785>and <00:26:58.901>I <00:26:59.018>want, <00:26:59.135>uh, <00:26:59.251>eight <00:26:59.368>coffee <00:26:59.485>because <00:26:59.601>we <00:26:59.718>are <00:26:59.835>living <00:26:59.951>in <00:27:00.068>the <00:27:00.185>US

592
00:27:03.825 --> 00:27:05.125
<00:27:03.825>and <00:27:04.117>right, <00:27:04.410>it's <00:27:04.702>very <00:27:04.995>expensive.

593
00:27:05.265 --> 00:27:09.965
<00:27:05.265>So <00:27:05.381>I <00:27:05.498>want <00:27:05.615>this <00:27:05.731>how <00:27:05.848>many <00:27:05.965>coffees <00:27:06.081>I <00:27:06.198>will <00:27:06.315>get <00:27:06.431>if <00:27:06.548>I <00:27:06.665>sell

594
00:27:10.705 --> 00:27:13.245
<00:27:10.705>my <00:27:10.985>a <00:27:11.264>hundred <00:27:11.544>shares <00:27:11.824>of <00:27:12.105>Starbucks.

595
00:27:13.575 --> 00:27:16.085
<00:27:13.575>Let's <00:27:13.775>see. <00:27:13.975>This <00:27:14.175>is <00:27:14.375>a <00:27:14.575>very <00:27:14.775>complex <00:27:14.975>tasks.

596
00:27:16.345 --> 00:27:20.445
<00:27:16.345>So <00:27:16.485>first <00:27:16.625>of <00:27:16.765>all, <00:27:16.905>we <00:27:17.045>need <00:27:17.185>to <00:27:17.325>understand <00:27:17.465>the <00:27:17.605>stock <00:27:17.745>price

597
00:27:20.665 --> 00:27:23.445
<00:27:20.665>of <00:27:20.945>Starbucks <00:27:21.225>and <00:27:21.504>also <00:27:21.784>will, <00:27:22.065>uh,

598
00:27:23.475 --> 00:27:25.565
<00:27:23.475>analyze <00:27:23.824>this <00:27:24.175>using <00:27:24.524>Python <00:27:24.875>ripple

599
00:27:25.835 --> 00:27:30.165
<00:27:25.835>because <00:27:25.975>it's, <00:27:26.115>um, <00:27:26.255>a <00:27:26.395>way <00:27:26.535>also <00:27:26.675>of <00:27:26.815>doing <00:27:26.955>some <00:27:27.095>complex <00:27:27.235>tasks

600
00:27:30.425 --> 00:27:32.725
<00:27:30.425>and <00:27:30.774>do <00:27:31.125>this <00:27:31.475>calculation <00:27:31.825>itself.

601
00:27:32.785 --> 00:27:35.765
<00:27:32.785>Mm-hmm. <00:27:32.940>So <00:27:33.096>everything <00:27:33.251>that <00:27:33.407>you <00:27:33.562>can <00:27:33.718>see <00:27:33.873>it's <00:27:34.029>running <00:27:34.185>locally

602
00:27:36.665 --> 00:27:40.645
<00:27:36.665>and <00:27:37.131>afterwards <00:27:37.598>will <00:27:38.065>hopefully,

603
00:27:41.665 --> 00:27:44.085
<00:27:41.665>if <00:27:41.898>you <00:27:42.131>see <00:27:42.365>it's <00:27:42.598>grabbing <00:27:42.831>the <00:27:43.065>information.

604
00:27:44.255 --> 00:27:47.885
<00:27:44.255>Let's <00:27:44.410>see <00:27:44.566>afterwards, <00:27:44.721>um, <00:27:44.877>a <00:27:45.032>little <00:27:45.188>bit <00:27:45.343>like, <00:27:45.499>uh, <00:27:45.655>bigger.

605
00:27:48.665 --> 00:27:49.925
<00:27:48.665>So <00:27:48.891>what's <00:27:49.118>going <00:27:49.345>on <00:27:49.572>in <00:27:49.799>here?

606
00:27:50.865 --> 00:27:53.845
<00:27:50.865>The <00:27:51.145>agent <00:27:51.425>that <00:27:51.705>is <00:27:51.985>in <00:27:52.265>here,

607
00:27:54.875 --> 00:27:56.485
<00:27:54.875>it's <00:27:55.225>from <00:27:55.575>the <00:27:55.925>front <00:27:56.275>end.

608
00:27:56.745 --> 00:28:01.685
<00:27:56.745>I'm <00:27:56.900>requesting <00:27:57.056>this, <00:27:57.211>uh, <00:27:57.367>information <00:27:57.522>is <00:27:57.678>going <00:27:57.833>to <00:27:57.989>the <00:27:58.145>backend

609
00:28:02.105 --> 00:28:04.165
<00:28:02.105>and <00:28:02.338>it's <00:28:02.571>going <00:28:02.805>to <00:28:03.038>my <00:28:03.271>local <00:28:03.505>LLM

610
00:28:04.785 --> 00:28:07.085
<00:28:04.785>and <00:28:05.018>the <00:28:05.251>local <00:28:05.485>LLM <00:28:05.718>understand <00:28:05.951>that <00:28:06.185>needs

611
00:28:07.105 --> 00:28:11.365
<00:28:07.105>to <00:28:07.232>rely <00:28:07.359>on <00:28:07.486>all <00:28:07.614>the <00:28:07.741>tools, <00:28:07.868>grab <00:28:07.995>this <00:28:08.123>information <00:28:08.250>and <00:28:08.377>hands <00:28:08.505>it

612
00:28:11.385 --> 00:28:14.885
<00:28:11.385>and <00:28:11.525>explain <00:28:11.665>it <00:28:11.805>to <00:28:11.945>me <00:28:12.085>through <00:28:12.225>the <00:28:12.365>backend <00:28:12.504>to <00:28:12.645>the <00:28:12.785>frontend.

613
00:28:15.015 --> 00:28:18.965
<00:28:15.015>Everything <00:28:15.215>that <00:28:15.415>you <00:28:15.615>can <00:28:15.815>see <00:28:16.015>here, <00:28:16.215>it's <00:28:16.415>local.

614
00:28:19.705 --> 00:28:22.685
<00:28:19.705>So <00:28:20.054>in <00:28:20.405>my <00:28:20.754>front <00:28:21.105>end

615
00:28:23.485 --> 00:28:26.085
<00:28:23.485>I <00:28:23.764>will <00:28:24.044>see <00:28:24.324>the <00:28:24.604>response <00:28:24.885>back.

616
00:28:26.305 --> 00:28:28.765
<00:28:26.305>So <00:28:26.505>the <00:28:26.705>latest, <00:28:26.905>uh, <00:28:27.105>the <00:28:27.305>latest <00:28:27.505>closing <00:28:27.705>price

617
00:28:28.785 --> 00:28:32.565
<00:28:28.785>of <00:28:29.018>Starbucks <00:28:29.251>is <00:28:29.485>almost <00:28:29.718>95 <00:28:29.951>uh, <00:28:30.185>bucks.

618
00:28:32.785 --> 00:28:37.685
<00:28:32.785>And <00:28:32.960>how <00:28:33.135>many, <00:28:33.310>uh, <00:28:33.485>shares <00:28:33.660>I <00:28:33.835>have <00:28:34.010>mm <00:28:34.185>of,

619
00:28:37.905 --> 00:28:41.445
<00:28:37.905>uh, <00:28:38.105>this <00:28:38.305>and <00:28:38.504>theoretically <00:28:38.705>I <00:28:38.905>have <00:28:39.105>more <00:28:39.305>than

620
00:28:42.505 --> 00:28:45.485
<00:28:42.505>1000 <00:28:42.680>coffees <00:28:42.855>that <00:28:43.030>I <00:28:43.205>can <00:28:43.380>take <00:28:43.555>with <00:28:43.730>these <00:28:43.905>shares.

621
00:28:45.485 --> 00:28:46.605
<00:28:45.485>So <00:28:45.686>it's <00:28:45.888>a <00:28:46.089>lot <00:28:46.291>of <00:28:46.493>coffee.

622
00:28:46.605 --> 00:28:48.005
<00:28:46.605>Yeah. <00:28:46.785>So <00:28:46.965>everyone <00:28:47.145>can <00:28:47.325>have <00:28:47.505>a <00:28:47.685>coffee. <00:28:47.865>We

623
00:28:48.005 --> 00:28:50.045
<00:28:48.005>Can, <00:28:48.180>we <00:28:48.355>can <00:28:48.530>share, <00:28:48.705>uh, <00:28:48.880>the <00:28:49.055>coffees <00:28:49.230>in <00:28:49.405>here

624
00:28:50.665 --> 00:28:53.405
<00:28:50.665>and <00:28:50.865>this <00:28:51.065>is <00:28:51.264>a <00:28:51.465>way <00:28:51.665>of <00:28:51.865>explaining, <00:28:52.065>um,

625
00:28:53.625 --> 00:28:56.445
<00:28:53.625>how <00:28:53.825>you <00:28:54.025>can <00:28:54.225>build <00:28:54.425>also <00:28:54.625>your <00:28:54.825>applications <00:28:55.025>locally

626
00:28:56.515 --> 00:28:59.725
<00:28:56.515>because <00:28:56.670>everything <00:28:56.826>that <00:28:56.981>we <00:28:57.137>saw <00:28:57.292>in <00:28:57.448>here <00:28:57.603>right, <00:28:57.759>runs <00:28:57.915>locally

627
00:29:00.185 --> 00:29:01.845
<00:29:00.185>so <00:29:00.465>you <00:29:00.745>can <00:29:01.024>start <00:29:01.304>working <00:29:01.585>out.

628
00:29:02.345 --> 00:29:04.085
<00:29:02.345>And, <00:29:02.625>uh, <00:29:02.905>last <00:29:03.185>but <00:29:03.465>not <00:29:03.745>least,

629
00:29:05.315 --> 00:29:05.925
<00:29:05.315>Yeah. <00:29:05.864>As

630
00:29:06.305 --> 00:29:07.845
<00:29:06.305>How <00:29:06.582>we <00:29:06.859>can <00:29:07.136>integrate <00:29:07.413>ai. <00:29:07.691>Yeah,

631
00:29:08.235 --> 00:29:09.605
<00:29:08.235>Well, <00:29:08.543>as <00:29:08.851>Roberta <00:29:09.159>said, <00:29:09.468>everything

632
00:29:09.605 --> 00:29:11.525
<00:29:09.605>that <00:29:09.885>we <00:29:10.165>did <00:29:10.445>today <00:29:10.725>was <00:29:11.005>local

633
00:29:11.785 --> 00:29:15.565
<00:29:11.785>and <00:29:11.912>whether <00:29:12.039>we <00:29:12.166>use <00:29:12.294>that <00:29:12.421>PO <00:29:12.548>AI <00:29:12.675>lab <00:29:12.803>tool <00:29:12.930>or <00:29:13.057>O <00:29:13.185>lama

634
00:29:15.565 --> 00:29:19.405
<00:29:15.565>or <00:29:15.692>VLLM, <00:29:15.819>you <00:29:15.946>can <00:29:16.074>essentially <00:29:16.201>swap <00:29:16.328>out <00:29:16.455>the <00:29:16.583>same <00:29:16.710>open <00:29:16.837>AI <00:29:16.965>a

635
00:29:19.515 --> 00:29:22.805
<00:29:19.515>open <00:29:19.715>API <00:29:19.915>or <00:29:20.115>open <00:29:20.315>AI, <00:29:20.515>API <00:29:20.715>compatible <00:29:20.915>endpoint.

636
00:29:23.105 --> 00:29:25.645
<00:29:23.105>So <00:29:23.245>you <00:29:23.385>can <00:29:23.525>switch <00:29:23.665>as <00:29:23.805>you're <00:29:23.945>going <00:29:24.085>from <00:29:24.225>local <00:29:24.365>to <00:29:24.505>production.

637
00:29:26.065 --> 00:29:28.725
<00:29:26.065>So <00:29:26.205>today <00:29:26.345>you <00:29:26.485>guys <00:29:26.625>saw <00:29:26.765>how <00:29:26.905>to <00:29:27.045>integrate <00:29:27.185>your <00:29:27.325>data, <00:29:27.465>how

638
00:29:28.725 --> 00:29:30.005
<00:29:28.725>to <00:29:29.109>integrate <00:29:29.493>your <00:29:29.877>IDE

639
00:29:30.145 --> 00:29:31.885
<00:29:30.145>and <00:29:30.378>how <00:29:30.611>to <00:29:30.845>build <00:29:31.078>a <00:29:31.311>agentic <00:29:31.545>applications.

640
00:29:32.065 --> 00:29:34.085
<00:29:32.065>And <00:29:32.220>I <00:29:32.376>hope <00:29:32.531>you <00:29:32.687>took <00:29:32.842>away <00:29:32.998>something <00:29:33.153>from <00:29:33.309>this <00:29:33.465>talk,

641
00:29:34.085 --> 00:29:36.165
<00:29:34.085>but <00:29:34.365>most <00:29:34.645>importantly, <00:29:34.925>that <00:29:35.205>open <00:29:35.485>source

642
00:29:36.345 --> 00:29:37.885
<00:29:36.345>and <00:29:36.691>AI <00:29:37.038>is <00:29:37.384>quickly <00:29:37.731>growing

643
00:29:38.345 --> 00:29:40.845
<00:29:38.345>and, <00:29:38.485>uh, <00:29:38.625>I <00:29:38.765>think <00:29:38.905>there's <00:29:39.045>a <00:29:39.185>bright <00:29:39.325>future <00:29:39.465>for <00:29:39.605>the <00:29:39.745>world

644
00:29:40.845 --> 00:29:44.045
<00:29:40.845>of <00:29:41.000>developers <00:29:41.156>and, <00:29:41.311>uh, <00:29:41.467>we <00:29:41.622>can <00:29:41.778>build <00:29:41.933>really <00:29:42.089>cool <00:29:42.245>applications

645
00:29:44.225 --> 00:29:45.605
<00:29:44.225>by <00:29:44.473>not <00:29:44.721>relying <00:29:44.970>on <00:29:45.218>third <00:29:45.467>parties,

646
00:29:45.705 --> 00:29:47.565
<00:29:45.705>but <00:29:45.985>by <00:29:46.264>being <00:29:46.544>able <00:29:46.824>to <00:29:47.105>self-host.

647
00:29:47.905 --> 00:29:49.125
<00:29:47.905>And <00:29:48.179>we <00:29:48.454>appreciate <00:29:48.728>your <00:29:49.003>time.

648
00:29:49.265 --> 00:29:50.685
<00:29:49.265>So <00:29:49.520>hope <00:29:49.776>you <00:29:50.031>enjoyed <00:29:50.287>the <00:29:50.543>session.

649
00:29:50.935 --> 00:29:52.965
<00:29:50.935>Check <00:29:51.110>out <00:29:51.284>the <00:29:51.460>slides, <00:29:51.635>connect <00:29:51.810>with <00:29:51.985>us <00:29:52.159>on <00:29:52.335>LinkedIn

650
00:29:52.965 --> 00:29:55.165
<00:29:52.965>and <00:29:53.092>we'll <00:29:53.219>also <00:29:53.346>be <00:29:53.474>at <00:29:53.601>the <00:29:53.728>booth <00:29:53.855>if <00:29:53.983>you <00:29:54.110>wanna <00:29:54.237>talk <00:29:54.365>shop

651
00:29:55.185 --> 00:29:56.365
<00:29:55.185>and, <00:29:55.361>uh, <00:29:55.539>geek <00:29:55.716>out <00:29:55.893>a <00:29:56.070>little <00:29:56.247>bit.

652
00:29:56.625 --> 00:29:58.085
<00:29:56.625>But <00:29:56.812>have <00:29:57.000>a <00:29:57.188>great <00:29:57.375>rest <00:29:57.563>of <00:29:57.751>the <00:29:57.939>conference.

653
00:29:58.215 --> 00:29:59.805
<00:29:58.215>Thank <00:29:58.495>you <00:29:58.774>very <00:29:59.054>much. <00:29:59.334>Thank <00:29:59.615>you.