WEBVTT

1
00:00:05.035 --> 00:00:06.775
<00:00:05.035>As <00:00:05.315>first <00:00:05.595>I <00:00:05.875>have <00:00:06.155>to <00:00:06.435>apologize

2
00:00:06.775 --> 00:00:09.415
<00:00:06.775>because <00:00:06.950>there <00:00:07.125>is <00:00:07.300>a <00:00:07.475>tag <00:00:07.650>that <00:00:07.825>says <00:00:08.000>life <00:00:08.175>coding,

3
00:00:09.715 --> 00:00:12.495
<00:00:09.715>but <00:00:09.855>as <00:00:09.995>my <00:00:10.135>talk <00:00:10.275>got <00:00:10.415>shortened <00:00:10.555>from <00:00:10.695>45 <00:00:10.834>to <00:00:10.975>30 <00:00:11.115>minutes,

4
00:00:12.975 --> 00:00:16.135
<00:00:12.975>I <00:00:13.102>said <00:00:13.229>I'm <00:00:13.356>gonna <00:00:13.484>talk <00:00:13.611>a <00:00:13.738>bit <00:00:13.865>more <00:00:13.993>about <00:00:14.120>the <00:00:14.247>experiences <00:00:14.375>to,

5
00:00:16.135 --> 00:00:19.175
<00:00:16.135>to <00:00:16.275>show <00:00:16.415>that <00:00:16.555>some <00:00:16.695>basic <00:00:16.835>lines <00:00:16.975>of <00:00:17.115>code <00:00:17.255>really <00:00:17.395>do <00:00:17.535>work.

6
00:00:19.415 --> 00:00:20.535
<00:00:19.415>I <00:00:19.616>hope <00:00:19.818>you <00:00:20.019>can <00:00:20.221>imagine <00:00:20.423>that.

7
00:00:21.275 --> 00:00:24.935
<00:00:21.275>Um, <00:00:21.555>and <00:00:21.834>um, <00:00:22.115>you're <00:00:22.395>gonna, <00:00:22.674>um,

8
00:00:25.605 --> 00:00:26.655
<00:00:25.605>forgive <00:00:25.920>me <00:00:26.235>that <00:00:26.550>part.

9
00:00:27.155 --> 00:00:29.135
<00:00:27.155>So <00:00:27.355>what <00:00:27.555>are <00:00:27.755>we <00:00:27.955>going <00:00:28.155>to <00:00:28.355>talk <00:00:28.555>about?

10
00:00:29.245 --> 00:00:31.335
<00:00:29.245>Like <00:00:29.525>it's <00:00:29.805>um, <00:00:30.085>a <00:00:30.365>short <00:00:30.645>introduction,

11
00:00:31.405 --> 00:00:33.495
<00:00:31.405>some <00:00:31.871>background <00:00:32.338>about <00:00:32.805>LLMs,

12
00:00:34.135 --> 00:00:35.455
<00:00:34.135>everyone <00:00:34.372>in <00:00:34.610>here <00:00:34.847>who's <00:00:35.085>a <00:00:35.323>specialist,

13
00:00:35.455 --> 00:00:39.095
<00:00:35.455>please <00:00:35.610>just <00:00:35.766>go <00:00:35.921>into <00:00:36.077>Brains <00:00:36.232>News <00:00:36.388>for <00:00:36.543>this <00:00:36.699>few <00:00:36.855>minutes

14
00:00:39.595 --> 00:00:42.215
<00:00:39.595>and <00:00:39.735>everybody <00:00:39.875>else, <00:00:40.015>I <00:00:40.155>hope <00:00:40.295>I <00:00:40.435>can <00:00:40.574>tell <00:00:40.714>you <00:00:40.855>something <00:00:40.995>new.

15
00:00:42.635 --> 00:00:46.415
<00:00:42.635>Um, <00:00:42.775>how <00:00:42.915>we <00:00:43.055>can <00:00:43.195>add <00:00:43.335>them <00:00:43.474>into <00:00:43.614>our <00:00:43.754>software, <00:00:43.894>of <00:00:44.035>course

16
00:00:46.415 --> 00:00:49.495
<00:00:46.415>with <00:00:46.615>some <00:00:46.815>practical <00:00:47.015>examples, <00:00:47.214>just <00:00:47.415>like <00:00:47.615>some <00:00:47.815>code

17
00:00:49.555 --> 00:00:52.415
<00:00:49.555>and <00:00:49.695>then <00:00:49.835>gonna <00:00:49.975>explain <00:00:50.115>like <00:00:50.255>what <00:00:50.394>worked <00:00:50.535>and <00:00:50.675>what <00:00:50.815>didn't <00:00:50.955>work

18
00:00:52.555 --> 00:00:56.255
<00:00:52.555>or <00:00:52.695>maybe <00:00:52.835>work <00:00:52.975>of <00:00:53.115>course <00:00:53.255>a <00:00:53.394>short <00:00:53.535>side <00:00:53.675>step <00:00:53.815>to <00:00:53.955>CPS

19
00:00:56.515 --> 00:00:59.895
<00:00:56.515>and <00:00:56.748>some <00:00:56.981>legal <00:00:57.215>implications <00:00:57.448>about <00:00:57.681>all <00:00:57.915>this.

20
00:01:00.475 --> 00:01:01.655
<00:01:00.475>So, <00:01:01.537>um,

21
00:01:02.965 --> 00:01:07.055
<00:01:02.965>when <00:01:03.120>LLM <00:01:03.276>started <00:01:03.431>like <00:01:03.587>3.5 <00:01:03.742>to <00:01:03.898>four <00:01:04.053>years <00:01:04.209>ago, <00:01:04.365>like

22
00:01:07.125 --> 00:01:10.775
<00:01:07.125>with <00:01:07.358>GPT-3 <00:01:07.591>0.5 <00:01:07.825>to <00:01:08.058>really <00:01:08.291>get <00:01:08.525>usable.

23
00:01:11.085 --> 00:01:14.655
<00:01:11.085>Yeah, <00:01:11.285>um, <00:01:11.485>there <00:01:11.685>was <00:01:11.885>this <00:01:12.085>absolutely <00:01:12.285>big <00:01:12.485>hype

24
00:01:14.655 --> 00:01:17.695
<00:01:14.655>that <00:01:14.830>I <00:01:15.005>can <00:01:15.180>just <00:01:15.355>use <00:01:15.530>this <00:01:15.705>tweet <00:01:15.880>to <00:01:16.055>summarize.

25
00:01:19.005 --> 00:01:20.855
<00:01:19.005>Like <00:01:19.354>there <00:01:19.705>is <00:01:20.054>these <00:01:20.405>dogs

26
00:01:21.075 --> 00:01:24.855
<00:01:21.075>and <00:01:21.191>they <00:01:21.308>knew <00:01:21.425>how <00:01:21.541>to <00:01:21.658>bark, <00:01:21.775>so <00:01:21.891>it <00:01:22.008>really <00:01:22.125>sounds <00:01:22.241>like <00:01:22.358>a <00:01:22.475>human,

27
00:01:25.235 --> 00:01:27.615
<00:01:25.235>but <00:01:25.515>maybe <00:01:25.795>they <00:01:26.075>aren't <00:01:26.355>really <00:01:26.635>human.

28
00:01:28.435 --> 00:01:32.135
<00:01:28.435>And <00:01:28.635>the <00:01:28.835>thing <00:01:29.035>is <00:01:29.235>like <00:01:29.435>LLMs <00:01:29.635>don't <00:01:29.835>like

29
00:01:33.315 --> 00:01:34.685
<00:01:33.315>work <00:01:33.726>in <00:01:34.137>that <00:01:34.548>way.

30
00:01:34.755 --> 00:01:36.485
<00:01:34.755>They <00:01:35.104>are <00:01:35.455>more <00:01:35.804>like <00:01:36.155>clouds.

31
00:01:36.875 --> 00:01:39.605
<00:01:36.875>They <00:01:37.108>look <00:01:37.341>like <00:01:37.575>a <00:01:37.808>dog, <00:01:38.041>but <00:01:38.275>not

32
00:01:39.605 --> 00:01:41.685
<00:01:39.604>because <00:01:39.804>the <00:01:40.005>clouds <00:01:40.204>look <00:01:40.404>like <00:01:40.604>a <00:01:40.804>dog, <00:01:41.005>but

33
00:01:41.685 --> 00:01:45.365
<00:01:41.685>because <00:01:41.812>of <00:01:41.939>our <00:01:42.066>ability <00:01:42.194>to <00:01:42.321>see <00:01:42.448>the <00:01:42.575>clouds <00:01:42.703>looking <00:01:42.830>like <00:01:42.957>a <00:01:43.085>dog.

34
00:01:45.365 --> 00:01:46.605
<00:01:45.365>And <00:01:45.588>it's <00:01:45.811>the <00:01:46.034>same <00:01:46.257>with <00:01:46.481>text

35
00:01:46.985 --> 00:01:49.765
<00:01:46.985>and <00:01:47.140>it's <00:01:47.296>really <00:01:47.451>easy <00:01:47.607>to <00:01:47.762>get <00:01:47.918>mistaking <00:01:48.073>on <00:01:48.229>this <00:01:48.385>part

36
00:01:50.065 --> 00:01:52.765
<00:01:50.065>and <00:01:50.298>to <00:01:50.531>see <00:01:50.765>like <00:01:50.998>in <00:01:51.231>this <00:01:51.465>text

37
00:01:53.995 --> 00:01:57.085
<00:01:53.995>that <00:01:54.170>is <00:01:54.345>standing <00:01:54.520>there <00:01:54.695>to <00:01:54.870>imagine <00:01:55.045>that <00:01:55.220>someone <00:01:55.395>had

38
00:01:57.085 --> 00:01:58.605
<00:01:57.085>to <00:01:57.427>think <00:01:57.769>to <00:01:58.110>write <00:01:58.452>this.

39
00:01:58.865 --> 00:02:02.085
<00:01:58.865>But <00:01:59.065>it <00:01:59.265>sounds <00:01:59.465>also <00:01:59.665>just <00:01:59.865>something <00:02:00.065>that <00:02:00.265>we

40
00:02:02.625 --> 00:02:04.245
<00:02:02.625>impose <00:02:03.091>on <00:02:03.558>that <00:02:04.025>text.

41
00:02:04.425 --> 00:02:06.005
<00:02:04.425>The <00:02:04.658>text <00:02:04.891>has <00:02:05.125>never <00:02:05.358>been <00:02:05.591>thought <00:02:05.825>through.

42
00:02:06.345 --> 00:02:07.965
<00:02:06.345>That's <00:02:06.578>one <00:02:06.811>point <00:02:07.045>I <00:02:07.278>really <00:02:07.511>wanna <00:02:07.745>make

43
00:02:07.965 --> 00:02:09.805
<00:02:07.965>that <00:02:08.315>LLMs <00:02:08.664>even <00:02:09.015>in <00:02:09.365>reasoning.

44
00:02:10.175 --> 00:02:14.885
<00:02:10.175>Never <00:02:10.350>do <00:02:10.525>think <00:02:10.700>that's <00:02:10.875>not <00:02:11.050>something <00:02:11.225>what <00:02:11.400>they <00:02:11.575>do.

45
00:02:15.705 --> 00:02:20.645
<00:02:15.705>So <00:02:15.832>this <00:02:15.959>is, <00:02:16.086>um, <00:02:16.214>what <00:02:16.341>is <00:02:16.468>the <00:02:16.595>current <00:02:16.723>state, <00:02:16.850>in <00:02:16.977>my <00:02:17.105>opinion,

46
00:02:20.985 --> 00:02:22.605
<00:02:20.985>uh, <00:02:21.335>in <00:02:21.685>much <00:02:22.035>of <00:02:22.385>ai.

47
00:02:22.625 --> 00:02:26.885
<00:02:22.625>We <00:02:22.765>had <00:02:22.905>a <00:02:23.045>very <00:02:23.185>nice <00:02:23.325>introductionary <00:02:23.465>talk <00:02:23.605>today <00:02:23.745>where <00:02:23.885>we <00:02:24.025>saw

48
00:02:26.885 --> 00:02:29.805
<00:02:26.885>that <00:02:27.165>never <00:02:27.445>rely <00:02:27.725>on <00:02:28.005>an <00:02:28.285>LLM

49
00:02:29.805 --> 00:02:31.005
<00:02:29.805>because <00:02:30.165>they're <00:02:30.525>not <00:02:30.885>able

50
00:02:31.005 --> 00:02:33.325
<00:02:31.005>to <00:02:31.160>build <00:02:31.316>a <00:02:31.471>snake <00:02:31.627>game <00:02:31.782>if <00:02:31.938>you <00:02:32.093>want <00:02:32.249>them <00:02:32.405>to.

51
00:02:33.635 --> 00:02:38.005
<00:02:33.635>Yeah, <00:02:33.790>and <00:02:33.946>that's <00:02:34.101>also <00:02:34.257>something <00:02:34.412>I <00:02:34.568>really <00:02:34.723>wanna, <00:02:34.879>um, <00:02:35.035>give.

52
00:02:38.265 --> 00:02:40.805
<00:02:38.265>Um, <00:02:38.440>a <00:02:38.614>a <00:02:38.790>point <00:02:38.964>I <00:02:39.140>wanna <00:02:39.315>make <00:02:39.489>is <00:02:39.665>that

53
00:02:41.635 --> 00:02:45.565
<00:02:41.635>LLMs <00:02:41.774>never <00:02:41.915>behaved <00:02:42.054>the <00:02:42.195>way <00:02:42.334>they <00:02:42.475>behaved <00:02:42.614>like <00:02:42.755>the <00:02:42.894>day <00:02:43.035>before

54
00:02:46.025 --> 00:02:48.165
<00:02:46.025>or <00:02:46.305>maybe <00:02:46.585>just <00:02:46.865>three <00:02:47.145>seconds <00:02:47.425>before.

55
00:02:48.785 --> 00:02:52.245
<00:02:48.785>So <00:02:48.984>the <00:02:49.185>thing <00:02:49.385>is <00:02:49.585>with <00:02:49.785>um, <00:02:49.984>prompting <00:02:50.185>LLMs,

56
00:02:52.245 --> 00:02:57.045
<00:02:52.245>it's <00:02:52.420>al <00:02:52.595>always <00:02:52.770>about <00:02:52.945>re <00:02:53.120>drying <00:02:53.295>at <00:02:53.470>least <00:02:53.645>once

57
00:02:57.045 --> 00:03:01.565
<00:02:57.045>because <00:02:57.325>the <00:02:57.605>endpoint <00:02:57.885>maybe <00:02:58.165>won't <00:02:58.445>answer

58
00:03:01.825 --> 00:03:03.645
<00:03:01.825>or <00:03:02.024>it <00:03:02.225>will <00:03:02.424>answer <00:03:02.625>in <00:03:02.825>a <00:03:03.024>different <00:03:03.225>way

59
00:03:03.785 --> 00:03:07.045
<00:03:03.785>or <00:03:03.940>it <00:03:04.096>won't <00:03:04.251>answer <00:03:04.407>in <00:03:04.562>Jason, <00:03:04.718>but <00:03:04.873>in <00:03:05.029>Japanese, <00:03:05.185>um,

60
00:03:07.265 --> 00:03:08.925
<00:03:07.265>or <00:03:07.545>in <00:03:07.825>some <00:03:08.105>other <00:03:08.385>different <00:03:08.665>way.

61
00:03:09.425 --> 00:03:14.165
<00:03:09.425>So, <00:03:09.580>um, <00:03:09.736>one <00:03:09.891>key <00:03:10.047>takeaway <00:03:10.202>is <00:03:10.358>that, <00:03:10.513>um, <00:03:10.669>LLMs <00:03:10.825>are

62
00:03:14.845 --> 00:03:17.725
<00:03:14.845>absolutely <00:03:15.125>not <00:03:15.405>reliable <00:03:15.685>in <00:03:15.965>any <00:03:16.245>way.

63
00:03:18.185 --> 00:03:22.365
<00:03:18.185>So, <00:03:18.325>um, <00:03:18.465>what <00:03:18.605>many <00:03:18.745>are <00:03:18.885>used <00:03:19.025>to <00:03:19.165>are <00:03:19.305>like <00:03:19.445>this, <00:03:19.585>um,

64
00:03:23.155 --> 00:03:25.005
<00:03:23.155>like <00:03:23.435>seeing <00:03:23.715>the <00:03:23.995>chat <00:03:24.275>interface, <00:03:24.555>yes,

65
00:03:25.465 --> 00:03:27.605
<00:03:25.465>but <00:03:25.640>this <00:03:25.815>is <00:03:25.990>what <00:03:26.165>we <00:03:26.340>as <00:03:26.515>a <00:03:26.690>developers <00:03:26.865>see.

66
00:03:27.605 --> 00:03:29.285
<00:03:27.605>Yeah, <00:03:27.838>we <00:03:28.071>sent <00:03:28.304>some <00:03:28.538>chase <00:03:28.771>in <00:03:29.005>there

67
00:03:29.585 --> 00:03:33.325
<00:03:29.585>and <00:03:29.740>we <00:03:29.896>say <00:03:30.051>like, <00:03:30.207>okay, <00:03:30.362>um, <00:03:30.518>maybe <00:03:30.673>please <00:03:30.829>do <00:03:30.985>that.

68
00:03:33.775 --> 00:03:35.765
<00:03:33.775>Maybe <00:03:34.055>write <00:03:34.335>me <00:03:34.615>some <00:03:34.895>user <00:03:35.175>stories

69
00:03:35.885 --> 00:03:37.725
<00:03:35.885>because <00:03:36.084>that's <00:03:36.285>something <00:03:36.484>we <00:03:36.685>do <00:03:36.885>on <00:03:37.084>a <00:03:37.285>project

70
00:03:38.545 --> 00:03:42.725
<00:03:38.545>and <00:03:38.720>we <00:03:38.895>sent <00:03:39.070>them <00:03:39.245>everything <00:03:39.420>the <00:03:39.595>customer <00:03:39.770>like <00:03:39.945>usual

71
00:03:43.155 --> 00:03:44.165
<00:03:43.155>sees <00:03:43.458>in <00:03:43.761>the <00:03:44.064>chat.

72
00:03:44.705 --> 00:03:48.485
<00:03:44.704>And <00:03:44.860>what <00:03:45.016>many <00:03:45.171>people <00:03:45.327>do <00:03:45.482>get <00:03:45.638>wrong <00:03:45.793>about <00:03:45.949>ai, <00:03:46.105>especially

73
00:03:48.485 --> 00:03:49.605
<00:03:48.485>of <00:03:48.821>course <00:03:49.157>not <00:03:49.493>developers

74
00:03:49.605 --> 00:03:53.965
<00:03:49.605>but <00:03:49.732>end <00:03:49.859>users, <00:03:49.986>is <00:03:50.114>to <00:03:50.241>think <00:03:50.368>that <00:03:50.495>an <00:03:50.623>ai, <00:03:50.750>um, <00:03:50.877>is <00:03:51.005>able

75
00:03:54.025 --> 00:03:56.605
<00:03:54.025>to <00:03:54.225>remember <00:03:54.425>stuff, <00:03:54.625>but <00:03:54.825>of <00:03:55.025>course <00:03:55.225>it's <00:03:55.425>not.

76
00:03:56.705 --> 00:04:00.365
<00:03:56.704>So <00:03:56.832>in <00:03:56.959>each <00:03:57.086>and <00:03:57.214>everything <00:03:57.341>that <00:03:57.468>we <00:03:57.595>want <00:03:57.723>the <00:03:57.850>AI <00:03:57.977>to <00:03:58.105>know,

77
00:04:00.625 --> 00:04:03.165
<00:04:00.625>we <00:04:00.825>have <00:04:01.025>to <00:04:01.225>send <00:04:01.425>it <00:04:01.625>to <00:04:01.825>the <00:04:02.025>LLM.

78
00:04:03.505 --> 00:04:07.525
<00:04:03.505>Of <00:04:03.644>course <00:04:03.785>it <00:04:03.924>has <00:04:04.065>its <00:04:04.204>world <00:04:04.345>knowledge <00:04:04.484>to <00:04:04.625>sound <00:04:04.765>like <00:04:04.905>that,

79
00:04:07.705 --> 00:04:09.885
<00:04:07.705>but <00:04:07.905>it <00:04:08.105>doesn't <00:04:08.305>know <00:04:08.505>anything <00:04:08.705>about <00:04:08.905>your <00:04:09.105>context

80
00:04:09.905 --> 00:04:11.725
<00:04:09.905>or <00:04:10.255>anything <00:04:10.605>you're <00:04:10.955>interacted <00:04:11.305>with.

81
00:04:12.145 --> 00:04:13.845
<00:04:12.145>So <00:04:12.378>each <00:04:12.611>and <00:04:12.845>every <00:04:13.078>time <00:04:13.311>you <00:04:13.545>wanted

82
00:04:13.985 --> 00:04:15.805
<00:04:13.985>to <00:04:14.265>know <00:04:14.545>what <00:04:14.825>it <00:04:15.105>did <00:04:15.385>before.

83
00:04:16.145 --> 00:04:19.885
<00:04:16.144>So <00:04:16.272>for <00:04:16.399>the <00:04:16.526>customer <00:04:16.654>for <00:04:16.781>example, <00:04:16.908>be <00:04:17.035>able <00:04:17.163>to <00:04:17.290>say, <00:04:17.417>um, <00:04:17.544>no,

84
00:04:19.885 --> 00:04:23.805
<00:04:19.885>you <00:04:20.012>did <00:04:20.139>that <00:04:20.266>wrong, <00:04:20.394>do <00:04:20.521>it <00:04:20.648>again <00:04:20.775>or <00:04:20.903>try <00:04:21.030>again <00:04:21.157>or <00:04:21.284>something.

85
00:04:23.985 --> 00:04:26.525
<00:04:23.985>You <00:04:24.185>have <00:04:24.385>to <00:04:24.585>resend <00:04:24.785>it <00:04:24.985>the <00:04:25.185>whole <00:04:25.385>history.

86
00:04:26.985 --> 00:04:30.445
<00:04:26.985>So <00:04:27.140>that's <00:04:27.296>of <00:04:27.451>course <00:04:27.607>gets <00:04:27.762>maybe <00:04:27.918>once <00:04:28.073>a <00:04:28.229>token <00:04:28.385>problem.

87
00:04:30.945 --> 00:04:35.885
<00:04:30.945>So <00:04:31.072>if <00:04:31.199>you <00:04:31.326>wanna <00:04:31.454>add <00:04:31.581>like <00:04:31.708>a <00:04:31.835>comment <00:04:31.963>structure <00:04:32.090>to <00:04:32.217>your <00:04:32.344>app,

88
00:04:36.185 --> 00:04:37.925
<00:04:36.185>you <00:04:36.418>really <00:04:36.651>also <00:04:36.885>have <00:04:37.118>to <00:04:37.351>think <00:04:37.585>about

89
00:04:38.455 --> 00:04:42.405
<00:04:38.455>where <00:04:38.582>is <00:04:38.709>your <00:04:38.836>cutoff <00:04:38.964>point, <00:04:39.091>how <00:04:39.218>much <00:04:39.345>text <00:04:39.473>will <00:04:39.600>you <00:04:39.727>send <00:04:39.854>it?

90
00:04:42.405 --> 00:04:45.845
<00:04:42.405>Because <00:04:42.560>if <00:04:42.716>you <00:04:42.871>send <00:04:43.027>it <00:04:43.182>every <00:04:43.338>text <00:04:43.493>again <00:04:43.649>and <00:04:43.804>again

91
00:04:45.865 --> 00:04:49.045
<00:04:45.865>and <00:04:46.020>again <00:04:46.176>from <00:04:46.331>the <00:04:46.487>beginning <00:04:46.642>when <00:04:46.798>they <00:04:46.953>start <00:04:47.109>to <00:04:47.265>interact

92
00:04:49.045 --> 00:04:51.885
<00:04:49.045>with <00:04:49.220>your <00:04:49.395>app, <00:04:49.570>there <00:04:49.745>will <00:04:49.920>be <00:04:50.095>coming <00:04:50.270>a <00:04:50.445>point

93
00:04:52.055 --> 00:04:54.085
<00:04:52.055>where <00:04:52.405>the <00:04:52.755>performance <00:04:53.105>really <00:04:53.455>degrades

94
00:04:54.085 --> 00:04:56.965
<00:04:54.085>because <00:04:54.260>the <00:04:54.435>context <00:04:54.609>windows <00:04:54.784>are <00:04:54.960>like <00:04:55.135>up <00:04:55.310>to <00:04:55.484>one

95
00:04:56.965 --> 00:04:57.965
<00:04:56.965>or <00:04:57.265>2 <00:04:57.565>million <00:04:57.865>now,

96
00:04:58.305 --> 00:05:02.445
<00:04:58.305>but <00:04:58.480>it's <00:04:58.655>not <00:04:58.830>able <00:04:59.005>to <00:04:59.180>really <00:04:59.355>concentrate <00:04:59.530>the <00:04:59.705>needle

97
00:05:02.505 --> 00:05:05.885
<00:05:02.505>and <00:05:02.660>taste <00:05:02.816>egg <00:05:02.971>things, <00:05:03.127>the <00:05:03.282>performance <00:05:03.438>tests, <00:05:03.593>you <00:05:03.749>know, <00:05:03.905>um,

98
00:05:06.475 --> 00:05:10.245
<00:05:06.475>they <00:05:06.650>don't <00:05:06.825>really <00:05:07.000>represent <00:05:07.175>the <00:05:07.350>real <00:05:07.525>world <00:05:07.700>use <00:05:07.875>case

99
00:05:10.465 --> 00:05:14.005
<00:05:10.465>of <00:05:10.620>an <00:05:10.776>LLM <00:05:10.931>focusing <00:05:11.087>on <00:05:11.242>things <00:05:11.398>that <00:05:11.553>it <00:05:11.709>should <00:05:11.864>know.

100
00:05:14.705 --> 00:05:17.445
<00:05:14.705>So <00:05:14.844>if <00:05:14.984>you <00:05:15.125>send <00:05:15.265>it <00:05:15.405>a <00:05:15.544>thousand <00:05:15.685>lines <00:05:15.825>of <00:05:15.965>of <00:05:16.104>stuff,

101
00:05:17.895 --> 00:05:20.645
<00:05:17.895>it'll <00:05:18.094>definitely <00:05:18.294>forget <00:05:18.495>at <00:05:18.695>least <00:05:18.895>20% <00:05:19.094>of <00:05:19.294>them

102
00:05:20.645 --> 00:05:21.725
<00:05:20.645>that <00:05:20.888>it <00:05:21.131>ever <00:05:21.374>seen <00:05:21.617>them.

103
00:05:22.145 --> 00:05:24.445
<00:05:22.145>If <00:05:22.320>you <00:05:22.495>reference <00:05:22.669>it <00:05:22.844>again <00:05:23.020>in <00:05:23.195>the <00:05:23.370>next <00:05:23.544>chat,

104
00:05:24.535 --> 00:05:26.925
<00:05:24.535>it'll <00:05:24.735>remember <00:05:24.935>again <00:05:25.135>and <00:05:25.335>say, <00:05:25.535>oh, <00:05:25.735>of <00:05:25.935>course.

105
00:05:27.225 --> 00:05:28.285
<00:05:27.225>Um, <00:05:27.543>I <00:05:27.861>forgot <00:05:28.179>that.

106
00:05:28.385 --> 00:05:29.605
<00:05:28.385>But <00:05:28.604>of <00:05:28.824>course <00:05:29.043>you <00:05:29.263>told <00:05:29.483>me

107
00:05:29.605 --> 00:05:32.685
<00:05:29.605>before <00:05:29.955>like <00:05:30.305>they're <00:05:30.655>very <00:05:31.005>apologetic,

108
00:05:33.185 --> 00:05:36.565
<00:05:33.185>but <00:05:33.360>um, <00:05:33.535>they <00:05:33.710>won't <00:05:33.885>be <00:05:34.060>better <00:05:34.235>because <00:05:34.410>of <00:05:34.585>that.

109
00:05:37.465 --> 00:05:40.165
<00:05:37.465>So, <00:05:37.698>um, <00:05:37.931>talk <00:05:38.165>about <00:05:38.398>about <00:05:38.631>the, <00:05:38.865>um,

110
00:05:40.655 --> 00:05:42.005
<00:05:40.655>about <00:05:40.897>some <00:05:41.140>patterns <00:05:41.383>we <00:05:41.627>have <00:05:41.870>here.

111
00:05:42.305 --> 00:05:45.005
<00:05:42.305>So <00:05:42.445>we <00:05:42.585>can <00:05:42.725>give <00:05:42.865>it <00:05:43.005>a <00:05:43.145>temperature <00:05:43.285>if <00:05:43.425>we <00:05:43.565>get <00:05:43.705>it

112
00:05:45.005 --> 00:05:46.405
<00:05:45.005>A <00:05:45.425>temperature <00:05:45.844>of <00:05:46.265>zero.

113
00:05:46.945 --> 00:05:49.605
<00:05:46.945>Uh, <00:05:47.295>LLMs <00:05:47.645>are <00:05:47.995>absolutely <00:05:48.344>deterministic.

114
00:05:49.825 --> 00:05:52.805
<00:05:49.825>So <00:05:49.941>you <00:05:50.058>give <00:05:50.175>the <00:05:50.291>same <00:05:50.408>thing <00:05:50.525>in, <00:05:50.641>you <00:05:50.758>get <00:05:50.875>the <00:05:50.991>same <00:05:51.108>thing <00:05:51.224>out.

115
00:05:52.865 --> 00:05:56.045
<00:05:52.865>So <00:05:52.992>you <00:05:53.119>can <00:05:53.246>even <00:05:53.374>unit <00:05:53.501>test <00:05:53.628>LLMs <00:05:53.755>if <00:05:53.883>you <00:05:54.010>really <00:05:54.137>want <00:05:54.265>to.

116
00:05:56.425 --> 00:05:58.485
<00:05:56.425>Um, <00:05:56.580>as <00:05:56.736>long <00:05:56.891>as <00:05:57.047>the <00:05:57.202>model <00:05:57.358>doesn't <00:05:57.513>change <00:05:57.669>of <00:05:57.825>course.

117
00:05:59.065 --> 00:06:02.045
<00:05:59.065>And <00:05:59.344>that <00:05:59.625>also <00:05:59.905>has <00:06:00.185>an <00:06:00.465>eristics

118
00:06:02.045 --> 00:06:05.685
<00:06:02.045>because <00:06:02.185>if <00:06:02.325>you <00:06:02.465>maybe <00:06:02.605>give <00:06:02.745>the <00:06:02.885>LLM <00:06:03.025>an <00:06:03.165>information <00:06:03.305>like, <00:06:03.445>oh,

119
00:06:05.695 --> 00:06:08.965
<00:06:05.695>today <00:06:05.835>is <00:06:05.974>the <00:06:06.115>10th <00:06:06.255>of <00:06:06.395>July, <00:06:06.534>then <00:06:06.675>the <00:06:06.815>response <00:06:06.955>will <00:06:07.094>change

120
00:06:08.965 --> 00:06:13.125
<00:06:08.965>because <00:06:09.120>the <00:06:09.276>slightest <00:06:09.431>change <00:06:09.587>in <00:06:09.742>the <00:06:09.898>input <00:06:10.053>will <00:06:10.209>generate <00:06:10.364>a

121
00:06:13.125 --> 00:06:14.125
<00:06:13.125>completely <00:06:13.575>different <00:06:14.025>output.

122
00:06:14.125 --> 00:06:15.685
<00:06:14.125>It's <00:06:14.405>more <00:06:14.685>like <00:06:14.965>a <00:06:15.245>hashing <00:06:15.525>algorithm.

123
00:06:15.685 --> 00:06:18.085
<00:06:15.685>Yeah, <00:06:15.885>you <00:06:16.085>change <00:06:16.285>one <00:06:16.485>bit <00:06:16.685>and <00:06:16.885>everything <00:06:17.085>changes

124
00:06:18.435 --> 00:06:22.365
<00:06:18.435>because <00:06:18.610>it <00:06:18.785>changes <00:06:18.960>the <00:06:19.135>entry <00:06:19.310>point <00:06:19.485>to <00:06:19.660>the, <00:06:19.835>um,

125
00:06:22.475 --> 00:06:24.005
<00:06:22.475>knowledge <00:06:22.819>base <00:06:23.163>of <00:06:23.507>the <00:06:23.852>LLM.

126
00:06:24.005 --> 00:06:28.125
<00:06:24.005>We'll <00:06:24.121>come <00:06:24.238>to <00:06:24.355>that. <00:06:24.471>Um, <00:06:24.588>the <00:06:24.705>second <00:06:24.821>point <00:06:24.938>is <00:06:25.055>that <00:06:25.171>the <00:06:25.288>top <00:06:25.405>p

127
00:06:28.125 --> 00:06:30.885
<00:06:28.125>that's <00:06:28.265>just <00:06:28.405>a <00:06:28.545>statistics <00:06:28.685>thing <00:06:28.825>where <00:06:28.965>you <00:06:29.105>can <00:06:29.245>tell <00:06:29.385>it <00:06:29.525>just

128
00:06:30.905 --> 00:06:33.245
<00:06:30.905>to <00:06:31.138>look <00:06:31.371>at <00:06:31.604>the <00:06:31.838>most <00:06:32.071>probable <00:06:32.304>answers

129
00:06:33.435 --> 00:06:34.965
<00:06:33.435>that <00:06:33.779>it <00:06:34.123>will <00:06:34.467>probably <00:06:34.812>generate.

130
00:06:35.185 --> 00:06:37.645
<00:06:35.185>So <00:06:35.385>like <00:06:35.585>the <00:06:35.785>next <00:06:35.985>token, <00:06:36.185>the <00:06:36.385>next <00:06:36.585>bit

131
00:06:37.665 --> 00:06:41.045
<00:06:37.665>of <00:06:37.820>word <00:06:37.976>will <00:06:38.131>only <00:06:38.287>be <00:06:38.442>taken <00:06:38.598>from <00:06:38.753>the <00:06:38.909>most, <00:06:39.065>um,

132
00:06:42.285 --> 00:06:43.725
<00:06:42.284>probable <00:06:43.581>possibilities

133
00:06:44.705 --> 00:06:48.125
<00:06:44.705>and <00:06:44.844>it's <00:06:44.984>very <00:06:45.125>common <00:06:45.265>to <00:06:45.405>just <00:06:45.544>leave <00:06:45.685>that <00:06:45.825>at <00:06:45.965>the <00:06:46.104>95%

134
00:06:48.505 --> 00:06:50.485
<00:06:48.505>and <00:06:48.738>then <00:06:48.971>you <00:06:49.205>give <00:06:49.438>it <00:06:49.671>some <00:06:49.905>penalties

135
00:06:50.505 --> 00:06:54.525
<00:06:50.505>for <00:06:50.632>words <00:06:50.759>already <00:06:50.886>being <00:06:51.014>there <00:06:51.141>and <00:06:51.268>give <00:06:51.395>it <00:06:51.523>a <00:06:51.650>max <00:06:51.777>of <00:06:51.905>tokens.

136
00:06:55.265 --> 00:06:58.445
<00:06:55.265>You <00:06:55.405>can <00:06:55.544>um, <00:06:55.685>even <00:06:55.825>give <00:06:55.965>it <00:06:56.104>a <00:06:56.245>stop <00:06:56.385>token <00:06:56.525>like, <00:06:56.664>um,

137
00:06:59.135 --> 00:07:00.725
<00:06:59.135>dosed <00:06:59.414>records <00:06:59.695>or <00:06:59.974>something <00:07:00.255>like <00:07:00.534>that.

138
00:07:01.185 --> 00:07:04.965
<00:07:01.185>So <00:07:01.312>it'll <00:07:01.439>stop <00:07:01.566>if <00:07:01.694>you <00:07:01.821>tell <00:07:01.948>it <00:07:02.075>to <00:07:02.203>provide <00:07:02.330>you <00:07:02.457>with <00:07:02.585>Chase.

139
00:07:05.705 --> 00:07:09.605
<00:07:05.705>So <00:07:05.844>if <00:07:05.984>we <00:07:06.125>send <00:07:06.265>that, <00:07:06.405>what <00:07:06.544>will <00:07:06.685>come <00:07:06.825>back, <00:07:06.965>something <00:07:07.104>like

140
00:07:09.605 --> 00:07:11.325
<00:07:09.605>that, <00:07:09.885>it <00:07:10.165>will <00:07:10.445>be <00:07:10.725>very <00:07:11.005>textual

141
00:07:11.785 --> 00:07:15.885
<00:07:11.785>and <00:07:11.940>it <00:07:12.096>will <00:07:12.251>um, <00:07:12.407>it <00:07:12.562>will <00:07:12.718>be <00:07:12.873>just <00:07:13.029>something <00:07:13.185>that's

142
00:07:16.405 --> 00:07:20.685
<00:07:16.405>a <00:07:16.521>bit <00:07:16.638>hard <00:07:16.755>to <00:07:16.871>use, <00:07:16.988>um, <00:07:17.104>that <00:07:17.221>will <00:07:17.338>come, <00:07:17.455>um, <00:07:17.571>how <00:07:17.688>to <00:07:17.804>structure

143
00:07:20.685 --> 00:07:21.845
<00:07:20.685>that <00:07:21.207>LA <00:07:21.729>later.

144
00:07:22.705 --> 00:07:26.205
<00:07:22.705>So <00:07:22.860>what <00:07:23.016>are <00:07:23.171>LLMs <00:07:23.327>and <00:07:23.482>how <00:07:23.638>can <00:07:23.793>we <00:07:23.949>use <00:07:24.104>them?

145
00:07:26.625 --> 00:07:29.685
<00:07:26.625>So <00:07:26.858>that's <00:07:27.091>my <00:07:27.325>most <00:07:27.558>favorite <00:07:27.791>example <00:07:28.025>of

146
00:07:29.985 --> 00:07:32.525
<00:07:29.985>how <00:07:30.218>an <00:07:30.451>LLM <00:07:30.685>um, <00:07:30.918>tends <00:07:31.151>to <00:07:31.385>work.

147
00:07:32.785 --> 00:07:36.445
<00:07:32.784>Um, <00:07:32.901>you <00:07:33.018>can <00:07:33.135>go <00:07:33.251>visit <00:07:33.368>this <00:07:33.484>URL, <00:07:33.601>it's <00:07:33.718>um, <00:07:33.835>a <00:07:33.951>really <00:07:34.068>cool <00:07:34.184>page.

148
00:07:37.065 --> 00:07:39.645
<00:07:37.065>Um, <00:07:37.298>whereas <00:07:37.531>scientists <00:07:37.765>has <00:07:37.998>just <00:07:38.231>run, <00:07:38.465>um,

149
00:07:40.275 --> 00:07:44.245
<00:07:40.275>like <00:07:40.450>the <00:07:40.625>same <00:07:40.799>starting <00:07:40.974>sentence <00:07:41.150>like, <00:07:41.325>I <00:07:41.500>don't <00:07:41.674>know,

150
00:07:44.245 --> 00:07:47.245
<00:07:44.245>like <00:07:44.400>a <00:07:44.556>hundred <00:07:44.711>thousand <00:07:44.867>times <00:07:45.022>through <00:07:45.178>the <00:07:45.333>LL <00:07:45.489>with <00:07:45.645>um,

151
00:07:48.485 --> 00:07:49.925
<00:07:48.485>a <00:07:48.809>temperature <00:07:49.133>not <00:07:49.457>zero <00:07:49.781>because

152
00:07:49.925 --> 00:07:51.925
<00:07:49.925>otherwise <00:07:50.125>you <00:07:50.325>would <00:07:50.525>always <00:07:50.725>get <00:07:50.925>the <00:07:51.125>same <00:07:51.325>reaction.

153
00:07:52.665 --> 00:07:56.685
<00:07:52.665>And <00:07:52.805>what <00:07:52.945>it <00:07:53.085>does <00:07:53.225>is <00:07:53.365>that <00:07:53.505>it <00:07:53.645>shows <00:07:53.785>you <00:07:53.925>the <00:07:54.065>percentage

154
00:07:56.945 --> 00:07:59.605
<00:07:56.945>of <00:07:57.178>the <00:07:57.411>possibilities <00:07:57.645>of <00:07:57.878>the <00:07:58.111>next <00:07:58.344>word.

155
00:08:00.225 --> 00:08:02.925
<00:08:00.225>And <00:08:00.352>what <00:08:00.479>you <00:08:00.606>see <00:08:00.734>at <00:08:00.861>the <00:08:00.988>left <00:08:01.115>side <00:08:01.243>is <00:08:01.370>a <00:08:01.497>3D <00:08:01.625>representation

156
00:08:03.345 --> 00:08:07.405
<00:08:03.345>of <00:08:03.545>the <00:08:03.745>token <00:08:03.945>space <00:08:04.145>inside <00:08:04.345>of <00:08:04.545>an <00:08:04.745>LLM.

157
00:08:08.225 --> 00:08:11.525
<00:08:08.225>So <00:08:08.380>an <00:08:08.536>LLM <00:08:08.691>of <00:08:08.847>course <00:08:09.002>doesn't <00:08:09.158>work <00:08:09.313>in <00:08:09.469>three <00:08:09.625>dimensions,

158
00:08:12.025 --> 00:08:15.965
<00:08:12.025>but <00:08:12.164>we <00:08:12.304>as <00:08:12.445>a <00:08:12.585>humans <00:08:12.724>are <00:08:12.864>really <00:08:13.005>bad <00:08:13.145>at <00:08:13.284>imagining <00:08:13.424>80

159
00:08:16.105 --> 00:08:17.725
<00:08:16.105>to <00:08:16.805>100 <00:08:17.505>dimensions.

160
00:08:18.065 --> 00:08:22.405
<00:08:18.065>So <00:08:18.192>we <00:08:18.319>just <00:08:18.446>um, <00:08:18.574>cooled <00:08:18.701>it <00:08:18.828>down <00:08:18.955>to <00:08:19.083>3D. <00:08:19.210>That's <00:08:19.337>really <00:08:19.465>cool.

161
00:08:22.825 --> 00:08:24.885
<00:08:22.825>And <00:08:23.058>you <00:08:23.291>see <00:08:23.525>like <00:08:23.758>on <00:08:23.991>the, <00:08:24.224>um,

162
00:08:25.185 --> 00:08:27.205
<00:08:25.185>bottom <00:08:25.418>right <00:08:25.651>there <00:08:25.885>is <00:08:26.118>the <00:08:26.351>starting <00:08:26.585>point

163
00:08:27.825 --> 00:08:30.765
<00:08:27.825>and <00:08:27.941>then <00:08:28.058>the <00:08:28.175>LLM <00:08:28.291>moved <00:08:28.408>a <00:08:28.525>bit <00:08:28.641>to <00:08:28.758>the <00:08:28.875>left <00:08:28.991>and <00:08:29.108>then <00:08:29.224>up

164
00:08:30.785 --> 00:08:31.925
<00:08:30.785>and <00:08:31.041>the <00:08:31.298>left <00:08:31.554>and <00:08:31.811>up.

165
00:08:32.185 --> 00:08:35.525
<00:08:32.184>And <00:08:32.418>those <00:08:32.651>are <00:08:32.885>vector <00:08:33.118>differences <00:08:33.351>between <00:08:33.584>tokens.

166
00:08:35.785 --> 00:08:37.445
<00:08:35.784>Of <00:08:36.018>course <00:08:36.251>they're <00:08:36.485>also <00:08:36.718>not <00:08:36.951>in <00:08:37.184>3D,

167
00:08:37.465 --> 00:08:41.805
<00:08:37.465>but <00:08:37.665>um, <00:08:37.865>so <00:08:38.065>depending <00:08:38.265>on <00:08:38.465>the <00:08:38.665>starting <00:08:38.865>point,

168
00:08:42.255 --> 00:08:45.365
<00:08:42.255>which <00:08:42.410>is <00:08:42.566>influenced <00:08:42.721>by <00:08:42.877>the <00:08:43.032>prompt, <00:08:43.188>you <00:08:43.343>give <00:08:43.499>it <00:08:43.655>like

169
00:08:45.945 --> 00:08:49.165
<00:08:45.945>the <00:08:46.100>date <00:08:46.256>time <00:08:46.411>and <00:08:46.567>your <00:08:46.722>text <00:08:46.878>and <00:08:47.033>stuff <00:08:47.189>like <00:08:47.345>that.

170
00:08:49.305 --> 00:08:52.405
<00:08:49.305>All <00:08:49.444>the <00:08:49.584>context <00:08:49.724>you <00:08:49.864>give <00:08:50.005>it, <00:08:50.145>it'll <00:08:50.285>start <00:08:50.425>somewhere <00:08:50.564>in <00:08:50.704>there

171
00:08:52.785 --> 00:08:54.165
<00:08:52.785>and <00:08:53.033>then <00:08:53.281>it <00:08:53.530>will <00:08:53.778>just <00:08:54.026>move

172
00:08:54.465 --> 00:08:57.565
<00:08:54.465>and <00:08:54.620>everything <00:08:54.776>it <00:08:54.931>finds <00:08:55.087>out <00:08:55.242>there, <00:08:55.398>it <00:08:55.553>will, <00:08:55.709>it'll <00:08:55.865>output.

173
00:08:58.425 --> 00:08:59.805
<00:08:58.425>Of <00:08:58.673>course <00:08:58.921>that's <00:08:59.170>a <00:08:59.418>simplified <00:08:59.666>model,

174
00:08:59.825 --> 00:09:02.365
<00:08:59.825>but <00:08:59.980>it's <00:09:00.136>really, <00:09:00.291>really, <00:09:00.447>um, <00:09:00.602>good <00:09:00.758>to <00:09:00.913>see <00:09:01.069>that <00:09:01.225>way.

175
00:09:02.785 --> 00:09:06.845
<00:09:02.785>And <00:09:02.985>because <00:09:03.185>what <00:09:03.385>this <00:09:03.584>also <00:09:03.785>shows <00:09:03.985>is <00:09:04.185>LLMs

176
00:09:07.375 --> 00:09:10.725
<00:09:07.375>don't <00:09:07.575>really <00:09:07.775>think <00:09:07.975>so <00:09:08.175>even <00:09:08.375>if <00:09:08.575>their <00:09:08.775>reasoning,

177
00:09:10.725 --> 00:09:13.165
<00:09:10.725>they're <00:09:10.925>just <00:09:11.125>moving <00:09:11.325>around <00:09:11.525>in <00:09:11.725>their <00:09:11.925>vector <00:09:12.125>space

178
00:09:13.585 --> 00:09:15.245
<00:09:13.585>and <00:09:13.935>outputting <00:09:14.285>everything <00:09:14.635>they <00:09:14.985>find.

179
00:09:15.785 --> 00:09:19.085
<00:09:15.785>So <00:09:15.940>reasoning <00:09:16.096>is <00:09:16.251>not <00:09:16.407>a <00:09:16.562>part <00:09:16.718>of <00:09:16.873>what <00:09:17.029>they <00:09:17.185>do.

180
00:09:21.425 --> 00:09:23.205
<00:09:21.425>So <00:09:21.704>why <00:09:21.984>does <00:09:22.265>it <00:09:22.545>even <00:09:22.824>work?

181
00:09:23.425 --> 00:09:26.365
<00:09:23.425>Of <00:09:23.599>course, <00:09:23.775>because <00:09:23.949>you <00:09:24.125>say <00:09:24.300>like <00:09:24.474>something <00:09:24.650>like <00:09:24.824>GPT,

182
00:09:26.625 --> 00:09:30.925
<00:09:26.625>it <00:09:26.765>has <00:09:26.905>read <00:09:27.045>everything <00:09:27.185>on <00:09:27.325>the <00:09:27.465>internet <00:09:27.605>up <00:09:27.745>to <00:09:27.885>the <00:09:28.025>point

183
00:09:31.375 --> 00:09:34.565
<00:09:31.375>where <00:09:31.608>um, <00:09:31.841>like <00:09:32.075>the <00:09:32.308>whole <00:09:32.541>of <00:09:32.775>Wikipedia

184
00:09:34.565 --> 00:09:37.005
<00:09:34.565>and <00:09:34.740>all <00:09:34.915>languages <00:09:35.090>is <00:09:35.265>about <00:09:35.440>3% <00:09:35.615>of <00:09:35.790>GP <00:09:35.965>four

185
00:09:37.025 --> 00:09:40.685
<00:09:37.025>and <00:09:37.141>we <00:09:37.258>know <00:09:37.375>GP <00:09:37.491>four <00:09:37.608>is <00:09:37.725>like <00:09:37.841>kind <00:09:37.958>of <00:09:38.074>small <00:09:38.191>in <00:09:38.308>the <00:09:38.425>meantime.

186
00:09:41.585 --> 00:09:45.005
<00:09:41.585>So <00:09:41.740>what <00:09:41.896>did <00:09:42.051>Google <00:09:42.207>do <00:09:42.362>to <00:09:42.518>create <00:09:42.673>more <00:09:42.829>training <00:09:42.985>data?

187
00:09:45.145 --> 00:09:46.485
<00:09:45.145>It <00:09:45.446>started <00:09:45.748>to <00:09:46.049>scrape <00:09:46.351>YouTube.

188
00:09:47.105 --> 00:09:51.645
<00:09:47.105>So <00:09:47.260>if <00:09:47.416>you're <00:09:47.571>an <00:09:47.727>avid <00:09:47.882>user <00:09:48.038>of <00:09:48.193>um, <00:09:48.349>transcript, <00:09:48.505>um,

189
00:09:51.835 --> 00:09:53.645
<00:09:51.835>text <00:09:52.535>below <00:09:53.235>interviews

190
00:09:53.665 --> 00:09:57.285
<00:09:53.665>and <00:09:53.805>stuff <00:09:53.944>like <00:09:54.084>that, <00:09:54.224>like <00:09:54.365>I <00:09:54.505>am, <00:09:54.645>you <00:09:54.785>can <00:09:54.925>just, <00:09:55.064>um,

191
00:09:58.615 --> 00:10:01.885
<00:09:58.615>thank <00:09:58.790>Google <00:09:58.965>and <00:09:59.140>LLMs <00:09:59.315>that <00:09:59.490>they <00:09:59.665>needed <00:09:59.840>more <00:10:00.015>data

192
00:10:02.075 --> 00:10:05.325
<00:10:02.075>because <00:10:02.250>they <00:10:02.425>just <00:10:02.600>double <00:10:02.775>used <00:10:02.950>all <00:10:03.125>the <00:10:03.300>transcripted <00:10:03.475>text.

193
00:10:06.785 --> 00:10:10.885
<00:10:06.785>So, <00:10:06.959>um, <00:10:07.135>what <00:10:07.310>happened? <00:10:07.485>There <00:10:07.660>are <00:10:07.834>some <00:10:08.010>emerging <00:10:08.185>abilities.

194
00:10:11.035 --> 00:10:13.365
<00:10:11.035>When <00:10:11.385>LLM <00:10:11.735>started <00:10:12.084>with <00:10:12.435>3.5,

195
00:10:13.735 --> 00:10:16.045
<00:10:13.735>there <00:10:14.015>were <00:10:14.295>instruction <00:10:14.575>no <00:10:14.855>instruct <00:10:15.135>models.

196
00:10:16.465 --> 00:10:20.205
<00:10:16.465>In <00:10:16.640>the <00:10:16.815>meantime, <00:10:16.990>nearly <00:10:17.165>every <00:10:17.340>model <00:10:17.515>is <00:10:17.690>is <00:10:17.865>output

197
00:10:20.225 --> 00:10:21.245
<00:10:20.225>as <00:10:20.531>an <00:10:20.837>instruct <00:10:21.143>model.

198
00:10:21.945 --> 00:10:26.245
<00:10:21.945>And <00:10:22.085>um, <00:10:22.225>so <00:10:22.365>what <00:10:22.505>did <00:10:22.645>happen <00:10:22.785>that <00:10:22.925>they <00:10:23.065>could <00:10:23.205>not <00:10:23.345>only

199
00:10:26.765 --> 00:10:28.925
<00:10:26.765>complete <00:10:26.965>anymore <00:10:27.165>because <00:10:27.365>at <00:10:27.564>the <00:10:27.765>beginning <00:10:27.965>it <00:10:28.165>were

200
00:10:28.955 --> 00:10:30.245
<00:10:28.955>more <00:10:29.342>like <00:10:29.729>fairytale <00:10:30.116>models.

201
00:10:30.385 --> 00:10:32.245
<00:10:30.385>You <00:10:30.735>said <00:10:31.085>like <00:10:31.435>please <00:10:31.785>continue

202
00:10:32.275 --> 00:10:35.805
<00:10:32.275>with <00:10:32.430>once <00:10:32.586>upon <00:10:32.741>a <00:10:32.897>while <00:10:33.052>there <00:10:33.208>was <00:10:33.363>um, <00:10:33.519>a <00:10:33.675>wolf

203
00:10:35.945 --> 00:10:38.925
<00:10:35.945>and <00:10:36.085>a <00:10:36.225>little <00:10:36.365>girl <00:10:36.505>and <00:10:36.645>they <00:10:36.785>were <00:10:36.925>in <00:10:37.065>a <00:10:37.205>forest <00:10:37.345>do.dot

204
00:10:38.925 --> 00:10:42.085
<00:10:38.925>and <00:10:39.080>it <00:10:39.236>would <00:10:39.391>complete <00:10:39.547>it <00:10:39.702>with <00:10:39.858>the <00:10:40.013>most <00:10:40.169>next, <00:10:40.324>uh,

205
00:10:42.085 --> 00:10:43.085
<00:10:42.085>most <00:10:42.535>probable <00:10:42.985>words.

206
00:10:43.465 --> 00:10:46.205
<00:10:43.465>And <00:10:43.640>in <00:10:43.815>the <00:10:43.990>meantime <00:10:44.165>you <00:10:44.340>can <00:10:44.515>give <00:10:44.690>it <00:10:44.865>instructions.

207
00:10:46.635 --> 00:10:50.405
<00:10:46.635>Like <00:10:46.790>you <00:10:46.946>maybe <00:10:47.101>have <00:10:47.257>heard <00:10:47.412>that <00:10:47.568>the <00:10:47.723>the <00:10:47.879>system <00:10:48.035>prompt

208
00:10:50.505 --> 00:10:52.725
<00:10:50.505>of <00:10:50.785>tropic <00:10:51.064>Claude <00:10:51.345>is <00:10:51.625>like, <00:10:51.905>um,

209
00:10:52.885 --> 00:10:55.245
<00:10:52.885>a <00:10:53.118>little <00:10:53.351>book <00:10:53.585>like <00:10:53.818>this, <00:10:54.051>this <00:10:54.285>thick.

210
00:10:55.435 --> 00:10:58.285
<00:10:55.435>Yeah, <00:10:55.668>because <00:10:55.901>they <00:10:56.135>give <00:10:56.368>it, <00:10:56.601>um, <00:10:56.834>ideas

211
00:10:58.595 --> 00:11:01.605
<00:10:58.595>what <00:10:58.770>it <00:10:58.945>should <00:10:59.120>do <00:10:59.295>in <00:10:59.470>like <00:10:59.645>if <00:10:59.820>it <00:10:59.995>discovers

212
00:11:01.605 --> 00:11:04.805
<00:11:01.605>that <00:11:01.780>it <00:11:01.955>not <00:11:02.130>know <00:11:02.305>something <00:11:02.480>when <00:11:02.655>it <00:11:02.830>should <00:11:03.005>resort

213
00:11:04.825 --> 00:11:06.245
<00:11:04.825>to <00:11:05.080>tools <00:11:05.336>and <00:11:05.591>stuff <00:11:05.847>like <00:11:06.103>that.

214
00:11:06.825 --> 00:11:11.085
<00:11:06.825>If <00:11:06.952>yourself <00:11:07.079>are <00:11:07.206>building <00:11:07.334>an <00:11:07.461>API, <00:11:07.588>if <00:11:07.715>you <00:11:07.843>are <00:11:07.970>using <00:11:08.097>the <00:11:08.225>API

215
00:11:11.185 --> 00:11:15.365
<00:11:11.185>of <00:11:11.312>cloud <00:11:11.439>open, <00:11:11.566>may <00:11:11.694>I <00:11:11.821>of <00:11:11.948>Azure <00:11:12.075>Open, <00:11:12.203>may <00:11:12.330>I <00:11:12.457>missal, <00:11:12.584>whatever,

216
00:11:15.985 --> 00:11:18.485
<00:11:15.985>you <00:11:16.185>have <00:11:16.385>to <00:11:16.585>create <00:11:16.785>this <00:11:16.985>system <00:11:17.185>prompt <00:11:17.385>yourself.

217
00:11:18.995 --> 00:11:22.605
<00:11:18.995>Yeah. <00:11:19.122>You <00:11:19.249>have <00:11:19.376>to <00:11:19.504>tell <00:11:19.631>it <00:11:19.758>when <00:11:19.885>it <00:11:20.013>can <00:11:20.140>resort <00:11:20.267>to, <00:11:20.395>um,

218
00:11:22.615 --> 00:11:25.805
<00:11:22.615>those <00:11:22.770>tools, <00:11:22.926>what <00:11:23.081>it <00:11:23.237>can <00:11:23.392>use <00:11:23.548>and <00:11:23.703>stuff <00:11:23.859>like <00:11:24.015>that.

219
00:11:26.265 --> 00:11:28.525
<00:11:26.265>Um, <00:11:26.465>of <00:11:26.665>course <00:11:26.865>it <00:11:27.064>has <00:11:27.265>some <00:11:27.465>basic <00:11:27.665>instructions

220
00:11:28.525 --> 00:11:30.325
<00:11:28.525>that <00:11:28.758>already <00:11:28.991>in <00:11:29.225>there <00:11:29.458>in <00:11:29.691>the <00:11:29.925>model,

221
00:11:30.625 --> 00:11:33.965
<00:11:30.625>but <00:11:30.800>you <00:11:30.975>can, <00:11:31.150>um, <00:11:31.325>like <00:11:31.500>it, <00:11:31.675>it <00:11:31.850>really, <00:11:32.025>most

222
00:11:33.965 --> 00:11:36.925
<00:11:33.965>of <00:11:34.120>the <00:11:34.276>models <00:11:34.431>are <00:11:34.587>really <00:11:34.742>the <00:11:34.898>first <00:11:35.053>things <00:11:35.209>they <00:11:35.365>get.

223
00:11:37.105 --> 00:11:38.405
<00:11:37.105>It <00:11:37.397>will <00:11:37.690>treat <00:11:37.982>as <00:11:38.275>law

224
00:11:38.425 --> 00:11:41.285
<00:11:38.425>and <00:11:38.580>will <00:11:38.736>try <00:11:38.891>not <00:11:39.047>to <00:11:39.202>break <00:11:39.358>it <00:11:39.513>in <00:11:39.669>any <00:11:39.824>case.

225
00:11:41.595 --> 00:11:45.645
<00:11:41.595>Yeah. <00:11:41.828>So, <00:11:42.061>um, <00:11:42.295>with <00:11:42.528>this <00:11:42.761>unexpected <00:11:42.995>capabilities,

226
00:11:45.825 --> 00:11:48.565
<00:11:45.825>we <00:11:46.025>could <00:11:46.225>get <00:11:46.425>it <00:11:46.625>to <00:11:46.825>respond <00:11:47.025>in <00:11:47.225>chase

227
00:11:48.565 --> 00:11:51.445
<00:11:48.565>and <00:11:48.798>respond <00:11:49.031>in <00:11:49.265>whatever <00:11:49.498>format <00:11:49.731>we <00:11:49.965>wanted.

228
00:11:51.705 --> 00:11:54.885
<00:11:51.705>We <00:11:51.905>can <00:11:52.105>use <00:11:52.305>it <00:11:52.505>to <00:11:52.705>summarize <00:11:52.905>one <00:11:53.105>bit

229
00:11:55.185 --> 00:11:58.885
<00:11:55.185>and <00:11:55.324>um, <00:11:55.464>introduce <00:11:55.604>it <00:11:55.744>to <00:11:55.885>another <00:11:56.025>thing <00:11:56.165>and <00:11:56.305>stuff <00:11:56.444>like <00:11:56.584>that.

230
00:12:00.125 --> 00:12:02.145
<00:12:00.125>So <00:12:00.405>what <00:12:00.685>are <00:12:00.965>the <00:12:01.245>current <00:12:01.525>limitations?

231
00:12:02.145 --> 00:12:04.745
<00:12:02.145>What <00:12:02.272>can, <00:12:02.399>can <00:12:02.526>we <00:12:02.654>do <00:12:02.781>and <00:12:02.908>what <00:12:03.035>can't <00:12:03.163>we <00:12:03.290>do <00:12:03.417>with <00:12:03.545>it?

232
00:12:05.205 --> 00:12:07.105
<00:12:05.205>Um, <00:12:05.405>the <00:12:05.605>next <00:12:05.805>slide <00:12:06.005>is <00:12:06.205>like <00:12:06.405>the <00:12:06.605>second

233
00:12:07.325 --> 00:12:09.025
<00:12:07.325>or <00:12:07.558>the <00:12:07.791>most <00:12:08.025>important <00:12:08.258>point <00:12:08.491>I <00:12:08.725>wanna,

234
00:12:09.385 --> 00:12:13.785
<00:12:09.385>I <00:12:09.512>wanna <00:12:09.639>give <00:12:09.766>you <00:12:09.894>like <00:12:10.021>from <00:12:10.148>my <00:12:10.275>point <00:12:10.403>of <00:12:10.530>view <00:12:10.657>is <00:12:10.785>like

235
00:12:14.335 --> 00:12:18.705
<00:12:14.335>what <00:12:14.535>LLMs <00:12:14.735>are <00:12:14.935>not <00:12:15.135>as <00:12:15.335>like <00:12:15.535>the <00:12:15.735>I

236
00:12:19.285 --> 00:12:21.425
<00:12:19.285>in <00:12:19.564>the <00:12:19.844>LLM <00:12:20.125>stands <00:12:20.405>for <00:12:20.685>intelligence.

237
00:12:21.975 --> 00:12:23.945
<00:12:21.975>They <00:12:22.441>are <00:12:22.908>not <00:12:23.375>intelligent.

238
00:12:24.365 --> 00:12:28.345
<00:12:24.365>Yes, <00:12:24.598>they <00:12:24.831>are <00:12:25.065>simply <00:12:25.298>following <00:12:25.531>patterns <00:12:25.765>in

239
00:12:28.965 --> 00:12:30.065
<00:12:28.965>all <00:12:29.212>the <00:12:29.460>way <00:12:29.707>they <00:12:29.955>can.

240
00:12:30.485 --> 00:12:35.465
<00:12:30.485>If <00:12:30.612>you <00:12:30.739>tell <00:12:30.866>GPT <00:12:30.994>that <00:12:31.121>there <00:12:31.248>is <00:12:31.375>um, <00:12:31.503>a <00:12:31.630>sheep <00:12:31.757>and <00:12:31.885>um,

241
00:12:35.565 --> 00:12:37.545
<00:12:35.565>and <00:12:35.720>some <00:12:35.876>other <00:12:36.031>guy <00:12:36.187>at <00:12:36.342>one <00:12:36.498>side <00:12:36.653>of <00:12:36.809>the <00:12:36.965>river

242
00:12:37.645 --> 00:12:41.345
<00:12:37.645>and <00:12:37.761>they <00:12:37.878>have <00:12:37.995>to <00:12:38.111>cross <00:12:38.228>it, <00:12:38.345>it'll <00:12:38.461>think <00:12:38.578>of <00:12:38.694>the, <00:12:38.811>of <00:12:38.928>the <00:12:39.045>riddle

243
00:12:41.345 --> 00:12:44.105
<00:12:41.345>with <00:12:41.472>the <00:12:41.599>sheep <00:12:41.726>and <00:12:41.854>the <00:12:41.981>cabbage <00:12:42.108>and <00:12:42.235>the <00:12:42.363>wolf <00:12:42.490>and <00:12:42.617>the <00:12:42.745>boat

244
00:12:44.285 --> 00:12:46.105
<00:12:44.285>and <00:12:44.564>it'll <00:12:44.844>tell <00:12:45.125>you <00:12:45.405>ah, <00:12:45.685>yes.

245
00:12:46.325 --> 00:12:48.745
<00:12:46.325>Um, <00:12:46.525>first <00:12:46.725>the <00:12:46.925>guy <00:12:47.125>can <00:12:47.325>cross <00:12:47.525>the <00:12:47.725>river,

246
00:12:48.815 --> 00:12:51.465
<00:12:48.815>then <00:12:48.970>it <00:12:49.126>can <00:12:49.281>come <00:12:49.437>back, <00:12:49.592>then <00:12:49.748>it <00:12:49.903>can <00:12:50.059>cross <00:12:50.215>again

247
00:12:51.495 --> 00:12:55.025
<00:12:51.495>with <00:12:51.635>the <00:12:51.775>sheep <00:12:51.915>try <00:12:52.055>it <00:12:52.195>like <00:12:52.335>in <00:12:52.475>three <00:12:52.615>to <00:12:52.755>four <00:12:52.895>um,

248
00:12:55.035 --> 00:12:56.625
<00:12:55.035>cases <00:12:55.314>out <00:12:55.594>of <00:12:55.875>out <00:12:56.155>of <00:12:56.435>five.

249
00:12:56.805 --> 00:12:58.905
<00:12:56.805>It <00:12:57.005>still <00:12:57.204>is <00:12:57.405>wrong <00:12:57.604>with <00:12:57.805>with <00:12:58.005>these <00:12:58.204>things

250
00:12:58.935 --> 00:13:02.065
<00:12:58.935>because <00:12:59.090>it's <00:12:59.246>so <00:12:59.401>stuck <00:12:59.557>with <00:12:59.712>solving <00:12:59.868>it <00:13:00.023>with <00:13:00.179>a <00:13:00.334>pattern.

251
00:13:02.655 --> 00:13:04.985
<00:13:02.655>It's <00:13:02.888>so <00:13:03.121>drain <00:13:03.355>of <00:13:03.588>solving <00:13:03.821>these <00:13:04.055>riddles

252
00:13:04.985 --> 00:13:06.625
<00:13:04.985>that <00:13:05.265>it <00:13:05.545>can <00:13:05.825>think <00:13:06.105>normal <00:13:06.385>anymore.

253
00:13:07.125 --> 00:13:10.665
<00:13:07.125>But <00:13:07.280>you <00:13:07.436>can <00:13:07.591>use <00:13:07.747>this <00:13:07.902>like <00:13:08.058>reasoning <00:13:08.213>is <00:13:08.369>exactly <00:13:08.525>this.

254
00:13:10.845 --> 00:13:14.265
<00:13:10.845>You <00:13:10.972>give <00:13:11.099>it, <00:13:11.226>um, <00:13:11.354>you <00:13:11.481>let <00:13:11.608>it <00:13:11.735>coach <00:13:11.863>yourself, <00:13:11.990>you <00:13:12.117>tell <00:13:12.245>it,

255
00:13:14.535 --> 00:13:16.145
<00:13:14.535>give <00:13:14.885>yourself <00:13:15.235>a <00:13:15.584>pattern, <00:13:15.935>search

256
00:13:16.165 --> 00:13:17.945
<00:13:16.165>for <00:13:16.398>a <00:13:16.631>pattern <00:13:16.865>and <00:13:17.098>then <00:13:17.331>use <00:13:17.564>this.

257
00:13:18.485 --> 00:13:21.265
<00:13:18.485>So <00:13:18.640>you <00:13:18.796>can <00:13:18.951>in <00:13:19.107>your <00:13:19.262>prompt <00:13:19.418>give <00:13:19.573>it <00:13:19.729>some <00:13:19.885>examples.

258
00:13:21.285 --> 00:13:24.625
<00:13:21.285>You <00:13:21.425>can <00:13:21.564>tell <00:13:21.704>it <00:13:21.844>to <00:13:21.985>like <00:13:22.125>if <00:13:22.265>you <00:13:22.405>see <00:13:22.545>something <00:13:22.685>like

259
00:13:24.625 --> 00:13:26.865
<00:13:24.625>that <00:13:24.858>first <00:13:25.091>do <00:13:25.325>that, <00:13:25.558>that <00:13:25.791>and <00:13:26.025>that.

260
00:13:26.965 --> 00:13:29.705
<00:13:26.965>Or <00:13:27.092>if <00:13:27.219>you <00:13:27.346>see <00:13:27.474>that <00:13:27.601>you <00:13:27.728>give <00:13:27.855>it, <00:13:27.983>you <00:13:28.110>give <00:13:28.237>it <00:13:28.365>examples

261
00:13:29.925 --> 00:13:32.945
<00:13:29.925>and <00:13:30.064>you <00:13:30.204>can <00:13:30.344>really <00:13:30.484>give <00:13:30.625>it <00:13:30.765>pages <00:13:30.905>and <00:13:31.045>pages <00:13:31.185>on <00:13:31.324>instructions

262
00:13:33.525 --> 00:13:35.225
<00:13:33.525>and <00:13:33.875>um, <00:13:34.225>on <00:13:34.574>the <00:13:34.925>examples

263
00:13:35.525 --> 00:13:38.745
<00:13:35.525>and <00:13:35.665>it <00:13:35.805>can <00:13:35.944>use <00:13:36.084>them <00:13:36.225>to <00:13:36.365>uh, <00:13:36.505>create <00:13:36.645>a <00:13:36.785>better <00:13:36.925>output.

264
00:13:39.125 --> 00:13:42.985
<00:13:39.125>But <00:13:39.358>LLMs <00:13:39.591>are <00:13:39.825>really <00:13:40.058>advanced <00:13:40.291>pattern <00:13:40.525>measure.

265
00:13:43.415 --> 00:13:46.385
<00:13:43.415>Yeah, <00:13:43.589>they're <00:13:43.765>good <00:13:43.939>with <00:13:44.115>that, <00:13:44.290>but <00:13:44.464>they're <00:13:44.640>not <00:13:44.814>thinking.

266
00:13:46.655 --> 00:13:49.945
<00:13:46.655>Yeah. <00:13:46.935>So <00:13:47.214>what <00:13:47.495>are <00:13:47.775>LLMs <00:13:48.055>really?

267
00:13:50.165 --> 00:13:54.345
<00:13:50.165>So <00:13:50.281>what <00:13:50.398>I <00:13:50.515>like <00:13:50.631>to <00:13:50.748>say <00:13:50.865>is <00:13:50.981>like <00:13:51.098>to <00:13:51.214>paraphrase <00:13:51.331>one <00:13:51.448>of, <00:13:51.564>um,

268
00:13:54.685 --> 00:13:59.505
<00:13:54.685>our <00:13:54.824>developers <00:13:54.964>is <00:13:55.104>that <00:13:55.244>LLMs <00:13:55.385>are <00:13:55.525>kind <00:13:55.665>of <00:13:55.805>a <00:13:55.944>new <00:13:56.084>API.

269
00:14:00.545 --> 00:14:03.395
<00:14:00.545>Back <00:14:00.700>then <00:14:00.856>you <00:14:01.011>had <00:14:01.167>an <00:14:01.322>API <00:14:01.478>like <00:14:01.633>maybe <00:14:01.789>from <00:14:01.944>Google

270
00:14:03.935 --> 00:14:06.835
<00:14:03.935>to <00:14:04.168>calculate <00:14:04.401>the <00:14:04.635>distance <00:14:04.868>between <00:14:05.101>two <00:14:05.334>places

271
00:14:07.295 --> 00:14:09.515
<00:14:07.295>or <00:14:07.574>maybe <00:14:07.854>to <00:14:08.135>calculate <00:14:08.415>something <00:14:08.694>else.

272
00:14:09.895 --> 00:14:13.595
<00:14:09.895>Now <00:14:10.050>you <00:14:10.206>have <00:14:10.361>an <00:14:10.517>an <00:14:10.672>API <00:14:10.828>that <00:14:10.983>can <00:14:11.139>calculate <00:14:11.295>text.

273
00:14:14.015 --> 00:14:17.235
<00:14:14.015>You <00:14:14.131>give <00:14:14.248>it <00:14:14.365>ideas <00:14:14.481>and <00:14:14.598>it <00:14:14.715>gives <00:14:14.831>you <00:14:14.948>text <00:14:15.064>back <00:14:15.181>in <00:14:15.298>any <00:14:15.415>format.

274
00:14:17.295 --> 00:14:20.755
<00:14:17.295>You <00:14:17.411>want <00:14:17.528>it <00:14:17.645>to <00:14:17.761>be <00:14:17.878>it <00:14:17.995>X <00:14:18.111>ML <00:14:18.228>or <00:14:18.344>Jason <00:14:18.461>or <00:14:18.578>whatever. <00:14:18.694>Yeah.

275
00:14:21.215 --> 00:14:24.395
<00:14:21.215>And <00:14:21.415>with <00:14:21.615>that <00:14:21.815>API, <00:14:22.015>you <00:14:22.215>can <00:14:22.415>even <00:14:22.615>go

276
00:14:24.855 --> 00:14:27.555
<00:14:24.855>and <00:14:25.321>control <00:14:25.788>your <00:14:26.255>app

277
00:14:27.975 --> 00:14:32.435
<00:14:27.975>and <00:14:28.130>it <00:14:28.286>can <00:14:28.441>suddenly <00:14:28.597>react <00:14:28.752>to <00:14:28.908>stuff <00:14:29.063>like <00:14:29.219>audio <00:14:29.375>inputs

278
00:14:32.435 --> 00:14:33.675
<00:14:32.435>and <00:14:32.658>commands <00:14:32.881>and <00:14:33.104>stuff <00:14:33.327>like <00:14:33.550>that.

279
00:14:34.335 --> 00:14:38.035
<00:14:34.335>So, <00:14:34.475>um, <00:14:34.615>yeah, <00:14:34.755>that <00:14:34.895>would <00:14:35.035>be <00:14:35.175>the <00:14:35.315>correct <00:14:35.455>slide <00:14:35.595>for <00:14:35.735>that.

280
00:14:38.575 --> 00:14:42.115
<00:14:38.575>Um, <00:14:38.715>but <00:14:38.855>the <00:14:38.995>thing <00:14:39.135>is <00:14:39.275>again, <00:14:39.415>with <00:14:39.555>that <00:14:39.695>also <00:14:39.835>with <00:14:39.975>audio

281
00:14:42.815 --> 00:14:45.675
<00:14:42.815>you <00:14:43.048>have <00:14:43.281>to <00:14:43.515>retry <00:14:43.748>once <00:14:43.981>or <00:14:44.215>twice.

282
00:14:46.065 --> 00:14:50.555
<00:14:46.065>Like <00:14:46.192>it's <00:14:46.319>far <00:14:46.446>better <00:14:46.574>since <00:14:46.701>GPD <00:14:46.828>3.5 <00:14:46.955>where <00:14:47.083>I <00:14:47.210>once <00:14:47.337>said, <00:14:47.465>uh,

283
00:14:50.555 --> 00:14:52.205
<00:14:50.555>okay, <00:14:50.788>we <00:14:51.021>need <00:14:51.255>to <00:14:51.488>retry <00:14:51.721>nine <00:14:51.954>times

284
00:14:52.795 --> 00:14:55.845
<00:14:52.795>because <00:14:52.950>like <00:14:53.106>it, <00:14:53.261>it <00:14:53.417>get <00:14:53.572>its <00:14:53.728>wrong <00:14:53.883>all <00:14:54.039>the <00:14:54.194>time.

285
00:14:56.065 --> 00:14:58.765
<00:14:56.065>It, <00:14:56.265>it <00:14:56.465>outputs <00:14:56.665>wrong <00:14:56.865>Chas <00:14:57.065>all <00:14:57.265>the <00:14:57.465>time.

286
00:14:59.265 --> 00:15:02.085
<00:14:59.265>Um, <00:14:59.545>LLMs <00:14:59.824>are <00:15:00.105>simply <00:15:00.385>not <00:15:00.665>reliable.

287
00:15:02.625 --> 00:15:04.885
<00:15:02.625>And <00:15:02.800>the <00:15:02.975>one <00:15:03.150>thing <00:15:03.325>is <00:15:03.500>that <00:15:03.675>you <00:15:03.850>should, <00:15:04.025>um,

288
00:15:05.505 --> 00:15:07.725
<00:15:05.505>should <00:15:05.785>retry <00:15:06.064>on <00:15:06.345>the <00:15:06.625>same <00:15:06.905>LLM,

289
00:15:08.145 --> 00:15:11.845
<00:15:08.145>but <00:15:08.300>um, <00:15:08.456>from <00:15:08.611>my <00:15:08.767>experience <00:15:08.922>it <00:15:09.078>is <00:15:09.233>very, <00:15:09.389>very <00:15:09.545>helpful

290
00:15:12.385 --> 00:15:15.165
<00:15:12.385>if <00:15:12.735>you <00:15:13.085>add <00:15:13.435>multiple <00:15:13.785>endpoints.

291
00:15:15.905 --> 00:15:19.845
<00:15:15.905>So <00:15:16.060>don't <00:15:16.216>just <00:15:16.371>stick <00:15:16.527>with <00:15:16.682>open <00:15:16.838>me <00:15:16.993>eye <00:15:17.149>on <00:15:17.305>Azure

292
00:15:20.225 --> 00:15:22.245
<00:15:20.225>or <00:15:20.575>just <00:15:20.925>Mytral <00:15:21.275>or <00:15:21.625>just

293
00:15:22.265 --> 00:15:25.045
<00:15:22.265>and <00:15:22.498>tropic <00:15:22.731>they <00:15:22.965>are <00:15:23.198>kind <00:15:23.431>of <00:15:23.665>interchangeable.

294
00:15:25.395 --> 00:15:29.245
<00:15:25.395>Yeah. <00:15:25.522>Like <00:15:25.649>the <00:15:25.776>bigger <00:15:25.904>models <00:15:26.031>are <00:15:26.158>kind <00:15:26.285>of <00:15:26.413>at <00:15:26.540>the <00:15:26.667>same <00:15:26.795>level.

295
00:15:29.425 --> 00:15:30.965
<00:15:29.425>Um, <00:15:29.656>what <00:15:29.887>they <00:15:30.117>will <00:15:30.349>respond <00:15:30.580>to <00:15:30.811>you.

296
00:15:31.385 --> 00:15:35.325
<00:15:31.385>So <00:15:31.512>at <00:15:31.639>the <00:15:31.766>same <00:15:31.894>time <00:15:32.021>you <00:15:32.148>can <00:15:32.275>use <00:15:32.403>Gemini <00:15:32.530>or <00:15:32.657>or <00:15:32.785>CLO

297
00:15:35.505 --> 00:15:36.765
<00:15:35.505>and <00:15:36.072>and <00:15:36.639>whatever.

298
00:15:37.225 --> 00:15:40.645
<00:15:37.225>Yes. <00:15:37.458>And <00:15:37.691>the <00:15:37.925>APIs <00:15:38.158>are <00:15:38.391>interchangeable <00:15:38.625>too

299
00:15:40.645 --> 00:15:43.485
<00:15:40.645>because <00:15:40.878>almost <00:15:41.111>every <00:15:41.345>endpoint <00:15:41.578>available <00:15:41.811>follows <00:15:42.045>the

300
00:15:44.225 --> 00:15:48.645
<00:15:44.225>AI <00:15:44.380>standard <00:15:44.536>so <00:15:44.691>you <00:15:44.847>can <00:15:45.002>use <00:15:45.158>that <00:15:45.313>to, <00:15:45.469>um, <00:15:45.625>work.

301
00:15:49.505 --> 00:15:52.405
<00:15:49.505>So, <00:15:49.738>but <00:15:49.971>I <00:15:50.205>said <00:15:50.438>they <00:15:50.671>calculate <00:15:50.905>text,

302
00:15:52.745 --> 00:15:55.845
<00:15:52.745>but <00:15:52.885>one <00:15:53.025>point <00:15:53.165>I <00:15:53.305>want <00:15:53.445>to <00:15:53.585>make <00:15:53.725>is <00:15:53.865>they <00:15:54.005>can <00:15:54.145>calculate.

303
00:15:56.905 --> 00:16:01.605
<00:15:56.905>So <00:15:57.012>this <00:15:57.120>was <00:15:57.228>a <00:15:57.335>study <00:15:57.443>of <00:15:57.551>um, <00:15:57.658>oh <00:15:57.766>one <00:15:57.874>mini <00:15:57.981>where <00:15:58.089>they <00:15:58.197>tried <00:15:58.305>like

304
00:16:02.025 --> 00:16:04.445
<00:16:02.025>how <00:16:02.375>um, <00:16:02.725>accurate <00:16:03.074>this <00:16:03.425>is

305
00:16:04.785 --> 00:16:08.005
<00:16:04.785>and <00:16:04.940>you <00:16:05.096>see <00:16:05.251>as <00:16:05.407>soon <00:16:05.562>as <00:16:05.718>you <00:16:05.873>surpass <00:16:06.029>the <00:16:06.185>um,

306
00:16:08.695 --> 00:16:13.085
<00:16:08.695>eight <00:16:08.870>times <00:16:09.045>eight <00:16:09.220>digits, <00:16:09.395>the <00:16:09.570>results <00:16:09.745>will <00:16:09.920>get <00:16:10.095>wrong.

307
00:16:13.465 --> 00:16:17.125
<00:16:13.465>And <00:16:13.698>why, <00:16:13.931>because <00:16:14.165>LLMs <00:16:14.398>cannot <00:16:14.631>calculate <00:16:14.865>tax.

308
00:16:17.155 --> 00:16:18.405
<00:16:17.155>They <00:16:17.717>cannot <00:16:18.280>multiply.

309
00:16:19.145 --> 00:16:22.205
<00:16:19.145>If <00:16:19.378>you <00:16:19.611>ask <00:16:19.845>GPT <00:16:20.078>like <00:16:20.311>what <00:16:20.545>is

310
00:16:22.775 --> 00:16:27.125
<00:16:22.775>50,000 <00:16:22.930>times <00:16:23.086>1 <00:16:23.241>million, <00:16:23.397>it'll <00:16:23.552>create <00:16:23.708>itself <00:16:23.863>a <00:16:24.019>Python <00:16:24.175>script

311
00:16:27.145 --> 00:16:29.085
<00:16:27.145>to <00:16:27.345>calculate <00:16:27.545>it <00:16:27.745>and <00:16:27.944>give <00:16:28.145>you <00:16:28.345>the <00:16:28.545>response,

312
00:16:29.545 --> 00:16:33.725
<00:16:29.545>but <00:16:29.700>it <00:16:29.856>cannot <00:16:30.011>calculate, <00:16:30.167>they <00:16:30.322>will <00:16:30.478>search <00:16:30.633>in <00:16:30.789>their <00:16:30.944>memories

313
00:16:33.785 --> 00:16:36.925
<00:16:33.785>for <00:16:33.940>stuff <00:16:34.096>where <00:16:34.251>people <00:16:34.407>talked <00:16:34.562>about <00:16:34.718>stuff <00:16:34.873>like <00:16:35.029>that <00:16:35.185>and,

314
00:16:37.105 --> 00:16:39.045
<00:16:37.105>and <00:16:37.455>try <00:16:37.805>to <00:16:38.155>go <00:16:38.505>close.

315
00:16:40.105 --> 00:16:44.925
<00:16:40.105>And <00:16:40.280>also <00:16:40.455>LLMs <00:16:40.630>will <00:16:40.805>absolutely <00:16:40.980>lie <00:16:41.155>to <00:16:41.330>your <00:16:41.505>face.

316
00:16:45.905 --> 00:16:48.965
<00:16:45.905>So <00:16:46.138>if <00:16:46.371>we're <00:16:46.605>talking <00:16:46.838>about <00:16:47.071>explainable <00:16:47.305>ai,

317
00:16:49.185 --> 00:16:51.165
<00:16:49.185>people <00:16:49.359>were <00:16:49.535>in <00:16:49.709>the <00:16:49.885>beginning, <00:16:50.060>they <00:16:50.234>were <00:16:50.410>like, <00:16:50.584>oh,

318
00:16:51.175 --> 00:16:53.845
<00:16:51.175>we'll <00:16:51.330>just <00:16:51.486>ask <00:16:51.641>it <00:16:51.797>why <00:16:51.952>it <00:16:52.108>came <00:16:52.263>to <00:16:52.419>that <00:16:52.574>conclusion.

319
00:16:54.385 --> 00:16:56.085
<00:16:54.385>So <00:16:54.735>what <00:16:55.085>did <00:16:55.435>Entropic <00:16:55.785>do?

320
00:16:56.355 --> 00:17:00.965
<00:16:56.355>They, <00:16:56.510>um, <00:16:56.666>kind <00:16:56.821>of <00:16:56.977>attached <00:16:57.132>like, <00:16:57.288>uh, <00:16:57.443>um, <00:16:57.599>neural, <00:16:57.755>um,

321
00:17:04.085 --> 00:17:08.485
<00:17:04.085>a <00:17:04.240>neural <00:17:04.396>measurement <00:17:04.551>system <00:17:04.707>to <00:17:04.862>the <00:17:05.018>neurons <00:17:05.173>of <00:17:05.329>the <00:17:05.485>LLM

322
00:17:08.905 --> 00:17:11.645
<00:17:08.905>and <00:17:09.138>try <00:17:09.371>to <00:17:09.605>see <00:17:09.838>which <00:17:10.071>parts <00:17:10.305>fire.

323
00:17:12.305 --> 00:17:16.045
<00:17:12.305>So <00:17:12.421>they, <00:17:12.538>um, <00:17:12.655>if <00:17:12.771>you <00:17:12.888>remember <00:17:13.005>the <00:17:13.121>3D <00:17:13.238>thing <00:17:13.355>is <00:17:13.471>like <00:17:13.588>they <00:17:13.705>kind

324
00:17:16.045 --> 00:17:17.925
<00:17:16.045>of <00:17:16.325>could, <00:17:16.605>um, <00:17:16.885>trace <00:17:17.165>the <00:17:17.445>way

325
00:17:18.785 --> 00:17:21.845
<00:17:18.785>and <00:17:18.985>they <00:17:19.185>discovered <00:17:19.385>what <00:17:19.585>the <00:17:19.785>LLM <00:17:19.985>tells <00:17:20.185>you

326
00:17:21.915 --> 00:17:24.605
<00:17:21.915>what <00:17:22.148>it <00:17:22.381>thought <00:17:22.615>is <00:17:22.848>completely <00:17:23.081>made <00:17:23.315>up.

327
00:17:24.985 --> 00:17:27.125
<00:17:24.984>And <00:17:25.218>of <00:17:25.451>course, <00:17:25.685>because <00:17:25.918>it <00:17:26.151>doesn't <00:17:26.385>know,

328
00:17:27.265 --> 00:17:29.765
<00:17:27.265>it <00:17:27.420>has <00:17:27.576>no <00:17:27.731>conscience, <00:17:27.887>it <00:17:28.042>cannot <00:17:28.198>think <00:17:28.353>like <00:17:28.509>you <00:17:28.665>do.

329
00:17:30.275 --> 00:17:31.645
<00:17:30.275>It's <00:17:30.583>really, <00:17:30.891>it's <00:17:31.199>really <00:17:31.508>important

330
00:17:31.665 --> 00:17:34.645
<00:17:31.665>to <00:17:31.840>get <00:17:32.014>the <00:17:32.190>differentiation <00:17:32.365>between <00:17:32.540>us <00:17:32.715>who <00:17:32.889>uses <00:17:33.065>text

331
00:17:34.705 --> 00:17:36.565
<00:17:34.705>as <00:17:34.938>an <00:17:35.171>output <00:17:35.405>of <00:17:35.638>our <00:17:35.871>thought <00:17:36.105>process

332
00:17:37.065 --> 00:17:39.925
<00:17:37.065>and <00:17:37.298>text <00:17:37.531>as <00:17:37.765>an <00:17:37.998>output <00:17:38.231>of <00:17:38.465>nothing.

333
00:17:40.635 --> 00:17:42.605
<00:17:40.635>It's <00:17:40.985>just <00:17:41.335>the <00:17:41.685>output. <00:17:42.035>Yeah.

334
00:17:43.385 --> 00:17:48.325
<00:17:43.385>And, <00:17:43.525>and <00:17:43.665>so <00:17:43.805>they <00:17:43.945>discovered <00:17:44.085>that <00:17:44.225>even <00:17:44.365>when <00:17:44.504>asking <00:17:44.645>stuff <00:17:44.785>like

335
00:17:48.435 --> 00:17:50.085
<00:17:48.435>what <00:17:48.784>is <00:17:49.135>nine <00:17:49.485>times <00:17:49.835>nine?

336
00:17:50.545 --> 00:17:53.565
<00:17:50.545>Um, <00:17:50.700>the <00:17:50.856>LLM <00:17:51.011>will <00:17:51.167>tell <00:17:51.322>you <00:17:51.478>some <00:17:51.633>elaborate <00:17:51.789>things <00:17:51.945>on

337
00:17:53.625 --> 00:17:55.045
<00:17:53.625>how <00:17:53.880>it <00:17:54.136>came <00:17:54.391>to <00:17:54.647>that <00:17:54.903>conclusion,

338
00:17:55.425 --> 00:17:57.805
<00:17:55.425>but <00:17:55.580>in <00:17:55.736>the <00:17:55.891>real <00:17:56.047>time <00:17:56.202>it <00:17:56.358>things <00:17:56.513>like, <00:17:56.669>ah, <00:17:56.825>okay,

339
00:17:57.845 --> 00:18:00.405
<00:17:57.845>I <00:17:58.045>saw <00:17:58.245>something <00:17:58.445>10 <00:17:58.645>times <00:17:58.845>nine <00:17:59.045>is <00:17:59.245>90,

340
00:18:00.585 --> 00:18:02.165
<00:18:00.585>so <00:18:00.818>it <00:18:01.051>should <00:18:01.285>be <00:18:01.518>around <00:18:01.751>90 <00:18:01.985>ish.

341
00:18:02.465 --> 00:18:03.805
<00:18:02.465>And <00:18:02.766>I <00:18:03.068>saw <00:18:03.369>that <00:18:03.671>calculation.

342
00:18:03.805 --> 00:18:06.245
<00:18:03.805>So <00:18:03.980>maybe <00:18:04.155>it's, <00:18:04.330>it's <00:18:04.505>80 <00:18:04.680>81, <00:18:04.855>I <00:18:05.030>don't <00:18:05.205>know.

343
00:18:06.395 --> 00:18:07.845
<00:18:06.395>Well <00:18:06.721>stuff <00:18:07.047>like <00:18:07.373>that. <00:18:07.700>Yeah.

344
00:18:08.945 --> 00:18:11.925
<00:18:08.945>So, <00:18:09.120>um, <00:18:09.294>I <00:18:09.470>have <00:18:09.645>to <00:18:09.820>speed <00:18:09.995>up <00:18:10.169>a <00:18:10.345>bit.

345
00:18:12.265 --> 00:18:14.325
<00:18:12.265>Um, <00:18:12.545>LMS <00:18:12.825>can <00:18:13.105>react <00:18:13.385>to <00:18:13.665>audio.

346
00:18:14.465 --> 00:18:16.885
<00:18:14.465>If <00:18:14.639>you <00:18:14.814>go <00:18:14.990>to <00:18:15.165>um, <00:18:15.340>if <00:18:15.514>you <00:18:15.689>use <00:18:15.865>Whisper

347
00:18:17.105 --> 00:18:19.445
<00:18:17.105>or <00:18:17.385>multilevel <00:18:17.665>mo <00:18:17.945>uh, <00:18:18.225>multimodal <00:18:18.505>models,

348
00:18:19.965 --> 00:18:22.245
<00:18:19.965>I <00:18:20.165>would <00:18:20.365>really <00:18:20.564>recommend <00:18:20.764>to <00:18:20.965>go <00:18:21.165>to <00:18:21.365>uh,

349
00:18:22.315 --> 00:18:26.325
<00:18:22.315>dedicated <00:18:22.515>models <00:18:22.715>like <00:18:22.915>Whisper <00:18:23.115>or <00:18:23.315>um, <00:18:23.515>other <00:18:23.715>endpoints

350
00:18:26.435 --> 00:18:28.685
<00:18:26.435>because <00:18:26.715>it's <00:18:26.995>uh, <00:18:27.274>far <00:18:27.554>more <00:18:27.835>reliable

351
00:18:28.785 --> 00:18:30.005
<00:18:28.785>and <00:18:29.059>the <00:18:29.334>results <00:18:29.608>are <00:18:29.883>better.

352
00:18:30.505 --> 00:18:33.685
<00:18:30.505>Um, <00:18:30.645>maybe <00:18:30.785>this <00:18:30.925>statement <00:18:31.065>will <00:18:31.205>get <00:18:31.345>untrue <00:18:31.485>in <00:18:31.625>the <00:18:31.765>next <00:18:31.905>year.

353
00:18:34.465 --> 00:18:37.885
<00:18:34.465>Um, <00:18:34.620>and <00:18:34.776>you <00:18:34.931>can <00:18:35.087>use <00:18:35.242>it, <00:18:35.398>um, <00:18:35.553>especially <00:18:35.709>good <00:18:35.865>well

354
00:18:37.885 --> 00:18:38.965
<00:18:37.885>with <00:18:38.209>a <00:18:38.533>command <00:18:38.857>pattern.

355
00:18:39.065 --> 00:18:42.125
<00:18:39.065>So <00:18:39.192>if <00:18:39.319>you <00:18:39.446>give <00:18:39.574>it <00:18:39.701>commands <00:18:39.828>to <00:18:39.955>interact <00:18:40.083>with <00:18:40.210>your <00:18:40.337>app, <00:18:40.465>um,

356
00:18:42.465 --> 00:18:44.365
<00:18:42.465>you <00:18:43.165>can <00:18:43.865>um,

357
00:18:44.955 --> 00:18:49.125
<00:18:44.955>just <00:18:45.095>send <00:18:45.235>all <00:18:45.375>the <00:18:45.514>commands <00:18:45.655>the <00:18:45.794>LLM <00:18:45.935>did <00:18:46.074>to <00:18:46.215>the <00:18:46.355>LLM

358
00:18:49.385 --> 00:18:52.085
<00:18:49.385>and <00:18:49.540>it <00:18:49.696>can <00:18:49.851>provide <00:18:50.007>you <00:18:50.162>with <00:18:50.318>an <00:18:50.473>easy <00:18:50.629>undo <00:18:50.785>pattern

359
00:18:52.085 --> 00:18:53.285
<00:18:52.085>because <00:18:52.355>you <00:18:52.625>just <00:18:52.895>say, <00:18:53.165>ah,

360
00:18:53.595 --> 00:18:55.365
<00:18:53.595>redo <00:18:53.795>my <00:18:53.995>last <00:18:54.195>command <00:18:54.395>or <00:18:54.595>stuff <00:18:54.795>like <00:18:54.995>that.

361
00:18:56.185 --> 00:18:58.285
<00:18:56.185>So <00:18:56.885>for <00:18:57.585>listening,

362
00:18:58.525 --> 00:19:01.645
<00:18:58.525>I <00:18:58.680>thought <00:18:58.836>the <00:18:58.991>nicest <00:18:59.147>example <00:18:59.302>would <00:18:59.458>be <00:18:59.613>actually <00:18:59.769>Lemon <00:18:59.925>Fox.

363
00:19:01.645 --> 00:19:04.285
<00:19:01.645>It's <00:19:01.772>a <00:19:01.899>shout <00:19:02.026>out <00:19:02.154>because <00:19:02.281>I <00:19:02.408>just <00:19:02.535>found <00:19:02.663>them <00:19:02.790>one <00:19:02.917>month <00:19:03.045>ago

364
00:19:04.545 --> 00:19:07.405
<00:19:04.545>and <00:19:04.700>they <00:19:04.856>have <00:19:05.011>the <00:19:05.167>face <00:19:05.322>fastest <00:19:05.478>whisper <00:19:05.633>a <00:19:05.789>PII <00:19:05.945>found

365
00:19:07.705 --> 00:19:08.725
<00:19:07.705>and <00:19:08.011>the <00:19:08.317>most <00:19:08.622>reliable.

366
00:19:09.105 --> 00:19:10.885
<00:19:09.105>And <00:19:09.305>it's <00:19:09.505>just <00:19:09.705>these <00:19:09.905>five <00:19:10.105>lines <00:19:10.305>of <00:19:10.505>code.

367
00:19:10.905 --> 00:19:13.845
<00:19:10.905>You <00:19:11.021>send <00:19:11.138>them <00:19:11.254>the <00:19:11.371>MP <00:19:11.488>three <00:19:11.605>and <00:19:11.721>you <00:19:11.838>get <00:19:11.955>back <00:19:12.071>all <00:19:12.188>the <00:19:12.305>text

368
00:19:14.345 --> 00:19:17.165
<00:19:14.345>and <00:19:14.545>it <00:19:14.745>works, <00:19:14.945>um, <00:19:15.145>especially <00:19:15.345>well <00:19:15.545>with, <00:19:15.745>um,

369
00:19:17.165 --> 00:19:18.445
<00:19:17.165>with <00:19:17.549>German <00:19:17.933>and <00:19:18.317>English.

370
00:19:18.505 --> 00:19:21.805
<00:19:18.505>So <00:19:18.645>that's <00:19:18.785>of <00:19:18.925>course <00:19:19.065>maybe <00:19:19.205>in <00:19:19.345>this <00:19:19.485>area <00:19:19.625>bit <00:19:19.765>more <00:19:19.905>important.

371
00:19:22.705 --> 00:19:26.245
<00:19:22.705>You <00:19:22.879>can <00:19:23.054>ask <00:19:23.230>it <00:19:23.405>to <00:19:23.580>just <00:19:23.754>describe <00:19:23.929>it <00:19:24.105>images,

372
00:19:26.505 --> 00:19:29.325
<00:19:26.505>of <00:19:26.645>course <00:19:26.785>you <00:19:26.925>have <00:19:27.065>to <00:19:27.205>say <00:19:27.345>like <00:19:27.485>in <00:19:27.625>what, <00:19:27.765>um, <00:19:27.905>in

373
00:19:29.325 --> 00:19:31.765
<00:19:29.325>what <00:19:29.525>circumstances <00:19:29.725>you <00:19:29.925>want <00:19:30.125>to <00:19:30.325>have <00:19:30.525>it, <00:19:30.725>like

374
00:19:31.765 --> 00:19:34.445
<00:19:31.765>what <00:19:31.940>do <00:19:32.115>you <00:19:32.290>need, <00:19:32.465>what <00:19:32.640>examples <00:19:32.815>do <00:19:32.990>you <00:19:33.165>need?

375
00:19:34.515 --> 00:19:35.645
<00:19:34.515>Like <00:19:34.684>do <00:19:34.854>you <00:19:35.023>need <00:19:35.193>it <00:19:35.362>in <00:19:35.532>Chase

376
00:19:35.645 --> 00:19:37.285
<00:19:35.645>and <00:19:35.820>do <00:19:35.995>you <00:19:36.170>need <00:19:36.345>attributes <00:19:36.520>and <00:19:36.695>stuff <00:19:36.870>like <00:19:37.045>that.

377
00:19:37.705 --> 00:19:39.885
<00:19:37.705>But <00:19:37.879>you <00:19:38.054>just <00:19:38.230>have <00:19:38.405>to <00:19:38.580>give <00:19:38.754>it <00:19:38.929>an <00:19:39.105>example

378
00:19:40.545 --> 00:19:43.685
<00:19:40.545>and <00:19:40.700>say <00:19:40.856>like, <00:19:41.011>give <00:19:41.167>me, <00:19:41.322>give <00:19:41.478>me <00:19:41.633>that <00:19:41.789>like, <00:19:41.945>um,

379
00:19:43.995 --> 00:19:47.325
<00:19:43.995>like <00:19:44.135>I <00:19:44.274>described <00:19:44.415>it <00:19:44.554>here <00:19:44.695>like <00:19:44.834>with <00:19:44.975>a <00:19:45.114>few <00:19:45.254>shot, <00:19:45.395>um,

380
00:19:47.515 --> 00:19:49.045
<00:19:47.515>example <00:19:48.892>medicine.

381
00:19:49.345 --> 00:19:52.405
<00:19:49.345>And <00:19:49.520>you <00:19:49.695>will <00:19:49.870>get <00:19:50.045>really, <00:19:50.220>really, <00:19:50.395>really <00:19:50.570>good <00:19:50.745>outputs.

382
00:19:52.475 --> 00:19:56.005
<00:19:52.475>Like <00:19:52.615>give <00:19:52.754>it <00:19:52.895>a <00:19:53.034>screenshot <00:19:53.175>of <00:19:53.314>a <00:19:53.455>pin, <00:19:53.594>um, <00:19:53.735>of, <00:19:53.875>of,

383
00:19:56.005 --> 00:20:00.205
<00:19:56.005>of <00:19:56.160>some <00:19:56.316>nodes <00:19:56.471>and <00:19:56.627>it <00:19:56.782>will <00:19:56.938>really <00:19:57.093>give <00:19:57.249>you <00:19:57.405>everything.

384
00:20:00.235 --> 00:20:02.685
<00:20:00.235>It's <00:20:00.435>the <00:20:00.635>best <00:20:00.834>OCR <00:20:01.034>um, <00:20:01.235>I've <00:20:01.435>ever <00:20:01.635>seen.

385
00:20:03.825 --> 00:20:07.525
<00:20:03.825>And <00:20:03.965>of <00:20:04.105>course <00:20:04.245>we <00:20:04.385>can <00:20:04.525>just <00:20:04.665>give <00:20:04.805>it <00:20:04.945>um, <00:20:05.085>a <00:20:05.225>reformulation.

386
00:20:07.835 --> 00:20:10.805
<00:20:07.835>Most <00:20:08.010>of <00:20:08.185>you <00:20:08.360>have <00:20:08.535>already <00:20:08.710>seen <00:20:08.885>some <00:20:09.060>re <00:20:09.235>rewrite

387
00:20:10.805 --> 00:20:12.085
<00:20:10.805>with <00:20:11.189>this <00:20:11.573>text <00:20:11.957>version

388
00:20:12.505 --> 00:20:14.845
<00:20:12.505>and <00:20:12.855>you <00:20:13.205>just, <00:20:13.555>um, <00:20:13.905>have

389
00:20:14.845 --> 00:20:17.885
<00:20:14.845>to <00:20:15.020>tell <00:20:15.195>it <00:20:15.370>like <00:20:15.545>please <00:20:15.720>respond <00:20:15.895>in <00:20:16.070>this <00:20:16.245>format.

390
00:20:18.505 --> 00:20:22.485
<00:20:18.505>So, <00:20:18.621>um, <00:20:18.738>what <00:20:18.855>some <00:20:18.971>of <00:20:19.088>you <00:20:19.205>may <00:20:19.321>be <00:20:19.438>wondering <00:20:19.555>why <00:20:19.671>I <00:20:19.788>am, <00:20:19.905>um,

391
00:20:22.915 --> 00:20:25.045
<00:20:22.915>enforcing <00:20:23.148>Jasons <00:20:23.381>in <00:20:23.615>this <00:20:23.848>in <00:20:24.081>this <00:20:24.315>way

392
00:20:25.195 --> 00:20:28.925
<00:20:25.195>because <00:20:25.350>of <00:20:25.506>course <00:20:25.661>there <00:20:25.817>is <00:20:25.972>something <00:20:26.128>called <00:20:26.283>the <00:20:26.439>Chases <00:20:26.595>mold

393
00:20:29.025 --> 00:20:30.445
<00:20:29.025>in <00:20:29.344>some <00:20:29.664>of <00:20:29.983>the <00:20:30.303>APIs.

394
00:20:30.625 --> 00:20:34.685
<00:20:30.625>But <00:20:30.765>the <00:20:30.905>thing <00:20:31.045>is <00:20:31.185>this <00:20:31.325>is <00:20:31.465>a <00:20:31.605>bit, <00:20:31.745>um, <00:20:31.885>proprietary, <00:20:32.025>um,

395
00:20:34.705 --> 00:20:37.285
<00:20:34.705>in <00:20:34.905>the <00:20:35.105>APIs <00:20:35.304>of <00:20:35.504>some <00:20:35.705>of <00:20:35.905>some <00:20:36.105>models.

396
00:20:37.825 --> 00:20:39.925
<00:20:37.825>So <00:20:38.058>you <00:20:38.291>can't <00:20:38.525>really <00:20:38.758>interchange <00:20:38.991>all <00:20:39.225>models

397
00:20:39.925 --> 00:20:41.085
<00:20:39.925>anymore <00:20:40.186>if <00:20:40.447>you <00:20:40.708>do <00:20:40.969>it.

398
00:20:41.345 --> 00:20:42.725
<00:20:41.345>If <00:20:41.522>you <00:20:41.699>just <00:20:41.877>add <00:20:42.054>it <00:20:42.232>to <00:20:42.409>the <00:20:42.587>prompt,

399
00:20:43.385 --> 00:20:45.845
<00:20:43.385>you'll <00:20:43.560>get <00:20:43.735>Jason <00:20:43.910>in <00:20:44.085>nine <00:20:44.260>out <00:20:44.435>of <00:20:44.610>10 <00:20:44.785>times

400
00:20:46.105 --> 00:20:47.245
<00:20:46.105>and <00:20:46.361>in <00:20:46.618>the <00:20:46.874>correct <00:20:47.130>format

401
00:20:47.825 --> 00:20:50.925
<00:20:47.825>and <00:20:47.980>you <00:20:48.136>can <00:20:48.291>interchange <00:20:48.447>all <00:20:48.602>modern <00:20:48.758>models <00:20:48.913>up <00:20:49.069>to <00:20:49.225>local

402
00:20:50.985 --> 00:20:53.085
<00:20:50.985>and <00:20:51.218>on-premises <00:20:51.451>models, <00:20:51.685>no <00:20:51.918>problem <00:20:52.151>at <00:20:52.385>all.

403
00:20:54.265 --> 00:20:56.845
<00:20:54.265>So <00:20:54.545>is <00:20:54.825>it <00:20:55.105>legal <00:20:55.385>to <00:20:55.665>use?

404
00:20:57.185 --> 00:21:00.805
<00:20:57.185>So <00:20:57.312>the <00:20:57.439>short <00:20:57.566>answer <00:20:57.694>is <00:20:57.821>yes, <00:20:57.948>the <00:20:58.075>lawyer <00:20:58.203>answer <00:20:58.330>is <00:20:58.457>yes, <00:20:58.585>but,

405
00:21:01.225 --> 00:21:03.045
<00:21:01.225>but <00:21:01.504>um, <00:21:01.784>to <00:21:02.064>make <00:21:02.344>it <00:21:02.625>short,

406
00:21:03.515 --> 00:21:06.765
<00:21:03.515>it's <00:21:03.670>just <00:21:03.826>like <00:21:03.981>there <00:21:04.137>is <00:21:04.292>no <00:21:04.448>law <00:21:04.603>against <00:21:04.759>using <00:21:04.915>LLM

407
00:21:06.765 --> 00:21:10.925
<00:21:06.765>or <00:21:06.881>stuff <00:21:06.998>like <00:21:07.115>that, <00:21:07.231>but <00:21:07.348>you <00:21:07.465>just <00:21:07.581>have <00:21:07.698>to <00:21:07.815>be <00:21:07.931>careful <00:21:08.048>on, <00:21:08.165>um,

408
00:21:11.235 --> 00:21:13.045
<00:21:11.235>what <00:21:11.468>you <00:21:11.701>send <00:21:11.935>out, <00:21:12.168>what <00:21:12.401>is <00:21:12.635>yours,

409
00:21:13.075 --> 00:21:15.485
<00:21:13.075>what <00:21:13.275>do <00:21:13.475>you <00:21:13.675>have <00:21:13.875>legal <00:21:14.075>requirements <00:21:14.275>with <00:21:14.475>your,

410
00:21:15.865 --> 00:21:17.365
<00:21:15.865>um, <00:21:16.315>with <00:21:16.765>your <00:21:17.215>clients.

411
00:21:17.705 --> 00:21:21.485
<00:21:17.705>But <00:21:17.832>the <00:21:17.959>point <00:21:18.086>I <00:21:18.214>wanna <00:21:18.341>make <00:21:18.468>is <00:21:18.595>like <00:21:18.723>don't <00:21:18.850>be <00:21:18.977>too <00:21:19.105>scared

412
00:21:21.485 --> 00:21:24.765
<00:21:21.485>because <00:21:21.625>if <00:21:21.764>you're <00:21:21.905>allowed <00:21:22.044>to <00:21:22.185>write <00:21:22.324>an <00:21:22.465>email <00:21:22.604>with <00:21:22.745>the <00:21:22.885>data,

413
00:21:24.855 --> 00:21:29.125
<00:21:24.855>which <00:21:24.971>is <00:21:25.088>in <00:21:25.205>oh <00:21:25.321>365, <00:21:25.438>which <00:21:25.555>is <00:21:25.671>in <00:21:25.788>Azure, <00:21:25.905>you <00:21:26.021>may <00:21:26.138>be <00:21:26.255>allowed

414
00:21:29.125 --> 00:21:31.925
<00:21:29.125>to <00:21:29.252>add <00:21:29.379>it <00:21:29.506>to <00:21:29.634>Azure <00:21:29.761>to <00:21:29.888>let <00:21:30.015>an <00:21:30.143>LLM <00:21:30.270>run <00:21:30.397>through <00:21:30.525>it.

415
00:21:32.305 --> 00:21:35.525
<00:21:32.305>If <00:21:32.432>you <00:21:32.559>have <00:21:32.686>it <00:21:32.814>in <00:21:32.941>OneNote, <00:21:33.068>it's <00:21:33.195>in <00:21:33.323>the <00:21:33.450>same <00:21:33.577>data <00:21:33.705>center.

416
00:21:36.025 --> 00:21:40.765
<00:21:36.025>So <00:21:36.132>like <00:21:36.240>there <00:21:36.348>is <00:21:36.455>some, <00:21:36.563>um, <00:21:36.671>fear <00:21:36.778>to <00:21:36.886>get <00:21:36.994>out <00:21:37.101>of <00:21:37.209>our <00:21:37.317>legal, <00:21:37.425>um,

417
00:21:41.075 --> 00:21:45.205
<00:21:41.075>departments, <00:21:41.355>but <00:21:41.635>um, <00:21:41.915>there <00:21:42.195>are <00:21:42.475>contracts

418
00:21:45.515 --> 00:21:50.365
<00:21:45.515>that <00:21:45.642>um, <00:21:45.769>do <00:21:45.896>not <00:21:46.024>allow <00:21:46.151>the <00:21:46.278>API <00:21:46.405>providers <00:21:46.533>to <00:21:46.660>use <00:21:46.787>your <00:21:46.915>data.

419
00:21:50.775 --> 00:21:53.365
<00:21:50.775>Every <00:21:51.055>paid <00:21:51.335>API <00:21:51.615>actually <00:21:51.895>does <00:21:52.175>not

420
00:21:53.415 --> 00:21:54.685
<00:21:53.415>allow <00:21:53.643>them <00:21:53.872>to <00:21:54.100>use <00:21:54.329>your <00:21:54.558>data.

421
00:21:54.835 --> 00:21:56.485
<00:21:54.835>Yeah. <00:21:55.068>So <00:21:55.301>don't <00:21:55.535>be <00:21:55.768>afraid <00:21:56.001>of <00:21:56.235>that.

422
00:21:56.905 --> 00:21:59.005
<00:21:56.905>Uh, <00:21:57.105>like <00:21:57.305>that <00:21:57.504>would <00:21:57.705>be <00:21:57.905>breaking <00:21:58.105>the <00:21:58.305>contract.

423
00:22:00.705 --> 00:22:05.005
<00:22:00.705>Um, <00:22:00.812>so <00:22:00.920>what <00:22:01.028>do <00:22:01.135>we, <00:22:01.243>what <00:22:01.351>can <00:22:01.458>we <00:22:01.566>do <00:22:01.674>to <00:22:01.781>make <00:22:01.889>it <00:22:01.997>real <00:22:02.105>illegal?

424
00:22:05.105 --> 00:22:07.565
<00:22:05.105>We <00:22:05.232>can <00:22:05.359>use <00:22:05.486>our <00:22:05.614>private <00:22:05.741>endpoints, <00:22:05.868>we <00:22:05.995>have <00:22:06.123>to <00:22:06.250>pay <00:22:06.377>for <00:22:06.505>them.

425
00:22:07.825 --> 00:22:09.245
<00:22:07.825>If <00:22:08.038>you <00:22:08.251>don't <00:22:08.464>pay, <00:22:08.677>you're <00:22:08.889>the <00:22:09.102>product.

426
00:22:09.505 --> 00:22:11.925
<00:22:09.505>The <00:22:09.738>old, <00:22:09.971>um, <00:22:10.205>the <00:22:10.438>old <00:22:10.671>saying, <00:22:10.905>um,

427
00:22:12.025 --> 00:22:14.125
<00:22:12.025>you <00:22:12.200>can <00:22:12.375>just <00:22:12.550>self <00:22:12.725>host <00:22:12.900>it <00:22:13.075>on <00:22:13.250>your <00:22:13.425>laptop,

428
00:22:14.145 --> 00:22:15.205
<00:22:14.145>on <00:22:14.463>your <00:22:14.781>phone, <00:22:15.099>whatever.

429
00:22:15.625 --> 00:22:17.845
<00:22:15.625>Um, <00:22:15.825>and <00:22:16.025>you <00:22:16.225>can <00:22:16.425>use <00:22:16.625>it <00:22:16.825>on <00:22:17.025>premises.

430
00:22:18.195 --> 00:22:20.805
<00:22:18.195>It's <00:22:18.395>getting <00:22:18.595>easier <00:22:18.794>every <00:22:18.995>day <00:22:19.195>with <00:22:19.395>all <00:22:19.595>Lama

431
00:22:20.805 --> 00:22:23.525
<00:22:20.805>and <00:22:21.038>other, <00:22:21.271>um, <00:22:21.505>node <00:22:21.738>CPP <00:22:21.971>and <00:22:22.205>stuff.

432
00:22:23.525 --> 00:22:25.485
<00:22:23.525>Like <00:22:23.805>it's <00:22:24.085>getting <00:22:24.365>easier <00:22:24.645>every <00:22:24.925>day

433
00:22:25.825 --> 00:22:27.485
<00:22:25.825>to <00:22:26.175>just <00:22:26.525>host <00:22:26.875>them <00:22:27.225>yourselves.

434
00:22:27.745 --> 00:22:30.725
<00:22:27.745>But <00:22:27.872>for <00:22:27.999>the <00:22:28.126>really <00:22:28.254>good <00:22:28.381>models <00:22:28.508>where <00:22:28.635>you <00:22:28.763>can <00:22:28.890>do <00:22:29.017>really <00:22:29.145>good,

435
00:22:30.905 --> 00:22:32.365
<00:22:30.905>um, <00:22:31.343>things <00:22:31.781>like <00:22:32.219>and,

436
00:22:32.465 --> 00:22:35.685
<00:22:32.465>and <00:22:32.665>really <00:22:32.865>solve <00:22:33.064>problems, <00:22:33.264>it's <00:22:33.465>really <00:22:33.665>better <00:22:33.865>just

437
00:22:35.985 --> 00:22:38.845
<00:22:35.985>to <00:22:36.159>use <00:22:36.334>the <00:22:36.510>API <00:22:36.685>provided <00:22:36.860>by <00:22:37.034>the <00:22:37.209>big <00:22:37.385>providers

438
00:22:39.145 --> 00:22:40.525
<00:22:39.145>and <00:22:39.393>just <00:22:39.641>use <00:22:39.890>the <00:22:40.138>big <00:22:40.387>models.

439
00:22:41.105 --> 00:22:45.365
<00:22:41.105>It <00:22:41.260>costs <00:22:41.416>sense <00:22:41.571>really <00:22:41.727>like <00:22:41.882>with <00:22:42.038>text <00:22:42.193>it's <00:22:42.349>really <00:22:42.505>hard

440
00:22:45.465 --> 00:22:48.205
<00:22:45.465>to <00:22:45.665>surpass <00:22:45.865>like <00:22:46.064>the <00:22:46.264>few <00:22:46.465>hundred <00:22:46.665>euros <00:22:46.865>of,

441
00:22:48.705 --> 00:22:50.645
<00:22:48.705>um, <00:22:49.171>of, <00:22:49.638>of <00:22:50.105>cost.

442
00:22:50.875 --> 00:22:54.125
<00:22:50.875>Yeah. <00:22:51.050>So, <00:22:51.225>um, <00:22:51.400>just <00:22:51.575>try <00:22:51.750>it <00:22:51.925>out <00:22:52.100>with <00:22:52.275>text.

443
00:22:54.225 --> 00:22:56.765
<00:22:54.225>You <00:22:54.365>can <00:22:54.504>do <00:22:54.645>a <00:22:54.784>lot <00:22:54.925>until <00:22:55.064>it <00:22:55.205>really <00:22:55.344>costs <00:22:55.485>you <00:22:55.625>money.

444
00:22:58.345 --> 00:23:02.005
<00:22:58.345>So, <00:22:58.485>um, <00:22:58.625>what <00:22:58.765>does <00:22:58.905>the <00:22:59.045>European <00:22:59.185>AI <00:22:59.325>Act <00:22:59.465>say <00:22:59.605>about <00:22:59.745>it?

445
00:23:02.345 --> 00:23:05.645
<00:23:02.345>It <00:23:02.500>says <00:23:02.656>that <00:23:02.811>AI <00:23:02.967>systems <00:23:03.122>should <00:23:03.278>be <00:23:03.433>overseen <00:23:03.589>by <00:23:03.745>people.

446
00:23:06.105 --> 00:23:08.005
<00:23:06.105>And <00:23:06.305>it's <00:23:06.505>something <00:23:06.705>that's <00:23:06.905>really <00:23:07.105>important <00:23:07.305>to <00:23:07.505>me

447
00:23:08.355 --> 00:23:12.245
<00:23:08.355>that <00:23:08.530>please, <00:23:08.705>every <00:23:08.880>one <00:23:09.055>of <00:23:09.230>you, <00:23:09.405>never, <00:23:09.580>ever <00:23:09.755>trust

448
00:23:12.305 --> 00:23:14.485
<00:23:12.305>and <00:23:12.585>LLM <00:23:12.865>trust <00:23:13.145>your <00:23:13.425>own <00:23:13.705>gut,

449
00:23:14.945 --> 00:23:18.205
<00:23:14.945>but <00:23:15.100>you <00:23:15.256>should <00:23:15.411>be <00:23:15.567>the <00:23:15.722>one <00:23:15.878>doing <00:23:16.033>like <00:23:16.189>the <00:23:16.345>fi,

450
00:23:18.205 --> 00:23:21.045
<00:23:18.205>having <00:23:18.360>the <00:23:18.516>final <00:23:18.671>saying, <00:23:18.827>or <00:23:18.982>at <00:23:19.138>least <00:23:19.293>your <00:23:19.449>end <00:23:19.605>user.

451
00:23:21.345 --> 00:23:24.885
<00:23:21.345>Please <00:23:21.500>do <00:23:21.656>not <00:23:21.811>allow <00:23:21.967>the <00:23:22.122>AI <00:23:22.278>to <00:23:22.433>directly <00:23:22.589>change <00:23:22.745>things.

452
00:23:25.425 --> 00:23:27.445
<00:23:25.425>Do <00:23:25.625>not <00:23:25.825>misuse <00:23:26.024>the <00:23:26.225>trust <00:23:26.425>of <00:23:26.625>your <00:23:26.825>users.

453
00:23:28.215 --> 00:23:29.725
<00:23:28.215>Allow <00:23:28.554>your <00:23:28.894>users <00:23:29.234>to <00:23:29.573>accept

454
00:23:29.745 --> 00:23:32.965
<00:23:29.745>or <00:23:29.900>not <00:23:30.056>accept <00:23:30.211>the <00:23:30.367>things <00:23:30.522>the <00:23:30.678>AI <00:23:30.833>wants <00:23:30.989>to <00:23:31.145>change

455
00:23:33.145 --> 00:23:34.285
<00:23:33.145>or <00:23:33.658>add, <00:23:34.171>please.

456
00:23:35.145 --> 00:23:37.845
<00:23:35.145>Um, <00:23:35.320>it's <00:23:35.495>the <00:23:35.670>law <00:23:35.845>by <00:23:36.020>the <00:23:36.195>way, <00:23:36.370>but <00:23:36.545>okay.

457
00:23:38.835 --> 00:23:40.085
<00:23:38.835>Doesn't <00:23:39.210>stop <00:23:39.585>many <00:23:39.960>people.

458
00:23:40.945 --> 00:23:44.245
<00:23:40.945>And <00:23:41.072>of <00:23:41.199>course <00:23:41.326>you <00:23:41.454>have <00:23:41.581>to <00:23:41.708>disclose, <00:23:41.835>you <00:23:41.963>may <00:23:42.090>have <00:23:42.217>heard <00:23:42.345>that

459
00:23:44.275 --> 00:23:47.005
<00:23:44.275>that <00:23:44.475>an <00:23:44.675>uh, <00:23:44.875>content <00:23:45.075>was <00:23:45.275>generated <00:23:45.475>by <00:23:45.675>AI

460
00:23:47.305 --> 00:23:50.685
<00:23:47.305>and <00:23:47.445>there <00:23:47.585>is <00:23:47.725>an <00:23:47.865>eristics, <00:23:48.005>it <00:23:48.145>does <00:23:48.285>not <00:23:48.425>apply <00:23:48.565>to <00:23:48.705>text.

461
00:23:51.065 --> 00:23:52.405
<00:23:51.065>Yes, <00:23:51.467>but <00:23:51.869>for <00:23:52.271>images

462
00:23:52.505 --> 00:23:55.965
<00:23:52.505>and <00:23:52.660>video <00:23:52.816>it's <00:23:52.971>already, <00:23:53.127>um, <00:23:53.282>like <00:23:53.438>next <00:23:53.593>month <00:23:53.749>or <00:23:53.905>something.

463
00:23:56.395 --> 00:23:58.445
<00:23:56.395>It's <00:23:56.675>um, <00:23:56.955>all <00:23:57.235>already <00:23:57.514>current <00:23:57.795>law

464
00:23:58.665 --> 00:24:01.725
<00:23:58.665>and <00:23:58.781>human <00:23:58.898>in <00:23:59.014>the <00:23:59.131>loop <00:23:59.248>is <00:23:59.365>a <00:23:59.481>very, <00:23:59.598>very <00:23:59.715>important <00:23:59.831>thing <00:23:59.948>I <00:24:00.065>want

465
00:24:01.725 --> 00:24:04.805
<00:24:01.725>to <00:24:01.899>take, <00:24:02.074>tell <00:24:02.250>you <00:24:02.425>to <00:24:02.600>take <00:24:02.774>away <00:24:02.949>with <00:24:03.125>that.

466
00:24:05.185 --> 00:24:07.485
<00:24:05.185>Yes. <00:24:05.885>So <00:24:06.585>another

467
00:24:08.045 --> 00:24:11.325
<00:24:08.045>interesting <00:24:08.220>question <00:24:08.395>is <00:24:08.570>like <00:24:08.745>you <00:24:08.920>a <00:24:09.095>attached <00:24:09.270>an <00:24:09.445>API

468
00:24:11.325 --> 00:24:13.805
<00:24:11.325>with <00:24:11.675>an <00:24:12.025>LLM <00:24:12.375>and <00:24:12.725>um,

469
00:24:13.865 --> 00:24:15.765
<00:24:13.865>who <00:24:14.065>is <00:24:14.265>now <00:24:14.465>responsible <00:24:14.665>if <00:24:14.865>something <00:24:15.065>goes <00:24:15.265>wrong

470
00:24:16.265 --> 00:24:18.725
<00:24:16.265>and <00:24:16.498>in <00:24:16.731>fact <00:24:16.965>it's <00:24:17.198>the <00:24:17.431>end <00:24:17.665>user.

471
00:24:19.545 --> 00:24:24.325
<00:24:19.545>So, <00:24:19.661>um, <00:24:19.778>it's <00:24:19.895>maybe <00:24:20.011>your <00:24:20.128>fault <00:24:20.245>if <00:24:20.361>you <00:24:20.478>don't <00:24:20.595>give <00:24:20.711>the <00:24:20.828>end <00:24:20.945>user

472
00:24:24.385 --> 00:24:26.525
<00:24:24.385>the <00:24:24.665>possibility <00:24:24.945>to <00:24:25.225>check <00:24:25.504>and <00:24:25.785>accept

473
00:24:26.585 --> 00:24:29.405
<00:24:26.585>and <00:24:26.725>be <00:24:26.865>the <00:24:27.005>human <00:24:27.145>in <00:24:27.285>the <00:24:27.425>loop <00:24:27.565>because <00:24:27.705>ah, <00:24:27.845>I <00:24:27.985>dunno.

474
00:24:29.785 --> 00:24:32.925
<00:24:29.785>Um, <00:24:29.985>but <00:24:30.185>you <00:24:30.385>have <00:24:30.585>that, <00:24:30.785>that's <00:24:30.985>another <00:24:31.185>reason

475
00:24:33.025 --> 00:24:34.245
<00:24:33.025>to <00:24:33.299>add <00:24:33.574>all <00:24:33.848>these <00:24:34.122>checks

476
00:24:34.395 --> 00:24:37.325
<00:24:34.395>because <00:24:34.595>if <00:24:34.795>they <00:24:34.995>accept <00:24:35.195>what <00:24:35.395>the <00:24:35.595>LLM <00:24:35.795>says,

477
00:24:37.435 --> 00:24:38.525
<00:24:37.435>it's <00:24:37.925>their <00:24:38.416>responsibility.

478
00:24:38.755 --> 00:24:41.085
<00:24:38.755>Yeah, <00:24:38.930>it's <00:24:39.105>not <00:24:39.280>the <00:24:39.455>fault <00:24:39.630>of <00:24:39.805>the <00:24:39.980>LLM <00:24:40.155>provider,

479
00:24:41.155 --> 00:24:43.845
<00:24:41.155>it's <00:24:41.355>not <00:24:41.555>your <00:24:41.754>fault <00:24:41.955>because <00:24:42.155>they've <00:24:42.355>seen <00:24:42.555>it.

480
00:24:44.065 --> 00:24:45.245
<00:24:44.065>And <00:24:44.277>this <00:24:44.489>is <00:24:44.702>also <00:24:44.914>a <00:24:45.127>way

481
00:24:45.265 --> 00:24:49.485
<00:24:45.265>to <00:24:45.405>make <00:24:45.545>yourself <00:24:45.685>a <00:24:45.825>bit <00:24:45.965>safe <00:24:46.105>against <00:24:46.245>like, <00:24:46.385>um, <00:24:46.525>like, <00:24:46.665>uh,

482
00:24:49.615 --> 00:24:52.525
<00:24:49.615>legal <00:24:49.815>stuff <00:24:50.015>that <00:24:50.215>could, <00:24:50.415>um, <00:24:50.615>apply <00:24:50.815>to <00:24:51.015>you.

483
00:24:53.585 --> 00:24:57.125
<00:24:53.585>So, <00:24:53.818>um, <00:24:54.051>short <00:24:54.285>side <00:24:54.518>step <00:24:54.751>to <00:24:54.985>cps,

484
00:24:57.125 --> 00:24:59.605
<00:24:57.125>like <00:24:57.591>CPS <00:24:58.058>are <00:24:58.525>wonderful.

485
00:24:59.715 --> 00:25:02.325
<00:24:59.715>Yeah, <00:24:59.995>they're, <00:25:00.274>uh, <00:25:00.554>standardized <00:25:00.834>AI <00:25:01.115>integration.

486
00:25:02.385 --> 00:25:05.525
<00:25:02.385>You <00:25:02.512>can <00:25:02.639>write <00:25:02.766>one <00:25:02.894>with <00:25:03.021>wipe <00:25:03.148>coding <00:25:03.275>in <00:25:03.403>three <00:25:03.530>minutes <00:25:03.657>I <00:25:03.785>guess

487
00:25:05.625 --> 00:25:09.725
<00:25:05.625>in <00:25:05.752>the <00:25:05.879>meantime, <00:25:06.006>and <00:25:06.134>it <00:25:06.261>can <00:25:06.388>connect <00:25:06.515>to <00:25:06.643>your <00:25:06.770>real, <00:25:06.897>um, <00:25:07.025>data

488
00:25:10.225 --> 00:25:13.485
<00:25:10.225>and <00:25:10.399>it's <00:25:10.574>very <00:25:10.750>easy <00:25:10.925>to <00:25:11.100>integrate <00:25:11.274>into <00:25:11.449>your <00:25:11.625>data,

489
00:25:13.865 --> 00:25:18.525
<00:25:13.865>but <00:25:13.992>it's <00:25:14.119>an <00:25:14.246>API <00:25:14.374>and <00:25:14.501>API <00:25:14.628>to <00:25:14.755>be <00:25:14.883>used <00:25:15.010>by <00:25:15.137>the <00:25:15.265>ai.

490
00:25:19.025 --> 00:25:23.235
<00:25:19.025>So <00:25:19.258>what <00:25:19.491>is <00:25:19.725>it <00:25:19.958>also <00:25:20.191>really <00:25:20.425>dangerous?

491
00:25:23.465 --> 00:25:28.235
<00:25:23.465>Because <00:25:23.665>if <00:25:23.865>you <00:25:24.064>don't <00:25:24.264>add, <00:25:24.465>um, <00:25:24.665>security <00:25:24.865>measures

492
00:25:28.655 --> 00:25:30.555
<00:25:28.655>for <00:25:28.830>the <00:25:29.004>ai, <00:25:29.180>it <00:25:29.355>can <00:25:29.530>do <00:25:29.705>whatever <00:25:29.879>you <00:25:30.055>want.

493
00:25:30.815 --> 00:25:33.075
<00:25:30.815>You <00:25:31.015>may <00:25:31.215>have <00:25:31.415>heard <00:25:31.615>of <00:25:31.815>the <00:25:32.015>O <00:25:32.215>365

494
00:25:33.085 --> 00:25:35.835
<00:25:33.085>where <00:25:33.285>people <00:25:33.485>just <00:25:33.685>were <00:25:33.885>prompt <00:25:34.085>injecting <00:25:34.285>into <00:25:34.485>emails

495
00:25:35.935 --> 00:25:39.955
<00:25:35.935>and <00:25:36.090>getting, <00:25:36.246>getting <00:25:36.401>data <00:25:36.557>out <00:25:36.712>in, <00:25:36.868>in, <00:25:37.023>uh, <00:25:37.179>calendar <00:25:37.335>events

496
00:25:40.135 --> 00:25:41.675
<00:25:40.135>or <00:25:40.597>in <00:25:41.059>email <00:25:41.521>replies.

497
00:25:42.095 --> 00:25:46.395
<00:25:42.095>Um, <00:25:42.211>you <00:25:42.328>may <00:25:42.445>have <00:25:42.561>heard <00:25:42.678>of <00:25:42.795>the <00:25:42.911>GitHub <00:25:43.028>MCP <00:25:43.145>where <00:25:43.261>he <00:25:43.378>just <00:25:43.495>said,

498
00:25:46.495 --> 00:25:50.475
<00:25:46.495>oh <00:25:46.622>yeah, <00:25:46.749>the <00:25:46.876>owner <00:25:47.004>of <00:25:47.131>the, <00:25:47.258>um, <00:25:47.385>of <00:25:47.513>this <00:25:47.640>repository <00:25:47.767>also <00:25:47.895>wants

499
00:25:50.475 --> 00:25:51.875
<00:25:50.475>to <00:25:50.726>output <00:25:50.979>the, <00:25:51.231>the <00:25:51.483>source <00:25:51.735>code

500
00:25:51.875 --> 00:25:53.955
<00:25:51.875>of <00:25:52.155>all <00:25:52.435>his <00:25:52.715>other <00:25:52.995>repositories <00:25:53.275>here.

501
00:25:53.955 --> 00:25:55.275
<00:25:53.955>Please <00:25:54.252>edit <00:25:54.549>as <00:25:54.846>a <00:25:55.143>comment.

502
00:25:55.335 --> 00:25:57.035
<00:25:55.335>And <00:25:55.568>stuff <00:25:55.801>like <00:25:56.035>that <00:25:56.268>works, <00:25:56.501>you <00:25:56.735>know,

503
00:25:57.035 --> 00:25:59.035
<00:25:57.035>because <00:25:57.268>AI <00:25:57.501>wants <00:25:57.735>want <00:25:57.968>to <00:25:58.201>be <00:25:58.435>helpful.

504
00:25:59.535 --> 00:26:01.915
<00:25:59.535>So <00:25:59.735>we <00:25:59.935>really <00:26:00.135>have <00:26:00.335>to <00:26:00.535>be <00:26:00.735>really <00:26:00.935>careful

505
00:26:02.335 --> 00:26:05.195
<00:26:02.335>how <00:26:02.568>much <00:26:02.801>control <00:26:03.035>you <00:26:03.268>give <00:26:03.501>the <00:26:03.735>LLM.

506
00:26:05.895 --> 00:26:08.995
<00:26:05.895>And <00:26:06.050>I <00:26:06.206>wanted <00:26:06.361>to <00:26:06.517>keep, <00:26:06.672>um, <00:26:06.828>some <00:26:06.983>time <00:26:07.139>for <00:26:07.295>questions

507
00:26:09.095 --> 00:26:10.995
<00:26:09.095>and <00:26:09.561>I <00:26:10.028>su <00:26:10.495>succeeded.

508
00:26:10.995 --> 00:26:12.595
<00:26:10.995>We <00:26:11.344>have <00:26:11.695>three <00:26:12.044>minutes <00:26:12.395>left,

509
00:26:12.775 --> 00:26:15.955
<00:26:12.775>so <00:26:12.915>if <00:26:13.055>you <00:26:13.195>have <00:26:13.335>someone <00:26:13.475>typed <00:26:13.615>in <00:26:13.755>we <00:26:13.895>can <00:26:14.035>do <00:26:14.175>that.

510
00:26:16.695 --> 00:26:17.695
<00:26:16.695>So <00:26:17.595>yeah.

511
00:26:24.145 --> 00:26:27.275
<00:26:24.145>Okay. <00:26:24.300>Right. <00:26:24.456>So <00:26:24.611>please <00:26:24.767>do <00:26:24.922>fire <00:26:25.078>questions <00:26:25.233>into <00:26:25.389>the <00:26:25.545>app.