WEBVTT 00:00:08.880 --> 00:00:11.310 align:start position:0% <00:00:08.880>All <00:00:09.080>right, <00:00:09.280>and <00:00:09.480>we <00:00:09.680>are <00:00:09.880>back <00:00:10.080>and <00:00:10.280>premiere 00:00:11.310 --> 00:00:11.320 align:start position:0% <00:00:11.310>All <00:00:11.311>right, <00:00:11.312>and <00:00:11.313>we <00:00:11.315>are <00:00:11.316>back <00:00:11.317>and <00:00:11.319>premiere 00:00:11.320 --> 00:00:13.350 align:start position:0% <00:00:11.320>All <00:00:11.420>right, <00:00:11.520>and <00:00:11.620>we <00:00:11.720>are <00:00:11.820>back <00:00:11.920>and <00:00:12.020>premiere <00:00:12.120>for <00:00:12.220>this <00:00:12.320>afternoon. <00:00:12.420>We <00:00:12.520>have <00:00:12.620>two <00:00:12.720>speakers 00:00:13.350 --> 00:00:13.360 align:start position:0% <00:00:13.350>for <00:00:13.351>this <00:00:13.353>afternoon. <00:00:13.354>We <00:00:13.356>have <00:00:13.357>two <00:00:13.359>speakers 00:00:13.360 --> 00:00:15.550 align:start position:0% <00:00:13.360>for <00:00:13.467>this <00:00:13.575>afternoon. <00:00:13.683>We <00:00:13.790>have <00:00:13.898>two <00:00:14.006>speakers <00:00:14.113>on <00:00:14.221>the <00:00:14.329>stage <00:00:14.436>so <00:00:14.544>twice <00:00:14.652>the <00:00:14.760>fun 00:00:15.550 --> 00:00:15.560 align:start position:0% <00:00:15.550>on <00:00:15.551>the <00:00:15.553>stage <00:00:15.554>so <00:00:15.556>twice <00:00:15.557>the <00:00:15.559>fun 00:00:15.560 --> 00:00:17.750 align:start position:0% <00:00:15.560>on <00:00:15.660>the <00:00:15.760>stage <00:00:15.860>so <00:00:15.960>twice <00:00:16.059>the <00:00:16.160>fun <00:00:16.260>and <00:00:16.360>probably <00:00:16.460>one <00:00:16.560>of <00:00:16.660>the <00:00:16.760>best <00:00:16.860>titles <00:00:16.960>all 00:00:17.750 --> 00:00:17.760 align:start position:0% <00:00:17.750>and <00:00:17.751>probably <00:00:17.752>one <00:00:17.753>of <00:00:17.755>the <00:00:17.756>best <00:00:17.757>titles <00:00:17.759>all 00:00:17.760 --> 00:00:20.150 align:start position:0% <00:00:17.760>and <00:00:17.853>probably <00:00:17.946>one <00:00:18.040>of <00:00:18.133>the <00:00:18.226>best <00:00:18.320>titles <00:00:18.413>all <00:00:18.506>day <00:00:18.600>LLM's <00:00:18.693>in <00:00:18.786>the <00:00:18.880>wild <00:00:18.973>building <00:00:19.066>an <00:00:19.160>AI 00:00:20.150 --> 00:00:20.160 align:start position:0% <00:00:20.150>day <00:00:20.151>LLM's <00:00:20.152>in <00:00:20.153>the <00:00:20.155>wild <00:00:20.156>building <00:00:20.157>an <00:00:20.159>AI 00:00:20.160 --> 00:00:22.350 align:start position:0% <00:00:20.160>day <00:00:20.276>LLM's <00:00:20.393>in <00:00:20.510>the <00:00:20.626>wild <00:00:20.743>building <00:00:20.860>an <00:00:20.976>AI <00:00:21.093>trend <00:00:21.210>that <00:00:21.326>survives <00:00:21.443>production. <00:00:21.560>We're 00:00:22.350 --> 00:00:22.360 align:start position:0% <00:00:22.350>trend <00:00:22.352>that <00:00:22.354>survives <00:00:22.356>production. <00:00:22.359>We're 00:00:22.360 --> 00:00:23.910 align:start position:0% <00:00:22.360>trend <00:00:22.499>that <00:00:22.639>survives <00:00:22.778>production. <00:00:22.918>We're <00:00:23.057>looking <00:00:23.197>forward <00:00:23.336>to <00:00:23.476>that. <00:00:23.615>So <00:00:23.755>please 00:00:23.910 --> 00:00:23.920 align:start position:0% <00:00:23.910>looking <00:00:23.911>forward <00:00:23.913>to <00:00:23.915>that. <00:00:23.917>So <00:00:23.919>please 00:00:23.920 --> 00:00:25.870 align:start position:0% <00:00:23.920>looking <00:00:24.036>forward <00:00:24.153>to <00:00:24.270>that. <00:00:24.386>So <00:00:24.503>please <00:00:24.620>welcome <00:00:24.736>to <00:00:24.853>the <00:00:24.970>stage <00:00:25.086>Steven <00:00:25.203>and <00:00:25.320>John 00:00:25.870 --> 00:00:25.880 align:start position:0% <00:00:25.870>welcome <00:00:25.871>to <00:00:25.873>the <00:00:25.874>stage <00:00:25.876>Steven <00:00:25.877>and <00:00:25.878>John 00:00:25.880 --> 00:00:27.870 align:start position:0% <00:00:25.880>welcome <00:00:26.035>to <00:00:26.191>the <00:00:26.346>stage <00:00:26.502>Steven <00:00:26.657>and <00:00:26.813>John <00:00:26.968>Paulo. <00:00:27.124>Welcome <00:00:27.279>guys. 00:00:27.870 --> 00:00:27.880 align:start position:0% <00:00:27.870>Paulo. <00:00:27.874>Welcome <00:00:27.878>guys. 00:00:27.880 --> 00:00:35.590 align:start position:0% <00:00:27.880>Paulo. <00:00:28.346>Welcome <00:00:28.813>guys. <00:00:29.279>Please. 00:00:35.600 --> 00:00:36.790 align:start position:0% <00:00:35.600>>> <00:00:36.135>Hello <00:00:36.671>everyone. 00:00:36.790 --> 00:00:36.800 align:start position:0% <00:00:36.790>>> <00:00:36.794>Hello <00:00:36.799>everyone. 00:00:36.800 --> 00:00:39.070 align:start position:0% <00:00:36.800>>> <00:00:37.000>Hello <00:00:37.199>everyone. <00:00:37.400>So <00:00:37.599>quick <00:00:37.800>show <00:00:38.000>of <00:00:38.199>hands. 00:00:39.070 --> 00:00:39.080 align:start position:0% <00:00:39.070>So <00:00:39.072>quick <00:00:39.074>show <00:00:39.076>of <00:00:39.079>hands. 00:00:39.080 --> 00:00:42.030 align:start position:0% <00:00:39.080>So <00:00:39.207>quick <00:00:39.334>show <00:00:39.461>of <00:00:39.589>hands. <00:00:39.716>Who <00:00:39.843>here <00:00:39.970>has <00:00:40.098>shipped <00:00:40.225>a <00:00:40.352>product <00:00:40.480>in 00:00:42.030 --> 00:00:42.040 align:start position:0% <00:00:42.030>Who <00:00:42.031>here <00:00:42.033>has <00:00:42.034>shipped <00:00:42.036>a <00:00:42.037>product <00:00:42.039>in 00:00:42.040 --> 00:00:46.910 align:start position:0% <00:00:42.040>Who <00:00:42.167>here <00:00:42.294>has <00:00:42.421>shipped <00:00:42.549>a <00:00:42.676>product <00:00:42.803>in <00:00:42.930>production <00:00:43.058>with <00:00:43.185>LLM <00:00:43.312>in <00:00:43.440>it? 00:00:46.920 --> 00:00:48.750 align:start position:0% Okay. 00:00:48.750 --> 00:00:48.760 align:start position:0% Okay. 00:00:48.760 --> 00:00:52.510 align:start position:0% <00:00:48.760>Okay. <00:00:48.934>Keep <00:00:49.110>your <00:00:49.285>hands <00:00:49.460>up <00:00:49.635>if <00:00:49.809>the <00:00:49.985>model <00:00:50.160>behaved 00:00:52.510 --> 00:00:52.520 align:start position:0% <00:00:52.510>Keep <00:00:52.511>your <00:00:52.512>hands <00:00:52.513>up <00:00:52.515>if <00:00:52.516>the <00:00:52.517>model <00:00:52.519>behaved 00:00:52.520 --> 00:00:55.510 align:start position:0% <00:00:52.520>Keep <00:00:52.613>your <00:00:52.706>hands <00:00:52.800>up <00:00:52.893>if <00:00:52.986>the <00:00:53.080>model <00:00:53.173>behaved <00:00:53.266>in <00:00:53.360>some <00:00:53.453>strange <00:00:53.546>ways <00:00:53.640>compared <00:00:53.733>to <00:00:53.826>when <00:00:53.920>it 00:00:55.510 --> 00:00:55.520 align:start position:0% <00:00:55.510>in <00:00:55.511>some <00:00:55.512>strange <00:00:55.513>ways <00:00:55.515>compared <00:00:55.516>to <00:00:55.517>when <00:00:55.519>it 00:00:55.520 --> 00:00:59.230 align:start position:0% <00:00:55.520>in <00:00:55.636>some <00:00:55.753>strange <00:00:55.870>ways <00:00:55.986>compared <00:00:56.103>to <00:00:56.220>when <00:00:56.336>it <00:00:56.453>was <00:00:56.570>working <00:00:56.686>in <00:00:56.803>development <00:00:56.920>phase. 00:00:59.230 --> 00:00:59.240 align:start position:0% <00:00:59.230>was <00:00:59.232>working <00:00:59.234>in <00:00:59.236>development <00:00:59.239>phase. 00:00:59.240 --> 00:01:00.870 align:start position:0% <00:00:59.240>was <00:00:59.473>working <00:00:59.706>in <00:00:59.940>development <00:01:00.173>phase. <00:01:00.406>Okay, <00:01:00.640>nice. 00:01:00.870 --> 00:01:00.880 align:start position:0% <00:01:00.870>Okay, <00:01:00.879>nice. 00:01:00.880 --> 00:01:02.790 align:start position:0% <00:01:00.880>Okay, <00:01:01.020>nice. <00:01:01.160>So <00:01:01.300>that's <00:01:01.440>what <00:01:01.580>we <00:01:01.720>want <00:01:01.860>to <00:01:02.000>focus <00:01:02.140>on <00:01:02.280>that 00:01:02.790 --> 00:01:02.800 align:start position:0% <00:01:02.790>So <00:01:02.791>that's <00:01:02.792>what <00:01:02.793>we <00:01:02.794>want <00:01:02.795>to <00:01:02.796>focus <00:01:02.797>on <00:01:02.799>that 00:01:02.800 --> 00:01:05.710 align:start position:0% <00:01:02.800>So <00:01:02.887>that's <00:01:02.974>what <00:01:03.062>we <00:01:03.150>want <00:01:03.237>to <00:01:03.324>focus <00:01:03.412>on <00:01:03.500>that <00:01:03.587>gap <00:01:03.675>between <00:01:03.762>it <00:01:03.849>was <00:01:03.937>working <00:01:04.024>in <00:01:04.112>my <00:01:04.200>prompt 00:01:05.710 --> 00:01:05.720 align:start position:0% <00:01:05.710>gap <00:01:05.711>between <00:01:05.712>it <00:01:05.713>was <00:01:05.715>working <00:01:05.716>in <00:01:05.717>my <00:01:05.719>prompt 00:01:05.720 --> 00:01:07.870 align:start position:0% <00:01:05.720>gap <00:01:05.827>between <00:01:05.935>it <00:01:06.043>was <00:01:06.150>working <00:01:06.258>in <00:01:06.366>my <00:01:06.473>prompt <00:01:06.581>and <00:01:06.689>it's <00:01:06.796>working <00:01:06.904>and <00:01:07.012>surviving <00:01:07.120>in 00:01:07.870 --> 00:01:07.880 align:start position:0% <00:01:07.870>and <00:01:07.871>it's <00:01:07.873>working <00:01:07.875>and <00:01:07.877>surviving <00:01:07.878>in 00:01:07.880 --> 00:01:10.830 align:start position:0% <00:01:07.880>and <00:01:07.987>it's <00:01:08.095>working <00:01:08.203>and <00:01:08.310>surviving <00:01:08.418>in <00:01:08.526>production. <00:01:08.633>So <00:01:08.741>this <00:01:08.849>will <00:01:08.956>be <00:01:09.064>a <00:01:09.172>story <00:01:09.280>of 00:01:10.830 --> 00:01:10.840 align:start position:0% <00:01:10.830>production. <00:01:10.831>So <00:01:10.832>this <00:01:10.833>will <00:01:10.835>be <00:01:10.836>a <00:01:10.837>story <00:01:10.839>of 00:01:10.840 --> 00:01:13.190 align:start position:0% <00:01:10.840>production. <00:01:10.933>So <00:01:11.026>this <00:01:11.120>will <00:01:11.213>be <00:01:11.306>a <00:01:11.400>story <00:01:11.493>of <00:01:11.586>how <00:01:11.680>we <00:01:11.773>build <00:01:11.866>our <00:01:11.960>AI <00:01:12.053>discovery <00:01:12.146>agent <00:01:12.240>at 00:01:13.190 --> 00:01:13.200 align:start position:0% <00:01:13.190>how <00:01:13.191>we <00:01:13.192>build <00:01:13.193>our <00:01:13.195>AI <00:01:13.196>discovery <00:01:13.197>agent <00:01:13.199>at 00:01:13.200 --> 00:01:16.670 align:start position:0% <00:01:13.200>how <00:01:13.375>we <00:01:13.550>build <00:01:13.725>our <00:01:13.900>AI <00:01:14.075>discovery <00:01:14.250>agent <00:01:14.425>at <00:01:14.600>GetYourGuide. 00:01:16.680 --> 00:01:18.830 align:start position:0% <00:01:16.680>Uh <00:01:16.855>here <00:01:17.030>is <00:01:17.205>the <00:01:17.380>plan <00:01:17.555>for <00:01:17.730>the <00:01:17.905>next <00:01:18.080>25 00:01:18.830 --> 00:01:18.840 align:start position:0% <00:01:18.830>Uh <00:01:18.831>here <00:01:18.832>is <00:01:18.833>the <00:01:18.834>plan <00:01:18.835>for <00:01:18.836>the <00:01:18.837>next <00:01:18.839>25 00:01:18.840 --> 00:01:21.230 align:start position:0% <00:01:18.840>Uh <00:01:18.927>here <00:01:19.015>is <00:01:19.102>the <00:01:19.190>plan <00:01:19.277>for <00:01:19.365>the <00:01:19.452>next <00:01:19.540>25 <00:01:19.627>minutes. <00:01:19.715>Uh <00:01:19.802>we <00:01:19.890>will <00:01:19.977>go <00:01:20.065>through <00:01:20.152>uh <00:01:20.240>talk 00:01:21.230 --> 00:01:21.240 align:start position:0% <00:01:21.230>minutes. <00:01:21.231>Uh <00:01:21.232>we <00:01:21.233>will <00:01:21.235>go <00:01:21.236>through <00:01:21.237>uh <00:01:21.238>talk 00:01:21.240 --> 00:01:23.750 align:start position:0% <00:01:21.240>minutes. <00:01:21.347>Uh <00:01:21.455>we <00:01:21.563>will <00:01:21.670>go <00:01:21.778>through <00:01:21.886>uh <00:01:21.993>talk <00:01:22.101>about <00:01:22.209>the <00:01:22.316>problem, <00:01:22.424>why <00:01:22.532>the <00:01:22.640>classic 00:01:23.750 --> 00:01:23.760 align:start position:0% <00:01:23.750>about <00:01:23.751>the <00:01:23.753>problem, <00:01:23.755>why <00:01:23.757>the <00:01:23.759>classic 00:01:23.760 --> 00:01:26.310 align:start position:0% <00:01:23.760>about <00:01:23.867>the <00:01:23.975>problem, <00:01:24.083>why <00:01:24.190>the <00:01:24.298>classic <00:01:24.406>search <00:01:24.513>was <00:01:24.621>not <00:01:24.729>working <00:01:24.836>well <00:01:24.944>for <00:01:25.052>us <00:01:25.160>and 00:01:26.310 --> 00:01:26.320 align:start position:0% <00:01:26.310>search <00:01:26.311>was <00:01:26.312>not <00:01:26.313>working <00:01:26.315>well <00:01:26.316>for <00:01:26.317>us <00:01:26.318>and 00:01:26.320 --> 00:01:28.270 align:start position:0% <00:01:26.320>search <00:01:26.419>was <00:01:26.520>not <00:01:26.619>working <00:01:26.720>well <00:01:26.820>for <00:01:26.919>us <00:01:27.020>and <00:01:27.119>was <00:01:27.220>not <00:01:27.320>suiting <00:01:27.419>our <00:01:27.520>case, <00:01:27.619>then <00:01:27.720>the 00:01:28.270 --> 00:01:28.280 align:start position:0% <00:01:28.270>was <00:01:28.271>not <00:01:28.273>suiting <00:01:28.274>our <00:01:28.276>case, <00:01:28.277>then <00:01:28.279>the 00:01:28.280 --> 00:01:30.790 align:start position:0% <00:01:28.280>was <00:01:28.387>not <00:01:28.495>suiting <00:01:28.603>our <00:01:28.710>case, <00:01:28.818>then <00:01:28.926>the <00:01:29.033>architecture <00:01:29.141>of <00:01:29.249>our <00:01:29.356>agent, <00:01:29.464>and <00:01:29.572>then <00:01:29.680>the 00:01:30.790 --> 00:01:30.800 align:start position:0% <00:01:30.789>architecture <00:01:30.791>of <00:01:30.792>our <00:01:30.794>agent, <00:01:30.795>and <00:01:30.797>then <00:01:30.798>the 00:01:30.800 --> 00:01:32.950 align:start position:0% <00:01:30.800>architecture <00:01:30.916>of <00:01:31.033>our <00:01:31.149>agent, <00:01:31.266>and <00:01:31.383>then <00:01:31.500>the <00:01:31.616>evaluation <00:01:31.733>framework <00:01:31.850>that <00:01:31.966>let <00:01:32.083>us <00:01:32.200>do 00:01:32.950 --> 00:01:32.960 align:start position:0% <00:01:32.950>evaluation <00:01:32.951>framework <00:01:32.953>that <00:01:32.955>let <00:01:32.957>us <00:01:32.959>do 00:01:32.960 --> 00:01:35.830 align:start position:0% <00:01:32.960>evaluation <00:01:33.076>framework <00:01:33.193>that <00:01:33.310>let <00:01:33.426>us <00:01:33.543>do <00:01:33.660>changes <00:01:33.776>and <00:01:33.893>build <00:01:34.010>on <00:01:34.126>it <00:01:34.243>without <00:01:34.360>having 00:01:35.830 --> 00:01:35.840 align:start position:0% <00:01:35.830>changes <00:01:35.831>and <00:01:35.833>build <00:01:35.834>on <00:01:35.836>it <00:01:35.837>without <00:01:35.839>having 00:01:35.840 --> 00:01:38.350 align:start position:0% <00:01:35.840>changes <00:01:35.940>and <00:01:36.040>build <00:01:36.140>on <00:01:36.240>it <00:01:36.340>without <00:01:36.440>having <00:01:36.540>problems. <00:01:36.640>And <00:01:36.740>then <00:01:36.840>we <00:01:36.940>will <00:01:37.040>go <00:01:37.140>with <00:01:37.240>some 00:01:38.350 --> 00:01:38.360 align:start position:0% <00:01:38.350>problems. <00:01:38.351>And <00:01:38.352>then <00:01:38.353>we <00:01:38.355>will <00:01:38.356>go <00:01:38.357>with <00:01:38.359>some 00:01:38.360 --> 00:01:42.150 align:start position:0% <00:01:38.360>problems. <00:01:38.476>And <00:01:38.593>then <00:01:38.710>we <00:01:38.826>will <00:01:38.943>go <00:01:39.060>with <00:01:39.176>some <00:01:39.293>uh <00:01:39.410>conclusion <00:01:39.526>and <00:01:39.643>key <00:01:39.760>takeaways. 00:01:42.160 --> 00:01:45.310 align:start position:0% <00:01:42.160>So <00:01:42.393>quick <00:01:42.626>intros. <00:01:42.860>I'm <00:01:43.093>John <00:01:43.326>Paulo, <00:01:43.560>senior 00:01:45.310 --> 00:01:45.320 align:start position:0% <00:01:45.310>So <00:01:45.311>quick <00:01:45.313>intros. <00:01:45.314>I'm <00:01:45.316>John <00:01:45.317>Paulo, <00:01:45.318>senior 00:01:45.320 --> 00:01:47.270 align:start position:0% <00:01:45.320>So <00:01:45.460>quick <00:01:45.600>intros. <00:01:45.740>I'm <00:01:45.880>John <00:01:46.020>Paulo, <00:01:46.160>senior <00:01:46.300>data <00:01:46.440>scientist <00:01:46.580>at <00:01:46.720>GetYourGuide. 00:01:47.270 --> 00:01:47.280 align:start position:0% <00:01:47.270>data <00:01:47.273>scientist <00:01:47.276>at <00:01:47.279>GetYourGuide. 00:01:47.280 --> 00:01:49.750 align:start position:0% <00:01:47.280>data <00:01:47.396>scientist <00:01:47.513>at <00:01:47.630>GetYourGuide. <00:01:47.746>Uh <00:01:47.863>over <00:01:47.980>the <00:01:48.096>last <00:01:48.213>3 <00:01:48.330>years <00:01:48.446>we <00:01:48.563>shipped <00:01:48.680>many 00:01:49.750 --> 00:01:49.760 align:start position:0% <00:01:49.750>Uh <00:01:49.751>over <00:01:49.752>the <00:01:49.753>last <00:01:49.754>3 <00:01:49.755>years <00:01:49.756>we <00:01:49.757>shipped <00:01:49.758>many 00:01:49.760 --> 00:01:51.590 align:start position:0% <00:01:49.759>Uh <00:01:49.859>over <00:01:49.960>the <00:01:50.059>last <00:01:50.160>3 <00:01:50.259>years <00:01:50.359>we <00:01:50.460>shipped <00:01:50.559>many <00:01:50.660>AI <00:01:50.759>solutions <00:01:50.859>at <00:01:50.960>GetYourGuide <00:01:51.059>over <00:01:51.160>our 00:01:51.590 --> 00:01:51.600 align:start position:0% <00:01:51.590>AI <00:01:51.591>solutions <00:01:51.593>at <00:01:51.595>GetYourGuide <00:01:51.597>over <00:01:51.598>our 00:01:51.600 --> 00:01:54.230 align:start position:0% <00:01:51.600>AI <00:01:51.740>solutions <00:01:51.880>at <00:01:52.020>GetYourGuide <00:01:52.160>over <00:01:52.300>our <00:01:52.440>product, <00:01:52.580>AI <00:01:52.720>itinerary <00:01:52.860>builder, <00:01:53.000>AI 00:01:54.230 --> 00:01:54.240 align:start position:0% <00:01:54.229>product, <00:01:54.232>AI <00:01:54.234>itinerary <00:01:54.236>builder, <00:01:54.239>AI 00:01:54.240 --> 00:01:56.910 align:start position:0% <00:01:54.240>product, <00:01:54.415>AI <00:01:54.590>itinerary <00:01:54.765>builder, <00:01:54.940>AI <00:01:55.115>classification, <00:01:55.290>smart <00:01:55.465>filters, <00:01:55.640>rack 00:01:56.910 --> 00:01:56.920 align:start position:0% <00:01:56.910>classification, <00:01:56.913>smart <00:01:56.916>filters, <00:01:56.919>rack 00:01:56.920 --> 00:01:59.270 align:start position:0% <00:01:56.920>classification, <00:01:57.120>smart <00:01:57.320>filters, <00:01:57.520>rack <00:01:57.720>system, <00:01:57.920>and <00:01:58.120>so <00:01:58.320>on. 00:01:59.270 --> 00:01:59.280 align:start position:0% <00:01:59.270>system, <00:01:59.273>and <00:01:59.276>so <00:01:59.279>on. 00:01:59.280 --> 00:02:01.870 align:start position:0% <00:01:59.280>system, <00:01:59.407>and <00:01:59.534>so <00:01:59.661>on. <00:01:59.789>Before <00:01:59.916>that, <00:02:00.043>I <00:02:00.170>worked <00:02:00.298>at <00:02:00.425>Amazon <00:02:00.552>as <00:02:00.680>a 00:02:01.870 --> 00:02:01.880 align:start position:0% <00:02:01.870>Before <00:02:01.871>that, <00:02:01.872>I <00:02:01.873>worked <00:02:01.875>at <00:02:01.876>Amazon <00:02:01.877>as <00:02:01.878>a 00:02:01.880 --> 00:02:04.190 align:start position:0% <00:02:01.880>Before <00:02:02.035>that, <00:02:02.191>I <00:02:02.346>worked <00:02:02.502>at <00:02:02.657>Amazon <00:02:02.813>as <00:02:02.968>a <00:02:03.124>production <00:02:03.280>ML. 00:02:04.190 --> 00:02:04.200 align:start position:0% <00:02:04.190>production <00:02:04.199>ML. 00:02:04.200 --> 00:02:06.710 align:start position:0% <00:02:04.200>production <00:02:04.375>ML. <00:02:04.550>>> <00:02:04.725>Hi <00:02:04.900>everybody. <00:02:05.075>I'm <00:02:05.250>Steven, <00:02:05.425>senior <00:02:05.600>ML 00:02:06.710 --> 00:02:06.720 align:start position:0% <00:02:06.710>>> <00:02:06.711>Hi <00:02:06.713>everybody. <00:02:06.714>I'm <00:02:06.716>Steven, <00:02:06.717>senior <00:02:06.719>ML 00:02:06.720 --> 00:02:09.109 align:start position:0% <00:02:06.720>>> <00:02:06.836>Hi <00:02:06.953>everybody. <00:02:07.070>I'm <00:02:07.186>Steven, <00:02:07.303>senior <00:02:07.420>ML <00:02:07.536>engineer <00:02:07.653>at <00:02:07.770>GetYourGuide. <00:02:07.886>Over <00:02:08.003>the <00:02:08.120>last 00:02:09.109 --> 00:02:09.119 align:start position:0% <00:02:09.109>engineer <00:02:09.110>at <00:02:09.112>GetYourGuide. <00:02:09.114>Over <00:02:09.116>the <00:02:09.118>last 00:02:09.119 --> 00:02:10.430 align:start position:0% <00:02:09.119>engineer <00:02:09.226>at <00:02:09.333>GetYourGuide. <00:02:09.440>Over <00:02:09.548>the <00:02:09.655>last <00:02:09.762>few <00:02:09.869>years <00:02:09.977>at <00:02:10.084>GetYourGuide, <00:02:10.191>I've <00:02:10.298>spent 00:02:10.430 --> 00:02:10.440 align:start position:0% <00:02:10.430>few <00:02:10.431>years <00:02:10.433>at <00:02:10.435>GetYourGuide, <00:02:10.437>I've <00:02:10.439>spent 00:02:10.440 --> 00:02:13.190 align:start position:0% <00:02:10.440>few <00:02:10.556>years <00:02:10.673>at <00:02:10.789>GetYourGuide, <00:02:10.906>I've <00:02:11.023>spent <00:02:11.140>building <00:02:11.256>most <00:02:11.373>of <00:02:11.490>our <00:02:11.606>ML <00:02:11.723>systems <00:02:11.840>within 00:02:13.190 --> 00:02:13.200 align:start position:0% <00:02:13.190>building <00:02:13.191>most <00:02:13.192>of <00:02:13.194>our <00:02:13.196>ML <00:02:13.197>systems <00:02:13.198>within 00:02:13.200 --> 00:02:15.070 align:start position:0% <00:02:13.200>building <00:02:13.339>most <00:02:13.480>of <00:02:13.619>our <00:02:13.760>ML <00:02:13.899>systems <00:02:14.040>within <00:02:14.179>our <00:02:14.320>customer <00:02:14.459>acquisition, <00:02:14.600>customer 00:02:15.070 --> 00:02:15.080 align:start position:0% <00:02:15.070>our <00:02:15.073>customer <00:02:15.076>acquisition, <00:02:15.079>customer 00:02:15.080 --> 00:02:17.430 align:start position:0% <00:02:15.080>our <00:02:15.280>customer <00:02:15.480>acquisition, <00:02:15.680>customer <00:02:15.880>retention, <00:02:16.080>and <00:02:16.280>supplier <00:02:16.480>domain. 00:02:17.430 --> 00:02:17.440 align:start position:0% <00:02:17.430>retention, <00:02:17.433>and <00:02:17.436>supplier <00:02:17.439>domain. 00:02:17.440 --> 00:02:18.910 align:start position:0% <00:02:17.440>retention, <00:02:17.660>and <00:02:17.881>supplier <00:02:18.101>domain. <00:02:18.322>In <00:02:18.542>the <00:02:18.763>last 00:02:18.910 --> 00:02:18.920 align:start position:0% <00:02:18.910>In <00:02:18.914>the <00:02:18.919>last 00:02:18.920 --> 00:02:20.230 align:start position:0% <00:02:18.920>In <00:02:19.116>the <00:02:19.313>last <00:02:19.509>yeah, <00:02:19.706>year, <00:02:19.902>I've <00:02:20.099>been 00:02:20.230 --> 00:02:20.240 align:start position:0% <00:02:20.230>yeah, <00:02:20.233>year, <00:02:20.236>I've <00:02:20.239>been 00:02:20.240 --> 00:02:21.790 align:start position:0% <00:02:20.240>yeah, <00:02:20.395>year, <00:02:20.550>I've <00:02:20.705>been <00:02:20.860>mostly <00:02:21.015>focusing <00:02:21.170>on <00:02:21.325>building <00:02:21.480>the <00:02:21.635>app 00:02:21.790 --> 00:02:21.800 align:start position:0% <00:02:21.790>mostly <00:02:21.791>focusing <00:02:21.793>on <00:02:21.795>building <00:02:21.797>the <00:02:21.799>app 00:02:21.800 --> 00:02:24.270 align:start position:0% <00:02:21.800>mostly <00:02:21.927>focusing <00:02:22.054>on <00:02:22.181>building <00:02:22.309>the <00:02:22.436>app <00:02:22.563>platform <00:02:22.690>that <00:02:22.818>helps <00:02:22.945>and <00:02:23.072>enables <00:02:23.200>our 00:02:24.270 --> 00:02:24.280 align:start position:0% <00:02:24.270>platform <00:02:24.271>that <00:02:24.273>helps <00:02:24.275>and <00:02:24.277>enables <00:02:24.279>our 00:02:24.280 --> 00:02:26.070 align:start position:0% <00:02:24.280>platform <00:02:24.420>that <00:02:24.560>helps <00:02:24.700>and <00:02:24.840>enables <00:02:24.980>our <00:02:25.120>GetYourGuide <00:02:25.260>tech <00:02:25.400>talk <00:02:25.540>to <00:02:25.680>ship 00:02:26.070 --> 00:02:26.080 align:start position:0% <00:02:26.070>GetYourGuide <00:02:26.072>tech <00:02:26.074>talk <00:02:26.076>to <00:02:26.078>ship 00:02:26.080 --> 00:02:28.910 align:start position:0% <00:02:26.079>GetYourGuide <00:02:26.235>tech <00:02:26.391>talk <00:02:26.546>to <00:02:26.702>ship <00:02:26.857>high-quality <00:02:27.013>AI <00:02:27.168>features <00:02:27.324>in <00:02:27.480>production. 00:02:28.910 --> 00:02:28.920 align:start position:0% <00:02:28.910>high-quality <00:02:28.912>AI <00:02:28.914>features <00:02:28.916>in <00:02:28.919>production. 00:02:28.920 --> 00:02:31.230 align:start position:0% <00:02:28.920>high-quality <00:02:29.036>AI <00:02:29.153>features <00:02:29.270>in <00:02:29.386>production. <00:02:29.503>>> <00:02:29.620>Yeah, <00:02:29.736>for <00:02:29.853>this <00:02:29.970>project, <00:02:30.086>I <00:02:30.203>focused <00:02:30.320>mainly 00:02:31.230 --> 00:02:31.240 align:start position:0% <00:02:31.230>>> <00:02:31.231>Yeah, <00:02:31.232>for <00:02:31.233>this <00:02:31.235>project, <00:02:31.236>I <00:02:31.237>focused <00:02:31.239>mainly 00:02:31.240 --> 00:02:32.790 align:start position:0% <00:02:31.240>>> <00:02:31.339>Yeah, <00:02:31.439>for <00:02:31.538>this <00:02:31.638>project, <00:02:31.738>I <00:02:31.837>focused <00:02:31.937>mainly <00:02:32.037>that <00:02:32.136>the <00:02:32.236>agent <00:02:32.336>is <00:02:32.435>giving <00:02:32.535>the <00:02:32.635>right 00:02:32.790 --> 00:02:32.800 align:start position:0% <00:02:32.790>that <00:02:32.791>the <00:02:32.793>agent <00:02:32.794>is <00:02:32.796>giving <00:02:32.797>the <00:02:32.799>right 00:02:32.800 --> 00:02:35.110 align:start position:0% <00:02:32.800>that <00:02:32.916>the <00:02:33.033>agent <00:02:33.150>is <00:02:33.266>giving <00:02:33.383>the <00:02:33.500>right <00:02:33.616>answers <00:02:33.733>and <00:02:33.850>find <00:02:33.966>the <00:02:34.083>right <00:02:34.200>activities. 00:02:35.110 --> 00:02:35.120 align:start position:0% <00:02:35.110>answers <00:02:35.111>and <00:02:35.113>find <00:02:35.115>the <00:02:35.117>right <00:02:35.119>activities. 00:02:35.120 --> 00:02:36.830 align:start position:0% <00:02:35.120>answers <00:02:35.220>and <00:02:35.320>find <00:02:35.420>the <00:02:35.520>right <00:02:35.620>activities. <00:02:35.720>>> <00:02:35.820>And <00:02:35.920>I <00:02:36.020>try <00:02:36.120>to <00:02:36.220>make <00:02:36.320>sure <00:02:36.420>that <00:02:36.520>whatever 00:02:36.830 --> 00:02:36.840 align:start position:0% <00:02:36.829>>> <00:02:36.831>And <00:02:36.832>I <00:02:36.833>try <00:02:36.834>to <00:02:36.835>make <00:02:36.836>sure <00:02:36.837>that <00:02:36.839>whatever 00:02:36.840 --> 00:02:39.030 align:start position:0% <00:02:36.840>>> <00:02:36.940>And <00:02:37.040>I <00:02:37.140>try <00:02:37.240>to <00:02:37.340>make <00:02:37.440>sure <00:02:37.540>that <00:02:37.640>whatever <00:02:37.740>Gianpaolo <00:02:37.840>wants <00:02:37.940>works <00:02:38.040>and <00:02:38.140>doesn't <00:02:38.240>wake 00:02:39.030 --> 00:02:39.040 align:start position:0% <00:02:39.030>Gianpaolo <00:02:39.031>wants <00:02:39.033>works <00:02:39.035>and <00:02:39.037>doesn't <00:02:39.039>wake 00:02:39.040 --> 00:02:40.750 align:start position:0% <00:02:39.040>Gianpaolo <00:02:39.147>wants <00:02:39.255>works <00:02:39.363>and <00:02:39.470>doesn't <00:02:39.578>wake <00:02:39.686>us <00:02:39.793>up <00:02:39.901>at <00:02:40.009>3:00 <00:02:40.116>a.m. <00:02:40.224>in <00:02:40.332>the <00:02:40.440>morning. 00:02:40.750 --> 00:02:40.760 align:start position:0% <00:02:40.750>us <00:02:40.751>up <00:02:40.752>at <00:02:40.753>3:00 <00:02:40.755>a.m. <00:02:40.756>in <00:02:40.757>the <00:02:40.759>morning. 00:02:40.760 --> 00:02:42.990 align:start position:0% <00:02:40.760>us <00:02:40.859>up <00:02:40.959>at <00:02:41.060>3:00 <00:02:41.160>a.m. <00:02:41.260>in <00:02:41.359>the <00:02:41.459>morning. <00:02:41.560>>> <00:02:41.660>So, <00:02:41.760>we <00:02:41.859>naturally <00:02:41.959>disagree <00:02:42.060>a <00:02:42.160>lot. 00:02:42.990 --> 00:02:43.000 align:start position:0% <00:02:42.990>>> <00:02:42.991>So, <00:02:42.993>we <00:02:42.994>naturally <00:02:42.996>disagree <00:02:42.997>a <00:02:42.999>lot. 00:02:43.000 --> 00:02:46.030 align:start position:0% <00:02:43.000>>> <00:02:43.107>So, <00:02:43.215>we <00:02:43.323>naturally <00:02:43.430>disagree <00:02:43.538>a <00:02:43.646>lot. <00:02:43.753>>> <00:02:43.861>In <00:02:43.969>a <00:02:44.076>kind <00:02:44.184>of <00:02:44.292>productive <00:02:44.400>way. 00:02:46.030 --> 00:02:46.040 align:start position:0% <00:02:46.030>>> <00:02:46.031>In <00:02:46.033>a <00:02:46.034>kind <00:02:46.036>of <00:02:46.037>productive <00:02:46.039>way. 00:02:46.040 --> 00:02:49.430 align:start position:0% <00:02:46.040>>> <00:02:46.140>In <00:02:46.239>a <00:02:46.340>kind <00:02:46.440>of <00:02:46.540>productive <00:02:46.640>way. <00:02:46.739>>> <00:02:46.840>So, <00:02:46.940>we <00:02:47.040>both <00:02:47.140>work <00:02:47.239>at <00:02:47.340>GetYourGuide, <00:02:47.440>the 00:02:49.430 --> 00:02:49.440 align:start position:0% <00:02:49.430>>> <00:02:49.431>So, <00:02:49.432>we <00:02:49.433>both <00:02:49.435>work <00:02:49.436>at <00:02:49.437>GetYourGuide, <00:02:49.439>the 00:02:49.440 --> 00:02:51.230 align:start position:0% <00:02:49.440>>> <00:02:49.567>So, <00:02:49.694>we <00:02:49.821>both <00:02:49.949>work <00:02:50.076>at <00:02:50.203>GetYourGuide, <00:02:50.330>the <00:02:50.458>global <00:02:50.585>marketplace <00:02:50.712>of <00:02:50.840>travel 00:02:51.230 --> 00:02:51.240 align:start position:0% <00:02:51.230>global <00:02:51.233>marketplace <00:02:51.236>of <00:02:51.239>travel 00:02:51.240 --> 00:02:53.710 align:start position:0% <00:02:51.240>global <00:02:51.380>marketplace <00:02:51.520>of <00:02:51.660>travel <00:02:51.800>experiences. <00:02:51.940>And <00:02:52.080>for <00:02:52.220>example, <00:02:52.360>if <00:02:52.500>you <00:02:52.640>are 00:02:53.710 --> 00:02:53.720 align:start position:0% <00:02:53.710>experiences. <00:02:53.711>And <00:02:53.713>for <00:02:53.714>example, <00:02:53.716>if <00:02:53.717>you <00:02:53.719>are 00:02:53.720 --> 00:02:56.390 align:start position:0% <00:02:53.720>experiences. <00:02:53.827>And <00:02:53.935>for <00:02:54.043>example, <00:02:54.150>if <00:02:54.258>you <00:02:54.366>are <00:02:54.473>looking <00:02:54.581>for <00:02:54.689>a <00:02:54.796>paella <00:02:54.904>cooking <00:02:55.012>class <00:02:55.120>in 00:02:56.390 --> 00:02:56.400 align:start position:0% <00:02:56.390>looking <00:02:56.391>for <00:02:56.393>a <00:02:56.394>paella <00:02:56.396>cooking <00:02:56.397>class <00:02:56.399>in 00:02:56.400 --> 00:03:00.350 align:start position:0% <00:02:56.400>looking <00:02:56.507>for <00:02:56.615>a <00:02:56.723>paella <00:02:56.830>cooking <00:02:56.938>class <00:02:57.046>in <00:02:57.153>Valencia <00:02:57.261>or <00:02:57.369>a <00:02:57.476>glacier <00:02:57.584>hike <00:02:57.692>in <00:02:57.800>Iceland, 00:03:00.350 --> 00:03:00.360 align:start position:0% <00:03:00.350>Valencia <00:03:00.351>or <00:03:00.353>a <00:03:00.354>glacier <00:03:00.356>hike <00:03:00.357>in <00:03:00.359>Iceland, 00:03:00.360 --> 00:03:02.670 align:start position:0% <00:03:00.360>Valencia <00:03:00.500>or <00:03:00.640>a <00:03:00.780>glacier <00:03:00.920>hike <00:03:01.060>in <00:03:01.200>Iceland, <00:03:01.340>or <00:03:01.480>even <00:03:01.620>like <00:03:01.760>a 00:03:02.670 --> 00:03:02.680 align:start position:0% <00:03:02.670>or <00:03:02.673>even <00:03:02.676>like <00:03:02.679>a 00:03:02.680 --> 00:03:05.150 align:start position:0% <00:03:02.680>or <00:03:02.807>even <00:03:02.934>like <00:03:03.061>a <00:03:03.189>quad <00:03:03.316>tour <00:03:03.443>in <00:03:03.570>the <00:03:03.698>desert, <00:03:03.825>those <00:03:03.952>are <00:03:04.080>the 00:03:05.150 --> 00:03:05.160 align:start position:0% <00:03:05.150>quad <00:03:05.151>tour <00:03:05.152>in <00:03:05.153>the <00:03:05.155>desert, <00:03:05.156>those <00:03:05.157>are <00:03:05.159>the 00:03:05.160 --> 00:03:08.270 align:start position:0% <00:03:05.160>quad <00:03:05.287>tour <00:03:05.414>in <00:03:05.541>the <00:03:05.669>desert, <00:03:05.796>those <00:03:05.923>are <00:03:06.050>the <00:03:06.178>experiences <00:03:06.305>that <00:03:06.432>we <00:03:06.560>provide. 00:03:08.270 --> 00:03:08.280 align:start position:0% <00:03:08.270>experiences <00:03:08.273>that <00:03:08.276>we <00:03:08.279>provide. 00:03:08.280 --> 00:03:09.990 align:start position:0% <00:03:08.280>experiences <00:03:08.396>that <00:03:08.513>we <00:03:08.630>provide. <00:03:08.746>To <00:03:08.863>give <00:03:08.980>you <00:03:09.096>a <00:03:09.213>sense <00:03:09.330>of <00:03:09.446>scale, <00:03:09.563>we <00:03:09.680>are 00:03:09.990 --> 00:03:10.000 align:start position:0% <00:03:09.990>To <00:03:09.991>give <00:03:09.992>you <00:03:09.993>a <00:03:09.994>sense <00:03:09.995>of <00:03:09.996>scale, <00:03:09.997>we <00:03:09.999>are 00:03:10.000 --> 00:03:11.949 align:start position:0% <00:03:10.000>To <00:03:10.093>give <00:03:10.186>you <00:03:10.280>a <00:03:10.373>sense <00:03:10.466>of <00:03:10.560>scale, <00:03:10.653>we <00:03:10.746>are <00:03:10.840>more <00:03:10.933>than <00:03:11.026>1,000 <00:03:11.120>employees <00:03:11.213>and <00:03:11.306>more <00:03:11.400>than 00:03:11.949 --> 00:03:11.959 align:start position:0% <00:03:11.949>more <00:03:11.950>than <00:03:11.952>1,000 <00:03:11.953>employees <00:03:11.955>and <00:03:11.956>more <00:03:11.958>than 00:03:11.959 --> 00:03:14.070 align:start position:0% <00:03:11.959>more <00:03:12.134>than <00:03:12.309>1,000 <00:03:12.484>employees <00:03:12.659>and <00:03:12.834>more <00:03:13.009>than <00:03:13.184>400 <00:03:13.359>engineers. 00:03:14.070 --> 00:03:14.080 align:start position:0% <00:03:14.070>400 <00:03:14.079>engineers. 00:03:14.080 --> 00:03:15.550 align:start position:0% <00:03:14.080>400 <00:03:14.227>engineers. <00:03:14.374>On <00:03:14.521>the <00:03:14.668>product <00:03:14.815>side, <00:03:14.962>we <00:03:15.109>have <00:03:15.256>more <00:03:15.403>than 00:03:15.550 --> 00:03:15.560 align:start position:0% <00:03:15.550>On <00:03:15.551>the <00:03:15.552>product <00:03:15.553>side, <00:03:15.555>we <00:03:15.556>have <00:03:15.557>more <00:03:15.559>than 00:03:15.560 --> 00:03:19.390 align:start position:0% <00:03:15.560>On <00:03:15.676>the <00:03:15.793>product <00:03:15.910>side, <00:03:16.026>we <00:03:16.143>have <00:03:16.260>more <00:03:16.376>than <00:03:16.493>200,000 <00:03:16.610>experiences <00:03:16.726>from <00:03:16.843>more <00:03:16.960>than 00:03:19.390 --> 00:03:19.400 align:start position:0% <00:03:19.390>200,000 <00:03:19.392>experiences <00:03:19.394>from <00:03:19.396>more <00:03:19.399>than 00:03:19.400 --> 00:03:22.310 align:start position:0% <00:03:19.400>200,000 <00:03:19.600>experiences <00:03:19.800>from <00:03:20.000>more <00:03:20.200>than <00:03:20.400>50,000 <00:03:20.600>experience <00:03:20.800>creators 00:03:22.310 --> 00:03:22.320 align:start position:0% <00:03:22.310>50,000 <00:03:22.314>experience <00:03:22.319>creators 00:03:22.320 --> 00:03:26.030 align:start position:0% <00:03:22.320>50,000 <00:03:22.519>experience <00:03:22.720>creators <00:03:22.920>in <00:03:23.120>more <00:03:23.320>than <00:03:23.519>18,000 <00:03:23.720>cities. 00:03:26.030 --> 00:03:26.040 align:start position:0% <00:03:26.030>in <00:03:26.032>more <00:03:26.034>than <00:03:26.036>18,000 <00:03:26.039>cities. 00:03:26.040 --> 00:03:28.910 align:start position:0% <00:03:26.040>in <00:03:26.147>more <00:03:26.255>than <00:03:26.363>18,000 <00:03:26.470>cities. <00:03:26.578>So, <00:03:26.686>why <00:03:26.793>did <00:03:26.901>we <00:03:27.009>build <00:03:27.116>an <00:03:27.224>agent <00:03:27.332>at <00:03:27.440>all? 00:03:28.910 --> 00:03:28.920 align:start position:0% <00:03:28.910>So, <00:03:28.911>why <00:03:28.912>did <00:03:28.913>we <00:03:28.914>build <00:03:28.915>an <00:03:28.916>agent <00:03:28.917>at <00:03:28.919>all? 00:03:28.920 --> 00:03:31.710 align:start position:0% <00:03:28.920>So, <00:03:29.020>why <00:03:29.120>did <00:03:29.220>we <00:03:29.320>build <00:03:29.420>an <00:03:29.520>agent <00:03:29.620>at <00:03:29.720>all? <00:03:29.820>The <00:03:29.920>search <00:03:30.020>landscape <00:03:30.120>was <00:03:30.220>changing <00:03:30.320>pretty 00:03:31.710 --> 00:03:31.720 align:start position:0% <00:03:31.710>The <00:03:31.711>search <00:03:31.713>landscape <00:03:31.715>was <00:03:31.717>changing <00:03:31.719>pretty 00:03:31.720 --> 00:03:35.229 align:start position:0% <00:03:31.720>The <00:03:31.847>search <00:03:31.974>landscape <00:03:32.101>was <00:03:32.229>changing <00:03:32.356>pretty <00:03:32.483>fast <00:03:32.610>under <00:03:32.738>our <00:03:32.865>feet. <00:03:32.992>In <00:03:33.120>late 00:03:35.229 --> 00:03:35.239 align:start position:0% <00:03:35.228>fast <00:03:35.230>under <00:03:35.232>our <00:03:35.234>feet. <00:03:35.236>In <00:03:35.238>late 00:03:35.239 --> 00:03:37.470 align:start position:0% <00:03:35.239>fast <00:03:35.439>under <00:03:35.639>our <00:03:35.839>feet. <00:03:36.039>In <00:03:36.239>late <00:03:36.439>December <00:03:36.639>2025, 00:03:37.470 --> 00:03:37.480 align:start position:0% <00:03:37.470>December <00:03:37.478>2025, 00:03:37.480 --> 00:03:41.790 align:start position:0% <00:03:37.480>December <00:03:37.655>2025, <00:03:37.829>we <00:03:38.005>saw <00:03:38.179>that <00:03:38.355>OpenAI <00:03:38.530>opened <00:03:38.704>up <00:03:38.880>submissions 00:03:41.790 --> 00:03:41.800 align:start position:0% <00:03:41.790>we <00:03:41.791>saw <00:03:41.793>that <00:03:41.794>OpenAI <00:03:41.796>opened <00:03:41.797>up <00:03:41.799>submissions 00:03:41.800 --> 00:03:45.070 align:start position:0% <00:03:41.800>we <00:03:41.916>saw <00:03:42.033>that <00:03:42.150>OpenAI <00:03:42.266>opened <00:03:42.383>up <00:03:42.500>submissions <00:03:42.616>for <00:03:42.733>developers. <00:03:42.850>So, <00:03:42.966>from <00:03:43.083>that <00:03:43.200>point, 00:03:45.070 --> 00:03:45.080 align:start position:0% <00:03:45.070>for <00:03:45.071>developers. <00:03:45.073>So, <00:03:45.075>from <00:03:45.077>that <00:03:45.078>point, 00:03:45.080 --> 00:03:47.750 align:start position:0% <00:03:45.079>for <00:03:45.196>developers. <00:03:45.313>So, <00:03:45.429>from <00:03:45.546>that <00:03:45.663>point, <00:03:45.779>anyone <00:03:45.896>can <00:03:46.013>put <00:03:46.130>their <00:03:46.246>product <00:03:46.363>inside <00:03:46.480>the 00:03:47.750 --> 00:03:47.760 align:start position:0% <00:03:47.750>anyone <00:03:47.751>can <00:03:47.753>put <00:03:47.754>their <00:03:47.756>product <00:03:47.757>inside <00:03:47.759>the 00:03:47.760 --> 00:03:49.509 align:start position:0% <00:03:47.760>anyone <00:03:47.935>can <00:03:48.109>put <00:03:48.285>their <00:03:48.459>product <00:03:48.635>inside <00:03:48.810>the <00:03:48.984>chat <00:03:49.160>experience. 00:03:49.509 --> 00:03:49.519 align:start position:0% <00:03:49.509>chat <00:03:49.518>experience. 00:03:49.519 --> 00:03:51.790 align:start position:0% <00:03:49.519>chat <00:03:49.674>experience. <00:03:49.830>So, <00:03:49.985>that's <00:03:50.141>where <00:03:50.296>we <00:03:50.452>started <00:03:50.607>to <00:03:50.763>have <00:03:50.919>this 00:03:51.790 --> 00:03:51.800 align:start position:0% <00:03:51.790>So, <00:03:51.791>that's <00:03:51.792>where <00:03:51.793>we <00:03:51.795>started <00:03:51.796>to <00:03:51.797>have <00:03:51.799>this 00:03:51.800 --> 00:03:54.110 align:start position:0% <00:03:51.800>So, <00:03:51.907>that's <00:03:52.015>where <00:03:52.123>we <00:03:52.230>started <00:03:52.338>to <00:03:52.446>have <00:03:52.553>this <00:03:52.661>fear <00:03:52.769>of <00:03:52.876>disruption <00:03:52.984>that <00:03:53.092>kicked <00:03:53.200>in 00:03:54.110 --> 00:03:54.120 align:start position:0% <00:03:54.110>fear <00:03:54.111>of <00:03:54.113>disruption <00:03:54.115>that <00:03:54.117>kicked <00:03:54.119>in 00:03:54.120 --> 00:03:56.270 align:start position:0% <00:03:54.120>fear <00:03:54.236>of <00:03:54.353>disruption <00:03:54.470>that <00:03:54.586>kicked <00:03:54.703>in <00:03:54.820>because <00:03:54.936>we <00:03:55.053>started <00:03:55.170>to <00:03:55.286>see <00:03:55.403>headlines <00:03:55.520>all 00:03:56.270 --> 00:03:56.280 align:start position:0% <00:03:56.270>because <00:03:56.271>we <00:03:56.273>started <00:03:56.274>to <00:03:56.276>see <00:03:56.277>headlines <00:03:56.279>all 00:03:56.280 --> 00:03:58.670 align:start position:0% <00:03:56.280>because <00:03:56.396>we <00:03:56.513>started <00:03:56.630>to <00:03:56.746>see <00:03:56.863>headlines <00:03:56.980>all <00:03:57.096>over <00:03:57.213>the <00:03:57.330>places. <00:03:57.446>For <00:03:57.563>example, <00:03:57.680>from 00:03:58.670 --> 00:03:58.680 align:start position:0% <00:03:58.670>over <00:03:58.671>the <00:03:58.673>places. <00:03:58.675>For <00:03:58.677>example, <00:03:58.679>from 00:03:58.680 --> 00:04:01.150 align:start position:0% <00:03:58.680>over <00:03:58.796>the <00:03:58.913>places. <00:03:59.030>For <00:03:59.146>example, <00:03:59.263>from <00:03:59.380>Skift, <00:03:59.496>share <00:03:59.613>of <00:03:59.730>travelers <00:03:59.846>who <00:03:59.963>said <00:04:00.080>they 00:04:01.150 --> 00:04:01.160 align:start position:0% <00:04:01.150>Skift, <00:04:01.151>share <00:04:01.153>of <00:04:01.154>travelers <00:04:01.156>who <00:04:01.157>said <00:04:01.159>they 00:04:01.160 --> 00:04:04.070 align:start position:0% <00:04:01.160>Skift, <00:04:01.276>share <00:04:01.393>of <00:04:01.510>travelers <00:04:01.626>who <00:04:01.743>said <00:04:01.859>they <00:04:01.976>use <00:04:02.093>ChatGPT <00:04:02.210>or <00:04:02.326>similar <00:04:02.443>tools <00:04:02.560>extensively 00:04:04.070 --> 00:04:04.080 align:start position:0% <00:04:04.070>use <00:04:04.071>ChatGPT <00:04:04.073>or <00:04:04.075>similar <00:04:04.077>tools <00:04:04.079>extensively 00:04:04.080 --> 00:04:07.070 align:start position:0% <00:04:04.080>use <00:04:04.220>ChatGPT <00:04:04.360>or <00:04:04.500>similar <00:04:04.640>tools <00:04:04.780>extensively <00:04:04.920>for <00:04:05.060>trip <00:04:05.200>planning <00:04:05.340>rose <00:04:05.480>124% 00:04:07.070 --> 00:04:07.080 align:start position:0% <00:04:07.070>for <00:04:07.072>trip <00:04:07.074>planning <00:04:07.076>rose <00:04:07.079>124% 00:04:07.080 --> 00:04:10.550 align:start position:0% <00:04:07.080>for <00:04:07.235>trip <00:04:07.391>planning <00:04:07.546>rose <00:04:07.702>124% <00:04:07.857>year-over-year <00:04:08.013>from <00:04:08.168>13% <00:04:08.324>to <00:04:08.480>30%. 00:04:10.550 --> 00:04:10.560 align:start position:0% <00:04:10.550>year-over-year <00:04:10.552>from <00:04:10.554>13% <00:04:10.556>to <00:04:10.559>30%. 00:04:10.560 --> 00:04:13.510 align:start position:0% <00:04:10.560>year-over-year <00:04:10.687>from <00:04:10.814>13% <00:04:10.941>to <00:04:11.069>30%. <00:04:11.196>Then <00:04:11.323>also, <00:04:11.450>will <00:04:11.578>agentic <00:04:11.705>AI <00:04:11.832>turn <00:04:11.960>online 00:04:13.510 --> 00:04:13.520 align:start position:0% <00:04:13.510>Then <00:04:13.511>also, <00:04:13.513>will <00:04:13.514>agentic <00:04:13.516>AI <00:04:13.517>turn <00:04:13.519>online 00:04:13.520 --> 00:04:15.830 align:start position:0% <00:04:13.520>Then <00:04:13.647>also, <00:04:13.774>will <00:04:13.901>agentic <00:04:14.029>AI <00:04:14.156>turn <00:04:14.283>online <00:04:14.410>travel <00:04:14.538>agencies <00:04:14.665>into <00:04:14.792>passive <00:04:14.920>order 00:04:15.830 --> 00:04:15.840 align:start position:0% <00:04:15.830>travel <00:04:15.832>agencies <00:04:15.834>into <00:04:15.836>passive <00:04:15.839>order 00:04:15.840 --> 00:04:17.190 align:start position:0% <00:04:15.840>travel <00:04:16.083>agencies <00:04:16.326>into <00:04:16.569>passive <00:04:16.812>order <00:04:17.055>takers? 00:04:17.190 --> 00:04:17.200 align:start position:0% takers? 00:04:17.200 --> 00:04:20.190 align:start position:0% <00:04:17.200>takers? <00:04:17.399>Or <00:04:17.599>even <00:04:17.800>from <00:04:18.000>TechCrunch, <00:04:18.200>that <00:04:18.399>Google <00:04:18.599>is 00:04:20.190 --> 00:04:20.200 align:start position:0% <00:04:20.190>Or <00:04:20.191>even <00:04:20.192>from <00:04:20.194>TechCrunch, <00:04:20.195>that <00:04:20.197>Google <00:04:20.199>is 00:04:20.200 --> 00:04:22.110 align:start position:0% <00:04:20.200>Or <00:04:20.339>even <00:04:20.479>from <00:04:20.620>TechCrunch, <00:04:20.760>that <00:04:20.899>Google <00:04:21.039>is <00:04:21.180>expanding <00:04:21.320>AI <00:04:21.459>shopping <00:04:21.599>with 00:04:22.110 --> 00:04:22.120 align:start position:0% <00:04:22.110>expanding <00:04:22.113>AI <00:04:22.116>shopping <00:04:22.119>with 00:04:22.120 --> 00:04:24.550 align:start position:0% <00:04:22.120>expanding <00:04:22.320>AI <00:04:22.520>shopping <00:04:22.720>with <00:04:22.920>conversational <00:04:23.120>search <00:04:23.320>and <00:04:23.520>agentic 00:04:24.550 --> 00:04:24.560 align:start position:0% <00:04:24.550>conversational <00:04:24.553>search <00:04:24.556>and <00:04:24.559>agentic 00:04:24.560 --> 00:04:26.110 align:start position:0% <00:04:24.560>conversational <00:04:24.908>search <00:04:25.257>and <00:04:25.606>agentic <00:04:25.955>checkout. 00:04:26.110 --> 00:04:26.120 align:start position:0% checkout. 00:04:26.120 --> 00:04:29.510 align:start position:0% <00:04:26.120>checkout. <00:04:26.295>So, <00:04:26.470>how <00:04:26.645>we <00:04:26.820>started? <00:04:26.995>Two <00:04:27.170>of <00:04:27.345>us, <00:04:27.520>hackathon 00:04:29.510 --> 00:04:29.520 align:start position:0% <00:04:29.510>So, <00:04:29.511>how <00:04:29.512>we <00:04:29.513>started? <00:04:29.515>Two <00:04:29.516>of <00:04:29.517>us, <00:04:29.519>hackathon 00:04:29.520 --> 00:04:33.190 align:start position:0% <00:04:29.520>So, <00:04:29.620>how <00:04:29.719>we <00:04:29.820>started? <00:04:29.919>Two <00:04:30.020>of <00:04:30.120>us, <00:04:30.219>hackathon <00:04:30.320>spirit, <00:04:30.419>and <00:04:30.520>a <00:04:30.620>very <00:04:30.719>serious <00:04:30.820>dress <00:04:30.919>code. 00:04:33.190 --> 00:04:33.200 align:start position:0% <00:04:33.190>spirit, <00:04:33.191>and <00:04:33.193>a <00:04:33.194>very <00:04:33.195>serious <00:04:33.197>dress <00:04:33.199>code. 00:04:33.200 --> 00:04:35.670 align:start position:0% <00:04:33.200>spirit, <00:04:33.300>and <00:04:33.400>a <00:04:33.500>very <00:04:33.599>serious <00:04:33.700>dress <00:04:33.800>code. <00:04:33.900>>> <00:04:34.000>The <00:04:34.099>water <00:04:34.200>gun <00:04:34.300>was <00:04:34.400>for <00:04:34.500>load <00:04:34.599>testing. 00:04:35.670 --> 00:04:35.680 align:start position:0% <00:04:35.670>>> <00:04:35.671>The <00:04:35.672>water <00:04:35.673>gun <00:04:35.675>was <00:04:35.676>for <00:04:35.677>load <00:04:35.679>testing. 00:04:35.680 --> 00:04:37.710 align:start position:0% <00:04:35.680>>> <00:04:35.767>The <00:04:35.855>water <00:04:35.942>gun <00:04:36.030>was <00:04:36.117>for <00:04:36.205>load <00:04:36.292>testing. <00:04:36.380>>> <00:04:36.467>We <00:04:36.555>started <00:04:36.642>with <00:04:36.730>a <00:04:36.817>simple <00:04:36.905>goal. <00:04:36.992>We <00:04:37.080>wanted 00:04:37.710 --> 00:04:37.720 align:start position:0% <00:04:37.710>>> <00:04:37.711>We <00:04:37.712>started <00:04:37.713>with <00:04:37.714>a <00:04:37.715>simple <00:04:37.716>goal. <00:04:37.717>We <00:04:37.719>wanted 00:04:37.720 --> 00:04:40.470 align:start position:0% <00:04:37.720>>> <00:04:37.807>We <00:04:37.895>started <00:04:37.982>with <00:04:38.070>a <00:04:38.157>simple <00:04:38.245>goal. <00:04:38.332>We <00:04:38.420>wanted <00:04:38.507>to <00:04:38.595>build <00:04:38.682>our <00:04:38.770>agent <00:04:38.857>that <00:04:38.945>would <00:04:39.032>work <00:04:39.120>and 00:04:40.470 --> 00:04:40.480 align:start position:0% <00:04:40.470>to <00:04:40.471>build <00:04:40.472>our <00:04:40.473>agent <00:04:40.475>that <00:04:40.476>would <00:04:40.477>work <00:04:40.479>and 00:04:40.480 --> 00:04:42.550 align:start position:0% <00:04:40.480>to <00:04:40.596>build <00:04:40.713>our <00:04:40.830>agent <00:04:40.946>that <00:04:41.063>would <00:04:41.180>work <00:04:41.296>and <00:04:41.413>find <00:04:41.530>experiences <00:04:41.646>that <00:04:41.763>would <00:04:41.880>work 00:04:42.550 --> 00:04:42.560 align:start position:0% <00:04:42.550>find <00:04:42.552>experiences <00:04:42.554>that <00:04:42.556>would <00:04:42.559>work 00:04:42.560 --> 00:04:44.950 align:start position:0% <00:04:42.560>find <00:04:42.715>experiences <00:04:42.871>that <00:04:43.026>would <00:04:43.182>work <00:04:43.337>naturally <00:04:43.493>inside <00:04:43.648>the <00:04:43.804>chat <00:04:43.960>conversation 00:04:44.950 --> 00:04:44.960 align:start position:0% <00:04:44.950>naturally <00:04:44.952>inside <00:04:44.954>the <00:04:44.956>chat <00:04:44.959>conversation 00:04:44.960 --> 00:04:48.070 align:start position:0% <00:04:44.960>naturally <00:04:45.099>inside <00:04:45.239>the <00:04:45.380>chat <00:04:45.520>conversation <00:04:45.659>that <00:04:45.799>could <00:04:45.940>actually <00:04:46.080>work <00:04:46.219>in <00:04:46.359>production. 00:04:48.070 --> 00:04:48.080 align:start position:0% <00:04:48.070>that <00:04:48.071>could <00:04:48.073>actually <00:04:48.075>work <00:04:48.077>in <00:04:48.079>production. 00:04:48.080 --> 00:04:50.350 align:start position:0% <00:04:48.080>that <00:04:48.196>could <00:04:48.313>actually <00:04:48.430>work <00:04:48.546>in <00:04:48.663>production. <00:04:48.780>So, <00:04:48.896>this <00:04:49.013>is <00:04:49.130>the <00:04:49.246>final <00:04:49.363>product. <00:04:49.479>The 00:04:50.350 --> 00:04:50.360 align:start position:0% <00:04:50.350>So, <00:04:50.351>this <00:04:50.353>is <00:04:50.354>the <00:04:50.356>final <00:04:50.357>product. <00:04:50.359>The 00:04:50.360 --> 00:04:53.150 align:start position:0% <00:04:50.360>So, <00:04:50.487>this <00:04:50.614>is <00:04:50.741>the <00:04:50.869>final <00:04:50.996>product. <00:04:51.123>The <00:04:51.250>GetYourGuide <00:04:51.378>app <00:04:51.505>running <00:04:51.632>inside <00:04:51.760>ChatGPT 00:04:53.150 --> 00:04:53.160 align:start position:0% <00:04:53.150>GetYourGuide <00:04:53.152>app <00:04:53.154>running <00:04:53.156>inside <00:04:53.159>ChatGPT 00:04:53.160 --> 00:04:55.870 align:start position:0% <00:04:53.159>GetYourGuide <00:04:53.287>app <00:04:53.414>running <00:04:53.541>inside <00:04:53.669>ChatGPT <00:04:53.796>powered <00:04:53.923>by <00:04:54.050>our <00:04:54.178>discovery <00:04:54.305>agent. <00:04:54.432>Let <00:04:54.559>me 00:04:55.870 --> 00:04:55.880 align:start position:0% <00:04:55.870>powered <00:04:55.871>by <00:04:55.873>our <00:04:55.874>discovery <00:04:55.876>agent. <00:04:55.877>Let <00:04:55.879>me 00:04:55.880 --> 00:04:59.070 align:start position:0% <00:04:55.880>powered <00:04:56.007>by <00:04:56.134>our <00:04:56.261>discovery <00:04:56.389>agent. <00:04:56.516>Let <00:04:56.643>me <00:04:56.770>show <00:04:56.898>you <00:04:57.025>how <00:04:57.152>it <00:04:57.280>works. 00:04:59.070 --> 00:04:59.080 align:start position:0% <00:04:59.070>show <00:04:59.072>you <00:04:59.074>how <00:04:59.076>it <00:04:59.079>works. 00:04:59.080 --> 00:05:01.070 align:start position:0% <00:04:59.080>show <00:04:59.219>you <00:04:59.359>how <00:04:59.500>it <00:04:59.640>works. <00:04:59.780>So, <00:04:59.919>a <00:05:00.060>traveler <00:05:00.200>describe <00:05:00.340>what <00:05:00.479>they're 00:05:01.070 --> 00:05:01.080 align:start position:0% <00:05:01.070>So, <00:05:01.071>a <00:05:01.073>traveler <00:05:01.075>describe <00:05:01.077>what <00:05:01.079>they're 00:05:01.080 --> 00:05:04.230 align:start position:0% <00:05:01.080>So, <00:05:01.196>a <00:05:01.313>traveler <00:05:01.430>describe <00:05:01.546>what <00:05:01.663>they're <00:05:01.780>looking <00:05:01.896>for <00:05:02.013>in <00:05:02.130>natural <00:05:02.246>language. <00:05:02.363>And <00:05:02.479>the 00:05:04.230 --> 00:05:04.240 align:start position:0% <00:05:04.230>looking <00:05:04.231>for <00:05:04.233>in <00:05:04.234>natural <00:05:04.236>language. <00:05:04.237>And <00:05:04.239>the 00:05:04.240 --> 00:05:07.150 align:start position:0% <00:05:04.240>looking <00:05:04.356>for <00:05:04.473>in <00:05:04.590>natural <00:05:04.706>language. <00:05:04.823>And <00:05:04.940>the <00:05:05.056>agent <00:05:05.173>turns <00:05:05.290>that <00:05:05.406>into <00:05:05.523>relevant <00:05:05.640>activity 00:05:07.150 --> 00:05:07.160 align:start position:0% <00:05:07.150>agent <00:05:07.151>turns <00:05:07.153>that <00:05:07.155>into <00:05:07.157>relevant <00:05:07.159>activity 00:05:07.160 --> 00:05:10.230 align:start position:0% <00:05:07.160>agent <00:05:07.300>turns <00:05:07.440>that <00:05:07.580>into <00:05:07.720>relevant <00:05:07.860>activity <00:05:08.000>recommendations. <00:05:08.140>It <00:05:08.280>looks <00:05:08.420>simple, <00:05:08.560>like 00:05:10.230 --> 00:05:10.240 align:start position:0% <00:05:10.230>recommendations. <00:05:10.232>It <00:05:10.234>looks <00:05:10.236>simple, <00:05:10.239>like 00:05:10.240 --> 00:05:12.630 align:start position:0% <00:05:10.240>recommendations. <00:05:10.380>It <00:05:10.520>looks <00:05:10.660>simple, <00:05:10.800>like <00:05:10.940>it's <00:05:11.080>just <00:05:11.220>a <00:05:11.360>conversation. <00:05:11.500>But <00:05:11.640>actually, 00:05:12.630 --> 00:05:12.640 align:start position:0% <00:05:12.630>it's <00:05:12.631>just <00:05:12.633>a <00:05:12.635>conversation. <00:05:12.637>But <00:05:12.639>actually, 00:05:12.640 --> 00:05:14.710 align:start position:0% <00:05:12.640>it's <00:05:12.747>just <00:05:12.855>a <00:05:12.963>conversation. <00:05:13.070>But <00:05:13.178>actually, <00:05:13.286>behind <00:05:13.393>it <00:05:13.501>there <00:05:13.609>is <00:05:13.716>a <00:05:13.824>lot <00:05:13.932>of <00:05:14.039>production 00:05:14.710 --> 00:05:14.720 align:start position:0% <00:05:14.710>behind <00:05:14.711>it <00:05:14.712>there <00:05:14.713>is <00:05:14.715>a <00:05:14.716>lot <00:05:14.717>of <00:05:14.719>production 00:05:14.720 --> 00:05:16.230 align:start position:0% <00:05:14.720>behind <00:05:14.824>it <00:05:14.929>there <00:05:15.033>is <00:05:15.138>a <00:05:15.242>lot <00:05:15.347>of <00:05:15.451>production <00:05:15.556>system <00:05:15.660>that <00:05:15.765>needs <00:05:15.869>to <00:05:15.974>understand <00:05:16.079>the 00:05:16.230 --> 00:05:16.240 align:start position:0% <00:05:16.230>system <00:05:16.231>that <00:05:16.233>needs <00:05:16.235>to <00:05:16.237>understand <00:05:16.239>the 00:05:16.240 --> 00:05:19.190 align:start position:0% <00:05:16.240>system <00:05:16.367>that <00:05:16.494>needs <00:05:16.621>to <00:05:16.749>understand <00:05:16.876>the <00:05:17.003>intent, <00:05:17.130>route, <00:05:17.258>find <00:05:17.385>between <00:05:17.512>all <00:05:17.640>the 00:05:19.190 --> 00:05:19.200 align:start position:0% <00:05:19.190>intent, <00:05:19.191>route, <00:05:19.193>find <00:05:19.195>between <00:05:19.197>all <00:05:19.199>the 00:05:19.200 --> 00:05:21.350 align:start position:0% <00:05:19.200>intent, <00:05:19.316>route, <00:05:19.433>find <00:05:19.550>between <00:05:19.666>all <00:05:19.783>the <00:05:19.900>thousands <00:05:20.016>of <00:05:20.133>activities <00:05:20.250>that <00:05:20.366>we <00:05:20.483>have <00:05:20.599>in 00:05:21.350 --> 00:05:21.360 align:start position:0% <00:05:21.350>thousands <00:05:21.351>of <00:05:21.353>activities <00:05:21.354>that <00:05:21.356>we <00:05:21.357>have <00:05:21.359>in 00:05:21.360 --> 00:05:24.909 align:start position:0% <00:05:21.360>thousands <00:05:21.460>of <00:05:21.560>activities <00:05:21.660>that <00:05:21.760>we <00:05:21.860>have <00:05:21.960>in <00:05:22.060>a <00:05:22.160>fast <00:05:22.260>way, <00:05:22.360>cheap <00:05:22.460>way, <00:05:22.560>and <00:05:22.660>also <00:05:22.760>being 00:05:24.909 --> 00:05:24.919 align:start position:0% <00:05:24.909>a <00:05:24.910>fast <00:05:24.911>way, <00:05:24.912>cheap <00:05:24.914>way, <00:05:24.915>and <00:05:24.916>also <00:05:24.918>being 00:05:24.919 --> 00:05:28.030 align:start position:0% <00:05:24.919>a <00:05:25.019>fast <00:05:25.119>way, <00:05:25.219>cheap <00:05:25.318>way, <00:05:25.419>and <00:05:25.519>also <00:05:25.619>being <00:05:25.719>able <00:05:25.818>to <00:05:25.919>sustain <00:05:26.019>changes <00:05:26.119>over <00:05:26.219>time. <00:05:26.318>That 00:05:28.030 --> 00:05:28.040 align:start position:0% <00:05:28.030>able <00:05:28.031>to <00:05:28.033>sustain <00:05:28.034>changes <00:05:28.036>over <00:05:28.037>time. <00:05:28.039>That 00:05:28.040 --> 00:05:29.990 align:start position:0% <00:05:28.040>able <00:05:28.140>to <00:05:28.240>sustain <00:05:28.340>changes <00:05:28.440>over <00:05:28.540>time. <00:05:28.640>That <00:05:28.740>when <00:05:28.840>we <00:05:28.940>change <00:05:29.040>the <00:05:29.140>model, <00:05:29.240>we <00:05:29.340>change <00:05:29.440>the 00:05:29.990 --> 00:05:30.000 align:start position:0% <00:05:29.990>when <00:05:29.991>we <00:05:29.992>change <00:05:29.993>the <00:05:29.995>model, <00:05:29.996>we <00:05:29.997>change <00:05:29.999>the 00:05:30.000 --> 00:05:31.350 align:start position:0% <00:05:30.000>when <00:05:30.151>we <00:05:30.303>change <00:05:30.455>the <00:05:30.607>model, <00:05:30.759>we <00:05:30.911>change <00:05:31.063>the <00:05:31.215>prompts. 00:05:31.350 --> 00:05:31.360 align:start position:0% prompts. 00:05:31.360 --> 00:05:32.270 align:start position:0% <00:05:31.360>prompts. <00:05:32.179>So, 00:05:32.270 --> 00:05:32.280 align:start position:0% So, 00:05:32.280 --> 00:05:34.950 align:start position:0% <00:05:32.280>So, <00:05:32.479>now <00:05:32.679>that's <00:05:32.880>the <00:05:33.080>experience <00:05:33.280>that <00:05:33.479>we <00:05:33.679>built, 00:05:34.950 --> 00:05:34.960 align:start position:0% <00:05:34.950>now <00:05:34.951>that's <00:05:34.953>the <00:05:34.954>experience <00:05:34.955>that <00:05:34.957>we <00:05:34.959>built, 00:05:34.960 --> 00:05:36.830 align:start position:0% <00:05:34.960>now <00:05:35.060>that's <00:05:35.159>the <00:05:35.260>experience <00:05:35.359>that <00:05:35.460>we <00:05:35.560>built, <00:05:35.659>and <00:05:35.760>I'll <00:05:35.859>hand <00:05:35.960>it <00:05:36.060>over <00:05:36.159>to <00:05:36.260>Steven <00:05:36.359>to 00:05:36.830 --> 00:05:36.840 align:start position:0% <00:05:36.830>and <00:05:36.831>I'll <00:05:36.832>hand <00:05:36.833>it <00:05:36.835>over <00:05:36.836>to <00:05:36.837>Steven <00:05:36.839>to 00:05:36.840 --> 00:05:39.110 align:start position:0% <00:05:36.840>and <00:05:36.940>I'll <00:05:37.040>hand <00:05:37.140>it <00:05:37.240>over <00:05:37.340>to <00:05:37.440>Steven <00:05:37.540>to <00:05:37.640>explain <00:05:37.740>the <00:05:37.840>customer <00:05:37.940>problem <00:05:38.040>and <00:05:38.140>how <00:05:38.240>did 00:05:39.110 --> 00:05:39.120 align:start position:0% <00:05:39.110>explain <00:05:39.111>the <00:05:39.113>customer <00:05:39.114>problem <00:05:39.116>and <00:05:39.117>how <00:05:39.119>did 00:05:39.120 --> 00:05:44.470 align:start position:0% <00:05:39.120>explain <00:05:39.275>the <00:05:39.431>customer <00:05:39.586>problem <00:05:39.742>and <00:05:39.897>how <00:05:40.053>did <00:05:40.208>we <00:05:40.364>solve <00:05:40.520>it. 00:05:44.480 --> 00:05:47.190 align:start position:0% <00:05:44.480>>> <00:05:44.830>Thanks <00:05:45.180>a <00:05:45.530>lot, <00:05:45.880>Giampaolo. 00:05:47.190 --> 00:05:47.200 align:start position:0% <00:05:47.190>>> <00:05:47.192>Thanks <00:05:47.194>a <00:05:47.196>lot, <00:05:47.199>Giampaolo. 00:05:47.200 --> 00:05:49.390 align:start position:0% <00:05:47.200>>> <00:05:47.316>Thanks <00:05:47.433>a <00:05:47.550>lot, <00:05:47.666>Giampaolo. <00:05:47.783>So, <00:05:47.900>I <00:05:48.016>would <00:05:48.133>like <00:05:48.250>to <00:05:48.366>start <00:05:48.483>with <00:05:48.599>showing 00:05:49.390 --> 00:05:49.400 align:start position:0% <00:05:49.390>So, <00:05:49.391>I <00:05:49.392>would <00:05:49.393>like <00:05:49.395>to <00:05:49.396>start <00:05:49.397>with <00:05:49.399>showing 00:05:49.400 --> 00:05:52.270 align:start position:0% <00:05:49.400>So, <00:05:49.493>I <00:05:49.586>would <00:05:49.679>like <00:05:49.773>to <00:05:49.866>start <00:05:49.960>with <00:05:50.053>showing <00:05:50.146>kind <00:05:50.239>of <00:05:50.333>like <00:05:50.426>examples <00:05:50.520>of <00:05:50.613>how <00:05:50.706>a <00:05:50.799>traveler 00:05:52.270 --> 00:05:52.280 align:start position:0% <00:05:52.270>kind <00:05:52.271>of <00:05:52.272>like <00:05:52.273>examples <00:05:52.275>of <00:05:52.276>how <00:05:52.277>a <00:05:52.279>traveler 00:05:52.280 --> 00:05:54.870 align:start position:0% <00:05:52.280>kind <00:05:52.407>of <00:05:52.534>like <00:05:52.661>examples <00:05:52.789>of <00:05:52.916>how <00:05:53.043>a <00:05:53.170>traveler <00:05:53.298>would <00:05:53.425>use <00:05:53.552>conversational <00:05:53.679>AI. 00:05:54.870 --> 00:05:54.880 align:start position:0% <00:05:54.870>would <00:05:54.873>use <00:05:54.876>conversational <00:05:54.879>AI. 00:05:54.880 --> 00:05:57.430 align:start position:0% <00:05:54.880>would <00:05:55.007>use <00:05:55.134>conversational <00:05:55.261>AI. <00:05:55.389>So, <00:05:55.516>in <00:05:55.643>this <00:05:55.770>example, <00:05:55.898>we <00:05:56.025>have <00:05:56.152>a <00:05:56.280>prompt 00:05:57.430 --> 00:05:57.440 align:start position:0% <00:05:57.430>So, <00:05:57.431>in <00:05:57.432>this <00:05:57.433>example, <00:05:57.435>we <00:05:57.436>have <00:05:57.437>a <00:05:57.439>prompt 00:05:57.440 --> 00:05:59.190 align:start position:0% <00:05:57.440>So, <00:05:57.540>in <00:05:57.640>this <00:05:57.740>example, <00:05:57.840>we <00:05:57.940>have <00:05:58.040>a <00:05:58.140>prompt <00:05:58.240>where <00:05:58.340>it <00:05:58.440>says, <00:05:58.540>"I'm <00:05:58.640>heading <00:05:58.740>to <00:05:58.840>Tenerife 00:05:59.190 --> 00:05:59.200 align:start position:0% <00:05:59.190>where <00:05:59.191>it <00:05:59.193>says, <00:05:59.194>"I'm <00:05:59.195>heading <00:05:59.197>to <00:05:59.199>Tenerife 00:05:59.200 --> 00:06:02.070 align:start position:0% <00:05:59.200>where <00:05:59.293>it <00:05:59.386>says, <00:05:59.479>"I'm <00:05:59.573>heading <00:05:59.666>to <00:05:59.760>Tenerife <00:05:59.853>next <00:05:59.946>week <00:06:00.039>with <00:06:00.133>my <00:06:00.226>wife <00:06:00.320>and <00:06:00.413>two <00:06:00.506>kids <00:06:00.599>age 00:06:02.070 --> 00:06:02.080 align:start position:0% <00:06:02.070>next <00:06:02.071>week <00:06:02.072>with <00:06:02.073>my <00:06:02.074>wife <00:06:02.075>and <00:06:02.076>two <00:06:02.077>kids <00:06:02.079>age 00:06:02.080 --> 00:06:03.710 align:start position:0% <00:06:02.080>next <00:06:02.207>week <00:06:02.334>with <00:06:02.461>my <00:06:02.589>wife <00:06:02.716>and <00:06:02.843>two <00:06:02.970>kids <00:06:03.098>age <00:06:03.225>5 <00:06:03.352>and <00:06:03.479>8. 00:06:03.710 --> 00:06:03.720 align:start position:0% <00:06:03.710>5 <00:06:03.714>and <00:06:03.719>8. 00:06:03.720 --> 00:06:05.430 align:start position:0% <00:06:03.720>5 <00:06:03.860>and <00:06:04.000>8. <00:06:04.140>We <00:06:04.280>want <00:06:04.420>to <00:06:04.560>see <00:06:04.700>the <00:06:04.840>volcano, <00:06:04.980>but <00:06:05.120>avoid 00:06:05.430 --> 00:06:05.440 align:start position:0% <00:06:05.430>We <00:06:05.431>want <00:06:05.432>to <00:06:05.433>see <00:06:05.435>the <00:06:05.436>volcano, <00:06:05.437>but <00:06:05.439>avoid 00:06:05.440 --> 00:06:07.190 align:start position:0% <00:06:05.440>We <00:06:05.580>want <00:06:05.719>to <00:06:05.860>see <00:06:06.000>the <00:06:06.140>volcano, <00:06:06.280>but <00:06:06.420>avoid <00:06:06.560>the <00:06:06.700>cable <00:06:06.840>car. 00:06:07.190 --> 00:06:07.200 align:start position:0% <00:06:07.190>the <00:06:07.194>cable <00:06:07.199>car. 00:06:07.200 --> 00:06:09.230 align:start position:0% <00:06:07.200>the <00:06:07.340>cable <00:06:07.479>car. <00:06:07.620>Any <00:06:07.760>guided <00:06:07.900>hikes <00:06:08.039>or <00:06:08.180>ways <00:06:08.320>to <00:06:08.460>enjoy <00:06:08.599>the 00:06:09.230 --> 00:06:09.240 align:start position:0% <00:06:09.230>Any <00:06:09.231>guided <00:06:09.232>hikes <00:06:09.233>or <00:06:09.235>ways <00:06:09.236>to <00:06:09.237>enjoy <00:06:09.239>the 00:06:09.240 --> 00:06:11.270 align:start position:0% <00:06:09.240>Any <00:06:09.415>guided <00:06:09.590>hikes <00:06:09.765>or <00:06:09.940>ways <00:06:10.115>to <00:06:10.290>enjoy <00:06:10.465>the <00:06:10.640>nature 00:06:11.270 --> 00:06:11.280 align:start position:0% nature 00:06:11.280 --> 00:06:13.150 align:start position:0% <00:06:11.280>nature <00:06:11.513>around <00:06:11.746>that <00:06:11.979>are <00:06:12.213>suitable <00:06:12.446>for <00:06:12.679>kids?" 00:06:13.150 --> 00:06:13.160 align:start position:0% <00:06:13.150>around <00:06:13.151>that <00:06:13.153>are <00:06:13.155>suitable <00:06:13.157>for <00:06:13.159>kids?" 00:06:13.160 --> 00:06:15.190 align:start position:0% <00:06:13.160>around <00:06:13.267>that <00:06:13.375>are <00:06:13.483>suitable <00:06:13.590>for <00:06:13.698>kids?" <00:06:13.806>And <00:06:13.913>this <00:06:14.021>prompt <00:06:14.129>itself <00:06:14.236>has <00:06:14.344>quite <00:06:14.452>a <00:06:14.560>lot 00:06:15.190 --> 00:06:15.200 align:start position:0% <00:06:15.190>And <00:06:15.191>this <00:06:15.192>prompt <00:06:15.193>itself <00:06:15.195>has <00:06:15.196>quite <00:06:15.197>a <00:06:15.199>lot 00:06:15.200 --> 00:06:17.310 align:start position:0% <00:06:15.200>And <00:06:15.355>this <00:06:15.511>prompt <00:06:15.666>itself <00:06:15.822>has <00:06:15.977>quite <00:06:16.133>a <00:06:16.288>lot <00:06:16.444>of <00:06:16.599>information. 00:06:17.310 --> 00:06:17.320 align:start position:0% <00:06:17.310>of <00:06:17.319>information. 00:06:17.320 --> 00:06:20.710 align:start position:0% <00:06:17.320>of <00:06:17.495>information. <00:06:17.670>We <00:06:17.844>see <00:06:18.020>filter <00:06:18.195>and <00:06:18.370>constraints <00:06:18.545>such <00:06:18.719>as 00:06:20.710 --> 00:06:20.720 align:start position:0% <00:06:20.710>We <00:06:20.711>see <00:06:20.712>filter <00:06:20.714>and <00:06:20.716>constraints <00:06:20.717>such <00:06:20.719>as 00:06:20.720 --> 00:06:22.430 align:start position:0% <00:06:20.720>We <00:06:20.827>see <00:06:20.935>filter <00:06:21.043>and <00:06:21.150>constraints <00:06:21.258>such <00:06:21.366>as <00:06:21.473>where <00:06:21.581>they <00:06:21.689>are <00:06:21.796>traveling, <00:06:21.904>when <00:06:22.012>they <00:06:22.120>are 00:06:22.430 --> 00:06:22.440 align:start position:0% <00:06:22.430>where <00:06:22.431>they <00:06:22.433>are <00:06:22.434>traveling, <00:06:22.436>when <00:06:22.437>they <00:06:22.439>are 00:06:22.440 --> 00:06:23.830 align:start position:0% <00:06:22.440>where <00:06:22.544>they <00:06:22.648>are <00:06:22.752>traveling, <00:06:22.857>when <00:06:22.961>they <00:06:23.065>are <00:06:23.169>traveling, <00:06:23.274>and <00:06:23.378>who <00:06:23.482>they <00:06:23.586>are <00:06:23.691>traveling 00:06:23.830 --> 00:06:23.840 align:start position:0% <00:06:23.830>traveling, <00:06:23.831>and <00:06:23.833>who <00:06:23.835>they <00:06:23.837>are <00:06:23.839>traveling 00:06:23.840 --> 00:06:25.190 align:start position:0% <00:06:23.840>traveling, <00:06:24.042>and <00:06:24.245>who <00:06:24.447>they <00:06:24.650>are <00:06:24.852>traveling <00:06:25.055>with. 00:06:25.190 --> 00:06:25.200 align:start position:0% with. 00:06:25.200 --> 00:06:26.510 align:start position:0% <00:06:25.200>with. <00:06:25.368>We <00:06:25.536>also <00:06:25.705>have <00:06:25.873>kind <00:06:26.042>of <00:06:26.210>like <00:06:26.379>the 00:06:26.510 --> 00:06:26.520 align:start position:0% <00:06:26.510>We <00:06:26.511>also <00:06:26.513>have <00:06:26.514>kind <00:06:26.515>of <00:06:26.517>like <00:06:26.519>the 00:06:26.520 --> 00:06:28.750 align:start position:0% <00:06:26.520>We <00:06:26.659>also <00:06:26.799>have <00:06:26.940>kind <00:06:27.080>of <00:06:27.219>like <00:06:27.359>the <00:06:27.500>preferences <00:06:27.640>of <00:06:27.780>our <00:06:27.919>travelers. 00:06:28.750 --> 00:06:28.760 align:start position:0% <00:06:28.750>preferences <00:06:28.753>of <00:06:28.756>our <00:06:28.759>travelers. 00:06:28.760 --> 00:06:30.390 align:start position:0% <00:06:28.760>preferences <00:06:28.887>of <00:06:29.014>our <00:06:29.141>travelers. <00:06:29.269>They <00:06:29.396>want <00:06:29.523>to <00:06:29.650>see <00:06:29.778>the <00:06:29.905>volcano, <00:06:30.032>and <00:06:30.159>they 00:06:30.390 --> 00:06:30.400 align:start position:0% <00:06:30.390>They <00:06:30.391>want <00:06:30.392>to <00:06:30.393>see <00:06:30.395>the <00:06:30.396>volcano, <00:06:30.397>and <00:06:30.399>they 00:06:30.400 --> 00:06:31.470 align:start position:0% <00:06:30.400>They <00:06:30.474>want <00:06:30.548>to <00:06:30.622>see <00:06:30.696>the <00:06:30.770>volcano, <00:06:30.844>and <00:06:30.918>they <00:06:30.992>want <00:06:31.066>to <00:06:31.140>have <00:06:31.214>something <00:06:31.288>suitable <00:06:31.363>for 00:06:31.470 --> 00:06:31.480 align:start position:0% <00:06:31.470>want <00:06:31.471>to <00:06:31.473>have <00:06:31.475>something <00:06:31.477>suitable <00:06:31.479>for 00:06:31.480 --> 00:06:33.110 align:start position:0% <00:06:31.480>want <00:06:31.713>to <00:06:31.946>have <00:06:32.180>something <00:06:32.413>suitable <00:06:32.646>for <00:06:32.880>kids. 00:06:33.110 --> 00:06:33.120 align:start position:0% kids. 00:06:33.120 --> 00:06:34.790 align:start position:0% <00:06:33.120>kids. <00:06:33.295>At <00:06:33.470>the <00:06:33.645>same <00:06:33.820>time, <00:06:33.995>they <00:06:34.170>also <00:06:34.345>are <00:06:34.520>telling 00:06:34.790 --> 00:06:34.800 align:start position:0% <00:06:34.790>At <00:06:34.791>the <00:06:34.792>same <00:06:34.793>time, <00:06:34.795>they <00:06:34.796>also <00:06:34.797>are <00:06:34.799>telling 00:06:34.800 --> 00:06:36.230 align:start position:0% <00:06:34.800>At <00:06:34.885>the <00:06:34.971>same <00:06:35.057>time, <00:06:35.143>they <00:06:35.229>also <00:06:35.314>are <00:06:35.400>telling <00:06:35.486>us <00:06:35.572>what <00:06:35.658>they <00:06:35.743>don't <00:06:35.829>really <00:06:35.915>want <00:06:36.001>to <00:06:36.087>do 00:06:36.230 --> 00:06:36.240 align:start position:0% <00:06:36.230>us <00:06:36.231>what <00:06:36.232>they <00:06:36.233>don't <00:06:36.235>really <00:06:36.236>want <00:06:36.237>to <00:06:36.239>do 00:06:36.240 --> 00:06:38.990 align:start position:0% <00:06:36.240>us <00:06:36.347>what <00:06:36.455>they <00:06:36.563>don't <00:06:36.670>really <00:06:36.778>want <00:06:36.886>to <00:06:36.993>do <00:06:37.101>such <00:06:37.209>as <00:06:37.316>exploring <00:06:37.424>the <00:06:37.532>cable <00:06:37.640>car. 00:06:38.990 --> 00:06:39.000 align:start position:0% <00:06:38.990>such <00:06:38.991>as <00:06:38.993>exploring <00:06:38.995>the <00:06:38.997>cable <00:06:38.999>car. 00:06:39.000 --> 00:06:40.990 align:start position:0% <00:06:39.000>such <00:06:39.127>as <00:06:39.254>exploring <00:06:39.381>the <00:06:39.509>cable <00:06:39.636>car. <00:06:39.763>And <00:06:39.890>finally, <00:06:40.018>they're <00:06:40.145>also <00:06:40.272>telling <00:06:40.400>us 00:06:40.990 --> 00:06:41.000 align:start position:0% <00:06:40.990>And <00:06:40.991>finally, <00:06:40.993>they're <00:06:40.995>also <00:06:40.997>telling <00:06:40.999>us 00:06:41.000 --> 00:06:42.390 align:start position:0% <00:06:41.000>And <00:06:41.156>finally, <00:06:41.312>they're <00:06:41.469>also <00:06:41.625>telling <00:06:41.781>us <00:06:41.938>kind <00:06:42.094>of <00:06:42.251>like 00:06:42.390 --> 00:06:42.400 align:start position:0% <00:06:42.390>kind <00:06:42.394>of <00:06:42.399>like 00:06:42.400 --> 00:06:43.630 align:start position:0% <00:06:42.400>kind <00:06:42.538>of <00:06:42.676>like <00:06:42.815>that <00:06:42.953>they <00:06:43.091>are <00:06:43.230>not <00:06:43.368>really <00:06:43.507>specifically 00:06:43.630 --> 00:06:43.640 align:start position:0% <00:06:43.630>that <00:06:43.631>they <00:06:43.633>are <00:06:43.635>not <00:06:43.637>really <00:06:43.639>specifically 00:06:43.640 --> 00:06:45.750 align:start position:0% <00:06:43.640>that <00:06:43.767>they <00:06:43.894>are <00:06:44.021>not <00:06:44.149>really <00:06:44.276>specifically <00:06:44.403>looking <00:06:44.530>for <00:06:44.658>something, <00:06:44.785>but <00:06:44.912>more <00:06:45.039>they're 00:06:45.750 --> 00:06:45.760 align:start position:0% <00:06:45.750>looking <00:06:45.751>for <00:06:45.753>something, <00:06:45.755>but <00:06:45.757>more <00:06:45.759>they're 00:06:45.760 --> 00:06:47.950 align:start position:0% <00:06:45.760>looking <00:06:45.887>for <00:06:46.014>something, <00:06:46.141>but <00:06:46.269>more <00:06:46.396>they're <00:06:46.523>open <00:06:46.650>for <00:06:46.778>recommendations, <00:06:46.905>and <00:06:47.032>they <00:06:47.159>want 00:06:47.950 --> 00:06:47.960 align:start position:0% <00:06:47.950>open <00:06:47.951>for <00:06:47.953>recommendations, <00:06:47.955>and <00:06:47.957>they <00:06:47.959>want 00:06:47.960 --> 00:06:49.390 align:start position:0% <00:06:47.960>open <00:06:48.067>for <00:06:48.174>recommendations, <00:06:48.281>and <00:06:48.389>they <00:06:48.496>want <00:06:48.603>to <00:06:48.710>have <00:06:48.818>something <00:06:48.925>that <00:06:49.032>is <00:06:49.139>kind <00:06:49.246>of 00:06:49.390 --> 00:06:49.400 align:start position:0% <00:06:49.390>to <00:06:49.391>have <00:06:49.393>something <00:06:49.394>that <00:06:49.395>is <00:06:49.397>kind <00:06:49.399>of 00:06:49.400 --> 00:06:51.350 align:start position:0% <00:06:49.400>to <00:06:49.555>have <00:06:49.711>something <00:06:49.866>that <00:06:50.022>is <00:06:50.177>kind <00:06:50.333>of <00:06:50.488>related <00:06:50.644>to <00:06:50.799>nature. 00:06:51.350 --> 00:06:51.360 align:start position:0% <00:06:51.350>related <00:06:51.354>to <00:06:51.359>nature. 00:06:51.360 --> 00:06:53.390 align:start position:0% <00:06:51.360>related <00:06:51.500>to <00:06:51.640>nature. <00:06:51.780>And <00:06:51.920>this <00:06:52.060>type <00:06:52.200>of <00:06:52.340>queries <00:06:52.480>are <00:06:52.620>kind <00:06:52.760>of 00:06:53.390 --> 00:06:53.400 align:start position:0% <00:06:53.390>And <00:06:53.391>this <00:06:53.392>type <00:06:53.393>of <00:06:53.395>queries <00:06:53.396>are <00:06:53.397>kind <00:06:53.399>of 00:06:53.400 --> 00:06:55.070 align:start position:0% <00:06:53.400>And <00:06:53.527>this <00:06:53.654>type <00:06:53.781>of <00:06:53.909>queries <00:06:54.036>are <00:06:54.163>kind <00:06:54.290>of <00:06:54.418>like <00:06:54.545>different <00:06:54.672>from <00:06:54.799>traditional 00:06:55.070 --> 00:06:55.080 align:start position:0% <00:06:55.070>like <00:06:55.073>different <00:06:55.075>from <00:06:55.079>traditional 00:06:55.080 --> 00:06:56.870 align:start position:0% <00:06:55.080>like <00:06:55.255>different <00:06:55.430>from <00:06:55.604>traditional <00:06:55.780>discovery <00:06:55.955>experiences. <00:06:56.130>If <00:06:56.305>you <00:06:56.479>think 00:06:56.870 --> 00:06:56.880 align:start position:0% <00:06:56.870>discovery <00:06:56.872>experiences. <00:06:56.874>If <00:06:56.876>you <00:06:56.879>think 00:06:56.880 --> 00:06:59.510 align:start position:0% <00:06:56.880>discovery <00:06:57.035>experiences. <00:06:57.191>If <00:06:57.346>you <00:06:57.502>think <00:06:57.657>about <00:06:57.813>it, <00:06:57.968>um <00:06:58.124>usually <00:06:58.280>traditional 00:06:59.510 --> 00:06:59.520 align:start position:0% <00:06:59.510>about <00:06:59.512>it, <00:06:59.514>um <00:06:59.516>usually <00:06:59.519>traditional 00:06:59.520 --> 00:07:01.270 align:start position:0% <00:06:59.520>about <00:06:59.675>it, <00:06:59.831>um <00:06:59.986>usually <00:07:00.142>traditional <00:07:00.297>discovery <00:07:00.453>experiences <00:07:00.608>are <00:07:00.764>on <00:07:00.919>keyword 00:07:01.270 --> 00:07:01.280 align:start position:0% <00:07:01.270>discovery <00:07:01.272>experiences <00:07:01.274>are <00:07:01.276>on <00:07:01.279>keyword 00:07:01.280 --> 00:07:03.670 align:start position:0% <00:07:01.280>discovery <00:07:01.407>experiences <00:07:01.534>are <00:07:01.661>on <00:07:01.789>keyword <00:07:01.916>base <00:07:02.043>and <00:07:02.170>giving <00:07:02.298>the <00:07:02.425>travelers <00:07:02.552>or <00:07:02.679>the 00:07:03.670 --> 00:07:03.680 align:start position:0% <00:07:03.670>base <00:07:03.671>and <00:07:03.673>giving <00:07:03.674>the <00:07:03.676>travelers <00:07:03.677>or <00:07:03.679>the 00:07:03.680 --> 00:07:06.430 align:start position:0% <00:07:03.680>base <00:07:03.780>and <00:07:03.880>giving <00:07:03.980>the <00:07:04.080>travelers <00:07:04.180>or <00:07:04.280>the <00:07:04.380>customer <00:07:04.480>on <00:07:04.580>what <00:07:04.680>they <00:07:04.780>might <00:07:04.880>want <00:07:04.980>to <00:07:05.080>do. 00:07:06.430 --> 00:07:06.440 align:start position:0% <00:07:06.430>customer <00:07:06.431>on <00:07:06.432>what <00:07:06.433>they <00:07:06.435>might <00:07:06.436>want <00:07:06.437>to <00:07:06.439>do. 00:07:06.440 --> 00:07:08.110 align:start position:0% <00:07:06.440>customer <00:07:06.540>on <00:07:06.640>what <00:07:06.740>they <00:07:06.840>might <00:07:06.940>want <00:07:07.040>to <00:07:07.140>do. <00:07:07.240>But <00:07:07.340>in <00:07:07.440>conversational <00:07:07.540>AI, <00:07:07.640>we <00:07:07.740>really <00:07:07.840>need 00:07:08.110 --> 00:07:08.120 align:start position:0% <00:07:08.110>But <00:07:08.111>in <00:07:08.113>conversational <00:07:08.114>AI, <00:07:08.116>we <00:07:08.117>really <00:07:08.119>need 00:07:08.120 --> 00:07:10.270 align:start position:0% <00:07:08.120>But <00:07:08.220>in <00:07:08.320>conversational <00:07:08.420>AI, <00:07:08.520>we <00:07:08.620>really <00:07:08.720>need <00:07:08.820>to <00:07:08.920>raise <00:07:09.020>the <00:07:09.120>bar <00:07:09.220>in <00:07:09.320>terms <00:07:09.420>of <00:07:09.520>relevance, 00:07:10.270 --> 00:07:10.280 align:start position:0% <00:07:10.270>to <00:07:10.271>raise <00:07:10.272>the <00:07:10.273>bar <00:07:10.275>in <00:07:10.276>terms <00:07:10.277>of <00:07:10.279>relevance, 00:07:10.280 --> 00:07:12.270 align:start position:0% <00:07:10.280>to <00:07:10.396>raise <00:07:10.513>the <00:07:10.630>bar <00:07:10.746>in <00:07:10.863>terms <00:07:10.979>of <00:07:11.096>relevance, <00:07:11.213>understand <00:07:11.330>the <00:07:11.446>traveler's <00:07:11.563>intent, <00:07:11.679>and 00:07:12.270 --> 00:07:12.280 align:start position:0% <00:07:12.270>understand <00:07:12.272>the <00:07:12.274>traveler's <00:07:12.276>intent, <00:07:12.279>and 00:07:12.280 --> 00:07:15.110 align:start position:0% <00:07:12.280>understand <00:07:12.407>the <00:07:12.534>traveler's <00:07:12.661>intent, <00:07:12.789>and <00:07:12.916>true <00:07:13.043>give <00:07:13.170>them <00:07:13.298>truly <00:07:13.425>few <00:07:13.552>truly <00:07:13.679>relevant 00:07:15.110 --> 00:07:15.120 align:start position:0% <00:07:15.110>true <00:07:15.111>give <00:07:15.113>them <00:07:15.114>truly <00:07:15.116>few <00:07:15.117>truly <00:07:15.119>relevant 00:07:15.120 --> 00:07:18.070 align:start position:0% <00:07:15.120>true <00:07:15.236>give <00:07:15.353>them <00:07:15.470>truly <00:07:15.586>few <00:07:15.703>truly <00:07:15.820>relevant <00:07:15.936>experiences <00:07:16.053>that <00:07:16.170>they <00:07:16.286>want <00:07:16.403>to <00:07:16.520>do. 00:07:18.070 --> 00:07:18.080 align:start position:0% <00:07:18.070>experiences <00:07:18.071>that <00:07:18.073>they <00:07:18.075>want <00:07:18.077>to <00:07:18.079>do. 00:07:18.080 --> 00:07:19.550 align:start position:0% <00:07:18.080>experiences <00:07:18.190>that <00:07:18.300>they <00:07:18.410>want <00:07:18.521>to <00:07:18.631>do. <00:07:18.741>And <00:07:18.851>at <00:07:18.962>the <00:07:19.072>beginning, <00:07:19.182>we <00:07:19.292>were <00:07:19.403>thinking 00:07:19.550 --> 00:07:19.560 align:start position:0% <00:07:19.550>And <00:07:19.551>at <00:07:19.553>the <00:07:19.554>beginning, <00:07:19.556>we <00:07:19.557>were <00:07:19.559>thinking 00:07:19.560 --> 00:07:21.030 align:start position:0% <00:07:19.560>And <00:07:19.654>at <00:07:19.749>the <00:07:19.843>beginning, <00:07:19.938>we <00:07:20.032>were <00:07:20.127>thinking <00:07:20.221>about <00:07:20.316>doing <00:07:20.410>it <00:07:20.505>in <00:07:20.599>kind <00:07:20.693>of <00:07:20.788>like <00:07:20.883>this 00:07:21.030 --> 00:07:21.040 align:start position:0% <00:07:21.030>about <00:07:21.031>doing <00:07:21.032>it <00:07:21.033>in <00:07:21.035>kind <00:07:21.036>of <00:07:21.037>like <00:07:21.039>this 00:07:21.040 --> 00:07:23.550 align:start position:0% <00:07:21.040>about <00:07:21.140>doing <00:07:21.240>it <00:07:21.340>in <00:07:21.440>kind <00:07:21.540>of <00:07:21.640>like <00:07:21.740>this <00:07:21.840>wrapper <00:07:21.940>way <00:07:22.040>where <00:07:22.140>hey, <00:07:22.240>let's <00:07:22.340>just <00:07:22.440>expose 00:07:23.550 --> 00:07:23.560 align:start position:0% <00:07:23.550>wrapper <00:07:23.551>way <00:07:23.553>where <00:07:23.554>hey, <00:07:23.556>let's <00:07:23.557>just <00:07:23.559>expose 00:07:23.560 --> 00:07:25.710 align:start position:0% <00:07:23.560>wrapper <00:07:23.660>way <00:07:23.760>where <00:07:23.860>hey, <00:07:23.960>let's <00:07:24.060>just <00:07:24.160>expose <00:07:24.260>strategy <00:07:24.360>PT <00:07:24.460>kind <00:07:24.560>of <00:07:24.660>like <00:07:24.760>our <00:07:24.860>APIs <00:07:24.960>as 00:07:25.710 --> 00:07:25.720 align:start position:0% <00:07:25.710>strategy <00:07:25.711>PT <00:07:25.712>kind <00:07:25.713>of <00:07:25.715>like <00:07:25.716>our <00:07:25.717>APIs <00:07:25.719>as 00:07:25.720 --> 00:07:28.070 align:start position:0% <00:07:25.720>strategy <00:07:25.820>PT <00:07:25.920>kind <00:07:26.020>of <00:07:26.120>like <00:07:26.220>our <00:07:26.320>APIs <00:07:26.420>as <00:07:26.520>tools. <00:07:26.620>Similarly <00:07:26.720>also <00:07:26.820>of <00:07:26.920>you <00:07:27.020>would <00:07:27.120>just 00:07:28.070 --> 00:07:28.080 align:start position:0% <00:07:28.070>tools. <00:07:28.071>Similarly <00:07:28.073>also <00:07:28.074>of <00:07:28.075>you <00:07:28.077>would <00:07:28.079>just 00:07:28.080 --> 00:07:30.390 align:start position:0% <00:07:28.080>tools. <00:07:28.207>Similarly <00:07:28.334>also <00:07:28.461>of <00:07:28.589>you <00:07:28.716>would <00:07:28.843>just <00:07:28.970>better <00:07:29.098>MCP <00:07:29.225>for <00:07:29.352>strategy <00:07:29.479>PT. 00:07:30.390 --> 00:07:30.400 align:start position:0% <00:07:30.390>better <00:07:30.392>MCP <00:07:30.394>for <00:07:30.396>strategy <00:07:30.399>PT. 00:07:30.400 --> 00:07:32.230 align:start position:0% <00:07:30.400>better <00:07:30.516>MCP <00:07:30.633>for <00:07:30.750>strategy <00:07:30.866>PT. <00:07:30.983>And <00:07:31.099>then <00:07:31.216>let <00:07:31.333>the <00:07:31.450>model <00:07:31.566>or <00:07:31.683>strategy <00:07:31.799>PT 00:07:32.230 --> 00:07:32.240 align:start position:0% <00:07:32.230>And <00:07:32.231>then <00:07:32.232>let <00:07:32.233>the <00:07:32.235>model <00:07:32.236>or <00:07:32.237>strategy <00:07:32.239>PT 00:07:32.240 --> 00:07:33.710 align:start position:0% <00:07:32.240>And <00:07:32.360>then <00:07:32.480>let <00:07:32.600>the <00:07:32.721>model <00:07:32.841>or <00:07:32.961>strategy <00:07:33.081>PT <00:07:33.202>the <00:07:33.322>platform <00:07:33.442>itself <00:07:33.563>orchestrate 00:07:33.710 --> 00:07:33.720 align:start position:0% <00:07:33.710>the <00:07:33.712>platform <00:07:33.716>itself <00:07:33.719>orchestrate 00:07:33.720 --> 00:07:34.909 align:start position:0% <00:07:33.720>the <00:07:33.987>platform <00:07:34.255>itself <00:07:34.522>orchestrate <00:07:34.790>everything. 00:07:34.909 --> 00:07:34.919 align:start position:0% everything. 00:07:34.919 --> 00:07:36.870 align:start position:0% <00:07:34.919>everything. <00:07:35.074>And <00:07:35.230>to <00:07:35.385>be <00:07:35.541>honest, <00:07:35.696>it <00:07:35.852>is <00:07:36.007>quite <00:07:36.163>fast <00:07:36.318>and 00:07:36.870 --> 00:07:36.880 align:start position:0% <00:07:36.870>And <00:07:36.871>to <00:07:36.872>be <00:07:36.873>honest, <00:07:36.874>it <00:07:36.875>is <00:07:36.876>quite <00:07:36.877>fast <00:07:36.879>and 00:07:36.880 --> 00:07:38.110 align:start position:0% <00:07:36.880>And <00:07:37.003>to <00:07:37.126>be <00:07:37.249>honest, <00:07:37.372>it <00:07:37.495>is <00:07:37.618>quite <00:07:37.741>fast <00:07:37.864>and <00:07:37.987>simple. 00:07:38.110 --> 00:07:38.120 align:start position:0% simple. 00:07:38.120 --> 00:07:39.750 align:start position:0% <00:07:38.120>simple. <00:07:38.295>But <00:07:38.470>then <00:07:38.645>you <00:07:38.820>start <00:07:38.995>to <00:07:39.170>think <00:07:39.345>about <00:07:39.520>it, 00:07:39.750 --> 00:07:39.760 align:start position:0% <00:07:39.750>But <00:07:39.751>then <00:07:39.752>you <00:07:39.753>start <00:07:39.755>to <00:07:39.756>think <00:07:39.757>about <00:07:39.759>it, 00:07:39.760 --> 00:07:41.830 align:start position:0% <00:07:39.760>But <00:07:39.853>then <00:07:39.946>you <00:07:40.039>start <00:07:40.133>to <00:07:40.226>think <00:07:40.320>about <00:07:40.413>it, <00:07:40.506>and <00:07:40.599>you <00:07:40.693>start <00:07:40.786>to <00:07:40.880>realize <00:07:40.973>that <00:07:41.066>hey, <00:07:41.159>you 00:07:41.830 --> 00:07:41.840 align:start position:0% <00:07:41.830>and <00:07:41.831>you <00:07:41.832>start <00:07:41.833>to <00:07:41.835>realize <00:07:41.836>that <00:07:41.837>hey, <00:07:41.839>you 00:07:41.840 --> 00:07:44.070 align:start position:0% <00:07:41.840>and <00:07:41.940>you <00:07:42.040>start <00:07:42.140>to <00:07:42.240>realize <00:07:42.340>that <00:07:42.440>hey, <00:07:42.540>you <00:07:42.640>can <00:07:42.740>have <00:07:42.840>potential <00:07:42.940>of <00:07:43.040>data <00:07:43.140>leakage. <00:07:43.240>In 00:07:44.070 --> 00:07:44.080 align:start position:0% <00:07:44.070>can <00:07:44.071>have <00:07:44.073>potential <00:07:44.074>of <00:07:44.075>data <00:07:44.077>leakage. <00:07:44.079>In 00:07:44.080 --> 00:07:45.910 align:start position:0% <00:07:44.080>can <00:07:44.196>have <00:07:44.313>potential <00:07:44.430>of <00:07:44.546>data <00:07:44.663>leakage. <00:07:44.780>In <00:07:44.896>order <00:07:45.013>to <00:07:45.130>make <00:07:45.246>the <00:07:45.363>experience <00:07:45.479>really 00:07:45.910 --> 00:07:45.920 align:start position:0% <00:07:45.909>order <00:07:45.911>to <00:07:45.913>make <00:07:45.915>the <00:07:45.917>experience <00:07:45.919>really 00:07:45.920 --> 00:07:47.830 align:start position:0% <00:07:45.920>order <00:07:46.047>to <00:07:46.174>make <00:07:46.301>the <00:07:46.429>experience <00:07:46.556>really <00:07:46.683>great <00:07:46.810>for <00:07:46.938>your <00:07:47.065>customers, <00:07:47.192>you <00:07:47.320>really 00:07:47.830 --> 00:07:47.840 align:start position:0% <00:07:47.830>great <00:07:47.831>for <00:07:47.833>your <00:07:47.835>customers, <00:07:47.837>you <00:07:47.839>really 00:07:47.840 --> 00:07:49.630 align:start position:0% <00:07:47.840>great <00:07:47.940>for <00:07:48.040>your <00:07:48.140>customers, <00:07:48.240>you <00:07:48.340>really <00:07:48.440>need <00:07:48.540>to <00:07:48.640>give <00:07:48.740>ChatGPT <00:07:48.840>a <00:07:48.940>lot <00:07:49.040>of <00:07:49.140>context <00:07:49.240>in 00:07:49.630 --> 00:07:49.640 align:start position:0% <00:07:49.630>need <00:07:49.631>to <00:07:49.632>give <00:07:49.633>ChatGPT <00:07:49.634>a <00:07:49.635>lot <00:07:49.636>of <00:07:49.637>context <00:07:49.639>in 00:07:49.640 --> 00:07:53.510 align:start position:0% <00:07:49.640>need <00:07:49.747>to <00:07:49.855>give <00:07:49.963>ChatGPT <00:07:50.070>a <00:07:50.178>lot <00:07:50.286>of <00:07:50.393>context <00:07:50.501>in <00:07:50.609>order <00:07:50.716>to <00:07:50.824>make <00:07:50.932>a <00:07:51.039>judgment. 00:07:53.520 --> 00:07:54.990 align:start position:0% <00:07:53.520>The <00:07:53.740>user <00:07:53.961>experience <00:07:54.181>is <00:07:54.402>also <00:07:54.622>not <00:07:54.843>that 00:07:54.990 --> 00:07:55.000 align:start position:0% <00:07:54.990>The <00:07:54.991>user <00:07:54.993>experience <00:07:54.994>is <00:07:54.996>also <00:07:54.997>not <00:07:54.999>that 00:07:55.000 --> 00:07:57.270 align:start position:0% <00:07:55.000>The <00:07:55.100>user <00:07:55.200>experience <00:07:55.300>is <00:07:55.400>also <00:07:55.500>not <00:07:55.600>that <00:07:55.700>great <00:07:55.800>because <00:07:55.900>you <00:07:56.000>end <00:07:56.100>up <00:07:56.200>having <00:07:56.300>many <00:07:56.400>and 00:07:57.270 --> 00:07:57.280 align:start position:0% <00:07:57.270>great <00:07:57.271>because <00:07:57.272>you <00:07:57.273>end <00:07:57.275>up <00:07:57.276>having <00:07:57.277>many <00:07:57.279>and 00:07:57.280 --> 00:07:59.150 align:start position:0% <00:07:57.280>great <00:07:57.419>because <00:07:57.559>you <00:07:57.700>end <00:07:57.840>up <00:07:57.979>having <00:07:58.119>many <00:07:58.260>and <00:07:58.400>slow <00:07:58.539>tool <00:07:58.679>calls. 00:07:59.150 --> 00:07:59.160 align:start position:0% <00:07:59.150>slow <00:07:59.154>tool <00:07:59.159>calls. 00:07:59.160 --> 00:08:01.190 align:start position:0% <00:07:59.159>slow <00:07:59.287>tool <00:07:59.414>calls. <00:07:59.541>So, <00:07:59.669>the <00:07:59.796>user <00:07:59.923>ends <00:08:00.050>up <00:08:00.178>waiting <00:08:00.305>in <00:08:00.432>kind <00:08:00.559>of 00:08:01.190 --> 00:08:01.200 align:start position:0% <00:08:01.190>So, <00:08:01.191>the <00:08:01.192>user <00:08:01.193>ends <00:08:01.194>up <00:08:01.195>waiting <00:08:01.196>in <00:08:01.197>kind <00:08:01.199>of 00:08:01.200 --> 00:08:03.990 align:start position:0% <00:08:01.200>So, <00:08:01.316>the <00:08:01.433>user <00:08:01.550>ends <00:08:01.666>up <00:08:01.783>waiting <00:08:01.900>in <00:08:02.016>kind <00:08:02.133>of <00:08:02.250>like <00:08:02.366>this <00:08:02.483>agentic <00:08:02.599>loop. 00:08:03.990 --> 00:08:04.000 align:start position:0% <00:08:03.990>like <00:08:03.993>this <00:08:03.996>agentic <00:08:03.999>loop. 00:08:04.000 --> 00:08:06.190 align:start position:0% <00:08:04.000>like <00:08:04.140>this <00:08:04.280>agentic <00:08:04.420>loop. <00:08:04.560>And <00:08:04.700>finally, <00:08:04.840>we <00:08:04.980>don't <00:08:05.120>have <00:08:05.260>any <00:08:05.400>control. 00:08:06.190 --> 00:08:06.200 align:start position:0% <00:08:06.190>And <00:08:06.191>finally, <00:08:06.193>we <00:08:06.194>don't <00:08:06.195>have <00:08:06.197>any <00:08:06.199>control. 00:08:06.200 --> 00:08:08.390 align:start position:0% <00:08:06.200>And <00:08:06.307>finally, <00:08:06.415>we <00:08:06.523>don't <00:08:06.630>have <00:08:06.738>any <00:08:06.846>control. <00:08:06.953>We <00:08:07.061>cannot <00:08:07.169>really <00:08:07.276>modify <00:08:07.384>or <00:08:07.492>adjust <00:08:07.599>how 00:08:08.390 --> 00:08:08.400 align:start position:0% <00:08:08.390>We <00:08:08.391>cannot <00:08:08.393>really <00:08:08.394>modify <00:08:08.395>or <00:08:08.397>adjust <00:08:08.399>how 00:08:08.400 --> 00:08:10.870 align:start position:0% <00:08:08.400>We <00:08:08.516>cannot <00:08:08.633>really <00:08:08.750>modify <00:08:08.866>or <00:08:08.983>adjust <00:08:09.099>how <00:08:09.216>ChatGPT <00:08:09.333>is <00:08:09.450>behaving. <00:08:09.566>So, <00:08:09.683>ChatGPT <00:08:09.799>is 00:08:10.870 --> 00:08:10.880 align:start position:0% <00:08:10.870>ChatGPT <00:08:10.871>is <00:08:10.873>behaving. <00:08:10.875>So, <00:08:10.877>ChatGPT <00:08:10.879>is 00:08:10.880 --> 00:08:12.270 align:start position:0% <00:08:10.880>ChatGPT <00:08:11.058>is <00:08:11.237>behaving. <00:08:11.416>So, <00:08:11.594>ChatGPT <00:08:11.773>is <00:08:11.952>doing <00:08:12.131>everything 00:08:12.270 --> 00:08:12.280 align:start position:0% <00:08:12.270>doing <00:08:12.279>everything 00:08:12.280 --> 00:08:14.510 align:start position:0% <00:08:12.280>doing <00:08:12.479>everything <00:08:12.679>how <00:08:12.880>they <00:08:13.080>want <00:08:13.280>to <00:08:13.479>do <00:08:13.679>it. 00:08:14.510 --> 00:08:14.520 align:start position:0% <00:08:14.510>how <00:08:14.511>they <00:08:14.513>want <00:08:14.515>to <00:08:14.517>do <00:08:14.519>it. 00:08:14.520 --> 00:08:16.430 align:start position:0% <00:08:14.520>how <00:08:14.636>they <00:08:14.753>want <00:08:14.870>to <00:08:14.986>do <00:08:15.103>it. <00:08:15.219>So, <00:08:15.336>that's <00:08:15.453>why <00:08:15.570>we <00:08:15.686>essentially <00:08:15.803>decided <00:08:15.919>to 00:08:16.430 --> 00:08:16.440 align:start position:0% <00:08:16.430>So, <00:08:16.431>that's <00:08:16.433>why <00:08:16.434>we <00:08:16.436>essentially <00:08:16.437>decided <00:08:16.439>to 00:08:16.440 --> 00:08:19.110 align:start position:0% <00:08:16.440>So, <00:08:16.547>that's <00:08:16.655>why <00:08:16.763>we <00:08:16.870>essentially <00:08:16.978>decided <00:08:17.086>to <00:08:17.193>kind <00:08:17.301>of <00:08:17.409>flip <00:08:17.516>the <00:08:17.624>relationship <00:08:17.732>and <00:08:17.840>start 00:08:19.110 --> 00:08:19.120 align:start position:0% <00:08:19.110>kind <00:08:19.111>of <00:08:19.113>flip <00:08:19.114>the <00:08:19.116>relationship <00:08:19.117>and <00:08:19.119>start 00:08:19.120 --> 00:08:21.070 align:start position:0% <00:08:19.120>kind <00:08:19.207>of <00:08:19.295>flip <00:08:19.382>the <00:08:19.470>relationship <00:08:19.557>and <00:08:19.645>start <00:08:19.732>to <00:08:19.820>do <00:08:19.907>it <00:08:19.995>in <00:08:20.082>a <00:08:20.170>more <00:08:20.257>agentic <00:08:20.345>way, <00:08:20.432>where <00:08:20.520>we 00:08:21.070 --> 00:08:21.080 align:start position:0% <00:08:21.070>to <00:08:21.070>do <00:08:21.072>it <00:08:21.073>in <00:08:21.074>a <00:08:21.075>more <00:08:21.075>agentic <00:08:21.077>way, <00:08:21.078>where <00:08:21.079>we 00:08:21.080 --> 00:08:23.030 align:start position:0% <00:08:21.080>to <00:08:21.162>do <00:08:21.244>it <00:08:21.327>in <00:08:21.409>a <00:08:21.491>more <00:08:21.574>agentic <00:08:21.656>way, <00:08:21.738>where <00:08:21.821>we <00:08:21.903>built <00:08:21.985>a <00:08:22.068>kind <00:08:22.150>of <00:08:22.232>like <00:08:22.315>the <00:08:22.397>agentic <00:08:22.479>loop 00:08:23.030 --> 00:08:23.040 align:start position:0% <00:08:23.030>built <00:08:23.031>a <00:08:23.032>kind <00:08:23.033>of <00:08:23.035>like <00:08:23.036>the <00:08:23.037>agentic <00:08:23.039>loop 00:08:23.040 --> 00:08:24.430 align:start position:0% <00:08:23.040>built <00:08:23.196>a <00:08:23.352>kind <00:08:23.509>of <00:08:23.665>like <00:08:23.821>the <00:08:23.978>agentic <00:08:24.134>loop <00:08:24.291>that 00:08:24.430 --> 00:08:24.440 align:start position:0% that 00:08:24.440 --> 00:08:26.710 align:start position:0% <00:08:24.440>that <00:08:24.673>orchestration <00:08:24.906>on <00:08:25.140>our <00:08:25.373>side <00:08:25.606>that <00:08:25.840>handles 00:08:26.710 --> 00:08:26.720 align:start position:0% <00:08:26.710>orchestration <00:08:26.711>on <00:08:26.713>our <00:08:26.715>side <00:08:26.717>that <00:08:26.719>handles 00:08:26.720 --> 00:08:28.230 align:start position:0% <00:08:26.720>orchestration <00:08:26.914>on <00:08:27.108>our <00:08:27.302>side <00:08:27.496>that <00:08:27.690>handles <00:08:27.884>the <00:08:28.079>state. 00:08:28.230 --> 00:08:28.240 align:start position:0% <00:08:28.230>the <00:08:28.239>state. 00:08:28.240 --> 00:08:30.590 align:start position:0% <00:08:28.240>the <00:08:28.395>state. <00:08:28.551>And <00:08:28.706>this <00:08:28.862>way, <00:08:29.017>we <00:08:29.173>can <00:08:29.328>refine <00:08:29.484>the <00:08:29.640>user 00:08:30.590 --> 00:08:30.600 align:start position:0% <00:08:30.590>And <00:08:30.591>this <00:08:30.592>way, <00:08:30.593>we <00:08:30.595>can <00:08:30.596>refine <00:08:30.597>the <00:08:30.599>user 00:08:30.600 --> 00:08:32.310 align:start position:0% <00:08:30.600>And <00:08:30.707>this <00:08:30.815>way, <00:08:30.923>we <00:08:31.030>can <00:08:31.138>refine <00:08:31.246>the <00:08:31.353>user <00:08:31.461>experiences, <00:08:31.569>right? <00:08:31.676>We <00:08:31.784>can <00:08:31.892>optimize <00:08:32.000>the 00:08:32.310 --> 00:08:32.320 align:start position:0% <00:08:32.309>experiences, <00:08:32.311>right? <00:08:32.313>We <00:08:32.315>can <00:08:32.317>optimize <00:08:32.319>the 00:08:32.320 --> 00:08:35.830 align:start position:0% <00:08:32.320>experiences, <00:08:32.436>right? <00:08:32.553>We <00:08:32.670>can <00:08:32.786>optimize <00:08:32.903>the <00:08:33.020>latency, <00:08:33.136>we <00:08:33.253>can <00:08:33.370>make <00:08:33.486>it <00:08:33.603>truly <00:08:33.720>relevant. 00:08:35.830 --> 00:08:35.840 align:start position:0% <00:08:35.830>latency, <00:08:35.831>we <00:08:35.833>can <00:08:35.834>make <00:08:35.836>it <00:08:35.837>truly <00:08:35.839>relevant. 00:08:35.840 --> 00:08:37.830 align:start position:0% <00:08:35.840>latency, <00:08:35.933>we <00:08:36.026>can <00:08:36.120>make <00:08:36.213>it <00:08:36.306>truly <00:08:36.400>relevant. <00:08:36.493>And <00:08:36.586>this <00:08:36.680>will <00:08:36.773>in <00:08:36.866>the <00:08:36.960>end <00:08:37.053>result <00:08:37.146>a <00:08:37.240>much 00:08:37.830 --> 00:08:37.840 align:start position:0% <00:08:37.830>And <00:08:37.831>this <00:08:37.832>will <00:08:37.833>in <00:08:37.834>the <00:08:37.835>end <00:08:37.836>result <00:08:37.837>a <00:08:37.839>much 00:08:37.840 --> 00:08:39.510 align:start position:0% <00:08:37.840>And <00:08:37.967>this <00:08:38.094>will <00:08:38.221>in <00:08:38.349>the <00:08:38.476>end <00:08:38.603>result <00:08:38.730>a <00:08:38.858>much <00:08:38.985>better <00:08:39.112>user <00:08:39.240>experience 00:08:39.510 --> 00:08:39.520 align:start position:0% <00:08:39.510>better <00:08:39.514>user <00:08:39.519>experience 00:08:39.520 --> 00:08:43.350 align:start position:0% <00:08:39.520>better <00:08:39.659>user <00:08:39.799>experience <00:08:39.939>than <00:08:40.079>we <00:08:40.220>were <00:08:40.360>just <00:08:40.500>be <00:08:40.640>exposing <00:08:40.780>API <00:08:40.919>calls. 00:08:43.350 --> 00:08:43.360 align:start position:0% <00:08:43.350>than <00:08:43.351>we <00:08:43.352>were <00:08:43.353>just <00:08:43.355>be <00:08:43.356>exposing <00:08:43.357>API <00:08:43.359>calls. 00:08:43.360 --> 00:08:45.670 align:start position:0% <00:08:43.360>than <00:08:43.460>we <00:08:43.560>were <00:08:43.659>just <00:08:43.760>be <00:08:43.860>exposing <00:08:43.960>API <00:08:44.060>calls. <00:08:44.159>And <00:08:44.260>lastly, <00:08:44.360>we <00:08:44.460>are <00:08:44.560>building <00:08:44.660>this <00:08:44.760>system 00:08:45.670 --> 00:08:45.680 align:start position:0% <00:08:45.670>And <00:08:45.671>lastly, <00:08:45.673>we <00:08:45.674>are <00:08:45.675>building <00:08:45.677>this <00:08:45.679>system 00:08:45.680 --> 00:08:47.590 align:start position:0% <00:08:45.680>And <00:08:45.796>lastly, <00:08:45.913>we <00:08:46.030>are <00:08:46.146>building <00:08:46.263>this <00:08:46.380>system <00:08:46.496>with <00:08:46.613>the <00:08:46.729>goal <00:08:46.846>of <00:08:46.963>gathering <00:08:47.079>data, 00:08:47.590 --> 00:08:47.600 align:start position:0% <00:08:47.590>with <00:08:47.591>the <00:08:47.593>goal <00:08:47.595>of <00:08:47.597>gathering <00:08:47.599>data, 00:08:47.600 --> 00:08:50.230 align:start position:0% <00:08:47.600>with <00:08:47.727>the <00:08:47.854>goal <00:08:47.981>of <00:08:48.109>gathering <00:08:48.236>data, <00:08:48.363>understanding <00:08:48.490>how <00:08:48.618>our <00:08:48.745>customers <00:08:48.872>would <00:08:49.000>do 00:08:50.230 --> 00:08:50.240 align:start position:0% <00:08:50.230>understanding <00:08:50.231>how <00:08:50.233>our <00:08:50.235>customers <00:08:50.237>would <00:08:50.239>do 00:08:50.240 --> 00:08:52.270 align:start position:0% <00:08:50.240>understanding <00:08:50.395>how <00:08:50.551>our <00:08:50.706>customers <00:08:50.862>would <00:08:51.017>do <00:08:51.173>conversational <00:08:51.328>travel <00:08:51.484>discovery, <00:08:51.640>and 00:08:52.270 --> 00:08:52.280 align:start position:0% <00:08:52.270>conversational <00:08:52.273>travel <00:08:52.276>discovery, <00:08:52.279>and 00:08:52.280 --> 00:08:53.030 align:start position:0% <00:08:52.280>conversational <00:08:52.448>travel <00:08:52.617>discovery, <00:08:52.786>and <00:08:52.954>then 00:08:53.030 --> 00:08:53.040 align:start position:0% then 00:08:53.040 --> 00:08:55.070 align:start position:0% <00:08:53.040>then <00:08:53.214>optimize <00:08:53.390>our <00:08:53.564>agent <00:08:53.740>to <00:08:53.915>also <00:08:54.089>be <00:08:54.265>able <00:08:54.439>to 00:08:55.070 --> 00:08:55.080 align:start position:0% <00:08:55.070>optimize <00:08:55.071>our <00:08:55.072>agent <00:08:55.073>to <00:08:55.075>also <00:08:55.076>be <00:08:55.077>able <00:08:55.079>to 00:08:55.080 --> 00:08:58.670 align:start position:0% <00:08:55.080>optimize <00:08:55.196>our <00:08:55.313>agent <00:08:55.430>to <00:08:55.546>also <00:08:55.663>be <00:08:55.780>able <00:08:55.896>to <00:08:56.013>use <00:08:56.130>it <00:08:56.246>in <00:08:56.363>our <00:08:56.480>product. 00:08:58.680 --> 00:09:01.350 align:start position:0% <00:08:58.680>So <00:08:58.854>then, <00:08:59.030>to <00:08:59.204>wrap <00:08:59.380>it <00:08:59.555>up, <00:08:59.729>this <00:08:59.905>the <00:09:00.079>summer 00:09:01.350 --> 00:09:01.360 align:start position:0% <00:09:01.350>So <00:09:01.351>then, <00:09:01.352>to <00:09:01.353>wrap <00:09:01.354>it <00:09:01.355>up, <00:09:01.356>this <00:09:01.357>the <00:09:01.359>summer 00:09:01.360 --> 00:09:03.390 align:start position:0% <00:09:01.360>So <00:09:01.447>then, <00:09:01.535>to <00:09:01.622>wrap <00:09:01.710>it <00:09:01.797>up, <00:09:01.885>this <00:09:01.972>the <00:09:02.060>summer <00:09:02.147>to <00:09:02.235>summarize <00:09:02.322>also <00:09:02.410>all <00:09:02.497>our <00:09:02.585>goals, <00:09:02.672>we <00:09:02.760>want 00:09:03.390 --> 00:09:03.400 align:start position:0% <00:09:03.390>to <00:09:03.391>summarize <00:09:03.392>also <00:09:03.393>all <00:09:03.395>our <00:09:03.396>goals, <00:09:03.397>we <00:09:03.399>want 00:09:03.400 --> 00:09:07.550 align:start position:0% <00:09:03.400>to <00:09:03.507>summarize <00:09:03.615>also <00:09:03.723>all <00:09:03.830>our <00:09:03.938>goals, <00:09:04.046>we <00:09:04.153>want <00:09:04.261>to <00:09:04.369>have <00:09:04.476>search <00:09:04.584>with <00:09:04.692>natural <00:09:04.800>languages. 00:09:07.550 --> 00:09:07.560 align:start position:0% <00:09:07.550>to <00:09:07.551>have <00:09:07.553>search <00:09:07.555>with <00:09:07.557>natural <00:09:07.559>languages. 00:09:07.560 --> 00:09:09.550 align:start position:0% <00:09:07.560>to <00:09:07.667>have <00:09:07.775>search <00:09:07.883>with <00:09:07.990>natural <00:09:08.098>languages. <00:09:08.206>We <00:09:08.313>want <00:09:08.421>to <00:09:08.529>have <00:09:08.636>much <00:09:08.744>deeper <00:09:08.852>search <00:09:08.959>than 00:09:09.550 --> 00:09:09.560 align:start position:0% <00:09:09.550>We <00:09:09.551>want <00:09:09.552>to <00:09:09.553>have <00:09:09.555>much <00:09:09.556>deeper <00:09:09.557>search <00:09:09.559>than 00:09:09.560 --> 00:09:10.950 align:start position:0% <00:09:09.560>We <00:09:09.699>want <00:09:09.838>to <00:09:09.977>have <00:09:10.116>much <00:09:10.255>deeper <00:09:10.394>search <00:09:10.533>than <00:09:10.672>ever <00:09:10.811>before. 00:09:10.950 --> 00:09:10.960 align:start position:0% <00:09:10.950>ever <00:09:10.959>before. 00:09:10.960 --> 00:09:12.550 align:start position:0% <00:09:10.960>ever <00:09:11.087>before. <00:09:11.214>And <00:09:11.341>at <00:09:11.469>same <00:09:11.596>time, <00:09:11.723>we <00:09:11.850>want <00:09:11.978>to <00:09:12.105>be <00:09:12.232>able <00:09:12.360>to 00:09:12.550 --> 00:09:12.560 align:start position:0% <00:09:12.550>And <00:09:12.550>at <00:09:12.551>same <00:09:12.553>time, <00:09:12.554>we <00:09:12.555>want <00:09:12.555>to <00:09:12.556>be <00:09:12.558>able <00:09:12.559>to 00:09:12.560 --> 00:09:14.470 align:start position:0% <00:09:12.560>And <00:09:12.653>at <00:09:12.746>same <00:09:12.839>time, <00:09:12.933>we <00:09:13.026>want <00:09:13.119>to <00:09:13.213>be <00:09:13.306>able <00:09:13.400>to <00:09:13.493>capture <00:09:13.586>the <00:09:13.680>nuances <00:09:13.773>and <00:09:13.866>preferences <00:09:13.959>of 00:09:14.470 --> 00:09:14.480 align:start position:0% <00:09:14.470>capture <00:09:14.471>the <00:09:14.473>nuances <00:09:14.475>and <00:09:14.477>preferences <00:09:14.479>of 00:09:14.480 --> 00:09:17.350 align:start position:0% <00:09:14.480>capture <00:09:14.680>the <00:09:14.880>nuances <00:09:15.080>and <00:09:15.280>preferences <00:09:15.480>of <00:09:15.680>our <00:09:15.880>customers 00:09:17.350 --> 00:09:17.360 align:start position:0% <00:09:17.350>our <00:09:17.359>customers 00:09:17.360 --> 00:09:19.470 align:start position:0% <00:09:17.360>our <00:09:17.593>customers <00:09:17.826>while <00:09:18.060>keeping <00:09:18.293>actually <00:09:18.526>the <00:09:18.760>retrieval 00:09:19.470 --> 00:09:19.480 align:start position:0% <00:09:19.470>while <00:09:19.472>keeping <00:09:19.474>actually <00:09:19.476>the <00:09:19.479>retrieval 00:09:19.480 --> 00:09:21.630 align:start position:0% <00:09:19.480>while <00:09:19.596>keeping <00:09:19.713>actually <00:09:19.830>the <00:09:19.946>retrieval <00:09:20.063>within <00:09:20.180>10 <00:09:20.296>to <00:09:20.413>15 <00:09:20.530>seconds <00:09:20.646>at <00:09:20.763>a <00:09:20.880>reasonable 00:09:21.630 --> 00:09:21.640 align:start position:0% <00:09:21.630>within <00:09:21.631>10 <00:09:21.632>to <00:09:21.633>15 <00:09:21.635>seconds <00:09:21.636>at <00:09:21.637>a <00:09:21.639>reasonable 00:09:21.640 --> 00:09:23.230 align:start position:0% <00:09:21.640>within <00:09:21.814>10 <00:09:21.990>to <00:09:22.165>15 <00:09:22.340>seconds <00:09:22.515>at <00:09:22.689>a <00:09:22.865>reasonable <00:09:23.040>cost. 00:09:23.230 --> 00:09:23.240 align:start position:0% cost. 00:09:23.240 --> 00:09:25.270 align:start position:0% <00:09:23.240>cost. <00:09:23.415>And <00:09:23.590>finally, <00:09:23.765>our <00:09:23.940>last <00:09:24.115>goal <00:09:24.290>is <00:09:24.465>to <00:09:24.640>keep 00:09:25.270 --> 00:09:25.280 align:start position:0% <00:09:25.270>And <00:09:25.271>finally, <00:09:25.272>our <00:09:25.273>last <00:09:25.275>goal <00:09:25.276>is <00:09:25.277>to <00:09:25.279>keep 00:09:25.280 --> 00:09:28.950 align:start position:0% <00:09:25.280>And <00:09:25.407>finally, <00:09:25.534>our <00:09:25.661>last <00:09:25.789>goal <00:09:25.916>is <00:09:26.043>to <00:09:26.170>keep <00:09:26.298>our <00:09:26.425>secret <00:09:26.552>sauce <00:09:26.680>secret. 00:09:28.960 --> 00:09:31.390 align:start position:0% <00:09:28.960>So <00:09:29.160>now <00:09:29.360>that <00:09:29.560>we <00:09:29.760>have <00:09:29.960>these <00:09:30.160>goals, <00:09:30.360>let's 00:09:31.390 --> 00:09:31.400 align:start position:0% <00:09:31.390>So <00:09:31.391>now <00:09:31.392>that <00:09:31.393>we <00:09:31.395>have <00:09:31.396>these <00:09:31.397>goals, <00:09:31.399>let's 00:09:31.400 --> 00:09:34.070 align:start position:0% <00:09:31.400>So <00:09:31.487>now <00:09:31.574>that <00:09:31.662>we <00:09:31.750>have <00:09:31.837>these <00:09:31.925>goals, <00:09:32.012>let's <00:09:32.100>actually <00:09:32.187>have <00:09:32.275>a <00:09:32.362>look <00:09:32.449>on <00:09:32.537>how <00:09:32.625>we <00:09:32.712>built <00:09:32.800>and 00:09:34.070 --> 00:09:34.080 align:start position:0% <00:09:34.070>actually <00:09:34.071>have <00:09:34.072>a <00:09:34.073>look <00:09:34.074>on <00:09:34.075>how <00:09:34.076>we <00:09:34.077>built <00:09:34.079>and 00:09:34.080 --> 00:09:36.829 align:start position:0% <00:09:34.080>actually <00:09:34.207>have <00:09:34.334>a <00:09:34.461>look <00:09:34.589>on <00:09:34.716>how <00:09:34.843>we <00:09:34.970>built <00:09:35.098>and <00:09:35.225>designed <00:09:35.352>our <00:09:35.480>agent. 00:09:36.829 --> 00:09:36.839 align:start position:0% <00:09:36.829>designed <00:09:36.833>our <00:09:36.838>agent. 00:09:36.839 --> 00:09:39.510 align:start position:0% <00:09:36.839>designed <00:09:36.966>our <00:09:37.093>agent. <00:09:37.220>So, <00:09:37.348>our <00:09:37.475>agent <00:09:37.602>is <00:09:37.729>kind <00:09:37.857>of <00:09:37.984>like <00:09:38.111>a <00:09:38.238>workflow 00:09:39.510 --> 00:09:39.520 align:start position:0% <00:09:39.510>So, <00:09:39.511>our <00:09:39.512>agent <00:09:39.513>is <00:09:39.514>kind <00:09:39.515>of <00:09:39.516>like <00:09:39.517>a <00:09:39.519>workflow 00:09:39.520 --> 00:09:41.830 align:start position:0% <00:09:39.520>So, <00:09:39.607>our <00:09:39.694>agent <00:09:39.782>is <00:09:39.870>kind <00:09:39.957>of <00:09:40.045>like <00:09:40.132>a <00:09:40.220>workflow <00:09:40.307>as <00:09:40.395>you <00:09:40.482>can <00:09:40.569>with <00:09:40.657>with <00:09:40.745>multiple <00:09:40.832>nodes. <00:09:40.920>We 00:09:41.830 --> 00:09:41.840 align:start position:0% <00:09:41.830>as <00:09:41.831>you <00:09:41.832>can <00:09:41.833>with <00:09:41.835>with <00:09:41.836>multiple <00:09:41.837>nodes. <00:09:41.839>We 00:09:41.840 --> 00:09:44.310 align:start position:0% <00:09:41.840>as <00:09:41.933>you <00:09:42.026>can <00:09:42.120>with <00:09:42.213>with <00:09:42.306>multiple <00:09:42.400>nodes. <00:09:42.493>We <00:09:42.586>first <00:09:42.680>start <00:09:42.773>with <00:09:42.866>the <00:09:42.960>intent <00:09:43.053>that <00:09:43.146>kind <00:09:43.240>of 00:09:44.310 --> 00:09:44.320 align:start position:0% <00:09:44.310>first <00:09:44.311>start <00:09:44.312>with <00:09:44.313>the <00:09:44.315>intent <00:09:44.316>that <00:09:44.317>kind <00:09:44.319>of 00:09:44.320 --> 00:09:46.230 align:start position:0% <00:09:44.320>first <00:09:44.427>start <00:09:44.535>with <00:09:44.643>the <00:09:44.750>intent <00:09:44.858>that <00:09:44.966>kind <00:09:45.073>of <00:09:45.181>tries <00:09:45.289>to <00:09:45.396>understand <00:09:45.504>what <00:09:45.612>the <00:09:45.720>traveler 00:09:46.230 --> 00:09:46.240 align:start position:0% <00:09:46.230>tries <00:09:46.231>to <00:09:46.233>understand <00:09:46.235>what <00:09:46.237>the <00:09:46.239>traveler 00:09:46.240 --> 00:09:47.270 align:start position:0% <00:09:46.240>tries <00:09:46.355>to <00:09:46.471>understand <00:09:46.587>what <00:09:46.703>the <00:09:46.819>traveler <00:09:46.935>wants <00:09:47.051>to <00:09:47.167>do. 00:09:47.270 --> 00:09:47.280 align:start position:0% <00:09:47.270>wants <00:09:47.274>to <00:09:47.279>do. 00:09:47.280 --> 00:09:49.470 align:start position:0% <00:09:47.280>wants <00:09:47.420>to <00:09:47.560>do. <00:09:47.699>That <00:09:47.839>intent <00:09:47.980>is <00:09:48.120>passed <00:09:48.260>to <00:09:48.400>a <00:09:48.540>planner <00:09:48.680>node 00:09:49.470 --> 00:09:49.480 align:start position:0% <00:09:49.470>That <00:09:49.471>intent <00:09:49.472>is <00:09:49.473>passed <00:09:49.475>to <00:09:49.476>a <00:09:49.477>planner <00:09:49.479>node 00:09:49.480 --> 00:09:51.270 align:start position:0% <00:09:49.480>That <00:09:49.587>intent <00:09:49.695>is <00:09:49.803>passed <00:09:49.910>to <00:09:50.018>a <00:09:50.126>planner <00:09:50.233>node <00:09:50.341>that <00:09:50.449>is <00:09:50.556>structurizing <00:09:50.664>the <00:09:50.772>free <00:09:50.880>text 00:09:51.270 --> 00:09:51.280 align:start position:0% <00:09:51.270>that <00:09:51.271>is <00:09:51.273>structurizing <00:09:51.275>the <00:09:51.277>free <00:09:51.279>text 00:09:51.280 --> 00:09:54.230 align:start position:0% <00:09:51.280>that <00:09:51.435>is <00:09:51.591>structurizing <00:09:51.746>the <00:09:51.902>free <00:09:52.057>text <00:09:52.213>query <00:09:52.368>into <00:09:52.524>structured <00:09:52.680>fields. 00:09:54.230 --> 00:09:54.240 align:start position:0% <00:09:54.230>query <00:09:54.233>into <00:09:54.236>structured <00:09:54.239>fields. 00:09:54.240 --> 00:09:56.230 align:start position:0% <00:09:54.240>query <00:09:54.380>into <00:09:54.520>structured <00:09:54.660>fields. <00:09:54.800>Now <00:09:54.940>that <00:09:55.080>we <00:09:55.220>have <00:09:55.360>our <00:09:55.500>structured <00:09:55.640>fields, 00:09:56.230 --> 00:09:56.240 align:start position:0% <00:09:56.230>Now <00:09:56.231>that <00:09:56.233>we <00:09:56.234>have <00:09:56.236>our <00:09:56.237>structured <00:09:56.239>fields, 00:09:56.240 --> 00:09:58.470 align:start position:0% <00:09:56.240>Now <00:09:56.340>that <00:09:56.440>we <00:09:56.540>have <00:09:56.640>our <00:09:56.740>structured <00:09:56.840>fields, <00:09:56.940>we <00:09:57.040>can <00:09:57.140>use <00:09:57.240>that <00:09:57.340>for <00:09:57.440>retrieval. <00:09:57.540>So, <00:09:57.640>here 00:09:58.470 --> 00:09:58.480 align:start position:0% <00:09:58.470>we <00:09:58.471>can <00:09:58.472>use <00:09:58.473>that <00:09:58.475>for <00:09:58.476>retrieval. <00:09:58.477>So, <00:09:58.479>here 00:09:58.480 --> 00:10:01.230 align:start position:0% <00:09:58.480>we <00:09:58.580>can <00:09:58.680>use <00:09:58.780>that <00:09:58.880>for <00:09:58.980>retrieval. <00:09:59.080>So, <00:09:59.180>here <00:09:59.280>we <00:09:59.380>essentially <00:09:59.480>do <00:09:59.580>a <00:09:59.680>search <00:09:59.780>with <00:09:59.880>the 00:10:01.230 --> 00:10:01.240 align:start position:0% <00:10:01.230>we <00:10:01.231>essentially <00:10:01.233>do <00:10:01.234>a <00:10:01.236>search <00:10:01.237>with <00:10:01.239>the 00:10:01.240 --> 00:10:03.710 align:start position:0% <00:10:01.240>we <00:10:01.380>essentially <00:10:01.520>do <00:10:01.660>a <00:10:01.800>search <00:10:01.940>with <00:10:02.080>the <00:10:02.220>information <00:10:02.360>that <00:10:02.500>we <00:10:02.640>have. 00:10:03.710 --> 00:10:03.720 align:start position:0% <00:10:03.710>information <00:10:03.713>that <00:10:03.716>we <00:10:03.719>have. 00:10:03.720 --> 00:10:05.830 align:start position:0% <00:10:03.720>information <00:10:03.836>that <00:10:03.953>we <00:10:04.070>have. <00:10:04.186>Then <00:10:04.303>the <00:10:04.420>next <00:10:04.536>step <00:10:04.653>is <00:10:04.770>to <00:10:04.886>pass <00:10:05.003>it <00:10:05.120>through 00:10:05.830 --> 00:10:05.840 align:start position:0% <00:10:05.830>Then <00:10:05.831>the <00:10:05.832>next <00:10:05.833>step <00:10:05.834>is <00:10:05.835>to <00:10:05.836>pass <00:10:05.837>it <00:10:05.839>through 00:10:05.840 --> 00:10:07.710 align:start position:0% <00:10:05.840>Then <00:10:05.940>the <00:10:06.040>next <00:10:06.140>step <00:10:06.240>is <00:10:06.340>to <00:10:06.440>pass <00:10:06.540>it <00:10:06.640>through <00:10:06.740>an <00:10:06.840>additional <00:10:06.940>relevant <00:10:07.040>step <00:10:07.140>that <00:10:07.240>looks 00:10:07.710 --> 00:10:07.720 align:start position:0% <00:10:07.710>an <00:10:07.711>additional <00:10:07.713>relevant <00:10:07.715>step <00:10:07.717>that <00:10:07.719>looks 00:10:07.720 --> 00:10:09.470 align:start position:0% <00:10:07.720>an <00:10:07.847>additional <00:10:07.974>relevant <00:10:08.101>step <00:10:08.229>that <00:10:08.356>looks <00:10:08.483>at <00:10:08.610>the <00:10:08.738>original <00:10:08.865>intent, <00:10:08.992>the <00:10:09.120>candidate 00:10:09.470 --> 00:10:09.480 align:start position:0% <00:10:09.470>at <00:10:09.471>the <00:10:09.473>original <00:10:09.475>intent, <00:10:09.477>the <00:10:09.479>candidate 00:10:09.480 --> 00:10:11.990 align:start position:0% <00:10:09.480>at <00:10:09.620>the <00:10:09.760>original <00:10:09.900>intent, <00:10:10.040>the <00:10:10.180>candidate <00:10:10.320>experiences <00:10:10.460>from <00:10:10.600>our <00:10:10.740>retriever <00:10:10.880>layer, 00:10:11.990 --> 00:10:12.000 align:start position:0% <00:10:11.990>experiences <00:10:11.992>from <00:10:11.994>our <00:10:11.996>retriever <00:10:11.999>layer, 00:10:12.000 --> 00:10:14.150 align:start position:0% <00:10:12.000>experiences <00:10:12.127>from <00:10:12.254>our <00:10:12.381>retriever <00:10:12.509>layer, <00:10:12.636>and <00:10:12.763>filters <00:10:12.890>out <00:10:13.018>the <00:10:13.145>experiences <00:10:13.272>that <00:10:13.400>are 00:10:14.150 --> 00:10:14.160 align:start position:0% <00:10:14.150>and <00:10:14.151>filters <00:10:14.153>out <00:10:14.154>the <00:10:14.156>experiences <00:10:14.157>that <00:10:14.159>are 00:10:14.160 --> 00:10:17.150 align:start position:0% <00:10:14.160>and <00:10:14.276>filters <00:10:14.393>out <00:10:14.510>the <00:10:14.626>experiences <00:10:14.743>that <00:10:14.860>are <00:10:14.976>not <00:10:15.093>really <00:10:15.209>relevant <00:10:15.326>for <00:10:15.443>the <00:10:15.560>intent. 00:10:17.150 --> 00:10:17.160 align:start position:0% <00:10:17.150>not <00:10:17.151>really <00:10:17.153>relevant <00:10:17.155>for <00:10:17.157>the <00:10:17.159>intent. 00:10:17.160 --> 00:10:19.030 align:start position:0% <00:10:17.160>not <00:10:17.267>really <00:10:17.375>relevant <00:10:17.483>for <00:10:17.590>the <00:10:17.698>intent. <00:10:17.806>And <00:10:17.913>then <00:10:18.021>lastly, <00:10:18.129>we <00:10:18.236>have <00:10:18.344>a <00:10:18.452>drafter <00:10:18.560>that 00:10:19.030 --> 00:10:19.040 align:start position:0% <00:10:19.030>And <00:10:19.031>then <00:10:19.032>lastly, <00:10:19.033>we <00:10:19.035>have <00:10:19.036>a <00:10:19.037>drafter <00:10:19.039>that 00:10:19.040 --> 00:10:21.790 align:start position:0% <00:10:19.040>And <00:10:19.140>then <00:10:19.240>lastly, <00:10:19.339>we <00:10:19.439>have <00:10:19.540>a <00:10:19.640>drafter <00:10:19.740>that <00:10:19.839>generates <00:10:19.939>a <00:10:20.040>free <00:10:20.140>text <00:10:20.240>message <00:10:20.339>for <00:10:20.439>our 00:10:21.790 --> 00:10:21.800 align:start position:0% <00:10:21.790>generates <00:10:21.791>a <00:10:21.793>free <00:10:21.794>text <00:10:21.795>message <00:10:21.797>for <00:10:21.799>our 00:10:21.800 --> 00:10:23.070 align:start position:0% <00:10:21.800>generates <00:10:21.963>a <00:10:22.126>free <00:10:22.289>text <00:10:22.453>message <00:10:22.616>for <00:10:22.779>our <00:10:22.943>customers. 00:10:23.070 --> 00:10:23.080 align:start position:0% customers. 00:10:23.080 --> 00:10:24.830 align:start position:0% <00:10:23.080>customers. <00:10:23.255>We <00:10:23.430>also <00:10:23.605>have <00:10:23.780>a <00:10:23.955>memory <00:10:24.130>component <00:10:24.305>that <00:10:24.480>is 00:10:24.830 --> 00:10:24.840 align:start position:0% <00:10:24.830>We <00:10:24.831>also <00:10:24.832>have <00:10:24.833>a <00:10:24.835>memory <00:10:24.836>component <00:10:24.837>that <00:10:24.839>is 00:10:24.840 --> 00:10:27.190 align:start position:0% <00:10:24.840>We <00:10:24.933>also <00:10:25.026>have <00:10:25.120>a <00:10:25.213>memory <00:10:25.306>component <00:10:25.400>that <00:10:25.493>is <00:10:25.586>added <00:10:25.680>on <00:10:25.773>top <00:10:25.866>of <00:10:25.960>the <00:10:26.053>intent <00:10:26.146>that <00:10:26.240>is 00:10:27.190 --> 00:10:27.200 align:start position:0% <00:10:27.190>added <00:10:27.191>on <00:10:27.192>top <00:10:27.193>of <00:10:27.195>the <00:10:27.196>intent <00:10:27.197>that <00:10:27.199>is 00:10:27.200 --> 00:10:29.230 align:start position:0% <00:10:27.200>added <00:10:27.316>on <00:10:27.433>top <00:10:27.550>of <00:10:27.666>the <00:10:27.783>intent <00:10:27.900>that <00:10:28.016>is <00:10:28.133>handling <00:10:28.250>personalization <00:10:28.366>and <00:10:28.483>stores <00:10:28.600>all 00:10:29.230 --> 00:10:29.240 align:start position:0% <00:10:29.230>handling <00:10:29.232>personalization <00:10:29.234>and <00:10:29.236>stores <00:10:29.239>all 00:10:29.240 --> 00:10:30.950 align:start position:0% <00:10:29.240>handling <00:10:29.380>personalization <00:10:29.520>and <00:10:29.660>stores <00:10:29.800>all <00:10:29.940>of <00:10:30.080>the <00:10:30.220>historical <00:10:30.360>context <00:10:30.500>from <00:10:30.640>our 00:10:30.950 --> 00:10:30.960 align:start position:0% <00:10:30.950>of <00:10:30.951>the <00:10:30.953>historical <00:10:30.955>context <00:10:30.957>from <00:10:30.959>our 00:10:30.960 --> 00:10:33.390 align:start position:0% <00:10:30.960>of <00:10:31.193>the <00:10:31.426>historical <00:10:31.660>context <00:10:31.893>from <00:10:32.126>our <00:10:32.360>customers. 00:10:33.390 --> 00:10:33.400 align:start position:0% customers. 00:10:33.400 --> 00:10:35.190 align:start position:0% <00:10:33.400>customers. <00:10:33.600>This <00:10:33.800>looks <00:10:34.000>quite <00:10:34.199>abstract, <00:10:34.400>so <00:10:34.600>let's <00:10:34.800>have 00:10:35.190 --> 00:10:35.200 align:start position:0% <00:10:35.190>This <00:10:35.191>looks <00:10:35.193>quite <00:10:35.194>abstract, <00:10:35.196>so <00:10:35.197>let's <00:10:35.199>have 00:10:35.200 --> 00:10:37.550 align:start position:0% <00:10:35.200>This <00:10:35.293>looks <00:10:35.386>quite <00:10:35.480>abstract, <00:10:35.573>so <00:10:35.666>let's <00:10:35.760>have <00:10:35.853>a <00:10:35.946>look <00:10:36.040>at <00:10:36.133>an <00:10:36.226>example <00:10:36.320>on <00:10:36.413>how <00:10:36.506>this <00:10:36.600>would 00:10:37.550 --> 00:10:37.560 align:start position:0% <00:10:37.550>a <00:10:37.551>look <00:10:37.552>at <00:10:37.553>an <00:10:37.554>example <00:10:37.555>on <00:10:37.556>how <00:10:37.557>this <00:10:37.559>would 00:10:37.560 --> 00:10:42.070 align:start position:0% <00:10:37.560>a <00:10:37.687>look <00:10:37.814>at <00:10:37.941>an <00:10:38.069>example <00:10:38.196>on <00:10:38.323>how <00:10:38.450>this <00:10:38.578>would <00:10:38.705>look <00:10:38.832>like <00:10:38.959>step-by-step. 00:10:42.080 --> 00:10:44.910 align:start position:0% <00:10:42.080>So, <00:10:42.280>the <00:10:42.480>first <00:10:42.680>stage <00:10:42.880>is <00:10:43.080>intent. <00:10:43.280>So, <00:10:43.480>it's 00:10:44.910 --> 00:10:44.920 align:start position:0% <00:10:44.910>So, <00:10:44.911>the <00:10:44.912>first <00:10:44.913>stage <00:10:44.915>is <00:10:44.916>intent. <00:10:44.917>So, <00:10:44.919>it's 00:10:44.920 --> 00:10:46.350 align:start position:0% <00:10:44.920>So, <00:10:45.027>the <00:10:45.134>first <00:10:45.241>stage <00:10:45.348>is <00:10:45.456>intent. <00:10:45.563>So, <00:10:45.670>it's <00:10:45.778>understand <00:10:45.885>and <00:10:45.992>compress. <00:10:46.099>So, <00:10:46.207>let's 00:10:46.350 --> 00:10:46.360 align:start position:0% <00:10:46.350>understand <00:10:46.352>and <00:10:46.354>compress. <00:10:46.356>So, <00:10:46.359>let's 00:10:46.360 --> 00:10:48.670 align:start position:0% <00:10:46.360>understand <00:10:46.487>and <00:10:46.614>compress. <00:10:46.741>So, <00:10:46.869>let's <00:10:46.996>imagine <00:10:47.123>I <00:10:47.250>have <00:10:47.378>this <00:10:47.505>user <00:10:47.632>prompt <00:10:47.760>that 00:10:48.670 --> 00:10:48.680 align:start position:0% <00:10:48.670>imagine <00:10:48.671>I <00:10:48.673>have <00:10:48.674>this <00:10:48.675>user <00:10:48.677>prompt <00:10:48.679>that 00:10:48.680 --> 00:10:50.270 align:start position:0% <00:10:48.680>imagine <00:10:48.796>I <00:10:48.913>have <00:10:49.030>this <00:10:49.146>user <00:10:49.263>prompt <00:10:49.380>that <00:10:49.496>says, <00:10:49.613>"I'm <00:10:49.729>visiting <00:10:49.846>Berlin <00:10:49.963>and <00:10:50.079>would 00:10:50.270 --> 00:10:50.280 align:start position:0% <00:10:50.270>says, <00:10:50.271>"I'm <00:10:50.273>visiting <00:10:50.275>Berlin <00:10:50.277>and <00:10:50.279>would 00:10:50.280 --> 00:10:52.310 align:start position:0% <00:10:50.280>says, <00:10:50.396>"I'm <00:10:50.513>visiting <00:10:50.630>Berlin <00:10:50.746>and <00:10:50.863>would <00:10:50.980>love <00:10:51.096>a <00:10:51.213>food <00:10:51.329>tour <00:10:51.446>in <00:10:51.563>the <00:10:51.680>afternoon. 00:10:52.310 --> 00:10:52.320 align:start position:0% <00:10:52.310>love <00:10:52.311>a <00:10:52.313>food <00:10:52.314>tour <00:10:52.316>in <00:10:52.317>the <00:10:52.319>afternoon. 00:10:52.320 --> 00:10:54.070 align:start position:0% <00:10:52.320>love <00:10:52.420>a <00:10:52.520>food <00:10:52.620>tour <00:10:52.720>in <00:10:52.820>the <00:10:52.920>afternoon. <00:10:53.020>Include <00:10:53.120>a <00:10:53.220>stop <00:10:53.320>at <00:10:53.420>a <00:10:53.520>local <00:10:53.620>market <00:10:53.720>and 00:10:54.070 --> 00:10:54.080 align:start position:0% <00:10:54.070>Include <00:10:54.071>a <00:10:54.072>stop <00:10:54.073>at <00:10:54.075>a <00:10:54.076>local <00:10:54.077>market <00:10:54.079>and 00:10:54.080 --> 00:10:55.990 align:start position:0% <00:10:54.080>Include <00:10:54.173>a <00:10:54.266>stop <00:10:54.360>at <00:10:54.453>a <00:10:54.546>local <00:10:54.640>market <00:10:54.733>and <00:10:54.826>keep <00:10:54.920>it <00:10:55.013>under <00:10:55.106>50 <00:10:55.200>euro <00:10:55.293>per <00:10:55.386>person. <00:10:55.480>What's 00:10:55.990 --> 00:10:56.000 align:start position:0% <00:10:55.990>keep <00:10:55.991>it <00:10:55.992>under <00:10:55.993>50 <00:10:55.995>euro <00:10:55.996>per <00:10:55.997>person. <00:10:55.999>What's 00:10:56.000 --> 00:10:57.990 align:start position:0% <00:10:56.000>keep <00:10:56.100>it <00:10:56.200>under <00:10:56.300>50 <00:10:56.400>euro <00:10:56.500>per <00:10:56.600>person. <00:10:56.700>What's <00:10:56.800>available?" <00:10:56.900>And <00:10:57.000>we <00:10:57.100>would <00:10:57.200>also <00:10:57.300>have <00:10:57.400>some 00:10:57.990 --> 00:10:58.000 align:start position:0% <00:10:57.990>available?" <00:10:57.991>And <00:10:57.993>we <00:10:57.994>would <00:10:57.996>also <00:10:57.997>have <00:10:57.999>some 00:10:58.000 --> 00:11:00.110 align:start position:0% <00:10:58.000>available?" <00:10:58.116>And <00:10:58.233>we <00:10:58.350>would <00:10:58.466>also <00:10:58.583>have <00:10:58.700>some <00:10:58.816>kind <00:10:58.933>of <00:10:59.050>historical <00:10:59.166>messages <00:10:59.283>that <00:10:59.400>we 00:11:00.110 --> 00:11:00.120 align:start position:0% <00:11:00.110>kind <00:11:00.111>of <00:11:00.113>historical <00:11:00.115>messages <00:11:00.117>that <00:11:00.119>we 00:11:00.120 --> 00:11:03.710 align:start position:0% <00:11:00.120>kind <00:11:00.275>of <00:11:00.431>historical <00:11:00.586>messages <00:11:00.742>that <00:11:00.897>we <00:11:01.053>persisted <00:11:01.208>over <00:11:01.364>multiple <00:11:01.520>sessions. 00:11:03.710 --> 00:11:03.720 align:start position:0% <00:11:03.710>persisted <00:11:03.713>over <00:11:03.716>multiple <00:11:03.719>sessions. 00:11:03.720 --> 00:11:05.870 align:start position:0% <00:11:03.720>persisted <00:11:03.860>over <00:11:04.000>multiple <00:11:04.140>sessions. <00:11:04.280>Our <00:11:04.420>intent <00:11:04.560>node <00:11:04.700>would <00:11:04.840>compress <00:11:04.980>all <00:11:05.120>of 00:11:05.870 --> 00:11:05.880 align:start position:0% <00:11:05.870>Our <00:11:05.871>intent <00:11:05.873>node <00:11:05.874>would <00:11:05.876>compress <00:11:05.877>all <00:11:05.879>of 00:11:05.880 --> 00:11:07.350 align:start position:0% <00:11:05.880>Our <00:11:05.990>intent <00:11:06.100>node <00:11:06.210>would <00:11:06.321>compress <00:11:06.431>all <00:11:06.541>of <00:11:06.651>that <00:11:06.762>information, <00:11:06.872>remove <00:11:06.982>all <00:11:07.092>of <00:11:07.203>the 00:11:07.350 --> 00:11:07.360 align:start position:0% <00:11:07.350>that <00:11:07.351>information, <00:11:07.353>remove <00:11:07.355>all <00:11:07.357>of <00:11:07.359>the 00:11:07.360 --> 00:11:08.950 align:start position:0% <00:11:07.360>that <00:11:07.487>information, <00:11:07.614>remove <00:11:07.741>all <00:11:07.869>of <00:11:07.996>the <00:11:08.123>noise, <00:11:08.250>and <00:11:08.378>rewrite <00:11:08.505>the <00:11:08.632>query <00:11:08.760>into 00:11:08.950 --> 00:11:08.960 align:start position:0% <00:11:08.950>noise, <00:11:08.951>and <00:11:08.953>rewrite <00:11:08.955>the <00:11:08.957>query <00:11:08.959>into 00:11:08.960 --> 00:11:11.230 align:start position:0% <00:11:08.960>noise, <00:11:09.087>and <00:11:09.214>rewrite <00:11:09.341>the <00:11:09.469>query <00:11:09.596>into <00:11:09.723>something <00:11:09.850>more <00:11:09.978>shorter, <00:11:10.105>such <00:11:10.232>as <00:11:10.360>Berlin 00:11:11.230 --> 00:11:11.240 align:start position:0% <00:11:11.230>something <00:11:11.231>more <00:11:11.233>shorter, <00:11:11.235>such <00:11:11.237>as <00:11:11.239>Berlin 00:11:11.240 --> 00:11:13.030 align:start position:0% <00:11:11.240>something <00:11:11.340>more <00:11:11.440>shorter, <00:11:11.540>such <00:11:11.640>as <00:11:11.740>Berlin <00:11:11.840>food <00:11:11.940>tour <00:11:12.040>that <00:11:12.140>include <00:11:12.240>a <00:11:12.340>stop <00:11:12.440>at <00:11:12.540>a <00:11:12.640>local 00:11:13.030 --> 00:11:13.040 align:start position:0% <00:11:13.030>food <00:11:13.031>tour <00:11:13.032>that <00:11:13.033>include <00:11:13.034>a <00:11:13.035>stop <00:11:13.036>at <00:11:13.037>a <00:11:13.039>local 00:11:13.040 --> 00:11:16.430 align:start position:0% <00:11:13.040>food <00:11:13.133>tour <00:11:13.226>that <00:11:13.319>include <00:11:13.413>a <00:11:13.506>stop <00:11:13.599>at <00:11:13.693>a <00:11:13.786>local <00:11:13.880>market <00:11:13.973>under <00:11:14.066>50 <00:11:14.160>euro <00:11:14.253>in <00:11:14.346>the <00:11:14.439>afternoon. 00:11:16.430 --> 00:11:16.440 align:start position:0% <00:11:16.430>market <00:11:16.431>under <00:11:16.433>50 <00:11:16.434>euro <00:11:16.436>in <00:11:16.437>the <00:11:16.439>afternoon. 00:11:16.440 --> 00:11:18.430 align:start position:0% <00:11:16.440>market <00:11:16.527>under <00:11:16.615>50 <00:11:16.702>euro <00:11:16.790>in <00:11:16.877>the <00:11:16.965>afternoon. <00:11:17.052>Now <00:11:17.140>that <00:11:17.227>we <00:11:17.315>cut <00:11:17.402>off <00:11:17.490>all <00:11:17.577>of <00:11:17.665>the <00:11:17.752>noise, <00:11:17.840>we 00:11:18.430 --> 00:11:18.440 align:start position:0% <00:11:18.430>Now <00:11:18.430>that <00:11:18.432>we <00:11:18.433>cut <00:11:18.434>off <00:11:18.435>all <00:11:18.436>of <00:11:18.437>the <00:11:18.438>noise, <00:11:18.439>we 00:11:18.440 --> 00:11:20.470 align:start position:0% <00:11:18.440>Now <00:11:18.533>that <00:11:18.626>we <00:11:18.720>cut <00:11:18.813>off <00:11:18.906>all <00:11:19.000>of <00:11:19.093>the <00:11:19.186>noise, <00:11:19.280>we <00:11:19.373>can <00:11:19.466>put <00:11:19.560>it <00:11:19.653>into <00:11:19.746>our <00:11:19.840>planner. 00:11:20.470 --> 00:11:20.480 align:start position:0% <00:11:20.470>can <00:11:20.471>put <00:11:20.473>it <00:11:20.475>into <00:11:20.477>our <00:11:20.479>planner. 00:11:20.480 --> 00:11:22.030 align:start position:0% <00:11:20.480>can <00:11:20.635>put <00:11:20.790>it <00:11:20.945>into <00:11:21.100>our <00:11:21.255>planner. <00:11:21.410>And <00:11:21.564>the <00:11:21.720>planner <00:11:21.875>is 00:11:22.030 --> 00:11:22.040 align:start position:0% <00:11:22.030>And <00:11:22.033>the <00:11:22.036>planner <00:11:22.039>is 00:11:22.040 --> 00:11:24.750 align:start position:0% <00:11:22.040>And <00:11:22.195>the <00:11:22.351>planner <00:11:22.506>is <00:11:22.662>structuring <00:11:22.817>that <00:11:22.973>request. <00:11:23.128>So, <00:11:23.284>in <00:11:23.439>this 00:11:24.750 --> 00:11:24.760 align:start position:0% <00:11:24.750>structuring <00:11:24.751>that <00:11:24.753>request. <00:11:24.755>So, <00:11:24.757>in <00:11:24.759>this 00:11:24.760 --> 00:11:26.870 align:start position:0% <00:11:24.760>structuring <00:11:24.915>that <00:11:25.071>request. <00:11:25.226>So, <00:11:25.382>in <00:11:25.537>this <00:11:25.693>case, <00:11:25.848>we <00:11:26.004>will <00:11:26.160>have 00:11:26.870 --> 00:11:26.880 align:start position:0% <00:11:26.870>case, <00:11:26.873>we <00:11:26.876>will <00:11:26.879>have 00:11:26.880 --> 00:11:28.110 align:start position:0% <00:11:26.880>case, <00:11:27.003>we <00:11:27.126>will <00:11:27.249>have <00:11:27.372>kind <00:11:27.495>of <00:11:27.618>like <00:11:27.741>different <00:11:27.864>fields, <00:11:27.987>the 00:11:28.110 --> 00:11:28.120 align:start position:0% <00:11:28.110>kind <00:11:28.111>of <00:11:28.113>like <00:11:28.115>different <00:11:28.117>fields, <00:11:28.119>the 00:11:28.120 --> 00:11:31.430 align:start position:0% <00:11:28.120>kind <00:11:28.236>of <00:11:28.353>like <00:11:28.470>different <00:11:28.586>fields, <00:11:28.703>the <00:11:28.820>destination, <00:11:28.936>the <00:11:29.053>a <00:11:29.170>budget, <00:11:29.286>the <00:11:29.403>time, <00:11:29.520>and 00:11:31.430 --> 00:11:31.440 align:start position:0% <00:11:31.430>destination, <00:11:31.431>the <00:11:31.433>a <00:11:31.434>budget, <00:11:31.436>the <00:11:31.437>time, <00:11:31.439>and 00:11:31.440 --> 00:11:33.510 align:start position:0% <00:11:31.440>destination, <00:11:31.567>the <00:11:31.694>a <00:11:31.821>budget, <00:11:31.949>the <00:11:32.076>time, <00:11:32.203>and <00:11:32.330>keywords. <00:11:32.458>And <00:11:32.585>these <00:11:32.712>are <00:11:32.840>structured 00:11:33.510 --> 00:11:33.520 align:start position:0% <00:11:33.510>keywords. <00:11:33.512>And <00:11:33.514>these <00:11:33.516>are <00:11:33.519>structured 00:11:33.520 --> 00:11:34.710 align:start position:0% <00:11:33.520>keywords. <00:11:33.617>And <00:11:33.714>these <00:11:33.812>are <00:11:33.909>structured <00:11:34.006>fields <00:11:34.104>that <00:11:34.201>we <00:11:34.298>can <00:11:34.396>also <00:11:34.493>use <00:11:34.591>for 00:11:34.710 --> 00:11:34.720 align:start position:0% <00:11:34.710>fields <00:11:34.711>that <00:11:34.713>we <00:11:34.714>can <00:11:34.716>also <00:11:34.717>use <00:11:34.719>for 00:11:34.720 --> 00:11:36.630 align:start position:0% <00:11:34.720>fields <00:11:34.895>that <00:11:35.070>we <00:11:35.245>can <00:11:35.420>also <00:11:35.595>use <00:11:35.770>for <00:11:35.945>downstream <00:11:36.120>services. 00:11:36.630 --> 00:11:36.640 align:start position:0% <00:11:36.630>downstream <00:11:36.639>services. 00:11:36.640 --> 00:11:37.670 align:start position:0% <00:11:36.640>downstream <00:11:36.794>services. <00:11:36.949>And <00:11:37.103>this <00:11:37.257>actually <00:11:37.412>sounds <00:11:37.567>quite 00:11:37.670 --> 00:11:37.680 align:start position:0% <00:11:37.670>And <00:11:37.672>this <00:11:37.674>actually <00:11:37.676>sounds <00:11:37.679>quite 00:11:37.680 --> 00:11:39.470 align:start position:0% <00:11:37.680>And <00:11:37.819>this <00:11:37.959>actually <00:11:38.099>sounds <00:11:38.239>quite <00:11:38.380>straightforward, <00:11:38.520>but <00:11:38.660>then <00:11:38.800>in <00:11:38.939>our <00:11:39.079>case, 00:11:39.470 --> 00:11:39.480 align:start position:0% <00:11:39.470>straightforward, <00:11:39.471>but <00:11:39.473>then <00:11:39.475>in <00:11:39.477>our <00:11:39.479>case, 00:11:39.480 --> 00:11:41.230 align:start position:0% <00:11:39.480>straightforward, <00:11:39.596>but <00:11:39.713>then <00:11:39.830>in <00:11:39.946>our <00:11:40.063>case, <00:11:40.180>we <00:11:40.296>have <00:11:40.413>more <00:11:40.530>than <00:11:40.646>10 <00:11:40.763>different <00:11:40.880>fields. 00:11:41.230 --> 00:11:41.240 align:start position:0% <00:11:41.230>we <00:11:41.231>have <00:11:41.233>more <00:11:41.234>than <00:11:41.236>10 <00:11:41.237>different <00:11:41.239>fields. 00:11:41.240 --> 00:11:43.350 align:start position:0% <00:11:41.240>we <00:11:41.347>have <00:11:41.455>more <00:11:41.563>than <00:11:41.670>10 <00:11:41.778>different <00:11:41.886>fields. <00:11:41.993>Each <00:11:42.101>of <00:11:42.209>them <00:11:42.316>have <00:11:42.424>different <00:11:42.532>edge <00:11:42.640>cases. 00:11:43.350 --> 00:11:43.360 align:start position:0% <00:11:43.350>Each <00:11:43.351>of <00:11:43.353>them <00:11:43.354>have <00:11:43.356>different <00:11:43.357>edge <00:11:43.359>cases. 00:11:43.360 --> 00:11:44.990 align:start position:0% <00:11:43.360>Each <00:11:43.453>of <00:11:43.546>them <00:11:43.640>have <00:11:43.733>different <00:11:43.826>edge <00:11:43.920>cases. <00:11:44.013>And <00:11:44.106>so, <00:11:44.200>if <00:11:44.293>you <00:11:44.386>think <00:11:44.480>about <00:11:44.573>it, <00:11:44.666>what <00:11:44.760>does 00:11:44.990 --> 00:11:45.000 align:start position:0% <00:11:44.990>And <00:11:44.991>so, <00:11:44.992>if <00:11:44.993>you <00:11:44.994>think <00:11:44.995>about <00:11:44.996>it, <00:11:44.997>what <00:11:44.999>does 00:11:45.000 --> 00:11:46.950 align:start position:0% <00:11:45.000>And <00:11:45.116>so, <00:11:45.233>if <00:11:45.350>you <00:11:45.466>think <00:11:45.583>about <00:11:45.700>it, <00:11:45.816>what <00:11:45.933>does <00:11:46.050>lunch <00:11:46.166>time <00:11:46.283>mean? <00:11:46.400>Or 00:11:46.950 --> 00:11:46.960 align:start position:0% <00:11:46.950>lunch <00:11:46.953>time <00:11:46.956>mean? <00:11:46.959>Or 00:11:46.960 --> 00:11:51.230 align:start position:0% <00:11:46.960>lunch <00:11:47.160>time <00:11:47.360>mean? <00:11:47.560>Or <00:11:47.760>afternoon, <00:11:47.960>and <00:11:48.160>so <00:11:48.360>on. 00:11:51.240 --> 00:11:53.270 align:start position:0% <00:11:51.240>Now <00:11:51.473>that <00:11:51.706>we <00:11:51.940>have <00:11:52.173>uh <00:11:52.406>structured <00:11:52.640>fields, 00:11:53.270 --> 00:11:53.280 align:start position:0% <00:11:53.270>Now <00:11:53.271>that <00:11:53.273>we <00:11:53.274>have <00:11:53.276>uh <00:11:53.277>structured <00:11:53.279>fields, 00:11:53.280 --> 00:11:55.430 align:start position:0% <00:11:53.280>Now <00:11:53.407>that <00:11:53.534>we <00:11:53.661>have <00:11:53.789>uh <00:11:53.916>structured <00:11:54.043>fields, <00:11:54.170>we <00:11:54.298>can <00:11:54.425>use <00:11:54.552>our <00:11:54.680>more 00:11:55.430 --> 00:11:55.440 align:start position:0% <00:11:55.430>we <00:11:55.432>can <00:11:55.434>use <00:11:55.436>our <00:11:55.439>more 00:11:55.440 --> 00:11:58.430 align:start position:0% <00:11:55.440>we <00:11:55.595>can <00:11:55.751>use <00:11:55.906>our <00:11:56.062>more <00:11:56.217>traditional <00:11:56.373>search <00:11:56.528>and <00:11:56.684>relevance <00:11:56.840>engine. 00:11:58.430 --> 00:11:58.440 align:start position:0% <00:11:58.430>traditional <00:11:58.432>search <00:11:58.434>and <00:11:58.436>relevance <00:11:58.439>engine. 00:11:58.440 --> 00:12:00.270 align:start position:0% <00:11:58.440>traditional <00:11:58.567>search <00:11:58.694>and <00:11:58.821>relevance <00:11:58.949>engine. <00:11:59.076>Here, <00:11:59.203>we <00:11:59.330>are <00:11:59.458>using <00:11:59.585>our <00:11:59.712>external <00:11:59.840>systems 00:12:00.270 --> 00:12:00.280 align:start position:0% <00:12:00.270>Here, <00:12:00.271>we <00:12:00.273>are <00:12:00.274>using <00:12:00.276>our <00:12:00.277>external <00:12:00.279>systems 00:12:00.280 --> 00:12:02.270 align:start position:0% <00:12:00.280>Here, <00:12:00.387>we <00:12:00.495>are <00:12:00.603>using <00:12:00.710>our <00:12:00.818>external <00:12:00.926>systems <00:12:01.033>that <00:12:01.141>we <00:12:01.249>have <00:12:01.356>built <00:12:01.464>years <00:12:01.572>optimizing <00:12:01.680>and 00:12:02.270 --> 00:12:02.280 align:start position:0% <00:12:02.270>that <00:12:02.271>we <00:12:02.273>have <00:12:02.274>built <00:12:02.276>years <00:12:02.277>optimizing <00:12:02.279>and 00:12:02.280 --> 00:12:03.870 align:start position:0% <00:12:02.280>that <00:12:02.480>we <00:12:02.680>have <00:12:02.880>built <00:12:03.079>years <00:12:03.280>optimizing <00:12:03.480>and <00:12:03.680>building. 00:12:03.870 --> 00:12:03.880 align:start position:0% building. 00:12:03.880 --> 00:12:05.190 align:start position:0% <00:12:03.880>building. <00:12:04.027>And <00:12:04.174>so, <00:12:04.322>we <00:12:04.469>start <00:12:04.616>off <00:12:04.764>with <00:12:04.911>a <00:12:05.059>hybrid 00:12:05.190 --> 00:12:05.200 align:start position:0% <00:12:05.190>And <00:12:05.191>so, <00:12:05.192>we <00:12:05.193>start <00:12:05.195>off <00:12:05.196>with <00:12:05.197>a <00:12:05.199>hybrid 00:12:05.200 --> 00:12:06.830 align:start position:0% <00:12:05.200>And <00:12:05.293>so, <00:12:05.386>we <00:12:05.480>start <00:12:05.573>off <00:12:05.666>with <00:12:05.760>a <00:12:05.853>hybrid <00:12:05.946>search <00:12:06.040>with <00:12:06.133>filters. <00:12:06.226>So, <00:12:06.320>we <00:12:06.413>take <00:12:06.506>all <00:12:06.600>of 00:12:06.830 --> 00:12:06.840 align:start position:0% <00:12:06.830>search <00:12:06.831>with <00:12:06.832>filters. <00:12:06.833>So, <00:12:06.835>we <00:12:06.836>take <00:12:06.837>all <00:12:06.839>of 00:12:06.840 --> 00:12:08.550 align:start position:0% <00:12:06.840>search <00:12:06.947>with <00:12:07.055>filters. <00:12:07.163>So, <00:12:07.270>we <00:12:07.378>take <00:12:07.486>all <00:12:07.593>of <00:12:07.701>our <00:12:07.809>experiences <00:12:07.916>in <00:12:08.024>our <00:12:08.132>database, <00:12:08.240>we 00:12:08.550 --> 00:12:08.560 align:start position:0% <00:12:08.550>our <00:12:08.551>experiences <00:12:08.553>in <00:12:08.555>our <00:12:08.557>database, <00:12:08.559>we 00:12:08.560 --> 00:12:10.670 align:start position:0% <00:12:08.560>our <00:12:08.676>experiences <00:12:08.793>in <00:12:08.910>our <00:12:09.026>database, <00:12:09.143>we <00:12:09.260>apply <00:12:09.376>the <00:12:09.493>filters <00:12:09.609>that <00:12:09.726>we <00:12:09.843>extracted, <00:12:09.959>and 00:12:10.670 --> 00:12:10.680 align:start position:0% <00:12:10.670>apply <00:12:10.671>the <00:12:10.673>filters <00:12:10.674>that <00:12:10.675>we <00:12:10.677>extracted, <00:12:10.679>and 00:12:10.680 --> 00:12:14.030 align:start position:0% <00:12:10.680>apply <00:12:10.780>the <00:12:10.880>filters <00:12:10.979>that <00:12:11.079>we <00:12:11.180>extracted, <00:12:11.280>and <00:12:11.380>then <00:12:11.479>do <00:12:11.579>a <00:12:11.680>BM25 <00:12:11.780>and <00:12:11.880>semantic <00:12:11.979>search <00:12:12.079>in 00:12:14.030 --> 00:12:14.040 align:start position:0% <00:12:14.030>then <00:12:14.031>do <00:12:14.032>a <00:12:14.033>BM25 <00:12:14.035>and <00:12:14.036>semantic <00:12:14.037>search <00:12:14.039>in 00:12:14.040 --> 00:12:15.870 align:start position:0% <00:12:14.040>then <00:12:14.180>do <00:12:14.319>a <00:12:14.459>BM25 <00:12:14.599>and <00:12:14.740>semantic <00:12:14.880>search <00:12:15.020>in <00:12:15.160>order <00:12:15.300>to <00:12:15.439>get 00:12:15.870 --> 00:12:15.880 align:start position:0% <00:12:15.870>order <00:12:15.874>to <00:12:15.879>get 00:12:15.880 --> 00:12:17.790 align:start position:0% <00:12:15.880>order <00:12:16.080>to <00:12:16.280>get <00:12:16.480>semantically <00:12:16.680>relevant <00:12:16.880>experience <00:12:17.080>for <00:12:17.280>the 00:12:17.790 --> 00:12:17.800 align:start position:0% <00:12:17.790>semantically <00:12:17.792>relevant <00:12:17.794>experience <00:12:17.796>for <00:12:17.799>the 00:12:17.800 --> 00:12:20.190 align:start position:0% <00:12:17.800>semantically <00:12:18.079>relevant <00:12:18.359>experience <00:12:18.640>for <00:12:18.920>the <00:12:19.199>request. 00:12:20.190 --> 00:12:20.200 align:start position:0% request. 00:12:20.200 --> 00:12:21.870 align:start position:0% <00:12:20.200>request. <00:12:20.400>Then <00:12:20.600>we <00:12:20.800>have <00:12:21.000>an <00:12:21.200>additional <00:12:21.400>deep <00:12:21.600>learning 00:12:21.870 --> 00:12:21.880 align:start position:0% <00:12:21.870>Then <00:12:21.871>we <00:12:21.873>have <00:12:21.874>an <00:12:21.876>additional <00:12:21.877>deep <00:12:21.879>learning 00:12:21.880 --> 00:12:23.150 align:start position:0% <00:12:21.880>Then <00:12:21.983>we <00:12:22.087>have <00:12:22.191>an <00:12:22.295>additional <00:12:22.399>deep <00:12:22.503>learning <00:12:22.607>ranking <00:12:22.711>system <00:12:22.815>that <00:12:22.919>is <00:12:23.023>a 00:12:23.150 --> 00:12:23.160 align:start position:0% <00:12:23.150>ranking <00:12:23.152>system <00:12:23.154>that <00:12:23.156>is <00:12:23.159>a 00:12:23.160 --> 00:12:25.310 align:start position:0% <00:12:23.160>ranking <00:12:23.315>system <00:12:23.471>that <00:12:23.626>is <00:12:23.782>a <00:12:23.937>transformer-based <00:12:24.093>model <00:12:24.248>that <00:12:24.404>is <00:12:24.560>trained 00:12:25.310 --> 00:12:25.320 align:start position:0% <00:12:25.310>transformer-based <00:12:25.312>model <00:12:25.314>that <00:12:25.316>is <00:12:25.319>trained 00:12:25.320 --> 00:12:28.310 align:start position:0% <00:12:25.320>transformer-based <00:12:25.475>model <00:12:25.631>that <00:12:25.786>is <00:12:25.942>trained <00:12:26.097>on <00:12:26.253>personalized <00:12:26.408>and <00:12:26.564>historical <00:12:26.720>data. 00:12:28.310 --> 00:12:28.320 align:start position:0% <00:12:28.310>on <00:12:28.312>personalized <00:12:28.314>and <00:12:28.316>historical <00:12:28.319>data. 00:12:28.320 --> 00:12:30.070 align:start position:0% <00:12:28.320>on <00:12:28.495>personalized <00:12:28.670>and <00:12:28.845>historical <00:12:29.020>data. <00:12:29.195>So, <00:12:29.370>we <00:12:29.545>then <00:12:29.720>get 00:12:30.070 --> 00:12:30.080 align:start position:0% <00:12:30.070>So, <00:12:30.073>we <00:12:30.076>then <00:12:30.079>get 00:12:30.080 --> 00:12:32.750 align:start position:0% <00:12:30.080>So, <00:12:30.207>we <00:12:30.334>then <00:12:30.461>get <00:12:30.589>we <00:12:30.716>reshuffle <00:12:30.843>the <00:12:30.970>order <00:12:31.098>to <00:12:31.225>really <00:12:31.352>get <00:12:31.480>the 00:12:32.750 --> 00:12:32.760 align:start position:0% <00:12:32.750>we <00:12:32.751>reshuffle <00:12:32.752>the <00:12:32.753>order <00:12:32.755>to <00:12:32.756>really <00:12:32.757>get <00:12:32.759>the 00:12:32.760 --> 00:12:34.230 align:start position:0% <00:12:32.760>we <00:12:32.907>reshuffle <00:12:33.054>the <00:12:33.201>order <00:12:33.348>to <00:12:33.495>really <00:12:33.642>get <00:12:33.789>the <00:12:33.936>relevant <00:12:34.083>experience, 00:12:34.230 --> 00:12:34.240 align:start position:0% <00:12:34.230>relevant <00:12:34.239>experience, 00:12:34.240 --> 00:12:36.990 align:start position:0% <00:12:34.240>relevant <00:12:34.395>experience, <00:12:34.551>and <00:12:34.706>then <00:12:34.862>take <00:12:35.017>the <00:12:35.173>top <00:12:35.328>20 <00:12:35.484>most <00:12:35.640>relevant 00:12:36.990 --> 00:12:37.000 align:start position:0% <00:12:36.990>and <00:12:36.991>then <00:12:36.992>take <00:12:36.993>the <00:12:36.995>top <00:12:36.996>20 <00:12:36.997>most <00:12:36.999>relevant 00:12:37.000 --> 00:12:38.829 align:start position:0% <00:12:37.000>and <00:12:37.116>then <00:12:37.233>take <00:12:37.350>the <00:12:37.466>top <00:12:37.583>20 <00:12:37.700>most <00:12:37.816>relevant <00:12:37.933>experiences <00:12:38.050>for <00:12:38.166>the <00:12:38.283>customer <00:12:38.400>as 00:12:38.829 --> 00:12:38.839 align:start position:0% <00:12:38.829>experiences <00:12:38.831>for <00:12:38.833>the <00:12:38.835>customer <00:12:38.838>as 00:12:38.839 --> 00:12:41.070 align:start position:0% <00:12:38.839>experiences <00:12:39.118>for <00:12:39.398>the <00:12:39.679>customer <00:12:39.959>as <00:12:40.238>candidates. 00:12:41.070 --> 00:12:41.080 align:start position:0% candidates. 00:12:41.080 --> 00:12:43.630 align:start position:0% <00:12:41.080>candidates. <00:12:41.255>Now <00:12:41.430>that <00:12:41.605>we <00:12:41.780>have <00:12:41.955>these <00:12:42.130>20 <00:12:42.305>candidates, <00:12:42.480>we 00:12:43.630 --> 00:12:43.640 align:start position:0% <00:12:43.630>Now <00:12:43.631>that <00:12:43.632>we <00:12:43.633>have <00:12:43.635>these <00:12:43.636>20 <00:12:43.637>candidates, <00:12:43.639>we 00:12:43.640 --> 00:12:46.110 align:start position:0% <00:12:43.640>Now <00:12:43.747>that <00:12:43.855>we <00:12:43.963>have <00:12:44.070>these <00:12:44.178>20 <00:12:44.286>candidates, <00:12:44.393>we <00:12:44.501>pass <00:12:44.609>them <00:12:44.716>to <00:12:44.824>our <00:12:44.932>relevance <00:12:45.040>filter. 00:12:46.110 --> 00:12:46.120 align:start position:0% <00:12:46.110>pass <00:12:46.111>them <00:12:46.113>to <00:12:46.115>our <00:12:46.117>relevance <00:12:46.119>filter. 00:12:46.120 --> 00:12:48.390 align:start position:0% <00:12:46.120>pass <00:12:46.236>them <00:12:46.353>to <00:12:46.470>our <00:12:46.586>relevance <00:12:46.703>filter. <00:12:46.820>This <00:12:46.936>is <00:12:47.053>kind <00:12:47.170>of <00:12:47.286>like <00:12:47.403>a <00:12:47.520>LM 00:12:48.390 --> 00:12:48.400 align:start position:0% <00:12:48.390>This <00:12:48.391>is <00:12:48.393>kind <00:12:48.394>of <00:12:48.396>like <00:12:48.397>a <00:12:48.399>LM 00:12:48.400 --> 00:12:50.750 align:start position:0% <00:12:48.400>This <00:12:48.507>is <00:12:48.615>kind <00:12:48.723>of <00:12:48.830>like <00:12:48.938>a <00:12:49.046>LM <00:12:49.153>model <00:12:49.261>that <00:12:49.369>is <00:12:49.476>looking <00:12:49.584>at <00:12:49.692>each <00:12:49.800>single 00:12:50.750 --> 00:12:50.760 align:start position:0% <00:12:50.750>model <00:12:50.751>that <00:12:50.753>is <00:12:50.754>looking <00:12:50.756>at <00:12:50.757>each <00:12:50.759>single 00:12:50.760 --> 00:12:52.990 align:start position:0% <00:12:50.760>model <00:12:50.935>that <00:12:51.110>is <00:12:51.285>looking <00:12:51.460>at <00:12:51.635>each <00:12:51.810>single <00:12:51.985>activity <00:12:52.160>individually, 00:12:52.990 --> 00:12:53.000 align:start position:0% <00:12:52.990>activity <00:12:52.999>individually, 00:12:53.000 --> 00:12:54.990 align:start position:0% <00:12:53.000>activity <00:12:53.175>individually, <00:12:53.350>and <00:12:53.525>checks <00:12:53.700>whether <00:12:53.875>they <00:12:54.050>are <00:12:54.225>relevant <00:12:54.400>or 00:12:54.990 --> 00:12:55.000 align:start position:0% <00:12:54.990>and <00:12:54.991>checks <00:12:54.993>whether <00:12:54.994>they <00:12:54.996>are <00:12:54.997>relevant <00:12:54.999>or 00:12:55.000 --> 00:12:56.430 align:start position:0% <00:12:55.000>and <00:12:55.183>checks <00:12:55.367>whether <00:12:55.551>they <00:12:55.735>are <00:12:55.919>relevant <00:12:56.103>or <00:12:56.286>not. 00:12:56.430 --> 00:12:56.440 align:start position:0% not. 00:12:56.440 --> 00:12:57.990 align:start position:0% <00:12:56.440>not. <00:12:56.788>So, <00:12:57.137>as <00:12:57.486>an <00:12:57.835>example, 00:12:57.990 --> 00:12:58.000 align:start position:0% <00:12:57.990>So, <00:12:57.993>as <00:12:57.996>an <00:12:57.999>example, 00:12:58.000 --> 00:12:59.950 align:start position:0% <00:12:58.000>So, <00:12:58.140>as <00:12:58.280>an <00:12:58.420>example, <00:12:58.560>from <00:12:58.700>from <00:12:58.840>the <00:12:58.980>retrieval <00:12:59.120>layer, <00:12:59.260>we <00:12:59.400>would 00:12:59.950 --> 00:12:59.960 align:start position:0% <00:12:59.950>from <00:12:59.951>from <00:12:59.953>the <00:12:59.954>retrieval <00:12:59.956>layer, <00:12:59.957>we <00:12:59.959>would 00:12:59.960 --> 00:13:02.390 align:start position:0% <00:12:59.960>from <00:13:00.067>from <00:13:00.175>the <00:13:00.283>retrieval <00:13:00.390>layer, <00:13:00.498>we <00:13:00.606>would <00:13:00.713>get <00:13:00.821>something <00:13:00.929>like <00:13:01.036>dinner <00:13:01.144>in <00:13:01.252>the <00:13:01.360>dark. 00:13:02.390 --> 00:13:02.400 align:start position:0% <00:13:02.390>get <00:13:02.391>something <00:13:02.393>like <00:13:02.394>dinner <00:13:02.396>in <00:13:02.397>the <00:13:02.399>dark. 00:13:02.400 --> 00:13:04.710 align:start position:0% <00:13:02.400>get <00:13:02.493>something <00:13:02.586>like <00:13:02.680>dinner <00:13:02.773>in <00:13:02.866>the <00:13:02.959>dark. <00:13:03.053>It <00:13:03.146>is <00:13:03.240>a <00:13:03.333>relevant <00:13:03.426>food <00:13:03.520>tour, <00:13:03.613>but <00:13:03.706>it's <00:13:03.800>not 00:13:04.710 --> 00:13:04.720 align:start position:0% <00:13:04.710>It <00:13:04.711>is <00:13:04.712>a <00:13:04.713>relevant <00:13:04.714>food <00:13:04.715>tour, <00:13:04.716>but <00:13:04.717>it's <00:13:04.719>not 00:13:04.720 --> 00:13:07.310 align:start position:0% <00:13:04.720>It <00:13:04.820>is <00:13:04.920>a <00:13:05.020>relevant <00:13:05.120>food <00:13:05.220>tour, <00:13:05.320>but <00:13:05.420>it's <00:13:05.520>not <00:13:05.620>something <00:13:05.720>relevant <00:13:05.820>to <00:13:05.920>our <00:13:06.020>local <00:13:06.120>market. 00:13:07.310 --> 00:13:07.320 align:start position:0% <00:13:07.310>something <00:13:07.311>relevant <00:13:07.313>to <00:13:07.315>our <00:13:07.317>local <00:13:07.319>market. 00:13:07.320 --> 00:13:09.310 align:start position:0% <00:13:07.320>something <00:13:07.420>relevant <00:13:07.520>to <00:13:07.620>our <00:13:07.720>local <00:13:07.820>market. <00:13:07.920>So, <00:13:08.020>in <00:13:08.120>this <00:13:08.220>case, <00:13:08.320>the <00:13:08.420>LM <00:13:08.520>decided <00:13:08.620>to <00:13:08.720>drop 00:13:09.310 --> 00:13:09.320 align:start position:0% <00:13:09.310>So, <00:13:09.311>in <00:13:09.312>this <00:13:09.313>case, <00:13:09.314>the <00:13:09.315>LM <00:13:09.316>decided <00:13:09.317>to <00:13:09.319>drop 00:13:09.320 --> 00:13:10.270 align:start position:0% <00:13:09.320>So, <00:13:09.415>in <00:13:09.510>this <00:13:09.605>case, <00:13:09.700>the <00:13:09.795>LM <00:13:09.890>decided <00:13:09.985>to <00:13:10.080>drop <00:13:10.175>it. 00:13:10.270 --> 00:13:10.280 align:start position:0% it. 00:13:10.280 --> 00:13:12.030 align:start position:0% <00:13:10.280>it. <00:13:10.560>And <00:13:10.839>the <00:13:11.120>Icebar <00:13:11.400>entrance <00:13:11.680>with 00:13:12.030 --> 00:13:12.040 align:start position:0% <00:13:12.030>And <00:13:12.032>the <00:13:12.034>Icebar <00:13:12.036>entrance <00:13:12.039>with 00:13:12.040 --> 00:13:13.870 align:start position:0% <00:13:12.040>And <00:13:12.195>the <00:13:12.351>Icebar <00:13:12.506>entrance <00:13:12.662>with <00:13:12.817>complimentary <00:13:12.973>drink <00:13:13.128>is <00:13:13.284>a <00:13:13.439>similar 00:13:13.870 --> 00:13:13.880 align:start position:0% <00:13:13.870>complimentary <00:13:13.872>drink <00:13:13.874>is <00:13:13.876>a <00:13:13.879>similar 00:13:13.880 --> 00:13:16.430 align:start position:0% <00:13:13.880>complimentary <00:13:14.160>drink <00:13:14.439>is <00:13:14.720>a <00:13:15.000>similar <00:13:15.280>example. 00:13:16.430 --> 00:13:16.440 align:start position:0% example. 00:13:16.440 --> 00:13:17.590 align:start position:0% <00:13:16.440>example. <00:13:16.587>So, <00:13:16.735>now <00:13:16.883>we <00:13:17.031>come <00:13:17.179>to <00:13:17.327>something <00:13:17.475>more 00:13:17.590 --> 00:13:17.600 align:start position:0% <00:13:17.590>So, <00:13:17.591>now <00:13:17.593>we <00:13:17.594>come <00:13:17.596>to <00:13:17.597>something <00:13:17.599>more 00:13:17.600 --> 00:13:19.670 align:start position:0% <00:13:17.600>So, <00:13:17.716>now <00:13:17.833>we <00:13:17.950>come <00:13:18.066>to <00:13:18.183>something <00:13:18.300>more <00:13:18.416>interesting, <00:13:18.533>which <00:13:18.650>is <00:13:18.766>downtown <00:13:18.883>food <00:13:19.000>tour 00:13:19.670 --> 00:13:19.680 align:start position:0% <00:13:19.670>interesting, <00:13:19.671>which <00:13:19.673>is <00:13:19.675>downtown <00:13:19.677>food <00:13:19.679>tour 00:13:19.680 --> 00:13:21.950 align:start position:0% <00:13:19.680>interesting, <00:13:19.835>which <00:13:19.991>is <00:13:20.146>downtown <00:13:20.302>food <00:13:20.457>tour <00:13:20.613>with <00:13:20.768>eight <00:13:20.924>local <00:13:21.079>tastings. 00:13:21.950 --> 00:13:21.960 align:start position:0% <00:13:21.950>with <00:13:21.953>eight <00:13:21.956>local <00:13:21.959>tastings. 00:13:21.960 --> 00:13:23.670 align:start position:0% <00:13:21.960>with <00:13:22.087>eight <00:13:22.214>local <00:13:22.341>tastings. <00:13:22.469>Here, <00:13:22.596>I <00:13:22.723>would <00:13:22.850>like <00:13:22.978>to <00:13:23.105>ask <00:13:23.232>the <00:13:23.360>audience, 00:13:23.670 --> 00:13:23.680 align:start position:0% <00:13:23.670>Here, <00:13:23.671>I <00:13:23.672>would <00:13:23.673>like <00:13:23.675>to <00:13:23.676>ask <00:13:23.677>the <00:13:23.679>audience, 00:13:23.680 --> 00:13:25.110 align:start position:0% <00:13:23.680>Here, <00:13:23.765>I <00:13:23.851>would <00:13:23.937>like <00:13:24.023>to <00:13:24.108>ask <00:13:24.194>the <00:13:24.280>audience, <00:13:24.366>uh <00:13:24.452>raise <00:13:24.538>your <00:13:24.623>hand, <00:13:24.709>who <00:13:24.795>thinks <00:13:24.881>that <00:13:24.967>this 00:13:25.110 --> 00:13:25.120 align:start position:0% <00:13:25.110>uh <00:13:25.111>raise <00:13:25.112>your <00:13:25.113>hand, <00:13:25.115>who <00:13:25.116>thinks <00:13:25.117>that <00:13:25.119>this 00:13:25.120 --> 00:13:26.790 align:start position:0% <00:13:25.120>uh <00:13:25.260>raise <00:13:25.400>your <00:13:25.540>hand, <00:13:25.680>who <00:13:25.820>thinks <00:13:25.960>that <00:13:26.100>this <00:13:26.240>is <00:13:26.380>a <00:13:26.520>relevant 00:13:26.790 --> 00:13:26.800 align:start position:0% <00:13:26.790>is <00:13:26.794>a <00:13:26.799>relevant 00:13:26.800 --> 00:13:31.670 align:start position:0% <00:13:26.800>is <00:13:26.955>a <00:13:27.111>relevant <00:13:27.266>activity <00:13:27.422>for <00:13:27.577>the <00:13:27.733>local <00:13:27.888>market <00:13:28.044>food <00:13:28.199>walk. 00:13:31.680 --> 00:13:33.630 align:start position:0% <00:13:31.680>Great. <00:13:31.880>I'm <00:13:32.079>not <00:13:32.280>seeing <00:13:32.479>so <00:13:32.680>many <00:13:32.880>hands, <00:13:33.079>so 00:13:33.630 --> 00:13:33.640 align:start position:0% <00:13:33.630>Great. <00:13:33.631>I'm <00:13:33.632>not <00:13:33.633>seeing <00:13:33.635>so <00:13:33.636>many <00:13:33.637>hands, <00:13:33.639>so 00:13:33.640 --> 00:13:35.990 align:start position:0% <00:13:33.640>Great. <00:13:33.733>I'm <00:13:33.826>not <00:13:33.920>seeing <00:13:34.013>so <00:13:34.106>many <00:13:34.199>hands, <00:13:34.293>so <00:13:34.386>most <00:13:34.480>of <00:13:34.573>you <00:13:34.666>are <00:13:34.760>right. <00:13:34.853>This <00:13:34.946>is <00:13:35.040>actually 00:13:35.990 --> 00:13:36.000 align:start position:0% <00:13:35.990>most <00:13:35.991>of <00:13:35.992>you <00:13:35.993>are <00:13:35.995>right. <00:13:35.996>This <00:13:35.997>is <00:13:35.999>actually 00:13:36.000 --> 00:13:39.430 align:start position:0% <00:13:36.000>most <00:13:36.107>of <00:13:36.215>you <00:13:36.323>are <00:13:36.430>right. <00:13:36.538>This <00:13:36.646>is <00:13:36.753>actually <00:13:36.861>something <00:13:36.969>that <00:13:37.076>is <00:13:37.184>not <00:13:37.292>really <00:13:37.400>relevant. 00:13:39.430 --> 00:13:39.440 align:start position:0% <00:13:39.430>something <00:13:39.431>that <00:13:39.433>is <00:13:39.435>not <00:13:39.437>really <00:13:39.439>relevant. 00:13:39.440 --> 00:13:40.910 align:start position:0% <00:13:39.440>something <00:13:39.550>that <00:13:39.660>is <00:13:39.770>not <00:13:39.881>really <00:13:39.991>relevant. <00:13:40.101>And <00:13:40.211>there's <00:13:40.322>something <00:13:40.432>that <00:13:40.542>you <00:13:40.652>can <00:13:40.762>see 00:13:40.910 --> 00:13:40.920 align:start position:0% <00:13:40.910>And <00:13:40.911>there's <00:13:40.913>something <00:13:40.914>that <00:13:40.915>you <00:13:40.917>can <00:13:40.919>see 00:13:40.920 --> 00:13:42.710 align:start position:0% <00:13:40.920>And <00:13:41.036>there's <00:13:41.153>something <00:13:41.270>that <00:13:41.386>you <00:13:41.503>can <00:13:41.620>see <00:13:41.736>when <00:13:41.853>you <00:13:41.969>read <00:13:42.086>through <00:13:42.203>this <00:13:42.319>food 00:13:42.710 --> 00:13:42.720 align:start position:0% <00:13:42.710>when <00:13:42.711>you <00:13:42.713>read <00:13:42.715>through <00:13:42.717>this <00:13:42.719>food 00:13:42.720 --> 00:13:44.430 align:start position:0% <00:13:42.720>when <00:13:42.827>you <00:13:42.935>read <00:13:43.043>through <00:13:43.150>this <00:13:43.258>food <00:13:43.366>description. <00:13:43.473>So, <00:13:43.581>you <00:13:43.689>need <00:13:43.796>to <00:13:43.904>go <00:13:44.012>to <00:13:44.120>the 00:13:44.430 --> 00:13:44.440 align:start position:0% <00:13:44.430>description. <00:13:44.431>So, <00:13:44.432>you <00:13:44.433>need <00:13:44.435>to <00:13:44.436>go <00:13:44.437>to <00:13:44.439>the 00:13:44.440 --> 00:13:46.350 align:start position:0% <00:13:44.440>description. <00:13:44.533>So, <00:13:44.626>you <00:13:44.720>need <00:13:44.813>to <00:13:44.906>go <00:13:45.000>to <00:13:45.093>the <00:13:45.186>product <00:13:45.280>page <00:13:45.373>and <00:13:45.466>read <00:13:45.560>through <00:13:45.653>it. <00:13:45.746>I <00:13:45.840>will 00:13:46.350 --> 00:13:46.360 align:start position:0% <00:13:46.350>product <00:13:46.351>page <00:13:46.352>and <00:13:46.353>read <00:13:46.355>through <00:13:46.356>it. <00:13:46.357>I <00:13:46.359>will 00:13:46.360 --> 00:13:47.830 align:start position:0% <00:13:46.360>product <00:13:46.454>page <00:13:46.549>and <00:13:46.643>read <00:13:46.738>through <00:13:46.832>it. <00:13:46.927>I <00:13:47.021>will <00:13:47.116>save <00:13:47.210>you <00:13:47.305>the <00:13:47.399>time <00:13:47.494>for <00:13:47.588>it, <00:13:47.683>but 00:13:47.830 --> 00:13:47.840 align:start position:0% <00:13:47.830>save <00:13:47.831>you <00:13:47.833>the <00:13:47.834>time <00:13:47.836>for <00:13:47.837>it, <00:13:47.839>but 00:13:47.840 --> 00:13:49.870 align:start position:0% <00:13:47.840>save <00:13:47.995>you <00:13:48.151>the <00:13:48.306>time <00:13:48.462>for <00:13:48.617>it, <00:13:48.773>but <00:13:48.928>essentially <00:13:49.084>the <00:13:49.240>product 00:13:49.870 --> 00:13:49.880 align:start position:0% <00:13:49.870>essentially <00:13:49.874>the <00:13:49.879>product 00:13:49.880 --> 00:13:51.630 align:start position:0% <00:13:49.880>essentially <00:13:50.113>the <00:13:50.346>product <00:13:50.580>talks <00:13:50.813>about <00:13:51.046>doner <00:13:51.280>kebab, 00:13:51.630 --> 00:13:51.640 align:start position:0% <00:13:51.630>talks <00:13:51.633>about <00:13:51.636>doner <00:13:51.639>kebab, 00:13:51.640 --> 00:13:53.550 align:start position:0% <00:13:51.640>talks <00:13:51.840>about <00:13:52.040>doner <00:13:52.240>kebab, <00:13:52.439>kind <00:13:52.640>of <00:13:52.840>like <00:13:53.040>beer, 00:13:53.550 --> 00:13:53.560 align:start position:0% <00:13:53.550>kind <00:13:53.553>of <00:13:53.555>like <00:13:53.559>beer, 00:13:53.560 --> 00:13:56.270 align:start position:0% <00:13:53.560>kind <00:13:53.793>of <00:13:54.026>like <00:13:54.260>beer, <00:13:54.493>currywurst, <00:13:54.726>and <00:13:54.959>also 00:13:56.270 --> 00:13:56.280 align:start position:0% <00:13:56.270>currywurst, <00:13:56.274>and <00:13:56.279>also 00:13:56.280 --> 00:13:57.910 align:start position:0% <00:13:56.280>currywurst, <00:13:56.435>and <00:13:56.591>also <00:13:56.746>a <00:13:56.902>pastry <00:13:57.057>shop. <00:13:57.213>So, <00:13:57.368>it's <00:13:57.524>not <00:13:57.680>really 00:13:57.910 --> 00:13:57.920 align:start position:0% <00:13:57.910>a <00:13:57.911>pastry <00:13:57.913>shop. <00:13:57.914>So, <00:13:57.915>it's <00:13:57.917>not <00:13:57.919>really 00:13:57.920 --> 00:14:01.830 align:start position:0% <00:13:57.920>a <00:13:58.120>pastry <00:13:58.319>shop. <00:13:58.520>So, <00:13:58.719>it's <00:13:58.920>not <00:13:59.120>really <00:13:59.319>relevant. 00:14:01.840 --> 00:14:03.390 align:start position:0% <00:14:01.840>Then <00:14:02.072>let's <00:14:02.305>come <00:14:02.537>to <00:14:02.770>our <00:14:03.002>final <00:14:03.235>example, 00:14:03.390 --> 00:14:03.400 align:start position:0% <00:14:03.390>Then <00:14:03.391>let's <00:14:03.393>come <00:14:03.394>to <00:14:03.396>our <00:14:03.397>final <00:14:03.399>example, 00:14:03.400 --> 00:14:05.670 align:start position:0% <00:14:03.400>Then <00:14:03.500>let's <00:14:03.600>come <00:14:03.699>to <00:14:03.800>our <00:14:03.900>final <00:14:04.000>example, <00:14:04.100>which <00:14:04.199>is <00:14:04.300>best <00:14:04.400>rated <00:14:04.500>food <00:14:04.600>tour <00:14:04.699>with <00:14:04.800>our 00:14:05.670 --> 00:14:05.680 align:start position:0% <00:14:05.670>which <00:14:05.671>is <00:14:05.672>best <00:14:05.673>rated <00:14:05.675>food <00:14:05.676>tour <00:14:05.677>with <00:14:05.679>our 00:14:05.680 --> 00:14:07.590 align:start position:0% <00:14:05.680>which <00:14:05.787>is <00:14:05.895>best <00:14:06.003>rated <00:14:06.110>food <00:14:06.218>tour <00:14:06.326>with <00:14:06.433>our <00:14:06.541>local <00:14:06.649>guides. <00:14:06.756>This <00:14:06.864>one <00:14:06.972>is <00:14:07.079>actually 00:14:07.590 --> 00:14:07.600 align:start position:0% <00:14:07.590>local <00:14:07.591>guides. <00:14:07.593>This <00:14:07.595>one <00:14:07.597>is <00:14:07.599>actually 00:14:07.600 --> 00:14:09.830 align:start position:0% <00:14:07.600>local <00:14:07.716>guides. <00:14:07.833>This <00:14:07.950>one <00:14:08.066>is <00:14:08.183>actually <00:14:08.300>relevant. <00:14:08.416>It <00:14:08.533>has <00:14:08.650>a <00:14:08.766>high <00:14:08.883>relevance <00:14:09.000>score. 00:14:09.830 --> 00:14:09.840 align:start position:0% <00:14:09.830>relevant. <00:14:09.831>It <00:14:09.833>has <00:14:09.834>a <00:14:09.836>high <00:14:09.837>relevance <00:14:09.839>score. 00:14:09.840 --> 00:14:12.230 align:start position:0% <00:14:09.840>relevant. <00:14:09.933>It <00:14:10.026>has <00:14:10.120>a <00:14:10.213>high <00:14:10.306>relevance <00:14:10.400>score. <00:14:10.493>And <00:14:10.586>the <00:14:10.680>reason <00:14:10.773>is <00:14:10.866>not <00:14:10.960>that <00:14:11.053>it <00:14:11.146>this <00:14:11.240>is 00:14:12.230 --> 00:14:12.240 align:start position:0% <00:14:12.230>And <00:14:12.231>the <00:14:12.232>reason <00:14:12.233>is <00:14:12.234>not <00:14:12.235>that <00:14:12.236>it <00:14:12.237>this <00:14:12.239>is 00:14:12.240 --> 00:14:14.630 align:start position:0% <00:14:12.240>And <00:14:12.327>the <00:14:12.415>reason <00:14:12.502>is <00:14:12.590>not <00:14:12.677>that <00:14:12.765>it <00:14:12.852>this <00:14:12.940>is <00:14:13.027>can <00:14:13.115>be <00:14:13.202>found <00:14:13.290>in <00:14:13.377>in <00:14:13.465>the <00:14:13.552>food <00:14:13.640>description. 00:14:14.630 --> 00:14:14.640 align:start position:0% <00:14:14.630>can <00:14:14.631>be <00:14:14.632>found <00:14:14.633>in <00:14:14.635>in <00:14:14.636>the <00:14:14.637>food <00:14:14.639>description. 00:14:14.640 --> 00:14:16.910 align:start position:0% <00:14:14.640>can <00:14:14.747>be <00:14:14.855>found <00:14:14.963>in <00:14:15.070>in <00:14:15.178>the <00:14:15.286>food <00:14:15.393>description. <00:14:15.501>Actually, <00:14:15.609>in <00:14:15.716>the <00:14:15.824>food <00:14:15.932>description <00:14:16.040>you 00:14:16.910 --> 00:14:16.920 align:start position:0% <00:14:16.910>Actually, <00:14:16.911>in <00:14:16.913>the <00:14:16.915>food <00:14:16.917>description <00:14:16.919>you 00:14:16.920 --> 00:14:18.870 align:start position:0% <00:14:16.920>Actually, <00:14:17.036>in <00:14:17.153>the <00:14:17.270>food <00:14:17.386>description <00:14:17.503>you <00:14:17.620>can <00:14:17.736>only <00:14:17.853>see <00:14:17.969>Bastille <00:14:18.086>Market, <00:14:18.203>which <00:14:18.319>is 00:14:18.870 --> 00:14:18.880 align:start position:0% <00:14:18.870>can <00:14:18.871>only <00:14:18.873>see <00:14:18.874>Bastille <00:14:18.876>Market, <00:14:18.877>which <00:14:18.879>is 00:14:18.880 --> 00:14:20.390 align:start position:0% <00:14:18.880>can <00:14:19.049>only <00:14:19.219>see <00:14:19.389>Bastille <00:14:19.559>Market, <00:14:19.729>which <00:14:19.899>is <00:14:20.069>not <00:14:20.239>really 00:14:20.390 --> 00:14:20.400 align:start position:0% <00:14:20.390>not <00:14:20.399>really 00:14:20.400 --> 00:14:22.150 align:start position:0% <00:14:20.400>not <00:14:20.750>really <00:14:21.100>a <00:14:21.449>big <00:14:21.800>indicator. 00:14:22.150 --> 00:14:22.160 align:start position:0% <00:14:22.150>a <00:14:22.154>big <00:14:22.159>indicator. 00:14:22.160 --> 00:14:24.550 align:start position:0% <00:14:22.160>a <00:14:22.276>big <00:14:22.393>indicator. <00:14:22.510>But <00:14:22.626>here <00:14:22.743>the <00:14:22.860>key <00:14:22.976>to <00:14:23.093>that <00:14:23.209>is <00:14:23.326>that <00:14:23.443>we <00:14:23.560>have 00:14:24.550 --> 00:14:24.560 align:start position:0% <00:14:24.550>But <00:14:24.550>here <00:14:24.551>the <00:14:24.553>key <00:14:24.554>to <00:14:24.555>that <00:14:24.555>is <00:14:24.556>that <00:14:24.558>we <00:14:24.559>have 00:14:24.560 --> 00:14:26.310 align:start position:0% <00:14:24.560>But <00:14:24.667>here <00:14:24.775>the <00:14:24.883>key <00:14:24.990>to <00:14:25.098>that <00:14:25.206>is <00:14:25.313>that <00:14:25.421>we <00:14:25.529>have <00:14:25.636>something <00:14:25.744>that's <00:14:25.852>called <00:14:25.959>itinerary 00:14:26.310 --> 00:14:26.320 align:start position:0% <00:14:26.310>something <00:14:26.313>that's <00:14:26.316>called <00:14:26.319>itinerary 00:14:26.320 --> 00:14:27.750 align:start position:0% <00:14:26.320>something <00:14:26.641>that's <00:14:26.963>called <00:14:27.285>itinerary <00:14:27.607>that 00:14:27.750 --> 00:14:27.760 align:start position:0% that 00:14:27.760 --> 00:14:29.510 align:start position:0% <00:14:27.760>that <00:14:27.960>a <00:14:28.160>supplier <00:14:28.360>can <00:14:28.560>provide <00:14:28.760>in <00:14:28.960>order <00:14:29.160>to 00:14:29.510 --> 00:14:29.520 align:start position:0% <00:14:29.510>a <00:14:29.511>supplier <00:14:29.513>can <00:14:29.514>provide <00:14:29.516>in <00:14:29.517>order <00:14:29.519>to 00:14:29.520 --> 00:14:30.950 align:start position:0% <00:14:29.520>a <00:14:29.627>supplier <00:14:29.734>can <00:14:29.841>provide <00:14:29.949>in <00:14:30.056>order <00:14:30.163>to <00:14:30.270>define <00:14:30.378>okay, <00:14:30.485>where <00:14:30.592>is <00:14:30.699>this <00:14:30.807>activity 00:14:30.950 --> 00:14:30.960 align:start position:0% <00:14:30.950>define <00:14:30.951>okay, <00:14:30.953>where <00:14:30.955>is <00:14:30.957>this <00:14:30.959>activity 00:14:30.960 --> 00:14:35.390 align:start position:0% <00:14:30.960>define <00:14:31.193>okay, <00:14:31.426>where <00:14:31.660>is <00:14:31.893>this <00:14:32.126>activity <00:14:32.360>heading? 00:14:35.400 --> 00:14:37.270 align:start position:0% <00:14:35.400>Great. <00:14:35.600>So, <00:14:35.800>now <00:14:36.000>that <00:14:36.199>we <00:14:36.400>have <00:14:36.600>seen <00:14:36.800>this 00:14:37.270 --> 00:14:37.280 align:start position:0% <00:14:37.270>Great. <00:14:37.271>So, <00:14:37.272>now <00:14:37.273>that <00:14:37.275>we <00:14:37.276>have <00:14:37.277>seen <00:14:37.279>this 00:14:37.280 --> 00:14:39.150 align:start position:0% <00:14:37.280>Great. <00:14:37.367>So, <00:14:37.454>now <00:14:37.542>that <00:14:37.630>we <00:14:37.717>have <00:14:37.805>seen <00:14:37.892>this <00:14:37.980>complex <00:14:38.067>happy <00:14:38.155>path, <00:14:38.242>we <00:14:38.329>have <00:14:38.417>a <00:14:38.505>lot <00:14:38.592>of <00:14:38.680>API 00:14:39.150 --> 00:14:39.160 align:start position:0% <00:14:39.150>complex <00:14:39.151>happy <00:14:39.152>path, <00:14:39.153>we <00:14:39.154>have <00:14:39.155>a <00:14:39.156>lot <00:14:39.157>of <00:14:39.159>API 00:14:39.160 --> 00:14:40.390 align:start position:0% <00:14:39.160>complex <00:14:39.283>happy <00:14:39.406>path, <00:14:39.529>we <00:14:39.651>have <00:14:39.775>a <00:14:39.898>lot <00:14:40.021>of <00:14:40.144>API <00:14:40.266>calls. 00:14:40.390 --> 00:14:40.400 align:start position:0% calls. 00:14:40.400 --> 00:14:42.310 align:start position:0% <00:14:40.400>calls. <00:14:40.555>The <00:14:40.711>next <00:14:40.866>question <00:14:41.022>is <00:14:41.177>how <00:14:41.333>do <00:14:41.488>we <00:14:41.644>make <00:14:41.800>it 00:14:42.310 --> 00:14:42.320 align:start position:0% <00:14:42.310>The <00:14:42.311>next <00:14:42.312>question <00:14:42.313>is <00:14:42.314>how <00:14:42.315>do <00:14:42.316>we <00:14:42.317>make <00:14:42.319>it 00:14:42.320 --> 00:14:44.190 align:start position:0% <00:14:42.320>The <00:14:42.420>next <00:14:42.520>question <00:14:42.620>is <00:14:42.720>how <00:14:42.820>do <00:14:42.920>we <00:14:43.020>make <00:14:43.120>it <00:14:43.220>fast, <00:14:43.320>resilient, <00:14:43.420>and <00:14:43.520>within <00:14:43.620>budget? <00:14:43.720>So, 00:14:44.190 --> 00:14:44.200 align:start position:0% <00:14:44.190>fast, <00:14:44.191>resilient, <00:14:44.193>and <00:14:44.195>within <00:14:44.197>budget? <00:14:44.199>So, 00:14:44.200 --> 00:14:46.670 align:start position:0% <00:14:44.200>fast, <00:14:44.307>resilient, <00:14:44.415>and <00:14:44.523>within <00:14:44.630>budget? <00:14:44.738>So, <00:14:44.846>here <00:14:44.953>I <00:14:45.061>would <00:14:45.169>like <00:14:45.276>to <00:14:45.384>show <00:14:45.492>some <00:14:45.600>patterns 00:14:46.670 --> 00:14:46.680 align:start position:0% <00:14:46.670>here <00:14:46.671>I <00:14:46.672>would <00:14:46.673>like <00:14:46.675>to <00:14:46.676>show <00:14:46.677>some <00:14:46.679>patterns 00:14:46.680 --> 00:14:48.070 align:start position:0% <00:14:46.680>here <00:14:46.805>I <00:14:46.930>would <00:14:47.055>like <00:14:47.180>to <00:14:47.305>show <00:14:47.430>some <00:14:47.555>patterns <00:14:47.680>that <00:14:47.805>we <00:14:47.931>applied. 00:14:48.070 --> 00:14:48.080 align:start position:0% <00:14:48.070>that <00:14:48.074>we <00:14:48.079>applied. 00:14:48.080 --> 00:14:49.870 align:start position:0% <00:14:48.080>that <00:14:48.235>we <00:14:48.391>applied. <00:14:48.546>We <00:14:48.702>do <00:14:48.857>essentially <00:14:49.013>in <00:14:49.168>order <00:14:49.324>to <00:14:49.480>improve 00:14:49.870 --> 00:14:49.880 align:start position:0% <00:14:49.870>We <00:14:49.871>do <00:14:49.873>essentially <00:14:49.874>in <00:14:49.876>order <00:14:49.877>to <00:14:49.879>improve 00:14:49.880 --> 00:14:52.310 align:start position:0% <00:14:49.880>We <00:14:49.980>do <00:14:50.080>essentially <00:14:50.180>in <00:14:50.280>order <00:14:50.380>to <00:14:50.480>improve <00:14:50.580>the <00:14:50.680>latency <00:14:50.780>fan <00:14:50.880>out <00:14:50.980>with <00:14:51.080>a <00:14:51.180>cutoff. <00:14:51.280>So, 00:14:52.310 --> 00:14:52.320 align:start position:0% <00:14:52.310>the <00:14:52.311>latency <00:14:52.312>fan <00:14:52.313>out <00:14:52.315>with <00:14:52.316>a <00:14:52.317>cutoff. <00:14:52.319>So, 00:14:52.320 --> 00:14:54.030 align:start position:0% <00:14:52.320>the <00:14:52.413>latency <00:14:52.506>fan <00:14:52.600>out <00:14:52.693>with <00:14:52.786>a <00:14:52.880>cutoff. <00:14:52.973>So, <00:14:53.066>we <00:14:53.160>parallelize <00:14:53.253>all <00:14:53.346>of <00:14:53.440>the <00:14:53.533>API <00:14:53.626>calls <00:14:53.720>and 00:14:54.030 --> 00:14:54.040 align:start position:0% <00:14:54.030>we <00:14:54.031>parallelize <00:14:54.032>all <00:14:54.033>of <00:14:54.035>the <00:14:54.036>API <00:14:54.037>calls <00:14:54.039>and 00:14:54.040 --> 00:14:55.829 align:start position:0% <00:14:54.040>we <00:14:54.140>parallelize <00:14:54.240>all <00:14:54.339>of <00:14:54.439>the <00:14:54.540>API <00:14:54.640>calls <00:14:54.740>and <00:14:54.839>we <00:14:54.939>don't <00:14:55.040>take <00:14:55.140>the <00:14:55.240>results <00:14:55.339>that <00:14:55.439>timed 00:14:55.829 --> 00:14:55.839 align:start position:0% <00:14:55.829>we <00:14:55.830>don't <00:14:55.832>take <00:14:55.833>the <00:14:55.834>results <00:14:55.836>that <00:14:55.838>timed 00:14:55.839 --> 00:14:57.670 align:start position:0% <00:14:55.839>we <00:14:56.039>don't <00:14:56.238>take <00:14:56.439>the <00:14:56.638>results <00:14:56.839>that <00:14:57.039>timed <00:14:57.238>out. 00:14:57.670 --> 00:14:57.680 align:start position:0% out. 00:14:57.680 --> 00:14:59.350 align:start position:0% <00:14:57.680>out. <00:14:57.854>Then <00:14:58.030>the <00:14:58.204>next <00:14:58.380>pattern <00:14:58.555>that <00:14:58.729>we <00:14:58.905>applied <00:14:59.079>is 00:14:59.350 --> 00:14:59.360 align:start position:0% <00:14:59.350>Then <00:14:59.351>the <00:14:59.352>next <00:14:59.353>pattern <00:14:59.355>that <00:14:59.356>we <00:14:59.357>applied <00:14:59.359>is 00:14:59.360 --> 00:15:00.829 align:start position:0% <00:14:59.360>Then <00:14:59.461>the <00:14:59.563>next <00:14:59.665>pattern <00:14:59.766>that <00:14:59.868>we <00:14:59.970>applied <00:15:00.071>is <00:15:00.173>essentially <00:15:00.275>design <00:15:00.377>for <00:15:00.478>failure. <00:15:00.580>So, <00:15:00.682>we 00:15:00.829 --> 00:15:00.839 align:start position:0% <00:15:00.829>essentially <00:15:00.830>design <00:15:00.832>for <00:15:00.834>failure. <00:15:00.836>So, <00:15:00.838>we 00:15:00.839 --> 00:15:04.470 align:start position:0% <00:15:00.839>essentially <00:15:00.979>design <00:15:01.119>for <00:15:01.259>failure. <00:15:01.399>So, <00:15:01.539>we <00:15:01.679>define <00:15:01.819>four <00:15:01.959>nodes <00:15:02.099>and <00:15:02.239>we 00:15:04.470 --> 00:15:04.480 align:start position:0% <00:15:04.470>define <00:15:04.472>four <00:15:04.474>nodes <00:15:04.476>and <00:15:04.479>we 00:15:04.480 --> 00:15:06.670 align:start position:0% <00:15:04.480>define <00:15:04.596>four <00:15:04.713>nodes <00:15:04.830>and <00:15:04.946>we <00:15:05.063>have <00:15:05.180>a <00:15:05.296>path <00:15:05.413>whenever <00:15:05.530>they <00:15:05.646>don't <00:15:05.763>have <00:15:05.880>any 00:15:06.670 --> 00:15:06.680 align:start position:0% <00:15:06.670>have <00:15:06.671>a <00:15:06.672>path <00:15:06.673>whenever <00:15:06.675>they <00:15:06.676>don't <00:15:06.677>have <00:15:06.679>any 00:15:06.680 --> 00:15:09.070 align:start position:0% <00:15:06.680>have <00:15:06.854>a <00:15:07.030>path <00:15:07.204>whenever <00:15:07.380>they <00:15:07.555>don't <00:15:07.729>have <00:15:07.905>any <00:15:08.079>results. 00:15:09.070 --> 00:15:09.080 align:start position:0% results. 00:15:09.080 --> 00:15:11.670 align:start position:0% <00:15:09.080>results. <00:15:09.280>And <00:15:09.480>then <00:15:09.680>lastly, <00:15:09.880>there's <00:15:10.080>this <00:15:10.280>we <00:15:10.480>tailor 00:15:11.670 --> 00:15:11.680 align:start position:0% <00:15:11.670>And <00:15:11.671>then <00:15:11.673>lastly, <00:15:11.674>there's <00:15:11.675>this <00:15:11.677>we <00:15:11.679>tailor 00:15:11.680 --> 00:15:13.470 align:start position:0% <00:15:11.680>And <00:15:11.787>then <00:15:11.895>lastly, <00:15:12.003>there's <00:15:12.110>this <00:15:12.218>we <00:15:12.326>tailor <00:15:12.433>the <00:15:12.541>AI <00:15:12.649>based <00:15:12.756>on <00:15:12.864>the <00:15:12.972>complexity. <00:15:13.079>So, 00:15:13.470 --> 00:15:13.480 align:start position:0% <00:15:13.470>the <00:15:13.471>AI <00:15:13.473>based <00:15:13.474>on <00:15:13.476>the <00:15:13.477>complexity. <00:15:13.479>So, 00:15:13.480 --> 00:15:14.910 align:start position:0% <00:15:13.480>the <00:15:13.587>AI <00:15:13.694>based <00:15:13.801>on <00:15:13.909>the <00:15:14.016>complexity. <00:15:14.123>So, <00:15:14.230>initially <00:15:14.338>when <00:15:14.445>we <00:15:14.552>are <00:15:14.659>building <00:15:14.766>the 00:15:14.910 --> 00:15:14.920 align:start position:0% <00:15:14.910>initially <00:15:14.911>when <00:15:14.913>we <00:15:14.915>are <00:15:14.917>building <00:15:14.919>the 00:15:14.920 --> 00:15:17.030 align:start position:0% <00:15:14.920>initially <00:15:15.047>when <00:15:15.174>we <00:15:15.301>are <00:15:15.429>building <00:15:15.556>the <00:15:15.683>system, <00:15:15.810>we <00:15:15.938>are <00:15:16.065>running <00:15:16.192>the <00:15:16.319>whole 00:15:17.030 --> 00:15:17.040 align:start position:0% <00:15:17.030>system, <00:15:17.031>we <00:15:17.033>are <00:15:17.035>running <00:15:17.037>the <00:15:17.039>whole 00:15:17.040 --> 00:15:19.990 align:start position:0% <00:15:17.040>system, <00:15:17.195>we <00:15:17.351>are <00:15:17.506>running <00:15:17.662>the <00:15:17.817>whole <00:15:17.973>workflow, <00:15:18.128>the <00:15:18.284>best <00:15:18.439>models 00:15:19.990 --> 00:15:20.000 align:start position:0% <00:15:19.990>workflow, <00:15:19.993>the <00:15:19.996>best <00:15:19.999>models 00:15:20.000 --> 00:15:21.630 align:start position:0% <00:15:20.000>workflow, <00:15:20.155>the <00:15:20.311>best <00:15:20.466>models <00:15:20.622>end-to-end <00:15:20.777>for <00:15:20.933>every <00:15:21.088>user <00:15:21.244>request. <00:15:21.400>But 00:15:21.630 --> 00:15:21.640 align:start position:0% <00:15:21.630>end-to-end <00:15:21.631>for <00:15:21.633>every <00:15:21.635>user <00:15:21.637>request. <00:15:21.639>But 00:15:21.640 --> 00:15:23.270 align:start position:0% <00:15:21.640>end-to-end <00:15:21.747>for <00:15:21.855>every <00:15:21.963>user <00:15:22.070>request. <00:15:22.178>But <00:15:22.286>then <00:15:22.393>we <00:15:22.501>saw <00:15:22.609>that <00:15:22.716>we <00:15:22.824>also <00:15:22.932>have <00:15:23.040>these 00:15:23.270 --> 00:15:23.280 align:start position:0% <00:15:23.270>then <00:15:23.271>we <00:15:23.272>saw <00:15:23.273>that <00:15:23.275>we <00:15:23.276>also <00:15:23.277>have <00:15:23.279>these 00:15:23.280 --> 00:15:25.830 align:start position:0% <00:15:23.280>then <00:15:23.396>we <00:15:23.513>saw <00:15:23.630>that <00:15:23.746>we <00:15:23.863>also <00:15:23.980>have <00:15:24.096>these <00:15:24.213>medium-size <00:15:24.329>complex <00:15:24.446>queries <00:15:24.563>such <00:15:24.680>as 00:15:25.830 --> 00:15:25.840 align:start position:0% <00:15:25.830>medium-size <00:15:25.832>complex <00:15:25.834>queries <00:15:25.836>such <00:15:25.839>as 00:15:25.840 --> 00:15:27.590 align:start position:0% <00:15:25.840>medium-size <00:15:25.967>complex <00:15:26.094>queries <00:15:26.221>such <00:15:26.349>as <00:15:26.476>"photos <00:15:26.603>in <00:15:26.730>Berlin <00:15:26.858>under <00:15:26.985>50 <00:15:27.112>in <00:15:27.240>the 00:15:27.590 --> 00:15:27.600 align:start position:0% <00:15:27.590>"photos <00:15:27.591>in <00:15:27.593>Berlin <00:15:27.594>under <00:15:27.596>50 <00:15:27.597>in <00:15:27.599>the 00:15:27.600 --> 00:15:28.750 align:start position:0% <00:15:27.600>"photos <00:15:27.747>in <00:15:27.895>Berlin <00:15:28.043>under <00:15:28.191>50 <00:15:28.339>in <00:15:28.487>the <00:15:28.635>afternoon". 00:15:28.750 --> 00:15:28.760 align:start position:0% afternoon". 00:15:28.760 --> 00:15:30.870 align:start position:0% <00:15:28.760>afternoon". <00:15:28.935>So, <00:15:29.110>there <00:15:29.285>we <00:15:29.460>just <00:15:29.635>look <00:15:29.810>at <00:15:29.985>a <00:15:30.160>high-level 00:15:30.870 --> 00:15:30.880 align:start position:0% <00:15:30.870>So, <00:15:30.871>there <00:15:30.872>we <00:15:30.873>just <00:15:30.875>look <00:15:30.876>at <00:15:30.877>a <00:15:30.879>high-level 00:15:30.880 --> 00:15:32.790 align:start position:0% <00:15:30.880>So, <00:15:30.980>there <00:15:31.080>we <00:15:31.180>just <00:15:31.280>look <00:15:31.380>at <00:15:31.480>a <00:15:31.580>high-level <00:15:31.680>description <00:15:31.780>of <00:15:31.880>the <00:15:31.980>activity, <00:15:32.080>use <00:15:32.180>also <00:15:32.280>a 00:15:32.790 --> 00:15:32.800 align:start position:0% <00:15:32.790>description <00:15:32.791>of <00:15:32.793>the <00:15:32.794>activity, <00:15:32.795>use <00:15:32.797>also <00:15:32.799>a 00:15:32.800 --> 00:15:34.070 align:start position:0% <00:15:32.800>description <00:15:32.942>of <00:15:33.085>the <00:15:33.228>activity, <00:15:33.371>use <00:15:33.514>also <00:15:33.657>a <00:15:33.800>much <00:15:33.943>smaller 00:15:34.070 --> 00:15:34.080 align:start position:0% <00:15:34.070>much <00:15:34.079>smaller 00:15:34.080 --> 00:15:35.630 align:start position:0% <00:15:34.080>much <00:15:34.235>smaller <00:15:34.390>uh <00:15:34.545>large <00:15:34.700>language <00:15:34.855>model <00:15:35.010>in <00:15:35.165>order <00:15:35.320>to <00:15:35.475>do 00:15:35.630 --> 00:15:35.640 align:start position:0% <00:15:35.630>uh <00:15:35.631>large <00:15:35.632>language <00:15:35.633>model <00:15:35.635>in <00:15:35.636>order <00:15:35.637>to <00:15:35.639>do 00:15:35.640 --> 00:15:36.910 align:start position:0% <00:15:35.640>uh <00:15:35.766>large <00:15:35.894>language <00:15:36.021>model <00:15:36.148>in <00:15:36.275>order <00:15:36.401>to <00:15:36.529>do <00:15:36.656>the <00:15:36.783>prediction. 00:15:36.910 --> 00:15:36.920 align:start position:0% <00:15:36.910>the <00:15:36.919>prediction. 00:15:36.920 --> 00:15:38.830 align:start position:0% <00:15:36.920>the <00:15:37.060>prediction. <00:15:37.199>And <00:15:37.339>also <00:15:37.479>at <00:15:37.620>the <00:15:37.760>same <00:15:37.900>time, <00:15:38.040>we <00:15:38.180>have <00:15:38.319>some 00:15:38.830 --> 00:15:38.840 align:start position:0% <00:15:38.830>And <00:15:38.831>also <00:15:38.832>at <00:15:38.833>the <00:15:38.834>same <00:15:38.835>time, <00:15:38.836>we <00:15:38.837>have <00:15:38.839>some 00:15:38.840 --> 00:15:40.590 align:start position:0% <00:15:38.840>And <00:15:38.933>also <00:15:39.026>at <00:15:39.120>the <00:15:39.213>same <00:15:39.306>time, <00:15:39.400>we <00:15:39.493>have <00:15:39.586>some <00:15:39.680>very <00:15:39.773>simple <00:15:39.866>queries <00:15:39.960>such <00:15:40.053>as <00:15:40.146>a <00:15:40.240>location 00:15:40.590 --> 00:15:40.600 align:start position:0% <00:15:40.590>very <00:15:40.591>simple <00:15:40.593>queries <00:15:40.594>such <00:15:40.596>as <00:15:40.597>a <00:15:40.599>location 00:15:40.600 --> 00:15:42.230 align:start position:0% <00:15:40.600>very <00:15:40.700>simple <00:15:40.800>queries <00:15:40.900>such <00:15:41.000>as <00:15:41.100>a <00:15:41.200>location <00:15:41.300>lookup <00:15:41.400>where <00:15:41.500>we <00:15:41.600>don't <00:15:41.700>need <00:15:41.800>to <00:15:41.900>add <00:15:42.000>an 00:15:42.230 --> 00:15:42.240 align:start position:0% <00:15:42.230>lookup <00:15:42.231>where <00:15:42.232>we <00:15:42.233>don't <00:15:42.235>need <00:15:42.236>to <00:15:42.237>add <00:15:42.239>an 00:15:42.240 --> 00:15:44.670 align:start position:0% <00:15:42.240>lookup <00:15:42.356>where <00:15:42.473>we <00:15:42.590>don't <00:15:42.706>need <00:15:42.823>to <00:15:42.940>add <00:15:43.056>an <00:15:43.173>additional <00:15:43.290>LM <00:15:43.406>filter <00:15:43.523>on <00:15:43.640>top. 00:15:44.670 --> 00:15:44.680 align:start position:0% <00:15:44.670>additional <00:15:44.672>LM <00:15:44.674>filter <00:15:44.676>on <00:15:44.679>top. 00:15:44.680 --> 00:15:46.550 align:start position:0% <00:15:44.680>additional <00:15:44.787>LM <00:15:44.895>filter <00:15:45.003>on <00:15:45.110>top. <00:15:45.218>And <00:15:45.326>all <00:15:45.433>of <00:15:45.541>this <00:15:45.649>is <00:15:45.756>done <00:15:45.864>by <00:15:45.972>our <00:15:46.079>planner 00:15:46.550 --> 00:15:46.560 align:start position:0% <00:15:46.550>And <00:15:46.551>all <00:15:46.552>of <00:15:46.553>this <00:15:46.554>is <00:15:46.555>done <00:15:46.556>by <00:15:46.557>our <00:15:46.559>planner 00:15:46.560 --> 00:15:48.990 align:start position:0% <00:15:46.560>And <00:15:46.653>all <00:15:46.746>of <00:15:46.839>this <00:15:46.933>is <00:15:47.026>done <00:15:47.119>by <00:15:47.213>our <00:15:47.306>planner <00:15:47.400>steps. <00:15:47.493>So, <00:15:47.586>it <00:15:47.680>takes <00:15:47.773>into <00:15:47.866>account <00:15:47.959>the 00:15:48.990 --> 00:15:49.000 align:start position:0% <00:15:48.990>steps. <00:15:48.991>So, <00:15:48.993>it <00:15:48.994>takes <00:15:48.996>into <00:15:48.997>account <00:15:48.999>the 00:15:49.000 --> 00:15:51.030 align:start position:0% <00:15:49.000>steps. <00:15:49.116>So, <00:15:49.233>it <00:15:49.350>takes <00:15:49.466>into <00:15:49.583>account <00:15:49.700>the <00:15:49.816>original <00:15:49.933>query <00:15:50.050>and <00:15:50.166>then <00:15:50.283>also <00:15:50.400>predicts 00:15:51.030 --> 00:15:51.040 align:start position:0% <00:15:51.030>original <00:15:51.031>query <00:15:51.033>and <00:15:51.035>then <00:15:51.037>also <00:15:51.039>predicts 00:15:51.040 --> 00:15:52.990 align:start position:0% <00:15:51.040>original <00:15:51.156>query <00:15:51.273>and <00:15:51.390>then <00:15:51.506>also <00:15:51.623>predicts <00:15:51.740>how <00:15:51.856>what's <00:15:51.973>the <00:15:52.089>complexity <00:15:52.206>of <00:15:52.323>the <00:15:52.439>user 00:15:52.990 --> 00:15:53.000 align:start position:0% <00:15:52.990>how <00:15:52.991>what's <00:15:52.993>the <00:15:52.994>complexity <00:15:52.996>of <00:15:52.997>the <00:15:52.999>user 00:15:53.000 --> 00:15:56.510 align:start position:0% <00:15:53.000>how <00:15:53.200>what's <00:15:53.400>the <00:15:53.600>complexity <00:15:53.800>of <00:15:54.000>the <00:15:54.200>user <00:15:54.400>request. 00:15:56.520 --> 00:15:58.150 align:start position:0% <00:15:56.520>Great. <00:15:56.800>So, <00:15:57.079>that's <00:15:57.360>essentially <00:15:57.640>the <00:15:57.920>whole 00:15:58.150 --> 00:15:58.160 align:start position:0% <00:15:58.150>Great. <00:15:58.151>So, <00:15:58.153>that's <00:15:58.155>essentially <00:15:58.157>the <00:15:58.159>whole 00:15:58.160 --> 00:16:00.550 align:start position:0% <00:15:58.160>Great. <00:15:58.287>So, <00:15:58.414>that's <00:15:58.541>essentially <00:15:58.669>the <00:15:58.796>whole <00:15:58.923>system <00:15:59.050>that <00:15:59.178>we <00:15:59.305>built, <00:15:59.432>powerful, <00:15:59.560>fast, 00:16:00.550 --> 00:16:00.560 align:start position:0% <00:16:00.550>system <00:16:00.551>that <00:16:00.553>we <00:16:00.555>built, <00:16:00.557>powerful, <00:16:00.559>fast, 00:16:00.560 --> 00:16:02.870 align:start position:0% <00:16:00.560>system <00:16:00.699>that <00:16:00.839>we <00:16:00.979>built, <00:16:01.119>powerful, <00:16:01.260>fast, <00:16:01.400>resilient <00:16:01.540>for <00:16:01.680>the <00:16:01.819>user, <00:16:01.959>uh 00:16:02.870 --> 00:16:02.880 align:start position:0% <00:16:02.870>resilient <00:16:02.872>for <00:16:02.874>the <00:16:02.876>user, <00:16:02.879>uh 00:16:02.880 --> 00:16:04.750 align:start position:0% <00:16:02.880>resilient <00:16:03.035>for <00:16:03.191>the <00:16:03.346>user, <00:16:03.502>uh <00:16:03.657>affordable <00:16:03.813>to <00:16:03.968>operate. <00:16:04.124>But, <00:16:04.280>actually 00:16:04.750 --> 00:16:04.760 align:start position:0% <00:16:04.750>affordable <00:16:04.752>to <00:16:04.754>operate. <00:16:04.756>But, <00:16:04.759>actually 00:16:04.760 --> 00:16:06.270 align:start position:0% <00:16:04.760>affordable <00:16:04.873>to <00:16:04.986>operate. <00:16:05.099>But, <00:16:05.213>actually <00:16:05.326>building <00:16:05.439>the <00:16:05.552>agent <00:16:05.665>was <00:16:05.779>kind <00:16:05.892>of <00:16:06.005>only <00:16:06.119>the 00:16:06.270 --> 00:16:06.280 align:start position:0% <00:16:06.270>building <00:16:06.271>the <00:16:06.272>agent <00:16:06.273>was <00:16:06.275>kind <00:16:06.276>of <00:16:06.277>only <00:16:06.279>the 00:16:06.280 --> 00:16:07.870 align:start position:0% <00:16:06.280>building <00:16:06.380>the <00:16:06.480>agent <00:16:06.579>was <00:16:06.680>kind <00:16:06.780>of <00:16:06.880>only <00:16:06.980>the <00:16:07.079>half <00:16:07.180>of <00:16:07.280>the <00:16:07.380>story <00:16:07.480>because <00:16:07.579>the <00:16:07.680>big 00:16:07.870 --> 00:16:07.880 align:start position:0% <00:16:07.870>half <00:16:07.871>of <00:16:07.873>the <00:16:07.874>story <00:16:07.876>because <00:16:07.877>the <00:16:07.879>big 00:16:07.880 --> 00:16:11.390 align:start position:0% <00:16:07.880>half <00:16:07.996>of <00:16:08.113>the <00:16:08.230>story <00:16:08.346>because <00:16:08.463>the <00:16:08.580>big <00:16:08.696>challenge <00:16:08.813>is <00:16:08.930>making <00:16:09.046>it <00:16:09.163>resilient <00:16:09.280>and 00:16:11.390 --> 00:16:11.400 align:start position:0% <00:16:11.390>challenge <00:16:11.391>is <00:16:11.393>making <00:16:11.395>it <00:16:11.397>resilient <00:16:11.399>and 00:16:11.400 --> 00:16:12.910 align:start position:0% <00:16:11.400>challenge <00:16:11.504>is <00:16:11.609>making <00:16:11.713>it <00:16:11.818>resilient <00:16:11.922>and <00:16:12.027>making <00:16:12.131>sure <00:16:12.236>that <00:16:12.340>every <00:16:12.445>change <00:16:12.549>that <00:16:12.654>we <00:16:12.759>do 00:16:12.910 --> 00:16:12.920 align:start position:0% <00:16:12.910>making <00:16:12.911>sure <00:16:12.912>that <00:16:12.913>every <00:16:12.915>change <00:16:12.916>that <00:16:12.917>we <00:16:12.919>do 00:16:12.920 --> 00:16:14.590 align:start position:0% <00:16:12.920>making <00:16:13.020>sure <00:16:13.120>that <00:16:13.219>every <00:16:13.319>change <00:16:13.420>that <00:16:13.520>we <00:16:13.620>do <00:16:13.719>don't <00:16:13.819>break <00:16:13.920>it. <00:16:14.020>And <00:16:14.120>that's <00:16:14.219>where <00:16:14.319>I'm 00:16:14.590 --> 00:16:14.600 align:start position:0% <00:16:14.590>don't <00:16:14.591>break <00:16:14.593>it. <00:16:14.594>And <00:16:14.596>that's <00:16:14.597>where <00:16:14.599>I'm 00:16:14.600 --> 00:16:17.670 align:start position:0% <00:16:14.600>don't <00:16:14.707>break <00:16:14.815>it. <00:16:14.923>And <00:16:15.030>that's <00:16:15.138>where <00:16:15.246>I'm <00:16:15.353>going <00:16:15.461>to <00:16:15.569>hand <00:16:15.676>it <00:16:15.784>over <00:16:15.892>to <00:16:16.000>Gianpaolo. 00:16:17.670 --> 00:16:17.680 align:start position:0% <00:16:17.670>going <00:16:17.671>to <00:16:17.673>hand <00:16:17.674>it <00:16:17.675>over <00:16:17.677>to <00:16:17.679>Gianpaolo. 00:16:17.680 --> 00:16:20.190 align:start position:0% <00:16:17.680>going <00:16:17.780>to <00:16:17.880>hand <00:16:17.979>it <00:16:18.079>over <00:16:18.180>to <00:16:18.280>Gianpaolo. <00:16:18.380>>> <00:16:18.479>Thanks, <00:16:18.579>Steven. <00:16:18.680>So, <00:16:18.780>now <00:16:18.880>the <00:16:18.979>part <00:16:19.079>where 00:16:20.190 --> 00:16:20.200 align:start position:0% <00:16:20.190>>> <00:16:20.191>Thanks, <00:16:20.192>Steven. <00:16:20.193>So, <00:16:20.195>now <00:16:20.196>the <00:16:20.197>part <00:16:20.199>where 00:16:20.200 --> 00:16:22.710 align:start position:0% <00:16:20.200>>> <00:16:20.287>Thanks, <00:16:20.375>Steven. <00:16:20.462>So, <00:16:20.550>now <00:16:20.637>the <00:16:20.725>part <00:16:20.812>where <00:16:20.900>we <00:16:20.987>show <00:16:21.075>how <00:16:21.162>we <00:16:21.250>were <00:16:21.337>able <00:16:21.425>to <00:16:21.512>make <00:16:21.600>changes 00:16:22.710 --> 00:16:22.720 align:start position:0% <00:16:22.710>we <00:16:22.711>show <00:16:22.712>how <00:16:22.713>we <00:16:22.714>were <00:16:22.715>able <00:16:22.716>to <00:16:22.717>make <00:16:22.719>changes 00:16:22.720 --> 00:16:25.790 align:start position:0% <00:16:22.720>we <00:16:22.836>show <00:16:22.953>how <00:16:23.070>we <00:16:23.186>were <00:16:23.303>able <00:16:23.420>to <00:16:23.536>make <00:16:23.653>changes <00:16:23.770>without <00:16:23.886>breaking <00:16:24.003>the <00:16:24.120>system. 00:16:25.790 --> 00:16:25.800 align:start position:0% <00:16:25.790>without <00:16:25.793>breaking <00:16:25.795>the <00:16:25.799>system. 00:16:25.800 --> 00:16:28.550 align:start position:0% <00:16:25.800>without <00:16:25.927>breaking <00:16:26.054>the <00:16:26.181>system. <00:16:26.309>So, <00:16:26.436>we <00:16:26.563>know <00:16:26.690>that <00:16:26.818>all <00:16:26.945>LM <00:16:27.072>projects <00:16:27.199>are 00:16:28.550 --> 00:16:28.560 align:start position:0% <00:16:28.550>So, <00:16:28.551>we <00:16:28.552>know <00:16:28.553>that <00:16:28.555>all <00:16:28.556>LM <00:16:28.557>projects <00:16:28.559>are 00:16:28.560 --> 00:16:31.030 align:start position:0% <00:16:28.560>So, <00:16:28.667>we <00:16:28.775>know <00:16:28.883>that <00:16:28.990>all <00:16:29.098>LM <00:16:29.206>projects <00:16:29.313>are <00:16:29.421>easy <00:16:29.529>until <00:16:29.636>they <00:16:29.744>aren't. <00:16:29.852>Everyone <00:16:29.959>starts 00:16:31.030 --> 00:16:31.040 align:start position:0% <00:16:31.030>easy <00:16:31.031>until <00:16:31.033>they <00:16:31.035>aren't. <00:16:31.037>Everyone <00:16:31.039>starts 00:16:31.040 --> 00:16:33.150 align:start position:0% <00:16:31.040>easy <00:16:31.147>until <00:16:31.255>they <00:16:31.363>aren't. <00:16:31.470>Everyone <00:16:31.578>starts <00:16:31.686>with <00:16:31.793>an <00:16:31.901>idea. <00:16:32.009>You <00:16:32.116>have <00:16:32.224>your <00:16:32.332>prompt, <00:16:32.439>you 00:16:33.150 --> 00:16:33.160 align:start position:0% <00:16:33.150>with <00:16:33.151>an <00:16:33.152>idea. <00:16:33.153>You <00:16:33.155>have <00:16:33.156>your <00:16:33.157>prompt, <00:16:33.159>you 00:16:33.160 --> 00:16:35.110 align:start position:0% <00:16:33.160>with <00:16:33.253>an <00:16:33.346>idea. <00:16:33.439>You <00:16:33.533>have <00:16:33.626>your <00:16:33.719>prompt, <00:16:33.813>you <00:16:33.906>have <00:16:34.000>some <00:16:34.093>example, <00:16:34.186>and <00:16:34.280>then <00:16:34.373>you <00:16:34.466>start <00:16:34.560>to 00:16:35.110 --> 00:16:35.120 align:start position:0% <00:16:35.110>have <00:16:35.111>some <00:16:35.112>example, <00:16:35.113>and <00:16:35.115>then <00:16:35.116>you <00:16:35.117>start <00:16:35.119>to 00:16:35.120 --> 00:16:37.550 align:start position:0% <00:16:35.120>have <00:16:35.220>some <00:16:35.320>example, <00:16:35.420>and <00:16:35.520>then <00:16:35.620>you <00:16:35.720>start <00:16:35.820>to <00:16:35.920>try <00:16:36.020>this <00:16:36.120>prompt <00:16:36.220>with <00:16:36.320>your <00:16:36.420>examples. <00:16:36.520>And 00:16:37.550 --> 00:16:37.560 align:start position:0% <00:16:37.550>try <00:16:37.551>this <00:16:37.553>prompt <00:16:37.554>with <00:16:37.555>your <00:16:37.557>examples. <00:16:37.559>And 00:16:37.560 --> 00:16:39.430 align:start position:0% <00:16:37.560>try <00:16:37.660>this <00:16:37.760>prompt <00:16:37.859>with <00:16:37.959>your <00:16:38.060>examples. <00:16:38.160>And <00:16:38.260>then <00:16:38.359>you <00:16:38.459>see, <00:16:38.560>yes, <00:16:38.660>it's <00:16:38.760>working <00:16:38.859>on <00:16:38.959>the 00:16:39.430 --> 00:16:39.440 align:start position:0% <00:16:39.430>then <00:16:39.431>you <00:16:39.432>see, <00:16:39.433>yes, <00:16:39.435>it's <00:16:39.436>working <00:16:39.437>on <00:16:39.439>the 00:16:39.440 --> 00:16:41.110 align:start position:0% <00:16:39.440>then <00:16:39.540>you <00:16:39.640>see, <00:16:39.740>yes, <00:16:39.840>it's <00:16:39.940>working <00:16:40.040>on <00:16:40.140>the <00:16:40.240>first <00:16:40.340>one, <00:16:40.440>then <00:16:40.540>it's <00:16:40.640>working <00:16:40.740>on <00:16:40.840>the 00:16:41.110 --> 00:16:41.120 align:start position:0% <00:16:41.110>first <00:16:41.111>one, <00:16:41.113>then <00:16:41.114>it's <00:16:41.116>working <00:16:41.117>on <00:16:41.119>the 00:16:41.120 --> 00:16:43.550 align:start position:0% <00:16:41.120>first <00:16:41.220>one, <00:16:41.320>then <00:16:41.420>it's <00:16:41.520>working <00:16:41.620>on <00:16:41.720>the <00:16:41.820>second <00:16:41.920>one, <00:16:42.020>also <00:16:42.120>the <00:16:42.220>third <00:16:42.320>one. <00:16:42.420>You <00:16:42.520>say, 00:16:43.550 --> 00:16:43.560 align:start position:0% <00:16:43.550>second <00:16:43.551>one, <00:16:43.552>also <00:16:43.553>the <00:16:43.555>third <00:16:43.556>one. <00:16:43.557>You <00:16:43.559>say, 00:16:43.560 --> 00:16:45.950 align:start position:0% <00:16:43.560>second <00:16:43.660>one, <00:16:43.760>also <00:16:43.859>the <00:16:43.959>third <00:16:44.060>one. <00:16:44.160>You <00:16:44.260>say, <00:16:44.359>"Oh, <00:16:44.459>wow, <00:16:44.560>it's <00:16:44.660>working <00:16:44.760>quite <00:16:44.859>well. <00:16:44.959>Seems 00:16:45.950 --> 00:16:45.960 align:start position:0% <00:16:45.950>"Oh, <00:16:45.951>wow, <00:16:45.953>it's <00:16:45.954>working <00:16:45.956>quite <00:16:45.957>well. <00:16:45.959>Seems 00:16:45.960 --> 00:16:48.350 align:start position:0% <00:16:45.960>"Oh, <00:16:46.076>wow, <00:16:46.193>it's <00:16:46.310>working <00:16:46.426>quite <00:16:46.543>well. <00:16:46.660>Seems <00:16:46.776>pretty <00:16:46.893>simple <00:16:47.010>problem. <00:16:47.126>Then, <00:16:47.243>let's <00:16:47.360>try 00:16:48.350 --> 00:16:48.360 align:start position:0% <00:16:48.350>pretty <00:16:48.351>simple <00:16:48.353>problem. <00:16:48.355>Then, <00:16:48.357>let's <00:16:48.359>try 00:16:48.360 --> 00:16:50.670 align:start position:0% <00:16:48.360>pretty <00:16:48.476>simple <00:16:48.593>problem. <00:16:48.710>Then, <00:16:48.826>let's <00:16:48.943>try <00:16:49.060>to <00:16:49.176>add <00:16:49.293>more <00:16:49.410>examples." <00:16:49.526>Let's <00:16:49.643>create <00:16:49.760>the 00:16:50.670 --> 00:16:50.680 align:start position:0% <00:16:50.670>to <00:16:50.671>add <00:16:50.673>more <00:16:50.674>examples." <00:16:50.675>Let's <00:16:50.677>create <00:16:50.679>the 00:16:50.680 --> 00:16:51.750 align:start position:0% <00:16:50.680>to <00:16:50.760>add <00:16:50.840>more <00:16:50.920>examples." <00:16:51.001>Let's <00:16:51.081>create <00:16:51.161>the <00:16:51.241>prompt <00:16:51.322>that's <00:16:51.402>a <00:16:51.482>little <00:16:51.562>bit <00:16:51.643>more 00:16:51.750 --> 00:16:51.760 align:start position:0% <00:16:51.750>prompt <00:16:51.751>that's <00:16:51.753>a <00:16:51.755>little <00:16:51.757>bit <00:16:51.759>more 00:16:51.760 --> 00:16:53.670 align:start position:0% <00:16:51.760>prompt <00:16:51.876>that's <00:16:51.993>a <00:16:52.110>little <00:16:52.226>bit <00:16:52.343>more <00:16:52.460>complicated <00:16:52.576>for <00:16:52.693>those <00:16:52.810>ones. <00:16:52.926>And <00:16:53.043>then <00:16:53.160>you 00:16:53.670 --> 00:16:53.680 align:start position:0% <00:16:53.670>complicated <00:16:53.671>for <00:16:53.673>those <00:16:53.674>ones. <00:16:53.675>And <00:16:53.677>then <00:16:53.679>you 00:16:53.680 --> 00:16:55.430 align:start position:0% <00:16:53.680>complicated <00:16:53.796>for <00:16:53.913>those <00:16:54.030>ones. <00:16:54.146>And <00:16:54.263>then <00:16:54.380>you <00:16:54.496>see, <00:16:54.613>yes, <00:16:54.729>your <00:16:54.846>previous <00:16:54.963>three <00:16:55.079>example 00:16:55.430 --> 00:16:55.440 align:start position:0% <00:16:55.430>see, <00:16:55.431>yes, <00:16:55.433>your <00:16:55.435>previous <00:16:55.437>three <00:16:55.439>example 00:16:55.440 --> 00:16:57.710 align:start position:0% <00:16:55.440>see, <00:16:55.547>yes, <00:16:55.655>your <00:16:55.763>previous <00:16:55.870>three <00:16:55.978>example <00:16:56.086>works, <00:16:56.193>also <00:16:56.301>the <00:16:56.409>new <00:16:56.516>one <00:16:56.624>that <00:16:56.732>you <00:16:56.840>add 00:16:57.710 --> 00:16:57.720 align:start position:0% <00:16:57.710>works, <00:16:57.711>also <00:16:57.712>the <00:16:57.713>new <00:16:57.715>one <00:16:57.716>that <00:16:57.717>you <00:16:57.719>add 00:16:57.720 --> 00:16:59.710 align:start position:0% <00:16:57.720>works, <00:16:57.813>also <00:16:57.906>the <00:16:58.000>new <00:16:58.093>one <00:16:58.186>that <00:16:58.280>you <00:16:58.373>add <00:16:58.466>that <00:16:58.560>kind <00:16:58.653>of <00:16:58.746>works. <00:16:58.840>So, <00:16:58.933>you <00:16:59.026>get <00:16:59.120>excited 00:16:59.710 --> 00:16:59.720 align:start position:0% <00:16:59.710>that <00:16:59.711>kind <00:16:59.712>of <00:16:59.713>works. <00:16:59.715>So, <00:16:59.716>you <00:16:59.717>get <00:16:59.719>excited 00:16:59.720 --> 00:17:01.390 align:start position:0% <00:16:59.720>that <00:16:59.827>kind <00:16:59.935>of <00:17:00.043>works. <00:17:00.150>So, <00:17:00.258>you <00:17:00.366>get <00:17:00.473>excited <00:17:00.581>like, <00:17:00.689>"Oh, <00:17:00.796>wow, <00:17:00.904>this <00:17:01.012>project <00:17:01.120>will 00:17:01.390 --> 00:17:01.400 align:start position:0% <00:17:01.390>like, <00:17:01.391>"Oh, <00:17:01.393>wow, <00:17:01.395>this <00:17:01.397>project <00:17:01.399>will 00:17:01.400 --> 00:17:03.390 align:start position:0% <00:17:01.400>like, <00:17:01.516>"Oh, <00:17:01.633>wow, <00:17:01.750>this <00:17:01.866>project <00:17:01.983>will <00:17:02.100>survive, <00:17:02.216>will <00:17:02.333>go <00:17:02.449>well. <00:17:02.566>Let's <00:17:02.683>push <00:17:02.800>it 00:17:03.390 --> 00:17:03.400 align:start position:0% <00:17:03.390>survive, <00:17:03.391>will <00:17:03.393>go <00:17:03.394>well. <00:17:03.396>Let's <00:17:03.397>push <00:17:03.399>it 00:17:03.400 --> 00:17:05.030 align:start position:0% <00:17:03.400>survive, <00:17:03.507>will <00:17:03.615>go <00:17:03.723>well. <00:17:03.830>Let's <00:17:03.938>push <00:17:04.046>it <00:17:04.153>forward. <00:17:04.261>Let's <00:17:04.369>move <00:17:04.476>it <00:17:04.584>in <00:17:04.692>in <00:17:04.800>the 00:17:05.030 --> 00:17:05.040 align:start position:0% <00:17:05.030>forward. <00:17:05.031>Let's <00:17:05.032>move <00:17:05.034>it <00:17:05.036>in <00:17:05.037>in <00:17:05.039>the 00:17:05.040 --> 00:17:07.110 align:start position:0% <00:17:05.040>forward. <00:17:05.180>Let's <00:17:05.319>move <00:17:05.460>it <00:17:05.599>in <00:17:05.740>in <00:17:05.879>the <00:17:06.020>company. <00:17:06.159>Let's <00:17:06.300>spread <00:17:06.440>it." 00:17:07.110 --> 00:17:07.120 align:start position:0% <00:17:07.109>company. <00:17:07.112>Let's <00:17:07.116>spread <00:17:07.118>it." 00:17:07.120 --> 00:17:09.030 align:start position:0% <00:17:07.119>company. <00:17:07.236>Let's <00:17:07.353>spread <00:17:07.469>it." <00:17:07.586>Then <00:17:07.703>you <00:17:07.819>say, <00:17:07.936>"Okay, <00:17:08.053>baby, <00:17:08.169>but <00:17:08.286>to <00:17:08.403>do <00:17:08.520>it, 00:17:09.030 --> 00:17:09.040 align:start position:0% <00:17:09.030>Then <00:17:09.031>you <00:17:09.032>say, <00:17:09.033>"Okay, <00:17:09.034>baby, <00:17:09.035>but <00:17:09.036>to <00:17:09.037>do <00:17:09.039>it, 00:17:09.040 --> 00:17:10.949 align:start position:0% <00:17:09.040>Then <00:17:09.127>you <00:17:09.214>say, <00:17:09.302>"Okay, <00:17:09.389>baby, <00:17:09.477>but <00:17:09.565>to <00:17:09.652>do <00:17:09.740>it, <00:17:09.827>I <00:17:09.915>also <00:17:10.002>Let <00:17:10.089>me <00:17:10.177>add <00:17:10.264>also <00:17:10.352>other <00:17:10.440>examples." 00:17:10.949 --> 00:17:10.959 align:start position:0% <00:17:10.949>I <00:17:10.950>also <00:17:10.951>Let <00:17:10.952>me <00:17:10.954>add <00:17:10.955>also <00:17:10.956>other <00:17:10.958>examples." 00:17:10.959 --> 00:17:12.750 align:start position:0% <00:17:10.959>I <00:17:11.046>also <00:17:11.134>Let <00:17:11.221>me <00:17:11.309>add <00:17:11.396>also <00:17:11.484>other <00:17:11.571>examples." <00:17:11.659>So, <00:17:11.746>you <00:17:11.834>created <00:17:11.921>a <00:17:12.009>little <00:17:12.096>bit <00:17:12.184>more <00:17:12.271>of <00:17:12.359>the 00:17:12.750 --> 00:17:12.760 align:start position:0% <00:17:12.750>So, <00:17:12.751>you <00:17:12.752>created <00:17:12.753>a <00:17:12.754>little <00:17:12.755>bit <00:17:12.756>more <00:17:12.757>of <00:17:12.759>the 00:17:12.760 --> 00:17:14.870 align:start position:0% <00:17:12.760>So, <00:17:12.853>you <00:17:12.946>created <00:17:13.040>a <00:17:13.133>little <00:17:13.226>bit <00:17:13.319>more <00:17:13.413>of <00:17:13.506>the <00:17:13.599>final <00:17:13.693>version <00:17:13.786>that <00:17:13.879>should <00:17:13.973>work. <00:17:14.066>You <00:17:14.160>have 00:17:14.870 --> 00:17:14.880 align:start position:0% <00:17:14.869>final <00:17:14.871>version <00:17:14.873>that <00:17:14.874>should <00:17:14.876>work. <00:17:14.877>You <00:17:14.879>have 00:17:14.880 --> 00:17:17.590 align:start position:0% <00:17:14.880>final <00:17:14.987>version <00:17:15.095>that <00:17:15.203>should <00:17:15.310>work. <00:17:15.418>You <00:17:15.526>have <00:17:15.633>your <00:17:15.741>first <00:17:15.849>example. <00:17:15.956>Yes, <00:17:16.064>yes, <00:17:16.172>kind <00:17:16.280>of 00:17:17.590 --> 00:17:17.600 align:start position:0% <00:17:17.589>your <00:17:17.591>first <00:17:17.592>example. <00:17:17.594>Yes, <00:17:17.596>yes, <00:17:17.597>kind <00:17:17.598>of 00:17:17.600 --> 00:17:19.790 align:start position:0% <00:17:17.599>your <00:17:17.699>first <00:17:17.800>example. <00:17:17.899>Yes, <00:17:18.000>yes, <00:17:18.099>kind <00:17:18.199>of <00:17:18.300>works. <00:17:18.399>Most <00:17:18.500>almost <00:17:18.599>all <00:17:18.699>of <00:17:18.800>them, <00:17:18.899>just <00:17:19.000>one 00:17:19.790 --> 00:17:19.800 align:start position:0% <00:17:19.790>works. <00:17:19.791>Most <00:17:19.792>almost <00:17:19.793>all <00:17:19.795>of <00:17:19.796>them, <00:17:19.797>just <00:17:19.799>one 00:17:19.800 --> 00:17:22.350 align:start position:0% <00:17:19.800>works. <00:17:19.893>Most <00:17:19.986>almost <00:17:20.079>all <00:17:20.173>of <00:17:20.266>them, <00:17:20.359>just <00:17:20.453>one <00:17:20.546>fails. <00:17:20.639>And <00:17:20.733>the <00:17:20.826>new <00:17:20.919>one <00:17:21.013>that <00:17:21.106>you <00:17:21.200>added, 00:17:22.350 --> 00:17:22.360 align:start position:0% <00:17:22.349>fails. <00:17:22.351>And <00:17:22.352>the <00:17:22.353>new <00:17:22.355>one <00:17:22.356>that <00:17:22.357>you <00:17:22.358>added, 00:17:22.360 --> 00:17:24.829 align:start position:0% <00:17:22.359>fails. <00:17:22.453>And <00:17:22.546>the <00:17:22.639>new <00:17:22.733>one <00:17:22.826>that <00:17:22.919>you <00:17:23.013>added, <00:17:23.106>some <00:17:23.199>of <00:17:23.293>them <00:17:23.386>works, <00:17:23.479>most <00:17:23.573>of <00:17:23.666>them <00:17:23.760>not. 00:17:24.829 --> 00:17:24.839 align:start position:0% <00:17:24.829>some <00:17:24.830>of <00:17:24.831>them <00:17:24.832>works, <00:17:24.834>most <00:17:24.835>of <00:17:24.836>them <00:17:24.838>not. 00:17:24.839 --> 00:17:26.110 align:start position:0% <00:17:24.839>some <00:17:24.926>of <00:17:25.014>them <00:17:25.102>works, <00:17:25.190>most <00:17:25.278>of <00:17:25.366>them <00:17:25.454>not. <00:17:25.542>So, <00:17:25.630>you <00:17:25.718>say, <00:17:25.806>"Yeah, <00:17:25.894>maybe <00:17:25.982>that's 00:17:26.110 --> 00:17:26.120 align:start position:0% <00:17:26.109>So, <00:17:26.111>you <00:17:26.113>say, <00:17:26.115>"Yeah, <00:17:26.117>maybe <00:17:26.118>that's 00:17:26.120 --> 00:17:28.510 align:start position:0% <00:17:26.119>So, <00:17:26.236>you <00:17:26.353>say, <00:17:26.469>"Yeah, <00:17:26.586>maybe <00:17:26.703>that's <00:17:26.819>probably <00:17:26.936>not <00:17:27.053>the <00:17:27.169>final <00:17:27.286>version <00:17:27.403>of <00:17:27.520>the 00:17:28.510 --> 00:17:28.520 align:start position:0% <00:17:28.510>probably <00:17:28.511>not <00:17:28.512>the <00:17:28.514>final <00:17:28.516>version <00:17:28.517>of <00:17:28.519>the 00:17:28.520 --> 00:17:32.190 align:start position:0% <00:17:28.520>probably <00:17:28.636>not <00:17:28.753>the <00:17:28.870>final <00:17:28.986>version <00:17:29.103>of <00:17:29.220>the <00:17:29.336>prompt. <00:17:29.453>We <00:17:29.570>need <00:17:29.686>to <00:17:29.803>tweak <00:17:29.920>it." 00:17:32.190 --> 00:17:32.200 align:start position:0% <00:17:32.190>prompt. <00:17:32.191>We <00:17:32.193>need <00:17:32.195>to <00:17:32.197>tweak <00:17:32.199>it." 00:17:32.200 --> 00:17:34.150 align:start position:0% <00:17:32.200>prompt. <00:17:32.293>We <00:17:32.386>need <00:17:32.480>to <00:17:32.573>tweak <00:17:32.666>it." <00:17:32.760>So, <00:17:32.853>that's <00:17:32.946>why <00:17:33.040>you <00:17:33.133>try <00:17:33.226>to <00:17:33.320>make <00:17:33.413>it <00:17:33.506>even <00:17:33.600>a 00:17:34.150 --> 00:17:34.160 align:start position:0% <00:17:34.150>So, <00:17:34.151>that's <00:17:34.152>why <00:17:34.153>you <00:17:34.154>try <00:17:34.155>to <00:17:34.156>make <00:17:34.157>it <00:17:34.158>even <00:17:34.159>a 00:17:34.160 --> 00:17:36.630 align:start position:0% <00:17:34.160>So, <00:17:34.247>that's <00:17:34.335>why <00:17:34.422>you <00:17:34.510>try <00:17:34.597>to <00:17:34.685>make <00:17:34.772>it <00:17:34.860>even <00:17:34.947>a <00:17:35.035>more <00:17:35.122>complex <00:17:35.210>prompt <00:17:35.297>to <00:17:35.385>trying <00:17:35.472>to <00:17:35.560>cover 00:17:36.630 --> 00:17:36.640 align:start position:0% <00:17:36.630>more <00:17:36.631>complex <00:17:36.633>prompt <00:17:36.634>to <00:17:36.636>trying <00:17:36.637>to <00:17:36.639>cover 00:17:36.640 --> 00:17:38.830 align:start position:0% <00:17:36.640>more <00:17:36.740>complex <00:17:36.840>prompt <00:17:36.940>to <00:17:37.040>trying <00:17:37.140>to <00:17:37.240>cover <00:17:37.340>the <00:17:37.440>edge <00:17:37.540>cases, <00:17:37.640>the <00:17:37.740>new <00:17:37.840>example <00:17:37.940>that <00:17:38.040>you 00:17:38.830 --> 00:17:38.840 align:start position:0% <00:17:38.830>the <00:17:38.831>edge <00:17:38.832>cases, <00:17:38.833>the <00:17:38.835>new <00:17:38.836>example <00:17:38.837>that <00:17:38.839>you 00:17:38.840 --> 00:17:41.150 align:start position:0% <00:17:38.840>the <00:17:38.927>edge <00:17:39.014>cases, <00:17:39.102>the <00:17:39.189>new <00:17:39.277>example <00:17:39.365>that <00:17:39.452>you <00:17:39.540>added, <00:17:39.627>to <00:17:39.715>try <00:17:39.802>to <00:17:39.889>find <00:17:39.977>a <00:17:40.064>new <00:17:40.152>version <00:17:40.240>of 00:17:41.150 --> 00:17:41.160 align:start position:0% <00:17:41.150>added, <00:17:41.151>to <00:17:41.152>try <00:17:41.153>to <00:17:41.154>find <00:17:41.155>a <00:17:41.156>new <00:17:41.157>version <00:17:41.159>of 00:17:41.160 --> 00:17:44.390 align:start position:0% <00:17:41.160>added, <00:17:41.247>to <00:17:41.335>try <00:17:41.422>to <00:17:41.510>find <00:17:41.597>a <00:17:41.685>new <00:17:41.772>version <00:17:41.860>of <00:17:41.947>it. <00:17:42.035>And <00:17:42.122>then <00:17:42.210>you <00:17:42.297>notice <00:17:42.385>that, <00:17:42.472>yes, <00:17:42.560>the 00:17:44.390 --> 00:17:44.400 align:start position:0% <00:17:44.390>it. <00:17:44.391>And <00:17:44.392>then <00:17:44.393>you <00:17:44.395>notice <00:17:44.396>that, <00:17:44.397>yes, <00:17:44.399>the 00:17:44.400 --> 00:17:46.470 align:start position:0% <00:17:44.400>it. <00:17:44.500>And <00:17:44.600>then <00:17:44.700>you <00:17:44.800>notice <00:17:44.900>that, <00:17:45.000>yes, <00:17:45.100>the <00:17:45.200>new <00:17:45.300>example <00:17:45.400>that <00:17:45.500>you <00:17:45.600>added, <00:17:45.700>the <00:17:45.800>last 00:17:46.470 --> 00:17:46.480 align:start position:0% <00:17:46.470>new <00:17:46.471>example <00:17:46.473>that <00:17:46.474>you <00:17:46.476>added, <00:17:46.477>the <00:17:46.479>last 00:17:46.480 --> 00:17:48.990 align:start position:0% <00:17:46.480>new <00:17:46.587>example <00:17:46.695>that <00:17:46.803>you <00:17:46.910>added, <00:17:47.018>the <00:17:47.126>last <00:17:47.233>ones <00:17:47.341>are <00:17:47.449>working, <00:17:47.556>but <00:17:47.664>then <00:17:47.772>the <00:17:47.880>previous 00:17:48.990 --> 00:17:49.000 align:start position:0% <00:17:48.990>ones <00:17:48.991>are <00:17:48.993>working, <00:17:48.994>but <00:17:48.996>then <00:17:48.997>the <00:17:48.999>previous 00:17:49.000 --> 00:17:50.270 align:start position:0% <00:17:49.000>ones <00:17:49.087>are <00:17:49.175>working, <00:17:49.263>but <00:17:49.351>then <00:17:49.439>the <00:17:49.527>previous <00:17:49.615>one <00:17:49.703>that <00:17:49.791>you <00:17:49.879>had <00:17:49.967>are <00:17:50.055>not <00:17:50.143>working 00:17:50.270 --> 00:17:50.280 align:start position:0% <00:17:50.270>one <00:17:50.271>that <00:17:50.273>you <00:17:50.274>had <00:17:50.276>are <00:17:50.277>not <00:17:50.279>working 00:17:50.280 --> 00:17:52.710 align:start position:0% <00:17:50.280>one <00:17:50.387>that <00:17:50.495>you <00:17:50.603>had <00:17:50.710>are <00:17:50.818>not <00:17:50.926>working <00:17:51.033>anymore. <00:17:51.141>So, <00:17:51.249>you <00:17:51.356>are <00:17:51.464>in <00:17:51.572>a <00:17:51.680>situation 00:17:52.710 --> 00:17:52.720 align:start position:0% <00:17:52.710>anymore. <00:17:52.711>So, <00:17:52.713>you <00:17:52.714>are <00:17:52.716>in <00:17:52.717>a <00:17:52.719>situation 00:17:52.720 --> 00:17:54.910 align:start position:0% <00:17:52.720>anymore. <00:17:52.827>So, <00:17:52.935>you <00:17:53.043>are <00:17:53.150>in <00:17:53.258>a <00:17:53.366>situation <00:17:53.473>where <00:17:53.581>you <00:17:53.689>don't <00:17:53.796>know <00:17:53.904>how <00:17:54.012>to <00:17:54.120>move 00:17:54.910 --> 00:17:54.920 align:start position:0% <00:17:54.910>where <00:17:54.911>you <00:17:54.913>don't <00:17:54.914>know <00:17:54.916>how <00:17:54.917>to <00:17:54.919>move 00:17:54.920 --> 00:17:57.270 align:start position:0% <00:17:54.920>where <00:17:55.027>you <00:17:55.135>don't <00:17:55.243>know <00:17:55.350>how <00:17:55.458>to <00:17:55.566>move <00:17:55.673>forward. <00:17:55.781>What <00:17:55.889>was <00:17:55.996>the <00:17:56.104>change <00:17:56.212>that <00:17:56.320>made 00:17:57.270 --> 00:17:57.280 align:start position:0% <00:17:57.270>forward. <00:17:57.271>What <00:17:57.273>was <00:17:57.274>the <00:17:57.276>change <00:17:57.277>that <00:17:57.279>made 00:17:57.280 --> 00:17:59.470 align:start position:0% <00:17:57.280>forward. <00:17:57.387>What <00:17:57.495>was <00:17:57.603>the <00:17:57.710>change <00:17:57.818>that <00:17:57.926>made <00:17:58.033>this <00:17:58.141>problem? <00:17:58.249>Was <00:17:58.356>it <00:17:58.464>the <00:17:58.572>previous <00:17:58.680>prompt 00:17:59.470 --> 00:17:59.480 align:start position:0% <00:17:59.470>this <00:17:59.471>problem? <00:17:59.473>Was <00:17:59.474>it <00:17:59.476>the <00:17:59.477>previous <00:17:59.479>prompt 00:17:59.480 --> 00:18:02.030 align:start position:0% <00:17:59.480>this <00:17:59.580>problem? <00:17:59.680>Was <00:17:59.780>it <00:17:59.880>the <00:17:59.980>previous <00:18:00.080>prompt <00:18:00.180>or <00:18:00.280>like <00:18:00.380>just <00:18:00.480>the <00:18:00.580>example <00:18:00.680>that <00:18:00.780>you <00:18:00.880>added? 00:18:02.030 --> 00:18:02.040 align:start position:0% <00:18:02.030>or <00:18:02.031>like <00:18:02.032>just <00:18:02.033>the <00:18:02.035>example <00:18:02.036>that <00:18:02.037>you <00:18:02.039>added? 00:18:02.040 --> 00:18:03.790 align:start position:0% <00:18:02.040>or <00:18:02.133>like <00:18:02.226>just <00:18:02.320>the <00:18:02.413>example <00:18:02.506>that <00:18:02.600>you <00:18:02.693>added? <00:18:02.786>And <00:18:02.879>and <00:18:02.973>then <00:18:03.066>you <00:18:03.159>are <00:18:03.253>in <00:18:03.346>a <00:18:03.440>situation 00:18:03.790 --> 00:18:03.800 align:start position:0% <00:18:03.790>And <00:18:03.791>and <00:18:03.792>then <00:18:03.793>you <00:18:03.795>are <00:18:03.796>in <00:18:03.797>a <00:18:03.799>situation 00:18:03.800 --> 00:18:06.390 align:start position:0% <00:18:03.800>And <00:18:03.899>and <00:18:04.000>then <00:18:04.100>you <00:18:04.200>are <00:18:04.300>in <00:18:04.399>a <00:18:04.500>situation <00:18:04.600>where <00:18:04.700>you <00:18:04.800>don't <00:18:04.899>know <00:18:05.000>who <00:18:05.100>caused <00:18:05.200>what. 00:18:06.390 --> 00:18:06.400 align:start position:0% <00:18:06.390>where <00:18:06.391>you <00:18:06.393>don't <00:18:06.394>know <00:18:06.396>who <00:18:06.397>caused <00:18:06.399>what. 00:18:06.400 --> 00:18:08.390 align:start position:0% <00:18:06.400>where <00:18:06.500>you <00:18:06.600>don't <00:18:06.700>know <00:18:06.800>who <00:18:06.900>caused <00:18:07.000>what. <00:18:07.100>Like, <00:18:07.200>is <00:18:07.300>it <00:18:07.400>the <00:18:07.500>model <00:18:07.600>that <00:18:07.700>you <00:18:07.800>changed 00:18:08.390 --> 00:18:08.400 align:start position:0% <00:18:08.390>Like, <00:18:08.391>is <00:18:08.392>it <00:18:08.393>the <00:18:08.395>model <00:18:08.396>that <00:18:08.397>you <00:18:08.399>changed 00:18:08.400 --> 00:18:09.950 align:start position:0% <00:18:08.400>Like, <00:18:08.499>is <00:18:08.599>it <00:18:08.698>the <00:18:08.798>model <00:18:08.898>that <00:18:08.997>you <00:18:09.097>changed <00:18:09.197>that's <00:18:09.296>not <00:18:09.396>able <00:18:09.496>to <00:18:09.595>understand <00:18:09.695>the <00:18:09.795>prompt 00:18:09.950 --> 00:18:09.960 align:start position:0% <00:18:09.950>that's <00:18:09.951>not <00:18:09.953>able <00:18:09.954>to <00:18:09.956>understand <00:18:09.957>the <00:18:09.959>prompt 00:18:09.960 --> 00:18:12.150 align:start position:0% <00:18:09.960>that's <00:18:10.067>not <00:18:10.175>able <00:18:10.283>to <00:18:10.390>understand <00:18:10.498>the <00:18:10.606>prompt <00:18:10.713>well <00:18:10.821>enough? <00:18:10.929>Is <00:18:11.036>it <00:18:11.144>the <00:18:11.252>prompt <00:18:11.360>changes 00:18:12.150 --> 00:18:12.160 align:start position:0% <00:18:12.150>well <00:18:12.151>enough? <00:18:12.153>Is <00:18:12.154>it <00:18:12.156>the <00:18:12.157>prompt <00:18:12.159>changes 00:18:12.160 --> 00:18:13.790 align:start position:0% <00:18:12.160>well <00:18:12.260>enough? <00:18:12.360>Is <00:18:12.460>it <00:18:12.560>the <00:18:12.660>prompt <00:18:12.760>changes <00:18:12.860>that <00:18:12.960>you <00:18:13.060>did <00:18:13.160>that <00:18:13.260>then <00:18:13.360>is <00:18:13.460>not <00:18:13.560>covering 00:18:13.790 --> 00:18:13.800 align:start position:0% <00:18:13.790>that <00:18:13.791>you <00:18:13.792>did <00:18:13.793>that <00:18:13.795>then <00:18:13.796>is <00:18:13.797>not <00:18:13.799>covering 00:18:13.800 --> 00:18:15.590 align:start position:0% <00:18:13.800>that <00:18:13.899>you <00:18:14.000>did <00:18:14.100>that <00:18:14.200>then <00:18:14.300>is <00:18:14.399>not <00:18:14.500>covering <00:18:14.600>well <00:18:14.700>the <00:18:14.800>example? <00:18:14.899>Or <00:18:15.000>are <00:18:15.100>the <00:18:15.200>example 00:18:15.590 --> 00:18:15.600 align:start position:0% <00:18:15.590>well <00:18:15.591>the <00:18:15.592>example? <00:18:15.594>Or <00:18:15.596>are <00:18:15.597>the <00:18:15.599>example 00:18:15.600 --> 00:18:18.150 align:start position:0% <00:18:15.600>well <00:18:15.716>the <00:18:15.833>example? <00:18:15.949>Or <00:18:16.066>are <00:18:16.183>the <00:18:16.300>example <00:18:16.416>just <00:18:16.533>not <00:18:16.649>representing <00:18:16.766>well <00:18:16.883>the <00:18:17.000>prompt 00:18:18.150 --> 00:18:18.160 align:start position:0% <00:18:18.150>just <00:18:18.151>not <00:18:18.153>representing <00:18:18.155>well <00:18:18.157>the <00:18:18.159>prompt 00:18:18.160 --> 00:18:19.550 align:start position:0% <00:18:18.160>just <00:18:18.264>not <00:18:18.368>representing <00:18:18.472>well <00:18:18.577>the <00:18:18.681>prompt <00:18:18.785>that <00:18:18.889>you <00:18:18.994>described <00:18:19.098>in <00:18:19.202>the <00:18:19.306>problem <00:18:19.411>that 00:18:19.550 --> 00:18:19.560 align:start position:0% <00:18:19.550>that <00:18:19.551>you <00:18:19.553>described <00:18:19.554>in <00:18:19.556>the <00:18:19.557>problem <00:18:19.559>that 00:18:19.560 --> 00:18:21.750 align:start position:0% <00:18:19.560>that <00:18:19.700>you <00:18:19.840>described <00:18:19.980>in <00:18:20.120>the <00:18:20.260>problem <00:18:20.399>that <00:18:20.540>you <00:18:20.679>want <00:18:20.820>to <00:18:20.960>solve? 00:18:21.750 --> 00:18:21.760 align:start position:0% <00:18:21.750>you <00:18:21.753>want <00:18:21.756>to <00:18:21.759>solve? 00:18:21.760 --> 00:18:24.270 align:start position:0% <00:18:21.760>you <00:18:21.887>want <00:18:22.014>to <00:18:22.141>solve? <00:18:22.269>In <00:18:22.396>our <00:18:22.523>case, <00:18:22.650>we <00:18:22.778>have <00:18:22.905>the <00:18:23.032>workflow. <00:18:23.160>And 00:18:24.270 --> 00:18:24.280 align:start position:0% <00:18:24.270>In <00:18:24.271>our <00:18:24.272>case, <00:18:24.273>we <00:18:24.275>have <00:18:24.276>the <00:18:24.277>workflow. <00:18:24.279>And 00:18:24.280 --> 00:18:26.110 align:start position:0% <00:18:24.280>In <00:18:24.379>our <00:18:24.480>case, <00:18:24.580>we <00:18:24.680>have <00:18:24.780>the <00:18:24.879>workflow. <00:18:24.980>And <00:18:25.080>each <00:18:25.180>one <00:18:25.280>of <00:18:25.379>those <00:18:25.480>nodes <00:18:25.580>can <00:18:25.680>have 00:18:26.110 --> 00:18:26.120 align:start position:0% <00:18:26.110>each <00:18:26.111>one <00:18:26.112>of <00:18:26.114>those <00:18:26.116>nodes <00:18:26.117>can <00:18:26.119>have 00:18:26.120 --> 00:18:28.190 align:start position:0% <00:18:26.120>each <00:18:26.236>one <00:18:26.353>of <00:18:26.469>those <00:18:26.586>nodes <00:18:26.703>can <00:18:26.820>have <00:18:26.936>different <00:18:27.053>failure <00:18:27.169>modes. <00:18:27.286>You <00:18:27.403>have <00:18:27.520>the 00:18:28.190 --> 00:18:28.200 align:start position:0% <00:18:28.190>different <00:18:28.191>failure <00:18:28.193>modes. <00:18:28.195>You <00:18:28.197>have <00:18:28.199>the 00:18:28.200 --> 00:18:30.790 align:start position:0% <00:18:28.200>different <00:18:28.316>failure <00:18:28.433>modes. <00:18:28.550>You <00:18:28.666>have <00:18:28.783>the <00:18:28.900>intent <00:18:29.016>that <00:18:29.133>can <00:18:29.250>just <00:18:29.366>take <00:18:29.483>the <00:18:29.600>wrong 00:18:30.790 --> 00:18:30.800 align:start position:0% <00:18:30.790>intent <00:18:30.791>that <00:18:30.793>can <00:18:30.794>just <00:18:30.796>take <00:18:30.797>the <00:18:30.799>wrong 00:18:30.800 --> 00:18:34.070 align:start position:0% <00:18:30.800>intent <00:18:30.907>that <00:18:31.015>can <00:18:31.123>just <00:18:31.230>take <00:18:31.338>the <00:18:31.446>wrong <00:18:31.553>scope <00:18:31.661>or <00:18:31.769>lose <00:18:31.876>context. <00:18:31.984>The <00:18:32.092>planner <00:18:32.200>can 00:18:34.070 --> 00:18:34.080 align:start position:0% <00:18:34.070>scope <00:18:34.071>or <00:18:34.072>lose <00:18:34.074>context. <00:18:34.076>The <00:18:34.077>planner <00:18:34.079>can 00:18:34.080 --> 00:18:36.350 align:start position:0% <00:18:34.080>scope <00:18:34.196>or <00:18:34.313>lose <00:18:34.429>context. <00:18:34.546>The <00:18:34.663>planner <00:18:34.780>can <00:18:34.896>just <00:18:35.013>not <00:18:35.129>understand <00:18:35.246>some <00:18:35.363>constraint <00:18:35.480>or 00:18:36.350 --> 00:18:36.360 align:start position:0% <00:18:36.350>just <00:18:36.351>not <00:18:36.353>understand <00:18:36.355>some <00:18:36.357>constraint <00:18:36.359>or 00:18:36.360 --> 00:18:38.750 align:start position:0% <00:18:36.360>just <00:18:36.467>not <00:18:36.575>understand <00:18:36.683>some <00:18:36.790>constraint <00:18:36.898>or <00:18:37.006>do <00:18:37.113>the <00:18:37.221>wrong <00:18:37.329>routing <00:18:37.436>to <00:18:37.544>the <00:18:37.652>wrong <00:18:37.760>LLM. 00:18:38.750 --> 00:18:38.760 align:start position:0% <00:18:38.750>do <00:18:38.751>the <00:18:38.752>wrong <00:18:38.753>routing <00:18:38.755>to <00:18:38.756>the <00:18:38.757>wrong <00:18:38.759>LLM. 00:18:38.760 --> 00:18:41.390 align:start position:0% <00:18:38.760>do <00:18:38.853>the <00:18:38.946>wrong <00:18:39.040>routing <00:18:39.133>to <00:18:39.226>the <00:18:39.320>wrong <00:18:39.413>LLM. <00:18:39.506>Or <00:18:39.600>the <00:18:39.693>retrieval <00:18:39.786>step <00:18:39.879>can <00:18:39.973>just <00:18:40.066>get <00:18:40.160>noise 00:18:41.390 --> 00:18:41.400 align:start position:0% <00:18:41.390>Or <00:18:41.391>the <00:18:41.392>retrieval <00:18:41.393>step <00:18:41.395>can <00:18:41.396>just <00:18:41.397>get <00:18:41.399>noise 00:18:41.400 --> 00:18:44.110 align:start position:0% <00:18:41.400>Or <00:18:41.500>the <00:18:41.600>retrieval <00:18:41.700>step <00:18:41.800>can <00:18:41.900>just <00:18:42.000>get <00:18:42.100>noise <00:18:42.200>or <00:18:42.300>drop <00:18:42.400>relevant <00:18:42.500>items. <00:18:42.600>Or <00:18:42.700>the <00:18:42.800>relevance 00:18:44.110 --> 00:18:44.120 align:start position:0% <00:18:44.110>or <00:18:44.111>drop <00:18:44.112>relevant <00:18:44.114>items. <00:18:44.116>Or <00:18:44.117>the <00:18:44.119>relevance 00:18:44.120 --> 00:18:46.670 align:start position:0% <00:18:44.120>or <00:18:44.219>drop <00:18:44.320>relevant <00:18:44.419>items. <00:18:44.520>Or <00:18:44.620>the <00:18:44.719>relevance <00:18:44.820>can <00:18:44.919>just <00:18:45.020>give <00:18:45.120>a <00:18:45.219>low <00:18:45.320>score <00:18:45.419>to <00:18:45.520>something 00:18:46.670 --> 00:18:46.680 align:start position:0% <00:18:46.670>can <00:18:46.671>just <00:18:46.672>give <00:18:46.673>a <00:18:46.675>low <00:18:46.676>score <00:18:46.677>to <00:18:46.679>something 00:18:46.680 --> 00:18:48.990 align:start position:0% <00:18:46.680>can <00:18:46.780>just <00:18:46.880>give <00:18:46.980>a <00:18:47.080>low <00:18:47.180>score <00:18:47.280>to <00:18:47.380>something <00:18:47.480>that <00:18:47.580>should <00:18:47.680>be <00:18:47.780>highly <00:18:47.880>matched. <00:18:47.980>So, <00:18:48.080>and 00:18:48.990 --> 00:18:49.000 align:start position:0% <00:18:48.990>that <00:18:48.991>should <00:18:48.993>be <00:18:48.994>highly <00:18:48.996>matched. <00:18:48.997>So, <00:18:48.999>and 00:18:49.000 --> 00:18:50.990 align:start position:0% <00:18:49.000>that <00:18:49.100>should <00:18:49.200>be <00:18:49.300>highly <00:18:49.400>matched. <00:18:49.500>So, <00:18:49.600>and <00:18:49.700>in <00:18:49.800>this <00:18:49.900>case, <00:18:50.000>every <00:18:50.100>change <00:18:50.200>that <00:18:50.300>you <00:18:50.400>do 00:18:50.990 --> 00:18:51.000 align:start position:0% <00:18:50.990>in <00:18:50.991>this <00:18:50.992>case, <00:18:50.993>every <00:18:50.995>change <00:18:50.996>that <00:18:50.997>you <00:18:50.999>do 00:18:51.000 --> 00:18:52.910 align:start position:0% <00:18:51.000>in <00:18:51.100>this <00:18:51.200>case, <00:18:51.300>every <00:18:51.400>change <00:18:51.500>that <00:18:51.600>you <00:18:51.700>do <00:18:51.800>can <00:18:51.900>break <00:18:52.000>everything. <00:18:52.100>And <00:18:52.200>you <00:18:52.300>don't <00:18:52.400>need 00:18:52.910 --> 00:18:52.920 align:start position:0% <00:18:52.910>can <00:18:52.911>break <00:18:52.913>everything. <00:18:52.914>And <00:18:52.916>you <00:18:52.917>don't <00:18:52.919>need 00:18:52.920 --> 00:18:54.950 align:start position:0% <00:18:52.920>can <00:18:53.027>break <00:18:53.135>everything. <00:18:53.243>And <00:18:53.350>you <00:18:53.458>don't <00:18:53.566>need <00:18:53.673>to <00:18:53.781>be <00:18:53.889>super <00:18:53.996>careful <00:18:54.104>whenever <00:18:54.212>things <00:18:54.320>you 00:18:54.950 --> 00:18:54.960 align:start position:0% <00:18:54.950>to <00:18:54.951>be <00:18:54.953>super <00:18:54.954>careful <00:18:54.956>whenever <00:18:54.957>things <00:18:54.959>you 00:18:54.960 --> 00:18:57.110 align:start position:0% <00:18:54.960>to <00:18:55.100>be <00:18:55.240>super <00:18:55.380>careful <00:18:55.520>whenever <00:18:55.660>things <00:18:55.800>you <00:18:55.940>do <00:18:56.080>in <00:18:56.220>this <00:18:56.360>case. 00:18:57.110 --> 00:18:57.120 align:start position:0% <00:18:57.110>do <00:18:57.112>in <00:18:57.116>this <00:18:57.119>case. 00:18:57.120 --> 00:19:00.470 align:start position:0% <00:18:57.120>do <00:18:57.236>in <00:18:57.353>this <00:18:57.469>case. <00:18:57.586>So, <00:18:57.703>how <00:18:57.820>do <00:18:57.936>we <00:18:58.053>solve <00:18:58.169>it <00:18:58.286>in <00:18:58.403>our <00:18:58.520>example 00:19:00.470 --> 00:19:00.480 align:start position:0% <00:19:00.470>So, <00:19:00.471>how <00:19:00.472>do <00:19:00.473>we <00:19:00.474>solve <00:19:00.475>it <00:19:00.476>in <00:19:00.477>our <00:19:00.479>example 00:19:00.480 --> 00:19:02.630 align:start position:0% <00:19:00.480>So, <00:19:00.587>how <00:19:00.695>do <00:19:00.803>we <00:19:00.910>solve <00:19:01.018>it <00:19:01.126>in <00:19:01.233>our <00:19:01.341>example <00:19:01.449>using <00:19:01.556>some <00:19:01.664>test-driven <00:19:01.772>development, <00:19:01.880>but 00:19:02.630 --> 00:19:02.640 align:start position:0% <00:19:02.630>using <00:19:02.632>some <00:19:02.634>test-driven <00:19:02.636>development, <00:19:02.639>but 00:19:02.640 --> 00:19:05.430 align:start position:0% <00:19:02.640>using <00:19:02.747>some <00:19:02.855>test-driven <00:19:02.963>development, <00:19:03.070>but <00:19:03.178>for <00:19:03.286>AI. <00:19:03.393>So, <00:19:03.501>there <00:19:03.609>we <00:19:03.716>have <00:19:03.824>a <00:19:03.932>data <00:19:04.040>set 00:19:05.430 --> 00:19:05.440 align:start position:0% <00:19:05.430>for <00:19:05.431>AI. <00:19:05.432>So, <00:19:05.433>there <00:19:05.434>we <00:19:05.435>have <00:19:05.436>a <00:19:05.437>data <00:19:05.439>set 00:19:05.440 --> 00:19:07.630 align:start position:0% <00:19:05.440>for <00:19:05.533>AI. <00:19:05.626>So, <00:19:05.720>there <00:19:05.813>we <00:19:05.906>have <00:19:06.000>a <00:19:06.093>data <00:19:06.186>set <00:19:06.280>with <00:19:06.373>the <00:19:06.466>input <00:19:06.560>cases <00:19:06.653>and <00:19:06.746>output <00:19:06.840>cases. 00:19:07.630 --> 00:19:07.640 align:start position:0% <00:19:07.630>with <00:19:07.631>the <00:19:07.633>input <00:19:07.634>cases <00:19:07.636>and <00:19:07.637>output <00:19:07.639>cases. 00:19:07.640 --> 00:19:10.150 align:start position:0% <00:19:07.640>with <00:19:07.740>the <00:19:07.840>input <00:19:07.940>cases <00:19:08.040>and <00:19:08.140>output <00:19:08.240>cases. <00:19:08.340>What <00:19:08.440>do <00:19:08.540>you <00:19:08.640>expect <00:19:08.740>from <00:19:08.840>the <00:19:08.940>LLM <00:19:09.040>to 00:19:10.150 --> 00:19:10.160 align:start position:0% <00:19:10.150>What <00:19:10.151>do <00:19:10.152>you <00:19:10.153>expect <00:19:10.155>from <00:19:10.156>the <00:19:10.157>LLM <00:19:10.159>to 00:19:10.160 --> 00:19:12.910 align:start position:0% <00:19:10.160>What <00:19:10.253>do <00:19:10.346>you <00:19:10.440>expect <00:19:10.533>from <00:19:10.626>the <00:19:10.720>LLM <00:19:10.813>to <00:19:10.906>answer? <00:19:11.000>Then <00:19:11.093>you <00:19:11.186>can <00:19:11.280>run <00:19:11.373>your <00:19:11.466>agent <00:19:11.560>on 00:19:12.910 --> 00:19:12.920 align:start position:0% <00:19:12.910>answer? <00:19:12.911>Then <00:19:12.912>you <00:19:12.913>can <00:19:12.915>run <00:19:12.916>your <00:19:12.917>agent <00:19:12.919>on 00:19:12.920 --> 00:19:14.990 align:start position:0% <00:19:12.920>answer? <00:19:13.020>Then <00:19:13.120>you <00:19:13.220>can <00:19:13.320>run <00:19:13.420>your <00:19:13.520>agent <00:19:13.620>on <00:19:13.720>the <00:19:13.820>whole <00:19:13.920>examples <00:19:14.020>or <00:19:14.120>just <00:19:14.220>partial <00:19:14.320>of 00:19:14.990 --> 00:19:15.000 align:start position:0% <00:19:14.990>the <00:19:14.991>whole <00:19:14.993>examples <00:19:14.994>or <00:19:14.996>just <00:19:14.997>partial <00:19:14.999>of 00:19:15.000 --> 00:19:17.270 align:start position:0% <00:19:15.000>the <00:19:15.107>whole <00:19:15.215>examples <00:19:15.323>or <00:19:15.430>just <00:19:15.538>partial <00:19:15.646>of <00:19:15.753>them. <00:19:15.861>You <00:19:15.969>measure <00:19:16.076>the <00:19:16.184>accuracy, <00:19:16.292>so <00:19:16.400>you 00:19:17.270 --> 00:19:17.280 align:start position:0% <00:19:17.270>them. <00:19:17.271>You <00:19:17.273>measure <00:19:17.274>the <00:19:17.276>accuracy, <00:19:17.277>so <00:19:17.279>you 00:19:17.280 --> 00:19:19.670 align:start position:0% <00:19:17.280>them. <00:19:17.373>You <00:19:17.466>measure <00:19:17.560>the <00:19:17.653>accuracy, <00:19:17.746>so <00:19:17.840>you <00:19:17.933>check <00:19:18.026>how <00:19:18.120>many <00:19:18.213>of <00:19:18.306>them <00:19:18.399>pass <00:19:18.493>or <00:19:18.586>fail, <00:19:18.680>and 00:19:19.670 --> 00:19:19.680 align:start position:0% <00:19:19.670>check <00:19:19.671>how <00:19:19.672>many <00:19:19.673>of <00:19:19.674>them <00:19:19.675>pass <00:19:19.676>or <00:19:19.677>fail, <00:19:19.679>and 00:19:19.680 --> 00:19:22.150 align:start position:0% <00:19:19.680>check <00:19:19.767>how <00:19:19.855>many <00:19:19.942>of <00:19:20.030>them <00:19:20.117>pass <00:19:20.205>or <00:19:20.292>fail, <00:19:20.380>and <00:19:20.467>then <00:19:20.555>you <00:19:20.642>repeat <00:19:20.730>this <00:19:20.817>loop <00:19:20.905>until <00:19:20.992>you <00:19:21.080>have 00:19:22.150 --> 00:19:22.160 align:start position:0% <00:19:22.150>then <00:19:22.151>you <00:19:22.152>repeat <00:19:22.153>this <00:19:22.155>loop <00:19:22.156>until <00:19:22.157>you <00:19:22.159>have 00:19:22.160 --> 00:19:25.270 align:start position:0% <00:19:22.160>then <00:19:22.260>you <00:19:22.360>repeat <00:19:22.460>this <00:19:22.560>loop <00:19:22.660>until <00:19:22.760>you <00:19:22.860>have <00:19:22.960>a <00:19:23.060>performance <00:19:23.160>bar <00:19:23.260>that <00:19:23.360>surpass <00:19:23.460>like <00:19:23.560>what 00:19:25.270 --> 00:19:25.280 align:start position:0% <00:19:25.270>a <00:19:25.271>performance <00:19:25.273>bar <00:19:25.274>that <00:19:25.276>surpass <00:19:25.277>like <00:19:25.279>what 00:19:25.280 --> 00:19:27.270 align:start position:0% <00:19:25.280>a <00:19:25.387>performance <00:19:25.495>bar <00:19:25.603>that <00:19:25.710>surpass <00:19:25.818>like <00:19:25.926>what <00:19:26.033>you <00:19:26.141>expect <00:19:26.249>from <00:19:26.356>this <00:19:26.464>project. <00:19:26.572>Most <00:19:26.680>of 00:19:27.270 --> 00:19:27.280 align:start position:0% <00:19:27.270>you <00:19:27.271>expect <00:19:27.273>from <00:19:27.274>this <00:19:27.276>project. <00:19:27.277>Most <00:19:27.279>of 00:19:27.280 --> 00:19:29.270 align:start position:0% <00:19:27.280>you <00:19:27.379>expect <00:19:27.480>from <00:19:27.580>this <00:19:27.680>project. <00:19:27.780>Most <00:19:27.879>of <00:19:27.980>the <00:19:28.080>time <00:19:28.180>it <00:19:28.280>should <00:19:28.379>like <00:19:28.480>it's <00:19:28.580>not <00:19:28.680>100% 00:19:29.270 --> 00:19:29.280 align:start position:0% <00:19:29.270>the <00:19:29.271>time <00:19:29.272>it <00:19:29.273>should <00:19:29.275>like <00:19:29.276>it's <00:19:29.277>not <00:19:29.279>100% 00:19:29.280 --> 00:19:30.750 align:start position:0% <00:19:29.280>the <00:19:29.368>time <00:19:29.456>it <00:19:29.544>should <00:19:29.632>like <00:19:29.721>it's <00:19:29.809>not <00:19:29.897>100% <00:19:29.985>but <00:19:30.073>you <00:19:30.162>need <00:19:30.250>to <00:19:30.338>find <00:19:30.426>your <00:19:30.514>bar, <00:19:30.603>your 00:19:30.750 --> 00:19:30.760 align:start position:0% <00:19:30.750>but <00:19:30.751>you <00:19:30.752>need <00:19:30.753>to <00:19:30.755>find <00:19:30.756>your <00:19:30.757>bar, <00:19:30.759>your 00:19:30.760 --> 00:19:33.310 align:start position:0% <00:19:30.760>but <00:19:30.853>you <00:19:30.946>need <00:19:31.040>to <00:19:31.133>find <00:19:31.226>your <00:19:31.320>bar, <00:19:31.413>your <00:19:31.506>balance <00:19:31.600>that <00:19:31.693>you <00:19:31.786>need <00:19:31.879>in <00:19:31.973>order <00:19:32.066>to <00:19:32.160>move 00:19:33.310 --> 00:19:33.320 align:start position:0% <00:19:33.310>balance <00:19:33.311>that <00:19:33.312>you <00:19:33.313>need <00:19:33.315>in <00:19:33.316>order <00:19:33.317>to <00:19:33.319>move 00:19:33.320 --> 00:19:36.710 align:start position:0% <00:19:33.320>balance <00:19:33.447>that <00:19:33.574>you <00:19:33.701>need <00:19:33.829>in <00:19:33.956>order <00:19:34.083>to <00:19:34.210>move <00:19:34.338>forward <00:19:34.465>with <00:19:34.592>this <00:19:34.720>loop. 00:19:36.720 --> 00:19:39.030 align:start position:0% <00:19:36.720>So, <00:19:37.000>let's <00:19:37.280>see <00:19:37.560>some <00:19:37.840>concrete <00:19:38.120>examples 00:19:39.030 --> 00:19:39.040 align:start position:0% <00:19:39.030>So, <00:19:39.031>let's <00:19:39.033>see <00:19:39.035>some <00:19:39.037>concrete <00:19:39.039>examples 00:19:39.040 --> 00:19:41.350 align:start position:0% <00:19:39.040>So, <00:19:39.147>let's <00:19:39.255>see <00:19:39.363>some <00:19:39.470>concrete <00:19:39.578>examples <00:19:39.686>that <00:19:39.793>we <00:19:39.901>were <00:19:40.009>using <00:19:40.116>in <00:19:40.224>our <00:19:40.332>workflow. <00:19:40.440>For 00:19:41.350 --> 00:19:41.360 align:start position:0% <00:19:41.350>that <00:19:41.351>we <00:19:41.352>were <00:19:41.353>using <00:19:41.355>in <00:19:41.356>our <00:19:41.357>workflow. <00:19:41.359>For 00:19:41.360 --> 00:19:43.110 align:start position:0% <00:19:41.360>that <00:19:41.467>we <00:19:41.575>were <00:19:41.683>using <00:19:41.790>in <00:19:41.898>our <00:19:42.006>workflow. <00:19:42.113>For <00:19:42.221>example, <00:19:42.329>the <00:19:42.436>planner, <00:19:42.544>we <00:19:42.652>have <00:19:42.760>exact 00:19:43.110 --> 00:19:43.120 align:start position:0% <00:19:43.110>example, <00:19:43.111>the <00:19:43.113>planner, <00:19:43.115>we <00:19:43.117>have <00:19:43.119>exact 00:19:43.120 --> 00:19:45.230 align:start position:0% <00:19:43.120>example, <00:19:43.260>the <00:19:43.399>planner, <00:19:43.540>we <00:19:43.679>have <00:19:43.820>exact <00:19:43.959>match. <00:19:44.100>Like <00:19:44.239>pretty <00:19:44.379>straightforward, <00:19:44.520>you 00:19:45.230 --> 00:19:45.240 align:start position:0% <00:19:45.230>match. <00:19:45.232>Like <00:19:45.234>pretty <00:19:45.236>straightforward, <00:19:45.239>you 00:19:45.240 --> 00:19:47.670 align:start position:0% <00:19:45.240>match. <00:19:45.356>Like <00:19:45.473>pretty <00:19:45.590>straightforward, <00:19:45.706>you <00:19:45.823>have <00:19:45.940>a <00:19:46.056>query, <00:19:46.173>small <00:19:46.290>group, <00:19:46.406>food <00:19:46.523>tour <00:19:46.640>in 00:19:47.670 --> 00:19:47.680 align:start position:0% <00:19:47.670>have <00:19:47.671>a <00:19:47.672>query, <00:19:47.673>small <00:19:47.675>group, <00:19:47.676>food <00:19:47.677>tour <00:19:47.679>in 00:19:47.680 --> 00:19:50.830 align:start position:0% <00:19:47.680>have <00:19:47.787>a <00:19:47.895>query, <00:19:48.003>small <00:19:48.110>group, <00:19:48.218>food <00:19:48.326>tour <00:19:48.433>in <00:19:48.541>Rome, <00:19:48.649>let's <00:19:48.756>say <00:19:48.864>Saturday <00:19:48.972>morning, <00:19:49.080>and 00:19:50.830 --> 00:19:50.840 align:start position:0% <00:19:50.830>Rome, <00:19:50.831>let's <00:19:50.833>say <00:19:50.835>Saturday <00:19:50.837>morning, <00:19:50.839>and 00:19:50.840 --> 00:19:53.070 align:start position:0% <00:19:50.840>Rome, <00:19:50.947>let's <00:19:51.055>say <00:19:51.163>Saturday <00:19:51.270>morning, <00:19:51.378>and <00:19:51.486>you <00:19:51.593>want <00:19:51.701>that <00:19:51.809>those <00:19:51.916>things <00:19:52.024>match <00:19:52.132>in <00:19:52.240>the 00:19:53.070 --> 00:19:53.080 align:start position:0% <00:19:53.070>you <00:19:53.071>want <00:19:53.072>that <00:19:53.073>those <00:19:53.075>things <00:19:53.076>match <00:19:53.077>in <00:19:53.079>the 00:19:53.080 --> 00:19:55.030 align:start position:0% <00:19:53.080>you <00:19:53.187>want <00:19:53.295>that <00:19:53.403>those <00:19:53.510>things <00:19:53.618>match <00:19:53.726>in <00:19:53.833>the <00:19:53.941>exact <00:19:54.049>fields <00:19:54.156>that <00:19:54.264>you <00:19:54.372>have, <00:19:54.480>like 00:19:55.030 --> 00:19:55.040 align:start position:0% <00:19:55.030>exact <00:19:55.031>fields <00:19:55.033>that <00:19:55.035>you <00:19:55.037>have, <00:19:55.039>like 00:19:55.040 --> 00:19:57.870 align:start position:0% <00:19:55.040>exact <00:19:55.167>fields <00:19:55.294>that <00:19:55.421>you <00:19:55.549>have, <00:19:55.676>like <00:19:55.803>destination <00:19:55.930>Rome, <00:19:56.058>the <00:19:56.185>date, <00:19:56.312>whatever <00:19:56.440>it 00:19:57.870 --> 00:19:57.880 align:start position:0% <00:19:57.870>destination <00:19:57.871>Rome, <00:19:57.873>the <00:19:57.875>date, <00:19:57.877>whatever <00:19:57.879>it 00:19:57.880 --> 00:20:00.790 align:start position:0% <00:19:57.880>destination <00:19:57.996>Rome, <00:19:58.113>the <00:19:58.230>date, <00:19:58.346>whatever <00:19:58.463>it <00:19:58.580>is, <00:19:58.696>time <00:19:58.813>window <00:19:58.930>morning, <00:19:59.046>and <00:19:59.163>etc. <00:19:59.280>Then 00:20:00.790 --> 00:20:00.800 align:start position:0% <00:20:00.790>is, <00:20:00.791>time <00:20:00.793>window <00:20:00.794>morning, <00:20:00.796>and <00:20:00.797>etc. <00:20:00.799>Then 00:20:00.800 --> 00:20:03.070 align:start position:0% <00:20:00.800>is, <00:20:00.907>time <00:20:01.015>window <00:20:01.123>morning, <00:20:01.230>and <00:20:01.338>etc. <00:20:01.446>Then <00:20:01.553>you <00:20:01.661>have <00:20:01.769>explicit <00:20:01.876>null <00:20:01.984>check <00:20:02.092>where <00:20:02.200>you 00:20:03.070 --> 00:20:03.080 align:start position:0% <00:20:03.070>you <00:20:03.071>have <00:20:03.072>explicit <00:20:03.074>null <00:20:03.076>check <00:20:03.077>where <00:20:03.079>you 00:20:03.080 --> 00:20:05.270 align:start position:0% <00:20:03.080>you <00:20:03.187>have <00:20:03.295>explicit <00:20:03.403>null <00:20:03.510>check <00:20:03.618>where <00:20:03.726>you <00:20:03.833>have <00:20:03.941>a <00:20:04.049>query, <00:20:04.156>you <00:20:04.264>have <00:20:04.372>some <00:20:04.480>expected 00:20:05.270 --> 00:20:05.280 align:start position:0% <00:20:05.270>have <00:20:05.271>a <00:20:05.273>query, <00:20:05.274>you <00:20:05.276>have <00:20:05.277>some <00:20:05.279>expected 00:20:05.280 --> 00:20:07.270 align:start position:0% <00:20:05.280>have <00:20:05.387>a <00:20:05.495>query, <00:20:05.603>you <00:20:05.710>have <00:20:05.818>some <00:20:05.926>expected <00:20:06.033>fields, <00:20:06.141>but <00:20:06.249>also <00:20:06.356>some <00:20:06.464>things <00:20:06.572>that <00:20:06.680>you 00:20:07.270 --> 00:20:07.280 align:start position:0% <00:20:07.270>fields, <00:20:07.271>but <00:20:07.273>also <00:20:07.274>some <00:20:07.276>things <00:20:07.277>that <00:20:07.279>you 00:20:07.280 --> 00:20:09.550 align:start position:0% <00:20:07.280>fields, <00:20:07.379>but <00:20:07.480>also <00:20:07.580>some <00:20:07.680>things <00:20:07.780>that <00:20:07.879>you <00:20:07.980>expect <00:20:08.080>to <00:20:08.180>be <00:20:08.280>null. <00:20:08.379>For <00:20:08.480>example, <00:20:08.580>in <00:20:08.680>this 00:20:09.550 --> 00:20:09.560 align:start position:0% <00:20:09.550>expect <00:20:09.551>to <00:20:09.552>be <00:20:09.553>null. <00:20:09.555>For <00:20:09.556>example, <00:20:09.557>in <00:20:09.559>this 00:20:09.560 --> 00:20:11.950 align:start position:0% <00:20:09.560>expect <00:20:09.653>to <00:20:09.746>be <00:20:09.840>null. <00:20:09.933>For <00:20:10.026>example, <00:20:10.120>in <00:20:10.213>this <00:20:10.306>case <00:20:10.399>keywords. <00:20:10.493>A <00:20:10.586>lot <00:20:10.679>of <00:20:10.773>times <00:20:10.866>we <00:20:10.960>tried 00:20:11.950 --> 00:20:11.960 align:start position:0% <00:20:11.950>case <00:20:11.951>keywords. <00:20:11.952>A <00:20:11.953>lot <00:20:11.955>of <00:20:11.956>times <00:20:11.957>we <00:20:11.959>tried 00:20:11.960 --> 00:20:13.790 align:start position:0% <00:20:11.960>case <00:20:12.060>keywords. <00:20:12.160>A <00:20:12.260>lot <00:20:12.360>of <00:20:12.460>times <00:20:12.560>we <00:20:12.660>tried <00:20:12.760>that <00:20:12.860>for <00:20:12.960>example <00:20:13.060>with <00:20:13.160>simple <00:20:13.260>queries <00:20:13.360>in 00:20:13.790 --> 00:20:13.800 align:start position:0% <00:20:13.790>that <00:20:13.791>for <00:20:13.793>example <00:20:13.794>with <00:20:13.796>simple <00:20:13.797>queries <00:20:13.799>in 00:20:13.800 --> 00:20:15.950 align:start position:0% <00:20:13.800>that <00:20:13.907>for <00:20:14.015>example <00:20:14.123>with <00:20:14.230>simple <00:20:14.338>queries <00:20:14.446>in <00:20:14.553>this <00:20:14.661>case, <00:20:14.769>some <00:20:14.876>keywords <00:20:14.984>were <00:20:15.092>get <00:20:15.200>filled 00:20:15.950 --> 00:20:15.960 align:start position:0% <00:20:15.950>this <00:20:15.951>case, <00:20:15.953>some <00:20:15.954>keywords <00:20:15.956>were <00:20:15.957>get <00:20:15.959>filled 00:20:15.960 --> 00:20:20.590 align:start position:0% <00:20:15.960>this <00:20:16.053>case, <00:20:16.146>some <00:20:16.240>keywords <00:20:16.333>were <00:20:16.426>get <00:20:16.520>filled <00:20:16.613>with <00:20:16.706>things <00:20:16.800>to <00:20:16.893>do <00:20:16.986>or <00:20:17.080>museums <00:20:17.173>or <00:20:17.266>tours <00:20:17.360>or 00:20:20.590 --> 00:20:20.600 align:start position:0% <00:20:20.590>with <00:20:20.591>things <00:20:20.592>to <00:20:20.593>do <00:20:20.594>or <00:20:20.595>museums <00:20:20.596>or <00:20:20.597>tours <00:20:20.599>or 00:20:20.600 --> 00:20:22.110 align:start position:0% <00:20:20.600>with <00:20:20.690>things <00:20:20.781>to <00:20:20.871>do <00:20:20.962>or <00:20:21.053>museums <00:20:21.143>or <00:20:21.234>tours <00:20:21.324>or <00:20:21.415>whatever. <00:20:21.505>In <00:20:21.596>this <00:20:21.687>case, <00:20:21.777>we <00:20:21.868>don't <00:20:21.958>want 00:20:22.110 --> 00:20:22.120 align:start position:0% <00:20:22.110>whatever. <00:20:22.111>In <00:20:22.112>this <00:20:22.114>case, <00:20:22.116>we <00:20:22.117>don't <00:20:22.119>want 00:20:22.120 --> 00:20:24.030 align:start position:0% <00:20:22.120>whatever. <00:20:22.219>In <00:20:22.320>this <00:20:22.419>case, <00:20:22.520>we <00:20:22.620>don't <00:20:22.719>want <00:20:22.820>nothing, <00:20:22.919>so <00:20:23.020>you <00:20:23.120>need <00:20:23.219>to <00:20:23.320>also <00:20:23.419>to <00:20:23.520>check 00:20:24.030 --> 00:20:24.040 align:start position:0% <00:20:24.030>nothing, <00:20:24.031>so <00:20:24.032>you <00:20:24.033>need <00:20:24.035>to <00:20:24.036>also <00:20:24.037>to <00:20:24.039>check 00:20:24.040 --> 00:20:26.150 align:start position:0% <00:20:24.040>nothing, <00:20:24.180>so <00:20:24.320>you <00:20:24.460>need <00:20:24.600>to <00:20:24.740>also <00:20:24.879>to <00:20:25.020>check <00:20:25.159>the <00:20:25.300>false <00:20:25.440>positives. 00:20:26.150 --> 00:20:26.160 align:start position:0% <00:20:26.150>the <00:20:26.154>false <00:20:26.159>positives. 00:20:26.160 --> 00:20:28.030 align:start position:0% <00:20:26.160>the <00:20:26.300>false <00:20:26.440>positives. <00:20:26.580>And <00:20:26.720>then <00:20:26.860>we <00:20:27.000>have <00:20:27.140>the <00:20:27.280>routing <00:20:27.420>flag. <00:20:27.560>For 00:20:28.030 --> 00:20:28.040 align:start position:0% <00:20:28.030>And <00:20:28.031>then <00:20:28.032>we <00:20:28.033>have <00:20:28.035>the <00:20:28.036>routing <00:20:28.037>flag. <00:20:28.039>For 00:20:28.040 --> 00:20:30.030 align:start position:0% <00:20:28.040>And <00:20:28.139>then <00:20:28.240>we <00:20:28.340>have <00:20:28.440>the <00:20:28.540>routing <00:20:28.639>flag. <00:20:28.740>For <00:20:28.840>example, <00:20:28.940>in <00:20:29.040>this <00:20:29.139>case <00:20:29.240>the <00:20:29.340>query <00:20:29.440>seems 00:20:30.030 --> 00:20:30.040 align:start position:0% <00:20:30.030>example, <00:20:30.031>in <00:20:30.033>this <00:20:30.034>case <00:20:30.036>the <00:20:30.037>query <00:20:30.039>seems 00:20:30.040 --> 00:20:32.870 align:start position:0% <00:20:30.040>example, <00:20:30.133>in <00:20:30.226>this <00:20:30.320>case <00:20:30.413>the <00:20:30.506>query <00:20:30.600>seems <00:20:30.693>kind <00:20:30.786>of <00:20:30.879>simple. <00:20:30.973>It's <00:20:31.066>like <00:20:31.159>if <00:20:31.253>you <00:20:31.346>want <00:20:31.440>to 00:20:32.870 --> 00:20:32.880 align:start position:0% <00:20:32.870>kind <00:20:32.871>of <00:20:32.872>simple. <00:20:32.873>It's <00:20:32.874>like <00:20:32.875>if <00:20:32.876>you <00:20:32.877>want <00:20:32.879>to 00:20:32.880 --> 00:20:35.550 align:start position:0% <00:20:32.880>kind <00:20:32.980>of <00:20:33.080>simple. <00:20:33.180>It's <00:20:33.280>like <00:20:33.380>if <00:20:33.480>you <00:20:33.580>want <00:20:33.680>to <00:20:33.780>visit <00:20:33.880>Giant's <00:20:33.980>Causeways <00:20:34.080>from <00:20:34.180>Dublin, <00:20:34.280>but 00:20:35.550 --> 00:20:35.560 align:start position:0% <00:20:35.550>visit <00:20:35.551>Giant's <00:20:35.553>Causeways <00:20:35.555>from <00:20:35.557>Dublin, <00:20:35.559>but 00:20:35.560 --> 00:20:37.430 align:start position:0% <00:20:35.560>visit <00:20:35.687>Giant's <00:20:35.814>Causeways <00:20:35.941>from <00:20:36.069>Dublin, <00:20:36.196>but <00:20:36.323>those <00:20:36.450>type <00:20:36.578>of <00:20:36.705>activities <00:20:36.832>are <00:20:36.960>pretty 00:20:37.430 --> 00:20:37.440 align:start position:0% <00:20:37.430>those <00:20:37.431>type <00:20:37.433>of <00:20:37.435>activities <00:20:37.437>are <00:20:37.439>pretty 00:20:37.440 --> 00:20:39.790 align:start position:0% <00:20:37.440>those <00:20:37.567>type <00:20:37.694>of <00:20:37.821>activities <00:20:37.949>are <00:20:38.076>pretty <00:20:38.203>difficult <00:20:38.330>to <00:20:38.458>find <00:20:38.585>because <00:20:38.712>they <00:20:38.840>start 00:20:39.790 --> 00:20:39.800 align:start position:0% <00:20:39.790>difficult <00:20:39.791>to <00:20:39.793>find <00:20:39.795>because <00:20:39.797>they <00:20:39.799>start 00:20:39.800 --> 00:20:41.790 align:start position:0% <00:20:39.800>difficult <00:20:39.907>to <00:20:40.015>find <00:20:40.123>because <00:20:40.230>they <00:20:40.338>start <00:20:40.446>from <00:20:40.553>Dublin <00:20:40.661>that <00:20:40.769>is <00:20:40.876>not <00:20:40.984>that <00:20:41.092>common <00:20:41.200>for 00:20:41.790 --> 00:20:41.800 align:start position:0% <00:20:41.790>from <00:20:41.791>Dublin <00:20:41.792>that <00:20:41.793>is <00:20:41.795>not <00:20:41.796>that <00:20:41.797>common <00:20:41.799>for 00:20:41.800 --> 00:20:45.150 align:start position:0% <00:20:41.800>from <00:20:41.893>Dublin <00:20:41.986>that <00:20:42.080>is <00:20:42.173>not <00:20:42.266>that <00:20:42.360>common <00:20:42.453>for <00:20:42.546>this <00:20:42.639>place. <00:20:42.733>So, <00:20:42.826>for <00:20:42.919>us, <00:20:43.013>this <00:20:43.106>we <00:20:43.200>expect 00:20:45.150 --> 00:20:45.160 align:start position:0% <00:20:45.150>this <00:20:45.151>place. <00:20:45.152>So, <00:20:45.153>for <00:20:45.155>us, <00:20:45.156>this <00:20:45.157>we <00:20:45.159>expect 00:20:45.160 --> 00:20:46.950 align:start position:0% <00:20:45.160>this <00:20:45.253>place. <00:20:45.346>So, <00:20:45.440>for <00:20:45.533>us, <00:20:45.626>this <00:20:45.720>we <00:20:45.813>expect <00:20:45.906>that <00:20:46.000>the <00:20:46.093>deep <00:20:46.186>filter <00:20:46.280>is <00:20:46.373>true. <00:20:46.466>So, <00:20:46.560>it 00:20:46.950 --> 00:20:46.960 align:start position:0% <00:20:46.950>that <00:20:46.951>the <00:20:46.952>deep <00:20:46.953>filter <00:20:46.955>is <00:20:46.956>true. <00:20:46.957>So, <00:20:46.959>it 00:20:46.960 --> 00:20:49.710 align:start position:0% <00:20:46.960>that <00:20:47.047>the <00:20:47.135>deep <00:20:47.222>filter <00:20:47.310>is <00:20:47.397>true. <00:20:47.485>So, <00:20:47.572>it <00:20:47.660>needs <00:20:47.747>to <00:20:47.835>call <00:20:47.922>a <00:20:48.010>stronger <00:20:48.097>LLM <00:20:48.185>to <00:20:48.272>find <00:20:48.360>the 00:20:49.710 --> 00:20:49.720 align:start position:0% <00:20:49.710>needs <00:20:49.711>to <00:20:49.712>call <00:20:49.713>a <00:20:49.714>stronger <00:20:49.715>LLM <00:20:49.716>to <00:20:49.717>find <00:20:49.719>the 00:20:49.720 --> 00:20:52.310 align:start position:0% <00:20:49.720>needs <00:20:49.836>to <00:20:49.953>call <00:20:50.070>a <00:20:50.186>stronger <00:20:50.303>LLM <00:20:50.420>to <00:20:50.536>find <00:20:50.653>the <00:20:50.770>right <00:20:50.886>use <00:20:51.003>cases <00:20:51.120>here. 00:20:52.310 --> 00:20:52.320 align:start position:0% <00:20:52.310>right <00:20:52.312>use <00:20:52.316>cases <00:20:52.319>here. 00:20:52.320 --> 00:20:54.110 align:start position:0% <00:20:52.320>right <00:20:52.460>use <00:20:52.600>cases <00:20:52.740>here. <00:20:52.879>Then <00:20:53.020>overlap <00:20:53.159>score. <00:20:53.300>We <00:20:53.439>said <00:20:53.580>that <00:20:53.720>there 00:20:54.110 --> 00:20:54.120 align:start position:0% <00:20:54.110>Then <00:20:54.111>overlap <00:20:54.112>score. <00:20:54.114>We <00:20:54.116>said <00:20:54.117>that <00:20:54.119>there 00:20:54.120 --> 00:20:56.270 align:start position:0% <00:20:54.120>Then <00:20:54.227>overlap <00:20:54.335>score. <00:20:54.443>We <00:20:54.550>said <00:20:54.658>that <00:20:54.766>there <00:20:54.873>is <00:20:54.981>a <00:20:55.089>chat <00:20:55.196>with <00:20:55.304>multiple <00:20:55.412>with <00:20:55.520>multiple 00:20:56.270 --> 00:20:56.280 align:start position:0% <00:20:56.270>is <00:20:56.271>a <00:20:56.273>chat <00:20:56.274>with <00:20:56.276>multiple <00:20:56.277>with <00:20:56.279>multiple 00:20:56.280 --> 00:20:58.230 align:start position:0% <00:20:56.280>is <00:20:56.379>a <00:20:56.480>chat <00:20:56.580>with <00:20:56.680>multiple <00:20:56.780>with <00:20:56.879>multiple <00:20:56.980>messages <00:20:57.080>in <00:20:57.180>the <00:20:57.280>same <00:20:57.379>chat. <00:20:57.480>So, <00:20:57.580>we <00:20:57.680>want 00:20:58.230 --> 00:20:58.240 align:start position:0% <00:20:58.230>messages <00:20:58.231>in <00:20:58.232>the <00:20:58.233>same <00:20:58.235>chat. <00:20:58.236>So, <00:20:58.237>we <00:20:58.239>want 00:20:58.240 --> 00:21:00.670 align:start position:0% <00:20:58.240>messages <00:20:58.327>in <00:20:58.415>the <00:20:58.502>same <00:20:58.590>chat. <00:20:58.677>So, <00:20:58.765>we <00:20:58.852>want <00:20:58.940>to <00:20:59.027>make <00:20:59.115>sure <00:20:59.202>that <00:20:59.290>our <00:20:59.377>intent <00:20:59.465>is <00:20:59.552>able <00:20:59.640>to 00:21:00.670 --> 00:21:00.680 align:start position:0% <00:21:00.670>to <00:21:00.671>make <00:21:00.672>sure <00:21:00.673>that <00:21:00.674>our <00:21:00.675>intent <00:21:00.676>is <00:21:00.677>able <00:21:00.679>to 00:21:00.680 --> 00:21:02.630 align:start position:0% <00:21:00.680>to <00:21:00.787>make <00:21:00.895>sure <00:21:01.003>that <00:21:01.110>our <00:21:01.218>intent <00:21:01.326>is <00:21:01.433>able <00:21:01.541>to <00:21:01.649>understand <00:21:01.756>the <00:21:01.864>different <00:21:01.972>messages <00:21:02.080>and 00:21:02.630 --> 00:21:02.640 align:start position:0% <00:21:02.630>understand <00:21:02.632>the <00:21:02.634>different <00:21:02.636>messages <00:21:02.639>and 00:21:02.640 --> 00:21:04.790 align:start position:0% <00:21:02.640>understand <00:21:02.767>the <00:21:02.894>different <00:21:03.021>messages <00:21:03.149>and <00:21:03.276>resolve <00:21:03.403>in <00:21:03.530>one <00:21:03.658>query <00:21:03.785>that <00:21:03.912>kind <00:21:04.040>of 00:21:04.790 --> 00:21:04.800 align:start position:0% <00:21:04.790>resolve <00:21:04.791>in <00:21:04.793>one <00:21:04.794>query <00:21:04.796>that <00:21:04.797>kind <00:21:04.799>of 00:21:04.800 --> 00:21:07.350 align:start position:0% <00:21:04.800>resolve <00:21:04.916>in <00:21:05.033>one <00:21:05.149>query <00:21:05.266>that <00:21:05.383>kind <00:21:05.500>of <00:21:05.616>compact <00:21:05.733>and <00:21:05.850>assign <00:21:05.966>everything <00:21:06.083>in <00:21:06.200>the 00:21:07.350 --> 00:21:07.360 align:start position:0% <00:21:07.350>compact <00:21:07.351>and <00:21:07.353>assign <00:21:07.355>everything <00:21:07.357>in <00:21:07.359>the 00:21:07.360 --> 00:21:09.230 align:start position:0% <00:21:07.360>compact <00:21:07.500>and <00:21:07.639>assign <00:21:07.780>everything <00:21:07.919>in <00:21:08.060>the <00:21:08.199>same <00:21:08.340>query <00:21:08.479>all <00:21:08.620>the <00:21:08.760>historical 00:21:09.230 --> 00:21:09.240 align:start position:0% <00:21:09.230>same <00:21:09.232>query <00:21:09.234>all <00:21:09.236>the <00:21:09.239>historical 00:21:09.240 --> 00:21:11.910 align:start position:0% <00:21:09.240>same <00:21:09.415>query <00:21:09.590>all <00:21:09.765>the <00:21:09.940>historical <00:21:10.115>conversation <00:21:10.290>that <00:21:10.465>you <00:21:10.640>had. 00:21:11.910 --> 00:21:11.920 align:start position:0% <00:21:11.910>conversation <00:21:11.913>that <00:21:11.916>you <00:21:11.919>had. 00:21:11.920 --> 00:21:13.710 align:start position:0% <00:21:11.920>conversation <00:21:12.047>that <00:21:12.174>you <00:21:12.301>had. <00:21:12.429>And <00:21:12.556>then <00:21:12.683>simple <00:21:12.810>keep <00:21:12.938>or <00:21:13.065>drop <00:21:13.192>for <00:21:13.320>the 00:21:13.710 --> 00:21:13.720 align:start position:0% <00:21:13.710>And <00:21:13.711>then <00:21:13.712>simple <00:21:13.713>keep <00:21:13.715>or <00:21:13.716>drop <00:21:13.717>for <00:21:13.719>the 00:21:13.720 --> 00:21:15.350 align:start position:0% <00:21:13.720>And <00:21:13.827>then <00:21:13.935>simple <00:21:14.043>keep <00:21:14.150>or <00:21:14.258>drop <00:21:14.366>for <00:21:14.473>the <00:21:14.581>relevance. <00:21:14.689>You <00:21:14.796>have <00:21:14.904>a <00:21:15.012>query, <00:21:15.120>some 00:21:15.350 --> 00:21:15.360 align:start position:0% <00:21:15.350>relevance. <00:21:15.351>You <00:21:15.353>have <00:21:15.355>a <00:21:15.357>query, <00:21:15.359>some 00:21:15.360 --> 00:21:17.390 align:start position:0% <00:21:15.360>relevance. <00:21:15.476>You <00:21:15.593>have <00:21:15.709>a <00:21:15.826>query, <00:21:15.943>some <00:21:16.060>example <00:21:16.176>of <00:21:16.293>activities, <00:21:16.409>and <00:21:16.526>which <00:21:16.643>one <00:21:16.760>you 00:21:17.390 --> 00:21:17.400 align:start position:0% <00:21:17.390>example <00:21:17.391>of <00:21:17.393>activities, <00:21:17.394>and <00:21:17.396>which <00:21:17.397>one <00:21:17.399>you 00:21:17.400 --> 00:21:19.470 align:start position:0% <00:21:17.400>example <00:21:17.500>of <00:21:17.600>activities, <00:21:17.700>and <00:21:17.800>which <00:21:17.900>one <00:21:18.000>you <00:21:18.100>expect <00:21:18.200>to <00:21:18.300>keep, <00:21:18.400>and <00:21:18.500>which <00:21:18.600>one <00:21:18.700>you <00:21:18.800>don't 00:21:19.470 --> 00:21:19.480 align:start position:0% <00:21:19.470>expect <00:21:19.471>to <00:21:19.472>keep, <00:21:19.473>and <00:21:19.475>which <00:21:19.476>one <00:21:19.477>you <00:21:19.479>don't 00:21:19.480 --> 00:21:22.190 align:start position:0% <00:21:19.480>expect <00:21:19.620>to <00:21:19.760>keep, <00:21:19.900>and <00:21:20.040>which <00:21:20.180>one <00:21:20.320>you <00:21:20.460>don't <00:21:20.600>expect <00:21:20.740>to <00:21:20.880>keep. 00:21:22.190 --> 00:21:22.200 align:start position:0% <00:21:22.190>expect <00:21:22.194>to <00:21:22.199>keep. 00:21:22.200 --> 00:21:23.870 align:start position:0% <00:21:22.200>expect <00:21:22.340>to <00:21:22.480>keep. <00:21:22.620>So, <00:21:22.760>as <00:21:22.900>I <00:21:23.040>said, <00:21:23.180>we <00:21:23.320>have <00:21:23.460>our <00:21:23.600>current 00:21:23.870 --> 00:21:23.880 align:start position:0% <00:21:23.870>So, <00:21:23.871>as <00:21:23.872>I <00:21:23.873>said, <00:21:23.875>we <00:21:23.876>have <00:21:23.877>our <00:21:23.879>current 00:21:23.880 --> 00:21:26.350 align:start position:0% <00:21:23.880>So, <00:21:23.987>as <00:21:24.095>I <00:21:24.203>said, <00:21:24.310>we <00:21:24.418>have <00:21:24.526>our <00:21:24.633>current <00:21:24.741>approach, <00:21:24.849>and <00:21:24.956>from <00:21:25.064>all <00:21:25.172>of <00:21:25.280>them, 00:21:26.350 --> 00:21:26.360 align:start position:0% <00:21:26.350>approach, <00:21:26.351>and <00:21:26.353>from <00:21:26.355>all <00:21:26.357>of <00:21:26.359>them, 00:21:26.360 --> 00:21:27.910 align:start position:0% <00:21:26.360>approach, <00:21:26.514>and <00:21:26.669>from <00:21:26.825>all <00:21:26.980>of <00:21:27.135>them, <00:21:27.290>actually, <00:21:27.445>that <00:21:27.600>was <00:21:27.755>the 00:21:27.910 --> 00:21:27.920 align:start position:0% <00:21:27.910>actually, <00:21:27.913>that <00:21:27.916>was <00:21:27.919>the 00:21:27.920 --> 00:21:30.990 align:start position:0% <00:21:27.920>actually, <00:21:28.036>that <00:21:28.153>was <00:21:28.270>the <00:21:28.386>phase <00:21:28.503>where <00:21:28.620>it <00:21:28.736>took <00:21:28.853>the <00:21:28.970>most <00:21:29.086>to <00:21:29.203>of <00:21:29.320>time 00:21:30.990 --> 00:21:31.000 align:start position:0% <00:21:30.990>phase <00:21:30.991>where <00:21:30.992>it <00:21:30.993>took <00:21:30.994>the <00:21:30.995>most <00:21:30.996>to <00:21:30.997>of <00:21:30.999>time 00:21:31.000 --> 00:21:33.590 align:start position:0% <00:21:31.000>phase <00:21:31.093>where <00:21:31.186>it <00:21:31.280>took <00:21:31.373>the <00:21:31.466>most <00:21:31.560>to <00:21:31.653>of <00:21:31.746>time <00:21:31.840>to <00:21:31.933>build <00:21:32.026>the <00:21:32.120>agent, <00:21:32.213>because <00:21:32.306>that's <00:21:32.400>where 00:21:33.590 --> 00:21:33.600 align:start position:0% <00:21:33.590>to <00:21:33.591>build <00:21:33.592>the <00:21:33.594>agent, <00:21:33.596>because <00:21:33.597>that's <00:21:33.599>where 00:21:33.600 --> 00:21:35.070 align:start position:0% <00:21:33.600>to <00:21:33.701>build <00:21:33.803>the <00:21:33.905>agent, <00:21:34.007>because <00:21:34.108>that's <00:21:34.210>where <00:21:34.312>you <00:21:34.414>need <00:21:34.515>to <00:21:34.617>write <00:21:34.719>those <00:21:34.821>examples. <00:21:34.923>You 00:21:35.070 --> 00:21:35.080 align:start position:0% <00:21:35.070>you <00:21:35.071>need <00:21:35.072>to <00:21:35.074>write <00:21:35.076>those <00:21:35.077>examples. <00:21:35.079>You 00:21:35.080 --> 00:21:35.870 align:start position:0% <00:21:35.080>you <00:21:35.127>need <00:21:35.174>to <00:21:35.222>write <00:21:35.269>those <00:21:35.317>examples. <00:21:35.364>You <00:21:35.411>need <00:21:35.459>to <00:21:35.506>know <00:21:35.553>all <00:21:35.601>of <00:21:35.648>them. <00:21:35.696>You <00:21:35.743>need <00:21:35.791>to 00:21:35.870 --> 00:21:35.880 align:start position:0% <00:21:35.870>need <00:21:35.871>to <00:21:35.872>know <00:21:35.873>all <00:21:35.874>of <00:21:35.875>them. <00:21:35.876>You <00:21:35.877>need <00:21:35.879>to 00:21:35.880 --> 00:21:38.230 align:start position:0% <00:21:35.880>need <00:21:35.962>to <00:21:36.044>know <00:21:36.127>all <00:21:36.209>of <00:21:36.291>them. <00:21:36.374>You <00:21:36.456>need <00:21:36.538>to <00:21:36.621>know <00:21:36.703>all <00:21:36.785>of <00:21:36.868>them. <00:21:36.950>You <00:21:37.032>can, <00:21:37.115>yes, <00:21:37.197>use <00:21:37.280>AI 00:21:38.230 --> 00:21:38.240 align:start position:0% <00:21:38.230>know <00:21:38.231>all <00:21:38.232>of <00:21:38.233>them. <00:21:38.234>You <00:21:38.235>can, <00:21:38.236>yes, <00:21:38.237>use <00:21:38.239>AI 00:21:38.240 --> 00:21:40.190 align:start position:0% <00:21:38.240>know <00:21:38.327>all <00:21:38.415>of <00:21:38.502>them. <00:21:38.590>You <00:21:38.677>can, <00:21:38.765>yes, <00:21:38.852>use <00:21:38.940>AI <00:21:39.027>to <00:21:39.115>help <00:21:39.202>writing <00:21:39.290>them, <00:21:39.377>but <00:21:39.465>you <00:21:39.552>need <00:21:39.640>to 00:21:40.190 --> 00:21:40.200 align:start position:0% <00:21:40.190>to <00:21:40.191>help <00:21:40.192>writing <00:21:40.193>them, <00:21:40.195>but <00:21:40.196>you <00:21:40.197>need <00:21:40.199>to 00:21:40.200 --> 00:21:41.950 align:start position:0% <00:21:40.200>to <00:21:40.307>help <00:21:40.415>writing <00:21:40.523>them, <00:21:40.630>but <00:21:40.738>you <00:21:40.846>need <00:21:40.953>to <00:21:41.061>actually <00:21:41.169>understand <00:21:41.276>all <00:21:41.384>of <00:21:41.492>them, <00:21:41.600>which 00:21:41.950 --> 00:21:41.960 align:start position:0% <00:21:41.950>actually <00:21:41.951>understand <00:21:41.953>all <00:21:41.955>of <00:21:41.957>them, <00:21:41.959>which 00:21:41.960 --> 00:21:44.230 align:start position:0% <00:21:41.960>actually <00:21:42.067>understand <00:21:42.175>all <00:21:42.283>of <00:21:42.390>them, <00:21:42.498>which <00:21:42.606>cases <00:21:42.713>are <00:21:42.821>they <00:21:42.929>covering, <00:21:43.036>why <00:21:43.144>do <00:21:43.252>you <00:21:43.360>have 00:21:44.230 --> 00:21:44.240 align:start position:0% <00:21:44.230>cases <00:21:44.231>are <00:21:44.232>they <00:21:44.233>covering, <00:21:44.235>why <00:21:44.236>do <00:21:44.237>you <00:21:44.239>have 00:21:44.240 --> 00:21:46.150 align:start position:0% <00:21:44.240>cases <00:21:44.340>are <00:21:44.440>they <00:21:44.540>covering, <00:21:44.640>why <00:21:44.740>do <00:21:44.840>you <00:21:44.940>have <00:21:45.040>need <00:21:45.140>them, <00:21:45.240>like <00:21:45.340>what <00:21:45.440>are <00:21:45.540>they <00:21:45.640>covering, 00:21:46.150 --> 00:21:46.160 align:start position:0% <00:21:46.150>need <00:21:46.151>them, <00:21:46.153>like <00:21:46.154>what <00:21:46.156>are <00:21:46.157>they <00:21:46.159>covering, 00:21:46.160 --> 00:21:48.070 align:start position:0% <00:21:46.160>need <00:21:46.260>them, <00:21:46.360>like <00:21:46.460>what <00:21:46.560>are <00:21:46.660>they <00:21:46.760>covering, <00:21:46.860>and <00:21:46.960>what's <00:21:47.060>the <00:21:47.160>scope <00:21:47.260>of <00:21:47.360>them. <00:21:47.460>So, <00:21:47.560>for 00:21:48.070 --> 00:21:48.080 align:start position:0% <00:21:48.070>and <00:21:48.071>what's <00:21:48.072>the <00:21:48.073>scope <00:21:48.075>of <00:21:48.076>them. <00:21:48.077>So, <00:21:48.079>for 00:21:48.080 --> 00:21:49.670 align:start position:0% <00:21:48.080>and <00:21:48.173>what's <00:21:48.266>the <00:21:48.360>scope <00:21:48.453>of <00:21:48.546>them. <00:21:48.639>So, <00:21:48.733>for <00:21:48.826>each <00:21:48.919>node, <00:21:49.013>you <00:21:49.106>need <00:21:49.199>to <00:21:49.293>find <00:21:49.386>all <00:21:49.480>the 00:21:49.670 --> 00:21:49.680 align:start position:0% <00:21:49.670>each <00:21:49.671>node, <00:21:49.672>you <00:21:49.673>need <00:21:49.675>to <00:21:49.676>find <00:21:49.677>all <00:21:49.679>the 00:21:49.680 --> 00:21:53.870 align:start position:0% <00:21:49.680>each <00:21:49.807>node, <00:21:49.934>you <00:21:50.061>need <00:21:50.189>to <00:21:50.316>find <00:21:50.443>all <00:21:50.570>the <00:21:50.698>scenario <00:21:50.825>and <00:21:50.952>expected <00:21:51.080>outcomes. 00:21:53.880 --> 00:21:56.110 align:start position:0% <00:21:53.880>And <00:21:54.080>once <00:21:54.280>you <00:21:54.480>have <00:21:54.680>this <00:21:54.880>golden <00:21:55.080>data <00:21:55.280>set, 00:21:56.110 --> 00:21:56.120 align:start position:0% <00:21:56.110>And <00:21:56.111>once <00:21:56.112>you <00:21:56.113>have <00:21:56.115>this <00:21:56.116>golden <00:21:56.117>data <00:21:56.119>set, 00:21:56.120 --> 00:21:58.750 align:start position:0% <00:21:56.120>And <00:21:56.219>once <00:21:56.320>you <00:21:56.419>have <00:21:56.520>this <00:21:56.620>golden <00:21:56.719>data <00:21:56.820>set, <00:21:56.919>in <00:21:57.020>this <00:21:57.120>case, <00:21:57.219>we're <00:21:57.320>using <00:21:57.419>deep <00:21:57.520>evals, 00:21:58.750 --> 00:21:58.760 align:start position:0% <00:21:58.750>in <00:21:58.751>this <00:21:58.753>case, <00:21:58.754>we're <00:21:58.756>using <00:21:58.757>deep <00:21:58.759>evals, 00:21:58.760 --> 00:22:00.830 align:start position:0% <00:21:58.760>in <00:21:58.860>this <00:21:58.960>case, <00:21:59.060>we're <00:21:59.160>using <00:21:59.260>deep <00:21:59.360>evals, <00:21:59.460>and <00:21:59.560>to <00:21:59.660>call <00:21:59.760>with <00:21:59.860>the <00:21:59.960>some <00:22:00.060>just <00:22:00.160>Git 00:22:00.830 --> 00:22:00.840 align:start position:0% <00:22:00.830>and <00:22:00.831>to <00:22:00.832>call <00:22:00.833>with <00:22:00.835>the <00:22:00.836>some <00:22:00.837>just <00:22:00.839>Git 00:22:00.840 --> 00:22:02.870 align:start position:0% <00:22:00.840>and <00:22:00.939>to <00:22:01.040>call <00:22:01.139>with <00:22:01.240>the <00:22:01.340>some <00:22:01.439>just <00:22:01.540>Git <00:22:01.639>commit <00:22:01.740>with <00:22:01.840>the <00:22:01.939>eval <00:22:02.040>section <00:22:02.139>that <00:22:02.240>will 00:22:02.870 --> 00:22:02.880 align:start position:0% <00:22:02.870>commit <00:22:02.871>with <00:22:02.873>the <00:22:02.874>eval <00:22:02.876>section <00:22:02.877>that <00:22:02.879>will 00:22:02.880 --> 00:22:05.230 align:start position:0% <00:22:02.880>commit <00:22:02.996>with <00:22:03.113>the <00:22:03.230>eval <00:22:03.346>section <00:22:03.463>that <00:22:03.580>will <00:22:03.696>trigger <00:22:03.813>this <00:22:03.930>agent <00:22:04.046>run <00:22:04.163>with <00:22:04.280>this 00:22:05.230 --> 00:22:05.240 align:start position:0% <00:22:05.230>trigger <00:22:05.231>this <00:22:05.233>agent <00:22:05.235>run <00:22:05.237>with <00:22:05.239>this 00:22:05.240 --> 00:22:07.390 align:start position:0% <00:22:05.240>trigger <00:22:05.356>this <00:22:05.473>agent <00:22:05.590>run <00:22:05.706>with <00:22:05.823>this <00:22:05.940>workflow. <00:22:06.056>And <00:22:06.173>then <00:22:06.290>you <00:22:06.406>get <00:22:06.523>directly <00:22:06.640>in 00:22:07.390 --> 00:22:07.400 align:start position:0% <00:22:07.390>workflow. <00:22:07.391>And <00:22:07.393>then <00:22:07.394>you <00:22:07.396>get <00:22:07.397>directly <00:22:07.399>in 00:22:07.400 --> 00:22:10.990 align:start position:0% <00:22:07.400>workflow. <00:22:07.487>And <00:22:07.575>then <00:22:07.662>you <00:22:07.750>get <00:22:07.837>directly <00:22:07.925>in <00:22:08.012>CI <00:22:08.100>if <00:22:08.187>uh <00:22:08.275>the <00:22:08.362>test <00:22:08.450>passed <00:22:08.537>or <00:22:08.625>not, <00:22:08.712>in <00:22:08.800>this 00:22:10.990 --> 00:22:11.000 align:start position:0% <00:22:10.990>CI <00:22:10.991>if <00:22:10.992>uh <00:22:10.993>the <00:22:10.994>test <00:22:10.995>passed <00:22:10.996>or <00:22:10.997>not, <00:22:10.998>in <00:22:10.999>this 00:22:11.000 --> 00:22:14.310 align:start position:0% <00:22:11.000>CI <00:22:11.077>if <00:22:11.155>uh <00:22:11.233>the <00:22:11.311>test <00:22:11.388>passed <00:22:11.466>or <00:22:11.544>not, <00:22:11.622>in <00:22:11.700>this <00:22:11.777>case <00:22:11.855>not. <00:22:11.933>And <00:22:12.011>then <00:22:12.088>you <00:22:12.166>get <00:22:12.244>also <00:22:12.322>a <00:22:12.400>report 00:22:14.310 --> 00:22:14.320 align:start position:0% <00:22:14.310>case <00:22:14.311>not. <00:22:14.312>And <00:22:14.313>then <00:22:14.314>you <00:22:14.315>get <00:22:14.316>also <00:22:14.317>a <00:22:14.319>report 00:22:14.320 --> 00:22:16.350 align:start position:0% <00:22:14.320>case <00:22:14.436>not. <00:22:14.553>And <00:22:14.669>then <00:22:14.786>you <00:22:14.903>get <00:22:15.020>also <00:22:15.136>a <00:22:15.253>report <00:22:15.370>in <00:22:15.486>the <00:22:15.603>Git <00:22:15.720>commit 00:22:16.350 --> 00:22:16.360 align:start position:0% <00:22:16.350>in <00:22:16.352>the <00:22:16.356>Git <00:22:16.359>commit 00:22:16.360 --> 00:22:18.230 align:start position:0% <00:22:16.360>in <00:22:16.500>the <00:22:16.639>Git <00:22:16.780>commit <00:22:16.919>with <00:22:17.060>the <00:22:17.199>some <00:22:17.340>comments <00:22:17.479>uh <00:22:17.620>from <00:22:17.760>the 00:22:18.230 --> 00:22:18.240 align:start position:0% <00:22:18.230>with <00:22:18.231>the <00:22:18.233>some <00:22:18.234>comments <00:22:18.236>uh <00:22:18.237>from <00:22:18.239>the 00:22:18.240 --> 00:22:20.950 align:start position:0% <00:22:18.240>with <00:22:18.356>the <00:22:18.473>some <00:22:18.590>comments <00:22:18.706>uh <00:22:18.823>from <00:22:18.940>the <00:22:19.056>workflow <00:22:19.173>on <00:22:19.290>which <00:22:19.406>metric <00:22:19.523>passed, <00:22:19.640>which 00:22:20.950 --> 00:22:20.960 align:start position:0% <00:22:20.950>workflow <00:22:20.951>on <00:22:20.953>which <00:22:20.955>metric <00:22:20.957>passed, <00:22:20.959>which 00:22:20.960 --> 00:22:24.030 align:start position:0% <00:22:20.960>workflow <00:22:21.067>on <00:22:21.175>which <00:22:21.283>metric <00:22:21.390>passed, <00:22:21.498>which <00:22:21.606>not, <00:22:21.713>what's <00:22:21.821>the <00:22:21.929>average <00:22:22.036>pass <00:22:22.144>rate, <00:22:22.252>to <00:22:22.360>so 00:22:24.030 --> 00:22:24.040 align:start position:0% <00:22:24.030>not, <00:22:24.031>what's <00:22:24.032>the <00:22:24.033>average <00:22:24.035>pass <00:22:24.036>rate, <00:22:24.037>to <00:22:24.039>so 00:22:24.040 --> 00:22:26.310 align:start position:0% <00:22:24.040>not, <00:22:24.139>what's <00:22:24.240>the <00:22:24.340>average <00:22:24.440>pass <00:22:24.540>rate, <00:22:24.639>to <00:22:24.740>so <00:22:24.840>that <00:22:24.940>you <00:22:25.040>know <00:22:25.139>what <00:22:25.240>happened <00:22:25.340>where, <00:22:25.440>and 00:22:26.310 --> 00:22:26.320 align:start position:0% <00:22:26.310>that <00:22:26.311>you <00:22:26.312>know <00:22:26.314>what <00:22:26.316>happened <00:22:26.317>where, <00:22:26.319>and 00:22:26.320 --> 00:22:29.870 align:start position:0% <00:22:26.320>that <00:22:26.447>you <00:22:26.574>know <00:22:26.701>what <00:22:26.829>happened <00:22:26.956>where, <00:22:27.083>and <00:22:27.210>where <00:22:27.338>you <00:22:27.465>should <00:22:27.592>look <00:22:27.720>for. 00:22:29.880 --> 00:22:32.270 align:start position:0% <00:22:29.880>And <00:22:30.055>we <00:22:30.230>also <00:22:30.405>use <00:22:30.580>the <00:22:30.755>pass <00:22:30.930>at <00:22:31.105>K <00:22:31.280>approach. 00:22:32.270 --> 00:22:32.280 align:start position:0% <00:22:32.270>And <00:22:32.271>we <00:22:32.272>also <00:22:32.273>use <00:22:32.274>the <00:22:32.275>pass <00:22:32.276>at <00:22:32.277>K <00:22:32.279>approach. 00:22:32.280 --> 00:22:35.030 align:start position:0% <00:22:32.280>And <00:22:32.373>we <00:22:32.466>also <00:22:32.560>use <00:22:32.653>the <00:22:32.746>pass <00:22:32.840>at <00:22:32.933>K <00:22:33.026>approach. <00:22:33.120>We <00:22:33.213>have <00:22:33.306>just <00:22:33.399>deterministic <00:22:33.493>evals <00:22:33.586>in <00:22:33.680>our 00:22:35.030 --> 00:22:35.040 align:start position:0% <00:22:35.030>We <00:22:35.031>have <00:22:35.033>just <00:22:35.034>deterministic <00:22:35.036>evals <00:22:35.037>in <00:22:35.039>our 00:22:35.040 --> 00:22:37.670 align:start position:0% <00:22:35.040>We <00:22:35.156>have <00:22:35.273>just <00:22:35.389>deterministic <00:22:35.506>evals <00:22:35.623>in <00:22:35.740>our <00:22:35.856>case. <00:22:35.973>The <00:22:36.090>problem <00:22:36.206>is, <00:22:36.323>almost <00:22:36.440>everyone 00:22:37.670 --> 00:22:37.680 align:start position:0% <00:22:37.670>case. <00:22:37.671>The <00:22:37.673>problem <00:22:37.675>is, <00:22:37.677>almost <00:22:37.679>everyone 00:22:37.680 --> 00:22:40.390 align:start position:0% <00:22:37.680>case. <00:22:37.807>The <00:22:37.934>problem <00:22:38.061>is, <00:22:38.189>almost <00:22:38.316>everyone <00:22:38.443>knows, <00:22:38.570>LLMs <00:22:38.698>are <00:22:38.825>not <00:22:38.952>deterministic. <00:22:39.080>So, 00:22:40.390 --> 00:22:40.400 align:start position:0% <00:22:40.390>knows, <00:22:40.391>LLMs <00:22:40.393>are <00:22:40.395>not <00:22:40.397>deterministic. <00:22:40.399>So, 00:22:40.400 --> 00:22:42.470 align:start position:0% <00:22:40.400>knows, <00:22:40.507>LLMs <00:22:40.615>are <00:22:40.723>not <00:22:40.830>deterministic. <00:22:40.938>So, <00:22:41.046>sometimes <00:22:41.153>you <00:22:41.261>can <00:22:41.369>run <00:22:41.476>the <00:22:41.584>same <00:22:41.692>test <00:22:41.800>and 00:22:42.470 --> 00:22:42.480 align:start position:0% <00:22:42.470>sometimes <00:22:42.471>you <00:22:42.472>can <00:22:42.473>run <00:22:42.475>the <00:22:42.476>same <00:22:42.477>test <00:22:42.479>and 00:22:42.480 --> 00:22:44.190 align:start position:0% <00:22:42.480>sometimes <00:22:42.620>you <00:22:42.760>can <00:22:42.900>run <00:22:43.040>the <00:22:43.180>same <00:22:43.320>test <00:22:43.460>and <00:22:43.600>get <00:22:43.740>different <00:22:43.880>results. 00:22:44.190 --> 00:22:44.200 align:start position:0% <00:22:44.190>get <00:22:44.194>different <00:22:44.199>results. 00:22:44.200 --> 00:22:46.510 align:start position:0% <00:22:44.200>get <00:22:44.316>different <00:22:44.433>results. <00:22:44.550>And <00:22:44.666>that's <00:22:44.783>why <00:22:44.900>we <00:22:45.016>use <00:22:45.133>pass <00:22:45.250>at <00:22:45.366>K, <00:22:45.483>so <00:22:45.600>if 00:22:46.510 --> 00:22:46.520 align:start position:0% <00:22:46.510>And <00:22:46.511>that's <00:22:46.512>why <00:22:46.513>we <00:22:46.514>use <00:22:46.514>pass <00:22:46.516>at <00:22:46.517>K, <00:22:46.518>so <00:22:46.519>if 00:22:46.520 --> 00:22:49.470 align:start position:0% <00:22:46.520>And <00:22:46.597>that's <00:22:46.675>why <00:22:46.753>we <00:22:46.831>use <00:22:46.908>pass <00:22:46.986>at <00:22:47.064>K, <00:22:47.142>so <00:22:47.220>if <00:22:47.297>any <00:22:47.375>of <00:22:47.453>those <00:22:47.531>K <00:22:47.608>run <00:22:47.686>pass, <00:22:47.764>we <00:22:47.842>consider <00:22:47.920>the 00:22:49.470 --> 00:22:49.480 align:start position:0% <00:22:49.470>any <00:22:49.471>of <00:22:49.472>those <00:22:49.473>K <00:22:49.474>run <00:22:49.475>pass, <00:22:49.476>we <00:22:49.477>consider <00:22:49.479>the 00:22:49.480 --> 00:22:51.470 align:start position:0% <00:22:49.480>any <00:22:49.573>of <00:22:49.666>those <00:22:49.760>K <00:22:49.853>run <00:22:49.946>pass, <00:22:50.040>we <00:22:50.133>consider <00:22:50.226>the <00:22:50.320>test <00:22:50.413>as <00:22:50.506>passed. <00:22:50.600>In <00:22:50.693>this <00:22:50.786>case, <00:22:50.880>for 00:22:51.470 --> 00:22:51.480 align:start position:0% <00:22:51.470>test <00:22:51.471>as <00:22:51.473>passed. <00:22:51.474>In <00:22:51.476>this <00:22:51.477>case, <00:22:51.479>for 00:22:51.480 --> 00:22:52.950 align:start position:0% <00:22:51.480>test <00:22:51.581>as <00:22:51.683>passed. <00:22:51.785>In <00:22:51.887>this <00:22:51.988>case, <00:22:52.090>for <00:22:52.192>example, <00:22:52.294>you <00:22:52.395>run <00:22:52.497>the <00:22:52.599>first <00:22:52.701>time, <00:22:52.803>it 00:22:52.950 --> 00:22:52.960 align:start position:0% <00:22:52.950>example, <00:22:52.951>you <00:22:52.953>run <00:22:52.954>the <00:22:52.956>first <00:22:52.957>time, <00:22:52.959>it 00:22:52.960 --> 00:22:54.670 align:start position:0% <00:22:52.960>example, <00:22:53.087>you <00:22:53.214>run <00:22:53.341>the <00:22:53.469>first <00:22:53.596>time, <00:22:53.723>it <00:22:53.850>failed, <00:22:53.978>then <00:22:54.105>automatically <00:22:54.232>run <00:22:54.360>the 00:22:54.670 --> 00:22:54.680 align:start position:0% <00:22:54.670>failed, <00:22:54.672>then <00:22:54.674>automatically <00:22:54.676>run <00:22:54.679>the 00:22:54.680 --> 00:22:56.390 align:start position:0% <00:22:54.680>failed, <00:22:54.807>then <00:22:54.934>automatically <00:22:55.061>run <00:22:55.189>the <00:22:55.316>second <00:22:55.443>time, <00:22:55.570>if <00:22:55.698>it <00:22:55.825>passed, <00:22:55.952>we <00:22:56.080>consider 00:22:56.390 --> 00:22:56.400 align:start position:0% <00:22:56.390>second <00:22:56.391>time, <00:22:56.393>if <00:22:56.394>it <00:22:56.396>passed, <00:22:56.397>we <00:22:56.399>consider 00:22:56.400 --> 00:22:58.630 align:start position:0% <00:22:56.400>second <00:22:56.540>time, <00:22:56.680>if <00:22:56.820>it <00:22:56.960>passed, <00:22:57.100>we <00:22:57.240>consider <00:22:57.380>the <00:22:57.520>test <00:22:57.660>as <00:22:57.800>good. 00:22:58.630 --> 00:22:58.640 align:start position:0% <00:22:58.630>the <00:22:58.633>test <00:22:58.636>as <00:22:58.639>good. 00:22:58.640 --> 00:23:01.070 align:start position:0% <00:22:58.640>the <00:22:58.767>test <00:22:58.894>as <00:22:59.021>good. <00:22:59.149>Uh <00:22:59.276>and <00:22:59.403>yeah, <00:22:59.530>that's <00:22:59.658>for <00:22:59.785>us <00:22:59.912>is <00:23:00.040>good 00:23:01.070 --> 00:23:01.080 align:start position:0% <00:23:01.070>Uh <00:23:01.071>and <00:23:01.072>yeah, <00:23:01.073>that's <00:23:01.075>for <00:23:01.076>us <00:23:01.077>is <00:23:01.079>good 00:23:01.080 --> 00:23:02.670 align:start position:0% <00:23:01.080>Uh <00:23:01.196>and <00:23:01.313>yeah, <00:23:01.429>that's <00:23:01.546>for <00:23:01.663>us <00:23:01.780>is <00:23:01.896>good <00:23:02.013>enough, <00:23:02.129>because <00:23:02.246>we <00:23:02.363>have <00:23:02.480>deterministic 00:23:02.670 --> 00:23:02.680 align:start position:0% <00:23:02.670>enough, <00:23:02.672>because <00:23:02.674>we <00:23:02.676>have <00:23:02.679>deterministic 00:23:02.680 --> 00:23:05.030 align:start position:0% <00:23:02.680>enough, <00:23:02.796>because <00:23:02.913>we <00:23:03.030>have <00:23:03.146>deterministic <00:23:03.263>one, <00:23:03.380>so <00:23:03.496>usually <00:23:03.613>we <00:23:03.730>don't <00:23:03.846>see <00:23:03.963>that <00:23:04.080>much 00:23:05.030 --> 00:23:05.040 align:start position:0% <00:23:05.030>one, <00:23:05.031>so <00:23:05.032>usually <00:23:05.033>we <00:23:05.035>don't <00:23:05.036>see <00:23:05.037>that <00:23:05.039>much 00:23:05.040 --> 00:23:07.070 align:start position:0% <00:23:05.040>one, <00:23:05.133>so <00:23:05.226>usually <00:23:05.320>we <00:23:05.413>don't <00:23:05.506>see <00:23:05.600>that <00:23:05.693>much <00:23:05.786>variance. <00:23:05.879>This <00:23:05.973>helps <00:23:06.066>us <00:23:06.159>also <00:23:06.253>a <00:23:06.346>lot <00:23:06.440>to 00:23:07.070 --> 00:23:07.080 align:start position:0% <00:23:07.070>variance. <00:23:07.071>This <00:23:07.072>helps <00:23:07.073>us <00:23:07.075>also <00:23:07.076>a <00:23:07.077>lot <00:23:07.079>to 00:23:07.080 --> 00:23:11.590 align:start position:0% <00:23:07.080>variance. <00:23:07.179>This <00:23:07.280>helps <00:23:07.379>us <00:23:07.480>also <00:23:07.580>a <00:23:07.679>lot <00:23:07.780>to <00:23:07.879>have <00:23:07.980>this <00:23:08.080>kind <00:23:08.179>of <00:23:08.280>low <00:23:08.379>K <00:23:08.480>number. 00:23:11.600 --> 00:23:14.270 align:start position:0% <00:23:11.600>And <00:23:11.800>why <00:23:12.000>does <00:23:12.199>this <00:23:12.399>matter? <00:23:12.600>In <00:23:12.800>our <00:23:13.000>case, 00:23:14.270 --> 00:23:14.280 align:start position:0% <00:23:14.270>And <00:23:14.271>why <00:23:14.272>does <00:23:14.273>this <00:23:14.275>matter? <00:23:14.276>In <00:23:14.277>our <00:23:14.279>case, 00:23:14.280 --> 00:23:18.750 align:start position:0% <00:23:14.280>And <00:23:14.387>why <00:23:14.495>does <00:23:14.603>this <00:23:14.710>matter? <00:23:14.818>In <00:23:14.926>our <00:23:15.033>case, <00:23:15.141>we <00:23:15.249>have <00:23:15.356>816 <00:23:15.464>handcrafted <00:23:15.572>test <00:23:15.680>cases. 00:23:18.750 --> 00:23:18.760 align:start position:0% <00:23:18.750>we <00:23:18.751>have <00:23:18.753>816 <00:23:18.755>handcrafted <00:23:18.757>test <00:23:18.759>cases. 00:23:18.760 --> 00:23:21.630 align:start position:0% <00:23:18.760>we <00:23:18.900>have <00:23:19.040>816 <00:23:19.180>handcrafted <00:23:19.320>test <00:23:19.460>cases. <00:23:19.600>Uh <00:23:19.740>these <00:23:19.879>were <00:23:20.020>able <00:23:20.160>to 00:23:21.630 --> 00:23:21.640 align:start position:0% <00:23:21.630>Uh <00:23:21.632>these <00:23:21.634>were <00:23:21.636>able <00:23:21.639>to 00:23:21.640 --> 00:23:24.910 align:start position:0% <00:23:21.640>Uh <00:23:21.767>these <00:23:21.894>were <00:23:22.021>able <00:23:22.149>to <00:23:22.276>uh <00:23:22.403>sustain <00:23:22.530>more <00:23:22.658>than <00:23:22.785>150 <00:23:22.912>prompt <00:23:23.040>changes 00:23:24.910 --> 00:23:24.920 align:start position:0% <00:23:24.910>uh <00:23:24.911>sustain <00:23:24.913>more <00:23:24.914>than <00:23:24.916>150 <00:23:24.917>prompt <00:23:24.919>changes 00:23:24.920 --> 00:23:26.870 align:start position:0% <00:23:24.920>uh <00:23:25.060>sustain <00:23:25.200>more <00:23:25.340>than <00:23:25.480>150 <00:23:25.620>prompt <00:23:25.760>changes <00:23:25.900>over <00:23:26.040>the <00:23:26.180>development <00:23:26.320>part. 00:23:26.870 --> 00:23:26.880 align:start position:0% <00:23:26.870>over <00:23:26.873>the <00:23:26.876>development <00:23:26.879>part. 00:23:26.880 --> 00:23:29.030 align:start position:0% <00:23:26.880>over <00:23:27.007>the <00:23:27.134>development <00:23:27.261>part. <00:23:27.389>And <00:23:27.516>we <00:23:27.643>were <00:23:27.770>able <00:23:27.898>to <00:23:28.025>get <00:23:28.152>zero <00:23:28.280>regression 00:23:29.030 --> 00:23:29.040 align:start position:0% <00:23:29.030>And <00:23:29.031>we <00:23:29.032>were <00:23:29.033>able <00:23:29.035>to <00:23:29.036>get <00:23:29.037>zero <00:23:29.039>regression 00:23:29.040 --> 00:23:33.990 align:start position:0% <00:23:29.040>And <00:23:29.156>we <00:23:29.273>were <00:23:29.389>able <00:23:29.506>to <00:23:29.623>get <00:23:29.740>zero <00:23:29.856>regression <00:23:29.973>incident <00:23:30.090>caused <00:23:30.206>by <00:23:30.323>code <00:23:30.440>changes. 00:23:34.000 --> 00:23:36.150 align:start position:0% <00:23:34.000>Let's <00:23:34.200>show <00:23:34.400>an <00:23:34.600>example <00:23:34.800>of <00:23:35.000>a <00:23:35.200>problem <00:23:35.400>that 00:23:36.150 --> 00:23:36.160 align:start position:0% <00:23:36.150>Let's <00:23:36.151>show <00:23:36.152>an <00:23:36.153>example <00:23:36.155>of <00:23:36.156>a <00:23:36.157>problem <00:23:36.159>that 00:23:36.160 --> 00:23:37.670 align:start position:0% <00:23:36.160>Let's <00:23:36.264>show <00:23:36.369>an <00:23:36.473>example <00:23:36.578>of <00:23:36.682>a <00:23:36.787>problem <00:23:36.891>that <00:23:36.996>you <00:23:37.100>can <00:23:37.205>get <00:23:37.309>while <00:23:37.414>you're <00:23:37.519>developing 00:23:37.670 --> 00:23:37.680 align:start position:0% <00:23:37.670>you <00:23:37.671>can <00:23:37.673>get <00:23:37.675>while <00:23:37.677>you're <00:23:37.679>developing 00:23:37.680 --> 00:23:40.190 align:start position:0% <00:23:37.680>you <00:23:37.787>can <00:23:37.895>get <00:23:38.003>while <00:23:38.110>you're <00:23:38.218>developing <00:23:38.326>this. <00:23:38.433>We <00:23:38.541>have <00:23:38.649>this <00:23:38.756>uh <00:23:38.864>input <00:23:38.972>of <00:23:39.080>the 00:23:40.190 --> 00:23:40.200 align:start position:0% <00:23:40.190>this. <00:23:40.191>We <00:23:40.192>have <00:23:40.193>this <00:23:40.195>uh <00:23:40.196>input <00:23:40.197>of <00:23:40.199>the 00:23:40.200 --> 00:23:42.830 align:start position:0% <00:23:40.200>this. <00:23:40.293>We <00:23:40.386>have <00:23:40.480>this <00:23:40.573>uh <00:23:40.666>input <00:23:40.760>of <00:23:40.853>the <00:23:40.946>query, <00:23:41.040>"Show <00:23:41.133>me <00:23:41.226>tours <00:23:41.320>in <00:23:41.413>Rome." <00:23:41.506>Uh <00:23:41.600>in 00:23:42.830 --> 00:23:42.840 align:start position:0% <00:23:42.830>query, <00:23:42.831>"Show <00:23:42.832>me <00:23:42.833>tours <00:23:42.835>in <00:23:42.836>Rome." <00:23:42.837>Uh <00:23:42.839>in 00:23:42.840 --> 00:23:44.950 align:start position:0% <00:23:42.840>query, <00:23:42.933>"Show <00:23:43.026>me <00:23:43.120>tours <00:23:43.213>in <00:23:43.306>Rome." <00:23:43.399>Uh <00:23:43.493>in <00:23:43.586>our <00:23:43.679>case, <00:23:43.773>we <00:23:43.866>expect <00:23:43.959>that <00:23:44.053>the <00:23:44.146>date <00:23:44.240>from 00:23:44.950 --> 00:23:44.960 align:start position:0% <00:23:44.950>our <00:23:44.951>case, <00:23:44.952>we <00:23:44.953>expect <00:23:44.955>that <00:23:44.956>the <00:23:44.957>date <00:23:44.959>from 00:23:44.960 --> 00:23:47.830 align:start position:0% <00:23:44.960>our <00:23:45.047>case, <00:23:45.135>we <00:23:45.222>expect <00:23:45.310>that <00:23:45.397>the <00:23:45.485>date <00:23:45.572>from <00:23:45.660>and <00:23:45.747>to <00:23:45.835>is <00:23:45.922>null. <00:23:46.010>But <00:23:46.097>in <00:23:46.185>the <00:23:46.272>context <00:23:46.360>of 00:23:47.830 --> 00:23:47.840 align:start position:0% <00:23:47.830>and <00:23:47.831>to <00:23:47.832>is <00:23:47.833>null. <00:23:47.834>But <00:23:47.835>in <00:23:47.836>the <00:23:47.837>context <00:23:47.839>of 00:23:47.840 --> 00:23:49.510 align:start position:0% <00:23:47.840>and <00:23:47.933>to <00:23:48.026>is <00:23:48.120>null. <00:23:48.213>But <00:23:48.306>in <00:23:48.399>the <00:23:48.493>context <00:23:48.586>of <00:23:48.679>the <00:23:48.773>query, <00:23:48.866>we <00:23:48.959>also <00:23:49.053>have <00:23:49.146>some <00:23:49.240>additional 00:23:49.510 --> 00:23:49.520 align:start position:0% <00:23:49.510>the <00:23:49.511>query, <00:23:49.513>we <00:23:49.514>also <00:23:49.516>have <00:23:49.517>some <00:23:49.519>additional 00:23:49.520 --> 00:23:51.350 align:start position:0% <00:23:49.520>the <00:23:49.636>query, <00:23:49.753>we <00:23:49.870>also <00:23:49.986>have <00:23:50.103>some <00:23:50.220>additional <00:23:50.336>parameter <00:23:50.453>like <00:23:50.570>the <00:23:50.686>today's <00:23:50.803>date. <00:23:50.920>Every 00:23:51.350 --> 00:23:51.360 align:start position:0% <00:23:51.350>parameter <00:23:51.351>like <00:23:51.353>the <00:23:51.355>today's <00:23:51.357>date. <00:23:51.359>Every 00:23:51.360 --> 00:23:53.470 align:start position:0% <00:23:51.360>parameter <00:23:51.476>like <00:23:51.593>the <00:23:51.709>today's <00:23:51.826>date. <00:23:51.943>Every <00:23:52.060>time <00:23:52.176>we <00:23:52.293>add <00:23:52.409>this <00:23:52.526>information <00:23:52.643>in <00:23:52.760>the 00:23:53.470 --> 00:23:53.480 align:start position:0% <00:23:53.470>time <00:23:53.471>we <00:23:53.473>add <00:23:53.474>this <00:23:53.476>information <00:23:53.477>in <00:23:53.479>the 00:23:53.480 --> 00:23:56.870 align:start position:0% <00:23:53.480>time <00:23:53.587>we <00:23:53.695>add <00:23:53.803>this <00:23:53.910>information <00:23:54.018>in <00:23:54.126>the <00:23:54.233>prompt. <00:23:54.341>So, <00:23:54.449>sometimes <00:23:54.556>the <00:23:54.664>model <00:23:54.772>was <00:23:54.880>just 00:23:56.870 --> 00:23:56.880 align:start position:0% <00:23:56.870>prompt. <00:23:56.871>So, <00:23:56.873>sometimes <00:23:56.874>the <00:23:56.876>model <00:23:56.877>was <00:23:56.879>just 00:23:56.880 --> 00:23:59.710 align:start position:0% <00:23:56.880>prompt. <00:23:56.987>So, <00:23:57.095>sometimes <00:23:57.203>the <00:23:57.310>model <00:23:57.418>was <00:23:57.526>just <00:23:57.633>filling <00:23:57.741>the <00:23:57.849>date <00:23:57.956>from <00:23:58.064>with <00:23:58.172>today's <00:23:58.280>date. 00:23:59.710 --> 00:23:59.720 align:start position:0% <00:23:59.710>filling <00:23:59.711>the <00:23:59.713>date <00:23:59.714>from <00:23:59.716>with <00:23:59.717>today's <00:23:59.719>date. 00:23:59.720 --> 00:24:02.110 align:start position:0% <00:23:59.720>filling <00:23:59.836>the <00:23:59.953>date <00:24:00.070>from <00:24:00.186>with <00:24:00.303>today's <00:24:00.420>date. <00:24:00.536>So, <00:24:00.653>those <00:24:00.770>were <00:24:00.886>actually <00:24:01.003>failing <00:24:01.120>with 00:24:02.110 --> 00:24:02.120 align:start position:0% <00:24:02.110>So, <00:24:02.111>those <00:24:02.113>were <00:24:02.115>actually <00:24:02.117>failing <00:24:02.119>with 00:24:02.120 --> 00:24:04.870 align:start position:0% <00:24:02.120>So, <00:24:02.227>those <00:24:02.335>were <00:24:02.443>actually <00:24:02.550>failing <00:24:02.658>with <00:24:02.766>these <00:24:02.873>queries. <00:24:02.981>So, <00:24:03.089>how <00:24:03.196>do <00:24:03.304>we <00:24:03.412>fix <00:24:03.520>it? 00:24:04.870 --> 00:24:04.880 align:start position:0% <00:24:04.870>these <00:24:04.871>queries. <00:24:04.872>So, <00:24:04.873>how <00:24:04.875>do <00:24:04.876>we <00:24:04.877>fix <00:24:04.879>it? 00:24:04.880 --> 00:24:07.710 align:start position:0% <00:24:04.880>these <00:24:04.980>queries. <00:24:05.080>So, <00:24:05.180>how <00:24:05.280>do <00:24:05.380>we <00:24:05.480>fix <00:24:05.580>it? <00:24:05.680>Like <00:24:05.780>the <00:24:05.880>simple <00:24:05.980>solution <00:24:06.080>is <00:24:06.180>just <00:24:06.280>writing 00:24:07.710 --> 00:24:07.720 align:start position:0% <00:24:07.710>Like <00:24:07.711>the <00:24:07.713>simple <00:24:07.714>solution <00:24:07.716>is <00:24:07.717>just <00:24:07.719>writing 00:24:07.720 --> 00:24:10.110 align:start position:0% <00:24:07.720>Like <00:24:07.827>the <00:24:07.935>simple <00:24:08.043>solution <00:24:08.150>is <00:24:08.258>just <00:24:08.366>writing <00:24:08.473>the <00:24:08.581>prompt, <00:24:08.689>"If <00:24:08.796>no <00:24:08.904>date <00:24:09.012>is <00:24:09.120>specified 00:24:10.110 --> 00:24:10.120 align:start position:0% <00:24:10.110>the <00:24:10.111>prompt, <00:24:10.112>"If <00:24:10.114>no <00:24:10.116>date <00:24:10.117>is <00:24:10.119>specified 00:24:10.120 --> 00:24:13.230 align:start position:0% <00:24:10.120>the <00:24:10.219>prompt, <00:24:10.320>"If <00:24:10.419>no <00:24:10.520>date <00:24:10.620>is <00:24:10.719>specified <00:24:10.820>from <00:24:10.919>the <00:24:11.020>query, <00:24:11.120>just <00:24:11.219>do <00:24:11.320>not <00:24:11.419>use <00:24:11.520>today's 00:24:13.230 --> 00:24:13.240 align:start position:0% <00:24:13.230>from <00:24:13.231>the <00:24:13.232>query, <00:24:13.233>just <00:24:13.235>do <00:24:13.236>not <00:24:13.237>use <00:24:13.239>today's 00:24:13.240 --> 00:24:16.030 align:start position:0% <00:24:13.240>from <00:24:13.340>the <00:24:13.440>query, <00:24:13.540>just <00:24:13.640>do <00:24:13.740>not <00:24:13.840>use <00:24:13.940>today's <00:24:14.040>date <00:24:14.140>as <00:24:14.240>the <00:24:14.340>default <00:24:14.440>one." <00:24:14.540>And <00:24:14.640>pretty 00:24:16.030 --> 00:24:16.040 align:start position:0% <00:24:16.030>date <00:24:16.031>as <00:24:16.033>the <00:24:16.034>default <00:24:16.036>one." <00:24:16.037>And <00:24:16.039>pretty 00:24:16.040 --> 00:24:17.710 align:start position:0% <00:24:16.040>date <00:24:16.139>as <00:24:16.240>the <00:24:16.340>default <00:24:16.440>one." <00:24:16.540>And <00:24:16.639>pretty <00:24:16.740>simple, <00:24:16.840>you <00:24:16.940>just <00:24:17.040>add <00:24:17.139>this <00:24:17.240>in <00:24:17.340>the <00:24:17.440>prompt, 00:24:17.710 --> 00:24:17.720 align:start position:0% <00:24:17.710>simple, <00:24:17.711>you <00:24:17.712>just <00:24:17.713>add <00:24:17.715>this <00:24:17.716>in <00:24:17.717>the <00:24:17.719>prompt, 00:24:17.720 --> 00:24:19.950 align:start position:0% <00:24:17.720>simple, <00:24:17.827>you <00:24:17.935>just <00:24:18.043>add <00:24:18.150>this <00:24:18.258>in <00:24:18.366>the <00:24:18.473>prompt, <00:24:18.581>it <00:24:18.689>should <00:24:18.796>cover <00:24:18.904>all <00:24:19.012>those <00:24:19.120>cases. 00:24:19.950 --> 00:24:19.960 align:start position:0% <00:24:19.950>it <00:24:19.951>should <00:24:19.953>cover <00:24:19.955>all <00:24:19.957>those <00:24:19.959>cases. 00:24:19.960 --> 00:24:22.150 align:start position:0% <00:24:19.960>it <00:24:20.076>should <00:24:20.193>cover <00:24:20.310>all <00:24:20.426>those <00:24:20.543>cases. <00:24:20.660>And <00:24:20.776>it <00:24:20.893>actually <00:24:21.010>fixed <00:24:21.126>that. <00:24:21.243>Like <00:24:21.360>all 00:24:22.150 --> 00:24:22.160 align:start position:0% <00:24:22.150>And <00:24:22.151>it <00:24:22.153>actually <00:24:22.154>fixed <00:24:22.156>that. <00:24:22.157>Like <00:24:22.159>all 00:24:22.160 --> 00:24:24.830 align:start position:0% <00:24:22.160>And <00:24:22.260>it <00:24:22.360>actually <00:24:22.460>fixed <00:24:22.560>that. <00:24:22.660>Like <00:24:22.760>all <00:24:22.860>those <00:24:22.960>tests <00:24:23.060>started <00:24:23.160>to <00:24:23.260>work. <00:24:23.360>There <00:24:23.460>is <00:24:23.560>no 00:24:24.830 --> 00:24:24.840 align:start position:0% <00:24:24.830>those <00:24:24.831>tests <00:24:24.832>started <00:24:24.833>to <00:24:24.835>work. <00:24:24.836>There <00:24:24.837>is <00:24:24.839>no 00:24:24.840 --> 00:24:27.150 align:start position:0% <00:24:24.840>those <00:24:24.939>tests <00:24:25.040>started <00:24:25.139>to <00:24:25.240>work. <00:24:25.340>There <00:24:25.439>is <00:24:25.540>no <00:24:25.639>date <00:24:25.740>inferred. <00:24:25.840>The <00:24:25.939>problem <00:24:26.040>was <00:24:26.139>when <00:24:26.240>we 00:24:27.150 --> 00:24:27.160 align:start position:0% <00:24:27.150>date <00:24:27.151>inferred. <00:24:27.153>The <00:24:27.154>problem <00:24:27.156>was <00:24:27.157>when <00:24:27.159>we 00:24:27.160 --> 00:24:29.510 align:start position:0% <00:24:27.160>date <00:24:27.276>inferred. <00:24:27.393>The <00:24:27.510>problem <00:24:27.626>was <00:24:27.743>when <00:24:27.860>we <00:24:27.976>started <00:24:28.093>to <00:24:28.210>trigger <00:24:28.326>the <00:24:28.443>CI <00:24:28.560>test 00:24:29.510 --> 00:24:29.520 align:start position:0% <00:24:29.510>started <00:24:29.511>to <00:24:29.513>trigger <00:24:29.515>the <00:24:29.517>CI <00:24:29.519>test 00:24:29.520 --> 00:24:32.270 align:start position:0% <00:24:29.520>started <00:24:29.627>to <00:24:29.735>trigger <00:24:29.843>the <00:24:29.950>CI <00:24:30.058>test <00:24:30.166>is <00:24:30.273>where <00:24:30.381>we <00:24:30.489>actually <00:24:30.596>saw <00:24:30.704>that <00:24:30.812>yes, <00:24:30.920>those 00:24:32.270 --> 00:24:32.280 align:start position:0% <00:24:32.270>is <00:24:32.271>where <00:24:32.272>we <00:24:32.273>actually <00:24:32.275>saw <00:24:32.276>that <00:24:32.277>yes, <00:24:32.279>those 00:24:32.280 --> 00:24:34.710 align:start position:0% <00:24:32.280>is <00:24:32.379>where <00:24:32.480>we <00:24:32.580>actually <00:24:32.680>saw <00:24:32.780>that <00:24:32.879>yes, <00:24:32.980>those <00:24:33.080>first <00:24:33.180>example <00:24:33.280>that <00:24:33.379>we <00:24:33.480>had <00:24:33.580>were <00:24:33.680>passing, 00:24:34.710 --> 00:24:34.720 align:start position:0% <00:24:34.710>first <00:24:34.711>example <00:24:34.713>that <00:24:34.714>we <00:24:34.716>had <00:24:34.717>were <00:24:34.719>passing, 00:24:34.720 --> 00:24:36.630 align:start position:0% <00:24:34.720>first <00:24:34.827>example <00:24:34.935>that <00:24:35.043>we <00:24:35.150>had <00:24:35.258>were <00:24:35.366>passing, <00:24:35.473>but <00:24:35.581>many <00:24:35.689>other <00:24:35.796>ones <00:24:35.904>were <00:24:36.012>not <00:24:36.120>passing 00:24:36.630 --> 00:24:36.640 align:start position:0% <00:24:36.630>but <00:24:36.631>many <00:24:36.633>other <00:24:36.634>ones <00:24:36.636>were <00:24:36.637>not <00:24:36.639>passing 00:24:36.640 --> 00:24:39.390 align:start position:0% <00:24:36.640>but <00:24:36.747>many <00:24:36.855>other <00:24:36.963>ones <00:24:37.070>were <00:24:37.178>not <00:24:37.286>passing <00:24:37.393>anymore. <00:24:37.501>All <00:24:37.609>the <00:24:37.716>ones <00:24:37.824>that <00:24:37.932>had <00:24:38.040>dates 00:24:39.390 --> 00:24:39.400 align:start position:0% <00:24:39.390>anymore. <00:24:39.391>All <00:24:39.393>the <00:24:39.394>ones <00:24:39.396>that <00:24:39.397>had <00:24:39.399>dates 00:24:39.400 --> 00:24:41.590 align:start position:0% <00:24:39.400>anymore. <00:24:39.516>All <00:24:39.633>the <00:24:39.750>ones <00:24:39.866>that <00:24:39.983>had <00:24:40.100>dates <00:24:40.216>that <00:24:40.333>were <00:24:40.450>not <00:24:40.566>specified <00:24:40.683>with <00:24:40.800>exact 00:24:41.590 --> 00:24:41.600 align:start position:0% <00:24:41.590>that <00:24:41.591>were <00:24:41.593>not <00:24:41.595>specified <00:24:41.597>with <00:24:41.599>exact 00:24:41.600 --> 00:24:44.310 align:start position:0% <00:24:41.600>that <00:24:41.716>were <00:24:41.833>not <00:24:41.949>specified <00:24:42.066>with <00:24:42.183>exact <00:24:42.300>dates, <00:24:42.416>but <00:24:42.533>with <00:24:42.649>like <00:24:42.766>temporal <00:24:42.883>time, <00:24:43.000>like 00:24:44.310 --> 00:24:44.320 align:start position:0% <00:24:44.310>dates, <00:24:44.311>but <00:24:44.312>with <00:24:44.314>like <00:24:44.316>temporal <00:24:44.317>time, <00:24:44.319>like 00:24:44.320 --> 00:24:46.710 align:start position:0% <00:24:44.320>dates, <00:24:44.436>but <00:24:44.553>with <00:24:44.669>like <00:24:44.786>temporal <00:24:44.903>time, <00:24:45.020>like <00:24:45.136>next <00:24:45.253>weekend, <00:24:45.370>tomorrow, <00:24:45.486>they <00:24:45.603>started <00:24:45.720>to 00:24:46.710 --> 00:24:46.720 align:start position:0% <00:24:46.710>next <00:24:46.711>weekend, <00:24:46.713>tomorrow, <00:24:46.715>they <00:24:46.717>started <00:24:46.719>to 00:24:46.720 --> 00:24:48.110 align:start position:0% <00:24:46.720>next <00:24:46.928>weekend, <00:24:47.137>tomorrow, <00:24:47.345>they <00:24:47.553>started <00:24:47.762>to <00:24:47.971>break. 00:24:48.110 --> 00:24:48.120 align:start position:0% break. 00:24:48.120 --> 00:24:50.590 align:start position:0% <00:24:48.120>break. <00:24:48.294>And <00:24:48.469>this <00:24:48.645>was <00:24:48.820>not <00:24:48.995>expected. <00:24:49.169>So, <00:24:49.344>the <00:24:49.520>key 00:24:50.590 --> 00:24:50.600 align:start position:0% <00:24:50.590>And <00:24:50.591>this <00:24:50.592>was <00:24:50.593>not <00:24:50.595>expected. <00:24:50.596>So, <00:24:50.597>the <00:24:50.599>key 00:24:50.600 --> 00:24:53.230 align:start position:0% <00:24:50.600>And <00:24:50.693>this <00:24:50.786>was <00:24:50.879>not <00:24:50.973>expected. <00:24:51.066>So, <00:24:51.159>the <00:24:51.253>key <00:24:51.346>takeaway <00:24:51.439>is <00:24:51.533>if <00:24:51.626>we <00:24:51.719>would <00:24:51.813>not <00:24:51.906>have <00:24:52.000>had 00:24:53.230 --> 00:24:53.240 align:start position:0% <00:24:53.230>takeaway <00:24:53.231>is <00:24:53.232>if <00:24:53.233>we <00:24:53.235>would <00:24:53.236>not <00:24:53.237>have <00:24:53.239>had 00:24:53.240 --> 00:24:55.430 align:start position:0% <00:24:53.240>takeaway <00:24:53.340>is <00:24:53.440>if <00:24:53.540>we <00:24:53.640>would <00:24:53.740>not <00:24:53.840>have <00:24:53.940>had <00:24:54.040>this <00:24:54.140>setup, <00:24:54.240>we <00:24:54.340>would <00:24:54.440>just <00:24:54.540>have <00:24:54.640>pushed 00:24:55.430 --> 00:24:55.440 align:start position:0% <00:24:55.430>this <00:24:55.431>setup, <00:24:55.433>we <00:24:55.434>would <00:24:55.436>just <00:24:55.437>have <00:24:55.439>pushed 00:24:55.440 --> 00:24:57.790 align:start position:0% <00:24:55.440>this <00:24:55.556>setup, <00:24:55.673>we <00:24:55.790>would <00:24:55.906>just <00:24:56.023>have <00:24:56.140>pushed <00:24:56.256>something <00:24:56.373>in <00:24:56.490>production <00:24:56.606>that <00:24:56.723>would <00:24:56.840>have 00:24:57.790 --> 00:24:57.800 align:start position:0% <00:24:57.790>something <00:24:57.791>in <00:24:57.793>production <00:24:57.795>that <00:24:57.797>would <00:24:57.799>have 00:24:57.800 --> 00:24:59.990 align:start position:0% <00:24:57.800>something <00:24:57.916>in <00:24:58.033>production <00:24:58.149>that <00:24:58.266>would <00:24:58.383>have <00:24:58.500>broken <00:24:58.616>many <00:24:58.733>other <00:24:58.850>use <00:24:58.966>cases <00:24:59.083>and <00:24:59.200>would 00:24:59.990 --> 00:25:00.000 align:start position:0% <00:24:59.990>broken <00:24:59.991>many <00:24:59.993>other <00:24:59.994>use <00:24:59.996>cases <00:24:59.997>and <00:24:59.999>would 00:25:00.000 --> 00:25:01.750 align:start position:0% <00:25:00.000>broken <00:25:00.140>many <00:25:00.280>other <00:25:00.420>use <00:25:00.560>cases <00:25:00.700>and <00:25:00.840>would <00:25:00.980>not <00:25:01.120>have <00:25:01.260>found <00:25:01.400>it. 00:25:01.750 --> 00:25:01.760 align:start position:0% <00:25:01.750>not <00:25:01.753>have <00:25:01.756>found <00:25:01.759>it. 00:25:01.760 --> 00:25:04.950 align:start position:0% <00:25:01.760>not <00:25:01.887>have <00:25:02.014>found <00:25:02.141>it. <00:25:02.269>So, <00:25:02.396>that <00:25:02.523>was <00:25:02.650>the <00:25:02.778>eval <00:25:02.905>setup <00:25:03.032>that <00:25:03.160>made 00:25:04.950 --> 00:25:04.960 align:start position:0% <00:25:04.950>So, <00:25:04.951>that <00:25:04.952>was <00:25:04.953>the <00:25:04.955>eval <00:25:04.956>setup <00:25:04.957>that <00:25:04.959>made 00:25:04.960 --> 00:25:07.310 align:start position:0% <00:25:04.960>So, <00:25:05.060>that <00:25:05.160>was <00:25:05.260>the <00:25:05.360>eval <00:25:05.460>setup <00:25:05.560>that <00:25:05.660>made <00:25:05.760>kind <00:25:05.860>of <00:25:05.960>the <00:25:06.060>invisible <00:25:06.160>visible <00:25:06.260>for <00:25:06.360>our 00:25:07.310 --> 00:25:07.320 align:start position:0% <00:25:07.310>kind <00:25:07.311>of <00:25:07.312>the <00:25:07.314>invisible <00:25:07.316>visible <00:25:07.317>for <00:25:07.319>our 00:25:07.320 --> 00:25:08.950 align:start position:0% <00:25:07.320>kind <00:25:07.520>of <00:25:07.720>the <00:25:07.919>invisible <00:25:08.120>visible <00:25:08.320>for <00:25:08.520>our <00:25:08.720>team. 00:25:08.950 --> 00:25:08.960 align:start position:0% team. 00:25:08.960 --> 00:25:09.870 align:start position:0% <00:25:08.960>team. <00:25:09.779>So, 00:25:09.870 --> 00:25:09.880 align:start position:0% So, 00:25:09.880 --> 00:25:12.270 align:start position:0% <00:25:09.880>So, <00:25:10.020>what <00:25:10.160>do <00:25:10.300>we <00:25:10.440>want <00:25:10.580>you <00:25:10.720>to <00:25:10.860>to <00:25:11.000>take <00:25:11.140>home <00:25:11.280>from 00:25:12.270 --> 00:25:12.280 align:start position:0% <00:25:12.270>what <00:25:12.271>do <00:25:12.272>we <00:25:12.273>want <00:25:12.274>you <00:25:12.275>to <00:25:12.276>to <00:25:12.277>take <00:25:12.278>home <00:25:12.279>from 00:25:12.280 --> 00:25:15.310 align:start position:0% <00:25:12.280>what <00:25:12.373>do <00:25:12.466>we <00:25:12.560>want <00:25:12.653>you <00:25:12.746>to <00:25:12.840>to <00:25:12.933>take <00:25:13.026>home <00:25:13.120>from <00:25:13.213>this <00:25:13.306>like <00:25:13.399>discussion <00:25:13.493>and <00:25:13.586>key <00:25:13.680>takeaways? 00:25:15.310 --> 00:25:15.320 align:start position:0% <00:25:15.310>this <00:25:15.311>like <00:25:15.313>discussion <00:25:15.315>and <00:25:15.317>key <00:25:15.319>takeaways? 00:25:15.320 --> 00:25:16.670 align:start position:0% <00:25:15.320>this <00:25:15.493>like <00:25:15.667>discussion <00:25:15.840>and <00:25:16.014>key <00:25:16.187>takeaways? <00:25:16.361>First <00:25:16.535>one, 00:25:16.670 --> 00:25:16.680 align:start position:0% <00:25:16.670>First <00:25:16.679>one, 00:25:16.680 --> 00:25:19.270 align:start position:0% <00:25:16.680>First <00:25:16.880>one, <00:25:17.080>build <00:25:17.280>the <00:25:17.480>eval <00:25:17.680>infrastructure <00:25:17.880>as <00:25:18.080>early 00:25:19.270 --> 00:25:19.280 align:start position:0% <00:25:19.270>build <00:25:19.271>the <00:25:19.273>eval <00:25:19.275>infrastructure <00:25:19.277>as <00:25:19.279>early 00:25:19.280 --> 00:25:21.790 align:start position:0% <00:25:19.280>build <00:25:19.396>the <00:25:19.513>eval <00:25:19.629>infrastructure <00:25:19.746>as <00:25:19.863>early <00:25:19.980>as <00:25:20.096>possible. <00:25:20.213>It <00:25:20.330>pays <00:25:20.446>for <00:25:20.563>itself <00:25:20.680>the 00:25:21.790 --> 00:25:21.800 align:start position:0% <00:25:21.790>as <00:25:21.791>possible. <00:25:21.793>It <00:25:21.794>pays <00:25:21.796>for <00:25:21.797>itself <00:25:21.799>the 00:25:21.800 --> 00:25:23.030 align:start position:0% <00:25:21.800>as <00:25:21.923>possible. <00:25:22.046>It <00:25:22.168>pays <00:25:22.292>for <00:25:22.415>itself <00:25:22.538>the <00:25:22.661>first <00:25:22.783>time <00:25:22.907>it 00:25:23.030 --> 00:25:23.040 align:start position:0% <00:25:23.030>first <00:25:23.034>time <00:25:23.039>it 00:25:23.040 --> 00:25:25.390 align:start position:0% <00:25:23.040>first <00:25:23.215>time <00:25:23.389>it <00:25:23.565>caught <00:25:23.740>a <00:25:23.915>couch <00:25:24.090>regression. <00:25:24.264>The <00:25:24.440>problem 00:25:25.390 --> 00:25:25.400 align:start position:0% <00:25:25.390>caught <00:25:25.391>a <00:25:25.393>couch <00:25:25.395>regression. <00:25:25.397>The <00:25:25.399>problem 00:25:25.400 --> 00:25:27.430 align:start position:0% <00:25:25.400>caught <00:25:25.516>a <00:25:25.633>couch <00:25:25.750>regression. <00:25:25.866>The <00:25:25.983>problem <00:25:26.100>is <00:25:26.216>you <00:25:26.333>don't <00:25:26.450>know <00:25:26.566>when <00:25:26.683>this <00:25:26.800>first 00:25:27.430 --> 00:25:27.440 align:start position:0% <00:25:27.430>is <00:25:27.431>you <00:25:27.433>don't <00:25:27.434>know <00:25:27.436>when <00:25:27.437>this <00:25:27.439>first 00:25:27.440 --> 00:25:29.270 align:start position:0% <00:25:27.440>is <00:25:27.540>you <00:25:27.640>don't <00:25:27.740>know <00:25:27.840>when <00:25:27.940>this <00:25:28.040>first <00:25:28.140>regression <00:25:28.240>will <00:25:28.340>be <00:25:28.440>if <00:25:28.540>you <00:25:28.640>don't <00:25:28.740>have <00:25:28.840>the 00:25:29.270 --> 00:25:29.280 align:start position:0% <00:25:29.270>regression <00:25:29.271>will <00:25:29.272>be <00:25:29.273>if <00:25:29.275>you <00:25:29.276>don't <00:25:29.277>have <00:25:29.279>the 00:25:29.280 --> 00:25:30.670 align:start position:0% <00:25:29.280>regression <00:25:29.418>will <00:25:29.558>be <00:25:29.697>if <00:25:29.836>you <00:25:29.975>don't <00:25:30.114>have <00:25:30.253>the <00:25:30.392>eval <00:25:30.531>setup. 00:25:30.670 --> 00:25:30.680 align:start position:0% <00:25:30.670>eval <00:25:30.679>setup. 00:25:30.680 --> 00:25:33.270 align:start position:0% <00:25:30.680>eval <00:25:30.820>setup. <00:25:30.960>Then <00:25:31.100>the <00:25:31.240>second <00:25:31.380>one, <00:25:31.520>try <00:25:31.660>to <00:25:31.800>use <00:25:31.940>as <00:25:32.080>many 00:25:33.270 --> 00:25:33.280 align:start position:0% <00:25:33.270>Then <00:25:33.271>the <00:25:33.272>second <00:25:33.273>one, <00:25:33.274>try <00:25:33.275>to <00:25:33.276>use <00:25:33.277>as <00:25:33.279>many 00:25:33.280 --> 00:25:35.870 align:start position:0% <00:25:33.280>Then <00:25:33.387>the <00:25:33.495>second <00:25:33.603>one, <00:25:33.710>try <00:25:33.818>to <00:25:33.926>use <00:25:34.033>as <00:25:34.141>many <00:25:34.249>deterministic <00:25:34.356>evals <00:25:34.464>as <00:25:34.572>possible. <00:25:34.680>Yes, 00:25:35.870 --> 00:25:35.880 align:start position:0% <00:25:35.870>deterministic <00:25:35.872>evals <00:25:35.874>as <00:25:35.876>possible. <00:25:35.879>Yes, 00:25:35.880 --> 00:25:37.950 align:start position:0% <00:25:35.880>deterministic <00:25:35.987>evals <00:25:36.095>as <00:25:36.203>possible. <00:25:36.310>Yes, <00:25:36.418>LLM <00:25:36.526>as <00:25:36.633>a <00:25:36.741>judge <00:25:36.849>sometimes <00:25:36.956>you <00:25:37.064>need <00:25:37.172>to <00:25:37.280>use 00:25:37.950 --> 00:25:37.960 align:start position:0% <00:25:37.950>LLM <00:25:37.951>as <00:25:37.952>a <00:25:37.953>judge <00:25:37.954>sometimes <00:25:37.955>you <00:25:37.956>need <00:25:37.957>to <00:25:37.959>use 00:25:37.960 --> 00:25:41.310 align:start position:0% <00:25:37.960>LLM <00:25:38.042>as <00:25:38.124>a <00:25:38.207>judge <00:25:38.289>sometimes <00:25:38.371>you <00:25:38.454>need <00:25:38.536>to <00:25:38.618>use <00:25:38.701>them, <00:25:38.783>but <00:25:38.865>if <00:25:38.948>you <00:25:39.030>can, <00:25:39.112>try <00:25:39.195>to <00:25:39.277>have <00:25:39.360>as 00:25:41.310 --> 00:25:41.320 align:start position:0% <00:25:41.310>them, <00:25:41.311>but <00:25:41.312>if <00:25:41.313>you <00:25:41.314>can, <00:25:41.315>try <00:25:41.316>to <00:25:41.317>have <00:25:41.319>as 00:25:41.320 --> 00:25:43.830 align:start position:0% <00:25:41.320>them, <00:25:41.419>but <00:25:41.520>if <00:25:41.620>you <00:25:41.720>can, <00:25:41.820>try <00:25:41.919>to <00:25:42.020>have <00:25:42.120>as <00:25:42.220>many <00:25:42.320>as <00:25:42.419>deterministic <00:25:42.520>one <00:25:42.620>before <00:25:42.720>you 00:25:43.830 --> 00:25:43.840 align:start position:0% <00:25:43.830>many <00:25:43.831>as <00:25:43.833>deterministic <00:25:43.835>one <00:25:43.837>before <00:25:43.839>you 00:25:43.840 --> 00:25:46.030 align:start position:0% <00:25:43.840>many <00:25:43.939>as <00:25:44.040>deterministic <00:25:44.139>one <00:25:44.240>before <00:25:44.340>you <00:25:44.439>start <00:25:44.540>to <00:25:44.639>go <00:25:44.740>through <00:25:44.840>the <00:25:44.939>LLM <00:25:45.040>as <00:25:45.139>a <00:25:45.240>judge 00:25:46.030 --> 00:25:46.040 align:start position:0% <00:25:46.030>start <00:25:46.031>to <00:25:46.032>go <00:25:46.033>through <00:25:46.034>the <00:25:46.035>LLM <00:25:46.036>as <00:25:46.037>a <00:25:46.039>judge 00:25:46.040 --> 00:25:47.190 align:start position:0% <00:25:46.040>start <00:25:46.155>to <00:25:46.270>go <00:25:46.385>through <00:25:46.500>the <00:25:46.615>LLM <00:25:46.730>as <00:25:46.845>a <00:25:46.960>judge <00:25:47.075>ones. 00:25:47.190 --> 00:25:47.200 align:start position:0% ones. 00:25:47.200 --> 00:25:49.430 align:start position:0% <00:25:47.200>ones. <00:25:47.550>Then <00:25:47.900>the <00:25:48.250>third <00:25:48.600>one, 00:25:49.430 --> 00:25:49.440 align:start position:0% <00:25:49.430>Then <00:25:49.433>the <00:25:49.436>third <00:25:49.439>one, 00:25:49.440 --> 00:25:51.990 align:start position:0% <00:25:49.440>Then <00:25:49.567>the <00:25:49.694>third <00:25:49.821>one, <00:25:49.949>we <00:25:50.076>suggest <00:25:50.203>you <00:25:50.330>to <00:25:50.458>have <00:25:50.585>the <00:25:50.712>right <00:25:50.840>model 00:25:51.990 --> 00:25:52.000 align:start position:0% <00:25:51.990>we <00:25:51.991>suggest <00:25:51.992>you <00:25:51.993>to <00:25:51.995>have <00:25:51.996>the <00:25:51.997>right <00:25:51.999>model 00:25:52.000 --> 00:25:54.670 align:start position:0% <00:25:52.000>we <00:25:52.087>suggest <00:25:52.175>you <00:25:52.262>to <00:25:52.350>have <00:25:52.437>the <00:25:52.525>right <00:25:52.612>model <00:25:52.700>for <00:25:52.787>the <00:25:52.875>right <00:25:52.962>use <00:25:53.050>case. <00:25:53.137>Uh <00:25:53.225>yes, <00:25:53.312>you <00:25:53.400>can 00:25:54.670 --> 00:25:54.680 align:start position:0% <00:25:54.670>for <00:25:54.671>the <00:25:54.672>right <00:25:54.673>use <00:25:54.674>case. <00:25:54.675>Uh <00:25:54.676>yes, <00:25:54.677>you <00:25:54.679>can 00:25:54.680 --> 00:25:56.030 align:start position:0% <00:25:54.680>for <00:25:54.766>the <00:25:54.853>right <00:25:54.940>use <00:25:55.027>case. <00:25:55.113>Uh <00:25:55.200>yes, <00:25:55.287>you <00:25:55.374>can <00:25:55.461>start <00:25:55.547>when <00:25:55.634>you're <00:25:55.721>developing <00:25:55.808>with <00:25:55.895>the 00:25:56.030 --> 00:25:56.040 align:start position:0% <00:25:56.030>start <00:25:56.031>when <00:25:56.033>you're <00:25:56.035>developing <00:25:56.037>with <00:25:56.039>the 00:25:56.040 --> 00:25:57.830 align:start position:0% <00:25:56.040>start <00:25:56.156>when <00:25:56.273>you're <00:25:56.389>developing <00:25:56.506>with <00:25:56.623>the <00:25:56.740>most <00:25:56.856>like <00:25:56.973>kind <00:25:57.090>of <00:25:57.206>expensive <00:25:57.323>one <00:25:57.440>and 00:25:57.830 --> 00:25:57.840 align:start position:0% <00:25:57.830>most <00:25:57.831>like <00:25:57.832>kind <00:25:57.834>of <00:25:57.836>expensive <00:25:57.837>one <00:25:57.839>and 00:25:57.840 --> 00:25:59.790 align:start position:0% <00:25:57.840>most <00:25:57.956>like <00:25:58.073>kind <00:25:58.189>of <00:25:58.306>expensive <00:25:58.423>one <00:25:58.540>and <00:25:58.656>powerful <00:25:58.773>one, <00:25:58.889>so <00:25:59.006>you <00:25:59.123>can <00:25:59.240>understand 00:25:59.790 --> 00:25:59.800 align:start position:0% <00:25:59.790>powerful <00:25:59.791>one, <00:25:59.793>so <00:25:59.795>you <00:25:59.797>can <00:25:59.799>understand 00:25:59.800 --> 00:26:03.190 align:start position:0% <00:25:59.800>powerful <00:25:59.916>one, <00:26:00.033>so <00:26:00.149>you <00:26:00.266>can <00:26:00.383>understand <00:26:00.500>what's <00:26:00.616>the <00:26:00.733>limitations <00:26:00.850>of <00:26:00.966>your <00:26:01.083>uh <00:26:01.200>scope 00:26:03.190 --> 00:26:03.200 align:start position:0% <00:26:03.190>what's <00:26:03.191>the <00:26:03.193>limitations <00:26:03.194>of <00:26:03.196>your <00:26:03.197>uh <00:26:03.199>scope 00:26:03.200 --> 00:26:04.830 align:start position:0% <00:26:03.200>what's <00:26:03.300>the <00:26:03.400>limitations <00:26:03.500>of <00:26:03.600>your <00:26:03.700>uh <00:26:03.800>scope <00:26:03.900>and <00:26:04.000>your <00:26:04.100>prompt <00:26:04.200>and <00:26:04.300>what <00:26:04.400>you <00:26:04.500>have, <00:26:04.600>the 00:26:04.830 --> 00:26:04.840 align:start position:0% <00:26:04.830>and <00:26:04.831>your <00:26:04.832>prompt <00:26:04.833>and <00:26:04.835>what <00:26:04.836>you <00:26:04.837>have, <00:26:04.839>the 00:26:04.840 --> 00:26:06.950 align:start position:0% <00:26:04.840>and <00:26:04.933>your <00:26:05.026>prompt <00:26:05.120>and <00:26:05.213>what <00:26:05.306>you <00:26:05.399>have, <00:26:05.493>the <00:26:05.586>maximum <00:26:05.679>that <00:26:05.773>you <00:26:05.866>can <00:26:05.959>reach, <00:26:06.053>and <00:26:06.146>then <00:26:06.240>you 00:26:06.950 --> 00:26:06.960 align:start position:0% <00:26:06.950>maximum <00:26:06.951>that <00:26:06.952>you <00:26:06.953>can <00:26:06.955>reach, <00:26:06.956>and <00:26:06.957>then <00:26:06.959>you 00:26:06.960 --> 00:26:09.590 align:start position:0% <00:26:06.960>maximum <00:26:07.067>that <00:26:07.175>you <00:26:07.283>can <00:26:07.390>reach, <00:26:07.498>and <00:26:07.606>then <00:26:07.713>you <00:26:07.821>start <00:26:07.929>to <00:26:08.036>use <00:26:08.144>smaller <00:26:08.252>models, <00:26:08.360>continuing 00:26:09.590 --> 00:26:09.600 align:start position:0% <00:26:09.590>start <00:26:09.591>to <00:26:09.593>use <00:26:09.595>smaller <00:26:09.597>models, <00:26:09.599>continuing 00:26:09.600 --> 00:26:11.550 align:start position:0% <00:26:09.600>start <00:26:09.707>to <00:26:09.815>use <00:26:09.923>smaller <00:26:10.030>models, <00:26:10.138>continuing <00:26:10.246>using <00:26:10.353>the <00:26:10.461>other <00:26:10.569>test <00:26:10.676>example <00:26:10.784>to <00:26:10.892>see <00:26:11.000>if 00:26:11.550 --> 00:26:11.560 align:start position:0% <00:26:11.550>using <00:26:11.551>the <00:26:11.552>other <00:26:11.553>test <00:26:11.555>example <00:26:11.556>to <00:26:11.557>see <00:26:11.559>if 00:26:11.560 --> 00:26:13.390 align:start position:0% <00:26:11.560>using <00:26:11.659>the <00:26:11.760>other <00:26:11.860>test <00:26:11.960>example <00:26:12.060>to <00:26:12.159>see <00:26:12.260>if <00:26:12.360>everything <00:26:12.460>still <00:26:12.560>pass <00:26:12.659>and <00:26:12.760>if <00:26:12.860>you <00:26:12.960>can 00:26:13.390 --> 00:26:13.400 align:start position:0% <00:26:13.390>everything <00:26:13.391>still <00:26:13.393>pass <00:26:13.394>and <00:26:13.396>if <00:26:13.397>you <00:26:13.399>can 00:26:13.400 --> 00:26:16.590 align:start position:0% <00:26:13.400>everything <00:26:13.527>still <00:26:13.654>pass <00:26:13.781>and <00:26:13.909>if <00:26:14.036>you <00:26:14.163>can <00:26:14.290>reduce <00:26:14.418>like <00:26:14.545>cost <00:26:14.672>and <00:26:14.800>latency. 00:26:16.590 --> 00:26:16.600 align:start position:0% <00:26:16.590>reduce <00:26:16.592>like <00:26:16.594>cost <00:26:16.596>and <00:26:16.599>latency. 00:26:16.600 --> 00:26:18.710 align:start position:0% <00:26:16.600>reduce <00:26:16.740>like <00:26:16.879>cost <00:26:17.020>and <00:26:17.159>latency. <00:26:17.300>Then <00:26:17.439>the <00:26:17.580>four <00:26:17.719>uh <00:26:17.860>graceful <00:26:18.000>degradation. 00:26:18.710 --> 00:26:18.720 align:start position:0% <00:26:18.710>Then <00:26:18.711>the <00:26:18.713>four <00:26:18.715>uh <00:26:18.717>graceful <00:26:18.719>degradation. 00:26:18.720 --> 00:26:20.950 align:start position:0% <00:26:18.720>Then <00:26:18.827>the <00:26:18.935>four <00:26:19.043>uh <00:26:19.150>graceful <00:26:19.258>degradation. <00:26:19.366>So, <00:26:19.473>if <00:26:19.581>you <00:26:19.689>can <00:26:19.796>rewind <00:26:19.904>for <00:26:20.012>your <00:26:20.120>system 00:26:20.950 --> 00:26:20.960 align:start position:0% <00:26:20.950>So, <00:26:20.951>if <00:26:20.952>you <00:26:20.953>can <00:26:20.955>rewind <00:26:20.956>for <00:26:20.957>your <00:26:20.959>system 00:26:20.960 --> 00:26:21.830 align:start position:0% <00:26:20.960>So, <00:26:21.057>if <00:26:21.155>you <00:26:21.253>can <00:26:21.351>rewind <00:26:21.449>for <00:26:21.547>your <00:26:21.645>system <00:26:21.743>that 00:26:21.830 --> 00:26:21.840 align:start position:0% that 00:26:21.840 --> 00:26:24.430 align:start position:0% <00:26:21.840>that <00:26:21.995>you <00:26:22.151>are <00:26:22.306>able <00:26:22.462>to <00:26:22.617>have <00:26:22.773>a <00:26:22.928>good <00:26:23.084>answer <00:26:23.240>with 00:26:24.430 --> 00:26:24.440 align:start position:0% <00:26:24.430>you <00:26:24.431>are <00:26:24.432>able <00:26:24.433>to <00:26:24.434>have <00:26:24.435>a <00:26:24.436>good <00:26:24.437>answer <00:26:24.439>with 00:26:24.440 --> 00:26:26.710 align:start position:0% <00:26:24.440>you <00:26:24.540>are <00:26:24.640>able <00:26:24.740>to <00:26:24.840>have <00:26:24.940>a <00:26:25.040>good <00:26:25.140>answer <00:26:25.240>with <00:26:25.340>some <00:26:25.440>partial <00:26:25.540>results, <00:26:25.640>try <00:26:25.740>to <00:26:25.840>optimize 00:26:26.710 --> 00:26:26.720 align:start position:0% <00:26:26.710>some <00:26:26.711>partial <00:26:26.713>results, <00:26:26.715>try <00:26:26.717>to <00:26:26.719>optimize 00:26:26.720 --> 00:26:29.390 align:start position:0% <00:26:26.720>some <00:26:26.860>partial <00:26:27.000>results, <00:26:27.140>try <00:26:27.280>to <00:26:27.420>optimize <00:26:27.560>for <00:26:27.700>that <00:26:27.840>whenever <00:26:27.980>you <00:26:28.120>can. 00:26:29.390 --> 00:26:29.400 align:start position:0% <00:26:29.390>for <00:26:29.392>that <00:26:29.394>whenever <00:26:29.396>you <00:26:29.399>can. 00:26:29.400 --> 00:26:31.670 align:start position:0% <00:26:29.400>for <00:26:29.516>that <00:26:29.633>whenever <00:26:29.750>you <00:26:29.866>can. <00:26:29.983>So, <00:26:30.100>if <00:26:30.216>you <00:26:30.333>want <00:26:30.450>just <00:26:30.566>to <00:26:30.683>remember <00:26:30.800>one 00:26:31.670 --> 00:26:31.680 align:start position:0% <00:26:31.670>So, <00:26:31.671>if <00:26:31.672>you <00:26:31.673>want <00:26:31.675>just <00:26:31.676>to <00:26:31.677>remember <00:26:31.679>one 00:26:31.680 --> 00:26:33.910 align:start position:0% <00:26:31.680>So, <00:26:31.780>if <00:26:31.880>you <00:26:31.980>want <00:26:32.080>just <00:26:32.180>to <00:26:32.280>remember <00:26:32.380>one <00:26:32.480>sentence <00:26:32.580>from <00:26:32.680>this <00:26:32.780>talk, <00:26:32.880>try <00:26:32.980>to <00:26:33.080>have 00:26:33.910 --> 00:26:33.920 align:start position:0% <00:26:33.910>sentence <00:26:33.911>from <00:26:33.913>this <00:26:33.914>talk, <00:26:33.916>try <00:26:33.917>to <00:26:33.919>have 00:26:33.920 --> 00:26:37.830 align:start position:0% <00:26:33.920>sentence <00:26:34.027>from <00:26:34.135>this <00:26:34.243>talk, <00:26:34.350>try <00:26:34.458>to <00:26:34.566>have <00:26:34.673>your <00:26:34.781>LLM <00:26:34.889>system <00:26:34.996>as <00:26:35.104>a <00:26:35.212>production <00:26:35.320>code. 00:26:37.830 --> 00:26:37.840 align:start position:0% <00:26:37.830>your <00:26:37.831>LLM <00:26:37.832>system <00:26:37.834>as <00:26:37.836>a <00:26:37.837>production <00:26:37.839>code. 00:26:37.840 --> 00:26:40.150 align:start position:0% <00:26:37.840>your <00:26:37.933>LLM <00:26:38.026>system <00:26:38.120>as <00:26:38.213>a <00:26:38.306>production <00:26:38.399>code. <00:26:38.493>Think <00:26:38.586>about <00:26:38.679>it <00:26:38.773>not <00:26:38.866>as <00:26:38.959>a <00:26:39.053>demo <00:26:39.146>that <00:26:39.240>just 00:26:40.150 --> 00:26:40.160 align:start position:0% <00:26:40.150>Think <00:26:40.151>about <00:26:40.152>it <00:26:40.153>not <00:26:40.154>as <00:26:40.155>a <00:26:40.156>demo <00:26:40.157>that <00:26:40.159>just 00:26:40.160 --> 00:26:42.430 align:start position:0% <00:26:40.160>Think <00:26:40.300>about <00:26:40.440>it <00:26:40.580>not <00:26:40.720>as <00:26:40.860>a <00:26:41.000>demo <00:26:41.140>that <00:26:41.280>just <00:26:41.420>got <00:26:41.560>promoted. 00:26:42.430 --> 00:26:42.440 align:start position:0% <00:26:42.430>got <00:26:42.439>promoted. 00:26:42.440 --> 00:26:44.310 align:start position:0% <00:26:42.440>got <00:26:42.673>promoted. <00:26:42.906>And <00:26:43.140>that's <00:26:43.373>it <00:26:43.606>for <00:26:43.840>us. 00:26:44.310 --> 00:26:44.320 align:start position:0% <00:26:44.310>And <00:26:44.312>that's <00:26:44.314>it <00:26:44.316>for <00:26:44.319>us. 00:26:44.320 --> 00:26:47.378 align:start position:0% <00:26:44.320>And <00:26:44.495>that's <00:26:44.669>it <00:26:44.845>for <00:26:45.020>us. <00:26:45.195>Thank <00:26:45.370>you <00:26:45.544>very <00:26:45.720>much. 00:26:47.388 --> 00:26:49.590 align:start position:0% <00:26:47.388>>> <00:26:48.788>[applause] 00:26:49.600 --> 00:26:51.710 align:start position:0% <00:26:49.600>>> <00:26:49.800>Very <00:26:50.000>good <00:26:50.199>talk. <00:26:50.399>Thank <00:26:50.600>you, <00:26:50.800>gentlemen. <00:26:51.000>Um 00:26:51.710 --> 00:26:51.720 align:start position:0% <00:26:51.710>>> <00:26:51.711>Very <00:26:51.712>good <00:26:51.713>talk. <00:26:51.715>Thank <00:26:51.716>you, <00:26:51.717>gentlemen. <00:26:51.719>Um 00:26:51.720 --> 00:26:53.390 align:start position:0% <00:26:51.720>>> <00:26:51.813>Very <00:26:51.906>good <00:26:52.000>talk. <00:26:52.093>Thank <00:26:52.186>you, <00:26:52.280>gentlemen. <00:26:52.373>Um <00:26:52.466>we <00:26:52.560>got <00:26:52.653>a <00:26:52.746>couple <00:26:52.840>of <00:26:52.933>questions. <00:26:53.026>Um <00:26:53.120>don't 00:26:53.390 --> 00:26:53.400 align:start position:0% <00:26:53.390>we <00:26:53.391>got <00:26:53.392>a <00:26:53.393>couple <00:26:53.395>of <00:26:53.396>questions. <00:26:53.397>Um <00:26:53.399>don't 00:26:53.400 --> 00:26:54.870 align:start position:0% <00:26:53.400>we <00:26:53.482>got <00:26:53.565>a <00:26:53.648>couple <00:26:53.730>of <00:26:53.813>questions. <00:26:53.896>Um <00:26:53.978>don't <00:26:54.061>know <00:26:54.144>if <00:26:54.226>you <00:26:54.309>can <00:26:54.392>run <00:26:54.474>through <00:26:54.557>all <00:26:54.640>of <00:26:54.723>them. 00:26:54.870 --> 00:26:54.880 align:start position:0% <00:26:54.870>know <00:26:54.871>if <00:26:54.872>you <00:26:54.873>can <00:26:54.874>run <00:26:54.875>through <00:26:54.876>all <00:26:54.877>of <00:26:54.879>them. 00:26:54.880 --> 00:26:57.270 align:start position:0% <00:26:54.880>know <00:26:54.962>if <00:26:55.044>you <00:26:55.127>can <00:26:55.209>run <00:26:55.291>through <00:26:55.374>all <00:26:55.456>of <00:26:55.538>them. <00:26:55.621>So, <00:26:55.703>if <00:26:55.785>I <00:26:55.868>can't <00:26:55.950>ask <00:26:56.032>all <00:26:56.115>of <00:26:56.197>them, <00:26:56.280>make 00:26:57.270 --> 00:26:57.280 align:start position:0% <00:26:57.270>So, <00:26:57.271>if <00:26:57.272>I <00:26:57.273>can't <00:26:57.274>ask <00:26:57.275>all <00:26:57.276>of <00:26:57.277>them, <00:26:57.279>make 00:26:57.280 --> 00:26:59.350 align:start position:0% <00:26:57.280>So, <00:26:57.373>if <00:26:57.466>I <00:26:57.560>can't <00:26:57.653>ask <00:26:57.746>all <00:26:57.840>of <00:26:57.933>them, <00:26:58.026>make <00:26:58.120>sure <00:26:58.213>you <00:26:58.306>talk <00:26:58.399>to <00:26:58.493>Stephen <00:26:58.586>and <00:26:58.680>Gianpaolo 00:26:59.350 --> 00:26:59.360 align:start position:0% <00:26:59.350>sure <00:26:59.351>you <00:26:59.352>talk <00:26:59.354>to <00:26:59.356>Stephen <00:26:59.357>and <00:26:59.359>Gianpaolo 00:26:59.360 --> 00:27:01.550 align:start position:0% <00:26:59.360>sure <00:26:59.467>you <00:26:59.575>talk <00:26:59.683>to <00:26:59.790>Stephen <00:26:59.898>and <00:27:00.006>Gianpaolo <00:27:00.113>right <00:27:00.221>side <00:27:00.329>stage. <00:27:00.436>So, <00:27:00.544>maybe <00:27:00.652>let's <00:27:00.760>start 00:27:01.550 --> 00:27:01.560 align:start position:0% <00:27:01.550>right <00:27:01.551>side <00:27:01.553>stage. <00:27:01.554>So, <00:27:01.556>maybe <00:27:01.557>let's <00:27:01.559>start 00:27:01.560 --> 00:27:03.870 align:start position:0% <00:27:01.560>right <00:27:01.676>side <00:27:01.793>stage. <00:27:01.909>So, <00:27:02.026>maybe <00:27:02.143>let's <00:27:02.260>start <00:27:02.376>with <00:27:02.493>Michael <00:27:02.610>or <00:27:02.726>Michael's <00:27:02.843>question. <00:27:02.960>So, 00:27:03.870 --> 00:27:03.880 align:start position:0% <00:27:03.870>with <00:27:03.871>Michael <00:27:03.873>or <00:27:03.875>Michael's <00:27:03.877>question. <00:27:03.879>So, 00:27:03.880 --> 00:27:05.310 align:start position:0% <00:27:03.880>with <00:27:03.997>Michael <00:27:04.114>or <00:27:04.231>Michael's <00:27:04.348>question. <00:27:04.465>So, <00:27:04.582>he <00:27:04.699>asked <00:27:04.816>which <00:27:04.933>which <00:27:05.050>technologies <00:27:05.167>are 00:27:05.310 --> 00:27:05.320 align:start position:0% <00:27:05.310>he <00:27:05.311>asked <00:27:05.313>which <00:27:05.315>which <00:27:05.317>technologies <00:27:05.319>are 00:27:05.320 --> 00:27:07.830 align:start position:0% <00:27:05.320>he <00:27:05.447>asked <00:27:05.574>which <00:27:05.701>which <00:27:05.829>technologies <00:27:05.956>are <00:27:06.083>you <00:27:06.210>using <00:27:06.338>to <00:27:06.465>observe <00:27:06.592>your <00:27:06.720>agent? 00:27:07.830 --> 00:27:07.840 align:start position:0% <00:27:07.830>you <00:27:07.831>using <00:27:07.833>to <00:27:07.835>observe <00:27:07.837>your <00:27:07.839>agent? 00:27:07.840 --> 00:27:11.430 align:start position:0% <00:27:07.840>you <00:27:07.947>using <00:27:08.055>to <00:27:08.163>observe <00:27:08.270>your <00:27:08.378>agent? <00:27:08.486>>> <00:27:08.593>Yes, <00:27:08.701>we <00:27:08.809>use <00:27:08.916>BrainTrust <00:27:09.024>and <00:27:09.132>as <00:27:09.240>a 00:27:11.430 --> 00:27:11.440 align:start position:0% <00:27:11.430>>> <00:27:11.431>Yes, <00:27:11.432>we <00:27:11.433>use <00:27:11.435>BrainTrust <00:27:11.436>and <00:27:11.437>as <00:27:11.439>a 00:27:11.440 --> 00:27:13.350 align:start position:0% <00:27:11.440>>> <00:27:11.547>Yes, <00:27:11.655>we <00:27:11.763>use <00:27:11.870>BrainTrust <00:27:11.978>and <00:27:12.086>as <00:27:12.193>a <00:27:12.301>observability <00:27:12.409>layer <00:27:12.516>and <00:27:12.624>then <00:27:12.732>we <00:27:12.840>use 00:27:13.350 --> 00:27:13.360 align:start position:0% <00:27:13.350>observability <00:27:13.351>layer <00:27:13.353>and <00:27:13.355>then <00:27:13.357>we <00:27:13.359>use 00:27:13.360 --> 00:27:17.310 align:start position:0% <00:27:13.360>observability <00:27:13.515>layer <00:27:13.671>and <00:27:13.826>then <00:27:13.982>we <00:27:14.137>use <00:27:14.293>LangGraph <00:27:14.448>to <00:27:14.604>orchestrate <00:27:14.760>everything. 00:27:17.310 --> 00:27:17.320 align:start position:0% <00:27:17.310>LangGraph <00:27:17.312>to <00:27:17.316>orchestrate <00:27:17.319>everything. 00:27:17.320 --> 00:27:18.390 align:start position:0% <00:27:17.320>LangGraph <00:27:17.407>to <00:27:17.495>orchestrate <00:27:17.582>everything. <00:27:17.670>>> <00:27:17.757>Excellent. <00:27:17.845>Thank <00:27:17.932>you. <00:27:18.020>That <00:27:18.107>was <00:27:18.195>a <00:27:18.283>brief 00:27:18.390 --> 00:27:18.400 align:start position:0% <00:27:18.390>>> <00:27:18.391>Excellent. <00:27:18.392>Thank <00:27:18.393>you. <00:27:18.395>That <00:27:18.396>was <00:27:18.397>a <00:27:18.399>brief 00:27:18.400 --> 00:27:20.630 align:start position:0% <00:27:18.400>>> <00:27:18.500>Excellent. <00:27:18.600>Thank <00:27:18.700>you. <00:27:18.800>That <00:27:18.900>was <00:27:19.000>a <00:27:19.100>brief <00:27:19.200>answer, <00:27:19.300>so <00:27:19.400>we <00:27:19.500>can <00:27:19.600>do <00:27:19.700>one <00:27:19.800>more. 00:27:20.630 --> 00:27:20.640 align:start position:0% <00:27:20.630>answer, <00:27:20.631>so <00:27:20.633>we <00:27:20.634>can <00:27:20.636>do <00:27:20.637>one <00:27:20.639>more. 00:27:20.640 --> 00:27:22.750 align:start position:0% <00:27:20.640>answer, <00:27:20.740>so <00:27:20.840>we <00:27:20.940>can <00:27:21.040>do <00:27:21.140>one <00:27:21.240>more. <00:27:21.340>So, <00:27:21.440>so <00:27:21.540>we <00:27:21.640>now <00:27:21.740>would <00:27:21.840>like <00:27:21.940>to <00:27:22.040>know, 00:27:22.750 --> 00:27:22.760 align:start position:0% <00:27:22.750>So, <00:27:22.751>so <00:27:22.752>we <00:27:22.753>now <00:27:22.755>would <00:27:22.756>like <00:27:22.757>to <00:27:22.759>know, 00:27:22.760 --> 00:27:24.870 align:start position:0% <00:27:22.760>So, <00:27:22.860>so <00:27:22.960>we <00:27:23.060>now <00:27:23.160>would <00:27:23.260>like <00:27:23.360>to <00:27:23.460>know, <00:27:23.560>have <00:27:23.660>you <00:27:23.760>faced <00:27:23.860>differences <00:27:23.960>when <00:27:24.060>you <00:27:24.160>had 00:27:24.870 --> 00:27:24.880 align:start position:0% <00:27:24.870>have <00:27:24.871>you <00:27:24.873>faced <00:27:24.874>differences <00:27:24.876>when <00:27:24.877>you <00:27:24.879>had 00:27:24.880 --> 00:27:27.830 align:start position:0% <00:27:24.880>have <00:27:25.007>you <00:27:25.134>faced <00:27:25.261>differences <00:27:25.389>when <00:27:25.516>you <00:27:25.643>had <00:27:25.770>multilingual <00:27:25.898>queries <00:27:26.025>for <00:27:26.152>for <00:27:26.280>retrieval? 00:27:27.830 --> 00:27:27.840 align:start position:0% <00:27:27.830>multilingual <00:27:27.832>queries <00:27:27.834>for <00:27:27.836>for <00:27:27.839>retrieval? 00:27:27.840 --> 00:27:30.550 align:start position:0% <00:27:27.840>multilingual <00:27:27.967>queries <00:27:28.094>for <00:27:28.221>for <00:27:28.349>retrieval? <00:27:28.476>Did <00:27:28.603>that <00:27:28.730>Did <00:27:28.858>that <00:27:28.985>make <00:27:29.112>any <00:27:29.240>difference? 00:27:30.550 --> 00:27:30.560 align:start position:0% <00:27:30.550>Did <00:27:30.551>that <00:27:30.553>Did <00:27:30.554>that <00:27:30.556>make <00:27:30.557>any <00:27:30.559>difference? 00:27:30.560 --> 00:27:32.430 align:start position:0% <00:27:30.560>Did <00:27:30.667>that <00:27:30.775>Did <00:27:30.883>that <00:27:30.990>make <00:27:31.098>any <00:27:31.206>difference? <00:27:31.313>Was <00:27:31.421>Did <00:27:31.529>the <00:27:31.636>quality <00:27:31.744>differ <00:27:31.852>or <00:27:31.960>something 00:27:32.430 --> 00:27:32.440 align:start position:0% <00:27:32.430>Was <00:27:32.431>Did <00:27:32.433>the <00:27:32.434>quality <00:27:32.436>differ <00:27:32.437>or <00:27:32.439>something 00:27:32.440 --> 00:27:32.950 align:start position:0% <00:27:32.440>Was <00:27:32.497>Did <00:27:32.554>the <00:27:32.612>quality <00:27:32.669>differ <00:27:32.726>or <00:27:32.784>something <00:27:32.841>like <00:27:32.899>that? 00:27:32.950 --> 00:27:32.960 align:start position:0% <00:27:32.950>like <00:27:32.959>that? 00:27:32.960 --> 00:27:35.790 align:start position:0% <00:27:32.960>like <00:27:33.100>that? <00:27:33.240>>> <00:27:33.380>Yeah, <00:27:33.520>so <00:27:33.660>we <00:27:33.800>have <00:27:33.940>some <00:27:34.080>test <00:27:34.220>cases <00:27:34.360>also 00:27:35.790 --> 00:27:35.800 align:start position:0% <00:27:35.790>>> <00:27:35.791>Yeah, <00:27:35.792>so <00:27:35.793>we <00:27:35.794>have <00:27:35.795>some <00:27:35.796>test <00:27:35.797>cases <00:27:35.799>also 00:27:35.800 --> 00:27:37.870 align:start position:0% <00:27:35.800>>> <00:27:35.899>Yeah, <00:27:36.000>so <00:27:36.100>we <00:27:36.200>have <00:27:36.300>some <00:27:36.399>test <00:27:36.500>cases <00:27:36.600>also <00:27:36.700>for <00:27:36.800>multilingual. <00:27:36.899>We <00:27:37.000>also <00:27:37.100>did <00:27:37.200>some 00:27:37.870 --> 00:27:37.880 align:start position:0% <00:27:37.870>for <00:27:37.871>multilingual. <00:27:37.873>We <00:27:37.875>also <00:27:37.877>did <00:27:37.879>some 00:27:37.880 --> 00:27:39.750 align:start position:0% <00:27:37.880>for <00:27:38.035>multilingual. <00:27:38.191>We <00:27:38.346>also <00:27:38.502>did <00:27:38.657>some <00:27:38.813>benchmarking <00:27:38.968>with <00:27:39.124>different <00:27:39.280>languages 00:27:39.750 --> 00:27:39.760 align:start position:0% <00:27:39.750>benchmarking <00:27:39.753>with <00:27:39.756>different <00:27:39.759>languages 00:27:39.760 --> 00:27:41.310 align:start position:0% <00:27:39.760>benchmarking <00:27:39.899>with <00:27:40.039>different <00:27:40.178>languages <00:27:40.318>and <00:27:40.457>actually <00:27:40.597>the <00:27:40.736>model <00:27:40.876>that <00:27:41.015>we're <00:27:41.155>using 00:27:41.310 --> 00:27:41.320 align:start position:0% <00:27:41.310>and <00:27:41.311>actually <00:27:41.312>the <00:27:41.314>model <00:27:41.316>that <00:27:41.317>we're <00:27:41.319>using 00:27:41.320 --> 00:27:43.110 align:start position:0% <00:27:41.320>and <00:27:41.436>actually <00:27:41.553>the <00:27:41.669>model <00:27:41.786>that <00:27:41.903>we're <00:27:42.020>using <00:27:42.136>with <00:27:42.253>GPT <00:27:42.370>were <00:27:42.486>pretty <00:27:42.603>powerful <00:27:42.720>with 00:27:43.110 --> 00:27:43.120 align:start position:0% <00:27:43.110>with <00:27:43.111>GPT <00:27:43.113>were <00:27:43.115>pretty <00:27:43.117>powerful <00:27:43.119>with 00:27:43.120 --> 00:27:45.230 align:start position:0% <00:27:43.120>with <00:27:43.236>GPT <00:27:43.353>were <00:27:43.469>pretty <00:27:43.586>powerful <00:27:43.703>with <00:27:43.820>different <00:27:43.936>languages. <00:27:44.053>And <00:27:44.169>there <00:27:44.286>is <00:27:44.403>also <00:27:44.520>a 00:27:45.230 --> 00:27:45.240 align:start position:0% <00:27:45.230>different <00:27:45.231>languages. <00:27:45.233>And <00:27:45.234>there <00:27:45.236>is <00:27:45.237>also <00:27:45.239>a 00:27:45.240 --> 00:27:48.230 align:start position:0% <00:27:45.240>different <00:27:45.367>languages. <00:27:45.494>And <00:27:45.621>there <00:27:45.749>is <00:27:45.876>also <00:27:46.003>a <00:27:46.130>translation <00:27:46.258>part <00:27:46.385>from <00:27:46.512>ChatGPT <00:27:46.640>sometimes. 00:27:48.230 --> 00:27:48.240 align:start position:0% <00:27:48.230>translation <00:27:48.232>part <00:27:48.234>from <00:27:48.236>ChatGPT <00:27:48.239>sometimes. 00:27:48.240 --> 00:27:51.630 align:start position:0% <00:27:48.240>translation <00:27:48.367>part <00:27:48.494>from <00:27:48.621>ChatGPT <00:27:48.749>sometimes. <00:27:48.876>So, <00:27:49.003>we <00:27:49.130>didn't <00:27:49.258>find <00:27:49.385>any <00:27:49.512>different <00:27:49.640>results 00:27:51.630 --> 00:27:51.640 align:start position:0% <00:27:51.630>So, <00:27:51.631>we <00:27:51.633>didn't <00:27:51.634>find <00:27:51.636>any <00:27:51.637>different <00:27:51.639>results 00:27:51.640 --> 00:27:53.750 align:start position:0% <00:27:51.640>So, <00:27:51.747>we <00:27:51.855>didn't <00:27:51.963>find <00:27:52.070>any <00:27:52.178>different <00:27:52.286>results <00:27:52.393>with <00:27:52.501>the <00:27:52.609>most <00:27:52.716>common, <00:27:52.824>I <00:27:52.932>would <00:27:53.040>say, 00:27:53.750 --> 00:27:53.760 align:start position:0% <00:27:53.750>with <00:27:53.751>the <00:27:53.753>most <00:27:53.754>common, <00:27:53.756>I <00:27:53.757>would <00:27:53.759>say, 00:27:53.760 --> 00:27:55.030 align:start position:0% <00:27:53.760>with <00:27:53.923>the <00:27:54.086>most <00:27:54.249>common, <00:27:54.413>I <00:27:54.576>would <00:27:54.739>say, <00:27:54.903>languages. 00:27:55.030 --> 00:27:55.040 align:start position:0% languages. 00:27:55.040 --> 00:27:56.630 align:start position:0% <00:27:55.040>languages. <00:27:55.215>>> <00:27:55.389>Excellent. <00:27:55.565>I <00:27:55.740>think <00:27:55.915>that <00:27:56.090>that's <00:27:56.264>time <00:27:56.440>for 00:27:56.630 --> 00:27:56.640 align:start position:0% <00:27:56.630>>> <00:27:56.631>Excellent. <00:27:56.632>I <00:27:56.633>think <00:27:56.635>that <00:27:56.636>that's <00:27:56.637>time <00:27:56.639>for 00:27:56.640 --> 00:27:58.430 align:start position:0% <00:27:56.640>>> <00:27:56.733>Excellent. <00:27:56.826>I <00:27:56.920>think <00:27:57.013>that <00:27:57.106>that's <00:27:57.200>time <00:27:57.293>for <00:27:57.386>one <00:27:57.480>more <00:27:57.573>selfie <00:27:57.666>you <00:27:57.760>wanted <00:27:57.853>to <00:27:57.946>do. <00:27:58.040>And 00:27:58.430 --> 00:27:58.440 align:start position:0% <00:27:58.430>one <00:27:58.431>more <00:27:58.432>selfie <00:27:58.433>you <00:27:58.435>wanted <00:27:58.436>to <00:27:58.437>do. <00:27:58.439>And 00:27:58.440 --> 00:27:59.790 align:start position:0% <00:27:58.440>one <00:27:58.591>more <00:27:58.743>selfie <00:27:58.895>you <00:27:59.047>wanted <00:27:59.199>to <00:27:59.351>do. <00:27:59.503>And <00:27:59.655>then 00:27:59.790 --> 00:27:59.800 align:start position:0% then 00:27:59.800 --> 00:28:01.470 align:start position:0% <00:27:59.800>then <00:28:00.000>we <00:28:00.200>have <00:28:00.399>a <00:28:00.600>10-minute <00:28:00.800>break. <00:28:01.000>Once <00:28:01.200>again, 00:28:01.470 --> 00:28:01.480 align:start position:0% <00:28:01.470>we <00:28:01.471>have <00:28:01.473>a <00:28:01.474>10-minute <00:28:01.476>break. <00:28:01.477>Once <00:28:01.479>again, 00:28:01.480 --> 00:28:03.590 align:start position:0% <00:28:01.480>we <00:28:01.587>have <00:28:01.695>a <00:28:01.803>10-minute <00:28:01.910>break. <00:28:02.018>Once <00:28:02.126>again, <00:28:02.233>penultimate <00:28:02.341>talk <00:28:02.449>coming <00:28:02.556>up. <00:28:02.664>So, <00:28:02.772>see <00:28:02.880>you 00:28:03.590 --> 00:28:03.600 align:start position:0% <00:28:03.590>penultimate <00:28:03.591>talk <00:28:03.592>coming <00:28:03.594>up. <00:28:03.596>So, <00:28:03.597>see <00:28:03.599>you 00:28:03.600 --> 00:28:08.200 align:start position:0% <00:28:03.600>penultimate <00:28:03.727>talk <00:28:03.854>coming <00:28:03.981>up. <00:28:04.109>So, <00:28:04.236>see <00:28:04.363>you <00:28:04.490>all <00:28:04.618>again <00:28:04.745>at <00:28:04.872>3:40. <00:28:05.000>3:40.