Bridging LLMs and Systems: Practical Automation with MCP Tools and Function Calls

In this session, we’ll look at how a .NET-based solution enables Large Language Models (LLMs) to act as automation engines inside a VR application. The system uses the Model Context Protocol (MCP) to give the LLM controlled access to real system data and APIs. When a user submits a request — for example, “make the workspace more ergonomic” — the LLM interprets the intent, fetches relevant user and environment data via MCP, and responds by invoking an API or function with precise parameters.

We’ll walk through how this automation pipeline is built: from the LLM prompt orchestration and MCP data interfaces, to function calling, execution in .NET, and real-time feedback in VR.

The talk will also include real scenarios such as automatically identifying unused areas in a VR layout, adjusting ergonomics, or optimizing object placement — all triggered by natural-language input.