Machine Learning Engineer I - Large Language Models - AI & Human Health Research

We are seeking a skilled LLM Engineer I to join our team in the SinAI Assurance Lab. The LLM Engineer will play a key role in designing, building, and deploying large language model (LLM) applications including retrieval-augmented generation (RAG) systems, agentic platforms, and clinical chatbots and will be responsible for designing, maintaining, and optimizing data infrastructure and model validation pipelines that ensure all AI systems, Generative and Non-Generative, deployed across the Mount Sinai Health System (MSHS) are rigorously validated for compliance, performance, and patient safety.

You will work closely with AI product teams, clinical and technical stakeholders, DevOps engineers, and the AI Governance Committee to engineer scalable data flows that support model validation, real-time monitoring, and Machine Learning Engineer I will be primarily responsible for contributing to the development and enhancement of machine learning applications and systems. They will work closely with other engineers and data scientists to design and implement scalable and efficient machine learning systems.

• Bachelor's degree in Computer Science, Statistics, Mathematics, or related field.
• Knowledge of at least one programming language among Scala, Python, Java, C, or C++.
• Knowledge of big data technologies (e.g., Hadoop, Spark)
• Knowledge of Software Development Lifecycle.
• Self-motivated with a demonstrated ability to work independently, and to exercise independent judgment in developing complex techniques or programs in a dynamic environment.

• Act as the major contributor in the development and operationalization of four different applications.
• Play a key technical role in maintaining deployed products

• Understanding of machine learning algorithms (Supervised, Unsupervised ML algorithms).
• Familiarity with SQL or other database languages.

Preferred:

• Master's degree in a quantitative discipline (e.g., Statistics, Operations Research, Bioinformatics, Economics, Computational Biology, Computer Science, Information Technology, Mathematics, Physics) or equivalent practical experience
• 2+ years of experience in data engineering, software engineering, or machine learning.
• Proficient in Python and SQL
• Proficiency in at least one cloud computing platforms (e.g., AWS, Azure, GCP)
• Intermediate knowledge of Machine Learning
• Familiarity with ML lifecycle management tools (e.g., MLflow, Kubeflow, Airflow)
• Experience on deployment and operationalization of ML Systems
• Experience with monitoring tools for AI model tracking
• Understanding of DevOps principles, CI/CD pipelines, and containerization (e.g., Docker, Kubernetes)
• Experience with version control systems (e.g., Git) Knowledge of big data technologies (e.g., Hadoop, Spark)
• Hands-on experience building and deploying LLM-based applications in production (chatbots, copilots, summarization, Q&A, or decision-support tools).
• Experience designing and implementing retrieval-augmented generation (RAG) architectures, including chunking strategies, embedding models, and vector databases (e.g., Pinecone, Weaviate, FAISS, pgvector, Milvus).
• Experience with agentic frameworks and orchestration libraries (e.g., LangChain, LlamaIndex, LangGraph, CrewAI, AutoGen, Semantic Kernel) including tool/function calling and multi-agent workflows.
• Experience building conversational AI / chatbot systems, including dialog state management, memory, and integration with enterprise systems.
• Familiarity with foundation model APIs and SDKs (e.g., OpenAI, Anthropic, Google, Azure OpenAI, AWS Bedrock) and open-weight model families (e.g., Llama, Mistral, Qwen, Gemma).
• Working knowledge of prompt engineering, prompt evaluation, and LLM observability/evaluation tooling (e.g., LangSmith, Langfuse, Arize, Ragas, DeepEval).
• Familiarity with fine-tuning and model adaptation techniques (e.g., supervised fine-tuning, LoRA/QLoRA, PEFT, instruction tuning, RLHF/DPO) and serving stacks (e.g., vLLM, TGI, Triton).
• Awareness of LLM safety, guardrails, and evaluation practices (hallucination, bias, sycophancy, jailbreak resistance) - experience with healthcare-specific evaluation is a plus.
• Strong problem-solving skills and ability to work in cross-functional teams

The Mount Sinai Health System (MSHS) provides salary ranges that comply with the New York City Law on Salary Transparency in Job Advertisements. The salary range for the role is $109,000.00 - $163,695.00 Annually. Actual salaries depend on a variety of factors, including experience, education, and operational need. The salary range or contractual rate listed does not include bonuses/incentive, differential pay or other forms of compensation or benefits.

Non-Bargaining Unit, J18 - AI and Human Health - ISM, Icahn School of Medicine