> Markdown version of [/jobs/ext/491371-machine-learning-engineer-inference-new](https://www.wearedevelopers.com/jobs/ext/491371-machine-learning-engineer-inference-new). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Machine Learning Engineer - Inference New

- **Company:** BYTEDANCE INC.
- **Location:** San Jose, CA, United States
- **Experience:** Experienced
- **Salary:** $156,000.0
- **Contract:** Permanent contract
- **Skills:** Machine Learning, Open Source Technology, Remote Direct Memory Access, Tensorflow, AI Infrastructure, High Performance Computing, Hardware Acceleration, TensorRT
- **Published:** May 31, 2026
- **Apply:** https://www.gamesjobsdirect.com/job/bytedance/machine-learning-engineer-inference/345955

## About the Role

Minimum Qualifications: - At least 3 years of experience in developing and deploying large-scale systems. - Experience contributing to an open sourced machine learning framework (tensorflow / jax / pytorch / torchscript / mxnet / tensorrt). - Strong background in one of the following fields: Hardware-Software Co-Design, High Performance Computing, ML Hardware Acceleration (e.g., GPU/RDMA) or ML for Systems.

## Description

The mission of our AML team is to push the next-generation AI infrastructure and recommendation platform for the ads ranking, search ranking, live & ecom ranking in our company. We also drive substantial impact on core businesses of the company. Currently, we are looking for Machine Learning Engineer - Inference to join our team to support and advance that mission. Responsibilities: - Responsible for the design and implementation of distributed inference infrastructure for feeds, ads and search ranking models. - Responsible for building monitoring/managing tools to oversee the reliability and scalability of online inference servers - Responsible for triaging system inefficiency and bottlenecks and improving system performance - Responsible for building tools to analyze bottlenecks and sources of instability and then design and implement solutions - Responsible for collaboration with product teams and providing general solutions to meet their requirements


## Related Videos

- [Machine learning in the browser with TensorFlowjs](https://www.wearedevelopers.com/videos/155-machine-learning-in-the-browser-with-tensorflowjs)
- [Tour de Force: Open-Source LLM Inference Optimization from Simple to Sophisticated](https://www.wearedevelopers.com/videos/100099-tour-de-force-open-source-llm-inference-optimization-from-simple-to-sophisticated)
- [Introduction to Azure Machine Learning](https://www.wearedevelopers.com/videos/368-introduction-to-azure-machine-learning)
- [Efficient deployment and inference of GPU-accelerated LLMs​](https://www.wearedevelopers.com/videos/929-efficient-deployment-and-inference-of-gpu-accelerated-llms)
- [How Machine Learning is turning the Automotive Industry upside down](https://www.wearedevelopers.com/videos/61-how-machine-learning-is-turning-the-automotive-industry-upside-down)
- [Trends, Challenges and Best Practices for AI at the Edge](https://www.wearedevelopers.com/videos/630-trends-challenges-and-best-practices-for-ai-at-the-edge)

## Related Articles

- [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers)
- [What Are Large Language Models?](https://www.wearedevelopers.com/magazine/304-what-are-large-language-models)
- [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer)
- [MLOps And AI Driven Development](https://www.wearedevelopers.com/magazine/82-mlops-and-ai-driven-development)
- [What Industries Outside of AI Are Hiring The Most AI Experts?](https://www.wearedevelopers.com/magazine/98-what-industries-outside-of-ai-are-hiring-the-most-ai-experts)
- [Stephan Gillich - Bringing AI Everywhere](https://www.wearedevelopers.com/magazine/489-stephan-gillich-bringing-ai-everywhere)