> Markdown version of [/jobs/ext/3607107-principal-software-engineer-data-processing-optimization](https://www.wearedevelopers.com/jobs/ext/3607107-principal-software-engineer-data-processing-optimization). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Principal Software Engineer, Data Processing Optimization

- **Company:** NetApp, Inc.
- **Location:** San Jose, CA, United States
- **Experience:** Expert
- **Salary:** $227,800.0 - $338,800.0
- **Contract:** Permanent contract
- **Skills:** Artificial Intelligence, Data Analysis, Computing Platforms, Software as a Service, Databases, Data Cleansing, Information Engineering, Data Warehousing, Python (Programming Language), Open Source Technology, Query Optimization, Software Engineering, SQL Databases, Data Processing, Enterprise Software Applications, Apache Spark, Information Technology, Apache Flink, Free and Open-Source Software, Presto
- **Published:** October 7, 2026
- **Apply:** https://jobs.netapp.com/talentcommunity/apply/1437491100/?locale=en_US

## About the Role

* Bachelor's degree in computer science. Master's or PhD preferred, especially in areas related to database systems.
 * 15+ years of experience developing query planning and query optimization techniques for enterprise data processing software platform. Prefer at least 5+ years of that experience on parallel platforms such as Apache Spark, Presto/Trino, or Apache Flink.
 * Deep knowledge of SQL semantics and expertise in SQL processing. Additional Python processing preferred. Solid understanding of databases, data warehouses, query engines.
 * Deep knowledge of query optimization state of the art, from published literature and open-source software. Prefer own open-source contributions and/or publications in this area.
 * Solid understanding of the architecture, deployment, and operations of data processing platforms in applications such as data engineering, data preparation, and analytics.
 * Demonstrated proficiency in all phases of software development and post-production support of enterprise software or SaaS. Adept in adopting AI effectively.
 * Experience owning the definition and delivery of major product capabilities through technical leadership and collaboration spanning multiple teams.
 * Strong collaboration and communication skills.

## Description

As a Senior/Principal Data Processing Optimization Engineer, you will be a key individual contributor in advancing planner and optimization capabilities so that DataPelago acceleration delivers the most performance and cost reduction for open-source software SQL and Python processing engines running on CPU-based and GPU-based infrastructure. This is a unique opportunity to make a significant impact on a category-defining product and work with a talented team of engineers., * Research, design, implement, and deliver logical and physical query plan optimization techniques for massively parallel data processing engines. Develop query planning and query optimization techniques that enable industry-leading performance and performance efficiency on execution engines based on heterogeneous accelerated compute elements.

 * Integrate DataPelago data processing acceleration and plan optimization capabilities with diverse open-source engines including Apache Spark and Apache Flink.
 * Ensure product performance on customers' real deployment environment and workload.
 * Establish metrics for plan quality, workload performance, reliability and lead the team in ensuring these metrics are continuously improved and achieved in customer deployments.
 * Provide technical leadership through all phases of product development and post-production support. Own development and support of major components and capabilities.
 * Collaborate with other engineering team members and product management in defining product features, architecture, interfaces, and development timelines. Mentor junior team members.