> Markdown version of [/jobs/ext/1958231-data-engineering-palantir-spark-pyspark-python-w2-role](https://www.wearedevelopers.com/jobs/ext/1958231-data-engineering-palantir-spark-pyspark-python-w2-role). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Data Engineering (Palantir, Spark, PySpark, Python)- W2 Role

- **Company:** CLOUDSCOUTS SOFTWARE SOLUTIONS LLC
- **Location:** Frisco, TX, United States (Remote available)
- **Experience:** Experienced
- **Contract:** Permanent contract
- **Skills:** Airflow, Amazon Web Services, Microsoft Azure, Big Data, Software Quality, Code Review, Computer Programming, Continuous Integration, Information Engineering, Data Infrastructure, Data Integration, Extract Transform Load (ETL), DevOps, Distributed Systems, Python (Programming Language), Performance Tuning, Release Management, SQL Databases, Data Streaming, Workflow Management Systems, Enterprise Data Management, Data Processing, Azure Data Factory, Sql Optimization, Apache Spark, Pyspark, Information Technology, Data Management, Data Pipelines, Devsecops
- **Published:** August 6, 2026
- **Apply:** https://www.wayup.com/i-j-Data-Engineering-Palantir-Spark-PySpark-Python-W2-Role-CLOUDSCOUTS-SOFTWARE-SOLUTIONS-LLC-368185690634843/

## About the Role

Bachelor's or Master's degree in Computer Science, Engineering, or related field
8+ years of experience in data engineering, with at least 2-3 years in a technical leadership role
Strong hands-on experience with:
Python & PySpark
Apache Spark
Advanced SQL
Experience with Palantir Foundry or similar modern data platforms
Deep understanding of data engineering principles (ETL/ELT, data modeling, distributed systems)
Experience designing and managing large-scale data architectures
Strong leadership, communication, and stakeholder management skills
Preferred Qualifications
Experience in retail domain (supply chain, inventory, merchandising, store analytics)
Experience with cloud platforms (Azure, AWS, or GCP)
Familiarity with orchestration tools (Airflow, Azure Data Factory, etc.)
Experience with real-time/streaming data pipelines
Exposure to DevSecOps practices
Key Skills
Technical Leadership
Data Engineering Architecture
PySpark & Spark
Python Programming
SQL Optimization
Palantir Foundry
Data Modeling & Warehousing
CI/CD & DevOps
Flexible work from home options available.

## Description

We are looking for a hands-on Technical Lead in Data Engineering to drive the design, development, and delivery of scalable data solutions in a large retail enterprise. This role requires strong expertise in Palantir Foundry, Spark/PySpark, SQL, and Python, along with the ability to lead engineering teams and partner with business stakeholders across supply chain, merchandising, and store operations., Lead the design and implementation of scalable data platforms and pipelines using PySpark, Spark, and Python
Drive adoption and best practices for Palantir Foundry (data pipelines, ontology, workflows, and operational applications)
Architect and optimize high-performance data processing solutions for large-scale datasets
Provide technical leadership and mentorship to data engineers, ensuring code quality and best practices
Collaborate with cross-functional teams (business, analytics, data science) to translate requirements into scalable solutions
Design robust data models, ETL/ELT frameworks, and data integration strategies
Ensure data quality, governance, security, and compliance across enterprise data platforms
Lead performance tuning, troubleshooting, and optimization of data pipelines
Drive CI/CD implementation, code reviews, and release management
Stay current with emerging data engineering technologies and recommend improvements


## Related Videos

- [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works)
- [PySpark - Combining Machine Learning & Big Data](https://www.wearedevelopers.com/videos/44-pyspark-combining-machine-learning-big-data)
- [Data Science in Retail](https://www.wearedevelopers.com/videos/586-data-science-in-retail)
- [From DevOps to Scaled DevOps: How We’re Rebuilding Continuous Delivery as a Platform](https://www.wearedevelopers.com/videos/100018-from-devops-to-scaled-devops-how-we-re-rebuilding-continuous-delivery-as-a-platform)
- [Empowering Retail Through Applied Machine Learning](https://www.wearedevelopers.com/videos/976-empowering-retail-through-applied-machine-learning)
- [Enjoying SQL data pipelines with dbt](https://www.wearedevelopers.com/videos/823-enjoying-sql-data-pipelines-with-dbt)

## Related Articles

- [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers)
- [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk)
- [Fully Remote Software Engineer Jobs](https://www.wearedevelopers.com/magazine/447-fully-remote-software-engineer-jobs)
- [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer)
- [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know)
- [A Guide to Green Tech and Green IT Careers](https://www.wearedevelopers.com/magazine/374-a-guide-to-green-tech-and-green-it-careers)