> Markdown version of [/jobs/ext/1971070-lead-data-engineer](https://www.wearedevelopers.com/jobs/ext/1971070-lead-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Lead Data Engineer - **Company:** Shiftcode Analytics, Inc - **Location:** Pittsburgh, PA, United States - **Experience:** Expert - **Contract:** Permanent contract - **Skills:** Application Programming Interfaces (APIs), Agile Methodology, Confluence, JIRA, Unit Testing, Big Data, Unix, Configuration Management, Code Coverage, Information Engineering, Data Recovery, Database Storage Structures, File Systems, Python (Programming Language), NumPy, Object-Oriented Software Development, SQL Databases, Parquet, Scripting, File Transfer Protocol (FTP), Data Storage Technologies, Fastapi, Pandas, Matplotlib, Data Lakes, Pyspark, Data Management, Multiplatform, Software Version Control, Data Pipelines, Microservices - **Published:** August 7, 2026 - **Apply:** https://www.careerbuilder.com/job-details/lead-data-engineer-pa--ee6b1d1b-3670-416b-a635-81497da50457 ## About the Role Roles and Responsibilities: Interact with Business to understand the requirement on a daily evolving need and adapt to the needs. Able to understand the business requirements, tech stack involved and guide the team towards Integral solution approach. Dedicated and committed approach, with good communication skills and participation in agile ceremonies. Experience with Object Oriented Programming including limitations. Familiarity with cross-platform development Able to handle all the database storage and retrieval mechanisms on Data Lake Platforms and extensive experience in handling. Extensive Unix/FTP/ File Handling Strong Hands-on experience with SQL databases Must Have Technical Skills: A deep understanding and multi-process architecture and the threading limitations of Python. Pyspark Data engineering skills Experience using libraries like Pandas, NumPy and MatPlotLib. With experience working with file based systems(supporting csv/parquet/) Extensive knowledge in understanding Data and creating Data Pipelines Hands on experience designing and implementing APIs Hands on experience building microservices using FastAPI or related technologies. Experience writing unit testing and ensure code coverage. Experience with Agile Methodology/JIRA/Confluence Experience in version control using GitFlex Flex Skills/Nice to Have: Added advantage if worked on Big Data, PySpark libraries. Soft Skills: Excellent written and verbal communication skills Team player Skills: Agile Programming Methodologies, Application Programming Interface (API), Atlassian JIRA, Big Data, Code Coverage, Communication Skills, Data Lake, Data Management, Data Recovery, Data Storage, Engineering, FTP (File Transfer Protocol), File Systems, Microservices, Multiplatform/Cross-Platform, Object Oriented Programming (OOP), Presentation/Verbal Skills, Python Programming/Scripting Language, SQL Databases, Source Code/Configuration Management (SCM), Team Player, Test Plan/Schedule, Unit Test, Unix Operating Systems, Writing Skills ## Related Videos - [Improving quality with Agentic AI with Rovo Dev and Xray](https://www.wearedevelopers.com/videos/2005-improving-quality-with-agentic-ai-with-rovo-dev-and-xray) - [WeAreDevelopers LIVE - Node and Package Security](https://www.wearedevelopers.com/videos/2138-wearedevelopers-live-node-and-package-security) - [Vectorize all the things! Using linear algebra and NumPy to make your Python code lightning fast.](https://www.wearedevelopers.com/videos/562-vectorize-all-the-things-using-linear-algebra-and-numpy-to-make-your-python-code-lightning-fast) - [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works) - [Collaboration Quantified: Lessons from Open Source Developer Networks](https://www.wearedevelopers.com/videos/1422-collaboration-quantified-lessons-from-open-source-developer-networks) - [Alibaba Big Data and Machine Learning Technology](https://www.wearedevelopers.com/videos/37-alibaba-big-data-and-machine-learning-technology) ## Related Articles - [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know) - [Navigating the AI Shift](https://www.wearedevelopers.com/magazine/629-navigating-the-ai-shift) - [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story) - [Dev Digest 120 - Apple and peers](https://www.wearedevelopers.com/magazine/455-dev-digest-120-apple-and-peers) - [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers) - [The 13 Best Python Libraries for Developers in 2025](https://www.wearedevelopers.com/magazine/371-the-13-best-python-libraries-for-developers-in-2025)