> Markdown version of [/jobs/ext/1658333-senior-data-engineer](https://www.wearedevelopers.com/jobs/ext/1658333-senior-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Senior Data Engineer - **Company:** CGI Technologies and Solutions, Inc. - **Location:** Pittsburgh, PA, United States - **Experience:** Expert - **Salary:** $79,600.0 - **Contract:** Permanent contract - **Skills:** Agile Methodology, Artificial Intelligence, Airflow, Amazon Web Services, Apache HTTP Server, Microsoft Azure, Big Data, Cloudera Impala, Code Review, Information Engineering, Data Governance, Extract Transform Load (ETL), Data Transformation Services, Distributed Systems, Apache Hadoop, Apache Hive, Python (Programming Language), Meta-Data Management, MySQL, Apache Oozie, Oracle (Applications), Performance Tuning, Scrum Methodology, Query Optimization, Teradata SQL, Workflow Management Systems, Enterprise Data Management, Google Cloud, Cloud Platform System, Large Language Models, Apache Spark, Generative AI, Git, Data Lakes, AI Platforms, Pyspark, Apache Kafka, Software Coding, Software Version Control, Data Pipelines - **Published:** July 8, 2026 - **Apply:** https://dejobs.org/x/x/CA1BACE740544E1CBE9E950FC883713F/job/ ## About the Role 6+ years of experience in Data Engineering, Big Data development, or enterprise data platforms. . Strong experience developing enterprise scale Big Data applications. . Hands on expertise with: o Hadoop ecosystem o Hive o Apache Spark o Impala o PySpark o Apache Iceberg . Strong SQL development and query optimization skills. . Experience designing scalable ETL/ELT pipelines. . Experience working with distributed computing environments and large volume datasets. . Strong analytical, troubleshooting, and problem solving skills. . Experience working in Agile/Scrum delivery environments. . Excellent communication and stakeholder collaboration skills. . Proven ability to lead technical initiatives across geographically distributed teams. . Experience with cloud based data platforms (AWS, Azure, or Google Cloud). . Experience with orchestration tools such as Airflow or Oozie. . Experience with version control systems such as Git and CI/CD pipelines. . Familiarity with data governance, metadata management, and data quality frameworks. . Experience with performance tuning and optimization of Big Data workloads. . Working knowledge of Large Language Models (LLMs) and Retrieval Augmented Generation (RAG) architectures is a plus., * Apache Kafka * AWS AI Services * Big Data,Analytics&Operations * Large Language Model (LLM) * MySQL * Oracle * Retrieval-Augmented Gen.(RAG) * Teradata ## Description We are seeking a highly motivated and experienced Senior Data Engineer to design, develop, and optimize large scale data transformation solutions on modern Big Data platforms. The ideal candidate is passionate about technology, thrives in a fast paced environment, and possesses a strong ownership mindset with a "can do" attitude. This role requires deep expertise in Hadoop ecosystem technologies, Python, and modern data lake architectures. The successful candidate will serve as a technical leader on a large scale digital transformation initiative, collaborating with cross functional business and technology teams while guiding both onshore and offshore development teams to deliver high quality, scalable data solutions. Experience with Generative AI technologies and AI assisted development is highly desirable. This position can be performed onsite five days a week at our client site in Strongsville, OH or Pittsburgh, PA or Dallas, TX. Future duties and responsibilities . Design, develop, and maintain scalable data pipelines and transformation frameworks using Hadoop ecosystem technologies. . Build high performance data transformation solutions utilizing Hive, Spark, Python, Impala, and Apache Iceberg. . Develop robust ETL/ELT processes supporting enterprise scale analytics and data products. . Design and optimize Hive, Spark SQL, and Impala queries for large datasets. . Implement scalable data lake solutions leveraging Apache Iceberg table formats and best practices. . Collaborate closely with Product Owners, Business Analysts, Subject Matter Experts, Architects, and Technical Managers to translate business requirements into technical solutions. . Lead technical design discussions and establish engineering best practices across the development team. . Provide technical leadership and mentoring to both onshore and offshore development teams. . Conduct code reviews and ensure adherence to coding standards, performance optimization, and data quality best practices. . Troubleshoot production issues and implement long term sustainable solutions. . Drive continuous improvement through automation, reusable frameworks, and engineering best practices. . Participate in Agile ceremonies including sprint planning, backlog refinement, and retrospectives. . Stay current with emerging Big Data, AWS cloud, and AI technologies and recommend innovative solutions. ## Related Videos - [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works) - [Alibaba Big Data and Machine Learning Technology](https://www.wearedevelopers.com/videos/37-alibaba-big-data-and-machine-learning-technology) - [MySQL Protocol Features You Should Be Aware Of](https://www.wearedevelopers.com/videos/100267-mysql-protocol-features-you-should-be-aware-of) - [How a Small Team Shrank a Microsoft Monorepo by 94%](https://www.wearedevelopers.com/videos/1236-how-a-small-team-shrank-a-microsoft-monorepo-by-94) - [Enjoying SQL data pipelines with dbt](https://www.wearedevelopers.com/videos/823-enjoying-sql-data-pipelines-with-dbt) - [Branch your database like your code: How schema changes and pull requests go hand in hand](https://www.wearedevelopers.com/videos/350-branch-your-database-like-your-code-how-schema-changes-and-pull-requests-go-hand-in-hand) ## Related Articles - [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know) - [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story) - [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers) - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk) - [Coffee with Developers - Maria Apazoglou - Making AI understandable for all in production](https://www.wearedevelopers.com/magazine/475-coffee-with-developers-maria-apazoglou-making-ai-understandable-for-all-in-production) - [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer)