Senior Data Engineer
Beautifulll Consulting Co.
Pittsburgh, PA, United States
24 days ago
- Discuss this with your agent
- Open in Claude
- Open in ChatGPT
Apply on www.indeed.com
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Role details
Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Experience required
7 years minimum
Working hours
Regular working hours
Job source
Tech stack
Automation of Tests
Continuous Integration
Information Engineering
Data Integration
Extract Transform Load (ETL)
Data Systems
Data Virtualization
Data Warehousing
Database Design
Database Queries
R (Programming Language)
Python (Programming Language)
+15 more
Operational Databases
Oracle (Applications)
Cloud Services
SQL Databases
Systems Integration
Tableau (Software)
Unstructured Data
Informatica Powercenter
Apache Spark
Pyspark
Tools for Reporting
Spotfire
Software Version Control
Data Pipelines
Databricks
Job description
- Developing and maintaining scalable data pipelines and reusable datasets within a Databricks lakehouse environment
- Ingesting, validating, cleaning and transforming structured and unstructured data from multiple sources using Python, PySpark and SQL
- Building data models and reporting datasets that support Tableau and established analytics tools such as Spotfire
- Integrating established data warehouse and reporting systems with modern cloud analytics platforms
- Improving pipeline reliability, data quality and processing performance
- Implementing automated testing, monitoring, documentation and CI/CD practices
- Partnering with analysts, architects, developers and business stakeholders to deliver practical data solutions
- Providing technical guidance, reviewing code and mentoring other data engineer
Requirements
- 7+ years of experience in data engineering, including building and supporting production data pipelines
- Strong SQL skills and a solid understanding of data modeling, database design and data warehousing
- Hands-on experience with Databricks, Apache Spark and cloud data platforms
- Strong proficiency in Python and PySpark
- Experience developing ETL/ELT pipelines and integrating data from multiple source systems
- Experience preparing reliable datasets for Tableau or comparable business intelligence tools
- Familiarity with source control, automated testing and CI/CD practices
- Ability to troubleshoot complex data issues and optimize pipeline performance
- Strong communication skills and experience collaborating with technical teams and business stakeholders
- Experience with Spotfire, Oracle, Informatica, R or data virtualization preferred
- Databricks certification preferred
Apply for this position
This job is hosted externally. Click below to view the full posting and apply.
Apply on www.indeed.com
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Good distractions
Loading talks and stories from around this roleβ¦