Data Engineer PySpark/Python
Highbrow LLC
Dallas, TX, United States
1 day ago
- Discuss this with your agent
- Open in Claude
- Open in ChatGPT
Apply on www.careerboard.com
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Role details
Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Experience required
8 years minimum
Working hours
Regular working hours
Job source
Tech stack
Agile Methodology
Airflow
Amazon Web Services
Microsoft Azure
Big Data
Continuous Integration
Information Engineering
Extract Transform Load (ETL)
Data Systems
Data Warehousing
Database Queries
Distributed Computing Environment
+17 more
Distributed Systems
Apache Hive
Python (Programming Language)
Query Optimization
Cloud Services
SQL Databases
Workflow Management Systems
Data Processing
Cloud Platform System
Azure Data Factory
Apache Spark
Git
Data Lakes
Pyspark
AWS Glue
Data Pipelines
Databricks
Requirements
We are looking for an experienced Data Engineer with strong PySpark expertise to design, develop, and maintain scalable data processing solutions. The ideal candidate will have extensive experience in big data technologies, distributed computing, ETL/ELT development, and cloud-based data platforms.
The candidate should be comfortable working with large datasets, building efficient data pipelines, optimizing Spark workloads, and collaborating with cross-functional teams to deliver reliable data solutions. Required Skills & Qualifications
- 8+ years of experience in Data Engineering or related roles.
- Strong hands-on experience with Python and PySpark.
- Extensive experience with Apache Spark, Spark SQL, and distributed data processing.
- Strong proficiency in SQL, including complex queries, joins, aggregations, and query optimization.
- Experience developing and maintaining large-scale ETL/ELT data pipelines.
- Strong understanding of data warehousing concepts, data modeling, and data lake architectures.
- Experience working with cloud data platforms such as AWS, Azure, or GCP.
- Experience with technologies such as Databricks, Delta Lake, Hive, or similar big data frameworks.
- Knowledge of workflow orchestration tools such as Apache Airflow, Azure Data Factory, or AWS Glue.
- Experience with Git, CI/CD pipelines, and Agile development methodologies.
- Strong analytical, troubleshooting, and problem-solving skills.
Apply for this position
This job is hosted externally. Click below to view the full posting and apply.
Apply on www.careerboard.com
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Good distractions
Loading talks and stories from around this roleβ¦