Data Engineer
TekFortune INC
Fremont, CA, United States
18 days ago
- Discuss this with your agent
- Open in Claude
- Open in ChatGPT
Apply on www.dice.com
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Role details
Contract type
Contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Experience required
10 years minimum
Working hours
Regular working hours
Job source
Tech stack
Application Programming Interfaces (APIs)
Information Engineering
Data Integration
Extract Transform Load (ETL)
Data Warehousing
Apache Hive
SQL Databases
Unstructured Data
Data Processing
Data Ingestion
Apache Spark
Pyspark
+3 more
Data Pipelines
Api Management
Databricks
Job description
- Design, develop, and maintain scalable ETL/ELT pipelines for structured and unstructured data.
- Build and optimize data processing solutions using Databricks, Apache Spark, PySpark, and Spark SQL.
- Develop efficient data integration solutions using APIs and various data sources.
- Design and implement database schemas, dimensional models, and data warehouse solutions.
- Implement and maintain Medallion Architecture (Bronze, Silver, and Gold layers).
- Optimize data pipelines, SQL queries, and Spark workloads for performance and scalability.
- Follow data warehousing best practices, development standards, and engineering methodologies.
- Independently manage technical deliverables and ensure timely project execution.
Requirements
We are looking for an experienced Data Engineering Lead with 10+ years of experience in designing, developing, and optimizing scalable data engineering solutions. The ideal candidate should have strong hands-on expertise in ETL/ELT development, SQL, PySpark, Databricks, Apache Spark, data modeling, and Medallion Architecture.
The candidate should be capable of independently driving data engineering initiatives, building robust data pipelines, and delivering high-quality solutions in a fast-paced environment., * 10+ years of overall experience in Data Engineering.
- Strong hands-on experience with ETL/ELT development.
- Expertise in SQL, PySpark, and Spark SQL.
- Extensive experience with Databricks and Apache Spark.
- Strong understanding of structured and unstructured data processing.
- Experience with API integrations and data ingestion.
- Expertise in schema design and dimensional data modeling.
- Hands-on experience with Medallion Architecture.
- Strong knowledge of Data Warehouse concepts and best practices.
Preferred Candidate Profile
- Strong analytical and problem-solving skills.
- Self-motivated with an independent working approach.
- Ability to work in a fast-paced and dynamic environment.
- Strong ownership, communication, and delivery mindset.
Apply for this position
This job is hosted externally. Click below to view the full posting and apply.
Apply on www.dice.com
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Good distractions
Loading talks and stories from around this roleβ¦