Data Engineer

Precision Inc.
Louisville, KY, United States
3 days ago
Apply on www.thejobnetwork.com
Prepare application

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Working hours
Regular working hours

Tech stack

Amazon Web Services Microsoft Azure Data Governance Extract Transform Load (ETL) Apache Hive Enterprise Data Management Apache Spark Data Lakes Pyspark Information Technology Data Lakehouse Data Pipelines
+1 more
Databricks

Job description

Design, develop, and maintain scalable batch and real-time data pipelines using Databricks.

Requirements

n

Bachelor’s degree in Computer Science, Information Technology, or a related field.

\n

Strong experience with Databricks, Apache Spark, PySpark, Spark SQL, and Delta Lake.

\n

Experience designing and developing large-scale data pipelines and ETL processes.

\n

Strong understanding of Data Lakehouse architecture and data modeling concepts.

\n

Experience with cloud platforms such as Azure, AWS, or GCP.

\n

Knowledge of data governance, security, and enterprise data management practices.

\n

Excellent problem-solving, analytical, and communication skills.

Benefits & conditions

Build robust data solutions using PySpark, Spark SQL, and Delta Lake.

\n

Implement data validation, quality checks, and performance tuning processes.

\n

Develop ETL/ELT workflows to support enterprise data integration needs.

\n

Data Lakehouse Management

\n

Design and manage data architecture within the Databricks Lakehouse Platform.

\n

Implement Delta Lake tables, partitioning strategies, optimization techniques, and schema evolution.

\n

Develop data models to support analytics, reporting, and business intelligence requirements.

\n

Ensure data reliability, scalability, governance, and availability.

\n

Performance Optimization

\n

Analyze and optimize Spark jobs for performance and cost efficiency.

\n

Monitor Databricks cluster utilization and implement best practices for resource management.

\n

Troubleshoot bottlenecks in data processing workflows.

\n

Improve processing times through caching, partitioning, and query optimization techniques.

\n

Data Governance & Security

\n

Implement role-based access controls (RBAC) and data security standards.

\n

Support data governance initiatives, data lineage tracking, and compliance requirements.

\n

Ensure adherence to enterprise data management policies and security frameworks.

\n

Collaboration & Delivery

\n

Work closely with Data Architects, Data Scientists, Business Analysts, and Application Teams.

\n

Participate in Agile development processes, sprint planning, and scrum ceremonies.

\n

Prepare technical design documentation and support knowledge transfer activities.

\n

Collaborate with stakeholders to gather requirements and translate them into scalable technical solutions.

\n

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.thejobnetwork.com
Prepare application

Good distractions

Loading talks and stories from around this role…