Data Engineer / Lead Data Engineer

Raas Infotek LLC

Texas City, TX, United States

1 day ago

Apply on www.dice.com

Prepare application

Role details

Contract type

Permanent contract

Employment type

Full-time (> 32 hours)

Experience level

Expert

Experience required

10 years minimum

Working hours

Regular working hours

Job source

www.dice.com

Tech stack

Application Programming Interfaces (APIs) Agile Methodology Airflow Amazon Web Services Microsoft Azure BigQuery Cloud Database Code Review Databases Continuous Integration Information Engineering Data Governance

+34 more

Data Integration Extract Transform Load (ETL) Data Mart Data Transformation Data Security Data Warehousing Relational Databases DevOps Dimensional Modeling Python (Programming Language) Operational Databases Software Tools SQL Stored Procedures SQL Databases Software Organization Freeform SQL Google Cloud Azure Data Factory Informatica Powercenter Snowflake Apache Spark Git Data Lakes Pyspark Kubernetes Information Technology Apache Kafka Data Management Video Streaming Azure Synapse Analytics Data Pipelines Docker Amazon Redshift Databricks

Job description

We are looking for an experienced Data Engineer with 10+ years of experience in designing, developing, and maintaining scalable data pipelines and data platforms. The ideal candidate should have strong expertise in ETL/ELT development, cloud data technologies, data warehousing, SQL, and modern data engineering tools.

The candidate will work closely with data analysts, data scientists, architects, and business teams to build reliable and scalable data solutions., * Design, develop, and maintain scalable data pipelines and ETL/ELT processes.

Develop complex SQL queries, stored procedures, and data transformations.
Build and optimize data warehouses, data lakes, and data marts.
Perform data integration from multiple sources including APIs, databases, files, and cloud platforms.
Develop data pipelines using tools such as Azure Data Factory, Databricks, Apache Spark, Airflow, Informatica, or equivalent technologies.
Work with cloud platforms such as AWS, Azure, or Google Cloud Platform.
Implement data quality, validation, monitoring, and error-handling processes.
Optimize data pipelines and queries for performance and scalability.
Collaborate with architects, developers, analysts, and business stakeholders.
Troubleshoot production data issues and provide timely resolutions.
Implement best practices for data security, governance, and lifecycle management.
Participate in code reviews, technical documentation, and Agile development processes.

Requirements

10+ years of experience in Data Engineering
Strong expertise in SQL and relational databases.
Strong experience with ETL/ELT development and data integration.
Experience with Python and/or Scala.
Hands-on experience with Apache Spark / PySpark.
Experience with at least one major cloud platform: AWS, Azure, or Google Cloud Platform.
Strong knowledge of data warehousing concepts and dimensional modeling.
Experience with technologies such as Databricks, Snowflake, Redshift, BigQuery, Azure Synapse, or equivalent.
Experience with data pipeline orchestration tools such as Airflow, Azure Data Factory, or similar.
Strong understanding of data modeling, data quality, and data governance.
Excellent problem-solving and communication skills., * Experience with CI/CD and DevOps practices.
Experience working with Kafka or other streaming technologies.
Knowledge of Docker/Kubernetes.
Experience with Git and modern software development practices.
Experience working in Agile/Scrum environments.
Bachelor’’s degree in Computer Science, Engineering, Information Technology, or a related field.