Postion: Data Engineer

Brilliant

Edison, NJ, United States

3 days ago

Apply on www.dice.com

Prepare application

Role details

Contract type

Permanent contract

Employment type

Full-time (> 32 hours)

Working hours

Regular working hours

Job source

www.dice.com

Tech stack

Application Programming Interfaces (APIs) Airflow Amazon Web Services Microsoft Azure Big Data Databases Data Architecture Data Integration Extract Transform Load (ETL) Data Transformation Data Security Data Systems

+29 more

Data Visualization Data Warehousing Relational Databases Database Theory Apache Hadoop Python (Programming Language) Power BI Cloud Services SQL Stored Procedures SQL Databases Tableau (Software) Unstructured Data Data Processing Freeform SQL Google Cloud Azure Data Factory Snowflake Apache Spark Git Data Lakes Pyspark Information Technology AWS Glue Data Analytics Apache Kafka Stream Processing Data Pipelines Databricks Programming Languages

Job description

We are looking for a skilled Data Engineer to design, develop, and maintain scalable data pipelines and data solutions. The ideal candidate should have strong experience with data integration, ETL/ELT processes, cloud data platforms, databases, and data analytics technologies., * Design, develop, and maintain scalable data pipelines and ETL/ELT workflows.

Develop and optimize data integration solutions for structured and unstructured data.
Work with cloud data platforms such as AWS, Azure, or Google Cloud Platform (Google Cloud Platform).
Build and maintain data warehouses, data lakes, and lakehouse solutions.
Develop complex SQL queries, stored procedures, and data transformation processes.
Work with big data technologies such as Spark, Databricks, or Hadoop.
Implement data quality, validation, governance, and security processes.
Integrate data from APIs, databases, applications, and other enterprise data sources.
Optimize data pipelines and queries for performance, scalability, and reliability.
Collaborate with Data Scientists, Data Analysts, Software Engineers, and business teams to understand data requirements.
Monitor data pipelines and troubleshoot data processing and integration issues.
Maintain technical documentation for data architecture, pipelines, workflows, and processes.

Requirements

Strong proficiency in SQL and relational databases.
Hands-on experience with Python, PySpark, or similar programming languages.
Experience with ETL/ELT tools and data pipeline development.
Experience with cloud platforms such as AWS, Azure, or Google Cloud Platform.
Knowledge of data warehousing concepts and technologies.
Experience with technologies such as Snowflake, Databricks, Azure Data Factory, AWS Glue, or Redshift.
Experience working with Apache Spark/PySpark.
Strong understanding of data modeling, data integration, and database concepts.
Experience with Git and CI/CD practices.
Good understanding of data security, governance, and data quality., * Bachelor’s degree in Computer Science, Information Technology, Engineering, Data Science, or a related field.
Experience with Kafka or other real-time data streaming technologies.
Knowledge of dbt, Airflow, or similar data orchestration tools.
Experience with data visualization tools such as Power BI or Tableau.
Experience working with modern cloud data architectures and lakehouse platforms.
Relevant certifications in AWS, Azure, Google Cloud Platform, Databricks, or Snowflake.