Data Engineer
Robotics Technologies LLC
San Jose, CA, United States
4 days ago
- Discuss this with your agent
- Open in Claude
- Open in ChatGPT
Apply on www.indeed.com
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Role details
Contract type
Temporary contract
Employment type
Full-time (> 32 hours)
Working hours
Regular working hours
Job source
Tech stack
Application Programming Interfaces (APIs)
Agile Methodology
Amazon Web Services
Amazon S3
Microsoft Azure
Big Data
Cloud Computing
Cloud Storage
Code Review
Computer Programming
Databases
Continuous Integration
+46 more
Data Validation
Data Cleansing
Information Engineering
Data Integration
Extract Transform Load (ETL)
Data Mart
Data Transformation
Data Security
Data Warehousing
Relational Databases
Database Theory
Software Debugging
Dimensional Modeling
Apache Hadoop
Python (Programming Language)
PostgreSQL
Microsoft SQL Server
MySQL
Oracle (Applications)
Scrum Methodology
Systems Development Life Cycle
Query Optimization
Power BI
Azure Data Lake
SQL Stored Procedures
SQL Databases
SQL Server Integration Services
Unstructured Data
Data Processing
Freeform SQL
Data Storage Technologies
Cloud Platform System
Azure Data Factory
Informatica Powercenter
Snowflake
Apache Spark
Git
Pyspark
Information Technology
Star Schema
Restful APIs
Azure Synapse Analytics
Software Version Control
Data Pipelines
Jenkins
Databricks
Job description
- Designed, developed, and maintained scalable data pipelines to extract, transform, and load data from multiple sources.
- Developed ETL/ELT workflows using Python, SQL, PySpark, and Azure Data Factory.
- Built and optimized data processing solutions using Apache Spark and Databricks.
- Worked with structured and unstructured data from relational databases, APIs, flat files, and cloud storage.
- Designed and implemented data integration solutions to support business intelligence and analytics requirements.
- Developed complex SQL queries, stored procedures, views, and functions to transform and process large datasets.
- Implemented data validation, data cleansing, data transformation, and data quality checks to ensure data accuracy.
- Worked with Azure Data Lake Storage and cloud-based data platforms to store and manage large volumes of data.
- Created and maintained data warehouses, data marts, and dimensional data models.
- Integrated data from multiple enterprise systems using REST APIs and database connectors.
- Monitored data pipelines, identified failures, and resolved performance and data processing issues.
- Implemented incremental data loading, partitioning, and optimization techniques to improve pipeline performance.
- Worked with Snowflake for data loading, transformation, query optimization, and warehouse management.
- Implemented data security, access controls, and governance practices to protect sensitive business information.
- Used Git and Azure DevOps for source control, CI/CD, and deployment management.
- Collaborated with data analysts, data scientists, and business teams to understand data requirements.
- Participated in Agile/Scrum ceremonies, technical discussions, code reviews, and production support.
Requirements
- Programming: Python, SQL, PySpark
- Big Data: Apache Spark, Hadoop, Databricks
- ETL Tools: Azure Data Factory, Informatica, SSIS
- Cloud Platforms: Microsoft Azure, AWS
- Data Storage: Azure Data Lake, Amazon S3
- Data Warehousing: Snowflake, Azure Synapse Analytics, SQL Server
- Databases: PostgreSQL, MySQL, Oracle, SQL Server
- Data Modeling: Star Schema, Snowflake Schema, Dimensional Modeling
- DevOps: Git, Azure DevOps, CI/CD, Jenkins
- Reporting: Power BI
- Methodologies: Agile, Scrum, SDLC, * Bachelor’s degree in Computer Science, Information Technology, or a related field.
- Experience in data engineering, ETL development, and data pipeline implementation.
- Strong programming knowledge in Python, SQL, and PySpark.
- Experience working with cloud platforms and data warehousing technologies.
- Understanding of data modeling, database concepts, and data integration.
- Strong analytical, problem-solving, debugging, and communication skills.
Apply for this position
This job is hosted externally. Click below to view the full posting and apply.
Apply on www.indeed.com
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Good distractions
Loading talks and stories from around this role…