Data Engineer

Design, build, monitor, and troubleshoot ETL pipelines (Azure Data Factory) to move and transform large datasets from sources like Salesforce and SFTP into Snowflake. Implement SQL-based data cleansing and quality checks, document data lineage and mapping, collaborate with QA/UAT, and recommend new integration tools and efficiencies., Job Duties: Build ETL (extract, transform, and loading) jobs using Azure Data Factory for internal projects and externally facing customers that use various platforms such as Azure, Salesforce, and SFTP technologies. Develop ETL jobs that manipulate and transfer large volumes of data and perform quality checks. Monitor active ETL jobs in production. When ETL jobs fail, log into the environment to determine what failed and work on the issue or collaborate with team members to resolve the failure. Create Azure Data Factory pipelines to process the data and manipulate and clean the data using SQL scripts. Assist with the build out data lineage artifacts to ensure all current and future systems are properly documented. Assist with the build out design/mapping documentation to ensure development is clear and testable for QA (quality assurance) and UAT (user acceptance testing) purposes. Help to assess current and future data transformation needs to recommend, develop, and train new data integration tool technologies. Build ETL jobs for the new Snowflake data platform initiative to help centralize all data for operational and analytical reporting needs. Assist with discovering efficiencies with shared data processes and batch schedules to help ensure no redundancy and smooth operations. Assist with the data quality analyst to implement checks and balances across all jobs to ensure data quality throughout the entire environment for current and future batch jobs. Use of Azure Data Factory, Apache Kafka, SQL, Python, Scala, Hadoop, Spark, Snowflake, Salesforce, Azure, AWS, and SQL Server.

Bachelor’s degree or foreign academic equivalent in Computer Science, Computer Engineering, Data Analytics, Information Systems, or a related field. Plus 1 year of experience in the job offered or a related occupation including Programmer Analyst or related experience. Position requires 1 year of experience with ETL, Python, and SQL. 40hrs/week.