Principal - Data Engineer

• Azure experience - ADF for orchestration, ADLS for storage, AzureDevOps for CI/CD
• Databricks experience - all compute/ETL leverages Databricks and is programmed leveraging Spark (PySpark, SparkSQL)
• PowerShell experience - this is our scripting language of choice
• SQL proficiency - it’s used everywhere (TSQL, PostgreSQL)

• Proficiency with parquet and delta formats Additionally - they will need experience in:

• SDLC + CI/CD - we follow a standard deployment process (dev, test, prod) that includes peer reviewed code. They need to be comfortable with standard DevOps practices.
• Should have a deep understanding of indexes and partitioning.
• Should be proficient optimizing code for performance (able to read a DAG, determine where CBO is using most resources)
• Should be proficient in writing code in a matter that it can run repeatedly and produce the same state (we have a custom SQL Deployment framework)

Skills: Code Reviews, Computer Programming, Continuous Deployment/Delivery, Continuous Integration, Database Extract Transform and Load (ETL), DevOps, Microsoft Transact-SQL (T-SQL), Microsoft Windows Azure, Performance Tuning/Optimization, PostgreSQL, Product Testing, SQL (Structured Query Language), Scripting (Scripting Languages), Software Development Lifecycle (SDLC), United States Citizen, Windows PowerShell

Georgia Tek Systems