Data Engineer
PERIGONSOFT LLC
Columbia, MD, United States
14 days ago
- Discuss this with your agent
- Open in Claude
- Open in ChatGPT
Apply on www.dice.com
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Role details
Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Experience required
8 years minimum
Working hours
Regular working hours
Job source
Tech stack
Artificial Intelligence
Airflow
Microsoft Azure
Cloud Database
Continuous Integration
Information Engineering
Extract Transform Load (ETL)
Data Systems
DevOps
Github
Python (Programming Language)
Performance Tuning
+21 more
Software Tools
Scala (Programming Language)
SQL Databases
Data Streaming
Systems Integration
Azure Data Factory
Large Language Models
Snowflake
Prompt Engineering
Apache Spark
Event Driven Architecture
Pyspark
Infrastructure Automation Frameworks
Information Technology
Apache Kafka
Data Management
Terraform
Data Pipelines
Serverless Computing
Key Vault
Databricks
Job description
We are looking for an experienced Data Engineer with 9+ years of experience to design, build, and maintain data pipelines using Databricks, Spark, and Azure cloud services. The ideal candidate has hands-on experience with modern data engineering tools, cloud automation, and event-driven architectures., * Design, develop, and optimize scalable data pipelines using Databricks and Spark (PySpark)
- Build and maintain ETL/ELT workflows integrating with Snowflake and other data platforms
- Develop and manage Azure-based data solutions using ADF, ADLS, Azure Functions, and Key Vault
- Implement CI/CD pipelines and infrastructure automation using GitHub Actions, Terraform, and DevOps best practices
- Build and maintain event-driven architectures using Kafka, including CDC and streaming integrations
- Work with AI/GenAI initiatives involving LLM fine-tuning, RAG, and prompt engineering
- Monitor, troubleshoot, and optimize data pipeline performance and reliability
- Collaborate with data architects, analysts, and business stakeholders to translate requirements into technical solutions
- Ensure data quality, security, and governance standards are met across pipelines
- Orchestrate and schedule workflows using Airflow
Requirements
- 8+ years of experience in Data Engineering
- Data Engineering: Databricks, Spark (PySpark), Python/Scala, SQL, Snowflake
- Azure: Azure Cloud Services, ADF, ADLS, Azure Functions, Key Vault, DevOps
- AI/GenAI: LLM Fine-Tuning, RAG, Prompt Engineering
- Automation & DevOps: Airflow, GitHub, GitHub Actions, CI/CD, Terraform, Infrastructure Automation
- Kafka, Event-Driven Architecture, CDC/Streaming Integrations
Preferred Qualifications:
- Bachelor’s or Master’s degree in Computer Science, Engineering, or related field
- Azure certifications (e.g., Azure Data Engineer Associate, Azure Solutions Architect) is a plus
- Experience working in large-scale enterprise or regulated industry environments (finance, healthcare, insurance)
- Strong communication skills with ability to work directly with client stakeholders
Apply for this position
This job is hosted externally. Click below to view the full posting and apply.
Apply on www.dice.com
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Good distractions
Loading talks and stories from around this role…