WeAreDevelopers Logo WeAreDevelopers
Sign in
For You LIVE
Back

Senior Big Data Software Engineer (AWS + Databricks)

SoftServe, Inc.
United States
7 days ago
Apply on arc.dev
Prepare application

Role details

Contract type
Permanent contract
Employment type
Part-time (≀ 32 hours)
Experience level
Expert
Experience required
5 years minimum
Working hours
Regular working hours
Languages
English
Job source
arc.dev

Tech stack

Java (Programming Language) Artificial Intelligence Airflow Amazon Web Services Data Analysis Big Data Continuous Integration Information Engineering Data Infrastructure Data Systems Relational Databases Distributed Computing Environment
+19 more
Distributed Data Store Python (Programming Language) PostgreSQL MySQL SQL Databases Data Processing Apache Spark AI Platforms Infrastructure Automation Frameworks AWS Glue Integration Frameworks Apache Kafka Data Management Machine Learning Operations Video Streaming Terraform Software Version Control Data Pipelines Databricks

Job description

In this role you will design and develop scalable data processing solutions within AWS environments, working with large-scale and distributed data. You will take ownership of reliable data pipelines and contribute to data platform development, collaborating with engineering, AI, and product teams to support analytics, automation, and AI-enabled solutions, * Design, develop, and maintain scalable batch and distributed data processing pipelines to support reliable data processing and integration

  • Build and optimize data processing solutions, taking ownership of performance, reliability, scalability, and cost efficiency across data pipelines
  • Collaborate with engineering, AI, and product teams to understand requirements and support analytics, automation, and AI-enabled use cases
  • Develop data solutions using Python, SQL, Apache Spark, Databricks, and AWS services such as AWS Glue, Amazon Athena, and Amazon Aurora
  • Work with relational databases such as PostgreSQL and MySQL to support data processing and integration workflows
  • Develop and maintain workflow orchestration using technologies such as Apache Airflow and contribute to reliable data processing workflows
  • Contribute to CI/CD, Infrastructure as Code, testing, troubleshooting, and production support activities using Terraform or similar technologies
  • Participate in architecture discussions and contribute to the continuous improvement of data platforms and engineering practices

Requirements

  • Strong experience in Big Data, Data Engineering, or related fields, with 5+ years of professional experience
  • Strong proficiency in Python and SQL, with Java experience considered a plus
  • Strong hands-on experience with Apache Spark, distributed data processing, and Databricks
  • Solid experience developing data solutions on AWS using services such as AWS Glue, Amazon Athena, and Amazon Aurora
  • Practical experience working with relational databases such as PostgreSQL and MySQL
  • Familiarity with workflow orchestration technologies such as Apache Airflow and streaming technologies such as Apache Kafka
  • Solid experience with CI/CD, version control, and Infrastructure as Code using Terraform or similar technologies
  • Familiarity with GenAI, MLOps, machine learning workflows, or AI platform integrations is considered a plus
  • Upper-intermediate or higher English proficiency for collaboration across teams

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on arc.dev
Prepare application

Good distractions

Loading talks and stories from around this role…