> Markdown version of [/jobs/ext/1499034-spark-scala-senior-engineer](https://www.wearedevelopers.com/jobs/ext/1499034-spark-scala-senior-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Spark / Scala Senior Engineer - **Company:** TalentOla View all jobs - **Location:** Jacksonville, FL, United States - **Experience:** Expert - **Salary:** $90,000.0 - $140,000.0 - **Contract:** Permanent contract - **Skills:** Agile Methodology, Automation of Tests, Code Review, Continuous Integration, Information Engineering, Data Warehousing, Apache Hive, Operational Databases, Performance Tuning, Query Optimization, Standard Sql, Scala (Programming Language), Simple Data Format, Data Streaming, Enterprise Data Management, Parquet, Data Processing, Data Ingestion, Snowflake, Apache Spark, Caching, Git, Data Lakes, Avro, Real Time Data, Apache Kafka, Apache Nifi, Spark Streaming, Functional Programming, Software Coding, Stream Processing, Software Version Control, Data Pipelines, Docker - **Published:** July 30, 2026 - **Apply:** https://www.careerjet.com/jobad/us0f48e28ad032c3ae13b5246a1d25c2e5 ## About the Role We are looking for a highly skilled Spark/Scala Senior Engineer with 8 to 10 years of experience in designing, developing, and optimizing large-scale batch and real-time data processing solutions. The ideal candidate will possess strong expertise in Scala, Apache Spark, Snowflake, Kafka/NiFi, and modern Data Engineering frameworks. This role requires hands-on technical leadership, performance optimization expertise, and the ability to mentor junior team members while driving engineering best practices, * Collections Framework * Implicits * Strong knowledge of Functional Programming concepts: * Immutability * Higher-Order Functions (HOFs) * Monads * Experience developing and maintaining production-grade Scala applications. Apache Spark * Strong expertise with: * Spark SQL * DataFrames * Datasets * Extensive experience with Spark Structured Streaming. * Deep understanding of Spark internals: * DAG Execution * Lazy Evaluation * Shuffle Operations * Stage Execution * Experience optimizing large-scale Spark workloads. * Hands-on experience with: * Parquet * Avro * ORC * Delta Lake * Apache Kafka * Apache NiFi Snowflake * Experience designing and querying Snowflake data warehouse schemas. * Familiarity with: * Snowflake Stages * File Formats * Streams * Tasks * Knowledge of Snowpark for Scala is preferred. Data Engineering & Development * Strong SQL and query optimization skills. * Experience with Git-based version control workflows. * Familiarity with Docker and containerized application development. * Experience working within Agile development environments., Engineering Degree BE/ME/BTech/MTech/BSc/MSc. Technical certification in multiple technologies is desirable. Good to have skills: - Pharmacy Domain ## Description Design, develop, and optimize large-scale batch and streaming data pipelines using Scala and Apache Spark. Build robust data processing solutions using Spark SQL, DataFrames, and Datasets. Develop and maintain Spark Structured Streaming applications for real-time data ingestion and processing. Integrate data pipelines with enterprise data platforms such as Snowflake. Implement scalable data ingestion frameworks utilizing Apache Kafka and/or Apache NiFi. Perform Spark performance tuning, including: Partition optimization Caching strategies Shuffle optimization Broadcast joins Resource allocation tuning Develop clean, maintainable, and testable Scala code following functional programming principles. Participate in code reviews and establish coding standards and best practices. Create automated tests and support CI/CD pipeline improvements. Monitor production data pipelines and troubleshoot performance or data quality issues. Conduct root cause analysis and implement preventive measures. Lead technical and architectural discussions across teams. Mentor junior engineers and provide technical guidance. Collaborate with cross-functional stakeholders to deliver scalable data solutions. Required Technical Skills Scala * Strong hands-on experience with Scala development. * Deep understanding of: * Case Classes * Traits ## Related Videos - [Flex your Energy: Building a Cloud-Native Platform for Renewable Energy Communities](https://www.wearedevelopers.com/videos/1990-flex-your-energy-building-a-cloud-native-platform-for-renewable-energy-communities) - [Docker Compose: Rediscovered](https://www.wearedevelopers.com/videos/1978-docker-compose-rediscovered) - [From event streaming to event sourcing 101](https://www.wearedevelopers.com/videos/91-from-event-streaming-to-event-sourcing-101) - [How a Small Team Shrank a Microsoft Monorepo by 94%](https://www.wearedevelopers.com/videos/1236-how-a-small-team-shrank-a-microsoft-monorepo-by-94) - [Docker build without Docker](https://www.wearedevelopers.com/videos/100114-docker-build-without-docker) - [Introducing JSON Structure](https://www.wearedevelopers.com/videos/100219-introducing-json-structure) ## Related Articles - [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers) - [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story) - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk) - [Is Software Engineering Over-Saturated?](https://www.wearedevelopers.com/magazine/418-is-software-engineering-over-saturated) - [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know) - [Fully Remote Software Engineer Jobs](https://www.wearedevelopers.com/magazine/447-fully-remote-software-engineer-jobs)