> Markdown version of [/jobs/ext/4803603-senior-data-engineer-software-engineer-sql-databricks-java-spark](https://www.wearedevelopers.com/jobs/ext/4803603-senior-data-engineer-software-engineer-sql-databricks-java-spark). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Senior Data Engineer / Software Engineer (SQL, Databricks, Java, Spark) - **Company:** Intuitive Insights Llc - **Location:** Houston, TX, United States - **Experience:** Expert - **Contract:** Temporary to permanent - **Skills:** Java (Programming Language), Artificial Intelligence, Amazon Web Services, Amazon S3, Software Applications, Automation of Tests, Unit Testing, Cloud Computing, Cloud Database, Code Generation, Software Quality, Relational Databases, Database Queries, Software Debugging, Jinja (Template Engine), Python (Programming Language), PostgreSQL, Microsoft SQL Server, Oracle (Applications), Oracle SQL Developer, Performance Tuning, Standard Sql, Secure Coding, Software Engineering, Software Systems, SQL Databases, Systems Architecture, Toolchain, Transact-SQL, Macros, Snowflake, Apache Spark, Spring-boot, Caching, AI Coding Agents, Data Lakes, Kubernetes, Production Code, Maintaining Code, Apache Kafka, Docker, Databricks, Programming Languages - **Published:** November 20, 2026 - **Apply:** https://www.dice.com/job-detail/47d14080-90f9-4a5f-b636-ea8d675b71be ## About the Role * Formal training or certification on software engineering concepts and 8+ years of applied experience * Hands-on practical experience in system design, application development, testing, and operational stability * Proficient in coding (Java or Python) * Strong knowledge of relational databases/ Transact SQL, caching technologies (Oracle/Postgres/SQL Server/elastic etc), Kafka, Apache Spark * Working knowledge of Databricks, AWS * Strong Experience in developing, debugging, and maintaining code in a large corporate environment with one or more modern programming languages and database querying languages * Overall knowledge of the Software Development Life Cycle * Hands-on experience using enterprise-authorized AI-assisted software development tools within the work environment (e.g., for coding, test creation, troubleshooting, or documentation) with demonstrated ability to critically evaluate, validate, and refine AI-generated outputs for correctness, performance, and security. * Understanding of responsible AI use in engineering workflows, including data sensitivity considerations, secure handling of inputs/outputs, and adherence to resiliency and security expectations; ability to guide peers on safe and effective usage within team practices. Preferred qualifications, capabilities, and skills * Exposure to cloud technologies (e.g. AWS, Databricks, Snowflake), * - >10-12 years of experience with strong Expertise Oracle PL/SQL. You can read and reason about complex legacy packages, views and procedural logic. * - Strong SQL and Databricks SQL. Good knowledge of Delta Lake (MERGE, time travel, partitioning, table versus view behaviour) and Unity Catalog. * - Hands-on dbt: models, snapshots, macros, Jinja, hooks, unit tests, and multi-environment configuration. * - Experience in Java, Scala/Spark, Spring Boot, Kafka Messaging, Kubernetes, Docker, S3, DeltaLake, Dremio * - Proven data modelling: dimensional and SCD2 history, grain design, keys, and point-in-time / as-of queries. * - Performance tuning on Spark or Databricks: query plans, join strategy, caching and materialization, partition pruning. * - Experience of legacy-to-cloud data migrations with formal parity and reconciliation. * - Strong written communication: you can turn code into clear business-rule documentation. ## Description * Participate in modernization and upgrade of the Credit Risk technology stack in accordance with the industry best practices * Executes software solutions, design, development, and technical troubleshooting with ability to think beyond routine or conventional approaches to build solutions or break down technical problems * Creates secure and high-quality production code and maintains algorithms that run synchronously with appropriate systems * Produces architecture and design artifacts for complex applications while being accountable for ensuring design constraints are met by software code development * Gathers, analyzes, synthesizes, and develops visualizations and reporting from large, diverse data sets in service of continuous improvement of software applications and systems * Proactively identifies hidden problems and patterns in data and uses these insights to drive improvements to coding hygiene and system architecture * Contributes to software engineering communities of practice and events that explore new and emerging technologies * Leverages enterprise-authorized AI coding assist tools within the work environment to improve code quality, delivery speed, and productivity across complex deliverables (e.g., code generation/refactoring, unit test creation, documentation), while validating outputs through peer review, automated testing, and secure coding standards; contributes learnings and reusable patterns to improve broader team effectiveness. * Applies knowledge of tools within the Software Development Life Cycle toolchain, including enterprise-authorized AI-assisted development and automation capabilities, to improve the value realized by automation.