Databricks Engineer

As a Databricks Engineer, you’ll work closely with engineering and business leaders to design and build data integration solutions that drive enterprise analytics and decision-making. You’ll leverage cloud technologies, CI/CD pipelines, and Power BI to deliver scalable, high-performance data solutions. The ideal candidate excels in ETL programming, data integration, and data modeling, enabling self-service analytics and ensuring data accessibility, governance, and integrity. You’ll be part of an agile team, participating in Scrum ceremonies and delivering continuous value to stakeholders., * As a technical member, collaborate with a small team of data engineers, driving excellence in data platform evolution. Promote a culture of innovation, accountability, and continuous improvement.

• Work with the Lead on the technical planning and requirements gathering.
• Serve as a technical lead and mentor. Provide technical support and leadership in the development and continual improvement of service.
• Develop and maintain effective working relationships with team members.
• Demonstrate the ability to adapt and work with team members of various experience levels.
• Work with the data engineering team to define and develop data ingestion, validation, transformation, and data engineering code.
• Lead the design, development, testing, and deployment of scalable batch/streaming data pipelines using Databricks. Data jobs include data ingestion, cleansing, transformation, aggregation, governance, and sharing.
• Define and enforce best practices for Databricks usage, including notebook development, job orchestration, metadata-driven frameworks, data quality checks, and optimization of Spark job performance.
• Develop Databricks platform components using Python, SQL, PySpark, Notebook Repositories (medallion architecture L2, L3, L4), and CI/CD.
• Integrate Databricks solutions with cloud platforms like Azure (ADF, ADLS, Event Hub, Azure DevOps) or AWS (S3, Lambda, Glue).
• Design and implement analytical data models and star schemas for BI and AI usage.
• Collaborate with data scientists and analysts to meet data requirements.
• Implement logging, monitoring, and exception handling for data workflows.
• Participate in code reviews and CI/CD pipeline development.
• Document code artifacts and participate in developing user documentation and runbooks
• Troubleshoot deployment to various environments and provide test support.
• Participate in design sessions, demos, and prototype sessions, testing, and training workshops with business users and IT associates, Build and operate scalable Databricks-on-AWS data pipelines using PySpark, Delta Lake, and lakehouse patterns. Optimize performance, implement data quality, monitoring, alerting, and automated remediation, and deliver curated datasets for BI and analytics partners. Collaborate with stakeholders on architecture and design while applying secure software engineering, CI/CD, agile, and operational stability practices. The role also uses AI-assisted development tools and supports workforce data analytics. Top Skills: AlteryxAmazon AthenaAmazon EmrAmazon S3Apache AirflowApache IcebergSparkAutosysAWSAws CloudwatchAws GlueAws LambdaBitbucketClaudeDatabricksDatabricks WorkflowsDelta LakeDelta Live TablesGitGithub CopilotJavaJenkinsOracleParquetPysparkPythonScalaSigmaSpinnakerSQLTableau CNX

• Passionate developer with 5+ years of data engineering experience, with at least 3 years in a lead or senior data engineer role.
• Minimum 3 years of hands-on Databricks experience using Python, SQL, Pyspark, Notebook Repositories (medallion architecture L2, L3, L4), CI/CD.
• Minimum 3 years of Databricks experience in developing large-scale data processing/data storage/data distribution systems.
• Experience with AWS and/or Azure cloud environment.
• Experience with various Notebook Repositories (medallion architecture L2, L3, L4) as well as unit testing frameworks.
• DevOps experience with Databricks CI/CD (Databricks Asset Bundles) and Git
• Minimum 3 years of experience working in a large multi-vendor environment with multiple teams as a part of a project
• Minimum 5 years of working with a complex Big Data/Lakehouse environment
• Minimum of 5 years of experience with JIRA/GitHub/Git and other code management toolsets
• Experience with ELT/ETL development, patterns, and tooling is recommended
• Experience with SQL, including Postgres, RDBMS platforms
• Experience with various IDE and code repositories as well as unit testing frameworks.
• Demonstrated ability to leverage AI tools to enhance productivity, streamline workflows, and support data-informed task execution.
• Familiarity with AI-enhanced platforms is a plus.
• A solid understanding of AI capabilities and limitations, including ethical considerations, is expected.
• Flexible and adaptable attitude, disciplined to manage multiple responsibilities and adjust to varied environments.
• Ability to produce high-quality products within deadlines

• Bachelor’s degree in Computer Science, Engineering, Information Systems, or a related field. Preferred Skills & Education:

• Certification in Databricks, Spark, AWS, Azure, or other cloud platforms.
• Experience with ETL/ELT data platforms, including Informatica, DataStage, Fivetran, Matillion, and dbt.
• Familiarity with ML capabilities such as MLflow, TensorFlow, or Scikit-learn is a plus.
• Master’s degree in Computer Science or related field.
• Client-facing or consulting experience is highly preferred.

In-Office or Remote 2 Locations 102K-204K Annually Senior level 102K-204K Annually Senior level Fitness * Healthtech * Retail * Pharmaceutical Build and operate an Azure Databricks data platform using Python frameworks, configuration-driven pipelines, CI/CD, Terraform, and infrastructure as code. Support production reliability through monitoring, incident response, root cause analysis, runbooks, disaster recovery, and automation. Maintain data security and governance with Unity Catalog, enable domain teams to onboard data sources, and improve self-service platform capabilities in a regulated healthcare environment. Top Skills: Adls Gen2SparkAzureAzure Key VaultCi/CdDatabricksDatabricks Asset BundlesDelta LakeDelta Live TablesGitGithub ActionsInfrastructure As CodeLakeflowMicrosoft Entra IdPysparkPytestPythonSQLTerraformUnity Catalog JPMorganChase, 5 Days Ago In-Office or Remote 106K-152K Annually Senior level 106K-152K Annually Senior level Energy Owns the end-to-end data lifecycle by designing, building, orchestrating, governing, and optimizing batch and streaming pipelines on Databricks. Develops lakehouse architectures, data models, quality controls, CI/CD practices, and secure Unity Catalog implementations across Azure and other cloud platforms. Leads technical design and code reviews, establishes engineering standards, mentors data engineers, monitors production pipelines, resolves issues, and collaborates with business and IT stakeholders to deliver analytics, reporting, and AI/ML data solutions. Top Skills: Adls Gen2Auto LoaderAWSAzure Data FactoryAzure DevopsAzure Event HubsAzure Key VaultDatabricksDatabricks Asset BundlesDatabricks WorkflowsDeclarative Automation BundlesDelta LakeDelta Live TablesGitGit FoldersGithub ActionsGoogle Cloud PlatformLakeflow ConnectLakeflow JobsLakeflow Spark Declarative PipelinesAzureMicrosoft Entra IdPandas Api On SparkPysparkPythonSpark SqlSQLUnity Catalog

What you need to know about the Colorado Tech Scene

With a business-friendly climate and research universities like CU Boulder and Colorado State, Colorado has made a name for itself as a startup ecosystem. The state boasts a skilled workforce and high quality of life thanks to its affordable housing, vibrant cultural scene and unparalleled opportunities for outdoor recreation. Colorado is also home to the National Renewable Energy Laboratory, helping cement its status as a hub for renewable energy innovation.

Key Facts About Colorado Tech

• Number of Tech Workers: 260,000; 8.5% of overall workforce (2024 CompTIA survey)
• Major Tech Employers: Lockheed Martin, Century Link, Comcast, BAE Systems, Level 3
• Key Industries: Software, artificial intelligence, aerospace, e-commerce, fintech, healthtech
• Funding Landscape: $4.9 billion in VC funding in 2024 (Pitchbook)
• Notable Investors: Access Venture Partners, Ridgeline Ventures, Techstars, Blackhorn Ventures
• Research Centers and Universities: Colorado School of Mines, University of Colorado Boulder, University of Denver, Colorado State University, Mesa Laboratory, Space Science Institute, National Center for Atmospheric Research, National Renewable Energy Laboratory, Gottlieb Institute

Perficient is a global AI-first consultancy. Our team of strategists, designers, technologists, and engineers partners with the world’s most innovative enterprises and admired brands to deliver real business results through the transformative power of AI. As part of our AI-First strategy, we empower every employee to build AI fluency and actively engage with AI tools to drive innovation and efficiency. We break boundaries, obsess over outcomes, and shape the future for our clients. Join a company where bold ideas and brilliant minds converge to redefine what’s possible - while building a career filled with growth, balance, and purpose., Perficient is always looking for the best and brightest talent, and we need you! We’re a quickly growing, global digital consulting leader, and we’re transforming the world’s largest enterprises and biggest brands. You’ll work with the latest technologies, expand your skills, and become a part of our global community of talented, diverse, and knowledgeable colleagues., About UsPerficient is the global AI and technology consulting firm disrupting the traditional consulting model. Powered by our 7,000+ advisors, engineers, and designers, Perficient implements AI-first solutions that break conventions and deliver outcomes that matter. Proudly serving clients that represent the world’s most innovative brands, and in collaboration with our powerful technology partner ecosystem, we bring deep industry expertise and data-driven design to redefine how businesses run and succeed. Perficient is different. For real. Learn more at perficient.com.