> Markdown version of [/jobs/ext/2572552-data-engineer](https://www.wearedevelopers.com/jobs/ext/2572552-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Data Engineer - **Company:** Eli Lilly and Company - **Location:** Indianapolis, IN, United States - **Experience:** Expert - **Salary:** $64,500.0 - $158,400.0 - **Contract:** Permanent contract - **Skills:** Java (Programming Language), Agile Methodology, Artificial Intelligence, Amazon Web Services, Amazon S3, Business Analytics Applications, JIRA, Microsoft Azure, BigQuery, Code Review, Information Systems, Data as a Services, Data Architecture, Information Engineering, Data Governance, Extract Transform Load (ETL), Data Transformation, Data Systems, Data Warehousing, Github, Python (Programming Language), Machine Learning, Meta-Data Management, Scrum Methodology, Cloud Services, Azure Data Lake, SQL Databases, Data Streaming, Policy as Code, Data Storage Management, Google Cloud, Data Ingestion, System Availability, Snowflake, Git, Pytest, Data Lakes, Pyspark, Information Technology, Data Pipelines, Serverless Computing, GXP, Amazon Redshift, Databricks, Programming Languages - **Published:** August 25, 2026 - **Apply:** https://www.biospace.com/logon?PipelinedPage=%2Fjob%2F3070338%2Fdata-engineer%3FAction%3DContinueJobApplication%23application-form ## About the Role * At least a Bachelor's degree in Computer Science, Engineering, Information Systems, or a related quantitative field. * 5+ years of experience in data engineering, ETL development, or a similar role. * Proficiency in SQL and at least one programming language (e.g., Python, Java, hands-on Databricks). * Experience with cloud data platforms (e.g., Databricks, AWS, Azure, GCP) and their associated data services (e.g., S3, Redshift, Snowflake, Azure Data Lake Storage, BigQuery). * Proficiency with Git-based CI/CD workflows (Git Actions or equivalent) for versioned data and pipeline artifacts. Qualified applicants must be authorized to work in the United States on a full-time basis now and in the future. Lilly will not provide support for or sponsor work authorization or visas for this role, including but not limited to F-1 CPT, F-1 OPT, F-1 STEM OPT, J-1, H-1B, TN, O-1, E-3, H-1B1, or L-1. What You Should Bring: * Excellent problem-solving skills; ability to translate architectural designs into working, tested pipelines. * Good communication and collaboration skills - able to work effectively with architects, product owners, and multi-functional partners. * Prior experience in the pharmaceutical or life sciences industry is preferred. ## Description Lilly is seeking a highly motivated and skilled Data Engineer to join our innovative team at Eli Lilly and Company. This role involves designing, building, and maintaining robust and scalable data pipelines and infrastructure to support our critical scientific and business initiatives, ultimately contributing to the discovery and development of life-changing medicines. What You Will Do: Data Engineering & Pipeline Development * Design, develop, and optimize scalable data pipelines using Databricks, PySpark, Python, SQL, and Delta Lake to ingest, transform, and load data from diverse sources into data warehouses and data lakes. * Build scalable, efficient Databricks pipelines implementing canonical data models across the medallion architecture (Bronze * Silver * Gold), scoped entirely within the CE trust boundary. * Evaluate and apply Databricks capabilities and integration patterns - Unity Catalog, Delta Lake, Databricks Workflows, serverless compute, Lakebase, and ingestion connectors - selecting the right tool for each pipeline given performance, cost, and scalability constraints. * Implement and maintain ELT/ETL workflows using Databricks Workflows, Auto Loader, Structured Streaming, and Delta Live Tables (DLT). * Build and maintain CI/CD pipelines (GitHub Actions, Git-based promotion dev * test * prod) for CE data, contract, policy, and agent artifacts. * Automate data ingestion and product creation to reduce manual pipeline maintenance and onboarding time for new CE data sources. Data Governance, Quality & Security * Implement and manage data governance policies, ensuring data quality, integrity, security, and compliance with regulatory requirements (e.g., GxP, HIPAA) and covered-entity constructs. * Implement row/column-level security, masking, and tokenization boundaries so PHI isolation is enforced at the platform layer, in partnership with the Policy-as-Code Engineer's OPA/Rego policies. * Support implementation of data governance capabilities including metadata management, lineage, and access control using Unity Catalog. * Establish and implement data quality, testing, and validation methodology (pytest, DLT/Great Expectations); build monitoring and alerting to proactively catch and resolve pipeline and data issues. Data Modeling & Architecture * Develop and maintain data models, schemas, and metadata for efficient data storage and retrieval. * Design data solutions following Lakehouse and Medallion Architecture (Bronze, Silver, Gold) design principles. * Develop reusable data transformation frameworks and automated data quality checks. * Partner with the CE Data Architect on reference architecture and patterns, providing implementation feedback that keeps designs buildable and performant at scale. * Develop and maintain documentation for data architecture, pipelines, and processes. Collaboration & Innovation * Collaborate with data scientists, analysts, architects, and business stakeholders to understand data requirements and translate them into scalable technical solutions. * Monitor data pipeline performance, troubleshoot issues, and implement solutions to ensure high availability and reliability. * Participate in code reviews, contribute to architectural discussions, and promote best practices in data engineering. * Evaluate and recommend new data technologies and tools to enhance the enterprise data platform capabilities. * Support the integration of machine learning models, AI Skills, and analytical tools into production environments - ensuring PHI classification and consent travel with the data into agentic consumption paths. * Participate in Agile ceremonies, sprint planning, and testing activities using Jira or equivalent tooling. ## Related Videos - [Blueprints for Success: Steering a Global Data & AI Architecture](https://www.wearedevelopers.com/videos/1577-blueprints-for-success-steering-a-global-data-ai-architecture) - [Alibaba Big Data and Machine Learning Technology](https://www.wearedevelopers.com/videos/37-alibaba-big-data-and-machine-learning-technology) - [Improving quality with Agentic AI with Rovo Dev and Xray](https://www.wearedevelopers.com/videos/2005-improving-quality-with-agentic-ai-with-rovo-dev-and-xray) - [How a Small Team Shrank a Microsoft Monorepo by 94%](https://www.wearedevelopers.com/videos/1236-how-a-small-team-shrank-a-microsoft-monorepo-by-94) - [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works) - [Collaboration Quantified: Lessons from Open Source Developer Networks](https://www.wearedevelopers.com/videos/1422-collaboration-quantified-lessons-from-open-source-developer-networks) ## Related Articles - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk) - [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer) - [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers) - [The Most Popular IT Jobs on the Market](https://www.wearedevelopers.com/magazine/376-the-most-popular-it-jobs-on-the-market) - [Top-Paying Tech Jobs (with Salaries)](https://www.wearedevelopers.com/magazine/372-top-paying-tech-jobs-with-salaries) - [Data Analyst Salary in the UK](https://www.wearedevelopers.com/magazine/278-data-analyst-salary-in-the-uk)