Lead - Data / AI Native Engineer

kastech software solutions
O'Fallon, MO, United States
5 days ago

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Experience required
3 years minimum
Working hours
Regular working hours
Job source

Tech stack

Artificial Intelligence Amazon Web Services Data Analysis Cloud Engineering Information Engineering Extract Transform Load (ETL) Data Systems Programming Tools Software Tools Software Engineering Data Ingestion GitHub Copilot
+6 more
Apache Spark Pyspark Data Management Data Pipelines Amazon Elastic Mapreduce (EMR) Databricks

Requirements

  • 8+ years of Data Engineering and Software Engineering experience.
  • 3+ years leading complex data engineering initiatives or engineering teams.
  • Proven experience building enterprise scale data platforms and pipelines.
  • Experience delivering cloud native analytics and AI solutions.

Technical Skills

  • Build highly available, secure, and scalable cloud native data solutions.
  • Develop ETL/ELT frameworks for ingesting, transforming, and publishing enterprise data.
  • Implement high volume distributed processing using Spark and PySpark.
  • Leverage GitHub Copilot, Claude Code CLI, and AI assisted development tools to accelerate engineering productivity.
  • Build Proof of Concepts (POCs) and MVPs using AI powered development approaches.
  • Develop cloud native solutions leveraging AWS and Databricks ecosystems.
  • Build scalable architectures for data ingestion, processing, storage, and analytics.

Key Expectations

  • Strong experience in building enterprise grade data pipelines, modern Lakehouse architectures, AI enabled data products, and cloud native platforms leveraging Databricks, AWS EMR, Spark, PySpark, and AI native engineering tools such as GitHub Copilot and Claude Code.
  • The role aligns with enterprise initiatives around AI assisted engineering, intelligent automation, and AI powered development workflows

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.dice.com

Good distractions

Talks and stories from around this role — technically off-topic, practically not.

4:32 min

Harnessing Spark with Python using PySpark and Py4J

Ayon Roy · LIVE

2:27 min

Managing traffic and tracking costs with Databricks Unity Catalog

Viktoria Semaan Viktoria Semaan · WWC Europe 2026

2:37 min

Classifying and anonymizing data during system design

Reto Kaeser · LIVE

3:03 min

Career evolution in data engineering and AI platforms

Maria Apazoglou · Coffee With Developers

4:15 min

Bridging operational and analytical systems using formal data contracts

Matthias Niehoff Matthias Niehoff · WWC 2024

2:50 min

Executing LoRA fine-tuning using serverless Databricks AI runtimes

Viktoria Semaan Viktoria Semaan · WWC Europe 2026

Videos

See all

Related articles

See all