Jr. Big Data Engineer

• Build and maintain data processing pipelines using Python and Apache Spark with guidance from senior engineers.
• Use AI development tools to accelerate coding, testing, debugging, documentation, and understanding of unfamiliar codebases.
• Review and validate AI-generated code for accuracy, maintainability, edge cases, and potential errors.
• Write automated unit and integration tests to support data quality and pipeline reliability.
• Troubleshoot pipeline failures and use AI-assisted development techniques to accelerate root-cause analysis.
• Rapidly prototype solutions and develop lightweight tools that automate repetitive engineering tasks.
• Work with AWS-based data environments, including Amazon S3 and related cloud services.
• Document technical solutions and share effective AI development workflows, prompts, and engineering practices with the team.

Big Data, Prototyping, Programming Tools, Big Data, Artificial Intelligence (AI), Problem Solving Skills, Software Engineering, Python Programming/Scripting Language, SQL (Structured Query Language), Data Management, We are seeking an early-career Big Data Engineer who takes an AI-first approach to software and data engineering. This opportunity is ideal for someone with approximately 1 2 years of experience who actively uses AI development tools to accelerate coding, problem-solving, testing, and prototyping.

You will work alongside experienced engineers to build and maintain scalable data pipelines using Python, Spark, SQL, and AWS. Deep expertise in big data technologies is not required; we are looking for strong technical fundamentals, curiosity, rapid learning ability, and the judgment to critically evaluate AI-generated output.

We are particularly interested in candidates who have built projects independently whether professionally, academically, through open source, or personally and can explain how they used AI to accelerate development, what challenges they encountered, and how they validated the results., * Bachelor’s degree in Computer Science, Information Systems, Engineering, or a related technical discipline.

• Approximately 1 2 years of relevant experience. Internships, co-ops, research, open-source contributions, and substantial personal projects may be considered.
• Master’s degree in a related discipline may substitute for professional experience.
• Solid programming fundamentals using Python, including the ability to write clean, readable, maintainable code.
• Working knowledge of SQL, including joins, aggregations, filtering, NULL handling, and duplicate management.
• Exposure to Apache Spark and AWS, including Amazon S3.
• Familiarity with Git, source control, and code-review practices.
• Strong problem-solving, communication, and learning skills.

AI Development Skills

• Hands-on experience using AI development tools such as GitHub Copilot, Amazon Q Developer, ChatGPT, Claude, Cursor, Kiro, or similar tools.
• Ability to use prompt engineering and iterative prompting to improve technical outputs.
• Ability to critically evaluate AI-generated code and identify issues such as incorrect logic, edge-case failures, hallucinated APIs, and other subtle errors.
• Experience using AI to rapidly move from an idea or requirement to a functional prototype.
• Ability to explain where AI-assisted development was effective and where human judgment or correction was required.

Project Experience

Candidates should be prepared to discuss at least one technical project in detail, including:

• What you built and why.
• Technologies used.
• How AI tools contributed to the development process.
• Problems or incorrect AI outputs encountered and how they were resolved.
• What you would approach differently today.

Independent projects, hackathons, open-source contributions, research projects, and self-directed technical work are highly valued.

Preferred Qualifications

• Exposure to Hive, Trino, Presto, or similar distributed query technologies.
• Familiarity with CI/CD pipelines.
• Experience working within Agile, Scrum, or Kanban teams.
• Experience with AI agents, MCP servers, RAG, or LLM APIs.
• Participation in hackathons, Kaggle competitions, technical blogging, or open-source development.
• AWS or related cloud certifications.
• Experience working with large or distributed datasets.
• Financial services industry exposure is a plus.

Welcome to ConsultNet, a premier national provider of technology talent and solutions. Our expertise spans across project services, contract-to-hire, direct search, and managed services onshore, nearshore, and hybrid. For over 25 years, we have connected thousands of consultants with meaningful roles through a personal, communication-driven approach, partnering with a diverse client base to build high-performing teams and create lasting impact. Our comprehensive service offerings cover a wide range of technology and engineering positions across key markets nationwide. Learn more at www.consultnet.com .