Big Data Engineer

We are seeking an experienced Big Data Engineer to design, build, and optimize large-scale data processing systems. This role partners closely with engineering, data, and analytics teams to deliver scalable, reliable data pipelines that support data-driven decision-making. The ideal candidate has strong experience with distributed systems, cloud platforms, and modern big data technologies., * Design, develop, and maintain scalable data pipelines using technologies such as Spark, Hadoop, Python, and Scala.

• Build and optimize data ingestion, transformation, and storage solutions for large-scale datasets.
• Optimize existing data pipelines for performance, scalability, and reliability.
• Collaborate with cross-functional teams to translate business requirements into technical solutions.
• Monitor, troubleshoot, and resolve data pipeline issues in production environments.
• Implement automated testing and data quality checks across pipelines.
• Stay current with emerging big data and cloud technologies to continuously improve the platform.
• Support data scientists and analysts by enabling reliable access to high-quality data.

• Bachelor's degree in Computer Science or a related field, or equivalent experience.
• 5 years of experience building enterprise-scale data solutions.
• Strong experience with big data technologies such as Spark, Hadoop, Hive, and Trino.
• Proficiency in Python or Scala, with solid object-oriented and/or functional programming skills.
• Strong SQL skills, including complex queries, joins, and window functions.
• Experience working with large datasets and troubleshooting performance, scalability, and data quality issues.
• Experience with cloud platforms, preferably AWS (e.g., S3, EMR, Glue, Athena, Lambda).
• Familiarity with Agile development practices and CI/CD pipelines.
• Strong communication skills and ability to work effectively in a fast-paced environment.

Preferred Qualifications

• Experience managing production ETL or data pipeline systems.
• Exposure to AI-assisted development tools (e.g., Copilot, ChatGPT, Q Developer).
• Experience with Spark performance tuning and optimization.
• AWS certifications or equivalent cloud experience.

Welcome to ConsultNet, a premier national provider of technology talent and solutions. Our expertise spans across project services, contract-to-hire, direct search, and managed services onshore, nearshore, and hybrid. For over 25 years, we have connected thousands of consultants with meaningful roles through a personal, communication-driven approach, partnering with a diverse client base to build high-performing teams and create lasting impact. Our comprehensive service offerings cover a wide range of technology and engineering positions across key markets nationwide. Learn more at www.consultnet.com .