Data Engineer

Are you passionate about building scalable data pipelines and working with complex datasets in cloud environments? NS2 Mission is seeking a skilled Data Engineer to support mission-focused programs in Chantilly, VA. In this role, you will design and maintain data pipelines, collaborate closely with development teams, and help deliver reliable data solutions that enable teams to transform data into actionable insights., * Design, build, and maintain scalable data pipelines that support the ingestion, transformation, and delivery of large datasets

• Collaborate with cross-functional development teams to integrate data solutions into applications and mission systems
• Process and transform data using technologies such as Spark within cloud-based environments
• Develop and optimize SQL queries to support data retrieval, transformation, and analysis
• Work with relational databases including PostgreSQL, Oracle, and MySQL to manage and access structured data
• Create automation and processing scripts using Python, Linux, and shell scripting
• Analyze and work with datasets across multiple formats including CSV, XML, JSON, Avro, and Parquet
• Write and maintain unit tests to validate data processing logic and ensure code quality
• Monitor, troubleshoot, and improve the performance and reliability of data pipelines and workflows

• 4+ years of experience building and managing data pipelines
• 4+ years working within highly collaborative development teams
• 2+ years of hands-on experience with Python
• Experience working in AWS cloud environments
• Experience processing data using Spark
• Experience working with relational databases such as PostgreSQL, Oracle, or MySQL, including writing SQL queries
• Experience working in Linux environments, including shell scripting
• Experience analyzing data across multiple formats such as CSV, XML, JSON, Avro, and Parquet
• Experience writing and maintaining unit tests, * Experience using Apache NiFi, Apache Airflow, or similar orchestration tools to manage data pipelines
• Development experience with Java or Scala
• Experience administering or supporting EMR or Spark clusters
• Background in performance tuning Spark jobs
• Experience working with Hive, Iceberg, or other technologies that provide SQL-based access to large datasets
• Familiarity with implementing cloud-based security solutions
• Experience following configuration management processes for code review and release deployments

Technical Environment

• Python * AWS * Spark * SQL * PostgreSQL * Oracle * MySQL * Linux * Shell Scripting * Apache NiFi * Apache Airflow * Java * Scala * Hive * Iceberg * EMR * JSON * XML * CSV * Avro * Parquet

Position Clearance Requirement:

TS/SCI with an FSP

Please be aware, many of our positions require the ability to obtain or maintain a U.S security clearance which requires U.S citizenship

NS2 Mission's culture of inclusion, focus on health and well-being, and flexible working models help ensure that everyone - regardless of background - feels included and can run at their best. At NS2 Mission, we believe we are made stronger by the unique capabilities and qualities that each person brings to our company, and we invest in our employees to inspire confidence and help everyone realize their full potential. We ultimately believe in unleashing all talent and creating a better world.