Data Engineer

• 3-10 years of directly related experience and B.A./B.S. or M.A./M.S. in a related field. \n

• Experience building within Palantir Foundry applications, including Workshop, Ontology Manager, and Gaia. \n

• Experience with Machine Learning models and/or Generative AI. \n

• Experience with APIs. \n

• Previous experience with Advana or the War Data Platform. \n

• Experience with the DoW mission and associated datasets. \n

• Understanding of data modeling/visualization, database design principles, and data governance practices. \n

• Experience working with large datasets and leveraging big data technologies to process and analyze data efficiently. \n

• Proficiency in ETL processes and firsthand experience with technologies, such as Databricks, Python, Spark, Scala, JavaScript/JSON, SQL, and Jupyter Notebooks. \n

• 3+ years of experience as a Data Engineer. \n, n \n

• Experience with Machine Learning models and/or Generative AI. \n

• Experience with APIs. \n

• Previous experience with Advana or the War Data Platform. \n

• Experience with the DoW mission and associated datasets. \n

• Understanding of data modeling/visualization, database design principles, and data governance practices. \n

• Experience working with large datasets and leveraging big data technologies to process and analyze data efficiently. \n

• Proficiency in ETL processes and firsthand experience with technologies, such as Databricks, Python, Spark, Scala, JavaScript/JSON, and SQL. \n

• Analyze science, engineering, business, and other data processing problems to develop and implement solutions to complex applications problems, system administration issues, or network concerns. \n

• Perform systems management and integration functions, improve existing computer systems, and review computer system capabilities, workflow, and schedule limitations. \n

• May analyze or recommend commercially available software. \n

• Implement and enforce data governance policies and procedures, ensuring compliance with regulatory requirements and industry best practices. \n

• Troubleshoot data-related problems and provide innovative solutions to address complex data challenges. \n

• Collaborate with data scientists, analysts, and stakeholders to provide timely and accurate data insights and support decision-making processes. \n

• Leverage big data technologies to efficiently process and analyze large datasets. \n

• Perform data cleansing, data integration, and data quality assurance activities to maintain the accuracy and integrity of large datasets. \n

• Create and optimize data pipelines from scratch, ensuring scalability, reliability, and high-performance processing. \n

• Work closely with cross-functional teams to understand data requirements and design optimal data models and architectures. \n

• Design, develop, and implement end-to-end data pipelines, utilizing ETL processes and technologies such as Databricks, Python, Spark, Scala, JavaScript/JSON, and SQL. \n

\n

\n