Mid-Level Data Engineer 130-006

The Mid-Level Data Engineer designs, builds, and operates the scalable data pipelines, ingestion frameworks, metadata governance systems, and RDF triple store infrastructure that power DIA’s enterprise MARS OBI platform. This role sits at the operational core, ensuring that high-quality, semantically consistent, provenance-tracked intelligence data flows reliably from authoritative sources into the analytic environment. All pipelines must integrate into DIA’s IT environment.

Duties may include:

• Design, implement, and optimize scalable ETL/ELT data pipelines using Apache NiFi, Databricks, Apache Kafka, and Python to ingest, transform, normalize, and load multi-INT intelligence datasets into the MARS RDF triple store.
• Develop and optimize data queries via multiple protocols explicitly: GraphQL, SPARQL, SHACL, and SQL - enabling semantic data retrieval and reasoning across knowledge graphs.
• Develop and maintain SKOS-based semantic mapping registries per Ontology & Knowledge Modeling, aligning every source field and code value to approved OBI/DICO term URIs across all authoritative data sources.
• Build and enforce SHACL validation shapes for cardinality, datatype, value range, and relationship constraints; execute fail-fast validation at ingestion and nightly bulk reconciliation across the full graph.
• Manage PROV-O provenance tagging on all generated triples, maintaining complete data lineage from source record through transformation and graph load for every intelligence object - supporting AI documentation requirements.
• Implement version-controlled GitOps deployment of SHACL shapes, mapping rules, ontology versions, and SPARQL CONSTRUCT queries per infrastructure-as-code and containerization requirements.
• Operate and maintain enterprise RDF triple store infrastructure (GraphDB Enterprise, Apache Jena Fuseki) per secure storage and highly available distributed access requirements; monitor P95/P99 query latency.
• Monitor pipeline health, ingestion throughput, SHACL validation pass rates, and provenance completeness metrics per Advanced Analytics & Modeling requirement; report status to quality dashboards.
• Upon Government activation, provide Data Engineer support to assigned Combatant Command locations; maintain pipeline operations from COCOM environments.
• Participate in SAFe ceremonies; contribute data engineering user stories and continuously deployed pipeline improvements at each Program Increment.

• Bachelor’s degree in Computer Science, Software Engineering, or related STEM field AND 3-6 years of data engineering experience in classified IC or DoD environments
• Proficiency in Python for ETL scripting, data transformation, and pipeline automation; familiarity with DIA baseline (Python, R, SQL, ArcGIS, Tableau)
• Experience with Apache NiFi, Databricks, or equivalent enterprise data orchestration platforms
• Proficiency in SQL; experience with NoSQL/graph databases; SPARQL query development for RDF data
• Experience with SHACL constraint validation, data quality monitoring, and provenance tracking in classified data environments
• Understanding of containerization (Docker, Kubernetes) and infrastructure-as-code principles per PWS §4.1.4
• Agile/SAFe software development environment experience

Desired:

• Direct experience with RDF triple stores: GraphDB Enterprise, Apache Jena Fuseki, or equivalent at production enterprise scale
• Experience with OWL 2 reasoning, GeoSPARQL, or geospatial data engineering for spatially-enabled intelligence per PWS §4.1.4 Advanced Geospatial Analysis
• Familiarity with DIEKM/DICO standards and DIA MARS or TALOS program data architecture
• Experience with Debezium CDC, Kafka Streams, or real-time change data capture for schema change detection
• Knowledge of PROV-O, SKOS, or other W3C provenance and vocabulary standards for data lineage
• Experience with AgreementMakerLight, LogMap, or ontology matching tooling for source vocabulary alignment
• Willingness and eligibility for COCOM deployment if activated per PWS §6.3 and §10
• SAFe Agile certification

Security Clearance:

• Active TS/SCI and the willingness to sit for a polygraph, if needed