Software & Interoperability Test Engineer
Role details
Job location
Tech stack
Job description
- Deliverable name: ADatP-5658 standard refinement engagement report.
- Output: Engagement Report summarizing the participation in the refinement activities of ADatP-5658 standard.
- Outcome: An improved and more consistent ADatP-5658 standard definition that better supports interoperability, implementation, and alignment with the standard's objectives. § Acceptance criteria and key performance indicators (KPIs):
o The report includes key decisions, action items and the Contractor's contributions
o The Contractor's contributions support the clarification, improvement, or extension of the ADatP-5658 standard.
- Key performance indicators (KPIs):
o Report Completeness (%):
- Inclusion of all required sections (decisions, actions, contributions)
- Target = 100%
o Contribution Relevance Score (1-5):
-
Stakeholder evaluation of usefulness of contributions
-
Target 4.0
-
Service: Reviewing and update the existing conformance criteria of ADatP-5658 with the aim of achieving completeness and consistency.
- Deliverable name: Updated ADatP-5658 conformance criteria list
- Output: An updated version of the ADatP-5658 conformance criteria that incorporates all required amendments and newly defined criteria covering the full standard, presented in the agreed-upon format.
- Outcome: A finalized and fully aligned set of ADatP-5658 conformance criteria ready for stakeholder use.
- Acceptance criteria:
o All amendments and new criteria are included in the updated document.
o All conformance criteria are deemed testable.
o The updated conformance criteria follow the agreed format consistently.
o The document is complete and ready for stakeholder review.
- Key performance indicators (KPIs):
o Completeness (%):
- Percentage of the standard covered by the acceptance criteria list
- Target = 100%
o Testability Rate (%):
- Percentage of conformance criteria that are testable
- Target = 100%
o Format Compliance (%):
- Adherence to agreed template/format
- Target 95%
o Clarity Score (1-5):
-
Stakeholder evaluation on clarity and readability
-
Target 4.0
-
Service: Analyse and identify the different ADatP-5658 intended use cases.
- Deliverable name: ADatP-5658 use cases analysis report
- Output: An analysis report describing the range of intended application scenarios or use cases for ADatP-5658, with a particular focus on the types of interactions the standard enables or requires with other systems.
- Outcome: A clear understanding of how ADatP-5658 is expected to be applied across relevant scenarios, supporting informed decision-making for integration, implementation, and interoperability planning.
- Acceptance criteria:
o The report identifies and describes all relevant intended application scenarios.
o Interactions with other systems are clearly analysed and articulated for each scenario.
o The report is complete, well-structured, and suitable for stakeholder review.
- Key performance indicators (KPIs):
o Use Case Coverage (%):
- Percentage of relevant scenarios identified
- Target 95%
o Clarity & Structure Score (1-5):
- Stakeholder evaluation on clarity and structure
- Target 4.0
o Analysis Depth Score (1-5):
-
Quality of scenario descriptions and interoperability insights
-
Target 4.0
-
Service: Design and prototype an ADatP-5658 Test Reference System able to run interoperability tests against other capabilities, for all ADatP-5658 intended use cases, during NATO Interoperability events.
- Deliverable name: ADatP-5658 Test Reference System design specification
- Output: A detailed design specification document describing the architecture, components, interfaces, and operational workflow of the ADatP-5658 test reference system.
- Outcome: A comprehensive and clearly defined system design that supports implementation, integration, and validation of the ADatP-5658 test reference system.
- Acceptance criteria:
o The document follows the System Design Specification Template provided by the NCIA POC.
o The document fully describes the system architecture, components, interfaces, and operational workflow.
o All design elements are presented clearly and consistently to support implementation.
o The specification is complete and suitable for stakeholder and technical review.
- Key performance indicators (KPIs):
o Template Compliance (%):
- Alignment with the System Design Specification Template
- Target 95%
o Architecture Coverage (%):
- Percentage of required elements documented (components, interfaces, workflows)
- Target 95%
o Clarity & Structure Score (1-5):
- Stakeholder evaluation on clarity and structure
- Target 4.0
Deliverable name: ADatP-5658 Test Reference System functional prototype
- Output: A functional prototype of the ADatP-5658 test reference system, capable of executing interoperability tests with other capabilities across all intended ADatP-5658 use cases during NATO Digital Interoperability Verification Experiment (DIVE) event.
- Outcome: Support of ADatP-5658 interoperability testing activities at NATO DIVE event to de-risk the standard.
- Acceptance criteria:
o The prototype can run interoperability tests for all intended ADatP-5658 use cases.
o The prototype successfully interfaces with and tests against other participating capabilities.
o The prototype is functional and ready for use during NATO Interoperability events.
- Key performance indicators (KPIs):
o Use Case Execution Coverage (%):
- Percentage of use cases supported by prototype
- Target = 100%
o System Stability (Failure Rate %):
- Failed executions during DIVE event
- Target 10%
Deliverable name: ADatP-5658 Test Reference System operational version
- Output: A fully functional version of the ADatP-5658 Test Reference System, providing a complete, operational implementation ready to entirely support ADatP-5658 interoperability testing and validation.
- Outcome: Complete support of ADatP-5658 interoperability testing activities at future NATO interoperability events.
- Acceptance criteria:
o The new version can support the validation of all the interoperability scenarios defined in ADatP-5658 standard.
o All approved changes, enhancements, and new requirements gathered during the usage of the ADatP-5658 Test Reference System prototype are incorporated into the new version.
- Key performance indicators (KPIs):
o Scenario Validation Coverage (%):
- Percentage of ADatP-5658 scenarios supported
- Target = 100%
o System Stability (Failure Rate %):
- Failed executions during interoperability testing activities
- Target 5%
Deliverable name: ADatP-5658 Test Reference System documentation set
- Output: An up-to-date documentation set encompassing all the relevant information regarding the ADatP-5658 Test Reference System, including architecture diagrams, workflow charts, user guides, and other relevant reference materials, provided in the format requested by the NCIA POC.
- Outcome: A complete and well-organized collection of documentation that supports understanding, use, maintenance, and evaluation of all ADatP-5658 Test Reference System.
- Acceptance criteria:
o All documentation (diagrams, workflows, guides, and reference materials) is included, complete, and up to date.
o The documentation follows the format and structure requested by the NCIA POC.
o The documentation set is clear, well-organized, and suitable for stakeholder use.
- Key performance indicators (KPIs):
o Documentation Completeness (%):
- Coverage of all required materials
- Target = 100%
o Clarity & Structure Score (1-5):
- Stakeholder evaluation on clarity and structure
- Target 4.0
- Deliverable name: Future ADatP-5658 Test Reference System and reference components enhancements and scalability recommendations
- Output: A set of recommendations outlining potential future functional and non- functional future enhancements of the ADatP-5658 Test Reference system and reference components.
- Outcome: A clear, forward-looking set of improvement and guidelines that inform the design and development of the future versions of the ADatP-5658 Test Reference and system reference components.
- Acceptance criteria:
o Recommendations address both functional and non-functional future enhancements.
o Each recommendation is clearly explained and supported with rationale.
o The set of recommendations is complete and suitable for informing future development planning.
- Key performance indicators (KPIs):
o Coverage of Recommendations (%):
- Percentage of functional and non-functional aspects addressed
- Target = 100%
o Recommendation Quality Score (1-5):
- Stakeholder evaluation on clarity, feasibility and technical depth
- Target 4.0
Service: Design and implement the required ADatP-5658 test reference components to run any future conformance tests related to all the identified intended use cases, as well as to integrate in the NCIA ASB INSPECT platform.
- Deliverable name: ADatP-5658 Test Reference Components design specification
- Output: A detailed design specification document describing the architecture, components, interfaces, and operational workflow of the ADatP-5658 test reference components, including the integration approach with the NCIA ASB INSPECT platform.
- Outcome: A comprehensive and clearly defined system design that supports the implementation and integration of the ADatP-5658 test reference components with the NCIA ASB INSPECT platform.
- Acceptance criteria:
o The document follows the System Design Specification Template provided by the NCIA POC.
o The document fully describes the architecture, components, interfaces, operational workflow, and integration approach with the ASB INSPECT platform.
o All design elements are presented clearly and consistently to support implementation and integration activities.
o The specification is complete and suitable for stakeholder and technical review.
- Key performance indicators (KPIs):
o Template Compliance (%):
- Alignment with the System Design Specification Template
- Target 95%
o Architecture Coverage (%):
- Percentage of required elements documented (components, interfaces, workflows)
- Target 95%
o Clarity & Structure Score (1-5):
- Stakeholder evaluation on clarity and structure
- Target 4.0
- Deliverable name: ADatP-5658 Test Reference Components functional implementations
- Output: Functional implementations of the ADatP-5658 test reference components, capable of executing conformance tests across all identified intended use cases and operable under the management of the NCIA ASB INSPECT platform.
- Outcome: A working set of test reference components that can run conformance tests for all intended ADatP-5658 use cases and integrate seamlessly with the ASB INSPECT platform for orchestration and management.
- Acceptance criteria:
o The components can execute conformance tests for all identified ADatP-5658 intended use cases.
o The components can be fully managed and controlled from the NCIA ASB INSPECT platform.
o The implementations are functional, stable, and ready for stakeholder evaluation.
- Key performance indicators (KPIs):
o Use Case Test Coverage (%):
- Percentage of ADatP-5658 scenarios supported
- Target = 100%
o INSPECT Platform Integration Success Rate (%):
- Percentage of successful actions that the reference components can execute in integration with the INSPECT platform
- Target 95%
o System Stability (Failure Rate %):
- Failed executions during testing activities
- Target 5%
Deliverable name: ADatP-5658 automated conformance test samples
- Output: A set of ADatP-5658 automated conformance test samples designed to validate the test reference components across all identified intended use cases, including tests that verify integration with the NCIA ASB INSPECT platform.
- Outcome: A representative suite of automated conformance tests that demonstrates test coverage for all intended ADatP-5658 use cases and confirms correct integration with the ASB INSPECT platform.
- Acceptance criteria:
o The test samples cover all identified ADatP-5658 intended use cases.
o The test samples include scenarios validating integration with the NCIA ASB INSPECT platform.
o The test set is complete and clearly documented, including test setup and test execution instructions.
- Key performance indicators (KPIs):
o Use Case Test Coverage (%):
- Percentage of ADatP-5658 use cases covered with automated tests
- Target = 100%
o Test Documentation Completeness (%):
- Coverage of all required materials
- Target = 100%
Deliverable name: ADatP-5658 Test Reference Components Documentation Set
- Output: An up-to-date documentation set encompassing all the relevant information regarding the ADatP-5658 Test Reference Components, including architecture diagrams, workflow charts, user guides, and other relevant reference materials, provided in the format requested by the NCIA POC.
- Outcome: A complete and well-organized collection of documentation that supports understanding, use, maintenance, and evaluation of all ADatP-5658 Test Reference Components.
- Acceptance criteria:
o All documentation (diagrams, workflows, guides, and reference materials) is included, complete, and up to date.
o The documentation follows the format and structure requested by the NCIA POC.
o The documentation set is clear, well-organized, and suitable for stakeholder use.
- Key performance indicators (KPIs):
o Documentation Completeness (%):
- Coverage of all required materials
- Target = 100%
o Clarity & Structure Score (1-5):
- Stakeholder evaluation on clarity and structure
- Target 4.0
Service: Provide support for the NATO Interoperability events' conferences and execution
phases.
- Deliverable name: ADatP-5658 Test Reference System architecture & workflow presentation
- Output: A set of slides describing the architecture and operational workflow of both the ADatP-5658 test reference components and the overall test reference system.
- Outcome: A clear and visually structured presentation that communicates how the test reference components and system are designed and operate, supporting stakeholder understanding and discussion.
- Acceptance criteria:
o The slides clearly describe the architecture and operational workflow of both the test reference components and the test reference system.
o The content is accurate, well-structured, and easy to understand.
o The slide deck is complete and suitable for presentation to stakeholders.
- Key performance indicators (KPIs):
o Content Completeness (%):
- Coverage of all the systems, components and workflows
- Target = 100%
o Clarity & Structure Score (1-5):
- Stakeholder evaluation on clarity and structure
- Target 4.0
- Deliverable name: ADatP-5658 Test Environment setup and troubleshooting
- Output: Support to NCIA in all test environment setup activities related to the ADatP-5658 Test Reference System and its test environment, including assistance with troubleshooting and resolving any issues that arise.
- Outcome: A fully prepared and operational test environment enabling effective execution of ADatP-5658 testing activities, with issues identified and resolved in a timely manner.
- Acceptance criteria:
o Test environment setup tasks for the ADatP-5658 Test Reference System are completed as required.
o Troubleshooting support is provided promptly, with issues effectively diagnosed and resolved.
o The environment is confirmed operational and ready for NCIA testing activities.
- Key performance indicators (KPIs):
o Setup Completeness (%):
- Percentage of the test environment ready for usage
- Target = 100%
o Average Issue Resolution Time (hours):
- Average time to fully fix any test environment related issues
- Target 24 hours
- Deliverable name: On-site operation of the ADatP-5658 Test Reference System during the NATO Digital Interoperability Verification Experiment (DIVE) event.
- Output: On-site operation of the ADatP-5658 Test Reference System to support the execution of interoperability tests throughout the DIVE execution phase at NCIA The Hague, The Netherlands.
- Outcome: Consistent and reliable operation of the Test Reference System, enabling successful execution of interoperability tests during the events.
- Acceptance criteria:
o The Test Reference System is operated reliably throughout the event's execution phase.
o Interoperability tests are supported and executed as required.
o Any operational issues are addressed promptly to maintain test continuity.
- Key performance indicators (KPIs):
o System Uptime (%):
- System availability during the event
- Target 95%
o Average Issue Resolution Time (hours):
- Average time to fully fix any operational issues
- Target 1 hours
- Travel requirements: At least one member of the Contractor's personnel shall be present, onsite, at NCIA The Hague, The Netherlands, from 5 to 9 October 2026.
Requirements
- At least 5 years of professional experience in the industry of software development;
- At least 3 years of professional experience working with programming or scripting languages (knowledge of Robot Framework and Python would be desirable);
- At least 1 year of professional experience working with Docker or Podman, and building and running Linux-based containers;
- At least 1 year of professional experience defining conformance criteria for software based standards. In the absence of such experience, proven expertise in writing acceptance criteria for software projects may be considered;
- Demonstrable knowledge of NATO digital standards (knowledge of ADatP 5658 standard would be desirable);
Language
- Ability to communicate in English with clarity, coherence, conciseness and engagement, both orally and in writing., * Valid National or NATO Secret personal security clearance
