Link Coverage 100%, ASPICE Assessment Still at Risk.

A central pillar of ASPICE is the V-model, which goes far beyond a simple phase model: it forms the methodical backbone for well thought out and traceable product development. ASPICE requires seamless, bidirectional linking across all development stages, from the original stakeholder requirements to the system and software design, through to the actual implementation and final tests. And - since ASPICE 4.0, anchored even more strictly in the assessment through precise definitions - it demands not just formal links, but genuine content consistency. As the official standard (Automotive SPICE PAM v4.0) unequivocally clarifies: “Traceability alone, e.g., the existence of links, does not necessarily mean that the information is consistent with each other.” And in Annex B on Consistency Evidence: “Evidence that the content of the referenced or mapped information coheres semantically along the traceability chain,…”.

This is precisely the difference this article addresses. The added value of traceability does not arise from the existence of links in a tool, but from the fact that they are consistent in terms of content and operate at the correct level of abstraction. If either of these two aspects is not fulfilled, the process fails to meet the target.

Therefore, when conducting assessments, the assessors never just check whether a linking field is checked in the tool. They read the texts, scrutinize the context and expose alibi links. The goal is to prove that a requirement is fully covered by the design, that no unrequested extra functionalities end up in the code and that every artifact is created where it belongs procedurally.

While traceability could already be demonstrated through classical metrics in the past, such as percentage-based link coverage rates or allocation matrices, the effort required to prove consistency was the responsibility of the developers. However, Large Language Models now make it possible for the first time to support engineers through automated, semantic consistency checking: they analyze the actual meaning of requirements, architectural specifications and test cases to identify logical deviations, parameter errors or context breaks in the text.

But it is not enough to give the language model access to requirements databases and prompt it to perform a consistency check. Rather, a systematic approach must be developed that combines the LLM’s language comprehension capabilities with a deterministic and reproducible evaluation metric. As part of the practical implementation of an LLM-supported ASPICE agent, the procedure outlined below illustrates how an automated consistency check can systematically uncover semantic errors:

Step 1: The Type Check as the Foundation

The Type Check automatically verifies whether development artifacts correspond in content to their assigned ASPICE process before the consistency of the links is checked.

Since language models are excellent at understanding language but tend toward variance when drawing conclusions (non-determinism), the evaluation is carried out in 2 stages:

Structured Content Check (LLM)

In the first analysis phase, the language model evaluates each artifact individually based on a predefined catalog of questions:

• Content Check: Does a quantifiable text even exist? Possible answers: yes/no.
• Level Check: Does the text remain at a single level of abstraction? Possible answers: yes/no.
• Abstraction Check: Only for requirements artifacts: is the formulation solution-free (what) and does it contain no implementation details (how)? Possible answers: yes/no.
• Category Check: Which ASPICE process does the content correspond to? Possible answers: fixed predefined categories or unclear.

This ensures that the LLM always performs the check as objectively as possible using the same questions with the same possible answers.

Deterministic Classification and Evaluation (Script)

Following the content check, the LLM’s answers are mapped to finding types via a predefined combinational schema:

• No Content: The artifact contains no quantifiable text.
• Mixed Levels: Mixes several levels of abstraction or detail within a requirement.
• Abstraction Drift: The category is fundamentally correct, but the requirement already contains concrete implementation details or internal signal names.
• Category Unclear: The content cannot be clearly assigned to a category.
• Wrong Category: The content of the text contradicts the specified ASPICE category.
• Correct Category: If all content checks are passed, the artifact is considered as fully compliant (content consistent).

Important: the Type Check serves as a pre-filter. If there are too many erroneous or unclear process assignments, a subsequent consistency check of the traceability is meaningless and the data basis must be reviewed.

Step 2: The Consistency-Check as the Evaluative Traceability Instance

The evaluation of the links is also carried out in two stages to combine the strengths of the language model with the clarity of classic scripts.

Pairwise Structured Content Check (LLM)

The LLM answers the following five questions strictly in sequence for each linked pair (possible answers: yes/no):

• Domain Check: Do the source and target artifacts belong to the same functional domain?
• Unambiguity Check: Does the target address exactly this specific source artifact (and not a neighboring element)?
• Name Check: Do signal, interface or test names match?
• Completeness Check: Does the target cover all aspects of the source?
• Conflict Check: Are there contradictory thresholds, times or behaviors?

Deterministic Classification and Evaluation (Script)

Here, too, the answers from the content check are deterministically mapped to finding types:

• Mislink: Artifacts are linked to the wrong partner; the correct linking target exists elsewhere.
• Wrong Domain: Source and target refer to independent functional areas.
• Wrong Entity: Same domain, but the target addresses a different specific entity.
• Value Conflict: Contradictory values, thresholds or behaviors between source and target.
• Terminology Mismatch: Same concept at different abstraction levels; different naming conventions.
• Coverage Gap: Target covers some, but not all, aspects of the source.
• Concept Gap: Related but not identical concepts.
• Scope Ambiguity: Unclear whether the source and target refer to exactly the same subsystem or scope.
• Aligned: No problems found - correct semantic assignment.

Step 3: The Consistency-Score

Based on the results of the previous analysis steps, a reproducible Consistency Score is calculated. It provides an indication along the NPLF scale known from ASPICE:

• N (not achieved)
• P (partially achieved)
• L (largely achieved)
• F (fully achieved)

The classification is important: the score does not replace an assessment or assign ratings. The assessor assigns these based on the evidence. But it continuously shows, long before the assessment, where the evidence is fragile. The principle of Conservative Bias applies throughout: in case of doubt, a finding is reported. This is essential because the costs of a “false positive,” where a human checks the finding and possibly rejects it, are significantly lower than the risks of a “false negative,” where an unrecognized inconsistency undermines confidence in the validity of the automated check.

Practical Examples

A System Requirement is entered into the tool.

• Input (Artifact Text): “The system must receive the sensor value from Temp_Sensor_01 via the CAN_ID_0x55A.”
• LLM Analysis:
  • Abstraction Check (Solution-free?): No. (The artifact contains specific implementation details such as communication interface and signal names).
• Script Evaluation (Mapping): Abstraction Drift
• Justification: Although the artifact belongs to the “System Requirement” category in terms of content, it violates the principle of solution neutrality. The Type Check triggers because the requirement operates at the abstraction level of implementation details instead of only describing the “what.”

A linked pair consisting of a System Requirement (source) and a Test Case Specification (target) is checked.

• Source (System Requirement): “The system must switch to safe mode in an emergency state within a maximum of 100 milliseconds.”
• Target (Test Case): “Verify that safe mode becomes active after triggering the emergency event within 500 milliseconds.”
• LLM Analysis:
  • Conflict Check: Yes. (There is a substantive contradiction in the time specifications).
• Script Evaluation (Mapping): Value Conflict
• Justification: The semantic check detects the contradiction between the requirement (100ms) and the test criterion (500ms). Since the values do not match, a “Value Conflict” is reported.

These examples illustrate: The system reports an irregularity in each case, forcing an engineer to either clean up the requirements level (Example 1) or check the consistency between design/test and requirement (Example 2).

Conclusion: Quality Lies in the Content, Not the Link

Traceability metrics often provide a false sense of security. Without semantic content verification, ASPICE compliance remains superficial. The approach outlined in this document - the combination of Type Check, deterministic Consistency Check and an objective Consistency Score - forms the methodical foundation for LLM-based, content-related quality assurance. In this context, the AI does not function as a replacement for humans, but as a pre-filter: by automating the mass of trivial consistency checks, it massively relieves the engineering team. Engineers thus gain the freedom to concentrate their expertise specifically on the critical deviations where human judgment is indispensable.

Requirements traceability in practice — How end-to-end traceability works in regulated projects, which tools have proven themselves, and when the RTM approach reaches its limits: Requirements Traceability at itemis →