Cisco discovered reasons why even seemingly perfect AI reports on cyber incidents should not be taken as truth

Cisco Talos Investigates the Accuracy of Large Language Models in Generating Cybersecurity Reports

The Cisco Talos Incident Response team conducted a test to determine how reliably modern LLMs (ChatGPT, Claude, and Gemini) can generate technical reports on cyber incidents. The results showed that even the most “professionally looking” documents contain factual errors, contradictions, and inconsistencies.

How the Study Was Conducted
* Preparation – Researchers provided each model with raw incident notes and asked it to produce a report.

* Output – All three LLMs produced visually polished documents, but detailed analysis revealed inaccuracies and unexpected conclusions.

Nate Pors, senior director of Incident Response at Cisco Talos, described the findings in detail on the corporate blog.

Why LLMs “Make Mistakes”
Cisco attributes the issues to the probabilistic nature of the models: they predict the next word based on statistical weights rather than understanding meaning. According to Pors, distortions manifest in four directions:

1. Different Data Fragments
With each request, the model relies on different parts of the input, making reproducible and standardized results impossible.

2. Inconsistent Conclusions
The same data can lead to different recommendations: one report may suggest a forced password change across the organization, while another recommends only targeted changes. The model often locks onto the first recommendation it generates, even if it is unsuitable.

3. Unconventional Structure
Because LLMs build text token by token, final documents can have varying structure and formatting, which is critical in environments that require templates for quality control.

4. Context Window Issues
When input volume exceeds the context limit, the model may “forget” important information from the start of the session and produce unpredictable or mixed results.

What This Means for Cybersecurity
Cisco acknowledges that theoretically the task could be limited to a specific report fragment, but such an approach deprives users of time savings. In cyber defense even a small error can be costly. Therefore, authors of final documents must carefully verify every word and not rely entirely on LLM recommendations.

Conclusion: Large language models can quickly generate reports, but their outputs require mandatory verification by specialists to avoid errors and inconsistencies in the critically important context of cyber incidents.