Pien Rooijendijk

PhD Candidate

Radboud University


2026

Who Said CVE? How Vulnerability Identifiers Are Mentioned by Humans, Bots, and Agents in Pull Requests

Pien Rooijendijk, Christoph Treude, Mairieli Wessel · International Conference on Mining Software Repositories (MSR) 2026

In this paper, we study how vulnerability IDs (like CVE, CWE, and GHSA) are mentioned in Github pull requests. By examining who mentions these IDs, we can better assess whether autonomous agents are truly participating in security maintenance tasks. The highlights of our work include:
  • Augmented Dataset: We are sharing an augmented dataset with additional pull requests from the same repositories found in the AIDev-pop data, a dataset designed to capture the activity of AI agents in real-world GitHub repositories.
  • Quantitative Analysis: Our data shows that bots account for 69.1% of all mentions. While humans and agents mention IDs less frequently, their mentions appear across titles and commit messages, whereas bots primarily mention vulnerabilities in PR descriptions.
  • Qualitative Analysis: Our manual review reveals that bots primarily use these IDs in automated dependency updates, while humans and agents use them to provide context for fixes.

Link to the paper

How Humans, Bots, and Agents Communicate About Vulnerabilities in Pull Requests

Pien Rooijendijk, Christoph Treude, Mairieli Wessel · International Conference on Software Maintenance and Evolution (Registered Report) 2026

In this study, we investigate how humans, bots, and coding agents communicate about vulnerabilities in pull requests. Using the AIDev-pop dataset, we analyze both explicit vulnerability references and implicit security-related language across pull request discussions, including titles, descriptions, reviews, commits, and comments. We also examine whether these discussions correspond to vulnerabilities introduced or fixed in the code and how they relate to the review process and its outcomes. Our goal is to provide a large-scale view of how vulnerability communication is changing in modern software development, particularly as automated tools and AI coding agents become increasingly involved in software engineering.

Link to arXiv