Every scientific breakthrough is supposed to become the foundation for the next one. But what happens when other researchers can’t readily find, run or build upon the work that came before?
That’s the challenge behind efforts to make scientific research both reproducible and reusable. Reproducibility means that an independent researcher can use the original methods, software and data from a study and obtain the same results. But scientific progress also depends on reusability: whether other researchers can find, understand, execute and adapt those materials to compare approaches, explore new questions and build on the original work.
For cybersecurity researchers, where technology changes rapidly, barriers to reuse have real consequences. Every week spent rebuilding old research is a week not spent developing better defenses.
“If researchers can easily reproduce and reuse earlier work, we could build more sophisticated solutions more quickly,” saidJelena Mirkovic, a principal scientist in theUSC Information Sciences Instituteand research associate professor at theUSC Thomas Lord Computer Science Departmentat theUSC Viterbi School of Engineeringand theUSC Stevens School of Computing and Artificial Intelligence.
Mirkovic andDavid Balenson,a senior supervising computer scientist at ISI and interim director of the Networking and Cybersecurity division, along with M.S. studentQishen Sam Liang, wanted to know whether the cybersecurity community’s efforts to improve reproducibility were actually making research artifacts easy for later researchers to reuse. Their study,“Measuring Ease of Reuse for Security Research Artifacts with Results-Reproduced Badges,”found significant room for improvement.
The paper was presented in late July at the 2026 ACM Conference on Reproducibility and Replicability.
A companion position paper by Mirkovic and Balenson,“Living Artifacts: Enabling Vertical Progress in Cybersecurity Research,”was presented in May at the Metascience and Critical Reflections in Security & Privacy (MetaCRiSP) workshop, held in conjunction with the 47th IEEE Symposium on Security and Privacy. It proposes a way to keep research useful and relevant long after publication.
Reproducible Versus Reusable
Many conferences now encourage researchers to submit the software and data behind their studies for artifact evaluation, an independent review in which researchers verify that the materials are complete and can reproduce the published results. Depending on what evaluators can verify, artifacts may receive badges indicating that they are available, functional, reusable or capable of reproducing key results.
Artifact evaluation provides valuable evidence that research materials worked near the time of publication. But a badge does not guarantee that another researcher will still be able to find, install, understand or adapt the artifact months or years later.
“Reproducible usually means I can run your code and your data set and get the same results,” Mirkovic said. “But if I want to reuse your research, change some pieces of your code or use it on a different data set, that can be much harder.”
Testing the Research
When evaluating research artifacts, Mirkovic, Liang, and Balenson put themselves in the shoes of scientists trying to leverage someone else’s work. They evaluated 97 code artifacts associated with papers from two major cybersecurity conferences: thePrivacy Enhancing Technologies Symposiumand theIEEE Annual Computer Security Applications Conference. Each artifact had already received one of the venues’ highest badges, indicating that independent evaluators had successfully used it near the time of publication and were able to obtain the same results as reported in the paper.
The team approached each artifact as a researcher trying to reuse it would. They tried to locate the artifact, determine whether it contained the necessary components, recreate its computing environment, and install and run it using the published documentation—without contacting the original authors. If they encountered problems during reuse, they tried to fix them using online search and web-based AI engines. They generally limited their effort to one hour per artifact.
The exercise exposed just how difficult reuse attempts can be.
Of the 97 artifacts, about 12 percent could not be found or were missing essential components. Of the 85 complete artifacts, 23 required hardware or software outside the broadly available environment assumed by the study. Among the remaining 62 candidates, 43 could be installed and run within the one-hour limit—but only 23 were considered easy to reuse, requiring less than 10 minutes of active effort.
The researchers also found that three artifacts that had worked during evaluation had become unusable only four months later, illustrating how quickly software dependencies and computing environments can change.
“We expected some problems,” Balenson said. “But the amount of friction that we found was just very striking.”
The study did not assess the validity of the papers’ scientific claims. Instead, it examined whether the artifacts supporting those claims remained findable and usable. Broken links, missing components, specialized resource requirements, inconsistent packaging, incomplete documentation and evolving software dependencies all created barriers.
Keeping Research Alive
To address the challenge of artifact obsolescence as software evolves, Mirkovic and Balenson propose what they call “living artifacts.”
The idea borrows from the open-arch software would remain available for other scientists to update, improve and adapt to new computing environments, with each new version linked from and to the original work
The published version would remain preserved, while maintained versions and derivatives could evolve as software, dependencies and computing environments change. Shared research infrastructure could help keep the artifacts executable on the same common platform, while versioning and attribution would preserve their connection to the original paper and recognize later contributors.
Although their work focused on cybersecurity, the implications extend much further. As more disciplines rely on software and data, making research reusablengs, compare new approaches and make progress without repeatedly rebuilding the foundations of prior work
“Reusable means the researcher can take published materials, use them as a starting point for new work, and they can do that much later, after the paper is published and without the original author’s help,” Balenson said. “That’s what really enables further progress.”
Last updated on August 31st, 2026
This article may feature some AI-assisted content for clarity, consistency, and to help explore
complex scientific concepts with greater depth and creative range.
