A comparative benchmark study of LLM-based threat elicitation tools: replication package

This replication package is made available in support of the submission entitled "A comparative benchmark study of LLM-based threat elicitation tools". It provides details, scripts and reproduction materials for:

baseline construction

tool outputs 

evaluation and comparison of threat models

detailed results for F1 scores, redundancy, etc

V2: (revision 1) The updated version of this package provides the results of additional validation (impact of the instruction to generate around 100 threats, test cases for semantic similarity threshold, etc)

V3 (revision 2) 

This revision includes:

Additional validation of the expert baselines

Results of sensitivity analysis to evaluate performance at different semantic similarity levels (0.7,0.75,0.8 and 0.9) 

Clean up of temporary artifacts and harmonization of structure of the archive.