MSR 2023 Dataset

This is the artifact that accompanies the paper titled: "Wasmizer: Curating WebAssembly-driven Projects on GitHub". <br> It contains: - the scripts we used to produce a dataset of WebAssembly binaries - a dataset of WebAssembly binaries (8915 .wasm files and 1384 .wat files) - the WASMIZER tool that can be used to automatically collect a new dataset, with refined information # Scripts <br> The `Scripts/` folder contains the scripts that we used to collect our dataset in December 2022. Since then, we have refined our scripts and made a tool, WASMIZER, described at the end of this README and accessible in the `WASMIZER/` folder. <br> In the `Scripts/` folder, there are three subfolders : <br> - `Collector/` that contains the scripts to collect GitHub projects that may be targeting WebAssembly as a compilation target - `Compiler/` that tries to compile a project and extracts the `.wasm` and `.wat` found after compilation. - `SmellsChecker/` that contains the checkers for the smells used in the case study of the paper. Each of these directories contain a README providing more details. <br> # Dataset The `Dataset/` folder contains our dataset of WebAssembly binaries collected in December 2022. It is structured as follows: <br> - The `wasm/` folder contains WebAssembly binaries in their binary format (`.wasm`) - The `wat/` folder contains WebAssembly binaries in their textual format. The basename of the files are their SHA checksum. Each binary is accompanied by a `.meta` file of the same name containing the project and the path within the project in which it was found. For example, for the binary `00047ad76615715bb2b36fa2102135b8dc32ac3c17f3488451168f808e2039f0.wasm`, there is a `00047ad76615715bb2b36fa2102135b8dc32ac3c17f3488451168f808e2039f0.meta` file containing: <br> ``` ./JuiceFV-Emscripten_OpenGL/application/dependencies/lib_sources/GLM/test/gtx/test-gtx_easing.wasm ``` <br> This indicates that we have found a WebAssembly binary in the `JuiceFV/Emscripten_OpenGL` GitHub project, at location `application/dependencies/lib_sources/GLM/test/gtx/test-gtx_easing.wasm`. When collecting this dataset, a number of metadata have not been collected and are thus missing from this initial snapshot. However, we have bundled a tool described below, which scrapes repositories and extract .wasm files after compilation, which collects more metadata. <br> # WASMIZER The WASMIZER tool is provided in the `WASMIZER/` directory. This is a refined version of our scripts used to collect the dataset. More details can be found in `WASMIZER/README.md`. To obtain the latest version, one can run `git pull` from the `WASMIZER/` directory, or access the repository online at [https://github.com/arash-mazidi/WASMIZER](https://github.com/arash-mazidi/WASMIZER). <br> The tool is deployed and regularly pushes newly found projects, .wasm and .wat files to the following shared folder : https://tucloud.tu-clausthal.de/index.php/s/MMRQMEZm66GRGXI (password: wasmizer).本研究产出物配套论文题为《Wasmizer：GitHub上WebAssembly驱动项目的整理》（原英文标题："Wasmizer: Curating WebAssembly-driven Projects on GitHub"）。本产出物包含以下内容： - 用于构建WebAssembly（Wasm）二进制文件数据集的脚本； - 一套WebAssembly二进制文件数据集（含8915个`.wasm`文件与1384个`.wat`文件）； - WASMIZER工具，可用于自动采集包含精细化信息的全新数据集。 ## 脚本（Scripts） `Scripts/` 文件夹内包含我们2022年12月用于采集本数据集的脚本。此后我们对脚本进行了优化，并开发了WASMIZER工具（详见本自述文件末尾），该工具存放于`WASMIZER/`文件夹中。 `Scripts/` 文件夹包含三个子目录： - `Collector/`：内含用于采集以WebAssembly为编译目标的GitHub项目的脚本； - `Compiler/`：可尝试编译项目并提取编译后生成的`.wasm`与`.wat`文件； - `SmellsChecker/`：包含本论文案例研究中所用的代码坏味检测脚本。每个子目录均配有自述文件，提供更多细节。 ## 数据集（Dataset） `Dataset/` 文件夹内包含我们2022年12月采集的WebAssembly二进制文件数据集，其结构如下： - `wasm/` 文件夹存放二进制格式的WebAssembly文件（`.wasm`）； - `wat/` 文件夹存放文本格式的WebAssembly文件（`.wat`）。文件的基础名为其SHA校验和。每个二进制文件均配有同名的`.meta`元数据文件，用于记录其所属项目及在项目内的路径。例如，对于二进制文件`00047ad76615715bb2b36fa2102135b8dc32ac3c17f3488451168f808e2039f0.wasm`，会存在对应的`00047ad76615715bb2b36fa2102135b8dc32ac3c17f3488451168f808e2039f0.meta`文件，其内容如下： ./JuiceFV-Emscripten_OpenGL/application/dependencies/lib_sources/GLM/test/gtx/test-gtx_easing.wasm 该内容表明，我们在`JuiceFV/Emscripten_OpenGL` GitHub项目的`application/dependencies/lib_sources/GLM/test/gtx/test-gtx_easing.wasm`路径下发现了该WebAssembly二进制文件。 在本次数据集采集过程中，部分元数据未被采集，因此该初始快照存在元数据缺失问题。不过我们已附带了下述工具，该工具可抓取仓库并在编译后提取`.wasm`文件，同时采集更多元数据。 ## WASMIZER工具 WASMIZER工具存放于`WASMIZER/`目录中，是我们优化后的数据集采集脚本。更多细节可查阅`WASMIZER/README.md`。若需获取最新版本，可在`WASMIZER/`目录中执行`git pull`命令，或在线访问仓库：https://github.com/arash-mazidi/WASMIZER。 该工具已完成部署，并会定期将新发现的项目、`.wasm`与`.wat`文件推送至以下共享文件夹：https://tucloud.tu-clausthal.de/index.php/s/MMRQMEZm66GRGXI（访问密码：wasmizer）。