Śmigiel: corpus of human-written and machine-generated text fragments in Polish

Introduction

Śmigiel (Spotting Machine-Generated Text from LLMs for Polish) is the first open dataset for training and evaluating machine-generated text (MGT) in Polish. It includes a collection of human-written text (HWT) fragments from six domains, which are used to prompt text generation by eight language models capable of producing credible Polish text. In addition to the raw corpus of over 462K generated texts, we also release a cleaned source- and domain-balanced dataset suitable for training and evaluating MGT detectors.

The dataset is described in an article presented at LREC 2026 conference (Śmigiel Dataset: Laying Foundations for Investigating Machine-Generated Text Detection in Polish) and was used in the shared task at PolEval 2025.

Human-written text (HWT)

To compile a corpus of Polish HWT passages, we use several recent datasets with permissive licenses:

literature

PLSC (Poświata et al., 2024)

open-coursebooks (Poświata, 2024)

Wikiźródła (2025)

reviews

PolEmo (Kocoń et al., 2019)

Allegro Reviews (Rybak et al., 2020)

Filmweb (Przybyła, 2024)

Filmweb+ (Narolski, 2020)

social

TwitterEmo (Bogdanowicz et al., 2023)

BAN-PL (Kolos et al., 2024)

wikipedia

Polish Wikipedia (2025)

news

Polish Wikinews (2025)

parlamint

ParlaMint (Erjavec et al., 2025)

Machine-generated text (MGT)

To compile a corpus of MGT passages, we prompt the following LLMs:

small

Bielik-7B-Instruct-v0.1 (Ociepa et al., 2025)

Llama-3.1-8B-Instruct (Meta Team, 2024)

Mistral-7B-Instruct-v0.3 (Jiang et al., 2023)

medium

Bielik-11B-v2.3-Instruct)

PLLuM-12B-nc-chat (Kocoń et al., 2025)

Mistral-Nemo-Instruct-2407 (Mistral AI Team, 2024)

large

gemma-3-27b-it (Gemma Team, 2025)

Llama-3.3-70B-Instruct

Śmigiel Dataset

The published data covers two stages of Śmigiel:

Raw text, including full HWT fragments and MGT generations,

Postprocessed text, following cleaning, sampling and trimming to obtain a clean and unbiased dataset for the purpose of training and evaluating MGT detection models.

The descriptions below are based on articles covering Śmigiel dataset and the shared task, so you should look at them for more information.

Raw text

The raw text is included in three files:

RAW_train_testalpha.csv:  fragments in the literature, reviews, social and wikipedia domains, including generations by all models except Llama-3.3-70B-Instruct -- used for the training portion and the test-alpha),

RAW_testbeta.csv: the same domains as train, but the generations are from Llama-3.3-70B-Instruct,

RAW_testgamma.csv: all of the fragments in the news and parlamint domains.

The files are in a CSV format with the following fields:

numerical ID,

HWT fragment,

prefix obtained from the HWT fragment,

full prompt provided to an LLM,

source dataset,

domain,

number of tokens generated,

sampling strategy,

text generated by the model,

LLM model used.

Postprocessed text

The postprocessed text includes the following portions:

training: 80% of the fragments in the literature, reviews, social and wikipedia domains, including generations by all models except Llama-3.3-70B-Instruct,

testing data split in one of two ways:

according to the source:

test_alpha: 10% of the fragments in the same domain as training, including generations by the same models,

test_beta: 10% of the fragments in the same domain as train, but using the Llama-3.3-70B-Instruct generations,

test_gamma: all of the fragments in the news and parlamint domains.

according to their use in shared task:

testA: 50% of test_alpha, 33% of test_beta and 33% of test_gamma,

testB: 50% of test_alpha, 67% of test_beta and 67% of test_gamma,

Thanks to this structure, the model trained on the training portion can be tested on data from the same distribution (test_alpha), from an unseen model (test_beta), from an unseen domain (test_gamma) or from a mixtures of these (testA, testB).

Each portion is saved in three files:

<name>.txt: newline-separated fragments,

<name>.key: newline-sparated labels (0: HWT, 1: MGT)

<name>.meta: TSV file with additional information: text checksum, source model (or human) and sampling strategy.

Statistics

The Śmigiel dataset is balanced across the main categories (MGT and HWT) and across text domains and LLM sizes.

It includes 32K HWT and 32K MGT examples. The MGT portion contains 10K samples from small LLMs, 10K from medium LLMs, and 12K from large LLMs. For domains, there are about 5.5K HWT and MGT samples each from four types: literature, reviews, social media, and wikipedia. Śmigiel also includes two unseen test domains: news (2.6K HWT and 2.6K MGT) and parlamint (7K samples per category).

More

For more information, you can:

check the Śmigiel dataset article, presented at LREC 2026 (Śmigiel Dataset: Laying Foundations for Investigating Machine-Generated Text Detection in Polish),

consult the webpage, the resources and the overview article of the Śmigiel shared task at PolEval 2025,

use the source code at GitHub or the dataset at HuggingFace,

contact the authors at [e-mail] or [e-mail].

This work was supported by the Ramón y Cajal grant RYC2024-050327-I, funded by the Spanish State Research Agency (MICIU/AEI/10.13039/501100011033) and by the European Social Fund Plus (ESF+) of the European Union. We also gratefully acknowledge Polish high-performance computing infrastructure PLGrid (HPC Centers: ACK Cyfronet AGH) for providing computer facilities and support within computational grant no. PLG/2025/018019.