Academic research · UNESP × Emergent Methods

Do you read the news the way our AI does?

We built a model that flags bias in news writing. Now we want to understand how its judgments compare with the way people actually read. You’ll go through a few short news paragraphs and tell us whether you see the bias the model sees.

The study runs on AskNews accounts. That’s how we keep one session per person and manage participation. Log in with your AskNews account, or create a free one right on the sign-in screen; you’ll be brought back here afterwards.

minutes: 10–15
short paragraphs: 6
questions + 1 practice: 12

What you’ll do

1
Practice first (not counted)
One paragraph in the exact format of the study, so you know what to expect before anything counts.
2
Judge 6 paragraphs
For each, two yes/no questions: does this paragraph show this kind of bias? “Not sure” is always an option.
3
See how the model compares with you
At the end, one number: how often the model agreed with your reading. Then, if you like, see the model’s reasoning for each paragraph and tell us where it got it wrong.

No preparation needed. There are no trick questions, and no “wrong” answers held against you.

What you’ll be looking for

The model recognizes 18 kinds of bias in news coverage. For this study we focus on the eight below, the ones a reader can judge fairly from a single short paragraph. You’ll see one paragraph at a time, without its headline or source, so a session stays quick and your reading is about the text alone. Each bias type is explained again, with examples, right before we ask about it.

Loaded word choice

The article uses labels or loaded wording that guides the reader toward a judgment.

"The mob flooded the streets after the vote."

Judgmental adjectives

The article uses descriptive words that tell readers how to feel about a person, group, event, or policy.

"The disturbing trend continued this week."

Spin

The article shapes facts into a more favorable or unfavorable story through framing and emphasis.

"The mayor scrambled to defend the routine budget update."

Political viewpoint bias

The article favors or criticizes a political side in a way that pushes the reader toward that view.

"The reckless opposition is ruining the country."

Opinion stated as fact

The article presents an interpretation or opinion as though it were a settled fact.

"The policy is proof that leaders do not care."

Sensational language

The article exaggerates or uses dramatic wording to create a strong emotional reaction.

"Chaos erupted as lawmakers traded blows over taxes."

Negative framing

The article emphasizes bad outcomes or frames events in an unusually negative way.

"The city is collapsing under failed leadership."

Assumed motive

The article claims to know what someone thinks, feels, wants, or intends without clear evidence.

"She clearly wanted to undermine the election."

Why this matters

The dataset behind this model was labeled by a consensus of several AI models, not by hand. Your judgments help us understand how well that approach matches human readers, bias type by bias type, and where the two see things differently.

There's no right answer

We measure agreement, not performance. Bias perception is genuinely subjective, and the places where readers and the model disagree are findings in themselves. Your score simply shows how often you and the model lined up.

Your data

Answers are stored in a private research database. No personal participant data is published; anything we report is aggregated and anonymous. Taking part is voluntary and you can stop at any time.

Informed consent

This is an academic study run by researchers at São Paulo State University (UNESP) and Emergent Methods. Taking part is voluntary, you may withdraw at any time without giving a reason, and no personal participant data will be published. You’ll be asked to confirm consent before the study begins.

Read the full consent form