Home | Accounting Experiments

The Fall of Constantinople for Amazon's Mechanical Turks?

The Fall of Constantinople for Amazon's Mechanical Turks?

While the Ottoman Turks caused turmoil in the late Middle Ages and the Age of Reformation, Amazon’s Mechanical Turks are causing a turmoil in experimental research at this moment. While early studies documented that Amazon’s Mechanical Turk participants were valid proxies for experimental accounting research, there are increasing concerns about the quality of Amazon’s Mechanical Turk (MTurk) data.

Christian Peters

• May 8, 2024

Using LLMs and ChatGPT in oTree experiments

Using LLMs and ChatGPT in oTree experiments

The oTree community has put together a useful oTree app. It allows participants to chat with ChatGPT through OpenAI's API. The app itself uses prompts so ChatGPT takes on a character or personality for participants to chat with. However, the possibilities and use-cases for experimental research are endless

Victor van Pelt

• July 18, 2023

Why you shouldn't trust mediation as process evidence

Why you shouldn't trust mediation as process evidence

Mediation is widely used in experimental accounting to obtain process evidence. The primary benefits of mediation are its low cost and easy integration. However, it has a hidden cost that weakens its effectiveness as process evidence. This post explains why it's the least effective method and suggests two better alternatives.

Victor van Pelt

• April 11, 2023

How to prevent bots and farms from taking over and ruining your online experiment

How to prevent bots and farms from taking over and ruining your online experiment

In this post, I share simple techniques to filter participants before they take part in your online experiment. These techniques filter bots and participants using automated scripts plus participants who fake their geolocation using VPN/VPS, proxies, and server farms.

Victor van Pelt

• December 23, 2022

Analyzing learning rates (pt. 2): Two approaches

Analyzing learning rates (pt. 2): Two approaches

Do you want to learn how to analyze learning? In this second post of a two-part series, Jake Zureich discusses two approaches when comparing learning curves.

Jake Zureich

• May 15, 2022

Replications can help improve relevance of accounting experiments

Replications can help improve relevance of accounting experiments

Both replications and practical relevance are awkward discussion topics for most experimental accounting researchers. Yet, replications offer a concrete way to address concerns we may have about the 'practical relevance' of experimental findings.

Jesse van der Geest

• May 3, 2022

Why We Ignore Free-form Communication and Why We Shouldn't

Why We Ignore Free-form Communication and Why We Shouldn't

People in workplace settings can typically communicate freely with each other, but many experiments scale communication down to a restricted form. Should we maintain this status quo or is there room for free-form communication? Read this post by Farah Arshad and Cardin Masselink to find out more.

Farah Arshad

• February 7, 2022

Analyzing learning rates (pt. 1): Common pitfalls

Analyzing learning rates (pt. 1): Common pitfalls

Do you want to learn how to analyze learning? In this first post of a two-part series, Jake Zureich discusses common pitfalls when comparing learning curves using an illustrative example.

Jake Zureich

• October 1, 2021

The role of exploratory analyses in accounting research

The role of exploratory analyses in accounting research

Developing theory after collecting data is problematic because the theoretical predictions are post hoc. However, does that imply that all exploratory analyses are pointless? In this post, Jeremy Bentley explains that exploratory analyses can still add value even when researchers prefer to pre-commit to ex-ante theoretical predictions.

Jeremy Bentley

• September 20, 2021

Running your experiment on MTurk

Running your experiment on MTurk

With the increasing popularity of online experiments, many have asked us for advice on how to conduct experiments on Amazon's Mechanical Turk. In this post, Christian Peters provides a hands-on guide.

Christian Peters

• August 31, 2021

Are experiments conducted online field experiments or laboratory experiments?

Are experiments conducted online field experiments or laboratory experiments?

Experiments that recruit from online participants pools such as MTurk and Prolific have become increasingly popular over the past two decades. However, since scholars have referred to such experiments as both laboratory and field experiments, which classification should we use?

Victor van Pelt

• August 6, 2021

Choosing the right participants

Choosing the right participants

Choosing the right participant pool for your experiment is challenging. Which experiments require professional participants? Does it matter whether you recruit students or online participants? In this post, Jeremy Bentley explains his approach to participant pool selection.

Jeremy Bentley

• July 18, 2021

Why PEQs do not provide the best process evidence

Why PEQs do not provide the best process evidence

Mediation is widely used in experimental accounting to obtain process evidence. The primary benefits of mediation are its low cost and easy integration. However, it has a hidden cost that weakens its effectiveness as process evidence. This post explains why it's the least effective method and suggests two better alternatives.

Farah Arshad

• April 23, 2021

Setting up a simple qualtrics experiment

Setting up a simple qualtrics experiment

Many doctoral students and researchers find it challenging to start conducting experiments. In this post, Razvan Ghita shows how to create a simple experiment using the Qualtrics platform.

Razvan Ghita

• March 17, 2021

Why ANOVA and linear regression are the same

Why ANOVA and linear regression are the same

Why do some experimentalists in accounting use ANOVA's while other use regressions? What's the difference? This post shows why they are merely different representations of the same thing.

Christian Peters

• February 26, 2021

Automize testing your experiments with 'bots'

Automize testing your experiments with 'bots'

Bots are a powerful yet often overlooked tool that helps experimental researchers test their applications more effectively and efficiently. In this post, Victor van Pelt explains their use and argues that their usefulness may even extend beyond testing.

Victor van Pelt

• December 4, 2020

What do participants think of accounting experiments?

What do participants think of accounting experiments?

Which design features of accounting experiments contribute the most to participant motivation, participant engagement, and perceived similarity to practice? Bart Dierynck and Victor van Pelt are in the process of providing an empirical answer

Bart Dierynck

• October 19, 2020

Effect sizes don't matter for experiments. Or do they?

Effect sizes don't matter for experiments. Or do they?

Some accounting researchers argue that effect sizes do not matter in experiments. In this post, I explain why effect sizes do matter and why they can be particularly valuable for experiments in the field of accounting.

Victor van Pelt

• September 17, 2020

When and how to cluster standard errors in experimental data?

When and how to cluster standard errors in experimental data?

Choosing whether and on which level to cluster standard errors in experimental data turns out to be less straightforward that I originally thought. However, some practical advice for experimental researchers is emerging.

Victor van Pelt

• July 18, 2020

Deploying your experiment to a server

Deploying your experiment to a server

The consequences of the Coronavirus have made it impossible to run experiments in the laboratory. This post shows how you can launch your experiment to participants on the internet.

Christian Peters

• April 28, 2020

Eliciting process variables using scripts

Eliciting process variables using scripts

This post shows how you can elicit process variables in an unobtrusive way using scripts.

Christian Peters

• March 10, 2020

Deception when generating random numbers

Deception when generating random numbers

Many experiments generate random numbers for participants. Yet, the code used to generate those numbers sometimes does not do what we think it does, which could lead to deception when reporting about the number generation process to participants.

Victor van Pelt

• February 24, 2020

Sliders with feedback and without anchoring

Sliders with feedback and without anchoring

Sliders are a great way to elicit input from participants. In this post, I share a few lines of code helping you program sliders with real-time feedback and without anchoring.

Victor van Pelt

• February 19, 2020

Embedding excel spreadsheets in your experiment

Embedding excel spreadsheets in your experiment

Some experiments ask participants to make use of Excel spreadsheets. This post shows how to embed Excel spreadsheets in the code of your experiment.

Christian Peters

• January 8, 2020