Why We Ignore Free-form Communication and Why We Shouldn't
When someone says: “I heard it through the grapevine”, what comes to your mind first? While you may not immediately think of the typical conversation with one of your coworkers at the coffee machine, almost 70% of all communication within organizations disseminates “through the grapevine”, a.k.a. informal communication (Wroblewski, 2018). Yes, this includes the gossip train your secretary is running at your department. But it also includes giving informal advice or feedback to a coworker, reinforcing social norms, building trust, and sharing knowledge.
Communication does not stop here. The remaining 30% of organizational communication happens through more official or formal channels, such as during departmental meetings, client visits, feedback systems, and so on. Such formal settings might be more task-oriented and hierarchical, but there is at least one thing they have in common with informal communication: they both entail free-form communication.
In experimental settings, free-form communication can be defined as communication that is unstructured, real-time, and that has no or limited restriction placed on the communication by the experimenter. This could be an interaction via e-mail or chat, but it also includes audio and face-to-face interactions. Free-form communication can also involve numerous participants simultaneously.
In many accounting experiments, regardless of whether they accommodate formal or informal communication, free-form interaction is often scaled down to a restricted communication setting. However, the overarching discussion is whether research questions in accounting necessitate more complex forms of communication. In this blog, we try to offer more guidance on why and when free-form communication can be helpful in accounting experiments.
Restricted communication
There is no doubt that free-form communication is insightful and, to some degree, reflective of what happens in practice. Still, most experiments in accounting stick to a more restricted form of communication. We believe that a large set of honesty and budgeting experiments offer a fair example of this because they allow participants to send a cost report in the form of a number (e.g., Evans, Hannan, Krishnan, & Moser, 2001; Hannan, Rankin, & Towry, 2010; Maas & Van Rinsum, 2013). Similar to listening to your favorite podcast on Spotify, these experiments accommodate one-sided communication and do not allow the receiver of the report to respond to it.
Other experiments do incorporate a bilateral feature that allows the other party to respond (e.g., Chow Hwang, Liao, & Wu, 1998; Fisher, Maines, Peffer, & Sprinkle, 2002; Majors, 2016), but participants remain restricted in the responses they can provide. Participants can respond either with a numeric value or with a pre-determined written message designed by the experimenter.
While the experimental designs mentioned above might not offer a full-blown representation of reality, the restricted communication context carries several benefits.
- First, there are three important principles in an experimenter’s life: safeguard the random assignment of treatments, ensure that the experiment replicates, and maintain experimental control at all costs. These principles are in fact so well-ingrained in us, experimental researchers, that it almost makes you want to abide by these rules in your daily life too. It goes without saying that we do not disregard these principles. As Steven Kachelmeier (2018) rightly said in his commentary: “Experiments are well-suited to test theories, not to simulate reality.” With this in mind, restricted communication ties in with the third principle and generally allows us to test the pre-selected theory while exercising tight control over the experiment and subsequent analysis.
- Another closely related reason is that pre-defined signals (e.g., a number or pre-formulated message) are often better suited to disentangle mechanisms than free-form communication. Consider the example of budget reporting settings again (e.g., Evans et al., 2001), where the main interest commonly lies in honesty as a psychological mechanism. In such settings, honesty is measured by computing the difference between the budgeted cost and actual cost (i.e., slack), which shows that restricting the communication content to numerical values is sufficiently reflective of the underlying mechanism. If one would allow for free-form communication instead, it could introduce a myriad of social preferences other than honesty.
Garofalo and Rott (2018) provide another insightful study where restricted communication facilitates the disentangling of mechanisms. They investigate whether individuals react differently depending on (1) who communicates bad news or unfair decisions (source) and (2) how that unfair decision is communicated (style). If the researchers had not opted for pre-formulated messages in their experimental design, it would have been unclear whether the communication source or style was driving their results.
Free-form communication
In an earlier post, Jeremy Bentley (2021) correctly explained that a proper experimental design depends on both the participant pool and the research question. Surprise! Because it is no different this time. Juggling the benefits of a restricted communication setting with the benefits of a free-form communication setting also depends, by and large, on the research question. In fact, a variety of research questions, such as those described below, accommodate the use of free-from communication.
The first type of research question that could benefit from the richness of free-form communication relates to the transfer of (private) information, such as sharing knowledge or convincing someone to make a certain decision. Cheap talk offers a good example (e.g., Zhang, 2008; Lundquist et al., 2009; Lunawat, Waymire, & Xin, 2021), where communication between players does not directly affect payoffs. Restricted cheap talk experiments often present a message space that is too limited because it only includes scripted intentions (Dugar & Shahriar, 2018). The authors show that cheap talk is a lot more effective if people can explain why they choose a specific action. It even happens in daily life. For instance, imagine you and your partner are deciding to go to McDonald’s (MD) or Burger King. You suggest choosing MD because you are lazy, and MD is closer to your house. Assuming both restaurants are of equal quality (up for debate), just saying you intend to choose MD would not make the talk to your partner truly cheap and make it less likely to convince him/her that MD is the better option.
The second type of research question that allows the integration of free-form communication pertains to the holistic nature of certain types of interactions, such as those between auditors and their clients or between feedback seekers and givers. If you were present at the 2017 AOS conference, you likely remember Steven Kachelmeier’s memorable presentation, to say the least. We would like to pat him on the back (quite literally) for having the courage to present his slides with his back turned to the audience and for offering us such a fitting illustration of the value of free-form communication. As Kachelmeier argues: “The ceteris paribus nature of their [ref. Bennett & Hatfield, 2018] experiment limits the extent to which their study can address something as rich and complex as human communication.” At the risk of some self-promotion, our own research shows that settings where employees seek real-time feedback from a supervisor (Dierynck & Masselink, 2020) or where calibration committees discuss employee performance (Arshad, Cardinaels, & Dierynck, 2020) are no different.
Why is free-form communication worthwhile?
Despite the nature of the research question, there are also more generic reasons why free-form communication might be useful, and we try to outline some of these reasons below.
- We may obtain different experimental results when restricting communication to numerical values or prefabricated messages. Just a brief reminder: we are not (yet) robots. We are programmed differently when we can communicate whatever is on our mind than when our communication is limited. Relative to restricted communication, free-form communication does not only enhance trustworthiness and trusting (Ben-Ner, Putterman, & Ren, 2011), cooperation in repeated games (Cooper & Kuhn, 2014), and the efficiency of promises (Charness & Dufwenberg, 2006, 2010), but it can also reduce lying (Lundquist et al., 2009).
- We gain mundane realism (i.e., perceived similarity to practice). To elicit the right beliefs, more realistic scenario settings can be specifically valuable in experiments that use more sophisticated participants, such as auditors or corporate directors (Bloomfield, Nelson, & Soltes, 2016). For instance, Bowlin, Hobson, & Piercey (2015) manipulate auditors’ access to an unstructured chat with a manager to investigate whether engaging in such interpersonal chat increases the rate of low-effort audits. It is important to note, however, that there is a tradeoff between mundane realism and internal validity where more mundane realism may come at the cost of internal validity. In this case, theory-testing experimentalists generally seem to favor internal validity, but there could be ways to overcome this issue (more on this in another blog post).
- We can collect richer data in a controlled environment. Especially if the experimenter’s main interest does not lie in a specific aspect of the communication process, collecting data on the communication content, social preferences or other social cues can be valuable. For instance, by quantifying the dialogues using textual analysis software or independent coding, you can shed more light on the mechanism driving the results (e.g., Zhang, 2008; Bowlin et al., 2015; Lunawat et al., 2021).
Given the above reasons, would you feel comfortable with incorporating free-form communication in an accounting experiment? In our experience, the receptiveness of the (management) accounting community to free-form communication is maturing. However, as we discuss in this blog post, a lot of times it all boils down to your research question. No method will be superior in all aspects.
So, whenever you are wearily waiting for your Monday morning coffee at the coffee machine, try to engage in a free-form interaction with your coworker, because real-life communication really is a lot more unique than the average prefabricated “Hi, my reported cost is equal to 5.00 Lira per unit” message. As George Bernard Shaw rightly said: “The single biggest problem in communication is the illusion that it has taken place.”
