Prediction Markets as institutional forecasting support systems

Abstract

An attractive feature of Prediction Markets (PMs) is that they provide economic incentives for informants to share unique information. It is unclear whether PMs are appropriate for applications with few knowledgeable informants as is the case for most institutional forecasting tasks. Hence, we compare the performance of small PMs with traditional judgment-based forecasting approaches. Our results show that forecasts from small PMs outperform traditional approaches in settings of high information-heterogeneity (i.e., where the amount of unique information possessed by informants is relatively high) and are no worse in settings of low information-heterogeneity.

Introduction

One of the most challenging tasks firms face is to make the most effective use of their extant internal information. That challenge applies especially to forecasting, which the forecasting literature (see [18], for example) describes as both “art and science”, with “art” referring to forms of managerial judgment and the “science” referring to statistical or econometric analysis based on historical data. But forecasts in organizations are often required in situations where there are no data-based approaches available. Furthermore, cost or secrecy concerns often preclude customer surveys as data sources for such forecasts. Hence, many organizations rely on informants (sales representatives, research analysts, business managers and project team members) for the development of (judgment-based) forecasts [20], what we will refer to here as “institutional forecasting.”

Forecast accuracy is often improved when informant-based forecasts rely on multiple rather than a single informant because (a) informants seldom have access to all relevant information and (b) using multiple informants lowers the error component of the group's forecast [3]. Group discussion enables informants to share information so that groups can access a larger pool of information than can any single informant acting alone [14], [43]. However, research has also shown that groups are sometimes ineffective at exchanging information [32] and that much unique information known to a single or only some individuals is never shared with the group [48].

Stasser and Titus [49] claim that groups can benefit from pooling members' information, particularly when members individually have partial and biased information but collectively can compose a less biased characterization of the decision alternatives. However, in an empirical application of their approach they find that group members often fail to effectively pool their information because discussion is dominated by commonly held information and information that supports members' existent viewpoints. This result implies that groups that share information interactively through group discussion will normally reach consensus but will neither appropriately correct for nor effectively pool members' complementary expertise and knowledge.

The increasing ease of interconnectivity and the proliferation of web conferencing tools like LiveMeeting and GotoMeeting are continually making it simpler for groups of people to work together through computer networks. A Group Forecasting Support System within such an environment is a communication and coordination process that structures the process of communication and information sharing. Dennis [14] studied the benefits of Group Support Systems (GSSs) and found that groups using a GSS exchanged 50% more information than verbally interacting groups, permitting them to include the best alternative amongst those considered for selection. However, he also found that very few groups ended up selecting the optimal decision, indicating that the GSS was not able to help the groups to process the information that they had optimally. Sia et al. [44] report that anonymous or dispersed computer-mediated communication settings are required for group discussion to lead to the strong polarization that may be necessary to identify a non-consensus choice.

Prediction Markets (PMs), also called information markets or virtual stock markets, represent an information technology based forecasting platform that may have the potential to address some of the challenges that traditional GSS's face in the institutional forecasting context. PMs enable informants to exchange information and should be considered as a type of organizational Group Forecasting Support System (GFSS). However, unlike traditional Group Support Systems [14], [16], PMs place an incentive on exchanging “unique” information with other informants in the market because doing so will lead to higher pay-offs for the informant possessing such unique information.

PMs have been successfully applied to predict election outcomes (e.g., the Iowa Electronic Markets [7]), the success of movies and impact of stars (e.g., the Hollywood Stock Exchange [17], [22], [38]), sports results [47], product concepts and new product ideas [12], [45]; and future economic outcomes (e.g., economicderivatives.com). In an inter-organizational setting Guo et al. [30] propose a Prediction Market for information sharing within supply chains. LaComb et al. [34] report that several GE businesses experimented with PMs to support idea generation and group-decision making. Ostrover [36] cites several PM applications in organizations: e.g. Hewlett Packard (HP) uses PMs to forecast sales, financial, and accounting results while Eli Lilly uses a PM to identify those drugs in the early stages of development most likely to win US Federal Drug Administration approval. The results of these applications demonstrate the potential value of the PM approach for institutional forecasting in settings with a large number of participants. However, little work has been reported on PMs in settings with few knowledgeable participants; indeed, scholars have stressed that such situations can lead to markets with low liquidity where small changes in supply and/or demand can have a large impact on market prices [23]. Research on traditional GSSs [24] suggests that these systems are also much more effective for larger groups, Yet, many practical institutional forecasting situations actually involve relatively few (knowledgeable) informants, which underlines the need for approaches that are effective for small groups.

Spann and Skiera [46] report a study of a (single) small PM with twelve employees at a large German mobile phone operator to forecast the usage of five different mobile phone services in a specific month. The PM showed better forecasting accuracy than several competing model-based forecasting approaches. The authors suggest that as the heterogeneity in market informants' knowledge increases, the PM's forecasting error declines relative to other approaches. If this suggestion is correct, it implies that small PMs can be effective in general and may do especially well relative to alternatives if the informants possess relatively large amounts of “unique” or “partially shared” information [14].

Our objective in this paper is to analyze whether the forecasting accuracy observed for PMs in other settings applies to forecasting settings with few knowledgeable informants. We consider whether, why and when small PMs are likely to outperform more traditional judgment-based approaches towards institutional forecasting [33], which we call the Combined Judgmental Forecasts (CJF) approach, or the Key Informant (KI) approach. We compare the accuracy of the PM approach with these alternatives for several forecasting tasks (i.e., forecasting the future value of two financial indices and forecasting the point spread of two college football games).

Our results show that the PM approach performs at least as well as these alternatives in markets with few participants. We find that for tasks we characterize as high in information-heterogeneity (i.e., predicting football point spreads), forecasts made through the PM approach are more accurate than those developed by the CJF approach or by the KI approach, while for low information-heterogeneity tasks (i.e., predicting financial indices), we find that the PM approach performs no worse than the other approaches.

The paper proceeds as follows: we first review alternative approaches to support institutional forecasting, including the opportunities afforded by the PM approach. Then we review literature on how PMs operate and how their operation may be affected by the number of market informants. We contrast the differences between the PM approach and the more traditional approaches (CJF or KI), which leads to our hypothesis about the expected differences in forecasting accuracy between those three approaches. We next describe the design of our empirical study and present our results. We conclude with a discussion of those results and their implications.