Rank-order choice-based conjoint experiments: Efficiency and design

https://doi.org/10.1016/j.jspi.2011.01.019Get rights and content

Abstract

In a rank-order choice-based conjoint experiment, the respondent is asked to rank a number of alternatives of a number of choice sets. In this paper, we study the efficiency of those experiments and propose a D-optimality criterion for rank-order experiments to find designs yielding the most precise parameter estimators. For that purpose, an expression of the Fisher information matrix for the rank-ordered conditional logit model is derived which clearly shows how much additional information is provided by each extra ranking step. A simulation study shows that, besides the Bayesian D-optimal ranking design, the Bayesian D-optimal choice design is also an appropriate design for this type of experiments. Finally, it is shown that considerable improvements in estimation and prediction accuracy are obtained by including extra ranking steps in an experiment.

Section snippets

The rank-ordered conditional logit model

Any rank order can be regarded as a sequence of choices made by the respondent. This was used as the starting point for the extension of the conditional logit model to the rank-ordered conditional logit model by Beggs et al. (1981). In their approach, each ranking of a choice set is converted into a number of independent pseudo-choices. In this way, each ranking of alternatives in a choice set is considered as a sequential and conditional choice task. The alternative ranked first is perceived

Designs for the rank-ordered conditional logit model

This section is devoted to the derivation of the Bayesian D-optimality criterion for the rank-ordered conditional logit model and the corresponding D-error. In addition, we discuss how to create efficient designs using this D-optimality criterion. Finally, benchmark designs, with which the Bayesian D-optimal ranking design will be compared, are introduced.

Evaluating the Bayesian Doptimal ranking design

In this section, we compare the different designs in terms of the Db-error and describe the findings of a simulation study by which the performance of the designs in terms of estimation and prediction accuracy was examined. In addition, we measure the improvement achieved by including extra ranking steps in the experiment. In the simulation study, we considered two cases: ranking nine choice sets of size four and ranking 12 choice sets of size three. In the following sections, we only report

Conclusion

In a rank-order choice-based conjoint experiment, the respondent is asked to rank a number of alternatives in each choice set. This way of performing a conjoint experiment offers the important advantage that extra information is extracted about the preferences of the respondent which results in better estimated part-worths and better predicted probabilities assuming constant error variances over the ranks in the rank-ordered conditional logit model.

Although rank-order choice-based conjoint

Acknowledgements

Bart Vermeulen was funded by project G.0611.05 of the Fund for Scientific Research Flanders. We are grateful to Qi Ding for his contribution to the development of the alternating-sample algorithm. He was funded by project NB/06/05 of the National Bank of Belgium.

References (21)

There are more references available in the full text version of this article.

Cited by (13)

  • Preference heterogeneity of local government for implementing ICT infrastructure and services through public-private partnership mechanism

    2022, Socio-Economic Planning Sciences
    Citation Excerpt :

    While making the decision to select the best alternative from the available alternative set, the decision maker compares his/her perceived level of utility from each alternative and selects the alternative that provides the highest level of utility. Discrete choice experiment is a widely applied tool in marketing research to judge the utility level of a product or service separately or in a combination with the attributes attached to the product or service [12]. The discrete choice experiment is done using a number of underpinning theoretical backgrounds to understand and explain consumer behavior [13].

  • Investigating the impact of design characteristics on statistical efficiency within discrete choice experiments: A systematic survey

    2018, Contemporary Clinical Trials Communications
    Citation Excerpt :

    Of the outcomes assessed for each scenario, four studies reported relative D-efficiency [1,30–32], two D-error [33,34], three Db-error (a Bayesian variation of D-error) [30,35,36], and two percentage changes in D-error [34,37]. Of the design characteristics explored, one study explored the impact of attributes on statistical efficiency [1], two explored alternatives [1,30], one explored choice tasks [1], two explored attribute levels [1,32], two explored choice behaviour [33,37], three explored priors [30,31,34], and four explored methods to create the design [30,34–36]. Results are further described below based for each design characteristic.

  • Conjoint analysis of researchers' hidden preferences for bibliometrics, altmetrics, and usage metrics

    2021, Journal of the Association for Information Science and Technology
View all citing articles on Scopus
View full text