Project 2 Outline: Improving belief elicitation through theory and experiments

​Supervisors

Prof Gerhard Riener (Economics), Dr Simon Columbus (Psychology)

Project Description

This interdisciplinary MScRES project will develop a new approach to measuring subjective beliefs by integrating incentive-compatible elicitation methods from experimental economics with psychometric latent variable models. While beliefs are central to economic models of decision-making under uncertainty, existing elicitation techniques are rarely evaluated as measurement instruments. From a behavioural and cognitive-science perspective, belief elicitation can be seen as the measurement of internal probabilistic representations under uncertainty, raising questions about how cognitive processing of the task influences observed responses.

The project bridges three complementary traditions: (i) the Manski approach to probabilistic expectations in surveys, (ii) incentivised behavioural measures using strictly proper scoring rules, and (iii) psychometrics’ focus on construct validity and measurement invariance. Its central question is whether different commonly used elicitation mechanisms capture the same latent belief or introduce systematic method-specific distortions driven by incentives, cognitive load, or individual differences. Economic theory contributes incentive-compatible measurement, psychometrics provides latent-variable modelling and tests of measurement, and cognitive science offers insight into how individuals mentally represent beliefs.

This project has three components, which may be emphasised depending on the successful applicant’s background and interests: (i) develop a theoretical framework within which to evaluate belief elicitation methods, (ii) re-analyse existing economic data sets with a focus on measurement reliability across important subgroups, and (iii) test theoretical predictions in a multi-method ‘horse race’ between different belief elicitation mechanisms.

In the main empirical part of the project, the student will implement a preregistered within-subjects experiment in which participants report beliefs about clearly defined probabilistic events using multiple elicitation formats (e.g., unincentivized probability scales and incentivized scoring rules).  The study will combine econometric and psychometric modelling approaches to identify method- and subgroup-specific heterogeneity in elicited beliefs. It is designed to be completed within a single academic year, ensuring feasibility within the given time frame. If time permits, open-ended “what comes to mind” responses could be analysed to study cognitive representations underlying belief formation.

By combining economic theory, experimental design, and latent-variable modelling, the project establishes a psychometrically grounded framework for belief elicitation which will improve the comparability of belief measures across populations and strengthens the foundations of behavioural research and policy-relevant survey measurement.

Skills & Requirements

Ideal candidates should have a bachelor’s degree in economics, psychology, or another quantitative (social) science, with a strong quantitative background. Experience with a statistical programming language (e.g. R, Python, Stata) is essential. Prior knowledge of belief elicitation methods or latent variable modelling is advantageous but not required.

Further details and application process

This project has been awarded St Andrews Business School Research Bursary of £1000 for research expenses. If you are planning to apply for this project, first contact the supervisors Prof Gerhard Reiner ([email protected]) and Dr Simon Columbus ([email protected]), to discuss eligibility and obtain approval. Once an agreement is reached, you can proceed to submit your formal application.