wtf is mirror?

Mirror is a research project and accompanying code release that tests how well AI language models can imitate a specific real person's survey answers, compared with a more traditional statistical approach. It uses real survey data from the long running General Social Survey along with a repeat of an earlier academic study on how opinions spread through partisan crowds. The main finding is that a small, purpose built statistical model that learns an embedding, a kind of numeric fingerprint, for each person turns out to predict that person's individual survey answers more accurately than having a language model play the role of that person through a written persona, even when using a strong, more expensive model. Language model personas do better in two other situations though: answering brand new questions the models were never trained on, and modeling how opinions shift inside a simulated crowd of people influencing each other, where they outperform a classic mathematical model of social influence. All of the results are checked against accuracy targets set before the tests were run, and results that did not support the researchers' hopes are reported alongside the ones that did. The repository contains the full data processing pipeline, the evaluation code, the statistical and neural models being compared, the code for generating and testing personas, and scripts to reproduce every experiment in order, starting with a script that fixes the random data split used for testing. Running it requires Python 3.12, and the neural network training step can run on a CUDA GPU, an Apple GPU, or plain CPU automatically depending on what is available. Since the underlying survey data is not owned by the project, you need to separately download it from the two original sources named in the instructions before the processing scripts will work. The written paper describing the study is included as a PDF, and the code is released under the MIT license.