Statistical Modeling, Causal Inference, and Social Science » Decision Theory

It’s the eternal question . . . what do you want, if given these three options: (a) 54 Jamaican beef patties. (b) 1/216 of a conference featuring some mixture of active and washed-up business e...

https://statmodeling.stat.columbia.edu/2024/04/24/for-that-price-he-couldve-had-54-jamaican-beef-patties-or-1-216-of-a-conference-featuring-gray-davis-grover-norquist-and-a-rabbi/

Emily Tanner-Smith writes: Remote/Hybrid Postdoc Opportunity—join us as a Post-Doctoral Scholar at the HEDCO Institute for Evidence-Based Educational Practice in the College of Education at the...

https://statmodeling.stat.columbia.edu/2024/04/23/postdoc-opportunity-at-the-hedco-institute-for-evidence-based-educational-practice-in-the-college-of-education-at-the-university-of-oregon/

James Heathers reports on the article, “Contagion or restitution? When bad apples can motivate ethical behavior,” by Gino, Gu, and Zhong (2009): There is some sentiment data reported in Exper...

https://statmodeling.stat.columbia.edu/2024/04/19/the-data-are-on-a-1-5-scale-the-mean-is-4-61-and-the-standard-deviation-is-1-64-whats-so-wrong-about-that/

I keep worrying, as with a loose tooth, about news media elites who are going for the UFOs-as-space-aliens theory. This one falls halfway between election denial (too upsetting for me to want to ...

https://statmodeling.stat.columbia.edu/2024/04/13/evidence-desire-support/

Ron Bloom points us to this wonderful article, “The Ethics of Belief,” by the mathematician William Clifford, also known for Clifford algebras. The article is related to some things I’ve wr...

https://statmodeling.stat.columbia.edu/2024/04/10/acq/

Here’s something from from Witold’s slides on baggr, an R package (built on Stan) that does hierarchical modeling for meta-analysis: Overall goals: 1. Implement all basic meta-analysis models...

https://statmodeling.stat.columbia.edu/2024/04/09/heres-something-you-should-do-when-beginning-a-project-and-in-the-middle-of-a-project-and-in-the-end-of-the-project-clearly-specify-your-goals-and-also-specify-whats-not-in-your-goals-set/

Following our recent post on the latest Dishonestygate scandal, we got into a discussion of the challenges of simulating fake data and performing a pre-analysis before conducting an experiment. Y...

https://statmodeling.stat.columbia.edu/2024/04/04/gold/

A recent issue of the New Yorker had two striking stories of bad parenting. Margaret Talbot reported on a child/adolescent-care center in Austria from the 1970s that was run by former Nazis who w...

https://statmodeling.stat.columbia.edu/2024/04/02/bad-parenting-in-the-news/

Here’s Cowen’s post, and here’s our paper: Moderation in the pursuit of moderation is no vice: the clear but limited advantages to being a moderate for Congressional elections Andrew Gelman...

https://statmodeling.stat.columbia.edu/2024/03/29/every-time-tyler-cowen-says-median-voter-theorem-still-underrated-hail-anthony-downs-im-gonna-point-him-to-this-paper/

This is Jessica. A couple weeks I wrote a post in response to Ben Recht’s critique of conformal prediction for quantifying uncertainty in a prediction. Compared to Ben, I am more open-minded ab...

https://statmodeling.stat.columbia.edu/2024/03/15/conformal-prediction-and-people/

Adam Zelizer writes: I saw your post about the underpowered COVID survey experiment on the blog and wondered if you’ve seen this paper, “Counter-stereotypical Messaging and Partisan Cues: Mov...

https://statmodeling.stat.columbia.edu/2024/03/13/their-signal-to-noise-ratio-was-low-so-they-decided-to-do-a-specification-search-use-a-one-tailed-test-and-go-with-a-p-value-of-0-1/

Tom Vladeck writes: I thought you may be interested in some internal research my company did using a conjoint experiment, with analysis using Stan! The upshot is that we found that vaccine hesita...

https://statmodeling.stat.columbia.edu/2024/03/11/putting-a-price-on-vaccine-hesitancy-bayesian-analysis-of-a-conjoint-experiment/

This is Jessica. In a previous post I mentioned methodological problems with studies of AI-assisted decision-making, such as are used to evaluate different model explanation strategies. The typic...

https://statmodeling.stat.columbia.edu/2024/03/06/defining-optimal-reliance-on-model-predictions-in-ai-assisted-decisions/

This is a long post, so let me give you the tl;dr right away: Don’t use the word “whom” in your dating profile. OK, now for the story. Fasten your seat belts, it’s going to be a bumpy nig...

https://statmodeling.stat.columbia.edu/2024/03/06/mindlessness-in-the-interpretation-of-a-study-on-mindlessness/

This is Jessica. Over on substack, Ben Recht has been posing some questions about the value of prediction bands with marginal guarantees, such as one gets from conformal prediction. It’s an int...

https://statmodeling.stat.columbia.edu/2024/02/20/when-do-we-expect-conformal-prediction-sets-to-be-helpful/

I just read the short book, “Uncertainty in games,” by Greg Costikyan. It was interesting. His main point, which makes sense to me, is that uncertainty a key part of the appeal of any game. H...

https://statmodeling.stat.columbia.edu/2024/02/15/uncertainty-in-games-how-to-get-that-balance-so-that-theres-a-motivation-to-play-well-but-you-can-still-have-a-chance-to-come-back-from-behind/

Mark Palko points us to a recent update by Robert Yeh et al. of the famous randomized parachute-jumping trial: Palko writes: I also love the way they dot all the i’s and cross all the t’s. Th...

https://statmodeling.stat.columbia.edu/2024/02/11/clinical-trials-that-are-designed-to-fail/

A question came up about the effects of school funding and student performance, and we were referred to this review article from a few years ago by Larry Hedges, Terri Pigott, Joshua Polanin, Ann...

https://statmodeling.stat.columbia.edu/2024/02/10/if-school-funding-doesnt-really-matter-why-does-everyone-want-their-kid-to-go-to-a-well-funded-school/

This one’s important. Matt Lerner points us to this report by Rosie Bettle, Replicability & Generalisability: A Guide to CEA discounts. “CEA” is cost-effectiveness analysis, and by “disco...

https://statmodeling.stat.columbia.edu/2024/02/07/replicability-generalisability-applying-a-discount-factor-to-cost-effectiveness-estimates/

This is Jessica. A while back on the blog I shared some opinions about studies of human-decision making, such as to understand how visualizations or displays of model predictions and explanations...

https://statmodeling.stat.columbia.edu/2024/01/30/minimum-criteria-for-studies-evaluating-human-decision-making/

Alexey Guzey asks: How much have you thought about AI and when will AI be able to do scientific research both cheaper and better than us, thus effectively obsoleting humans? My first reply: I gue...

https://statmodeling.stat.columbia.edu/2024/01/29/when-will-ai-be-able-to-do-scientific-research-both-cheaper-and-better-than-us-thus-effectively-obsoleting-humans/

Here’s the link: Learning from mistakes Andrew Gelman, Department of Statistics and Department of Political Science, Columbia University We learn so much from mistakes! How can we structure our...

https://statmodeling.stat.columbia.edu/2024/01/23/learning-from-mistakes-my-online-talk-for-the-american-statistical-association-230pm-tues-30-jan-2024/

I’ve often appealed to “common sense” or “face validity” when considering unusual research claims. For example, the statement that single women during certain times of the month were 20...

https://statmodeling.stat.columbia.edu/2024/01/20/regarding-the-use-of-common-sense-when-evaluating-research-claims/

Since Aki and Andrew are doing it…  Published: Dongping Zhang, Jason Hartline, and Jessica Hullman (2024). Designing Shared Information Displays for Agents of Varying Strategic Sophistication....

https://statmodeling.stat.columbia.edu/2024/01/16/progress-in-2023-jessica-edition/

I just read this magazine article by Nikhil Krishnan on the philosophy of Aristotle. As a former physics student, I’ve never had anything but disdain for that ancient philosopher, who’s famou...

https://statmodeling.stat.columbia.edu/2024/01/16/this-post-is-not-really-about-aristotle/

Greg Mayer points to this news article, “The Simple Nudge That Raised Median Donations by 80%,” which states: A start-up used the Hebrew word “chai” and its numerical match, 18, to bump u...

https://statmodeling.stat.columbia.edu/2024/01/05/what-to-trust-in-the-newspaper-example-of-the-simple-nudge-that-raised-median-donations-by-80/

How horrible. I remember when The Onion started. They were so funny and on point. And now . . . What’s the point of even having The Onion if it’s running plagiarized material? I mean, yeah, s...

https://statmodeling.stat.columbia.edu/2024/01/01/the-onion-ok-an-onion-affiliate-site-is-plagiarizing/

This is Jessica. A paper called “Decoupling Judgment and Decision Making: A Tale of Two Tails” by Oral, Dragicevic, Telea, and Dimara showed up in my feed the other day. The premise of the pa...

https://statmodeling.stat.columbia.edu/2023/12/28/judgments-versus-decisions/

Ben Recht writes: I’m a fan of physician-blogger John Mandrola. He had a nice response to your blog, using it as a jumping-off point for a short tutorial on his rather conservative approach to ...

https://statmodeling.stat.columbia.edu/2023/12/26/john-mandrolas-tips-for-assessing-medical-evidence/

Earlier today we posted something on our recent paper with Erik van Zwet et al., “A New Look at P Values for Randomized Clinical Trials.” The post had the provocative title, “Bayesians movi...

https://statmodeling.stat.columbia.edu/2023/12/24/explaining-that-line-bayesians-moving-from-defense-to-offense/

“One of the things that makes scientific research hard is that one is usually not sure what hat one should be wearing in the given situation.” — David H. Krantz, 1938-2023 It’s been a bun...

https://statmodeling.stat.columbia.edu/2023/12/19/dave-krantz/

OK, here’s a hypothetical scenario. You’re a researcher. You look carefully at one of the most cited-papers in an important subfield—perhaps the most influential paper published there in th...

https://statmodeling.stat.columbia.edu/2023/12/18/suppose-you-realize-that-a-paper-on-an-important-topic-with-thousands-of-citations-is-fatally-flawed-where-should-the-correction-be-published/

Robert Farley shares this amusing story of a city that contracted out the routing of its school buses to a company that “uses artificial intelligence to generate the routes with the intent of r...

https://statmodeling.stat.columbia.edu/2023/12/14/ai-bus-route-fail-typically-the-most-important-thing-is-not-how-you-do-the-optimization-but-rather-what-you-decide-to-optimize/

1. When can we trust? How can we navigate social science with skepticism? 2. Why I’m not convinced by that Quebec child-care study 3. 20 years on 1. When can we trust? How can we navigate socia...

https://statmodeling.stat.columbia.edu/2023/12/13/chal/

The post below addresses a bunch of specifics about Harvard, but for the key point, jump to the last paragraph of the post. Problems about Harvard A colleague pointed me to this post by Christoph...

https://statmodeling.stat.columbia.edu/2023/12/11/concerns-about-misconduct-at-harvards-department-of-government-and-elsewhere/

Following up on the post that appeared this morning . . . “Using simulation from the null hypothesis to study statistical artifacts” is another way of saying “hypothesis testing.” I like ...

https://statmodeling.stat.columbia.edu/2023/12/10/why-i-like-hypothesis-testing-its-another-way-to-say-fake-data-simulation/

Greg Mayer writes: Have you seen this paper by Frank Corotto, recently posted to a university depository? It advocates a way of doing box plots using “comparative confidence intervals” based ...

https://statmodeling.stat.columbia.edu/2023/12/02/on-a-proposal-to-scale-confidence-intervals-so-that-their-overlap-can-be-more-easily-interpreted/

Following up on today’s post, “Why I continue to support the science reform movement despite its flaws,” it seems worth linking to this post from 2019, about the way in which some mainstrea...

https://statmodeling.stat.columbia.edu/2023/11/29/back-to-basics-how-is-statistics-relevant-to-scientific-discovery-2/

Someone pointed me to a controversial article written a couple years ago. The article remains controversial. I replied that it’s a topic that I’ve not followed any detail and I’ll just defe...

https://statmodeling.stat.columbia.edu/2023/11/16/oooh-im-not-gonna-touch-that-tar-baby/

Last week we ran a post, “Simulations of measurement error and the replication crisis: Maybe Loken and I have a mistake in our paper?”, reporting some questions that neuroscience student Fede...

https://statmodeling.stat.columbia.edu/2023/11/11/simulations-of-measurement-error-and-the-replication-crisis-update/

The webpage is maintained by John Hunter, son of Box’s collaborator William Hunter, and I came across it because I was searching for background on the paper-helicopter example that we use in ou...

https://statmodeling.stat.columbia.edu/2023/11/10/hey-here-are-some-amazing-articles-by-george-box-from-around-1990/

This has come up before here, and it’s also in Section 16.4 of Regression and Other Stories (chapter 16: “Design and sample size decisions,” Section 16.4: “Interactions are harder to esti...

https://statmodeling.stat.columbia.edu/2023/11/09/you-need-16-times-the-sample-size-to-estimate-an-interaction-than-to-estimate-a-main-effect-explained/

Zac McEachran writes: I am a Hydrologist and Flood Forecaster at the National Weather Service in the Midwest. I use some Bayesian statistical methods in my research work on hydrological processes...

https://statmodeling.stat.columbia.edu/2023/10/31/hydrology-corner-how-to-compare-outputs-from-two-models-one-bayesian-and-one-non-bayesian/

My colleague and I wrote a paper, and someone suggested we submit it to the journal Nature Communications. Sounds fine, right? But then we noticed this: Hey! We wrote the damn article, right? The...

https://statmodeling.stat.columbia.edu/2023/10/28/we-were/

Sam Berchuck writes: I wanted to bring your attention to a postdoc opportunity in my group at Duke University in the Department of Biostatistics & Bioinformatics. The full job ad is here: https:/...

https://statmodeling.stat.columbia.edu/2023/10/24/postdoc-on-bayesian-methodological-and-applied-work-to-optimize-patient-care-using-stan-in-north-carolina/

Palko points to this amusing juxtaposition: I was curious so I googled to find the original story, “Forecast for US Recession Within Year Hits 100% in Blow to Biden,” by Josh Wingrove, which ...

https://statmodeling.stat.columbia.edu/2023/10/22/bloomberg-news-and-its-embarrassing-calibration-error/

Bethan Staton and Chris Cook write: A Cambridge university professor who copied parts of an undergraduate’s essays and published them as his own work will remain in his job, despite an investig...

https://statmodeling.stat.columbia.edu/2023/10/18/academia-corner-new-candidate-for-american-statistical-associations-founders-award-enduring-contribution-award-from-the-american-political-science-association-and-edge-foundation-just-dropped/

Bert Gunter writes: This article in today’s NYTimes is a hoot, and might be grist for the lighter side of your blog … or maybe the heavier if you want to get into statistical fake detection, ...

https://statmodeling.stat.columbia.edu/2023/10/17/why-arent-there-more-fake-reviews-on-yelp-etc/

Hey, it’s a free country. I guess it makes sense that the guy who thinks he discovered a new continent wants to hang out with the dude who bombed an old one. Bummer that the Theranos founders w...

https://statmodeling.stat.columbia.edu/2023/10/14/nudgelords-2/

Igor Grossmann writes: Together with colleagues from Innsbruck, Vienna, and Waterloo, we are launching an exciting new project and we would like to invite you to participate in (and your PostDocs...

https://statmodeling.stat.columbia.edu/2023/10/02/call-for-participation-crowd-sourcing-wisdom-of-the-crowd-mechanisms-win-10000/