Sample audit · robustness implementation

When a robustness check changes the ranking it was said to preserve.

Pärnamets et al. (2026), open-mindedness across 68 countries. Audit question: does the publicly archived reliability correction actually preserve the paper's ordinal predictor conclusions?

Executive finding

No — not for the conspiracy-beliefs outcome.

The archived final-revision R Markdown reuses the physical-hygiene reliability object for 13 later multi-item measures. A bug-only repair, propagated through an independently reconstructed posterior analysis, shifts the focal open-mindedness versus collective-narcissism ordering from essentially certain CN dominance to 96.7% posterior probability of OM dominance.

Median |βOM| − |βCN| · buggy
−0.03729
Median |βOM| − |βCN| · repaired
+0.02078
P(OM > CN) · repaired
96.67%
Interpretation boundary: the repaired 95% equal-tailed interval for the focal difference narrowly includes zero. The audit establishes that the published ordinal robustness conclusion is not preserved for this outcome; it does not establish the reverse substantive theory as true.
Verified implementation defect

A stale object is reused inside the reliability correction.

new reliability stored in s
→
old ss still appended
→
wrong reliability vector
→
disattenuated matrix
→
partial coefficients
→
ordinal claim
Inspected vector

21 entries

Two use their own calculated reliability correctly.

Stale reuse

13 later multi-item entries

New reliability is calculated, but the old hygiene object is still read.

Hard-coded

6 entries at .85

Kept separate from the stale-object pattern.

Independent reconstruction

The consequence is traced rather than inferred from the code smell.

CheckResult
Posterior draws4 chains × 1,500 post-warmup = 6,000
Sampler health0 divergences; 0 max-treedepth hits; minimum E-BFMI 0.660
Repair-target convergence210 residual correlations; max R-hat 1.003655; min bulk ESS 3981; min tail ESS 3496
Archived-posterior consistency210 / 210 new residual-correlation medians fall within archived 95% intervals
Paired correction effectΔD median +0.05825; 95% CrI [+0.05542, +0.06112]; ΔD > 0 in all 6,000 draws
Outcome specificity

The defect does not collapse every result.

OutcomeBuggy top orderingAfter bug-only repair
Physical contactOM rank 1: 100%OM rank 1: 100%
Policy supportOM rank 1: 100%OM rank 1: 100%
Physical hygieneBelonging: 100%Belonging: 99.93%
Conspiracy beliefsCN: 99.98%OM: 96.67%
Supported

Stale-object reliability defect; material shift in the focal posterior ordering; published ordinal robustness conclusion not preserved for this outcome.

Not established

Invalidity of the original uncorrected multivariate regression; change in every outcome; fraud, intent, misconduct, or falsity of the paper as a whole.