Jooyeol Kim Independent researcher

Working paper · Version 1.2 · July 2026 · Not peer reviewed
Submitted to SSRN and currently under review. SSRN Abstract ID: 7174178.

You Agreed to the Rule. Did You Agree to the Experiment?

ICML 2026 and the missing boundary between review duty and experimental participation

Agreement to perform a role under institutional rules does not automatically settle agreement to have that role used as an experimental condition.

Read the working paper (PDF) View the SSRN record View Figure 1
Editorial illustration of an audition contestant being directed toward experiment rooms A and B after saying that she came to audition.
Editorial illustration. A fictional idol-audition contestant says, “I came here to audition.” The producer answers, “Great. Now you choose A or B.” This image is a public-facing illustration of the paper’s central problem, not Figure 1 from the manuscript.

The question

ICML 2026 assigned reviewers enforceable AI-use rules while also using some policy assignments to compare outcomes under Policy A and Policy B. Some reviewers violated the assigned rule, and that breach matters. It does not answer a different question:

When did agreement to a review rule become agreement to participate in an experiment?

The paper does not allege a legal violation, an IRB failure, or research misconduct. It asks a narrower governance question: whether a reviewer could decline the experimental use of an assignment while remaining in the ordinary reviewing role, particularly where reciprocal-review conduct could affect the reviewer’s own submissions.

The missing third option

The paper uses a fictional televised idol audition to isolate the distinction. A contestant enters to audition. The organizer then assigns her to one of two experimental teaching conditions. Leaving means surrendering the opportunity altogether.

The missing option is not another experimental condition. It is:

I will remain in the underlying process, but I do not agree to the experimental use layered onto it.

Why the comparison matters

The ICML policy made the rule, enforcement mechanism, and possible experimental comparison visible. What its public materials did not separately describe was a route for declining experimental use while remaining in the reviewer role.

NeurIPS 2026 provides a contrast. Its AI-assisted reviewing experiment used a separate opt-in boundary for authors and reviewers, while ordinary reviewing obligations and conference authority remained in place. ICML’s own 2023 ranking experiment also separated analysis from that year’s paper decisions.

These comparisons do not prove wrongdoing. They show that an institution can preserve an ordinary gate while marking where experimental participation begins.

Figure 1 · The missing third option

Figure comparing an idol-audition analogy, ICML 2026 policy assignment and experimental use, and NeurIPS 2026 separate opt-in structure.
Figure 1. The audition analogy illustrates why leaving an opportunity is not equivalent to declining only an experiment. The lower panels contrast ICML’s publicly described combination of policy assignment, enforcement, and experimental comparison with NeurIPS’s separate opt-in boundary. The diagram is a simplified representation of the relevant participation structures.

Core conclusion

ICML explained the rule, documented the breach, defended enforcement, and described the experimental comparison. The remaining boundary question is whether reviewers had a separately described route to decline experimental use while remaining in the ordinary role.

You agreed to the rule. Did you agree to the experiment?

A serious institution should be able to answer both questions separately.

Citation

Kim, Jooyeol. 2026. “You Agreed to the Rule. Did You Agree to the Experiment? ICML 2026 and the Missing Boundary Between Review Duty and Experimental Participation.” Working paper, Version 1.2.

Status and disclosure