Protocols and Prespecified Analysis

#Record the plan before examining results

A protocol describes what a study is intended to do and how it will be conducted. It usually sets out the research question, eligibility rules, recruitment, measurements, outcomes and follow-up. An analysis plan adds detail about how the data will be prepared and examined. Both should have identifiable versions and dates.

Prespecification means recording decisions before access to the results that could influence them. The relevant point depends on the study. For existing data, researchers should disclose what they already know and when they gained access. A dated plan cannot erase earlier knowledge.

#Make important analytical choices explicit

Useful plans identify the main outcome, comparison and analysis population. They explain handling of missing values, exclusions, repeated measurements and relevant background differences between groups. For a model, they should describe development, tuning and evaluation procedures, including how evaluation data will be kept separate from decisions about the model.

When many outcomes or analyses are planned, specify how the risk of chance findings will be addressed. Describe subgroup analyses and any rules for stopping or changing the study. Advance planning reduces room for selective decisions, but it does not guarantee that the chosen methods are sound.

#Keep changes and exploration visible

Studies sometimes need changes because of practical problems, new information or errors in the original plan. Preserve earlier versions and document what changed, why it changed and whether relevant results were already known. Public registration or a time-stamped repository can make the sequence easier to assess.

Exploratory analysis remains valuable for discovering patterns and developing questions. Label it clearly rather than presenting it as planned confirmation. Report important departures and unfavorable or inconclusive findings. Transparency helps readers distinguish evidence that tests a stated expectation from evidence that suggests what to investigate next.

#Common misunderstandings

A prespecified analysis plan is not a guarantee that a study is well designed or that its findings are correct. It makes intended methods visible, but those methods still need to suit the research question, the data and the study design.

“Prespecified” also does not necessarily mean “recorded before the study began.” Plans may be completed at different stages. What matters is whether researchers could already see results that might influence their choices, and whether the timing is documented.

Changes to a plan are not automatically signs of poor practice. Unexpected measurement problems or mistaken assumptions may justify revisions. The important distinction is between a clearly explained change and an undisclosed decision shaped by favourable results.

Exploratory analyses are not inherently unreliable or unhelpful. They can identify patterns worth investigating. However, a finding discovered through exploration should not be presented as though it answered a question chosen in advance; it generally needs further testing.

#Questions worth asking a clinician

• Where can I find the dated study protocol and analysis plan showing which decisions were recorded before relevant results were known?
• Which primary and secondary outcomes, measurement time points, and statistical analyses were prespecified?
• What rules for excluding participants or observations and handling missing data were specified before results were known?
• What amendments were made to the protocol or analysis plan, why, and were relevant results known at the time?
• Which reported findings came from exploratory analyses, and how are they clearly distinguished from prespecified findings?