Scientific work: TestAS practice testlets

An experiment tests whether one factor causes a change in another. The factor that the experimenter deliberately changes is the independent variable; the quantity that is measured is the dependent variable. All other factors that could influence the result are controlled variables and must be kept constant, so that any difference in the result can be attributed to the independent variable. Such an experiment is often called a fair test.

A control group, or control condition, is treated in the same way as the experimental group except for the factor being tested. It provides a baseline for comparison. Without a control, an observed change might be due to something else entirely, such as time, the season or the act of measuring.

A confounding variable is a factor that changes together with the independent variable and also affects the dependent variable. If plants given a new fertiliser are also placed nearer the window, better growth could be due to the light rather than the fertiliser. Random assignment of subjects to groups helps to spread unknown confounding variables evenly. Repeating measurements and using several subjects per group (replication) reduces the influence of chance and allows the variability of results to be estimated.

Good experimental design also includes blinding where people are involved, a clear hypothesis stated before the experiment, and a plan for how results will be analysed. After the experiment, a careful scientist asks whether the results could have another explanation and whether they would hold under other conditions.

Before collecting data, scientists often write down exactly what they will measure and how, so that decisions cannot be influenced by the results. This practice, called pre-registration, is increasingly common in medicine and psychology, where it helps to prevent selective reporting.

A student wants to test whether the concentration of salt in water affects how quickly cress seeds germinate. She has 200 seeds, eight identical dishes and a greenhouse. She places 25 seeds on damp paper in each dish and waters each dish with 20 mL of solution every day. Her plan is shown below.

A scientific hypothesis is a testable statement that proposes an explanation or predicts a relationship. To be useful, it must be specific enough that some possible observations would contradict it. The philosopher Karl Popper argued that scientific hypotheses can never be proved true once and for all, because future observations might contradict them, but they can be falsified by a single reliable contrary observation. In practice, scientists speak of data that support, weaken or refute a hypothesis, and they accept hypotheses that have survived many serious tests.

Whether data support a hypothesis depends on what the hypothesis predicts. Data support it if they agree with its prediction and would have been unlikely if it were false. Data refute it if they clearly contradict its prediction. Data are uninformative if the hypothesis and its alternatives predict the same result; such data cannot decide between them, however much they seem to fit.

A frequent mistake is confirmation bias: seeking or noticing only the data that agree with one's hypothesis. Another is to conclude that a hypothesis is proven because the data are consistent with it, without checking whether other explanations would fit equally well. Good experiments are designed so that competing hypotheses make different predictions.

A biologist studies a population of lizards on an island. Her hypothesis is: "Lizards with longer legs run faster and therefore survive better when predators are present." She considers several observations, each from a separate study, which are listed below. For each observation, she asks whether it supports the hypothesis, weakens or refutes it, or cannot decide.

The biologist also notes that longer legs might have other effects, for example on climbing ability, and that a single observation rarely settles a question on its own; results from different kinds of studies must be weighed together.

The strength of evidence also depends on the quality of the data: a large, well-controlled study counts for more than a single anecdote.