## What is science?

8 juillet 2011

Science isn’t about what’s known. Nor is it about what isn’t known. At its most basic level, SCIENCE is nothing more than a process of playing games and making puzzles that may or may not tell us something about the world or ourselves in that world. When thought of in this way, it’s obvious that we all ‘do science‘ hundreds of times a day every day, which is about discovering and exploring through interaction. When interaction is made conscious and combined with reflection, that is ‘science‘.

Beau Lotto (on « Street Science« )

Publicités

## Marcher dans les nuages, puis au-delà

28 juin 2011

A force de l’écouter ce passage, elle en venait à ressentir un étrange et long frisson la parcourir. C’est un peu par surprise qu’elle était retombée dessus, d’ailleurs. Bien sûr, le passage avait déjà éveillé sa curiosité, mais sans la retenir ni l’attirer si loin. Le thème est annoncé dès le début pourtant, mais les explosions de cuivres étaient toujours parvenus à lui faire oublier. Puis les rouleaux des cordes suivent, ponctués de touches boisées s’amplifiant en triomphes cuivrés. Et puis, au détour d’une phrase de conclusion temporaire, les vents nous invitent à leur suite, bientôt accompagnés puis remplacés par une vision de doigts innombrables mais légers, comme autant de petits pas sur le sable, des petits pieds fragiles mais décidés, desquels apparaissent des poignets dont les fines attaches déclenchent à chaque tour des volutes s’élevant de manière bonhomme, avant de s’évanouir dans l’azur, presque aussi vite qu’elles sont apparues.

Aux environs de la cinquantième écoute, elle finit par relier cette vision musicale au souvenir de l’ascension. A l’issue de ces deux jours, alors à moitié assoupie sur les banquettes en noyer du vieux train la ramenant dans la vallée, elle avait pris conscience, quasiment violemment, que c’en était fini de cette marche. Non pas du sommet, parce qu’il y a toujours les photos pour orgueilleusement rappeler aux autres ce que l’on a « fait », mais bien de l’ascension, de cette traversée d’un pied sur l’autre, vers un quelque part que l’on rencontre au détour d’un chemin. C’est d’abord sur le symbole même de la cordée que son esprit embrumé s’était accroché, ses yeux suivant rêveusement les flocons de lumière des encordés s’enfonçant vers le ciel noir. Encordé, oui, elle l’était, mais seule aussi. Insondable paradoxe que celui d’être à la fois si proche de ses partenaires d’aventure, relié physiquement à eux par un cordon quasi maternel, et à la fois si loin, concentré sur le fait de garder ce cordon tendu, un peu mais pas trop, loin des crampons tranchants, et regrettant cette distance qui étouffe les paroles à peine imaginées, forçant les marcheurs à se parler à eux-même, du monde et d’eux-même. Et c’est là, à cet instant, après que la lune se soit couchée, alors que le cône d’ombre projeté par le sommet enfonçait les nuages environnants, que la glaciale blancheur s’est muée en un rose timide, à la fois doux et brillant, et chaque pas la faisait alors entrer dans un monde où les yeux s’agrippent à cette couleur que l’on voudrait caresser, où le souffle s’allonge, s’apaise, et le tumulte cérébral s’évanouit silencieusement, sur la pointe des pieds.

(Il s’agit du du deuxième mouvement de la symphonie n° 5 de Beethoven, et merci à LG pour la photo.)

## Qu’est-ce que l’engagement ?

9 juin 2011

L’engagement, cette décision pour une cause imparfaite.

Jorge Semprun

(via l’hommage de Régis Debray)

## Uncertainty and collective science

22 mai 2011

There are some things that you, the reader of this preface, know to be true, and others that you know to be false; yet, despite this extensive knowledge that you have, there remain many things whose truth or falsity is not known to you. We say that you are uncertain about them. You are uncertain, to varying degrees, about everything in the future; much of the past is hidden from you; and there is a lot of the present about which you do not have full information. Uncertainty is everywhere and you cannot escape from it. Truth and falsity are the subjects of logic, which has a long history going back at least to classical Greece. The object of this book is to tell you about work that has been done in the twentieth century about uncertainty.

[…]

Research is carried out by individuals and often the best research is the product of one person thinking deeply on their own. For example, relativity is essentially the result of Einstein’s thoughts. Yet, in a sense, the person is irrelevant, for most scientists feel that if he had not discovered relativity, then someone else would; that relativity is somehow ‘‘out there’’ waiting to be revealed, the revelation necessarily being made by human beings but not necessarily by that human being. This may not be true in the arts so that, for example, if Shakespeare had not written his plays it would not follow that someone else would have produced equivalent writing. Science is a collective activity, much more so than art, and although some scientists stand out from the rest, the character of science depends to only a very small extent on individuals and what little effect they have disappears over time as their work is absorbed into the work of others.

Dennis Lindley, Understanding uncertainty

Je ne sais pas pour vous, mais moi, quand je lis ça, j’ai envie de m’installer dans un fauteuil et lire toute la journée.

## Do you know what heritability is, do you?

9 mai 2011

Genetics is the study of genes. When they were « invented » (i.e. conceptualized) in the 19th century, genes were defined as units of heredity. Thanks to the revolution of molecular biology, we now know that a gene is a section of DNA (although it is still possible to debate on the precise meaning of this statement). But what I am interested here, is to ask a few questions about quantitative genetics, and especially heritability.

Indeed, this notion of heritability is key in genetics (and, by extension, in biology as a whole) but it is often obscure for many biologists (notably for me, until I decided to seriously read some papers, and thus write this post to be sure I understood properly). And you, the reader, even if you think you know everything about heritability, I hope it can still be worth the reading.

* * *

But let’s start with basics: Gregor Mendel and the birth of genetics. This Austrian monk was interested in inheritance (passing of traits from parents to offsprings). To study this phenomenon, he worked with pea plants and chose seven characters to study, but let’s focus on one: seed color.

Mendel observed that some pea plants had green seeds while others had yellow seeds, we will speak of a « green » phenotype and a « yellow » phenotype. Intrigued about what would happen with offsprings, he started by crossing the peas having green seeds among themselves during several generations, idem for the peas with yellow seeds (selfing is possible in this species). After doing this for several generations, he always observed that peas with the « green » phenotype always had offsprings with the « green » phenotype, and reciprocally for the « yellow » phenotype. These offsprings resulting of several generations of selfing were called « pure lines« , as they were obtained by crossing plants with the same phenotype.

Then, Mendel went on to cross a yellow male plant (denoted as P1) with a green female plant (denoted as P2), and named the offsprings the F1 generation. He observed that all F1 plants had green seeds, as if the yellow material disappeared. He also observed such a result when doing the reciprocal cross: a green male plant crossed with yellow female plant gave green offsprings. He therefore decided to qualify the « green » phenotype as dominant and the « yellow » phenotype as recessive.

Then, he continued his experiment, and crossed F1 plants together to obtain F2 plants. And here, what is interesting, is that some F2 plants had the « yellow » phenotype (although most had the « green » one): as if the « yellow » material, somehow, jumped from the P generation to the F2 generation…! At this point, Mendel had the genial idea of counting the plants: he found that 3/4 had the « green » phenotype and 1/4 had the « yellow » phenotype, i.e. a 3:1 ratio.

In front of such a striking observation, Mendel went on to characterize the F2 generation. And here again, he observed something strange: when crossing « green » F2 with themselves, some had only « green » offsprings but some had a mix of « green » and « yellow » offsprings, here also in a 3:1 ratio. This was not the case when selfing the F2 « yellow » plants. Therefore, the F2 « yellow » plant seemed to be « pure » as the P « yellow » plants, but 2/3 of the F2 « green » plants were like the F1 plants, and 1/3 were « pure » like the P « green » plants.

Mendel then realized that the behind the 3:1 ratio was a more fundamental 1:2:1 ratio, 3/4 F2 green is in fact 1/4 « pure » green and 2/4 « impure » green, whereas 1/4 F2 yellow is 1/4 « pure » yellow:

From all this, Mendel was able to develop his theory, summarized in the 5 points below:

1. existence of genes, i.e. discrete units (« atomic particles ») of heredity;
2. genes are in pairs, i.e. a gene may have different forms called alleles;
3. each gamete carries only one member of each gene pair;
4. the members of each gene pair segregate equally into the gametes;
5. random fertilization, i.e. gametes combine together to form an organism without regard to which allele is carried.

Now let’s recapitulate by noting « A » the « green » allele, and « a » the « yellow » allele. In the case of the pea described above, Mendel hypothesized that the P « green » plants had an « AA » gene pair, called their genotype, while the P « yellow » plants had the « aa » genotype. As each parent can make only one kind of gametes, « A » for the green and « a » for the yellow, the F1 plants must have the « Aa » genotype. But these plants can make « A » gametes as well as « a » gametes, in equal proportions. This gives rise to F2 plants, 1/4 have the « AA » genotype and thus have the green phenotype, 1/2 have the « Aa » genotype and thus the green phenotype also, and 1/4 have the « aa » genotype and therefore the yellow phenotype:

This is Mendel’s explanation of the 1:2:1 ratio. It seems likely that he got it by imagining that F1 needed to have a « bit » of yellow as well as a « bit » of green somewhere. But still, this will be, forever, one of the greatest scientific discovery…

* * *

Ok, now, let’s look at another classical example which, at the end, introduces quantitative genetics, and from there on, heritability. We are now in the early 20th century and the American geneticist, Edward M. East, also has fun with crossing plants. He worked with a species related to tobacco, some of the plants having a long corolla, ~90mm, some having a short one, ~40mm. As Mendel, he started by selfing plants with a long corolla, idem for the plants with a short corolla. Thus, after several round of selfing, he got two « pure » lines: one pure line with a long corolla (we will call them the P1), and one with a short one (P2). He crossed plants from P1 with plants from P2 and got F1 plants having a medium-size corolla, ~65mm.

Until here, everything seemed fine. But when he crossed F1 together to obtain F2 plants, he didn’t get the 3:1 ratio of corolla length. Instead, he got plants with a corolla of ~65mm on average, and a much larger variability (the appropriate mathematical term is variance). To understand a bit more what happened, he chose F2 plants with a small corolla, crossed them, and obtained F3 plants also with a small corolla. When he crossed F2 plants with a medium corolla, he also obtained F3 plants with a medium corolla. And so on (see the picture below).

This means that the inheritance of the corolla length has indeed a genetic component, but not as simple as the Mendelian case seen above. It is very likely that, instead of a single one, it is several genes that influence the length of the corolla.

Let’s imagine that 5 genes are involved, each with two alleles, + and -, and that the each + allele lengthens the corolla by 1mm, whereas each – allele shortens the corolla by 1mm. The P1 plants with a small corolla are likely to have only the – allele at each of the 5 genes, whereas the P2 plants with the long corolla have the + allele at each of the 5 gene. As usual, the F1 plants have 5 – alleles, coming from their P1 parent, and 5 + alleles from their P2 parent. However, when producing the F2 generation, recombination occurs in all gametes from the F1 parents, and thus F2 offsprings will have different numbers of + and – alleles. Consequently, F2 offsprings will have a wider range of corolla lengths than their F1 parents.

There comes quantitative genetics: a trait is called quantitative if it varies continuously. The seed color of Mendel was a discrete trait (qualitative), while the corolla length is a continuous trait (quantitative). And therefore, quantitative genetics is the study of the genetic basis of quantitative traits. Why is it so important? Well, the overwhelming majority of traits are quantitative…

Here are the questions that quantitative geneticists try to answer:

• Is the observed variation in a character influenced at all by genetic variation? (versus environmental variation)
• If there is genetic variation, what are the norms of reaction of the various genotypes?
• How important is genetic variation as a source of total phenotypic variation?
• Do many loci (or only a few) contribute to the variation in the character? How are they distributed throughout the genome?

* * *

I won’t answer all of them because it would be way too long (!), and in these times of whole-genome sequencing, lots of ongoing research is still underway… But I can still say a few words more.

Misconception: what differentiates a Mendelian trait from a quantitative one is the number of genes involved.

Although, the earlier part may suggest it, this is false. The critical difference between Mendelian and quantitative traits is not the number of segregating loci, but the size of phenotypic differences between genotypes compared with the individual variation within genotypic classes. The scheme below should make this clear. It represents the phenotypic distribution according to the genotype at a given bi-allelic locus, height being the phenotype of interest.

Hence, the definition of a quantitative character becomes: a quantitative character is one for which the average phenotypic differences between genotypes are small compared with the variation between individuals within genotypes.

* * *

After all this, what is heritability? Well, it’s simple. Let’s take a given phenotype, whatever it is (seed color, corolla length, hair color, disease status, growth rate, milk production…). The phenotype is the result of the interaction between genotype and environment. That is, a given genotype in a given environment may not lead to the same phenotype as the same genotype in a different environment, or as a different genotype in the same environment. That’s why the notion of reaction norm is essential.

But still, people always want to know how much genes contribute to some phenotypes. Is it possible? Here again, it’s tricky. Let’s take the example of two bricklayers building a wall. If both of them work in parallel, that is one builds the left of the wall and the other builds the right, it is possible to assess their respective contribution: we just have to count the number of bricks each made. But if now one makes mortar, and the other lays bricks, it is not possible anymore to compare their work (it would be absurd to do it). And it is the same for genes.

Thus, what do we do? Instead of trying to assess the contribution of genes to the phenotype (compare to the contribution of the environment), we can try to assess the contribution of genes to variations of the phenotype, using what statisticians call the analysis of variance. When looking at a given phenotype among many individuals, we try to partition the variability of their phenotype ($\sigma^2_P$) into a variability due to the fact that they have different alleles for the genes involved in the phenotype ($\sigma^2_G$), and into a variability due to the fact that they live in different environments ($\sigma^2_E$):

$\sigma^2_P = \sigma^2_G + \sigma^2_E$

Last but not least, the heritability $H^2$ can now be defined:

$H^2 = \sigma^2_G / \sigma^2_P$

The question “Is a trait heritable?” is a question about the role that differences in genes play in the phenotypic differences between individuals or groups of individuals.

Misconception: a high heritability means that a character is unaffected by the environment.

Hell, no! Because genotype and environment interact to produce the phenotype, no partition of variation into its genetic and environmental components can actually separate causes of variation. But a highly heritable trait means that the genetic component is much more important than the environmental component in contributing to the variation in phenotype.

* * *

There is still much to say of course, and one can find lots of very well explained details in the reference book Introduction to Genetic Analysis from Griffiths and colleagues. Indeed, I even allowed myself to scan some of its figures as they are very self-explanatory. (But  due to copyright issues, I will remove them if the authors and/or publisher ask me to do so.)

## « Allons-y! » gueula Dean.

27 avril 2011

### Première partie

#### Chapitre I

Mais alors ils s’en allaient, dansant dans les rues comme des clochedingues, et je traînais derrière eux comme je l’ai fait toute ma vie derrière les gens qui m’intéressent, parce que les seuls gens qui existent pour moi sont les déments, ceux qui ont la démence de vivre, la démence de discourir, la démence d’être sauvés, qui veulent jouir de tout dans un seul instant, ceux qui ne savent pas bailler, ni sortir un lieu commun mais qui brûlent, qui brûlent, pareils aux fabuleux feux jaunes des chandelles romaines explosant comme des poêles à frire à travers les étoiles et, au milieu, on voit éclater le bleu du pétard central et chacun fait: « Aaaah ! » Quel nom donnait-on à cette jeunesse-là dans l’Allemagne de Goethe ?

Un gars de l’Ouest, de la race solaire, tel était Dean.

Quelque part sur le chemin, je savais qu’il y aurait des filles, des visions, tout, quoi; quelque part sur le chemin on me tendrait la perle rare.

#### Chapitre IX

Ils étaient comme le type avec sa dalle de cachot et ses ténèbres, surgissant du souterrain, les épaves des bas-fonds de l’Amérique, une nouvelle génération foutue que j’étais en train de rallier à petits pas.

#### Chapitre XIII

D’un petit air dégoûté, son joli nez au vent, elle quitta les lieux et nous allâmes tous deux errer dans l’obscurité le long des fossés de la grand-route. Je portais les bagages. La buée sortait de nos bouches dans l’air froid de la nuit. Je pris enfin la décision de dérober au monde une nuit encore avec elle, et au diable le lendemain matin. Nous entrâmes dans la cour d’un motel et louâmes un petit appartement confortable pour environ quatre dollars, avec douche, serviette de bain, radio murale, et tout. On se serra très fort l’un contre l’autre. On eut une longue et sérieuse conversation et on se baigna et on agita des problèmes avec la lumière allumée d’abord puis la lumière éteinte. Une question venait sur le tapis, je soutenais une opinion à laquelle elle se ralliait et nous concluions le pacte dans l’obscurité, haletants, puis heureux, comme de petits agneaux.

Je courbais les épaules sous le vent froid et pluvieux, laissant errer mon regard sur les tristes vignobles d’octobre qui couvraient la vallée. Dans ma tête résonnait cette magnifique chanson, « Homme d’amour », telle que Billie Holiday la chante; c’était mon concert à moi, en pleine cambrousse. « Un jour on se trouvera et tu sècheras toutes mes larmes et tu me murmureras de douces, de petites choses à l’oreille, tu m’enlaceras et là m’embrasseras, ah, tu me manques, homme d’amour, ah où peux-tu être… » Ce n’est pas tant les paroles que la mélodie, magnifique, et la façon dont Billie chante ça, comme une femme qui passe la main dans les cheveux de son homme, à la lueur tamisée d’une lampe. Les vents hurlaient. J’avais froid.

#### Chapitre XIV

N’est-il pas vrai qu’au départ de la vie on est un petit enfant sage qui croit à tout ce qui se présente sous le toit paternel ? Puis vient le jour laodicéen où l’on sait qu’on est pauvre et misérable et malheureux et aveugle et nu, et, avec le visage macabre et désolé d’un spectre, on traverse en frissonnant une vie de cauchemar.

### Deuxième partie

#### Chapitre IV

Il jouait des opéras de Verdi et les mimait drapé dans son pyjama, avec une grande déchirure qui lui descendait dans le dos. Il se foutait royalement de tout. C’est un homme de grande érudition qui déambule en titubant le long des quais de New York avec ds manuscrits originaux de musiciens du XVIIe siècle sous le bras, tout en gueulant. Il se traîne dans les rues comme une grosse araignée. Son excitation jaillissait de ses yeux par éclairs démoniaques. Il ployait sa nuque dans une extase spasmodique. Il zézayait, se tordait en convulsions, s’affalait, gémissait, hurlait, tombait à la renverse de désespoir. Il pouvait à peine placer un mot tellement ça l’excitait de vivre.

#### Chapitre V

La prison est l’endroit où l’on se promet à soi-même le droit de vivre.

#### Chapitre VIII

C’était triste de voir sa haute silhouette diminuer dans l’obscurité à mesure qu’on s’éloignait, exactement comme les autres silhouettes à New York et à la Nouvelle-Orléans: ils vacillent sous l’immensité étoilée et tout ce qu’ils sont est englouti. Où aller ? Que faire ? Dans quel but ? … Dormir. Mais cette équipe de déments était bandée vers l’avenir.

#### Chapitre IX

La Californie de Dean, pays délirant et suant, pays d’importance capitale, c’était celui où les amants solitaires, exilés et bizarres, viennent se rassembler comme les oiseaux, le pays où tout le monde, d’une manière ou d’une autre, ressemble aux acteurs de cinéma détraqués, beaux et décadents.

### Troisième partie

#### Chapitre III

Je regardais par la fenêtre. Il était seul devant la porte, savourant la rue. Les rancunes, les récriminations, les bons conseils, la morale, la tristesse, tout était derrière lu et, au-devant de lui, c’était, déguenillée et extatique, la pure volupté d’être.

#### Chapitre V

Voilà un gars et tout le monde autour, hein ? C’est à lui de mettre en forme ce qui est dans la tête de chacun. Il attaque le premier chorus puis il déroule ses idées, bonnes gens, bien sûr, bien sûr, mais tâchez de saisir, et alors ils e hausse jusqu’à son destin et c’est à ce niveau qu’il doit souffler. Tout à coup, quelque part au milieu du chorus, il ferre le it; tout le monde sursaute et comprend; on écoute; il le repique et s’en empare. le temps s’arrête. Il remplit le vide de l’espace avec la substance de nos vies.

Nos bagages cabossés étaient de nouveau empilés sur le trottoir; nous avions encore bien du chemin à faire. Mais qu’importait, la route, c’est la vie.

#### Chapitre IX

Quelque part derrière nous ou devant nous dans la nuit immense, son père était couché avec sa cuite dans un taillis, cela du moins était certain – avec de la bave au menton, de l’urine sur son froc, de la mélasse aux oreilles, des croutes dans le nez, peut-être du sang dans les cheveux, et la lune qui l’illuminait.

#### Chapitre XI

Nous tous, que brûlons-nous de faire ? Que voulons-nous ? Elle ne savait pas. Elle bâilla. Elle avait sommeil. C’était trop lui demander. Personne ne pouvait le dire. Personne ne le dirait jamais. Un point c’est tout. Elle était âgée de dix-huit ans et très charmante, mais foutue.

### Quatrième partie

#### Chapitre V

L’approche était délicate sans un langage commun. Et de nouveau tout le monde retrouva la paix et la sérénité et l’ivresse et se contenta de jouir de la brise du désert et de ruminer respectivement, eu égard à sa nation, à sa race et à sa personnalité, des pensées de haute éternité.

Assailli par les myriades fourmillantes des phosphènes célestes, il me fallait lutter pour voir la silhouette de Dean et il ressemblait à Dieu. J’étais tellement ivre que je devais appuyer la tête sur le dossier; les cahots de l’auto me fichaient des couteaux d’extase à travers le corps.

Extraits de Sur la route de Jack Kerouac (traduction de Jacques Houbard).

## Proof that P-values under the null are uniformly distributed

22 avril 2011

I often hear in talks from statisticians that P-values are uniformly distributed under the null. But how can this be? And what does it mean? As the demonstration is pretty straightforward but nonetheless hard to find on the Internet, here it is.

Everything starts with an experiment (or at least with the observation of a natural phenomenon, be it part of an experiment or not). The aim is to assess whether or not the hypothesis we have about this phenomenon seems to be true. But first, let’s recall that a parametric test (see Wikipedia) is constituted of:

• data: the $n$ observations $x_1$, $x_2$, …, $x_n$ are realizations of $n$ random variables $X_1$, $X_2$, …, $X_n$ assumed to be identically distributed;
• statistical model: the probability distribution of the $X_1$, $X_2$, …, $X_n$ depends on parameter(s) $\theta$;
• hypothesis: an assertion concerning $\theta$, noted $H_0$ for the null (e.g. $\theta=a$), and $H_1$ for the alternative (e.g. $\theta=b$ with $b > a$);
• decision rule: given a test statistic $T$, if it belongs to the critical region $C$, the null hypothesis $H_0$ is rejected.

In practice, $T$ follows a given distribution under $H_0$ (e.g. a Normal distribution, or a Student distribution) that does not depend on $\theta$ but on $n$. We use the observations to compute a realization, noted $t$, of $T$.

The P-value, noted $P$, can be seen as a random variable, and its realization, noted $p$, depends on the observations. According to the notations, the formal definition of the P-value for the given observations is:

$p = \mathbb{P} ( T \ge t | H_0 )$

Therefore, according to Wikipedia, a P-value is the probability of obtaining a test statistic at least as extreme as the one that was actually observed, assuming that the null hypothesis is true. According to Matthew Stephens (source), a p value is the proportion of times that you would see evidence stronger than what was observed, against the null hypothesis, if the null hypothesis were true and you hypothetically repeated the experiment (sampling of individuals from a population) a large number of times.

Very importantly, note that the 2nd definition emphasizes the fact that, although it is computed from the data, a P-value does not correspond to the probability that $H_0$ is true given the data we actually observed!!

$p \ne \mathbb{P} ( H_0 | x_1, x_2,..., x_n )$

A P-value simply gives information in the case we would repeat the experiment a large number of times… (That’s why P-values are often decried.)

Ok, back on topic now. From the formula above, we can also write:

$p = 1 - \mathbb{P} ( T < t | H_0 )$

By noting $F_0$ the cumulative distribution function (cdf, fonction de répartition in French) of $T$ under $H_0$, we obtain:

$p = 1 - F_0( t )$

And here is the trick, thanks to the fact that the cdf is monotonic, increasing and (left-)continuous:

$\mathbb{P} ( T \ge t | H_0 ) = \mathbb{P} ( F_0(T) \ge F_0(t) ) = 1 - \mathbb{P} ( F_0(T) < F_0(t) )$

Therefore, we have:

$\mathbb{P} ( F_0(T) < F_0(t) ) = F_0( t )$

Which means that $F_0(T)$ is following a uniform distribution. And, as this means also that $1 - F_0(T)$ is uniformly distributed, then we can conclude that P-values are uniformly distributed under the null hypothesis.

cqfd.

But what does it mean? Well, we usually consider a significance level, noted $alpha$ (small, e.g. 5%, 1%, 0.1%…), and if the P-value falls below this threshold, we reject the null and decide that the alternative is significant. However, let’s say we re-do the same experiment $N$ times and compute a P-value for each of them. Since P-values are uniformly distributed under the null, it is as likely to find some of them between 0.8 and 0.85 than to find some of them below 0.05, if $H_0$ is indeed true. That is, some of them will fall below the significance threshold, just by chance. The experiments corresponding to these P-values are called false-positives: we think they are positives, i.e. we decide to accept $H_1$, while in fact they are really false, i.e. $H_0$ is true and should not be rejected.

Last but not least, if we re-do the same experiment 100 times and consider a threshold of 5%:

• if $H_0$ is false (although we are not supposed to know it before doing the experiment), how many P-values will fall below this threshold just by chance? 5, on average;
• if now $H_0$ is supposed to be true 50% of the time, what proportion of P-values will be around $5\% +- \epsilon$? at least 23%, and typically 50% (see the paper of Sellke et al in 2001). In other words, when $H_0$ is true 50% of the time, a P-value of 5% doesn’t tell us anything, as half of the experiments from which they were calculated correspond to a true $H_0$, and half to a false $H_0$