Expression profiling without statistics

[hegesmith]

Dear qPCRforum,
I have some quenstions about gene expression profiling and statistical analyses. I attended a qPCR course in Biostatistics in Prague in June 2008, where we discussed that statistics does not always have to be applied to an experiment if you only want to look at a trend or an expression profile of a gene.
I have infected fish cells with pathogenic bacteria and studied the expression profile of several virulence genes as infection progressed. I have sampled from 6 time points and looked at the profile of 12 genes. When I submitted my paper to an international journal, the reviewers asked about the statistics. "Why were the results from in vitro gene expression not statistically analysed? Either they should be or a good reason for not doing so should be given".

During the course in Prague I learned that you can compare expression profiles without comparing differences in magnitude, and if you perform statistics on a large number of groups and genes you will meet the "multiple testing problem".

Furthermore, I compared expression of a few genes from two conditions using a standard t-test i GenEx. The reviewers asked: "Were the obtained values normally distributed? Perhaps due to small sample sizes a distribution-free Wilcoxon two group test would be an appropriate alternative in this case to compare expression of targeted genes in the two tissues". Is this an option in GenEx?

Many thanks in advance,

Hege

[Anders Bergkvist]

Dear Hege,

We distinguish between exploratory and confirmatory statistics. Central to the distinction between these approaches is the statistical hypothesis. In an exploratory study we basically browse through the data in search for a suitable hypothesis. In a confirmatory study we use a pre-defined hypothesis and analyze its validity under some pre-defined statistical criteria.

If you perform an exploratory study I would suggest you state that in your manuscript together with the one or several hypotheses that you would propose for further confirmatory studies. In this case, statistical details are not very important since you will not claim that your hypotheses are valid in any sense.

On the other hand, if you perform a confirmatory study you need to state your hypothesis together with the statistical criteria you use to estimate its validity. Note that a confirmatory study should not be performed on a data set that was previously used to define the hypothesis! Data needs to be recollected for the confirmatory study (or in other words: the hypothesis and criteria needs to be defined before the data is collected)!

Yes, Wilcoxon is available in GenEx among the "Non-parametric tests" under the Statistics tab. Running this (or one of the parametric t-tests) will also produce a kolmogorov-smirnov estimate of the normality of your data set in GenEx.

Hope this helps, Best wishes,
Anders

[hegesmith]

Dear Anders,
Thank you so much for your help.
I will argue that my study is exploratory and state the hypotheses that come out of this study more clearly.

Concerning comparison of multiple genes, I understand that it is common to use false discovery rate (FDR) corrections instead of Bonferroni when data are normally distributed (and a t-test is applied). This is due to the difficulty of obtaining significance when using Bonferroni corrections. Is this only valid for microarray data or do you use this on real time data as well? I can not see that this is an option in GenEx.

Many thanks for your reply,

Hege

[Anders Bergkvist]

Dear Hege,

The FDR correction is an alternative approach to handling challenges with multiple testing. To some extent it could be considered complementary to the Bonferroni correction. Basically the Bonferroni correction reduces the number of positive samples, whereas the FDR correction lets you keep all positive samples but tells you that a fraction of them are likely false positives.

At the time being, the FDR correction is not implemented in GenEx. However, it is a good suggestion and I'll forward the idea to our developers.

I want to stress though that for confirmatory studies you need to define strict criteria for significance and then FDR correction is not that useful. And when you perform exploratory studies the choice of correction approach is in any case not very important since you are just looking at trends or patterns in the data to allow you to define one or several hypotheses.

Best wishes,
Anders