The perils of null modeling

Community assembly processes are often inferred from null models of trait or species diversity. Can we trust these results? Caroline Tucker at evol-eco has a good perspective, discussing a recent paper of de Bello et al.

de Bello, F. The quest for trait convergence and divergence in community assembly:are null-models the magic wand? Global Ecology and Biogeography, (Global Ecol.Biogeogr.) (2012) 21, 312–317

The relevance of neutral versus niche-based community assembly rules (i.e. the processes sorting species present in a larger geographical region into local communities) remains to be demonstrated in ecology and biogeography. To attempt to do this, a number of complex null models are increasingly being used that compare observed community functional diversity (FD, i.e. the extent of trait dissimilarity between coexisting species) with randomly simulated FD. However, little is known about the performance of these null models in detecting non-neutral community assembly rules such as trait convergence and divergence of communities (supposedly revealing habitat selection and limiting similarity, respectively). Here, using both simulated and field communities, I show that assembly rule detection varies systematically with the magnitude of the observed FD, so that these null models do not really succeed in breaking down the observed functional relationships between species. This is a particular concern, making detection of community assembly dependent on: (1) the pool of samples considered, and (2) the capacity of observed FD to correctly discriminate these rules. Null models should be more thoroughly described and validated before being considered as a magic wand to reveal assembly patterns.

See also the useful perspective of Jeremy Fox at Oikos.


5 Comments on “The perils of null modeling”

  1. Jeremey Fox essentially summarized my thoughts on this paper. These new methods are just discovering what community ecologists have been debating for quite some time now. There really isnt anything that is greatly new here . .

    See Jeremey’s comments on the associated links – in particular he says

    “So, the Narcissus Effect (Colwell & Winkler 1984, IIRC) is rediscovered yet again…

    Time to refight the null model wars!

    As for why modern phylogenetic and functional approaches aren’t recognized as suffering from the same problems as old approaches, one natural hypothesis is that most of the people using these shiny new quantitative tools are largely unfamiliar with the older literature. “

    • Jeremy Fox says:

      Thanks for the shout-out, Brian. Good to know I’m not the only one thinking along these lines. Although I will say that just because the de Bello paper is just rediscovering old ideas doesn’t mean it isn’t important. Unfortunately, it seems like sometimes the only way to prevent people from repeating old mistakes is to write new papers pointing out the mistakes. That’s what I’ve been doing with my posts on the “zombie ideas” underpinning the intermediate disturbance hypothesis, for instance. I’m not saying anything that Peter Chesson and others didn’t say years ago, I’m just repeating it (and adding in zombie jokes) so that people pay attention and hopefully take it to heart. And in 20 years someone else (not me!) will probably have to do it again…

      • I agree Jeremy . . and thank you for emphasizing that just because others (quite some time ago) made these same points does not mean that the paper is not important. I’ve had fun conversations with senior community ecologists about ‘all of these new phylo- trait- community ecology papers’. Their reactions are surprisingly similar “been there done that” as well as warriness that we are retreading old patterns BUT also that “the new rigor and insights are truely moving the field further now in a way that was not possible then.” The worry is that we are potentially wasting time in retreading these same ol’ debates as well as not paying proper credit to those who first articulated these ideas in print.

        This is all reminsent of discussions between senior faculty in the hallway I remember when I was a grad student “the students these days just don’t read the classic literature!” Gosh, that was a while back now . . .

  2. Nate Swenson says:

    Ah boy, ok i’ll bite… i can’t resist. This just came across my radar via a colleague…

    True some rediscovery happens and the sensitivity of results to different null models is being quantified by phylo and trait people. What does irk me here though is the sense that phylo and trait people in general aren’t considering old literature. Rather it is de Bello that is not acknowledging the older literature. There are several papers from the phylo world that have acknowledged previous null model debates and tried to determine how those biases may apply to phylo analyses (including a paper by Brian and I in 2006; Kembel & Hubbell 2006; work by Olivier Hardy; work by Kraft). Indeed Kraft et al. 2007 Am Nat did a phylo and trait assembly sensitivity analysis that used Colwell and Winkler’s approach and generated R code based on the computer program from C&W. So if de Bello were to consult the contemporary literature a little more closely he may find citations to the foundational/older literature. He may have also found that contemporary work has already shown that the influences of null model choice demonstrated in the 80’s in species assembly studies generally are found in trait and phylo based assembly studies.

    From the trait side most of us are aware of the older debates and frequently cite them. de Bello is an outlier when it comes to this. For example Dan Rabosky has a paper in print or press at Am Nat that seeks to quantify the Narcissus Effect in a trait data set in Australia I believe. The term “Narcissus Effect” is in the title. So I generally agree with what most people are talking about in these blogs (oikos, here, elsewhere)(ie that we should be acknowledging and reading older literature that is germane to our present work), but I reject the notion that all of trait and phylo work is/has been ignorant of the null model ‘wars’ from the past.

    What is needed is advice to youngsters about the existence of this literature…i have brian to thank for that!

  3. Francesco de Bello says:

    Dear all,

    many thanks for this very stimulating discussion. I am happy that this work created such a constructive discussion. The take home message from my paper is probably not a very original one, although I feel that a more clear description of null-models used would be generally useful in the existing literature. I would like to mention here that some key old works are mentioned in the GEB paper (including Strong et al 1984) while the newer ones are also cited in a recent review ( which you might find interesting. Unfortunately is not possible to cite everybody all the times and I acknowledge that my small Italian brain can retain only a limited amount of information. Sorry for that.

    Anyway, I am not sure the point is really weather I am, or not, aware of other publications but weather null-models are really helpful or should be simply described a bit better. In a recent paper, in minor revisions in Ecology, we claim that there are are other options to assess community assembly rules with functional or phylogenetic diversity, not necessarily using null-models. Otherwise I am also generally happy to apply null-models (e.g. when they are clearly designed to answer a specific research question and, more important, sufficiently described in the methods.

    With greetings,


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s