Skip to main content
Advertisement
  • Loading metrics

The limitations, dangers, and benefits of simple methods for testing identifiability

Abstract

In their Commentary paper, Villaverde and Massonis (On testing structural identifiability by a simple scaling method: relying on scaling symmetries can be misleading) have commented on our paper in which we proposed a simple scaling method to test structural identifiability. Our scaling invariance method (SIM) tests for scaling symmetries only, and Villaverde and Massonis correctly show the SIM may fail to detect identifiability problems when a model has other types of symmetries. We agree with the limitations raised by these authors but, also, we emphasize that the method is still valuable for its applicability to a wide variety of models, its simplicity, and even as a tool to introduce the problem of identifiability to investigators with little training in mathematics.

In their Commentary paper, Villaverde and Massonis (On testing structural identifiability by a simple scaling method: relying on scaling symmetries can be misleading [1]) have commented on our paper in which we proposed a simple scaling method to test structural identifiability [2]. Our scaling invariance method (SIM) tests for scaling symmetries only, and Villaverde and Massonis correctly show the SIM may fail to detect identifiability problems when a model has other types of symmetries (we indeed indicated but not investigated the importance of generalizing the method to other symmetries). Thus, we agree that our simple method provides a necessary but not sufficient condition for identifiability, and we appreciate their careful analysis and constructive criticism.

We nevertheless think that the simple method remains useful because it is so simple. Even for investigators with little training in mathematics, the method provides a necessary condition for structural identifiability that can be derived in a few minutes with pen and paper. Similarly, we have found its pedagogic strength by teaching the method to our own graduate students and colleagues. More advanced methods (such as STRIKE-GOLDD [3,4], COMBOS [5], or SIAN [6]) are typically intimidating for researchers with a background in Biology or Bioinformatics. This simple method can help those practitioners to familiarize themselves with the identifiability problem and better understand their models.

Finally, it is worth noting that if scaling invariance is the only symmetry (as it was in all the cases we analyzed), our SIM remains valuable (albeit uncontrolled), and surprisingly effective for a wide variety of problems (as the extensive list collected in the Supplementary Material our paper [2]). We guess that the SIM especially fails when applied to linear models (as more potential rotations of the variables leave the system invariant), and in non-linear scenarios where some parameters are identical. For instance, the FitzHugh-Nagumo model raised by Villaverde and Massonis, could have been written as where λ1 = λ2 = λ3 = 1/λ4 = c. One of the reasons why our method fails, in this case, might be these additional symmetries introduced in this more elaborate notation of the model.

Hence, it is worth understanding generic conditions under which the SIM method is expected to be fragile, possibly using STRIKE-GOLDD to test large families of nonlinear models.

As a final remark, we appreciate that Villaverde and Massonis have shared their source code, so researchers might have a gold standard to test identifiability.

References

  1. 1. Villaverde AF, Feixas GM. On testing structural identifiability by a simple scaling method: relying on scaling symmetries can be misleading. bioRxiv. 2020.
  2. 2. Castro M, de RJ. Testing structural identifiability by a simple scaling method. PLoS Comput Biol. 2020;16 (11):e1008248. pmid:33141821
  3. 3. Villaverde AF, Barreiro A, Papachristodoulou A. Structural Identifiability Analysis via Extended Observability and Decomposition. IFAC-PapersOnLine. 2016;49 (26):171–7.
  4. 4. Villaverde AF, Barreiro A, Papachristodoulou A. Structural Identifiability of Dynamic Systems Biology Models. PLoS Comput Biol. 2016;12 (10):1–22. pmid:27792726
  5. 5. Meshkat N, Kuo CEz DSJ. On finding and using identifiable parameter combinations in nonlinear dynamic systems biology models and COMBOS: a novel web implementation. PLoS One. 2014;9 (10):e110261. pmid:25350289
  6. 6. Hong H, Ovchinnikov A, Pogudin G, Yap C. SIAN: software for structural identifiability analysis of ODE models. Bioinformatics. 2019;35 (16):2873–4. pmid:30601937