Revisiting the “satisfaction of spatial restraints” approach of MODELLER for protein homology modeling

Giacomo Janson; Alessandro Grottesi; Marco Pietrosanto; Gabriele Ausiello; Giulia Guarguaglini; Alessandro Paiardini

doi:10.1371/journal.pcbi.1007219

Abstract

The most frequently used approach for protein structure prediction is currently homology modeling. The 3D model building phase of this methodology is critical for obtaining an accurate and biologically useful prediction. The most widely employed tool to perform this task is MODELLER. This program implements the “modeling by satisfaction of spatial restraints” strategy and its core algorithm has not been altered significantly since the early 1990s. In this work, we have explored the idea of modifying MODELLER with two effective, yet computationally light strategies to improve its 3D modeling performance. Firstly, we have investigated how the level of accuracy in the estimation of structural variability between a target protein and its templates in the form of σ values profoundly influences 3D modeling. We show that the σ values produced by MODELLER are on average weakly correlated to the true level of structural divergence between target-template pairs and that increasing this correlation greatly improves the program’s predictions, especially in multiple-template modeling. Secondly, we have inquired into how the incorporation of statistical potential terms (such as the DOPE potential) in the MODELLER’s objective function impacts positively 3D modeling quality by providing a small but consistent improvement in metrics such as GDT-HA and lDDT and a large increase in stereochemical quality. Python modules to harness this second strategy are freely available at https://github.com/pymodproject/altmod. In summary, we show that there is a large room for improving MODELLER in terms of 3D modeling quality and we propose strategies that could be pursued in order to further increase its performance.

Author summary

Proteins are fundamental biological molecules that carry out countless activities in living beings. Since the function of proteins is dictated by their three-dimensional atomic structures, acquiring structural details of proteins provides deep insights into their function. Currently, the most frequently used computational approach for protein structure prediction is template-based modeling. In this approach, a target protein is modeled using the experimentally-derived structural information of a template protein assumed to have a similar structure to the target. MODELLER is the most frequently used program for template-based 3D model building. Despite its success, its predictions are not always accurate enough to be useful in Biomedical Research. Here, we show that it is possible to greatly increase the performance of MODELLER by modifying two aspects of its algorithm. First, we demonstrate that providing the program with accurate estimations of local target-template structural divergence greatly increases the quality of its predictions. Additionally, we show that modifying MODELLER’s scoring function with statistical potential energetic terms also helps to improve modeling quality. This work will be useful in future research, since it reports practical strategies to improve the performance of this core tool in Structural Bioinformatics.

Citation: Janson G, Grottesi A, Pietrosanto M, Ausiello G, Guarguaglini G, Paiardini A (2019) Revisiting the “satisfaction of spatial restraints” approach of MODELLER for protein homology modeling. PLoS Comput Biol 15(12): e1007219. https://doi.org/10.1371/journal.pcbi.1007219

Editor: Bert L. de Groot, Max Planck Institute for Biophysical Chemistry, GERMANY

Received: June 25, 2019; Accepted: November 13, 2019; Published: December 17, 2019

Copyright: © 2019 Janson et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: All relevant data are within the manuscript and its Supporting Information files. The data underlying the results presented in the study are also available from https://github.com/pymodproject/altmod/tree/master/data.

Funding: GJ and AP received support from Associazione Italiana Ricerca sul Cancro (AIRC, https://www.airc.it/) MFAG 20447 and Progetti Ateneo Sapienza University of Rome (https://www.uniroma1.it). GG received support from Associazione Italiana Ricerca sul Cancro (AIRC, https://www.airc.it/) IG Grant 17390. AP, AG and GJ acknowledge the CINECA award under the ISCRA initiative, for the availability of high performance computing resources and support (IsC68_altmod). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Competing interests: The authors have declared that no competing interests exist.

Introduction

In silico protein structure prediction constitutes an invaluable tool in Biomedical Research, since it allows to obtain structural information on a large number of proteins currently lacking an experimentally-determined 3D structure [1]. Template-based modeling (TBM) is the most frequently employed prediction strategy. In the past years it has been considered as the most accurate one [2], but recently it has been shown that template-free strategies have reached comparable levels of performance with protein targets that lack good templates [3] (for example, with members of several membrane protein families [4]). Despite this fact, TBM methods, thanks to their speed, flexibility and growing template libraries [5], currently remain the instrument of choice for many researchers.

Homology modeling (HM) is a fast and reliable TBM method in which a target protein is modeled by using as a structural template an homologous protein. HM predictions usually consist of three phases. In the first, the sequence of the target is used to search for suitable templates in the PDB [6–7]. In the second, a sequence alignment between the target and templates is built with the goal of inferring the equivalences between their residues [8]. In the final, the information of the templates is used to build a 3D atomic model of the target.

The overall accuracy of HM has remarkably increased in the last 25 years. While a major factor for this advancement has been the increase of the size of sequence and structural databases [5], it has been shown that progress in HM algorithms has also played a key role [9]. These improvements have consisted mainly in advances in template searching and alignment building algorithms, while only minor advances have been witnessed in the 3D model building step [10]. However, recent breakthroughs in protein structure refinement methods [11–12] envisage a large room for improvement in HM which could originate from advances in 3D model building.

MODELLER [13] is the most frequently used program for 3D model building in HM. One of the main reasons of its success has been its accurate [14], yet fast algorithm. In MODELLER, the information contained in an input target-template alignment is used to generate a series of homology-derived spatial restraints (HDSRs), acting on the atoms of the 3D protein model. Sigma (“σ”) values of homology-derived distance restraints (HDDRs) determine the amount of conformational freedom which the model is allowed to have with respect to its templates. MODELLER uses a statistical “histogram-based” strategy to estimate σ values [15]. These restraints are incorporated into an objective function which also includes physical energetic terms from CHARMM22 [16]. A fast, but effective optimization algorithm based on a combination of conjugate gradients (CG) and molecular dynamics with simulated annealing (MDSA) is then used to identify a model conformation that satisfies as much as possible the HDSRs, while retaining stereochemical realism.

The core MODELLER algorithm was developed in the early 1990s and it was essentially left unchanged over the years. Despite its importance, there have been relatively few attempts to improve it.

In 2015, Meier and Söding designed a novel probabilistic framework for building HDDRs [10], whose aim was to help MODELLER tolerate alignment errors and to combine the information from multiple templates in a statistically rigorous way. This system increased 3D modeling quality, especially for multiple-template modeling. However, since it is integrated in the HHsuite project [17] it can be employed only when the first two phases of HM are carried out by programs of the HHsuite package.

Researchers from Lee’s group developed a modified version of MODELLER which they have been using in CASP experiments [18–20]. First, they replaced the MODELLER optimization algorithm with the more thorough conformational space annealing (CSA) method [21]. Secondly, they pioneered a new strategy to assign σ values to HDDRs relying on machine learning [22]. Finally, they included a series of additional terms to the MODELLER objective function, such as terms for the DFIRE [23] and DFA [24] knowledge-based potentials, for hydrogen bond formation [25] and to enforce in models predictions of structural properties. In terms of 3D modeling quality, this system outperformed the original MODELLER [20]. Unfortunately, the separated contribution of several of these modifications is not reported and much of this system remains in-house (only the CSA algorithm is publicly available).

Although these seminal studies have shown that the core MODELLER algorithm has room for improvement, most of its users employ its original version, probably because existing modifications either depend on additional packages to install, or are computationally too expensive (e.g., the CSA algorithm alone was reported to increase computational times by a factor of ~130). Since MODELLER is a core tool in Structural Bioinformatics, it is of paramount importance to investigate in detail the inner working of its algorithm and to develop it further. Here, we have explored two computationally light strategies to improve it in terms of 3D modeling quality.

Particular attention has been dedicated in understanding how the level of accuracy in the estimation of structural variability between the target and templates expressed as σ values influences 3D modeling. Although in this work we have not modified the MODELLER algorithm for σ values assignment, we propose strategies that could be likely pursued in the next-future in order to greatly increase the performance of the program. Additionally, we have investigated how the incorporation of statistical potential terms, such as DOPE [26], in the program’s objective function is able to impact positively 3D modeling and under certain conditions (for example in single-template modeling) it can be coupled synergistically to the previous strategy.

To rigorously validate these approaches, we have benchmarked them using protein targets from a diverse set of high-resolution structures from the PDB and we quantified the individual impact on 3D modeling of each modification. This information will be useful in future research, since it shows in which areas there is still room for improvement and in which areas it might be difficult to advance further.

Materials and methods

Outline of MODELLER’s homology-derived distance restraints

The MODELLER approach relies on the generation of HDSRs for interatomic distances and dihedral angles [15]. Each HDSR is treated as a probability density function (pdf). HDSRs acting on interatomic distances (that is, HDDRs) have a predominant role in determining the 3D structure of a model. The way they are built is summarized here.

For a couple of atoms i and j of the model, the program finds in the template the equivalent atoms k and l which have a distance in space of d_t. The distance d_m between i and j is assumed to be normally distributed around d_t with a standard deviation σ and the pdf restraining it is: (1)

In MODELLER pdfs are converted in objective function terms as follows: (2) therefore Gaussian HDDRs correspond to harmonic potential terms. Since HDDRs are considered to be independent, their objective function terms are summed. HDDRs are built for four groups of atoms: the Cα-Cα, backbone NO, side chain-main chain (SCMC) and side chain-side chain (SCSC) groups (see S2 Table). MODELLER generates its σ values (hereinafter named σ_MOD values) through an histogram-based approach [15].

MODELLER allows to take advantage of multiple templates, a strategy that (when templates are chosen adequately) usually outperforms single-template modeling [27]. When employing U templates to restrain a distance d_m, MODELLER uses the following pdf: (3) where u is the template index, w_u is a template-specific weight, d_t,u and σ_u are the distance observed in template u and its σ value respectively. In MODELLER, w_u is a function of the local sequence similarity between the target and template u.

The total objective function of MODELLER (F_TOT) can be expressed as follows: (4) where F_PHYS contains five physical terms (see S1 Table) and F_HOM contains HDSRs terms. In this work, the weights for F_PHYS and F_HOM were always left to 1.0 (therefore they are omitted from the formula above).

Benchmarking MODELLER modifications with an analysis set

In order to benchmark modifications of MODELLER, we built an analysis set of selected target proteins. We obtained 926 X-ray structure chains from PISCES [28], using the following criteria to filter the PDB:

the maximum mutual sequence identity (SeqId) among the chains was 10%;
their structures had a resolution < 2.0 Å and R-factor < 0.25;
they contained no missing residues due to lacking electron density;
their length was between 70 and 700 residues.

These chains were our target candidates. To obtain their templates, we culled from PISCES another set using similar filters, except that this time the maximum mutual SeqId was 90%. We removed from this larger set all the targets, obtaining 6224 chains. Each target was then aligned to these chains using TM-align [29] and we selected as template candidates the chains meeting the following criteria:

the SeqId in the structural alignment built by TM-align was between 15% and 95%;
the two TM-scores [30] produced by TM-align (each score is normalized by the length of one of the aligned proteins) were at least 0.6, a threshold to consider two proteins as homologous [31].

We retained for each target only its top five templates in terms of TM-score (normalized on the target length). In this way, we obtained a final set of 225 target chains (suitable templates could not be found for 701 targets, a result of using only high-resolution template structures). For each target, we performed single-template modeling only with its top template and therefore we had 225 single-template models, which constituted the Analysis Single-template (AS) set. 118 targets had at least two templates (with an average of 3.3), which constituted the Analysis Multiple-templates (AM) set.

The average SeqId for the AS target-template alignments is 0.38. Improving the performance of MODELLER with targets having templates with a SeqId < 0.40 is important, because these cases are the most frequent ones in Biomedical Research [5] and the accuracy of TBM is often low in this regimen. The well-equilibrated distributions of SeqId, target coverage, target length and of CATH structural classes [32] of the analysis set (see S1 Fig) assure that our results have a general validity.

Alignment building

In order to align target-template pairs we employed the accurate HHalign program [7], which confronts two profile hidden Markov models. To build input profiles for HHalign, we ran HHblits [33] with its default parameters and three search iterations against the uniprot20_2016_02 database. After employing HHalign to align pairs of target-template profiles, we extracted from the program’s output their pairwise alignments. Multiple target-templates alignments were obtained by joining pairwise alignments.

Whenever specified, we also employed target-template alignments built with TM-align in order to assess the effect on 3D modeling of HDDRs derived from error-free structural alignments.

3D model building and evaluation

For all benchmarks we used MODELLER version 9.21. In order to modify its objective function terms and optimization schedules we interfaced with its Python API. To modify the restraints parameters we employed Python scripts to edit the default restraints files generated by the program (see the “Restraints files building” section).

In MODELLER, the final quality of a model is largely determined in the MDSA phase. In this work, unless otherwise stated, we employed the default very_fast MDSA protocol of the program (corresponding to a 5.4 ps run). When specified, we also employed the more thorough slow protocol (corresponding to a 18.4 ps run). The CG protocol was always left to its default parameters.

The approach used to evaluate the quality of an homology model was to build 16 different copies of it (hereinafter defined as decoys), and to report as an overall quality score (see below) the average score of the 16 decoys.

To evaluate the quality of the backbones we used the GDT-HA metric [9] computed by the TM-score program. In order to evaluate the quality of local structures and side chains, we used the lDDT metric [34], computed by the lDDT program. Detailed descriptions of these two metrics are given in S1 Text. To evaluate the stereochemical quality of models we employed MolProbity scores computed by the MolProbity suite [35]. A MolProbity score expresses the global stereochemical quality of a 3D model. The lower it is, the higher is the quality of the model.

Optimal σ values for homology-derived distance restraints

σ values of HDDRs have a fundamental role in MODELLER. A natural question is: given a target-template alignment, what is the set of σ values which will maximize 3D modeling accuracy? The concept of optimal σ values in single-template modeling was addressed for the first time by the Lee group [22]. They reported that for a Gaussian HDDR acting on a distance d_m between atoms i and j in a 3D model, the optimal σ value is: (5) where d_t is the distance between the template atoms equivalent to i and j and d_n is the distance between i and j observed in the experimentally-determined native target structure. We show that the use of |Δd_n| values for Gaussian HDDRs is supported by theory, as it can be analytically proven that they maximize the likelihood of obtaining a model in which each restrained d_m is equal to its corresponding d_n (see S2 Text).

In the case of multiple-template HDDRs, we demonstrate that the combination of optimal σ values and weights can be found again analytically (see S3 Text). In this situation, the optimal σ values are again |Δd_n| values. The associated template weighting scheme assigns a weight of 0 to all templates with the exception of the template with the lowest σ, which should have a weight of 1. We termed this scheme as the “only-lowest” (OL) scheme. Note that the OL scheme is an extreme case of the weighting scheme proposed in [36] (see S3 Text).

Whenever using |Δd_n| values as σ parameters, we had to modify them by setting their minimum value at 0.05 Å. Raw |Δd_n| values are extracted directly from pairs of homologous protein structures and they are often close to 0 Å (see Fig 1A). In MODELLER, HDDRs having very small σ values will seldom be satisfied because their quadratic objective function terms will penalize enormously even minimal deviations from templates. In fact, using unmodified |Δd_n| values often leads to modeling failures, since the total objective function of models surpasses the allowed limit of MODELLER, stopping the model building process. Setting a lower limit to their value, allows their use in 3D modeling.

Download:

Fig 1. Distribution of |Δd_n| and σ_MOD values.

Distributions of the |Δd_n| (A) and σ_MOD (B) values observed in the AS models for the four HDDR groups of MODELLER. Beside the names of the restraints groups, their mean values are reported.

https://doi.org/10.1371/journal.pcbi.1007219.g001

Restraints file building

In MODELLER, the restraints used in the 3D model building phase are supplied in a specific file. In this work, we explored how the choice of parameters for the HDDRs influences 3D modeling. Therefore, our approach for building restraints files was to let MODELLER generate its default restraints files and then to modify it by leaving unaltered all the stereochemical and homology-derived dihedral angle restraints and by modifying only the HDDRs parameters. The Python code we used to customize restraints files is available at https://github.com/pymodproject/altmod. This code allows to modify the list of HDDRs of a model by supplying a list of user-specified parameters for them. Users may specify both the location and scale parameters of the Gaussian HDDRs, see Eq (1). Additionally, we provide code for running MODELLER with optimal HDDRs, which can be employed whenever users can supply a native structure file for the target protein.

Perturbing optimal |Δd_n| values

To understand the effect of using error-containing |Δd_n| estimations on 3D modeling, we randomly perturbed the |Δd_n| values of target-template pairs. Our aim was to simulate a series of |Δd_n| estimators having different performances in terms of the Pearson correlation coefficient (PCC) between the perturbed values and their unperturbed counterparts. To perturb a |Δd_n| list of a target-template pair in order to reach a selected PCC (referred to as PCC_SEL), we added to the Δd_n values of the pair a list ε of errors in order to obtain a list p of perturbed values, whose elements are: (6) where p_i is the i-th perturbed value and ε_i is a random error extracted from a Laplace distribution with location 0 and scale parameter b. We chose Laplace distributions since adding errors extracted from them results in p_i values being distributed approximately as exponentials, which resemble the original |Δd_n| distributions (see Fig 1A and also Fig F in S2 Fig). To obtain the desired PCC_SEL, a |Δd_n| list was perturbed in 5000 trials by drawing ε from Laplace distributions with different b values and in each trial the PCCs between p and the original |Δd_n| list was computed (b was linearly increased from 0.005*m_obs to 25.0*m_obs, where m_obs is the mean of the |Δd_n| list). At the end of these trials, the p list having the observed PCC as close as possible to PCC_SEL was selected. The |Δd_n| values of each HDDR group (that is, the Cα-Cα, NO, SCMC and SCSC groups) were perturbed independently. This heuristic procedure usually selects PCCs being not more than 0.003 units away from PCC_SEL (thus giving practically the same level of perturbation of PCC_SEL). Note that larger b values result in larger amounts of noise (and in lower PCCs), but they also increase the mean of p lists far beyond the typical values observed for |Δd_n| data. Therefore, the p lists of the four HDDR groups were always scaled so that their mean (referred to as m_pt) would be equal to: (7) where m_grp is the mean |Δd_n| value observed in all our AS models for HDDR group grp (see Fig 1A). In this way, if PCC_SEL is near 1, the mean of a p list tends to the mean of the unperturbed |Δd_n| list, while if PCC_SEL is near 0, its mean tends to the “global” mean observed for the corresponding group in our whole |Δd_n| data sets. This whole perturbation scheme has three important characteristics. (i) Adding noise to every |Δd_n| value of a target-template pair allows us to properly simulate the effect of a real-life |Δd_n| estimator in which every estimation would have some uncertainty. (ii) Since 3D modeling quality tends to decrease when the average σ value of a model increases (see Fig 2A and 2B), the scaling procedure ensures that when employing highly perturbed |Δd_n| lists, alterations in the quality of 3D models will not be caused by just unrealistically increasing their mean σ. (iii) The choice of Laplace distributions as error-generators ensures that alterations in quality will not be caused by drastically changing the shape of the perturbed values distributions with respect to the observed |Δd_n| ones. An example of the effects of this perturbation scheme are found in S2 Fig (while the code that implements it is available in our Git repository, see above).

Download:

Fig 2. Modeling with uniform σ values.

Average GDT-HA (A) and lDDT (B) scores of the AS models as a function of the uniform σ value (ranging from 0.01 to 7.0 Å) applied to their HDDRs. The horizontal dashed lines represent the average scores obtained with the original σ_MOD values.

https://doi.org/10.1371/journal.pcbi.1007219.g002

To simulate various levels of accuracy in |Δd_n| estimation, we used 10 PCC_SEL values (linearly spacing from 0.0 to 0.9). For each PCC_SEL, we generated 5 sets of perturbed |Δd_n| values per target-template pair, which allowed to better sample the effect of perturbations. For each perturbed set, we built 8 decoys per target with MODELLER (resulting in a total of 5*8 = 40 decoys per target for each PCC_SEL value). For a certain PCC_SEL value, the quality score for a 3D model was recorded as the average score of all its 40 decoys.

To quantify in terms of PCC the actual amount of perturbation introduced in the |Δd_n| values of a single model, we used a score defined as PCC_MODEL. This score is computed as: (8) where n_R is the number of perturbed |Δd_n| sets (in our case 5), r is the index for these sets, U is the number of templates of the model and PCC_u,r indicates the observed PCC between the list of |Δd_n| values associated with the u-th template and the corresponding list of perturbed values in set r. For each HDDR group, the relationship between PCC_SEL values and the average PCC_MODEL observed in the AS and AM sets is almost perfectly linear (see Fig G and H in S2 Fig), confirming the efficacy of the perturbation scheme.

Inclusion of statistical potential terms in the objective function of MODELLER

In this work, we explored the effect of including in the objective function of MODELLER terms for interatomic distance statistical potentials. These potentials are developed with the aim of recognizing native-like protein conformations [37], therefore their use could help MODELLER to approach these conformations [38].

We employed the DOPE potential [26], which is integrated in the MODELLER package where it is commonly used to evaluate qualities of 3D models. DOPE is an “all atom” potential. Its 12561 terms are approximated with interpolating cubic splines, which can be differentiated analytically and used in the gradient-based optimization algorithm of the program.

The Lee group previously included the DFIRE [23] potential in the MODELLER objective function [18]. To compare their performances in 3D model building, we also integrated DFIRE in MODELLER (DFIRE parameters were obtained from its source code).

When including statistical potential terms, the MODELLER objective function becomes: (9) where F_SP contains the statistical potentials terms and w_SP is their weight. For obtaining best 3D modeling results, we tested several values of w_SP.

We employed statistical potentials using a contact shell value of 8.0 Å. Higher values can be safely avoided because the terms of DOPE and DFIRE start to acquire a flat shape over the 8.0 Å threshold (see Fig A in S3 Fig). The code we used to employ these potentials in MODELLER is freely available at https://github.com/pymodproject/altmod.

Results

Effects of optimal σ values on 3D modeling

Effects on single-template modeling.

Gaussian HDDRs are the heart of the MODELLER approach. At first, we explored how the use of optimal σ values (that is, |Δd_n| values) influences single-template modeling. The Lee group already reported it to bring significant improvements for a small number of proteins. Here, we extended the analysis to a larger set to derive general conclusions. As shown in Table 1, employing restraints bearing |Δd_n| values greatly increases 3D modeling accuracy. In terms of global Cα backbone quality, the average GDT-HA score of the AS models increases by 6.0% with respect to the score obtained with σ_MOD values. An improvement is also observed for local all-atom quality, as the average lDDT score increases by 4.2%. Increments in GDT-HA and lDDT are seen for 224/225 and 225/225 AS models respectively (see Fig 3A and 3B).

Download:

Fig 3. The use of optimal parameters for HDDRs improves 3D modeling quality.

(A) and (B) GDT-HA and lDDT scores of the AS models built with σ_MOD (reported on the x-axis) and with optimal |Δd_n| (y-axis) values. (C) and (D) GDT-HA and lDDT scores for the AM models obtained with MODELLER-generated (x-axis) and optimal (y-axis) HDDRs.

https://doi.org/10.1371/journal.pcbi.1007219.g003

Download:

Table 1. 3D modeling qualities of the AS single-template models built with optimal HDDRs and alignments.

https://doi.org/10.1371/journal.pcbi.1007219.t001

Increasing target-template alignment quality is one of the current challenges in TBM. In our AS models, the average accuracy of HHalign sequence alignments with respect to error-free TM-align structural alignments is 0.87 (see S4 Fig). When rebuilding the AS models using σ_MOD values and TM-align alignments, the average GDT-HA and lDDT scores improve by 6.1% and 5.9% respectively over the scores obtained with σ_MOD values and HHalign alignments (see Table 1). These results show that by optimizing parameters of the 3D model building phase of single-template HM, the same improvement obtainable by optimizing alignment building can be reached.

It might be thought that |Δd_n| values aid 3D modeling by compensating for alignment errors, that is, by assigning misaligned residues more conformational freedom to help MODELLER repositioning them in a correct way. However, their effect can not be explained only by this mechanism, since they yield a 6.6% and 4.4% increase in GDT-HA and lDDT also when models are built with TM-align alignments (see Table 1).

Effects on multiple-template modeling.

Next, we explored the effect of optimal HDDRs in multiple-template modeling, which has never been assessed before. As shown in Table 2, applying an optimal set of σ values and template weights results in an enormous improvement in the quality of 3D models (see also Fig 3C and 3D). When building the AM models with optimal restraints, their average GDT-HA and lDDT scores improve by 38.9% and 18.9% over the scores obtained by using MODELLER-generated restraints. These increments are larger than the one observed when performing multiple-template modeling with MODELLER-generated restraints and error-free TM-align structural alignments, which results in a 5.7% and 5.1% improvements in GDT-HA and lDDT.

Download:

Table 2. 3D modeling qualities of the AM multiple-template models built with optimal HDDRs and alignments.

https://doi.org/10.1371/journal.pcbi.1007219.t002

Optimal HDDRs increase even more the beneficial effect of using multiple templates. With MODELLER-generated restraints, employing multiple templates leads to an improvement of 1.9% and 2.0% in the average GDT-HA and lDDT of the AM models over single-template modeling performed with top-templates (see the MODELLER-ST strategy in Table 2). On the other hand, with optimal HDDRs, it leads to an improvement of 33.2% and 16.0% in GDT-HA and lDDT over single-template modeling performed with optimal HDDRs (see the OPTIMAL-ST strategy in Table 2).

The reason for this large improvement is the following. In MODELLER, the pdf for a multiple-template HDDR includes a weighted contribution from each template. In optimal HDDRs, |Δd_n| values are employed as σ values in conjunction with the OL weighting scheme (see the “Methods” section). In this scheme, only the contribution of the best template is selected for each HDDR (when considering a single HDDR, the best template is defined as the one having a distance d_t as close as possible to the target distance d_n, that is, the template with lowest |Δd_n| value). On the other hand, in MODELLER-generated HDDRs, the weights are usually non-zero for every template, meaning that the contribution of the best template is always weakened. This effect increases the allowed conformational space for the restrained distance, thus making it less likely to build a model with a near-native distance.

The importance of the template-weighting scheme [10] is illustrated by the fact that when employing |Δd_n| values and a uniform weighting scheme (that is, for an HDDR with U templates each template is given a weight w_u = 1/U), the average GDT-HA and lDDT scores of the AM models improve only by 18.3% and 8.9% over the standard MODELLER (see the OPTIMAL-U strategy in Table 2).

Our data shows that if the best template can be identified for each restrained distance, a substantial improvement in 3D modeling quality can be reached. A relevant matter is therefore to understand whether for a single residue (on which several HDDRs are usually acting) or for some stretch of contiguous residues, the best template always happens to be the same, or instead if multiple templates are effectively used together. S4 Text shows that in the AM set, for regions of the target sequences covered by multiple templates, all templates are frequently used at the same time. When every template has the same level of sequence similarity with the target (e.g.: all of them have around 30% SeqId), usually no template dominates over some extended region of the target sequence. Even when considering only the HDDRs acting on a single residue, different best templates are most often used concomitantly (see Fig A and C in S4 Text). In situations where there is a large difference in SeqId among the templates, even if the template with the highest SeqId tends to be picked more frequently, also other templates may maintain a significant contribution throughout the target sequence (see Fig B in S4 Text). Therefore, in order to optimally model most target residues, multiple templates have to be effectively used. This fact suggests that in order to harness the full potential of multiple-templates in homology modeling, the concept of the “best template” for single restrained distances should be considered. In a real-life protein structure prediction scenario (where the structure of the target protein is unknown), the ability to select the best template for each distance would be related to our ability to directly estimate |Δd_n| values. If our accuracy in estimation is sufficiently high, the impact on multiple-template 3D modeling quality will be largely beneficial (see the “Perturbing optimal σ values” section).

Effects on stereochemical quality.

In both single and multiple-template modeling, the use of optimal HDDRs appears to decrease the stereochemical quality of models, as seen by increased MolProbity scores (see Table 1 and Table 2). The increment is more prominent in multiple-template modeling (2.4%) than in single-template modeling (0.7%). While optimal restraints may guide the models in conformations near the native state, at the same time they probably force stereochemical inaccuracies. However, employing a more through MDSA protocol is sufficient to almost entirely relax these inaccuracies, while maintaining high GDT-HA and lDDT scores (see the strategies with the “SLOW” suffix in the tables).

Perturbing optimal σ values

As first demonstrated in [22], σ_MOD values are weakly correlated with their optimal counterparts. In the AS models, the distributions of |Δd_n| and σ_MOD values are markedly different (see Fig 1A and 1B) and the average PCCs between them are 0.262, 0.277, 0.183 and 0.221 for the Cα-Cα, NO, SCMC and SCSC restraints groups respectively (see Fig 4A). Even with accurate alignments built through TM-align, the histogram-based approach of MODELLER produces σ values which are weakly correlated to |Δd_n| values (see Fig 4B).

Download:

Fig 4. Correlation between σ_MOD and |Δd_n| values in the AS models.

(A) Distributions for the PCCs between σ_MOD and |Δd_n| values for the HDDRs of the 225 AS models. (B) PCC distributions for the AS models rebuilt with TM-align alignments.

https://doi.org/10.1371/journal.pcbi.1007219.g004

In the previous section we have seen that the use of optimal σ values greatly improves MODELLER’s predictions. However, since |Δd_n| values can not be directly inferred without the prior knowledge of the actual 3D structure that we are trying to predict, a strategy to improve MODELLER would consist in accurately estimating them. Irrespective of the predictive algorithm, it is reasonable to suppose that |Δd_n| estimations will always bear a certain amount of error. In order to understand how 3D modeling quality changes as a function of this error, we rebuilt the models of the analysis set by perturbing their |Δd_n| values with random noise.