Code modifications

We modified the existing OpenMS C++ API to include a set of functions which allow efficient processing of large XML datastructures with minimal memory expenditure. Specifically, we added the IndexedMzMLFileLoader class to the API which allows loading of an indexed mzML file and returns an object of type OnDiscMSExperiment to the client. This provides random read access (“lazy loading”) to any spectrum or chromatogram present in the indexed mzML file by loading only the currently requested data structure into memory. In addition, we added the CachedmzML class to the API which is able to read a cached mzML file from disk. For event-driven data processing, we implemented transform functions (inspired by the std::transform concept in the C++ standard template library) for the MzMLHandler and the MzXMLHandler objects which apply a requested transformation to a given file on disk without ever fully loading the file into memory. The user can provide the transformation operation through a class derived from Interfaces::IMSDataConsumer (or in Python, a class that implements all four required functions, see online documentation and S1 Text).

Several implementations of the Interfaces::IMSDataConsumer are provided and implement basic functions such as writing to the mzML format, writing to a cached mzML format, applying a given function pointer to all data, chaining consumers together to perform multiple operations in a pre-defined order or simply perform no operation (to be used at a place where a Interfaces::IMSDataConsumer object is expected but no operation is desired).

Code analysis

By dynamic analysis of the XML-parsing code of OpenMS using the “callgrind” profiling tool, we were able to identify critical performance bottlenecks. Specifically, we identified performance issues during conversion of Base64 encoded data strings (as found in the mzML and mzXML data formats) to floating point values as well as during the construction of MSSpectrum instances. We were able to improve the performance through optimized Base64 parsing and through parallelization of the critical code sections, achieving an overall performance boost of a factor of 4 or more compared to the default OpenMS 1.11 code.

Performance measurement

All measurements of speed and system memory were performed on the same Red Hat Enterprise Linux server using a custom built version of OpenMS which incorporated all the described patches and reporting code for speed and memory requirements. A small sample application, TICCalculator, was written to compare the speed and memory requirements of the different algorithms described in the text. To evaluate single-threaded parsing, the -threads command line parameter was set to 1 while it was set to 16 for the multi-threaded comparisons. To measure the parsing speed of the ProteoWizard library, a sample program was built against the ProteoWizard software library (revision 7261, committed 2015-03-06); all related code can be found at https://github.com/hroest/pwiz_readspeed. Resident size was used for all memory measurements. All measurements were performed three times (after two burn-in runs) and averages are reported. As a test file, a single file from Röst et al. [14] was chosen and test files of different sizes were generated using the OpenMS FileFilter utility by selecting a retention time range of 100, 250, 500 and 750 seconds from the original input file, starting 2000 seconds into the original file. The comparison of all algorithms was only performed on the small, filtered files since the memory inefficient algorithms could not handle the full file (60 GB size). We decided against reporting CPU time and in favor of walltime since walltime is closer to the user-experienced wait time which is relevant here. The source code for the TICCalculator sample application can be downloaded from https://github.com/hroest/OpenMS/tree/feature/measureMemoryConsumption. All comparisons shown in the main text refer to the C++ API, see S1 Text for performance benchmarks of the Python API.

Availability

All described is currently available through https://github.com/OpenMS/OpenMS and has been merged into the main OpenMS development branch. Documentation for the different file access modes by read and write use-case are provided through the OpenMS documentation including three fully functional and nightly tested sample programs (see documentation at http://ftp.mi.fu-berlin.de/OpenMS/documentation/html/tutorial_format.html).

Random read access to disk

A new class has been added to the API which provides read access to spectra present on the hard disk using a pre-computed index. The addition of the index at the end of the XML file is implemented according to the extended mzML_idx standard (mzML1.1.1_idx.xsd), which is now fully supported by the modified OpenMS library using our extensions. The client C++ or Python code is able to get access to the total number of spectra and chromatograms and retrieve individual data points directly by their identifier. This allows random access to mass spectrometric data in files of virtually any size.

For certain applications, performing random seeks inside an XML file on disk may be prohibitively expensive since for each access, the underlying XML data format will have to be parsed and decoded. This may lead the algorithm to parse the same XML data multiple times if access requests cannot be bundled together, resulting in performance bottlenecks. To address this specific problem, we next implemented a simple binary representation of mzML inside OpenMS which provides random access to mass spectrometric data as well as all meta-data on request. The implemented API encapsulates the binary representation and provides convenient access to raw data and meta-data in a transparent fashion such that the client application does not need to have knowledge about the actual representation of the data. In practice, the user would first cache the mzML file to produce a .cachedMzML file which contains all spectral and chromatogram raw data in a simple, lightweight binary format. However, the fast access comes at a cost of the initial step of producing the cached data structure and potentially lost portability between different processor architectures. We therefore recommend to produce the binary format as intermediate format only and not for long-term data storage. The processing time to produce the cached file is not included in our measurements and needs to be considered in real-world applications as well (in general this time is similar to the time the “indexed” or the “event-driven” API takes to process a single file).

Event-driven processing

Since not all algorithms need random access to mass spectrometric data, we have implemented an API for sequential (event-driven) parsing of mzML and mzXML files. Just as SAX (Simple API for XML) parsers allow client code to handle a specific XML tag as soon as it is encountered, our API allows client code to handle spectra and chromatograms as soon as they are read from disk. This allows for very efficient processing of mzML or mzXML files even in cases where a plain mzML (without an index) or a compressed file (where the index offsets do not match the compressed stream) is encountered. The ability to efficiently process compressed files with minimal memory overhead makes this implementation suitable for high throughput algorithms or for applications in constrained memory and/or disk space environments.

One potential application of the event-driven approach is to chain multiple processing steps directly to the read event. Each individual processing step corresponds to a C++ or Python class implementing a specific operation on a spectrum (or chromatogram). The processing functions of these classes will then be called sequentially, therefore implementing whole data analysis pipelines through this approach. We implemented one example for such a pipeline by chaining (i) a smoothing operation (ii) a peak picking operation and (iii) output to disk, allowing us to retrieve a spectrum immediately after it has been read in, process it and write out the processed result to disk while only one single spectrum is kept in memory at any given time (see S1 Text).

Improved parsing speed of mzML/mzXML

During our work on the OpenMS core, we also implemented several substantial performance optimizations that increase the processing speed of the mzXML and mzML parsing modules in OpenMS. We have documented these instances and produced patches for the upstream project, thus making our improvements in parsing available for all future releases of OpenMS (Figs 1 and 2, see comparison to “OpenMS 1.11”). These improvements of the OpenMS code allow it to decode XML data in parallel when multiple computing cores are available. Furthermore, we identified several performance bottlenecks which we have eliminated through optimized string parsing, identification and speed-up of “hot-paths” through the code as well as the omission of unnecessary checks that decreased performance.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Processing time (s) for the different algorithms.

The time to process a single 6.9 GB file (containing 456 million peaks) is depicted for the different algorithms with and without multithreading enabled. The processing time using the ProteoWizard library is added for comparison. While our in-memory and our event-driven data access routines are substantially faster than data access using the OpenMS 1.11 code, the novel “cached” data format is an order of magnitude faster than all other data access methods. Note that some implementations, such as OpenMS 1.11 or ProteoWizard, are not capable of utilizing multiple threads. Comparisons were performed using the TICCalculator and files described in the text using a single thread or 16 threads for the multithreaded bar.

https://doi.org/10.1371/journal.pone.0125108.g001

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Execution times and memory requirements to calculate the total ion current (TIC) for the described data access implementations.

(a)-(b) Processing time normalized to the number of peaks processed per second for the different different implementations using a single thread (a) or up to 16 threads (b). No error bars for “Cached” are depicted due to graphical reasons (see S1 Text). For comparison purposes, the previous implementation (“Release 1.11”) and the ProteoWizard library also depicted (both only support single-threaded XML parsing). The best performing implementation (“Cached”) provides a speed-up of more than 200-fold when using multiple threads. (c)-(d) Memory requirements (c) and execution times (d) as a function of the number of peaks for the different algorithms. While runtime scales linearly for all algorithms, the “event-driven” and “cached” algorithms have constant memory requirements. The implementations were allowed to use parallel processing (up to 16 threads). Comparisons were performed using the TICCalculator and files described in the text.

https://doi.org/10.1371/journal.pone.0125108.g002

Application

To show the versatility and applicability of our improvements, we have implemented an algorithm that reports summary statistics about a mass spectrometric run such as the number of peaks measured or the total ion current (TIC). We chose a question that required iterating through every single peak in a file and therefore would allow straight-forward comparison in CPU time and memory usage between the four different implementations for raw data access. The benchmark results (Fig 1) show that our novel API can achieve speedups of up to 200-fold in read speed while consuming constant memory; specifically our “cached” implementation was able to completely process a 7 GB mzML file in 1.3 seconds using less than 15 MB of memory (Fig 2). Even a complete SWATH-MS file from Röst et al. [14] with a file size of 60 GB (containing 3.95 billion peaks) was processed in 15 seconds using less than 20 MB of system memory with the cached algorithm (compared to 92 seconds using the “indexed” algorithm and to 8 minutes using the event-driven algorithm; the “in-memory” algorithm could not be compared due to system memory limits).

Next, we benchmarked our improved OpenMS kernel against the widely used ProteoWizard software library [15, 16] by implementing an algorithm that uses the ProteoWizard API to read the raw data and then computes the TIC on the given input file. In a single-threaded environment, all implementations (improved OpenMS and ProteoWizard, except “cached”) showed almost equal processing performance (maximal performance difference was 15%). Notably, the “cached” implementation proved to at least a factor of 15 faster than all other implementations (see Fig 1). Additionally, when using multi-threaded parsing, the improved OpenMS kernel is substantially faster than the ProteoWizard kernel (Fig 2a and 2b). The measured speed-up compared to ProteoWizard ranged from 20% to 50% (“in-memory” and “event-driven”) to 7 fold and almost 50 fold (“indexed” and “cached”).

Our results show a clear improvement of our modified OpenMS kernel compared to the 1.11 kernel. Depending on the chosen access mode, a speedup of 200-fold was achieved and a reduction from O(n) memory complexity to constant O(1) memory complexity was obtained for specific implementations (Fig 2). Compared to the memory requirement of over 7.9 GB, all three newly implemented access methods had much lower requirements with the “cached” method at 15 MB, the “event-driven” processing at 49 MB and the “indexed” mzML implementation at 150 MB. In terms of processing speed, all three new implementations and the original “in-memory” implementation were faster than the OpenMS 1.11 kernel. Speedups ranged from over 200 fold (“caching”) to 4 fold for the “in-memory” parsing.