Skip to content

Faster parser

Christophe Geuzaine requested to merge faster-parser into master

Parsing .pro files with many (thousands or millions of) objects is slow. The culprit is the sequential searches performed with List_ISearchSeq() when constructing new objects, to assign their index. GetDP relies on these indices in lists (in contiguous memory), which the parser computes on the fly when reading the .pro file - so lists cannot be sorted for fast search.

This merge request rewrites the search logic for the 2 objects with usually the most instances (groups and functions/expressions), following the profiling of a test case from CERN with 7 million groups and 200 thousand expressions. An additional map between the object name is stored in the Problem structure: the link between object name and index is maintained/updated as needed while parsing, with the map being used to speed-up the searches.

This change should be carefully tested, as any mistake in the synchronisation between the indices stored in the map and the actual indices in the lists could lead to hard to track bugs and potentially silently invalid calculations (e.g. an expressions used instead of another).

@sabariego @inno @louisdenis @julien.dular @schnaubelt

If necessary, the same logic could be easily applied to other objects (formulations, terms, ...) - but until we really need it I think we can just focus on groups and functions for now.

Edited by Christophe Geuzaine

Merge request reports