Skip to content

ParaView plugin: read separate views of a specific partition stored in a single file under the msh2 format

Michel Rasquin requested to merge separateViewsAndPartitionsForMsh2 into master

In order to ensure the best IO performance for partitioned mesh and solution files, the GmshReader plugin for ParaView relies on the distributed file format (one .msh file per partition). Indeed, for a partitioned domain (mesh and solution), ParaView requests the information for a given partition independently from the other partitions. Storing a solution in a unique global view in a single file is therefore not optimal for very large data set partitioned in thousands of partitions. However, the drawback of this one .msh file per partition approach is the number of distributed solution files that needs to be written by the solver (contention on storage servers when too many files are written in the same directory) and then handled/maintained by the user (tar, scp, etc).

This branch fixes this bottleneck with the following three steps:

  1. An extension to the msh2 format used by our solver to write separate views in a single .msh file. More precisely, the solution for each partition is stored in the same single file as a separate view per partition (hence the name of the branch). The same name is used for each view (e.g. Velocity) but the partition id specified in the header of the view is of course different. This configuration allows each partition header + data block to be written collectively by the solver with MPI-IO. Note that Gmsh was already able to handle this concept before the implementation of this branch.

  2. A new argument "partitionToRead" has been added in MergeFile() in Gmsh (with a default value of -1). This argument allows the ParaView plugin to request a specific partition stored as a view in a single .msh file. This argument is then passed to PView::readMSH() implemented in PViewIO.cpp.

  3. A new (5th) integer tag in the view header stored after (a) the time step of the view, (b) the number of components, (c) the number of entities and (d) the partition id. This additional integer tag represents the size of the data block stored in bytes that follows the header. This integer allows the reader to jump over the data block without reading it (fseek) if the partition id of the current view does not correspond to the partition requested by the plugin.

If no partition is requested (default value of -1), the default behavior of the msh2 reader is restored.

Merge request reports