OpenM++ wiki

Home

This is the home of the OpenM++ wiki. It consists mostly of links to other topics, organized into sections. For a brief description of what OpenM++ can bring to a micro-simulation or agent-based modelling project please see the Features section. Our Glossary contains brief explanations of some of the terms used in this wiki.

Contents

Introduction to OpenM++

OpenM++ is an open source platform to develop, use, and deploy micro-simulation or agent-based models. OpenM++ was designed to enable non-programmers to develop simple or complex models. Click here for an overview of OpenM++ features.

[back to contents]

Getting started

This section describes how to get OpenM++ installed and working on Windows, Linux, or MacOS, for model users or for model developers. The installation kits include a collection of simple illustrative models. That same collection of models is also present in the cloud, where it can be accessed from any web browser, with no installation required. For more information on the OpenM++ cloud collection, please Contact us.

[back to contents]

Model development

Platform-independent information:

Platform-specific information:

Modgen-specific information:

[back to contents]

Model use

This section describes how to use a model once built.

Modgen-specific information:

  • Modgen: CsvToDat utility: Command-line utility to convert CSV parameters to DAT format

[back to contents]

Model API and how to run models in cloud

The model API provides programmatic access to scenario management, model inputs, model runs, and model outputs. It is implemented by the OpenM++ oms web service, which uses standard JSON to communicate with a controlling application. The worked examples in Model scripting provide practical illustrations of how to use the model API and the oms service to automate an analysis. Incidentally, the browser-based OpenM++ user interface uses the model API and the oms service for all model-specific operations. It is also possible to create workspace for model users in cloud using oms web-service.

[back to contents]

Model scripting

The topics in this section illustrate model-based analysis in two different scripting environments: Python and R. The Model API is used in these environments to create scenarios, run the model iteratively, and retrieve results for graphical presentation in the scripting environment.

[back to contents]

Docker

Docker is a technology used here to quickly replicate preconfigured operating system environments containing OpenM++ functionality.

[back to contents]

Features

Here is a summary of some OpenM++ features:

General features:

  • open source: OpenM++ and all components are licensed under the very broad MIT license.
  • cross-platform: Model development and use on Windows, Linux, or MacOS.
  • standards-based: Uses industry standard formats and technologies.
  • zero-footprint: File-based installation requires no elevation of privileges.

Model developer features:

  • high-level language: Model types, parameters, entities, events, tables, etc. are specified using a compact domain-specific language targeted to microsimulation.
  • scalable complexity: From simple 'toy' models to highly complex models.
  • modularity: New events and processes can be added to a model in a new module, often with little or no modification to existing modules.
  • continuous or discrete time, or a mixture.
  • supports multiple versions: Multiple OpenM++ versions can be installed and a single environment variable used to choose among them.
  • result compare: Supports rapid comparison of all model outputs during incremental model development.

Computational features:

  • scalable computation: Designed to scale linearly with population size or replicates when possible, N log N scaling for typical interacting populations.
  • grid-enabled, cloud-enabled: Supports MPI for multi-processing to distribute execution of replicates to a small or large computational grid or to the cloud, with automatic result assembly.
  • multi-threaded: Supports multi-threading for parallel execution of replicates on desktop or server.
  • on-the-fly tabulation: Tables are computed during the simulation, eliminating the need to output voluminous microdata for subsequent tabulation.
  • computationally efficient: The model specification is transformed to C++ which is processed by an optimizing C++ compiler to produce a highly efficient executable program.

Usability features:

  • generated UI: A model-specific UI is generated from the model specification.
  • browser-based UI: The UI requires only a browser, and runs on almost any modern browser.
  • cloud-enabled: Models can be deployed to a cloud and accessed remotely over the web, from a browser.
  • multilingual support: For UI and for model, with real-time language switching

Analyst features:

  • continuous time tabulation: Powerful but easy to use language constructs to tabulate time-in-state, empirical hazards, transitions counts, state changes, etc.
  • replicate support: All tables can have underlying replicate simulations to assess the uncertainty of any cell of any output table. Statistical measures of uncertainty are computed for all cells of all tables.
  • automation: Models can be controlled by scripts, eg Python or R.
  • import/export: Models and runs can be moved between databases, or to standard formats for upstream preparation of inputs or for downstream analysis of outputs.
  • dynamic run control: A computational grid can process runs dynamically to enable whole-model estimation or calibration, with a controlling script reading run results and preparing new runs for execution.

The OpenM++ language is based on the Modgen↗ language developed at Statistics Canada. With minor modifications to model source code, existing Modgen models can work with either Modgen or OpenM++.

[back to contents]

OpenM++ development

This section contains technical information for programmers interested in OpenM++ itself, as opposed to model developers or model users. It describes how to set up a programming environment to build and modify OpenM++.

[back to contents]

OpenM++ design

This section contains technical and project information of interest to programmers or system architects. It dates from the inception and 'alpha' days of the OpenM++ project. The road map diagram remains somewhat relevant and may be useful for a broad overview of the major components of OpenM++ from the perspective of a programmer or system architect.

Project Status: production stable since February 2016

[back to contents]

OpenM++ source code

This section contains technical information for programmers interested in OpenM++ itself, as opposed to model developers or model users. It contains links to the OpenM++ source code and to the documentation of that source code.

[back to contents]

Contact Us

[back to contents]

Windows: Quick Start for Model Users

Where is OpenM++

It is recommended to start from desktop version of openM++.

You need to use cluster version of openM++ to run the model on multiple computers in your network, in cloud or HPC cluster environment. OpenM++ is using MPI to run the models on multiple computers. Please check Model Run: How to Run the Model page for more details.

You can use Docker containers to avoid installation of multiple additional components in your host computer. Because all necessary software will be installed in container your host system will be clean.

Prerequisites

You may need to install Microsoft Visual C++ redistributable runtime, unless it is already installed as a part of some other software package:

If it is not present then please follow Microsoft instructions about: Visual C++ Redistributable.

Run on Windows computer

  • download and unzip Windows desktop binaries openmpp_win_YYYYMMDD.zip into C:\SomeDir\

  • run modelOne model with single subsample on local machine: 

    C:
cd \SomeDir\openmpp_win_20180205\models\bin
modelOne.exe
    2014-03-17 17:14:24.0023 Model: modelOne
2014-03-17 17:14:24.0070 Reading Parameters
2014-03-17 17:14:24.0085 Running Simulation
2014-03-17 17:14:24.0101 Writing Output Tables
2014-03-17 17:14:24.0179 Done.

  • run modelOne model with 16 subsamples and 4 threads: 

    modelOne.exe -OpenM.Subvalues 16 -OpenM.Threads 4
    2017-06-06 17:35:29.0421 modelOne
2017-06-06 17:35:29.0435 One-time initialization
2017-06-06 17:35:29.0454 Run: 106
2017-06-06 17:35:29.0456 Reading Parameters
2017-06-06 17:35:29.0460 Running Simulation
2017-06-06 17:35:29.0464 Writing Output Tables
........................
2017-06-06 17:35:29.0870 Done.

  • run other models (i.e. NewCaseBased, NewTimeBased, RiskPaths):

    NewCaseBased.exe -OpenM.Subvalues 8 -OpenM.Threads 2

  • run RiskPaths model with new parameter value CanDie = true and all other parameter values the same as in previous model run:

    RiskPaths.exe -Parameter.CanDie true -OpenM.BaseRunId 102
    2020-08-14 17:27:48.574 RiskPaths
2020-08-14 17:27:48.610 Run: 103
2020-08-14 17:27:48.618 Sub-value: 0
2020-08-14 17:27:48.628 member=0 Simulation progress=0% cases=0
........................
2020-08-14 17:27:54.883 Done.

  • run modelOne to compute modeling task "taskOne": 

    modelOne.exe -OpenM.Subvalues 16 -OpenM.Threads 4 -OpenM.TaskName taskOne
    2017-06-06 17:39:24.0757 modelOne
2017-06-06 17:39:24.0782 One-time initialization
2017-06-06 17:39:24.0800 Run: 107
2017-06-06 17:39:24.0802 Reading Parameters
2017-06-06 17:39:24.0807 Running Simulation
........................
2017-06-06 17:39:25.0232 Run: 108
2017-06-06 17:39:25.0234 Reading Parameters
........................
2017-06-06 17:39:25.0661 Done.

  • in case if previous model run fail, for example, due to power outage, then it can be "restarted": 

    modelOne.exe -OpenM.RestartRunId 1234

    output may vary depending on the stage where previous modelOne run failed, but still similar to above.

Note: We recommend to use normal Windows command line cmd.exe. If you are using Windows PowerShell then it may be necessary to put "quotes" around command line options, e.g:

model.exe "-OpenM.Subvalues" 16

Run on multiple computers over network, in HPC cluster or cloud

  • download and unzip Windows cluster binaries openmpp_win_mpi_YYYYMMDD.zip into C:\AnyDir. Please notice name of cluster version archive has mpi in it, i.e. openmpp_win_mpi_20180205.zip and is located in a subdirectory mpi.

  • if you are using regular Windows computers in your organization network (like Windows 7 or 10 and not MS HPC servers or Azure) then:

    • make sure you have latest version of Microsoft MPI Redistributable installed.
    • or pull Docker image docker pull openmpp/openmpp-run:windows-1903 to run models inside the container (see below).

  • run modelOne model with single subsample on local machine: 

    C:
cd \AnyDir\openmpp_win_mpi_20180205\models\bin
modelOne_mpi.exe
    2014-03-17 17:14:24.0023 Model: modelOne
2014-03-17 17:14:24.0070 Reading Parameters
2014-03-17 17:14:24.0085 Running Simulation
2014-03-17 17:14:24.0101 Writing Output Tables
2014-03-17 17:14:24.0179 Done.

  • run two instances of modelOne to compute 16 subsamples and 4 threads: 

    mpiexec -n 2 modelOne_mpi.exe -OpenM.Subvalues 16 -OpenM.Threads 4
    2017-06-06 17:52:06.0143 modelOne
2017-06-06 17:52:06.0145 modelOne
2017-06-06 17:52:06.0179 Parallel run of 2 modeling processes, 4 thread(s) each
2017-06-06 17:52:06.0179 One-time initialization
2017-06-06 17:52:06.0179 One-time initialization
2017-06-06 17:52:06.0192 Run: 106
2017-06-06 17:52:06.0192 Run: 106
2017-06-06 17:52:06.0192 Reading Parameters
........................
2017-06-06 17:52:06.0532 Writing Output Tables
2017-06-06 17:52:06.0599 Done.
2017-06-06 17:52:06.0599 Done.

  • run other models (i.e. NewCaseBased, NewTimeBased, RiskPaths):

    mpiexec -n 8 NewCaseBased_mpi.exe -OpenM.Subvalues 64 -OpenM.Threads 4

Microsoft recommends to install HPC Pack which simplifies your computational resources management rather than using mpiexec as above. It is also possible to use Microsoft Azure cloud where compute nodes available for you on demand.

Run models using Docker container

  • download and unzip openmpp_win_YYYYMMDD.zip into C:\AnyDir.

  • make sure you have Docker for Windows installed, see Microsoft documentation for more details.

  • pull Docker image:

    docker pull openmpp/openmpp-run:windows-1903

  • run modelOne model with single subsample:

    docker run -v C:\AnyDir\models\bin:C:\ompp openmpp/openmpp-run:windows-1903 modelOne.exe
    2014-03-17 17:14:24.0023 Model: modelOne
2014-03-17 17:14:24.0070 Reading Parameters
2014-03-17 17:14:24.0085 Running Simulation
2014-03-17 17:14:24.0101 Writing Output Tables
2014-03-17 17:14:24.0179 Done.

  • run two instances of modelOne to compute 16 subsamples and 4 threads: 

    docker run -v C:\AnyDir\models\bin:C:\ompp openmpp/openmpp-run:windows-1903 mpiexec -n 2 modelOne_mpi.exe -OpenM.Subvalues 16 -OpenM.Threads 4
    2017-06-06 17:52:06.0143 modelOne
2017-06-06 17:52:06.0145 modelOne
2017-06-06 17:52:06.0179 Parallel run of 2 modeling processes, 4 thread(s) each
2017-06-06 17:52:06.0179 One-time initialization
2017-06-06 17:52:06.0179 One-time initialization
2017-06-06 17:52:06.0192 Run: 106
2017-06-06 17:52:06.0192 Run: 106
2017-06-06 17:52:06.0192 Reading Parameters
........................
2017-06-06 17:52:06.0532 Writing Output Tables
2017-06-06 17:52:06.0599 Done.
2017-06-06 17:52:06.0599 Done.

  • run other models (i.e. NewCaseBased, NewTimeBased, RiskPaths):

    docker run -v C:\AnyDir\models\bin:C:\ompp openmpp/openmpp-run:windows-1903 mpiexec -n 8 NewCaseBased_mpi.exe -OpenM.Subvalues 64 -OpenM.Threads 4

Windows: Quick Start for Model Developers

Step by Step

  • Download desktop version zip archive: openmpp_win_YYYYMMDD.zip binary files and source code
  • Extract zip archive to C:\openmpp_win_20210112\
  • Build the example RiskPaths model and run the Default scenario
    • Open C:\openmpp_win_20210112\models\RiskPaths\RiskPaths-ompp.sln using Visual Studio 2019
    • 'Rebuild' in Visual Studio 2019 to build the model and run the Default scenario
    • (optional) Enable in project Properties -> OpenM++ -> "Run scenario after build" to examine model run results
    • (optional) Enable "Export model run results into csv files" to create CSV files containing values of model parameters and output tables
    • (optional) Enable "Open model web UI" to modify parameters, run the model and view model results
  • (optional) to enable model development from any directory, independent of C:\openmpp_win_20210112 location, do any of:
    • open a Command Prompt window and type the command: setx OM_ROOT C:\openmpp_win_20210112
    • open your model Model.vcrproj file in Notepad and update line:
      <OM_ROOT>C:\openmpp_win_20210112</OM_ROOT>
  • How to: create and debug models on Windows

OpenM++ Models: desktop? clusters? MPI?

It is recommended to start from desktop version of openM++.

You need to use cluster version of openM++ to run the model on multiple computers in your network, in cloud or HPC cluster environment. OpenM++ is using MPI to run the models on multiple computers. Please check Model Run: How to Run the Model page for more details.

Build on Windows

Tested platforms:

  • Windows 10, 2016 (64 bit)
  • Visual Studio 2019, including Community Edition
  • (optional) Microsoft MPI SDK Redistributable Package

Note: It may work on any Windows 7 and above or 2008R2 and above, 32 and 64 bits, with Visual Studio 2017. However we are not testing it on older versions of Windows or Visual Studio.

Build debug version of the model

You can use any of test models solution, except of modelOne, as starting point to develop your own model. Below we are using NewCaseBased model as example.

To build and run debug version of the model use desktop (non-MPI) version of openM++:

  • download and unzip openmpp_win_YYYYMMDD.zip Windows desktop binaries into C:\openmpp_win_20210112\
  • build Debug version of the model using solution: C:\openmpp_win_20210112\models\NewCaseBased\NewCaseBased-ompp.sln
  • (optional) Rebuild the model and run it:
    • go to menu: Project -> Properties -> Configuration Properties -> OpenM++
    • change: Run Options -> Run the scenario after build -> Yes
    • Rebuild project

At bottom Output window of Visual Studio you will see something like:

1>Model.vcxproj -> C:\openmpp_win_20210112\models\NewCaseBased\ompp\bin//NewCaseBasedD.exe
1>2017-06-06 18:21:08.0092 NewCaseBased
1>2017-06-06 18:21:08.0160 Run: 102
1>2017-06-06 18:21:08.0163 Get fixed and missing parameters
1>2017-06-06 18:21:08.0166 Get scenario parameters
1>2017-06-06 18:21:08.0172 Sub-value 0
1>2017-06-06 18:21:08.0175 compute derived parameters
1>2017-06-06 18:21:08.0177 Initialize invariant entity data
1>2017-06-06 18:21:08.0180 Member=0 simulation progress=0%
........
1>2017-06-06 18:21:08.0688 member=0 write output tables - finish
1>2017-06-06 18:21:08.0697 Writing Output Tables Expressions
1>2017-06-06 18:21:08.0727 Done.
1>Done building project "Model.vcxproj".
======= Build: 1 succeeded, 0 failed, 0 up-to-date, 0 skipped =======

Build cluster version of the model to run on multiple computers over network

Make sure you have latest version of Microsoft MPI SDK and MPI Redistributable installed.

  • download and unzip openmpp_win_mpi_YYYYMMDD.zip Windows cluster binaries into C:\openmpp_win_mpi_20180205. Please notice name of cluster version archive has mpi in it, i.e. openmpp_win_mpi_20180205.zip.

  • Rebuild the model and run it:

    • go to menu: Project -> Properties -> Configuration Properties -> OpenM++
    • change: Build Options -> Grid computing (MPI) -> MPI-enabled Executable (MPI)
    • change: Run Options -> Number of processes to launch -> use 2 or more (depends on your cluster configuration)
    • change: Run Options -> Run the scenario after build -> Yes
    • Rebuild Model project

At bottom Output window of Visual Studio you will see something like:

1>Model.vcxproj -> C:\openmpp_win_mpi_20180205\models\RiskPaths\ompp\bin\RiskPaths_mpi.exe
1>2021-06-01 20:57:28.146 RiskPaths
1>2021-06-01 20:57:28.146 RiskPaths
1>2021-06-01 20:57:28.146 RiskPaths
1>2021-06-01 20:57:28.163 RiskPaths
........
1>2021-06-01 20:57:28.366 OpenM++ build  : Windows 64 bit Release MPI
1>2021-06-01 20:57:28.367 Parallel run of 4 modeling processes, 1 thread(s) each
........
1>2021-06-01 20:57:28.859 member=3 Simulation progress=100% cases=2000
1>2021-06-01 20:57:28.867 member=3 Simulation summary: cases=2000, events/case=112.9, entities/case=1.0, elapsed=0.453989s
1>2021-06-01 20:57:28.868 member=3 Write output tables - start
1>2021-06-01 20:57:28.873 member=3 Write output tables - finish
1>2021-06-01 20:57:29.233 member=0 Write output tables - finish
1>2021-06-01 20:57:29.919 Writing into aggregated output tables, run: 102
1>2021-06-01 20:57:32.607 Done.
1>2021-06-01 20:57:32.607 Done.
1>2021-06-01 20:57:32.607 Done.
1>2021-06-01 20:57:32.607 Done.
1>Done building project "Model.vcxproj".
========== Rebuild All: 1 succeeded, 0 failed, 0 skipped ==========

Note: you can build Debug version of the model and run it on cluster, but actual debuging on cluster is far from being trivial.

Using older versions of Visual Studio

OpenM++ tested on current version of Windows 10 and Visual Studio and it is likely works on previous versions too, but it is not tested. If you experiencing an issues with model build please try below recepies.

If you getting link unresolved external symbol errors:

You may get linkage errors if your model .obj files incompatible with object files in openM++ library or Microsoft VC++ libraries. For example, build error messages may look like:

1>libopenm.lib(main.obj) : error LNK2001: unresolved external symbol __imp____std_init_once_begin_initialize@16
1>libopenm.lib(main.obj) : error LNK2001: unresolved external symbol __imp____std_init_once_complete@12
1>libopenm.lib(file.obj) : error LNK2001: unresolved external symbol ___std_system_error_allocate_message@8
1>libopenm.lib(file.obj) : error LNK2001: unresolved external symbol ___std_system_error_deallocate_message@4
1>C:\openmpp_win_20210304\models\RiskPaths\ompp\bin\RiskPaths.exe : fatal error LNK1120: 4 unresolved externals

  1. Clean existing model intermediate files and build model again. Assuming your model directory is C:\openmpp_win_20210304\models\RiskPaths then remove following directories:

    C:\openmpp_win_20210304\models\RiskPaths\ompp\bin\
C:\openmpp_win_20210304\models\RiskPaths\ompp\build\
C:\openmpp_win_20210304\models\RiskPaths\ompp\src\

  2. If you are using Visual Studio 2019 or 2017 then recepie above may not solve the problem. In that case you need to rebuild libopenm openM++ model run-time libarary.

  3. Open solution C:\openmpp_win_20210304\openm\openm.sln:

  • Rebuild libopenm library:
    • Visual Studio menu -> Build -> Batch Build...
    • select all libopenm projects: Debug / Release / x64 / Win32
    • click on Rebuild

  • Open your model solution and do rebuild. It is expected to work and you should be able to debug your model even with Visual Studio 2017:

If you getting build error MSB8036:

C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\Common7\IDE\VC\VCTargets\Platforms\Win32\PlatformToolsets\v141\Toolset.targets(34,5): 
error MSB8036: The Windows SDK version 10.0.14393.0 was not found. 
Install the required version of Windows SDK or change the SDK version in the project property pages or by right-clicking the solution and selecting "Retarget solution".

then do one of:

  • "Retarget solution"
  • use Visual Studio 2019
  • install Windows 8.1 SDK and UCRT SDK:

Linux: Quick Start for Model Users

Where is OpenM++

It is recommended to start from desktop version of openM++.

You need to use cluster version of openM++ to run the model on multiple computers in your network, in cloud or HPC cluster environment. OpenM++ is using MPI to run the models on multiple computers. Please check Model Run: How to Run the Model page for more details.

You can use Docker containers to avoid installation of multiple additional components in your host computer. Because all necessary software will be installed in container your host system will be clean.

Run on Linux computer

  • download and unpack openM++, i.e.:

    wget https://github.com/openmpp/main/releases/download/v1.2.0/openmpp_debian_20190508.tar.gz
tar xzf openmpp_debian_20190508.tar.gz

  • run modelOne model with single subsample on local machine: 

    cd openmpp_debian_20190508/models/bin/
./modelOne
    2017-06-06 19:24:53.0747 modelOne
2017-06-06 19:24:53.0763 Run: 105
2017-06-06 19:24:53.0763 Reading Parameters
2017-06-06 19:24:53.0764 Running Simulation
2017-06-06 19:24:53.0765 Writing Output Tables
2017-06-06 19:24:53.0790 Done.

  • run modelOne model with 16 subsamples and 4 threads: 

    ./modelOne -OpenM.Subvalues 16 -OpenM.Threads 4
    2017-06-06 19:25:38.0721 modelOne
2017-06-06 19:25:38.0735 Run: 106
2017-06-06 19:25:38.0735 Reading Parameters
........................
2017-06-06 19:25:38.0906 Done.

  • run other models (i.e. NewCaseBased, NewTimeBased, RiskPaths):

    ./NewCaseBased -OpenM.Subvalues 32 -OpenM.Threads 4

  • run RiskPaths model with new parameter value CanDie = true and all other parameter values the same as in previous model run:

    RiskPaths -Parameter.CanDie true -OpenM.BaseRunId 102
    2020-08-14 17:27:48.574 RiskPaths
2020-08-14 17:27:48.610 Run: 103
2020-08-14 17:27:48.618 Sub-value: 0
2020-08-14 17:27:48.628 member=0 Simulation progress=0% cases=0
........................
2020-08-14 17:27:54.883 Done.

  • run modelOne to compute modeling task "taskOne": 

    ./modelOne -OpenM.Subvalues 16 -OpenM.Threads 4 -OpenM.TaskName taskOne
    2017-06-06 19:27:08.0401 modelOne
2017-06-06 19:27:08.0421 Run: 107
2017-06-06 19:27:08.0421 Reading Parameters
........................
2017-06-06 19:27:08.0593 Run: 108
2017-06-06 19:27:08.0593 Reading Parameters
........................
2017-06-06 19:27:08.0704 Writing Output Tables
2017-06-06 19:27:08.0812 Done.

  • in case if previous model run fail, for example, due to power outage, then it can be "restarted":

    ./modelOne -OpenM.RestartRunId 1234

    output may vary depending on the stage where previous modelOne run failed, but still similar to above.

Run on multiple computers over network, in HPC cluster or cloud

  • make sure you have MPI run-time installed and ready to use. For example, on RedHat you may need to load it by following commands:

    module load mpi/openmpi-x86_64

    As an alternative to MPI installation you can pull Docker image docker pull openmpp/openmpp-run:debian to run models inside the container (see below).

  • download and unpack cluster version of openM++, i.e.:

    wget https://github.com/openmpp/main/releases/download/v1.2.0/openmpp_debian_mpi_20190508.tar.gz
tar xzf openmpp_debian_mpi_20190508.tar.gz

    please notice name of cluster version archive has mpi in it, i.e. openmpp_debian_mpi_20190508.tar.gz

  • run modelOne model with single subsample on local machine: 

    cd openmpp_debian_mpi_20190508/models/bin/
./modelOne_mpi
    2017-06-06 19:30:52.0690 Run: 105
2017-06-06 19:30:52.0690 Reading Parameters
2017-06-06 19:30:52.0691 Running Simulation
2017-06-06 19:30:52.0691 Writing Output Tables
2017-06-06 19:30:52.0716 Done.

  • run two instances of modelOne to compute 16 subsamples and 4 threads: 

    mpiexec -n 2 modelOne_mpi -OpenM.Subvalues 16 -OpenM.Threads 4
    2017-06-06 19:43:01.0486 modelOne
2017-06-06 19:43:01.0487 modelOne
2017-06-06 19:43:01.0742 Parallel run of 2 modeling processes, 4 thread(s) each
2017-06-06 19:43:01.0750 Run: 106
2017-06-06 19:43:01.0750 Reading Parameters
2017-06-06 19:43:01.0750 Run: 106
2017-06-06 19:43:01.0750 Reading Parameters
..........
2017-06-06 19:43:01.0800 Writing Output Tables
2017-06-06 19:43:01.0878 Done.
2017-06-06 19:43:01.0880 Done.

  • run other models (i.e. NewCaseBased, NewTimeBased, RiskPaths):

    mpiexec -n 8 NewCaseBased_mpi -OpenM.Subvalues 64 -OpenM.Threads 4

It is recommended to install SLURM or Torque to simplify your computational resources management rather than using mpiexec as above. It is also possible to use Google Cloud, Amazon or even Microsoft Azure cloud where compute nodes available for you on demand.

Run models using Docker container

  • make sure you have Docker installed, for example, on Ubuntu: sudo apt-get install docker.

  • pull Docker image:

    docker pull openmpp/openmpp-run:debian

  • image build for user ompp, UID=1999, GID=1999 and you may need to do one of:

    • add same user ompp, UID=1999, GID=1999 to your host system and login as user ompp
    • or as shown below use environment variables OMPP_* to map your current user name, UID, GID, HOME to container user

  • download and unpack cluster version of openM++, i.e.:

    wget https://github.com/openmpp/main/releases/download/v1.2.0/openmpp_debian_mpi_20200621.tar.gz
tar xzf openmpp_debian_mpi_20200621.tar.gz

    please notice name of cluster version archive has mpi in it, i.e. openmpp_debian_mpi_20200621.tar.gz

  • run modelOne model with single subsample on local machine: 

    docker run \
-v $HOME/models/bin:/home/models \
-e OMPP_USER=models -e OMPP_GROUP=models -e OMPP_UID=$UID -e OMPP_GID=`id -g` \
openmpp/openmpp-run:debian \
./modelOne_mpi
    2017-06-06 19:30:52.0690 Run: 105
2017-06-06 19:30:52.0690 Reading Parameters
2017-06-06 19:30:52.0691 Running Simulation
2017-06-06 19:30:52.0691 Writing Output Tables
2017-06-06 19:30:52.0716 Done.

    For explanation of:

    -v $HOME/models/bin:/home/models \
-e OMPP_USER=models -e OMPP_GROUP=models -e OMPP_UID=$UID -e OMPP_GID=`id -g` \

    please take a look at User, group, home_directory topic.

  • run two instances of modelOne to compute 16 subsamples and 4 threads: 

    docker run \
-v $HOME/models/bin:/home/models \
-e OMPP_USER=models -e OMPP_GROUP=models -e OMPP_UID=$UID -e OMPP_GID=`id -g` \
openmpp/openmpp-run:debian \
mpiexec -n 2 modelOne_mpi -OpenM.Subvalues 16 -OpenM.Threads 4
    2017-06-06 19:43:01.0486 modelOne
2017-06-06 19:43:01.0487 modelOne
2017-06-06 19:43:01.0742 Parallel run of 2 modeling processes, 4 thread(s) each
2017-06-06 19:43:01.0750 Run: 106
2017-06-06 19:43:01.0750 Reading Parameters
2017-06-06 19:43:01.0750 Run: 106
2017-06-06 19:43:01.0750 Reading Parameters
..........
2017-06-06 19:43:01.0800 Writing Output Tables
2017-06-06 19:43:01.0878 Done.
2017-06-06 19:43:01.0880 Done.

  • run other models (i.e. NewCaseBased, NewTimeBased, RiskPaths):

    docker run \
....user, UID, GID, HOME.... \
openmpp/openmpp-run:debian \
mpiexec -n 8 NewCaseBased_mpi -OpenM.Subvalues 64 -OpenM.Threads 4

Linux: Quick Start for Model Developers

Where is OpenM++

It is recommended to start from desktop version of openM++.

You need to use cluster version of openM++ to run the model on multiple computers in your network, in cloud or HPC cluster environment. OpenM++ is using MPI to run the models on multiple computers. Please check Model Run: How to Run the Model page for more details.

Build on Linux

Tested platforms:

  • Debian 10, MX Linux 19, Ubuntu 20.04, RedHat 8
  • g++ >= 8.3
  • (optional) MPI, i.e.: OpenMPI >= 3.1 or MPICH (other MPI implementations expected to work but not tested)
  • (optional) OpenMPI >= 4.0 on RedHat >= 8.3 (OpenMPI was broken on RedHat 8.1)

It is also occasionally tested on openSUSE, Mint, Manjaro, Solus and others.

It is not supported, but may also work on older versions, for example Ubuntu 18.04.

Build on Ubuntu 20.04

There is a minor incompatibility of shared libraries between Ubuntu 20.04 and Debian 10. As result you need to rebuild our model run-time libraries before building you own model:

  • download and unpack openM++ into any folder:
    wget https://github.com/openmpp/main/releases/download/v1.8.6/openmpp_debian_20210415.tar.gz
tar xzf openmpp_debian_20210415.tar.gz
  • rebuild model run-time libraries:
    cd openmpp_debian_20210415/openm
wget https://github.com/openmpp/main/releases/download/v1.8.6/openmpp_debian_20210415.tar.gz
tar xzf openmpp_debian_20210415.tar.gz

Build debug version of the model

You can use any of test models makefile, except of modelOne, as starting point to develop your own model. Below we are using NewCaseBased model as example.

To build and run debug version of the model use desktop (non-MPI) version of openM++:

  • check your g++ --version:

    g++ (Debian 8.3.0-6) 8.3.0
g++ (Ubuntu 9.3.0-10ubuntu2) 9.3.0
g++ (GCC) 8.3.1 20191121 (Red Hat 8.3.1-5)

  • download and unpack openM++

    wget https://github.com/openmpp/main/releases/download/v1.8.3/openmpp_debian_20210304.tar.gz
tar xzf openmpp_debian_20210304.tar.gz

  • build debug version of NewCaseBased model and "publish" it ("publish" do create NewCaseBased.sqlite database with default input data set)

    cd openmpp_debian_20210304/models/NewCaseBased/
make all publish

  • run the model

    cd ompp-linux/bin
./NewCaseBasedD
    2017-06-06 19:59:12.0429 NewCaseBased
2017-06-06 19:59:12.0449 Run: 103
2017-06-06 19:59:12.0449 Get fixed and missing parameters
2017-06-06 19:59:12.0449 Get scenario parameters
2017-06-06 19:59:12.0450 Sub-value 0
2017-06-06 19:59:12.0450 compute derived parameters
2017-06-06 19:59:12.0450 Initialize invariant entity data
2017-06-06 19:59:12.0450 Member=0 simulation progress=0%
..........
2017-06-06 19:59:12.0505 member=0 write output tables - finish
2017-06-06 19:59:12.0508 Writing Output Tables Expressions
2017-06-06 19:59:12.0520 Done.

Build release version of the model

Make executable, "publish" and run NewCaseBased test model:

cd openmpp_debian_20210304/models/NewCaseBased/
make RELEASE=1 clean-all
make RELEASE=1 all publish
cd ompp-linux/bin
./NewCaseBased

Rebuild all test models

Make executables, "publish" (create model.sqlite database file) and run all test models:

cd openmpp_debian_20210304/models/
make RELEASE=1 clean-all
make RELEASE=1 all publish run publish-all

results are in openmpp_debian_20210304/models/bin directory

OM_ROOT: How to separate model folder and openM++ release folder

If you want to keep model development folder(s) outside of openM++ release directory then set OM_ROOT environment variable to specify openM++ release location. For example if your model is in $HOME/my-models/BestModel then to build it do any of:

cd my-models/BestModel
OM_ROOT=openmpp_debian_20210304 make all publish run

Or edit $HOME/my-models/BestModel/makefile to set OM_ROOT:

ifndef OM_ROOT
  OM_ROOT = $(HOME)/openmpp_debian_20210304
endif

Or add export OM_ROOT=$HOME/openmpp_debian_20210304 into your .bash_profile

Build cluster version of the model to run on multiple computers over network

Make sure you have MPI installed and configured. For example, on RedHat you may need to load MPI module: module load mpi/openmpi-x86_64

  • download and unpack cluster version of openM++, i.e.:

    wget https://github.com/openmpp/main/releases/download/v1.8.3/openmpp_debian_mpi_20210304.tar.gz
tar xzf openmpp_debian_mpi_20210304.tar.gz

    please notice name of cluster version archive has mpi in it, i.e. openmpp_debian_mpi_20210304.tar.gz

  • make executable and "publish" (create model.sqlite database file) of NewCaseBased test model:

    cd openmpp_debian_mpi_20210304/models/NewCaseBased/
make RELEASE=1 OM_MSG_USE=MPI all publish

  • run 3 instances of NewCaseBased on 3 hosts to compute 16 subsamples using 4 threads

    cd ompp-linux/bin
mpirun -n 3 -H omm,om1,om2 NewCaseBased_mpi -OpenM.Subvalues 16 -OpenM.Threads 4
    2017-06-06 20:15:12.0050 NewCaseBased
2017-06-06 20:15:12.0173 NewCaseBased
2017-06-06 20:15:12.0200 NewCaseBased
2017-06-06 20:15:13.0148 Parallel run of 3 modeling processes, 4 thread(s) each
2017-06-06 20:15:13.0162 Run: 102
2017-06-06 20:15:13.0163 Get fixed and missing parameters
2017-06-06 20:15:13.0163 Get scenario parameters
2017-06-06 20:15:13.0164 compute derived parameters
2017-06-06 20:15:13.0164 Initialize invariant entity data
2017-06-06 20:15:13.0161 Run: 102
..........
2017-06-06 20:15:13.0224 member=0 write output tables - finish
2017-06-06 20:15:13.0354 Done.
2017-06-06 20:15:13.0352 Done.
2017-06-06 20:15:13.0353 Done.

You can use any of test models makefile, except of modelOne, as starting point to develop your own model.

MacOS: Quick Start for Model Users

Where is OpenM++

You can have multiple versions of openM++ installed on your computer. OpenM++ distributed as tar.gz archive, you can unpack into any directory and it is ready to use. In the documentation that directory called OM_ROOT.

OpenM++ does not update any system shared resources and you can remove it any time by simply deleting openM++ directory.

It is possible to run openM++ models:

On Linux and/or Windows you also can run model in cloud or on high perfomance cluster (HPC). Please also check Model Run: How to Run the Model page for more details.

Run openM++ models from terminal command line

  • download and unpack openM++ using Safari or, for example, curl:

    curl -L -o om.tar.gz https://github.com/openmpp/main/releases/download/v1.6.0/openmpp_mac_20200621.tar.gz
tar xzf om.tar.gz

  • run modelOne model with single sub-sample on local machine: 

    cd openmpp_mac_20200621/models/bin/
./modelOne
    2017-06-06 19:24:53.0747 modelOne
2017-06-06 19:24:53.0763 Run: 105
2017-06-06 19:24:53.0763 Reading Parameters
2017-06-06 19:24:53.0764 Running Simulation
2017-06-06 19:24:53.0765 Writing Output Tables
2017-06-06 19:24:53.0790 Done.

  • run modelOne model with 16 sub-samples and 4 threads: 

    ./modelOne -OpenM.Subvalues 16 -OpenM.Threads 4
    2017-06-06 19:25:38.0721 modelOne
2017-06-06 19:25:38.0735 Run: 106
2017-06-06 19:25:38.0735 Reading Parameters
........................
2017-06-06 19:25:38.0906 Done.

  • run other models (i.e. NewCaseBased, NewTimeBased, RiskPaths):

    ./NewCaseBased -OpenM.Subvalues 32 -OpenM.Threads 4

  • run RiskPaths model with new parameter value CanDie = true and all other parameter values the same as in previous model run:

    RiskPaths -Parameter.CanDie true -OpenM.BaseRunId 102
    2020-08-14 17:27:48.574 RiskPaths
2020-08-14 17:27:48.610 Run: 103
2020-08-14 17:27:48.618 Sub-value: 0
2020-08-14 17:27:48.628 member=0 Simulation progress=0% cases=0
........................
2020-08-14 17:27:54.883 Done.

  • run modelOne to compute modeling task "taskOne": 

    ./modelOne -OpenM.Subvalues 16 -OpenM.Threads 4 -OpenM.TaskName taskOne
    2017-06-06 19:27:08.0401 modelOne
2017-06-06 19:27:08.0421 Run: 107
2017-06-06 19:27:08.0421 Reading Parameters
........................
2017-06-06 19:27:08.0593 Run: 108
2017-06-06 19:27:08.0593 Reading Parameters
........................
2017-06-06 19:27:08.0704 Writing Output Tables
2017-06-06 19:27:08.0812 Done.

  • in case if previous model run fail, for example, due to power outage, then it can be "restarted":

    ./modelOne -OpenM.RestartRunId 1234

    output may vary depending on the stage where previous modelOne run failed, but still similar to above.

MacOS: Quick Start for Model Developers

Where is OpenM++

Also, please check Model Run: How to Run the Model page for more details.

Prerequisites

  • Tested on: MacOS 10.15 Catalina And Big Sur >= 11.1.
  • Install Xcode and command line developer tools, if not installed already by Xcode: xcode-select --install.
  • (optional) Install Visual Studio Code for cross-platform development: https://code.visualstudio.com/docs/?dv=osx
  • Check if clang, make and sqlite3 are installed on your computer:
g++ --version
....
Apple clang version 11.0.0 (clang-1100.0.33.12)

make --version
....
GNU Make 3.81

sqlite3 --version
....
3.28.0 2019-04-15 14:49:49

MacOS security issue

Make sure you are using tight security settings on your Mac and antivirus software, if necessary. We are trying our best to keep development machines clean, but cannot provide any guarantee.

On Big Sur it is very likely to get an security error when you are trying to run any downloaded executable:

  • please reply "Cancel" to that question (click "Cancel" button).
  • remove quarantine attribute from openM++ installation directory, for example:
    xattr -r -d com.apple.quarantine ~/openmpp_mac_20200621

Build debug version of the model from terminal command line

You can use any of test models makefile, except of modelOne, as starting point to develop your own model. Below we are using NewCaseBased model as example.

To build and run debug version of the model:

  • download and unpack latest openM++ release using Safari or curl:

    curl -L -o om.tar.gz https://github.com/openmpp/main/releases/download/v1.6.0/openmpp_mac_20200621.tar.gz
tar -xzf om.tar.gz

  • remove quarantine attribute from openM++ installation directory:

    xattr -r -d com.apple.quarantine openmpp_mac_20200621

  • build debug version of NewCaseBased model and "publish" it ("publish" do create NewCaseBased.sqlite database with default input data set)

    cd openmpp_mac_20200621/models/NewCaseBased/
make all publish

  • run the model

    cd ompp-mac/bin
./NewCaseBasedD
    2017-06-06 19:59:12.0429 NewCaseBased
2017-06-06 19:59:12.0449 Run: 103
2017-06-06 19:59:12.0449 Get fixed and missing parameters
2017-06-06 19:59:12.0449 Get scenario parameters
2017-06-06 19:59:12.0450 Sub-value 0
2017-06-06 19:59:12.0450 compute derived parameters
2017-06-06 19:59:12.0450 Initialize invariant entity data
2017-06-06 19:59:12.0450 Member=0 simulation progress=0%
..........
2017-06-06 19:59:12.0505 member=0 write output tables - finish
2017-06-06 19:59:12.0508 Writing Output Tables Expressions
2017-06-06 19:59:12.0520 Done.

  • you can also build and run the model using make:

    make all publish run
..........
2017-06-06 19:59:12.0429 NewCaseBased
2017-06-06 19:59:12.0449 Run: 103
..........
2017-06-06 19:59:12.0508 Writing Output Tables Expressions
2017-06-06 19:59:12.0520 Done.

Build release version of the model from terminal command line

Make executable, "publish" and run NewCaseBased test model:

cd openmpp_mac_20200621/models/NewCaseBased/
make RELEASE=1 clean-all
make RELEASE=1 all publish
cd ompp-mac/bin
./NewCaseBased

Rebuild all test models

Make executables, "publish" (create model.sqlite database file) and run all test models:

cd openmpp_mac_20200621/models/
make RELEASE=1 clean-all
make RELEASE=1 all publish run publish-all

results are in openmpp_mac_20200621/models/bin directory

OM_ROOT: How to separate model folder and openM++ release folder

If you want to keep model development folder(s) outside of openM++ release directory then set OM_ROOT environment variable to specify openM++ release location. For example if your model is in $HOME/my-models/BestModel then to build it do any of:

cd my-models/BestModel
OM_ROOT=openmpp_mac_20200621 make all publish run

Or edit $HOME/my-models/BestModel/makefile to set OM_ROOT:

ifndef OM_ROOT
  OM_ROOT = $(HOME)/openmpp_mac_20200621
endif

Or add export OM_ROOT=$HOME/openmpp_mac_20200621 into your .zprofile

Build openM++ sample model using Xcode

Download and unpack latest openM++ release using Safari or curl:

curl -L -o om.tar.gz https://github.com/openmpp/main/releases/download/v1.6.0/openmpp_mac_20200621.tar.gz
tar xzf om.tar.gz

Start Xcode and open any example model workspace, for example: ~/openmpp_mac_20200621/models/NewCaseBased/Model.xcworkspace