Equivalent configurations using the same in-memory representation

[knoepfel]

DUNE US S&C R&D items 069 & 070

Translate equivalent framework configurations from multiple languages into a single in-memory class.

1. For C++
2. For Python

We understand this to mean that we must demonstrate that we can represent a configuration in multiple configuration languages using the same C++ class (and equivalently the same Python class) for intelligible communication of the configuration to the various components and user plugins.

We therefore need to identify the following attributes of the Configuration subsystem:

1. The ability to parse multiple configuration languages (currently understood to be FHiCL, Jsonnet, YAML)
2. The structure of an evaluated configuration that will satisfy the needs of as-yet unspecified framework components and user plugins
3. The different ways that a component would need to locate and obtain applicable configuration data

[sabasehrish]

Subsystems

Required

1. Configuration

Nice to have

1. Plugin management
2. Task management
3. Logging

[knoepfel]

Related to:

• Create configuration subsystem for F1 demonstrators #35

[knoepfel]

Responsible developer: @greenc-FNAL

[marcpaterno]

We should talk with some of the authors of Near Detector Python code to determine what kind of YAML they use to configure their programs. We need to know if they use YAML parser features that allow recovering instances of user-defined types ("unsafe mode") or do they use the version that yields dicts-of-stuff, containing only built-in Python types like dict, list, int, float.

We should also find out if anyone actually uses complex numbers in configuration.

[wlav]

We also need to know what tools they use to parse the YAML and why (that is, what they specifically needed from those tools) and if they have custom converters or any type of post-processing of the configuration once loaded.

[greenc-FNAL]

The code demonstrating this milestone may be found at https://github.com/Framework-R-D/dune-framework-design/tree/main/f1_config_demonstration