From 0ba8da53bceaa376e49fcdd5992d65a907dac891 Mon Sep 17 00:00:00 2001 From: Kyle Knoepfel Date: Fri, 17 Jul 2026 15:53:37 -0500 Subject: [PATCH 1/5] Migrate remaining 'product name' instances to 'product suffix' --- doc/ch_conceptual_design/data_organization.rst | 2 +- doc/ch_conceptual_design/hofs/transforms.rst | 15 ++++++++------- 2 files changed, 9 insertions(+), 8 deletions(-) diff --git a/doc/ch_conceptual_design/data_organization.rst b/doc/ch_conceptual_design/data_organization.rst index d981bcf..1ce928e 100644 --- a/doc/ch_conceptual_design/data_organization.rst +++ b/doc/ch_conceptual_design/data_organization.rst @@ -92,7 +92,7 @@ Such metadata include: - the *creator*, the name of the algorithm that created the data product - an identifier for the *data cells* with which the data product is associated (e.g. `Spill`, `Run`, `Calibration Interval`, or other experiment-defined layer) - the *stage name*, an identifier for the job in which the data product was created -- an individual *name* for the data product (which may be empty), to distinguish between multiple products of the same type created by the same algorithm. +- an individual *suffix* for the data product (which may be empty), to distinguish between multiple products of the same type created by the same algorithm. In addition to these metadata, a data product is also specified by its *type*. diff --git a/doc/ch_conceptual_design/hofs/transforms.rst b/doc/ch_conceptual_design/hofs/transforms.rst index 886bf8c..1d74532 100644 --- a/doc/ch_conceptual_design/hofs/transforms.rst +++ b/doc/ch_conceptual_design/hofs/transforms.rst @@ -53,7 +53,7 @@ To illustrate the different ways a transform's algorithm can be registered with // Return type: first number = number of good hits // second number = number of all hits -**Transform with one argument (default output product name)** +**Transform with one argument (default output-product suffix)** .. code:: c++ @@ -63,7 +63,7 @@ To illustrate the different ways a transform's algorithm can be registered with .input_family(product_selector{.suffix = "Waveforms", .layer = "APA"}); } -**Transform with one argument (user-specified output product name)** +**Transform with one argument (user-specified output-product suffix)** *As shown in* :numref:`workflow` *and described in* :numref:`ch_conceptual_design/registration:Framework Registration` @@ -76,7 +76,7 @@ To illustrate the different ways a transform's algorithm can be registered with .output_product_suffixes("GoodHits"); } -**Transform with two arguments (default output product name)** +**Transform with two arguments (default output-product suffix)** *As shown in* :numref:`workflow` *and described in* :numref:`ch_conceptual_design/registration:Data Products from Different Data Layers` @@ -88,11 +88,10 @@ To illustrate the different ways a transform's algorithm can be registered with .input_family( product_selector{.suffix = "Geometry", .layer = "Job"}, product_selector{.suffix = "GoodTracks", .layer = "APA"} - ) - .output_product_suffixes("Vertices"); + ); } -**Transform creating two data products (user-specified output product names)** +**Transform creating two data products (user-specified output-product suffixes)** .. code:: c++ @@ -100,5 +99,7 @@ To illustrate the different ways a transform's algorithm can be registered with { m.transform("hit_counter", count_good_hits, concurrency::unlimited) .input_family(product_selector{.suffix = "GoodHits", .layer = "APA"}) - .output_product_suffixes("NumGoodHits", "NumAllHits"); // <= One name per tuple slot + .output_product_suffixes( + "NumGoodHits", "NumAllHits" // <= One suffix per tuple slot + ); } From 2ebac72dca9d8a8ae171c4c5fe7c5331a0c3ac55 Mon Sep 17 00:00:00 2001 From: Kyle Knoepfel Date: Fri, 17 Jul 2026 16:53:19 -0500 Subject: [PATCH 2/5] Describe logical vs. in-memory containment --- doc/ch_conceptual_design/data_organization.rst | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/doc/ch_conceptual_design/data_organization.rst b/doc/ch_conceptual_design/data_organization.rst index 1ce928e..45d4da2 100644 --- a/doc/ch_conceptual_design/data_organization.rst +++ b/doc/ch_conceptual_design/data_organization.rst @@ -30,7 +30,7 @@ The `Job` layer always includes a single data cell. Rectangles with labels :math:`\textsf{Run}_i`, :math:`\textsf{Spill}_{i,j}`, and :math:`\textsf{APA}_{i,j,k}` represent data cells. The pale green rectangles show two data-cell families; these are identified as families because they are the result of executing the :mathfunc:`unfold(into_apas)` node shown in :numref:`workflow`. A solid line from one data cell to another data cell represents a logical association between the two data cells. - The bottom rectangle shows that :math:`\textsf{Waveforms}_{1,1,1}` is in the data cell :math:`\textsf{APA}_{1,1,1}`, etc. + Each dashed line between a data product and a data cell (e.g., between :math:`\textsf{Waveforms}_{1,1,1}` and :math:`\textsf{APA}_{1,1,1}`) indicates that the data product is logically contained by the data cell (see text). Each pale purple rectangle indicates the data-product family created by unfolding each :product:`SimDepos` object as shown in :numref:`workflow`. :name: data-organization @@ -70,6 +70,16 @@ As illustrated in :numref:`data-organization`, data products are organized into They can be unfolded into finer-grained units, enabling detailed analysis or reprocessing at different scales :need:`DUNE 43`. This provides the ability to process data too large to fit into memory at one time :need:`DUNE 25`. +.. admonition:: Logical vs. in-memory containment + + From a mathematical perspective, data cells *logically* contain data products. + It is from this mathematical viewpoint that a data product equivalently *belongs to*, *is associated with*, *is contained by*, *is in*, or *is a member of* one or more data cells. + This does not mean, however, that the framework must represent a given data cell as an object that keeps all of its logically-contained data products in memory at one time. + The framework program owns the in-memory instances of data products rather than data-cell objects or user-defined algorithms. + The framework program, therefore, controls the in-memory lifetimes of data-product instances and may retain in memory only those instances necessary for performing a particular task (see :numref:`ch_conceptual_design/data_organization:Data Product Management`). + This means that a data product may logically belong to a data cell even if it does not reside in memory. + + Data Product Management ----------------------- From 97ff6668450b3e8855ea6eb45e2316549b1fd7ae Mon Sep 17 00:00:00 2001 From: Kyle Knoepfel Date: Wed, 29 Jul 2026 11:49:50 -0500 Subject: [PATCH 3/5] Fix fold operator for arithmetic mean --- doc/ch_preliminaries/functional_programming.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/doc/ch_preliminaries/functional_programming.rst b/doc/ch_preliminaries/functional_programming.rst index 5d95a71..dfd408a 100644 --- a/doc/ch_preliminaries/functional_programming.rst +++ b/doc/ch_preliminaries/functional_programming.rst @@ -86,10 +86,10 @@ Such a HOF is called a map or *transform*: In such a scenario, the average :math:`\overline{c}` could be expressed as: .. math:: - \overline{c} = \frac{1}{n}\ \fold{+}{0}\ \transform{w}\ \fami{e} = \frac{1}{n}\ \fold{+ \comp w}{0}\ \fami{e} + \overline{c} = \frac{1}{n}\ \fold{+}{0}\ \transform{w}\ \fami{e} = \frac{1}{n}\ \fold{+ \comp (w \times w)}{0}\ \fami{e} The second equality holds by the fold-map fusion law [Bird]_, which states that the application of a :math:`\text{transform}` followed by a :math:`\text{fold}` can be reduced to a single :math:`\text{fold}`. -The operator to this single fold is ':math:`+ \comp w`', indicating that the function :math:`w` should be applied first before invoking the :math:`+` operation. +The operator to this single fold is ':math:`+ \comp (w \times w)`', indicating that the function :math:`w` is applied to both input arguments of the :math:`+` operation. Relying on such mathematical laws permits the replacement of chained calculations with a single calculation, often leading to efficiency improvements without affecting the result. .. _hofs_in_phlex: From cd1ef313d9ee8a7a4c7724d881064d936e01ab65 Mon Sep 17 00:00:00 2001 From: Kyle Knoepfel Date: Tue, 18 Aug 2026 10:55:43 -0500 Subject: [PATCH 4/5] Distinguish between operators and algorithms --- algorithm_description.rst | 2 +- doc/appendices/requirements.rst | 2 +- doc/ch_conceptual_design/algorithms.rst | 33 ++++++------ doc/ch_conceptual_design/registration.rst | 51 ++++++++++--------- doc/ch_conceptual_design/resources.rst | 2 +- .../functional_programming.rst | 2 +- doc/conceptual_design.rst | 2 +- doc/introduction.rst | 3 +- 8 files changed, 51 insertions(+), 46 deletions(-) diff --git a/algorithm_description.rst b/algorithm_description.rst index 3f674f8..b097071 100644 --- a/algorithm_description.rst +++ b/algorithm_description.rst @@ -37,7 +37,7 @@ The subsystem defines a canonical descriptor object for each registered algorith - Detailed description (what it does, capabilities, use cases) - Author/maintainer information (optional) -**Signature** (derived from :numref:`ch_conceptual_design/algorithms:Algorithms`) +**Signature** (derived from :numref:`ch_conceptual_design/algorithms:Operators and Algorithms`) - Ordered list of data-product parameters, each with: - Parameter kind (``P const&``, ``P const*``, ``P``, ``phlex::handle

``) - Concrete type name (fully qualified) diff --git a/doc/appendices/requirements.rst b/doc/appendices/requirements.rst index ff69d51..1808640 100644 --- a/doc/appendices/requirements.rst +++ b/doc/appendices/requirements.rst @@ -227,7 +227,7 @@ Conceptual Requirements The framework shall support the registration of algorithms that are independent of framework interface. -- See :numref:`introduction:Usability`, :numref:`introduction:Framework Independence`, :numref:`ch_conceptual_design/data_organization:data layers, data cells, and families`, :numref:`ch_conceptual_design/algorithms:Algorithms` +- See :numref:`introduction:Usability`, :numref:`introduction:Framework Independence`, :numref:`ch_conceptual_design/data_organization:data layers, data cells, and families`, :numref:`ch_conceptual_design/algorithms:Operators and Algorithms` .. req:: Safely executing thread-safe and non-thread-safe algorithms :collapse: diff --git a/doc/ch_conceptual_design/algorithms.rst b/doc/ch_conceptual_design/algorithms.rst index 29b7963..4da7d55 100644 --- a/doc/ch_conceptual_design/algorithms.rst +++ b/doc/ch_conceptual_design/algorithms.rst @@ -1,8 +1,11 @@ -Algorithms -========== +Operators and Algorithms +======================== -As mentioned in :numref:`ch_preliminaries/functional_programming:Families of Data and Higher-Order Functions`, an algorithm is registered with the framework as an operator to a higher-order function (HOF). -In general, Phlex supports the registration of C++ algorithms with function signatures like (see :numref:`ch_conceptual_design/supported_hofs:Supported Higher-Order Functions` for a list of supported HOFs): +As mentioned in :numref:`ch_preliminaries/functional_programming:Families of Data and Higher-Order Functions`, a user-defined algorithm is bound to the framework through an operator to a higher-order function (HOF). +It is the operator registered with the framework that is executed as part of the data-flow graph. +As will be illustrated in :numref:`ch_conceptual_design/registration:Framework Registration`, a framework-agnostic algorithm can often serve directly as the HOF operator, without any framework-specific wrapper code. + +In general, Phlex supports the registration of C++ operators with function signatures like (see :numref:`ch_conceptual_design/supported_hofs:Supported Higher-Order Functions` for a list of supported HOFs): .. code:: c++ @@ -10,9 +13,9 @@ In general, Phlex supports the registration of C++ algorithms with function sign where the types :cpp:`P1, Pn...` denote types of data products and the types :cpp:`Rm...` indicate :term:`resources `. The bracketed :cpp:`[quals]` term indicates that Phlex allows for class member functions that have trailing qualifiers (e.g. :cpp:`const`). -Each registered function must accept at least one data product. +Each registered operator must accept at least one data product. -The signature of a Python algorithm needs to be available through reflection, either because the function is JITed (e.g. with `Numba`), bound (e.g. with `ctypes`), or annotated. +The signature of a Python operator needs to be available through reflection, either because the function is JITed (e.g. with `Numba`), bound (e.g. with `ctypes`), or annotated. The latter is good practice regardless and commonly required by Python coding conventions: .. rstcheck: ignore-next-code-block @@ -25,7 +28,7 @@ We will first discuss the data-product and resource types in :numref:`ch_concept Input Parameters ---------------- -A data product of type :cpp:`P` may be presented to a C++ algorithm if the corresponding input parameter (i.e. the relevant :cpp:`P1, ..., PN` type) is one of the following: +A data product of type :cpp:`P` may be presented to a C++ operator if the corresponding input parameter (i.e. the relevant :cpp:`P1, ..., PN` type) is one of the following: - :cpp:`P const&` — read-only access to a data product provided through a reference - :cpp:`P const*` — read-only access to a data product provided through a pointer @@ -33,7 +36,7 @@ A data product of type :cpp:`P` may be presented to a C++ algorithm if the corre - :cpp:`phlex::handle

` — a lightweight object that provides read-only access to a data product as well as any metadata associated with it For each of these cases, the data product itself remains immutable. -A Python algorithm can receive a `phlex::handle` or a direct reference to the data product. +A Python operator can receive a `phlex::handle` or a direct reference to the data product. There is no equivalent language support for read-only access, but it will be enforced where possible. Whereas data products may be copied, resources of type :cpp:`R` may not. @@ -49,16 +52,16 @@ Resources are described in more detail in :numref:`ch_conceptual_design/resource Return Types ------------ -The meaning of an algorithm's return type depends on the HOF and is discussed in the :numref:`ch_conceptual_design/supported_hofs:Supported Higher-Order Functions`. +The meaning of an operator's return type depends on the HOF and is discussed in :numref:`ch_conceptual_design/supported_hofs:Supported Higher-Order Functions`. However, to simplify the discussion we introduce to concept of the *created data-product type*. -For Phlex to appropriately schedule the execution of algorithms and manage the lifetimes of data products, the framework itself must retain ownership of the data products. -This means that the data products created by algorithms must have types that connote unique ownership. -An algorithm's returned object must therefore model a created data-product type, which can be: +For Phlex to appropriately schedule the execution of operators and manage the lifetimes of data products, the framework itself must retain ownership of the data products. +This means that the data products created by operators must have types that connote unique ownership. +An operator's returned object must therefore model a created data-product type, which can be: - a *value* of type :cpp:`T`, or - a :cpp:`std::unique_ptr`, where the created object is non-null. -For Python, this means that an algorithm should not retain any external hard references to a returned object. +For Python, this means that an operator should not retain any external hard references to a returned object. The following types (or their equivalents) are forbidden as created data-product types because they do not imply unambiguous ownership: @@ -68,8 +71,8 @@ The following types (or their equivalents) are forbidden as created data-product Function Names and Qualifiers ----------------------------- -The :cpp:`function_name` in :numref:`ch_conceptual_design/algorithms:Algorithms` above may be any function name supported by the C++ language. -Code authors should aim to implement algorithms as free functions. +The :cpp:`function_name` in :numref:`ch_conceptual_design/algorithms:Operators and Algorithms` above may be any function name supported by the C++ language. +Code authors should aim to implement operators (or, equivalently, algorithms) as free functions. However, in some cases it may be necessary for class member functions to be used instead. When member functions are required, the qualifier :cpp:`const` should be specified to indicate that the class instance remains immutable during the execution of the member function [#f2]_. diff --git a/doc/ch_conceptual_design/registration.rst b/doc/ch_conceptual_design/registration.rst index a312b35..2ee7e78 100644 --- a/doc/ch_conceptual_design/registration.rst +++ b/doc/ch_conceptual_design/registration.rst @@ -25,7 +25,7 @@ This can be achieved by in terms of the C++ *registration stanza*: { m.transform( // 1. Higher-order function "hit_finder", // 2. Name assigned to HOF - find_hits, // 3. Algorithm/HOF operation + find_hits, // 3. HOF operator (here, the algorithm itself) concurrency::unlimited // 4. Allowed CPU concurrency ) .input_family( // 5. Specification of input data-product family (see text) @@ -38,26 +38,25 @@ This can be achieved by in terms of the C++ *registration stanza*: The registration stanza is included in a C++ file that is compiled into a :term:`module`, a compiled library that is dynamically loadable by Phlex. -A Python algorithm can be registered with its own companion C++ module or through the Python import helpers that make use of a pre-built, configurable, Phlex module. -For the sake of consistency and ease of understaning, the helpers have the same naming and follow the same conventions as the C++ registration. +A Python operator can be registered with its own companion C++ module or through the Python import helpers that make use of a pre-built, configurable, Phlex module. +For the sake of consistency and ease of understanding, the helpers have the same naming and follow the same conventions as the C++ registration. The stanza is introduced by an *opener*—e.g. :cpp:`PHLEX_REGISTER_ALGORITHMS()`—followed by a *registration block*, a block of code between two curly braces that contains one or more *registration statements*. A registration statement is a programming statement intended to model the equation described in :numref:`ch_conceptual_design/supported_hofs:Supported Higher-Order Functions` [#statement_ordering]_: -Loading the library created from the :term:`module` causes the creation of one workflow node for each registration statement in the stanza. .. math:: \ifamily{b}{\text{output}} = \text{HOF}(f_1,\ f_2,\ \dots)\ \ifamily{a}{\text{input}} -Specifically, in the registration stanza above, we have the following: +Specifically, in the registration statement above, we have the following: :cpp:`transform(...)` Fully specifying the mathematical expression :math:`\text{HOF}(f_1,\ f_2,\ \dots)` requires several items: 1. The HOF to be used, 2. The name to assign to the configured HOF, - 3. The algorithm(s)/HOF operator(s) to be used (i.e. :math:`f_1,\ f_2,\ \dots`), and - 4. The maximum number of CPU threads the framework can use when invoking the algorithm :need:`DUNE 152`. + 3. The HOF operator(s) to be used (i.e. :math:`f_1,\ f_2,\ \dots`), and + 4. The maximum number of CPU threads the framework can use when invoking the operator :need:`DUNE 152`. Replace with: The maximum number of CPU threads from which the framework can simultaneously invoke the node. Note that this does not address :need:`DUNE 152`. We propose to get rid of :need:`DUNE 152`. @@ -69,14 +68,14 @@ Specifically, in the registration stanza above, we have the following: Because the product suffix is empty, the :cpp:`.suffix` field could have been omitted altogether. :cpp:`output_product_suffixes(...)` - 6. This is the specification of the output products :math:`\ifamily{b}{\text{output}}`, which is formed from specification(s) of the data product(s) created by the algorithm :need:`DUNE 156`. + 6. This is the specification of the output products :math:`\ifamily{b}{\text{output}}`, which is formed from specification(s) of the data product(s) created by the operator :need:`DUNE 156`. The arguments to :cpp:`output_product_suffixes(...)` is one suffix for each product created by the transform operator. In this case, because the operator is producing a single product of type :cpp:`hits`, the product suffix could have been an empty string. An equivalent to that would have been to omit the :cpp:`output_product_suffixes(...)` clause altogether, in which case the framework would have assigned a default suffix of an empty string to the output product. -The set of information required by the framework for registering an algorithm largely depends on the HOF being used (see the :numref:`ch_conceptual_design/supported_hofs:Supported Higher-Order Functions` for specific interface). -However, in general, the registration code will specify which data products are required/produced by the algorithm :need:`DUNE 111` and the hardware resources required by the algorithm :need:`DUNE 9`. -Note that the input and output data-product specifications are matched with the corresponding types of the registered algorithm's function signature. +The set of information required by the framework for registering an algorithm through an operator largely depends on the HOF being used (see the :numref:`ch_conceptual_design/supported_hofs:Supported Higher-Order Functions` for specific interface). +However, in general, the registration statement will specify which data products are required/produced by the algorithm :need:`DUNE 111` and the hardware resources required by the algorithm :need:`DUNE 9`. +Note that the input and output data-product specifications are matched with the corresponding types of the registered operator's function signature. In other words: - :cpp:`"Waveforms"` specifies a data product whose C++ type is that of the first (and, in this case, only) input parameter to :cpp:`find_hits` (i.e. :cpp:`waveforms`). @@ -85,17 +84,18 @@ In other words: When executed, the above code creates a :term:`configured higher-order function`, which serves as a node in the function-centric data-flow graph. The registration block may contain any code supported by C++. -The block, however, must contain a registration statement to execute an algorithm. +The block, however, must contain at least one registration statement to execute an algorithm. .. important:: A module must contain only one registration stanza. Note that multiple registration statements may be made in each stanza. + Loading the library created from the module causes the creation of one workflow node for each registration statement in the stanza. Algorithms with Multiple Input Data Products -------------------------------------------- -The registration example given above in :numref:`ch_conceptual_design/registration:Framework Registration` creates an output family by applying a one-parameter algorithm :cpp:`find_hits` to each element of the input family, as specified by :cpp:`input_family(product_selector{.suffix = "Waveforms", .layer = "APA"})`. +The registration example given above in :numref:`ch_conceptual_design/registration:Framework Registration` creates an output family by applying a one-parameter operator (specifically, the algorithm :cpp:`find_hits`) to each element of the input family, as specified by :cpp:`input_family(product_selector{.suffix = "Waveforms", .layer = "APA"})`. In many cases, however, the algorithm will require more than one data product. Consider another algorithm :cpp:`find_hits_subtract_pedestals`, which forms hits by first subtracting pedestal values from the waveforms, both of which are presented to the algorithm as data products from the `APA`. The interface of the algorithm and its registration would look like: @@ -117,15 +117,16 @@ The interface of the algorithm and its registration would look like: .output_product_suffixes("GoodHits"); } -The elements of the input family are thus pairs of the data products labeled :cpp:`"Waveforms"` and :cpp:`"Pedestals"` in each APA. [#zip]_ +As in the earlier example shown above, the HOF operator is the algorithm :cpp:`find_hits_subtract_pedestals` itself. +The elements of the input family are pairs of the data products labeled :cpp:`"Waveforms"` and :cpp:`"Pedestals"` in each APA. [#zip]_ In this case, the data cell for both data products is the same—i.e. for a given invocation of :cpp:`find_hits_subtract_pedestals`, both data products will be associated with the same APA. There are cases, however, where an algorithm needs to operate on data products from *different* data cells :need:`DUNE 89`. .. note:: - The number of arguments presented to the :cpp:`family(...)` clause must match the number of input parameters to the registered algorithm. - The order of the :cpp:`family(...)` arguments also corresponds to the order of the algorithm's input parameters. + The number of arguments presented to the :cpp:`family(...)` clause must match the number of input parameters to the registered operator. + The order of the :cpp:`family(...)` arguments also corresponds to the order of the operator's input parameters. Data Products from Different Data Layers ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -156,7 +157,7 @@ So long as a relation can be defined between specific `Spill` data cells and spe How the relation between data cells is defined is referred to as *data marshaling*, and it is described further in :numref:`ch_technical_design/task_management:Data-Marshaling`. Data Products from Adjacent Data Cells -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ In some cases, it may be necessary to simultaneously access data products from adjacent data-products sets :need:`DUNE 91`, where *adjacency* is defined by the user :need:`DUNE 92`. The notion of adjacency can be critical for (e.g.) time-windowed processing (see :numref:`ch_conceptual_design/hofs/windows:Windows`), where the details of the "next" time bin are needed to accurately calculate properties of the "current" time bin. @@ -182,20 +183,20 @@ To do this, an additional argument (e.g. :cpp:`config`) is passed to the registr .. note:: - As discussed in :numref:`ch_technical_design/configuration:Programmatic access to the configuration`, the registration code will have access only to the configuration relevant to the algorithm being registered, and to certain framework-level configuration such as debug level, verbosity, or parallelization options. + As discussed in :numref:`ch_technical_design/configuration:Programmatic access to the configuration`, the registration stanza will have access only to the configuration relevant to the module being loaded, and to certain framework-level configuration such as debug level, verbosity, or parallelization options. -Except for the specification of :cpp:`find_hits` as the algorithm to be invoked, and :cpp:`transform` as the HOF, all other pieces of information may be provided through the configuration. +Except for the specification of :cpp:`transform` as the HOF and :cpp:`find_hits` as the HOF operator, all other pieces of information may be provided through the configuration. Framework Dependence in Registration Code ----------------------------------------- -Usually, classes like :cpp:`waveforms` and :cpp:`hits` and algorithms like :cpp:`find_hits` are framework-independent (see :numref:`introduction:Framework Independence`). +Classes like :cpp:`waveforms` and :cpp:`hits` and algorithms like :cpp:`find_hits` are usually framework-independent (see :numref:`introduction:Framework Independence`). There may be scenarios, however, where dependence on framework interface is required, especially if framework-specific metadata types are used by the algorithm. In such cases, it is strongly encouraged to keep framework dependence within the module itself and, more specifically, within the registration stanza. -This can be often achieved by registering closure objects that are generated by lambda expressions. +This can often be achieved by registering closure objects produced by lambda expressions as operators. For example, suppose a physicist would like to create an algorithm :cpp:`find_hits_debug` that reports a spill number when making tracks. -By specifying a lambda expression that takes a :cpp:`phlex::handle` object, the data product can be passed to the :cpp:`find_hits_debug` function, along with the spill number from the metadata accessed from the handle: +By specifying a lambda expression as the HOF operator that takes a :cpp:`phlex::handle` object, the data product can be passed to the :cpp:`find_hits_debug` algorithm, along with the spill number from the metadata accessed from the handle: .. code:: c++ @@ -212,7 +213,7 @@ By specifying a lambda expression that takes a :cpp:`phlex::handle` o .output_product_suffixes("GoodHits"); } -The lambda expression *does* depend on framework interface; the :cpp:`find_hits_debug` function, however, retains its framework independence. +The operator generated from the lambda expression *does* depend on framework interface; the :cpp:`find_hits_debug` algorithm, however, retains its framework independence. Member Functions of Classes --------------------------- @@ -247,7 +248,7 @@ The :cpp:`hit_finder::find` member function's address is registered in the :cpp: .. note:: - Algorithm authors should first attempt to implement algorithms as free functions (see :numref:`ch_preliminaries/functional_programming:Pure functions`). + Algorithm authors should first try to implement algorithms as free functions (see :numref:`ch_preliminaries/functional_programming:Pure functions`) and register those free functions directly as HOF operators. Registering class instances and their member functions with the framework should only be considered when: - multiple processing steps must work together, relying on shared internal data, or @@ -257,7 +258,7 @@ Overloaded Functions -------------------- Phlex performs a substantial amount of type deduction through the :cpp:`transform(...)` clause. -This works well except in cases where the registered algorithms are overloaded functions. +This works well except in cases where the registered operators are overloaded functions. For example, suppose one wants to register C++'s overloaded :cpp:`std::sqrt(...)` function with the framework. Simply specifying :cpp:`transform(..., std::sqrt)` will fail at compile time as the compiler will not be able to determine which overload is desired. diff --git a/doc/ch_conceptual_design/resources.rst b/doc/ch_conceptual_design/resources.rst index 5decffb..e390518 100644 --- a/doc/ch_conceptual_design/resources.rst +++ b/doc/ch_conceptual_design/resources.rst @@ -1,7 +1,7 @@ Resources ========= -As mentioned in :numref:`ch_conceptual_design/algorithms:Algorithms`, the typical form of a C++ function that can be registered with the framework is: +As mentioned in :numref:`ch_conceptual_design/algorithms:Operators and Algorithms`, the typical form of a C++ function that can be registered with the framework is: .. code:: c++ diff --git a/doc/ch_preliminaries/functional_programming.rst b/doc/ch_preliminaries/functional_programming.rst index dfd408a..2a2ad76 100644 --- a/doc/ch_preliminaries/functional_programming.rst +++ b/doc/ch_preliminaries/functional_programming.rst @@ -138,7 +138,7 @@ A calculation using a HOF is then generally expressed in terms of: .. index:: Algorithm Phlex supports the HOFs listed in :numref:`hofs_in_phlex`. -As discussed later, each HOF's *operator* is an :term:`algorithm` registered with the framework. +As discussed later, each HOF's *operator* binds a user-defined :term:`algorithm` to the framework. Phlex will likely support other higher order functions as well. .. rubric:: Footnotes diff --git a/doc/conceptual_design.rst b/doc/conceptual_design.rst index 43392d0..e348616 100644 --- a/doc/conceptual_design.rst +++ b/doc/conceptual_design.rst @@ -37,7 +37,7 @@ Various framework aspects are demonstrated by that figure: From a functional programming perspective these providers are impure functions that map a data cell index to one of the data products within the indicated data cell. *HOFs and user-provided algorithms* - Arguably the most important aspect of the framework is how user-provided algorithms are bound to HOFs and registered with the framework (see :numref:`ch_conceptual_design/algorithms:Algorithms`, :numref:`ch_conceptual_design/registration:Framework Registration` and :numref:`ch_conceptual_design/supported_hofs:Supported Higher-Order Functions`). + Arguably the most important aspect of the framework is how user-provided algorithms are bound to HOFs and registered with the framework (see :numref:`ch_conceptual_design/algorithms:Operators and Algorithms`, :numref:`ch_conceptual_design/registration:Framework Registration` and :numref:`ch_conceptual_design/supported_hofs:Supported Higher-Order Functions`). All seven HOFs supported by Phlex (see :numref:`hofs_in_phlex`) are used in :numref:`workflow`. For the main processing chain of creating vertices: diff --git a/doc/introduction.rst b/doc/introduction.rst index 14aa19e..4b17f1e 100644 --- a/doc/introduction.rst +++ b/doc/introduction.rst @@ -157,7 +157,8 @@ Consequently, part of the Phlex design is simply a summary of prior research tha Programming Languages ===================== -The framework will support user :term:`algorithms ` written in multiple programming languages :need:`DUNE 14`. Specifically, an algorithm may be written in either C++ [#f2]_ :need:`DUNE 81` or Python [#f3]_ :need:`DUNE 82`. +The framework will support user :term:`algorithms ` written in multiple programming languages :need:`DUNE 14`. +Specifically, an algorithm may be written in either C++ [#f2]_ :need:`DUNE 81` or Python [#f3]_ :need:`DUNE 82`. If there is a need to support user algorithms written in another programming language, a corresponding stakeholder requirement should be created. Note that the language is left unspecified for the implementation of the framework itself. From eb1a0be03ebbd6aaea401c335cd9e340402e25b6 Mon Sep 17 00:00:00 2001 From: Kyle Knoepfel Date: Fri, 21 Aug 2026 10:51:44 -0500 Subject: [PATCH 5/5] Address @marcpaterno's comment --- doc/ch_conceptual_design/registration.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/ch_conceptual_design/registration.rst b/doc/ch_conceptual_design/registration.rst index 2ee7e78..7247605 100644 --- a/doc/ch_conceptual_design/registration.rst +++ b/doc/ch_conceptual_design/registration.rst @@ -25,7 +25,7 @@ This can be achieved by in terms of the C++ *registration stanza*: { m.transform( // 1. Higher-order function "hit_finder", // 2. Name assigned to HOF - find_hits, // 3. HOF operator (here, the algorithm itself) + find_hits, // 3. HOF operator (here, the user's algorithm itself) concurrency::unlimited // 4. Allowed CPU concurrency ) .input_family( // 5. Specification of input data-product family (see text)