impl/lola: Added shm-size calculation by analysis for DATA/CONTROL by crimson11 · Pull Request #760 · eclipse-score/communication

crimson11 · 2026-07-23T20:12:00Z

Currently we determine shm-sizes of the
CTRL and DATA shm-objects by a simulation run.
I.e. we initialize the content of both section
1st within a heap-allocated resource. At the end
we use the sizes to correctly size the shm-objects. This "SIMULATION" method is exact but has high runtime costs and might consume lots of memory during startup.

This change now redesigns the containers/dynamic data types being used within the DATA section, to use only classes, which we are in control of and where we exactly know based on our configuration, who much size they will need. Thus the whole simulation canbe skipped and we calculate the size correctly from the configuration settins for the service instance.

We re-introduce therefore the ESTIMATION mode in parallel to the SIMULATION mode.

Tackles issue #761

castler · 2026-07-24T07:20:32Z

 enum class ShmSizeCalculationMode : std::uint8_t
 {
    kSimulation,
+    kEstimation,


This is no longer an estimation - this is now an calculation! We should be clear in the wording.

bemerybmw · 2026-07-28T14:56:33Z

+        return key_equal_;
+    }
+
+    mapped_type& at(const Key& key)


Suggested change

mapped_type& at(const Key& key)

mapped_type& at(const Key& key)&

I think this should have the trailing & so that it can't be called on a temporary(would lead to a dangling reference).

Hm. I'm not convinced ;) Why?

we resemble a std::map interface here! map doesn't lvalue qualify at() neither. This confuses imho.

then I would need to be consistent! Also would have to add it o plenty of other methods! Like begin()/end() etc. they are all returning iterators/pointers which may be dangling in case of a temporary ...

There was some misra rule I believe about this. But I'm fine if you follow the std::map interface and we can revisit this if the warning pops up

bemerybmw · 2026-07-29T07:22:44Z

+    {
+        const auto field_it = lola_service_instance_deployment_.fields_.find(name);
+        SCORE_LANGUAGE_FUTURECPP_ASSERT_PRD_MESSAGE(field_it != lola_service_instance_deployment_.fields_.cend(),
+                                                    "Could not find field in deployment configuration.");


The message should not just be repeating what the check is in words. It should either be explaining why the assertion is checked / expected or just omit it. It's not adding any new information here. Same for the other assertions

bemerybmw · 2026-07-29T07:29:56Z

+    return total_size;
+}
+
+std::size_t SkeletonMemoryManager::GetNumberOfSampleSlotsFromConfig(const std::string_view service_element_name,


What about something like this to avoid code duplication?

template <ServiceElementType ServiceElementType> std::size_t SkeletonMemoryManager::GetNumberOfSampleSlotsFromConfig(const std::string_view service_element_name) const { SCORE_LANGUAGE_FUTURECPP_PRECONDITION(ServiceElementType == ServiceElementType::EVENT || ServiceElementType == ServiceElementType::FIELD); const auto& lola_service_element_deployment = [this, service_element_name]() { if constexpr (ServiceElementType == ServiceElementType::EVENT) { return lola_service_instance_deployment_.events_.at(std::string{service_element_name}); } else { return lola_service_instance_deployment_.fields_.at(std::string{service_element_name}) .lola_event_instance_deployment_; } }(); const std::string name{service_element_name}; const auto field_it = lola_service_element_deployment.fields_.find(name); SCORE_LANGUAGE_FUTURECPP_ASSERT_PRD_MESSAGE(field_it != lola_service_instance_deployment_.fields_.cend(), "Could not find field in deployment configuration."); const auto number_of_slots = field_it->second.lola_event_instance_deployment_.GetNumberOfSampleSlots(); SCORE_LANGUAGE_FUTURECPP_ASSERT_PRD_MESSAGE(number_of_slots.has_value(), "Number of sample slots not specified for field."); return number_of_slots.value(); }

Same for GetMaxSubscribersFromConfig

Hm. I'm not convinced. Just compare the src-code sizes. It is not really much less source-code and the original was also "easy" to grasp imho. Now you are templating it and in reality we will have larger code-size, because we have now the impl. instantiated twice ... and as I said - the new impl. still has a cosnstexpr if-else, which makes the code not really simpler?

If you still insist -> I can change ;)

My point wasn't really about number of lines, it was more about avoiding code duplication. We're currently duplicating the "algorithm" to extract number of slots from a deployment. So if we for example changed the structure of the deployments, we'd have to change it (correctly) in 2 places instead of 1. But it's such a small function, I'm fine if you leave it as is.

will leave it.

Currently we determine shm-sizes of the CTRL and DATA shm-objects by a simulation run. I.e. we initialize the content of both section 1st within a heap-allocated resource. At the end we use the sizes to correctly size the shm-objects. This "SIMULATION" method is exact but has high runtime costs and might consume lots of memory during startup. This change now redesigns the containers/dynamic data types being used within the DATA section, to use only classes, which we are in control of and where we exactly know based on our configuration, who much size they will need. Thus the whole simulation canbe skipped and we calculate the size correctly from the configuration settins for the service instance. We re-introduce therefore the ESTIMATION mode in parallel to the SIMULATION mode.

Added analytical estimate/ANALYSIS mode for the shm-size calculation for the CONTROL section.

Adapted signature of LinearSearchMap to support custom KeqEqual just like std::unordered_map.

In the skeleton component tests we were testing the lola::Skeleton with a event and field service element. But we only provided the config/ deployment info for these elements, but did never register them at their parent Skeleton. Thus, essential tests checking the shm-size calculation were off! The tests now correctly register the elements.

Fixed review comments for ne ANALYSIS based size calculations. Added unit test for parsing of new ANALYSIS mode.

Moved the shm-size calc for ServiceDataStorage and ServiceDataControl out of SkeletonMemoryManager to the correspoinding data structures itself.

bemerybmw · 2026-08-06T10:39:00Z

+
+TEST_F(LinearSearchMapFixture, IsEmptyAfterConstruction)
+{
+    // Given a freshly constructed LinearSearchMap with a capacity of 4


Suggested change

// Given a freshly constructed LinearSearchMap with a capacity of 4

bemerybmw · 2026-08-06T10:39:18Z

+{
+    // Given a freshly constructed LinearSearchMap with a capacity of 4
+
+    // When constructing the map


When constructing a LinearSearchMap with a capacity of 4

Same for the test below

bemerybmw · 2026-08-06T10:41:44Z

+    TestMap unit{4U, memory_};
+
+    // When emplacing a new key/value pair
+    const auto result = unit.emplace(1, 100);


We should minimise the use of magic numbers which are reused. The key and value should be variables which are reused in the Then section.

bemerybmw · 2026-08-06T10:44:31Z

+    // When searching via the const overload of find
+    const auto it = const_unit.find(5);
+
+    // Then an existing key is found and a missing key yields the const end iterator


Suggested change

// Then an existing key is found and a missing key yields the const end iterator

// Then an existing key is found

bemerybmw · 2026-08-06T10:51:25Z

+    // Then the returned predicate exhibits the custom equality semantics
+    EXPECT_TRUE(predicate(-7, 7));
+    EXPECT_FALSE(predicate(-7, 8));
+}


What do we want to happen when we try to emplace more elements than we reserved capacity for? I guess that it would have to terminate because otherwise emplace would return false but what iterator to return? I think this should be specified in the header docs and also probably should have a test here. I know the behaviour is implemented in NonRelocatableVector, but IMO, this is an important variant of this class which should be documented and tested.

Added test and doc.

bemerybmw · 2026-08-07T09:23:52Z

+    return total_size;
+}
+
+std::size_t SkeletonMemoryManager::GetNumberOfSampleSlotsFromConfig(const std::string_view service_element_name,


My point wasn't really about number of lines, it was more about avoiding code duplication. We're currently duplicating the "algorithm" to extract number of slots from a deployment. So if we for example changed the structure of the deployments, we'd have to change it (correctly) in 2 places instead of 1. But it's such a small function, I'm fine if you leave it as is.

bemerybmw · 2026-08-07T09:25:07Z

+        return key_equal_;
+    }
+
+    mapped_type& at(const Key& key)


There was some misra rule I believe about this. But I'm fine if you follow the std::map interface and we can revisit this if the warning pops up

bemerybmw · 2026-08-07T09:29:43Z

+        number_of_service_elements * sizeof(ServiceDataStorage::EventMetaInfoMap::value_type), kMaxAlign);
+
+    // The size of the EventDataStorage control structure (a DynamicArray) is independent of the concrete sample-type
+    // (it only holds an offset-pointer, an allocator and two size_t members).


We're making a lot of assumptions here about the internals of a DynamicArray. Would it make sense that this is added to the DynamicArray itself?

Is this our discussion regarding splitting up/making a recursive approach for CalculateServiceDataStorageShmSize ? I.e. also implement a corresponding calculate-method in DynamicArray ?
This we could do, but I would like to postpone it after we have switched to type-erased storage for the binding ... as it will change a lot again.

bemerybmw · 2026-08-07T09:34:37Z

+    const std::string name{service_element_name};
+    if (is_field)
+    {
+        const auto field_it = lola_service_instance_deployment_.fields_.find(name);


You can actually use GetServiceElementInstanceDeployment from score/mw/com/impl/configuration/lola_service_instance_deployment.h here.

bemerybmw · 2026-08-07T09:34:53Z

+                                                    "Number of sample slots not specified for field.");
+        return number_of_slots.value();
+    }
+    const auto event_it = lola_service_instance_deployment_.events_.find(name);


You can actually use GetServiceElementInstanceDeployment from score/mw/com/impl/configuration/lola_service_instance_deployment.h here.

bemerybmw · 2026-08-07T09:36:29Z

+///        service-elements (events + fields), which is the fixed capacity the event_controls_ container is constructed
+///        with.
+/// \return needed size (in bytes) for a single control shm-object.
+std::size_t CalculateServiceDataControlShmSize(


We're missing tests for this?

Added tests

bemerybmw · 2026-08-07T09:36:36Z

+///        service-elements (events + fields), which is the fixed capacity the ServiceDataStorage containers are
+///        constructed with.
+/// \return needed size (in bytes) for the data shm-object.
+std::size_t CalculateServiceDataStorageShmSize(


We're missing tests for this?

Added tests

Added death-test for LinearSearchMap for capacity overflow. SkeletonMemoryManager now uses GetServiceElementInstanceDeployment helper.

        {
            return ShmSizeCalculationMode::kSimulation;
        }
+        else if (shm_size_calc_mode_value == kShmSizeCalcModeAnalysis)


Added tests for ServiceDataControl for size calculation.

Added tests for ServiceDataStorage for size calculation.

crimson11 requested review from LittleHuba, bemerybmw, castler, hoe-jo and limdor as code owners July 23, 2026 20:12

github-project-automation Bot added this to COM - Communication FT Jul 23, 2026

github-project-automation Bot moved this to Backlog in COM - Communication FT Jul 23, 2026

crimson11 force-pushed the mf_new_shm_size_calculation branch from c72a678 to a090932 Compare July 23, 2026 21:01

crimson11 changed the title ~~impl/lola: Added shm-size calc for CTRL~~ impl/lola: Added shm-size calc by estimation for DATA/CONTROL Jul 23, 2026

crimson11 force-pushed the mf_new_shm_size_calculation branch from a090932 to 59cd98d Compare July 23, 2026 21:24

castler reviewed Jul 24, 2026

View reviewed changes

crimson11 force-pushed the mf_new_shm_size_calculation branch from 59cd98d to e088dba Compare July 24, 2026 09:46

crimson11 requested a review from castler July 24, 2026 09:57

castler changed the title ~~impl/lola: Added shm-size calc by estimation for DATA/CONTROL~~ impl/lola: Added shm-size calculation for DATA/CONTROL Jul 27, 2026

castler changed the title ~~impl/lola: Added shm-size calculation for DATA/CONTROL~~ impl/lola: Added shm-size calculation by analysis for DATA/CONTROL Jul 27, 2026

anmittag assigned crimson11 Jul 29, 2026

bemerybmw requested changes Jul 29, 2026

View reviewed changes

bemerybmw self-assigned this Jul 29, 2026

crimson11 force-pushed the mf_new_shm_size_calculation branch 5 times, most recently from f8aab17 to d04e2bf Compare August 1, 2026 16:18

crimson11 added 5 commits August 4, 2026 13:33

impl/lola: Added shm-size calc for CTRL

23688e2

Added analytical estimate/ANALYSIS mode for the shm-size calculation for the CONTROL section.

impl/lola: Refine LinearSearchMap

1ca1635

Adapted signature of LinearSearchMap to support custom KeqEqual just like std::unordered_map.

impl/lola: Fixed review comments

7c24d15

Fixed review comments for ne ANALYSIS based size calculations. Added unit test for parsing of new ANALYSIS mode.

crimson11 force-pushed the mf_new_shm_size_calculation branch from d04e2bf to 93af38c Compare August 4, 2026 11:35

github-advanced-security AI found potential problems Aug 4, 2026

View reviewed changes

com/impl/lola: Refactor shm-size-calc

798fdc0

Moved the shm-size calc for ServiceDataStorage and ServiceDataControl out of SkeletonMemoryManager to the correspoinding data structures itself.

crimson11 force-pushed the mf_new_shm_size_calculation branch from 93af38c to 798fdc0 Compare August 4, 2026 12:50

bemerybmw reviewed Aug 7, 2026

View reviewed changes

com/impl/lola: extended tests for LinearSearchMap

e401813

Added death-test for LinearSearchMap for capacity overflow. SkeletonMemoryManager now uses GetServiceElementInstanceDeployment helper.

github-advanced-security AI found potential problems Aug 7, 2026

View reviewed changes

Comment thread score/mw/com/impl/configuration/configuration_json_parsing_strategy.cpp

{

return ShmSizeCalculationMode::kSimulation;

}

else if (shm_size_calc_mode_value == kShmSizeCalcModeAnalysis)

crimson11 added 2 commits August 7, 2026 15:28

mw/com/impl/lola: Added ServiceDataControl test

fc9ba8a

Added tests for ServiceDataControl for size calculation.

mw/com/impl/lola: Added ServiceDataStorage tests

e3af40f

Added tests for ServiceDataStorage for size calculation.

crimson11 requested a review from bemerybmw August 7, 2026 14:06

	mapped_type& at(const Key& key)
	mapped_type& at(const Key& key)&

	// Then an existing key is found and a missing key yields the const end iterator
	// Then an existing key is found

Uh oh!

Conversation

crimson11 commented Jul 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

crimson11 commented Jul 23, 2026 •

edited

Loading