@Aquaticfuller @DiyouS as discussed this morning:
Here I'm not sure how much of this can already be achieved in the current system, so let me just summarize the idea:
Summary
For a kernel like the RLC kernel, a lot of data can be tile-local. For example, we need an RLC entity descriptor with some fields (the list of PDCP packets received, the list of PDUs sent but not yet acknowledged, etc. We can assign a "home tile" for each RLC entity which means that unless that tile is overloaded, we prefer to keep the data there and do the processing in the same tile. Similarly, the linked list for an UE would allocate from such a tile-local pool. In the end, all of this is still a cache, the real memory in DRAM. But this way, we can select where the cached data will end up.
Or to frame it the other way round: we spread UEs across all tiles, but the assignment is deterministic. Then when we need to select a core to process an UE on, we use a core in the tile where the data is located.
However, at the same point we will have address regions where data is not tile-local (e.g. the payload region) and data should just be spread over all tiles.
Idea
N configurable windows having:
- An aligned base address
- A total size / bit width
- A slice width.
Usage example:
For example, we could configure one region:
- Start 0x82000000
- Stride size: 0x80000
- total size: 32MB window.
With this, range 0x82000000 – 0x8207FFFF will map to tile 0, range 0x82080000 – 0x820FFFFF to tile 1, etc up to tile 64. Then a core can trivially do local allocation, by just using memory from a pool in that range. (And SW can just put static data there as well).
At the same time, figuring out what tile a address ends up on is trivial in the SW.
Within the per-tile window and tile, everything is still line-interleaved across the banks in the tile. Also the non-configured range would have default behavior, likely spread across all tiles.
Suggested implementation
It is important that none of the data here is private: All data can be accessed from anywhere. Also, there's one shared address space, no per-core address space. It's just a flexible way to decide on which tile an address is cached. This enables flexible "work stealing": If the cores in a home tile are overloaded, the UE can still be processed in another tile. All cores have access to all data, it will just have higher noc latency and cause more data transfer / bandwidth restrictions. So unlike a SPM system, work stealing is very simple to do.
I think maybe most of this is already there? However, I think to make full use of it, we need to be able to configure at least 2 ranges (the "private"/locally allcoated range and everything else).
What do you think?
@Aquaticfuller @DiyouS as discussed this morning:
Here I'm not sure how much of this can already be achieved in the current system, so let me just summarize the idea:
Summary
For a kernel like the RLC kernel, a lot of data can be tile-local. For example, we need an RLC entity descriptor with some fields (the list of PDCP packets received, the list of PDUs sent but not yet acknowledged, etc. We can assign a "home tile" for each RLC entity which means that unless that tile is overloaded, we prefer to keep the data there and do the processing in the same tile. Similarly, the linked list for an UE would allocate from such a tile-local pool. In the end, all of this is still a cache, the real memory in DRAM. But this way, we can select where the cached data will end up.
Or to frame it the other way round: we spread UEs across all tiles, but the assignment is deterministic. Then when we need to select a core to process an UE on, we use a core in the tile where the data is located.
However, at the same point we will have address regions where data is not tile-local (e.g. the payload region) and data should just be spread over all tiles.
Idea
N configurable windows having:
Usage example:
For example, we could configure one region:
With this, range 0x82000000 – 0x8207FFFF will map to tile 0, range 0x82080000 – 0x820FFFFF to tile 1, etc up to tile 64. Then a core can trivially do local allocation, by just using memory from a pool in that range. (And SW can just put static data there as well).
At the same time, figuring out what tile a address ends up on is trivial in the SW.
Within the per-tile window and tile, everything is still line-interleaved across the banks in the tile. Also the non-configured range would have default behavior, likely spread across all tiles.
Suggested implementation
It is important that none of the data here is private: All data can be accessed from anywhere. Also, there's one shared address space, no per-core address space. It's just a flexible way to decide on which tile an address is cached. This enables flexible "work stealing": If the cores in a home tile are overloaded, the UE can still be processed in another tile. All cores have access to all data, it will just have higher noc latency and cause more data transfer / bandwidth restrictions. So unlike a SPM system, work stealing is very simple to do.
I think maybe most of this is already there? However, I think to make full use of it, we need to be able to configure at least 2 ranges (the "private"/locally allcoated range and everything else).
What do you think?