feat: Implement ADRP+ADD to NOP+ADR relaxation for AArch64 - #2319
feat: Implement ADRP+ADD to NOP+ADR relaxation for AArch64#2319deepakshirkem wants to merge 1 commit into
Conversation
ec27067 to
692cef7
Compare
davidlattimore
left a comment
There was a problem hiding this comment.
I haven't looked in much detail as yet, but is there any reason why this can't be done similar to how the other aarch64 relaxations are implemented?
|
The main challenge is that ADRP+ADD relaxation requires inspecting the NEXT relocation (ADD) to verify same symbol and register, which isn't available in new_relaxation. The existing AArch64 relaxations are single-instruction transformations. If you have a preferred approach for handling two-instruction relaxations in Wild's existing infrastructure. |
|
I think we'll need to adjust the API so that the code that decides relaxations can access the next relocation. Either that or add a new API for handling relaxations - e.g. one that always receives pairs of adjacent relocations. I'm not sure which would be best. But either way, aarch64 relaxation code definitely needs to be in the elf_aarch64 module. Where it is now, it gets run for all architectures and the relocation codes it's looking for are likely to be used for something completely different on those other architectures. |
68b9d88 to
ae65e29
Compare
| // LLD does some different relaxations to us | ||
| "rel.missing-opt.R_AARCH64_ADR_GOT_PAGE.ReplaceWithNop.*", | ||
| "rel.missing-opt.R_AARCH64_ADR_PREL_PG_HI21.ReplaceWithNop.*", | ||
| "rel.extra-opt.R_AARCH64_ADR_PREL_PG_HI21.ReplaceWithNop.*", |
There was a problem hiding this comment.
@davidlattimore Is this line correct? The CI is now green with this line. My understanding is that we're comparing the output against GNU ld, but Wild has an optimization that GNU ld doesn't. This line seems to skip that comparison.
There was a problem hiding this comment.
I think that's reasonable to add that global ignore. I guess ideally we'd have ignore rules that changed depending on which linker we were comparing against, but currently we don't. The placement of the ignore rule is possibly a bit misleading. It's under a comment that refers to "LLD", but from what I understand, LLD does this optimisation too and it's GNU ld that doesn't.
| //#ReferenceLinkers:lld | ||
| //#LinkArgs:--no-gc-sections -Ttext=0x200ffc | ||
| //#RunEnabled:false | ||
| //#DiffIgnore:file-header.entry |
There was a problem hiding this comment.
Do you think we can improve this test as well?
There was a problem hiding this comment.
Do you mean with regard to the ignore of file-header.entry? It looks like that's effectively a bug in linker-diff where it's not doing a good job of handling the fact that the entry point has multiple symbols that resolve to that address.
Usually I try to not disable running of the executable, but it's possible that it's not worth trying to run the executable in this case.
|
|
||
| fn apply_pair_relaxation( | ||
| first_kind: object::elf::RelocationType, | ||
| _second_kind: object::elf::RelocationType, |
There was a problem hiding this comment.
Why aren't you checking the type of the second relocation? Doesn't that mean that if it's a different kind of relocation, we might apply the relaxation when it shouldn't apply? Could be good to add a test for this.
| None | ||
| } | ||
|
|
||
| fn apply_pair_relaxation( |
There was a problem hiding this comment.
To be honest, don't like the new entry point - can't we rather base the relaxation on what was suggested by David - passing the next relocation to the A::next_relaxation based on the following patch:
diff --git a/libwild/src/elf_writer.rs b/libwild/src/elf_writer.rs
index 97e18ca6..b80893ac 100644
--- a/libwild/src/elf_writer.rs
+++ b/libwild/src/elf_writer.rs
@@ -2417,7 +2417,7 @@ fn apply_relocations<
object: &ObjectLayout<'data, elf::Elf<C>>,
out: &mut [u8],
section_index: object::SectionIndex,
- mut relocations: I,
+ relocations: I,
layout: &ElfLayout<'data, C>,
table_writer: &mut TableWriter<'_, '_, C>,
trace: &TraceOutput,
@@ -2434,6 +2434,7 @@ fn apply_relocations<
let relax_deltas = object.section_relax_deltas.get(section_index.0);
let mut relax_cursor = relax_deltas.map(|deltas| deltas.cursor());
+ let mut relocations = relocations.peekable();
while let Some(rel) = relocations.next() {
let rel = rel?;
relocation_count += 1;
@@ -2456,10 +2457,12 @@ fn apply_relocations<
None => rel.offset(),
};
+ let rel_next = relocations.peek().copied().transpose()?;
modifier = apply_relocation::<C, A, R, _>(
object,
offset_in_section,
&rel,
+ rel_next,
SectionInfo {
section_address,
is_writable: object_section.is_writable(),
@@ -2718,6 +2721,9 @@ fn write_eh_frame_relocations<
object,
rel_offset - input_pos as u64,
rel,
+ // Relaxations based on the ability to look-up the next relocation are not
+ // expected to happen for the section.
+ None,
SectionInfo {
section_address: output_pos as u64 + table_writer.eh_frame_start_address,
is_writable: false,
@@ -3040,6 +3046,7 @@ fn apply_relocation<
object_layout: &ObjectLayout<'data, elf::Elf<C>>,
mut offset_in_section: u64,
rel: &R,
+ _next_rel: Option<R>,
section_info: SectionInfo<linker_utils::elf::SectionFlags>,
layout: &ElfLayout<'data, C>,
out: &mut [u8],| // Get destination register from ADRP instruction | ||
| let adrp_instr = u32::from_le_bytes(out[offset..offset + 4].try_into().unwrap()); | ||
| let add_instr = u32::from_le_bytes(out[offset + 4..offset + 8].try_into().unwrap()); | ||
| let adrp_dest_reg = adrp_instr & 0x1f; |
There was a problem hiding this comment.
can we rather use already used pattern: extract_bit_range?
| let add_dest_reg = add_instr & 0x1f; | ||
| let add_src_reg = (add_instr >> 5) & 0x1f; | ||
| // Verify ADRP and ADD instructions use same register | ||
| if (adrp_instr & 0x9f000000) != 0x90000000 |
There was a problem hiding this comment.
I see the code just extracted some source and destination registers - why are we then comparing the masks out of adrp_instr and add_instr? Don't get it.
| return false; | ||
| } | ||
| // Write NOP at ADRP position | ||
| out[offset..offset + 4].copy_from_slice(&[0x1f, 0x20, 0x03, 0xd5]); |
There was a problem hiding this comment.
You should rather define a new RelaxationKind enum variant and make the transformation of all 8B in RelaxationKind::apply function.
| } | ||
| // Write NOP at ADRP position | ||
| out[offset..offset + 4].copy_from_slice(&[0x1f, 0x20, 0x03, 0xd5]); | ||
| // Write ADR base at ADD position |
There was a problem hiding this comment.
You might be able to reuse: RelaxationKind::AddToAdr (and so your RelaxationKind will be effectively a composition of 2 already supported relaxations)?
| @@ -0,0 +1,27 @@ | |||
| // This verifies that Wild relaxes ADRP+ADD to NOP+ADR when the symbol is | |||
| // within ADR range (±1MB) and registers match, but keeps ADRP+ADD when: | |||
| // - the destination registers differ (x2 vs x3 in third pair) | |||
There was a problem hiding this comment.
These 2 lines are duplicate - they are later documented again at the assembly level.
|
@marxin Thank you. I actually started with the peekable/next_rel approach but found it required changes to many architectures (updating all new_relaxation implementations). I then went with the apply_pair_relaxation approach to keep changes isolated. However looking at your patch, it's much cleaner passing next_rel through apply_relocation avoids the need for a new trait method entirely. I'll implement this approach later today and update the PR. |
49cdc54 to
bcd370b
Compare
|
@marxin @davidlattimore Please do one more cycle of review when you get a chance? |
marxin
left a comment
There was a problem hiding this comment.
We're getting closer I think, please try to address my comments.
ead5f74 to
a9a30d5
Compare
|
@marxin Ping :)). |
marxin
left a comment
There was a problem hiding this comment.
Thanks for the iteration on the patch - I am pretty happy about it now. But let's make the final decision to @davidlattimore.
One general recommendation - please try to instruct your LLM (I guess you're using Claude) to emit more information intensive comments. Or ideally, try to compress the comments.
| symbol_db.output_kind, | ||
| section_flags, | ||
| true, | ||
| 1, // sym_addr: assume non-zero (actual address not available in layout phase) |
There was a problem hiding this comment.
@davidlattimore - what about using an Option<(u64, u64), or a new type instead of the part of u64 types? In particular, don't like the 1 special value which mimics the non-zero value, but is pretty non-intuitive.
a9a30d5 to
ba4803b
Compare
davidlattimore
left a comment
There was a problem hiding this comment.
I see a 2.5% performance loss when linking wild for aarch64:
Benchmark 1 (3495 runs): /home/david/save/wild-aarch64/run-with /home/d/wild-builds/2026-08-09.cg1 --strip-debug --no-fork
measurement mean ± σ min … max outliers delta
wall_time 51.4ms ± 1.06ms 47.4ms … 59.1ms 31 ( 1%) 0%
peak_rss 138MB ± 832KB 135MB … 142MB 28 ( 1%) 0%
cpu_cycles 1.03G ± 29.6M 826M … 1.16G 84 ( 2%) 0%
instructions 1.23G ± 13.6M 1.16G … 1.31G 78 ( 2%) 0%
cache_references 25.5M ± 550K 23.5M … 27.6M 38 ( 1%) 0%
cache_misses 5.93M ± 144K 5.26M … 6.49M 15 ( 0%) 0%
branch_misses 2.26M ± 80.4K 2.03M … 2.87M 100 ( 3%) 0%
Benchmark 2 (3411 runs): /home/david/save/wild-aarch64/run-with target/cg1/wild --strip-debug --no-fork
measurement mean ± σ min … max outliers delta
wall_time 52.7ms ± 1.05ms 48.8ms … 57.5ms 35 ( 1%) 💩+ 2.5% ± 0.1%
peak_rss 138MB ± 814KB 135MB … 141MB 22 ( 1%) - 0.0% ± 0.0%
cpu_cycles 1.04G ± 30.3M 826M … 1.20G 87 ( 3%) 💩+ 1.2% ± 0.1%
instructions 1.26G ± 14.1M 1.18G … 1.35G 77 ( 2%) 💩+ 2.4% ± 0.1%
cache_references 25.7M ± 564K 23.3M … 28.9M 37 ( 1%) + 1.0% ± 0.1%
cache_misses 5.92M ± 149K 5.26M … 6.63M 17 ( 0%) - 0.2% ± 0.1%
branch_misses 2.25M ± 49.0K 1.98M … 2.48M 62 ( 2%) - 0.3% ± 0.1%
| { | ||
| // Check symbol is within ADR range (±1MB) from ADD position | ||
| let add_place = section_address + next_offset; | ||
| let diff = (sym_addr as i64).wrapping_sub(add_place as i64); |
There was a problem hiding this comment.
Do we need to also take the addend into account here? Otherwise we might apply the relaxation when the addend puts us just outside the allowed range..
| sym_addr: u64, | ||
| section_address: u64, | ||
| _relax_deltas: Option<&linker_utils::relaxation::SectionRelaxDeltas>, | ||
| next_relocation: Option<(object::elf::RelocationType, u64)>, |
There was a problem hiding this comment.
It looks like we're not checking which symbol the relocation refers to, so if the second symbol was for a different symbol, we might still apply the relaxation when we shouldn't. It'd be a bit weird for the compiler to emit that, especially given that the instructions use the same register, but we still shouldn't relax it.
|
Speaking of the numbers, I’d also be interested in how many such relaxations can be triggered for a project like Wild. |
|
The slowdown for x86_64 was small enough that the benchmark didn't flag it. It showed it as 0.8%, which I guess is small enough that it didn't have high confidence that it wasn't noise. My guess is that the optimiser was able to eliminate some of the arguments since they were never used on the x86_64 version. |
|
Note that I did benchmark an earlier version - I think the version that was adding the new API to |
e0780c5 to
48e2ba5
Compare
|
@marxin @davidlattimore For performance, I'll try a few other approaches as well. I'm very open to experimenting with different approaches to achieve the same result with this relaxation. I'd be very happy to try another approach. |
davidlattimore
left a comment
There was a problem hiding this comment.
That still shows about 2.4% slowdown on aarch64. What about the possibility of processing the pair when we get to the ADD rather than the ADRP? That would mean we'd need to pass in the previous relocation rather than the next relocation, which might be a bit easier to implement in a performant way. The RelocationCache already stores the previous relocation. The relocations are applied in order, so a relaxation on the later relocation can overwrite whatever the previous relocation did if it decides to do something different.
6081cd4 to
8e340a8
Compare
Adds sym_addr, section_address and previous_relocation parameters to new_relaxation to enable pair-based relaxations without performance overhead. The previous_relocation comes from RelocationCache which already stored it, so no peekable() iterator is needed. AArch64 relaxes ADRP+ADD to NOP+ADR when processing the ADD relocation by looking back at the previous relocation (ADRP) via previous_relocation: - Previous must be R_AARCH64_ADR_PREL_PG_HI21 at consecutive offset - Both relocations must reference the same symbol (checked in elf_writer.rs) - ADRP and ADD must use the same destination/source register - Symbol must be within ADR range checked via AllowedRange::from_bit_size(21) New AdrpAddToNopAdr RelaxationKind applied at ADD position: - Uses ReplaceWithNop for NOP at ADRP position (offset-4) - Writes ADR base instruction at ADD position - Applies R_AARCH64_ADR_PREL_LO21 relocation value This matches lld's tryRelaxAdrpAdd optimization with no performance regression since RelocationCache.previous was already available.
8e340a8 to
5422ff6
Compare
marxin
left a comment
There was a problem hiding this comment.
Thanks for iterating on that change - in general looks fine to me, but I would like to see the performance numbers after the latest attempt @deepakshirkem?
| sym_addr: u64, | ||
| section_address: u64, | ||
| previous_relocation: Option<( | ||
| <Self::Platform as Platform>::RelocationInfo, |
There was a problem hiding this comment.
Should be a new struct instead of the tuple.
| && prev_addend == 0 | ||
| { | ||
| let offset = offset_in_section as usize; | ||
| if offset >= 4 && offset + 4 <= section_bytes.len() { |
There was a problem hiding this comment.
Instead of the boundary checking, please use directly section_bytes.get and handle the Option return value.
| relocation_cache | ||
| .previous | ||
| .as_ref() | ||
| .filter(|r| r.symbol() == rel.symbol() && r.addend() == 0 && rel.addend() == 0) |
There was a problem hiding this comment.
Seems the filtering is a duplicate logic of new_relaxation and it should not happen here.
When an ADRP+ADD pair references a symbol within ADR range (±1MB) and the instructions use the same register, relax to NOP+ADR.
This matches lld's
tryRelaxAdrpAddoptimization and reduces code size for frequently accessed local symbols.Issue #2247