Skip to content

feat: Implement ADRP+ADD to NOP+ADR relaxation for AArch64 - #2319

Open
deepakshirkem wants to merge 1 commit into
wild-linker:mainfrom
deepakshirkem:feat/aarch64-adrp-add-relaxation
Open

feat: Implement ADRP+ADD to NOP+ADR relaxation for AArch64#2319
deepakshirkem wants to merge 1 commit into
wild-linker:mainfrom
deepakshirkem:feat/aarch64-adrp-add-relaxation

Conversation

@deepakshirkem

Copy link
Copy Markdown
Member

When an ADRP+ADD pair references a symbol within ADR range (±1MB) and the instructions use the same register, relax to NOP+ADR.

This matches lld's tryRelaxAdrpAdd optimization and reduces code size for frequently accessed local symbols.

Issue #2247

@deepakshirkem
deepakshirkem force-pushed the feat/aarch64-adrp-add-relaxation branch from ec27067 to 692cef7 Compare August 2, 2026 19:23

@davidlattimore davidlattimore left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't looked in much detail as yet, but is there any reason why this can't be done similar to how the other aarch64 relaxations are implemented?

@deepakshirkem

Copy link
Copy Markdown
Member Author

The main challenge is that ADRP+ADD relaxation requires inspecting the NEXT relocation (ADD) to verify same symbol and register, which isn't available in new_relaxation. The existing AArch64 relaxations are single-instruction transformations.

If you have a preferred approach for handling two-instruction relaxations in Wild's existing infrastructure.

@davidlattimore

Copy link
Copy Markdown
Member

I think we'll need to adjust the API so that the code that decides relaxations can access the next relocation. Either that or add a new API for handling relaxations - e.g. one that always receives pairs of adjacent relocations. I'm not sure which would be best. But either way, aarch64 relaxation code definitely needs to be in the elf_aarch64 module. Where it is now, it gets run for all architectures and the relocation codes it's looking for are likely to be used for something completely different on those other architectures.

@deepakshirkem
deepakshirkem force-pushed the feat/aarch64-adrp-add-relaxation branch 2 times, most recently from 68b9d88 to ae65e29 Compare August 5, 2026 14:10
Comment thread linker-diff/src/lib.rs
// LLD does some different relaxations to us
"rel.missing-opt.R_AARCH64_ADR_GOT_PAGE.ReplaceWithNop.*",
"rel.missing-opt.R_AARCH64_ADR_PREL_PG_HI21.ReplaceWithNop.*",
"rel.extra-opt.R_AARCH64_ADR_PREL_PG_HI21.ReplaceWithNop.*",

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@davidlattimore Is this line correct? The CI is now green with this line. My understanding is that we're comparing the output against GNU ld, but Wild has an optimization that GNU ld doesn't. This line seems to skip that comparison.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that's reasonable to add that global ignore. I guess ideally we'd have ignore rules that changed depending on which linker we were comparing against, but currently we don't. The placement of the ignore rule is possibly a bit misleading. It's under a comment that refers to "LLD", but from what I understand, LLD does this optimisation too and it's GNU ld that doesn't.

//#ReferenceLinkers:lld
//#LinkArgs:--no-gc-sections -Ttext=0x200ffc
//#RunEnabled:false
//#DiffIgnore:file-header.entry

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you think we can improve this test as well?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you mean with regard to the ignore of file-header.entry? It looks like that's effectively a bug in linker-diff where it's not doing a good job of handling the fact that the entry point has multiple symbols that resolve to that address.

Usually I try to not disable running of the executable, but it's possible that it's not worth trying to run the executable in this case.

@deepakshirkem
deepakshirkem marked this pull request as ready for review August 5, 2026 14:41
Comment thread libwild/src/elf_aarch64.rs Outdated

fn apply_pair_relaxation(
first_kind: object::elf::RelocationType,
_second_kind: object::elf::RelocationType,

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why aren't you checking the type of the second relocation? Doesn't that mean that if it's a different kind of relocation, we might apply the relaxation when it shouldn't apply? Could be good to add a test for this.

Comment thread libwild/src/elf_aarch64.rs Outdated
None
}

fn apply_pair_relaxation(

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be honest, don't like the new entry point - can't we rather base the relaxation on what was suggested by David - passing the next relocation to the A::next_relaxation based on the following patch:

diff --git a/libwild/src/elf_writer.rs b/libwild/src/elf_writer.rs
index 97e18ca6..b80893ac 100644
--- a/libwild/src/elf_writer.rs
+++ b/libwild/src/elf_writer.rs
@@ -2417,7 +2417,7 @@ fn apply_relocations<
     object: &ObjectLayout<'data, elf::Elf<C>>,
     out: &mut [u8],
     section_index: object::SectionIndex,
-    mut relocations: I,
+    relocations: I,
     layout: &ElfLayout<'data, C>,
     table_writer: &mut TableWriter<'_, '_, C>,
     trace: &TraceOutput,
@@ -2434,6 +2434,7 @@ fn apply_relocations<
     let relax_deltas = object.section_relax_deltas.get(section_index.0);
     let mut relax_cursor = relax_deltas.map(|deltas| deltas.cursor());

+    let mut relocations = relocations.peekable();
     while let Some(rel) = relocations.next() {
         let rel = rel?;
         relocation_count += 1;
@@ -2456,10 +2457,12 @@ fn apply_relocations<
             None => rel.offset(),
         };

+        let rel_next = relocations.peek().copied().transpose()?;
         modifier = apply_relocation::<C, A, R, _>(
             object,
             offset_in_section,
             &rel,
+            rel_next,
             SectionInfo {
                 section_address,
                 is_writable: object_section.is_writable(),
@@ -2718,6 +2721,9 @@ fn write_eh_frame_relocations<
                     object,
                     rel_offset - input_pos as u64,
                     rel,
+                    // Relaxations based on the ability to look-up the next relocation are not
+                    // expected to happen for the section.
+                    None,
                     SectionInfo {
                         section_address: output_pos as u64 + table_writer.eh_frame_start_address,
                         is_writable: false,
@@ -3040,6 +3046,7 @@ fn apply_relocation<
     object_layout: &ObjectLayout<'data, elf::Elf<C>>,
     mut offset_in_section: u64,
     rel: &R,
+    _next_rel: Option<R>,
     section_info: SectionInfo<linker_utils::elf::SectionFlags>,
     layout: &ElfLayout<'data, C>,
     out: &mut [u8],

Comment thread libwild/src/elf_aarch64.rs Outdated
// Get destination register from ADRP instruction
let adrp_instr = u32::from_le_bytes(out[offset..offset + 4].try_into().unwrap());
let add_instr = u32::from_le_bytes(out[offset + 4..offset + 8].try_into().unwrap());
let adrp_dest_reg = adrp_instr & 0x1f;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we rather use already used pattern: extract_bit_range?

Comment thread libwild/src/elf_aarch64.rs Outdated
let add_dest_reg = add_instr & 0x1f;
let add_src_reg = (add_instr >> 5) & 0x1f;
// Verify ADRP and ADD instructions use same register
if (adrp_instr & 0x9f000000) != 0x90000000

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see the code just extracted some source and destination registers - why are we then comparing the masks out of adrp_instr and add_instr? Don't get it.

Comment thread libwild/src/elf_aarch64.rs Outdated
return false;
}
// Write NOP at ADRP position
out[offset..offset + 4].copy_from_slice(&[0x1f, 0x20, 0x03, 0xd5]);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should rather define a new RelaxationKind enum variant and make the transformation of all 8B in RelaxationKind::apply function.

Comment thread libwild/src/elf_aarch64.rs Outdated
}
// Write NOP at ADRP position
out[offset..offset + 4].copy_from_slice(&[0x1f, 0x20, 0x03, 0xd5]);
// Write ADR base at ADD position

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You might be able to reuse: RelaxationKind::AddToAdr (and so your RelaxationKind will be effectively a composition of 2 already supported relaxations)?

@@ -0,0 +1,27 @@
// This verifies that Wild relaxes ADRP+ADD to NOP+ADR when the symbol is
// within ADR range (±1MB) and registers match, but keeps ADRP+ADD when:
// - the destination registers differ (x2 vs x3 in third pair)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These 2 lines are duplicate - they are later documented again at the assembly level.

@deepakshirkem

Copy link
Copy Markdown
Member Author

@marxin Thank you. I actually started with the peekable/next_rel approach but found it required changes to many architectures (updating all new_relaxation implementations). I then went with the apply_pair_relaxation approach to keep changes isolated.

However looking at your patch, it's much cleaner passing next_rel through apply_relocation avoids the need for a new trait method entirely. I'll implement this approach later today and update the PR.

@deepakshirkem
deepakshirkem force-pushed the feat/aarch64-adrp-add-relaxation branch from 49cdc54 to bcd370b Compare August 8, 2026 11:24
@deepakshirkem
deepakshirkem requested a review from marxin August 8, 2026 11:29
@deepakshirkem

Copy link
Copy Markdown
Member Author

@marxin @davidlattimore Please do one more cycle of review when you get a chance?

@marxin marxin left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're getting closer I think, please try to address my comments.

Comment thread libwild/src/elf_aarch64.rs Outdated
Comment thread libwild/src/elf_x86_64.rs
Comment thread libwild/src/elf_aarch64.rs Outdated
Comment thread linker-utils/src/aarch64.rs Outdated
Comment thread linker-utils/src/aarch64.rs Outdated
Comment thread libwild/src/elf_aarch64.rs Outdated
@deepakshirkem
deepakshirkem requested a review from marxin August 8, 2026 13:47
@deepakshirkem
deepakshirkem force-pushed the feat/aarch64-adrp-add-relaxation branch from ead5f74 to a9a30d5 Compare August 8, 2026 14:03
@deepakshirkem

Copy link
Copy Markdown
Member Author

@marxin Ping :)).

@marxin marxin left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the iteration on the patch - I am pretty happy about it now. But let's make the final decision to @davidlattimore.

One general recommendation - please try to instruct your LLM (I guess you're using Claude) to emit more information intensive comments. Or ideally, try to compress the comments.

Comment thread libwild/src/elf_riscv64.rs Outdated
Comment thread linker-utils/src/aarch64.rs Outdated
Comment thread linker-utils/src/aarch64.rs Outdated
Comment thread linker-utils/src/aarch64.rs Outdated
Comment thread libwild/src/elf.rs Outdated
symbol_db.output_kind,
section_flags,
true,
1, // sym_addr: assume non-zero (actual address not available in layout phase)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@davidlattimore - what about using an Option<(u64, u64), or a new type instead of the part of u64 types? In particular, don't like the 1 special value which mimics the non-zero value, but is pretty non-intuitive.

@deepakshirkem
deepakshirkem force-pushed the feat/aarch64-adrp-add-relaxation branch from a9a30d5 to ba4803b Compare August 8, 2026 18:39
@deepakshirkem
deepakshirkem requested a review from marxin August 8, 2026 18:44

@davidlattimore davidlattimore left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see a 2.5% performance loss when linking wild for aarch64:

Benchmark 1 (3495 runs): /home/david/save/wild-aarch64/run-with /home/d/wild-builds/2026-08-09.cg1 --strip-debug --no-fork
  measurement          mean ± σ            min … max           outliers         delta
  wall_time          51.4ms ± 1.06ms    47.4ms … 59.1ms         31 ( 1%)        0%
  peak_rss            138MB ±  832KB     135MB …  142MB         28 ( 1%)        0%
  cpu_cycles         1.03G  ± 29.6M      826M  … 1.16G          84 ( 2%)        0%
  instructions       1.23G  ± 13.6M     1.16G  … 1.31G          78 ( 2%)        0%
  cache_references   25.5M  ±  550K     23.5M  … 27.6M          38 ( 1%)        0%
  cache_misses       5.93M  ±  144K     5.26M  … 6.49M          15 ( 0%)        0%
  branch_misses      2.26M  ± 80.4K     2.03M  … 2.87M         100 ( 3%)        0%
Benchmark 2 (3411 runs): /home/david/save/wild-aarch64/run-with target/cg1/wild --strip-debug --no-fork
  measurement          mean ± σ            min … max           outliers         delta
  wall_time          52.7ms ± 1.05ms    48.8ms … 57.5ms         35 ( 1%)        💩+  2.5% ±  0.1%
  peak_rss            138MB ±  814KB     135MB …  141MB         22 ( 1%)          -  0.0% ±  0.0%
  cpu_cycles         1.04G  ± 30.3M      826M  … 1.20G          87 ( 3%)        💩+  1.2% ±  0.1%
  instructions       1.26G  ± 14.1M     1.18G  … 1.35G          77 ( 2%)        💩+  2.4% ±  0.1%
  cache_references   25.7M  ±  564K     23.3M  … 28.9M          37 ( 1%)          +  1.0% ±  0.1%
  cache_misses       5.92M  ±  149K     5.26M  … 6.63M          17 ( 0%)          -  0.2% ±  0.1%
  branch_misses      2.25M  ± 49.0K     1.98M  … 2.48M          62 ( 2%)          -  0.3% ±  0.1%

{
// Check symbol is within ADR range (±1MB) from ADD position
let add_place = section_address + next_offset;
let diff = (sym_addr as i64).wrapping_sub(add_place as i64);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to also take the addend into account here? Otherwise we might apply the relaxation when the addend puts us just outside the allowed range..

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any comment on this?

Comment thread libwild/src/elf_aarch64.rs Outdated
sym_addr: u64,
section_address: u64,
_relax_deltas: Option<&linker_utils::relaxation::SectionRelaxDeltas>,
next_relocation: Option<(object::elf::RelocationType, u64)>,

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like we're not checking which symbol the relocation refers to, so if the second symbol was for a different symbol, we might still apply the relaxation when we shouldn't. It'd be a bit weird for the compiler to emit that, especially given that the instructions use the same register, but we still shouldn't relax it.

@marxin

marxin commented Aug 9, 2026

Copy link
Copy Markdown
Collaborator

Speaking of the numbers, I’d also be interested in how many such relaxations can be triggered for a project like Wild.
@davidlattimore am I right we should see a similar slowdown even for non-AArch64 architecture as we're passing more data to the relaxation parsing?

@davidlattimore

Copy link
Copy Markdown
Member

The slowdown for x86_64 was small enough that the benchmark didn't flag it. It showed it as 0.8%, which I guess is small enough that it didn't have high confidence that it wasn't noise. My guess is that the optimiser was able to eliminate some of the arguments since they were never used on the x86_64 version.

@davidlattimore

Copy link
Copy Markdown
Member

Note that I did benchmark an earlier version - I think the version that was adding the new API to Platform for relaxing a pair of relocations. That version didn't show a slowdown. I'm not saying that we necessarily need to go back to that version - there might be other options that can be experimented with.

@deepakshirkem
deepakshirkem force-pushed the feat/aarch64-adrp-add-relaxation branch 2 times, most recently from e0780c5 to 48e2ba5 Compare August 9, 2026 07:28
@deepakshirkem

Copy link
Copy Markdown
Member Author

@marxin @davidlattimore For performance, I'll try a few other approaches as well. I'm very open to experimenting with different approaches to achieve the same result with this relaxation. I'd be very happy to try another approach.

@davidlattimore davidlattimore left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That still shows about 2.4% slowdown on aarch64. What about the possibility of processing the pair when we get to the ADD rather than the ADRP? That would mean we'd need to pass in the previous relocation rather than the next relocation, which might be a bit easier to implement in a performant way. The RelocationCache already stores the previous relocation. The relocations are applied in order, so a relaxation on the later relocation can overwrite whatever the previous relocation did if it decides to do something different.

@marxin
marxin removed their request for review August 12, 2026 09:58
@deepakshirkem
deepakshirkem force-pushed the feat/aarch64-adrp-add-relaxation branch 2 times, most recently from 6081cd4 to 8e340a8 Compare August 12, 2026 12:07
Adds sym_addr, section_address and previous_relocation parameters to
new_relaxation to enable pair-based relaxations without performance overhead.
The previous_relocation comes from RelocationCache which already stored it,
so no peekable() iterator is needed.

AArch64 relaxes ADRP+ADD to NOP+ADR when processing the ADD relocation
by looking back at the previous relocation (ADRP) via previous_relocation:
- Previous must be R_AARCH64_ADR_PREL_PG_HI21 at consecutive offset
- Both relocations must reference the same symbol (checked in elf_writer.rs)
- ADRP and ADD must use the same destination/source register
- Symbol must be within ADR range checked via AllowedRange::from_bit_size(21)

New AdrpAddToNopAdr RelaxationKind applied at ADD position:
- Uses ReplaceWithNop for NOP at ADRP position (offset-4)
- Writes ADR base instruction at ADD position
- Applies R_AARCH64_ADR_PREL_LO21 relocation value

This matches lld's tryRelaxAdrpAdd optimization with no performance
regression since RelocationCache.previous was already available.
@deepakshirkem
deepakshirkem force-pushed the feat/aarch64-adrp-add-relaxation branch from 8e340a8 to 5422ff6 Compare August 12, 2026 12:40

@marxin marxin left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for iterating on that change - in general looks fine to me, but I would like to see the performance numbers after the latest attempt @deepakshirkem?

sym_addr: u64,
section_address: u64,
previous_relocation: Option<(
<Self::Platform as Platform>::RelocationInfo,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be a new struct instead of the tuple.

&& prev_addend == 0
{
let offset = offset_in_section as usize;
if offset >= 4 && offset + 4 <= section_bytes.len() {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of the boundary checking, please use directly section_bytes.get and handle the Option return value.

Comment thread libwild/src/elf_writer.rs
relocation_cache
.previous
.as_ref()
.filter(|r| r.symbol() == rel.symbol() && r.addend() == 0 && rel.addend() == 0)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems the filtering is a duplicate logic of new_relaxation and it should not happen here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants