document ABI compatibility by RalfJung · Pull Request #115476 · rust-lang/rust

RalfJung · 2023-09-02T10:57:01Z

I don't think we have any central place where we document our ABI compatibility rules, so let's create one. The fn() pointer type seems like a good place since ABI questions can only become relevant when invoking a function through a function pointer.

This will likely need T-lang FCP.

rustbot · 2023-09-02T10:57:06Z

r? @Mark-Simulacrum

(rustbot has picked a reviewer for you, use r? to override)

RalfJung · 2023-09-02T10:58:51Z

This is currently not true on sparc64, but this is required for repr(transparent) to work so that's likely a target bug and we'll probably have to change the ABI of some Rust types. Also see the discussion on IRLO here and here.

RalfJung · 2023-09-02T11:00:17Z

I wasn't sure if this should be documented here, or with the NonZero* and Option types, respectively -- or both.
The Option part is justified by an RFC. The same RFC also mentions Result but so far we don't document any layout nor ABI guarantees for Result so I left that out of this PR.

RalfJung · 2023-09-02T11:01:43Z

Sign in to view

This goes quite far and I'm a bit worried we might one day discover that it doesn't actually always work. But so far I haven't been able to find a counterexample... and chances are high that people will be relying on it even if we don't guarantee it, since it seems so natural (and since AFAIK it is true in C, at least in de-facto C -- I hear the standard has some odd caveats involving the name of the struct type).

It seems reasonable to restrict some guarantees to just the ABIs that are guaranteed to exist on all platforms, which would include system.

I'm not clear on why this particular guarantee would ever break, though, and I think you're right that people will rely on ABI compatibility being "recursive" or "covariant" in this sense. So if we do have reason to believe it's not guaranteed, we should definitely say so very loudly somewhere.

Sign in to view

soqb

i think there are a few style/clarity nits to iron out but i really like this documentation!

pnkfelix · 2023-09-05T18:39:58Z

The T-lang design team discussed this PR briefly in triage meeting.

We agreed with @RalfJung that this is something that needs documentation, and it appears that this text is not adding anything new, but rather documenting the existing rules as they are understood. Therefore, we are happy to go ahead with a T-lang FCP.

In addition, however, we thought that this seemed like it may overlap with the expertise of T-opsem, and therefore in addition to the T-lang FCP, we also are nominating this for T-opsem to look at to ensure that they have a chance to sign off on the text as written here.

@rfcbot fcp merge

rfcbot · 2023-09-05T18:40:01Z

Team member @pnkfelix has proposed to merge this. The next step is review by the rest of the tagged team members:

Concerns:

~~make-sure-t-opsem-has-chance-to-look-at-this-before-merging-it~~ resolved by document ABI compatibility #115476 (comment)

Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

cc @rust-lang/lang-advisors: FCP proposed for lang, please feel free to register concerns.
See this document for info about what commands tagged team members can give me.

RalfJung · 2023-09-05T18:41:05Z

Did you mean to include T-opsem in the FCP?

pnkfelix · 2023-09-05T18:45:50Z

Did you mean to include T-opsem in the FCP?

No, I did not want to do that yet.

I wanted to instead nominate this for T-opsem discussion and let them decide whether they need an FCP here.

…ity comes up

and add an and

Mark-Simulacrum

r=me with this comment resolved (either way)

Mark-Simulacrum · 2023-11-17T01:48:37Z

+/// is also used rarely. So, most likely you do not have to worry about ABI compatibility.
+///
+/// But assuming such circumstances, what are the rules? For this section, we are specifically
+/// concerned with the case where both the caller and the callee are defined in Rust.


Suggested change

/// concerned with the case where both the caller and the callee are defined in Rust.

/// concerned with the case where both the caller and the callee are defined in Rust (and using the same compiler version).

Right? We're not making guarantees about inter-op between future and past compiler versions here, in terms of what the ABI of some primitive is?

If you start linking separately compiled Rust objects with each other there's a ton of things you need to consider. Using the same Rust version is necessary but far from sufficient. I don't even know the full list of things, but at the very least you must use the same target triple, and there's a bunch of other flags that must be set the same, as far as I understand.

I'm not sure if we want to get into listing all these requirements here. I want to discuss ABI, not linking.

RalfJung · 2023-11-17T07:03:05Z

@bors r=Mark-Simulacrum rollup

bors · 2023-11-17T07:03:08Z

📌 Commit 8f03a55 has been approved by Mark-Simulacrum

It is now in the queue for this repository.

…iaskrgr Rollup of 3 pull requests Successful merges: - rust-lang#115476 (document ABI compatibility) - rust-lang#117688 (Misc changes to StableMIR required to Kani use case.) - rust-lang#117998 (On resolve error of `[rest..]`, suggest `[rest @ ..]`) r? `@ghost` `@rustbot` modify labels: rollup

Rollup merge of rust-lang#115476 - RalfJung:abi-compat-docs, r=Mark-Simulacrum document ABI compatibility I don't think we have any central place where we document our ABI compatibility rules, so let's create one. The `fn()` pointer type seems like a good place since ABI questions can only become relevant when invoking a function through a function pointer. This will likely need T-lang FCP.

…tmcm do not allow ABI mismatches inside repr(C) types In rust-lang#115476 we allowed ABI mismatches inside `repr(C)` types. This wasn't really discussed much; I added it because from how I understand calling conventions, this should actually be safe in practice. However I entirely forgot to actually allow this in Miri, and in the mean time I have learned that too much ABI compatibility can be a problem for CFI (it can reject fewer calls so that gives an attacker more room to play with). So I propose we take back that part about ABI compatibility in `repr(C)`. It is anyway something that C and C++ do not allow, as far as I understand. In the future we might want to introduce a class of ABI compatibilities where we say "this is a bug and it may lead to aborting the process, but it won't lead to arbitrary misbehavior -- worst case it'll just transmute the arguments from the caller type to the callee type". That would give CFI leeway to reject such calls without introducing the risk of arbitrary UB. (The UB can still happen if the transmute leads to bad results, of course, but it wouldn't be due to ABI weirdness.) rust-lang#115476 hasn't reached beta yet so if we land this before Dec 22nd we can just pretend this all never happened. ;) Otherwise we should do a beta backport (of the docs change at least). Cc `@rust-lang/opsem` `@rust-lang/types`

do not allow ABI mismatches inside repr(C) types In rust-lang/rust#115476 we allowed ABI mismatches inside `repr(C)` types. This wasn't really discussed much; I added it because from how I understand calling conventions, this should actually be safe in practice. However I entirely forgot to actually allow this in Miri, and in the mean time I have learned that too much ABI compatibility can be a problem for CFI (it can reject fewer calls so that gives an attacker more room to play with). So I propose we take back that part about ABI compatibility in `repr(C)`. It is anyway something that C and C++ do not allow, as far as I understand. In the future we might want to introduce a class of ABI compatibilities where we say "this is a bug and it may lead to aborting the process, but it won't lead to arbitrary misbehavior -- worst case it'll just transmute the arguments from the caller type to the callee type". That would give CFI leeway to reject such calls without introducing the risk of arbitrary UB. (The UB can still happen if the transmute leads to bad results, of course, but it wouldn't be due to ABI weirdness.) #115476 hasn't reached beta yet so if we land this before Dec 22nd we can just pretend this all never happened. ;) Otherwise we should do a beta backport (of the docs change at least). Cc `@rust-lang/opsem` `@rust-lang/types`

Pkgsrc changes: * Adapt checksums and patches. Upstream chnages: Version 1.76.0 (2024-02-08) ========================== Language -------- - [Document Rust ABI compatibility between various types] (rust-lang/rust#115476) - [Also: guarantee that char and u32 are ABI-compatible] (rust-lang/rust#118032) - [Warn against ambiguous wide pointer comparisons] (rust-lang/rust#117758) Compiler -------- - [Lint pinned `#[must_use]` pointers (in particular, `Box<T>` where `T` is `#[must_use]`) in `unused_must_use`.] (rust-lang/rust#118054) - [Soundness fix: fix computing the offset of an unsized field in a packed struct] (rust-lang/rust#118540) - [Soundness fix: fix dynamic size/align computation logic for packed types with dyn Trait tail] (rust-lang/rust#118538) - [Add `$message_type` field to distinguish json diagnostic outputs] (rust-lang/rust#115691) - [Enable Rust to use the EHCont security feature of Windows] (rust-lang/rust#118013) - [Add tier 3 {x86_64,i686}-win7-windows-msvc targets] (rust-lang/rust#118150) - [Add tier 3 aarch64-apple-watchos target] (rust-lang/rust#119074) - [Add tier 3 arm64e-apple-ios & arm64e-apple-darwin targets] (rust-lang/rust#115526) Refer to Rust's [platform support page][platform-support-doc] for more information on Rust's tiered platform support. Libraries --------- - [Add a column number to `dbg!()`] (rust-lang/rust#114962) - [Add `std::hash::{DefaultHasher, RandomState}` exports] (rust-lang/rust#115694) - [Fix rounding issue with exponents in fmt] (rust-lang/rust#116301) - [Add T: ?Sized to `RwLockReadGuard` and `RwLockWriteGuard`'s Debug impls.] (rust-lang/rust#117138) - [Windows: Allow `File::create` to work on hidden files] (rust-lang/rust#116438) Stabilized APIs --------------- - [`Arc::unwrap_or_clone`] (https://doc.rust-lang.org/stable/std/sync/struct.Arc.html#method.unwrap_or_clone) - [`Rc::unwrap_or_clone`] (https://doc.rust-lang.org/stable/std/rc/struct.Rc.html#method.unwrap_or_clone) - [`Result::inspect`] (https://doc.rust-lang.org/stable/std/result/enum.Result.html#method.inspect) - [`Result::inspect_err`] (https://doc.rust-lang.org/stable/std/result/enum.Result.html#method.inspect_err) - [`Option::inspect`] (https://doc.rust-lang.org/stable/std/option/enum.Option.html#method.inspect) - [`type_name_of_val`] (https://doc.rust-lang.org/stable/std/any/fn.type_name_of_val.html) - [`std::hash::{DefaultHasher, RandomState}`] (https://doc.rust-lang.org/stable/std/hash/index.html#structs) These were previously available only through `std::collections::hash_map`. - [`ptr::{from_ref, from_mut}`] (https://doc.rust-lang.org/stable/std/ptr/fn.from_ref.html) - [`ptr::addr_eq`](https://doc.rust-lang.org/stable/std/ptr/fn.addr_eq.html) Cargo ----- See [Cargo release notes] (https://github.com/rust-lang/cargo/blob/master/CHANGELOG.md#cargo-176-2024-02-08). Rustdoc ------- - [Don't merge cfg and doc(cfg) attributes for re-exports] (rust-lang/rust#113091) - [rustdoc: allow resizing the sidebar / hiding the top bar] (rust-lang/rust#115660) - [rustdoc-search: add support for traits and associated types] (rust-lang/rust#116085) - [rustdoc: Add highlighting for comments in items declaration] (rust-lang/rust#117869) Compatibility Notes ------------------- - [Add allow-by-default lint for unit bindings] (rust-lang/rust#112380) This is expected to be upgraded to a warning by default in a future Rust release. Some macros emit bindings with type `()` with user-provided spans, which means that this lint will warn for user code. - [Remove x86_64-sun-solaris target.] (rust-lang/rust#118091) - [Remove asmjs-unknown-emscripten target] (rust-lang/rust#117338) - [Report errors in jobserver inherited through environment variables] (rust-lang/rust#113730) This [may warn](rust-lang/rust#120515) on benign problems too. - [Update the minimum external LLVM to 16.] (rust-lang/rust#117947) - [Improve `print_tts`](rust-lang/rust#114571) This change can break some naive manual parsing of token trees in proc macro code which expect a particular structure after `.to_string()`, rather than just arbitrary Rust code. - [Make `IMPLIED_BOUNDS_ENTAILMENT` into a hard error from a lint] (rust-lang/rust#117984) - [Vec's allocation behavior was changed when collecting some iterators] (rust-lang/rust#110353) Allocation behavior is currently not specified, nevertheless changes can be surprising. See [`impl FromIterator for Vec`] (https://doc.rust-lang.org/nightly/std/vec/struct.Vec.html#impl-FromIterator%3CT%3E-for-Vec%3CT%3E) for more details. - [Properly reject `default` on free const items] (rust-lang/rust#117818)

do not allow ABI mismatches inside repr(C) types In rust-lang/rust#115476 we allowed ABI mismatches inside `repr(C)` types. This wasn't really discussed much; I added it because from how I understand calling conventions, this should actually be safe in practice. However I entirely forgot to actually allow this in Miri, and in the mean time I have learned that too much ABI compatibility can be a problem for CFI (it can reject fewer calls so that gives an attacker more room to play with). So I propose we take back that part about ABI compatibility in `repr(C)`. It is anyway something that C and C++ do not allow, as far as I understand. In the future we might want to introduce a class of ABI compatibilities where we say "this is a bug and it may lead to aborting the process, but it won't lead to arbitrary misbehavior -- worst case it'll just transmute the arguments from the caller type to the callee type". That would give CFI leeway to reject such calls without introducing the risk of arbitrary UB. (The UB can still happen if the transmute leads to bad results, of course, but it wouldn't be due to ABI weirdness.) #115476 hasn't reached beta yet so if we land this before Dec 22nd we can just pretend this all never happened. ;) Otherwise we should do a beta backport (of the docs change at least). Cc `@rust-lang/opsem` `@rust-lang/types`

@nikic

…-Simulacrum declare C and C-unwind as mutually ABI-compatible We left this conservative in rust-lang#115476. That means the following code is currently UB: ```rust extern "C-unwind" fn does_not_unwind_but_could() {} fn main() { let f: extern "C-unwind" fn() = does_not_unwind_but_could; let f: extern "C" fn() = unsafe { std::mem::transmute(f) }; f(); } ``` I think this code should be allowed. [Zulip discussion](https://rust-lang.zulipchat.com/#narrow/channel/136281-t-opsem/topic/ABI.20compatibility.20of.20.22C.22.20vs.20.22C-unwind.22) also led to the conclusion that this should be fine in LLVM -- the `nounwind` attribute does not affect the ABI. This is widely relied upon in the C ecosystem when doing calls between C and C++. Cc @nikic to confirm. I am not sure why rust-lang#115476 left this conservative. There was discussion about explicitly listing the ABIs rather than saying this holds universally for all `*-unwind` that we may add in the future, but there was no discussion I could find about allowing a mismatch both ways. Cc @rust-lang/opsem @rust-lang/lang

@nikic

Rollup merge of #161904 - RalfJung:unwind-abi-compat, r=Mark-Simulacrum declare C and C-unwind as mutually ABI-compatible We left this conservative in #115476. That means the following code is currently UB: ```rust extern "C-unwind" fn does_not_unwind_but_could() {} fn main() { let f: extern "C-unwind" fn() = does_not_unwind_but_could; let f: extern "C" fn() = unsafe { std::mem::transmute(f) }; f(); } ``` I think this code should be allowed. [Zulip discussion](https://rust-lang.zulipchat.com/#narrow/channel/136281-t-opsem/topic/ABI.20compatibility.20of.20.22C.22.20vs.20.22C-unwind.22) also led to the conclusion that this should be fine in LLVM -- the `nounwind` attribute does not affect the ABI. This is widely relied upon in the C ecosystem when doing calls between C and C++. Cc @nikic to confirm. I am not sure why #115476 left this conservative. There was discussion about explicitly listing the ABIs rather than saying this holds universally for all `*-unwind` that we may add in the future, but there was no discussion I could find about allowing a mismatch both ways. Cc @rust-lang/opsem @rust-lang/lang

rustbot assigned Mark-Simulacrum Sep 2, 2023

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-libs Relevant to the library team, which will review and decide on the PR/issue. labels Sep 2, 2023

RalfJung commented Sep 2, 2023

View reviewed changes

Comment thread library/std/src/primitive_docs.rs Outdated

RalfJung commented Sep 2, 2023

View reviewed changes

RalfJung force-pushed the abi-compat-docs branch from 8a2a026 to 7983afe Compare September 2, 2023 10:59

RalfJung commented Sep 2, 2023

View reviewed changes

This comment has been minimized.

Sign in to view

RalfJung force-pushed the abi-compat-docs branch from 7983afe to ee09160 Compare September 2, 2023 11:31

This comment has been minimized.

Sign in to view

RalfJung force-pushed the abi-compat-docs branch from cddd5b7 to b1a7983 Compare September 2, 2023 14:53

Noratrieb reviewed Sep 2, 2023

View reviewed changes

Comment thread library/core/src/primitive_docs.rs Outdated

RalfJung force-pushed the abi-compat-docs branch from 5479e4c to 81dadf6 Compare September 3, 2023 08:01

RalfJung commented Sep 3, 2023

View reviewed changes

Comment thread library/std/src/primitive_docs.rs Outdated

Comment thread library/std/src/primitive_docs.rs Outdated

Comment on lines 1554 to 1555

This comment was marked as resolved.

Sign in to view

soqb reviewed Sep 3, 2023

View reviewed changes

Comment thread library/std/src/primitive_docs.rs Outdated

Comment thread library/std/src/primitive_docs.rs Outdated

Comment thread library/std/src/primitive_docs.rs Outdated

est31 reviewed Sep 3, 2023

View reviewed changes

Comment thread library/core/src/primitive_docs.rs Outdated

RalfJung force-pushed the abi-compat-docs branch from 01838d4 to f9bcda9 Compare September 4, 2023 09:46

RalfJung commented Sep 4, 2023

View reviewed changes

Comment thread library/std/src/primitive_docs.rs Outdated

RalfJung added T-lang Relevant to the language team S-waiting-on-team I-lang-nominated Nominated for discussion during a lang team meeting. and removed T-libs Relevant to the library team, which will review and decide on the PR/issue. labels Sep 5, 2023

rfcbot added proposed-final-comment-period Proposed to merge/close by relevant subteam, see T-<team> label. Will enter FCP once signed off. disposition-merge This issue / PR is in PFCP or FCP with a disposition to merge it. labels Sep 5, 2023

add 'import functions' to the list of situations where ABI compatibil…

044d057

…ity comes up

jonhoo reviewed Nov 11, 2023

View reviewed changes

Comment thread library/core/src/primitive_docs.rs Outdated

jonhoo reviewed Nov 11, 2023

View reviewed changes

Comment thread library/core/src/primitive_docs.rs Outdated

RalfJung force-pushed the abi-compat-docs branch 2 times, most recently from 38cdc45 to 9db35cb Compare November 11, 2023 09:57

jonhoo reviewed Nov 11, 2023

View reviewed changes

Comment thread library/core/src/primitive_docs.rs Outdated

clarify ABI compatibility of fn ptr types and ptr types

52d22ea

and add an and

RalfJung force-pushed the abi-compat-docs branch from 9db35cb to 52d22ea Compare November 11, 2023 12:36

Mark-Simulacrum reviewed Nov 17, 2023

View reviewed changes

linking in general has more pitfalls than just call ABI

8f03a55

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Nov 17, 2023

matthiaskrgr mentioned this pull request Nov 17, 2023

Rollup of 3 pull requests #118003

Merged

bors merged commit 1cabedc into rust-lang:master Nov 17, 2023

rustbot added this to the 1.76.0 milestone Nov 17, 2023

RalfJung deleted the abi-compat-docs branch November 18, 2023 07:11

RalfJung mentioned this pull request Dec 17, 2023

do not allow ABI mismatches inside repr(C) types #119037

Merged

traviscross mentioned this pull request May 26, 2026

Handling release notes for language guarantees made in the Reference #156990

Open

RalfJung mentioned this pull request Aug 28, 2026

declare C and C-unwind as mutually ABI-compatible #161904

Merged

	/// concerned with the case where both the caller and the callee are defined in Rust.
	/// concerned with the case where both the caller and the callee are defined in Rust (and using the same compiler version).

Uh oh!

Conversation

RalfJung commented Sep 2, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rustbot commented Sep 2, 2023

Uh oh!

Uh oh!

RalfJung Sep 2, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

RalfJung Sep 2, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

RalfJung Sep 2, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

This comment was marked as resolved.

Uh oh!

BatmanAoD Sep 3, 2023

Choose a reason for hiding this comment

Uh oh!

This comment has been minimized.

This comment has been minimized.

Uh oh!

Uh oh!

This comment was marked as resolved.

Uh oh!

soqb left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

pnkfelix commented Sep 5, 2023

Uh oh!

rfcbot commented Sep 5, 2023 • edited by scottmcm Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

RalfJung commented Sep 5, 2023

Uh oh!

pnkfelix commented Sep 5, 2023

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Mark-Simulacrum left a comment

Choose a reason for hiding this comment

Uh oh!

Mark-Simulacrum Nov 17, 2023

Choose a reason for hiding this comment

Uh oh!

RalfJung Nov 17, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

RalfJung commented Nov 17, 2023

Uh oh!

bors commented Nov 17, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants

RalfJung commented Sep 2, 2023 •

edited

Loading

RalfJung Sep 2, 2023 •

edited

Loading

RalfJung Sep 2, 2023 •

edited

Loading

RalfJung Sep 2, 2023 •

edited

Loading

rfcbot commented Sep 5, 2023 •

edited by scottmcm

Loading

RalfJung Nov 17, 2023 •

edited

Loading