Skip to content

Conversation

@ionicmc-rs
Copy link

@ionicmc-rs ionicmc-rs commented Jan 21, 2026

Allows for creating custom ABIs in Rust.

this should still map to something that works in LLVM, the exact details are still to be decided

Important

When responding to RFCs, try to use inline review comments (it is possible to leave an inline review comment for the entire file at the top) instead of direct comments for normal comments and keep normal comments for procedural matters like starting FCPs.

This keeps the discussion more organized.

Rendered

@ehuss ehuss added T-lang Relevant to the language team, which will review and decide on the RFC. T-compiler Relevant to the compiler team, which will review and decide on the RFC. labels Jan 22, 2026
Comment on lines +38 to +55
unsafe impl Abi for CAbi { // unsafe impl
// might use slices instead.
const REGISTER_COUNT: u8 = 6;
const FLOAT_REGISTER_COUNT: u8 = 2;

const INPUT_REGISTERS: [Register; Self::REGISTER_COUNT] = [
Register::Rdi,
// ...
];
const INPUT_FLOATS: [Register; Self::FLOAT_REGISTER_COUNT] = [
Register::Xmm0,
// ...
]

const STACK_ALIGN: usize = 16;

// more ABI fields here.
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This just isn't enough to represent anything more than trivial ABIs. Different types may have different stack alignment, different float/int sizes may get passed in different registers, values may be split across registers, ADTs need to be figured out, etc...

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I also thought that could be a problem.

I'm not an expert on what an ABI really does, but there should be some possible way to allow for ABI defs in rust.

I'm going to try and gather a bit more information and adjust the implementation

Copy link
Author

@ionicmc-rs ionicmc-rs Jan 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok i have thought of a better way to do this, im going to say it here before changing the RFC:

We can have special "abi" crates (like proc-macro) that can only export ABI defs.

Example:

Cargo.toml

[lib]
abi = true
# proc-macro = true
# this fails

lib.rs:

extern crate abi;

use abi::{Alignment, AbiTable, BasicType}; // basic types are unsigned interger

#[abi(align_for /*, method specific options */)]
pub fn align_for(ty: BasicType) -> Alignment {
    match ty {
         // ... 
         BasicType::U8 => Alignment::One
    }
}

#[abi(table, name="custom_abi")]
pub const CUSTOM_ABI: AbiTable = AbiTable::new()
     .set_align_for(ty);

Usage:

use abi_crate::CUSTOM_ABI; // required import

extern "custom_abi" fn do_nothing() {}

This offers more functionality, while keeping familiar API to users of Rust

I want to state again that this may not be the best way, and that i am not an expert.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not an expert on what an ABI really does

You should not be writing a proposal for a full ABI definition framework which is permanently stable and cannot be changed unless you know what an ABI is and what it does. This doesn't mean that the idea doesn't have merit, but as someone who also does not fully understand ABIs, I at least am past the Dunning-Kruger curve enough to realise that defining an ABI using a simple trait does not seem sufficient in the slightest.

Not even LLVM seems to even attempt to tackle this goal. What makes you think that Rust can and should?

Just take a brief look at the closest similar proposal, to add crABI, and see if your proposal covers 1% of the requirements or answers 1% of the questions left open: rust-lang/compiler-team#631

@@ -0,0 +1,189 @@
- Feature Name: `abi_descriptors`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I kind of hinted at this in my other comment but IMO, any sort of proposal to express ABI in code is a non-starter. There is an immense amount of complexity (see all the LLVM tablegen files related to ABI) and little reason not to do things in assembly.

There are certainly problems with handwritten assembly that could be improved if you are interested, but I think this is better framed as “how can we make it easier to integrate handwritten assembly (like trampolines) with normal code” rather than trying to, effectively, get rustc to emit a very specific assembly without writing it.

Copy link
Member

@workingjubilee workingjubilee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LLVMIR, despite being referred to as "portable assembly" by some, can be remarkably weak around the less-portable parts. It offers no contract that allows describing the actual calling convention you desire instead of selecting one of its built-in sets of rules and hoping that works the way you think it will based on the function signature you provide.

Thus, this is functionally impossible to implement, except via using inline assembly. At that point you are accepting the significant constraints of inline assembly, but are at least simply asking for inline assembly, which we have.

The other backends do not seem more capable in this regard.

Comment on lines +29 to +38
there are 2 possible ways this freature may be implemented.

### Trait
Example for C ABI
```rust
use core::abi::{Abi, Register}; // in core

struct CAbi;

unsafe impl Abi for CAbi { // unsafe impl
Copy link
Member

@workingjubilee workingjubilee Jan 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You simply cannot ergonomically describe an ABI in a single trait, because it is a matter of dire amounts of polymorphism.

Speaking from the general view, even if we consider "how to handle an ABI" as a sort of stack machine-ish thing with multiple stacks for each type of register (so the machine mostly is reducing the program-argument stack while picking which machine-argument stack to push values into, essentially), a programmer-visible ABI is not required to translate to what we would consider a single mechanical implementation of a calling convention. It may select a different mechanical calling convention after considering the total function signature, which can have different stack or register usage rules.

Often these rules are not easily described via simple constants, but instead must be modeled like executable code, e.g. some ABIs will insert "fake" arguments at various points, considering numerous small rules, to handle certain alignment constraints.

This is not necessarily described entirely by stack alignment as a constraint, because an ABI may wish to consider "register alignment", which is not the same as stack alignment, and not the same as type alignment, but comes up quite often in actual CPU ISAs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

T-compiler Relevant to the compiler team, which will review and decide on the RFC. T-lang Relevant to the language team, which will review and decide on the RFC.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants