CODEWITHSUNDEEP
c#inliningnative-aot
user1576055
user1576055

Reputation: 586

Is it possible to increase C# AOT inline method length limit?

When I force the C# AOT compiler with MethodImpl(MethodImplOptions.AggressiveInlining) to do a really long method:

enter image description here

At the very end, it stops inlining and starts calling every possible method (even simple getters):

enter image description here

Is there a way to prevent it from stopping the inlining? Is there some method inline limit that can be increased?

EDIT: I was hoping it wouldn't devolve into a "you shouldn't do that" response. Why I can't just ask the question without justifying? So, here is the justifying. My application is meant to be time performance oriented. I don't care if the compiled code is 10000x bigger. What matters to me is time performance and I can measure about 20% better time performance if this code is inlined.

EDIT2: I've made some reasearch and it seems that there exists configuration knobs for JIT https://github.com/steveharter/dotnet_coreclr/blob/master/Documentation/project-docs/clr-configuration-knobs.md Specifically there is a JITInlineSize knob. I've tried to set its value via ENV variable, but it doesn't help.

EDIT3: I've made a reproducible example. Please see https://github.com/dagid4/AotInlineLimitProof

Upvotes: 2

Views: 325

Answers (2)

user1576055
user1576055

Reputation: 586

It seems that it is not possible to increase the inline limit.

https://github.com/dotnet/runtime/issues/93393

There is no supported way to do this.

The problem is known as the Inliner's time budget and may be reconsidered in .NET 9.0.

https://github.com/dotnet/runtime/issues/93069

Once inliner runs out of its Time Budget it starts to even ignore [AggressiveInlining] attributes which breaks user's assumptions and produces slower code. The idea is to re-think how it's calculated/checked and, possibly, redesign inliner to start from hot blocks first. The beneficiaries are:

  1. Library folks sometimes have to re-shuffle code or use NoInlining to make inliner happy - we should improve that experience
  2. 1P/3P code should hit less often and gain performance improvements from that

The (1) actually describes my problem. I'm creating a library which I want to be fast. I have already re-shuffled code and made some methods NoInlining. But still, I'm afraid that in some cases, the time budget may be hit and from that point on, the performance will be broken.

I hope for better solution in future .NET, for example:

https://github.com/dotnet/runtime/pull/81565#issuecomment-1415987399

It seems we can think about doing a proper rejection in JIT instead. Imagine we have a callgraph like this:

A()
    B()
        C()
        D()
        E()
        F()

B() has [AggressiveInlining] and it eats the whole inliner's budget (because B() has a lot of IL) - so we successfully inline B() into A() but then we have no budget for C()...F() despite them being small and super useful to inline. In this case we should sort of go back and reject inlining decision of B() into A() so we can optimize B() properly.

Upvotes: 1

Corey
Corey

Reputation: 16584

Pretty sure that's not how that works.

The MethodImplAttribute applies to your method, and AggressiveInlining tells the JIT (and thus AOT) that it should inline your method if possible. If your method is too large - based apparently on the number of bytes it compiles to - then it will be called normally rather than inlined.

Inlining can save a few cycles per invocation, and will happen automatically for trivial methods when optimization is enabled. AggressiveInlining raises the threshold for inlining but doesn't guarantee that it will happen, or that the output code will be any better.

Here's a direct quote from the docs:

Unnecessary use of this attribute can reduce performance. The attribute might cause implementation limits to be encountered that will result in slower generated code.

And that's what's happening here.

You've told the compiler that it should do what it can to inline your "really long" (whatever that means) method, and it does its best to comply. Your method apparently calls a bunch of other methods that would normally be inlined, but doing so will make your method exceed the inlining limit. Since you've insisted that it be inlined, and inlining those other methods would prevent that, the compiler is balancing the books in favor of what you told it to do. If inlining those 'simple getter' calls would make your method too large to inline, they don't get inlined.

AggressiveInlining should be used rarely if at all. If you look at the runtime source you'll find that the vast majority of cases where the standard libraries use it, it's a performance tweak for commonly-called methods that would be inlined under normal circumstances anyway.

For example, AsyncTaskMethodBuilder has this:

public Task Task
{
    [MethodImpl(MethodImplOptions.AggressiveInlining)]
    get => m_task ?? InitializeTaskAsPromise();
}

That's a trivial getter that would normally be inlined, except in cases like yours where the JIT is already running low on space. It's important that accessing the Task be as quick as possible though, thus the attribute. It should take priority over optimizations because it's a definite win to inline this.

What it's not intended for is what you're trying to do: force the JIT to duplicate large chunks of your code inline.

Upvotes: 2

Related Questions