GC live lock

[filipnavara]

Android framework version

net9.0-android, net10.0-android (Preview)

Affected platform version

.NET 9, .NET 10

Description

We have an application that sometimes ends up in a vicious GC cycle live lock. The bug happens when the app goes through rapid UI section switching which results in a large number of Java proxy objects being allocated in a short time. What happens next is the following:

• GRef exhaustion: The number of Java Global References (GRefs) exceeds 90% of the system threshold (approximately > 46,800). Every subsequent GRef allocation (essentially any access to Java objects) triggers a Full GC.
• Full GC triggers a relatively expensive GC Bridge, where garbage collection is synchronized across both .NET and Java. This involves complex logic to reconcile references from one managed heap to the other; roughly 22k objects (give or take) enter this process.
• The GC bridge clears almost nothing, so the GRef count remains critical. The system gets stuck in a loop of constant garbage collection.
• Finalizer Bottleneck: The lack of memory recovery is partly because Java objects have finalizers, all of which run on the Finalizer Thread.
• The Finalizer Thread is preoccupied because every Full GC triggers a Gen2GcCallback, which runs trimming of various ArrayPool<T>.Shared array pools.

This creates a vicious cycle from which the application cannot recover.

Steps to Reproduce

See above. We don't have repro app but we do have GC dumps and sampling profiler traces from the app.

Did you find any workaround?

No, but I believe it may potentially help to call GC.WaitForPendingFinalizers() before GC.Collect() when the GRef threshold is reached here:

android/src/Mono.Android/Android.Runtime/AndroidRuntime.cs

Line 190 in db86d33

System.GC.Collect ();

Relevant log output

[filipnavara]

GC dump and sample profile: gcdump-and-profile.zip
GC log of one of the iterations of full GC: https://gist.github.com/filipnavara/1d24f2d55e8484b29709f2c1b7568aab

[filipnavara]

Notably, we definitely have GREF handle leak somewhere which gets us to the situation in the first place. I haven't been able to track it down yet but the sequence of operations that eventually leads to the failure contains large number of UI changes and we see a growing number of View_OnFocusChangeListenerImplementor instances:

12-18 13:49:52.584 19649 19726 I ient.mailclient: 4016 of mono.android.view.View_OnFocusChangeListenerImplementor (4016 unique instances)

We don't subscribe to the View.FocusChange ourselves, so this presumably comes from MAUI with some external trigger.

[filipnavara]

We tracked down the underlying memory leak to an implementation of the CollectionView handler in MAUI. Under specific circumstances it creates ItemContentView classes that are hooked on the MeasureInvalidated event and not unhooked when destroying the RecycleView adapter or handler itself. In some circumstances it may eventually get garbage collected but in our cases it would cause a chain to remain hooked on a global app theme changed event and never to be freed. This deserves a separate issue in dotnet/maui repository.

However, I am keeping this open since I believe that there's GC.WaitForPendingFinalizers() missing that could reduce the impact of GCs in this rare case. It would not help enough to make a difference in our case, but it could still help in some scenarios.