Skip to content

Move RewriteMarshalMethods to the inner (per-RID) build #11066

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
sbomer wants to merge 2 commits into
base: main
Choose a base branch
from
+137 −82
Open

Move RewriteMarshalMethods to the inner (per-RID) build #11066

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
e0d2900
[build] Move RewriteMarshalMethods to the inner (per-RID) build
sbomer Apr 1, 2026
0ec9a28
[build] Fix ConvertedMarshalMethodEntry NativeCallback returning wrap…
sbomer Apr 1, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 20 additions & 7 deletions ...id.Build.Tasks/Microsoft.Android.Sdk/targets/Microsoft.Android.Sdk.TypeMap.LlvmIr.targets
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -78,13 +78,6 @@
EnableNativeRuntimeLinking="$(_AndroidEnableNativeRuntimeLinking)">
</GenerateJavaStubs>

<RewriteMarshalMethods
Condition=" '$(_AndroidUseMarshalMethods)' == 'true' And '$(AndroidIncludeDebugSymbols)' == 'false' "
EnableManagedMarshalMethodsLookup="$(_AndroidUseManagedMarshalMethodsLookup)"
Environments="@(_EnvironmentFiles)"
IntermediateOutputDirectory="$(IntermediateOutputPath)">
</RewriteMarshalMethods>

<GenerateTypeMappings
AndroidRuntime="$(_AndroidRuntime)"
Debug="$(AndroidIncludeDebugSymbols)"
Expand Down Expand Up @@ -250,6 +243,26 @@
Deterministic="$(Deterministic)" />
</Target>

<!--
Rewrite assemblies to use marshal methods (static native callbacks with [UnmanagedCallersOnly])
instead of dynamic JNI registration. Runs in the inner (per-RID) build after ILLink and
_PostTrimmingPipeline but before ReadyToRun/crossgen2 or NativeAOT compilation, so these
tools operate on the already-rewritten assemblies.
-->
<Target Name="_RewriteMarshalMethods"
AfterTargets="_PostTrimmingPipeline"
BeforeTargets="CreateReadyToRunImages;IlcCompile"
Condition=" '$(_AndroidUseMarshalMethods)' == 'true' and '$(AndroidIncludeDebugSymbols)' == 'false' and '$(PublishTrimmed)' == 'true' ">
<ItemGroup>
<_RewriteMarshalMethodsAssembly Include="@(ResolvedFileToPublish)" Condition=" '%(Extension)' == '.dll' " />
</ItemGroup>
<RewriteMarshalMethods
Assemblies="@(_RewriteMarshalMethodsAssembly)"
EnableManagedMarshalMethodsLookup="$(_AndroidUseManagedMarshalMethodsLookup)"
Environments="@(AndroidEnvironment);@(LibraryEnvironments)"
RuntimeIdentifier="$(RuntimeIdentifier)" />
</Target>

<!-- Inject _TypeMapKind into the property cache -->
<Target Name="_SetTypemapProperties"
BeforeTargets="_CreatePropertiesCache">
Expand Down
162 changes: 89 additions & 73 deletions src/Xamarin.Android.Build.Tasks/Tasks/RewriteMarshalMethods.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
#nullable enable
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.Linq;
using Java.Interop.Tools.Cecil;
using Microsoft.Android.Build.Tasks;
using Microsoft.Build.Framework;
using Xamarin.Android.Tools;
Expand All @@ -13,15 +14,22 @@ namespace Xamarin.Android.Tasks;
/// attributes, significantly improving startup performance and reducing runtime overhead for Android applications.
/// </summary>
/// <remarks>
/// This task operates on the marshal method classifications produced by earlier pipeline stages and:
///
/// 1. Retrieves marshal method classifications from the build pipeline state
/// 2. Parses environment files to determine exception transition behavior
/// 3. Rewrites assemblies to replace dynamic registration with static marshal methods
/// 4. Optionally builds managed lookup tables for runtime marshal method resolution
/// 5. Reports statistics on marshal method generation and any fallback to dynamic registration
///
/// The rewriting process creates native callback wrappers for methods that have non-blittable
/// This task runs in the inner (per-RID) build after ILLink and <c>_PostTrimmingPipeline</c> but before
/// <c>CreateReadyToRunImages</c> or <c>IlcCompile</c>. It is fully self-contained: it creates its own
/// assembly resolver, scans assemblies for marshal method candidates, classifies them, and rewrites them
/// in a single pass.
///
/// The rewriting process:
///
/// 1. Derives the target architecture from the <see cref="RuntimeIdentifier"/>
/// 2. Creates an assembly resolver with ReadWrite+InMemory mode for Cecil
/// 3. Scans assemblies to discover marshal method candidates via <see cref="MarshalMethodsCollection.FromAssemblies"/>
/// 4. Parses environment files to determine exception transition behavior
/// 5. Rewrites assemblies to replace dynamic registration with static marshal methods
/// 6. Optionally builds managed lookup tables for runtime marshal method resolution
/// 7. Reports statistics on marshal method generation and any fallback to dynamic registration
///
/// The rewriting creates native callback wrappers for methods that have non-blittable
/// parameters or return types, ensuring compatibility with the [UnmanagedCallersOnly] attribute
/// while maintaining proper marshaling semantics.
/// </remarks>
Expand All @@ -32,6 +40,13 @@ public class RewriteMarshalMethods : AndroidTask
/// </summary>
public override string TaskPrefix => "RMM";

/// <summary>
/// Gets or sets the resolved assemblies to scan and rewrite.
/// These are the post-ILLink assemblies from <c>ResolvedFileToPublish</c> in the inner build.
/// </summary>
[Required]
public ITaskItem [] Assemblies { get; set; } = [];

/// <summary>
/// Gets or sets whether to enable managed marshal methods lookup tables.
/// When enabled, generates runtime lookup structures that allow dynamic resolution
Expand All @@ -47,42 +62,53 @@ public class RewriteMarshalMethods : AndroidTask
public ITaskItem [] Environments { get; set; } = [];

/// <summary>
/// Gets or sets the intermediate output directory path. Required for retrieving
/// build state objects that contain marshal method classifications.
/// Gets or sets the runtime identifier for this inner build (e.g., "android-arm64").
/// Used to derive the target architecture and ABI for assembly scanning and rewriting.
/// </summary>
[Required]
public string IntermediateOutputDirectory { get; set; } = "";
public string RuntimeIdentifier { get; set; } = "";

/// <summary>
/// Executes the marshal method rewriting task. This is the main entry point that
/// coordinates the entire assembly rewriting process across all target architectures.
/// coordinates the entire assembly rewriting process for the current target architecture.
/// </summary>
/// <returns>
/// true if the task completed successfully; false if errors occurred during processing.
/// </returns>
/// <remarks>
/// The execution flow is:
///
/// 1. Retrieve native code generation state from previous pipeline stages
/// 2. Parse environment files for configuration (e.g., broken exception transitions)
/// 3. For each target architecture:
/// - Rewrite assemblies to use marshal methods
/// - Add special case methods (e.g., TypeManager methods)
/// - Optionally build managed lookup tables
/// 4. Report statistics on marshal method generation
/// 5. Log warnings for methods that must fall back to dynamic registration
///
///
/// 1. Derive the target architecture and ABI from the RuntimeIdentifier
/// 2. Build a dictionary of assemblies and set Abi metadata on each item
/// 3. Create an assembly resolver with ReadWrite+InMemory mode
/// 4. Scan assemblies and classify marshal methods via MarshalMethodsCollection.FromAssemblies
/// 5. Parse environment files for configuration (e.g., broken exception transitions)
/// 6. Rewrite assemblies to use marshal methods
/// 7. Add special case methods (e.g., TypeManager methods)
/// 8. Optionally build managed lookup tables
/// 9. Report statistics on marshal method generation
/// 10. Log warnings for methods that must fall back to dynamic registration
///
/// The task handles the ordering dependency between special case methods and managed
/// lookup tables - special cases must be added first so they appear in the lookup tables.
/// </remarks>
public override bool RunTask ()
{
// Retrieve the stored NativeCodeGenState from the build pipeline
// This contains marshal method classifications from earlier stages
var nativeCodeGenStates = BuildEngine4.GetRegisteredTaskObjectAssemblyLocal<ConcurrentDictionary<AndroidTargetArch, NativeCodeGenState>> (
MonoAndroidHelper.GetProjectBuildSpecificTaskObjectKey (GenerateJavaStubs.NativeCodeGenStateRegisterTaskKey, WorkingDirectory, IntermediateOutputDirectory),
RegisteredTaskObjectLifetime.Build
);
AndroidTargetArch arch = MonoAndroidHelper.RidToArch (RuntimeIdentifier);
string abi = MonoAndroidHelper.RidToAbi (RuntimeIdentifier);

// Build the assemblies dictionary and set Abi metadata required by XAJavaTypeScanner
var assemblies = new Dictionary<string, ITaskItem> (Assemblies.Length);
foreach (var asm in Assemblies) {
asm.SetMetadata ("Abi", abi);
assemblies [asm.ItemSpec] = asm;
}

// Create a resolver with ReadWrite+InMemory mode so Cecil can modify assemblies in place
using XAAssemblyResolver resolver = MonoAndroidHelper.MakeResolver (Log, useMarshalMethods: true, arch, assemblies);

// Scan and classify marshal methods
MarshalMethodsCollection classifier = MarshalMethodsCollection.FromAssemblies (arch, [.. assemblies.Values], resolver, Log);

// Parse environment files to determine configuration settings
// We need to parse the environment files supplied by the user to see if they want to use broken exception transitions. This information is needed
Expand All @@ -91,42 +117,33 @@ public override bool RunTask ()
var environmentParser = new EnvironmentFilesParser ();
bool brokenExceptionTransitionsEnabled = environmentParser.AreBrokenExceptionTransitionsEnabled (Environments);

// Process each target architecture
foreach (var kvp in nativeCodeGenStates) {
NativeCodeGenState state = kvp.Value;

if (state.Classifier is null) {
Log.LogError ("state.Classifier cannot be null if marshal methods are enabled");
return false;
}

// Handle the ordering dependency between special case methods and managed lookup tables
if (!EnableManagedMarshalMethodsLookup) {
// Standard path: rewrite first, then add special cases
RewriteMethods (state, brokenExceptionTransitionsEnabled);
state.Classifier.AddSpecialCaseMethods ();
} else {
// Managed lookup path: add special cases first so they appear in lookup tables
// We need to run `AddSpecialCaseMethods` before `RewriteMarshalMethods` so that we can see the special case
// methods (such as TypeManager.n_Activate_mm) when generating the managed lookup tables.
state.Classifier.AddSpecialCaseMethods ();
state.ManagedMarshalMethodsLookupInfo = new ManagedMarshalMethodsLookupInfo (Log);
RewriteMethods (state, brokenExceptionTransitionsEnabled);
}

// Report statistics on marshal method generation
Log.LogDebugMessage ($"[{state.TargetArch}] Number of generated marshal methods: {state.Classifier.MarshalMethods.Count}");
if (state.Classifier.DynamicallyRegisteredMarshalMethods.Count > 0) {
Log.LogWarning ($"[{state.TargetArch}] Number of methods in the project that will be registered dynamically: {state.Classifier.DynamicallyRegisteredMarshalMethods.Count}");
}

// Count and report methods that need blittable workaround wrappers
var wrappedCount = state.Classifier.MarshalMethods.Sum (m => m.Value.Count (m2 => m2.NeedsBlittableWorkaround));

if (wrappedCount > 0) {
// TODO: change to LogWarning once the generator can output code which requires no non-blittable wrappers
Log.LogDebugMessage ($"[{state.TargetArch}] Number of methods in the project that need marshal method wrappers: {wrappedCount}");
}
// Handle the ordering dependency between special case methods and managed lookup tables
ManagedMarshalMethodsLookupInfo? managedLookupInfo = null;
if (!EnableManagedMarshalMethodsLookup) {
// Standard path: rewrite first, then add special cases
RewriteAssemblies (arch, classifier, resolver, managedLookupInfo, brokenExceptionTransitionsEnabled);
classifier.AddSpecialCaseMethods ();
} else {
// Managed lookup path: add special cases first so they appear in lookup tables
// We need to run `AddSpecialCaseMethods` before `RewriteMarshalMethods` so that we can see the special case
// methods (such as TypeManager.n_Activate_mm) when generating the managed lookup tables.
classifier.AddSpecialCaseMethods ();
managedLookupInfo = new ManagedMarshalMethodsLookupInfo (Log);
RewriteAssemblies (arch, classifier, resolver, managedLookupInfo, brokenExceptionTransitionsEnabled);
}
Comment on lines +120 to +133
Copy link

Copilot AI Apr 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

EnableManagedMarshalMethodsLookup currently builds a ManagedMarshalMethodsLookupInfo instance (managedLookupInfo) but it’s not persisted anywhere after rewriting. Downstream native codegen relies on NativeCodeGenState.ManagedMarshalMethodsLookupInfo to populate AssemblyIndex/ClassIndex/MethodIndex (and will throw if missing), so managed lookup builds can regress. Consider emitting the lookup info (or the computed indexes) as a task output/file and rehydrating it where NativeCodeGenStateObject is created (or recomputing it from the rewritten assemblies).

Copilot uses AI. Check for mistakes.

// Report statistics on marshal method generation
Log.LogDebugMessage ($"[{arch}] Number of generated marshal methods: {classifier.MarshalMethods.Count}");
if (classifier.DynamicallyRegisteredMarshalMethods.Count > 0) {
Log.LogWarning ($"[{arch}] Number of methods in the project that will be registered dynamically: {classifier.DynamicallyRegisteredMarshalMethods.Count}");
}

// Count and report methods that need blittable workaround wrappers
var wrappedCount = classifier.MarshalMethods.Sum (m => m.Value.Count (m2 => m2.NeedsBlittableWorkaround));

if (wrappedCount > 0) {
// TODO: change to LogWarning once the generator can output code which requires no non-blittable wrappers
Log.LogDebugMessage ($"[{arch}] Number of methods in the project that need marshal method wrappers: {wrappedCount}");
}

return !Log.HasLoggedErrors;
Expand All @@ -137,7 +154,10 @@ public override bool RunTask ()
/// Creates and executes the <see cref="MarshalMethodsAssemblyRewriter"/> that handles
/// the low-level assembly modification operations.
/// </summary>
/// <param name="state">The native code generation state containing marshal method classifications and resolver.</param>
/// <param name="arch">The target Android architecture.</param>
/// <param name="classifier">The marshal methods classifier containing method classifications.</param>
/// <param name="resolver">The assembly resolver used to load and resolve assemblies.</param>
/// <param name="managedLookupInfo">Optional managed marshal methods lookup info for building lookup tables.</param>
/// <param name="brokenExceptionTransitionsEnabled">
/// Whether to generate code compatible with broken exception transitions.
/// This affects how wrapper methods handle exceptions during JNI calls.
Expand All @@ -150,13 +170,9 @@ public override bool RunTask ()
/// - Modifying assembly references and imports
/// - Building managed lookup table entries
/// </remarks>
void RewriteMethods (NativeCodeGenState state, bool brokenExceptionTransitionsEnabled)
void RewriteAssemblies (AndroidTargetArch arch, MarshalMethodsCollection classifier, XAAssemblyResolver resolver, ManagedMarshalMethodsLookupInfo? managedLookupInfo, bool brokenExceptionTransitionsEnabled)
{
if (state.Classifier == null) {
return;
}

var rewriter = new MarshalMethodsAssemblyRewriter (Log, state.TargetArch, state.Classifier, state.Resolver, state.ManagedMarshalMethodsLookupInfo);
var rewriter = new MarshalMethodsAssemblyRewriter (Log, arch, classifier, resolver, managedLookupInfo);
rewriter.Rewrite (brokenExceptionTransitionsEnabled);
}
}
30 changes: 28 additions & 2 deletions src/Xamarin.Android.Build.Tasks/Utilities/MarshalMethodCecilAdapter.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -68,24 +68,50 @@ static NativeCodeGenStateObject CreateNativeCodeGenState (AndroidTargetArch arch
obj.MarshalMethods.Add (group.Key, methods);
}

// Also include methods that were already rewritten in the inner build.
// After the inner build rewrites assemblies, the outer build's GenerateJavaStubs
// re-classifies them and puts already-rewritten methods (with _mm_wrapper +
// [UnmanagedCallersOnly]) into ConvertedMarshalMethods instead of MarshalMethods.
// We must include these so that GenerateNativeMarshalMethodSources produces
// correct native callback tables.
foreach (var group in state.Classifier.ConvertedMarshalMethods) {
if (!obj.MarshalMethods.TryGetValue (group.Key, out var methods)) {
methods = new List<MarshalMethodEntryObject> (group.Value.Count);
obj.MarshalMethods.Add (group.Key, methods);
}

foreach (var method in group.Value) {
var entry = CreateEntry (method, state.ManagedMarshalMethodsLookupInfo);
methods.Add (entry);
}
}
Comment on lines +71 to +87
Copy link

Copilot AI Apr 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ConvertedMarshalMethods entries are now merged into obj.MarshalMethods, but CreateEntry will only populate AssemblyIndex/ClassIndex/MethodIndex when state.ManagedMarshalMethodsLookupInfo is non-null. With RewriteMarshalMethods moved out of the outer-build state object, this lookup info is likely to be null, which will break managed lookup native codegen (MarshalMethodsNativeAssemblyGenerator throws when indexes are missing). Consider rebuilding ManagedMarshalMethodsLookupInfo from ConvertedMarshalMethods here (or ensuring it is propagated into state before adaptation).

Copilot uses AI. Check for mistakes.

return obj;
}

static MarshalMethodEntryObject CreateEntry (MarshalMethodEntry entry, ManagedMarshalMethodsLookupInfo? info)
{
// For converted entries, use the wrapper method (ConvertedNativeCallback) for native
// callback tables — it has the correct MetadataToken and [UnmanagedCallersOnly] attribute.
// For regular entries, NativeCallback already returns the wrapper after the rewriter sets
// NativeCallbackWrapper.
MethodDefinition nativeCallbackMethod = entry is ConvertedMarshalMethodEntry converted
? converted.ConvertedNativeCallback
: entry.NativeCallback;

var obj = new MarshalMethodEntryObject (
declaringType: CreateDeclaringType (entry.DeclaringType),
implementedMethod: CreateMethod (entry.ImplementedMethod),
isSpecial: entry.IsSpecial,
jniTypeName: entry.JniTypeName,
jniMethodName: entry.JniMethodName,
jniMethodSignature: entry.JniMethodSignature,
nativeCallback: CreateMethod (entry.NativeCallback),
nativeCallback: CreateMethod (nativeCallbackMethod),
registeredMethod: CreateMethodBase (entry.RegisteredMethod)
);

if (info is not null) {
(uint assemblyIndex, uint classIndex, uint methodIndex) = info.GetIndex (entry.NativeCallback);
(uint assemblyIndex, uint classIndex, uint methodIndex) = info.GetIndex (nativeCallbackMethod);

obj.NativeCallback.AssemblyIndex = assemblyIndex;
obj.NativeCallback.ClassIndex = classIndex;
Expand Down
Loading