System.RuntimeMethodHandle.InvokeMethod error in 4 GB + Model

[harish2020Code]

Is there any known limitation with model size? I build the multiple models with different range of the data to find out which one is giving the better result. I have a model with 2 GB size and everything works fine and when I use 4 GB+ model and I am getting the following

Line of the code - ITransformer mlModel = mlContext.Model.Load(modelPath, out var modelInputSchema);

Error
December 13, 2024 08:49:24,476 [MLCodeModule] ERROR - Error PredictionEngine at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor)
at System.Reflection.RuntimeMethodInfo.UnsafeInvokeInternal(Object obj, Object[] parameters, Object[] arguments)
at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
at Microsoft.ML.Runtime.ComponentCatalog.LoadableClassInfo.CreateInstanceCore(Object[] ctorArgs)
at Microsoft.ML.Runtime.ComponentCatalog.TryCreateInstance[TRes](IHostEnvironment env, Type signatureType, TRes& result, String name, String options, Object[] extra)
at Microsoft.ML.Runtime.ComponentCatalog.TryCreateInstance[TRes,TSig](IHostEnvironment env, TRes& result, String name, String options, Object[] extra)
at Microsoft.ML.ModelLoadContext.TryLoadModelCore[TRes,TSig](IHostEnvironment env, TRes& result, Object[] extra)
at Microsoft.ML.ModelLoadContext.TryLoadModel[TRes,TSig](IHostEnvironment env, TRes& result, RepositoryReader rep, Entry ent, String dir, Object[] extra)
at Microsoft.ML.ModelLoadContext.LoadModel[TRes,TSig](IHostEnvironment env, TRes& result, RepositoryReader rep, Entry ent, String dir, Object[] extra)
at Microsoft.ML.ModelLoadContext.LoadModelOrNull[TRes,TSig](IHostEnvironment env, TRes& result, RepositoryReader rep, String dir, Object[] extra)
at Microsoft.ML.ModelLoadContext.LoadModel[TRes,TSig](IHostEnvironment env, TRes& result, RepositoryReader rep, String dir, Object[] extra)
at Microsoft.ML.ModelOperationsCatalog.Load(Stream stream, DataViewSchema& inputSchema)
at Microsoft.ML.ModelOperationsCatalog.Load(String filePath, DataViewSchema& inputSchema)
at MLClassification.ConsumeModel.CreatePredictionEngine()

Please let me know if any other information is needed?

Thanks
Harish

[michaelgsharp]

I don't believe there should be an issue with models of that size (we use a Stream and its limit is... large). Granted, I also don't know if we have tested a single model that is that large. We will have to test more. Can you share how you created your specific model?
Did the model train/save just fine?

[dotnet-policy-service]

This issue has been marked needs-author-action and may be missing some important information.

[dotnet-policy-service]

This issue has been automatically marked no-recent-activity because it has not had any activity for 14 days. It will be closed if no further activity occurs within 14 more days. Any new comment (by anyone, not necessarily the author) will remove no-recent-activity.