WithExpressionExpanding not expanding LINQ queries properly

[JKamsker]

I'm experiencing an issue with WithExpressionExpanding where it does not properly expand LINQ queries. The application throws a System.InvalidOperationException when using a custom expression within a query. However, manually expanding the expression using Expand() works as expected.

Steps to Reproduce

Here's a simplified example that demonstrates the issue:

1. Create a custom expression:

public class Version
{
    public int Id { get; set; }
    public int Major { get; set; }
    public int Minor { get; set; }
    public int Patch { get; set; }
    public string Tag { get; set; }
}

public class ProductVersion
{
    public int Id { get; set; }
    public Version Version { get; set; }
}

Expression<Func<Version, Version, bool>> IsEqual = 
            (version1, version2) => (version1.Major == version2.Major && version1.Minor == version2.Minor && version1.Patch == version2.Patch && version1.Tag == version2.Tag);

Expression<Func<ProductVersion, bool>> b = x => IsEqual.Invoke(x.Version, toFind);

Configure EF Core with `WithExpressionExpanding`:

  // Configure EF Core
  builder.Services.AddDbContextPool<UpdateServerContext>
  (
      options => options
          .UseSqlServer(connectionString, sql => { sql.EnableRetryOnFailure(10); })
          .EnableSensitiveDataLogging(false)
          .WithExpressionExpanding()
  );

Execute the query:

  var result = ctx.ProductVersions
    .Where(b)
    .ToList();

Expected Behavior

The query should execute successfully and return the expected results.

Actual Behavior

A System.InvalidOperationException is thrown:

System.InvalidOperationException
  HResult=0x80131509
  Message=The LINQ expression 'DbSet<ProductVersion>()
    .Join(
        inner: DbSet<Version>(), 
        outerKeySelector: p => EF.Property<int?>(p, "VersionId"), 
        innerKeySelector: a => EF.Property<int?>(a, "Id"), 
        resultSelector: (o, i) => new TransparentIdentifier<ProductVersion, Version>(
            Outer = o, 
            Inner = i
        ))
    .Where(p => p.Inner.Major == __toFind_0.Major && p.Inner.Minor == __toFind_0.Minor && p.Inner.Patch == __toFind_0.Patch && p.Inner.Tag == __toFind_0.Tag)' could not be translated. Either rewrite the query in a form that can be translated, or switch to client evaluation explicitly by inserting a call to 'AsEnumerable', 'AsAsyncEnumerable', 'ToList', or 'ToListAsync'. See https://go.microsoft.com

Workaround 1

Manually expanding the expression using Expand() works:

var expanded = b.Expand();
var result = ctx.ProductVersions
  .Where(expanded)
  .ToList();

Workaround 2

Using the AsExpandable() extension method works:

var result = ctx.ProductVersions
  .AsExpandable()
  .Where(b)
  .ToList();

Environment

• LinqKit.Microsoft.EntityFrameworkCore version: 7.1.4
• .NET version: 6
• Microsoft.EntityFrameworkCore: 7.0.5
• Database provider: Microsoft.EntityFrameworkCore.SqlServer
• Operating system: Windows 10

[JKamsker]

Maybe @sdanyliv can help

[SpaceOgre]

If you are using EF Core 7 or higher you can create your own query interceptor that handles the expansion:

public class ExpandingQueryExpressionInterceptor : IQueryExpressionInterceptor
{
    public Expression QueryCompilationStarting(Expression queryExpression, QueryExpressionEventData eventData)
        => queryExpression.Expand();
}

Then just register it like any other interceptor:

services.AddDbContext<MyDbContext>(options =>
    options.UseSqlServer(
        configuration.GetConnectionString(nameof(MyDbContext)),
        x => x.MigrationsAssembly(Assembly.GetAssembly(typeof(MyDbContext))?.FullName)
    )
    .AddInterceptors(new ExpandingQueryExpressionInterceptor()) //<-- Add this line
);

It might be good to change the WithExpressionExpanding to use something like this for EF Core 7 and above, since it is way less code to write.

[SpaceOgre]

This seem to not work as well as the .AsExpanded() though I'm getting some errors:

System.InvalidOperationException: variable '__EntityId_1' of type 'System.Nullable`1[System.Guid]' referenced from scope '', but it is not defined

But the same problem exists with the WithExpressionExpanding, so it might be something to do with what EF core does with expression before sending it on in the pipe. Or that the ExpressionExpanding wrapper for the IQueryable is doing something more than the Expand() method with regards to scope/expression replacing.

[sdanyliv]

I can help. But I still cannot compile this huge solution with a lot of outdated stuff.

[ClassTerr]

I have exactly the same issue.
As I understand, the problem lies in the EF core pipeline.
The rough EF core pipeline is:

1. Receive query via materialization methods (.ToList(), .Count(), etc.) or foreach loop.
2. Parameter extraction
3. Check if the same query (with extracted parameters) is already compiled
   4.1. If so, just use it
   4.2. Otherwise, call all IQueryExpressionInterceptor, compile and store to cache

When using WithExpressionExpanding, the interceptor receives an expression tree with already extracted parameters,
but when using AsExpandable the expression tree is still not preprocessed by the EF Core.

To me, the current implementation of WithExpressionExpanding is error-prone and I wasted a few hours of my time to investigate.
I think there should be a disclosure that this is problematic.

However, I can't see how it is possible to implement it, since EF core does not expose a hook that works before query parameter extraction. IMO, for most use cases these parameters are crucial for such dynamic queries.

All we can hope for is that it will work with simple parameters like string, int, etc.
For my use-case, I wanted to pass an object as a parameter, and it is failed.

For now, I can't see other solution except to use .AsExpandable() everywhere, because EF Core team refuses to expose the hook.