IEnumerable and IQueryable - the best of both worlds

Question

For performance reasons many of my services are returning IQueryables right now, so the caller can decide what elements to filter before they get materialized. For design reasons (interfaces etc.) I'd rather like to return IEnumerables but that sure removes the advantages of IQueryables: materialization of the complete result set plus no -Async methods.

I've started writing some extension methods that prefer IQueryable over IEnumerable, e.g.:

public static IEnumerable<TSource> Where<TSource>(this IEnumerable<TSource> source, Expression<Func<TSource, bool>> predicate)
    => source is IQueryable<TSource> ? Queryable.Where(source as IQueryable<TSource>, predicate) : Enumerable.Where(source, predicate.Compile());

public static Task<TSource[]> ToArrayAsync<TSource>(this IEnumerable<TSource> source)
    => (source as IQueryable<TSource>)?.ToArrayAsync() ?? Task.FromResult(source.ToArray());

I can't be the only one thinking about this. Any improvements for these methods? Is there already a good collection / solution out there? Or is this a bad design? So please tell me reasons.

Thank you!

Answer 1

Personally it would make me cautious; sequences and queries are similar but different enough that I very much want to be sure how it is being handled. There is also some non-trivial overhead involved in generating an expression tree and then compiling it, vs just using a direct compiler-generated method on the capture-context.the overhead of the expression tree is justifiable if it is going to be used as part of query composition and translation, but isn't needed in the sequence case.

Perhaps a more obvious approach would be to use AsQueryable() on a sequence to simulate a true query.

But: what you have should work, I guess. As a minor syntactic note, in recent C# versions you can capture the typed value as part of is to avoid a double cast:

public static IEnumerable<TSource> Where<TSource>(this IEnumerable<TSource> source, Expression<Func<TSource, bool>> predicate)
    => source is IQueryable<TSource> q ? Queryable.Where(q, predicate) : Enumerable.Where(source, predicate.Compile());

It may also be interesting to experiment with the boolean flag on Compile() - using interpreted mode gives slower execution but avoids IL generation. It can be a positive effect for small data sets, and negative for large.

As a side note: there are multiple async implementations for queryable sources; EF Core has one, and IX (part of Reactive) has another. They are incompatible with each-other and use different approaches, but both can work depending on your scenario.