How to complete the dynamic construction of complex queries?

Sometimes, you need to dynamically construct a more complex query condition and transfer it to the database for query. The conditions themselves may come from front-end requests or configuration files. Then at this time, the expression tree can help you. In this article, we will use a few short examples to understand how to complete these operations.

Microsoft MVP Lab Researcher

You may also have received these requests:

(The picture is queried from the model)

(Based on configuration query)

Today we look at how expression trees fulfill these requirements.

Fixed conditions can be passed in in Where

The following is a simple unit test case. Next, we will change this test case beyond recognition.

[Test]
public void Normal()
{
    var re = Enumerable.Range(0, 10).AsQueryable() // 0-9
        .Where(x => x >= 1 && x < 5).ToList(); // 1 2 3 4
    var expectation = Enumerable.Range(1, 4); // 1 2 3 4
    re.Should().BeEquivalentTo(expectation);
}

Where in Queryable is an expression tree

Because it is a Queryable relationship, the expression in Where is actually an expression, so let's define it separately, by the way, the length of the article.

[Test]
public void Expression00()
{
    Expression<Func<int, bool>> filter = x => x >= 1 && x < 5;
    var re = Enumerable.Range(0, 10).AsQueryable()
        .Where(filter).ToList();
    var expectation = Enumerable.Range(1, 4);
    re.Should().BeEquivalentTo(expectation);
}

Expressions can be implicitly converted by Lambda

On the right side of Expression is a Lambda, so variables in the context can be captured.
In this way, you can define minValue and maxValue separately.
So you can get minValue and maxValue from other places to change the filter.

[Test]
public void Expression01()
{
    var minValue = 1;
    var maxValue = 5;
    Expression<Func<int, bool>> filter = x => x >= minValue && x < maxValue;
    var re = Enumerable.Range(0, 10).AsQueryable()
        .Where(filter).ToList();
    var expectation = Enumerable.Range(1, 4);
    re.Should().BeEquivalentTo(expectation);
}

You can use methods to create expressions

In this case, we can also use a method to create Expression.
This method can actually be considered as the factory method of this Expression.

[Test]
public void Expression02()
{
    var filter = CreateFilter(1, 5);
    var re = Enumerable.Range(0, 10).AsQueryable()
        .Where(filter).ToList();
    var expectation = Enumerable.Range(1, 4);
    re.Should().BeEquivalentTo(expectation);

    Expression<Func<int, bool>> CreateFilter(int minValue, int maxValue)
    {
        return x => x >= minValue && x < maxValue;
    }
}

Func can be used to combine conditions more flexibly

Then you can use minValue and maxValue as parameters to make factory methods, so of course you can use delegates.
Therefore, we can define the left side and the right side as two Func respectively, so that the specific comparison method of left and right is determined by the outside.

[Test]
public void Expression03()
{
    var filter = CreateFilter(x => x >= 1, x => x < 5);
    var re = Enumerable.Range(0, 10).AsQueryable()
        .Where(filter).ToList();
    var expectation = Enumerable.Range(1, 4);
    re.Should().BeEquivalentTo(expectation);

    Expression<Func<int, bool>> CreateFilter(Func<int, bool> leftFunc, Func<int, bool> rightFunc)
    {
        return x => leftFunc.Invoke(x) && rightFunc.Invoke(x);
    }
}

You can also build the expression manually

In fact, the two left and right are not only two Func, but can also be directly two expressions.
But a little bit different is that the combination of expressions needs to be created with the relevant methods in the Expression type.
We can find that there is no change in the calling place this time, because Lambda can be implicitly converted to Func or implicitly converted to Expression.
The meaning of each method can be seen from the comments.

[Test]
public void Expression04()
{
    var filter = CreateFilter(x => x >= 1, x => x < 5);
    var re = Enumerable.Range(0, 10).AsQueryable()
        .Where(filter).ToList();
    var expectation = Enumerable.Range(1, 4);
    re.Should().BeEquivalentTo(expectation);

    Expression<Func<int, bool>> CreateFilter(Expression<Func<int, bool>> leftFunc,
        Expression<Func<int, bool>> rightFunc)
    {
        // x
        var pExp = Expression.Parameter(typeof(int), "x");
        // (a => leftFunc(a))(x)
        var leftExp = Expression.Invoke(leftFunc, pExp);
        // (a => rightFunc(a))(x)
        var rightExp = Expression.Invoke(rightFunc, pExp);
        // (a => leftFunc(a))(x) && (a => rightFunc(a))(x)
        var bodyExp = Expression.AndAlso(leftExp, rightExp);
        // x => (a => leftFunc(a))(x) && (a => rightFunc(a))(x)
        var resultExp = Expression.Lambda<Func<int, bool>>(bodyExp, pExp);
        return resultExp;
    }
}

Introduce deconstruction of expressions to make it easier

However, the above method can actually be optimized. Avoid direct calls to left and right expressions.
Using a method called Unwrap, Lambda Expression can be deconstructed into an expression containing only the Body part.
This is a custom extension method, you can introduce this method ObjectVisitor
Due to space limitations, we cannot discuss the implementation of Unwrap here. We only need to pay attention to the difference from the comment in the previous example.

ObjectVisitor：https://github.com/newbe36524/Newbe.ObjectVisitor

[Test]
public void Expression05()
{
    var filter = CreateFilter(x => x >= 1, x => x < 5);
    var re = Enumerable.Range(0, 10).AsQueryable()
        .Where(filter).ToList();
    var expectation = Enumerable.Range(1, 4);
    re.Should().BeEquivalentTo(expectation);

    Expression<Func<int, bool>> CreateFilter(Expression<Func<int, bool>> leftFunc,
        Expression<Func<int, bool>> rightFunc)
    {
        // x
        var pExp = Expression.Parameter(typeof(int), "x");
        // leftFunc(x)
        var leftExp = leftFunc.Unwrap(pExp);
        // rightFunc(x)
        var rightExp = rightFunc.Unwrap(pExp);
        // leftFunc(x) && rightFunc(x)
        var bodyExp = Expression.AndAlso(leftExp, rightExp);
        // x => leftFunc(x) && rightFunc(x)
        var resultExp = Expression.Lambda<Func<int, bool>>(bodyExp, pExp);
        return resultExp;
    }
}

More expressions can be spliced

We can optimize the following and extend the CreateFilter method to support multiple sub-expressions and the connection mode of customizable sub-expressions.
Thus, we can get a JoinSubFilters method.

[Test]
public void Expression06()
{
    var filter = JoinSubFilters(Expression.AndAlso, x => x >= 1, x => x < 5);
    var re = Enumerable.Range(0, 10).AsQueryable()
        .Where(filter).ToList();
    var expectation = Enumerable.Range(1, 4);
    re.Should().BeEquivalentTo(expectation);

    Expression<Func<int, bool>> JoinSubFilters(Func<Expression, Expression, Expression> expJoiner,
        params Expression<Func<int, bool>>[] subFilters)
    {
        // x
        var pExp = Expression.Parameter(typeof(int), "x");
        var result = subFilters[0];
        foreach (var sub in subFilters[1..])
        {
            var leftExp = result.Unwrap(pExp);
            var rightExp = sub.Unwrap(pExp);
            var bodyExp = expJoiner(leftExp, rightExp);

            result = Expression.Lambda<Func<int, bool>>(bodyExp, pExp);
        }

        return result;
    }
}

Use factory methods instead of fixed sub-expressions

With previous experience, we know. In fact, x => x >= 1 This expression can be built by a factory method.
So, we use a CreateMinValueFilter to create this expression.

[Test]
public void Expression07()
{
    var filter = JoinSubFilters(Expression.AndAlso,
        CreateMinValueFilter(1),
        x => x < 5);
    var re = Enumerable.Range(0, 10).AsQueryable()
        .Where(filter).ToList();
    var expectation = Enumerable.Range(1, 4);
    re.Should().BeEquivalentTo(expectation);

    Expression<Func<int, bool>> CreateMinValueFilter(int minValue)
    {
        return x => x >= minValue;
    }

    Expression<Func<int, bool>> JoinSubFilters(Func<Expression, Expression, Expression> expJoiner,
        params Expression<Func<int, bool>>[] subFilters)
    {
        // x
        var pExp = Expression.Parameter(typeof(int), "x");
        var result = subFilters[0];
        foreach (var sub in subFilters[1..])
        {
            var leftExp = result.Unwrap(pExp);
            var rightExp = sub.Unwrap(pExp);
            var bodyExp = expJoiner(leftExp, rightExp);

            result = Expression.Lambda<Func<int, bool>>(bodyExp, pExp);
        }

        return result;
    }
}

The factory method can also be manually created using Expression

Of course, you can only use Expression-related methods to create x => x >= 1 .

[Test]
public void Expression08()
{
    var filter = JoinSubFilters(Expression.AndAlso,
        CreateMinValueFilter(1),
        x => x < 5);
    var re = Enumerable.Range(0, 10).AsQueryable()
        .Where(filter).ToList();
    var expectation = Enumerable.Range(1, 4);
    re.Should().BeEquivalentTo(expectation);

    Expression<Func<int, bool>> CreateMinValueFilter(int minValue)
    {
        // x
        var pExp = Expression.Parameter(typeof(int), "x");
        // minValue
        var rightExp = Expression.Constant(minValue);
        // x >= minValue
        var bodyExp = Expression.GreaterThanOrEqual(pExp, rightExp);
        var result = Expression.Lambda<Func<int, bool>>(bodyExp, pExp);
        return result;
    }

    Expression<Func<int, bool>> JoinSubFilters(Func<Expression, Expression, Expression> expJoiner,
        params Expression<Func<int, bool>>[] subFilters)
    {
        // x
        var pExp = Expression.Parameter(typeof(int), "x");
        var result = subFilters[0];
        foreach (var sub in subFilters[1..])
        {
            var leftExp = result.Unwrap(pExp);
            var rightExp = sub.Unwrap(pExp);
            var bodyExp = expJoiner(leftExp, rightExp);

            result = Expression.Lambda<Func<int, bool>>(bodyExp, pExp);
        }

        return result;
    }
}

In the same way, sub-expressions can be created like this

Now that Expression is used to create sub-expressions, just make a little improvement and make x => x <5 also get it from the factory method.

[Test]
public void Expression09()
{
    var filter = JoinSubFilters(Expression.AndAlso,
        CreateValueCompareFilter(Expression.GreaterThanOrEqual, 1),
        CreateValueCompareFilter(Expression.LessThan, 5));
    var re = Enumerable.Range(0, 10).AsQueryable()
        .Where(filter).ToList();
    var expectation = Enumerable.Range(1, 4);
    re.Should().BeEquivalentTo(expectation);

    Expression<Func<int, bool>> CreateValueCompareFilter(Func<Expression, Expression, Expression> comparerFunc,
        int rightValue)
    {
        var pExp = Expression.Parameter(typeof(int), "x");
        var rightExp = Expression.Constant(rightValue);
        var bodyExp = comparerFunc(pExp, rightExp);
        var result = Expression.Lambda<Func<int, bool>>(bodyExp, pExp);
        return result;
    }

    Expression<Func<int, bool>> JoinSubFilters(Func<Expression, Expression, Expression> expJoiner,
        params Expression<Func<int, bool>>[] subFilters)
    {
        // x
        var pExp = Expression.Parameter(typeof(int), "x");
        var result = subFilters[0];
        foreach (var sub in subFilters[1..])
        {
            var leftExp = result.Unwrap(pExp);
            var rightExp = sub.Unwrap(pExp);
            var bodyExp = expJoiner(leftExp, rightExp);

            result = Expression.Lambda<Func<int, bool>>(bodyExp, pExp);
        }

        return result;
    }
}

Add a little configuration and you're done

Finally, we are using a little trick to create the sub-expression. Determined by external parameters. This basically completes the dynamic construction of a multi-And value comparison query condition.

[Test]
public void Expression10()
{
    var config = new Dictionary<string, int>
    {
        { ">=", 1 },
        { "<", 5 }
    };
    var subFilters = config.Select(x => CreateValueCompareFilter(MapConfig(x.Key), x.Value)).ToArray();
    var filter = JoinSubFilters(Expression.AndAlso, subFilters);
    var re = Enumerable.Range(0, 10).AsQueryable()
        .Where(filter).ToList();
    var expectation = Enumerable.Range(1, 4);
    re.Should().BeEquivalentTo(expectation);

    Func<Expression, Expression, Expression> MapConfig(string op)
    {
        return op switch
        {
            ">=" => Expression.GreaterThanOrEqual,
            "<" => Expression.LessThan,
            _ => throw new ArgumentOutOfRangeException(nameof(op))
        };
    }

    Expression<Func<int, bool>> CreateValueCompareFilter(Func<Expression, Expression, Expression> comparerFunc,
        int rightValue)
    {
        var pExp = Expression.Parameter(typeof(int), "x");
        var rightExp = Expression.Constant(rightValue);
        var bodyExp = comparerFunc(pExp, rightExp);
        var result = Expression.Lambda<Func<int, bool>>(bodyExp, pExp);
        return result;
    }

    Expression<Func<int, bool>> JoinSubFilters(Func<Expression, Expression, Expression> expJoiner,
        params Expression<Func<int, bool>>[] subFilters)
    {
        // x
        var pExp = Expression.Parameter(typeof(int), "x");
        var result = subFilters[0];
        foreach (var sub in subFilters[1..])
        {
            var leftExp = result.Unwrap(pExp);
            var rightExp = sub.Unwrap(pExp);
            var bodyExp = expJoiner(leftExp, rightExp);

            result = Expression.Lambda<Func<int, bool>>(bodyExp, pExp);
        }

        return result;
    }
}

Summarize

If the logical relationship is more complicated, there are multiple levels of nesting like a tree, and there are many ways to compare, and even include methods, what should be done?
You can refer to the following examples:

• https://github.com/newbe36524/Newbe.Demo/tree/main/src/BlogDemos/Newbe.ExpressionsTests/Newbe.ExpressionsTests/FilterFactory

If you are interested in this content, you can also browse the video I recorded before to learn more:

Play fine sharing C# expression tree, the first season

• https://www.bilibili.com/video/BV15y4y1r7pK

Play fine sharing C# expression tree, the second season

• https://www.bilibili.com/video/BV1Mi4y1L7oR

You can also refer to the previous entry:

"As long as ten steps, you can apply expression trees to optimize dynamic calls"

• https://www.newbe.pro/Newbe.Claptrap/Using-Expression-Tree-To-Build-Delegate/index.html

Or look at the MSDN documentation, I think you can also gain something:

• https://docs.microsoft.com/dotnet/csharp/programming-guide/concepts/expression-trees/?WT.mc_id=DX-MVP-5003606

This related code can be obtained from the following address:

• https://github.com/newbe36524/Newbe.Demo/blob/main/src/BlogDemos/Newbe.ExpressionsTests/Newbe.ExpressionsTests/Examples/Z01SingleWhereTest.cs

If you think this article is good, remember to bookmark, like, comment, and forward. Tell me what else I want to know!

Microsoft's most valuable expert (MVP)

Microsoft's Most Valuable Expert is a global award granted by Microsoft to third-party technology professionals. For 28 years, technology community leaders around the world have won this award for sharing their expertise and experience in online and offline technology communities.

MVP is a rigorously selected team of experts. They represent the most skilled and intelligent people. They are experts who are passionate and helpful to the community. MVP is committed to helping others through speeches, forum questions and answers, creating websites, writing blogs, sharing videos, open source projects, organizing conferences, etc., and to help users in the Microsoft technology community use Microsoft technology to the greatest extent.
For more details, please visit the official website:
https://mvp.microsoft.com/zh-cn

---

Welcome to follow the Microsoft China MSDN subscription account for more latest releases!