Can't include nested fields with spring data aggregation

Question

I have a problem with the inclusion and exclusion of nested fields in the aggregation.. I have a collection that contains nested objects, so when I try to build the query by including only some fields of nested objects it only works with the query, but it doesn't work with aggregation

This is a simple preview of my collection

{
    "id": "1234",
    "name": "place name",
    "address": {
        "city": "city name",
        "gov": "gov name",
        "country": "country name",
        "location": [0.0, 0.0],
        //some other data
    },
    //some other data
}

when i configure my fields with query it works

query.fields().include("name").include("address.city").include("address.gov")

Also when I do it with aggregation using the shell it works

db.getCollection("places").aggregate([
    { $project: {
        "name": 1,
        "address.city": 1,
        "address.gov": 1
    } },
])

but it doesn't work with aggregation in spring

val aggregation = Aggregation.newAggregation(
        Aggregation.match(criteria)
        Aggregation.project().andInclude("name").andInclude("address.city").andInclude("address.gov")
)

This aggregation with spring always returns address field as null, but when I put the "address" field without its nested fields the result will contain the full address object, not just its nested fields that I want to include.

Can someone tell me how to fix that?

Answer 1

I found a solution, it's by using the nested() function

val aggregation = Aggregation.newAggregation(
        Aggregation.match(criteria)
        Aggregation.project().andInclude("name")
            .and("address").nested(Fields.fields("address.city", "address.gov"))
)

But it only works with hardcoded fields.. So if you want to have a function to which you pass the list of fields to include or exclude, you can use this solution

fun fieldsConfig(fields : List<String>) : ProjectionOperation
{
    val mainFields = fields.filter { !it.contains(".") }

    var projectOperation = Aggregation.project().andInclude(*mainFields.toTypedArray())

    val nestedFields = fields.filter { it.contains(".") }.map { it.substringBefore(".") }.distinct()

    nestedFields.forEach { mainField ->

        val subFields = fields.filter { it.startsWith("${mainField}.") }

        projectOperation = projectOperation.and(mainField).nested(Fields.fields(*subFields.toTypedArray()))
    }

    return projectOperation
}

The problem with this solution is that there is a lot of code, memory allocation for objects, and configuration to include fields.. In addition, it works only with inclusion, if you use it to exclude fields, it throws an exception.. Also, it does not work with the deepest fields of your document.

So I implemented a simpler and more elegant solution, which covers most cases of inclusion and exclusion of fields.

This builder class allows you to create an object containing the fields you want to include or exclude.

class DbFields private constructor(private val list : List<String>, val include : Boolean) : List<String>
{
    override val size : Int get() = list.size

    //overridden functions of List class.


    /**
     * the builder of the fields.
     */
    class Builder
    {
        private val list = ArrayList<String>()
        private var include : Boolean = true


        /**
         * add a new field.
         */
        fun withField(field : String) : Builder
        {
            list.add(field)

            return this
        }


        /**
         * add a new fields.
         */
        fun withFields(fields : Array<String>) : Builder
        {
            fields.forEach {
                list.add(it)
            }

            return this
        }


        fun include() : Builder
        {
            include = true
            return this
        }


        fun exclude() : Builder
        {
            include = false
            return this
        }


        fun build() : DbFields
        {
            if (include && !list.contains("id"))
            {
                list.add("id")
            }
            else if (!include && list.contains("id"))
            {
                list.remove("id")
            }

            return DbFields(list.distinct(), include)
        }
    }
}

To build your fields configuration

    val fields = DbFields.Builder()
        .withField("fieldName")
        .withField("fieldName")
        .withField("fieldName")
        .include()
        .build()

This object you can pass it to your repository to configure the inclusion or exlusion.

I also created this class to configures the inclusion and exclusion using raw documents that will be transformed to a custom aggregation operation.

class CustomProjectionOperation(private val fields : DbFields) : ProjectionOperation()
{
    override fun toDocument(context : AggregationOperationContext) : Document
    {
        val fieldsDocument = BasicDBObject()

        fields.forEach {
            fieldsDocument.append(it, fields.include)
        }

        val operation = Document()
        operation.append("\$project", fieldsDocument)

        return operation
    }
}

now, you have just to use this class in your aggregation

class RepoCustomImpl : RepoCustom
{
    @Autowired
    private lateint mongodb : MongoTemplate


    override fun getList(fields : DbFields) : List<Result>
    {
        val aggregation = Aggregation.newAggregation(
            CustomProjectionOperation(fields)
        )

        return mongodb.aggregate(aggregation, Result::class.java, Result::class.java).mappedResults
    }
}