CODEWITHSUNDEEP
javaspringmongodbspring-data-mongodb
Dylan
Dylan

Reputation: 2219

How to utilize Pageable when running a custom delete query in Spring JPA for mongodb?

I am working on creating a tool allowing admins to purge data from the database. Our one collection has millions of records making deletes seize up the system. Originally I was just running a query with that returns a Page and dropping that into the standard delete. Ideally i'd prefer to run the query and delete in one go.

@Query(value = "{ 'timestamp' : {$gte : ?0, $lte: ?1 }}")
public Page deleteByTimestampBetween(Date from, Date to, Pageable pageable);

Is this possible, using the above code the system behaves the same where the program doesnt continue the delete function and the data isnt removed from mongo. Or is there a better approach?

Upvotes: 1

Views: 1391

Answers (1)

s7vr
s7vr

Reputation: 75934

I don't think it is possible using Pageable/Query annotation. You can use Bulk Write to process deletes in batches.

Something like

int count = 0;
int batch = 100; //Send 100 requests at a time
BulkOperations bulkOps = mongoTemplate.bulkOps(BulkOperations.BulkMode.UNORDERED, YourPojo.class);
List<DateRange> dateRanges = generateDateRanges(from, to, step); //Add a function to generate date ranges with the defined step.

for (DateRange dateRange: dateRanges){
    Query query = new Query();
    Criteria criteria = new Criteria().andOperator(Criteria.where("timestamp").gte(dateRange.from), Criteria.where("timestamp").lte(dateRange.to));
    query.addCriteria(criteria);
    bulkOps.remove(query);
    count++;
    if (count == batch) {
        bulkOps.execute();
        count = 0;
    }
}

if (count > 0) {
    bulkOps.execute();
}

Upvotes: 1

Related Questions