CODEWITHSUNDEEP
mongodbgomongodb-query
Madu Alikor
Madu Alikor

Reputation: 2662

How to use mongodb/mongo-go-driver to perform efficient paging

I read in the following article that it is more efficient to use the natural ordering of _id to perform pagination because skip always starts from the beginning of the collection.

Fast and Efficient Pagination in MongoDB

// Page 1
db.students.find().limit(10)

// Page 2
last_id = ...  # logic to get last_id
db.students.find({'_id': {'$gt': last_id}}).limit(10)

But I have no idea how to perform the above using the mongodb/mongo-go-driver.

Upvotes: 1

Views: 11506

Answers (3)

Mamad
Mamad

Reputation: 199

Set page=0 and limit=0 if no pagination is required.

func GetUsers (page, limit int) {
    filter := bson.D{{}} // selects all documents
    options := new(options.FindOptions)
    if limit != 0 {
        if page == 0 {
            page = 1
        }
        options.SetSkip(int64((page - 1) * limit))
        options.SetLimit(int64(limit))
    }

    cursor, err := mongoCollection.Find(context.TODO(), filter, options)
...
}

Upvotes: 0

Fajrul Aulia
Fajrul Aulia

Reputation: 145

you can create a new func, dont forget to pass http.writer to read parameter.

func Pagination(r *http.Request, FindOptions *options.FindOptions) (int64, int64) {
    if r.URL.Query().Get("page") != "" && r.URL.Query().Get("limit") != "" {
        page, _ := strconv.ParseInt(r.URL.Query().Get("page"), 10, 32)
        limit, _ := strconv.ParseInt(r.URL.Query().Get("limit"), 10, 32)
        if page == 1 {
            FindOptions.SetSkip(0)
            FindOptions.SetLimit(limit)
            return page, limit
        }

        FindOptions.SetSkip((page - 1) * limit)
        FindOptions.SetLimit(limit)
        return page, limit

    }
    FindOptions.SetSkip(0)
    FindOptions.SetLimit(0)
    return 0, 0
}

just call

Pagination(r, options)

example

options := options.Find()
page, limit := parameter.Pagination(r, options)
// page, limit as response for header payload

Upvotes: 3

Wan B.
Wan B.

Reputation: 18835

The cursor.skip() method requires the server to scan from the beginning of the input results set before beginning to return results. As the offset increases, cursor.skip() will become slower. While range queries can use indexes to avoid scanning unwanted documents, typically yielding better performance as the offset grows compared to using cursor.skip() for pagination. See more information on MongoDB: Pagination Example

Using the current version of mongo-go-driver (v0.0.15).An example to perform pagination showing latest entry first: 

func Paginate(collection *mongo.Collection, startValue objectid.ObjectID, nPerPage int64) ([]bson.Document, *bson.Value, error) {

    // Query range filter using the default indexed _id field. 
    filter := bson.VC.DocumentFromElements(
        bson.EC.SubDocumentFromElements(
            "_id",
            bson.EC.ObjectID("$gt", startValue),
        ),
    )

    var opts []findopt.Find
    opts = append(opts, findopt.Sort(bson.NewDocument(bson.EC.Int32("_id", -1))))
    opts = append(opts, findopt.Limit(nPerPage))

    cursor, _ := collection.Find(context.Background(), filter, opts...)

    var lastValue *bson.Value
    var results []bson.Document
    for cursor.Next(context.Background()) {
        elem := bson.NewDocument()
        err := cursor.Decode(elem)
        if err != nil {
            return results, lastValue, err
        }
        results = append(results, *elem)
        lastValue = elem.Lookup("_id")
    }

    return results, lastValue, nil
}

An example to call the pagination function above: 

database := client.Database("databaseName")
collection := database.Collection("collectionName")
startObjectID, _ := objectid.FromHex("5bbafea2b5e14ee3a298fa4a")

// Paginate only the latest 20 documents 
elements, lastID, err := Paginate(collection, startObjectID, 20)
for _, e := range elements {
    fmt.Println(&e)
}
// Last seen ObjectID can be used to call next Paginate() 
fmt.Println("Last seen ObjectID: ", lastID.ObjectID())

Note that you can also substitute the _id field with another indexed field.

Upvotes: 1

Related Questions