CODEWITHSUNDEEP
pythonmongodb
Jack022
Jack022

Reputation: 1257

How can I query data from MongoDB in real-time?

I created a MongoDB database, and I'm sending data to it. At the same time, I'm running a Python script to fetch data from that Database. I would like my script to print the new entry to my console as soon as it's pushed to the DB, but I don't know how to accomplish this.

This is my current work, but I don't like it because each time it will print the whole data on the db, even though I only want the last entry/entries as soon as they are updated:

from pymongo import MongoClient
import time
import random
from pprint import pprint

client = MongoClient(port=27017)

arr = []

db = client.one

mycol = client["coll"]



while True:
    cursor = db.mycol.find()
    for document in cursor:
        print(document['num'])
    time.sleep(2)

How can I resolve this?

Upvotes: 0

Views: 1378

Answers (2)

user10503628
user10503628

Reputation:

There are a few ways to handle this, but the easiest might be to store an auto-incrementing "primaryKey" (or insert timestamp or whatever), and only print the results that occur after that key. Here is a quick example to demonstrate:

# we start at one...
highest_previous_primary_key = 1

while True:
    cursor = db.mycol.find()
    for document in cursor:

        # get the current primary key, and if it's greater than the previous one
        # we print the results and increment the variable to that value
        current_primary_key = document['primaryKey']
        if current_primary_key > highest_previous_primary_key:
            print(document['num'])
            highest_previous_primary_key = current_primary_key

    time.sleep(2)

This is perhaps the laziest approach to do. But beyond this, you may try doing:

  1. Adjusting the query itself, so it only gets items > the primaryKey (imagine if you had a billion results and each time you fetched all results).

Upvotes: 2

Klaus D.
Klaus D.

Reputation: 14369

Mongo DB since version 3.6 supports a feature call "Change Streams". In the documentation you will find this simple Python example among some others:

cursor = db.inventory.watch()
document = next(cursor)

If next() is supported on the cursor your should also be able to use it in loops, generators and even asyncio.

Upvotes: 5

Related Questions