CODEWITHSUNDEEP
pythoncollectionsdocumentmongoengine
Rvdixit23
Rvdixit23

Reputation: 193

How to generate a model for a mongoengine Document from an already existing collection

There is an existing collection in mongo called students. Is there a way where I don't have to type out the schema for all the fields and directly import all the fields from the collection?

class Student(DynamicDocument):
    meta = {'collection': 'students'}
    name = StringField() # I want to avoid writing this for all the fields in the collection
    rollNo = IntField()
    address = StringField()

Upvotes: 8

Views: 2194

Answers (3)

Dori Lahav Waisberg
Dori Lahav Waisberg

Reputation: 970

You can't do that as MongoDB doesn't have schemas built into it. Schemas are only a driver thing, and even then some drivers support them and some don't.

What you can try to do is create a simple script to map all the fields in the documents in your collection, then construct a schema out of it.

Upvotes: 3

wowkin2
wowkin2

Reputation: 6355

You can generate user_properties (as in this answer) dynamically iterating document by document in your collection and adding new values to that dict.

from pymongo import MongoClient

db = MongoClient(MONGODB_URI).get_database()
documents = db['users'].find()

user_properties = {
   # Example of structure:
   # '_id': StringField(required=False),
   # 'name': StringField(required=False),
   # 'email': StringField(required=False),
}
for doc in documents:
    for field_name, value in doc.items():        
        # Some smart recognition can be here
        field_definition = StringField(required=False)

        user_properties[field_name] = field_definition


# Your new class for MongoEngine:
User = type("User", (Document, ), user_properties)

users = User.objects(email__endswith='.com')
print(users)

Upvotes: 5

amanb
amanb

Reputation: 5463

The jinja template engine has features that can be utilized to create a mongo class template that can be used to populate the property schemas in the class definition based on the bsonType of each property and finally generate the desired Python class. The following is an example illustrating this concept. We start with the following project structure, which has a sample parent folder called Templates and three files:

Templates\
    - mongo_sample.schema.json
    - mongoclass_template.jinja
    - auto_mongo_class.py

Let's explore these files:

1. mongo_sample.schema.json:

This is the MongoDB schema for a sample document in a sample collection(say "students"). There are 3 properties namely name, rollNo & address, their bsonTypes have also been specified. :

{
    "bsonType": "object",
    "required": ["name", "rollNo", "address"],
    "properties": {
      "name": {
        "bsonType": "string"
      },
      "rollNo": {
        "bsonType": "int",
        "minimum": 1        
      },
      "address":{
          "bsonType":"string"
      }
    }
}

2. mongoclass_template.jinja:

This is the Jinja template for a Mongo class which will be used to render the desired Python Mongo class in the end. It makes use of the Jinja template language. Here's the reference for the Template Designer Documentation:

from mongoengine import DynamicDocument, StringField, IntField

class Student(DynamicDocument):
    meta = {"collection": "{{collection_name}}"}    
    {%- for prop, val in properties.items() %}        
    {% if val.bsonType == "string" %}
    {{prop}} = StringField()
    {% elif val.bsonType == "int" %}
    {{prop}} = IntField()    
    {% else %} 
    {{prop}} = StringField()       
    {% endif %}
{%- endfor %}

3. auto_mongo_class.py:

This Python script parses through the JSON schema above to get the properties object, passes the necessary variables to the template render() method which will be used while rendering the template and finally writes to a Python file called models.py which is the final Mongo class:

import json
from jinja2 import Template

with open(r"mongo_sample.schema.json", "r") as schema_json:
    schema_tree = json.load(schema_json)
properties = schema_tree["properties"]
template = Template(open(r"mongoclass_template.jinja").read()).render(properties=properties, \
            collection_name="students")

with open("models.py", 'w') as file_:
    file_.write(template)

Now, let's run this script:

python auto_mongo_class.py

A new Python script called models.py will appear in the project folder with the following contents:

from mongoengine import DynamicDocument, StringField, IntField

class Student(DynamicDocument):
    meta = {"collection": "students"}        
    
    name = StringField()
            
    
    rollNo = IntField()    
            
    
    address = StringField()

This is a basic example of a simple BSON schema. It can be extended further for more complex schemas.

Upvotes: 2

Related Questions