How to import complex json do django model object

Question

I got from some network service list of following dict:

{
    '_id': 'bcEsX4Mhridf7Fdz59nYcm',
    'stats': {
        'acqs': 0,
        'alerts': 0,
        'blocks': 0,
        'false_positive_alerts': 0,
        'false_positive_alerts_by_source': {},
        'malware_false_positive_alerts': 0
    },
    'hostname': 'SomePC',
    'domain': 'local',
    'url': '/hx/api/v3/hosts/bcEsX4Mhridf7Fdz59nYcm',
    'sysinfo': {
        'url': '/hx/api/v3/hosts/bcEsX4Mhridf7Fdz59nYcm/sysinfo'
    },
    'os': {
        'product_name': 'Windows 10 Pro',
        'patch_level': None,
        'bitness': '64-bit',
        'platform': 'win',
        'kernel_version': None
    },
}

I want to import it to django model object. My models.py contain following code:

class Host(models.Model):
    _id = models.CharField(max_length=25, primary_key=True)
    hostname = models.CharField(max_length=25)
    domain = models.CharField(max_length=253)
    url = models.CharField(max_length=50)
    def __str__(self):
        """String for representing the Model object."""
    return '{0}_({1})'.format(self.hostname, self._id)

class stats(models.Model):
    host = models.OneToOneField(Host, on_delete=models.CASCADE, null=True)
    acqs = models.IntegerField(default=0)
    alerts = models.IntegerField(default=0)
    blocks = models.IntegerField(default=0)
    false_positive_alerts = models.IntegerField(default=0)
    malware_false_positive_alerts = models.IntegerField(default=0)
    def __str__(self):
        """String for representing the Model object."""
        return '{0}/{1}/{2}'.format(
            self.acqs, 
            self.alerts, 
            self.blocks
            )

class sysinfo(models.Model):
    host = models.OneToOneField(Host, on_delete=models.CASCADE, null=True)
    url = models.CharField(max_length=45, blank=True)
    def __str__(self):
        """String for representing the Model object."""
        return '{0}'.format(self.url)

class os(models.Model):
    host = models.OneToOneField(Host, on_delete=models.CASCADE, null=True)
    product_name = models.CharField(max_length=253)
    patch_level = models.CharField(max_length=253, blank=True, null=True)
    bitness = models.CharField(max_length=10, blank=True, null=True)
    platform = models.CharField(max_length=10, blank=True, null=True)
    kernel_version = models.CharField(max_length=10, blank=True, null=True)
    def __str__(self):
        """String for representing the Model object."""
        return '{0}'.format(self.product_name)

If mentioned dict would be a flat one I can import it to model object by following code:

h = Host(**my_dict)
h.save()

But cannot find the way to put nested/complex dict to my model objects.
When I try h = Host(**my_dict) I got error message:

ValueError                                Traceback (most recent call last)
<ipython-input-37-1250f570aeb3> in <module>
----> 1 h_new2 = Host(**my_dict2)
C:\Program Files\Python37\lib\site-packages\django\db\models\base.py in __init__(self, *args, **kwargs)
    493                     if prop in property_names or opts.get_field(prop):
    494                         if kwargs[prop] is not _DEFERRED:
--> 495                             _setattr(self, prop, kwargs[prop])
    496                         del kwargs[prop]
    497                 except (AttributeError, FieldDoesNotExist):

C:\Program Files\Python37\lib\site-packages\django\db\models\fields\related_descriptors.py in __set__(self, instance, value)
    462                     instance._meta.object_name,
    463                     self.related.get_accessor_name(),
--> 464                     self.related.related_model._meta.object_name,
    465                 )
    466             )
ValueError: Cannot assign "{'url': '/hx/api/v3/hosts/po1qmpQ2L3jdwQKKWS1CJm/sysinfo'}": "Host.sysinfo" must be a "sysinfo" instance.

What is wrong with my models?

Answer 1

Django does not handle the construction of related model instances automatically. You need to take care of this yourself.

I would do the following in your case.

# Assume "data" holds your data structure

stats_data = data.pop('stats')
os_data = data.pop('os')  # Clashes with os module - please don't
sysinfo_data = data.pop('sysinfo')

# store the FK in data
data['stats'] = stats(**stats_data)
data['os'] = os(**os_data)
data['sysinfo'] = sysinfo(**sysinfo_data)

host_instance = Host(**data)

I also recommend to use atomic transaction context manager, so you don't have any data left if something goes wrong during the construction of the Host instance.

from django.db import transaction


with transaction.atomic():

    # Code from above

Further to increase readability of your code you should follow the PEP8 and Django code guidelines. This makes it also easier to help you, otherwise people might get confused about what a model, method, etc. is. https://docs.djangoproject.com/en/dev/internals/contributing/writing-code/coding-style/#model-style

For example: Modelnames should be CamelCase

Answer 2

I can suggest you to make a new model class like: StatsItem

which is:

class StatsItem:
def __init__(self, data):
    self.acqs = data.get("acqs")
    self.alerts = data.get("alerts")
    ..
    ..
    self.blocks = data.get("blocks")

then give json file to constructor:

stats = StatsItem(data.get('stats'))

now you have stats variable and u can reach every value easily:

blocks_value = stats.blocks

and if you create a function inside of StatsItem (I called it request_data):

def to_request_data(self):
    return dict(
        acqs=self.acqs if self.acqs else None,
        alerts=self.alerts,
        ..
        ..
        blocks=self.blocks
    )

this way you can return dict which is also easy to access:

data = stats.requested_data() ## stats is the variable that we initialize above.
blocks = data.get('blocks')

hope that helps,