CODEWITHSUNDEEP
pythondjangofile-io
Ty.
Ty.

Reputation: 1820

Django: Access primary key in models.filefield(upload_to) location

I'd like to save my files using the primary key of the entry.

Here is my code:

def get_nzb_filename(instance, filename):
    if not instance.pk:
        instance.save() # Does not work.
    name_slug = re.sub('[^a-zA-Z0-9]', '-', instance.name).strip('-').lower()
    name_slug = re.sub('[-]+', '-', name_slug)
    return u'files/%s_%s.nzb' % (instance.pk, name_slug)

class File(models.Model):
    nzb = models.FileField(upload_to=get_nzb_filename)
    name = models.CharField(max_length=256)

I know the first time an object is saved the primary key isn't available, so I'm willing to take the extra hit to save the object just to get the primary key, and then continue on.

The above code doesn't work. It throws the following error:

maximum recursion depth exceeded while calling a Python object

I'm assuming this is an infinite loop. Calling the save method would call the get_nzb_filename method, which would again call the save method, and so on.

I'm using the latest version of the Django trunk.

How can I get the primary key so I can use it to save my uploaded files?

Update @muhuk:

I like your solution. Can you help me implement it? I've updated my code to the following and the error is 'File' object has no attribute 'create'. Perhaps I'm using what you've written out of context?

def create_with_pk(self):
    instance = self.create()
    instance.save()
    return instance

def get_nzb_filename(instance, filename):
    if not instance.pk:
        create_with_pk(instance)
    name_slug = re.sub('[^a-zA-Z0-9]', '-', instance.name).strip('-').lower()
    name_slug = re.sub('[-]+', '-', name_slug)
    return u'files/%s_%s.nzb' % (instance.pk, name_slug)

class File(models.Model):
    nzb = models.FileField(upload_to=get_nzb_filename, blank=True, null=True)
    name = models.CharField(max_length=256)

Instead of enforcing the required field in my model I'll do it in my Form class. No problem.

Upvotes: 15

Views: 7811

Answers (8)

Ranu Vijay
Ranu Vijay

Reputation: 1277

Well I'm not sure of my answer but -

use nested models, if you can -

class File(models.Model):
    name = models.CharField(max_length=256)

class FileName(models.Model):
    def get_nzb_filename(instance, filename):
        return instance.name
    name = models.ForeignKey(File)
    nzb = models.FileField(upload_to=get_nzb_filename)

And in create method -

File_name = validated_data.pop(file_name_data)
file = File.objects.create(validated_data)
F = FileName.objects.create(name=file, **File_name)

Upvotes: 0

awesoon
awesoon

Reputation: 33671

Here are 2 possible solutions:

Retrieve id before inserting a row

For simplicity I use postgresql db, although it is possible to adjust implementation for your db backend.

By default django creates id as bigserial (or serial depending on DEFAULT_AUTO_FIELD). For example, this model:

class File(models.Model):
    nzb = models.FileField(upload_to=get_nzb_filename)
    name = models.CharField(max_length=256)

Produces the following DDL:

CREATE TABLE "example_file" ("id" bigserial NOT NULL PRIMARY KEY, "nzb" varchar(100) NOT NULL, "name" varchar(256) NOT NULL);

There is no explicit sequence specification. By default bigserial creates sequence name in the form of tablename_colname_seq (example_file_id_seq in our case)

The solution is to retrieve this id using nextval :

def get_nextval(model, using=None):
    seq_name = f"{model._meta.db_table}_id_seq"
    if using is None:
        using = "default"
    with connections[using].cursor() as cursor:
        cursor.execute("select nextval(%s)", [seq_name])
        return cursor.fetchone()[0]

And set it before saving the model:

class File(models.Model):
    # fields definition

    def save(
        self, force_insert=False, force_update=False, using=None, update_fields=None
    ):
        if not self.pk:
            self.pk = get_nextval(self, using=using)
            force_insert = True
        super().save(
            force_insert=force_insert,
            force_update=force_update,
            using=using,
            update_fields=update_fields,
        )

Note that we rely on force_insert behavior, so make sure to read documentation and cover your code with tests:

from django.core.files.uploadedfile import SimpleUploadedFile
from django.forms import ModelForm
from django.test import TestCase

from example import models


class FileForm(ModelForm):
    class Meta:
        model = models.File
        fields = (
            "nzb",
            "name",
        )


class FileTest(TestCase):
    def test(self):
        form = FileForm(
            {
                "name": "picture",
            },
            {
                "nzb": SimpleUploadedFile("filename", b"content"),
            },
        )
        self.assertTrue(form.is_valid())
        form.save()

        self.assertEqual(models.File.objects.count(), 1)
        f = models.File.objects.first()
        self.assertRegexpMatches(f.nzb.name, rf"files/{f.pk}_picture(.*)\.nzb")

Insert without nzt then update with actual nzt value

The idea is self-explanatory - we basically pop nzt on the object creation and save object again after we know id:

def save(
    self, force_insert=False, force_update=False, using=None, update_fields=None
):
    nzb = None
    if not self.pk:
        nzb = self.nzb
        self.nzb = None

    super().save(
        force_insert=force_insert,
        force_update=force_update,
        using=using,
        update_fields=update_fields,
    )

    if nzb:
        self.nzb = nzb
        super().save(
            force_insert=False,
            force_update=True,
            using=using,
            update_fields=["nzb"],
        )

Test is updated to check actual queries:

def test(self):
    form = FileForm(
        {
            "name": "picture",
        },
        {
            "nzb": SimpleUploadedFile("filename", b"content"),
        },
    )
    self.assertTrue(form.is_valid())
    with CaptureQueriesContext(connection) as ctx:
        form.save()

    self.assertEqual(models.File.objects.count(), 1)
    f = models.File.objects.first()
    self.assertRegexpMatches(f.nzb.name, rf"files/{f.pk}_picture(.*)\.nzb")

    self.assertEqual(len(ctx.captured_queries), 2)
    insert, update = ctx.captured_queries
    self.assertEqual(
        insert["sql"],
        '''INSERT INTO "example_file" ("nzb", "name") VALUES ('', 'picture') RETURNING "example_file"."id"''',
    )
    self.assertRegexpMatches(
        update["sql"],
        rf"""UPDATE "example_file" SET "nzb" = 'files/{f.pk}_picture(.*)\.nzb' WHERE "example_file"."id" = {f.pk}""",
    )

Upvotes: 1

Max Koniev
Max Koniev

Reputation: 141

You can create pre_save and post_save signals. Actual file saving will be in post_save, when pk is already created. Do not forget to include signals in app.py so they work. Here is an example:

_UNSAVED_FILE_FIELD = 'unsaved_file'


@receiver(pre_save, sender=File)
def skip_saving_file_field(sender, instance: File, **kwargs):
    if not instance.pk and not hasattr(instance, _UNSAVED_FILE_FIELD):
        setattr(instance, _UNSAVED_FILE_FIELD, instance.image)
        instance.nzb = None


@receiver(post_save, sender=File)
def save_file_field(sender, instance: Icon, created, **kwargs):
    if created and hasattr(instance, _UNSAVED_FILE_FIELD):
        instance.nzb = getattr(instance, _UNSAVED_FILE_FIELD)
        instance.save()

Upvotes: 2

Mikolaj Buchwald
Mikolaj Buchwald

Reputation: 270

You can use the next available primary key ID:

class Document(models.Model):
    def upload_path(self, filename):
        if not self.pk:
            document_next_id = Document.objects.order_by('-id').first().id + 1
            self.id = self.pk = document_next_id
        return "my/path/document-%s" % str(self.pk)
    document = models.FileField(upload_to=upload_path)

Details

My example is a modification of @vinyll's answer, however, the problem Giles mentioned in his comment (two objects being created) is resolved here.

I am aware that my answer is not perfect, and there can be issues with the "next available ID", e.g., when more users will attempt to submit many forms at once. Giles's answer is more robust, mine is simpler (no need to generate temp files, then moving files, and deleting them). For simpler applications, this will be enough.

Also credits to Tjorriemorrie for the clear example on how to get the next available ID of an object.

Upvotes: 0

vinyll
vinyll

Reputation: 11409

Context

Had the same issue. Solved it attributing an id to the current object by saving the object first.

Method

  1. create a custom upload_to function
  2. detect if object has pk
  3. if not, save instance first, retrieve the pk and assign it to the object
  4. generate your path with that

Sample working code :

class Image(models.Model):
    def upload_path(self, filename):
        if not self.pk:
            i = Image.objects.create()
            self.id = self.pk = i.id
        return "my/path/%s" % str(self.id)
    file = models.ImageField(upload_to=upload_path)

Upvotes: 2

Giles
Giles

Reputation: 362

You can do this by setting upload_to to a temporary location and by creating a custom save method.

The save method should call super first, to generate the primary key (this will save the file to the temporary location). Then you can rename the file using the primary key and move it to it's proper location. Call super one more time to save the changes and you are good to go! This worked well for me when I came across this exact issue.

For example:

class File( models.Model ):
    nzb = models.FileField( upload_to='temp' )

    def save( self, *args, **kwargs ):
        # Call save first, to create a primary key
        super( File, self ).save( *args, **kwargs )

        nzb = self.nzb
        if nzb:
            # Create new filename, using primary key and file extension
            oldfile = self.nzb.name
            dot = oldfile.rfind( '.' )
            newfile = str( self.pk ) + oldfile[dot:]

            # Create new file and remove old one
            if newfile != oldfile:
                self.nzb.storage.delete( newfile )
                self.nzb.storage.save( newfile, nzb )
                self.nzb.name = newfile 
                self.nzb.close()
                self.nzb.storage.delete( oldfile )

        # Save again to keep changes
        super( File, self ).save( *args, **kwargs )

Upvotes: 4

Dana Woodman
Dana Woodman

Reputation: 4522

Ty, is there a reason you rolled your own slugify filter?

Django ships with a built-in slugify filter, you can use it like so:

from django.template.defaultfilters import slugify

slug = slugify(some_string)

Not sure if you were aware it was available to use...

Upvotes: 0

muhuk
muhuk

Reputation: 16085

It seems you'll need to pre-generate your File models with empty file fields first. Then pick up one and save it with the given file object.

You can have a custom manager method like this;

def create_with_pk(self):
    instance = self.create()
    instance.save()     # probably this line is unneeded
    return instance

But this will be troublesome if either of your fields is required. Because you are initially creating a null object, you can't enforce required fields on the model level.

EDIT

create_with_pk is supposed to be a custom manager method, in your code it is just a regular method. Hence self is meaningless. It is all properly documented with examples.

Upvotes: 4

Related Questions