value too long for type character varying(100)----recently switched database, didn't do anything in db

Question

I recently switched to postgresql, I thought everything worked fine until I realized when I make post I get value too long for type character varying(100). Now I googled it, and saw some similar questions but when I tried some solutions there none of them worked. I will explain why my question is different in my opinion. I have this code in models.py

class Post(models.Model):
    url = models.URLField(max_length=250, blank=True, null=True)

    slug = models.CharField(max_length=255, unique=True)
    objects = models.Manager()    

@property

def save(self, *args, **kwargs):
    self.slug = uuslug(self.title, instance=self, max_length=255)
    super(Post, self).save(*args, **kwargs)

As I saw some recommended solutions, I did try changing my max_length to 100. I have no idea why this is happening, I don't have anything in db. I recently switched to postgresql. Can you help me why this error is occurring and how I can fix it? should I move away from uuslug?

Full model

.
class Category(models.Model): 

    name = models.CharField(max_length=128, unique=True)
    description = models.TextField(verbose_name=('describe'))
    author = models.ForeignKey(settings.AUTH_USER_MODEL)


    def __unicode__(self): 
        return self.name

    def get_absolute_url(self):
        return "/category/%s/" %self.name

def my_handler(sender, instance, created, **kwargs):
    action.send(instance.author, verb='following', target=Category)
post_save.connect(my_handler, sender=Category)


class Post(models.Model):
    category = models.ForeignKey(Category, verbose_name=('community'))
    pub_date = models.DateTimeField(auto_now_add = True)
    url = models.URLField(max_length=250, blank=True, null=True)
    video = EmbedVideoField(verbose_name='link',help_text="Youtube", blank=True, null=True) 

    title = models.CharField(max_length = 50)
    moderator = models.ForeignKey(User)
    views = models.IntegerField(default=0)
    slug = models.CharField(max_length=255, unique=True)
    objects = models.Manager()            # default manager
    content = RichTextUploadingField(config_name='default')
    rank_score = models.FloatField(default= 1)
    image = models.ImageField(upload_to='images',blank=True, null=True)
    thumbnail = models.ImageField(upload_to='images', blank=True, null=True)


    @property
    def domain(self):
        long_url = urlparse(self.url).netloc if self.url else "be kind to one another"
        return long_url.split('.', 1)[1] if long_url.split('.', 1)[0] == 'www' else long_url
    def save(self, *args, **kwargs):
        self.slug = uuslug(self.title, instance=self, max_length=255)
        super(Post, self).save(*args, **kwargs)
    def __unicode__(self):
        return self.title 

Here's full Traceback

T

raceback:
File "env/local/lib/python2.7/site-packages/django/core/handlers/base.py" in get_response
  132.                     response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "env/local/lib/python2.7/site-packages/django/views/generic/base.py" in view
  71.             return self.dispatch(request, *args, **kwargs)
File "env/local/lib/python2.7/site-packages/django/utils/decorators.py" in _wrapper
  34.             return bound_func(*args, **kwargs)
File "env/local/lib/python2.7/site-packages/django/contrib/auth/decorators.py" in _wrapped_view
  22.                 return view_func(request, *args, **kwargs)
File "env/local/lib/python2.7/site-packages/django/utils/decorators.py" in bound_func
  30.                 return func.__get__(self, type(self))(*args2, **kwargs2)
File "ebagu/main/views.py" in dispatch
  191.      return super(PostCreateView, self).dispatch(request, *args, **kwargs)       
File "env/local/lib/python2.7/site-packages/django/views/generic/base.py" in dispatch
  89.         return handler(request, *args, **kwargs)
File "env/local/lib/python2.7/site-packages/django/views/generic/edit.py" in post
  249.         return super(BaseCreateView, self).post(request, *args, **kwargs)
File "env/local/lib/python2.7/site-packages/django/views/generic/edit.py" in post
  215.             return self.form_valid(form)
File "ebagu/main/views.py" in form_valid
  186.          self.object.save()
File "ebagu/main/models.py" in save
  66.       super(Post, self).save(*args, **kwargs)
File "env/local/lib/python2.7/site-packages/django/db/models/base.py" in save
  734.                        force_update=force_update, update_fields=update_fields)
File "env/local/lib/python2.7/site-packages/django/db/models/base.py" in save_base
  762.             updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields)
File "env/local/lib/python2.7/site-packages/django/db/models/base.py" in _save_table
  846.             result = self._do_insert(cls._base_manager, using, fields, update_pk, raw)
File "env/local/lib/python2.7/site-packages/django/db/models/base.py" in _do_insert
  885.                                using=using, raw=raw)
File "env/local/lib/python2.7/site-packages/django/db/models/manager.py" in manager_method
  127.                 return getattr(self.get_queryset(), name)(*args, **kwargs)
File "env/local/lib/python2.7/site-packages/django/db/models/query.py" in _insert
  920.         return query.get_compiler(using=using).execute_sql(return_id)
File "env/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py" in execute_sql
  974.                 cursor.execute(sql, params)
File "env/local/lib/python2.7/site-packages/django/db/backends/utils.py" in execute
  79.             return super(CursorDebugWrapper, self).execute(sql, params)
File "env/local/lib/python2.7/site-packages/django/db/backends/utils.py" in execute
  64.                 return self.cursor.execute(sql, params)
File "env/local/lib/python2.7/site-packages/django/db/utils.py" in __exit__
  97.                 six.reraise(dj_exc_type, dj_exc_value, traceback)
File "env/local/lib/python2.7/site-packages/django/db/backends/utils.py" in execute
  64.                 return self.cursor.execute(sql, params)

Exception Type: DataError at /add_post/
Exception Value: value too long for type character varying(100)

Answer 1

The error is due to already saved data in your database being longer than the given characters. You may want to look at the data and consider fixing it or whether the given characters are really the limit you want.

If you are maintaining history for model than you also need to delete data in history table.

Answer 2

Removing the migration files from previous database should solve your problem.

Answer 3

Look into database.

---

Quick psql tutorial

Start postgres client shell, eg. with python manage.py dbshell. Something like this shows:

ebagu=>

The lines

File "ebagu/main/models.py" in save
  66.       super(Post, self).save(*args, **kwargs)

show you are trying to save model Post in what I assume is app main. If you didn't change table in Post's Meta, the table should be something like main_post.

You want to display all tables in postgres shell with command:

\d

It will list tables in a similar fashion:

                          List of relations
 Schema |               Name                |     Type      | Owner  
--------+-----------------------------------+---------------+--------
 public | django_migrations                 | table         | hruske
 public | django_migrations_id_seq          | sequence      | hruske
 public | django_session                    | table         | hruske

If you want to check the details of the table django_session you can do:

\d django_session

And you get something like this:

             Table "public.django_session"
    Column    |           Type           | Modifiers 
--------------+--------------------------+-----------
 session_key  | character varying(40)    | not null
 session_data | text                     | not null
 expire_date  | timestamp with time zone | not null
Indexes:
    "django_session_pkey" PRIMARY KEY, btree (session_key)
    "django_session_de54fa62" btree (expire_date)
    "django_session_session_key_461cfeaa630ca218_like" btree (session_key varchar_pattern_ops)

Here you can see which fields are defined as varchar(100).

If you want to know more psql commands, type \?

Answer 4

I don't think you need help fixing this problem, so much as you need help debugging it. Once the problem is clear, the solution seems clear as well. The Traceback is perhaps a bit unclear because it's going through so much Django source code and it's not telling you which of your fields is having an issue.

Background To This Issue

To start with, we're having problems saving a Post instance. Well, look at all of these fields you have in your model definition:

 ...
  url = models.URLField(max_length=250, blank=True, null=True)
  video = EmbedVideoField(verbose_name='link',help_text="Youtube", blank=True, null=True) 
  content = RichTextUploadingField(config_name='default')
  image = models.ImageField(upload_to='images',blank=True, null=True)
  thumbnail = models.ImageField(upload_to='images', blank=True, null=True)

These may not look like text fields, but a lot of them are variations on text fields because, if you think about it, you're probably not going to store whole files in your database. What you will do instead (and what Django does by default) is store the file somewhere on some disk and then in the database you would store the path to that file so you can retrieve it when you need to.

Further, it's probably a waste to store file-paths in the db as LongText or whatever, so every FileField we have means we've got a field with a max_length whether we specify it or not. Thus, all of the above fields have an implicit max_length. You can actually find this out by reading the Django source code.

Source Examples

I've never used EmbedVideoField, for example, but it turns out to be a subclass of models.URLField, which means it has a max_length by default set to 200 if you do not specify one.

Moreover, your various ImageFields are just subclasses of FileField, which has a max_length default of 100.

How to Debug Problems Like This In the Future?

Now, this doesn't help us to know which of your fields is throwing an error in this case. For that, I would probably set a breakpoint somewhere in the code, probably here:

File "ebagu/main/models.py" in save
   66.       super(Post, self).save(*args, **kwargs)

By "set a breakpoint", I mean the following:

Go to line 65 in the module mentioned above, ebagu/main/models.py and enter the following and save the module: import pdb; pdb.set_trace()

(I actually have a strong preference for ipdb myself, but that requires Ipython, which I also have a strong preference for...)

Run your local server, and run through the steps that produced this issue. You will end up submitting your form and if you look at the console where you started your server, you'll eventually be dumped into a shell right at line 65. This shell is a pdb shell, which has different rules from a normal shell, but you can evaluate your about-to-be-saved Post instance, by looking at the various fields on the instance itself, self, and running Python code in the context of that method call:

(pdb) len(self.image.path)

Using that, I would manually evaluate the various fields and look at which one has this really long entry that is choking the save (probably one of your ImageFields).

Solution with Warnings

Alternately, you can just add a max_length to all of these, but be forewarned that you will most likely need to perform database migrations for any limited text field you change because your database is still going to verify the length of the input against how the column is defined. Here's a good StackOverflow answer looking at exactly this problem.

Footnote

Why didn't this come up before you switched to Postgresql? There are a variety of potential reasons, but it probably has to do with how the previous database was setup vs how the Postgresql database was set up (manually vs Django migrations?).

It may also have to do with whether or not you changed where these things are being stored. Did you change your MEDIA settings so the paths where files are stored got a lot longer?

What you really should be doing is looking directly at your database. Open up a psql instance and ask it to describe your tables for you. It'll tell you which fields are limited to 100 characters and those are the fields giving you problems.