flask-migrate/alembic fails with `ValueError: not enough values to unpack (expected 2, got 1)` when running db migrate

Question

I have created a Flask app which currently has around ~30 models. So far, when I was testing everything locally, I had a local Postgresql docker container on which I would use SQLAlchemy().create_all() to create all tables. That worked great so far. However, I figured that in order to make this app more production-ready, I should probably start using migrations rather than running create_all() every time I make schema changes. So, I implemented the migration commands by following Flask-migrate docs, ran python manage.py db init successfully but then when I run python manage.py db migrate I get a pretty long stack trace which ends with ValueError: not enough values to unpack (expected 2, got 1). I have found this previously-asked question which matches my error, and I tried following the workaround (I only have one table with __table_args__ which specifies two check constraints, like this):

    __table_args__ = (
        db.CheckConstraint('minutes >= 0', name='check_minutes_positive'),
        db.CheckConstraint('seconds between 0 and 59', name='check_seconds_valid')
    )

but even when I commented that out, I'm still getting the same error. I even went as far as commenting-out every field in every model except for the primary keys, but still got the same error.

I've tried increasing the log levels to DEBUG on every logger in the alembic.ini file, but that didn't help in pointing me out to the root cause of this error. The last call in the stack trace points to api/.venv/lib/python3.8/site-packages/alembic/autogenerate/render.py", line 808, in _fk_colspec, but that doesn't tell me much since I can't figure out what might be missing on my end which is causing the ValueError. I'd really appreciate any help with understanding how to find the problem.

Here's the full output of the migrate command (I have only removed most of the initial INFO lines showing table names and index created, since I figured they aren't really relevant. Only left the first and last ones):

INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO  [alembic.runtime.migration] Will assume transactional DDL.
INFO  [alembic.autogenerate.compare] Detected added table 'tAbstractQuestions'
INFO  [alembic.autogenerate.compare] Detected added index 'ix_tAbstractQuestions_questionTypeId' on '['questionTypeId']'
INFO  [alembic.autogenerate.compare] Detected added table 'tQuestionsOptions'
INFO  [alembic.autogenerate.compare] Detected added index 'ix_tQuestionsOptions_questionId' on '['questionId']'
INFO  [alembic.autogenerate.compare] Detected added index 'ix_tQuestionsOptions_questionOptionId' on '['questionOptionId']'
Traceback (most recent call last):
  File "manage.py", line 70, in <module>
    manager.run()
  File "/api/.venv/lib/python3.8/site-packages/flask_script/__init__.py", line 417, in run
    result = self.handle(argv[0], argv[1:])
  File "/api/.venv/lib/python3.8/site-packages/flask_script/__init__.py", line 386, in handle
    res = handle(*args, **config)
  File "/api/.venv/lib/python3.8/site-packages/flask_script/commands.py", line 216, in __call__
    return self.run(*args, **kwargs)
  File "/api/.venv/lib/python3.8/site-packages/flask_migrate/__init__.py", line 96, in wrapped
    f(*args, **kwargs)
  File "/api/.venv/lib/python3.8/site-packages/flask_migrate/__init__.py", line 210, in migrate
    command.revision(config, message, autogenerate=True, sql=sql,
  File "/api/.venv/lib/python3.8/site-packages/alembic/command.py", line 221, in revision
    scripts = [script for script in revision_context.generate_scripts()]
  File "/api/.venv/lib/python3.8/site-packages/alembic/command.py", line 221, in <listcomp>
    scripts = [script for script in revision_context.generate_scripts()]
  File "/api/.venv/lib/python3.8/site-packages/alembic/autogenerate/api.py", line 513, in generate_scripts
    yield self._to_script(generated_revision)
  File "/api/.venv/lib/python3.8/site-packages/alembic/autogenerate/api.py", line 425, in _to_script
    render._render_python_into_templatevars(
  File "/api/.venv/lib/python3.8/site-packages/alembic/autogenerate/render.py", line 49, in _render_python_into_templatevars
    _render_cmd_body(upgrade_ops, autogen_context)
  File "/api/.venv/lib/python3.8/site-packages/alembic/autogenerate/render.py", line 71, in _render_cmd_body
    lines = render_op(autogen_context, op)
  File "/api/.venv/lib/python3.8/site-packages/alembic/autogenerate/render.py", line 87, in render_op
    lines = util.to_list(renderer(autogen_context, op))
  File "/api/.venv/lib/python3.8/site-packages/alembic/autogenerate/render.py", line 179, in _add_table
    for rcons in [
  File "/api/.venv/lib/python3.8/site-packages/alembic/autogenerate/render.py", line 180, in <listcomp>
    _render_constraint(cons, autogen_context)
  File "/api/.venv/lib/python3.8/site-packages/alembic/autogenerate/render.py", line 775, in _render_constraint
    return renderer(constraint, autogen_context)
  File "/api/.venv/lib/python3.8/site-packages/alembic/autogenerate/render.py", line 871, in _render_foreign_key
    "refcols": ", ".join(
  File "/api/.venv/lib/python3.8/site-packages/alembic/autogenerate/render.py", line 872, in <genexpr>
    repr(_fk_colspec(f, apply_metadata_schema))
  File "/api/.venv/lib/python3.8/site-packages/alembic/autogenerate/render.py", line 808, in _fk_colspec
    tname, colname = tokens[-2:]
ValueError: not enough values to unpack (expected 2, got 1)

Thanks!

Answer 1

I posted the same issue on Alembic's GitHub repo, and was pointed to the right direction there. Link to the issue.

Pasting the solution below as well:

Found it! I decided I'm diving deeper. So what I did is to comment-out every table that had a foreign key in it and ran migrate - worked. Then I started adding one table at a time from the ones that did have a foreign key and voila! I found that one of the tables had a db.foreignKey('tableName') (without specifying the actual column on which the constraint should apply 🤦 ). So no bug here, but I guess a more descriptive error would've saved me a good bit of time spent over this, lol. To verify- after fixing the misconfiguration, I cleaned up the migrations folder and restarted my PostgreSQL container (which started a brand new db from scratch) and ran the migrate command to create a single revision which includes all models at once- and it works! 🎉