CODEWITHSUNDEEP
pythonpostgresqlsqlalchemy
abinitio
abinitio

Reputation: 91

sqlalchemy.exc.DataError: (psycopg2.DataError) integer out of range

I am trying to save data into postgresql via SQLAchemy ORM. I encountered an error below:

sqlalchemy.exc.DataError: (psycopg2.DataError) integer out of range

I pinpointed the place where it goes wrong. I have a large number which is 2468432255.0. If I change to smaller number like 468432255.0, then it works.

The thing confused me is that: I defined the column as volume = Column(Numeric). As far as I understand, Numeric should be able to handle this large number. Additionally, I tried other data type like BigInt etc... they all gave me the same error.

Any idea?

Thanks, Chengjun

Upvotes: 9

Views: 11825

Answers (2)

Brian Burns
Brian Burns

Reputation: 22012

I was getting this same error -

Here's some test code - it's okay when using the object interface, but bombs using plain SQL:

from sqlalchemy import create_engine
from sqlalchemy import MetaData, Table, Column, BigInteger

dsn = 'postgres://postgres@localhost/testdb'

engine = create_engine(dsn)
metadata = MetaData(engine)

# create table
tbl = Table('test', metadata, Column('id', BigInteger))
tbl.drop(checkfirst=True)
tbl.create(checkfirst=True)

# use object chaining - works
cmd = tbl.insert().values(id=123456789012)
engine.execute(cmd)

# pass a dictionary of values - works
cmd = tbl.insert(values = dict(id=123456789013))
engine.execute(cmd)

# pass a list of values - works
cmd = tbl.insert(values = [123456789014])
engine.execute(cmd)

# use plain sql - bombs!
# error - sqlalchemy.exc.DataError: (psycopg2.DataError) integer out of range
# sql = "INSERT INTO test (id) VALUES (123456789015);"
# engine.execute(sql)

# show table contents
cmd = tbl.select(True)
result = engine.execute(cmd)
for row in result:
    print(row)
# (123456789012,)
# (123456789013,)
# (123456789014,)

# show table definition
for t in metadata.sorted_tables:
    print("Table name: ", t.name)
    for c in t.columns:
        print("  ", c.name, c.type)
# Table name:  test
#    id BIGINT

I don't know why it bombs - at some point psycopg2 must be trying to convert the bigint to an int.

Upvotes: -2

Ricky Levi
Ricky Levi

Reputation: 7987

You can define anything you want in the SQlAlchemy schema in your local code, It doesn't mean it will be honored by the DB you're inserting the data into.

The schema that is defined in SQLAlchemy is being enforced by the code itself.

While the DB has its own schema that is also enforcing it when you're trying to insert/delete etc .. ( has its own constraints etc .. ) that SQLAlchemy knows nothing about ( until you declare it ).

In my opinion you can simply generate the SQLAlchemy schema automatically - and it will take the DB column name & types from the DB schema.

from sqlalchemy import create_engine

class SomeTable(Base):
    """
    Class, that represents SomeTable
    """
    __tablename__ = "my_table_in_db"

    __table__ = Table(__tablename__, Base.metadata,
        autoload=True,
        autoload_with=create_engine(DB_URL))

And after you'll create a SomeTable object, you can access its columns simply by

SomeTable.colname

colname is a column that currently exists in the DB

Upvotes: 2

Related Questions