CODEWITHSUNDEEP
pythonsqlalchemy
lowercase00
lowercase00

Reputation: 2576

Bulk INSERT IGNORE using Flask-SQLAlchemy

I'm trying to update a database using API-gathered data, and I need to make sure all tables are being updated.

Sometime I will receive data that's already in the database, so I want to do an INSERT IGNORE.

My current code is something like this:

def update_orders(new_orders):

    entries = []
    for each_order in new_orders:

            shipping_id = each_order['id']
            title = each_order['title']
            price = each_order['price']
            code = each_order['code']
            source = each_order['source']
            phone = each_order['phone']
            category = each_order['delivery_category']
            carrier = each_order['carrier_identifier']

            new_entry = Orders(
                id=shipping_id,
                title=title,
                code=code,
                source=source,
                phone=phone,
                category=category,
                carrier=carrier,
                price=price
            )

            entries.append(new_entry)

    if len(entries) == 0:
        print('No new orders.')
        break
    else:   
        print('New orders:', len(entries))
        db.session.add_all(entries)
        db.session.commit()

This works well when I'm creating the database from scratch, but it will give me an error if there's duplicate data, and I'm not able to commit the inserts.

I've been reading for a while, and found a workaround that uses prefix_with:

    print('New orders:', len(entries))

    if len(entries) == 0:
        print('No new orders.')
    else:   
        insert_command = Orders.__table__.insert().prefix_with('OR IGNORE').values(entries)
        db.session.execute(insert_command)
        db.session.commit()

The problem is that values(entries) is a bunch of objects: <shop.database.models.Orders object at 0x11986def0> instead of being the instance of the class, is the class instance object in memory.

Anybody has any suggestion on approaching this problem? Feel free to suggest a different approach, or just an adjustment.

Thanks a lot.

Upvotes: 2

Views: 2062

Answers (1)

taka-li-e
taka-li-e

Reputation: 21

What database are you using ? Under MySQL, "INSERT OR IGNORE" is not valid syntax, instead one should use "INSERT IGNORE". I had the same situation and got my query to work with the following:

insert_command = Orders.__table__.insert().prefix_with(' IGNORE').values(entries)

Upvotes: 1

Related Questions