CODEWITHSUNDEEP
databasepostgresqlsqlitedatabase-migrationdump
luqui
luqui

Reputation: 60463

How to convert a postgres database to SQLite?

We're working on a website, and when we develop locally (one of us from Windows), we use sqlite3, but on the server (linux) we use postgres. We'd like to be able to import the production database into our development process, so I'm wondering if there is a way to convert from a postgres database dump to something sqlite3 can understand (just feeding it the postgres's dumped SQL gave many, many errors). Or would it be easier just to install postgres on windows? Thanks.

Upvotes: 37

Views: 65078

Answers (7)

tutuDajuju
tutuDajuju

Reputation: 10860

2024 update - doing this by updating a SQL dump can be very complicated, or very simple, depending on what Postgres-specific features (e.g hstore columns) you use.

For a simple use case, I found this blog entry which guides you to do these steps:

  1. Create a dump of the PostgreSQL database.

     ssh -C [email protected] pg_dump --data-only --inserts YOUR_DB_NAME > dump.sql

  2. Remove/modify the dump.

    1. Remove the lines starting with SET
    2. Remove the lines starting with SELECT pg_catalog.setval
    3. Replace true for ‘t’
    4. Replace false for ‘f’

  3. Add BEGIN; as first line and END; as last line

  4. Recreate an empty development database. bundle exec rake db:migrate

  5. Import the dump.

     sqlite3 db/development.sqlite3
 sqlite> delete from schema_migrations;
 sqlite> .read dump.sql

Of course connecting via ssh and creating a new db using rake are optional.

If you'd like to expand on this, please feel free to add more details here and I'll edit the answer.

Upvotes: 28

Kirguduck
Kirguduck

Reputation: 796

when I faced with same issue I did not find any useful advices on Internet. My source PostgreSQL db had very complicated schema.
You just need to remove from your db-file manually everything besides table creating
More details - here

Upvotes: 0

luqui
luqui

Reputation: 60463

Even though there are many very good helpful answers here, I just want to mark this as answered. We ended up going with the advice of the comments:

I'd just switch your development environment to PostgreSQL, developing on top of one database (especially one as loose and forgiving as SQLite) but deploying on another (especially one as strict as PostgreSQL) is generally a recipe for aggravation and swearing. – @mu is too short

To echo mu's response, DON'T DO THIS..DON'T DO THIS..DON'T DO THIS. Develop and deploy on the same thing. It's bad engineering practice to do otherwise. – @Kuberchaun

So we just installed postgres on our dev machines. It was easy to get going and worked very smoothly.

Upvotes: 3

caiiiycuk
caiiiycuk

Reputation: 1596

You can use pg2sqlite for converting pg_dump output to sqlite.

#  Making dump
pg_dump -h host -U user -f database.dump database

#  Making sqlite database
pg2sqlite -d database.dump -o sqlite.db

Schemas is not supported by pg2sqlite, and if you dump contains schema then you need to remove it. You can use this script:

# sed 's/<schema name>\.//' -i  database.dump
sed 's/public\.//' -i  database.dump
pg2sqlite -d database.dump -o sqlite.db

Upvotes: 9

Artem Zaytsev
Artem Zaytsev

Reputation: 1695

STEP1: make a dump of your database structure and data

pg_dump --create --inserts -f myPgDump.sql \
   -d myDatabaseName -U myUserName -W myPassword

STEP2: delete everything except CREATE TABLES and INSERT statements out of myPgDump.sql (using text editor)

STEP3: initialize your SQLite database passing structure and data of your Postgres dump

sqlite3 myNewSQLiteDB.db -init myPgDump.sql

STEP4: use your database ;)

Upvotes: 14

random_user
random_user

Reputation: 848

In case one needs a more automatized solution, here's a head start:

#!/bin/bash

$table_name=TABLENAMEHERE

PGPASSWORD="PASSWORD" /usr/bin/pg_dump --file "results_dump.sql" --host "yourhost.com" --username "username" --no-password --verbose --format=p --create --clean --disable-dollar-quoting --inserts --column-inserts --table "public.${table_name}" "memseq"

# Some clean ups
perl -0777 -i.original -pe "s/.+?(INSERT)/\1/is" results_dump.sql
perl -0777 -i.original -pe "s/--.+//is" results_dump.sql

# Remove public. prefix from table name
sed -i "s/public.${table_name}/${table_name}/g" results_dump.sql

# fix binary blobs
sed -i "s/'\\\\x/x'/g" results_dump.sql

# use transactions to make it faster
echo 'BEGIN;' | cat - results_dump.sql > temp && mv temp results_dump.sql
echo 'END;' >> results_dump.sql

# clean the current table 
sqlite3 results.sqlite "DELETE FROM ${table_name};"

# finally apply changes
sqlite3 results.sqlite3 < results_dump.sql && \
rm results_dump.sql && \
rm results_dump.sql.original

Upvotes: 0

jpw
jpw

Reputation: 19247

It was VERY easy for me to do using the taps gem as described here: http://railscasts.com/episodes/342-migrating-to-postgresql

And I've started using the Postgres.app on my Mac (no install needed, drop the app in your Applications directory, although might have to add one line to your PATH envirnment variable as described in the documentation), with Induction.app as a GUI tool to view/query the database.

Upvotes: -2

Related Questions