Is there a way to get a schema of a database from within python?

Question

I'm trying to find out a way to find the names of tables in a database(if any exist). I find that from a sqlite cli I can use:

>.tables

Then for the fields:

>PRAGMA TABLE_INFO(table_name)

This obviously doesn't work within python. Is there even a way to do this with python or should I just be using the sqlite command-line?

Answer 1

Assuming the name of the database is my_db and the name of the table is my_table, to get the name of the columns and the datatypes:

con = sqlite.connect(my_db)
cur = con.cursor()
query =  "pragma table_info({})".format(my_table)
table_info = cur.execute(query).fetchall()

It returns a list of tuples. Each tuple has the order, name of the column and data type.

Answer 2

make the connection to the database

connection = connect_db('./database_name.db')

print the table names

table_names = [t[0] for t in connection.execute("SELECT name FROM sqlite_master WHERE type='table';")]
print(table_names)

Answer 3

To get the schema information, IMHO, below also works:

select sql from sqlite_master where type='table';

Answer 4

result sets have a description that you can get some information from. It reveals some basic metadata like column name and number of columns.

>>> rs = c.execute('''SELECT * FROM news WHERE 1=0''');
>>> dir(rs)
['__class__', '__delattr__', '__doc__', '__format__',  
'__getattribute__', '__hash__', '__init__', '__iter__', '__new__',  
'__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__',
'__str__', '__subclasshook__', 'arraysize', 'close', 'connection',
**'description'**, 'execute', 'executemany', 'executescript', 'fetchall',
'fetchmany', 'fetchone', 'lastrowid', 'next', 'row_factory',
'rowcount', 'setinputsizes', 'setoutputsize']



>>> print(rs.description)
(('id', None, None, None, None, None, None), 
('imageUrl', None, None, None, None, None, None), 
('headline', None, None, None, None, None, None), 
('who', None, None, None, None, None, None))

Answer 5

I just tried

SELECT name FROM my_db.sqlite_master WHERE type='table';

to combine Tom Kerr's answer and the attempt to retrieve information on an attached database. At first it didn't work. Turns out I first have to attach the other database this way:

ATTACH DATABASE 'file:my_other_database_file.db?cache=shared' as my_db;

otherwise the database will fail to obtain a read lock for the attached database's sqlite_master (and all queries will succeed with zero results). Just a hint in case anybody else stumbles upon that part of the issue.

Answer 6

Here's a convenient printer I wrote based Martijn's response:

def printSchema(connection):
    for (tableName,) in connection.execute(
        """
        select NAME from SQLITE_MASTER where TYPE='table' order by NAME;
        """
    ):
        print("{}:".format(tableName))
        for (
            columnID, columnName, columnType,
            columnNotNull, columnDefault, columnPK,
        ) in connection.execute("pragma table_info('{}');".format(tableName)):
            print("  {id}: {name}({type}){null}{default}{pk}".format(
                id=columnID,
                name=columnName,
                type=columnType,
                null=" not null" if columnNotNull else "",
                default=" [{}]".format(columnDefault) if columnDefault else "",
                pk=" *{}".format(columnPK) if columnPK else "",
            ))

Answer 7

Use sqlite row object. A row object has keys() that will give you the schema.

from docs.python.org

conn.row_factory = sqlite3.Row
c = conn.cursor()
c.execute('select * from stocks')
  <sqlite3.Cursor object at 0x7f4e7dd8fa80>
r = c.fetchone()
type(r)
  <type 'sqlite3.Row'>
  r
  (u'2006-01-05', u'BUY', u'RHAT', 100.0, 35.14)
r.keys()
  ['date', 'trans', 'symbol', 'qty', 'price']

Answer 8

From the sqlite FAQ:

From within a C/C++ program (or a script using Tcl/Ruby/Perl/Python bindings) you can get access to table and index names by doing a SELECT on a special table named "SQLITE_MASTER". Every SQLite database has an SQLITE_MASTER table that defines the schema for the database. The SQLITE_MASTER table looks like this:

CREATE TABLE sqlite_master (
  type TEXT,
  name TEXT,
  tbl_name TEXT,
  rootpage INTEGER,
  sql TEXT
);

So to get a list of all table names execute:

SELECT name FROM sqlite_master
WHERE type='table'
ORDER BY name;

To get column names for a given table, use the pragma table_info command:

This pragma returns one row for each column in the named table. Columns in the result set include the column name, data type, whether or not the column can be NULL, and the default value for the column.

This command works just fine from python:

>>> import sqlite3
>>> conn = sqlite3.connect(':mem:')
>>> for row in conn.execute("pragma table_info('sqlite_master')").fetchall():
...     print row
... 
(0, u'type', u'text', 0, None, 0)
(1, u'name', u'text', 0, None, 0)
(2, u'tbl_name', u'text', 0, None, 0)
(3, u'rootpage', u'integer', 0, None, 0)
(4, u'sql', u'text', 0, None, 0)

Unfortunately pragma statements do not work with parameters; you'll have to manually insert the table name (make sure it's not sourced from an untrusted source and escape it properly).

Answer 9

To get the field names, use cur.description after the query:

import sqlite3.dbapi2 as sqlite
con = sqlite.connect(":memory:")
cur = con.cursor()
con.executescript("""
    create table test (name, address);
    insert into test (name, address) values ("Jer", "Monterey Street");
""")

cur.execute("select * from test where 1=0")
rs = cur.fetchall()  ## will be [] because of where clause
field_names = [r[0] for r in cur.description]

Answer 10

You should be able access the table names from the sqlite_master table.

SELECT name FROM sqlite_master WHERE type='table';

The names of the columns are not directly accessible. The easiest way to get them is to query the table and get the column names from the query result.

SELECT * FROM table_name LIMIT 1;