CODEWITHSUNDEEP
pythondatabasesqlalchemy
drgxfs
drgxfs

Reputation: 1127

List database tables with SQLAlchemy

I want to implement a function that gives information about all the tables (and their column names) that are present in a database (not only those created with SQLAlchemy). While reading the documentation it seems to me that this is done via reflection but I didn't manage to get something working. Any suggestions or examples on how to do this?

Upvotes: 62

Views: 61174

Answers (5)

Zephaniah Grunschlag
Zephaniah Grunschlag

Reputation: 1084

I'm proposing another solution as I was not satisfied by any of the previous in the case of postgres which uses schemas. I hacked this solution together by looking into the pandas source code.

from sqlalchemy import MetaData, create_engine
from typing import List

def list_tables(pg_uri: str, schema: str) -> List[str]:
    with create_engine(pg_uri).connect() as conn:
        meta = MetaData(conn, schema=schema)
        meta.reflect(views=True)
        return list(meta.tables.keys())

In order to get a list of all tables in your schema, you need to form your postgres database uri pg_uri (e.g. "postgresql://u:p@host/database" as in the zzzeek's answer) as well as the schema's name schema. So if we use the example uri as well as the typical schema public we would get all the tables and views with:

list_tables("postgresql://u:p@host/database", "public")

Upvotes: 3

Rohit Lal
Rohit Lal

Reputation: 3299

First set up the sqlalchemy engine.

from sqlalchemy import create_engine, inspect, text
from sqlalchemy.engine import url

connect_url = url.URL(
    'oracle',
    username='db_username',
    password='db_password',
    host='db_host',
    port='db_port',
    query=dict(service_name='db_service_name'))

engine = create_engine(connect_url)

try:
    engine.connect()
except Exception as error:
    print(error)
    return

Like others have mentioned, you can use the inspect method to get the table names.

But in my case, the list of tables returned by the inspect method was incomplete.

So, I found out another way to find table names by using pure SQL queries in sqlalchemy.

query = text("SELECT table_name FROM all_tables where owner = '%s'"%str('db_username'))

table_name_data = self.session.execute(query).fetchall()

Just for sake of completeness of answer, here's the code to fetch table names by inspect method (if it works good in your case).

inspector = inspect(engine)
table_names = inspector.get_table_names()

Upvotes: 5

user2682863
user2682863

Reputation: 3217

While reflection/inspection is useful, I had trouble getting the data out of the database. I found sqlsoup to be much more user-friendly. You create the engine using sqlalchemy and pass that engine to sqlsoup.SQlSoup. ie:

import sqlsoup

def create_engine():
    from sqlalchemy import create_engine
    return create_engine(f"mysql+mysqlconnector://{database_username}:{database_pw}@{database_host}/{database_name}")

def test_sqlsoup():
    engine = create_engine()
    db = sqlsoup.SQLSoup(engine)
    # Note: database must have a table called 'users' for this example
    users = db.users.all()
    print(users)

if __name__ == "__main__":
    test_sqlsoup()

If you're familiar with sqlalchemy then you're familiar with sqlsoup. I've used this to extract data from a wordpress database.

Upvotes: 0

PepperoniPizza
PepperoniPizza

Reputation: 9102

Hey I created a small module that helps easily reflecting all tables in a database you connect to with SQLAlchemy, give it a look: EZAlchemy

from EZAlchemy.ezalchemy import EZAlchemy

DB = EZAlchemy(
    db_user='username',
    db_password='pezzword',
    db_hostname='127.0.0.1',
    db_database='mydatabase',
    d_n_d='mysql'   # stands for dialect+driver
)

# this function loads all tables in the database to the class instance DB
DB.connect()

# List all associations to DB, you will see all the tables in that database
dir(DB)

Upvotes: 3

zzzeek
zzzeek

Reputation: 75127

start with an engine:

from sqlalchemy import create_engine
engine = create_engine("postgresql://u:p@host/database")

quick path to all table /column names, use an inspector:

from sqlalchemy import inspect
inspector = inspect(engine)

for table_name in inspector.get_table_names():
   for column in inspector.get_columns(table_name):
       print("Column: %s" % column['name'])

docs: http://docs.sqlalchemy.org/en/rel_0_9/core/reflection.html?highlight=inspector#fine-grained-reflection-with-inspector

alternatively, use MetaData / Tables:

from sqlalchemy import MetaData
m = MetaData()
m.reflect(engine)
for table in m.tables.values():
    print(table.name)
    for column in table.c:
        print(column.name)

docs: http://docs.sqlalchemy.org/en/rel_0_9/core/reflection.html#reflecting-all-tables-at-once

Upvotes: 118

Related Questions