CODEWITHSUNDEEP
python-3.xboto3aws-glueaws-glue-data-catalog
user3476463
user3476463

Reputation: 4575

get list of tables in database using boto3

I’m trying to get a list of the tables from a database in my aws data catalog. I’m trying to use boto3. I’m running the code below on aws, in a sagemaker notebook. It runs forever (like over 30 minutes) and doesn’t return any results. The test_db only has 4 tables in it. My goal is to run similar code as part of an aws glue etl job, that I would run in an edited aws etl job script. Does anyone see what the issue might be or suggest how to do this?

code:

import boto3
from pprint import pprint

glue = boto3.client('glue', region_name='us-east-2')

response = glue.get_tables(
    DatabaseName=‘test_db’
)

print(pprint(response['TableList']))

Upvotes: 8

Views: 16878

Answers (2)

OrangeElyisa
OrangeElyisa

Reputation: 16

You can use boto3 with the list_table_metadata method of the Athena client as long as you know the name of your data catalog https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/athena/client/list_table_metadata.html

import boto3

client = boto3.client("athena", region_name="us-east-2")
table_metadata = client.list_table_metadata(CatalogName=`CatalogName`, DatabaseName=`test_db`)
print(table_metadata)

This will return a dictionary of metadata about your tables in the database. If you wanted to just get the name of the tables you could then use some list comprehension:

[table["Name"] for table in table_metadata['TableMetadataList']]

Upvotes: 0

Saleem
Saleem

Reputation: 139

db = session.resource('dynamodb', region_name="us-east-2")
tables = list(db.tables.all())
print(tables)

resource https://boto3.amazonaws.com/v1/documentation/api/latest/guide/dynamodb.html

Upvotes: 8

Related Questions