CODEWITHSUNDEEP
pythonpandasgenerator
Marcelo Soares
Marcelo Soares

Reputation: 187

Get columns from generator object

I'm using Scrapetube to get videos from a channel, and it brings a generator object. From the very simple documentation, I know it includes the parameter "videoId", but how can I know what other parameters I can get from there? Can I transform a generator object into, say, a dataframe?

Upvotes: 0

Views: 267

Answers (1)

ozacha
ozacha

Reputation: 1352

Generators allow you to efficiently iterate over (potentially infinite) sequences.

In your case, you probably want to first convert the generator into a list to expose all items in the sequence.

Then you can inspect what the returned elements look like and extract the information you need.

You can then create a dataframe for instance from a list of dictionaries:

result_gen = scrapetube.xxx()
result_list = list(result_gen)

# Inspect first element
print(result_list[0])
# Inspect attributes of the first element
print(dir(result_list[0]))

# Convert/extract information of interest into a dictionary
def to_dict(element):
    ...

result_df = pd.DataFrame([to_dict(element) for element in result_list])

Upvotes: 1

Related Questions