Reputation: 81
from bs4 import BeautifulSoup
import requests
import pandas as pd
import ast
s = requests.Session()
page=1
traits = []
#Get URL and extract content
class Scraper():
while page != 10:
content = s.get('https://bullsontheblock.com/api/tokens/{}'.format(page))
soup = BeautifulSoup(content.text, 'html.parser')
page = page + 1
traits = ast.literal_eval(soup.text)['attributes']
df = pd.DataFrame(traits).set_index('value').to_numpy()
trait_count = len(df)
print(trait_count)
Whenever I use the above code I get integers separated by line like this:
9
8
8
8
6
9
8
8
7
How do I create a column that tells me the the count of how many times a number appears so it looks something like this instead:
9 - 2
8 - 5
7 - 1
6 - 1
Basically the code above pulls the count of how many traits are in a list, but I have multiple lists so I would like to pull the count of how many times a list with a certain number of traits appear so it can look like the above. How do I go about this?
Upvotes: 0
Views: 88
Reputation: 81
Just in case anyone else ever has the same problem I'm having: @garlic_rat's answer was spot on but I was still having trouble with trying to get it into a dataframe but I solved my problem by using the following code:
# Import libraries
from bs4 import BeautifulSoup
import requests
import pandas as pd
import time
import ast
start_time = time.time()
s = requests.Session()
#Get URL and extract content
page=1
traits = []
traits_e = []
while page != 10:
content = s.get('https://bullsontheblock.com/api/tokens/{}'.format(page))
soup = BeautifulSoup(content.text, 'html.parser')
page = page + 1
traits = ast.literal_eval(soup.text)['attributes']
df = pd.DataFrame(traits).set_index('value').to_numpy()
trait_count = len(df)
traits_e.append(trait_count)
a = pd.DataFrame({'Trait Count': traits_e})
value_counts1 = a['Trait Count'].value_counts()
l1 = [f"{key} - {value_counts1[key]}" for key in value_counts1.keys()]
traits = l1
dfx = pd.DataFrame(
traits, columns=['Trait Count'])
print(dfx)
print("--- %s seconds ---" % (time.time() - start_time))
By doing this I was able to receive the same data as @garlic_rat's answer but this also allowed me to put it into a dataframe column.
Upvotes: 0
Reputation: 109
As mentioned in comments and @PyxlDavon's answer, you likely want to use a dictionary:
from bs4 import BeautifulSoup
import requests
import pandas as pd
import ast
s = requests.Session()
page=1
traits = []
traits_d = {}
#Get URL and extract content
class Scraper():
while page != 10:
content = s.get('https://bullsontheblock.com/api/tokens/{}'.format(page))
soup = BeautifulSoup(content.text, 'html.parser')
page = page + 1
traits = ast.literal_eval(soup.text)['attributes']
df = pd.DataFrame(traits).set_index('value').to_numpy()
trait_count = len(df)
if trait_count in traits_d:
traits_d[trait_count] += 1
else:
traits_d[trait_count] = 1
traits_d contains the trait and number of times seen:
traits_d
{9: 2, 8: 5, 6: 1, 7: 1}
To print the keys out as you show, loop through the sorted keys:
for key in sorted(traits_d.keys(), reverse=True):
print(key, '-', traits_d[key])
9 - 2
8 - 5
7 - 1
6 - 1
Upvotes: 1
Reputation: 1814
Add each numbers to a string, then count the frequency of each number
nums = "988869887"
fre = {}
for num in nums:
if num in fre:
fre[int(num)] += 1
else:
fre[int(num)] = 1
print(fre)
Output
{9: 2, 8: 5, 6: 1, 7: 1}
Upvotes: 1