CODEWITHSUNDEEP
pythonpython-2.7web-scrapingbeautifulsoup
dbakathaillist
dbakathaillist

Reputation: 11

Trouble Writing BeautifulSoup Output to File

Hello stackoverflow community!

I'm still learning the ins and outs of coding in Python, so please excuse my code that I am about to post.

I am currently trying to write a script that will scrape a list of the latest media torrents from http://kat.ph using BS4 and save it to a file. However, I am having trouble getting the output from BS4 to print to this file. When you open the text file, it's blank, but when you run the script in the terminal, it works just fine. Ultimately, I want to have python send the bs4 output in an email (which is where I originally ran into this problem and decided to see if I can write to .txt file).

I don't have the script that I made on my home computer at the moment, but I recreated another one where I did pretty much the same thing.

Any help/suggestions are much appreciated!

from bs4 import BeautifulSoup
import requests
import time

#The goal of this script was to scrape the names of the latest media torrents and  write them to a text file.
#When I run the script on my computer, I can see the prompt give me the list of torrents just fine.
#When I try to write to a file or send an email, it doesn't print anything.

req = requests.get('http://kat.ph')

site = req.text

soup = BeautifulSoup(site) #Tried making this 'soup = str(BeautifulSoup(site)) to no avail.

def writingFunction():
    #I imported time module because I had my script display the time and date here.
    counter = 1
    for i in soup.find_all('div', {'class': 'markeredBlock torType filmType'}):

        print str(counter) + '.' + ' ' + i.text
        counter = counter + 1

textFile = open('C:/python27/file.txt', 'a')
textFile.write(writingFunction()) #I've tried making this a str and I've also tried  assigning the function to a variable
textFile.close()

Upvotes: 1

Views: 1800

Answers (1)

Padraic Cunningham
Padraic Cunningham

Reputation: 180411

just write in the function, currently you are just printing the output:

def writing_function():
    with open('C:/python27/file.txt', 'a') as f:
        for ind, i in enumerate(soup.find_all('div', {'class': 'markedBlock torType filmType'}),1):
            f.write("{}. {}".format(ind, i.text.replace("\n","")))

print str(counter) + '.' + ' ' + i.content.replace('\n', '') is printing each line, you are not returning anything so textFile.write(writingFunction()) is futile, just do your writing in the function.

If you don't want to hardcode the file name just pass it as a parameter:

def writing_function(my_file):
    with open(my_file, 'a') as f:

Using enumerate with a start index of 1 will do what your counter variable is doing.

There is no .content in bs4, it is contents which is a list, if you want the text use i.text.

Upvotes: 1

Related Questions