CODEWITHSUNDEEP
pythoncsvweb-scrapingscrapyexport-to-csv
jack
jack

Reputation: 31

Add Scrapy data to csv without the header row

We have a local website that tracks the number of people using a certain license. I have create a scraper with that should run every hour. The only issue I have it's creating data that looks like this.

active_users,date,time
35,22/03/2022,11:38:30.397745
active_users,date,time
36,22/03/2022,11:44:04.753589

the issue I find is that every time scrapy crawl users is ran it adds that header. I know scrapy has CsvItemExporter() that can remove the header but I'm not too sure how to use it.

I just need the output csv to look like

active_users,date,time
35,22/03/2022,11:38:30.397745
36,22/03/2022,11:44:04.753589

Upvotes: 3

Views: 556

Answers (1)

msenior_
msenior_

Reputation: 2110

If you are using scrapy version 2.4 and above you can directly change this setting when defining the FEED such as below.

custom_settings = {"FEEDS": { "items.csv": {"format": "csv", "item_export_kwargs": {"include_headers_line": False}}}}

This will append to the contents of the file instead of adding a new header each time.

Upvotes: 1

Related Questions