CODEWITHSUNDEEP
javascriptyoutube-api
Fsmv
Fsmv

Reputation: 1176

How to get my entire YouTube watch history?

I'm trying to get a full list of watched videos for a given user in my YouTube API application. I want to add up total duration of all videos.

When I get the list of videos from history playlist, the API caps it at 50 items. There's pagination but total amount of items is 50 (not just per page); I can't access more data with the API it appears.

Is there any way I can get this playlist without the data cap? I'm hoping for another method (of using the API) or a way to do it without the API. I know YouTube stores this data because I can view my entire history (far more that 50 videos).

I'm using this code:

var requestOptions = {
    playlistId: playlistId,
    part: 'snippet',
    maxResults: 50
};
gapi.client.youtube.playlistItems.list(requestOptions);

where playlistId is the id of the history playlist I got from a gapi.client.youtube.channels.list request.

Edit (2017): I want to clarify that it was always my intention to download my own history, just out of interest to see how much time I have spent watching videos. I still have not been able to do this.

Upvotes: 34

Views: 34196

Answers (6)

Sergio Pereira
Sergio Pereira

Reputation: 1

I was looking for some way to get the list of YouTube history. I just found out that Google has a tool for this. In Google Takeout you have a option taht you can get the entire list of watched videos. My list went back util 2011.

To get explanation short there are two videos explaining how to do this:

https://www.youtube.com/watch?v=zlzzO1e6dws https://www.youtube.com/watch?v=dto8jGMxHxY

Upvotes: 0

Stefan M
Stefan M

Reputation: 48

While this isn't currently possible using just the YouTube API, there is an (albeit slightly involved) method to calculate your watch time):

  1. download a list of your watch history as a JSON file using Google Takeout.
  2. Unfortunately the JSON file doesn't include the video durations, so the next step is to extract the video IDs (the part after "watch?v=" in the "titleURL" object
  3. Now take your list of video IDs, and send a request to the youtube API that looks something like this:
 function execute() {
    return gapi.client.youtube.videos.list({
      "part": [
        "contentDetails"
      ],
      "id": [
        "VIDEO IDs"
      ],
      "fields": "items(contentDetails(duration))"
    })

(Code created using YouTube API Explorer)

Note: You may need to break the list of video IDs into smaller lists (I had to) or the API may reject the request. As [pointed out by stvar in the comments] the ID list maximum length is 50, so this is the maximum length your lists can be. (full disclosure: I was using Python to send the requests)

  1. Finally, just extract the duration values and add them up (though this might not be quite as easy as it sounds)

The best part of this is I don't believe this actually violates any ToS.

Upvotes: 3

Zv_oDD
Zv_oDD

Reputation: 1878

I wrote a scraper(in Python 2.7(updated for 3.5) and Scrapy) for this task a while ago. Sans official API, it uses a logged in session cookie and html parsing. Dumps to SQLite by default. https://github.com/zvodd/Youtube-Watch-History-Scraper

How it's done: essentially it opens the url 

https://www.youtube.com/feed/history'

with a valid(logged in) session cookie taken from Chrome. Scrapes all video entries for name, vid(url), channel/user, description, length. Then it finds the button at the bottom of the page with the attribute data-uix-load-more-href which contains the link to the next page, something like:

"/browse_ajax?action_continuation=1&continuation=98h32hfoasau0fu928hf2hf908h98hr%253D%253D&target_id=item-section-552363&direct_render=1"

... re-scrapes the video entries from there and dumps them all into an sqlite database; which you can search entries by any of the fields (name, length, user, description, etc).

So until they change their feed/history page, it's doable and done. I might even update it.

Upvotes: 17

noderman
noderman

Reputation: 1944

Brainstorming, never tried: Have you tried not using the API and instead parsing the https://www.youtube.com/feed/history URL?

Theoretically, the user browsing could be emulated, including the pagination. I am not aware of how hard though (probably very), since you need to deal with authentication and YouTube probably tries to verify that a human is browsing.

Upvotes: 0

user1400995
user1400995

Reputation:

It seems like this is a known bug originally reported in 2013. The exact same behavior is explained on a Google Code thread: https://code.google.com/p/gdata-issues/issues/detail?id=4642

Upvotes: 3

vol7ron
vol7ron

Reputation: 42109

The API currently only retrieves the last two weeks of Watch History. For more information refer to the Bug Issue reported: https://code.google.com/p/gdata-issues/issues/detail?id=4642

Note: There is a similar question on SO asked here: YouTube API v3 returns truncated watch history

Upvotes: 17

Related Questions