Python API for Google play store scraping

Question

I want to scrape reviews of apps from google play store. I was thinking to use Python API to scrape the reviews. I know there are API like selenium and beautiful soup, where you provide the weblink to scrape the data. I am more interested in using API like StackAPI (made for scraping stack overflow) where you don't need to provide any weblink. When I searched on the internet, I found there is no comprehensive API available for scraping the reviews from the play store. For instance, play_scraper API in Python gives detailed information about API but does not scrape the reviews associated with the app. Does any one guide me which one should I use for google play store scraping?

Answer 1

You could check out SerpApi. We handle proxies, solve captchas, and parse all rich structured data for you.

With this API you can retrieve up to 199 reviews per page, and then use next_page_token to get the rest.

Example python code for retrieving YouTube reviews (available in other libraries also):

from serpapi import GoogleSearch

params = {
  "api_key": "SECRET_API_KEY",
  "engine": "google_play_product",
  "store": "apps",
  "gl": "us",
  "product_id": "com.google.android.youtube",
  "all_reviews": "true"
}

search = GoogleSearch(params)
results = search.get_dict()

Example JSON output:

  "reviews": [
    {
      "title": "Qwerty Jones",
      "avatar": "https://play-lh.googleusercontent.com/a/AATXAJwSQC_a0OIQqkAkzuw8nAxt4vrVBgvkmwoSiEZ3=mo",
      "rating": 3,
      "snippet": "Overall a great app. Lots of videos to see, look at shorts, learn hacks, etc. However, every time I want to go on the app, it says I need to update the game and that it's \"not the current version\". I've done it about 3 times now, and it's starting to get ridiculous. It could just be my device, but try to update me if you have any clue how to fix this. Thanks :)",
      "likes": 586,
      "date": "November 26, 2021"
    },
    {
      "title": "matthew baxter",
      "avatar": "https://play-lh.googleusercontent.com/a/AATXAJy9NbOSrGscHXhJu8wmwBvR4iD-BiApImKfD2RN=mo",
      "rating": 1,
      "snippet": "App is broken, every video shows no dislikes even after I hit the button. I've tested this with multiple videos and now my recommended is all messed up because of it. The ads are longer than the videos that I'm trying to watch and there is always a second ad after the first one. This app seriously sucks. I would not recommend this app to anyone.",
      "likes": 352,
      "date": "November 28, 2021"
    },
    {
      "title": "Operation Blackout",
      "avatar": "https://play-lh.googleusercontent.com/a-/AOh14GjMRxVZafTAmwYA5xtamcfQbp0-rUWFRx_JzQML",
      "rating": 2,
      "snippet": "YouTube used to be great, but now theyve made questionable and arguably stupid decisions that have effectively ruined the platform. For instance, you now have the grand chance of getting 30 seconds of unskipable ad time before the start of a video (or even in the middle of it)! This happens so frequently that its actually a feasible option to buy an ad blocker just for YouTube itself... In correlation with this, YouTube is so sensitive twords the public they decided to remove dislikes. Why????",
      "likes": 370,
      "date": "November 24, 2021"
    },
    ...
  ],
  "serpapi_pagination": {
    "next": "https://serpapi.com/search.json?all_reviews=true&engine=google_play_product&gl=us&hl=en&next_page_token=CpEBCo4BKmgKR_8AwEEujFG0VLQA___-9zuazVT_jmsbmJ6WnsXPz8_Pz8_PxsfJx5vJns3Gxc7FiZLFxsrLysnHx8rIx87Mx8nNzsnLyv_-ECghlTCOpBLShpdQAFoLCZiJujt_EovhEANgmOjCATIiCiAKHmFuZHJvaWRfaGVscGZ1bG5lc3NfcXNjb3JlX3YyYQ&product_id=com.google.android.youtube&store=apps",
    "next_page_token": "CpEBCo4BKmgKR_8AwEEujFG0VLQA___-9zuazVT_jmsbmJ6WnsXPz8_Pz8_PxsfJx5vJns3Gxc7FiZLFxsrLysnHx8rIx87Mx8nNzsnLyv_-ECghlTCOpBLShpdQAFoLCZiJujt_EovhEANgmOjCATIiCiAKHmFuZHJvaWRfaGVscGZ1bG5lc3NfcXNjb3JlX3YyYQ"
  }

Check out the documentation for more details.

Test the search live on the playground.

Disclaimer: I work at SerpApi.

Answer 2

You might consider using the SearchApi, specifically the Google Play Product API. This API provides detailed product information, and the first page returns 40 reviews. Subsequent pages can yield over 4000 reviews per single request.

Below is a simple Python code snippet illustrating how to retrieve 2000 reviews from the TikTok app:

import requests

payload = {
    'api_key': 'key',               # key is received upon registration.
    'engine': 'google_play_product',
    'store': 'apps',
    'product_id': 'com.zhiliaoapp.musically',
    'num': '2000',
    'next_page_token': 'CnIKcApuMCwxMDAxMDAwLjc0NjM2MzUyMDYsODQ3MzU0MzQ3NTg3LCJodHRwOi8vbWFya2V0LmFuZHJvaWQuY29tL2RldGFpbHM_aWQ9djI6Y29tLnpoaWxpYW9hcHAubXVzaWNhbGx5OjEiLDEsZmFsc2U'
}
response = requests.get('https://www.searchapi.io/api/v1/search', params=payload)

print(response.text)

Here's an JSON example of 3 reviews:

{
  "reviews": [
    {
      "id": "ce3c3d5d-0138-44c7-a980-617270b022ac",
      "product_version": "29.3.4",
      "username": "Abby",
      "avatar": "https://play-lh.googleusercontent.com/a/AAcHTtfTZo0ehDfPzsn68ifvszx0WFwQ1gRsm-5y3lFwPrCJ=mo",
      "rating": 4,
      "link": "https://play.google.com/store/apps/details?id=&reviewId=ce3c3d5d-0138-44c7-a980-617270b022ac",
      "text": "Honestly, I think tiktok is a very good app and it is also really fun to do dances find songs you really like. You could even connect with some people. I just think it's a very good app! There's so much more to it. You can cooking recipes, life hacks. Some stuff can be false. That's the only bad thing over all. I really really love the app!",
      "likes": 610,
      "date": "May 10, 2023"
    },
    {
      "id": "f3cb1893-bb89-43c5-a9c2-d2e814be9add",
      "product_version": "29.5.4",
      "username": "Dr3amwalker",
      "avatar": "https://play-lh.googleusercontent.com/a-/AD_cMMQJUo-A3fxIceFTdQGr1ejTGvAxrpxae9HSo3Sl02cMECs",
      "rating": 5,
      "link": "https://play.google.com/store/apps/details?id=&reviewId=f3cb1893-bb89-43c5-a9c2-d2e814be9add",
      "text": "Edited review:: I've used TikTok quite a bit, and while there ARE parts of tiktok that I dont like, or parts that may be problematic, the app itself is great for what it is. It's taught me things I never thought I'd know. It's helped with making me feel less isolated. - Disclaimer: Moderation is key. Learn how the algorithm works [its super easy] and you can tailor your experience near perfectly.",
      "likes": 1272,
      "date": "May 17, 2023"
    },
    {
      "id": "54efe07b-2cb5-4716-98e4-7735d036ce28",
      "product_version": "29.5.5",
      "username": "owns 7 hats",
      "avatar": "https://play-lh.googleusercontent.com/a-/AD_cMMTgpgkOR7dJZ2S5y6JsDIApQuPQy91QqaMfKtqFFhVqMyCO",
      "rating": 1,
      "link": "https://play.google.com/store/apps/details?id=&reviewId=54efe07b-2cb5-4716-98e4-7735d036ce28",
      "text": "Recent update with text and picture slides is obnoxious. Can't pause the sound without a pop up that glitches out way too badly. Not to mention the series feature, when there's a part 2,3,4 or whatever the Playlist shouldn't be playing the most recent episode first. You don't watch a movie backwards and the autoplay prevents you from seeing it in the right order almost entirely with how annoying it is. I'm about to delete the app because these features are not worth my time.",
      "likes": 671,
      "date": "May 17, 2023"
    }
  ]
}

You can find the documentation for Google Play Product API here: https://www.searchapi.io/docs/google-play-product

Disclaimer: I work for SearchApi.

Answer 3

In case you haven't found the one, here is the link for python package to scrape reviews from google play store.

https://pypi.org/project/google-play-scraper/

I gave it a try and its working very fine.