Android RemoteMediator calls API again and again

Question

I have created a remote mediator which gets movies from API call and adds it to database which is then used as a source to load the data on screen. It is pretty cliche implementation done same as Google developers video of paging3 from YouTube, different articles etc.

@ExperimentalPagingApi
class RemoteMediator(
    val moviesRetrofitClient: MoviesRetrofitClient,
    private val movieDatabase: MovieDatabase
) : RemoteMediator<Int, MovieData>() {

    override suspend fun load(
        loadType: LoadType,
        state: PagingState<Int, MovieData>
    ): MediatorResult {

        try {
            val pageKeyData = getKeyPageData(loadType , state)
            val page = when(pageKeyData){
                is MediatorResult.Success -> {

                    Utils.debug("mediator result success = $pageKeyData")
                    return pageKeyData
                 }
                else -> {
                    Utils.debug("mediator result failed = $pageKeyData")
                    pageKeyData as Int
                 }
            }
            Utils.debug("page we got = $page")
            val movieResponse = moviesRetrofitClient.getNowPlayingMovies(page)
            val movies = movieResponse.movies

            var totalPages = movieResponse.totalPages
            val endOfPaginationReached = (page == totalPages)

            movieDatabase.withTransaction {
                if (loadType == LoadType.REFRESH){
                    movieDatabase.movieDao().deleteMovie()
                    movieDatabase.moviePagingKeyDao().deleteAllPagingKeys()
                }

                val prevPage = if (page == 1) null else (page-1)
                val nextPage = if (endOfPaginationReached) null else (page+1)

                val keys = movies.map {
                    MoviePagingKeys(it.id , prevPage = prevPage , nextPage = nextPage)
                }

                movieDatabase.moviePagingKeyDao().addAllPagingKeys(keys)
                movieDatabase.movieDao().addMovies(movies)
            }
            return MediatorResult.Success(endOfPaginationReached = endOfPaginationReached)
        }catch (e : Exception){
            Utils.error("exception Error : ${e.message.toString()}")
            return MediatorResult.Error(e)
        }catch (ioException : IOException){
            Utils.error("IO Error : ${ioException.message.toString()}")
            return MediatorResult.Error(ioException)
        }
    }

    private suspend fun getKeyPageData(loadType: LoadType, state: PagingState<Int, 
MovieData>): Any {
        return when(loadType){

            LoadType.REFRESH -> {
                Utils.debug("Refresh called")
                val remoteKeys = getRemoteKeyClosestToCurrentPosition(state)
                remoteKeys?.nextPage?.minus(1) ?: 1
            }

            LoadType.APPEND -> {
                Utils.debug("Append called")
                val remoteKeys = getLastRemoteKey(state)
                val nextKey = remoteKeys?.nextPage
                return nextKey ?: MediatorResult.Success(endOfPaginationReached = false)
            }

            LoadType.PREPEND -> {
                Utils.debug("Prepend Called")
                val remoteKeys = getFirstRemoteKey(state)
                val prevKey = remoteKeys?.prevPage ?: return MediatorResult.Success(
                    endOfPaginationReached = false
                )
                prevKey
            }
        }
    }

    private suspend fun getFirstRemoteKey(state: PagingState<Int, MovieData>): 
MoviePagingKeys?{
        return state.pages
            .firstOrNull { it.data.isNotEmpty() }
            ?.data?.firstOrNull()
            ?.let { movie -> movieDatabase.moviePagingKeyDao().getMoviePagingKey(movie.id) }
    }

    private suspend fun getLastRemoteKey(state: PagingState<Int, MovieData>): MoviePagingKeys? 
{
        return state.pages
            .lastOrNull { it.data.isNotEmpty() }
            ?.data?.lastOrNull()
            ?.let { movie -> movieDatabase.moviePagingKeyDao().getMoviePagingKey(movie.id) }
    }

    private suspend fun getRemoteKeyClosestToCurrentPosition(state: PagingState<Int, 
MovieData>): MoviePagingKeys? {
        return state.anchorPosition?.let {position ->
            state.closestItemToPosition(position)?.id?.let { movieId ->
                movieDatabase.moviePagingKeyDao().getMoviePagingKey(movieId)
            }
        }
    }

    override suspend fun initialize(): InitializeAction {
        return InitializeAction.LAUNCH_INITIAL_REFRESH
    }
}

This is my API response:

{
"dates": {
    "maximum": "2022-09-11",
    "minimum": "2022-07-25"
},
"page": 1,
"results": [
    {
        "adult": false,
        "backdrop_path": "/2RSirqZG949GuRwN38MYCIGG4Od.jpg",
        "genre_ids": [
            53
        ],
        "id": 985939,
        "original_language": "en",
        "original_title": "Fall",
        "overview": "For best friends Becky and Hunter, life is all about conquering fears and pushing limits. But after they climb 2,000 feet to the top of a remote, abandoned radio tower, they find themselves stranded with no way down. Now Becky and Hunter’s expert climbing skills will be put to the ultimate test as they desperately fight to survive the elements, a lack of supplies, and vertigo-inducing heights.",
        "popularity": 9791.409,
        "poster_path": "/9f5sIJEgvUpFv0ozfA6TurG4j22.jpg",
        "release_date": "2022-08-11",
        "title": "Fall",
        "video": false,
        "vote_average": 7.5,
        "vote_count": 455
    },...]
"total_pages": 83,
"total_results": 1645

}

The results are the movies which needs to be displayed. Since an array of movies are already fetched during the API call, I am checking if the remote mediator is success or not by comparing the page number with the total pages.

val endOfPaginationReached = (page == totalPages)

The problem is, the load method is called continuously again and again even after the first page is fetched. Hence making it call the API continuously. I understand the data which I gave might not be enough for a solution, but I do not know how to express the problem.

I want to know how is the load method called, like on what condition. Please help.

This is all the classes which is being used, I am not adding the unrelated classes like Daos and ViewModels. I am sure those does not have any problems.

Repository class with the config :-

class MovieRepository @Inject constructor(
val moviesRetrofitClient: MoviesRetrofitClient,
val movieDatabase: MovieDatabase) {

    fun getMovies() = Pager(
        config = PagingConfig(pageSize = Constants.PAGE_SIZE, maxSize = Constants.MAX_PAGE_COUNT),
        remoteMediator = RemoteMediator(moviesRetrofitClient , movieDatabase)){
        movieDatabase.movieDao().getMovies()
    }.liveData
}

Retrofit client:

@InstallIn(SingletonComponent::class)
@Module
class MoviesRetrofitClient @Inject constructor() {

    @Singleton
    @Provides
    fun getInterceptor() : Interceptor{
       val requestInterceptor = Interceptor{

           val url = it.request()
               .url
               .newBuilder()
               .addQueryParameter("api_key" , API_KEY)
               .build()

           val request = it.request()
               .newBuilder()
               .url(url)
               .build()

           return@Interceptor it.proceed(request)
       }
        return requestInterceptor
    }

    @Singleton
    @Provides
    fun getGsonConverterFactory() : GsonConverterFactory{
        return GsonConverterFactory.create()
    }

    @Singleton
    @Provides
    fun getOkHttpClient() : OkHttpClient{

        var httLog : HttpLoggingInterceptor = HttpLoggingInterceptor()
        httLog.setLevel(HttpLoggingInterceptor.Level.BODY)

        val okHttpClient = OkHttpClient.Builder()
           .addInterceptor(getInterceptor()).addInterceptor(httLog)
           .connectTimeout(60 , TimeUnit.SECONDS)
           .build()

        return okHttpClient
    }

    @Singleton
    @Provides
    fun getMoviesApiServiceRx() : MoviesApiService{
        var retrofit : Retrofit = Retrofit.Builder()
            .baseUrl(BASE_URL)
            .client(getOkHttpClient())
            .addConverterFactory(getGsonConverterFactory())
            .addCallAdapterFactory(RxJava2CallAdapterFactory.create())
            .build()

        return  retrofit.create(MoviesApiService::class.java)
    }

    @Singleton
    @Provides
    suspend fun getNowPlayingMovies(pageNo : Int): NowPlayingMoviesData {
        return getMoviesApiServiceRx().getNowPlayingMovies(pageNo)
    }

}

Paging Adapter:

class MoviesAdapter() : PagingDataAdapter<MovieData,MoviesAdapter.MovieViewHolder>(COMPARATOR) {

    override fun onCreateViewHolder(parent: ViewGroup, viewType: Int): MovieViewHolder {
        val binding = MovieViewBinding.inflate(LayoutInflater.from(parent.context), parent , false)
        return MovieViewHolder(context = parent.context , binding)
    }
    override fun onBindViewHolder(holder: MovieViewHolder, position: Int) {
        val movie = getItem(position)
        if (movie != null){
            holder.bindData(movie)
        }
    }

    inner class MovieViewHolder(private val context: Context, private val movieViewDataBinding : MovieViewBinding)
        : RecyclerView.ViewHolder(movieViewDataBinding.root){

        init {
            movieViewDataBinding.root.setOnClickListener{
                // TODO: "implement movie details screen"
                Utils.toast(context , "movie Clicked")
            }
        }

        fun bindData(movieData: MovieData){
            movieViewDataBinding.movie = calculateRating(movieData)
        }

        //change the ratings to the multiple of 5 , so that it can be fit in the rating view.
        private fun calculateRating(movieData: MovieData) : MovieData{
           movieData.voteAverage = (movieData.voteAverage?.times(5))?.div(10)
            return movieData
        }

    }

    companion object {
        private val COMPARATOR = object : DiffUtil.ItemCallback<MovieData>(){

            override fun areItemsTheSame(oldItem: MovieData, newItem: MovieData): Boolean {
                return oldItem.id == newItem.id
            }

            override fun areContentsTheSame(oldItem: MovieData, newItem: MovieData): Boolean {
                return oldItem == newItem
            }

        }
    }
}

Loading adapter for progress circle when scrolling:

class LoaderAdapter : LoadStateAdapter<LoaderAdapter.LoaderHolder>() {

    override fun onCreateViewHolder(parent: ViewGroup, loadState: LoadState): LoaderHolder {
        val view = LayoutInflater.from(parent.context).inflate(R.layout.loader , parent , false)
        return LoaderHolder(view)
    }

    override fun onBindViewHolder(holder: LoaderHolder, loadState: LoadState) {
        holder.bind(loadState)
    }

    inner class LoaderHolder(itemView: View) : RecyclerView.ViewHolder(itemView){

        val progress = itemView.findViewById<ProgressBar>(R.id.movieProgressBar)

        fun bind(loadState: LoadState){
           progress.isVisible = loadState is LoadState.Loading
        }

    }

}

Edit :

This is my main Activity:

class MainActivity : AppCompatActivity(), SwipeRefreshLayout.OnRefreshListener,
    View.OnClickListener{

    lateinit var movieViewModel : MoviesViewModel
    lateinit var moviesAdapter : MoviesAdapter
    lateinit var movieRecyclerView: RecyclerView
    lateinit var connectivityLiveStatus: ConnectionLiveStatus

    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        setContentView(R.layout.activity_main)
        init()
    }

    private fun init(){

        connectivityLiveStatus = ConnectionLiveStatus(this)
        observeConnectivity()

        swipeToRefresh.setOnRefreshListener(this)

        movieRecyclerView = findViewById(R.id.moviesRecyclerView)
        moviesAdapter = MoviesAdapter()
        movieViewModel = ViewModelProvider(this)[MoviesViewModel::class.java]

        movieRecyclerView.layoutManager = LinearLayoutManager(this)
        movieRecyclerView.setHasFixedSize(true)
        movieRecyclerView.adapter = moviesAdapter.withLoadStateHeaderAndFooter(
            header = LoaderAdapter(),
            footer = LoaderAdapter()
        )

        nowPlayingTV.setOnClickListener(this)
        observeViewModel()
    }

    //observe connectivity change
    private fun observeConnectivity(){
        connectivityLiveStatus.observe(this , Observer {status ->
            handleConnectivityChange(status)
        })
    }

    //Observe the movie data change
    private fun observeViewModel(){
        movieViewModel.movieList.observe(this) {
            moviesAdapter.submitData(lifecycle, it)
            if (swipeToRefresh.isRefreshing) swipeToRefresh.isRefreshing = false
        }
    }

    private fun handleConnectivityChange(status : Boolean){
        networkConnectivityStatusTv.visibility = if (status) View.INVISIBLE else View.VISIBLE
        nowPlayingTV.visibility = if (status) View.VISIBLE else View.GONE
        moviesAdapter.retry()

        //change the status bar color according to network status.
        val window = window
        window.statusBarColor = if (status) applicationContext.resources.getColor(R.color.app_background_color) else applicationContext.resources.getColor(
            R.color.network_connectivity_alert_color
        )
    }

    //refresh when swipe
    override fun onRefresh() {
        moviesAdapter.refresh()
    }

    override fun onClick(p0: View?) {
       when(p0?.id) {
           R.id.nowPlayingTV -> {
               movieRecyclerView.smoothScrollToPosition(0)
           }
       }
    }

}

And this line of code, which I used to display loading progress while scrolling (using the LoadAdapter). When I remove these lines, The entire paging stops working, No API gets called. What exactly does this line of code do; is there any other way for this? Could this be calling the load from remote mediator again and again?

Answer 1

I got the same problem, in my case page in RemoteMediator is not increasing. the list size is 20 (I am calling 20 records every time) after every API call. that's why infinite API call is happening.

there is 1 condition to stop the API calls. your remote list should be empty after the API call or less than 20 records, then only it will stop

in your case, you are getting 20 records in 1 API call RemoteMediator think there may be more records RemoteMediator again calling the API but the page is not getting increased you are getting the same records list size is still 20 because you are using the DiffUtil list it does not allow duplicates. RemoteMediator again calls API.

The problem is your List is not in ascending order.

private suspend fun getLastRemoteKey(state: PagingState<Int, MovieData>)

inside this function, you call the last records

state.pages.lastOrNull

but it's not returning the last record.

you can do like this

  @Query("SELECT * FROM MovieDatabase Order by createdAt ASC")
  fun getAllCustomers(): PagingSource<Int, MovieDataEntity>

add

val createdAt: Long = System.currentTimeMillis()

inside your entity class

after adding this

state.pages.lastOrNull

will return the last record every time

Answer 2

You are refreshing the list everytime it's visited:

override suspend fun initialize(): InitializeAction {
        return InitializeAction.LAUNCH_INITIAL_REFRESH
    }

use this one:

 return InitializeAction.SKIP_INITIAL_REFRESH 

you can read further here: https://developer.android.com/reference/kotlin/androidx/paging/RemoteMediator#initialize()