Read large (+- 50Mb) Json file using Kotlin

Question

I'm starting to work on a weather app using "openweathermap.org" API, and they provide you with a list of available cities in Json format.

Before i continue with the project, i would like to able to work with the data from this Json file.

The problem is that i get Null whenever i try to read and parse that file.

Here is the code:

Main Activity:

class MainActivity : AppCompatActivity() {
    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        setContentView(R.layout.activity_main)

        val jsonFileString = getJsonDataFromAsset(applicationContext, "citylist.json")
        Log.i("gabs Data", jsonFileString ?: "Empty Data")
        val gson = Gson()
        val listOfCities = object : TypeToken<List<CityList>>() {}.type
        var cities: List<CityList> = gson.fromJson(jsonFileString, listOfCities)
        cities.forEachIndexed { idx, city -> Log.i("data", "> Item $idx:\n$city") }
    }
}

Utils.kt:

fun getJsonDataFromAsset(context: Context, fileName: String): String? {
    val jsonString: String
    try {
        jsonString = context.assets.open(fileName).bufferedReader().use { it.readText() }
    } catch (ioException: IOException) {
        ioException.printStackTrace()
        return null
    }
    return jsonString
}

And the data class (Array of cities data):

class CityList : ArrayList<CityList.CityListItem>(){
    data class CityListItem(
        @SerializedName("coord")
        val coord: Coord,
        @SerializedName("country")
        val country: String,
        @SerializedName("id")
        val id: Double,
        @SerializedName("name")
        val name: String,
        @SerializedName("state")
        val state: String
    ) {
        data class Coord(
            @SerializedName("lat")
            val lat: Double,
            @SerializedName("lon")
            val lon: Double
        )
    }
}

And the error:

java.lang.RuntimeException: Unable to start activity ComponentInfo{com.example.weatherdisplay/com.example.weatherdisplay.ui.activities.MainActivity}: java.lang.NullPointerException: gson.fromJson(jsonFileString, listOfCities) must not be null.

Caused by: java.lang.NullPointerException: gson.fromJson(jsonFileString, listOfCities) must not be null at com.example.weatherdisplay.ui.activities.MainActivity.onCreate(MainActivity.kt:21)

Answer 1

There were some problems in your code:

1. You were not closing the BufferedReader
2. You should not load the file on the Main thread since it will block the UI

I created some sample data corresponding to your data structure:

[
  {
    "id": 1,
    "country": "Germany",
    "state": "Saxony",
    "name": "Dresden",
    "coord": {
      "lat": 0.0,
      "lon": 0.0
    }
  },
  {
    "id": 2,
    "country": "Germany",
    "state": "Berlin",
    "name": "Berlin",
    "coord": {
      "lat": 0.0,
      "lon": 0.0
    }
  },
  {
    "id": 3,
    "country": "Germany",
    "state": "Baden-Wuerttemberg",
    "name": "Stuttgart",
    "coord": {
      "lat": 0.0,
      "lon": 0.0
    }
  },
  {
    "id": 4,
    "country": "Germany",
    "state": "Hessen",
    "name": "Frankfurth",
    "coord": {
      "lat": 0.0,
      "lon": 0.0
    }
  },
  {
    "id": 5,
    "country": "Germany",
    "state": "Nordrhine-Westphalia",
    "name": "Cologne",
    "coord": {
      "lat": 0.0,
      "lon": 0.0
    }
  }
]

Your activity:

class MainActivity : AppCompatActivity() {

    companion object {
        const val TAG = "MyApplication"
    }

    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        setContentView(R.layout.activity_main)

        lifecycleScope.launchWhenStarted {
            launch(Dispatchers.IO) {
                var reader: BufferedReader? = null
                try {
                    // Create a reader and read the file contents
                    reader = assets.open("data.json").bufferedReader()
                    val rawData = reader.use { it.readText() }

                    // Create a Type token that Gson knows how to parse the raw data
                    val cityListType = object : TypeToken<List<City>>() {}.type

                    // Parse the raw data using Gson
                    val data: List<City> = Gson().fromJson(rawData, cityListType)

                    // TODO: Do something with the data
                } catch (e: IOException) {
                    // Handle IOException: Gets thrown when the file wasn't found or something similar
                    Log.e(TAG, "An error occurred while reading in the data:", e)
                } catch (e: JsonParseException) {
                    // Handle JsonParseException: Gets thrown when there is a problem with the contents of the file
                    Log.e(TAG, "An error occurred while reading in the data:", e)
                }
                finally {
                    // Close the reader to release system resources
                    reader?.close()
                }
            }
        }
    }

}

Your data structure:

data class City(
    @SerializedName("id")
    val id: Int,

    @SerializedName("country")
    val country: String,

    @SerializedName("state")
    val state: String,

    @SerializedName("name")
    val name: String,

    @SerializedName("coord")
    val coordinate: Coordinate
) {
    override fun toString(): String {
        return "#$id[$name $state $country]@[${coordinate.lat}|${coordinate.lon}]"
    }
}

data class Coordinate(
    @SerializedName("lat")
    val lat: Double,

    @SerializedName("lon")
    val lon: Double
)

In the best case you would put the code in which you get the file contents and parse the data in a ViewModel, but this would to go beyond the scope for this answer.

Additional information about ViewModels: https://developer.android.com/topic/libraries/architecture/viewmodel