How to handle errors when extracting data from untyped JSON in serde_json?

Question

I have a serde_json::Value which I expect to contain an array of objects. From those objects I want to extract 2 values and return an error if anything fails. This is my code so far:

use std::collections::HashMap;
use anyhow::Result;


fn get_stock(response: serde_json::Value) -> Result<HashMap<String, u32>>{
    response["products"]
        .as_array()?.iter()
        .map(|product| {
            let product = product.as_object()?;
            (
                product["name"].as_str()?.to_owned(),
                //as_u64 fails for some reason
                product["stock"].as_str()?.parse::<u32>()?,
            )
        })
        .collect()?
}

When I used .unwrap() this worked fine, but after changing the return type to Result and replacing unwraps with ? I'm getting the following compilation errors:

error[E0277]: the `?` operator can only be used on `Result`s, not `Option`s, in a function that returns `Result`
  --> src/bin/products.rs:7:20
   |
5  | / fn get_stock(response: serde_json::Value) -> Result<HashMap<String, u32>>{
6  | |     response["products"]
7  | |         .as_array()?.iter()
   | |                    ^ use `.ok_or(...)?` to provide an error compatible with `Result<HashMap<std::string::String, u32>, anyhow::Error>`
8  | |         .map(|product| {
...  |
16 | |         .collect()?
17 | | }
   | |_- this function returns a `Result`
   |
   = help: the trait `FromResidual<Option<Infallible>>` is not implemented for `Result<HashMap<std::string::String, u32>, anyhow::Error>`
   = note: required by `from_residual`

error[E0277]: the `?` operator can only be used in a closure that returns `Result` or `Option` (or another type that implements `FromResidual`)
  --> src/bin/products.rs:9:46
   |
8  |           .map(|product| {
   |  ______________-
9  | |             let product = product.as_object()?;
   | |                                              ^ cannot use the `?` operator in a closure that returns `(std::string::String, u32)`
10 | |             (
11 | |                 product["name"].as_str()?.to_owned(),
...  |
14 | |             )
15 | |         })
   | |_________- this function should return `Result` or `Option` to accept `?`
   |
   = help: the trait `FromResidual<Option<Infallible>>` is not implemented for `(std::string::String, u32)`
   = note: required by `from_residual`

error[E0277]: the `?` operator can only be used in a closure that returns `Result` or `Option` (or another type that implements `FromResidual`)
  --> src/bin/products.rs:11:41
   |
8  |           .map(|product| {
   |  ______________-
9  | |             let product = product.as_object()?;
10 | |             (
11 | |                 product["name"].as_str()?.to_owned(),
   | |                                         ^ cannot use the `?` operator in a closure that returns `(std::string::String, u32)`
...  |
14 | |             )
15 | |         })
   | |_________- this function should return `Result` or `Option` to accept `?`
   |
   = help: the trait `FromResidual<Option<Infallible>>` is not implemented for `(std::string::String, u32)`
   = note: required by `from_residual`

error[E0277]: the `?` operator can only be used in a closure that returns `Result` or `Option` (or another type that implements `FromResidual`)
  --> src/bin/products.rs:13:42
   |
8  |           .map(|product| {
   |  ______________-
9  | |             let product = product.as_object()?;
10 | |             (
11 | |                 product["name"].as_str()?.to_owned(),
12 | |                 //as_u64 fails for some reason
13 | |                 product["stock"].as_str()?.parse::<u32>()?,
   | |                                          ^ cannot use the `?` operator in a closure that returns `(std::string::String, u32)`
14 | |             )
15 | |         })
   | |_________- this function should return `Result` or `Option` to accept `?`
   |
   = help: the trait `FromResidual<Option<Infallible>>` is not implemented for `(std::string::String, u32)`
   = note: required by `from_residual`

error[E0277]: the `?` operator can only be used in a closure that returns `Result` or `Option` (or another type that implements `FromResidual`)
  --> src/bin/products.rs:13:58
   |
8  |           .map(|product| {
   |  ______________-
9  | |             let product = product.as_object()?;
10 | |             (
11 | |                 product["name"].as_str()?.to_owned(),
12 | |                 //as_u64 fails for some reason
13 | |                 product["stock"].as_str()?.parse::<u32>()?,
   | |                                                          ^ cannot use the `?` operator in a closure that returns `(std::string::String, u32)`
miav@battlestation catbot % cargo run --bin products
   Compiling catbot v0.1.0 (/Users/miav/Documents/Personal/Projects/programming/catbot)
error[E0277]: the `?` operator can only be used on `Result`s, not `Option`s, in a function that returns `Result`
  --> src/bin/products.rs:7:20
   |
5  | / fn get_stock(response: serde_json::Value) -> Result<HashMap<String, u32>>{
6  | |     response["products"]
7  | |         .as_array()?.iter()
   | |                    ^ use `.ok_or(...)?` to provide an error compatible with `Result<HashMap<std::string::String, u32>, anyhow::Error>`
8  | |         .map(|product| {
...  |
16 | |         .collect()?
17 | | }
   | |_- this function returns a `Result`
   |
   = help: the trait `FromResidual<Option<Infallible>>` is not implemented for `Result<HashMap<std::string::String, u32>, anyhow::Error>`
   = note: required by `from_residual`

error[E0277]: the `?` operator can only be used in a closure that returns `Result` or `Option` (or another type that implements `FromResidual`)
  --> src/bin/products.rs:9:46
   |
8  |           .map(|product| {
   |  ______________-
9  | |             let product = product.as_object()?;
   | |                                              ^ cannot use the `?` operator in a closure that returns `(std::string::String, u32)`
10 | |             (
11 | |                 product["name"].as_str()?.to_owned(),
...  |
14 | |             )
15 | |         })
   | |_________- this function should return `Result` or `Option` to accept `?`
   |
   = help: the trait `FromResidual<Option<Infallible>>` is not implemented for `(std::string::String, u32)`
   = note: required by `from_residual`

error[E0277]: the `?` operator can only be used in a closure that returns `Result` or `Option` (or another type that implements `FromResidual`)
  --> src/bin/products.rs:11:41
   |
8  |           .map(|product| {
   |  ______________-
9  | |             let product = product.as_object()?;
10 | |             (
11 | |                 product["name"].as_str()?.to_owned(),
   | |                                         ^ cannot use the `?` operator in a closure that returns `(std::string::String, u32)`
...  |
14 | |             )
15 | |         })
   | |_________- this function should return `Result` or `Option` to accept `?`
   |
   = help: the trait `FromResidual<Option<Infallible>>` is not implemented for `(std::string::String, u32)`
   = note: required by `from_residual`

error[E0277]: the `?` operator can only be used in a closure that returns `Result` or `Option` (or another type that implements `FromResidual`)
  --> src/bin/products.rs:13:42
   |
8  |           .map(|product| {
   |  ______________-
9  | |             let product = product.as_object()?;
10 | |             (
11 | |                 product["name"].as_str()?.to_owned(),
12 | |                 //as_u64 fails for some reason
13 | |                 product["stock"].as_str()?.parse::<u32>()?,
   | |                                          ^ cannot use the `?` operator in a closure that returns `(std::string::String, u32)`
14 | |             )
15 | |         })
   | |_________- this function should return `Result` or `Option` to accept `?`
   |
   = help: the trait `FromResidual<Option<Infallible>>` is not implemented for `(std::string::String, u32)`
   = note: required by `from_residual`

error[E0277]: the `?` operator can only be used in a closure that returns `Result` or `Option` (or another type that implements `FromResidual`)
  --> src/bin/products.rs:13:58
   |
8  |           .map(|product| {
   |  ______________-
9  | |             let product = product.as_object()?;
10 | |             (
11 | |                 product["name"].as_str()?.to_owned(),
12 | |                 //as_u64 fails for some reason
13 | |                 product["stock"].as_str()?.parse::<u32>()?,
   | |                                                          ^ cannot use the `?` operator in a closure that returns `(std::string::String, u32)`
14 | |             )
15 | |         })
   | |_________- this function should return `Result` or `Option` to accept `?`
   |
   = help: the trait `FromResidual<Result<Infallible, ParseIntError>>` is not implemented for `(std::string::String, u32)`
   = note: required by `from_residual`

error: aborting due to 5 previous errors

I don't know the exact structure of the JSON in advance, so I cannot parse it into a struct. All I know is that it is most likely an array of objects which have a name string field and a stock integer field, which I want to extract into a map. If that doesn't happen to be the case, then I want to return an error. What is the simplest way to do this?

UPDATE: After following Lagerbaer's suggestions and doing some digging I came up with the following solution.

use anyhow::{anyhow, Result};
use std::collections::HashMap;

fn get_stock(response: serde_json::Value) -> Result<HashMap<String, u32>> {
    response
        .get("products")
        .ok_or(anyhow!("'products' not found"))?
        .as_array()
        .ok_or(anyhow!("'products' is not an array"))?
        .iter()
        .map(|product| -> Result<(String, u32)> {
            let product = product.as_object().unwrap();
            Ok((
                product
                    .get("name")
                    .ok_or(anyhow!("'name' not found"))?
                    .as_str()
                    .ok_or(anyhow!("'name' is not a string"))?
                    .trim()
                    .to_owned(),
                //as_u64 fails for some reason
                product
                    .get("stock")
                    .ok_or(anyhow!("'stock' not found"))?
                    .as_str()
                    .ok_or(anyhow!("'stock' is not a string"))?
                    .parse::<u32>()?,
            ))
        })
        .collect()
}

Used ok_or() to map Option to Result, had to use the anyhow! macro to make it compatible with the anyhow result type. It also turns out that collect() actually accepts an iterator of Result<(String, u32)> to produce a Result<HashMap<String, u32>> with the exact behavior I wanted, that is, returning the first error or the complete hash map if there were no errors.

Answer 1

So there's a couple issues here and the compiler tries its best to tell you.

I'll get you started on the first, and then I encourage you to try going it alone for a bit.

So the very first error is that, apparently, as_array returns an Option and not Result and, hence, you can't use the ? operator there.

Luckily, the compiler tells you what to do: Option has a method ok_or that you should call. It will turn the Option into Result. If the Option is Some(value) you'll get a Ok(value), and if the Option is None, you'll get whatever error you specified in the ok_or argument.

Another easy to spot mistake is this: Your function returns a Result, so the return value should, obviously, be a Result. Hence the very final last ? should be removed, because that ? would take a Result and turn it into the "pure" value (if the result was Ok).

And then what's left is to figure out exactly what the closure should return. Maybe you can try that after fixing the other mistakes that I explained above.