CODEWITHSUNDEEP
rustserde
zino
zino

Reputation: 1472

Serde untagged attribute on enum results in stack overflow

Im using the untagged attribute on an enum to serialize and deserialize JSON.

// Just an example of using `untagged` - not actual code with the issue.
#[derive(Serialize, Deserialize)]
#[serde(untagged)]
enum Message {
    String(String),
    X(X),
}

I get a fatal runtime error: stack overflow for some of my types at runtime.

These types may have recursive definitions by using Box.

In general what would cause a stack overflow when using an untagged attribute?

The docs do not state any limitations so it seems like it should work for any code that compiles.

This is a stack trace I collected with Instruments/Sampler on Mac OS, __rust_probestack seems to be the last function called, at that point it's around 70 functions deep.

stack

Upvotes: 2

Views: 6554

Answers (2)

zino
zino

Reputation: 1472

https://github.com/serde-rs/serde/issues/1062#issuecomment-335659227

Fixed by setting the stack to 16MB:

export RUST_MIN_STACK=16777216 && cargo test

Upvotes: 1

Jmb
Jmb

Reputation: 23359

According to what you said in the comments, you have recursive types. The problem comes from the way untagged works. Here's a minimal example that reproduces the problem:

use serde::{Deserialize, Serialize};

#[derive(Serialize, Deserialize, Debug)]
#[serde(untagged)]
enum Foo {
    Foo(Box<Foo>),
    Bar(String),
}

fn main() {
    let data = r#""Bar""#;

    let _v: Foo = serde_json::from_str(&data).unwrap();
}

Playground

The problem is that with untagged, Serde tries to parse the data as the first variant of the enum, then if that fails it backtracks and tries the second variant and so on. Here's what happens with the example above:

  1. Serde is asked to parse a value of type Foo, which is untagged, so Serde tries to parse a value matching the first variant.
  2. The first variant is a Box<Foo>, so Serde tries to parse a value of type Foo, which is untagged, so Serde tries to parse a value matching the first variant.
  3. The first variant is a Box<Foo>, so Serde tries to parse a value of type Foo, which is untagged, so Serde tries to parse a value matching the first variant.
  4. And so on.

Note that this has nothing to do with the input data, this code fails before even attempting to read a single byte of data!

To avoid the issue, you must make sure that Serde will always consume some data before recursing, eg.:

use serde::{Deserialize, Serialize};

#[derive(Serialize, Deserialize, Debug)]
#[serde(untagged)]
enum Foo {
    Foo(String, Box<Foo>),
    Bar(String),
}

fn main() {
    let data = r#""Bar""#;

    let _v: Foo = serde_json::from_str(&data).unwrap();
}

Playground

Upvotes: 4

Related Questions