null value issue in models in grpc-go

Question

I have two servers

1. User Server: handle all the user CRUD operation

2. Product Server: Handle Product CRUD operation and fetch user info from the user server via gRPC call

   type User struct {
       ID string `json:"id"`
       FirstName      string     `json:"firstName"`
       MiddleName     *string    `json:"middleName,omitempty"`
       LastName       string     `json:"lastName"`
       Email          string     `json:"email"`
       Disabled       bool       `json:"disabled"`
       LastSignedInAt *time.Time `json:"lastSignedInAt,omitempty"`
       Bio            *string    `json:"bio,omitempty"`
       BirthDate      *time.Time `json:"birthDate,omitempty"`
   }
   

   Here some fields are optionals and as I am using cockroachDB(extended postgreSQL), I kept them as a pointer so it's easy in scaning variable form query result.

And here is my proto file:

message User {
    int64 id = 1;
    string firstName = 2;
    string lastName = 3;
    string email = 5;
    bool disabled = 6;
    string lastSignedInAt = 8;
    string bio = 9;
    string birthdate = 10;
    string profession = 14;
}

Now generated model from above proto file is like this:"

type User struct {
    Id                   int64                 `protobuf:"varint,1,opt,name=id,proto3" json:"id,omitempty"`
    FirstName            string                `protobuf:"bytes,2,opt,name=firstName,proto3" json:"firstName,omitempty"`
    LastName             string                `protobuf:"bytes,3,opt,name=lastName,proto3" json:"lastName,omitempty"`
    Email                string                `protobuf:"bytes,4,opt,name=email,json=email,proto3" json:"email,omitempty"`
    Disabled             bool                  `protobuf:"varint,6,opt,name=disabled,proto3" json:"disabled,omitempty"`
    LastSignedInAt       string                `protobuf:"bytes,8,opt,name=lastSignedInAt,proto3" json:"lastSignedInAt,omitempty"`
    Bio                  string                `protobuf:"bytes,9,opt,name=bio,proto3" json:"bio,omitempty"`
    Birthdate            string                `protobuf:"bytes,10,opt,name=birthdate,proto3" json:"birthdate,omitempty"`
}

Now the problem is as I am using a pointer for the optional field it will store null in case of no value but on the opposite site gRPC won't understand the null value and throw the error.

I have tried google.protobuf.StringValue value as a grpc type like this

google.protobuf.StringValue lastSignedInAt = 8;

This works but the problem is I have to write condition for each field in the handler:

if lastSignedInAt != nil {
    user.LastSignedInAt = &wrappers.StringValue{Value:*lastSignedInAt}
}

What is the best way to tackle this issue? Should I change the database model or any changes in the gRPC model?

Answer 1

As another answer above noted, if the protocol buffer message has fields that can be nil it is on you to check for them. There isn't much you can do about this, unless there's a utility package out there that does it.

If you want defaults above and beyond the defaults protocol buffers generates, you'll have to do exactly what you noted and check for nil.

I do have some questions though:

• Why use pointers for the "optional" fields in first User struct? If you used plain-old string they would be populated with the empty string on construction and untouched by the json unmarshal call if the field is missing. Same for the time fields. In this case the default value for string (the empty string) should be an invalid middle name, and the default value for time (0001-01-01 00:00:00 +0000 UTC) is an invalid timestamp (probably?). Getting rid of the pointers allows you to use the defaults.
• For the timestamp in the proto struct you could still use string and check for the empty string. Or you could use google.protobuf.Timestamp and use ptypes to handle the conversions to/from the non-proto struct. See https://godoc.org/github.com/golang/protobuf/ptypes#TimestampProto. You'll have to check for nil in this case.

Answer 2

grpc not recommended to use point field, If you insist on use pointers

the one way is use reflect to convert it

func ToGrpcData(in, out interface{}) error {
    inVal := reflect.ValueOf(in)
    if inVal.Kind() == reflect.Ptr {
        inVal = inVal.Elem()
    }
    inTyp := inVal.Type()

    outVal := reflect.ValueOf(out)
    if outVal.Kind() != reflect.Ptr {
        return errors.New("out data must be point value")
    }

    outVal = outVal.Elem()
    outTyp := outVal.Type()

    strWrapperType := reflect.TypeOf(wrappers.StringValue{})
    // range all 'in' fields
    for i := 0; i < inVal.NumField(); i++ {
        fTyp := inTyp.Field(i)
        fVal := inVal.Field(i)
        if fTyp.Type.Kind() == reflect.Ptr {
            switch fTyp.Type.Elem().Kind() {
            case reflect.String: // only implement string in this test
                oFTyp, ok := outTyp.FieldByName(fTyp.Name)
                if ok == false {
                    return errors.New("not match field in out value")
                }
                if oFTyp.Type.Elem() != strWrapperType {
                    return errors.New("not match field in out value")
                }
                if fVal.IsNil() == false {
                    outVal.FieldByName(fTyp.Name).Set(
                        reflect.ValueOf(&wrappers.StringValue{
                            Value: fVal.Elem().String(),
                        }),
                    )
                }
            }
        } else {
            outVal.FieldByName(fTyp.Name).Set(fVal)
        }
    }
    return nil
}

usage

type User struct {
    Name  string
    Value *string
}
type ServerUser struct {
    Name  string
    Value *wrappers.StringValue
}
usValue := "123"
u1 := User{
    Name:  "tom",
    Value: &usValue,
}
u2 := ServerUser{}
err := ToGrpcData(&u1, &u2)
// error process ...
fmt.Println(u2)

Answer 3

You cannot set the null value instead of you you can use

oneof Examples {
    Example1 example1 = 1;
    Example2 example2 = 2;
}

when you use oneof you have to set only one value either you can set example1 or example2 you cannot use both at same time. This will resolve your issue as compared to setting the nil value.

Approach 2:

And by default gRPC have all the variable has initial value ex: string: ""

One thing you can also do don't set nil value check with the condition if your value is nil then set nothing.