CODEWITHSUNDEEP
pythontensorflowkerasprotocol-buffersgrpc
SSEADGR
SSEADGR

Reputation: 11

Send a Keras model with gRPC

I am trying to implement a Federated Learning system with gRPC. Tensorflow Federated currently supports multi-machine remote learning, but something looks weird to me that it prepares client dataset on the server side. I expected that the dataset for client only resides and is prepared only in client's device, not on server's memory.

Tensors such as weight vectors are transferred well if i use tensor_shape.proto, tensor.proto, types.proto, resource_handle.proto and tensor_util.make_tensor_proto()which Tensorflow has been already using

hist = model.fit(...)
vector = model.trainable_variables
sending_tensors = []
for v in vector:
    tensor = tensor_util.make_tensor_proto(v.numpy(), shape=v.numpy().shape)
    sending_tensors.append(tensor)
#Some sending logic
...

But I couldn't find any kinds of "model.proto". I want to implement like the below as similar way as the above does.

#server
model = tf.keras.models.Sequential([ ... ])
model.compile(...)
model_proto = some_package.make_model_proto()

req = my_proto_pb2.Request(model=model_proto)
client_stub.some_grpc_service(req)

#client
class SomeServicer(...):
  def some_grpc_service(self, request, context):
    model_proto = request.model
    model = some_package.model_from_proto(model_proto)

    model.compile(...)
    model.fit(...)
    ...

with a protobuf defined as

message Request {
    ModelProto model = 1;
}

This is not about the Serving, just about the transferring a keras model through gRPC from server to clients. Any ways to do it ?

If not possible, am i supposed to send it simply as a stream of bytes read from .h5 file created by model.save() ?

Upvotes: 1

Views: 523

Answers (1)

Keith Rush
Keith Rush

Reputation: 1405

A quick note on TFF--currently many of the examples do have datasets materialized server-side, simply because of the FL-research-first design, for example improving federated optimization algorithms. TFF can currently support the kind of application desired here by simply using a different method to materialize datasets on the clients.

Since TF is the local computation engine, and TFF supports executing arbitrary TensorFlow, as long as you can load the dataset in TensorFlow based on the local client environment, TFF can support this. One example might be selecting some set of IDs in the outer Python layer, writing a TF function which takes these IDs and materializes a dataset from the local client environment, and mapping this function over these IDs. Such a pattern might look like:

@tff.tf_computation(tf.int32)
def materialize_dataset_on_client(x):
  ds = ... # inspect local environment, and load dataset as usual in TF
  return ds

@tff.federated_computation(
    tff.FederatedType(
        tff.TensorType(tf.int32, shape=[]), tff.CLIENTS),
    ... ) # model type in here as well presumably
def train(ids_and_model):
  ids, model = ids_and_model # simply unpacking here
  datasets = tff.federated_map(materialize_datasets_on_clients, ids)
  train_result = run_training(datasets, model) # Assume run_training_defined elsewhere
  return train_result

This kind of pattern would not result in the datasets being materialized at the server, only the result of training.

Upvotes: 1

Related Questions