Sam Dean
Reputation: 404
Google App Engine communicating with Google Cloud Run using GRPC gives "Error: 14 UNAVAILABLE: Connection dropped"
tl;dr
A gRPC server written in C++ in an image (Docker locally, Cloud Run online) works locally but doesn't work online when gRPC client written in node.js sends a request to it. This is true whether the client is local or on Google App engine. The image definitely works because it responds correctly when grpcurl sends a request to it.
Full details
The client code in Node.js
import {loadPackageDefinition, credentials} from '@grpc/grpc-js';
import {loadSync} from '@grpc/proto-loader';
const packageDef = loadSync(
'./path/to/file.proto',
{keepCase: true,
longs: String,
enums: String,
defaults: true,
oneofs: true,
},
);
const packageNameProto = loadPackageDefinition(packageDef).packageName;
const client = new packageProto.ServiceName(
`${host}:${port}`,
credentials.createInsecure(),
);
client.calculation(tempInputs, function(err, result) {
// code
});
The server code in C++.
void RunServer()
{
int port = 50051;
char* port_ptr = std::getenv("PORT");
if (port_ptr != nullptr) {
port = std::atoi(port_ptr);
}
if (port < MIN_PORT or MAX_PORT < port) {
port = 50051;
}
std::ostringstream server_address;
server_address << "0.0.0.0:" << port;
// std::string server_address("0.0.0.0:50051");
ServiceNameServiceImpl service;
grpc::EnableDefaultHealthCheckService(true);
grpc::reflection::InitProtoReflectionServerBuilderPlugin();
ServerBuilder builder;
// Listen on the given address without any authentication mechanism.
builder.AddListeningPort(server_address.str(), grpc::InsecureServerCredentials());
// Register "service" as the instance through which we'll communicate with
// clients. In this case it corresponds to an *synchronous* service.
builder.RegisterService(&service);
// Finally assemble the server.
std::unique_ptr<Server> server(builder.BuildAndStart());
std::cout << "Server listening on " << server_address.str() << std::endl;
// Wait for the server to shutdown. Note that some other thread must be
// responsible for shutting down the server for this call to ever return.
server->Wait();
}
int main(int argc, char** argv)
{
RunServer();
return 0;
}
Client and server, GCP and local compatibility table
| Local Server on Docker in WSL2 | Cloud Run Server | |
|---|---|---|
| Local Client from command line in Ubuntu on WSL2 | Works using localhost:50051 and 'my-external-ip address':50051 port forwarded to my machine | Doesn't work using 'cloud-run-url':443, giving the "Error: 14 UNAVAILABLE: Connection dropped" |
| App Engine Client | Works using 'my-external-ip address':50051 port forwarded to my machine | Doesn't work using 'cloud-run-url':443, giving the "Error: 14 UNAVAILABLE: Connection dropped" |
| grpcurl command in Ubuntu on WSL2 | X | Works using 'cloud-run-url':443 as cloud run wants requests on 443. Command is 'docker run -i -v pwd:/protos fullstorydev/grpcurl -d @ -proto protos/file.proto cloud.run.url.run.app:443 protoPackage.ProtoService.proto_function < inputs.json' |
Server image is built and deployed according to Google documentation.
Client is deployed with gcloud app deploy.
In the cloud run service triggers I have 'Ingress' set to 'Allow all traffic' and 'Authentication' set to 'Allow unauthenticated invocations'.
HTTP/2 connections are enabled. I have also tried with them disabled as the 'Using gRPC' guide says you only need to set it up if you're using gRPC streaming, which I'm not.
Error log: (private info removed)
{
"textPayload": "Error: 14 UNAVAILABLE: Connection dropped\n at Object.callErrorFromStatus (/workspace/node_modules/@grpc/grpc-js/build/src/call.js:31:26)\n at Object.onReceiveStatus (/workspace/node_modules/@grpc/grpc-js/build/src/client.js:180:52)\n at Object.onReceiveStatus (/workspace/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:336:141)\n at Object.onReceiveStatus (/workspace/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:299:181)\n at /workspace/node_modules/@grpc/grpc-js/build/src/call-stream.js:160:78\n at processTicksAndRejections (internal/process/task_queues.js:77:11) {",
"insertId": "61adfe9d000b03a0c#######",
"resource": {
"type": "gae_app",
"labels": {
"project_id": "$PROJECT_ID",
"module_id": "default",
"zone": "europe-west2-3",
"version_id": "20211206t121319"
}
},
"timestamp": "2021-12-06T12:14:21.721824Z",
"severity": "ERROR",
"labels": {
"clone_id": "00c61b117ca5f61b7b4b7b7dde5eddded10f5bead78cf0cf0eee3fe1111a982786d972c65976ff49ddd14eaab4b708ed52349b06cc1538deacc7a89b8d6641b55a14f5799#######"
},
"logName": "projects/$PROJECT_ID/logs/stdout",
"receiveTimestamp": "2021-12-06T12:14:22.008561030Z"
}
Error log with 'GRPC_TRACE=all GRPC_VERBOSITY=DEBUG': (private info removed)
D 2021-12-06T12:25:38.616Z | resolving_load_balancer | dns:$CLOUD_RUN_URL:443 CONNECTING -> CONNECTING
D 2021-12-06T12:25:38.616Z | channel | (22) dns:$CLOUD_RUN_URL:443 callRefTimer.unref | configSelectionQueue.length=0 pickQueue.length=0
D 2021-12-06T12:25:38.616Z | channel | (22) dns:$CLOUD_RUN_URL:443 Pick result: QUEUE subchannel: undefined status: undefined undefined
D 2021-12-06T12:25:38.616Z | channel | (22) dns:$CLOUD_RUN_URL:443 callRefTimer.ref | configSelectionQueue.length=0 pickQueue.length=1
D 2021-12-06T12:25:38.616Z | connectivity_state | (22) dns:$CLOUD_RUN_URL:443 CONNECTING -> CONNECTING
D 2021-12-06T12:25:38.617Z | subchannel | (24) $CLOUD_RUN_IP_ADD_1:443 CONNECTING -> READY
D 2021-12-06T12:25:38.618Z | pick_first | Pick subchannel with address $CLOUD_RUN_IP_ADD_1:443
D 2021-12-06T12:25:38.618Z | pick_first | CONNECTING -> READY
D 2021-12-06T12:25:38.618Z | resolving_load_balancer | dns:$CLOUD_RUN_URL:443 CONNECTING -> READY
D 2021-12-06T12:25:38.618Z | channel | (22) dns:$CLOUD_RUN_URL:443 callRefTimer.unref | configSelectionQueue.length=0 pickQueue.length=0
D 2021-12-06T12:25:38.618Z | channel | (22) dns:$CLOUD_RUN_URL:443 Pick result: COMPLETE subchannel: $CLOUD_RUN_IP_ADD_1:443 status: undefined undefined
D 2021-12-06T12:25:38.618Z | connectivity_state | (22) dns:$CLOUD_RUN_URL:443 CONNECTING -> READY
D 2021-12-06T12:25:38.618Z | subchannel_refcount | (24) $CLOUD_RUN_IP_ADD_1:443 refcount 2 -> 3
D 2021-12-06T12:25:38.618Z | subchannel_refcount | (23) $CLOUD_RUN_IPV6_1::a:443 refcount 3 -> 2
D 2021-12-06T12:25:38.618Z | subchannel_refcount | (24) $CLOUD_RUN_IP_ADD_1:443 refcount 3 -> 2
D 2021-12-06T12:25:38.619Z | subchannel_refcount | (25) $CLOUD_RUN_IPV6_2::a:443 refcount 3 -> 2
D 2021-12-06T12:25:38.619Z | subchannel_refcount | (26) $CLOUD_RUN_IP_ADD_2:443 refcount 3 -> 2
D 2021-12-06T12:25:38.619Z | subchannel_refcount | (27) $CLOUD_RUN_IPV6_3::a:443 refcount 3 -> 2
D 2021-12-06T12:25:38.619Z | subchannel_refcount | (28) $CLOUD_RUN_IP_ADD_3:443 refcount 3 -> 2
D 2021-12-06T12:25:38.619Z | subchannel_refcount | (29) $CLOUD_RUN_IPV6_4::a:443 refcount 3 -> 2
D 2021-12-06T12:25:38.619Z | subchannel_refcount | (30) $CLOUD_RUN_IP_ADD_4:443 refcount 3 -> 2
D 2021-12-06T12:25:38.619Z | subchannel_refcount | (31) $CLOUD_RUN_IPV6_3::35:443 refcount 2 -> 1
D 2021-12-06T12:25:38.619Z | subchannel_refcount | (32) $CLOUD_RUN_IP_ADD_5:443 refcount 3 -> 2
D 2021-12-06T12:25:38.619Z | subchannel_refcount | (23) $CLOUD_RUN_IPV6_1::a:443 refcount 2 -> 1
D 2021-12-06T12:25:38.619Z | subchannel_refcount | (26) $CLOUD_RUN_IP_ADD_2:443 refcount 2 -> 1
D 2021-12-06T12:25:38.619Z | subchannel_refcount | (25) $CLOUD_RUN_IPV6_2::a:443 refcount 2 -> 1
D 2021-12-06T12:25:38.619Z | subchannel_refcount | (28) $CLOUD_RUN_IP_ADD_3:443 refcount 2 -> 1
D 2021-12-06T12:25:38.619Z | subchannel_refcount | (27) $CLOUD_RUN_IPV6_3::a:443 refcount 2 -> 1
D 2021-12-06T12:25:38.619Z | subchannel_refcount | (30) $CLOUD_RUN_IP_ADD_4:443 refcount 2 -> 1
D 2021-12-06T12:25:38.619Z | subchannel_refcount | (29) $CLOUD_RUN_IPV6_4::a:443 refcount 2 -> 1
D 2021-12-06T12:25:38.619Z | subchannel_refcount | (32) $CLOUD_RUN_IP_ADD_5:443 refcount 2 -> 1
D 2021-12-06T12:25:38.619Z | call_stream | Starting stream on subchannel (24) $CLOUD_RUN_IP_ADD_1:443 with headers
grpc-accept-encoding: identity,deflate,gzip
accept-encoding: identity
:authority: $CLOUD_RUN_URL:443
user-agent: grpc-node-js/1.4.4
content-type: application/grpc
:method: POST
:path: /protoPackage.ProtoService/proto_function
te: trailers
D 2021-12-06T12:25:38.620Z | call_stream | [3] attachHttp2Stream from subchannel $CLOUD_RUN_IP_ADD_1:443
D 2021-12-06T12:25:38.620Z | subchannel_refcount | (24) $CLOUD_RUN_IP_ADD_1:443 callRefcount 0 -> 1
D 2021-12-06T12:25:38.620Z | call_stream | [3] sending data chunk of length 223 (deferred)
D 2021-12-06T12:25:38.620Z | call_stream | [3] calling end() on HTTP/2 stream
D 2021-12-06T12:25:38.627Z | call_stream | [3] ended with status: code=14 details="Connection dropped"
D 2021-12-06T12:25:38.628Z | subchannel_refcount | (24) $CLOUD_RUN_IP_ADD_1:443 callRefcount 1 -> 0
D 2021-12-06T12:25:38.628Z | call_stream | [3] close http2 stream with code 8
ERROR!
Error: 14 UNAVAILABLE: Connection dropped
at Object.callErrorFromStatus (/home/sam/nodejs/elevate-online-njs/node_modules/@grpc/grpc-js/build/src/call.js:31:26)
at Object.onReceiveStatus (/home/sam/nodejs/elevate-online-njs/node_modules/@grpc/grpc-js/build/src/client.js:180:52)
at Object.onReceiveStatus (/home/sam/nodejs/elevate-online-njs/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:336:141)
at Object.onReceiveStatus (/home/sam/nodejs/elevate-online-njs/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:299:181)
at /home/sam/nodejs/elevate-online-njs/node_modules/@grpc/grpc-js/build/src/call-stream.js:160:78
at processTicksAndRejections (internal/process/task_queues.js:77:11) {
code: 14,
details: 'Connection dropped',
metadata: Metadata { internalRepr: Map(0) {}, options: {} }
}
D 2021-12-06T12:25:38.631Z | subchannel | (24) $CLOUD_RUN_IP_ADD_1:443 connection closed
D 2021-12-06T12:25:38.632Z | subchannel | (24) $CLOUD_RUN_IP_ADD_1:443 READY -> IDLE
D 2021-12-06T12:25:38.632Z | subchannel_refcount | (24) $CLOUD_RUN_IP_ADD_1:443 refcount 2 -> 1
D 2021-12-06T12:25:38.632Z | pick_first | READY -> IDLE
D 2021-12-06T12:25:38.632Z | resolving_load_balancer | dns:$CLOUD_RUN_URL:443 READY -> IDLE
D 2021-12-06T12:25:38.632Z | connectivity_state | (22) dns:$CLOUD_RUN_URL:443 READY -> IDLE
D 2021-12-06T12:25:38.632Z | call_stream | [3] HTTP/2 stream closed with code 8
D 2021-12-06T12:25:39.275Z | subchannel | (2) $CLOUD_RUN_IPV6_1::a:443 TRANSIENT_FAILURE -> CONNECTING
D 2021-12-06T12:25:39.275Z | subchannel | (2) $CLOUD_RUN_IPV6_1::a:443 creating HTTP/2 session
D 2021-12-06T12:25:39.277Z | subchannel | (2) $CLOUD_RUN_IPV6_1::a:443 connection closed with error connect ENETUNREACH $CLOUD_RUN_IPV6_1::a:443 - Local (:::0)
D 2021-12-06T12:25:39.277Z | subchannel | (2) $CLOUD_RUN_IPV6_1::a:443 connection closed
D 2021-12-06T12:25:39.277Z | subchannel | (2) $CLOUD_RUN_IPV6_1::a:443 CONNECTING -> TRANSIENT_FAILURE
D 2021-12-06T12:25:39.278Z | subchannel | (2) $CLOUD_RUN_IPV6_1::a:443 TRANSIENT_FAILURE -> IDLE
D 2021-12-06T12:25:39.278Z | subchannel | (4) $CLOUD_RUN_IPV6_2::a:443 TRANSIENT_FAILURE -> CONNECTING
D 2021-12-06T12:25:39.279Z | subchannel | (4) $CLOUD_RUN_IPV6_2::a:443 creating HTTP/2 session
D 2021-12-06T12:25:39.280Z | subchannel | (4) $CLOUD_RUN_IPV6_2::a:443 connection closed with error connect ENETUNREACH $CLOUD_RUN_IPV6_2::a:443 - Local (:::0)
D 2021-12-06T12:25:39.281Z | subchannel | (4) $CLOUD_RUN_IPV6_2::a:443 connection closed
D 2021-12-06T12:25:39.281Z | subchannel | (4) $CLOUD_RUN_IPV6_2::a:443 CONNECTING -> TRANSIENT_FAILURE
D 2021-12-06T12:25:39.281Z | subchannel | (4) $CLOUD_RUN_IPV6_2::a:443 TRANSIENT_FAILURE -> IDLE
D 2021-12-06T12:25:39.281Z | subchannel | (6) $CLOUD_RUN_IPV6_3::a:443 TRANSIENT_FAILURE -> CONNECTING
D 2021-12-06T12:25:39.282Z | subchannel | (6) $CLOUD_RUN_IPV6_3::a:443 creating HTTP/2 session
D 2021-12-06T12:25:39.284Z | subchannel | (6) $CLOUD_RUN_IPV6_3::a:443 connection closed with error connect ENETUNREACH $CLOUD_RUN_IPV6_3::a:443 - Local (:::0)
D 2021-12-06T12:25:39.284Z | subchannel | (6) $CLOUD_RUN_IPV6_3::a:443 connection closed
D 2021-12-06T12:25:39.284Z | subchannel | (6) $CLOUD_RUN_IPV6_3::a:443 CONNECTING -> TRANSIENT_FAILURE
D 2021-12-06T12:25:39.284Z | subchannel | (6) $CLOUD_RUN_IPV6_3::a:443 TRANSIENT_FAILURE -> IDLE
D 2021-12-06T12:25:39.284Z | subchannel | (8) $CLOUD_RUN_IPV6_4::a:443 TRANSIENT_FAILURE -> CONNECTING
D 2021-12-06T12:25:39.285Z | subchannel | (8) $CLOUD_RUN_IPV6_4::a:443 creating HTTP/2 session
D 2021-12-06T12:25:39.286Z | subchannel | (8) $CLOUD_RUN_IPV6_4::a:443 connection closed with error connect ENETUNREACH $CLOUD_RUN_IPV6_4::a:443 - Local (:::0)
D 2021-12-06T12:25:39.287Z | subchannel | (8) $CLOUD_RUN_IPV6_4::a:443 connection closed
D 2021-12-06T12:25:39.288Z | subchannel | (8) $CLOUD_RUN_IPV6_4::a:443 CONNECTING -> TRANSIENT_FAILURE
D 2021-12-06T12:25:39.289Z | subchannel | (8) $CLOUD_RUN_IPV6_4::a:443 TRANSIENT_FAILURE -> IDLE
D 2021-12-06T12:25:39.290Z | subchannel | (10) $CLOUD_RUN_IPV6_5::200a:443 TRANSIENT_FAILURE -> CONNECTING
D 2021-12-06T12:25:39.291Z | subchannel | (10) $CLOUD_RUN_IPV6_5::200a:443 creating HTTP/2 session
D 2021-12-06T12:25:39.293Z | subchannel | (10) $CLOUD_RUN_IPV6_5::200a:443 connection closed with error connect ENETUNREACH $CLOUD_RUN_IPV6_5::200a:443 - Local (:::0)
D 2021-12-06T12:25:39.293Z | subchannel | (10) $CLOUD_RUN_IPV6_5::200a:443 connection closed
D 2021-12-06T12:25:39.294Z | subchannel | (10) $CLOUD_RUN_IPV6_5::200a:443 CONNECTING -> TRANSIENT_FAILURE
D 2021-12-06T12:25:39.294Z | subchannel | (10) $CLOUD_RUN_IPV6_5::200a:443 TRANSIENT_FAILURE -> IDLE
D 2021-12-06T12:25:39.602Z | subchannel | (23) $CLOUD_RUN_IPV6_1::a:443 TRANSIENT_FAILURE -> CONNECTING
D 2021-12-06T12:25:39.602Z | subchannel | (23) $CLOUD_RUN_IPV6_1::a:443 creating HTTP/2 session
D 2021-12-06T12:25:39.604Z | subchannel | (23) $CLOUD_RUN_IPV6_1::a:443 connection closed with error connect ENETUNREACH $CLOUD_RUN_IPV6_1::a:443 - Local (:::0)
D 2021-12-06T12:25:39.604Z | subchannel | (23) $CLOUD_RUN_IPV6_1::a:443 connection closed
D 2021-12-06T12:25:39.604Z | subchannel | (23) $CLOUD_RUN_IPV6_1::a:443 CONNECTING -> TRANSIENT_FAILURE
D 2021-12-06T12:25:39.605Z | subchannel | (23) $CLOUD_RUN_IPV6_1::a:443 TRANSIENT_FAILURE -> IDLE
D 2021-12-06T12:25:39.605Z | subchannel | (25) $CLOUD_RUN_IPV6_2::a:443 TRANSIENT_FAILURE -> CONNECTING
D 2021-12-06T12:25:39.606Z | subchannel | (25) $CLOUD_RUN_IPV6_2::a:443 creating HTTP/2 session
D 2021-12-06T12:25:39.607Z | subchannel | (25) $CLOUD_RUN_IPV6_2::a:443 connection closed with error connect ENETUNREACH $CLOUD_RUN_IPV6_2::a:443 - Local (:::0)
D 2021-12-06T12:25:39.607Z | subchannel | (25) $CLOUD_RUN_IPV6_2::a:443 connection closed
D 2021-12-06T12:25:39.607Z | subchannel | (25) $CLOUD_RUN_IPV6_2::a:443 CONNECTING -> TRANSIENT_FAILURE
D 2021-12-06T12:25:39.608Z | subchannel | (25) $CLOUD_RUN_IPV6_2::a:443 TRANSIENT_FAILURE -> IDLE
D 2021-12-06T12:25:39.608Z | subchannel | (27) $CLOUD_RUN_IPV6_3::a:443 TRANSIENT_FAILURE -> CONNECTING
D 2021-12-06T12:25:39.608Z | subchannel | (27) $CLOUD_RUN_IPV6_3::a:443 creating HTTP/2 session
D 2021-12-06T12:25:39.610Z | subchannel | (27) $CLOUD_RUN_IPV6_3::a:443 connection closed with error connect ENETUNREACH $CLOUD_RUN_IPV6_3::a:443 - Local (:::0)
D 2021-12-06T12:25:39.610Z | subchannel | (27) $CLOUD_RUN_IPV6_3::a:443 connection closed
D 2021-12-06T12:25:39.610Z | subchannel | (27) $CLOUD_RUN_IPV6_3::a:443 CONNECTING -> TRANSIENT_FAILURE
D 2021-12-06T12:25:39.611Z | subchannel | (27) $CLOUD_RUN_IPV6_3::a:443 TRANSIENT_FAILURE -> IDLE
D 2021-12-06T12:25:39.611Z | subchannel | (29) $CLOUD_RUN_IPV6_4::a:443 TRANSIENT_FAILURE -> CONNECTING
D 2021-12-06T12:25:39.611Z | subchannel | (29) $CLOUD_RUN_IPV6_4::a:443 creating HTTP/2 session
D 2021-12-06T12:25:39.612Z | subchannel | (29) $CLOUD_RUN_IPV6_4::a:443 connection closed with error connect ENETUNREACH $CLOUD_RUN_IPV6_4::a:443 - Local (:::0)
D 2021-12-06T12:25:39.612Z | subchannel | (29) $CLOUD_RUN_IPV6_4::a:443 connection closed
D 2021-12-06T12:25:39.612Z | subchannel | (29) $CLOUD_RUN_IPV6_4::a:443 CONNECTING -> TRANSIENT_FAILURE
D 2021-12-06T12:25:39.612Z | subchannel | (29) $CLOUD_RUN_IPV6_4::a:443 TRANSIENT_FAILURE -> IDLE
D 2021-12-06T12:25:39.613Z | subchannel | (31) $CLOUD_RUN_IPV6_3::35:443 TRANSIENT_FAILURE -> CONNECTING
D 2021-12-06T12:25:39.613Z | subchannel | (31) $CLOUD_RUN_IPV6_3::35:443 creating HTTP/2 session
D 2021-12-06T12:25:39.614Z | subchannel | (31) $CLOUD_RUN_IPV6_3::35:443 connection closed with error connect ENETUNREACH $CLOUD_RUN_IPV6_3::35:443 - Local (:::0)
D 2021-12-06T12:25:39.614Z | subchannel | (31) $CLOUD_RUN_IPV6_3::35:443 connection closed
D 2021-12-06T12:25:39.614Z | subchannel | (31) $CLOUD_RUN_IPV6_3::35:443 CONNECTING -> TRANSIENT_FAILURE
D 2021-12-06T12:25:39.614Z | subchannel | (31) $CLOUD_RUN_IPV6_3::35:443 TRANSIENT_FAILURE -> IDLE
D 2021-12-06T12:25:48.270Z | subchannel_refcount | (2) $CLOUD_RUN_IPV6_1::a:443 refcount 1 -> 0
D 2021-12-06T12:25:48.270Z | subchannel_refcount | (4) $CLOUD_RUN_IPV6_2::a:443 refcount 1 -> 0
D 2021-12-06T12:25:48.270Z | subchannel_refcount | (5) $CLOUD_RUN_IP_ADD_2:443 refcount 1 -> 0
D 2021-12-06T12:25:48.271Z | subchannel | (5) $CLOUD_RUN_IP_ADD_2:443 CONNECTING -> TRANSIENT_FAILURE
D 2021-12-06T12:25:48.271Z | subchannel_refcount | (6) $CLOUD_RUN_IPV6_3::a:443 refcount 1 -> 0
D 2021-12-06T12:25:48.271Z | subchannel_refcount | (7) $CLOUD_RUN_IP_ADD_3:443 refcount 1 -> 0
D 2021-12-06T12:25:48.271Z | subchannel | (7) $CLOUD_RUN_IP_ADD_3:443 CONNECTING -> TRANSIENT_FAILURE
D 2021-12-06T12:25:48.271Z | subchannel_refcount | (8) $CLOUD_RUN_IPV6_4::a:443 refcount 1 -> 0
D 2021-12-06T12:25:48.271Z | subchannel_refcount | (9) $CLOUD_RUN_IP_ADD_4:443 refcount 1 -> 0
D 2021-12-06T12:25:48.272Z | subchannel | (9) $CLOUD_RUN_IP_ADD_4:443 CONNECTING -> TRANSIENT_FAILURE
D 2021-12-06T12:25:48.272Z | subchannel_refcount | (10) $CLOUD_RUN_IPV6_5::200a:443 refcount 1 -> 0
D 2021-12-06T12:25:48.272Z | subchannel_refcount | (11) $CLOUD_RUN_IP_ADD_5:443 refcount 1 -> 0
D 2021-12-06T12:25:48.272Z | subchannel | (11) $CLOUD_RUN_IP_ADD_5:443 CONNECTING -> TRANSIENT_FAILURE
D 2021-12-06T12:25:48.272Z | subchannel_refcount | (23) $CLOUD_RUN_IPV6_1::a:443 refcount 1 -> 0
D 2021-12-06T12:25:48.272Z | subchannel_refcount | (24) $CLOUD_RUN_IP_ADD_1:443 refcount 1 -> 0
D 2021-12-06T12:25:48.272Z | subchannel_refcount | (25) $CLOUD_RUN_IPV6_2::a:443 refcount 1 -> 0
D 2021-12-06T12:25:48.272Z | subchannel_refcount | (26) $CLOUD_RUN_IP_ADD_2:443 refcount 1 -> 0
D 2021-12-06T12:25:48.272Z | subchannel | (26) $CLOUD_RUN_IP_ADD_2:443 CONNECTING -> TRANSIENT_FAILURE
D 2021-12-06T12:25:48.273Z | subchannel_refcount | (27) $CLOUD_RUN_IPV6_3::a:443 refcount 1 -> 0
D 2021-12-06T12:25:48.273Z | subchannel_refcount | (28) $CLOUD_RUN_IP_ADD_3:443 refcount 1 -> 0
D 2021-12-06T12:25:48.273Z | subchannel | (28) $CLOUD_RUN_IP_ADD_3:443 CONNECTING -> TRANSIENT_FAILURE
D 2021-12-06T12:25:48.273Z | subchannel_refcount | (29) $CLOUD_RUN_IPV6_4::a:443 refcount 1 -> 0
D 2021-12-06T12:25:48.273Z | subchannel_refcount | (30) $CLOUD_RUN_IP_ADD_4:443 refcount 1 -> 0
D 2021-12-06T12:25:48.273Z | subchannel | (30) $CLOUD_RUN_IP_ADD_4:443 CONNECTING -> TRANSIENT_FAILURE
D 2021-12-06T12:25:48.273Z | subchannel_refcount | (31) $CLOUD_RUN_IPV6_3::35:443 refcount 1 -> 0
D 2021-12-06T12:25:48.273Z | subchannel_refcount | (32) $CLOUD_RUN_IP_ADD_5:443 refcount 1 -> 0
D 2021-12-06T12:25:48.274Z | subchannel | (32) $CLOUD_RUN_IP_ADD_5:443 CONNECTING -> TRANSIENT_FAILURE
D 2021-12-06T12:25:48.274Z | subchannel | (5) $CLOUD_RUN_IP_ADD_2:443 TRANSIENT_FAILURE -> IDLE
D 2021-12-06T12:25:48.274Z | subchannel | (7) $CLOUD_RUN_IP_ADD_3:443 TRANSIENT_FAILURE -> IDLE
D 2021-12-06T12:25:48.274Z | subchannel | (9) $CLOUD_RUN_IP_ADD_4:443 TRANSIENT_FAILURE -> IDLE
D 2021-12-06T12:25:48.274Z | subchannel | (11) $CLOUD_RUN_IP_ADD_5:443 TRANSIENT_FAILURE -> IDLE
D 2021-12-06T12:25:48.274Z | subchannel | (26) $CLOUD_RUN_IP_ADD_2:443 TRANSIENT_FAILURE -> IDLE
D 2021-12-06T12:25:48.275Z | subchannel | (28) $CLOUD_RUN_IP_ADD_3:443 TRANSIENT_FAILURE -> IDLE
D 2021-12-06T12:25:48.275Z | subchannel | (30) $CLOUD_RUN_IP_ADD_4:443 TRANSIENT_FAILURE -> IDLE
D 2021-12-06T12:25:48.275Z | subchannel | (32) $CLOUD_RUN_IP_ADD_5:443 TRANSIENT_FAILURE -> IDLE
Upvotes: 0
Views: 5188
Answers (1)
Sam Dean
Reputation: 404
From the grpc documentation all 3 arguments for credentials.createSsl() are optional. However,
If using a client-side certificate, both the second and third arguments must be passed.
So because I was using a client-side certificate and not passing the private_key or cert_chain I was having to disable 'NODE_TLS_REJECT_UNAUTHORIZED'.
So, the documentation indicates that it's fine to just use credentials.createSsl() without passing it a certificate. However the official code examples don't give it as an example use case.
My current working code is
import {loadPackageDefinition, credentials} from '@grpc/grpc-js';
import {loadSync} from '@grpc/proto-loader';
const packageDef = loadSync(
'./path/to/file.proto',
{keepCase: true,
longs: String,
enums: String,
defaults: true,
oneofs: true,
},
);
const packageNameProto = loadPackageDefinition(packageDef).packageName;
const ssl_creds = credentials.createSsl();
const client = new packageNameProto.Uppeak(
`${host}:${port}`,
ssl_creds,
);
Thanks to Pablo for pointing this out.
Old answer
We now have a solution to get this working but it's definitely not an ideal solution.
Instead of using grpc.credentials.createInsecure(), we're instead creating a self-signed SSL certificate by following this guide (steps outlined at the bottom).
Then using it as follows in the node.js code. Note that setting NODE_TLS_REJECT_UNAUTHORIZED to '0' will throw a warning.
process.env['NODE_TLS_REJECT_UNAUTHORIZED'] = '0'; // new
import {loadPackageDefinition, credentials} from '@grpc/grpc-js';
import {loadSync} from '@grpc/proto-loader';
import {readFileSync} from 'fs'; // new
const packageDef = loadSync(
'./path/to/file.proto',
{keepCase: true,
longs: String,
enums: String,
defaults: true,
oneofs: true,
},
);
const packageNameProto = loadPackageDefinition(packageDef).packageName;
const root_cert = readFileSync('./server.crt'); // new
const ssl_creds = credentials.createSsl(root_cert); // new
const client = new packageNameProto.Uppeak(
`${host}:${port}`,
ssl_creds,
); // changed
Creating a self signed certificate
- You need Apache and openssl on Linux.
- Generate a server key:
openssl genrsa -des3 -out server.key 4096
- Create a certificate signing request with it. This command will prompt for a series of things (country, state or province, etc.). Make sure that "Common Name (eg, YOUR name)" matches the registered fully qualified domain name of your box (or your IP address if you don't have one):
openssl req -new -key server.key -out server.csr
- Now sign the certificate signing request. This example lasts 365 days:
openssl x509 -req -days 365 -in server.csr -signkey server.key -out server.crt
Upvotes: 1
Related Questions
- VPC Network error when configuring App Engine for Google Tag Manager Server Side
- Problem with Listening gRPC Requests Over HTTP on Cloud Run
- Error: 14 UNAVAILABLE: read ECONNRESET in gRPC Client connection
- GCP Cloud Run : Application exec likely failed
- GRPC on Google Cloud Run : upstream connect error or disconnect/reset before headers. reset reason: remote reset
- Communication between Google App Engine and Google Cloud Run
- InactiveRpcError when trying to connect to Google Cloud Run rpc service
- Can not connect between Cloud Run and Compute engine using Internal IP
- Cannot hit grpc server after successfully deploying to cloud run
- Run a Python gRPC server on Google Cloud Run