Error: Failed to connect before the deadline.

[yashykt]

What version of gRPC and what language are you using?
gRPC version = 1.8.14, Language = javascript;

What operating system (Linux, Windows,...) and version?
Linux

What did you do?
Please provide either
I have been using gRPC for a long time. I was using it with an inscure connection. Then we decided to make the connection secure using mtls.
I tested it in my local system and I am able to connect it securely but when I moved it to the server it started facing error. I have no clue how to debug it further.
The error I keep getting is Error: Failed to connect before the deadline.

This is what I have added in the code for server

const credentials = grpc.ServerCredentials.createSsl(fs.readFileSync(String(process.env.GRPC_NOTIFICATION_CA_CERT)), [{
cert_chain: fs.readFileSync(String(process.env.GRPC_NOTIFICATION_SERVER_CERT)),
private_key: fs.readFileSync(String(process.env.GRPC_NOTIFICATION_SERVER_KEY))
}], true);

server.addService(package.service, {
youService
});

server.bindAsync("0.0.0.0:3006", credentials, (err) => {
if (err) {
console.error(err);
return;
}
console.log("GRPC server running at port 3006");
server.start();
});

client code

const credentialsClient1 = grpc.credentials.createSsl(
fs.readFileSync(String(process.env.GRPC_NOTIFICATION_CA_CERT)),
fs.readFileSync(String(process.env.GRPC_NOTIFICATION_CLIENT_KEY)),
fs.readFileSync(String(process.env.GRPC_NOTIFICATION_CLIENT_CERT))
);
const client2 = new package("notification-srv:3005", credentialsClient1);

client2?.waitForClientReady(Date.now() + 5000, (err) => {
if (err) {
console.error(err);
}
else {
console.info("GRPC client connected");
}
});
Another thing I am using Kubernetes for my application.

What did you expect to see?
It should have gotten connected since I am passing a valid certificate.

What did you see instead?
It is giving me error

Copied over from grpc/grpc#38820
@shubham-rapidops

[shubham-rapidops]

@yashykt I have some new updates. We are using a microservice architecture and have added Istio to our cluster, enforcing mTLS with Istio. However, when we enable mTLS in both Istio and gRPC, the gRPC server starts, but the client is unable to connect. On the other hand, when mTLS is disabled in Istio, the client can connect successfully. Is there an industry best practice that prevents using both together?

[murgatroid99]

First, that is a fairly old version of the library. The latest version is 1.12.6. I recommend upgrading because there are relevant improvements, including better error messages for connection failures. However, the waitForClientReady method is not a good way to get error details; you are better off making a request and seeing what the error is.

If the error message you get from doing that isn't sufficient, you can have Node output TLS debug information with the environment variable NODE_DEBUG=tls, and you can also turn on Node TLS tracing with the channel argument grpc-node.tls_enable_trace: 1.