Is X509 certificates authentication supported?

[liumiaojq]

I'd like to use this library to connect a MQTT broker running in Azure event grid. In their sample: https://github.com/Azure-Samples/MqttApplicationSamples, we need to setup X509 certificates on both broker and clients for authentication. However it's unclear if this type of authentication is supported or not in this library. In the sample app using this library, I see authentication is done with a user name/password instead of a X509 certificate.

[adam-fowler]

You can use X509 certificates. Here is some documentation related to using them with AWS https://swift-server-community.github.io/mqtt-nio/documentation/mqttnio/mqttnio-aws

[liumiaojq]

You can use X509 certificates. Here is some documentation related to using them with AWS https://swift-server-community.github.io/mqtt-nio/documentation/mqttnio/mqttnio-aws

thanks for sharing the document. Is there any sample codes using X.509 certificate for authentication? The sample codes in the document only uses AWS Signature V4. I'd like to see how I can setup the cert and key file with the MQTT client.

[liumiaojq]

Found this document: https://swift-server-community.github.io/mqtt-nio/documentation/mqttnio/mqttnio-connections. will have a try to see if that works with Azure event grid MQTT broker

[liumiaojq]

I'm setting up the client with these codes and I get an exception when loading the client certificate:

        let trustRoortCertPath = Bundle.main.path(forResource: "intermediate_ca", ofType: "der")
        let clientCertPath = Bundle.main.path(forResource: "sample_client3", ofType: "p12")
        let trustRootCert = try TSTLSConfiguration.Certificates.der(trustRoortCertPath!)
        let clientIdentity = try TSTLSConfiguration.Identity.p12(filename: clientCertPath!, password: "") 

The exception is thrown at this line within TSTLSConfiguration.Identity.p12:
guard SecPKCS12Import(data as CFData, options as CFDictionary, &rawItems) == errSecSuccess else { throw TSTLSConfiguration.Error.invalidData }

I generated p12 file without a password, so I pass empty string. Also tried to generate the p12 file with a password and still get the same error. I created the p12 file on Windows wsl with this command:
step certificate p12 sample_client3.p12 sample_client3.pem sample_client3.key --password-file=password.txt

sample_client3.pem sample_client3.key are generated using a self-signed root ca:
step certificate create
sample_client3 sample_client3.pem sample_client3.key
--ca ~/.step/certs/intermediate_ca.crt
--ca-key ~/.step/secrets/intermediate_ca_key
--no-password --insecure
--not-after 2400h

Create self signed ca:
step ca init
--deployment-type standalone
--name MqttAppSamplesCA
--dns localhost
--address 127.0.0.1:443
--provisioner MqttAppSamplesCAProvisioner

Did some debugging, SecPKCS12Import returns -26275, which means "Unable to decode the provided data" (error message provided by SecCopyErrorMessageString)

[adam-fowler]

There is a script in the project for generating certificates scripts/generate-certs.sh for the tests. It uses OpenSSL. Maybe you could do use something similar.

[liumiaojq]

There is a script in the project for generating certificates scripts/generate-certs.sh for the tests. It uses OpenSSL. Maybe you could do use something similar.

thanks for the pointer. I tried to use that script to generate all certificates, but I get this error when connecting to my MQTT broker on iOS client:

Trust failed: “DigiCert Global Root G3” certificate is not trusted
boringssl_context_handle_fatal_alert(2170) [C1.1.1:1][0x106812680] write alert, level: fatal, description: certificate unknown
boringssl_context_error_print(2160) [C1.1.1:1][0x106812680] Error: 4405186896:error:1000007d:SSL routines:OPENSSL_internal:CERTIFICATE_VERIFY_FAILED:/Library/Caches/com.apple.xbs/Sources/boringssl_Sim/ssl/handshake.cc:399:
boringssl_session_handshake_incomplete(241) [C1.1.1:1][0x106812680] SSL library error
boringssl_session_handshake_error_print(44) [C1.1.1:1][0x106812680] Error: 4405186896:error:1000007d:SSL routines:OPENSSL_internal:CERTIFICATE_VERIFY_FAILED:/Library/Caches/com.apple.xbs/Sources/boringssl_Sim/ssl/handshake.cc:399:
nw_protocol_boringssl_handshake_negotiate_proceed(780) [C1.1.1:1][0x106812680] handshake failed at state 12288: not completed
nw_endpoint_flow_failed_with_error [C1.1.1 40.64.128.45:8883 in_progress socket-flow (satisfied (Path is satisfied), interface: en0[802.11], uses wifi)] already failing, returning
nw_endpoint_flow_failed_with_error [C1.1.1 40.64.128.45:8883 cancelled socket-flow ((null))] already failing, returning
warning: EmCuteetee was compiled with optimization - stepping may behave oddly; variables may not be available.

Looks like I need to install/trust the self signed ca on iOS device. However even I manually installed ca cert and enable trust on iOS device, I still get this error. My ca root cert doesn't have "DigiCert Global Root G3" in CN.

openssl x509 -in ./../mosquitto/certs/ca.pem -text -noout
Certificate:
Data:
Version: 3 (0x2)
Serial Number:
72:b0:12:d2:43:b2:50:9c:ce:10:e3:ec:8b:09:94:26:ea:10:79:70
Signature Algorithm: sha256WithRSAEncryption
Issuer: C=UK, ST=Edinburgh, L=Edinburgh, O=MQTTNIO, OU=CA, CN=qiangj-mqtt-poc-namespace.westus2-1.ts.eventgrid.azure.net
Validity
Not Before: Mar 5 21:56:30 2025 GMT
Not After : Mar 4 21:56:30 2030 GMT
Subject: C=UK, ST=Edinburgh, L=Edinburgh, O=MQTTNIO, OU=CA, CN=qiangj-mqtt-poc-namespace.westus2-1.ts.eventgrid.azure.net
Subject Public Key Info:
Public Key Algorithm: rsaEncryption
RSA Public-Key: (2048 bit)
Modulus:
00:be:5f:bb:91:ca:7b:1f:05:ef:a6:38:47:91:bf:
a0:ee:49:e7:2e:8e:36:87:b7:50:07:e3:d6:fd:1f:
5f:8f:fe:85:b3:91:b5:60:64:e1:fd:ea:8c:41:9d:
3e:ec:69:5a:19:31:57:40:8a:d6:92:e4:0d:cb:c0:
78:a2:e7:b0:7a:e1:18:dc:ec:8f:fd:b8:6b:1f:7f:
46:6c:48:55:f8:77:11:98:39:b7:a5:ac:17:84:62:
a4:90:e1:38:6d:b3:2c:1b:c8:a5:52:0c:a3:8b:62:
9f:cf:f3:6d:76:50:27:a3:dd:e5:aa:2c:6a:a7:cf:
3f:2a:7d:b8:83:70:fa:2a:17:d4:d7:28:09:e7:31:
ad:29:51:dd:6b:59:07:87:8a:15:2f:9c:58:40:6f:
ca:99:cf:12:b5:b2:0c:11:c8:c8:32:6c:13:d4:bd:
30:5b:ed:e2:9b:af:16:e9:50:29:fa:89:fc:ab:a1:
d3:87:1b:c7:f0:86:3d:f8:d6:e1:e8:98:57:9f:ad:
57:2a:a4:4f:aa:f4:28:79:91:84:0a:7a:0a:09:88:
7c:75:1a:be:51:de:ed:6a:b4:3d:87:94:60:5c:66:
73:40:d5:dd:16:64:26:b1:3a:ab:25:3a:1e:c7:60:
32:c0:78:22:9f:ab:c9:aa:dd:d5:94:ee:86:73:2d:
36:d1
Exponent: 65537 (0x10001)
X509v3 extensions:
X509v3 Subject Key Identifier:
11:10:AA:2E:B0:5C:FE:2F:5B:21:88:5D:AC:C5:EC:EF:50:82:A7:0B
X509v3 Authority Key Identifier:
keyid:11:10:AA:2E:B0:5C:FE:2F:5B:21:88:5D:AC:C5:EC:EF:50:82:A7:0B

        X509v3 Basic Constraints: critical
            CA:TRUE
Signature Algorithm: sha256WithRSAEncryption
     b8:66:92:34:f9:68:f2:20:80:9f:24:9d:4e:7a:78:e5:b1:60:
     20:04:eb:a5:ed:e7:23:16:13:37:ef:0d:08:9e:ac:4c:3b:c1:
     3e:3b:6f:ad:32:ee:26:80:6e:a8:a6:d8:d7:ad:5c:1a:5b:1d:
     e2:b7:a1:ee:0d:c1:81:12:ba:26:41:22:67:57:3d:c1:51:c6:
     66:60:f7:fe:46:67:dd:0a:3b:3d:7e:9d:2f:6f:58:70:e4:4e:
     52:49:03:20:5f:38:e6:bd:15:40:e9:34:1c:e5:d2:2b:81:af:
     bf:5d:6c:14:09:d7:47:55:ba:5b:2d:79:a0:6a:f9:0c:19:07:
     ee:1d:28:2d:d2:68:ff:38:85:f4:93:28:40:d9:8f:4d:95:8f:
     fc:11:87:57:0a:1f:b3:f9:07:03:3c:b7:d9:12:9a:da:f5:8b:
     e2:72:ed:73:11:aa:fc:af:93:aa:a2:17:1a:69:15:85:e9:13:
     e7:d0:55:36:dc:7b:a0:14:56:9c:7b:8b:4c:1b:f4:16:5e:bf:
     02:ea:ef:2e:8c:95:ef:ce:b0:c0:e5:51:2a:e9:3d:f0:10:c8:
     31:90:a4:8b:cb:a6:f5:0b:03:3d:e3:63:d8:37:06:e6:77:31:
     63:73:ed:36:61:8d:cf:57:99:ba:60:8a:eb:72:eb:a0:a7:6c:
     d6:b3:d7:04

I manually download/installed that CA from https://www.digicert.com/kb/digicert-root-certificates.htm on iOS device, but still getting same error.

client cert:
openssl x509 -in ./../mosquitto/certs/sample_client6.pem -text -noout
Certificate:
Data:
Version: 3 (0x2)
Serial Number:
6e:ea:49:56:b7:a4:36:f4:84:89:b4:f2:1a:ec:16:30:f3:35:11:88
Signature Algorithm: sha256WithRSAEncryption
Issuer: C=UK, ST=Edinburgh, L=Edinburgh, O=MQTTNIO, OU=CA, CN=qiangj-mqtt-poc-namespace.westus2-1.ts.eventgrid.azure.net
Validity
Not Before: Mar 5 21:56:31 2025 GMT
Not After : Mar 4 21:56:31 2030 GMT
Subject: C=UK, ST=Edinburgh, L=Edinburgh, O=MQTTNIO, OU=Client, CN=sample_client6
Subject Public Key Info:
Public Key Algorithm: rsaEncryption
RSA Public-Key: (2048 bit)
Modulus:
00:b3:aa:3d:14:f2:43:08:f6:3b:f0:42:19:53:e9:
4a:ec:05:77:79:8b:ca:0f:dd:aa:c7:09:01:38:21:
f5:be:77:c5:cd:27:bf:fe:95:d7:16:71:33:ff:84:
c0:8e:b6:c5:9c:9c:dd:77:44:ec:ed:4a:9d:74:b7:
45:8c:4e:64:be:31:8f:40:87:25:0e:39:7a:ee:93:
25:52:cd:fd:62:f9:be:cd:f7:55:a7:a9:af:b6:a4:
ac:b6:47:ef:31:ee:07:92:e3:cf:09:62:a4:3f:51:
07:93:5e:56:8a:1d:1b:e8:a4:95:a8:a9:9e:6b:8c:
9b:32:75:0f:4e:3f:ec:02:39:42:c2:a0:46:9d:24:
08:7f:14:1c:d4:5d:3a:b2:48:bf:98:ae:72:91:e6:
b5:6d:5c:a3:2f:3a:0e:1c:d4:7e:9e:ec:0f:ae:3e:
81:a5:a8:51:a5:53:59:02:08:53:4b:eb:1f:f7:77:
3a:7d:34:34:2b:21:19:8f:c3:f2:bf:42:bc:5c:a5:
f5:00:56:39:cc:d6:8d:66:ff:ad:65:51:30:c2:30:
8f:a1:3d:d8:c8:5c:91:cf:1c:52:30:d1:8d:08:06:
f1:92:11:2c:46:a1:fc:72:91:82:85:5e:05:76:46:
ae:53:61:f4:40:42:ec:bb:da:ea:37:2c:b3:0b:6a:
b5:71
Exponent: 65537 (0x10001)
X509v3 extensions:
X509v3 Subject Key Identifier:
F8:C1:6A:6D:04:6F:2C:40:F0:AE:51:7A:4A:1B:9F:B0:39:64:F6:B7
X509v3 Authority Key Identifier:
keyid:11:10:AA:2E:B0:5C:FE:2F:5B:21:88:5D:AC:C5:EC:EF:50:82:A7:0B

Signature Algorithm: sha256WithRSAEncryption
     46:3d:c0:d3:1f:c5:85:36:fe:b0:97:78:53:53:10:2c:b4:c2:
     34:49:54:7d:aa:fd:ae:ba:d3:1a:e5:ff:3a:ce:a3:dc:ad:5a:
     98:9c:e1:9a:c8:03:71:43:68:bf:95:fa:a9:8c:b3:d0:2d:81:
     4d:16:4d:4e:3a:6f:18:ab:5d:de:df:b4:24:e0:82:d2:65:61:
     76:54:81:46:f1:43:7b:47:13:e6:a5:45:c6:f8:b4:5a:d1:69:
     52:76:dd:cd:d2:9a:9d:e5:de:7e:95:67:13:93:a5:f6:cf:ba:
     f0:1b:02:c6:88:31:18:f6:49:9e:3c:9e:28:01:36:d7:04:fe:
     8f:f9:7b:d6:bc:79:b5:b9:a5:bf:5e:ce:32:16:d4:03:85:c0:
     88:41:8e:8b:47:ea:1f:57:07:d1:03:91:92:b8:94:8b:1f:2d:
     5d:de:e0:8a:e6:bc:71:c4:26:43:1a:d0:75:7c:a0:6e:6e:77:
     de:91:26:1e:86:ec:c6:4e:04:36:c8:98:4f:ff:75:64:fa:f1:
     8c:fc:75:5a:ec:9a:2e:91:61:0f:40:e1:d5:01:49:80:4e:60:
     2f:b7:40:ed:a3:48:7a:36:26:4c:95:1c:1c:1b:95:d6:1e:e8:
     09:5a:41:20:0c:b8:f3:ef:a5:af:dc:1e:7c:43:db:6f:71:a0:
     d5:6b:20:87

[liumiaojq]

I fixed a cert import issue on iOS 17 and lower: https://github.com/liumiaojq/mqtt-nio/pull/1/files.
In that PR I also made some changes to use client name as subject name in client cert, which is required by Azure Event Grid MQTT broker: https://github.com/Azure-Samples/MqttApplicationSamples/tree/main/scenarios/getting_started#lock-create-the-client-certificate

With all these changes, I'm still running into the same error even I manually installed and trusted this cert: “DigiCert Global Root G3”.

Trust failed: “DigiCert Global Root G3” certificate is not trusted
boringssl_context_handle_fatal_alert(2072) [C1.1.1:1][0x103e54b50] write alert, level: fatal, description: certificate unknown
boringssl_context_error_print(2062) [C1.1.1:1][0x103e54b50] Error: 4389557808:error:1000007d:SSL routines:OPENSSL_internal:CERTIFICATE_VERIFY_FAILED:/Library/Caches/com.apple.xbs/Sources/boringssl_Sim/ssl/handshake.cc:419:
boringssl_session_handshake_incomplete(210) [C1.1.1:1][0x103e54b50] SSL library error
boringssl_session_handshake_error_print(44) [C1.1.1:1][0x103e54b50] Error: 4389557808:error:1000007d:SSL routines:OPENSSL_internal:CERTIFICATE_VERIFY_FAILED:/Library/Caches/com.apple.xbs/Sources/boringssl_Sim/ssl/handshake.cc:419:
nw_protocol_boringssl_handshake_negotiate_proceed(779) [C1.1.1:1][0x103e54b50] handshake failed at state 12288: not completed
Snapshotting a view (0x103f6c3f0, _UIButtonBarStackView) that is not in a visible window requires afterScreenUpdates:YES.
-[RTIInputSystemClient remoteTextInputSessionWithID:performInputOperation:] perform input operation requires a valid sessionID. inputModality = Keyboard, inputOperation = dismissAutoFillPanel, customInfoType = UIUserInteractionRemoteInputOperations
Snapshotting a view (0x12300ce00, UIKeyboardImpl) that is not in a visible window requires afterScreenUpdates:YES.

[adam-fowler]

I'm not really that experienced in the certificate generation and have never used step. Looking at what's needed to setup the CA and then the client it looks slightly different from my script. You need a CA and an intermediary CA, then create your client key from the intermediary CA. The script doesn't create an intermediary key.

One thing I do know is Network.framework is particularly fussy about certificates. Certificates that worked using NIOSSL failed when using Network.framework. I had to setup a openssl.cnf with additional setup for the keys. I'm not sure if this is required for you

[liumiaojq]

Yes you are correct that instructions are slightly different. I'm using the CA generated with your script (with my changes) as the intermediate CA and upload it to the server. I think that's equivalent?

[liumiaojq]

I'm trying to generate the root and intermediate CA using step certificate for broker, and use the intermediate CA to sign client certificate generated by openssl to work around the cert import issue on iOS. However I'm still running into the same error. I summarized all my changes and process in this PR: liumiaojq/EmCuTeeTee#1

[liumiaojq]

Attach cert dump for new intermediate and client certs:
openssl x509 -in intermediate_ca.crt -text -noout
Certificate:
Data:
Version: 3 (0x2)
Serial Number:
38:58:23:7e:54:f5:1e:af:fe:ae:89:f8:61:2f:13:86
Signature Algorithm: ecdsa-with-SHA256
Issuer: O=MqttAppSamplesCA, CN=MqttAppSamplesCA Root CA
Validity
Not Before: Mar 7 23:09:23 2025 GMT
Not After : Mar 5 23:09:23 2035 GMT
Subject: O=MqttAppSamplesCA, CN=MqttAppSamplesCA Intermediate CA
Subject Public Key Info:
Public Key Algorithm: id-ecPublicKey
Public-Key: (256 bit)
pub:
04:85:ca:69:b6:e6:e5:2c:c0:62:34:7e:73:c3:ea:
76:d1:a4:bf:fd:93:6a:22:97:e8:b0:93:16:02:d6:
6f:2b:6d:a3:0e:78:7d:86:f9:60:0d:e7:3d:c3:ca:
b1:c9:d6:95:cb:ff:bc:51:29:b6:b2:8f:92:df:b4:
bd:17:a2:cf:5f
ASN1 OID: prime256v1
NIST CURVE: P-256
X509v3 extensions:
X509v3 Key Usage: critical
Certificate Sign, CRL Sign
X509v3 Basic Constraints: critical
CA:TRUE, pathlen:0
X509v3 Subject Key Identifier:
E3:4B:F5:74:EC:8A:D5:47:06:69:68:67:3D:E3:91:87:87:BB:12:B7
X509v3 Authority Key Identifier:
46:77:CD:6D:04:C8:06:6A:0A:5F:3D:A7:58:7C:FC:0A:E9:20:3F:A4
Signature Algorithm: ecdsa-with-SHA256
Signature Value:
30:44:02:20:71:f9:a0:20:0e:88:0e:fd:0c:fe:a4:9a:ec:30:
a4:36:e1:ee:26:b0:d8:6d:fb:aa:6e:7b:b8:13:e2:43:3c:ae:
02:20:06:5d:51:76:07:49:35:97:41:53:b5:27:30:df:ef:3c:
f9:85:7d:71:14:ce:7e:37:04:8a:88:36:d5:14:e0:58

openssl x509 -in sample_client10.pem -text -noout
Certificate:
Data:
Version: 3 (0x2)
Serial Number:
1a:58:b3:46:95:83:71:ad:dc:88:e6:34:d0:ce:fc:30:45:9f:41:03
Signature Algorithm: ecdsa-with-SHA256
Issuer: O=MqttAppSamplesCA, CN=MqttAppSamplesCA Intermediate CA
Validity
Not Before: Mar 7 23:10:54 2025 GMT
Not After : Mar 6 23:10:54 2030 GMT
Subject: C=UK, ST=Edinburgh, L=Edinburgh, O=MQTTNIO, OU=Client, CN=sample_client10
Subject Public Key Info:
Public Key Algorithm: rsaEncryption
Public-Key: (2048 bit)
Modulus:
00:ac:ac:7f:0a:fb:44:e7:59:9a:82:29:66:e3:20:
ee:1f:e3:ab:54:fe:9b:c2:38:b6:41:d6:5c:d9:76:
be:15:57:0a:41:74:d5:65:8d:08:62:6b:a8:17:c5:
75:9a:69:c4:18:ed:f2:bf:eb:3e:ef:be:28:09:62:
ef:cd:51:88:0e:16:9b:5b:b9:11:f2:d2:08:c3:6c:
8b:d7:2a:5f:04:87:47:fb:5a:bb:10:8d:e6:d2:59:
c0:03:9f:10:88:99:d9:8b:2b:e7:e4:09:d6:d3:f4:
c6:42:fc:79:ab:2b:cd:23:4a:89:9d:61:ea:ff:82:
8a:be:c7:73:e3:cc:e2:4c:6a:a7:6b:0a:72:72:72:
82:81:a8:eb:b0:53:d7:8c:78:fe:55:de:76:c2:51:
20:73:f9:41:07:7a:e9:85:7b:35:8c:01:23:59:c5:
e9:de:4f:29:e8:89:2f:c1:19:59:2b:c6:6a:6e:d1:
db:cd:35:e5:ed:4f:4e:49:e6:91:3e:d4:f1:28:9a:
cc:5b:cc:2e:b0:70:b9:03:85:5a:1d:e3:1b:57:67:
a6:cd:22:8b:6a:d4:27:f3:65:cf:11:ef:fc:3c:2a:
f0:a9:32:46:8b:0e:c2:95:a1:83:63:9b:ac:3a:4e:
a4:fb:60:6d:1a:eb:55:14:77:6d:a6:0f:c1:bd:84:
fe:49
Exponent: 65537 (0x10001)
X509v3 extensions:
X509v3 Key Usage:
Digital Signature, Key Encipherment
X509v3 Extended Key Usage:
TLS Web Server Authentication, TLS Web Client Authentication
X509v3 Subject Alternative Name:
DNS:sample_client10
X509v3 Subject Key Identifier:
19:99:86:7F:1A:5B:99:0E:44:48:68:94:A1:76:8E:9D:03:F6:FE:BB
X509v3 Authority Key Identifier:
E3:4B:F5:74:EC:8A:D5:47:06:69:68:67:3D:E3:91:87:87:BB:12:B7
Signature Algorithm: ecdsa-with-SHA256
Signature Value:
30:45:02:21:00:f0:71:71:25:64:e4:41:29:30:1f:9c:7d:b5:
0f:c4:0d:99:bd:c9:b9:31:d7:bb:47:d8:34:69:ac:ca:ff:5f:
d6:02:20:69:ae:91:e4:90:4e:4a:3a:e8:02:20:51:bf:e2:c0:
9c:0d:82:29:b9:e4:8c:94:c6:e4:bd:ea:a5:92:4d:fb:b9

[liumiaojq]

I tried to run sample app on a real iPhone and the certificate validation errors are gone in debug logs. I get this error in sample app: liumiaojq/EmCuTeeTee#1

Failed to connect
connectTimeout(NIOCore.TimeAmount(nanoseconds: 10000000000))

I did some debugging, the most inner exception is thrown here:

@adam-fowler Can you provide some tips how to debug this?

[adam-fowler]

A timeout implies it can't see your server. Is there any reason this would be the case?

[liumiaojq]

@adam-fowler For some reason I started to see this error on real iPhone as well:
Trust failed: “DigiCert Global Root G3” certificate is not trusted.

After doing some debugging, this error is printed here, after calling SecTrustEvaluateAsyncWithError:

However if you see my steps to generate/sign the client cert (in liumiaojq/EmCuTeeTee#1), I didn't use that root CA at all so it's quite confusing why that API complains about it. I'm going to post this question to apple developer forum and please help share your insight as well.