General help, S/MIME not working as expected

[roostercactus]

Hi, how are you? Great project...thanks so much for this.

I have a couple of questions. I don't know if it's just something I'm doing wrong but I can't get it to work.

My environment:

Office365 business account (I use https://outlook.office365.com/mail to access my webmail)
Linux distro - Devuan daedalus (6.1.0-26-amd64), it's basically Debian Bookworm 12.7
Chromium - Version 130.0.6723.91 (Official Build) built on Debian GNU/Linux 12 (bookworm) (64-bit)
Python - Python 3.11.2
Openssl - OpenSSL 3.0.14 4 Jun 2024 (Library: OpenSSL 3.0.14 4 Jun 2024)

I copied the json file here, ~/.config/chromium/NativeMessagingHosts/com.microsoft.outlook.smime.chromenativeapp.json
I copied the python file to /usr/bin/owa-smime.py (had to update the shebang to #!/usr/bin/python3...)
Downloaded and installed the extension from here - https://res-1.cdn.office.net/owasmime/20.21.531.1/Microsoft.Outlook.Smime.crx

Here's my config.json

{
"key-id": null,
"key-id::DESCRIPTION": "If using a smart card put your key id here, you can list you key ids using e.g. "pkcs15-tool --list-keys"",
"private-key": "/home/xxx/.config/owa-smime4linux/cert.pem",
"private-key::DESCRIPTION": "If using a private key from a file put the location of the pem formated file here.",
"cert-chain": "/home/xxx/.config/owa-smime4linux/chain.pem",
"cert-chain::DESCRIPTION": "(optional) Put the location of your pem formated certificate chain here."
}

Here's my cert and chain.pem (obtained using the openssl commands in the README)

cert.pem

Bag Attributes
localKeyID: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
subject=CN = "Last, First", O = Work Name, organizationIdentifier = "string"
issuer=C = US, O = "DigiCert, Inc.", CN = DigiCert Assured G2 mPKI SMIME RSA4096 SHA384 2023 CA1
-----BEGIN CERTIFICATE-----

-----END CERTIFICATE-----
Bag Attributes:
subject=C = US, O = "DigiCert, Inc.", CN = DigiCert Assured G2 mPKI SMIME RSA4096 SHA384 2023 CA1
issuer=C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert Assured ID Root G2
-----BEGIN CERTIFICATE-----

-----END CERTIFICATE-----
Bag Attributes
localKeyID: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Key Attributes:
-----BEGIN PRIVATE KEY-----

-----END PRIVATE KEY-----

chain.pem

Bag Attributes:
subject=C = US, O = "DigiCert, Inc.", CN = DigiCert Assured G2 mPKI SMIME RSA4096 SHA384 2023 CA1
issuer=C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert Assured ID Root G2
-----BEGIN CERTIFICATE-----

-----END CERTIFICATE-----

When I login to Office365 (https://outlook.office365.com/mail) and click on a S/MIME signed email (my company only signs...no encryption) it says the email has a digital signature but is unable to verify it because it's not supported on my browser or platform.

This is with my default User Agent (Chromium - Linux). There are no options to sign an email and in the Outlook settings under Mail there are no options for S/MIME.

When I change the User Agent to 'Chrome - Windows' as outlined in the README and click on an email it now says the S/MIME control is out of date and wants me to update. It also now says the digital signature is valid but not matching the sender of this message.

If I click on the 'install latest version' button it downloads a file named, 'SmimeOutlookWebChrome.msi'. Which I can't install since this is for Windows.

Now if I check the Outlook settings under Mail it does show the S/MIME extension controls. It also says it's out of date and to update, which again downloads the same SmimeOutlookWebChrome.msi file. I have it set to sign only (no encryption) and to automatically choose the best cert for signing. Again, these settings only appear when the User Agent is set to Windows.

If I try to send an email it tries to sign it (since I have the automatically sign option enabled) but fails and will not send the email.

In the native.log I can see this error, "No signing cert and no encryption cert given, aborting!".

I can upload more data as needed.

Any ideas? Thanks!

[schorschii]

Hi,

Unfortunately, I don't have an Office365 business account to test it. This project was tested with an Exchange 2019 On-Prem installation. I don't know if they changed something in the native messaging communication flow.

Why does your cert.pem contain 2 certificates? It should only contain your personal certificate, everything else goes into chain.pem. Also, try to reverse the order: put your private key first followed by your certificate.

You can also try replacing the advertised version (SMIME_CONTROL_VERSION = '4.0700.19.19.814.1') in the script with the latest version you can download as MSI to get rid of the update message.

Let me know the results.

[roostercactus]

Hi, thanks for the reply.

I was able to edit the SMIME_CONTROL_VERSION to be the latest version (4.0800.20.21.804.1). This does fix the problem with it thinking I need to update the extension. So that is solved.

I also removed the intermediate cert from cert.pem so only my personal cert and key are in the file. I also moved the key to the top of the file.

The chain.pem now contains the root and intermediate certs.

It is behaving the same though. When I set my user agent to Windows and click on an email it says the "digital signature is valid but not matching the sender of this message".

When I try to send an email it will try, but fails with an S/MIME encoding error (an unexpected S/MIME error occurred).

In the native.log I still see the "No signing cert and no encryption cert given, aborting!" error.

Along with the native.log file there is a signer.pem file which has my personal cert in it.

There's also a message-out.txt file that looks like this (which has the body of the email I was trying to send (just the number 44):

MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="smime-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"

--smime-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Content-Type: text/plain; charset=utf-8

44

--smime-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx--

Here's what native.log looks like when I try to send an email to myself, I've edited some strings to not contain personal data...

<< {"data": {"__type": "AcknowledgePartialSmimeRequestArrived:#Microsoft.Exchange.Clients.BrowserExtension.Smime", "PartIndex": -1, "StartOffset": -1, "NextStartOffset": -1, "Status": 1}, "messageType": "UploadPartialRequest", "portId": 2105334111, "requestId": 19}

{'__type': 'ExtensionMessage:#Microsoft.Exchange.Clients.BrowserExtension.Smime', 'messageType': 'DownloadPartialResult', 'requestId': 19, 'data': {'__type': 'FetchPartialSmimeResult:#Microsoft.Exchange.Clients.BrowserExtension.Smime', 'PartIndex': 0, 'StartOffset': 0, 'MaxPartSize': 262144}, 'portId': 2105334111}

<< {"data": {"__type": "ReturnPartialSmimeResult:#Microsoft.Exchange.Clients.BrowserExtension.Smime", "PartIndex": 0, "StartOffset": 0, "EndOffset": 100000, "IsLastPart": true, "PartialData": "{"Data": "MIME-Version: 1.0\nFrom: \"Last, First\" <my.email.address>\nTo: \"Last, First\" <my.email.address>\nSubject: 44\nImportance: Normal\nSensitivity: Normal\nDate: Mon, 11 Nov 2024 21:37:37 \nX-Generated-By: OWA-SMIME4Linux 4.0800.20.21.804.1\nContent-Type: application/x-pkcs7-mime; name=\"smime.p7m\"; smime-type=signed-data\nContent-Transfer-Encoding: base64\nContent-Disposition: attachment; filename=\"smime.p7m\"\n\nLOnGSTrinG/w

{'__type': 'ExtensionMessage:#Microsoft.Exchange.Clients.BrowserExtension.Smime', 'messageType': 'UploadPartialRequest', 'requestId': 20, 'data': {'__type': 'PostPartialSmimeRequest:#Microsoft.Exchange.Clients.BrowserExtension.Smime', 'PartIndex': 0, 'StartOffset': 0, 'EndOffset': 1472, 'IsLastPart': True, 'PartialData': '{"__type":"CreateSmimeFromMessageParams:#Microsoft.Exchange.Clients.BrowserExtension.Smime","SmimeType":11,"EmailMessage":"{\"__type\":\"Message:#Exchange\",\"Subject\":\"44\",\"Importance\":\"Normal\",\"IsReadReceiptRequested\":false,\"IsDeliveryReceiptRequested\":false,\"ToRecipients\":[{\"EmailAddress\":\"my.email.address\",\"Name\":\"Last, First\",\"MailboxType\":\"InternalMailbox\",\"RoutingType\":\"SMTP\"}],\"CcRecipients\":[],\"BccRecipients\":[],\"Sensitivity\":\"Normal\",\"Body\":{\"__type\":\"BodyContentType:#Exchange\",\"BodyType\":\"Text\",\"Value\":\"44\",\"DataUriCount\":0},\"Attachments\":[],\"ExtendedProperty\":[{\"type\":\"ExtendedPropertyType:#Exchange\",\"Value\":\"newMail\",\"ExtendedFieldURI\":{\"', 'TotalSize': 1472, 'RequestType': 'CreateSmimeFromMessage'}, 'portId': 2105334111}

No signing cert and no encryption cert given, aborting!

<< {"data": {"__type": "AcknowledgePartialSmimeRequestArrived:#Microsoft.Exchange.Clients.BrowserExtension.Smime", "PartIndex": -1, "StartOffset": -1, "NextStartOffset": -1, "Status": 1}, "messageType": "UploadPartialRequest", "portId": 2105334111, "requestId": 20}

{'__type': 'ExtensionMessage:#Microsoft.Exchange.Clients.BrowserExtension.Smime', 'messageType': 'DownloadPartialResult', 'requestId': 20, 'data': {'__type': 'FetchPartialSmimeResult:#Microsoft.Exchange.Clients.BrowserExtension.Smime', 'PartIndex': 0, 'StartOffset': 0, 'MaxPartSize': 262144}, 'portId': 2105334111}

<< {"data": {"__type": "ReturnPartialSmimeResult:#Microsoft.Exchange.Clients.BrowserExtension.Smime", "PartIndex": 0, "StartOffset": 0, "EndOffset": 100000, "IsLastPart": true, "PartialData": "{"Data": "MIME-Version: 1.0\nFrom: \"Last, First\" <my.email.address>\nTo: \"Last, First\" <my.email.address>\nSubject: 44\nImportance: Normal\nSensitivity: Normal\nDate: Mon, 11 Nov 2024 21:37:37 \nX-Generated-By: OWA-SMIME4Linux 4.0800.20.21.804.1\nContent-Type: application/x-pkcs7-mime; name=\"smime.p7m\"; smime-type=signed-data\nContAlso I wish I knew python to try and understand what the script is doing exactly so I could troubleshoot more on my end. But I'm not much of a programmer. ent-Transfer-Encoding: base64\nContent-Disposition: attachment; filename=\"smime.p7m\"\n\nLOnGSTrinG/w

I searched my drive for a file named smime.p7m but there is not one. It may only get created if the email is able to successfully send.

Thanks so much for looking. I know you don't have an Office365 account to test this on, so I understand if there's not much you can do. It could be something on the Office365 side and nothing to do with your script.

Thanks again!

[schorschii]

It looks like Microsoft changed the way in O365 how the signing cert is determined. Currently, available signing certs will be sent to OWA and then OWA returns the signing cert which should be used along the message to sign. In your logs, OWA does not supply the signing cert to use, which is why you get "No signing cert and no encryption cert given, aborting!".

It's really hard to debug this without an O365 test account, but we can try to use our own cert for signing even if it is missing in the OWA response. Can you please test the version in the o365 branch? 50de8f2

[roostercactus]

Hi, thanks so much for looking at this. If you don't have time to debug this, I completely understand. I know it will be hard without an Office365 account. Here's my update though.

I tried the updated script, but it's still the same result. Here's the native log for when I try to send myself an email.

{'__type': 'ExtensionMessage:#Microsoft.Exchange.Clients.BrowserExtension.Smime', 'messageType': 'GetSettings', 'requestId': -1, 'data': {}}

<< {"AllowedDomainsByPolicy": ["outlook.office365.com"]}

[EXIT]

{'__type': 'ExtensionMessage:#Microsoft.Exchange.Clients.BrowserExtension.Smime', 'messageType': 'UploadPartialRequest', 'requestId': 2, 'data': {'__type': 'PostPartialSmimeRequest:#Microsoft.Exchange.Clients.BrowserExtension.Smime', 'PartIndex': 0, 'StartOffset': 0, 'EndOffset': 2362, 'IsLastPart': True, 'PartialData': '{"__type":"InitializeParams:#Microsoft.Exchange.Clients.BrowserExtension.Smime","Settings":"{\"CodePage\":65001,\"Culture\":\"en-US\",\"AttachmentSizeLimit\":104857600,\"UseKeyIdentifier\":false,\"CertificateSubjects\":[\"my.email.address\",\"my.email.address\",\"my.email.address\",\"/o=ExchangeLabs/ou=Exchange Administrative Group (FY23I43039403)/cn=Recipients/cn=String-Last, First\",\"my.email.address\",\"SPO_UUID\",\"9999999;phone-context=UUID\",\"my.email.address;phone-context=UUID', 'TotalSize': 2362, 'RequestType': 'Initialize'}, 'portId': 850678111}

<< {"data": {"__type": "AcknowledgePartialSmimeRequestArrived:#Microsoft.Exchange.Clients.BrowserExtension.Smime", "PartIndex": -1, "StartOffset": -1, "NextStartOffset": -1, "Status": 1}, "messageType": "UploadPartialRequest", "portId": 850678111, "requestId": 2}

{'__type': 'ExtensionMessage:#Microsoft.Exchange.Clients.BrowserExtension.Smime', 'messageType': 'DownloadPartialResult', 'requestId': 2, 'data': {'__type': 'FetchPartialSmimeResult:#Microsoft.Exchange.Clients.BrowserExtension.Smime', 'PartIndex': 0, 'StartOffset': 0, 'MaxPartSize': 262144}, 'portId': 850678111}

<< {"data": {"__type": "ReturnPartialSmimeResult:#Microsoft.Exchange.Clients.BrowserExtension.Smime", "PartIndex": 0, "StartOffset": 0, "EndOffset": 100000, "IsLastPart": true, "PartialData": "{"Data": {"__type": "SmimeControlCapabilities:#Microsoft.Exchange.Clients.Smime", "SupportsAsyncMethods": true, "Version": "4.0800.20.21.804.1"}, "ErrorCode": 0}", "TotalSize": 161}, "messageType": "DownloadPartialResult", "portId": 850678111, "requestId": 2}

{'__type': 'ExtensionMessage:#Microsoft.Exchange.Clients.BrowserExtension.Smime', 'messageType': 'UploadPartialRequest', 'requestId': 3, 'data': {'__type': 'PostPartialSmimeRequest:#Microsoft.Exchange.Clients.BrowserExtension.Smime', 'PartIndex': 0, 'StartOffset': 0, 'EndOffset': 157, 'IsLastPart': True, 'PartialData': '{"__type":"GetSigningCertificateParams:#Microsoft.Exchange.Clients.BrowserExtension.Smime","SubjectKeyIdentifier":"STRING"}', 'TotalSize': 157, 'RequestType': 'GetSigningCertificate'}, 'portId': 850678111}

<< {"data": {"__type": "AcknowledgePartialSmimeRequestArrived:#Microsoft.Exchange.Clients.BrowserExtension.Smime", "PartIndex": -1, "StartOffset": -1, "NextStartOffset": -1, "Status": 1}, "messageType": "UploadPartialRequest", "portId": 850678111, "requestId": 3}

{'__type': 'ExtensionMessage:#Microsoft.Exchange.Clients.BrowserExtension.Smime', 'messageType': 'DownloadPartialResult', 'requestId': 3, 'data': {'__type': 'FetchPartialSmimeResult:#Microsoft.Exchange.Clients.BrowserExtension.Smime', 'PartIndex': 0, 'StartOffset': 0, 'MaxPartSize': 262144}, 'portId': 850678111}

<< {"data": {"__type": "ReturnPartialSmimeResult:#Microsoft.Exchange.Clients.BrowserExtension.Smime", "PartIndex": 0, "StartOffset": 0, "EndOffset": 100000, "IsLastPart": true, "PartialData": "{"Data": "REALLYLLLLOOOOOOONNNNNGGGGGGGGSSSSSSTTTTTRRRRRIIIIINNNNNGGGGG

{'__type': 'ExtensionMessage:#Microsoft.Exchange.Clients.BrowserExtension.Smime', 'messageType': 'UploadPartialRequest', 'requestId': 4, 'data': {'__type': 'PostPartialSmimeRequest:#Microsoft.Exchange.Clients.BrowserExtension.Smime', 'PartIndex': 0, 'StartOffset': 0, 'EndOffset': 3876, 'IsLastPart': True, 'PartialData': '{"__type":"CreateSmimeFromMessageParams:#Microsoft.Exchange.Clients.BrowserExtension.Smime","SmimeType":11,"EmailMessage":"{\"__type\":\"Message:#Exchange\",\"Subject\":\"test 47\",\"Importance\":\"Normal\",\"IsReadReceiptRequested\":false,\"IsDeliveryReceiptRequested\":false,\"ToRecipients\":[{\"EmailAddress\":\"my.email.address\",\"Name\":\"Last, First\",\"MailboxType\":\"InternalMailbox\",\"RoutingType\":\"SMTP\"}],\"CcRecipients\":[],\"BccRecipients\":[],\"Sensitivity\":\"Normal\",\"Body\":{\"__type\":\"BodyContentType:#Exchange\",\"BodyType\":\"Text\",\"Value\":\"47\",\"DataUriCount\":0},\"Attachments\":[],\"ExtendedProperty\":[{\"__type\":\"ExtendedPropertyType:#Exchange\",\"ExtendedFieldURI\":{\"__type\":\"Extended', 'TotalSize': 3876, 'RequestType': 'CreateSmimeFromMessage'}, 'portId': 850678437}

OFFICE 365 TEST TRIGGERED!

<< {"data": {"__type": "AcknowledgePartialSmimeRequestArrived:#Microsoft.Exchange.Clients.BrowserExtension.Smime", "PartIndex": -1, "StartOffset": -1, "NextStartOffset": -1, "Status": 1}, "messageType": "UploadPartialRequest", "portId": 850678437, "requestId": 4}

{'__type': 'ExtensionMessage:#Microsoft.Exchange.Clients.BrowserExtension.Smime', 'messageType': 'DownloadPartialResult', 'requestId': 4, 'data': {'__type': 'FetchPartialSmimeResult:#Microsoft.Exchange.Clients.BrowserExtension.Smime', 'PartIndex': 0, 'StartOffset': 0, 'MaxPartSize': 262144}, 'portId': 850678437}

<< {"data": {"__type": "ReturnPartialSmimeResult:#Microsoft.Exchange.Clients.BrowserExtension.Smime", "PartIndex": 0, "StartOffset": 0, "EndOffset": 100000, "IsLastPart": true, "PartialData": "{"Data": "MIME-Version: 1.0\nFrom: \"Last, First\" <my.email.address>\nTo: \"Last, First\" <my.email.address>\nSubject: test 47\nImportance: Normal\nSensitivity: Normal\nDate: Tue, 12 Nov 2024 09:54:00 \nX-Generated-By: OWA-SMIME4Linux 4.0800.20.21.804.1\nContent-Type: application/x-pkcs7-mime; name=\"smime.p7m\"; smime-type=signed-data\nContent-Transfer-Encoding: base64\nContent-Disposition: attachment; filename=\"smime.p7m\"\n\nLongString

{'__type': 'ExtensionMessage:#Microsoft.Exchange.Clients.BrowserExtension.Smime', 'messageType': 'UploadPartialRequest', 'requestId': 5, 'data': {'__type': 'PostPartialSmimeRequest:#Microsoft.Exchange.Clients.BrowserExtension.Smime', 'PartIndex': 0, 'StartOffset': 0, 'EndOffset': 1477, 'IsLastPart': True, 'PartialData': '{"__type":"CreateSmimeFromMessageParams:#Microsoft.Exchange.Clients.BrowserExtension.Smime","SmimeType":11,"EmailMessage":"{\"__type\":\"Message:#Exchange\",\"Subject\":\"test 47\",\"Importance\":\"Normal\",\"IsReadReceiptRequested\":false,\"IsDeliveryReceiptRequested\":false,\"ToRecipients\":[{\"EmailAddress\":\"my.email.address\",\"Name\":\"Last, First\",\"MailboxType\":\"InternalMailbox\",\"RoutingType\":\"SMTP\"}],\"CcRecipients\":[],\"BccRecipients\":[],\"Sensitivity\":\"Normal\",\"Body\":{\"__type\":\"BodyContentType:#Exchange\",\"BodyType\":\"Text\",\"Value\":\"47\",\"DataUriCount\":0},\"Attachments\":[],\"ExtendedProperty\":[{\"__type\":\"ExtendedPropertyType:#Exchange\",\"Value\":\"newMail\",\"ExtendedFieldURI\":', 'TotalSize': 1477, 'RequestType': 'CreateSmimeFromMessage'}, 'portId': 850678111}

OFFICE 365 TEST TRIGGERED!

<< {"data": {"__type": "AcknowledgePartialSmimeRequestArrived:#Microsoft.Exchange.Clients.BrowserExtension.Smime", "PartIndex": -1, "StartOffset": -1, "NextStartOffset": -1, "Status": 1}, "messageType": "UploadPartialRequest", "portId": 850678111, "requestId": 5}

{'__type': 'ExtensionMessage:#Microsoft.Exchange.Clients.BrowserExtension.Smime', 'messageType': 'DownloadPartialResult', 'requestId': 5, 'data': {'__type': 'FetchPartialSmimeResult:#Microsoft.Exchange.Clients.BrowserExtension.Smime', 'PartIndex': 0, 'StartOffset': 0, 'MaxPartSize': 262144}, 'portId': 850678111}

<< {"data": {"__type": "ReturnPartialSmimeResult:#Microsoft.Exchange.Clients.BrowserExtension.Smime", "PartIndex": 0, "StartOffset": 0, "EndOffset": 100000, "IsLastPart": true, "PartialData": "{"Data": "MIME-Version: 1.0\nFrom: \"Last, First\" <my.email.address>\nTo: \"Last, First\" <my.email.address>\nSubject: test 47\nImportance: Normal\nSensitivity: Normal\nDate: Tue, 12 Nov 2024 09:54:19 \nX-Generated-By: OWA-SMIME4Linux 4.0800.20.21.804.1\nContent-Type: application/x-pkcs7-mime; name=\"smime.p7m\"; smime-type=signed-data\nContent-Transfer-Encoding: base64\nContent-Disposition: attachment; filename=\"smime.p7m\"\n\nLongString

[EXIT]

Thanks again.

[schorschii]

Thanks for sending the logs. Unfortunately, I'm not able to debug this further without a O365 account. I created issue #6 with the label "help wanted", so maybe someone with an O365 account can have a look at it in the future.