[Bug]: External share with an e-mail address won't work when the receiver is using an URL-rewriter (security feature) in his mailbox

[WAG-Adm]

⚠️ This issue respects the following points: ⚠️

• This is a bug, not a question or a configuration/webserver/proxy issue.
• This issue is not already reported on Github OR Nextcloud Community Forum (I've searched it).
• Nextcloud Server is up to date. See Maintenance and Release Schedule for supported versions.
• I agree to follow Nextcloud's Code of Conduct.

Bug description

We are sending some e-mails with a share to exchange datas between our customer / provider through the nextcloud UI and we have determined, that the one's using an URL-rewriter in their mailboxes cannot open correctly the links. Sometimes they don't access at all or some times they cannot upload anything and receive the message: Process blocked by access control. And we can guarantee that's not the case. If we are using a simple test mailbox outside our network without any URL-rewriter function, it will work flawlessly. We cannot say if it's only with snap or with every nextcloud. We are using only Nextcloud with snap.

Steps to reproduce

1.Files -> some folder -> the user icon with the plus -> external shares -> write an e-mail address -> custom permission without delete -> Save share
The screenshots are received from our contacts. We don't have something to show by ourselves right now.

The mail is sent without any problem. But the only contacts who complain are writing us back and we have seen that all our URL of the e-mail was changed -> something like urldefense.com (Proofpoint) or urlsand.esvalabs.com (Libraesva srl)
All the other ones haven't any problem.
We have not every time the same state. For urlsand.esvalabs.com for example the call in browser of the URL shows nothing. For urldefense.com the user can access the files, but cannot upload something every time there's a problem.

Our contacts every time are sending the files / documents through Google Drive and it's working every time. That's not so good to spread Nextcloud.
2.
3.

Expected behavior

The contacts opens the link and can upload something without any messsage like "Process blocked by access control" - I repeat that I can upload the same file with the same name without any problems if no URL rewriting program is installed on the mailbox, and we can say that the trend toward using such software is increasing.

Nextcloud Server version

31

Operating system

Debian/Ubuntu

PHP engine version

PHP 8.3

Web server

Apache (supported)

Database engine version

MySQL

Is this bug present after an update or on a fresh install?

Fresh Nextcloud Server install

Are you using the Nextcloud Server Encryption module?

Encryption is Disabled

What user-backends are you using?

• Default user-backend (database)
• LDAP/ Active Directory
• SSO - SAML
• Other

Configuration report

{
    "system": {
        "apps_paths": [
            {
                "path": "\/snap\/nextcloud\/current\/htdocs\/apps",
                "url": "\/apps",
                "writable": false
            },
            {
                "path": "\/var\/snap\/nextcloud\/current\/nextcloud\/extra-apps",
                "url": "\/extra-apps",
                "writable": true
            }
        ],
        "supportedDatabases": [
            "mysql"
        ],
        "memcache.locking": "\\OC\\Memcache\\Redis",
        "memcache.local": "\\OC\\Memcache\\Redis",
        "redis": {
            "host": "***REMOVED SENSITIVE VALUE***",
            "port": 0
        },
        "log_type": "file",
        "logfile": "\/var\/snap\/nextcloud\/current\/logs\/nextcloud.log",
        "logfilemode": 416,
        "maintenance_window_start": 1,
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "xx.xxx.xxx.xxx",
            "xxx.xxx.xx.xxx",
            "xx.xx.xxx.xxx",
            "xxxx.xxxxx",
            "xxxx.xxxx.xxxx"
        ],
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "dbtype": "mysql",
        "version": "31.0.8.1",
        "overwrite.cli.url": "https:\/\/xxx.xxxx.xxxx",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "",
        "dbtableprefix": "oc_",
        "mysql.utf8mb4": true,
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "trusted_proxies": "***REMOVED SENSITIVE VALUE***",
        "overwriteprotocol": "https",
        "overwritehost": "xxx.xxxxx.xxxx",
        "maintenance": false,
        "mail_smtpmode": "smtp",
        "overwirtecondaddr": "^XX.X.XXX.XXX$",
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_sendmailmode": "smtp",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpport": "XX",
        "default_phone_region": "DE",
        "default_language": "en",
        "loglevel": 2,
        "skeletondirectory": "",
        "theme": "",
        "simpleSignUpLink.shown": false,
        "mail_smtpstreamoptions": {
            "ssl": {
                "allow_self_signed": true,
                "verify_peer": false,
                "verify_peer_name": false
            }
        }
    }
}

List of activated Apps

Enabled:
  - activity: 4.0.0
  - app_api: 5.0.2
  - bruteforcesettings: 4.0.0
  - circles: 31.0.0
  - cloud_federation_api: 1.14.0
  - comments: 1.21.0
  - contacts: 7.3.1
  - contactsinteraction: 1.12.0
  - dashboard: 7.11.0
  - dav: 1.33.0
  - deck: 1.15.2
  - federatedfilesharing: 1.21.0
  - federation: 1.21.0
  - files: 2.3.1
  - files_antivirus: 6.0.4
  - files_downloadlimit: 4.0.0
  - files_pdfviewer: 4.0.0
  - files_reminders: 1.4.0
  - files_sharing: 1.23.1
  - files_versions: 1.24.0
  - groupfolders: 19.1.3
  - guests: 4.5.1
  - logreader: 4.0.0
  - lookup_server_connector: 1.19.0
  - notifications: 4.0.0
  - oauth2: 1.19.1
  - photos: 4.0.0
  - previewgenerator: 5.10.0
  - privacy: 3.0.0
  - profile: 1.0.0
  - provisioning_api: 1.21.0
  - recommendations: 4.0.0
  - related_resources: 2.0.0
  - richdocuments: 8.7.4
  - serverinfo: 3.0.0
  - settings: 1.14.0
  - sharebymail: 1.21.0
  - side_menu: 5.1.1
  - survey_client: 3.0.0
  - systemtags: 1.21.1
  - text: 5.0.0
  - theming: 2.6.1
  - theming_customcss: 1.18.0
  - twofactor_backupcodes: 1.20.0
  - user_status: 1.11.0
  - viewer: 4.0.0
  - weather_status: 1.11.0
  - webhook_listeners: 1.2.0
  - whiteboard: 1.2.0
  - workflowengine: 2.13.0
Disabled:
  - admin_audit: 1.21.0
  - encryption: 2.19.0
  - files_external: 1.23.0
  - files_trashbin: 1.21.0 (installed 1.20.1)
  - firstrunwizard: 4.0.0 (installed 3.0.0)
  - nextcloud_announcements: 3.0.0 (installed 2.0.0)
  - onlyoffice: 9.10.0 (installed 9.10.0)
  - password_policy: 3.0.0 (installed 2.0.0)
  - support: 3.0.0 (installed 2.0.0)
  - suspicious_login: 9.0.1
  - twofactor_nextcloud_notification: 5.0.0
  - twofactor_totp: 13.0.0-dev.0
  - user_ldap: 1.22.0

Nextcloud Signing status

No errors have been found.

Nextcloud Logs

The problems in the URLs are on the client side, not the server side.

Additional info

[solracsf]

Bottom line is, what can Nextcloud server do if URLs are rewritten on the client side, completely out of control from Nextcloud? 🧐

[joshtrichards]

Can you tell whether these attempts are even reaching your infrastructure (i.e. per your reverse proxy, web server, or Nextcloud logs)?¹

Do the rewritten links decode and look reasonable (such as the share token getting messed with): e.g. https://www.vircom.com/urldefensedecoder/

Can you provide a sample real (i.e. test document) link that has been send/rewritten from both url services? ²

I'm not sure off-hand how the various security policies in a Nextcloud would interact with the intermediate redirect (and whatever else) that is introduced by these services; to debug this really requires end-to-end access to capture end client (browser console logs) and server-side logs. I could see there being potential problems with HSTS, CSP, cookies, etc.

There's a chance this works for services like Google due to whitelisting (either on the URL Defense side or even by Google); or maybe there is something else to be done here. Unfortunately neither of the filtering services you mentioned appear to have a well written and detailed document on this topic.

As an aside, this line in your config doesn't look right:

        "overwirtecondaddr": "^XX.X.XXX.XXX$",

Footnotes

1. Re: access control. The Operation is blocked by access control is generated client-side (not server-side) by the Nextcloud Web UI client and appears when the HTTP response is a 403 during an upload attempt: https://github.com/nextcloud/server/blob/c53a9bfde87c326b10f6914c156ac547664bddfc/apps/files/src/views/FilesList.vue#L746-L748 While normally that would be the Nextcloud Server sending the 403 to the client/visitor, it could also be an intermediate server in the path. It could also indicate the share token is getting messed up, etc. ↩

2. Obviously don't send one that is for a confidential shared document. Maybe create a dummy test file and share it for testing purposes with a couple of the impacted parties... then have them send it back to you? Then you can test the rewritten URL directly yourself and monitor the browser console for the specific errors (and even send it along here). ↩

[WAG-Adm]

Bottom line is, what can Nextcloud server do if URLs are rewritten on the client side, completely out of control from Nextcloud? 🧐

I am not an expert on URL rewriting security. Our contacts say they have no problems when using shared folders for uploads on Google Drive (and they will send the files through Google Drive instead of Nextcloud that's bad). Without getting too technical, what does Google do to make it work that Nextcloud may not be able to do, or am I perhaps missing the right information to set the correct parameters to make it work? We cannot undo the security features that are not implemented in our network, even though they are very popular in mobile and network security. Nevertheless, we need to find a solution to communicate with customers and partners.

[WAG-Adm]

Can you tell whether these attempts are even reaching your infrastructure (i.e. per your reverse proxy, web server, or Nextcloud logs)?1

Do the rewritten links decode and look reasonable (such as the share token getting messed with): e.g. https://www.vircom.com/urldefensedecoder/

Can you provide a sample real (i.e. test document) link that has been send/rewritten from both url services? 2

I'm not sure off-hand how the various security policies in a Nextcloud would interact with the intermediate redirect (and whatever else) that is introduced by these services; to debug this really requires end-to-end access to capture end client (browser console logs) and server-side logs. I could see there being potential problems with HSTS, CSP, cookies, etc.

There's a chance this works for services like Google due to whitelisting (either on the URL Defense side or even by Google); or maybe there is something else to be done here. Unfortunately neither of the filtering services you mentioned appear to have a well written and detailed document on this topic.

As an aside, this line in your config doesn't look right:

        "overwirtecondaddr": "^XX.X.XXX.XXX$",

Footnotes

1. Re: access control. The `Operation is blocked by access control` is generated client-side (not server-side) by the Nextcloud Web UI client and appears when the HTTP response is a 403 during an upload attempt: https://github.com/nextcloud/server/blob/c53a9bfde87c326b10f6914c156ac547664bddfc/apps/files/src/views/FilesList.vue#L746-L748 While normally that would be the Nextcloud Server sending the 403 to the client/visitor, it could also be an intermediate server in the path. It could also indicate the share token is getting messed up, etc. [↩](#user-content-fnref-acl-f6b4575e4576c37c4b7a96dcd257b58a)

2. Obviously don't send one that is for a confidential shared document. Maybe create a dummy test file and share it for testing purposes with a couple of the impacted parties... then have them send it back to you? Then you can test the rewritten URL directly yourself and monitor the browser console for the specific errors (and even send it along here). [↩](#user-content-fnref-sample-f6b4575e4576c37c4b7a96dcd257b58a)

overwritecondaddr was masked for security reason. The mix-up between the letters in the parameter was my mistake, please accept my apologies.
I will try to obtain as much information as possible. Communication was handled by our sales department. They received this type of feedback from our contacts, but until now we are receiving only from end-users, they're directly send us a link in the next mail through Google drive, if I rightly understood our sales department.

Thank you for your help.

Regards,

Joel.

[nextcloud-command]

This issue has been automatically marked as stale because it has not had recent activity and seems to be missing some essential information. It will be closed if no further activity occurs. Thank you for your contributions.

[WAG-Adm]

Hello everyone,

Does anyone have any ideas on how to solve this problem?

Best regards,
Joel.

[nextcloud-command]

This issue has been automatically marked as stale because it has not had recent activity and seems to be missing some essential information. It will be closed if no further activity occurs. Thank you for your contributions.