[bug]: Stalwart doesn't care for seen flag set by sieve filters

[nomadturk]

What happened?

I have a Trusted script, running for most incoming emails, trying to sort, flag, forward some emails.
As part of that, one part of it tries mark clutter as read.

The problem is, Stalwart does not register these emails as "Read".
So there is always a big chunk of unread emails in such mailboxes.

And

In Snappy and Roundcube, they are listed as "Unread" even if they have the Seen flag.
In thunderbird, they are listed as if they are not marked as read at all.

Same email clients do not display the same behaviour with other mail servers.

How can we reproduce the problem?

I can reproduce the problem by doing the following steps:

Create a trusted sieve script.
Try to change a filter with the action changing the flag of some emails with something like below:

require ["body","imap4flags"];
if header :contains "subject" [
	"Auto:"
	]
	{
		setflag "\\Seen";
		stop;
	}

Call that script from your mailbox by adding

include :global "thatfilter";

Send yourself a few emails that should fit the criteria.

Check it with Snappy or Roundcube or Thunderbird.
Messages that should have been marked as "Read" will come as "Unread".

Version

v0.10.x

What database are you using?

PostgreSQL

What blob storage are you using?

PostgreSQL

Where is your directory located?

Internal

What operating system are you using?

Linux

Relevant log output

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[mdecimus]

Trusted scripts are global and operate at the SMTP transaction level, for this reason they do not have access to Sieve functions that modify mailboxes such as fileinto or setflag. In order to use such functions you need to run the script on the user's mailbox as an untrusted script.

[nomadturk]

Hmmm...

Looks like I also have an untrusted sieve script, with the same name and id.
I wonder which one is running during message acceptance when I call them with include global

[mdecimus]

Yes, a global script will do the trick. Have in mind thought thatglobal filters are not used unless they are included from each account's main Sieve filter.

[nomadturk]

Well...

I'm already calling the global filter via main sieve filter. Not as a script to be called on X stage for all accounts.
Shouldn't it have worked then?

[mdecimus]

Perhaps this is related to the case sensitivity bug? Have you tried including "globalfilters" in lowercase from the mailbox's main script?

[nomadturk]

The script is already included and runs, I can see the headers being added by it.
And I'm using lowercase script calls due to the sensitivity bug.

But, I had 2 scripts with the same ID, 1 under system 1 under user scripts. (This is a bug as well I guess?)
Is it possible that it's calling the one that's not able to add the flags properly?

[mdecimus]

But, I had 2 scripts with the same ID, 1 under system 1 under user scripts. (This is a bug as well I guess?)

That is fine, you can have the same script id on the trusted and untrusted interpreters. These are kept separately.

Is it possible that it's calling the one that's not able to add the flags properly?

No, the untrusted interpreter doe not have access to the trusted scripts and viceversa.

[nomadturk]

@mdecimus

Nope. this doesn't work.
If I make the script a System Script I get this error when I call them from inside the main script.

If I create a User Script, and call it inside the main script, these flags become useless.
Like when I want to list all emails with Flags, both Roundcube and Snappy return 0 results. Even when they are kind of flagged.

This is a bug.
Global scripts should be able to do these just as well.

[mdecimus]

The Script not found error you are seeing is due to the case sensitivity bug importing scripts, but there is a separate issue for that.

Regarding trusted vs untrusted scripts, let me clarify what each of these do:

• The trusted interpreter runs scripts at the SMTP DATA stage. These script are defined in the configuration file and run in a global context rather than on the users' mailboxes. This means that they cannot make changes to mailboxes such as seting flags or filing messages in folders. The only actions they can perform on a message is to add/remove/modify headers and modify its contents.
• The untrusted interpreter runs scripts that are associated with a user mailbox. These scripts are managed by the user and have to be uploaded using ManageSieve or JMAP for Sieve. They can perform any kind of action on the message including setting flags and filing messages into different folders.

[nomadturk]

It is not a case sensitivity.
I'm calling the script in lowercase. And I have created the script in lowercase.

Also, we should be able to call the global script from inside a mailbox sieve, and set everything up.

So, setting up flags properly shouldn't be an issue.
I think your interpretation is wrong here @mdecimus
That's how it works with all other email servers and that was how it was defined in the RFC.

As per RFC6609, there are no such differences.

And it also states that

  :global
      Indicates that the named script is stored in a site-wide Sieve
      repository, accessible to all users of the Sieve system.

Basically, it's meant to reduce using same scripts over and over and making it a repository for all users to use.
And users should be able to do all things they can do regularly, like setting flags, changing headers, filing into folders and so on.

So, if we are meant to run a global script via the "trusted" interpreter, we must have access to all of the functionality.

[mdecimus]

This has been fixed today, there was a similar report on #1059.

[nomadturk]

Thanks.
I'll check and update this post after the new release.