Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for Webhooks to notify the mail app that new mail has arrived #9855

Open
1 task done
SebastianKrupinski opened this issue Jul 15, 2024 · 6 comments · May be fixed by #10221
Open
1 task done

Add support for Webhooks to notify the mail app that new mail has arrived #9855

SebastianKrupinski opened this issue Jul 15, 2024 · 6 comments · May be fixed by #10221

Comments

@SebastianKrupinski
Copy link
Contributor

SebastianKrupinski commented Jul 15, 2024

Is your feature request related to a problem? Please describe.

No

Describe the solution you'd like

Some mail server now support webhooks to notify mail clients that new messages have arrived. It would be nice to have the mail server trigger a mail sync when new messages arrive instead of waiting for the mail app to update.

Describe alternatives you've considered

No response

Work packages

  • Clarify if we can use implement a standardized endpoint that can be used by Dovecot and Stalwart

Additional context

No response

@SebastianKrupinski
Copy link
Contributor Author

Dovecot and Stalwart do support push notifications, but are missing some functionality at the moment to make this feature work.

There is a push notification standard called "Internet Message Store Events RFC 5423", the standard defines what events should be accessible and what parameters should be included. BUT the standard does NOT define how the information should be delivered to the consuming service. Therefor there is no standardization on authentication methods or data format (JSON, XML, Other)

Dovecot

Does support the Internet Message Store Events RFC 5423, but does not have built in capabilities to trigger HTTP endpoints on the consuming system. Instead Dovecot uses a plugin structure that triggers methods for each event with the required data according to the RFC. Dovecot does have a pre made plugin that supports Lua scripting which could be used to either directly add changes to the NC database or used to trigger a endpoint on NC.

Pros

  • Implements Internet Message Store Events RFC 5423

Cons

  • Requires custom plugin/script to transmit notifications

Stalwart

Does have built in capabilities to trigger HTTP endpoints on a consuming system. BUT does not support the Internet Message Store Events RFC 5423 and does not have the ability to use a plugin or scripting to accomplish this. Another issue with Stalwart is the lack of required information in mail operation notification, mainly the accountName (user login or email address) as all operation notification only contain a internal "accountId".

Pros

  • Can trigger HTTP endpoints out of the box (JSON)
  • Supports all mail operations

Cons

  • Does not support RFC required parameters
  • Would require revers connections to retrieve required information

@ChristophWurst
Copy link
Member

Then let's go with Dovecot + Script for now

@SebastianKrupinski SebastianKrupinski linked a pull request Oct 3, 2024 that will close this issue
@ChristophWurst
Copy link
Member

For documentation purposes, @SebastianKrupinski please keep this ticket updated with the blockers you've found with Stalwart. Thanks.

@SebastianKrupinski
Copy link
Contributor Author

Update:

Dovecot - With the use of a lua script most functions are working properly (Mailbox Create/Rename/Move/Delete), but we are missing some information on the server side to make Message Create/Move/Delete work properly. Waiting for contact information for dovecot

Stalwart - Although Stalwart has webhooks support they do not use the Internet Message Store Events RFC 5423 format. The main issue is missing information in the webhook messages, as all folder references are internal id's "10" and not IMAP folder names "Inbox\Test Folder" due to this there is no way for us to translate this information properly for our use.

Image

@SebastianKrupinski
Copy link
Contributor Author

Update:

Had a conversation with Dovecot, the information we are looking for in the hook request, is not available. Therefore we still have a situation where MessageAppend (Create/Move/Delete) works but is missing some important information like Date and Preview.

A solution to this would be to:

  • make a outbound IMAP call to the server to retrieve the missing information on a per message basis. Down side is if the a user moves 20/30 messages at once, this would cause one connection to the server per message.
  • make a outbound IMAP call to the server to sync the whole mailbox. This could be done on time interval, by tracking when the mailbox was last sync'd and only syncing every few minutes, versus on every hook trigger.

All though both these are viable options, they both have a side effect. From my tests, hook calls in Dovecot are blocking, the IMAP response is paused until the hook call returns a status.

Possible ways around this issue:

  • Do early status return but continue the processing (not sure if this is possible, at the moment)
  • Mark the folder as dirty and schedule a one time background job to sync that folder
  • Hijack task processing (if available) with a custom job to run a sync instantly in the back ground (this would be the best solution)

@ChristophWurst ChristophWurst moved this from 🏗️ In progress to 📄 To do in 💌 📅 👥 Groupware team Dec 13, 2024
@ChristophWurst
Copy link
Member

Thanks for the update. The effort seems rather high right now for a feature that will only be used by a small number of setups. Therefore we will put a hold on the change.

@ChristophWurst ChristophWurst moved this from 📄 To do to 🧭 Planning evaluation in 💌 📅 👥 Groupware team Dec 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: 🧭 Planning evaluation
Development

Successfully merging a pull request may close this issue.

2 participants