Mail attachment does not work / header problems

[spiro-trikaliotis]

Steps to reproduce

1. Open mail app (web interface)
2. Go to "Neue Nachricht" (new message)
3. Select a recipient, write subject and message body
4. Go to "..." at the bottom of the mail interface, then select "Anhang hochladen" (upload attachment)
5. Select an attachment from the File Selection Box
6. Press "SEND"
7. Look at the mail that was sent

I am using Nextcloud Hub 8 29.0.5 with Mail 3.7.7, but this also happened with older nextcloud versions 28.* and 29.0.x with corresponding older mail clients. Language is German.

Expected behavior

After step 7, I should see a mail with the attachment in another mail client (or in the mail app, if I sent a mail to myself)

Actual behavior

I see a mail, but the attachment is not there. I see a text "This message is in MIME format"

Mail app version

3.7.7

Mailserver or service

exim as mailserver; happens with SMPT transpart, as well as with sendmail interface

Operating system

Debian Linux

PHP engine version

PHP 8.2

Web server

Apache (supported)

Database

MariaDB

Additional info

Looking at the mail in maildiir format, I can see that the headers seem not to be formatted correctly:

From: XXXredactedXXX <XXXredactedXXX@XXXredactedXXX>
Cc:
Message-ID: <20240828123116.Horde.GesoMkoURhU5Gojp60h8exU@XXXredactedXXX>
User-Agent: Horde Application Framework 5
Date: Wed, 28 Aug 2024 12:31:16 +0000
Content-Type: multipart/mixed; boundary="=_gPXHtDyO8TUAmLNl8o7IgH_"
MIME-Version: 1.0
Message-Id: <E1sjHpU-00DBWN-1B@XXXredactedXXX>

Please note the space as first char in the lines Cc:, Message-ID: (the first one), User-Agent:, Date:, Content-Type: and MIME-Version:. Somehow, it seems the lines are not properly ended (perhaps, some CR/LF vs LF vs CR issue?)

That's also why the MTA creates a new Message-ID for this mail. This explains why the MIME data is not processed, because the headers are missing.

[spiro-trikaliotis]

I forgot to mention: If I send this mail to another mail address (also exim, but more strict), I get an "SMTP error: 550 header syntax"

[ChristophWurst]

Dear @spiro-trikaliotis,

thank you very much for your bug report.

I believe @GVodyanov ran into this as well. @GVodyanov do you remember what you had to do to make the attachment stay and get sent?

[GVodyanov]

@ChristophWurst I was only able to reproduce this on Firefox, and my temporary solution at the time was using chrome. I thought it was probably one of my extensions messing the attachments up, but seems like if other people have it too, there is some investigating to do... debugging this will be fun

[ChristophWurst]

Could you add this to your list? Debugging the frontend part would already help a lot. Like, is the attachment added to the message data, and is it part of the data added to the outbox?

[spiro-trikaliotis]

Thank you for your answers.

I normally work with Firefox, but I think I had also tested it in Chrome.
To make sure, I just tested it on Chrome v131.0.6778.69 (Official Build) (64-bit) with Nextcloud Hub 9 (30.0.2) and Mail 4.0.2: The problem also happens there.

Please also note that the attachment is there! I just took a message I had sent and removed the spaces at the first position of the header lines. After that, mutt could open the mail without problems. I copied this modified mail into the INBOX (with the help of mutt); now, even Nextcloud mail can correctly open the mail and the attachment.

Thus, it is only these spaces that are the problem. I do not understand where these come from, though.

[kesselb]

• You are not seeing this problem when using Roundcube, Rainloop or Thunderbird?
• Is there something unusual with the from header? For example using an emoji or so for the name.
• SMIME or Mailvelope?

Could you send me an email to [email protected] for an example?

[spiro-trikaliotis]

Hello Daniel,

I just sent a mail to the mail address you gave me.

• with Thunderbird, this problem does not exist. I did not test Roundcube or Rainloop, though, as I do not use them at all
• I don't think that the from is unusual. You can have a look at it yourself in the test mail.
• There is no S/MIME and no Mailvelope

[kesselb]

Hi,

I cannot spot an email from you in my inbox. Could you try to resend? It might help to send one from Thunderbird and one from Nextcloud Mail.

Thanks

[kesselb]

Hi,

Your email from 15:02:31 found its way into my inbox. I assume our spam filter delayed it a bit because it looked like spam due to the broken headers.

The email was delivered by www-data to your mx.
You are using 'app.mail.transport' => 'php-mail'?

If so, this means we don’t establish a direct connection with your SMTP server, but instead hand the message over to PHP’s mail function for delivery.

Are you observing the same issue when using 'app.mail.transport' => 'smtp'? I noticed you mentioned SMTP transport and the sendmail interface in your post, but these terms are often confused with Nextcloud email server configuration. I’d like to clarify this point to be sure.

Since I don’t have a working setup for PHP’s mail() locally, I could only inspect it briefly using the debugger. From what I’ve seen, we’re passing the headers to the mail() function like this:

This could indeed be an issue with the line endings, as you suggested. The documentation specifies that headers should use CRLF, although it later recommends LF to avoid issues with qmail.

The default implementation in horde/mail (the library we use for IMAP and SMTP) defaults to PHP_EOL: https://github.com/horde/Mail/blob/c6089da9ca82c932c5e2b8e68663c006980d69a6/lib/Horde/Mail/Transport.php#L61.

For SMTP it's overwritten to use CRLF: https://github.com/horde/Mail/blob/c6089da9ca82c932c5e2b8e68663c006980d69a6/lib/Horde/Mail/Transport/Smtphorde.php#L109.

However, the implementation for PHP’s mail() function does not override it: https://github.com/horde/Mail/blob/FRAMEWORK_6_0/lib/Horde/Mail/Transport/Mail.php.

It might be worth modifying the constructor of Horde/Mail/Transport/Mail.php to include $this->sep = "\r\n"; and testing if that resolves the issue.

That said, I’d still recommend switching to the SMTP transport and avoiding PHP’s mail() function where possible. It’s an outdated fallback and is hopefully not widely used anymore.

Let me know how it goes!

[spiro-trikaliotis]

Hello,

thank you very much, you helped me a lot!

It was indeed the php-mail transport that is the problem. If I change to SMTP, the problem is gone.

I was not even aware that I had php-mail in my configuration; it must have been a leftover from trying to get it working.

Additionally, you are also right: Adding the separator \r\n into the nextcloud-mail-php-mail.diff.gz fixes the problem also for php-mail.

[jiriks74]

Encountered this too. Wanted to delay an email for cca 4 hours, checked it in the outbox and the PDF I attached had a size of NaN.

[ChristophWurst]

Dear @jiriks74,
Thank you for your feedback. I'm sorry to hear you experienced this as well. Are you using the php-mailer as well?

@kesselb I'm wondering if we should remove this "feature". I've rarely heard of anyone using it successfully.

[jiriks74]

I'm sorry but I don't think I understand.

Php-mailer is a PHP library for interacting with mail servers, AFAIK, and I am not a PHP dev.

If Mail uses it then I guess I do?

[ChristophWurst]

Thank you for your message @jiriks74.

See #10066 (comment) and #10066 (comment).

Perhaps you can provide a sample message sent via Mail to demonstrate that what you are reporting is in fact the same problem that @spiro-trikaliotis reported. Would that be possible?

[jiriks74]

OK I'll look into it soon as I have only my phone RN.

[jiriks74]

Are you using the php-mailer as well?
No, I do not have 'app.mail.transport' => 'php-mail' set. Everything goes through my email server otherwise DKIM wouldn't work.

Screenshot

I made a new email to show this as the email in question had personal information in it.

Screenshots

[ChristophWurst]

Thank you for your message @jiriks74! It looks like you are running into a problem with editing of outbox messages, and an incorrectly displayed attachment size. This is not the problem reported in this ticket. Could you please submit a new ticket? Thank you.

[jiriks74]

OK so it's a UI issue?
I sent the test email to myself and got it with a working attachment.

I'm not sure how I feel about this. I guess I'll try sending more emails but this seems as a separate issue.

You'll understand that I didn't want to risk things on an important email.

[jiriks74]

It looks like you are running into a problem with editing of outbox messages, and an incorrectly displayed attachment size.

This sound on par with not being able to save a modified message in the outbox sometimes.

This is not the problem reported in this ticket. Could you please submit a new ticket? Thank you.

Will do soon.