openssl-users: DKIM, DMARC and all that jazz, and what it means to us

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

openssl-users: DKIM, DMARC and all that jazz, and what it means to us

Richard Levitte - VMS Whacker-2
Hi all,

It seem like DMARC, SPF, DKIM, or *something* is tripping us up quite
a bit.  Earlier this afternoon (that's what it is in Sweden at least),
us postmasters got a deluge of bounce reports from mailman, basically
telling us that it got something like this:

<[hidden email]>: host aspmx.l.google.com[74.125.140.26] said:
    550-5.7.1 This message does not have authentication information or fails to
    pass 550-5.7.1 authentication checks. To best protect our users from spam,
    the 550-5.7.1 message has been blocked. Please visit 550-5.7.1
    https://support.google.com/mail/answer/81126#authentication for more 550
    5.7.1 information. f1si3266960wro.105 - gsmtp (in reply to end of DATA
    command)

There's very little fact of what actually triggered these bounces, but
they always come from Google, so we're guessing that they're becoming
increasingly aggressive in their checks of DKIM, SPF, ARC, who knows
(they don't seem to check DMARC, 'cause we do have one with p=none and
an address to sent DMARC reports to, and I'm hearing absolutely
nothing from Google, but I do hear from others)

So, to mitigate the problem, we've removed all extra decoration of the
messages, i.e. the list footer that's usually added and the subject
tag that indicates what list this is (I added the "openssl-users:"
that you see manually).

So IF you're filtering the messages to get list messages in a
different folder, based on the subject line, you will unfortunately
have to change it.  If I may suggest something, it's to look at this:

    List-Id: <openssl-users.openssl.org>

Cheers,
Richard ( role: postmaster )

--
Richard Levitte         [hidden email]
OpenSSL Project         http://www.openssl.org/~levitte/
Reply | Threaded
Open this post in threaded view
|

Re: openssl-users: DKIM, DMARC and all that jazz, and what it means to us

OpenSSL - User mailing list
On 15/02/2019 16:03, Richard Levitte wrote:

> Hi all,
>
> It seem like DMARC, SPF, DKIM, or *something* is tripping us up quite
> a bit.  Earlier this afternoon (that's what it is in Sweden at least),
> us postmasters got a deluge of bounce reports from mailman, basically
> telling us that it got something like this:
>
> <[hidden email]>: host aspmx.l.google.com[74.125.140.26] said:
>      550-5.7.1 This message does not have authentication information or fails to
>      pass 550-5.7.1 authentication checks. To best protect our users from spam,
>      the 550-5.7.1 message has been blocked. Please visit 550-5.7.1
>      https://support.google.com/mail/answer/81126#authentication for more 550
>      5.7.1 information. f1si3266960wro.105 - gsmtp (in reply to end of DATA
>      command)
>
> There's very little fact of what actually triggered these bounces, but
> they always come from Google, so we're guessing that they're becoming
> increasingly aggressive in their checks of DKIM, SPF, ARC, who knows
> (they don't seem to check DMARC, 'cause we do have one with p=none and
> an address to sent DMARC reports to, and I'm hearing absolutely
> nothing from Google, but I do hear from others)
>
> So, to mitigate the problem, we've removed all extra decoration of the
> messages, i.e. the list footer that's usually added and the subject
> tag that indicates what list this is (I added the "openssl-users:"
> that you see manually).
>
> So IF you're filtering the messages to get list messages in a
> different folder, based on the subject line, you will unfortunately
> have to change it.  If I may suggest something, it's to look at this:
>
>      List-Id: <openssl-users.openssl.org>
>
> Cheers,
> Richard ( role: postmaster )
>
I have had some fruitless discussion with the mailman authors a while
back.  They seemed to insist that DMARC etc. were bad ideas and that
senders complying were broken and needed half-assed workarounds.

In my own role as postmaster, I see all these systems implementing
variations of the same concepts, that have pretty simply implications
for mailing list software:

1. The global mail system is increasingly implementing checks for
   spoofed source addresses.  List gateways thus need to be
   increasingly conservative in what they send out.

2. If posts contain any kind of digital signature (PGP, S/MIME,
   DKIM etc.), the mailing list software must either preserve the
   validity by not changing anything signed or remove that signature.
    Leaving a now-invalid signature in place makes the post obviously
   bogus to anyone checking, resulting in bounces and lost mail.
    (Optionally, the list gateway may add headers describing the
   validity of those signatures before processing, perhaps even
   rejecting posts with invalid signatures to reduce spam).

3. When sending out the mails, the various from addresses must be
   appropriately authorized, either by being the list gateway itself or
   by satisfying all the known checks for any preserved addresses.

4. As mass senders of mail, mailing list gateways should themselves
   implement all the checkable features in the standards: strict SPF
   records for its own domain, DKIM signatures for any mails with
   the list gateway as source, DMARC records telling recipients to
   discard/reject messages pretending to be from the list without
   satisfying all these checks.

5. The enforcement strictness in DMARC, SPF and DKIM DNS records
   should not be taken as license to violate the requirements.  For
   example an SPF rule of +all should not be treated as permission
   to use the posters domain in envelope-from.
    Frustration with spam may lead recipient systems to enforce
   more strictly than requested by the source domain.

More specific rules:

A. SPF requires the envelope-from (SMTP MAIL FROM) address to always
   be that of the list gateway, even if the posters domain has no SPF
   record.

B. A valid DKIM signature from the posters domain can allow keeping
   the poster as From: address if the DKIM signature is undamaged.

C. Some DKIM signatures allow appending a mailing list footer to the
   end of plaintext mails and adding the mailing list headers to the
   mail, others do not.  In practice this is determined by
   implementation details in the posters mail server (for example,
   most versions of exim don't sign in that permissive way).
    Programmatically this can be determined by directly comparing the
   coverage indicated in the DKIM header to the intended mail
   modifications.

D. DMARC DNS records indicate if a sending domain wants to restrict
   header-From (etc.) pointing to that domain to only be used with
   at least one of DKIM and SPF passing for header-From.  Rule 5
   applies, but so does rule C.


Enjoy

Jakob
--
Jakob Bohm, CIO, Partner, WiseMo A/S.  https://www.wisemo.com
Transformervej 29, 2860 Søborg, Denmark.  Direct +45 31 13 16 10
This public discussion message is non-binding and may contain errors.
WiseMo - Remote Service Management for PCs, Phones and Embedded

Reply | Threaded
Open this post in threaded view
|

Re: openssl-users: DKIM, DMARC and all that jazz, and what it means to us

Lewis Rosenthal
In reply to this post by Richard Levitte - VMS Whacker-2
Hi, Richard...

I'm not going to place my reply after Jakob's, as his makes a number of
excellent points, with many of which I wholeheartedly agree (I'm not big
on DKIM and DMARC, myself). However, a few points specific to the case
at hand, if I may:

Richard Levitte wrote:

> Hi all,
>
> It seem like DMARC, SPF, DKIM, or *something* is tripping us up quite
> a bit.  Earlier this afternoon (that's what it is in Sweden at least),
> us postmasters got a deluge of bounce reports from mailman, basically
> telling us that it got something like this:
>
> <[hidden email]>: host aspmx.l.google.com[74.125.140.26] said:
>      550-5.7.1 This message does not have authentication information or fails to
>      pass 550-5.7.1 authentication checks. To best protect our users from spam,
>      the 550-5.7.1 message has been blocked. Please visit 550-5.7.1
>      https://support.google.com/mail/answer/81126#authentication for more 550
>      5.7.1 information. f1si3266960wro.105 - gsmtp (in reply to end of DATA
>      command)
>
> There's very little fact of what actually triggered these bounces, but
> they always come from Google, so we're guessing that they're becoming
> increasingly aggressive in their checks of DKIM, SPF, ARC, who knows
> (they don't seem to check DMARC, 'cause we do have one with p=none and
> an address to sent DMARC reports to, and I'm hearing absolutely
> nothing from Google, but I do hear from others)
>

The onus for getting the attention of the mail admins at Google needs to
be on those who use their services for mail, and not on a third party.
If this were a non-technical list (the high school soccer team
schedule), I might not expect all of the list members to be able to
discuss in technical terms with the Google mail admins what the problems
may be, but people on this list should be able to get the relevant
points across, citing RFC numbers and so forth.

I often find myself assisting other admins (aren't we all on alternating
sides of that coin?) when we have delivery problems. The biggest hurdle
is getting to the right admin on the "problem" side, which is why the
initial contact needs to come from one of their customers who has been
affected.

> So, to mitigate the problem, we've removed all extra decoration of the
> messages, i.e. the list footer that's usually added and the subject
> tag that indicates what list this is (I added the "openssl-users:"
> that you see manually).
>

I strongly encourage you to re-think this. Everyone else on this list
whose server has been properly configured to not trash legitimate
messages must now be inconvenienced by the needs of a seemingly
tone-deaf provider. (FWIW, I go through this with yahoo.com addresses
all the time; the fault lies there, not in the list configuration - so
long as the list configuration follows the applicable RFC guidelines.)

> So IF you're filtering the messages to get list messages in a
> different folder, based on the subject line, you will unfortunately
> have to change it.  If I may suggest something, it's to look at this:
>
>      List-Id: <openssl-users.openssl.org>
>

Yes, this can be done, but without the list ID in square brackets in the
subject, what is liable to happen is that the entire string will be
replaced along the line when thread subjects change (e.g., "blah-blah
(was: blah)") and we would all have to remember to type "openssl-users:"
in order to get "organized" subjects (yes, I know; filtering to a
particular folder on the List-Id header should effectively "organize"
list messages by corralling them, but that's not my point). Threading is
liable to go at least slightly off the rails for some of us (depending
upon mail client), and there are a host of potential side effects, all
for what? The next time Google decides to change their filters, should
list managers hop-to and make further changes?

My own thinking is that if list messages cannot proliferate across
Google's infrastructure, then those list members should find alternative
means of subscribing. Undoubtedly, this is not the only list which would
be so affected for them.

--
Lewis
-------------------------------------------------------------
Lewis G Rosenthal, CNA, CLP, CLE, CWTS, EA
Rosenthal & Rosenthal, LLC                www.2rosenthals.com
visit my IT blog                www.2rosenthals.net/wordpress
-------------------------------------------------------------

Reply | Threaded
Open this post in threaded view
|

Re: openssl-users: DKIM, DMARC and all that jazz, and what it means to us

Richard Weinberger
In reply to this post by Richard Levitte - VMS Whacker-2
Am Freitag, 15. Februar 2019, 16:03:42 CET schrieb Richard Levitte:
> So, to mitigate the problem, we've removed all extra decoration of the
> messages, i.e. the list footer that's usually added and the subject
> tag that indicates what list this is (I added the "openssl-users:"
> that you see manually).

Hmm, and as side effect you have forcefully re-enabled mail delivery for all
mailinglist members?
I disable mail delivery usually and read mails via gmail.

Thanks,
//richard


Reply | Threaded
Open this post in threaded view
|

Re: openssl-users: DKIM, DMARC and all that jazz, and what it means to us

Richard Levitte - VMS Whacker-2
I did re-enable everyone that had [B] (for bounce) as reason for not receiving mail, but I may have gotten one or two that were disabled by choice. Sorry about that...

Cheers
Richard

Richard Weinberger <[hidden email]> skrev: (15 februari 2019 18:46:14 CET)

>Am Freitag, 15. Februar 2019, 16:03:42 CET schrieb Richard Levitte:
>> So, to mitigate the problem, we've removed all extra decoration of
>the
>> messages, i.e. the list footer that's usually added and the subject
>> tag that indicates what list this is (I added the "openssl-users:"
>> that you see manually).
>
>Hmm, and as side effect you have forcefully re-enabled mail delivery
>for all
>mailinglist members?
>I disable mail delivery usually and read mails via gmail.
>
>Thanks,
>//richard

--
Sent from my Android device with K-9 Mail. Please excuse my brevity.
Reply | Threaded
Open this post in threaded view
|

Re: openssl-users: DKIM, DMARC and all that jazz, and what it means to us

Richard Levitte - VMS Whacker-2
In reply to this post by Lewis Rosenthal
On Fri, 15 Feb 2019 18:33:30 +0100,
Lewis Rosenthal wrote:
>
> Hi, Richard...
>
> I'm not going to place my reply after Jakob's, as his makes a number
> of excellent points, with many of which I wholeheartedly agree (I'm
> not big on DKIM and DMARC, myself). However, a few points specific to
> the case at hand, if I may:

Yes you may.  Quite frankly, I'm frustrated with the situation, and
it...  well, kinda exploded today (getting a huge bunch of messages
from mailman tell us that it had disabled this and that user, it
turned out to be quite a lot of them...).  Either way, I'll take any
help I can get to get some clarity and a path forward.

> Richard Levitte wrote:
> > Hi all,
> >
> > It seem like DMARC, SPF, DKIM, or *something* is tripping us up quite
> > a bit.  Earlier this afternoon (that's what it is in Sweden at least),
> > us postmasters got a deluge of bounce reports from mailman, basically
> > telling us that it got something like this:
> >
> > <[hidden email]>: host aspmx.l.google.com[74.125.140.26] said:
> >      550-5.7.1 This message does not have authentication information or fails to
> >      pass 550-5.7.1 authentication checks. To best protect our users from spam,
> >      the 550-5.7.1 message has been blocked. Please visit 550-5.7.1
> >      https://support.google.com/mail/answer/81126#authentication for more 550
> >      5.7.1 information. f1si3266960wro.105 - gsmtp (in reply to end of DATA
> >      command)
> >
> > There's very little fact of what actually triggered these bounces, but
> > they always come from Google, so we're guessing that they're becoming
> > increasingly aggressive in their checks of DKIM, SPF, ARC, who knows
> > (they don't seem to check DMARC, 'cause we do have one with p=none and
> > an address to sent DMARC reports to, and I'm hearing absolutely
> > nothing from Google, but I do hear from others)
> >
>
> The onus for getting the attention of the mail admins at Google needs
> to be on those who use their services for mail, and not on a third
> party. If this were a non-technical list (the high school soccer team
> schedule), I might not expect all of the list members to be able to
> discuss in technical terms with the Google mail admins what the
> problems may be, but people on this list should be able to get the
> relevant points across, citing RFC numbers and so forth.
>
> I often find myself assisting other admins (aren't we all on
> alternating sides of that coin?) when we have delivery problems. The
> biggest hurdle is getting to the right admin on the "problem" side,
> which is why the initial contact needs to come from one of their
> customers who has been affected.
>
> > So, to mitigate the problem, we've removed all extra decoration of the
> > messages, i.e. the list footer that's usually added and the subject
> > tag that indicates what list this is (I added the "openssl-users:"
> > that you see manually).
> >
>
> I strongly encourage you to re-think this. Everyone else on this list
> whose server has been properly configured to not trash legitimate
> messages must now be inconvenienced by the needs of a seemingly
> tone-deaf provider. (FWIW, I go through this with yahoo.com addresses
> all the time; the fault lies there, not in the list configuration - so
> long as the list configuration follows the applicable RFC guidelines.)

Well, if we change the subject of a DKIM signed message, don't we
break it?  (I'm not sure how applicable that's with Google, as we
received the same kind of bounce for message originating at
openssl.org (there is a DMARC record with p=none, so shouldn't affect
anything as far as I understand) and no DKIM signature...  but still,
when there is one...

> > So IF you're filtering the messages to get list messages in a
> > different folder, based on the subject line, you will unfortunately
> > have to change it.  If I may suggest something, it's to look at this:
> >
> >      List-Id: <openssl-users.openssl.org>
> >
>
> Yes, this can be done, but without the list ID in square brackets in
> the subject, what is liable to happen is that the entire string will
> be replaced along the line when thread subjects change (e.g.,
> "blah-blah (was: blah)") and we would all have to remember to type
> "openssl-users:" in order to get "organized" subjects (yes, I know;
> filtering to a particular folder on the List-Id header should
> effectively "organize" list messages by corralling them, but that's
> not my point). Threading is liable to go at least slightly off the
> rails for some of us (depending upon mail client), and there are a
> host of potential side effects, all for what? The next time Google
> decides to change their filters, should list managers hop-to and make
> further changes?
>
> My own thinking is that if list messages cannot proliferate across
> Google's infrastructure, then those list members should find
> alternative means of subscribing. Undoubtedly, this is not the only
> list which would be so affected for them.

Well, Google users is a *large* part of our subscribers, and some of
them are Google Apps users, possibly not of their own choice.  I
believe that Google users aren't quite as easy to dismiss as, say,
hotmail back when that provider tumbled down the reputation shute.

Cheers,
Richard

--
Richard Levitte         [hidden email]
OpenSSL Project         http://www.openssl.org/~levitte/
Reply | Threaded
Open this post in threaded view
|

Re: openssl-users: DKIM, DMARC and all that jazz, and what it means to us

OpenSSL - User mailing list
On 16/02/2019 00:02, Richard Levitte wrote:

> On Fri, 15 Feb 2019 18:33:30 +0100, Lewis Rosenthal wrote:
>> ...
>>
>> I strongly encourage you to re-think this. Everyone else on this list
>> whose server has been properly configured to not trash legitimate
>> messages must now be inconvenienced by the needs of a seemingly
>> tone-deaf provider. (FWIW, I go through this with yahoo.com addresses
>> all the time; the fault lies there, not in the list configuration - so
>> long as the list configuration follows the applicable RFC guidelines.)
> Well, if we change the subject of a DKIM signed message, don't we
> break it?  (I'm not sure how applicable that's with Google, as we
> received the same kind of bounce for message originating at
> openssl.org (there is a DMARC record with p=none, so shouldn't affect
> anything as far as I understand) and no DKIM signature...  but still,
> when there is one...
Indeed it does break it (unless the signature unusually doesn't
cover the Subject).   According to the RFC, a DKIM signature can
choose an almost arbitrary subset of headers to cover (including
covering the absence of a header type), plus a choice between
signing the entire body or only the first N lines (for arbitrary
N).  That "first N lines" option is how to create a DKIM signature
that allows appending a list footer.

As for p=none, this is what my rule 5 covered, just because a DMARC
record says p=none doesn't remove the requirement for messages to
be correct, only lowers the default error handling to a warning (I
receive daily mails listing which IP addresses spoofed our domains
by sending out mails with the not doing so, as is required by the
DMARC RFC, and I did so when I had p=none).

Having a DMARC record without DKIM signatures (including DKIM
signing mails relayed with openssl.org as From: address) is either
an RFC violation or very close to one.  So I would suggest setting
that up.  There are probably generic plugins for Postfix already,
but check the DMARC and DKIM RFC rules for how to handle the various
special address combinations that occur in mailing list traffic
(such as having Sender and From with different domains).  Because
the plugins may not have been tested for that.


Enjoy

Jakob
--
Jakob Bohm, CIO, Partner, WiseMo A/S.  https://www.wisemo.com
Transformervej 29, 2860 Søborg, Denmark.  Direct +45 31 13 16 10
This public discussion message is non-binding and may contain errors.
WiseMo - Remote Service Management for PCs, Phones and Embedded

Reply | Threaded
Open this post in threaded view
|

Re: openssl-users: DKIM, DMARC and all that jazz, and what it means to us

Richard Levitte - VMS Whacker-2
On Mon, 18 Feb 2019 22:51:09 +0100,
Jakob Bohm wrote:
> Having a DMARC record without DKIM signatures (including DKIM
> signing mails relayed with openssl.org as From: address) is either
> an RFC violation or very close to one.

I suspected that.  We're not quite ready for full blown DKIM yet, so
I'll remove the DMARC record for now.

Thank you.

(I know that you have sent other recommendations, but haven't read
them yet...  be assured that I will give them consideration)

Cheers,
Richard

--
Richard Levitte         [hidden email]
OpenSSL Project         http://www.openssl.org/~levitte/