- 2022
- Aug
- 27
email misadventures - security is important!
What happened?
About a year ago, I swapped my domains over to a new registrar. The old one had become stale and was gaining a weird reputation, and they had cut their extremely attractive email plans down to crap Outlook plans. The final straw was when the free-with-reg email they provided was stopped. They charged for things like domain privacy, and it was just becoming more expensive than it was worth.
The new registrar had a better email plan using a non-Outlook service, offered free domain privacy and relatively inexpensive website SSL, so the choice was clear. However, the email plans didn’t auto-config as well as they claimed.
I knew there was something screwy with my email. Gmail, the great Satan of services (but if Google likes you, everyone does) would sometimes tell me that it couldn’t verify my email was sent by pygg.xyz. That TLD is somewhat suspect due to it’s use by spammers, but that wouldn’t cause a warning to happen, it would just bounce as spam. Something else was wrong. It finally came to a head when I tried to send a mail to a particularly tight system and it bounced as blocked.
I learned three new things: SPF, DMARC, and DKIM.
In the examples below, some personally identifying or secure information has been blurred, so do not use these as your own records.
SPF
SPF is a term that stands for “Sender Policy Framework.” It looks something like this:
This record is not one of the main named types used in Internet routing, so it simply gets a type of “TXT,” or as you probably guessed, text. The 1200 is the TTL (Time to live) which I simply left at the default.
What this record means, is that for my mail service of privateemail.com, mail coming from my domain of pygg.xyz is allowed to send mail. It’s simple “match this” mechanism, and it generally works as it’s a yes/no thing. Other mail is quarantined for inspection, which is what the ‘~all’ means. It’s called a softfail. The other option here is “-all” which means that any non-matching sender is rejected. This is called a hardfail. This can be dangerous if you’re not set up correctly, so I’ve left it as soft.
The vendor’s SPF record was orginally set up with a number of IPV4 addresses hard-coded in, which can be dangerous if the vendor’s address changes (such as adding new servers.) I found it better to match the mailserver’s name as opposed to it’s address. In this case, even if an address changes, you still match the name. I’m not sure, but it seems that the vendor may have had more mailservers than the record allowed. This fixed the SPF errors recorded by various testing services online.
DMARC
DMARC is a term that stands for “Domain-based Message Authentication, Reporting, and Conformance.” It looks like this:
This record is also not one of the named types, so it also gets a record type of TXT.
What this record means is simple. If someone tries to send email as you (i.e. spoofing,) the mailserver will take this action and send any suspicious mails to the addresses specifies. It works in conjunction with SPF and DKIM records. This record didn’t exist on my domain, so receiving servers looked at my email with a suspicious eye since they had no idea what my mail service did with spoofed. This record is one purely of self-policing trust. If I can’t take care of my affairs, it makes you less likely to deal with me. It was probably the most heavily weighted reason as to why I was being bounced.
To create this record, instead of wandering through what all of the options mean, I used a helper service. In particular, a site called dmarcian, located here: https://dmarcian.com … dmarc-record-wizard/. This leads you through all the things you need to fill out and automagically creates the record. It’s up to you to create the host and type, but you can follow mine or many other examples online.
One thing of note: You can either QUARANTINE or REJECT mail that doesn’t match your sending policy. There’s also another thing you can do, NONE - this means don’t do anything, just collect data. This is usually seen by receiving systems as a failure and is not suggested - it’s essentially the same as having no record at all.
DKIM
DKIM is the last thing that was messed up on my mail server, and part of it was my fault, part of it was the vendor’s fault. My fault was that, unless you’ve done this before, you have to trust that the vendor is giving you the correct information. As I had not done this before, I simply grabbed what was presented to me and pasted it in. That was wrong. The vendor didn’t clearly say that only a portion of what they presented was needed, and didn’t really mention that you had to append extra fields to use the long strings they generated.
DKIM is a term that stands for “DomainKeys Identified Mail”. It’s a public-private key pair that’s used to “sign” your emails and encode them, and if the receiving server can decode the mail with your public key, it’s assumed that you sent it since only you should have your key. It looks like this:
This record is an object that tells the receiving server what it is, what method to use during decoding, and what the actual key is. It’s (again) a TXT record, but this one has a single characteristic that is quite important. If you look at the first field, you’ll see “default._” - this is the object’s actual name, and MUST be used. In my case, the vendor used “default” as the object name when generating the key, although yours may be different. “s1”, “server1”, “bob” - whatever your vendor’s key generator uses MUST be used here as well. I didn’t understand that and used a name I found others mentioning, that of “s1”. This had the effect of the receiving server failing to find my key, as it was looking for “s1” and the server was using “default”. Changing the object’s name to “default” immediately fixed that and the remote server found the key and decrypted it.
My vendor also presented the key wrapped in a description that they created. That’s what really threw me off - when I pasted that in, remote sites still complained about a bad key, so I needed to determine the actual requirements of the key itself. The only parts of the key you need start at “v=DKIM” (the type of key) and end at the end of the actual key itself (the long string of characters.) Some sites suggested ending the record with a semicolon, so I did. I’m not sure if that’s needed, but it didn’t complain.
DKIM is a two-part thing on your end. You must have DKIM, and it must be valid. A receiver gives you a rating based on both. You can have a key, but if it’s invalid, that may still make your mail pass. Having both a key and a good key is imperative.
After that, I used some email test services that I found online. You send an email, and it analyzes the mail for proper construction. There are a number of those online. The only thing I can say about them is they are usually used for people who are sending newsletters, so they often analyze the content of the message and look for unsubscribe links. You can safely ignore those suggestions if you’re just setting up a personal email.
So what did I learn?
Modern email systems have a lot of security features that must be in place for other email systems to trust you. SPF, DMARC, and DKIM are all required, and even that may not save you.
For example, I’ve found out that my domain, which uses a TLD of .xyz, loses trust points simply for being an .xyz domain. Apparently, that’s used a lot for spammers, but there’s not much I can do about that. I can set my security up, and if you still don’t want to trust me then that’s your business, not mine.
What should you walk away with here?
This wasn’t meant to be a tutorial on how to set this stuff up, every email vendor has different ways of doing it and using my vendor’s system as an example is going to be useless for you. I mostly wanted to tell you about what you should look for when manually setting up your email system. It’s going to be up to you to learn how to implement those practices on your service.
Did the “tight” system accept your email afterwards?
No. It blocks .xyz wholesale, as best I can tell.
Gmail, on the other hand, now loves it and happily accepts it without error.