Brightball

Enterprise Challenges with DMARC Deployment

DMARC | Email | Security | dmarc-guide | - July 25, 2022 // Barry

DMARC deployment projects in larger organizations come with their own variety of challenges. A great many more people are involved, so there will be more communication, more approvals and more politics. Others will object on the basis of size. "Our company is simply too large!" some will say.

In the final section of our DMARC guide, we will discuss these common concerns and how to address the challenges. If 74% of the US Federal goverment did this in about a year, you can too.

If you just skipped to this section, I'm going to suggest that you read the prior entries in the series before continuing as we won't be rehashing everything that was covered there, only referencing it. They will provide you with a clear picture of the overall DMARC deployment experience while here we will talk about the nuances of an enterprise deployment, including scale, project management, politics and tooling.

All caught up? Then let's continue.

ANNOUNCEMENT: At BSides on October 28th, 2023 I announced the upcoming Q1 2024 private beta of dmarcSTAR, a service that approaches DMARC in what I believe is the ideal manner after over a decade of experience with the specification. Read more at dmarcSTAR.com.

How many email sources are there?

Last week, we talked about DMARC deployment in a startup or smaller organization where you probably know all of the different places you're sending email. Now put yourself in the shoes of an enterprise that's 40+ years old, has hundreds or thousands of servers.

Many servers could be sending email directly. Some departments could have built their own tools while others are using 3rd party services. The company could even have many different domain names in use for all of these different tools. Has the company grown through mergers or acquisition?

There's no telling what the scope of your project could be in a larger company. Luckily, DMARC is built to help with this problem. You start with the Recon step, setting a DMARC p=none; policy on each known domain so that we can gather reports.

Maybe as the project goes on, you discover new domains. Just add that record and start collecting reports. You'll probably want some type of service or tool to collect these for you to help make sense of what is in them...but the deployment process is exactly the same.

We discover services and IP addresses being used by our domains and we work on our implementation to make them both SPF and DKIM compliant. We make them compliant by finding out who has access to these tools, look up the relevant implementation steps for SPF and DKIM, then apply them. Or we consolidate delivery by routing multiple servers through a more centralized and compliant company email service.

Rinse and repeat, until everything is compliant and then we enforce our policy. In an enterprise there will be more communication, more people involved and a bigger project to manage...but it's all achievable.

Selling DMARC in the Enterprise

In an enterprise DMARC deployment there are a lot more people involved. Different departments and department heads, which means internal politics. It's entirely possible that portions of your network may have been delegated to other DNS systems so that they could be managed directly by different departments for the sake of efficiency. You could have delegated all or part of a subdomain to a web application firewall (WAF) provider like Cloudflare. There's more moving parts here behind each email system and that has to be taken into account.

Differences in priorities will color a great deal of the perspective on DMARC within an organization. A lot of experienced people will tell you that security comes last in many organizations. In some cases this is by necessity rather than negligence, as there's often only so much budget to go around and a lot of competition to capture market share. To get DMARC setup, just getting the project approved in the first place will take some doing. Here's a few things to know that will help.

The Department of Homeland Security (DHS) 18-01 Binding Operational Directive required all federal agencies to have a DMARC policy of p=reject by October 16, 2018. 74% of agencies did it in time according to a report from Agari. Similar requirements happened in the UK in 2016. The Dutch government went so far as to add DMARC to their email standards checklist that's part of the "Comply or Explain" list for business operations in the country. Businesses can depart from the compliance requirements, but they must provide a written explanation to justify the course of action. More an more it's becoming a baseline expectation of doing business, particularly with government agencies. Adoption is being pushed heavily across the US, EU, Japan and more.

It's probably only a matter of time before DMARC is folded into security standards like SOC2 Type II and ISO 27001.

Business Email Compromise (BEC) for Executives

The executive team will probably respond to BEC, also known as Spear Phishing or Whale Phishing. BEC scams target financial departments at companies by having those companies issue checks and vendor payment through social engineering. These scams work because the criminals are able to successfully impersonate company executives. Often times an entire fake email chain will be created that appears to be a multi-day conversation between an executive and a fictional vendor wondering why they haven't been paid. The executive will then forward the entire email chain to the finance department to get these people paid promptly. It's believable because, via email, these executives can be impersonated if the protections aren't in place.

You read something like that and your first thought it, "that could never happen" but these scams are responsible for $12.5 billion in losses from 2013-2018 according to the Global Cyber Alliance. How accessible is your CEO for a verification phone call if you were to receive an email like that, particularly from their company email address? It's easy to do without DMARC in place. Even with DMARC in place it can and does still happen, but perpetrators have to convince you they are the same executive but on a personal email account with Gmail or some other provider. It's much easier to spot. Many email and service providers like Greathorn and Google provide tools to help warn people about them...but those tools can't do anything about messages that come from the corporate email domain.

BIMI is a Carrot for the Marketing Department

Is it probably necessary at a large company to get the marketing department on board? Yes. Because marketing might fight you on setting it up. I've spoken about DMARC during an email marketing conference and had the conversations first hand. This will sound harsh, but your marketing department probably does not care at all about email security. What they care about are delivery, inbox placement, open rates and clicks. A DMARC project will likely scare the marketing folks more than anything as they wonder and speculate about a potential negative impact to these metrics. DMARC won't hurt these at all. If anything, it helps protect the brand by making it more difficult to impersonate.

And many will go as far as fighting against the DMARC deployment because of it. After all, what's in it for them? Well, now there's something for them too.

Once DMARC has been rolled out successfully you're eligible to setup Brand Indicators for Message Identification (BIMI), which will allow a trademarked logo to appear next to your emails right in the customer inbox. So far, it's supported by Gmail, Yahoo, AOL and Fastmail.

As a person try to improve security for your company and your users, you may be wondering if this has any positive impact on security by providing some type of reassuring trust indicator next to the email. This, unfortunately is a resounding, "No." Users do not respond to trust indicators because there are so many messages and websites they come across that do not have them. Because of that, users simply ignore missing indicators.

As a parallel you may remember Extended Validation Certificates security certificates. Companies paid a lot of money to go through a validation process that would turn the address bar in the browser green if they'd been validated. These were completely ineffective. The same certificate authorities who validated EV certificates are the Mark Verifying Authorities (MVA) doing the validation for BIMI to get you a Verified Mark Certificate (VMC). A 2018 study presented at M3AAWG also revealed no impact (positive or negative), aside from more email senders contacting email providers asking how they could get the indicators too.

BIMI is the Make My Logo Bigger Cream of email. It's a dangling carrot, but if it gets marketing on board with the project and it does no harm it is probably worth it for the overall security improvement. You still have to fully deploy DMARC first.

Reigning in Shadow IT for Security

One of the biggest struggles that IT and Security teams deal with at large companies is Shadow IT. Shadow IT happens when employees start using devices, tools and services that have not been approved by the company to meet with various internal standards and compliance.

Now, it's important to remember that Shadow IT rarely comes from malicious intent. The cause is traditionally an expected very high barrier to approval, numerous hoops to jump through or long timelines from IT itself which encourage people to go around IT to get their jobs done. Shadow IT is often a symptom of a process problem.

All that said, DMARC reports provide real insight on the scope of this problem within your organization. If you start collecting reports with dozens of legitimate 3rd party services that haven't been approved...you have a problem.

Does private data exist within these tools? Employee data? Customer data? Is PII collected? How are logins managed? Is access being removed during employee turnover? Are their legal or audit implications?

By knowing the tools exist, you gain the ability to get their usage documented, approved, secured properly and rolled into more polished company process. If you're unable to find out who's responsible you can always use the time honored IT sleuthing process of unplugging it until somebody complains. Enforcing your DMARC policy will have the same effect.

Scale and Subdomains

There's no scale that DMARC can't handle. Almost the entire US federal government has adopted it at this point as well as half of the Fortune 100. We've already established how easy this is in smaller organizations too.

Scale does come with it's own implementation challenges, so let's start with the most well known.

Overcome the SPF 10 Lookup Limit

When a mail server tries to verify that a message passes SPF, it contacts the domain DNS to look for the SPF record and checks to see if the IP address of the server that sent the message exists within the record. This could mean directly listed IP addresses, ranges of IP address or include: entries which list another domain name that contains another list of IPs.

Every time another DNS query must be performed, it adds to the "lookup count". If you're using Google Workspace and included _spf.google.com in your top level SPF record you may think that's only 1 lookup. By querying that record though, we will see that it contains 3 additional lookups.

"v=spf1 include:_netblocks.google.com include:_netblocks2.google.com include:_netblocks3.google.com ~all"

These all consist of IP address ranges without any additional look ups. Here's _netblocks3.google.com as an example.

_netblocks3.google.com.    300    IN    TXT    "v=spf1 ip4:172.217.0.0/19 ip4:172.217.32.0/20 ip4:172.217.128.0/19 ip4:172.217.160.0/20 ip4:172.217.192.0/19 ip4:172.253.56.0/21 ip4:172.253.112.0/20 ip4:108.177.96.0/19 ip4:35.191.0.0/16 ip4:130.211.0.0/22 ~all"

That means that by including _spf.google.com we've added 4 lookups to our SPF record and that's only for our corporate email account! What happens when we start adding all of those other services?

Exactly. You'll quickly go over the 10 lookup limit if everything gets added to a single SPF record. Which is why you aren't supposed to have a single SPF record. Each subdomain can and should have it's own SPF record. If a record isn't present at the subdomain then the lookup will bubble up to the root domain.

The easiest way to think about this problem is in terms of what services are being performed. Let's use our pirate gym, Slimmer Ye Timbers as an example.

  • newsletter.slimmeryetimbers.com - Marketing newsletters
  • invoice.slimmeryetimbers.com - Invoicing systems
  • classes.slimmeryetimbers.com - Class schedules and reminders
  • receipt.slimmeryetimbers.com - Transaction receipts
  • it.slimmeryetimbers.com - IT department systems
  • hr.slimmeryetimbers.com - HR department and tooling systems
  • sales.slimmeryetimbers.com - Sales tools like CRM systems
  • mail.slimmeryetimbers.com - General website emails

Any of these could be further broken down if needed. The point is that there's no need to try to cram everything into a single top level SPF record. As a bonus, you gain a number of benefits by using subdomains.

SPF Isolation and Error Containment

One thing many people don't realize is that a broken SPF record will invalidate your entire DMARC policy. If mail servers see you have an enforced DMARC policy setup, but also see that your SPF record is broken the error will force them to act as if you have no DMARC policy at all.

By isolating each SPF record on its own subdomain, you contain the damage from a broken record to only that subdomain.

You may be wondering how an SPF record could break after it's been setup correctly? One of the dangers of include: is the potential for change by the vendor.

Just as an example from 2012, we were using a vendor to manage sales channel communications. Not knowing any better, we added their include: record to our top level SPF and everything was working great. We had a fully implemented p=reject; policy and our phishing complaints had basically stopped at that point.

One day, all the phishing complaints started back again suddenly. And worse yet, the phishing emails were using our domain again. We hadn't touched anything with the email system so we couldn't understand what happened.

After inspecting all of our records, the Kitterman SPF Testing Tool helped us discover that our SPF record was broken. The sales channel communication company switched which domain they were using for include records and the old domain was no longer valid! This change invalidated our entire SPF record and the rest of our email policy along with it.

If the CRM tool had been on its own subdomain, only that tool would have been affected by a change outside of our control. We didn't know about the change because the email notification about the change only went to the person at the company who signed up for it in the first place. And they didn't pass it along.

The above scenario also underscores the benefit of long term monitoring for changes in your SPF records.

You don't need "SPF Flattening"

There are tools out there which provide a service called "SPF Flattening", which monitors your SPF record for changes and compresses it into the minimum possible number of SPF lookups. You have to pay for the service and it can still outgrow the 10 SPF lookup limit.

You don't need these services. You need subdomains.

No single service can function if it alone would outgrow the SPF lookup limit. By simply isolating each service on its own subdomain, you have "limit proofed" it against this problem.

The only potential benefit to an SPF flattening service is error insulation from 3rd party changes. If the flattening service does error detection against changes in the SPF record and refuses to publish any change that would cause an error, your tooling would be insulated against those 3rd party errors. That's a good thing.

However, it's only a short lived protection. Eventually the error has to be resolved or the flattened records may no longer be valid. A decent flattening service should notify you of these types of errors so that you can correct them.

If an error preventing cache buffer sounds worth the cost to you, then maybe it's worth it. Don't depend on the SPF flattener to prevent your longer term lookup limit issues. You're still better off with subdomains even when using a flattening service.

Return-Path and Subdomains

Both SPF and DKIM have an option in your DMARC configuration to allow both "strict" and "relaxed" (default) mode. In relaxed mode, the Return-Path used to send email bounces can use a subdomain while still showing the visible From: domain as only the primary root domain. There are examples included in the DMARC specification.

Most servers will configure themselves in the Return-Path by default, so if you have 1,000 servers like web1.slimmeryetimbers.com or api3. or mail7. or staging3. they can also still send DMARC aligned emails for the root domain, @slimmeryetimbers.com while using subdomains. An SPF entry referencing that server IP on the web1. subdomain will allow messages from that domain to pass an aligned SPF check without actually having to send email From:[email protected] which might not be visibly appealing.

This illustrates why, no matter how big your organization may be, no matter how many servers you have, subdomains will scale with you without growing your total lookup limit.

You could certainly reorganize the above example for simpler management by publishing them under a grouped subdomain, like web1.it.slimmeryetimbers.com. Then an SPF record could be posted on it.slimmeryetimbers.com with an IP range including the scope of the servers underneath it rather than adding and removing individual IP addresses.

Subdomains for Project Scope

One thing that people often forget is that subdomains themselves are just like the primary domain. Each subdomain can have it's own _dmarc. entry if you want. It can have it's own set of DKIM keys under _domainkey. as well.

Which means you can have a different email address to collect aggregate reports from DMARC report providers on each subdomain, just as if another domain entirely was being used.

And that means that your organization can have multiple DMARC deployment projects if needed. You can let HR roll out DMARC on it's own schedule while the rest of the company gets setup for p=reject; on the main project, for example.

Invisible Email Servers

In some circumstances, it's possible for an email server to be completely hidden from DMARC reports. This can happen if you have a server which only sends mail to servers that do not provide DMARC reports, like Microsoft Office365 and Proofpoint.

If you visit dmarcian's public DMARC Data Providers report, you can inspect the volume of reports sent by different email providers. You'll notice the absence both Microsoft and Proofpoint if you search the results. The fact that these two companies play such a huge role in the global email space while refusing to participate in one of the most important aspects of its security needs to be called out more often.

Sean Whalen, a consultant, wrote about both of these issues in detail, including workaround guides. Proofpoint initially tried to force customers to pay for their Email Fraud Defense service just to get aggregate reports from their own email gateway, but have since rolled the option into Proofpoint Essentials as a setting for an administrator to turn on. Microsoft claims to support sending reports as a Public Preview feature, but none were visible in dmarcian's public report. DMARC is a decade old and widely adopted at this point. There's really no excuse for the actions of these two.

But there are consequences. If your company is using an email system that doesn't provide reports and you have an email system that only sends emails internally, it won't show up on the reports at all. So it's invisible.

Sean came up with a way to work around this situation with Proofpoint in his above article:

As a less than ideal workaround for this problem, Proofpoint customers can create a Policy Route that matches on message From headers that end with their domains, and then create a DMARC policy in Proofpoint that applies to that route, and configure the policy to copy any messages that fail DMARC to a separate quarantine folder for later review. That way, they can at least get samples of the emails that failed DMARC, even though they won’t show up in third party analytics.

Ulrich Baum, a DMARC consultant from RedSift shared a client story on LinkedIn about this exact situation.

They couldn't identify all sender's, and missed a pet project of a board member which couldn't send for a couple days, which lead to the entire DMARC project being cancelled.

The good news is that a situation like this is only likely to happen with internal projects as described above. Email sent to customers will likely be delivered to a variety of mail servers, some of which will provide proper reports. Sean's workaround is an ideal way to expose this problem on your own email gateways.

International Rules, GDPR and PII

Large organizations also face global compliance challenges. If you're operating in Europe, you must be conscious of GDPR. There are a number of EU based companies that don't want any of their data to even travel overseas.

If that's a concern, you can send the DMARC reports for the domains or subdomains that apply to different regions to email addresses that deliver to servers only in those regions.

In some cases IP addresses are even considered personal information (PII) for purposes of GDPR style privacy regulations. The cases are subjective though. For example, in the hands of an Internet Service Provider who can definitively associate an IP address with a particular person an IP address is considered PII. If you are not an ISP and cannot do that, an IP address is not PII so you can disregard it for those purposes.

If you have concerns about this, consult an attorney because I Am Not A Lawyer. The case material seems fairly clear though.

There are exceptions for using this information for security purposes, such as blocking abusive IP addresses, etc. If your use case is along those lines, you likely have nothing to worry about. The IP addresses contained within a DMARC report are all either associated with your business or fraudulently impersonating your business, which leaves no real concerns.

Forensic/Failure Reports and PII

One aspect of the DMARC specification that we haven't discussed at all to this point are failure reports, which have sometimes been called forensic reports. These are an optional aspect of the DMARC specification where an ruf entry can be added to your DMARC record with an email address to receive these reports.

Failure entries will provide more specifics about messages which we see that didn't pass our DMARC policy, such as specific mail headers. That's probably going to be a lot of mail since it's going to include everything, including the mail that isn't being sent by us.

Because of that, there are concerns about PII contained within these mail samples. Many DMARC report providers won't even bother sending these because of those PII concerns. The format and structure of the messages are all over the place too.

In my experience, there's very little benefit trying to harvest these especially between the PII concerns, report inconsistency and signal to noise ratio.

Project and Process Management

A big aspect of DMARC rollout in an enterprise is the management of the project itself. Who's communicating with people responsible for different services? Who's monitoring progress? Who's reporting on roadblocks and providing education where needed?

It could be you. It could be another employee within the company. Because DMARC Deployment itself is a relatively short term project, 3-6 months for most enterprises, you may be better off hiring a consultant to drive the project forward rather than sidetracking a member of your staff. There are number of companies out there, like dmarcian who offer these types of consulting services. I consult on a number of different issues, so feel free to reach out to me on LinkedIn if you'd like to discuss a DMARC deployment project.

Longer term, adapting internal processes to maintain DMARC compliance within the organization is the real goal though. As we discussed last week, that means getting to p=reject; to ensure that nothing works without being compliant. Service provisioning channels within your organization should be adapted to make sure DMARC compliance is included as well.

Wrapping Up

There are additional challenges that could create difficulties with your deployment, but we've addressed the most common questions here today. Hopefully our 3 part DMARC series has pulled back the curtain on the complexity while encouraging you to move forward with better email security!

ANNOUNCEMENT: At BSides on October 28th, 2023 I announced the upcoming Q1 2024 private beta of dmarcSTAR, a service that approaches DMARC in what I believe is the ideal approach after over 10 years working with the specification. Read more and signup for updates at dmarcSTAR.com.