Proofpoint with Exchange Server: connectors, TLS and the rejections admins see

With Proofpoint in front of Exchange Server, a rejection can come from either product, and the fix depends on which one wrote it. This post covers the Exchange side of Proofpoint Essentials and Proofpoint Enterprise (hosted or on-premises): connectors, recipient lists, TLS, the rejections you'll meet in each direction, and how to trace one message across both logs. Commands are for Exchange Server SE and 2019.

Symptoms#

Match the text to the product that wrote it:

The Essentials wordings are reported by admins and in Proofpoint partners' support articles rather than in Proofpoint's public documentation, and those partner articles disagree on whether Relay access denied comes with 550 or 554. Proofpoint Enterprise can word its responses differently, so copy the exact text from the bounce or the message log. Anything containing RESOLVER, SMTP; or TlsCertificateName is Exchange talking.

Why it happens#

Proofpoint and Exchange meet at four seams, and every rejection belongs to one of them:

• Who may connect. Exchange must accept Proofpoint's delivery addresses, and Proofpoint must recognise your servers' public egress IP before relaying outbound mail. A firewall or NAT change on either side quietly breaks this.
• Who exists. Proofpoint keeps its own list of valid recipients, from directory sync or, in Essentials, SMTP Discovery. A stale list rejects real users ("marked as invalid"). A loose one lets mail for nonexistent users through, and an Exchange Mailbox server without the Recipient Filter agent accepts those recipients and then sends the RecipNotFound NDR, often to a forged sender.
• Which domains. Every domain Proofpoint delivers must be an accepted domain in Exchange (otherwise 5.7.54). If a domain is InternalRelay and Proofpoint routes it back to Exchange, unknown recipients loop until 554 5.4.6.
• TLS and capacity. Connectors still point at a replaced certificate (454 4.7.5), and Proofpoint's parallel deliveries hit Exchange's per-source connection limit (421 4.3.2) or arrive during back pressure (452 4.3.1).

How to fix it#

1. Decide which side rejected the message#

Get the Message-ID and the exact response first. Outbound problems show in the queue for the smart host connector:

Get-TransportService | Get-Queue | Where-Object { $_.DeliveryType -eq 'SmartHostConnectorDelivery' } |
    Format-List Identity, Status, MessageCount, LastError

Inbound, search Proofpoint's message log for the recipient. If Proofpoint never tried to deliver, the cause is in Proofpoint (policy, recipient list, quarantine). If it tried and quotes an Exchange reply, fix Exchange.

2. Lock the inbound path to Proofpoint#

New-ReceiveConnector -Name "Inbound from Proofpoint" -Server EX01 -TransportRole FrontendTransport -Custom `
    -Bindings 0.0.0.0:25 -RemoteIPRanges 203.0.113.0/26,198.51.100.0/24 `
    -PermissionGroups AnonymousUsers -AuthMechanism Tls -ProtocolLoggingLevel Verbose
Set-ReceiveConnector "EX01\Inbound from Proofpoint" -MaxInboundConnectionPerSource 100
Get-AcceptedDomain | Format-Table DomainName, DomainType -AutoSize

Use the delivery ranges Proofpoint documents for your Essentials stack or Enterprise data center (or your appliances' addresses), create the connector on every Mailbox server Proofpoint delivers to, and allow TCP 25 only from those ranges at the firewall. Keep AnonymousUsers rather than "Externally secured"; that choice, InternalSMTPServers and the header firewall are covered in the vendor-neutral gateway checklist. Add any missing accepted domain, make fully hosted domains Authoritative, and for 452 4.3.1 fix the resource Exchange is short of (back pressure guide).

3. Keep Proofpoint's recipient list true#

• Prefer directory-based provisioning (Active Directory or LDAP sync) to learning addresses from mail flow, so new mailboxes, aliases and groups are valid on day one.
• In Essentials, an address on SMTP Discovery's invalid list is rejected with "marked as invalid" even after the mailbox exists. Remove it from that list.
• Don't make Exchange reject unknown recipients with the Recipient Filter agent instead: Microsoft advises against configuring it on Mailbox servers.

4. Let Exchange relay outbound through Proofpoint#

New-SendConnector -Name "Outbound via Proofpoint" -AddressSpaces "*" -DNSRoutingEnabled $false `
    -SmartHosts "<outbound host from Proofpoint>" -SourceTransportServers EX01,EX02
Test-NetConnection "<outbound host from Proofpoint>" -Port 25

Then make Proofpoint recognise you. In Essentials, enable outbound relaying and register the public IP addresses your Exchange servers send from; in Enterprise, have them added to the outbound relay configuration. Proofpoint sees your post-NAT address, so check its log for the IP Exchange really connects from; an unregistered address is a common cause of Relay access denied. If your service authenticates relays instead, use -SmartHostAuthMechanism BasicAuthRequireTLS with -AuthenticationCredential. Add the SPF include Proofpoint gives you before you switch.

5. Fix TLS on both legs#

Inbound, Proofpoint can only use TLS if Exchange offers STARTTLS with a valid certificate for the connector FQDN; check for event ID 12014 and the expiry date. Outbound, 454 4.7.5 means TlsCertificateName names a certificate that's gone, usually after a renewal:

Get-ExchangeCertificate | Format-Table Thumbprint, Services, NotAfter, Subject -AutoSize
$cert = Get-ExchangeCertificate -Thumbprint "<new thumbprint>"
Set-SendConnector "Outbound via Proofpoint" -TlsCertificateName "<I>$($cert.Issuer)<S>$($cert.Subject)"
Retry-Queue -Server EX01 -Filter "Status -eq 'Retry'"

Verify the fix#

$id = "<Message-ID from the Proofpoint log>"
Get-TransportService | Get-MessageTrackingLog -MessageId $id -Start (Get-Date).AddHours(-2) |
    Sort-Object Timestamp | Format-Table Timestamp, EventId, Source, ConnectorId, RecipientStatus -AutoSize

• Inbound: Proofpoint's log shows delivery to Exchange accepted with a 250 reply, and Exchange shows RECEIVE (SMTP) then DELIVER. The receive protocol log shows the session on Inbound from Proofpoint with STARTTLS.
• Outbound: Exchange shows a SEND through Outbound via Proofpoint, the smart host queue is empty, and Proofpoint logs the message arriving from your registered IP.
• A test to a nonexistent address is rejected by Proofpoint, not NDR'd later by Exchange.

Prevent it next time#

• Add a new domain to Proofpoint and to Exchange's accepted domains in the same change.
• Keep Proofpoint's ranges in sync across the receive connector, InternalSMTPServers and the firewall, and register new egress IPs before any NAT change.
• After every certificate renewal, update TlsCertificateName on your connectors.
• Alert on the smart host queue: growth there is the first sign Proofpoint is refusing Exchange.
• Other gateways: Mimecast, Cisco Secure Email Gateway and the rejection codes quick reference.

References#

• Receive connectors in Exchange Server (Microsoft Learn)
• Enable antispam functionality on Mailbox servers (Microsoft Learn)
• Microsoft 365 Integration (MX Deployment) (Proofpoint Essentials documentation; written for Microsoft 365, shows Essentials as the inbound and outbound gateway)