SBC mTLS: Mutual TLS Authentication for SIP Signaling

Standard TLS proves that your Session Border Controller is the device its hostname claims it is. It does not prove anything about the device on the other end of the connection. For an internet-facing SIP edge, that is the wrong half of the problem to solve. Anyone on the public internet can open a TCP socket to port 5061, complete a one-way TLS handshake against your certificate, and start sending INVITE messages. TLS protects the channel; it does not authenticate the peer.

Mutual TLS, usually written mTLS, closes that gap. Both endpoints present X.509 certificates during the handshake, and both endpoints validate the certificate they receive against their own trust policy. If the peer does not present a certificate, or presents one your SBC does not trust, the handshake fails before a single SIP byte is exchanged. This is the mechanism most Tier 1 carriers expect on interconnects, and the mechanism that is steadily replacing IP allowlists as the default trust boundary between SIP peers.

This guide goes deeper than the high-level definition. It walks through what actually happens during the mTLS handshake, the difference between a truststore and a keystore (one of the most consistently confused points in real deployments), the four trust models you can choose between, why trust between two peers is almost never symmetric, how revocation checking works in practice, and the failure modes that are specific to mTLS rather than to TLS in general. For the per-trunk TLS and SRTP configuration walkthrough that sits underneath all of this, see the SBC TLS and SRTP Configuration Guide.

What One-Way TLS Leaves Open

In a standard server-authenticated TLS handshake, only one side proves its identity. When a SIP client opens a TLS connection to your SBC on port 5061, the SBC presents its certificate, the client validates that certificate against its truststore, and the encrypted tunnel is established. The client never has to prove who it is. From the SBC’s perspective, any device that completes the handshake gets to start sending SIP messages.

That works for the consumer web, where the server is the asset worth protecting and the client is a browser whose identity is enforced higher up the stack (usernames, passwords, cookies). It does not work for a SIP edge, where the server is the asset worth protecting and the client is another network element with no human at the keyboard. SIP digest authentication exists, but it runs above the TLS layer and is not a substitute for authenticating the host device.

The practical risk is straightforward. An attacker scans the internet, finds your SBC on port 5061, opens a TLS connection, and starts probing. They cannot decrypt your existing traffic, but they can now start SIP calls of their own. Whether those calls complete depends entirely on whatever authentication, allowlisting, and policy you have configured above TLS. mTLS pushes the authentication boundary down into the handshake itself, so the attacker’s connection is rejected before the SIP stack ever sees it.

How the mTLS Handshake Actually Works

The server requests a certificate from the client, the client provides one, and both sides validate what they receive before any application data flows.

1. The ClientHello opens the connection and offers the client’s supported TLS versions, cipher suites, and extensions. No certificate is exchanged yet.
2. The ServerHello and Server Certificate follow once the SBC has selected a cipher suite, and the SBC presents its own certificate chain at this stage. This is the same step you see in one-way TLS.
3. A CertificateRequest is sent instead of finishing the handshake, asking the client to present its own certificate. The message includes a list of acceptable Certificate Authorities so the client knows which of its certificates to choose.
4. The client Certificate and CertificateVerify messages come back together: the client returns its certificate (and the intermediate chain) along with a CertificateVerify message containing a signature over the handshake transcript, made with the private key that matches the certificate. This proves the client actually holds the key, not just a copy of someone else’s certificate.
5. The mutual validation step happens on both sides at once. The SBC validates the client’s chain against its truststore, checks the validity period, verifies the CertificateVerify signature, and (if configured) performs a revocation check. The client performs the same validation against the SBC’s certificate. Either failure aborts the handshake with a TLS alert.
6. The Finished messages are exchanged on both sides, the encrypted application channel opens, and SIP traffic begins.

Two implications matter operationally. First, mTLS is bidirectional by construction: a misconfiguration on either side breaks the connection, and the error surface lives in the TLS layer, not the SIP layer. Tools like a SIP trace will tell you the call never started; only a TLS-level capture (or the SBC’s TLS error log) will tell you why. Second, every successful call on an mTLS-protected trunk has already proven, at the handshake stage, that the peer holds a private key matching a certificate your SBC chose to trust. That is a far stronger statement than “the source IP is on the allowlist.”

The Four Pieces of an mTLS Configuration

Most operational confusion about mTLS comes from collapsing distinct concepts into one. There are four independent pieces, and each lives in a different part of the SBC configuration.

Server identity is the SBC’s own certificate and matching private key, used when the SBC acts as the TLS server (a peer is connecting inbound to port 5061). The SAN must include every FQDN inbound peers will use to reach you. For multi-tenant deployments a wildcard or multi-SAN certificate is common; see the multi-tenant SBC for Teams Direct Routing guide for the wildcard-certificate pattern Microsoft expects.

Client identity is the certificate the SBC presents when it acts as the TLS client (the SBC is initiating an outbound TLS connection to a carrier or to Teams). This is often the same certificate as the server identity on smaller deployments, and a different certificate on larger ones where outbound traffic uses a dedicated identity. Either way, it must be stored as a certificate plus matching private key, not just a certificate file.

Truststore covers which Certificate Authorities (or individual peer certificates) the SBC will accept when validating a peer. This is independent of your own identity. A truststore that contains “any public CA the operating system trusts” is far too permissive for SIP, because it authorises every Let’s Encrypt and DigiCert certificate on the internet to talk to you. A correct SIP truststore lists only the CAs that issue certificates to your actual peers.

Per-trunk policy ties the three above to specific SIP trunks.

The keystore-versus-truststore confusion is worth understanding because it produces a specific class of failure. If the SBC’s own certificate and private key are filed in the truststore instead of the keystore, the SBC has no identity to present: the TLS stack typically fails to bring the listener up at all, or aborts the handshake with a generic handshake_failure (alert 40) before any certificate is exchanged. If mTLS is configured and a peer’s CA is filed in the keystore instead of the truststore, the SBC will not trust the certificate that peer presents during client authentication, and the handshake fails with unknown_ca (alert 48).

Choosing a Trust Model

Once you have decided to require client authentication, you have a separate decision about which authorities issue the certificates you will accept. There are four practical options, and the right one depends on who the peer is and how much control you have over their PKI.

The pattern most likely to produce surprises is the broad public-CA-root approach. Putting “DigiCert Global Root G2” in your truststore means the SBC will accept any certificate issued by DigiCert for any subject, including any unintended actors. The defence is to layer FQDN validation on top of CA validation: even if the certificate chains to a trusted root, reject it unless the SAN matches the FQDN the trunk is supposed to be talking to.

The Asymmetric Trust Problem – the Microsoft Teams example

Trust between two peers in mTLS is rarely symmetric, and the documentation almost never says so explicitly. Your SBC trusts certificates issued by a specific set of CAs; the peer trusts certificates issued by a different (usually overlapping but not identical) set. Both lists matter, both can be wrong independently, and a connection works only when both lists accept the certificate the other side presents.

One such implementation gaining in popularity is Microsoft’s Teams Direct Routing. Microsoft’s truststore on the Teams side accepts certificates from a published list of public CAs (the list is updated periodically, most recently for the June 2026 root CA refresh). Your SBC, meanwhile, has to trust the CA that signs Microsoft’s own certificates on the Teams interface (currently DigiCert Global Root G2, soon also Microsoft RSA Root Certificate Authority 2017). If you update only the SBC side or only the Teams side, the trunk breaks. If you forget the new Microsoft root entirely, call establishment will no longer be possible. Asymmetric trust is the reason the Teams certificate change is being announced months in advance.

The operational discipline that handles this well is to think of mTLS trust per direction rather than per trunk. For each direction (inbound from peer, outbound to peer), keep a record of which CAs the SBC trusts, which CAs the peer trusts, when those CAs roll, and who is responsible for telling you when they do. Once you have that, the “TLS handshake started failing overnight” incident becomes a routine ticket instead of an outage.

Revocation: CRL, OCSP, and the Soft-Fail Choice

A certificate’s validity period tells you when it expires naturally. It does not tell you whether the CA has revoked it early because of compromise, contract termination, or rekeying. Revocation is enforced through one of two mechanisms.

CRL (Certificate Revocation List) is a signed file published by the CA listing every revoked certificate’s serial number. CRLs are simple and reliable, but they can grow large and the freshness depends on the polling interval.

OCSP (Online Certificate Status Protocol) is a real-time query the SBC sends to the CA’s OCSP responder asking specifically “is this certificate still valid?” The response is signed and short-lived. OCSP stapling lets the peer fetch its own OCSP response in advance and include it inside the TLS handshake, so the SBC does not need to make its own query at all. Stapling is the cleanest pattern when the peer supports it.

The decision that matters more than CRL versus OCSP is the soft-fail versus hard-fail policy. If the OCSP responder is unreachable or the CRL distribution point times out, do you accept the connection (soft-fail) or reject it (hard-fail)? Hard-fail is more secure, because a network attacker cannot block revocation checks to keep a compromised certificate alive. Soft-fail is more available, because a CA-side outage does not take your voice service down with it. Most carrier interconnects run soft-fail; high-security deployments and STIR/SHAKEN signing flows run hard-fail. Whichever you pick, pick it deliberately, and monitor revocation-check failures so a silent soft-fail does not hide a real problem.

Rotating Certificates Without Dropping Calls

Every certificate eventually expires. The rotation problem is harder for mTLS than for one-way TLS, because both peers have to rotate in coordination: when you replace your client certificate, every peer that has pinned your old certificate has to add the new one to its truststore before you cut over, and when a peer rotates, you have to add the new certificate before they cut over. Forget either step and the trunk goes silent.

The clean pattern is the dual-trust window. For at least 30 days before a rotation, both the old and the new certificate (or both the old and the new issuing CA) are trusted on both sides. The peer presents the new certificate as soon as it is provisioned; the SBC accepts it because the new CA is already in the truststore. Once every peer has confirmed it is presenting the new certificate, the old certificate is removed from the truststore. This avoids any all-or-nothing moment.

For SBC-presented certificates, the same pattern applies in reverse. Provision the new certificate alongside the old one, switch each trunk to the new certificate during a low-traffic window, monitor for handshake failures from peers that have not updated their truststore, and remove the old certificate only after every trunk has been cut over cleanly. Some SBCs (ProSBC included) let you stage a new certificate and bind it to a trunk without a service restart, so the rotation happens at the connection level rather than the process level.

Calendar discipline matters more than the mechanism. Track expiry dates per certificate, alert at 90, 60, and 30 days, and treat any certificate inside 14 days of expiry as an active incident. The number of TLS-handshake-failure incidents that turn out to be “we forgot the certificate expired today” is depressingly high.

mTLS-Specific Failure Modes

Most TLS troubleshooting guides cover handshake failures generically. The failures listed below are the ones that occur specifically because mTLS requires both sides to agree, and they look different from standard TLS errors.

A “no certificate available” alert means the peer’s client identity is missing or unreadable. On a fresh deployment this usually means the certificate was uploaded without the matching private key, or the private key file’s permissions block the SBC process from reading it. The peer’s TLS log will show the CertificateRequest arriving and an empty Certificate message going back.

An “unknown CA” alert covers the case where the certificate the peer presented chains to a CA the SBC does not trust. The fix is almost always to add the missing intermediate or root to the truststore, not to weaken the validation policy. If you do not recognise the issuing CA, do not trust it.

A “certificate verify failed” alert means the CertificateVerify signature did not match the certificate’s public key. This is rarer and usually happens because the peer is presenting a certificate whose private key it does not actually hold (often because someone copied a certificate file between hosts but not its key). Treat it as a security event, not a configuration drift.

An FQDN mismatch despite a valid chain means the chain validates and the certificate is in date, but the SAN does not include the FQDN your SBC is dialling. This is the failure mode that proves you are checking the SAN, which is correct. The peer needs a certificate with the right SAN, not a broader truststore on your side.

The classic “worked yesterday, broken today, no config change” symptom almost always traces back to a certificate that expired, a CRL the SBC could not refresh, or a CA root the SBC trusts that has been removed from the peer’s truststore (the Microsoft Teams case is the canonical example). Check expiry dates and revocation-source reachability before anything else.

A one-way handshake success happens when the SBC’s certificate is trusted by the peer, but the SBC rejects the peer’s certificate. The trunk works in one direction and fails in the other. This is the asymmetric-trust failure from the previous section, and it almost always means a missing CA on one side.

Where mTLS Belongs in a Layered Defence

mTLS authenticates the connection. It does not, on its own, address most of the other threats at the SIP edge. A complete SBC security posture treats mTLS as one of several layers.

Above mTLS, the SBC still has to police SIP-layer behaviour: rate limits on INVITEs, REGISTERs, and OPTIONS, dynamic blacklisting for sources that exceed thresholds, and protection against malformed messages. A peer that completes mTLS successfully and then floods you with INVITEs is authenticated, but still abusive.

Beside mTLS, the media path needs its own encryption. mTLS protects SIP signaling. The actual RTP media flows over UDP and is protected by SRTP, with keys exchanged either in SDES (inside the TLS-protected signaling) or via DTLS-SRTP (handshake on the media path itself). Encrypting only the signaling is a common mistake; SDES keys travel in the signaling, so unencrypted signaling defeats SRTP too.

Below mTLS, network controls still matter. IP allowlists are weaker than cryptographic authentication, but they cut the noise floor by keeping random internet traffic away from the TLS stack. The combination of “IP must be allowlisted and certificate must be trusted” is significantly stronger than either alone, and both fail closed.

Monitoring and Alerting Specific to mTLS

Generic SIP monitoring tells you when calls fail. mTLS-aware monitoring tells you why a TLS-protected trunk failed before the call was ever attempted, and gives you advance warning before the next certificate expires.

Four signals are worth alerting on specifically. The first is peer certificate expiry per trunk, surfaced as days remaining. This is the only way to catch a peer who is about to roll a certificate without telling you, and it has to be measured by reading the certificate the peer actually presents, not by trusting the contract documentation. The second is handshake-failure rate per peer, broken down by TLS alert reason. A sudden spike of “unknown CA” alerts from one peer almost always means they rotated their CA without coordinating. The third is revocation-check failure rate, separate from handshake-failure rate. A soft-fail policy will hide CRL/OCSP outages inside successful handshakes, so the failure rate is the only way to see them. The fourth is your own certificate’s days-to-expiry, with alerts at 90, 60, 30, and 14 days. Renewing in advance is straightforward; renewing at expiry-minus-one is a call-affecting incident.

For service providers running this at scale, Monitoring as a Service can fold certificate-expiry, handshake-failure, and revocation-failure signals into the same dashboard as call quality metrics, with email, Slack, or Teams alerting. Doing it well in-house is also feasible if you commit to per-trunk certificate inspection rather than per-server polling.

Use Cases Beyond Microsoft Teams

Teams Direct Routing is the use case that put mTLS on most SBC operators’ radar, but it is not the only one and it should not be the only one your design considers.

Carrier interconnects are increasingly built on mTLS rather than IP allowlists. Tier 1 carriers have published mTLS-required peering for several years; mid-tier carriers have started following. The advantage for both sides is that a certificate-bound trunk survives an IP renumbering with a configuration change rather than a security review.

Contact-center BYOC deployments (Genesys, Five9, NICE, Talkdesk on an operator’s own SIP trunks) use mTLS to authenticate the SBC to the CCaaS platform without relying on shared IP infrastructure. The CPaaS and BYOC pattern is functionally similar to Teams Direct Routing minus the Microsoft-specific requirements.

B2B SIP federation between two enterprises, or between an enterprise and a UCaaS platform, is a natural fit for mTLS because both sides have a known identity and a stable FQDN. This is where pinning or a private CA tends to work better than a public CA, because the certificate is not visible to anyone outside the relationship.

Multi-tenant MSP deployments use mTLS plus a wildcard certificate to authenticate each customer tenant’s traffic on a shared SBC. The asymmetric-trust discipline matters most here, because the failure of one tenant’s certificate must not affect the others.

Regulated industries (healthcare, financial services, government) treat mTLS as a baseline rather than a feature, often paired with hard-fail revocation, private CAs, and certificate-pinning for the most sensitive connections. The compliance frameworks rarely mandate mTLS by name, but the controls they do mandate (mutual authentication, key custody, revocation enforcement) add up to it in practice.

mTLS on ProSBC

ProSBC handles all four configuration pieces (server identity, client identity, truststore, per-trunk policy) through its standard certificate management and Network Access Point (NAP) configuration. The relevant settings live alongside the per-trunk TLS configuration covered in the SBC TLS and SRTP configuration guide, so this section focuses on what is mTLS-specific rather than re-walking the per-trunk TLS setup.

The truststore can hold any combination of public CAs, private CAs, and pinned peer certificates, with per-trunk selection of which entries are valid for which NAP. This avoids the broad “trust every public CA” failure mode by letting different trunks accept different authorities. For example, root CAs and a private internal CA can coexist on the same SBC without one weakening the other.

Each NAP carries its own TLS policy with the four standard settings (no TLS, TLS without client auth, TLS with optional client auth, TLS with required client auth), so the same SBC can serve a strict-mTLS carrier interconnect, an optional-mTLS migration window, and a one-way-TLS legacy trunk simultaneously. Per-NAP policy also makes it possible to enforce different revocation policies per trunk if needed.

For service providers and MSPs without dedicated PKI staff, the ProSBC Managed Service includes certificate-lifecycle support as part of the Level 3 engineering coverage, which removes the rotation-coordination burden from the operator. For self-hosted deployments, the ProSBC Lab license is enough to validate an mTLS configuration end-to-end against a Teams test tenant or a sandbox carrier before deploying in production.

Frequently Asked Questions