TLS Certificates, Hostname Validation, Internal Cluster Authentication, and Key Rotation

Learning objectives

1. TLS protects the channel only when identity validation remains enabled

Transport Layer Security (TLS) encrypts traffic and lets a client verify the server certificate. A Certificate Authority (CA) signs the server certificate. The certificate's Subject Alternative Name (SAN) should contain the DNS name or IP address clients actually use. A connection that merely encrypts bytes but accepts the wrong hostname can still be redirected to an unintended server.

2. Build a local CA and a server certificate whose SAN says exactly localhost

The certificate below is synthetic and valid only for a short-lived learning environment. Production deployments should use an organizational or trusted CA, protect private keys, issue per-host certificates, monitor expiration/revocation, and automate replacement. The deliberate SAN choice gives us a clean negative test: localhost is valid; a different name is not.

rm -rf ch22-l3-certs && mkdir ch22-l3-certsopenssl req -x509 -newkey rsa:2048 -sha256 -nodes -days 7 \  -subj "/CN=AtlasMart Chapter 22 Lab CA" \  -keyout ch22-l3-certs/ca.key -out ch22-l3-certs/ca.crtcat > ch22-l3-certs/server.cnf <<'EOF'[req]prompt = nodistinguished_name = dnreq_extensions = req_ext[dn]CN = localhostO = AtlasMart Lab[req_ext]subjectAltName = @alt[alt]DNS.1 = localhostEOFopenssl req -newkey rsa:2048 -nodes -keyout ch22-l3-certs/server.key \  -out ch22-l3-certs/server.csr -config ch22-l3-certs/server.cnfopenssl x509 -req -in ch22-l3-certs/server.csr \  -CA ch22-l3-certs/ca.crt -CAkey ch22-l3-certs/ca.key -CAcreateserial \  -days 7 -sha256 -extfile ch22-l3-certs/server.cnf -extensions req_ext \  -out ch22-l3-certs/server.crtcat ch22-l3-certs/server.key ch22-l3-certs/server.crt > ch22-l3-certs/server.pemopenssl x509 -in ch22-l3-certs/server.crt -noout -subject -issuer -serial -ext subjectAltName

3. Require TLS and run a positive/negative hostname test

The server publishes only to host loopback. The first command uses localhost, which matches the certificate SAN. The second uses 127.0.0.1; because the certificate deliberately lacks an IP SAN, a conforming client should reject the server identity. The repair is to issue a certificate containing the actual names/addresses clients use—not to normalize an insecure bypass.

docker rm -f atlasmart-ch22-l3 2>/dev/null || truedocker run -d --name atlasmart-ch22-l3 \  -p 127.0.0.1:27177:27017 \  -v "$PWD/ch22-l3-certs:/certs:ro" \  mongodb/mongodb-community-server:8.3.8-ubuntu2204-slim \  --bind_ip_all --tlsMode requireTLS \  --tlsCertificateKeyFile /certs/server.pem --tlsCAFile /certs/ca.crtuntil docker logs atlasmart-ch22-l3 2>&1 | grep -q "Waiting for connections"; do sleep 1; done# Expected success: requested hostname is present in SAN.mongosh "mongodb://localhost:27177/?tls=true" --tls --tlsCAFile ch22-l3-certs/ca.crt \  --quiet --eval 'db.runCommand({hello:1})'# Expected failure: 127.0.0.1 is not in this certificate's SAN.mongosh "mongodb://127.0.0.1:27177/?tls=true" --tls --tlsCAFile ch22-l3-certs/ca.crt \  --quiet --eval 'db.runCommand({ping:1})' || true

4. Internal cluster authentication is a separate membership boundary

Replica-set members and sharded-cluster components authenticate to each other using either a shared keyfile or internal X.509 certificates. Enabling internal authentication also enables client authorization. A keyfile acts as a shared membership secret and uses SCRAM internally; every communicating member must share at least one key. X.509 membership authentication instead validates certificate identity. When TLS is enabled, MongoDB also uses the configured certificate on internal connections, so certificate SAN and usage requirements matter even if the membership mechanism remains keyfile.

5. Why multi-key keyfiles allow rolling rotation

MongoDB keyfiles can contain multiple YAML key strings. Rotation therefore has an overlap phase: add the replacement key while retaining the old key everywhere; restart members one at a time; only after every member accepts the replacement do you remove the old key and roll again. The following deterministic model makes the invariant explicit without pretending it reproduces SCRAM, elections, or network failures.

members = {    "mongo1": {"old"},    "mongo2": {"old"},    "mongo3": {"old"},}def connected(a, b):    return bool(members[a] & members[b])def print_matrix(stage):    print(stage, {m: sorted(v) for m, v in members.items()})    for a in members:        for b in members:            if a < b:                print(a, b, connected(a, b))print_matrix("start")for m in members:    members[m] = {"old", "new"}    print_matrix(f"{m}: accepts old+new")for m in members:    members[m] = {"new"}    print_matrix(f"{m}: new only")

A real replica-set rotation additionally preserves a writable primary by restarting secondaries first, stepping down the primary when needed, and verifying replication/health after every member. Do not remove the old key from one member while peers still know only the old key. For a sharded cluster, the same overlap principle spans config servers, every shard replica set, and mongos routers.

6. Rotate TLS certificates online; do not silently replace trust

MongoDB 5.0+ supports rotateCertificates on Community and Enterprise self-managed deployments. Replace the certificate/CA/CRL files at the same configured paths, then invoke the command with a principal that has the rotateCertificates action (for example via hostManager). Existing connections continue using the old certificate; new connections use the replacement. A bad replacement causes rotation to fail without discarding the currently working TLS configuration.

const admin = db.getSiblingDB("admin");printjson(admin.runCommand({  rotateCertificates: 1,  message: "AtlasMart Chapter 22 controlled certificate rotation"}));

7. Verification, cleanup, and production judgment

• openssl x509 shows the expected CA, serial, and DNS:localhost SAN.
• A TLS connection to localhost succeeds with the lab CA.
• The same endpoint addressed as 127.0.0.1 fails hostname validation because the IP SAN is absent.
• The key-rotation model never allows two members to lose all common keys during a correct rollout.
• Certificate and key rotation procedures include an explicit overlap/rollback stage instead of a one-shot overwrite.

In production, issue per-host certificates with SANs matching real connection names, require certificate validation, protect private keys, monitor expiration, and rehearse rotation before an emergency. Use keyfiles for a simpler shared-secret membership boundary or X.509 where certificate-based membership fits the environment; neither removes the need for client least privilege. Atlas manages TLS/certificate rotation for the managed service. The next lesson moves outward from identity to network reachability and separates application and administration paths.

docker rm -f atlasmart-ch22-l3 2>/dev/null || truerm -rf ch22-l3-certs

Authoritative references

Version-sensitive behavior in this lesson was checked against current MongoDB documentation at generation time. Re-check these sources before applying the procedures to a later server or managed-service release.

• MongoDB Security
• Security Checklist for Self-Managed Deployments
• Localhost Exception in Self-Managed Deployments
• Role-Based Access Control in Self-Managed Deployments
• SCRAM Authentication
• createUser Command
• Built-In Roles
• Privilege Actions
• User-Defined Roles
• Configure MongoDB Instances for TLS
• Connection String TLS Options
• Self-Managed Internal/Membership Authentication
• Rotate Keys for Self-Managed Replica Sets
• rotateCertificates Command
• IP Binding in Self-Managed Deployments
• Auditing
• Configure Audit Filters
• System Event Audit Messages
• Atlas Database Auditing
• MongoDB 8.3 Release Notes
• mongosh Release Notes
• PyMongo Release Notes