Internode Encryption vs Client TLS: Certificates, Trust, Rotation, and Failure Modes

Learning outcomes

Password authentication does not encrypt network traffic. Enabling client TLS does not encrypt gossip, repair or streaming, and it does not secure JMX. This lesson treats each path separately and rehearses certificate rotation failure modes.

Terms before the security mechanisms

CQL is Cassandra Query Language. A coordinator is the node accepting a client request; replicas store copies of the target partition. A partition is the row group located by a partition key; its hash maps to a token. vnodes are multiple token ranges assigned per node. A keyspace defines replication. RF is replication factor and CL is consistency level. An SSTable is Cassandra's immutable on-disk table file set; compaction merges SSTables; repair compares replica ranges and streams differences. SAI is Storage-Attached Indexing. A driver is the client library speaking Cassandra's native protocol.

Authentication proves an identity; Cassandra's authenticator implements that decision. Authorization decides what an authenticated identity may do; the authorizer checks permissions on Cassandra resources. A role is Cassandra's principal/inheritance object; LOGIN=true makes it directly authenticatable. A superuser bypasses normal authorization and therefore belongs only in tightly controlled administration or break-glass recovery. Least privilege grants only the permissions needed.

TLS (Transport Layer Security) protects network traffic and authenticates endpoints with certificates. Client TLS protects application/cqlsh ↔ Cassandra native protocol; internode TLS protects Cassandra node ↔ node gossip/replication/streaming. mTLS (mutual TLS) means both sides present certificates. A keystore holds private key/certificate material and a truststore holds trust anchors. JMX (Java Management Extensions) is Cassandra's management interface used by nodetool. Defense in depth layers identity, permissions, encryption, network isolation, secrets, backups, audit, patching and recovery instead of trusting one control.

1. Three network planes, three controls

Cassandra 5.0's preferred model is a single native transport port; the older dedicated SSL port is deprecated. A rolling migration can temporarily accept encrypted and unencrypted traffic with optional=true, but the end state should enforce the intended policy.

2. Generate disposable lab certificates

New-Item -ItemType Directory -Force .\ch26-tls | Out-Nullif (-not $env:ATLASMART_TLS_STORE_PASSWORD) { throw "Set the store secret from an approved source." }1..3 | ForEach-Object {  $n=$_  docker run --rm -e ATLASMART_TLS_STORE_PASSWORD="$env:ATLASMART_TLS_STORE_PASSWORD" `    -v "${PWD}/ch26-tls:/tls" --entrypoint keytool cassandra:5.0.9 `    -genkeypair -alias "atlasmart-sec-$n" -keyalg RSA -keysize 2048 -validity 30 `    -dname "CN=atlasmart-sec-$n,OU=Training,O=AtlasMart,C=XX" `    -ext "SAN=dns:atlasmart-sec-$n" -keystore "/tls/node$n.p12" -storetype PKCS12 `    -storepass:env ATLASMART_TLS_STORE_PASSWORD -keypass:env ATLASMART_TLS_STORE_PASSWORD  docker run --rm -e ATLASMART_TLS_STORE_PASSWORD="$env:ATLASMART_TLS_STORE_PASSWORD" `    -v "${PWD}/ch26-tls:/tls" --entrypoint keytool cassandra:5.0.9 `    -exportcert -rfc -alias "atlasmart-sec-$n" -keystore "/tls/node$n.p12" `    -storetype PKCS12 -storepass:env ATLASMART_TLS_STORE_PASSWORD -file "/tls/node$n.crt"}

The 30-day self-signed identities make expiry observable in a free local lab. They are not a production PKI recommendation. Production certificates need approved CA profiles, Subject Alternative Names (SANs), key custody, renewal/revocation and trust-chain governance.

3. Configure client and internode TLS independently

server_encryption_options:  internode_encryption: all  optional: false  legacy_ssl_storage_port_enabled: false  keystore: /run/atlasmart-tls/node.p12  keystore_password: <INJECT_FROM_SECRET_SOURCE>  truststore: /run/atlasmart-tls/truststore.p12  truststore_password: <INJECT_FROM_SECRET_SOURCE>  require_client_auth: true  require_endpoint_verification: trueclient_encryption_options:  enabled: true  optional: false  keystore: /run/atlasmart-tls/node.p12  keystore_password: <INJECT_FROM_SECRET_SOURCE>  require_client_auth: false

CLIENT TLS1. enabled=true, optional=true; rolling restart.2. Configure every cqlsh/driver with trust validation and verify TLS.3. Inspect system_views.clients for ssl_enabled/protocol/cipher/user.4. Set optional=false; rolling restart.5. Prove plaintext fails while TLS succeeds.INTERNODE TLS1. internode_encryption=all, optional=true; rolling restart.2. Verify UN state, restart, repair and streaming with encrypted peers.3. Set optional=false; rolling restart.4. If mTLS is required, verify every peer certificate/trust/SAN.5. Repeat node restart/repair/streaming tests after enforcement.

4. Observe TLS instead of trusting YAML

SELECT address,port,driver_name,driver_version,protocol_version,       ssl_enabled,ssl_protocol,ssl_cipher_suite,usernameFROM system_views.clients;

A row with ssl_enabled=true proves that listed active connection negotiated TLS. It does not prove all clients use TLS, endpoint verification is enabled, internode traffic is encrypted, or JMX is secure.

keytool -printcert -file /run/atlasmart-tls/node.crt# After replacing a file-based keystore/truststore in a controlled rotation:docker exec atlasmart-sec-1 nodetool reloadssl# Open new connections and repeat TLS/certificate evidence.

5. What to capture

• Authentication success/failure over TLS.
• Certificate subject, SAN, fingerprint and expiry without private-key disclosure.
• system_views.clients SSL evidence for expected users.
• Node health plus repair/streaming after internode TLS enforcement.
• Plaintext denial after the transition.
• Rotation success and invalid/untrusted certificate failure.

Production judgment

Security controls affect latency, availability and operator skill. Record Cassandra/JVM/driver versions, DC/racks/RF/CL, auth/TLS/JMX/network state, role inheritance, credential/certificate expiry, audit retention, backup encryption/access, patch status, repair/restore credentials and the application routing/retry/idempotency model. Re-run p50/p95/p99 latency and replica-failure tests after encryption/authorization changes. Data modeling still matters: partition/cardinality/TTL/tombstone/compaction/repair/SAI/vector choices can change sensitive-data exposure and resource-denial risk.

Do not treat a private subnet, one superuser password, client TLS or one audit stream as sufficient. Managed Cassandra may operate some server certificates, JMX, upgrades or backups, but application identities, permissions, client trust, data classification, secret use and incident response still have explicit owners. Migration/rollback must retain a tested administrative path and prevent coordinator-dependent security drift. Lesson 4 protects the other privileged surfaces: JMX, listener exposure, credentials, configuration secrets and backups.

Summary and next step

This lesson’s concepts, evidence path, failure boundaries, and production judgment should now be explicit enough to verify rather than assume. Re-run the check-your-understanding prompts and preserve any lab evidence you need before changing or cleaning up the environment.

Next, continue to Protect JMX/Management Interfaces, Credentials, Network Paths, Backups, and Configuration Secrets.

Authoritative references

Security defaults, algorithms, tooling and deprecations are version-sensitive. Re-check the target Cassandra/JVM/driver distribution and organizational cryptographic policy before production rollout.

• Apache Cassandra releases
• Apache Cassandra security operations
• cassandra.yaml reference
• CQL security and access control
• CREATE ROLE
• GRANT
• REVOKE
• hash_password
• cqlsh security/history options
• Virtual tables
• Audit logging
• nodetool reloadssl