Sensitive Data in Logs, Error Messages, Traces, Backups, and Development Diagnostics

Learning outcomes

1. Security failures often leak through the tools used to diagnose other failures

ServiceHub now has logging, command interceptors, traces, test artifacts, migration backups and production runbooks. Those tools are valuable precisely because they capture context. The same context can contain passwords, bearer tokens, tenant IDs, email addresses, work-order summaries, raw SQL parameters, internal hostnames, schema names, stack traces or entire database rows.

Personally identifiable information (PII) is information that identifies or can reasonably be linked to a person. A secret grants access or authority. Business-confidential data may be neither PII nor a credential but still require restricted handling. Classification determines what may be captured, where, for how long, and who can access it.

2. Sensitive-data logging changes what EF exposes

EF Core intentionally hides data values from many log/exception messages by default. EnableSensitiveDataLogging() can include data values for debugging, which is useful in a controlled development environment and dangerous as a production default. EnableDetailedErrors() can improve materialization diagnostics by adding extra exception handling; it is a different feature and still requires environment/performance judgment.

builder.Services.AddDbContext<ServiceHubContext>((sp, options) =>{    var env = sp.GetRequiredService<IHostEnvironment>();    options.UseSqlite(builder.Configuration.GetConnectionString("ServiceHub"));    if (env.IsDevelopment())    {        options.EnableDetailedErrors();        // EnableSensitiveDataLogging only for a short, controlled diagnostic        // session when the captured data is acceptable.    }});

3. Structured logs should carry identifiers for correlation, not payload copies

Useful production logs can include event name, provider, operation type, duration, rows affected, retry number, exception class, migration ID, and a trace/correlation ID. Avoid raw connection strings, access tokens, full SQL parameter values, and unbounded user content.

logger.LogWarning(    "Database command failed. Operation={Operation}; Provider={Provider}; " +    "TraceId={TraceId}; ExceptionType={ExceptionType}",    "WorkOrder.Search",    db.Database.ProviderName,    Activity.Current?.TraceId.ToString(),    ex.GetType().Name);

If tenant-level correlation is operationally necessary, add a separately designed telemetry identifier whose sensitivity and cardinality have been reviewed. A casual hash of TenantId is not magic anonymization when the input space is guessable or the hashing design is weak.

4. Deliberate failure: log the whole request, connection string and exception object

catch (Exception ex){    logger.LogError(ex,        "DB failure. Conn={ConnectionString}; Request={@Request}; Tenant={TenantId}",        connectionString,        request,        tenantId);    throw;}

This can copy passwords, search text, personal data, tenant identifiers and stack details into every configured sink/exporter. The repair is to define an explicit diagnostic schema, classify each field, redact/tokenize where appropriate, and keep sensitive values out by construction rather than hoping downstream log scrubbing catches everything.

5. Traces and telemetry exporters are data egress paths

An OpenTelemetry span or vendor APM event can leave the host and cross organizational boundaries. Query tags, SQL text, exception messages, baggage and custom attributes all need the same classification review as logs. Avoid high-cardinality raw user values as span attributes; they increase cost and can create privacy leakage.

6. Developer exception pages and support errors require environment boundaries

Detailed exception pages can expose stack traces, paths, SQL/provider details, configuration names, headers or request data. Keep them for local development or tightly controlled environments. Production clients should receive a stable problem response/correlation ID while detailed diagnostics remain in protected telemetry.

{  "type": "https://servicehub.example/problems/database-operation-failed",  "title": "The operation could not be completed.",  "status": 500,  "traceId": "00-..."}

Do not echo the database exception message, SQL statement, table name, connection endpoint, or stack trace to an untrusted client merely because it helps support reproduce the issue.

7. Backups, test databases and dumps are data copies with their own access lifecycle

A sanitized application log may be safe while a copied servicehub.db, SQL backup, memory dump, test fixture, or CI artifact contains complete tenant data and credentials in memory. Treat copies as data assets: minimize them, encrypt/protect where supported, restrict access, define retention, verify deletion, and never use production customer data casually in tests.

artifact owner identifiedclassification: secret / PII / confidential / operationalpurpose and minimum fields recordedaccess group restrictedretention/expiry definedencryption/platform protection verifiedexport/share path reviewedfinal deletion/disposal verifiedincident access audited

8. Build an incident diagnostic escalation ladder

Start with low-risk signals and escalate capture only when justified:

Every escalation needs scope, owner, retention and rollback/disable steps. “Temporarily enable everything” without a shutdown plan is how diagnostic leakage becomes permanent telemetry.

9. Mandatory lab: produce useful diagnostics with zero secret payloads

1. Run a disposable SQLite ServiceHub failure such as a unique/FK/concurrency violation with sensitive-data logging disabled.
2. Capture operation name, provider, duration, trace ID, exception type, migration state, and a stable query tag/fingerprint—but no connection string or parameter values.
3. Add a deliberate unsafe logging statement in a local-only branch and prove which fields would leak; then remove it.
4. Create a sanitized incident package containing the log event, generated SQL shape without values, schema/migration ID and reproduction steps.
5. Create a temporary database copy and classify it as sensitive; restrict access and delete it after verification.
6. Write the escalation/retention owner and maximum diagnostic level permitted in ordinary production operations.

10. Production judgment and bridge to the capstone

Security observability is a minimization problem: capture enough stable evidence to diagnose failures, not every available byte. Parameterized queries, bounded dynamic APIs, protected secrets, least-privileged identities, tenant defense in depth, and controlled diagnostics now form one coherent security model. Chapter 25 uses those controls in the final production architecture/capstone, where persistence abstractions, outbox/CQRS choices, deployment and incident runbooks are evaluated together.

Authoritative references

• EF Core simple logging — EnableSensitiveDataLogging, LogTo and production logging considerations
• EF Core Microsoft.Extensions.Logging — structured logging and EnableDetailedErrors guidance
• EF Core interceptors — command/save diagnostics without indiscriminate payload capture
• Safe storage of app secrets (.NET 10) — credential handling and environment/User Secrets warnings
• OWASP Logging Cheat Sheet — event-data classification, sensitive-data exclusions and operational logging guidance
• OWASP Secrets Management Cheat Sheet — credential lifecycle and exposure-response guidance