Configure Structured Logging, Sensitive-Data Logging, Detailed Errors, and Environment Safety

Learning outcomes

1. The problem: a failed query is invisible—or the logs expose too much

ServiceHub is now large enough that “turn on SQL logging” is not an operational strategy. A production incident needs to answer which EF operation failed, how long it took, which application request/job triggered it, which provider/database was involved, and whether the failure came from translation, connection, execution, materialization, concurrency, or transaction behavior. At the same time, a work-order query can carry customer names, external references, summaries, addresses, and other personally identifiable information (PII) or commercially sensitive values. An observability system that solves diagnosis by copying those values into logs creates a second security problem.

Structured logging means emitting events with stable categories, event identifiers, levels, message templates, and properties that a logging backend can index. Simple logging is EF Core's LogTo callback: useful for a local lab, but not a replacement for a production logging pipeline. Sensitive-data logging allows EF to include application values—especially keys and parameters—in diagnostic messages. Detailed errors add extra exception detail around value-reading/materialization failures at a runtime cost. Define these mechanisms separately before enabling any of them.

2. Start with Microsoft.Extensions.Logging for a real application

ASP.NET Core and generic-host applications already use Microsoft.Extensions.Logging. EF Core integrates with that pipeline, so category filtering, providers, scopes, sampling, and retention can be configured with the rest of the application rather than hard-coded in DbContext. ServiceHub should keep a broad information baseline for application events while narrowing EF categories according to operational need.

{  "Logging": {    "LogLevel": {      "Default": "Information",      "Microsoft.EntityFrameworkCore": "Warning",      "Microsoft.EntityFrameworkCore.Database.Command": "Information",      "Microsoft.EntityFrameworkCore.Database.Connection": "Warning",      "Microsoft.EntityFrameworkCore.Database.Transaction": "Warning",      "Microsoft.EntityFrameworkCore.Update": "Warning"    }  }}

This example intentionally exposes command completion/failure at Information while keeping most EF chatter at Warning. It is not a universal production policy: a high-volume service may need sampling or a higher command threshold, while a low-volume internal system may retain more. The important property is that the choice is explicit and reviewable.

3. Use LogTo for a disposable local investigation—not as a production sink

var options = new DbContextOptionsBuilder<ServiceHubContext>()    .UseSqlite("Data Source=servicehub-observability-lab.db")    .LogTo(        Console.WriteLine,        new[]        {            DbLoggerCategory.Database.Command.Name,            DbLoggerCategory.Database.Transaction.Name        },        LogLevel.Information)    .Options;await using var db = new ServiceHubContext(options);

LogTo is attractive because it needs no additional logging packages. That simplicity is exactly why it belongs in the lab: it writes formatted text to a callback. Production systems usually need structured sinks, centralized retention, correlation fields, redaction, sampling, and access control. Do not send LogTo(Console.WriteLine) into an unmanaged container log stream and assume the observability problem is solved.

4. Read the categories as an execution pipeline

A useful incident timeline combines categories instead of searching for one “EF log line.” Chapter 17 already separated query translation, driver/database execution, materialization, and end-to-end latency. Logging preserves that same phase distinction in production.

5. Sensitive-data logging changes the threat model

By default EF does not include application data values in exception messages and many diagnostics because those values can be confidential. EnableSensitiveDataLogging() relaxes that protection. In ServiceHub, a diagnostic query could expose a customer name, work-order number, summary fragment, external reference, address component, tenant identifier, or token-like value depending on the model. Once the event is exported to a centralized log service, that copy may live longer and be accessible to more people than the source database.

optionsBuilder    .UseSqlite(connectionString)    .EnableSensitiveDataLogging()    .EnableDetailedErrors(); // enabled for every environment

The problem is not SQL injection; parameterization can still be correct. The problem is data disclosure through telemetry. Production log access, backups, exports, support bundles, and third-party processors become additional data surfaces.

services.AddDbContext<ServiceHubContext>((sp, options) =>{    var env = sp.GetRequiredService<IHostEnvironment>();    var cs = sp.GetRequiredService<IConfiguration>()        .GetConnectionString("ServiceHub")        ?? throw new InvalidOperationException("ServiceHub connection is missing.");    options.UseSqlite(cs);    if (env.IsDevelopment())    {        options.EnableDetailedErrors();        // Sensitive values stay OFF by default even in Development.        // Enable only for a short, disposable local diagnostic session.    }});

6. Detailed errors improve diagnosis but are not free

EnableDetailedErrors() is different from sensitive-data logging. EF wraps value reads with additional error handling so a materialization failure—for example, a database NULL flowing into a non-nullable model property—can produce more useful context. That extra work has a cost. The safe default is still to enable it for development/test or a deliberately scoped diagnostic environment, not because “more detail” sounds universally better.

Safe production-style signal:  InvalidOperationException / provider read failure  category=Microsoft.EntityFrameworkCore.Query  trace_id=... operation=workorder.queueControlled diagnostic session with detailed errors:  Adds EF context about the property/value read that failed.  Still keep sensitive-data logging OFF unless values are truly required.

7. Parameter redaction is a verification task, not an assumption

Run the same parameterized query twice: once under the production-style configuration and once under a disposable local configuration where sensitive logging is explicitly enabled. The exact formatted text can vary by EF version/provider, so verify behavior on the pinned packages rather than copying a screenshot from another release.

var customer = "Contoso Field Lab";var rows = await db.WorkOrders    .AsNoTracking()    .Where(w => w.CustomerName == customer)    .OrderBy(w => w.Id)    .Take(5)    .ToListAsync(cancellationToken);

The safe acceptance criterion is not a particular number of question marks in the formatted log. It is that the exported production event does not contain the literal customer value or other secret/PII fields. Build an automated redaction test around representative sensitive strings.

8. ConfigureWarnings can turn known hazards into test failures

Some EF events are warnings because their severity depends on the application. In development or integration tests, you can promote a selected warning to an exception when the team has decided that the pattern is always wrong for this codebase. The inverse—globally ignoring warnings—can hide provider or query-shape problems.

optionsBuilder.ConfigureWarnings(warnings =>{    // Example pattern: promote a specifically reviewed EF warning.    // Use the actual CoreEventId / RelationalEventId that matches    // your pinned EF version and policy.});

Do not paste an event ID from an old release without checking the EF Core 10 API. The production policy should usually log and alert on important warnings; CI can be stricter when the warning represents a code-quality invariant.

9. Failure case: a support toggle becomes permanent production configuration

During an incident, an engineer enables sensitive logging to find one failing key. The incident closes, but the flag remains in an environment variable. Over the next month, millions of customer values enter a centralized log service. The query itself was secure; the observability configuration created the leak.

Repair: use an allow-listed diagnostic configuration with an expiry/change ticket, isolate it to a disposable environment or a narrow process, keep retention short, and test the disabled state on every normal deployment. If a production-only reproduction is unavoidable, prefer correlation identifiers and database-side evidence before collecting application values.

10. Mandatory lab: prove the safe default

1. Copy the current ServiceHub SQLite lab database to servicehub-observability-lab.db.
2. Configure Microsoft.Extensions.Logging categories or a local LogTo equivalent at command level.
3. Execute a parameterized customer/work-order query with a deliberately recognizable fake sensitive string.
4. Capture the production-style output and assert the literal sensitive value is absent.
5. In a disposable local process only, enable sensitive-data logging and observe how the diagnostic output changes; delete that capture afterward.
6. Trigger a controlled materialization/data-shape error and compare normal versus EnableDetailedErrors diagnostics.
7. Record EF Core/provider/runtime versions, environment name, logging categories/levels, and whether sensitive logging was enabled.

11. Production judgment

Use Microsoft.Extensions.Logging as the production logging substrate; use LogTo for focused local experiments. Keep sensitive-data logging off by default everywhere, and treat detailed errors as a diagnostic cost/benefit decision. Retain stable operation identifiers, event IDs, trace IDs, duration, provider, database target class, and exception type—not arbitrary values. The next lesson adds process-wide diagnostic events, metrics, and trace correlation so one EF command can be placed inside an HTTP request or background-job trace.

Authoritative references

Observability and diagnostics are version-, provider-, and topology-sensitive. Re-check these primary sources before standardizing a production telemetry contract.

• Using Microsoft.Extensions.Logging - EF Core — production logging integration, filtering, categories and event IDs
• Simple logging - EF Core — LogTo, sensitive-data logging and detailed errors
• Overview of logging and interception - EF Core — choosing logs, events, interceptors and diagnostic listeners
• DbContext configuration - EF Core — EnableSensitiveDataLogging, EnableDetailedErrors and AddInterceptors options
• Logging in .NET — structured Microsoft.Extensions.Logging configuration
• Security and privacy guidance for logging — logging pipeline and configuration context; apply organizational data-classification policy