Chapter 01 · Search Engine Foundations, Elasticsearch vs OpenSearch, Deployment Models, and Lab Setup
Self-Managed, Elastic Cloud/Serverless, Amazon OpenSearch Service, and Other Managed Responsibility Boundaries
Map self-managed and managed search offerings to explicit control-plane, security, upgrade, recovery, cost and exit responsibilities.
Learning outcomes
AtlasMart’s search API can run on a laptop, a self-managed VM fleet, Elastic Cloud, Amazon OpenSearch Service, or another managed distribution. The application endpoint may look similar while the control plane, failure responsibilities, upgrade authority, network model, backup controls and cost levers differ. This lesson turns “managed” into a precise responsibility matrix.
Separate data-plane search APIs from control-plane infrastructure responsibilities.
Compare fully self-managed Elasticsearch/OpenSearch with Elastic Cloud Hosted, Elastic Cloud Serverless and Amazon OpenSearch Service without claiming feature equivalence.
Identify which responsibilities remain with AtlasMart even when nodes, upgrades or scaling are managed.
Recognize managed-service restrictions as API/architecture constraints rather than documentation trivia.
Write a deployment decision record that includes portability, observability, security, recovery and exit strategy.
This chapter pins Elasticsearch 9.5.3 (released 3
September 2026) and OpenSearch 3.8.0 (released 4
August 2026) for reproducible examples. OpenSearch 3.9.0 is
scheduled for 29 September 2026 and is therefore not treated
as current. Re-check both projects before reusing these
commands later. Elasticsearch and OpenSearch are independent
products: shared Lucene ancestry does not make their APIs,
plugins, security, lifecycle, vector features, clients, or
managed offerings interchangeable.
The environment used to generate this lesson does not provide
Docker, Elasticsearch, OpenSearch, Kibana, or OpenSearch
Dashboards. The commands and API shapes were reviewed against
the current official documentation but were not executed here.
Expected output is described by invariant and field shape
rather than presented as captured benchmark evidence. Every
destructive action is scoped to
atlasmart-* course containers, volumes, indices,
and local loopback ports.
1. “Managed” moves responsibilities; it does not eliminate them
A search deployment has at least two planes. The data plane receives indexing and search requests. The control plane creates clusters/projects/domains, chooses capacity, upgrades software, manages network exposure, replaces failed infrastructure, and governs service-specific features. Self-managed deployments expose the most control because your team owns both planes. Managed services intentionally hide or automate parts of the control plane.
Application responsibilities remain regardless: field semantics, document ownership, mappings, relevance judgments, authorization logic, ingestion idempotency, data quality, retention intent, SLOs, test coverage and incident decisions still belong to AtlasMart. A provider can replace a failed node without knowing whether “wireless keyboard” should rank above “keyboard cover.”
| Deployment model | Provider/platform handles more of | AtlasMart still must own |
|---|---|---|
| Self-managed Elasticsearch/OpenSearch | Nothing about your infrastructure by default; you control hosts/containers, network, upgrades and recovery tooling. | Data model, security config, capacity, shard design, upgrades, snapshots, monitoring, OS/JVM, failure testing and application correctness. |
| Elastic Cloud Hosted | Elastic-operated infrastructure/orchestration with deployment controls and managed platform features. | Data/index design, many deployment choices, relevance, security policy, performance/SLOs, data protection decisions and application behavior. |
| Elastic Cloud Serverless | Elastic manages nodes, shard distribution, scaling, upgrades and much of platform resilience. | Project/data design, access, ingest/query behavior, retention/performance settings exposed by the service, relevance and application-level recovery/continuity decisions. |
| Amazon OpenSearch Service | AWS manages service infrastructure for OpenSearch domains and service integrations. | Domain configuration choices, data/index/query design, IAM/resource policies, workload capacity/cost, snapshots/retention choices, application correctness and supported-version constraints. |
| Other managed offerings | Varies by provider and contract. | Verify the exact engine build, plugins, security, network, snapshots, upgrade policy, SLA/support and export path; never infer from the product name alone. |
2. Elastic Cloud Hosted and Serverless are different operational contracts
Elastic documentation distinguishes Hosted deployments from Serverless projects. Hosted gives users more deployment-level resource and configuration choices, while Serverless intentionally abstracts nodes, shard distribution and scaling and performs upgrades automatically. That difference can invalidate self-managed operational recipes even though applications still use Elasticsearch-oriented APIs.
For example, a serverless service may not expose low-level shard operations that are central to a self-managed capacity lesson. That is not a missing button; it is part of the service architecture. Course examples must therefore name the deployment model next to every control-plane command.
Managed-service capabilities, regions, versions, subscription tiers and restrictions change independently of upstream server releases. Re-check the provider documentation on the day you design or migrate; do not freeze this chapter as a purchasing matrix.
3. Amazon OpenSearch Service is a service contract, not just “OpenSearch in AWS”
AWS describes Amazon OpenSearch Service as a managed service for deploying, operating and scaling OpenSearch clusters (“domains”) in AWS. The service has its own supported-version schedule, IAM/network integration, configuration surface and serverless offering. Upstream OpenSearch documentation remains essential for engine concepts, but service behavior must be checked against AWS documentation.
AtlasMart should record at least: engine/service version, domain or serverless collection type, region/AZ strategy, VPC/public endpoint decision, authentication/IAM model, encryption, snapshot behavior, scaling controls, plugin/feature availability, maintenance policy, telemetry, quotas and export/migration path.
4. A deployment decision record forces hidden assumptions into the open
# AtlasMart Search Deployment DecisionWorkload:- catalog documents and growth rate- indexing/update rate- search QPS and concurrency- p95/p99 latency SLO- accepted search freshness lag- relevance evaluation set- retention and recovery objectivesSecurity:- endpoint exposure / private networking- identity source and least privilege- TLS and key/certificate ownership- tenant/document authorization boundaryOperations:- who patches OS/JVM/engine?- who replaces nodes?- who controls shard topology?- who performs upgrades and when?- snapshot/restore responsibility and restore-test evidence- logs/metrics/alerts available to operatorsPortability:- source-of-truth location- mappings/templates/pipelines in version control- client/API abstraction- data export/reindex path- judged relevance and benchmark suite- rollback windowCost:- provisioned vs usage-based resources- storage/egress/snapshot costs- operational labor and support/subscription costsCopy a self-managed runbook into a serverless service and assume missing node/shard controls can be “worked around.” The failure is a control-plane mismatch. Repair by separating application/data-plane requirements from infrastructure controls, then choose a deployment whose exposed controls satisfy the real requirements.
5. Free local lab: simulate the responsibility boundary
No paid account is required. Use the two local containers from Lesson 4 and mark every task your team must perform: choose version, allocate memory, bind ports, manage credentials/certificates, persist storage, inspect logs, create/delete indices, collect metrics, back up data later, upgrade later, and remove resources. Then compare that list with current provider documentation for Hosted/Serverless/Amazon OpenSearch Service.
| Task | Local self-managed evidence | Managed-service question |
|---|---|---|
| Version | Pinned Docker image tag and root response | Can you select the version? Who schedules upgrades? |
| Capacity | Container memory and host disk | Which resources are user-sized vs autoscaled? |
| Network | Loopback port binding and Docker network | Public/private endpoint options and provider network integration? |
| Security | Local TLS/auth setup | Which identity, certificate and key controls are exposed? |
| Recovery | You must create/test snapshot strategy | What is automatic, what is user-triggered, and how is restore tested? |
| Observability | You collect logs/nodes stats/health | Which metrics/logs/audit signals are available and retained? |
Production judgment
A managed service is valuable when transferred operational work and provider automation outweigh lost control, service-specific restrictions and migration cost. Self-managed is valuable when control, locality or custom integration justifies owning patching, capacity, security and recovery. The correct answer can differ between AtlasMart environments.
Next, build the local evidence baseline that future chapters can reproduce regardless of cloud access.
Check your understanding
- What is the difference between the search data plane and deployment control plane?
- Name three responsibilities that remain with the application team in a fully managed service.
- Why can a self-managed shard command be invalid guidance for Serverless?
- Why must Amazon OpenSearch Service be documented separately from upstream OpenSearch?
- What is the purpose of an exit strategy if migration is not currently planned?
Review the answers
1. The data plane serves indexing/search requests; the control plane provisions, scales, upgrades, networks and operates the search infrastructure/service.
2. Examples: mapping/data semantics, relevance quality, authorization intent, ingest idempotency, SLOs, source-of-truth/reconciliation and application correctness.
3. Serverless deliberately abstracts/automates low-level infrastructure and may not expose the same shard/node controls; the operational contract is different.
4. AWS adds a managed control plane, supported-version schedule, IAM/network integrations, quotas and service-specific features/restrictions beyond the upstream engine.
5. It keeps source data, schemas, tests, portability boundaries and recovery options explicit so a future cost, compliance, feature or outage decision is not blocked by undocumented dependence.
Summary and next step
Deployment model is part of search semantics whenever it changes available controls, upgrade timing, security, topology, snapshots or failure responsibility. Next, start pinned local clusters and collect the first reproducible evidence from both platforms.
Authoritative references
- Elastic deployment comparison — Official comparison across self-managed and Elastic orchestration/cloud options.
- Elastic self-managed clusters — Official statement of self-managed responsibility.
- Elastic Cloud Serverless — Current Serverless architecture and responsibility model.
- Elastic Cloud Hosted — Current Hosted deployment model.
- Amazon OpenSearch Service overview — AWS-managed OpenSearch service definition and concepts.
- OpenSearch documentation — Upstream OpenSearch engine documentation.