Chapter 01 · Search Engine Foundations, Elasticsearch vs OpenSearch, Deployment Models, and Lab Setup
Start Local Clusters, Use Kibana/OpenSearch Dashboards Dev Tools or curl, and Inspect Cluster/Node Health
Start pinned local Elasticsearch and OpenSearch clusters, keep TLS/auth visible, and diagnose identity and health from observable evidence.
Learning outcomes
This lesson creates the chapter’s observable baseline: two
pinned search engines running side by side on local loopback
ports with product identity, authentication/TLS state, cluster
health, node/JVM evidence and deterministic AtlasMart data.
curl is mandatory because it exposes the HTTP
boundary directly; Kibana/OpenSearch Dashboards Dev Tools are
optional interfaces over the same ideas.
Start Elasticsearch 9.5.3 and OpenSearch 3.8.0 in separate persistent Docker containers without publishing their ports beyond loopback.
Keep Elasticsearch auto-configured HTTPS/auth and OpenSearch demo HTTPS/auth visibly distinct.
Use root, cluster-health, nodes-information and index inventory APIs to prove which server answered.
Interpret green/yellow/red health in shard-allocation terms rather than as a generic “up/down” light.
Diagnose wrong scheme, wrong credentials, missing CA trust and wrong port as separate failure modes.
This chapter pins Elasticsearch 9.5.3 (released 3
September 2026) and OpenSearch 3.8.0 (released 4
August 2026) for reproducible examples. OpenSearch 3.9.0 is
scheduled for 29 September 2026 and is therefore not treated
as current. Re-check both projects before reusing these
commands later. Elasticsearch and OpenSearch are independent
products: shared Lucene ancestry does not make their APIs,
plugins, security, lifecycle, vector features, clients, or
managed offerings interchangeable.
The environment used to generate this lesson does not provide
Docker, Elasticsearch, OpenSearch, Kibana, or OpenSearch
Dashboards. The commands and API shapes were reviewed against
the current official documentation but were not executed here.
Expected output is described by invariant and field shape
rather than presented as captured benchmark evidence. Every
destructive action is scoped to
atlasmart-* course containers, volumes, indices,
and local loopback ports.
1. Preconditions and resource budget
Install Docker Desktop on Windows/macOS or Docker Engine on Linux. Elastic’s current Docker guidance recommends at least 4 GB allocated to Docker Desktop for its development workflow; running Elasticsearch and OpenSearch simultaneously needs enough host memory for both JVMs plus Docker and the browser. This chapter intentionally uses one node per product and no Dashboards containers by default to reduce memory pressure.
Create one private Docker network and two named data volumes.
Host ports bind to 127.0.0.1 so another machine
cannot reach the lab through a normal external interface. The
containers still communicate on the private Docker network.
docker network create atlasmart-searchdocker volume create atlasmart-es-datadocker volume create atlasmart-os-data
2. Start pinned Elasticsearch with its default security path
The official Elasticsearch Docker workflow starts a pinned
image, generates security material, prints the initial
elastic password/enrollment token, and allows the
HTTP CA certificate to be copied from the container. We keep
that secure default rather than disabling security for
convenience.
docker pull docker.elastic.co/elasticsearch/elasticsearch:9.5.3docker run -d --name atlasmart-es --hostname atlasmart-es --network atlasmart-search -p 127.0.0.1:9200:9200 -m 1GB -v atlasmart-es-data:/usr/share/elasticsearch/data docker.elastic.co/elasticsearch/elasticsearch:9.5.3docker logs --tail 120 atlasmart-es
Wait for startup. If you did not preserve the initially printed password, reset it using the supported tool, then store the new value only in your local shell/session—not in a committed lesson file:
docker exec -it atlasmart-es /usr/share/elasticsearch/bin/elasticsearch-reset-password -u elasticdocker cp atlasmart-es:/usr/share/elasticsearch/config/certs/http_ca.crt ./atlasmart-es-http-ca.crt
Set ELASTIC_PASSWORD in your shell to the
generated/reset value. In PowerShell use
$env:ELASTIC_PASSWORD = "..."; in Bash use
export ELASTIC_PASSWORD='...'. Do not commit it.
curl --cacert atlasmart-es-http-ca.crt -u "elastic:$ELASTIC_PASSWORD" https://localhost:9200/curl --cacert atlasmart-es-http-ca.crt -u "elastic:$ELASTIC_PASSWORD" "https://localhost:9200/_cluster/health?pretty"curl --cacert atlasmart-es-http-ca.crt -u "elastic:$ELASTIC_PASSWORD" "https://localhost:9200/_nodes?filter_path=cluster_name,nodes.*.name,nodes.*.version,nodes.*.roles,nodes.*.jvm.version"
Windows PowerShell users should call curl.exe when
they want real curl semantics. A successful TLS request with
--cacert proves the presented certificate chains to
the copied course CA and the credentials were accepted for that
request; it does not prove backups, high availability or
production hardening.
3. Start pinned OpenSearch with demo security explicitly labeled
Current OpenSearch Docker distributions run the Security plugin demo configuration unless it is disabled. OpenSearch 2.12+ requires a custom initial admin password for that demo setup. The demo configuration uses self-signed/demo certificates and must not be reused as a production certificate/identity design.
# Bashexport OPENSEARCH_INITIAL_ADMIN_PASSWORD='replace-with-a-strong-local-lab-secret'# PowerShell equivalent:# $env:OPENSEARCH_INITIAL_ADMIN_PASSWORD = 'replace-with-a-strong-local-lab-secret'
docker pull opensearchproject/opensearch:3.8.0docker run -d --name atlasmart-os --hostname atlasmart-os --network atlasmart-search -p 127.0.0.1:9201:9200 -p 127.0.0.1:9601:9600 -e discovery.type=single-node -e OPENSEARCH_INITIAL_ADMIN_PASSWORD -e "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m" -v atlasmart-os-data:/usr/share/opensearch/data opensearchproject/opensearch:3.8.0docker logs --tail 120 atlasmart-os
curl -k -u "admin:$OPENSEARCH_INITIAL_ADMIN_PASSWORD" https://localhost:9201/curl -k -u "admin:$OPENSEARCH_INITIAL_ADMIN_PASSWORD" "https://localhost:9201/_cluster/health?pretty"curl -k -u "admin:$OPENSEARCH_INITIAL_ADMIN_PASSWORD" "https://localhost:9201/_nodes?filter_path=cluster_name,nodes.*.name,nodes.*.version,nodes.*.roles,nodes.*.jvm.version"
-k appears only in this demo
OpenSearch documents -k/--insecure for its demo
self-signed certificate flow. It keeps the transport encrypted
but skips normal server-certificate verification/hostname
checks. That is acceptable only inside this loopback
disposable lab. Production must use certificates/hostnames
that clients can validate; later security lessons replace demo
credentials and certificates.
4. Health is shard allocation, not “the process exists”
| Evidence | Question it answers | What it does not prove |
|---|---|---|
docker ps |
Is the container process still running? | That the HTTP endpoint is ready, authenticated or healthy. |
Root GET / |
Which product/version/cluster identity answered? | That all shards are allocated or data is correct. |
/_cluster/health |
Are primary/replica shards allocated at green/yellow/red levels? | That latency, capacity, backups or relevance meet SLOs. |
/_nodes |
Which nodes/roles/JVM versions are reported? | That every client route or network path is correct. |
| CAT indices | Human-readable index/shard/doc inventory. | A stable machine API contract; use structured APIs for applications. |
In both product families, green/yellow/red health is about shard allocation. A single-node cluster can be yellow when a primary is allocated but a configured replica has nowhere else to go. That is not equivalent to a red cluster, but it is evidence that the requested redundancy is not satisfied. For the Chapter 01 fixture we use zero replicas explicitly so the lab does not pretend one node is highly available.
PUT /atlasmart-products-v1{ "settings": {"number_of_shards": 1, "number_of_replicas": 0}, "mappings": { "properties": { "product_id": {"type":"keyword"}, "name": {"type":"text"}, "category": {"type":"keyword"}, "description": {"type":"text"} } }}
5. Diagnose failures by layer
| Deliberate mistake | Expected signal | Diagnosis / repair |
|---|---|---|
Use http://localhost:9200 for secure
Elasticsearch
|
TLS/HTTP protocol error or unusable response | Use HTTPS and the copied HTTP CA; do not “fix” by globally disabling security. |
| Wrong Elasticsearch password | 401 authentication failure | Reset/store the lab password; distinguish authentication from TLS trust. |
| Omit Elasticsearch CA trust | Certificate verification failure |
Use --cacert atlasmart-es-http-ca.crt; do not
make -k the default.
|
| Send OpenSearch request to port 9200 | You hit Elasticsearch instead |
Inspect root identity and keep ES_URL/OS_URL
explicit.
|
| Use weak/missing OpenSearch initial admin password | Container setup can fail | Set a sufficiently strong local secret before starting; inspect logs rather than repeatedly restarting blindly. |
Publish both containers as -p 9200:9200, disable
security, and tell learners to discover which server answered.
The port collision either prevents startup or hides product
identity; disabling auth/TLS removes the very boundary the
chapter must teach. Repair with separate loopback ports,
explicit TLS/auth paths, root/version evidence and named
containers.
If you prefer Dev Tools, start the version-matched
Kibana/OpenSearch Dashboards using current official instructions
and execute the same REST request bodies there. The UI is a
convenience layer; curl remains the portable
chapter baseline.
Production judgment
A process that returns HTTP 200 is only the first layer of readiness. Production acceptance also needs authenticated/authorized access, valid certificate verification, shard allocation, capacity headroom, snapshot/restore, monitoring, tested upgrades and application-level correctness. Next, turn this ad-hoc startup into a repeatable course lab with data, metrics and reset automation.
Check your understanding
- Why are Elasticsearch and OpenSearch bound to different loopback ports?
-
What security property does Elasticsearch
--cacertprovide that OpenSearch demo-kintentionally skips? - Why can a single-node cluster be yellow without being red?
- What does the root endpoint prove?
- Why is CAT output better for operators than for application automation?
Review the answers
1. It prevents host-port collision and makes the target product explicit while keeping both services unreachable from normal external interfaces.
2. It verifies the server certificate
against the trusted CA (and hostname rules), whereas
-k encrypts the connection but skips normal
certificate verification for the demo cert.
3. A primary can be allocated while a requested replica cannot be placed on another node; red indicates an unallocated primary for affected data.
4. It identifies the answering server/cluster/version metadata for that request; it does not prove full cluster health, data correctness or resilience.
5. CAT APIs are optimized for human-readable command-line inspection and their formatting is not the preferred structured contract for applications.
Summary and next step
You now have a safe identity and health evidence path for both products, with TLS/auth differences visible rather than hidden. Next, make the AtlasMart fixture persistent and repeatable, collect metrics, and define reset procedures that cannot accidentally target unrelated data.
Authoritative references
- Elastic: single-node Docker cluster — Current secure Docker workflow, CA copy, password and REST verification.
- Elasticsearch cluster health API — Official green/yellow/red shard-allocation semantics.
- Elasticsearch CAT health — Human-oriented CAT health guidance.
- OpenSearch Docker installation — Current Docker setup and admin-password requirement.
- OpenSearch security demo configuration — Demo certificates/security behavior and production warning.
- OpenSearch quickstart — Official local Docker verification examples.