Client/SDK and Query-Language Differences, Deprecated APIs, Feature Replacements, and Testing Strategy

Learning outcomes

1. AtlasMart problem: the code compiles, the migration still fails

The AtlasMart application wraps search in a repository class. It creates a client, sends a Query DSL body, catches one exception type, retries 429s, and parses aggregations. A URL switch may still fail because client version checks, API namespaces, auth middleware, retry defaults, request serialization, response wrappers, and feature-specific APIs have diverged.

A migration-safe application boundary exposes business operations—searchProducts, bulkUpsertCatalog, healthProbe—rather than leaking product-specific client objects throughout the codebase.

2. Current client-family boundary

3. Build a product-neutral application contract

from typing import Protocol, Iterable

class SearchBackend(Protocol):
    def search_products(self, tenant_id: str, text: str, limit: int) -> list[dict]: ...
    def bulk_upsert(self, docs: Iterable[dict]) -> dict: ...
    def health(self) -> dict: ...

# Implement ElasticBackend with elasticsearch-py.
# Implement OpenSearchBackend with opensearch-py.
# Run the same contract tests against both.

The adapters may generate different requests internally. That is a strength: the business contract stays stable while product-specific behavior is explicit and testable.

4. Query DSL overlap needs semantic tests

Many basic bool, term, match, range, sort, and aggregation shapes remain familiar. Advanced search is where drift accumulates: vector mappings/query parameters, semantic fields, hybrid ranking, search pipelines, rank fusion, query insights/profile details, data streams, security filters, and product-specific query features. Maintain a test corpus for every production query template.

{
  "name":"catalog-hiking-tenant-a",
  "request": {
    "query": {
      "bool": {
        "filter":[{"term":{"tenant_id":"tenant-a"}},{"term":{"available":true}}],
        "must":[{"match":{"name":"hiking"}}]
      }
    },
    "size":3,
    "sort":[{"_score":"desc"},{"sku":"asc"}]
  },
  "expected": {
    "must_include":["P-1001"],
    "must_exclude":["P-1004"],
    "minimum_recall_at_3":1.0
  }
}

5. Query languages are not portability layers

Word-for-word syntax translation is especially dangerous for ranking and analytics because similarly named functions can differ in null handling, time bucketing, limits, result shape, or execution cost.

6. Deprecation/replacement ledger

contracts:
  - id: api-001
    source: "ES 9.5.3 _search bool/match"
    target: "OS 3.8.0 _search bool/match"
    action: test
    evidence: tests/golden/catalog_hiking.json
  - id: analytics-004
    source: "ES|QL sales query"
    target: "OpenSearch PPL or DSL aggregation"
    action: rewrite
    evidence: tests/analytics/sales_parity.json
  - id: lifecycle-002
    source: "Elastic ILM atlasmart-logs-retention"
    target: "OpenSearch ISM atlasmart-logs-retention"
    action: translate
    evidence: tests/lifecycle/retention_state.md
  - id: client-009
    source: "elasticsearch-py 9.5.1"
    target: "opensearch-py 3.2.0"
    action: adapter
    evidence: tests/backend_contract/

7. Error behavior belongs in the contract

Test authentication failures, authorization 403s, 404s, malformed queries, connection timeout, request timeout, 429/rejection behavior, bulk per-item failures, and server unavailability. A client migration that changes retry count or retries non-idempotent writes can corrupt behavior even if every search request succeeds in staging.

DO NOT treat HTTP 200 from _bulk as success.
Parse every item:
- 2xx item => accepted
- 409 conflict => classify by write/idempotency policy
- 429 => bounded retry with backoff/jitter if safe
- 4xx mapping/auth => fail fast, do not infinite-retry
- 5xx => bounded retry only when the operation is idempotent/replayable
Record retry_count, final_status, and source event/document identity.

8. Plugins are version-coupled software, not copied files

Inventory every installed plugin and whether it is first-party, third-party, custom, or managed-service-only. Reinstall the correct target product/version plugin or redesign the feature. Native binaries, Java APIs, security hooks, and plugin descriptors are not cross-product promises. A plugin absence can be a migration blocker even when the index data is portable.

9. Application integration test matrix

10. Local lab baseline for this lesson

Use the established disposable local endpoints: Elasticsearch 9.5.3 at https://localhost:9200 with ELASTIC_PASSWORD and CA file atlasmart-es-http-ca; OpenSearch 3.8.0 at https://localhost:9201 with OPENSEARCH_INITIAL_ADMIN_PASSWORD. Both remain on Docker network atlasmart-search. The OpenSearch demo certificate may be bypassed with -k only in this local lab. Production migration requires verified TLS and separate least-privilege source, target, application, and migration identities.

11. Mini lab: run one application contract suite against both products

1. Create the same five-document AtlasMart migration fixture on both local endpoints.
2. Run the golden tenant-a hiking query and record top IDs, facet output, HTTP status, and only the response fields the application actually consumes.
3. Implement the same SearchBackend business interface with elasticsearch-py 9.5.1 for Elasticsearch and opensearch-py 3.2.0 for OpenSearch.
4. Inject one 403 and one malformed query. Verify both adapters normalize failures into the application-level error contract instead of leaking client-specific exception types.
5. Send a Bulk request with one deliberately invalid document and prove the application detects the per-item failure even if the HTTP response itself is successful.
6. Cleanup only the disposable atlasmart-migrate-source-v1 and atlasmart-migrate-target-v1 indices after saving the contract-test report.

Expected invariant: product-specific client code may differ, but the application contract, security decisions, idempotency rules, and acceptance evidence remain stable.

References and current-version checks

• Elastic Stack 9.5.3 release
• Elasticsearch snapshot and restore compatibility
• Elasticsearch restore snapshot guidance
• Elasticsearch reindex and reindex-from-remote
• Elasticsearch reindex settings
• Elasticsearch Python client compatibility
• Elasticsearch Python client release notes
• OpenSearch 3.8 version history
• OpenSearch upgrade or migrate guidance
• OpenSearch Reindex Documents API
• OpenSearch reindex data guidance
• OpenSearch language clients and compatibility
• OpenSearch Migration Assistant
• Migration Assistant supported migration paths
• Amazon OpenSearch Service snapshot migration