PotentiallyUnmappedKeywordEsField (used for fields loaded from _source
when unmapped_fields="load") had isAggregatable=true, which caused
isPushableFieldAttribute to short-circuit past the SearchStats check and
push filters down to Lucene. On shards where the field is not indexed,
the Lucene query returns no results instead of letting the compute
engine evaluate the filter on _source-loaded values.

Guard isPushableFieldAttribute against PotentiallyUnmappedKeywordEsField
so these fields are always filtered in the compute engine.

Co-authored-by: Cursor <cursoragent@cursor.com>

The isPushableFieldAttribute fix (rejecting PotentiallyUnmappedKeywordEsField)
already covers sort pushdown since PushTopNToSource uses the same method.
This test verifies correct sort order when a field is mapped in one index
but unmapped in another under unmapped_fields="load".

Co-authored-by: Cursor <cursoragent@cursor.com>

Remove _index from the SORT clause — it prevented the entire sort from
being pushed to Lucene (canPushDownOrders requires all fields pushable),
masking the bug. With only SORT message, the sort is pushed down and
produces wrong order on the unmapped shard without the fix.

Co-authored-by: Cursor <cursoragent@cursor.com>

Capture single-index unmapped behavior for nullify/load and switch GoldenTestCase to read unmapped settings from parsed statements so SET-based golden tests run like type_conflicts.

Co-authored-by: Cursor <cursoragent@cursor.com>
Made-with: Cursor

- Add OPTIONAL_FIELDS_V2 capability for pushdown-elimination BWC.
- Remove fieldAliasAndNonExistent test; fix capabilities on pushdown tests.
- Remove SORT from filterOnPartiallyUnmappedField; add fully unmapped
  multi-sort, and cast tests in unmapped-load.csv-spec.
- Add comment in LucenePushdownPredicates for PotentiallyUnmappedKeywordEsField.
- Add cross-refs between PushdownGoldenTests and LocalPhysicalPlanOptimizerTests.
- Fix assumeTrue: use OPTIONAL_FIELDS_V2 for load, remove redundant nullify.
- Write query.esql to nested path in GoldenTestCase. This was actually a
  pretty "noisy" change, since it also affect
  SubstituteRoundToGoldenTests!
- Add filter-only, conjunction pushable/non-pushable golden tests.

Made-with: Cursor

…ton-based queries (elastic#142150)

Datasource tests fail on Windows CI and FIPS CI builds due to two
independent issues introduced with the external sources feature.

**Windows:** `StoragePath.of()` cannot parse `file://` URIs with
Windows drive letters. A path like `file://C:\bk\path\file.txt`
causes the colon after the drive letter `C` to be misinterpreted
as a port separator, resulting in `NumberFormatException`. Both the
production code (`LocalStorageProvider.toStoragePath()`) and the tests
construct file URIs via manual string concatenation instead of using
the existing `StoragePath.fileUri()` helper that normalizes Windows
paths correctly.

**FIPS:** `ExternalDistributedSpecIT` starts a test cluster with
`xpack.security.enabled=false`, but FIPS mode requires security to
be enabled. The Elasticsearch process dies during startup before any
test method runs. Since the test relies on plain HTTP S3 fixtures
that are inherently incompatible with FIPS, the test is now skipped
in FIPS mode.

Developed using AI-assisted tooling

…l sources (elastic#143420)

* ESQL: Add extended distribution tests and fault injection for external sources

Extends the test coverage for external source distributed execution with
property tests for weighted round-robin and coalescing, and adds fault
injection infrastructure for testing resilience under storage failures.

- ExtendedDistributionPropertyTests: weighted RR load balancing bounds,
  coalescing preservation, coalesced+distribution integration
- FaultInjectionRetryTests: retry policy behavior under transient and
  persistent fault patterns (503, connection reset, timeout)
- FaultInjectingS3HttpHandler: configurable S3 fault injection with
  countdown, path filtering, and auto-clearing
- FaultInjectingS3HttpHandlerIT: real HTTP server tests for the handler

Developed using AI-assisted tooling

* Update docs/changelog/143420.yaml

* Fix CSV-escaped quotes rendering in generated docs examples

DocsV3Support.renderTableLine() now unescapes RFC 4180 CSV quoting
(strips outer quote delimiters and replaces "" with ") so that JSON
strings in function example tables render correctly — e.g.
{"key":"value"} instead of {""key"":""value""}.

Affects json_extract and to_tdigest doc examples.

* Refine CSV unescaping to only unescape RFC 4180 doubled quotes; add tests

The previous approach stripped outer quotes from ALL quoted values, breaking
simple quoted values like "POINT(...)" and "foo". Now only cells with actual
RFC 4180 doubled-quote escaping ("") are unescaped, leaving simple quoted
values unchanged.

Added tests: testRenderingExampleResultCsvJsonUnescaping verifies JSON
unescaping works, testRenderingExampleResultSimpleQuotesPreserved verifies
simple quoted values are not modified.

Also adds changelog YAML for the PR.

…stic#143402)

* Fix ES|QL function list versioning metadata

Audit all _snippets/lists/ files against Java @FunctionAppliesTo
annotations. Adds missing applies_to tags, corrects wrong versions,
and applies cumulative preview→ga tags where functions graduated.

Also adds missing applies_to front matter to
time-series-aggregation-functions.md landing page.

* update tags on commands lists

…rsionTests testLoadAll elastic#143471

Stopping the transform at the end of test, before the reset, can help
other tests running in parallel.

Resolve elastic#122980

Update the pattern_text downgrade test so that it includes adding docs and querying for docs.\
Specifically the test now does the following:
  1. Create data stream with trial license and pattern text field
  2. Index docs, verify they're searchable.
  3. Downgrade to basic license.
  4. Index more docs in same backing index, verify all docs searchable, verify settings unchanged.
  5. Rollover the data stream, verify the new backing index has disable_templating=true.
  6. Index more docs into the new backing index, verify all docs searchable across both indices.
  7. Search with "fields": ["pattern_field"] to verify the valueFetcher() code path works across both backing indices.

…lastic#143389)

This change disables index.mapping.use_doc_values_skipper in about 10% 
of tests executions of the main TSDB with synthetic id tests, as well as in 
some unit tests. It also makes the necessary changes in the postings format 
to account for missing doc values skippers.

Relates ES-14224

)

In order to determine if shards have been "skipped" it's useful to have a helper so that a dev can see it as an error state to be handled (if desired). This commit adds the `getUnavailableShards` method as that helper.

…142969)

* Adding headers file

* Using base class for task settings

* Refactoring base class to allow for returning empty instance

* Fixing test failures

* Fixing test

* Update docs/changelog/142969.yaml

* [CI] Auto commit changes from spotless

* clean up and consolidating request header logic

* Returning validation exception instead of xcontent when possible

* Adding unwrapper utils tests

* Update x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/services/azureopenai/AzureOpenAiService.java

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/services/azureopenai/request/AzureOpenAiRequest.java

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* [CI] Auto commit changes from spotless

* Address chatgpt feedback

* Adding test for unwrap exception

* Working refactor

* Removing duplicate method

* Adding tests for statefulvalue

* Using a single model reference

* Fixing test and renaming

* Fixing tests and refactoring validation

* [CI] Auto commit changes from spotless

* Removing exception unwrap logic

* Addressing feedback

* [CI] Auto commit changes from spotless

* Allowing null in update api

* Using orElse

---------

Co-authored-by: elasticsearchmachine <infra-root+elasticsearchmachine@elastic.co>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* add test to verify timestamp.us mapping and timestamp.us coercion behavior

* explicitly map timestamp.us to long

* apm-data: bump resource version

…ta (elastic#143367)

…n Java 21 (elastic#143479)

On Java 21, FFI disallows passing heap-backed MemorySegments to native downcalls. Java 22+ removes this restriction.

IndexInputUtils.withSlice is now the single path through which all simdvec scorers obtain MemorySegments from IndexInput data, and several of those scorers pass the segment directly to native downcalls. Rather than patching each and every call site across scorer classes individually, this fix takes a "safety by design" approach: withSlice itself now guarantees that the segment it hands to callers is always native-safe, regardless of Java version. No current or future caller needs to worry about the heap-segment restriction, correctness is enforced at the source.

I also added an assertion to the DirectAccessInput path that the byte buffer is direct, documenting the invariant that real implementations always provide native-backed buffers.

The tradeoff here is that on Java 21 only, the copyAndApply fallback path incurs an extra native allocation and copy (via a confined Arena) where a heap-backed segment would have sufficed, since a number of call sites only use the Panama Vector API and never touch native downcalls. On Java 22+ the behavior is unchanged. This is an acceptable cost: the fallback path is already the slowest path (mmap and direct buffer paths are preferred), and the alternative - requiring every call site to independently guard against heap segments, is fragile and has already proven easy to miss. We can do some further cleanup at the call site after this fix has been merged.

caused by elastic#141718
closes elastic#143441

… and elsewhere

Made-with: Cursor

# Conflicts:
#	x-pack/plugin/esql/qa/testFixtures/src/main/resources/unmapped-load.csv-spec
#	x-pack/plugin/esql/qa/testFixtures/src/main/resources/unmapped-nullify.csv-spec
#	x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/core/type/MultiTypeEsField.java
#	x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/core/type/PotentiallyUnmappedKeywordEsField.java
#	x-pack/plugin/esql/src/test/java/org/elasticsearch/xpack/esql/CsvTests.java
#	x-pack/plugin/esql/src/test/java/org/elasticsearch/xpack/esql/analysis/AnalyzerUnmappedTests.java