apm-data: explicit map of timestamp.us to long by endorama · Pull Request #143173 · elastic/elasticsearch

endorama · 2026-02-26T17:25:12Z

We recently discovered a bug related to a missing mapping: elastic/apm-server#20496

This PR fixes it by adding the proper mapping in template for field timestamp.us on traces data streams.

Bumped resource version to ensure upgrade path works too.

Closes elastic/apm-server#20496

We would like to backport this fix to 8.19, 9.2 9.3, but I'm not sure how to trigger the automation.

Add known issue for elastic/elasticsearch#143173 and elastic#20496

endorama · 2026-02-27T12:32:17Z

I tested 2 different paths for this fix:

Sending data to the development build that includes this fix
Upgrading from 9.3.1 to the development build that includes this fix

The setup was APM Server => Logstash => Elasticsearch, to trigger the bug.

What we want is for timestamp.us to be mapped always as long.

Checking the type of timestamp.us field in the current version gives:

❯ curl -s "http://localhost:9200/_field_caps?fields=timestamp.us&index=traces-apm*&pretty"
{
  "indices" : [
    ".ds-traces-apm-default-2026.02.26-000001"
  ],
  "fields" : {
    "timestamp.us" : {
      "float" : {
        "type" : "float",
        "metadata_field" : false,
        "searchable" : true,
        "aggregatable" : true
      }
    },
    "timestamp" : {
      "object" : {
        "type" : "object",
        "metadata_field" : false,
        "searchable" : false,
        "aggregatable" : false
      }
    }
  }
}

Using the development build with this fix

An example Java application sending a single trace on requests to / was my telemetry generator.

Running the development build, we can see that the type has changed to long.

❯ curl -s "http://localhost:9200/_field_caps?fields=timestamp.us&index=traces-apm*&pretty"
{
  "indices" : [
    ".ds-traces-apm-default-2026.02.26-000001"
  ],
  "fields" : {
    "timestamp.us" : {
      "long" : {
        "type" : "long",
        "metadata_field" : false,
        "searchable" : true,
        "aggregatable" : true
      }
    },
    "timestamp" : {
      "object" : {
        "type" : "object",
        "metadata_field" : false,
        "searchable" : false,
        "aggregatable" : false
      }
    }
  }
}

Upgrading from `9.3.1` to the development build with this fix

❯ curl -s "http://localhost:9200/_field_caps?fields=timestamp.us&index=traces-apm*&pretty"

{
  "fields": {
    "timestamp": {
      "object": {
        "aggregatable": false,
        "metadata_field": false,
        "searchable": false,
        "type": "object"
      }
    },
    "timestamp.us": {
      "float": {
        "aggregatable": true,
        "metadata_field": false,
        "searchable": true,
        "type": "float"
      }
    }
  },
  "indices": [
    ".ds-traces-apm-default-2026.02.27-000001"
  ]
}

after upgrading to development version:

{
  "fields": {
    "timestamp": {
      "object": {
        "aggregatable": false,
        "metadata_field": false,
        "searchable": false,
        "type": "object"
      }
    },
    "timestamp.us": {
      "long": {
        "aggregatable": true,
        "metadata_field": false,
        "searchable": true,
        "type": "long"
      }
    }
  },
  "indices": [
    ".ds-traces-apm-default-2026.02.27-000002"
  ]
}

We can observe a rollover happened from .ds-traces-apm-default-2026.02.27-000001 to .ds-traces-apm-default-2026.02.27-000002 and the field from float is now long.

@lucabelluccini

…arch output is used (#20508) * Known issue - timestamp.us as float Add known issue for elastic/elasticsearch#143173 and #20496 * Apply suggestion from @lucabelluccini * Apply suggestion from @lucabelluccini * Apply suggestion from @lucabelluccini

elasticsearchmachine · 2026-02-27T13:42:14Z

Pinging @elastic/es-storage-engine (Team:StorageEngine)

elasticsearchmachine · 2026-02-27T13:42:15Z

Hi @endorama, I've created a changelog YAML for you.

carsonip

code change lgtm, 1 nit on test

x-pack/plugin/apm-data/src/yamlRestTest/resources/rest-api-spec/test/10_apm.yml

endorama · 2026-03-03T14:52:05Z

part-2 checks are successful: https://buildkite.com/elastic/elasticsearch-pull-request/builds/124971/steps/canvas?sid=019cb3a4-e794-480c-9a62-3750dc7c7187&tab=output

The failures looks unrelated to current changes. Updating main.

elasticsearchmachine · 2026-03-03T16:35:45Z

💔 Backport failed

You can use sqren/backport to manually backport by running backport --upstream elastic/elasticsearch --pr 143173

* add test to verify timestamp.us mapping and timestamp.us coercion behavior * explicitly map timestamp.us to long * apm-data: bump resource version

…locations * upstream/main: (51 commits) ESQL: Remaining serialization tests (elastic#143470) Eagerly release resources in `TransportAwaitClusterStateVersionAppliedAction` (elastic#143477) Stop and relocate sliced reindex on shutdown (elastic#143183) Documentation for query_vector base64 parameter (elastic#142675) ES|QL: Fix LIMIT after all columns are dropped (elastic#143463) Update docs-build.yml (elastic#142958) Fix KnnIndexTester to work with byte vectors (elastic#143493) Fix IndexInputUtils.withSlice to produce native-safe MemorySegments on Java 21 (elastic#143479) CPS fix: include only relevant projects in the search response metadata (elastic#143367) apm-data: explicit map of timestamp.us to long (elastic#143173) [Inference API] Add custom headers for Azure OpenAI Service (elastic#142969) ESQL: Add name IDs to golden tests and fix synthetic names (elastic#143450) Add getUnavailableShards to BaseBroadcastResponse (elastic#143406) Add description to reindex API without sensitive info (elastic#143112) SQL: fix CLI tests (elastic#143451) ES|QL: Add note of future removal of FORK implicit LIMIT (elastic#143457) [Test] Randomly disable doc values skippers in time-series indices (elastic#143389) Improve pattern text downgrade license test (elastic#143102) [Transform] Stop transforms at the end of tests (elastic#139783) Mute org.elasticsearch.compute.lucene.read.ValueSourceReaderTypeConversionTests testLoadAll elastic#143471 ...

* add test to verify timestamp.us mapping and timestamp.us coercion behavior * explicitly map timestamp.us to long * apm-data: bump resource version

* add test to verify timestamp.us mapping and timestamp.us coercion behavior * explicitly map timestamp.us to long * apm-data: bump resource version (cherry picked from commit 9b887f1)

endorama · 2026-03-04T16:30:18Z

💚 All backports created successfully

Status	Branch	Result
✅	9.3
✅	9.2
✅	8.19

Questions ?

Please refer to the Backport tool documentation

* add test to verify timestamp.us mapping and timestamp.us coercion behavior * explicitly map timestamp.us to long * apm-data: bump resource version (cherry picked from commit 9b887f1) # Conflicts: # x-pack/plugin/apm-data/src/main/resources/resources.yaml

* add test to verify timestamp.us mapping and timestamp.us coercion behavior * explicitly map timestamp.us to long * apm-data: bump resource version (cherry picked from commit 9b887f1)

* add test to verify timestamp.us mapping and timestamp.us coercion behavior * explicitly map timestamp.us to long * apm-data: bump resource version (cherry picked from commit 9b887f1) # Conflicts: # x-pack/plugin/apm-data/src/main/resources/resources.yaml

endorama added 3 commits February 26, 2026 17:26

add test

bc77ac3

explicitly map timestamp.us to long

db2698a

apm-data: bump resource version

d38e760

elasticsearchmachine added the v9.4.0 label Feb 26, 2026

endorama mentioned this pull request Feb 26, 2026

Broken waterfall view due to wrong mapping of timestamp.us field elastic/apm-server#20496

Closed

lucabelluccini added a commit to lucabelluccini/apm-server that referenced this pull request Feb 26, 2026

Known issue - timestamp.us as float

7a984d0

Add known issue for elastic/elasticsearch#143173 and elastic#20496

lucabelluccini mentioned this pull request Feb 26, 2026

Known issue - timestamp.us as float for APM Server when non-Elasticsearch output is used elastic/apm-server#20508

Merged

endorama marked this pull request as ready for review February 27, 2026 12:32

endorama requested a review from a team as a code owner February 27, 2026 12:32

elasticsearchmachine added the needs:triage Requires assignment of a team area label label Feb 27, 2026

endorama added Team:obs-ds-intake-services Meta label for Observability Intake Services team >bug labels Feb 27, 2026

elasticsearchmachine removed the Team:obs-ds-intake-services Meta label for Observability Intake Services team label Feb 27, 2026

endorama added the Team:obs-ds-intake-services Meta label for Observability Intake Services team label Feb 27, 2026

elasticsearchmachine removed the Team:obs-ds-intake-services Meta label for Observability Intake Services team label Feb 27, 2026

endorama self-assigned this Feb 27, 2026

lahsivjar approved these changes Feb 27, 2026

View reviewed changes

endorama added the :StorageEngine/Data streams Data streams and their lifecycles label Feb 27, 2026

elasticsearchmachine added the Team:StorageEngine label Feb 27, 2026

elasticsearchmachine removed the needs:triage Requires assignment of a team area label label Feb 27, 2026

Update docs/changelog/143173.yaml

377c75e

Merge branch 'main' into apm-data-timestampus_mapping

8850d2a

carsonip approved these changes Mar 2, 2026

View reviewed changes

x-pack/plugin/apm-data/src/yamlRestTest/resources/rest-api-spec/test/10_apm.yml Show resolved Hide resolved

add test for timestamp.us cohercion behavior

82efca8

carsonip approved these changes Mar 2, 2026

View reviewed changes

clarify role of timestamp.us mapping test

8ef97c1

carsonip approved these changes Mar 3, 2026

View reviewed changes

Merge branch 'main' into apm-data-timestampus_mapping

aaef3a1

endorama added auto-backport Automatically create backport pull requests when merged v9.3.2 v8.19.13 v9.2.7 labels Mar 3, 2026

endorama enabled auto-merge (squash) March 3, 2026 15:45

endorama merged commit 9b887f1 into elastic:main Mar 3, 2026
31 of 34 checks passed

elasticsearchmachine added the backport pending label Mar 3, 2026

ericywl mentioned this pull request Mar 4, 2026

iserver-test: Fix rollover expectation elastic/apm-server#20560

Merged

This was referenced Mar 4, 2026

[9.3] apm-data: explicit map of timestamp.us to long (#143173) #143593

Merged

[9.2] apm-data: explicit map of timestamp.us to long (#143173) #143594

Merged

endorama mentioned this pull request Mar 4, 2026

[8.19] apm-data: explicit map of timestamp.us to long (#143173) #143599

Merged

This was referenced Mar 6, 2026

[ML] Wait for cluster state in test #143767

Merged

[Transform] Disable PIT for CPS #143876

Closed

This was referenced Mar 11, 2026

8.19.13 Test Plan elastic/apm-server#20631

Closed

9.3.2 Test Plan elastic/apm-server#20633

Closed

9.2.7 Test Plan elastic/apm-server#20632

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

apm-data: explicit map of timestamp.us to long#143173

apm-data: explicit map of timestamp.us to long#143173
endorama merged 12 commits intoelastic:mainfrom
endorama:apm-data-timestampus_mapping

endorama commented Feb 26, 2026 •

edited

Loading

Uh oh!

endorama commented Feb 27, 2026 •

edited

Loading

Uh oh!

elasticsearchmachine commented Feb 27, 2026

Uh oh!

elasticsearchmachine commented Feb 27, 2026

Uh oh!

carsonip left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

endorama commented Mar 3, 2026

Uh oh!

Uh oh!

elasticsearchmachine commented Mar 3, 2026

Uh oh!

endorama commented Mar 4, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

endorama commented Feb 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

endorama commented Feb 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Using the development build with this fix

Upgrading from 9.3.1 to the development build with this fix

Uh oh!

elasticsearchmachine commented Feb 27, 2026

Uh oh!

elasticsearchmachine commented Feb 27, 2026

Uh oh!

carsonip left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

endorama commented Mar 3, 2026

Uh oh!

Uh oh!

elasticsearchmachine commented Mar 3, 2026

💔 Backport failed

Uh oh!

endorama commented Mar 4, 2026

💚 All backports created successfully

Questions ?

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

endorama commented Feb 26, 2026 •

edited

Loading

endorama commented Feb 27, 2026 •

edited

Loading

Upgrading from `9.3.1` to the development build with this fix