geo_point runtime field implementation by iverase · Pull Request #63164 · elastic/elasticsearch

iverase · 2020-10-02T09:12:33Z

Run time field that emits geo points from lat/lon values. For example, given a mapping like:

{
    "mappings": {
        "properties": {
            "location": {
                "lat": {"type" : "double"},
                "lon" : {"type" "double"}
            }
        }
    }
}

A geo_point runtime field can be declared as:

{
    "runtime_geo_point": {
        "type" : "runtime",
         "runtime_type": "geo_point",
         "script": {
             "source" : "emit(doc['location'].lat, doc['location'].lon);"
         }
}

where the emit function takes two arguments, the first argument is the latitude and the second argument the longitude.

elasticmachine · 2020-10-02T09:12:35Z

Pinging @elastic/es-analytics-geo (:Analytics/Geo)

elasticmachine · 2020-10-02T09:12:36Z

Pinging @elastic/es-search (:Search/Search)

...st/java/org/elasticsearch/xpack/runtimefields/mapper/GeoPointScriptMappedFieldTypeTests.java

iverase · 2020-10-05T12:58:53Z

@elasticmachine run elasticsearch-ci/packaging-sample-windows

iverase · 2020-10-05T13:28:42Z

@elasticmachine run elasticsearch-ci/1

nik9000

I took a look and it looks pretty reasonable.

...plugin/runtime-fields/src/main/java/org/elasticsearch/xpack/runtimefields/RuntimeFields.java

nik9000 · 2020-10-05T14:45:23Z

...elds/src/main/java/org/elasticsearch/xpack/runtimefields/mapper/GeoPointScriptFieldType.java

+        return new GeoPointScriptFieldGeoShapeQuery(script, leafFactory(context), fieldName, geometries);
+    }
+
+    private static LatLonGeometry[] toLuceneGeometry(String name, QueryShardContext context, Geometry geometry) {


Is this something we can move into core? It looks like something we'd have there already?

Thats is my plan but it seems more difficult that it looks at first sight as every time we need to map a geometry to a Lucene geometry, it seems to be slightly different. As part of that effort I opened https://issues.apache.org/jira/browse/LUCENE-9552 but I didn't have much support so far. I try to move it.

nik9000 · 2020-10-05T14:47:34Z

.../test/java/org/elasticsearch/xpack/runtimefields/mapper/AbstractScriptFieldTypeTestCase.java

        return mockContext(allowExpensiveQueries, null);
    }

+    protected boolean supportsTermQueries() {


Maybe it'd be better to make a new superclass that doesn't have these methods?

The problem is that we already have AbstractNonTextScriptFieldTypeTestCase that extends this class. Those methods are worthy running so adding a superclass you will miss that. My feeling is that we need an abstract class for every type of query and field types will implement all the ones they support?

So instead of an abstract superclass for the type of field it is per query? I could see that being better, maybe. I'm ok with this as is if we plan to refactor it soon enough anyway.

iverase · 2020-10-05T17:21:14Z

@elasticmachine run elasticsearch-ci/packaging-sample-windows

nik9000

This looks entirely sensible to me.

x-pack/plugin/src/test/resources/rest-api-spec/test/runtime_fields/100_geo_point.yml

iverase · 2020-10-08T12:28:14Z

Moved the PR to ready for review (although thanks Nik, you've done all the work already).

javanna

LGTM thanks @iverase . One meta ask is if you could update the description of the PR to include how you can define a geo point runtime field in the mappings, especially around the fact that emit in this case takes two arguments.

@nik9000 would you like to have one last look?

nik9000

I'm happy with it! I feel like I should find something worth changing because runtime fields are new and we don't have super established patterns, but I think this is the right things to do.

Run time field that emits geo points from lat/lon values.

geo_point runtime implementation

fd9eff3

iverase added >enhancement :Analytics/Geo Indexing, search aggregations of geo points and shapes :Search/Search Search-related issues that do not fall into other categories labels Oct 2, 2020

iverase requested review from javanna and nik9000 October 2, 2020 09:12

elasticmachine added Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) Team:Search Meta label for search team labels Oct 2, 2020

iverase marked this pull request as draft October 2, 2020 09:15

iverase added 2 commits October 2, 2020 11:32

Add geo_distance sort yaml test

47e4eb3

iter

767d398

iverase commented Oct 2, 2020

View reviewed changes

...st/java/org/elasticsearch/xpack/runtimefields/mapper/GeoPointScriptMappedFieldTypeTests.java Outdated Show resolved Hide resolved

iverase added 2 commits October 2, 2020 11:46

Shorten field type class name

2a6509a

cleanup tests

5a88e14

nik9000 reviewed Oct 5, 2020

View reviewed changes

iverase added 2 commits October 5, 2020 18:16

iter

9117fd9

iter

d94874c

iverase added 4 commits October 6, 2020 10:50

Move creation of LatLonGeometries to GeoShapeUtils

f53b662

Merge branch 'master' into runtime_geopoint

bb6cab1

compile errors

e7d84ca

precommit

2e9ec53

nik9000 reviewed Oct 7, 2020

View reviewed changes

x-pack/plugin/src/test/resources/rest-api-spec/test/runtime_fields/100_geo_point.yml Outdated Show resolved Hide resolved

iverase added 2 commits October 8, 2020 14:07

Merge branch 'master' into runtime_geopoint

8af770c

remove copy/paste nosense

a77aaa2

iverase marked this pull request as ready for review October 8, 2020 12:27

javanna approved these changes Oct 16, 2020

View reviewed changes

nik9000 approved these changes Oct 16, 2020

View reviewed changes

iverase added 2 commits October 19, 2020 14:54

Merge branch 'master' into runtime_geopoint

c983c00

Merge branch 'master' into runtime_geopoint

503d332

iverase merged commit 265a214 into elastic:master Oct 21, 2020

iverase mentioned this pull request Oct 21, 2020

geo_point runtime field implementation (#63164) #63968

Merged

iverase added v7.11.0 v8.0.0 labels Oct 21, 2020

iverase added a commit to iverase/elasticsearch that referenced this pull request Oct 21, 2020

geo_point runtime field implementation (elastic#63164)

5fdfc75

Run time field that emits geo points from lat/lon values.

iverase added a commit that referenced this pull request Oct 21, 2020

geo_point runtime field implementation (#63164) (#63968)

7104e21

Run time field that emits geo points from lat/lon values.

iverase deleted the runtime_geopoint branch October 21, 2020 10:06

pugnascotia pushed a commit to pugnascotia/elasticsearch that referenced this pull request Oct 21, 2020

geo_point runtime field implementation (elastic#63164)

1376afd

Run time field that emits geo points from lat/lon values.

javanna mentioned this pull request Nov 18, 2020

Runtime fields to support runtime type geo-shape #61299

Closed

stevejgordon mentioned this pull request Dec 17, 2020

7.11.0 Meta Ticket elastic/elasticsearch-net#5198

Closed

jakelandis added v8.0.0-alpha1 and removed v8.0.0 labels Jul 26, 2021

Conversation

iverase commented Oct 2, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

elasticmachine commented Oct 2, 2020

Uh oh!

elasticmachine commented Oct 2, 2020

Uh oh!

Uh oh!

iverase commented Oct 5, 2020

Uh oh!

iverase commented Oct 5, 2020

Uh oh!

nik9000 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

nik9000 Oct 5, 2020

Choose a reason for hiding this comment

Uh oh!

iverase Oct 5, 2020

Choose a reason for hiding this comment

Uh oh!

nik9000 Oct 5, 2020

Choose a reason for hiding this comment

Uh oh!

iverase Oct 5, 2020

Choose a reason for hiding this comment

Uh oh!

nik9000 Oct 5, 2020

Choose a reason for hiding this comment

Uh oh!

iverase commented Oct 5, 2020

Uh oh!

nik9000 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

iverase commented Oct 8, 2020

Uh oh!

javanna left a comment

Choose a reason for hiding this comment

Uh oh!

nik9000 left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

iverase commented Oct 2, 2020 •

edited

Loading