[WIP] Update elasticsearch connector to handle up to v6 #343
Conversation
|
@n-riesco I would be interested in working with you on these fixes. Do you want to create a separate branch on the falcon-sql-client that we can merge into? One other todo that I would like to suggest is that we add the ability to define a query for Elastic Search |
c6d2688 to
9b0ad9d
Compare
* Moved elasticsearch specs into their own file. * Do not catch failures in elasticsearch specs. * Updated connect spec to test all indices. * Updated elasticsearchMappings spec to test only the test indices. * Added scripts to run elasticsearch and datastores specs.
* Added datastores.elasticsearch-v2.spec.js with specs against the mocked responses from an Elasticsearch server v2. * Added script test-unit-elasticsearch-v2.
* Added datastores.elasticsearch-v5.spec.js with specs against the mocked responses from an Elasticsearch server v5. * Added script test-unit-elasticsearch-v5. * Added Dockerfile to build a container that runs an elasticsearch server with some sample indices for testing. This server was used to generate the mocked responses. * The main differences I've found between v2 and v5 is that `type: 'string'` in v2 becomes `type: 'text'` in v5. * The dockerised server uses 5 nodes. The main consequence of this setup is that the order of the documents returned by queries may change. * Also, note that when I created the index/type text-scroll/200k, I had to set max_response_window to 200001. * Added script docker:elasticsearch:build to build a dockerised elasticsearch server for testing. * Added script docker:elasticsearch:start to launched the dockerised test server.
9b0ad9d to
bbc28bd
Compare
|
@shannonlal I'm sorry I haven't moved this branch into falcon's repo yet. But I didn't want to lose the freedom to do a forced git push, before I got the Dockerfile right. This is how far I got:
The main result from the work so far is that Falcon's tests pass with either elasticsearch v2.4 or v5.6.7. @shannonlal Are you still able to reproduce the errors in this comment? |
|
After talking discussion with @n-riesco we agreed to merge this in 💃 . |
TODO
connect: fails if user didn't provide the port number(moved to Check in fix for ES behind Nginx #370)_cat/indicesto validate the connection (see comment ElasticSearch v 5 #316 (comment))works behind Nginx(moved to Check in fix for ES behind Nginx #370)Closes #316
Closes #317