HTTP CRUD+ API for Index Patterns#83576
Merged
streamich merged 110 commits intoelastic:masterfrom Dec 8, 2020
streamich:index-patterns-http-api
Merged
HTTP CRUD+ API for Index Patterns#83576streamich merged 110 commits intoelastic:masterfrom streamich:index-patterns-http-api
streamich merged 110 commits intoelastic:masterfrom
streamich:index-patterns-http-api
Conversation
mattkime
reviewed
Dec 6, 2020
Contributor
mattkime
left a comment
There was a problem hiding this comment.
A few items to be addressed
src/plugins/data/common/index_patterns/index_patterns/index_patterns.ts
Outdated
Show resolved
Hide resolved
src/plugins/data/server/index_patterns/index_patterns_service.ts
Outdated
Show resolved
Hide resolved
src/plugins/data/server/index_patterns/routes/scripted_fields/create_scripted_field.ts
Outdated
Show resolved
Hide resolved
src/plugins/data/server/index_patterns/routes/scripted_fields/put_scripted_field.ts
Outdated
Show resolved
Hide resolved
src/plugins/data/server/index_patterns/routes/util/handle_errors.ts
Outdated
Show resolved
Hide resolved
src/plugins/discover/public/application/helpers/popularize_field.ts
Outdated
Show resolved
Hide resolved
mattkime
approved these changes
Dec 7, 2020
Contributor
💚 Build SucceededMetrics [docs]Distributable file count
Page load bundle
History
To update your PR or re-run it, just comment with: |
jloleysens
added a commit
to jloleysens/kibana
that referenced
this pull request
Dec 8, 2020
…or-disable-searchable-snapshot-fields * 'master' of github.com:elastic/kibana: (29 commits) HTTP CRUD+ API for Index Patterns (elastic#83576) Don't list packages in devDependencies already listed in dependencies (elastic#85120) Remove unused devDependencies (elastic#85121) [ILM] Reposition form toggles (elastic#85143) [APM] Make sure jest script can be run from anywhere (elastic#85111) Add EuiButtonEmptyTo components (elastic#85213) skip flaky suite (elastic#85216) skip flaky suite (elastic#83775) skip flaky suite (elastic#83774) skip flaky suite (elastic#83773) skip flaky suite (elastic#83793) skip flaky suite (elastic#85215) skip flaky suite (elastic#85217) [Usage Collection] Kibana Overview Page UI Metrics (elastic#81937) [Alerting UI] Replaced AlertsContextProvider with KibanaContextProvider and exposed components in API (elastic#84604) skip flaky suite (elastic#85208) [Security Solutions][Detection Engine] Fixes cypress errors by using latest signals mapping (elastic#84600) Small fixes to Kibana search bar (elastic#85079) Change link text to say Fleet (elastic#85083) [Metrics UI] Refactor Process List API to fetch only top processes (elastic#84716) ... # Conflicts: # x-pack/plugins/index_lifecycle_management/public/application/sections/edit_policy/components/phases/hot_phase/hot_phase.tsx # x-pack/plugins/index_lifecycle_management/public/application/sections/edit_policy/components/phases/shared_fields/searchable_snapshot_field/searchable_snapshot_field.tsx
streamich
added a commit
that referenced
this pull request
Dec 8, 2020
* refactor: 💡 extract out createIndexPatternsService() method * feat: 🎸 add create index pattern route * feat: 🎸 add index patterns http context * feat: 🎸 add schema for index pattern create method * refactor: 💡 compose schema out of variables * feat: 🎸 allow to not make the new index pattern a default one * feat: 🎸 implement index pattern create endpoint * docs: ✏️ add index pattern create endpoint docs * fix: 🐛 add error handling wrapper * feat: 🎸 add index pattern GET route * docs: ✏️ add index pattern GET mention to docs * feat: 🎸 add higher order route handler to check for context * feat: 🎸 add index pattern deletion endpoint * docs: ✏️ add delete endpoint docs entry * feat: 🎸 add index pattern update route * feat: 🎸 add ability to update timeFieldName * feat: 🎸 add ability to edit intervalName, sourceFilters, fields * feat: 🎸 allow updating index pattern type and typeMeta * feat: 🎸 add ability to update index_pattern fields * docs: ✏️ add index pattern update endpoint docs * docs: ✏️ fix typo * feat: 🎸 disable XSRF and add Content-Type * fix: 🐛 enable xsrf * feat: 🎸 send 404 status code when index pattern is not found * fix: 🐛 correctly report empty change set error * feat: 🎸 fields API add route for adding field to IP * docs: ✏️ add index pattern field create route * fix: 🐛 throw on creating an existing index pattern field * feat: 🎸 add field upsert route * feat: 🎸 add field get api endpoint * feat: 🎸 add field delete endpoint * refactor: 💡 improve 404 error handling * refactor: 💡 improve index pattern 404 error class * feat: 🎸 add field update api endpoint * docs: ✏️ typo * refactor: 💡 rename ski_field_refresh to refresh_fields in creat * refactor: 💡 rename to refresh_fields param in update call * refactor: 💡 rename parameter to refresh_fields in fields api * feat: 🎸 allow creating and deleting only scripted fields * style: 💄 fix TypeScript/Prettier error * test: 💍 add index pattern creation negative tests * test: 💍 add index pattern create method tests * feat: 🎸 remove make_default flag * docs: ✏️ update docs * test: 💍 add index patter api get tests * test: 💍 add delete index pattern tests * fix: 🐛 fix TypeScript error after refactor * docs: ✏️ update autogenerated docs * docs: ✏️ update autogenerated docs * refactor: 💡 change Fields API to Scripted Fields API * feat: 🎸 allow only scripted fields in Scripted Fields API * test: 💍 add index patter api update tests * test: 💍 add scripted field create api tests * test: 💍 add scripted field delete api tests * test: 💍 add scripted fields fetch api tests * test: 💍 add scripted field field put api tests * test: 💍 add scripted field update api test * feat: 🎸 add index pattern field update endpoint * feat: 🎸 improve index patterns fields api * test: 💍 fix test * docs: ✏️ update docs README * test: 💍 add field metadata deletion teests * docs: ✏️ improve index pattern api readme * fix: 🐛 fix TypeScript errors * test: 💍 update jest tests * docs: ✏️ update auto-generated docs * feat: 🎸 remove intervalName from index pattern create API * test: 💍 fix integration tests * fix: 🐛 make refresh_fields default to false * chore: 🤖 remove searchable and aggregatable from public API * chore: 🤖 remove various field properties * chore: 🤖 remove "index" attribute from public field schema * docs: ✏️ update docs to match API * chore: 🤖 fix TypeScript errors * test: 💍 try fixing functional tests * docs: ✏️ update auto-generated docs * chore: 🤖 check if this fixes security tests * fix: 🐛 create index patterns context only if user is authc * fix: 🐛 assert on index patterns service * test: 💍 try fixing integration tests * fix: 🐛 try fixing integration tests * chore: 🤖 update auto-generated docs * refactor: 💡 directly inject index pattern provider into create * refactor: 💡 create index pattern service directly in each route * refactor: 💡 remove index pattern route context provider * chore: 🤖 remove unused code * chore: 🤖 remove refresh_fields from scripted fields * style: 💄 change template param names * refactor: 💡 make field.count always be an integer * docs: ✏️ update auto-generated docs * refactor: 💡 inject index patterns using getStartServices * docs: ✏️ update autogenerated docs * docs: ✏️ update docs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes #72360
Closes #12756
Adds REST API for index patterns—a wrapper around the Index Patterns service.
POST /api/index_patterns/index_patternGET /api/index_patterns/index_pattern/{id}DELETE /api/index_patterns/index_pattern/{id}POST /api/index_patterns/index_pattern/{id}titletimeFieldNameintervalNamefieldssourceFiltersfieldFormatMaptypetypeMetacount,customLabel,format).PUT /api/index_patterns/index_pattern/{id}/fieldsPOST /api/index_patterns/index_pattern/{id}/scripted_fieldGET /api/index_patterns/index_pattern/{id}/scripted_field/{name}POST /api/index_patterns/index_pattern/{id}/scripted_field/{name}PUT /api/index_patterns/index_pattern/{id}/scripted_fieldDELETE /api/index_patterns/index_pattern/{id}/scripted_field/{name}Other:
See Postman API collection
{ "info": { "_postman_id": "c01b4f06-1a9a-4625-bb7e-8cc70000922c", "name": "Index Patterns", "schema": "https://schema.getpostman.com/json/collection/v2.1.0/collection.json" }, "item": [ { "name": "Fields API", "item": [ { "name": "Create field", "request": { "method": "POST", "header": [ { "key": "Content-Type", "name": "Content-Type", "type": "text", "value": "application/json" }, { "key": "kbn-xsrf", "type": "text", "value": "true" } ], "body": { "mode": "raw", "raw": "{\n \"field\": {\n \"name\": \"test3\"\n }\n}\n" }, "url": { "raw": "{{origin}}/api/index_patterns/index_pattern/ab2fb6d0-2b49-11eb-8840-4d2b9eed180a/field", "host": [ "{{origin}}" ], "path": [ "api", "index_patterns", "index_pattern", "ab2fb6d0-2b49-11eb-8840-4d2b9eed180a", "field" ] } }, "response": [] }, { "name": "Upsert field", "request": { "method": "PUT", "header": [ { "key": "Content-Type", "name": "Content-Type", "type": "text", "value": "application/json" }, { "key": "kbn-xsrf", "type": "text", "value": "true" } ], "body": { "mode": "raw", "raw": "{\n \"field\": {\n \"name\": \"test4\",\n \"type\": \"number\"\n }\n}\n" }, "url": { "raw": "{{origin}}/api/index_patterns/index_pattern/ab2fb6d0-2b49-11eb-8840-4d2b9eed180a/field", "host": [ "{{origin}}" ], "path": [ "api", "index_patterns", "index_pattern", "ab2fb6d0-2b49-11eb-8840-4d2b9eed180a", "field" ] } }, "response": [] }, { "name": "Read field", "protocolProfileBehavior": { "disableBodyPruning": true }, "request": { "method": "GET", "header": [ { "key": "Content-Type", "name": "Content-Type", "type": "text", "value": "application/json" }, { "key": "kbn-xsrf", "type": "text", "value": "true" } ], "body": { "mode": "raw", "raw": "{\n \"field\": {\n \"name\": \"test4\",\n \"type\": \"number\"\n }\n}\n" }, "url": { "raw": "{{origin}}/api/index_patterns/index_pattern/ab2fb6d0-2b49-11eb-8840-4d2b9eed180a/field/test4", "host": [ "{{origin}}" ], "path": [ "api", "index_patterns", "index_pattern", "ab2fb6d0-2b49-11eb-8840-4d2b9eed180a", "field", "test4" ] } }, "response": [] }, { "name": "Update field", "request": { "method": "POST", "header": [ { "key": "Content-Type", "name": "Content-Type", "type": "text", "value": "application/json" }, { "key": "kbn-xsrf", "type": "text", "value": "true" } ], "body": { "mode": "raw", "raw": "{\n \"field\": {\n \"count\": 0\n }\n}" }, "url": { "raw": "{{origin}}/api/index_patterns/index_pattern/ab2fb6d0-2b49-11eb-8840-4d2b9eed180a/field/test4", "host": [ "{{origin}}" ], "path": [ "api", "index_patterns", "index_pattern", "ab2fb6d0-2b49-11eb-8840-4d2b9eed180a", "field", "test4" ] } }, "response": [] }, { "name": "Delete field", "request": { "method": "DELETE", "header": [ { "key": "Content-Type", "name": "Content-Type", "type": "text", "value": "application/json" }, { "key": "kbn-xsrf", "type": "text", "value": "true" } ], "body": { "mode": "raw", "raw": "{\n \"field\": {\n \"name\": \"test4\",\n \"type\": \"number\"\n }\n}\n" }, "url": { "raw": "{{origin}}/api/index_patterns/index_pattern/ab2fb6d0-2b49-11eb-8840-4d2b9eed180a/field/test4", "host": [ "{{origin}}" ], "path": [ "api", "index_patterns", "index_pattern", "ab2fb6d0-2b49-11eb-8840-4d2b9eed180a", "field", "test4" ] } }, "response": [] } ] }, { "name": "Create index pattern (simple)", "request": { "method": "POST", "header": [ { "key": "Content-Type", "name": "Content-Type", "value": "application/json", "type": "text" }, { "key": "kbn-xsrf", "value": "true", "type": "text" } ], "body": { "mode": "raw", "raw": "{\n \"index_pattern\": {\n \"title\": \"test\"\n }\n}\n" }, "url": { "raw": "{{origin}}/api/index_patterns/index_pattern", "host": [ "{{origin}}" ], "path": [ "api", "index_patterns", "index_pattern" ] } }, "response": [] }, { "name": "Update index pattern (title)", "request": { "method": "POST", "header": [ { "key": "Content-Type", "name": "Content-Type", "type": "text", "value": "application/json" }, { "key": "kbn-xsrf", "value": "true", "type": "text" } ], "body": { "mode": "raw", "raw": "{\n \"index_pattern\": {\n \"title\": null\n }\n}\n" }, "url": { "raw": "{{origin}}/api/index_patterns/index_pattern/e1a2d0d0-2b3f-11eb-87a6-8b5177386415", "host": [ "{{origin}}" ], "path": [ "api", "index_patterns", "index_pattern", "e1a2d0d0-2b3f-11eb-87a6-8b5177386415" ] } }, "response": [] }, { "name": "Read index pattern", "protocolProfileBehavior": { "disableBodyPruning": true }, "request": { "method": "GET", "header": [ { "key": "Content-Type", "name": "Content-Type", "type": "text", "value": "application/json" } ], "body": { "mode": "raw", "raw": "{\n \"index_pattern\": {\n \"title\": \"foo\"\n }\n}\n" }, "url": { "raw": "{{origin}}/api/index_patterns/index_pattern/e1a2d0d0-2b3f-11eb-87a6-8b5177386415", "host": [ "{{origin}}" ], "path": [ "api", "index_patterns", "index_pattern", "e1a2d0d0-2b3f-11eb-87a6-8b5177386415" ] } }, "response": [] }, { "name": "Delete index pattern", "request": { "method": "DELETE", "header": [ { "key": "Content-Type", "name": "Content-Type", "type": "text", "value": "application/json" }, { "key": "kbn-xsrf", "value": "true", "type": "text" } ], "body": { "mode": "raw", "raw": "{\n \"index_pattern\": {\n \"title\": \"foo\"\n }\n}\n" }, "url": { "raw": "{{origin}}/api/index_patterns/index_pattern/c03a9d60-2b35-11eb-bbe5-9f48d9a2a98f", "host": [ "{{origin}}" ], "path": [ "api", "index_patterns", "index_pattern", "c03a9d60-2b35-11eb-bbe5-9f48d9a2a98f" ] } }, "response": [] } ], "auth": { "type": "basic", "basic": [ { "key": "password", "value": "changeme", "type": "string" }, { "key": "username", "value": "elastic", "type": "string" } ] }, "event": [ { "listen": "prerequest", "script": { "id": "ea2d6011-8cc5-4123-94e8-317e44ca1e00", "type": "text/javascript", "exec": [ "" ] } }, { "listen": "test", "script": { "id": "3d326b30-1ad1-4079-a160-388876d120ef", "type": "text/javascript", "exec": [ "" ] } } ], "variable": [ { "id": "0602ae6f-3ba7-4639-9bfc-e091521d6204", "key": "origin", "value": "", "type": "string" } ] }Checklist
Delete any items that are not applicable to this PR.
For maintainers
Release note
Index Patterns service now has public HTTP API, which third parties can use to manage index patterns, index pattern field metadata, and scripted fields programatically. API consists of:
POST /api/index_patterns/index_pattern{id}—GET /api/index_patterns/index_pattern/{id}{id}—DELETE /api/index_patterns/index_pattern/{id}{id}—POST /api/index_patterns/index_pattern/{id}POST /api/index_patterns/index_pattern/{id}/fieldsPOST /api/index_patterns/index_pattern/{id}/scripted_fieldPUT /api/index_patterns/index_pattern/{id}/scripted_fieldGET /api/index_patterns/index_pattern/{id}/scripted_field/{name}DELETE /api/index_patterns/index_pattern/{id}/scripted_field/{name}POST /api/index_patterns/index_pattern/{id}/scripted_field/{name}Documentation
Index Patterns HTTP API
Index patterns provide Rest-like HTTP CRUD+ API with the following endpoints:
POST /api/index_patterns/index_pattern{id}—GET /api/index_patterns/index_pattern/{id}{id}—DELETE /api/index_patterns/index_pattern/{id}{id}—POST /api/index_patterns/index_pattern/{id}POST /api/index_patterns/index_pattern/{id}/fieldsPOST /api/index_patterns/index_pattern/{id}/scripted_fieldPUT /api/index_patterns/index_pattern/{id}/scripted_fieldGET /api/index_patterns/index_pattern/{id}/scripted_field/{name}DELETE /api/index_patterns/index_pattern/{id}/scripted_field/{name}POST /api/index_patterns/index_pattern/{id}/scripted_field/{name}Index Patterns API
Index Patterns CURD API allows you to create, retrieve and delete index patterns. I also
exposes an update endpoint which allows you to update specific fields without changing
the rest of the index pattern object.
Create an index pattern
Create an index pattern with a custom title.
Customize creation behavior with:
override— if set totrue, replaces an existing index pattern if anindex pattern with the provided title already exists. Defaults to
false.refresh_fields— if set totruereloads index pattern fields afterthe index pattern is stored. Defaults to
false.At creation all index pattern fields are option and you can provide them.
The endpoint returns the created index pattern object.
{ "index_pattern": {} }Fetch an index pattern by ID
Retrieve and index pattern by its ID.
Returns an index pattern object.
{ "index_pattern": { "id": "...", "version": "...", "title": "...", "type": "...", "intervalName": "...", "timeFieldName": "...", "sourceFilters": [], "fields": {}, "typeMeta": {}, "fieldFormats": {}, "fieldAttrs": {} } }Delete an index pattern by ID
Delete and index pattern by its ID.
Returns an '200 OK` response with empty body on success.
Partially update an index pattern by ID
Update part of an index pattern. Only provided fields will be updated on the
index pattern, missing fields will stay as they are persisted.
These fields can be update partially:
titletimeFieldNameintervalNamefields(optionally refresh fields)sourceFiltersfieldFormatMaptypetypeMetaUpdate a title of an index pattern.
All update fields are optional, you can specify the following fields.
refresh_fields— if set totruereloads index pattern fields afterthe index pattern is stored. Defaults to
false.This endpoint returns the updated index pattern object.
{ "index_pattern": { } }Fields API
Fields API allows to change field metadata, such as
count,customLabel, andformat.Update fields
Update endpoint allows you to update fields presentation metadata, such as
count,customLabel, andformat. You can update multiple fields in one request. Updatesare merges with persisted metadata. To remove existing metadata specify
nullas value.Set popularity
countfor fieldfoo:Update multiple metadata values and fields in one request:
Use
nullvalue to delete metadata:This endpoint returns the updated index pattern object.
{ "index_pattern": { } }Scripted Fields API
Scripted Fields API provides CRUD API for scripted fields of an index pattern.
Create a scripted field
Create a field by simply specifying its name, will default to
stringtype. Returnsan error if a field with the provided name already exists.
Create a field by specifying all field properties.
Upsert a scripted field
Creates a new field or updates an existing one, if one already exists with the same name.
Create a field by simply specifying its name.
Create a field by specifying all field properties.
Fetch a scripted field
Fetch an existing index pattern field by field name.
Returns the field object.
{ "field": {} }Delete a scripted field
Delete a field of an index pattern.
Update a an existing scripted field
Updates an exiting field by mergin provided properties with the existing ones. If
there is no existing field with the specified name, returns a
404 Not Founderror.You can specify any field properties, except
namewhich is specified in the URL path.