Skip to content

Commit 3053fbe

Browse files
kibanamachinekibanamachine
authored
Merge branch 'master' into nls/map-backend-link
2 parents 2cc48b1 + 923eca0 commit 3053fbe

158 files changed

Lines changed: 8023 additions & 4915 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎api_docs/timelines.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10918,7 +10918,7 @@
1091810918
"label": "alertConsumers",
1091910919
"description": [],
1092010920
"signature": [
10921-
"ALERTS_CONSUMERS",
10921+
"AlertConsumers",
1092210922
"[] | undefined"
1092310923
],
1092410924
"path": "x-pack/plugins/timelines/common/search_strategy/timeline/index.ts",

‎docs/maps/asset-tracking-tutorial.asciidoc‎

Lines changed: 340 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,340 @@
1+
[role="xpack"]
2+
[[asset-tracking-tutorial]]
3+
== Track, visualize, and alert on assets in real time
4+
5+
Are you interested in asset tracking? Good news! Visualizing and analyzing data that moves is easy with *Maps*. You can track the location of an IoT device and monitor a package or vehicle in transit.
6+
7+
In this tutorial, you’ll look at live urban transit data from the city of Portland, Oregon. You’ll watch the city buses, use the data to visualize congestion, and notify a dispatch team when a bus enters a construction zone.
8+
9+
You’ll learn to:
10+
11+
- Use Logstash to ingest the TriMet REST API into Elasticsearch.
12+
- Create a map with layers that visualize asset tracks and last-known locations.
13+
- Use symbols and colors to style data values and show which direction an asset is heading.
14+
- Set up tracking containment alerts to monitor moving vehicles.
15+
16+
When you complete this tutorial, you’ll have a map that looks like this:
17+
18+
[role="screenshot"]
19+
image::maps/images/asset-tracking-tutorial/construction_zones.png[]
20+
21+
[float]
22+
=== Prerequisites
23+
24+
- If you don’t already have {kib}, set it up with https://www.elastic.co/cloud/elasticsearch-service/signup?baymax=docs-body&elektra=docs[our free trial]. Download the deployment credentials.
25+
- Obtain an API key for https://developer.trimet.org/[TriMet web services] at https://developer.trimet.org/appid/registration/.
26+
- https://www.elastic.co/guide/en/logstash/current/getting-started-with-logstash.html[Install Logstash].
27+
28+
[float]
29+
=== Part 1: Ingest the Portland bus data
30+
To get to the fun of visualizing and alerting on Portland buses, you must first create a Logstash pipeline to ingest the TriMet Portland bus data into {es}.
31+
32+
[float]
33+
==== Step 1: Set up an Elasticsearch index
34+
35+
. In Kibana, open the main menu, then click *Dev Tools*.
36+
. In *Console*, create the `tri_met_tracks` index:
37+
+
38+
[source,js]
39+
----------------------------------
40+
PUT tri_met_tracks
41+
----------------------------------
42+
43+
. To configure the `tri_met_tracks` index mappings, run:
44+
+
45+
[source,js]
46+
----------------------------------
47+
PUT tri_met_tracks/_mapping
48+
{
49+
"properties": {
50+
"in_congestion": {
51+
"type": "boolean"
52+
},
53+
"location": {
54+
"type": "geo_point"
55+
},
56+
"route": {
57+
"type": "keyword"
58+
},
59+
"time": {
60+
"type": "date",
61+
"format": "epoch_millis"
62+
},
63+
"type": {
64+
"type": "keyword"
65+
},
66+
"vehicle_id": {
67+
"type": "keyword"
68+
}
69+
}
70+
}
71+
----------------------------------
72+
73+
[float]
74+
==== Step 2: Start Logstash
75+
76+
. In your `logstash/config` folder, create the file `trimet-pipeline.conf`.
77+
. Copy the pipeline script into your `trimet-pipeline.conf` file.
78+
+
79+
[source,yaml]
80+
----------------------------------
81+
input {
82+
http_poller {
83+
urls => {
84+
trimet => "https://developer.trimet.org/ws/v2/vehicles?appID=<tri_met_app_id>"
85+
}
86+
request_timeout => 60
87+
schedule => { cron => "* * * * * UTC"}
88+
codec => "json"
89+
}
90+
}
91+
92+
filter {
93+
split {
94+
field => "[resultSet][vehicle]"
95+
}
96+
97+
if ![resultSet][vehicle][inCongestion] {
98+
mutate {
99+
update => {
100+
"[resultSet][vehicle][inCongestion]" => "false"
101+
}
102+
}
103+
}
104+
105+
mutate {
106+
add_field => {
107+
"bearing" => "%{[resultSet][vehicle][bearing]}"
108+
"in_congestion" => "%{[resultSet][vehicle][inCongestion]}"
109+
"location" => "%{[resultSet][vehicle][latitude]},%{[resultSet][vehicle][longitude]}"
110+
"route" => "%{[resultSet][vehicle][routeNumber]}"
111+
"time" => "%{[resultSet][vehicle][time]}"
112+
"type" => "%{[resultSet][vehicle][type]}"
113+
"vehicle_id" => "%{[resultSet][vehicle][vehicleID]}"
114+
}
115+
remove_field => [ "resultSet", "@version", "@timestamp" ]
116+
}
117+
118+
mutate {
119+
convert => {
120+
"bearing" => "float"
121+
"in_congestion" => "boolean"
122+
"time" => "integer"
123+
}
124+
}
125+
}
126+
127+
output {
128+
stdout {
129+
codec => rubydebug
130+
}
131+
132+
elasticsearch {
133+
cloud_auth => "<username:password>"
134+
cloud_id => "<cloud_id>"
135+
index => "tri_met_tracks"
136+
document_id => "%{[vehicle_id]}_%{[time]}"
137+
}
138+
}
139+
----------------------------------
140+
141+
. Replace `<tri_met_app_id>` with your TriMet application id.
142+
. Replace `<username:password>` with your Elastic Cloud deployment credentials.
143+
. Replace `<cloud_id>` with your {ece}/ece-cloud-id.html[elastic cloud id].
144+
. Open a terminal window, and then navigate to the Logstash folder.
145+
. In your `logstash` folder, run Logstash with the TriMet pipeline:
146+
+
147+
[source,bash]
148+
----------------------------------
149+
bin/logstash -f config/trimet-pipeline.conf
150+
----------------------------------
151+
152+
. Wait for Logstash to initialize and confirm data is flowing. You should see messages similar to this:
153+
+
154+
[role="screenshot"]
155+
image::maps/images/asset-tracking-tutorial/logstash_output.png[]
156+
. Leave the terminal window open and Logstash running throughout this tutorial.
157+
158+
[float]
159+
==== Step 3: Create a {kib} index pattern for the tri_met_tracks {es} index
160+
161+
. In Kibana, open the main menu, and click *Stack Management > Index Patterns*.
162+
. Click *Create index pattern*.
163+
. Give the index pattern a name: *tri_met_tracks**.
164+
. Click *Next step*.
165+
. Set the *Time field* to *time*.
166+
. Click *Create index pattern*.
167+
168+
{kib} shows the fields in your index pattern.
169+
170+
[role="screenshot"]
171+
image::maps/images/asset-tracking-tutorial/index_pattern.png[]
172+
173+
[float]
174+
==== Step 4: Explore the Portland bus data
175+
176+
. Open the main menu, and click *Discover*.
177+
. Set the index pattern to *tri_met_tracks**.
178+
. Open the <<set-time-filter, time filter>>, and set the time range to the last 15 minutes.
179+
. Expand a document and explore some of the fields that you will use later in this tutorial: `bearing`, `in_congestion`, `location`, and `vehicle_id`.
180+
181+
[role="screenshot"]
182+
image::maps/images/asset-tracking-tutorial/discover.png[]
183+
184+
[float]
185+
=== Part 2: Build an operational map
186+
It's hard to get an overview of Portland buses by looking at individual events. Let's create a map to show the bus routes and current location for each bus, along with the direction the buses are headed.
187+
188+
[float]
189+
==== Step 1: Create your map
190+
Create your map and set the theme for the default layer to dark mode.
191+
192+
. Open the main menu, and click *Maps*.
193+
. Click *Create map*.
194+
. In the *Layers* list, click *Road map*, and then click *Edit layer settings*.
195+
. Open the *Tile service* dropdown, and select *Road map - dark*.
196+
. Click *Save & close*.
197+
198+
[float]
199+
==== Step 2. Add a tracks layer
200+
201+
Add a layer to show the bus routes for the last 15 minutes.
202+
203+
. Click *Add layer*.
204+
. Click *Tracks*.
205+
. Select the *tri_met_tracks** index pattern.
206+
. Define the tracks:
207+
.. Set *Entity* to *vehicle_id*.
208+
.. Set *Sort* to *time*.
209+
. Click *Add layer*.
210+
. In Layer settings:
211+
.. Set *Name* to *Buses*.
212+
.. Set *Opacity* to 80%.
213+
. Scroll to *Layer Style*, and set *Border color* to pink.
214+
. Click *Save & close*.
215+
. In the *Layers* list, click *Buses*, and then click *Fit to data*.
216+
217+
At this point, you have a map with lines that represent the routes of the buses as they move around the city.
218+
219+
[role="screenshot"]
220+
image::maps/images/asset-tracking-tutorial/tracks_layer.png[]
221+
222+
[float]
223+
==== Step 3. Indicate the direction of the bus tracks
224+
225+
Add a layer that uses attributes in the data to set the style and orientation of the buses. You’ll see the direction buses are headed and what traffic is like.
226+
227+
. Click *Add layer*, and then select *Top Hits per entity*.
228+
. Select the *tri_met_tracks** index pattern.
229+
. To display the most recent location per bus:
230+
.. Set *Entity* to *vehicle_id*.
231+
.. Set *Documents per entity* to 1.
232+
.. Set *Sort field* to *time*.
233+
.. Set *Sort order* to *descending*.
234+
. Click *Add layer*.
235+
. Scroll to *Layer Style*.
236+
.. Set *Symbol type* to *icon*.
237+
.. Set *Icon* to *arrow-es*.
238+
.. Set the *Fill color*:
239+
... Select *By value* styling, and set the field to *in_congestion*.
240+
... Use a *Custom color palette*.
241+
... Set the *Other* color to black.
242+
... Add a green class for *false*, meaning the bus is not in traffic.
243+
... Add a red class for *true*, meaning the bus is in congestion.
244+
.. Set *Border width* to 0.
245+
.. Change *Symbol orientation* to use *By value* and the *bearing* field.
246+
+
247+
[role="screenshot"]
248+
image::maps/images/asset-tracking-tutorial/top_hits_layer_style.png[]
249+
. Click *Save & close*.
250+
. Open the <<set-time-filter, time filter>>, and set *Refresh every* to 10 seconds, and click *Start*.
251+
252+
Your map should automatically refresh every 10 seconds to show the lastest bus positions and tracks.
253+
254+
[role="screenshot"]
255+
image::maps/images/asset-tracking-tutorial/tracks_and_top_hits.png[]
256+
257+
[float]
258+
=== Part 3: Setup geo-fencing alerts
259+
Let's make TriMet Portland bus data actionable and alert when buses enter construction zones.
260+
261+
[float]
262+
==== Step 1. Add a construction zone
263+
264+
Add a layer for construction zones, which you will draw on the map. The construction zones will be used as your geofence boundary or threshold that serves as the basis for triggering alerts.
265+
266+
. Click *Add layer*.
267+
. Click *Create index*.
268+
. Set *Index name* to *construction_zones*.
269+
. Click *Create index*.
270+
. Draw 2 or 3 construction zones on your map:
271+
.. In the toolbar on left side of the map, select the bounding box icon image:maps/images/asset-tracking-tutorial/bounding_box_icon.png[bounding box icon].
272+
.. To draw a construction zone, click a start point on the map and drag.
273+
.. Click an endpoint to finish.
274+
. When you finish drawing the construction zones, click *Exit* under the layer name in the legend.
275+
. In *Layer settings*, set *Name* to *Construction zones*.
276+
. Scroll to *Layer Style*, and set *Fill color* to yellow.
277+
. Click *Save & close*.
278+
. *Save* the map.
279+
.. Give the map a title.
280+
.. Under *Add to dashboard*, select *None*.
281+
.. Click *Save and add to library*.
282+
283+
The map now represents an operational view of live bus traffic. You’ll see the direction that the buses are traveling, and whether they are near or have entered a construction zone.
284+
285+
Your map is now complete.
286+
287+
[role="screenshot"]
288+
image::maps/images/asset-tracking-tutorial/construction_zones.png[]
289+
290+
291+
[float]
292+
==== Step 2. Configure an alert
293+
294+
Create a new alert by defining a rule and a connector. The rule includes the conditions that will trigger the alert, and the connector defines what action takes place once the alert is triggered. In this case, each alert will log a message to the Kibana log.
295+
296+
. Open *Stack Management*, and then click *Rules and Connectors*.
297+
. Click *Create rule*.
298+
. Name the rule *Bus Alerts*.
299+
. Set *Check every* to *5 seconds*.
300+
. Notify *Only on status change*.
301+
+
302+
[role="screenshot"]
303+
image::maps/images/asset-tracking-tutorial/rule_configuration.png[]
304+
. Select the *Tracking containment* rule type.
305+
. Set *Select entity*:
306+
.. Set *INDEX* to *tri_met_tracks**.
307+
.. Set *BY* to *vehicle_id*.
308+
. Set *Select boundary* *INDEX* to *construction_zones*.
309+
+
310+
[role="screenshot"]
311+
image::maps/images/asset-tracking-tutorial/tracking_containment_configuration.png[]
312+
. Under *Actions*, select the *Server log* connector type.
313+
. Click *Create a connector*.
314+
. In the *Server log connector*:
315+
.. Set *Connector name* to *Bus alert connector*.
316+
.. Click *Save*.
317+
. Complete the *Actions* configuration.
318+
.. Set *Message* to :
319+
+
320+
[source,js]
321+
----------------------------------
322+
{
323+
"entityId": "{{context.entityId}}",
324+
"entityDateTime": "{{context.entityDateTime}}",
325+
"entityDocumentId": "{{context.entityDocumentId}}",
326+
"detectionDateTime": "{{context.detectionDateTime}}",
327+
"entityLocation": "{{context.entityLocation}}",
328+
"containingBoundaryId": "{{context.containingBoundaryId}}",
329+
"containingBoundaryName": "{{context.containingBoundaryName}}"
330+
}
331+
----------------------------------
332+
333+
. Click *Save*.
334+
335+
The *Bus Alert connector* is added to the *Rules and Connectors* page. For more information on common connectors, refer to the <<slack-action-type, Slack>> and <<email-action-type, Email>> connectors.
336+
337+
[role="screenshot"]
338+
image::maps/images/asset-tracking-tutorial/rules_and_connectors.png[]
339+
340+
Congratulations! You have completed the tutorial and have the recipe for tracking assets. You can now try replicating this same analysis with your own data.

‎docs/maps/images/asset-tracking-tutorial/bounding_box_icon.png‎

3.8 KB
Loading

‎docs/maps/images/asset-tracking-tutorial/construction_zones.png‎

2.44 MB
Loading

‎docs/maps/images/asset-tracking-tutorial/discover.png‎

214 KB
Loading

‎docs/maps/images/asset-tracking-tutorial/index_pattern.png‎

248 KB
Loading

‎docs/maps/images/asset-tracking-tutorial/logstash_output.png‎

487 KB
Loading

‎docs/maps/images/asset-tracking-tutorial/rule_configuration.png‎

40 KB
Loading

‎docs/maps/images/asset-tracking-tutorial/rules_and_connectors.png‎

146 KB
Loading

‎docs/maps/images/asset-tracking-tutorial/top_hits_layer_style.png‎

121 KB
Loading

0 commit comments

Comments
 (0)