{"id":17180,"date":"2021-02-09T09:00:06","date_gmt":"2021-02-09T17:00:06","guid":{"rendered":"https:\/\/engineering.fb.com\/?p=17180"},"modified":"2021-02-11T21:49:30","modified_gmt":"2021-02-12T05:49:30","slug":"minesweeper","status":"publish","type":"post","link":"https:\/\/engineering.fb.com\/2021\/02\/09\/developer-tools\/minesweeper\/","title":{"rendered":"Minesweeper automates root cause analysis as a first-line defense against bugs"},"content":{"rendered":"

Root cause analysis (RCA) is an important part of fixing any bug. After all, you can\u2019t solve a problem without getting to the heart of it. But RCA isn\u2019t always simple<\/a>, especially at a scale like Facebook\u2019s. When billions of people are using an app on a variety of platforms and devices, a single bug can create several different issues on its own and multiple bugs can happen simultaneously.<\/span><\/p>\n

There was a time when on-call engineers had to spend hours, or even days, manually combing through error reports, looking for patterns to help them debug. Today, they can use<\/span> Minesweeper<\/span> \u2014 a technique we\u2019ve developed<\/a> for automating RCA that identifies the causes of bugs based on their symptoms.<\/span><\/p>\n

Minesweeper-based RCA is completely automated and scalable, and it\u2019s grounded in formal statistical concepts. Our own evaluations of Minesweeper using real-world bug reports from Facebook\u2019s apps have proven that it can perform RCA for tens of thousands of reports in minutes and can identify the root cause of bugs with 85 percent accuracy.<\/span><\/p>\n

Since it was developed, Minesweeper has become Facebook\u2019s first line of defense against bugs and has helped us prevent potentially wide-scale disruptions from affecting people on our apps.<\/span><\/p>\n

How does Minesweeper work?<\/span><\/h2>\n

Anytime someone reports a bug through the Facebook app, the error reporting system<\/a> typically captures a chronological trace of actions (or \u201cevents\u201d) that person performed in the app prior to encountering the bug. The idea is to get a snapshot of what might have caused the error to happen, such as the example below:<\/span><\/p>\n

\"\"<\/p>\n

Minesweeper scans these traces to look for distinctive patterns that could point to the cause of a bug. Traces that contain the bug (the test group) are compared with traces that do not (the control group). Minesweeper finds patterns of events that are statistically distinctive to the test group as opposed to the control group. These patterns are likely to be correlated to the bug and can thereby point toward its root cause.<\/span><\/p>\n

Here\u2019s an example. Suppose (hypothetically) that 10 people are using the Facebook app. Five of these people report a problem, and there are eight possible events tracked in the app (<\/span>a<\/span><\/i>, <\/span>b<\/span><\/i>, …<\/span> through <\/span>h<\/span><\/i>). We have 10 traces in total on these events, five in the test group (<\/span>T<\/span><\/i>) from people who encountered the bug, and five in the control group (<\/span>C<\/span><\/i>) from the remaining people who did not.<\/span><\/p>\n

When Minesweeper is applied to these traces, it begins by extracting sequential patterns in <\/span>T<\/span><\/i> and <\/span>C<\/span><\/i>. A sequential pattern is simply a chronological sequence of events that happened, but not necessarily one after another (i.e., there might be other events in between that were not significant).<\/span><\/p>\n

We could visualize the data like this:<\/span><\/p>\n

Events: { <\/span>a, b, \u2026, h <\/span><\/i>}<\/span><\/p>\n\n\n\n\n\n\n\n\n
Test group<\/b> T<\/span><\/i><\/td>\n\u00a0 \u00a0 \u00a0 Control group<\/b> C<\/span><\/i><\/td>\n<\/tr>\n
t<\/span><\/i>1<\/sub><\/span>:\u00a0 <\/span>a <\/span><\/i>\u2192<\/span> b <\/span><\/i>\u2192<\/span> c <\/span><\/i>\u2192<\/span> d<\/span><\/i><\/td>\n\u00a0 \u00a0 \u00a0 \u00a0t<\/span><\/i>6<\/sub><\/span>:\u00a0 <\/span>a <\/span><\/i>\u2192<\/span> b <\/span><\/i>\u2192<\/span> d<\/span><\/i><\/td>\n<\/tr>\n
t<\/span><\/i>2<\/sub><\/span>:\u00a0 <\/span>a <\/span><\/i>\u2192<\/span> b <\/span><\/i>\u2192<\/span> c<\/span><\/i><\/td>\n\u00a0 \u00a0 \u00a0 \u00a0t<\/span><\/i>7<\/sub><\/span>:\u00a0 <\/span>a <\/span><\/i>\u2192<\/span> c <\/span><\/i>\u2192<\/span> d<\/span><\/i><\/td>\n<\/tr>\n
t<\/span><\/i>3<\/sub><\/span>:\u00a0 <\/span>b <\/span><\/i>\u2192<\/span> c<\/span><\/i><\/td>\n\u00a0 \u00a0 \u00a0 t<\/span><\/i>8<\/sub><\/span>:\u00a0 <\/span>a <\/span><\/i>\u2192<\/span> c<\/span><\/i><\/td>\n<\/tr>\n
t<\/span><\/i>4<\/sub><\/span>:\u00a0 <\/span>e <\/span><\/i>\u2192<\/span> f <\/span><\/i>\u2192<\/span> g <\/span><\/i>\u2192<\/span> h<\/span><\/i><\/td>\n\u00a0 \u00a0 \u00a0 t<\/span><\/i>9<\/sub><\/span>:\u00a0 <\/span>f <\/span><\/i>\u2192<\/span> g<\/span><\/i><\/td>\n<\/tr>\n
t<\/span><\/i>5<\/sub><\/span>:\u00a0 <\/span>e <\/span><\/i>\u2192<\/span> g<\/span><\/i><\/td>\n\u00a0 \u00a0 \u00a0 t<\/span><\/i>10<\/sub><\/span>:\u00a0 <\/span>e <\/span><\/i>\u2192<\/span> f <\/span><\/i>\u2192<\/span> h<\/span><\/i><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n

An example of a pattern that the system can extract here is <\/span>a <\/span><\/i>\u2192<\/span> c<\/span><\/i>. For each pattern, it computes the number of traces in which it appears in each group (<\/span>T<\/span><\/i> and <\/span>C<\/span><\/i>). In the example above, this pattern appears twice in <\/span>T<\/span><\/i> (<\/span>t<\/span><\/i>1<\/sub><\/span>and <\/span>t<\/span><\/i>2<\/sub><\/span>), and so its support in <\/span>T<\/span><\/i> is 2. In this case, its support in <\/span>C<\/span><\/i> is also 2 (<\/span>t<\/span><\/i>7 <\/sub><\/span>and <\/span>t<\/span><\/i>8<\/sub><\/span>). Note that the pattern space is combinatorial in nature, and therefore it is crucial to employ algorithms that can search this space efficiently without an exponential blowup.<\/span><\/p>\n

Once all patterns are extracted along with their supports in <\/span>T<\/span><\/i> and <\/span>C<\/span><\/i>, the system performs statistical isolation. For each pattern, <\/span>P<\/span><\/i>, it computes precision and recall using its support:<\/span><\/p>\n

\"\"<\/p>\n

Informally, precision describes how accurate <\/span>P<\/span><\/i> is in detecting whether a given trace is in the test group rather than the control group, and recall describes how much of the test group <\/span>P<\/span><\/i> can cover. For example, the pattern <\/span>b<\/span><\/i> has a precision of 0.75 because it occurs in four traces in total, three of which are in the test group (i.e., it is 75 percent specific to the test group). It also has a recall of 0.6 because it occurs in three out of five traces in the test group (i.e., it covers 60 percent of the test group).<\/span><\/p>\n

The harmonic mean of the two, 0.67, is the F1-score. The system computes the F1-score of all patterns in this manner and returns the list of all patterns ranked by F1-score, as shown in the table below.<\/span><\/p>\n

\"\"<\/p>\n

In this example, the result shows that the pattern <\/span>b <\/span><\/i>\u2192<\/span> c<\/span><\/i> is the highest ranked. In a real setting, an engineer debugging the reports can infer that events <\/span>b<\/span><\/i> and <\/span>c<\/span><\/i>, occurring in that order, are suspicious and deserve close inspection.<\/span><\/p>\n

Automating RCA at scale<\/span><\/h2>\n

To make Minesweeper work at Facebook\u2019s scale and complexity, we borrowed an idea from the data mining community \u2014<\/span> sequential pattern mining<\/span><\/a> \u2014 to help track the order that events appear in traces.<\/span><\/p>\n

At Facebook, it\u2019s not unusual to encounter tens of thousands of traces related to a bug. To handle this, we leveraged the<\/span> PrefixSpan<\/span><\/a> algorithm, which is well known for being highly efficient at sequential pattern mining. To rank patterns according to their distinctiveness to the test group, thereby being useful for RCA, we utilized the aforementioned statistical approach based on the precision and recall of patterns. Finally, to make Minesweeper practical from a usability standpoint, we addressed several human-centric challenges, such as avoiding \u201credundant\u201d patterns that are similar in explaining the root cause.<\/span><\/p>\n

The figure below gives a high-level overview of the automated RCA architecture.\u00a0<\/span><\/p>\n

\"\"<\/p>\n

The impact of Minesweeper<\/span><\/h2>\n

Minesweeper has played an important role in helping Facebook\u2019s engineers analyze and diagnose regressions<\/a> \u2014 sudden spikes in a group of crash or bug reports \u2014 providing insights in minutes that once could have taken days to gather. With the complexity of Facebook\u2019s apps and the product release cycle, it is common for multiple regressions to take place simultaneously, especially after a new version is released. But thanks to Minesweeper, engineers working on user-facing regressions can analyze them promptly, making it easier than ever for them to reduce and mitigate disruptions to Facebook\u2019s services. <\/span><\/p>\n

More technical information is available in our paper \u201c<\/span>Scalable statistical root cause analysis on app telemetry<\/span><\/a>.\u201d<\/span><\/p>\n","protected":false},"excerpt":{"rendered":"

Root cause analysis (RCA) is an important part of fixing any bug. After all, you can\u2019t solve a problem without getting to the heart of it. But RCA isn\u2019t always simple, especially at a scale like Facebook\u2019s. When billions of people are using an app on a variety of platforms and devices, a single bug […]<\/p>\n

Read More…<\/a><\/p>\n","protected":false},"author":51,"featured_media":17270,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[66,70,67,47,1],"tags":[],"coauthors":[218,1693,1694,1468],"class_list":["post-17180","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-developer-tools","category-networking-traffic","category-production-engineering","category-security","category-uncategorized"],"yoast_head":"\nMinesweeper automates root cause analysis as a first-line defense against bugs<\/title>\n<meta name=\"description\" content=\"Minesweeper is a new technique for automating root cause analysis (RCA) that identifies the causes of bugs based on their symptoms.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/engineering.fb.com\/2021\/02\/09\/developer-tools\/minesweeper\/\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Vijay Murali, Edward Yao, Umang Mathur, Satish Chandra\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"6 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/engineering.fb.com\\\/2021\\\/02\\\/09\\\/developer-tools\\\/minesweeper\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/engineering.fb.com\\\/2021\\\/02\\\/09\\\/developer-tools\\\/minesweeper\\\/\"},\"author\":{\"@id\":\"https:\\\/\\\/engineering.fb.com\\\/2021\\\/02\\\/09\\\/developer-tools\\\/minesweeper\\\/#author\",\"name\":\"\"},\"headline\":\"Minesweeper automates root cause analysis as a first-line defense against bugs\",\"datePublished\":\"2021-02-09T17:00:06+00:00\",\"dateModified\":\"2021-02-12T05:49:30+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/engineering.fb.com\\\/2021\\\/02\\\/09\\\/developer-tools\\\/minesweeper\\\/\"},\"wordCount\":1075,\"publisher\":{\"@id\":\"https:\\\/\\\/engineering.fb.com\\\/#organization\"},\"image\":{\"@id\":\"https:\\\/\\\/engineering.fb.com\\\/2021\\\/02\\\/09\\\/developer-tools\\\/minesweeper\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/engineering.fb.com\\\/wp-content\\\/uploads\\\/2021\\\/02\\\/Minesweeper-Hero.png\",\"articleSection\":[\"DevInfra\",\"Networking & Traffic\",\"Production Engineering\",\"Security & Privacy\"],\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/engineering.fb.com\\\/2021\\\/02\\\/09\\\/developer-tools\\\/minesweeper\\\/\",\"url\":\"https:\\\/\\\/engineering.fb.com\\\/2021\\\/02\\\/09\\\/developer-tools\\\/minesweeper\\\/\",\"name\":\"Minesweeper automates root cause analysis as a first-line defense against bugs\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/engineering.fb.com\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/engineering.fb.com\\\/2021\\\/02\\\/09\\\/developer-tools\\\/minesweeper\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/engineering.fb.com\\\/2021\\\/02\\\/09\\\/developer-tools\\\/minesweeper\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/engineering.fb.com\\\/wp-content\\\/uploads\\\/2021\\\/02\\\/Minesweeper-Hero.png\",\"datePublished\":\"2021-02-09T17:00:06+00:00\",\"dateModified\":\"2021-02-12T05:49:30+00:00\",\"description\":\"Minesweeper is a new technique for automating root cause analysis (RCA) that identifies the causes of bugs based on their symptoms.\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/engineering.fb.com\\\/2021\\\/02\\\/09\\\/developer-tools\\\/minesweeper\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/engineering.fb.com\\\/2021\\\/02\\\/09\\\/developer-tools\\\/minesweeper\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/engineering.fb.com\\\/2021\\\/02\\\/09\\\/developer-tools\\\/minesweeper\\\/#primaryimage\",\"url\":\"https:\\\/\\\/engineering.fb.com\\\/wp-content\\\/uploads\\\/2021\\\/02\\\/Minesweeper-Hero.png\",\"contentUrl\":\"https:\\\/\\\/engineering.fb.com\\\/wp-content\\\/uploads\\\/2021\\\/02\\\/Minesweeper-Hero.png\",\"width\":1920,\"height\":1080,\"caption\":\"Minesweeper\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/engineering.fb.com\\\/2021\\\/02\\\/09\\\/developer-tools\\\/minesweeper\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/engineering.fb.com\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Minesweeper automates root cause analysis as a first-line defense against bugs\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/engineering.fb.com\\\/#website\",\"url\":\"https:\\\/\\\/engineering.fb.com\\\/\",\"name\":\"Engineering at Meta\",\"description\":\"Engineering at Meta Blog\",\"publisher\":{\"@id\":\"https:\\\/\\\/engineering.fb.com\\\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/engineering.fb.com\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\\\/\\\/engineering.fb.com\\\/#organization\",\"name\":\"Meta\",\"url\":\"https:\\\/\\\/engineering.fb.com\\\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/engineering.fb.com\\\/#\\\/schema\\\/logo\\\/image\\\/\",\"url\":\"https:\\\/\\\/engineering.fb.com\\\/wp-content\\\/uploads\\\/2023\\\/08\\\/Meta_lockup_positive-primary_RGB.jpg\",\"contentUrl\":\"https:\\\/\\\/engineering.fb.com\\\/wp-content\\\/uploads\\\/2023\\\/08\\\/Meta_lockup_positive-primary_RGB.jpg\",\"width\":29011,\"height\":12501,\"caption\":\"Meta\"},\"image\":{\"@id\":\"https:\\\/\\\/engineering.fb.com\\\/#\\\/schema\\\/logo\\\/image\\\/\"},\"sameAs\":[\"https:\\\/\\\/www.facebook.com\\\/Engineering\\\/\",\"https:\\\/\\\/x.com\\\/fb_engineering\"]},[]]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Minesweeper automates root cause analysis as a first-line defense against bugs","description":"Minesweeper is a new technique for automating root cause analysis (RCA) that identifies the causes of bugs based on their symptoms.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/engineering.fb.com\/2021\/02\/09\/developer-tools\/minesweeper\/","twitter_misc":{"Written by":"Vijay Murali, Edward Yao, Umang Mathur, Satish Chandra","Est. reading time":"6 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/engineering.fb.com\/2021\/02\/09\/developer-tools\/minesweeper\/#article","isPartOf":{"@id":"https:\/\/engineering.fb.com\/2021\/02\/09\/developer-tools\/minesweeper\/"},"author":{"@id":"https:\/\/engineering.fb.com\/2021\/02\/09\/developer-tools\/minesweeper\/#author","name":""},"headline":"Minesweeper automates root cause analysis as a first-line defense against bugs","datePublished":"2021-02-09T17:00:06+00:00","dateModified":"2021-02-12T05:49:30+00:00","mainEntityOfPage":{"@id":"https:\/\/engineering.fb.com\/2021\/02\/09\/developer-tools\/minesweeper\/"},"wordCount":1075,"publisher":{"@id":"https:\/\/engineering.fb.com\/#organization"},"image":{"@id":"https:\/\/engineering.fb.com\/2021\/02\/09\/developer-tools\/minesweeper\/#primaryimage"},"thumbnailUrl":"https:\/\/engineering.fb.com\/wp-content\/uploads\/2021\/02\/Minesweeper-Hero.png","articleSection":["DevInfra","Networking & Traffic","Production Engineering","Security & Privacy"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/engineering.fb.com\/2021\/02\/09\/developer-tools\/minesweeper\/","url":"https:\/\/engineering.fb.com\/2021\/02\/09\/developer-tools\/minesweeper\/","name":"Minesweeper automates root cause analysis as a first-line defense against bugs","isPartOf":{"@id":"https:\/\/engineering.fb.com\/#website"},"primaryImageOfPage":{"@id":"https:\/\/engineering.fb.com\/2021\/02\/09\/developer-tools\/minesweeper\/#primaryimage"},"image":{"@id":"https:\/\/engineering.fb.com\/2021\/02\/09\/developer-tools\/minesweeper\/#primaryimage"},"thumbnailUrl":"https:\/\/engineering.fb.com\/wp-content\/uploads\/2021\/02\/Minesweeper-Hero.png","datePublished":"2021-02-09T17:00:06+00:00","dateModified":"2021-02-12T05:49:30+00:00","description":"Minesweeper is a new technique for automating root cause analysis (RCA) that identifies the causes of bugs based on their symptoms.","breadcrumb":{"@id":"https:\/\/engineering.fb.com\/2021\/02\/09\/developer-tools\/minesweeper\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/engineering.fb.com\/2021\/02\/09\/developer-tools\/minesweeper\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/engineering.fb.com\/2021\/02\/09\/developer-tools\/minesweeper\/#primaryimage","url":"https:\/\/engineering.fb.com\/wp-content\/uploads\/2021\/02\/Minesweeper-Hero.png","contentUrl":"https:\/\/engineering.fb.com\/wp-content\/uploads\/2021\/02\/Minesweeper-Hero.png","width":1920,"height":1080,"caption":"Minesweeper"},{"@type":"BreadcrumbList","@id":"https:\/\/engineering.fb.com\/2021\/02\/09\/developer-tools\/minesweeper\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/engineering.fb.com\/"},{"@type":"ListItem","position":2,"name":"Minesweeper automates root cause analysis as a first-line defense against bugs"}]},{"@type":"WebSite","@id":"https:\/\/engineering.fb.com\/#website","url":"https:\/\/engineering.fb.com\/","name":"Engineering at Meta","description":"Engineering at Meta Blog","publisher":{"@id":"https:\/\/engineering.fb.com\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/engineering.fb.com\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/engineering.fb.com\/#organization","name":"Meta","url":"https:\/\/engineering.fb.com\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/engineering.fb.com\/#\/schema\/logo\/image\/","url":"https:\/\/engineering.fb.com\/wp-content\/uploads\/2023\/08\/Meta_lockup_positive-primary_RGB.jpg","contentUrl":"https:\/\/engineering.fb.com\/wp-content\/uploads\/2023\/08\/Meta_lockup_positive-primary_RGB.jpg","width":29011,"height":12501,"caption":"Meta"},"image":{"@id":"https:\/\/engineering.fb.com\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/Engineering\/","https:\/\/x.com\/fb_engineering"]},[]]}},"jetpack_featured_media_url":"https:\/\/engineering.fb.com\/wp-content\/uploads\/2021\/02\/Minesweeper-Hero.png","jetpack_shortlink":"https:\/\/wp.me\/pa0Lhq-4t6","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/engineering.fb.com\/wp-json\/wp\/v2\/posts\/17180","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/engineering.fb.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/engineering.fb.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/engineering.fb.com\/wp-json\/wp\/v2\/users\/51"}],"replies":[{"embeddable":true,"href":"https:\/\/engineering.fb.com\/wp-json\/wp\/v2\/comments?post=17180"}],"version-history":[{"count":12,"href":"https:\/\/engineering.fb.com\/wp-json\/wp\/v2\/posts\/17180\/revisions"}],"predecessor-version":[{"id":17269,"href":"https:\/\/engineering.fb.com\/wp-json\/wp\/v2\/posts\/17180\/revisions\/17269"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/engineering.fb.com\/wp-json\/wp\/v2\/media\/17270"}],"wp:attachment":[{"href":"https:\/\/engineering.fb.com\/wp-json\/wp\/v2\/media?parent=17180"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/engineering.fb.com\/wp-json\/wp\/v2\/categories?post=17180"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/engineering.fb.com\/wp-json\/wp\/v2\/tags?post=17180"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/engineering.fb.com\/wp-json\/wp\/v2\/coauthors?post=17180"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}