{"id":1534,"date":"2011-04-10T19:17:28","date_gmt":"2011-04-11T00:17:28","guid":{"rendered":"http:\/\/www.phpied.com\/?p=1534"},"modified":"2011-04-10T19:18:41","modified_gmt":"2011-04-11T00:18:41","slug":"httpwatch-automation-javascript","status":"publish","type":"post","link":"https:\/\/www.phpied.com\/httpwatch-automation-javascript\/","title":{"rendered":"HTTPWatch automation with JavaScript"},"content":{"rendered":"<h3>Background<\/h3>\n<ul>\n<li><a href=\"http:\/\/www.httpwatch.com\">HTTPWatch<\/a> (<a href=\"http:\/\/apihelp.httpwatch.com\/#Automation%20Overview.html\">automation<\/a>)<\/li>\n<li>...<a href=\"https:\/\/www.phpied.com\/automating-httpwatch-with-php\/\">with PHP<\/a> (and <a href=\"https:\/\/www.phpied.com\/automating-httpwatch-with-php-2\/\">again<\/a> and <a href=\"https:\/\/www.phpied.com\/automating-httpwatch-with-php-3\/\">again<\/a>, and <a href=\"http:\/\/blog.httpwatch.com\/2011\/03\/11\/a-guide-to-automating-httpwatch-with-php\/\">response<\/a>)<\/li>\n<li><a href=\"https:\/\/www.phpied.com\/javascript-shell-scripting\/\">JavaScript shell scripting<\/a><\/li>\n<\/ul>\n<p>I gave this short presentation at the recent Yahoo FE summit's <a href=\"http:\/\/www.meetup.com\/BayJax\/events\/16564259\/\">open mic<\/a>, here are the slides and some notes.<\/p>\n<h3>Slides and screencast vid<\/h3>\n<div style=\"width:595px\" id=\"__ss_7424078\"> <b style=\"display:block;margin:12px 0 4px\"><a href=\"http:\/\/www.slideshare.net\/stoyan\/javascript-shell-scripting\" title=\"JavaScript shell scripting\">JavaScript shell scripting<\/a><\/b> <iframe loading=\"lazy\" src=\"http:\/\/www.slideshare.net\/slideshow\/embed_code\/7424078?rel=0\" width=\"595\" height=\"497\" frameborder=\"0\" marginwidth=\"0\" marginheight=\"0\" scrolling=\"no\"><\/iframe> <\/p>\n<div style=\"padding:5px 0 12px\"> View more <a href=\"http:\/\/www.slideshare.net\/\">presentations<\/a> from <a href=\"http:\/\/www.slideshare.net\/stoyan\">Stoyan Stefanov<\/a> <\/div>\n<\/p><\/div>\n<p>Screencast to see the thing in motion:<\/p>\n<p><iframe loading=\"lazy\" title=\"YouTube video player\" width=\"640\" height=\"510\" src=\"http:\/\/www.youtube.com\/embed\/38rpb5gUhjk?rel=0&amp;hd=1\" frameborder=\"0\" allowfullscreen><\/iframe><\/p>\n<h3>Notes<\/h3>\n<p>Here's the transcript of the slides as produced by slideshare.net. I've added some notes here and there to make it more readable when the presenter is missing.<\/p>\n<h4>JavaScript shell scripting - Presentation Transcript<\/h4>\n<ol>\n<li>\n<pre>JavaScript is everywhere #42:\r\n      C:&gt; WSH<\/pre>\n<\/li>\n<li>\n<pre>Stoyan<\/pre>\n<p>      I do programming.\n      <\/li>\n<li>\n<pre>Programming<\/pre>\n<p>      There are many options to choose from when you decide top practice the<br \/>\n      art and craft of programming.\n      <\/li>\n<li>\n<pre>JavaScript<\/pre>\n<p>      ... is a very good one. Simply because JavaScript...\n      <\/li>\n<li>\n<pre>... is everywhere<\/pre>\n<\/li>\n<li>\n<pre>On the server<\/pre>\n<p>      node.js, asp.net<\/li>\n<li>\n<pre>Mobile<\/pre>\n<p>      html5, phonegap, titanium<\/li>\n<li>\n<pre>Desktop<\/pre>\n<p>      XULRunner lets you create cross-OS GUI apps<\/li>\n<li>\n<pre>Browser extensions<\/pre>\n<p>      FF, Chrome, bookmarklets...<\/li>\n<li>\n<pre>Photoshop<\/pre>\n<p>      yep, that too<br \/>\n      Several Adobe products actually let you script common\/uncommon\/programmable tasks<\/li>\n<li>\n<pre>Form validation too!<\/pre>\n<p>      (this was supposed to be funny)<\/li>\n<li>\n<pre>Shell scripting<\/pre>\n<p>      let's talk about shell scripting with JavaScript<\/li>\n<li>\n<pre>In Windows<\/pre>\n<\/li>\n<li>\n<pre>WSH: Windows Scripting Host<\/pre>\n<p>      All reasonable Windows machines (at least as old as Win2000) have this Windows Scripting Host in there.<br \/>\n      You can write VBScript or JavaScript (OK, JScript) to ... well, script.<br \/>\n      How does it work?\n      <\/li>\n<li>\n<pre>C:&gt;edit hello.js<\/pre>\n<p>      You create a file.\n      <\/li>\n<li>\n<pre>var a = \"Hello\",\r\n    b = \" WSH!\",\r\n    c = a + b;\r\n        \r\nWScript.Echo(c);<\/pre>\n<p>    Put any old JavaScript in there and print out some results\n    <\/li>\n<li>\n<pre>C:&gt;cscript hello.js\r\nHello WSH!<\/pre>\n<p>      And this is how you run it.<\/p>\n<p>      Or this:<\/p>\n<pre>C:&gt;wscript hello.js<\/pre>\n<\/li>\n<li>\n<pre>Open apps<\/pre>\n<p>      In addition to regular sysadmin tasks (copy, write files, move) you can open and script applications too.\n      <\/li>\n<li>\n<pre>var ie = new ActiveXObject(\"InternetExplorer.Application\");\r\nie.Visible = true;\r\nie.navigate(yahoo.com);<\/pre>\n<p>      This is how you open IE and point it to a page.<\/p>\n<p>      Notice something familiar? <code>ActiveXObject<\/code> - the thing we used in IE before it got native <code>XMLHttpRequest<\/code>\n      <\/li>\n<li>\n<pre>Firefox?<\/pre>\n<p>      Can you also open FF?<\/p>\n<p>      Not really, as it doesn't have COM interface (whatever that is).<\/p>\n<p>      But there's an easy solution<\/li>\n<li>\n<pre>HTTPWatch<\/pre>\n<p>      Finally we come to the topic of the talk.\n      <\/li>\n<li>\n<pre>Speed<\/pre>\n<p>      Performance is critical for the success of any web app.<\/p>\n<p>      Really, it is.<\/p>\n<p>      When talking about improving speed there are two main steps:\n      <\/li>\n<li>\n<pre>1. Fix with YSlow<\/pre>\n<p>      Take a slow page, run YSlow, do what it says.<\/p>\n<p>      Voila - a fast(er) page.\n      <\/li>\n<li>\n<pre>2. No regressions<\/pre>\n<p>      The second step is to not allow regressions.<\/p>\n<p>      Whatever you fix in step 1 will be slow in the next few months.<\/p>\n<p>      Even less than months the bigger the team or the rate of changes.<\/p>\n<p>      So to prevent regressions, you need to constantly...\n      <\/li>\n<li>\n<pre>Monitor<\/pre>\n<\/li>\n<li>\n<pre>Set limits<\/pre>\n<p>      The simplest way to prevent regressions is to set some limits.<\/p>\n<p>      If you go over the limits, an email is sent, an alarm sounds, panic instills and you've got to fix whatever cause it was.<\/p>\n<\/li>\n<li>\n<pre>e.g.\r\nmax 2 scripts\r\nmax 2 styles\r\nmax 9 images\r\nmax 0 redirects<\/pre>\n<\/li>\n<li>\n<pre>Scripting HTTPWatch<\/pre>\n<p>      Watching for violations of the limits manually every day is not a job anyone would want.<\/p>\n<p>      So automating it will help a great way towards employee satisfaction \ud83d\ude42\n      <\/li>\n<li>\n<pre>var http = new ActiveXObject(\"HTTPWatch.Controller\"),\r\n    ie = http.IE.New(),\r\n    ff = http.Firefox.New();<\/pre>\n<p>    This is how you open IE and FF with HTTPWatch's help.<\/p>\n<p>    FF - yey!\n    <\/li>\n<li>\n<pre>\/\/ browser cache\r\nie.clearCache();\r\n\r\n\/\/ show HTTPWatch\r\nie.OpenWindow(false);<\/pre>\n<p>    Examples of stuff you can do with the HTTPWatch API.<\/p>\n<p>    You can for example hit the page with empty cache and then again with full cache.<\/p>\n<p>    Best of all - these are the real browsers with their sometimes kinky behaviors.<br \/>\n    Actually if you setup several machines for the monitoring (or somehow do multiple IEs)<br \/>\n    you can test with different versions of the browsers. Nothing synthetic!\n    <\/li>\n<li>\n<pre>ie.Record();\r\nie.GotoUrl(\"yahoo.com\");\r\nhttp.Wait(ie, -1);\r\nie.Stop();<\/pre>\n<p>      Start monitoring, go to a page, stop monitoring after the page \"settles\" meaning some time after onload.<\/p>\n<pre>ie.CloseBrowser();<\/pre>\n<\/li>\n<li>\n<pre>new HTTPWatch()\r\n      <a href=\"http:\/\/github.com\/stoyan\/etc\/\">http:\/\/github.com\/stoyan\/etc\/<\/a><\/pre>\n<p>      I did this JavaScript thingie to make everything a little easier.\n      <\/li>\n<li>\n<pre>var http = new HTTPWatch(ff);\r\nhttp.go(search.yahoo.com);\r\nhttp.done();<\/pre>\n<p>    Example usage.\n<\/li>\n<li>\n<pre><\/pre>\n<\/li>\n<li>\n<pre>var har = http.toHAR();\r\nhar = eval(( + har + ));\r\n\r\nprint(har.log.browser.name);\r\nprint(har.log.browser.version);\r\nprint(# requests: );\r\nprint(har.log.entries.length);<\/pre>\n<p>    Opening and navigating browsers is cool. But we need some data back.<\/p>\n<p>    HTTPWatch can export a HAR (HTTP Archive) file. I have this <code>toHAR()<\/code> method.<br \/>\n    It writes the file, than reads and returns it.<br \/>\n    You can than <code>eval()<\/code> it because it's just a JSON string.<br \/>\n    And you get the data back in convenient JS objects and arrays.\n<\/li>\n<li>\n<pre>Internet Explorer 6.0.29...\r\n# requests: 10\r\n\r\nFirefox 3.5.6\r\n# requests: 15<\/pre>\n<p>      Result of running the above.\n      <\/li>\n<li>\n<pre><\/pre>\n<\/li>\n<li>\n<pre>var comps = http.getComponentsByType();\r\n      \r\nfor (var i in comps) {\r\n  print(i);\r\n  print(comps[i].length);\r\n}<\/pre>\n<p>    Another method I thought would be useful is <code>getComponentsByType()<\/code>\n    <\/li>\n<li>\n<pre>redirect: 1\r\ntext\/html: 3\r\nimage\/gif: 4\r\nimage\/png: 3\r\ntext\/javascript: 1<\/pre>\n<p>    Results of the code above.\n<\/li>\n<li>\n<pre>But wait...<\/pre>\n<p>      There's more \ud83d\ude42\n      <\/li>\n<li>\n<pre>What about DOM?<\/pre>\n<p>      So far we only talked about HTTP traffic inspection - headers and such.<\/p>\n<p>      Good news is that you can also inspect the DOM (in IE only) for any potential red flags.<\/p>\n<p>      For example having the number of DOM elements sharply increase.\n      <\/li>\n<li>\n<pre><\/pre>\n<\/li>\n<li>\n<pre>var http = new HTTPWatch();\r\nhttp.go(search.yahoo.com);\r\n\r\nvar d = http.watch.container.document;\r\n\r\nprint(d.getElementsByTagName(*).length);\r\nprint(d.documentElement.innerHTML);<\/pre>\n<p>    That works!<\/p>\n<p>    All your DOM voodoo skillz are suddenly reusable.\n    <\/li>\n<li>\n<pre>require(statz.js);\r\n      \r\nvar doc = http.watch.container.document;\r\nvar html = http.har.log.entries[0].response.content.text;\r\n\r\nvar out = statz(document, html);\r\nprint(out.join(\"\\n\"));<\/pre>\n<p>    This is me repurposing <a href=\"https:\/\/www.phpied.com\/statsy-more-data-points-for-markup-quality\/\">two<\/a> <a href=\"https:\/\/www.phpied.com\/content-to-markup-ratio-bookmarklet\/\">old<\/a> bookmarklets that gather some interesting stats (one of them was even featured <a href=\"http:\/\/ajaxian.com\/archives\/calculate-your-content-to-markup-ratio\">on Ajaxian<\/a>, remember?). <\/p>\n<p>    It was pretty easy to repurpose the bookmarklets, because it's just JavaScript.<\/p>\n<p>    The stats thingie can inspect both raw HTML that went over the wire, as well as innerHTML that was the result of any additional DOM manipulations.\n    <\/li>\n<li>\n<pre>JS attributes (e.g. onclick): 1207 bytes\r\nCSS style attributes: 883\r\nInline JS: 5243\r\nInline CSS: 5015\r\nAll innerHTML: 17283\r\n# DOM elements: 134\r\n\r\nTotal size: 14124 bytes\r\nContent size: 401 bytes\r\nContent-to-markup ratio: 0.03<\/pre>\n<p>      Sample results.\n      <\/li>\n<li>\n<pre>To summarize...<\/pre>\n<\/li>\n<li>\n<pre>JavaScript\r\nWSH\r\nHTTPWatch\r\nMonitor\r\nDOM and HTTP\r\nIE and Firefox<\/pre>\n<\/li>\n<li>\n<pre>Thanks\r\n      \r\nphpied.com<\/pre>\n<\/li>\n<\/ol>\n","protected":false},"excerpt":{"rendered":"<p>Background HTTPWatch (automation) &#8230;with PHP (and again and again, and response) JavaScript shell scripting I gave this short presentation at the recent Yahoo FE summit&#8217;s open mic, here are the slides and some notes. Slides and screencast vid JavaScript shell scripting View more presentations from Stoyan Stefanov Screencast to see the thing in motion: Notes [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[5,43,141,22],"tags":[],"_links":{"self":[{"href":"https:\/\/www.phpied.com\/wp-json\/wp\/v2\/posts\/1534"}],"collection":[{"href":"https:\/\/www.phpied.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.phpied.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.phpied.com\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.phpied.com\/wp-json\/wp\/v2\/comments?post=1534"}],"version-history":[{"count":0,"href":"https:\/\/www.phpied.com\/wp-json\/wp\/v2\/posts\/1534\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.phpied.com\/wp-json\/wp\/v2\/media?parent=1534"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.phpied.com\/wp-json\/wp\/v2\/categories?post=1534"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.phpied.com\/wp-json\/wp\/v2\/tags?post=1534"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}