Skip to content

jerry: add heap profiler#240

Merged
yorkie merged 39 commits intomasterfrom
heap-profiler
Aug 13, 2018
Merged

jerry: add heap profiler#240
yorkie merged 39 commits intomasterfrom
heap-profiler

Conversation

@algebrait
Copy link
Copy Markdown
Contributor

@algebrait algebrait commented Jul 31, 2018

Checklist
  • npm test passes
  • tests and/or benchmarks are included
    • test/run_pass/test_profilerjs
  • documentation is changed or added
    • docs/api/Profiler.md
    • docs/devs/Optimization-Tips.md
  • object node
  • string node
  • number node
  • bytecode node
  • synthetic node
  • edge in mark-sweep gc's reference

@algebrait
Copy link
Copy Markdown
Contributor Author

algebrait commented Jul 31, 2018

Work in progress!

build

./tools/build.py --jerry-heap-profiler

usage

run below code

var profiler = require('profiler');
profiler.takeSnapshot('heapsnapshot.txt');

heapsnapshot.txt will be generated.

convert to v8 heapsnapshot

node deps/jerry/tools/j2v8snap.js heapsnapshot.txt  v8.heapsnapshot

see full doc in
docs/api/Profiler.md
docs/devs/Optimization-Tips.md

TODO

  • Most shallow size and retained size are same, which must be a bug.
  • Move script of cpu profiler from shadow to jerry. Wrap cpu profiler and heap profiler into one flag. These work will be done in other PR.

@yorkie yorkie added the jerry label Jul 31, 2018
jerry_assert_api_available ();

#ifndef JERRY_HEAP_PROFILER
JERRY_UNUSED (path);
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If JERRY_HEAP_PROFILER not defined, we should not do operate on any file handle.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#define JERRY_UNUSED(x) ((void) (x))
JERRY_UNUSED does nothing on it's param. It just suppresses compiler warning "define but not used".


iotjs_string_t filepath = JS_GET_ARG(0, string);
jerry_heap_profiler_take_snapshot ((const char*)iotjs_string_data(&filepath));
return jerry_create_boolean(true);
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Call iotjs_string_destroy(&filepath) is required before return.

return JS_CREATE_ERROR(COMMON, "filepath should be required.");

iotjs_string_t filepath = JS_GET_ARG(0, string);
jerry_heap_profiler_take_snapshot ((const char*)iotjs_string_data(&filepath));
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove the space after the function jerry_heap_profiler_take_snapshot, that might be only in the JerryScript source.

@@ -0,0 +1,122 @@
var fs = require('fs');
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Strict mode is required.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where should this file be? The CPU profiling tool are https://github.com/Rokid/ShadowNode/tree/master/tools/profiler.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would better place CPU and heap profiler into jerry source tree, because it is not related to shadow-node. The CPU profiler scripts will also be moved in other PR.

const EDGE_FIELD_OFFSET_NAME = 1;
const EDGE_FIELD_OFFSET_TO_NODE = 2;

const DUMMY_STRING = "<dummy>";
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use the single quote for strings always in JavaScript.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

3x!
The eslint has already give me many warning. I'll fix them all.

console.log(","+JSON.stringify(stringItem.chars));
}

console.log("]}");
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would think here a more readable generator based on JSON is needed :)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

3x! I will learn it.

@algebrait
Copy link
Copy Markdown
Contributor Author

My understanding of strings in heap is wrong.
JERRY_CONTEXT (string_list_first_p) is only list of parser literal strings.
It seems there is no simple iterator of all strings.
The heap dumper and converter should be re-implemented.

@algebrait algebrait changed the title [WIP] profiler: add heap profiler profiler: add heap profiler Aug 11, 2018
@algebrait algebrait changed the title profiler: add heap profiler jerry: add heap profiler Aug 11, 2018
Copy link
Copy Markdown
Member

@yorkie yorkie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@algebrait leaves some considerable comments, please check and fix them :)

ecma_extended_object_t *ext_object_p = (ecma_extended_object_t*) jmem_heap_alloc_block (size);
#ifdef JERRY_HEAP_PROFILER
ext_object_p->size = (uint32_t) size;
#endif /* JERRY_HEAP_PROFILER */
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The pair of macros are not alignment with the above JMEM_STATS :(

JERRY_UNUSED (edge_type);
JERRY_UNUSED (args);

if (!ecma_is_value_object(to))
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess this might have a coding style issue.

ecma_value_t value = property_pair_p->values[index].value;

if (ecma_is_value_object (value))
// if (ecma_is_value_object (value))
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If a statement is not used, how about removing from source or add a TODO/FIXME?

}
else
{
// TODO: when happen?
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's better that wrap the author name within TODO :)

// TODO(algebrait): when happen?

false,
type_error_thrower,
LIT_MAGIC_STRING_TYPE_ERROR_UL)
LIT_MAGIC_STRING_ERROR_UL)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should still be LIT_MAGIC_STRING_TYPE_ERROR_UL?

* `path` {String} profiling data file path
* Returns {null}

The `profiler.takeSnapshot()` methd take a Heap profiler file which path is `path`.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

JavaScript doesn't have override, we should merge these 2 ways into:

profiler.takeSnapshot([path])

$ ./tools/build.py --jerry-heap-profiler
```

Add Javascript code like docs/api/Profiler.md, Iot.js will generate a Jerry Heap Profiler file, say Profiler-123.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Javascript => JavaScript, and it's better that the docs/api/Profiler.md would be linkable :)


Users can generate v8 heap snapshot file.
```shell
node deps/jerry/tools/profiler/j2v8snap.js Profiler-123 v8.heapsnapshot
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add the prefix $ for shell block :p

```

Then open in Chrome browser following below site.
https://developer.chrome.com/devtools/docs/heap-profiling
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make this to be clickable?

jerry_value_t profiler = jerry_create_object();
iotjs_jval_set_method(profiler, "takeSnapshot", TakeSnapshot);
iotjs_jval_set_method(profiler, "startProfiling", StartProfiling);
iotjs_jval_set_method(profiler, "stopProfiling", StopProfiling);
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make these strings are magic ones?


/* jerry heap snapshot to V8 heap snapshot converter */

let fs = require('fs');
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be const.


class SnapshotConverter {
constructor(jerrySnapshotPath, outputPath) {
let inputData = fs.readFileSync(jerrySnapshotPath, 'utf8');
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto.

genNodeEdges(edges) {
for (let i = 0; i < edges.length; i++) {
let edge = edges[i];
this.edgesOutput.push (edge.edge_type);
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove the space after the function name :)

this.edgesOutput.push (edge.edge_type);
let name;
name = this.stringMap.get(edge.name);
this.edgesOutput.push (name);
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto.

this.edgesOutput.push (name);
// v8 node items is a flattern array,
// so to_node is to_node_id * node_size
this.edgesOutput.push (this.nodeMap.get(edge.to) * 6);
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto.

Copy link
Copy Markdown
Member

@yorkie yorkie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@algebrait LGTM mostly, but still some JS coding style nits 🐰

Copy link
Copy Markdown
Member

@yorkie yorkie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me

@yorkie yorkie merged commit bc244fe into master Aug 13, 2018
@yorkie yorkie deleted the heap-profiler branch August 13, 2018 17:40
@yorkie yorkie mentioned this pull request Oct 19, 2018
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants