Memorybug

From FirebugWiki

(Difference between revisions)
Jump to: navigation, search
(Memory Profiler APIs)
m (moved Firebug Memory Profiler to Memorybug: Page is describing Memorybug, not the Memory Profiler integrated into Firebug.)
 
(6 intermediate revisions not shown)
Line 3: Line 3:
[http://code.google.com/p/fbug/issues/detail?id=179  CSS changes]) is a support for analyzing  
[http://code.google.com/p/fbug/issues/detail?id=179  CSS changes]) is a support for analyzing  
memory that is consumed by a web page.
memory that is consumed by a web page.
-
 
=Goals=
=Goals=
-
A memory profiler UI should be smoothly integrated with Firebug UI (preferably linking and reusing existing UI elements) and providing such visual information that helps the user to answer/examine following questions.
+
A memory profiler UI should be smoothly integrated with Firebug UI (preferably linking and reusing existing UI elements) and providing such visual information, that helps the user to answer/examine following questions:
-
 
+
* What is the state of the memory consumed by the current page just now (a snapshot)?
* What is the state of the memory consumed by the current page just now (a snapshot)?
* How can I see a list of all JS objects created by the page?
* How can I see a list of all JS objects created by the page?
-
* What are the relations among created objects (prototype, constructor, parent, references, referents)?
+
* How can I see relations among created objects (prototype, constructor, parent, references, referents)?
-
* How much memory is consumed by this object?
+
* How much memory is consumed by this particular object?
-
* Where this object was crated in the source code (url, line number)?
+
* Where has this object been created in the source code (url, line number)?
-
* How to automatically execute memory profiler and analyze gathered data over page-life time?
+
* How to automatically execute memory profiler and compare results gathered at different moments of the page-life time?
-
* I know there are memory leaks, how can I find them?
+
* I know there are memory leaks. How can I find them?
-
* Are there any objects ready for garbage collecting at this moment?
+
* Are there any objects ready for garbage collection at this moment?
-
 
+
=Memory Profiler APIs=
=Memory Profiler APIs=
-
The currently available memory-profiler APIs allow to examine all JS objects in ''Firefox runtime'' (i.e get some meta-data for each object). It's possible to freeze it and create additional ''memory profiling runtime'', which is consequently used to access the original ''Firefox runtime''. There is also a way how to provide custom Javascript code that is executed within the ''memory profiling runtime'' and collect required info (see [http://www.toolness.com/wp/?p=604 more]).
+
The currently available memory-profiler APIs allow to examine all JS objects in ''Firefox runtime'' (i.e get some meta-data for each object). It's possible to freeze it and create additional ''memory profiling runtime'', which is consequently used to access the original ''Firefox runtime''. There is also a way how to provide custom Javascript code that is executed within the ''memory profiling runtime'' and collect required info (see [http://www.toolness.com/wp/?p=604 more]).
Every object in a ''runtime'' is referred by a unique ID and so, the first usual thing to do is getting a list of all existing JS objects (<code>getObjectTable</code>). Another function (<code>getObjectInfo</code> allows to get a JSON-able data structure for every object according to specified ID.
Every object in a ''runtime'' is referred by a unique ID and so, the first usual thing to do is getting a list of all existing JS objects (<code>getObjectTable</code>). Another function (<code>getObjectInfo</code> allows to get a JSON-able data structure for every object according to specified ID.
-
 
The meta-data structure can contain following fields (depends on the actual JS object).
The meta-data structure can contain following fields (depends on the actual JS object).
-
* '''id''' - Unique JS object ID.
+
* '''id''' - Unique JS object ID
* '''nativeClass''' - e.g. Function, Object, etc.
* '''nativeClass''' - e.g. Function, Object, etc.
-
* '''size''' - The total storage requirements for the object, including the (shared) memory used for property descriptors (<code>JS_GetObjectTotalSize()</code>).
+
* '''size''' - The total storage requirements for the object, including the (shared) memory used for property descriptors (<code>JS_GetObjectTotalSize()</code>)
* '''parent''' (id)
* '''parent''' (id)
* '''prototype''' (id)
* '''prototype''' (id)
-
* '''functionSize''' - Size of the function object plus size of the script (<code>JS_GetFunctionTotalSize()</code>).
+
* '''functionSize''' - Size of the function object plus size of the script (<code>JS_GetFunctionTotalSize()</code>)
-
* '''scriptSize''' - Memory used to store the bytecode corresponding to the script, including things like atoms i.e. constant strings in the code (<code>JS_GetScriptTotalSize()</code>).
+
* '''scriptSize''' - Memory used to store the bytecode corresponding to the script including things like atoms i.e. constant strings in the code (<code>JS_GetScriptTotalSize()</code>)
* '''name''' - Name of a function (not available for instances)
* '''name''' - Name of a function (not available for instances)
-
* '''filename''' - URL of the file where a function is defined (not available for instances).
+
* '''filename''' - URL of the file where a function is defined (not available for instances)
-
* '''lineStart''' - Line number where a function object definition begins (not available for instances).
+
* '''lineStart''' - Line number where a function object definition begins (not available for instances)
-
* '''lineEnd''' - Line number where a function object definition ends (not available for instances).
+
* '''lineEnd''' - Line number where a function object definition ends (not available for instances)
-
* '''children''' - List of referred objects.
+
* '''children''' - List of referred objects
-
 
+
The collected data structure represents an effective graph of meta-data about all existing JS objects linked together using IDs - a snapshot of the memory state.
The collected data structure represents an effective graph of meta-data about all existing JS objects linked together using IDs - a snapshot of the memory state.
-
 
== Missing APIs ==
== Missing APIs ==
-
 
+
* Even if there are fields (size, functionSize, scriptSize) that provide some '''size''' info, the actual number of bytes used by an object is missing. This is probably the most critical as Firebug users would probably like to see this information.
-
* Even if there are fields (size, functionSize, scripSize) that provide some '''size''' info, the actual number of bytes used by an object is missing. This is probably the most critical as Firebug users would probably like to see this information.
+
* Name field is not available for simple JS objects (e.g. <code>var o = {a:1, b:2}</code> doesn't produce a label 'o' to be seen in the name field). These can be only recognized by a list of existing properties, which makes the association with the actual JS object on the page harder.
* Name field is not available for simple JS objects (e.g. <code>var o = {a:1, b:2}</code> doesn't produce a label 'o' to be seen in the name field). These can be only recognized by a list of existing properties, which makes the association with the actual JS object on the page harder.
-
* It would be also useful if meta-data have a field ''constructor'', which would help to link it to the function that was used for instantiation (and consequently to the line of code where the object has been created).
+
* It would be also useful if meta-data have a field ''constructor'', which would help to link it to the function, that was used for instantiation (and consequently to the line of code where the object has been created).
 +
 
 +
One way how to get an association between a JS object existing on a page (e.g. <code>var o = {a:1, b:2}</code>) and meta-data structure returned by the profiler, is passing the object into the <code>profileMemory</code> method and return it's ID. So, if all objects on the page were collected (can be time expensive) and passed to the profiler, we would have an effective mechanism how to connect returned meta-data with objects displayed in the [[DOM Panel]]. It's not sure if this is a workaround of a feature, but there is apparently no other way how to get the ID for a JS object (I think JSD only allows to get a ''tag'' for script objects where the ''tag'' === ''id''). This needs examination yet.
=Memorybug=
=Memorybug=
-
There is a working Firebug extension called <b>Memorybug</b> that shows what kind of information is possible to get from the currently available memory-profiling APIs. So, far this extension is in real alpha phase.
+
There is a working Firebug extension called '''Memorybug''', that shows what kind of information is possible to get from the current memory-profiling APIs (based on work made by Atul Varma and Dion Almer). So far this extension is in real alpha phase.
-
 
+
Here is the recommended configuration to run Memorybug.
Here is the recommended configuration to run Memorybug.
Line 60: Line 54:
* [http://www.mozilla-europe.org/en/firefox/ Firefox 3.6]
* [http://www.mozilla-europe.org/en/firefox/ Firefox 3.6]
* [http://getfirebug.com/releases/firebug/1.6X/ Firebug 1.6]
* [http://getfirebug.com/releases/firebug/1.6X/ Firebug 1.6]
-
* [https://addons.mozilla.org/cs/firefox/addon/12025 Jetpack] (contains binary components with memory-profiler APIs)
+
* [https://addons.mozilla.org/en/firefox/addon/12025 Jetpack] (contains binary components with memory-profiler APIs; not needed for FF 4.0+)
* [https://getfirebug.com/releases/memorybug/ Memorybug]
* [https://getfirebug.com/releases/memorybug/ Memorybug]
-
As soon as you have all setup, load a [http://www.softwareishard.com/firebug/tests/memory-profiler/testMemory.html test page] and follow instructions on it.
+
As soon as you have all setup, load the [http://www.softwareishard.com/firebug/tests/memory-profiler/testMemory2.html test page] and follow instructions on it.
 +
== Memorybug 0.1a4 UI ==
 +
Version [https://getfirebug.com/releases/memorybug/memorybug-0.1.a4.xpi Memorybug 0.1a4] introduces new modified UI (next iteration) that improves presentation of all relations among JS objects in memory. See the chapter [[Link:Memorybug 0.1a2 UI]] for screenshots from the previous version.
-
== Current UI ==
+
Following screenshots are taken from [http://softwareishard.com/firebug/tests/memory-profiler/testMemory2.html this] test page.
-
Memorybug plug itself into the Firebug UI as a new '''Memory''' panel.
+
 
 +
The default state of the Memory Panel is the same. There is a ''Memory Snapshot'' button that starts memory profiling. After clicking on this button you should see a list of all global objects on the page (including inner iframes).
 +
 
 +
[[Image:Memory1-global-object-list.png]]
 +
 
 +
Every objects is presented as an entry in the table together with some related information. Size of the object is computed from number of fields, so size of linked objects is not included. There is also a constructor field (a link to the [[Script Panel]]) and most importantly a list of all objects that link back, it's the  ''Referenced By'' column.
 +
 
 +
The other screenshot shows what is displayed after an object is expanded.
 +
 
 +
[[Image:Memory1-object-expanded.png]]
 +
 
 +
All properties of given object (''Caroline'' in this case) are now presented similarly to how [[DOM Panel]] does it. There is the same info for all inner objects, but not for primitive values (the memory profiler gives info about objects only). In this case only ''children'' can be further expanded.
 +
 
 +
[[Image:Memory1-object-expanded-2.png]]
 +
 
 +
All references are also links and by clicking on it the UI navigates the user (still within the ''Memory'' Panel) to the appropriate row (there is also a tooltip showing preview of the target referent).
 +
 
 +
[[Image:Memory1-reference-links.png]]
 +
 
 +
== Memorybug 0.1a2 UI ==
 +
Memorybug plug itself into the Firebug UI as a new '''Memory''' Panel.
[[Image:memory-panel.png]]
[[Image:memory-panel.png]]
-
After clicking the ''Memory Snapshot'' button, Jetpack's <code>profileMemory</code> function is used to get all JS objects from the ''Firefox runtime''. The list of objects is filtered and so, only those that belongs to the current page (window) or an embedded iframe (window) are further analyzed.
+
After clicking the ''Memory Snapshot'' button Jetpack's <code>profileMemory</code> function is used to get meta-data about JS objects from the ''Firefox runtime''. The list of objects is filtered and so, only those, that belongs to the current page (window) or an embedded iframe (window) are analyzed.
[[Image:memory-functions.png]]
[[Image:memory-functions.png]]
-
This screen-shot shows several groups displayed as the result of profiling. The ''Functions'' group shows all JS Functions object found on the page and in an embedded iframes. The function name is clickable and navigates the the user into the '''Script''' panel.
+
This screen-shot shows several groups displayed as the result of profiling. The ''Functions'' group shows all JS Functions object found on the page and its iframes. The function name is clickable and navigates the user into the '''Script''' Panel.
-
 
+
One problem is that dynamically injected code breaks line numbers. See the next screenshot, showing what happens if the Firebug [[Console Panel]] is enabled.
-
One problem is that dynamically injected code breaks line numbers. See the next screenshot, that shows what happens if the Firebug '''Console''' panel is enabled.
+
[[Image:memory-injected-functions.png]]
[[Image:memory-injected-functions.png]]
-
All functions defined on the well known ''console'' object are also visible. However these are injected dynamically by Firebug, which breaks ''lineStart'' and ''lineEnd'' data.
+
All functions defined on the well known ''console'' object are also visible. However these are injected dynamically by Firebug, which breaks ''lineStart'' and ''lineEnd'' data (try it with installed Memorybug).
-
 
+
Another screenshot shows list of JS objects found on the page together with number of instances.
-
Another screenshot shows list of JS objects found on the page and number of their instances.
+
[[Image:memory-injected-functions.png]]
[[Image:memory-injected-functions.png]]
-
 
Since there are no real classes in Javascript (like e.g. in C++ or Java) there is currently no way how to identify an instance, except of listing its properties and defining something called a ''shape''.
Since there are no real classes in Javascript (like e.g. in C++ or Java) there is currently no way how to identify an instance, except of listing its properties and defining something called a ''shape''.
Line 94: Line 107:
[[Image:memory-instances.png]]
[[Image:memory-instances.png]]
-
Firebug users/developers can easily recognize the injected '''console''' object (2 instances, since there is an iframe) and also an object with ''name, age, desc'' properties (2 instances). The later corresponds to the following script on the page.
+
Firebug users/developers can easily recognize the injected '''console''' object (2 instances, since there is one in an iframe) and also an object with ''name, age, desc'' properties (2 instances). The later corresponds to the following script on the page.
-
<pre>
+
<source lang="javascript">
<script type="text/javascript">
<script type="text/javascript">
function Person(name, age, desc) {
function Person(name, age, desc) {
Line 107: Line 120:
var p3 = new Person("Peter", 27, "Bad guy");
var p3 = new Person("Peter", 27, "Bad guy");
</script>
</script>
-
</pre>
+
</source>
-
 
+
The last screenshot shows list of windows/iframes (URLs) with additional info for each window.
The last screenshot shows list of windows/iframes (URLs) with additional info for each window.
Line 115: Line 127:
This is another approach used by [3]. Showing list of objects that are crated within specific URL also grouped by a line at which they were created. In this case 1 function object was created at line: 11 in innerFrame.html. The source code at this line is <code>function _TestFromIframe()</code>
This is another approach used by [3]. Showing list of objects that are crated within specific URL also grouped by a line at which they were created. In this case 1 function object was created at line: 11 in innerFrame.html. The source code at this line is <code>function _TestFromIframe()</code>
-
 
== Future UI ==
== Future UI ==
-
There are surely better ways how to integrate the memory-profiling feature with the rest of Firebug UI (i.e. the other panels) and also how to present all gathered information to the user. These are only limited by the power of available APIs.  
+
There are surely better ways how to integrate the memory-profiling feature with the rest of Firebug UI (i.e. the other panels) and also better ways how to present all gathered information to the user. But limited by the power of available APIs.  
Following list summarizes some ideas:
Following list summarizes some ideas:
-
* There should be a new <code>console.memoryProfile()</code> method that allows to automated profiling. The result info can be logged directly into the '''Console''' panel similarly to what ''console.profileStart'' and ''console.profileEnd'' methods do.
+
* There should be a new <code>console.memoryProfile()</code> method that allows to automated profiling. The result info can be logged directly into the [[Console Panel]] similarly to what ''console.profileStart'' and ''console.profileEnd'' methods do.
-
* The '''Dom''' panel should displays number of raw bytes consumed for every displayed object (such number would be available as soon as the profiler has been launched).
+
* The '''Dom''' Panel should displays number of raw bytes consumed for every displayed object (such number would be available as soon as the profiler has been launched).
-
* The '''Memory''' panel should also support a ''Persist'' feature (like '''Net''' and '''Console''' panels) and archive memory-snapshots made at different times within the page life cycle (both manual or automatic profiling should be respected). In such a case the history should allow to see how the memory consumption increased/decreased.
+
* The '''Memory''' Panel should also support a ''Persist'' feature (like [[Net Panel|Net]] and [[Console Panel]]) and archive memory-snapshots made at different times within the page life cycle (both manual or automatic profiling should be respected). In such a case the history should allow to see how the memory consumption increased/decreased.
-
* Every JS object appearing in memory-profiling results should be linked with the '''Script''' panel showing it's location in the source and also linked with the '''DOM''' panel showing the location in DOM hierarchy.
+
* Every JS object appearing in memory-profiling results should be linked with the [[Script Panel]] showing it's location in the source and also linked with the [[DOM Panel]] showing the location in DOM hierarchy.
-
* Finally, similarly to the '''Net''' panel, it should be possible to export all the info into a file.
+
* Finally, similarly to the [[Net Panel]], it should be possible to export all the info into a file.
=Resources=
=Resources=

Latest revision as of 09:58, 18 October 2011

Contents

[edit] Motivation

One of the most requested features (I think the one just after saving CSS changes) is a support for analyzing memory that is consumed by a web page.

[edit] Goals

A memory profiler UI should be smoothly integrated with Firebug UI (preferably linking and reusing existing UI elements) and providing such visual information, that helps the user to answer/examine following questions:

  • What is the state of the memory consumed by the current page just now (a snapshot)?
  • How can I see a list of all JS objects created by the page?
  • How can I see relations among created objects (prototype, constructor, parent, references, referents)?
  • How much memory is consumed by this particular object?
  • Where has this object been created in the source code (url, line number)?
  • How to automatically execute memory profiler and compare results gathered at different moments of the page-life time?
  • I know there are memory leaks. How can I find them?
  • Are there any objects ready for garbage collection at this moment?

[edit] Memory Profiler APIs

The currently available memory-profiler APIs allow to examine all JS objects in Firefox runtime (i.e get some meta-data for each object). It's possible to freeze it and create additional memory profiling runtime, which is consequently used to access the original Firefox runtime. There is also a way how to provide custom Javascript code that is executed within the memory profiling runtime and collect required info (see more).

Every object in a runtime is referred by a unique ID and so, the first usual thing to do is getting a list of all existing JS objects (getObjectTable). Another function (getObjectInfo allows to get a JSON-able data structure for every object according to specified ID.

The meta-data structure can contain following fields (depends on the actual JS object).

  • id - Unique JS object ID
  • nativeClass - e.g. Function, Object, etc.
  • size - The total storage requirements for the object, including the (shared) memory used for property descriptors (JS_GetObjectTotalSize())
  • parent (id)
  • prototype (id)
  • functionSize - Size of the function object plus size of the script (JS_GetFunctionTotalSize())
  • scriptSize - Memory used to store the bytecode corresponding to the script including things like atoms i.e. constant strings in the code (JS_GetScriptTotalSize())
  • name - Name of a function (not available for instances)
  • filename - URL of the file where a function is defined (not available for instances)
  • lineStart - Line number where a function object definition begins (not available for instances)
  • lineEnd - Line number where a function object definition ends (not available for instances)
  • children - List of referred objects

The collected data structure represents an effective graph of meta-data about all existing JS objects linked together using IDs - a snapshot of the memory state.

[edit] Missing APIs

  • Even if there are fields (size, functionSize, scriptSize) that provide some size info, the actual number of bytes used by an object is missing. This is probably the most critical as Firebug users would probably like to see this information.
  • Name field is not available for simple JS objects (e.g. var o = {a:1, b:2} doesn't produce a label 'o' to be seen in the name field). These can be only recognized by a list of existing properties, which makes the association with the actual JS object on the page harder.
  • It would be also useful if meta-data have a field constructor, which would help to link it to the function, that was used for instantiation (and consequently to the line of code where the object has been created).

One way how to get an association between a JS object existing on a page (e.g. var o = {a:1, b:2}) and meta-data structure returned by the profiler, is passing the object into the profileMemory method and return it's ID. So, if all objects on the page were collected (can be time expensive) and passed to the profiler, we would have an effective mechanism how to connect returned meta-data with objects displayed in the DOM Panel. It's not sure if this is a workaround of a feature, but there is apparently no other way how to get the ID for a JS object (I think JSD only allows to get a tag for script objects where the tag === id). This needs examination yet.

[edit] Memorybug

There is a working Firebug extension called Memorybug, that shows what kind of information is possible to get from the current memory-profiling APIs (based on work made by Atul Varma and Dion Almer). So far this extension is in real alpha phase.

Here is the recommended configuration to run Memorybug.

As soon as you have all setup, load the test page and follow instructions on it.

[edit] Memorybug 0.1a4 UI

Version Memorybug 0.1a4 introduces new modified UI (next iteration) that improves presentation of all relations among JS objects in memory. See the chapter Link:Memorybug 0.1a2 UI for screenshots from the previous version.

Following screenshots are taken from this test page.

The default state of the Memory Panel is the same. There is a Memory Snapshot button that starts memory profiling. After clicking on this button you should see a list of all global objects on the page (including inner iframes).

Memory1-global-object-list.png

Every objects is presented as an entry in the table together with some related information. Size of the object is computed from number of fields, so size of linked objects is not included. There is also a constructor field (a link to the Script Panel) and most importantly a list of all objects that link back, it's the Referenced By column.

The other screenshot shows what is displayed after an object is expanded.

Memory1-object-expanded.png

All properties of given object (Caroline in this case) are now presented similarly to how DOM Panel does it. There is the same info for all inner objects, but not for primitive values (the memory profiler gives info about objects only). In this case only children can be further expanded.

Memory1-object-expanded-2.png

All references are also links and by clicking on it the UI navigates the user (still within the Memory Panel) to the appropriate row (there is also a tooltip showing preview of the target referent).

Memory1-reference-links.png

[edit] Memorybug 0.1a2 UI

Memorybug plug itself into the Firebug UI as a new Memory Panel.

Memory-panel.png

After clicking the Memory Snapshot button Jetpack's profileMemory function is used to get meta-data about JS objects from the Firefox runtime. The list of objects is filtered and so, only those, that belongs to the current page (window) or an embedded iframe (window) are analyzed.

Memory-functions.png

This screen-shot shows several groups displayed as the result of profiling. The Functions group shows all JS Functions object found on the page and its iframes. The function name is clickable and navigates the user into the Script Panel.

One problem is that dynamically injected code breaks line numbers. See the next screenshot, showing what happens if the Firebug Console Panel is enabled.

Memory-injected-functions.png

All functions defined on the well known console object are also visible. However these are injected dynamically by Firebug, which breaks lineStart and lineEnd data (try it with installed Memorybug).

Another screenshot shows list of JS objects found on the page together with number of instances.

Memory-injected-functions.png

Since there are no real classes in Javascript (like e.g. in C++ or Java) there is currently no way how to identify an instance, except of listing its properties and defining something called a shape.

Memory-instances.png

Firebug users/developers can easily recognize the injected console object (2 instances, since there is one in an iframe) and also an object with name, age, desc properties (2 instances). The later corresponds to the following script on the page.

<script type="text/javascript">
function Person(name, age, desc) {
    this.name = name;
    this.age = age;
    this.desc = desc;
}
 
var p2 = new Person("Jack", 35, "Good guy");
var p3 = new Person("Peter", 27, "Bad guy");
</script>

The last screenshot shows list of windows/iframes (URLs) with additional info for each window.

Memory-windows.png

This is another approach used by [3]. Showing list of objects that are crated within specific URL also grouped by a line at which they were created. In this case 1 function object was created at line: 11 in innerFrame.html. The source code at this line is function _TestFromIframe()

[edit] Future UI

There are surely better ways how to integrate the memory-profiling feature with the rest of Firebug UI (i.e. the other panels) and also better ways how to present all gathered information to the user. But limited by the power of available APIs.

Following list summarizes some ideas:

  • There should be a new console.memoryProfile() method that allows to automated profiling. The result info can be logged directly into the Console Panel similarly to what console.profileStart and console.profileEnd methods do.
  • The Dom Panel should displays number of raw bytes consumed for every displayed object (such number would be available as soon as the profiler has been launched).
  • The Memory Panel should also support a Persist feature (like Net and Console Panel) and archive memory-snapshots made at different times within the page life cycle (both manual or automatic profiling should be respected). In such a case the history should allow to see how the memory consumption increased/decreased.
  • Every JS object appearing in memory-profiling results should be linked with the Script Panel showing it's location in the source and also linked with the DOM Panel showing the location in DOM hierarchy.
  • Finally, similarly to the Net Panel, it should be possible to export all the info into a file.

[edit] Resources

  1. Web Application Memory Profiling, Take Two
  2. Memory Profiler 0.0.5 AMO
  3. Another Memory Profiler, Techno Barje
  4. Fun with SpiderMonkey
  5. Jetpack Binary Components
  6. Memorybug 0.1 Source
Personal tools