Creating a Test Case

From FirebugWiki

(Difference between revisions)
Jump to: navigation, search
(Page describing how to create test cases)
m (Replaced links to templates)
(12 intermediate revisions not shown)
Line 1: Line 1:
 +
[[Image:testCase.png|thumb|400px|We need a [http://getfirebug.com/tests/manual/script/issue1619.html simple page], that we can open in the browser and use for manual testing to work on an issue]]
 +
 +
== General info ==
We can only work on issues, that we can reproduce. Therefore it is very important to have simple test cases for the bug you're seeing or the feature you are missing in Firebug.
We can only work on issues, that we can reproduce. Therefore it is very important to have simple test cases for the bug you're seeing or the feature you are missing in Firebug.
-
To help you with creating test cases we have created some HTML and JavaScript templates, which are listed below.
 
-
[http://fbug.googlecode.com/svn/tests/content/templates/net/issueXXXX.js JavaScript template for automated tests] - Template for automated tests used in [[FBTest]]
+
'''A test case MUST include precise step by step instructions how to reproduce it and a description of what you expect to see.'''
 +
 
 +
Also, if your steps to reproduce require a file, then please provide that file instead of posting lines of code. This makes it a lot easier for other people to reproduce your test case.
 +
 
 +
If you provide a test case file, you should also provide the following additional information:
 +
 
 +
* The issue number
 +
* The issue summary
 +
* Observed results (in case of bugs)
 +
* Contact information (so we can get back to you in case of questions)
== Manual tests ==
== Manual tests ==
-
In order to create a simple manual test case you can use one of the templates below:
+
=== How to create ===
-
# [http://fbug.googlecode.com/svn/tests/content/templates/default/issueXXXX.html Default HTML template] - Common template used for normal purposes
+
To create a manual test case you can either provide a publicly accessible URL or create a specific a test case file.
-
# [http://fbug.googlecode.com/svn/tests/content/templates/default/issueXXXXSeveralCases.html Enhanced HTML template] - Template including two cases and some example elements
+
 
 +
'''Please note again, that it is essential to give clear steps to reproduce your issue.'''
 +
 
 +
You can add additional material like screenshots, videos, links to discussions etc. or offer RDP access to your computer.
 +
Though they are '''not a replacement''' for a proper test case, since others should also be able to reproduce the problem by their own.
 +
 
 +
=== Test cases for enhancements ===
 +
Also for enhancements we want you to create a simple test case, so we are able to implement a feature/make a change as you imagine it.
 +
 
 +
So how can you provide a test case for something, that doesn't exist yet?
 +
Pretty simple: Like for bug descriptions you can create step by step instructions of how you imagine the changes.
 +
 
 +
==== Example ====
 +
You want a new option inside the [[Net Panel]], that allows you to copy the file name of a request.
 +
 
 +
<source lang="text">
 +
1. Open Firebug on this page
 +
2. Enable and switch to the Net panel
 +
3. Reload the page via F5
 +
4. Right-click the request for "Creating_a_Test_Case"
 +
  => The context menu for the request appears.
 +
5. Click the menu item "Copy file name" inside the context menu (not existing yet)
 +
 
 +
=> The file name "Creating_a_Test_Case" should be copied to the clipboard
 +
</source>
 +
 
 +
=== Templates ===
 +
To help you with creating test cases we provide some templates. You can use one of the templates below:
 +
* [https://getfirebug.com/tests/templates/manual/issueXXXX.html Default HTML template] - Common template used for normal purposes
 +
* [https://getfirebug.com/tests/templates/manual/issueXXXXSeveralCases.html Enhanced HTML template] - Template including two cases and some example elements
To adjust these templates please follow the steps below:
To adjust these templates please follow the steps below:
Line 20: Line 60:
# Remove all template comments
# Remove all template comments
-
There are already some examples, which you can use as reference:
+
=== Examples ===
-
* [https://getfirebug.com/tests/content/branches/1.8/css/537/issue537.html Issue 537]
+
There are already some examples, which can be used as reference:
-
* [https://getfirebug.com/tests/content/branches/1.8/css/1338/issue1338.html Issue 1338]
+
 
-
* [https://getfirebug.com/tests/content/branches/1.8/css/3652/issue3652.html Issue 3652]
+
* [https://getfirebug.com/tests/manual/console/4504/issue4504.html Issue 4504] ([[Console Panel]] related)
-
* [https://getfirebug.com/tests/content/branches/1.8/css/4180/issue4180.html Issue 4180]
+
* [https://getfirebug.com/tests/manual/html/issue1318.html Issue 1318] ([[HTML Panel]] related)
-
* [https://getfirebug.com/tests/content/branches/1.8/commandLine/4218/issue4218.html Issue 4218]
+
* [https://getfirebug.com/tests/manual/css/4543/issue4543.html Issue 4543] ([[CSS Panel]] related)
-
* [https://getfirebug.com/tests/content/branches/1.8/css/4543/issue4543.html Issue 4543]
+
* [https://getfirebug.com/tests/manual/script/4248/issue4248.html Issue 4248] ([[Script Panel]] related)
 +
* [https://getfirebug.com/tests/manual/net/issue3071.html Issue 3071] ([[Net Panel]] related)
 +
* [https://getfirebug.com/tests/manual/issues/5591/issue5591.php Issue 5591] ([[Cookies Panel]] related)
== Automated tests ==
== Automated tests ==
-
For creation of an automated test, which will be part of the [[FBTest]] suite you need at least two parts. An HTML file and a JavaScript file, which executes the test.
+
=== Requirements ===
-
To create the HTML page for the automated test case please follow the steps for the [[#Manual tests|manual tests]].
+
To run the FBTests you first need to install [[FBTest]].
 +
 
 +
In order to run the test suite on your machine you'll need to set up a local web server. For example you can use the [http://httpd.apache.org/ Apache HTTP Server]. To be able to access the FBTests through the web server, you have to create a mapping.
 +
For Apache you can achieve this by adding an alias to your <code>httpd.conf</code> file, which could look like this:
 +
 
 +
<source lang="apache">
 +
# Firebug FBTests
 +
Alias /fbtests "/path/to/your/fbtests/folder"
 +
<Directory "/path/to/your/fbtests/folder">
 +
    Options Indexes FollowSymLinks
 +
    AllowOverride all
 +
    Order allow,deny
 +
    Allow from all
 +
</Directory>
 +
</source>
 +
 
 +
Doing so you can access the test cases via <code><nowiki>http://127.0.0.1/fbtests</nowiki></code>.
 +
 
 +
=== How to create ===
 +
For the creation of an automated test, which will be part of the [[FBTest]] suite you need at least two parts. An HTML file and a JavaScript file, which executes the test.
 +
 
 +
To create the HTML page for the automated test case please follow the steps for the [[#Manual tests|manual tests]]. The only thing you do '''not''' need for FBTests is the section with the observed results.
 +
 
 +
The automated (JavaScript based) test case should include the exact same steps as when manually executing the test case, i. e. instead of calling an internal Firebug function directly you should call the UI functions, that will call the internal function. So for example instead of calling the <code>editNewAttribute()</code> method for a specific node inside the [[HTML Panel]] you should programmatically open the context menu at it and choose the option "New Attribute...". [[FBTest]] already provides several [[Firebug Automated Test API|APIs]], which encapsulates such logic, like in this case the function <code>FBTest.executeContextMenuCommand()</code>.
 +
 
 +
=== Steps for creating FBTests ===
 +
# Copy the two templates linked below to a subdirectory named after the issue number of the right group inside the <code>[https://github.com/firebug/firebug/tree/master/tests/content /tests/content/]</code> directory of your local copy of the repository.<br/>''Example:'' <code>/tests/content/net/5324/</code>.
 +
# Add a new line to <code>[https://github.com/firebug/firebug/blob/master/tests/content/firebug.html /tests/content/firebug.html]</code> and specify the group, the absolute URL path to the JavaScript file of your test case, a description of the form <code>"Issue xxxx: Issue summary"</code> and the absolute URL path to the HTML page.<br/>''Example:'' <code>{group: "commandLine", uri: "commandLine/5042/issue5042.js", desc: "Issue 5042: Command Line should not prevent tabbing out when empty", testPage: "commandLine/5042/issue5042.html"}</code>
 +
# Edit the HTML file following the steps described in the template.
 +
# Edit the JavaScript file following the steps described in the template. An FBTest normally opens the HTML file in a new browser tab via <code>FBTest.openNewTab()</code>. Furthermore it must contain a call to <code>FBTest.testDone()</code>, which indicates the end of the test.
 +
 
 +
=== Templates ===
 +
Two templates are available for automated tests:
 +
 
 +
'''Single tests:'''
 +
* [https://getfirebug.com/tests/templates/automated/issueXXXX.html Default HTML template] - Common template used for normal purposes
 +
* [https://getfirebug.com/tests/templates/automated/issueXXXX.js Template for single tests] - JavaScript template for single automated tests covering an issue
 +
 
 +
'''Multiple tests:'''
 +
* [https://getfirebug.com/tests/templates/automated/issueXXXXSeveralCases.html Enhanced HTML template] - Template including two cases and some example elements
 +
* [https://getfirebug.com/tests/templates/automated/issueXXXXSeveralCases.js Template test suites] - JavaScript template for several automated tests covering an issue
-
The JavaScript test can be created using the [http://fbug.googlecode.com/svn/tests/content/templates/net/issueXXXX.js JavaScript template for automated tests].
 
You will have to adjust this template using the [[Firebug Automated Test API|automated test API]].
You will have to adjust this template using the [[Firebug Automated Test API|automated test API]].
 +
=== Examples ===
There are also some [[Firebug Automated Test Examples|examples]] for how to create automated tests.
There are also some [[Firebug Automated Test Examples|examples]] for how to create automated tests.
 +
 +
Also see some live examples:
 +
 +
* [https://getfirebug.com/tests/head/css/537/issue537.html Issue 537]
 +
* [https://getfirebug.com/tests/head/css/1338/issue1338.html Issue 1338]
 +
* [https://getfirebug.com/tests/head/css/3652/issue3652.html Issue 3652]
 +
* [https://getfirebug.com/tests/head/css/4180/issue4180.html Issue 4180]
 +
* [https://getfirebug.com/tests/head/commandLine/4218/issue4218.html Issue 4218]
 +
* [https://getfirebug.com/tests/head/css/4543/issue4543.html Issue 4543]

Revision as of 07:51, 12 September 2012

We need a simple page, that we can open in the browser and use for manual testing to work on an issue

Contents

General info

We can only work on issues, that we can reproduce. Therefore it is very important to have simple test cases for the bug you're seeing or the feature you are missing in Firebug.

A test case MUST include precise step by step instructions how to reproduce it and a description of what you expect to see.

Also, if your steps to reproduce require a file, then please provide that file instead of posting lines of code. This makes it a lot easier for other people to reproduce your test case.

If you provide a test case file, you should also provide the following additional information:

  • The issue number
  • The issue summary
  • Observed results (in case of bugs)
  • Contact information (so we can get back to you in case of questions)

Manual tests

How to create

To create a manual test case you can either provide a publicly accessible URL or create a specific a test case file.

Please note again, that it is essential to give clear steps to reproduce your issue.

You can add additional material like screenshots, videos, links to discussions etc. or offer RDP access to your computer. Though they are not a replacement for a proper test case, since others should also be able to reproduce the problem by their own.

Test cases for enhancements

Also for enhancements we want you to create a simple test case, so we are able to implement a feature/make a change as you imagine it.

So how can you provide a test case for something, that doesn't exist yet? Pretty simple: Like for bug descriptions you can create step by step instructions of how you imagine the changes.

Example

You want a new option inside the Net Panel, that allows you to copy the file name of a request.

1. Open Firebug on this page
2. Enable and switch to the Net panel
3. Reload the page via F5
4. Right-click the request for "Creating_a_Test_Case"
   => The context menu for the request appears.
5. Click the menu item "Copy file name" inside the context menu (not existing yet)
 
=> The file name "Creating_a_Test_Case" should be copied to the clipboard

Templates

To help you with creating test cases we provide some templates. You can use one of the templates below:

To adjust these templates please follow the steps below:

  1. Replace "xxxx" by number of your issue
  2. Replace "Issue summary" by title of your issue
  3. Put inspectable elements, form fields etc. into the "content" section
  4. Add the exact steps to reproduce your issue under "Steps to reproduce"
  5. Describe the currently seen result under "Observed result"
  6. Describe what you would expect to see under "Expected result"
  7. Add your contact information
  8. Remove all template comments

Examples

There are already some examples, which can be used as reference:

Automated tests

Requirements

To run the FBTests you first need to install FBTest.

In order to run the test suite on your machine you'll need to set up a local web server. For example you can use the Apache HTTP Server. To be able to access the FBTests through the web server, you have to create a mapping. For Apache you can achieve this by adding an alias to your httpd.conf file, which could look like this:

# Firebug FBTests
Alias /fbtests "/path/to/your/fbtests/folder"
<Directory "/path/to/your/fbtests/folder">
    Options Indexes FollowSymLinks
    AllowOverride all
    Order allow,deny
    Allow from all
</Directory>

Doing so you can access the test cases via http://127.0.0.1/fbtests.

How to create

For the creation of an automated test, which will be part of the FBTest suite you need at least two parts. An HTML file and a JavaScript file, which executes the test.

To create the HTML page for the automated test case please follow the steps for the manual tests. The only thing you do not need for FBTests is the section with the observed results.

The automated (JavaScript based) test case should include the exact same steps as when manually executing the test case, i. e. instead of calling an internal Firebug function directly you should call the UI functions, that will call the internal function. So for example instead of calling the editNewAttribute() method for a specific node inside the HTML Panel you should programmatically open the context menu at it and choose the option "New Attribute...". FBTest already provides several APIs, which encapsulates such logic, like in this case the function FBTest.executeContextMenuCommand().

Steps for creating FBTests

  1. Copy the two templates linked below to a subdirectory named after the issue number of the right group inside the /tests/content/ directory of your local copy of the repository.
    Example: /tests/content/net/5324/.
  2. Add a new line to /tests/content/firebug.html and specify the group, the absolute URL path to the JavaScript file of your test case, a description of the form "Issue xxxx: Issue summary" and the absolute URL path to the HTML page.
    Example: {group: "commandLine", uri: "commandLine/5042/issue5042.js", desc: "Issue 5042: Command Line should not prevent tabbing out when empty", testPage: "commandLine/5042/issue5042.html"}
  3. Edit the HTML file following the steps described in the template.
  4. Edit the JavaScript file following the steps described in the template. An FBTest normally opens the HTML file in a new browser tab via FBTest.openNewTab(). Furthermore it must contain a call to FBTest.testDone(), which indicates the end of the test.

Templates

Two templates are available for automated tests:

Single tests:

Multiple tests:

You will have to adjust this template using the automated test API.

Examples

There are also some examples for how to create automated tests.

Also see some live examples:

Personal tools