Creating a Test Case

From FirebugWiki

(Difference between revisions)
Jump to: navigation, search
(Added 'Requirements' section)
m (Replaced links to templates)
(One intermediate revision not shown)
Line 47: Line 47:
=== Templates ===
=== Templates ===
To help you with creating test cases we provide some templates. You can use one of the templates below:
To help you with creating test cases we provide some templates. You can use one of the templates below:
-
* [https://github.com/firebug/firebug/blob/master/tests/templates/default/issueXXXX.html Default HTML template] - Common template used for normal purposes
+
* [https://getfirebug.com/tests/templates/manual/issueXXXX.html Default HTML template] - Common template used for normal purposes
-
* [https://github.com/firebug/firebug/blob/master/tests/templates/default/issueXXXXSeveralCases.html Enhanced HTML template] - Template including two cases and some example elements
+
* [https://getfirebug.com/tests/templates/manual/issueXXXXSeveralCases.html Enhanced HTML template] - Template including two cases and some example elements
To adjust these templates please follow the steps below:
To adjust these templates please follow the steps below:
Line 75: Line 75:
To run the FBTests you first need to install [[FBTest]].
To run the FBTests you first need to install [[FBTest]].
-
In order to run the test suite locally you'll need to set up a web server. For example you can use the [http://httpd.apache.org/ Apache HTTP Server]. To be able to access the FBTests through the web server, you have to create a mapping.
+
In order to run the test suite on your machine you'll need to set up a local web server. For example you can use the [http://httpd.apache.org/ Apache HTTP Server]. To be able to access the FBTests through the web server, you have to create a mapping.
For Apache you can achieve this by adding an alias to your <code>httpd.conf</code> file, which could look like this:
For Apache you can achieve this by adding an alias to your <code>httpd.conf</code> file, which could look like this:
Line 108: Line 108:
'''Single tests:'''
'''Single tests:'''
-
* [https://github.com/firebug/firebug/blob/master/tests/templates/default/issueXXXX.html Default HTML template] - Common template used for normal purposes
+
* [https://getfirebug.com/tests/templates/automated/issueXXXX.html Default HTML template] - Common template used for normal purposes
-
* [https://github.com/firebug/firebug/blob/master/tests/templates/default/issueXXXX.js Template for single tests] - JavaScript template for single automated tests covering an issue
+
* [https://getfirebug.com/tests/templates/automated/issueXXXX.js Template for single tests] - JavaScript template for single automated tests covering an issue
'''Multiple tests:'''
'''Multiple tests:'''
-
* [https://github.com/firebug/firebug/blob/master/tests/templates/default/issueXXXXSeveralCases.html Enhanced HTML template] - Template including two cases and some example elements
+
* [https://getfirebug.com/tests/templates/automated/issueXXXXSeveralCases.html Enhanced HTML template] - Template including two cases and some example elements
-
* [https://github.com/firebug/firebug/blob/master/tests/templates/default/issueXXXXSeveralCases.js Template test suites] - JavaScript template for several automated tests covering an issue
+
* [https://getfirebug.com/tests/templates/automated/issueXXXXSeveralCases.js Template test suites] - JavaScript template for several automated tests covering an issue
You will have to adjust this template using the [[Firebug Automated Test API|automated test API]].
You will have to adjust this template using the [[Firebug Automated Test API|automated test API]].

Revision as of 07:51, 12 September 2012

We need a simple page, that we can open in the browser and use for manual testing to work on an issue

Contents

General info

We can only work on issues, that we can reproduce. Therefore it is very important to have simple test cases for the bug you're seeing or the feature you are missing in Firebug.

A test case MUST include precise step by step instructions how to reproduce it and a description of what you expect to see.

Also, if your steps to reproduce require a file, then please provide that file instead of posting lines of code. This makes it a lot easier for other people to reproduce your test case.

If you provide a test case file, you should also provide the following additional information:

  • The issue number
  • The issue summary
  • Observed results (in case of bugs)
  • Contact information (so we can get back to you in case of questions)

Manual tests

How to create

To create a manual test case you can either provide a publicly accessible URL or create a specific a test case file.

Please note again, that it is essential to give clear steps to reproduce your issue.

You can add additional material like screenshots, videos, links to discussions etc. or offer RDP access to your computer. Though they are not a replacement for a proper test case, since others should also be able to reproduce the problem by their own.

Test cases for enhancements

Also for enhancements we want you to create a simple test case, so we are able to implement a feature/make a change as you imagine it.

So how can you provide a test case for something, that doesn't exist yet? Pretty simple: Like for bug descriptions you can create step by step instructions of how you imagine the changes.

Example

You want a new option inside the Net Panel, that allows you to copy the file name of a request.

1. Open Firebug on this page
2. Enable and switch to the Net panel
3. Reload the page via F5
4. Right-click the request for "Creating_a_Test_Case"
   => The context menu for the request appears.
5. Click the menu item "Copy file name" inside the context menu (not existing yet)
 
=> The file name "Creating_a_Test_Case" should be copied to the clipboard

Templates

To help you with creating test cases we provide some templates. You can use one of the templates below:

To adjust these templates please follow the steps below:

  1. Replace "xxxx" by number of your issue
  2. Replace "Issue summary" by title of your issue
  3. Put inspectable elements, form fields etc. into the "content" section
  4. Add the exact steps to reproduce your issue under "Steps to reproduce"
  5. Describe the currently seen result under "Observed result"
  6. Describe what you would expect to see under "Expected result"
  7. Add your contact information
  8. Remove all template comments

Examples

There are already some examples, which can be used as reference:

Automated tests

Requirements

To run the FBTests you first need to install FBTest.

In order to run the test suite on your machine you'll need to set up a local web server. For example you can use the Apache HTTP Server. To be able to access the FBTests through the web server, you have to create a mapping. For Apache you can achieve this by adding an alias to your httpd.conf file, which could look like this:

# Firebug FBTests
Alias /fbtests "/path/to/your/fbtests/folder"
<Directory "/path/to/your/fbtests/folder">
    Options Indexes FollowSymLinks
    AllowOverride all
    Order allow,deny
    Allow from all
</Directory>

Doing so you can access the test cases via http://127.0.0.1/fbtests.

How to create

For the creation of an automated test, which will be part of the FBTest suite you need at least two parts. An HTML file and a JavaScript file, which executes the test.

To create the HTML page for the automated test case please follow the steps for the manual tests. The only thing you do not need for FBTests is the section with the observed results.

The automated (JavaScript based) test case should include the exact same steps as when manually executing the test case, i. e. instead of calling an internal Firebug function directly you should call the UI functions, that will call the internal function. So for example instead of calling the editNewAttribute() method for a specific node inside the HTML Panel you should programmatically open the context menu at it and choose the option "New Attribute...". FBTest already provides several APIs, which encapsulates such logic, like in this case the function FBTest.executeContextMenuCommand().

Steps for creating FBTests

  1. Copy the two templates linked below to a subdirectory named after the issue number of the right group inside the /tests/content/ directory of your local copy of the repository.
    Example: /tests/content/net/5324/.
  2. Add a new line to /tests/content/firebug.html and specify the group, the absolute URL path to the JavaScript file of your test case, a description of the form "Issue xxxx: Issue summary" and the absolute URL path to the HTML page.
    Example: {group: "commandLine", uri: "commandLine/5042/issue5042.js", desc: "Issue 5042: Command Line should not prevent tabbing out when empty", testPage: "commandLine/5042/issue5042.html"}
  3. Edit the HTML file following the steps described in the template.
  4. Edit the JavaScript file following the steps described in the template. An FBTest normally opens the HTML file in a new browser tab via FBTest.openNewTab(). Furthermore it must contain a call to FBTest.testDone(), which indicates the end of the test.

Templates

Two templates are available for automated tests:

Single tests:

Multiple tests:

You will have to adjust this template using the automated test API.

Examples

There are also some examples for how to create automated tests.

Also see some live examples:

Personal tools