Information
Background
Previous session at UDS-M
Summary
"Laptop Testing" is a community-driven project started with Lucid Lynx Alpha 3. Our goal is get Ubuntu working great on as many different makes and models of laptops as possible, mainly testing point releases of the development cycle. This could be done both in all the Community and LoCos, sharing hardware info about Laptops to avoid duplicates and trying to attract as many new contributors as possible.
Project has been started by the Italian LoCo Testing team on launchpad. It's organized at moment with Wiki pages, a Mailing List and an IRC channel #ubuntu-it-test. In the Testing Team Wiki page this project is reported under Activities along with an English page.
In this session we would like to discuss how to improve this project.
Activity
Lucid Cycle Data
- Laptop tested.
- 49
- 40 people (7 of them are also active members of the Italian Testing Team).
- 17 of them useless (for various reasons: bugs not reported to Launchpad, tests never done or results never reported etc.)
- 20 ok.
- → 54% ok
Milestone |
People |
Test cases performed * |
10.04 A3 |
15 |
167 |
10.04 Beta 1 |
16 |
211 |
10.04 Beta 2 |
12 |
171 |
10.04 RC |
9 |
145 |
10.04 Final |
7 |
117 |
Maverick Cycle Data
- Laptop tested.
- 24
- 21 people (7 of them are also active members of the Italian Testing Team.)
- 7 useless (for various reasons: bugs not reported to Launchpad, tests never done or results never reported etc.)
- 15 ok.
- → 71,4% ok
Milestone |
People |
Test cases performed * |
10.10 A1 |
4 |
49 |
10.10 A2 |
4 |
88 |
10.10 A3 |
2 |
48 (Holidays) |
10.10 Beta |
7 |
167 |
10.10 RC |
5 |
81 |
10.10 Final |
5 |
54 |
* Counted as done even tests performed in "useless" reports (those marked as such because test failed but a bug have not been reported). Tests done several days after the final release have not been considered.
Thoughts
- Less people have partecipate in Maverick so less testcases have been performed.
- A large part of this decrease is due to a significant drop of wrong/useless reports.
- Number of testers from the Italian testing team (moderated) has been stable .
- List of common error found in reporting:
- Testcase marked as "Failed" without opening a bug or linking the bug on the page
Laptop report not added in the Reports page
- Laptop report linked wrong in the Reports page (typos)
Name/nick of tester not added with the @SIG@ macro (users can't be identified if the nick is different or has typos)
- Due to the open structure of wiki pages, some people add lots of unnecessary/unstructured or not properly formatted info
- A script to import Laptop testcases in checkbox is available but is useless without a plan to use/integrate checkbox database
How to improve
- Dedicated ISO Tracker. This would resolve a lot of issues and advantages are listed below:
- Users needs to be registered and providing the launchpad nickname are recognizable
- Reports are structured and data validated and can be aggregated or filtered
- Really easy to report a test result
- No more failed test with bugs not linked
- Tag system already implemented
- ISO Tracker / Checkbox integration
