Information

Background

Previous session at UDS-M
Previous session at UDS-N

Summary

"Laptop Testing" is a community-driven project started with Lucid Lynx Alpha 3. Our goal is get Ubuntu working great on as many different makes and models of laptops as possible, mainly testing point releases of the development cycle. This could be done both in all the Community and LoCos, sharing hardware info about Laptops to avoid duplicates and trying to attract as many new contributors as possible.

Project has been started by the Italian LoCo Testing team on launchpad. It's organized at moment with Wiki pages, a Mailing List and an IRC channel #ubuntu-it-test. In the Testing Team Wiki page this project is reported under Activities along with an English page.

In this session we would like to discuss how to improve this project.

Activity

Lucid Cycle Data

Laptop tested.
- 49
How many people filed a report:
- 40 people (7 of them are also active members of the Italian Testing Team).
How results have been reported:
- 17 of them useless (for various reasons: bugs not reported to Launchpad, tests never done or results never reported etc.)
- 20 ok.
  - → 54% ok

Milestone	People	Test cases performed *
10.04 A3	15	167
10.04 Beta 1	16	211
10.04 Beta 2	12	171
10.04 RC	9	145
10.04 Final	7	117

Maverick Cycle Data

Laptop tested.
- 24
How many people filed a report:
- 21 people (7 of them are also active members of the Italian Testing Team.)
How results have been reported:
- 7 useless (for various reasons: bugs not reported to Launchpad, tests never done or results never reported etc.)
- 15 ok.
  - → 71,4% ok

Milestone	People	Test cases performed *
10.10 A1	4	49
10.10 A2	4	88
10.10 A3	2	48 (Holidays)
10.10 Beta	7	167
10.10 RC	5	81
10.10 Final	5	54

* Counted as done even tests performed in "useless" reports (those marked as such because test failed but a bug have not been reported). Tests done several days after the final release have not been considered.

Natty Cycle Data

Mixed reports, filed using wiki until Alpha 3, with Laptop Tracker from Beta 1.

Laptop tested (Natty only, the tracker has also 7 for Maverick and 4 for Lucid).
- 8+32 = 40 Wiki method + Laptop tracker
How many people filed a report:
- 46 people (11 of them with the wiki method) out of 73 members of ubuntu-laptop-testing team in Launchpad.
How results have been reported:
- 4 reports with test results marked as failed but bugs not reported to Launchpad (all using wiki method) 44 ok

Milestone	People/Laptop	Test cases performed *
11.04 A1	4	120
11.04 A2	7	164
11.04 A3	5	86
11.04 Beta 1	19	338
11.04 Beta 2	17	366
11.04 Final	6	142

Graph

Thoughts

After the introduction of the Laptop Tracker:

Sensible increase in partecipants/tests performed.
a consistent drop of errors in filing a report, using wiki the most common were:
1. Testcase marked as "Failed" without opening a bug or linking the bug on the page.
2. Laptop report done not added in the Reports summary page
3. Various kind of typos, in some cases lack of important info,in others lots of unnecessary/unstructured or not properly formatted one etc...

Achievements:

Much more easy to report a results.
Reports are structured and data validated and can be aggregated or filtered.
Users can be easily contacted via tracker/launchpad and milestone and can be notificated by email when a new milestone is available.

How to improve

Extend the actual iso-testing tag system implementing a tag scheme specific for laptop (Audio-Video-I/O etc)
Try to automate the testing process using tools like kernel-qa scripts (using USB live disks).

CategoryHomepage

-  ⇤ ← Versione 8 del 06/05/2011 15.11.15 → 
  Dimensione: 5895
  Autore: SergioZanchetta
  Commento:
+   ← Versione 9 del 06/05/2011 15.22.40 → ⇥
  Dimensione: 5890
  Autore: SergioZanchetta
  Commento:
-Le cancellazioni sono segnalate in questo modo.
+Le aggiunte sono segnalate in questo modo.
 Linea 93:
-{{attachment:laptop-test-results.png}}
+{{attachment:laptop-results.png}}