Wiki Ubuntu-it

Indice
Partecipa
FAQ
Wiki Blog
------------------
Ubuntu-it.org
Forum
Chiedi
Chat
Cerca
Planet
  • Pagina non alterabile
  • Informazioni
  • Allegati
  • Differenze per "PaoloSammicheli/LaptopTesting-O"
Differenze tra le versioni 1 e 28 (in 27 versioni)
⇤ ← Versione 1 del 05/05/2011 13.54.48 →
Dimensione: 4960
Autore: PaoloSammicheli
Commento:
← Versione 28 del 11/05/2011 08.56.27 → ⇥
Dimensione: 6216
Autore: SergioZanchetta
Commento:
Le cancellazioni sono segnalate in questo modo. Le aggiunte sono segnalate in questo modo.
Linea 13: Linea 13:
"[[https://wiki.ubuntu.com/Testing|Laptop Testing]]" is a community-driven project started with Lucid Lynx Alpha 3. Our goal is get Ubuntu working great on as many different makes and models of laptops as possible, mainly testing point releases of the development cycle. This could be done both in all the Community and !LoCos, sharing hardware info about Laptops to avoid duplicates and trying to attract as many new contributors as possible. "[[https://wiki.ubuntu.com/Testing|Laptop Testing]]" is a community-driven project started with Lucid Lynx Alpha 3. Our goal is get Ubuntu working great on as many different makes and models of laptops as possible, mainly testing point releases of the development cycle to help devs in triaging bugs. This could be done both in all the Community and !LoCos, sharing hardware info about Laptops and trying to attract as many new contributors as possible.
Linea 15: Linea 15:
Project has been started by the [[https://launchpad.net/~ubuntu-it-testing|Italian LoCo Testing]] team on launchpad. It's organized at moment with [[http://wiki.ubuntu-it.org/GruppoTest|Wiki pages]], a [[http://liste.ubuntu-it.org/cgi-bin/mailman/listinfo/gruppo-test|Mailing List]] and an IRC channel #ubuntu-it-test. In the [[https://wiki.ubuntu.com/Testing|Testing Team Wiki page]] this project is reported under [[https://wiki.ubuntu.com/Testing/Activities|Activities]] along with an [[https://wiki.ubuntu.com/Testing/Laptop|English page]]. Project has a [[https://launchpad.net/~ubuntu-laptop-testing|Testing Team]] on launchpad. It uses the [[https://lists.ubuntu.com/mailman/listinfo/ubuntu-qa|QA Mailing List ]] and IRC channel #ubuntu-testing. In the [[https://wiki.ubuntu.com/Testing|Testing Team Wiki page]] this project is reported under [[https://wiki.ubuntu.com/Testing/Activities|Activities]] along with an [[https://wiki.ubuntu.com/Testing/Laptop|English page]].
Linea 17: Linea 17:
In this session we would like to discuss how to improve this project. In this session we would like to present goals reached and some suggestions to improve the project.
Linea 21: Linea 21:

 Laptop tested.
  * 49

 How many people filed a report:
  * 40 people (7 of them are also active members of the Italian Testing Team).

 How results have been reported:
 
    * 17 of them useless (for various reasons: bugs not reported to Launchpad, tests never done or results never reported etc.)
    * 20 ok.
                   → 54% ok

||<tablestyle="font-size: 0,9em; text-align: center; border: none; width: 50%; font-weight: bold; background-color: #f1f1ed; table-layout: fixed;"> Milestone || People || Test cases performed * ||		  * Tests [[https://wiki.ubuntu.com/Testing/Laptop/Lucid/Reports|here]]
||<tablestyle="font-size: 0,9em; text-align: center; border: none; width: 50%; font-weight: bold; background-color: #f1f1ed; table-layout: fixed;"> Milestone || People || Test cases performed ||
Linea 43: Linea 31:

 Laptop tested.
  * 24

 How many people filed a report:
  * 21 people (7 of them are also active members of the Italian Testing Team.)

 How results have been reported:
 
    * 7 useless (for various reasons: bugs not reported to Launchpad, tests never done or results never reported etc.)
           * 15 ok.
                      → 71,4% ok

||<tablestyle="font-size: 0,9em; text-align: center; border: none; width: 50%; font-weight: bold; background-color: #f1f1ed; table-layout: fixed;"> Milestone || People || Test cases performed * ||		  * Tests [[https://wiki.ubuntu.com/Testing/Laptop/Maverick/Reports|here]]
||<tablestyle="font-size: 0,9em; text-align: center; border: none; width: 50%; font-weight: bold; background-color: #f1f1ed; table-layout: fixed;"> Milestone || People || Test cases performed ||
Linea 65: Linea 41:
* Counted as done even tests performed in "useless" reports (those marked as such because test failed but a bug have not been reported). Tests done several days after the final release have not been considered. == Natty Cycle Data ==
 * Tests [[https://wiki.ubuntu.com/Testing/Laptop/OldReports|here]] (Wiki)
 * Tests [[http://laptop.qa.ubuntu.com/qatracker/laptop|here]] (Laptop Tracker)
||<tablestyle="font-size: 0,9em; text-align: center; border: none; width: 50%; font-weight: bold; background-color: #f1f1ed; table-layout: fixed;"> Milestone || People/Laptop || Test cases performed * ||
||< : #FFFFFF> 11.04 A1 ||< : #FFFFFF> 4 ||< : #FFFFFF> 120 ||
||< : #FFFFFF> 11.04 A2 ||< : #FFFFFF> 7 ||< : #FFFFFF> 164 ||
||< : #FFFFFF> 11.04 A3 ||< : #FFFFFF> 5 ||< : #FFFFFF> 86 ||
||< : #FFFFFF> 11.04 Beta 1 ||< : #FFFFFF> 19 ||< : #FFFFFF> 338 ||
||< : #FFFFFF> 11.04 Beta 2 ||< : #FFFFFF> 17 ||< : #FFFFFF> 366 ||
||< : #FFFFFF> 11.04 Final ||< : #FFFFFF> 11 ||< : #FFFFFF> 201 ||

* N.B. testcases here stand for "case-id" to compare them with wiki data (e.g. tracker has 18 testcases but most of them contain more than one case-id)


Mixed reports, filed using wiki until Alpha 3, with Laptop Tracker from Beta 1.

 Laptop tested (Natty only, the tracker has also 7 for Maverick and 4 for Lucid).
  * 8+33 = 41 Wiki method + Laptop tracker


 How many people filed a report (dev and stable releases):
  * 49 people out of 73 members of ubuntu-laptop-testing team in Launchpad.

 How results have been reported (natty):
 
         * 4 reports with test results marked as failed but bugs not reported to Launchpad (all using wiki method)
  * 45 ok


= Graph (dev cycles) =

{{attachment:laptop-tests-graph.png}}

##<<Immagine(/PaoloSammicheli/LaptopTesting-O/tests-graph.png,850,center)>>


= Laptop Tracker =
 * [[https://blueprints.launchpad.net/ubuntu/+spec/ubuntutheproject-qa-n-laptop-testing-tracker| Blueprint on Launchpad]]

 * Website: [[http://laptop.qa.ubuntu.com/|laptop.qa.ubuntu.com]]

== How it works ==

 * Profile setup

  {{attachment:laptop-info550.png}}

<<BR>>

 * Main page

  {{attachment:releases450.png}}

<<BR>>

 * [[http://laptop.qa.ubuntu.com/qatracker/test/5759|Testcases]]

  {{attachment:testcase450.png}}

<<BR>>

 * [[http://laptop.qa.ubuntu.com/qatracker/laptop/Natty/28712|Report example]]

  {{attachment:laptop-report.png}}

<<BR>>

 * [[http://laptop.qa.ubuntu.com/qatracker/laptop|Laptops summary]]

  {{attachment:laptop-summary550.png}}
Linea 68: Linea 115:
 * Less people have partecipate in Maverick so less testcases have been performed.
 * A large part of this decrease is due to a significant drop of wrong/useless reports.
 * Number of testers from the Italian testing team (moderated) has been stable .
 * List of common error found in reporting:
  1. Testcase marked as "Failed" without opening a bug or linking the bug on the page
  1. Laptop report not added in the [[https://wiki.ubuntu.com/Testing/Laptop/Reports|Reports page]]
  1. Laptop report linked wrong in the Reports page (typos)
  1. Name/nick of tester not added with the ''@SIG''@ macro (users can't be identified if the nick is different or has typos)
  1. Due to the open structure of wiki pages, some people add lots of unnecessary/unstructured or not properly formatted info
  1. A script to import Laptop testcases in checkbox is available but is useless without a plan to use/integrate checkbox database
 * Sensible increase in partecipants/tests performed.
 * A consistent drop of errors in filing a report, using wiki the most common were:
  1. Various kind of typos, in some cases lack of important info,in others lots of unnecessary/unstructured or not properly formatted one etc...
  1. Testcase marked as "Failed" without opening a bug or linking the bug on the page.
  1. Laptop report done not added in the Reports summary page.

== Achievements ==
  1. Much more easy and quick for a user to report a results.
  1. Reports are structured and data validated and can be aggregated or filtered.
  1. Testcases are grouped by category (Audio, Video, I/O etc.)
  1. Users can be easily contacted via tracker/launchpad and milestone and can be notificated by email when a new milestone is available.


= What's next =
Linea 79: Linea 131:
  1. Extend the actual iso-testing tag system implementing a tag scheme specific for laptop (Audio-Video-I/O etc)
  2. Add more testcases.
  3. Try to automate the testing process using tools like [[https://wiki.ubuntu.com/Kernel/kteam-tools |kernel-qa]] scripts (using USB live disks).
  4. Patches are welcome! :-)
Linea 80: Linea 136:
= How to improve =
 * Dedicated ISO Tracker. This would resolve a lot of issues and advantages are listed below:
  1. Users needs to be registered and providing the launchpad nickname are recognizable
  1. Reports are structured and data validated and can be aggregated or filtered
  1. Really easy to report a test result
  1. No more failed test with bugs not linked
  1. Tag system already implemented
   
 * ISO Tracker / Checkbox integration

Information

Background

  • Previous session at UDS-M

  • Previous session at UDS-N

Summary

"Laptop Testing" is a community-driven project started with Lucid Lynx Alpha 3. Our goal is get Ubuntu working great on as many different makes and models of laptops as possible, mainly testing point releases of the development cycle to help devs in triaging bugs. This could be done both in all the Community and LoCos, sharing hardware info about Laptops and trying to attract as many new contributors as possible.

Project has a Testing Team on launchpad. It uses the QA Mailing List and IRC channel #ubuntu-testing. In the Testing Team Wiki page this project is reported under Activities along with an English page.

In this session we would like to present goals reached and some suggestions to improve the project.

Activity

Lucid Cycle Data

Milestone

People

Test cases performed

10.04 A3

15

167

10.04 Beta 1

16

211

10.04 Beta 2

12

171

10.04 RC

9

145

10.04 Final

7

117

Maverick Cycle Data

Milestone

People

Test cases performed

10.10 A1

4

49

10.10 A2

4

88

10.10 A3

2

48 (Holidays)

10.10 Beta

7

167

10.10 RC

5

81

10.10 Final

5

54

Natty Cycle Data

  • Tests here (Wiki)

  • Tests here (Laptop Tracker)

Milestone

People/Laptop

Test cases performed *

11.04 A1

4

120

11.04 A2

7

164

11.04 A3

5

86

11.04 Beta 1

19

338

11.04 Beta 2

17

366

11.04 Final

11

201

* N.B. testcases here stand for "case-id" to compare them with wiki data (e.g. tracker has 18 testcases but most of them contain more than one case-id)

Mixed reports, filed using wiki until Alpha 3, with Laptop Tracker from Beta 1.

  • Laptop tested (Natty only, the tracker has also 7 for Maverick and 4 for Lucid).
    • 8+33 = 41 Wiki method + Laptop tracker
    How many people filed a report (dev and stable releases):
    • 49 people out of 73 members of ubuntu-laptop-testing team in Launchpad.
    How results have been reported (natty):
    • 4 reports with test results marked as failed but bugs not reported to Launchpad (all using wiki method)
    • 45 ok

Graph (dev cycles)

laptop-tests-graph.png

Laptop Tracker

How it works

  • Profile setup

    • laptop-info550.png


  • Main page

    • releases450.png




Thoughts

  • Sensible increase in partecipants/tests performed.
  • A consistent drop of errors in filing a report, using wiki the most common were:
    1. Various kind of typos, in some cases lack of important info,in others lots of unnecessary/unstructured or not properly formatted one etc...
    2. Testcase marked as "Failed" without opening a bug or linking the bug on the page.
    3. Laptop report done not added in the Reports summary page.

Achievements

  1. Much more easy and quick for a user to report a results.
  2. Reports are structured and data validated and can be aggregated or filtered.
  3. Testcases are grouped by category (Audio, Video, I/O etc.)
  4. Users can be easily contacted via tracker/launchpad and milestone and can be notificated by email when a new milestone is available.

What's next

  1. Extend the actual iso-testing tag system implementing a tag scheme specific for laptop (Audio-Video-I/O etc)
  2. Add more testcases.

  3. Try to automate the testing process using tools like kernel-qa scripts (using USB live disks).

  4. Patches are welcome! :-)

CategoryHomepage