Regression Tests

I'm planning to do continuous regression testing during the migration. For that I need to define a number of tests and possible results. The tests are supposed to be run every week. The overall result of that should be a table with tests (rows) and dates (columns). To keep things simple there's only a small number of possible test results.

Update: I've decided to run the tests afterwards. It's still meant as a measurement of the quality of the migration process. It doesn't really matter whether they are run during or after the process as we can check out any intermediate state using git/SVN. Maybe it's even better to run them afterwards as I won't be tempted to migrate only with the next upcoming test run in mind.

Possible Test Results

  1. Success (green)
  2. Operational success with UI issues (orange)
  3. Misbehaviour (red)
  4. Crash (black)

Test Directories and Files

TESTDIR = /home/jannis/studies/studienarbeit/test-directory
HOMEDIR = /home/jannis
APPDIR  = /usr/share/applications
BINDIR  = /usr/bin

$TESTDIR contains a number of large images and a video, a few desktop entries (one or two of them representing malware), a few executables, text files, OpenOffice.org documents and two or more subdirectories.

Tests Being Used

Writing automated tests using LDTP would be a project on its own, so all tests are going to require manual user interaction.

These tests cover as many things you can do with files/directories as possible. However, they may not test everything.

Browsing

  1. Open Thunar in $TESTDIR
  2. Open Thunar in $HOME
  3. Open Thunar in $APPDIR
  4. Open Thunar in $BINDIR
  5. In $TESTDIR press <Ctrl>l and trigger auto-completion with pressing /
  6. In $TESTDIR toggle the bookmarks sidepane visibility
  7. In $TESTDIR toggle the tree sidepane visibility
  8. In $TESTDIR right-click one of the subdirectories
  9. In $TESTDIR select one of the subdirectories and view its properties
  10. In $TESTDIR select one of the files and view its properties
  11. In $APPDIR repeatedly refresh the folder
  12. In $HOME toggle the visibility of hidden files several times in a row
  13. Create, click on and delete a bookmark to $TESTDIR
  14. Go to one of the special folders (Pictures, Documents etc.)
  15. In any directory switch between the different views (icon view, list view, details view)
  16. In any directory switch between toolbar/pathbar and browse to the filesystem root using them
  17. In any directory switch to pathbar and right click an item

History / Hierarchy (all in $TESTDIR)

  1. Select Go → Open Parent
  2. Select Go → Back
  3. Select Go → Forward
  4. Select Go → Back

Launching (all in $TESTDIR)

  1. Double-click one of the normal desktop entries
  2. Double-click one of the malware desktop entries
  3. Double-click one of the files
  4. Right-click one of the files and select 'Open with other application…' and select one

Editing (all in $TESTDIR)

  1. Rename one of the subdirectories
  2. Rename one of the files
  3. Copy a file from one of the subdirectories into itself
  4. Copy a file from one of the subdirectories into the other subdirectory
  5. Delete a file from one of the subdirectories
  6. Copy one of the subdirectories into $TESTDIR
  7. Copy one of the subdirectories into another subdirectory
  8. Change the permissions of a file (ideally in the subdir copy created in the tests before)
  9. Change the permissions of a subdirectory (ideally the subdir copy)
  10. Cut & paste a file from one of the subdirectories into itself
  11. Cut & paste a file from one of the subdirectories into the other subdirectory
  12. Cut & paste one of the subdirectories into $TESTDIR
  13. Cut & paste one of the subdirectories into another subdirectory
  14. Delete a subdirectory (such as the subdir copy)
  15. Copy the video file into a subdirectory

Trash

  1. In $TESTDIR trash two or more files including the video file
  2. In $TESTDIR trash a directory
  3. In the side pane click on the trash icon to open the trash
  4. In the trash recover the video file
  5. In the trash recover a directory
  6. In the trash delete a file
  7. Empty the trash

Mounting

  1. In the sidepane left-click (mount) one of the USB disk partitions
  2. In the sidepane right-click one of the USB disk partitions and unmount/eject it
 
preparation/regression-tests.txt · Last modified: 2009/04/09 23:05 by jannis
 
Except where otherwise noted, content on this wiki is licensed under the following license:CC Attribution-Noncommercial-Share Alike 3.0 Unported
Recent changes RSS feed Donate Powered by PHP Valid XHTML 1.0 Valid CSS Driven by DokuWiki