Best practices

null Event Based Beta Testing

Event Based Beta Testing

We propose a cyclic, event based Beta testing process.  We will conduct a week of testing focused on a specific Test Theme.  At the conclusion of the week, we will review the test results with the test team.  The following week the development team will address the issues for re-delivery to the test team the following Monday. The process is described below.

The Proposed Event Testing Pattern


Testing Event Week

Monday AM

Software is available on FTP and SFTP sites

Kickoff Meeting with users (DUSA and Testers)

Issues reported via FogBugz

Thursday PM

Wrap up meeting review issues (DUSA and Testers)

Data Stores posted by Testers


DUSA review and reproduction of issues


Development Week

Monday AM

Development check-in meeting (DUSA and DNL)

Issues will have been moved to Jira

Thursday PM

Coding and development testing complete

Software available to DUSA


Development wrap up check in (DUSA and DNL)

DUSA validation of corrections


Proposed Test Themes

Theme 1 SA Testing:

Imports validated

Pre-processing validated

Exports validated

Models run: state updating, forecasts, ensembles

TimeSeries Display functions (zoom, text …) validated

Plot Thumbnails

Data Editing validated

Spatial Display functions (zoom, thumbnails …) validated

Docking/undocking/layout saving validated

Manual Forecast Display

Edit run options

Forecaster Help display

Archive export and import


Theme 2 SA Testing:

Validation of fixes.

Test ALL modifiers

Status icon updating based on model runs and modifier interactions


Theme 3 SA Testing:

Validation of fixes.

Special configurations (data qc)

Rating display

Correlation display

Statistics on the fly

Snow Updating

Search and Select Forecasts

Tabular ConfigFiles Display

Schematic Status Displays

F-12 Key functions


Theme 4 Live System:

Transition to Live System (1 week install, then testing)

Validation of fixes.

System performance (speed)

Data Synchronization OC/FSS

Scheduled jobs run as expected

System monitor reports as expected

Forecast Manager reports as expected

Admin Interface functions

Config Manager

PI-Service imports, exports, archive access


Theme 5 Live System:

Open and Continuous testing

Proposed Schedule


November 15 – Beta Test Planning call


December 1 – Development Testing begins

            DNL test team and developers validating updates


December 15 – Go/No-go meeting for January 29 Beta start with Beta Testers

            Presentation of the build status with rationale for Go/No-go recommendation


March 5 – Alpha Testing –

Live systems in Silver Spring installed


April 2 –Beta Testing Theme 1 – SA Testing


April 9 – Development Week 1


April 16 – Beta Testing Theme 2 – SA Testing


April 23 – Development Week 2


April 30 – Beta Testing Theme 3 – SA Testing


May 7 – Development Week 3


May 14 – Live System Installs


May 21 – Beta Testing Theme 4 – Live System Testing


May 28 – Development Week 4


June 4 – Open and continuous testing begins

A period of testing and fixing with software updates as needed


June 15 – End of open testing

                        Individual acceptance to close testing


June 20 – Full Release available