We propose a cyclic, event based Beta testing process.  We will conduct a week of testing focused on a specific Test Theme.  At the conclusion of the week, we will review the test results with the test team.  The following week the development team will address the issues for re-delivery to the test team the following Monday. The process is described below.

The Proposed Event Testing Pattern

 

Testing Event Week

Monday AM

Software is available on FTP and SFTP sites

Kickoff Meeting with users (DUSA and Testers)

Issues reported via FogBugz

Thursday PM

Wrap up meeting review issues (DUSA and Testers)

Data Stores posted by Testers

Friday

DUSA review and reproduction of issues

 

Development Week

Monday AM

Development check-in meeting (DUSA and DNL)

Issues will have been moved to Jira

Thursday PM

Coding and development testing complete

Software available to DUSA

Friday

Development wrap up check in (DUSA and DNL)

DUSA validation of corrections

 

Proposed Test Themes

Theme 1 SA Testing:

Imports validated

Pre-processing validated

Exports validated

Models run: state updating, forecasts, ensembles

TimeSeries Display functions (zoom, text …) validated

Plot Thumbnails

Data Editing validated

Spatial Display functions (zoom, thumbnails …) validated

Docking/undocking/layout saving validated

Manual Forecast Display

Edit run options

Forecaster Help display

Archive export and import

 

Theme 2 SA Testing:

Validation of fixes.

Test ALL modifiers

Status icon updating based on model runs and modifier interactions

 

Theme 3 SA Testing:

Validation of fixes.

Special configurations (data qc)

Rating display

Correlation display

Statistics on the fly

Snow Updating

Search and Select Forecasts

Tabular ConfigFiles Display

Schematic Status Displays

F-12 Key functions

 

Theme 4 Live System:

Transition to Live System (1 week install, then testing)

Validation of fixes.

System performance (speed)

Data Synchronization OC/FSS

Scheduled jobs run as expected

System monitor reports as expected

Forecast Manager reports as expected

Admin Interface functions

Config Manager

PI-Service imports, exports, archive access

 

Theme 5 Live System:

Open and Continuous testing

Proposed Schedule

 

November 15 – Beta Test Planning call

 

December 1 – Development Testing begins

            DNL test team and developers validating updates

 

December 15 – Go/No-go meeting for January 29 Beta start with Beta Testers

            Presentation of the build status with rationale for Go/No-go recommendation

 

March 5 – Alpha Testing –

Live systems in Silver Spring installed

 

April 2 –Beta Testing Theme 1 – SA Testing

 

April 9 – Development Week 1

 

April 16 – Beta Testing Theme 2 – SA Testing

 

April 23 – Development Week 2

 

April 30 – Beta Testing Theme 3 – SA Testing

 

May 7 – Development Week 3

 

May 14 – Live System Installs

 

May 21 – Beta Testing Theme 4 – Live System Testing

 

May 28 – Development Week 4

 

June 4 – Open and continuous testing begins

A period of testing and fixing with software updates as needed

 

June 15 – End of open testing

                        Individual acceptance to close testing

 

June 20 – Full Release available