Event Based Beta Testing - Delft-FEWS
Event Based Beta Testing
11 June 2019
We propose a cyclic, event based Beta testing process. We will conduct a week of testing focused on a specific Test Theme. At the conclusion of the week, we will review the test results with the test team. The following week the development team will address the issues for re-delivery to the test team the following Monday. The process is described below.
The Proposed Event Testing Pattern
Testing Event Week
Monday AM
Software is available on FTP and SFTP sites
Kickoff Meeting with users (DUSA and Testers)
Issues reported via FogBugz
Thursday PM
Wrap up meeting review issues (DUSA and Testers)
Data Stores posted by Testers
Friday
DUSA review and reproduction of issues
Development Week
Monday AM
Development check-in meeting (DUSA and DNL)
Issues will have been moved to Jira
Thursday PM
Coding and development testing complete
Software available to DUSA
Friday
Development wrap up check in (DUSA and DNL)
DUSA validation of corrections
Proposed Test Themes
Theme 1 SA Testing:
Imports validated
Pre-processing validated
Exports validated
Models run: state updating, forecasts, ensembles
TimeSeries Display functions (zoom, text …) validated
Plot Thumbnails
Data Editing validated
Spatial Display functions (zoom, thumbnails …) validated
Docking/undocking/layout saving validated
Manual Forecast Display
Edit run options
Forecaster Help display
Archive export and import
Theme 2 SA Testing:
Validation of fixes.
Test ALL modifiers
Status icon updating based on model runs and modifier interactions
Theme 3 SA Testing:
Validation of fixes.
Special configurations (data qc)
Rating display
Correlation display
Statistics on the fly
Snow Updating
Search and Select Forecasts
Tabular ConfigFiles Display
Schematic Status Displays
F-12 Key functions
Theme 4 Live System:
Transition to Live System (1 week install, then testing)
Validation of fixes.
System performance (speed)
Data Synchronization OC/FSS
Scheduled jobs run as expected
System monitor reports as expected
Forecast Manager reports as expected
Admin Interface functions
Config Manager
PI-Service imports, exports, archive access
Theme 5 Live System:
Open and Continuous testing
Proposed Schedule
November 15 – Beta Test Planning call
December 1 – Development Testing begins
DNL test team and developers validating updates
December 15 – Go/No-go meeting for January 29 Beta start with Beta Testers
Presentation of the build status with rationale for Go/No-go recommendation
March 5 – Alpha Testing –
Live systems in Silver Spring installed
April 2 –Beta Testing Theme 1 – SA Testing
April 9 – Development Week 1
April 16 – Beta Testing Theme 2 – SA Testing
April 23 – Development Week 2
April 30 – Beta Testing Theme 3 – SA Testing
May 7 – Development Week 3
May 14 – Live System Installs
May 21 – Beta Testing Theme 4 – Live System Testing
May 28 – Development Week 4
June 4 – Open and continuous testing begins
A period of testing and fixing with software updates as needed
June 15 – End of open testing
Individual acceptance to close testing
June 20 – Full Release available