null 5 years of ‘Making North-American Hydrological Forecasting great again…'

5 years of ‘Making North-American Hydrological Forecasting great again…'

Now we are rapidly approaching the holiday season, and our calendars are full of conflicting Christmas parties, we typically take time for some sort of reflection. Allow me to use this blog for a personal as well as a more intercontinental hydrological contemplation, now I am transitioned back in Delft after 5 year working for Deltares USA. A (very) general comparison between hydrological forecasting on both sides of the pond, and a Delft-FEWS community that strengthens both.

Although Deltares USA (3 people upon arrival in 2012, currently 6) is closely aligned with Big Brother in Delft, and use of Skype and Gotomeeting in the family is daily practice, I quickly discovered some striking differences: no secretary, no IT support, no financial staff, no cleaners. On the other hand, limited amount of meetings, agenda freedom, and active involvement in the whole project cycle from project acquisition to full execution and (sometimes painful) invoicing administration. In short, developing useful insight what it means to run a small business. People complaining about Deltares' rates should ask their higher management what they pay for lawyers and accountantsJ. And Deltares colleagues complaining about the IT helpdesk, secretary or the financial Maconomy system should work for Deltares USA for 5 years.

The Delft-FEWS community gradually expanded to grow after NOAA's National Weather Service adopted Delft-FEWS in 2008 as the core of their Community Hydrologic Prediction System (CHPS). The C of CHPS was not just to complete the acronym. Soon other agencies joined the community of Delft-FEWS users, like hydro power authorities Bonneville Power Administration (BPA), Tennessee Valley Authority (TVA) and the Canadian province of New Brunswick. Currently there over 10 agencies in North America that use Delft-FEWS 24/7 for their operational forecasting business, covering an impressive part of the North-American watersheds.

Expansion in time of Delft-FEWS agencies in North-America

Having been involved in operational hydrological forecasting projects in Europe, and the new experiences gained with various clients in North America, made me realize that differences exist between hydrological forecasters on both sides of the pond. Or perhaps more importantly, in the way they generate their forecasts.

Generally spoken:

  • forecasters in North America have, in addition to a solid hydrologic knowledge base, an impressive feeling for IT and scripting. Like  the Wild West exploration back in the days, forecasters find their way if something get stuck, and are not limited by just the knowledge of the forecasting software itself.
  • Probably related to the difference in scale, but hydrological models in North America used for operational forecasting tend to be much more conceptual and simple compared to models used in Europe. As a result, there is more need for manual interaction between the forecaster and the model, in terms of state updating, time series adjustments and parameter tweaking. Forecasters in Europe tend to rely more on their more physically based, well calibrated models in combination with automatic data assimilation techniques like ensemble kalman filtering and ARMA.

The beauty of this reflection is to see the Delft-FEWS community at work, way beyond the level of gathering in Delft once a year. An example you ask? Before 2009 the level of model interaction was very limited in Delft-FEWS, apparently there hardly was any operational need for clients in Europe. Now, 8 years later, probably every client has configured the so called Interactive Forecasting Display, initially requested by North American clients, allowing forecasters to be more actively involved in their hydrological and hydraulic models. Most European clients have now developed their own type of ‘modifiers'.  On the other hand, more and more you see data assimilation techniques, like automatic calibration and state updating, applied in North American Delft-FEWS applications. The Delft-FEWS community helps forecaster to broaden their perspective, and to reflect their current operational forecasting paradigm on experiences and ideas initialized and practiced elsewhere.

It is not on this intercontinental scale we see the benefits of a (Delft-FEWS) community. Between clients in North-America, there is cooperation expressed in the communication of forecasting products (i.e.  NERFC vs New Brunswick, NERFC vs Quebec, BC Hydro vs BPA, TVA vs LMRFC), OHD model calibrations (i.e.  NWS vs TVA and BPA) and even Delft-FEWS configuration (i.e.  TVA vs BPA vs IdahoPower). These agencies have found and strengthened each other, and I think the Delft-FEWS community has contributed to this collaboration. The first North-American Delft-FEWS User Days in 2016 therefore was a great success, and we are looking forward to the next one, planned in 2019.

Also within agencies we see the Delft-FEWS community at work. Different groups within agencies with different roles and responsibilities resulting often in different cultures have found each other now they work with the same software for their operational needs. For example at BPA, TVA and Idaho Power I believe Delft-FEWS has enabled improved internal communication and IT structures, drastically reducing the number of back end scripts, and therefore improving quality of forecasting products.

While I have truly enjoyed my time in America, both work and private, I look forward helping other agencies on the globe in their operational forecasting needs. I have learned a lot at each agency I was involved with, and realized again that the two way traffic between Deltares and Delft-FEWS users is crucial. While Deltares as a research institute should keep focusing on bridging the gap between research and 24/7 operations and therefore bringing state of the art hydrological expertise to operational agencies, we learn from every client that robustness, reliability, backwards compatibility and simplicity are high operational values. The challenge remains to balance this ‘simplexity'.

 I wish you a balanced 2018!

Matthijs Lemans