Forum

Interpolation inconsistent? possible bug? ["IMPORT"]

MM
Mason Marchildon, modified 2 Months ago.

Interpolation inconsistent? possible bug? ["IMPORT"]

Rookie Crystal gazer Posts: 11 Join Date: 5/1/17 Recent Posts
I've just completed a rather large interpolation from ~40 scalar locations to a set (~2500) of catchment areas. All seems to be good except for a couple spurious locations. See for example the attached images, I have two adjacent basins where temperature has been interpolated, but something happened to the interpolation at one of the basins.  Has anyone seen this before?
IM
Ivo Miltenburg, modified 2 Months ago.

RE: Interpolation inconsistent? possible bug? ["IMPORT"]

Keen Forecaster Posts: 7 Join Date: 3/12/13 Recent Posts
Hi Mason,
It would be useful if you post the moduleConfigFile that does the spatial inteprolation. There are various options for that transformation (for starters the "type" of spatial interpolation) that might play a role here. I am sure you have looked at: https://publicwiki.deltares.nl/display/FEWSDOC/Interpolation+Spatial+Transformations
An interpolationSpatial inverseDistance might make most sense in your case? 
There might be a dodgy station that influences the results at one basin? It also is a good habit to combine the spatial interpolation with a serial interpolation in case some of the nearby stations have missing data at the same time. 

Also note that it is possible for a number of interpolation transformations to define a "validationRule" to control what inputs can be used and if an output flag should be written to e.g. show some doubtful inputs being used:https://publicwiki.deltares.nl/display/FEWSDOC/20+Transformation+Module+-+Improved+schema#id-20TransformationModule-Improvedschema-Validationrules 

I hope this helps, let us know if this improved the situation...
MM
Mason Marchildon, modified 2 Months ago.

RE: Interpolation inconsistent? possible bug? ["IMPORT"]

Rookie Crystal gazer Posts: 11 Join Date: 5/1/17 Recent Posts
Ivo,
Here's the module config file ($DISTANCEGEODATUM$ = UTM17N):



Nothing complex, only a Thiessen polygon interpolation, which is why I suspect an issue with FEWS, and not my configuration (honestly, I hope to be eating these words emoticon. My rationale is that the stations I'm using (locationSetId: MSChourly) is spaced much greater than the basins (locationSetId: $MODELNAME0$SubBasins) I'm interpolating to. Basins that are showing this error tend to be isolated/random.  What's weird is that basins surrounding the one in error seem to be fine, so I'm doubtful that a "dodgy" station is affecting the process due to the lack of pattern (see for example the attachments of my last post). Note that there are periods where stations have missing data, but I wouldn't call that dodgy, I made sure prior to exercising this transformation.

Here are the nearest stations I'm interpolating from:



And here is their data (note: not all stations selected above have data during the shown time period):



again compare that to a particular basin surrounded by the above stations:



So I'm not clear how a validation will help in this case since there is always available data to interpolate from at every time steps.
IM
Ivo Miltenburg, modified 2 Months ago.

RE: Interpolation inconsistent? possible bug? ["IMPORT"]

Keen Forecaster Posts: 7 Join Date: 3/12/13 Recent Posts
Mason,what you might wanna do for debugging purposes is the following:
1. Create a thiessenPolygon gridded output timeseries: https://publicwiki.deltares.nl/display/FEWSDOC/InterpolationSpatialContainingPolygonIt should produce grid timeseries likt thisMaybe if you inspect these grids for some suspect timesteps this gives a hint?
2. Rerun the transformation, but with just a single location as output (one that gives weird values). Is it consistent?
3. Rerun the workflow in "debug" mode: select the workflow in the Forecast tree -> press F12 -> Select modules to log debug messages. You might want to run this for just a single output location (see (2) ... Anything suspicious in the debug messages?

Good luck!
IM
Ivo Miltenburg, modified 2 Months ago.

RE: Interpolation inconsistent? possible bug? ["IMPORT"]

Keen Forecaster Posts: 7 Join Date: 3/12/13 Recent Posts
My suspicion is that your polygon is on the boundary of multiple ThiessenPolygons. I know that a recent improvement was made to the 2019.02 code for the "InterpolationSpatialContainingPolygon" transformation: the transformation will now copy the value of the polygon which has the largest overlapping area with the grid cell in case the grid cell intersects multiple polygons. This prevents fuzzy boundaries - previously a weighted  method was used.
In case this mechanism is indeed the culprit, you should contact delft-fews productmangement to see if/how you could get a patch containing the improvement.
MM
Mason Marchildon, modified 2 Months ago.

RE: Interpolation inconsistent? possible bug? ["IMPORT"]

Rookie Crystal gazer Posts: 11 Join Date: 5/1/17 Recent Posts
Thanks, I'll look into your above suggestions, but to make things more interesting, in digging a little deeper, I found a couple of basins (to be clear, I'm not interpolating to a grid, rather a polygon LocationSet) where the interpolation

Notice that none of the gauges I'm interpolating from has temperatures >35°C. This sure smells buggy to me.
MM
Mason Marchildon, modified 1 Month ago.

RE: Interpolation inconsistent? possible bug? ["IMPORT"]

Rookie Crystal gazer Posts: 11 Join Date: 5/1/17 Recent Posts
Thank you Ivo for the suggestion: I opted for inverseDistance in place of thiessenPolygon, all as expected, no issue. I will refrain from using thiessenPolygon.