Forum

RE: Removing unwanted data

MM
Mason Marchildon, modified 2 Months ago.

Removing unwanted data

Rookie Crystal gazer Posts: 19 Join Date: 5/1/17 Recent Posts
I have now been working with FEWS for about a year and have suffered many growing pains; thanks to this forum, I've been relieved.  There are some data that have been (regrettably) stored in my FEWS project.  In the interest of saving space, how would I go about deleting such data? Could it be as simple as removing location sets?
IM
Ivo Miltenburg, modified 1 Month ago.

RE: Removing unwanted data

Rookie Crystal gazer Posts: 13 Join Date: 3/12/13 Recent Posts
Hi Mason
The deletion of data in Delft-FEWS is typically orchestrated by setting an "ExpiryTime" for each timeSeries that is imported/created in Delft-FEWS, and a ""Rolling Barrel" process that regurly deletes data thas has expired. This is a typical setup for an operation system (what Delft-FEWS is designed to be. However, I suggest you have a Stand Alone and I guess you have not set these expiry times, so this does not help you a lot. For a Stand Alone application, it is more difficult to remove data from the Delft-FEWS interface directly. But with this background, please take a look at the following links, they give some suggestions for approaches:
https://publicwiki.deltares.nl/display/FEWSDOC/02+Data+Handling+in+Delft-FEWS#id-02DataHandlinginDelft-FEWS-expiryTime
https://publicwiki.deltares.nl/display/FEWSDOC/How+to+influence+the+expiry+time+of+rows+%28records%29+in+the+central+database

https://publicwiki.deltares.nl/display/FEWSDOC/How+to+remove+records+from+the+client-server+system
https://publicwiki.deltares.nl/display/FEWSDOC/HOW-TO+connect+to+a+localdatastore+with+DBVisualizer

https://publicwiki.deltares.nl/display/FEWSDOC/Local+datastore
https://publicwiki.deltares.nl/pages/viewpage.action?pageId=137136610

The DBVisualizer approach is powerful, but requires you learning SQL queries. There are some suggestions on the wiki pages, but this can be quite difficult and specific - depending on your use case. Always keep a copy of your database. It is easy to corrupt your datastore.
The F12 - P Database options might also be useful, but are less specific
An alternative approach is to export all data that you do want to keep and import it in an empty system. This does require setting up custom exports/imports.

Good luck
AV
Arnejan van Loenen, modified 1 Month ago.

RE: Removing unwanted data

Keen Forecaster Posts: 2 Join Date: 8/20/21 Recent Posts
If the data are forecasts that are also visible in the Forecast Management Display, you can easily remove them:
1) Open Forecast Management Display
2) select "Forecast Expiry Time" tab
3) select the forecasts you want to have removed
4) Press delete button
5) go to map display, press F12, go to database and select "rolling barrel local datastore"
6) possibly also needed: press F12, go to database and select "repair and defrag"
AV
Arnejan van Loenen, modified 1 Month ago.

RE: Removing unwanted data

Keen Forecaster Posts: 2 Join Date: 8/20/21 Recent Posts
It is indeed also possible to remove data from the database manually. But to start: best approach would be to be able to just start with an empty database and import and process the data that you really need. The configuration should have been set-up to allow this.

If this is not feasible and you have a localDatastore with data that is difficult to reproduce, it is indeed possible to delete data using DBVisualizer. In my experience it is not that difficult. First you need to make a connection as described here:
https://publicwiki.deltares.nl/display/FEWSDOC/HOW-TO+connect+to+a+localdatastore+with+DBVisualizer
(make sure FEWS is not running). Then use SQL to query the data you want to have removed. Can be simple:
DELETE FROM TimeSeries WHERE moduleInstanceId='some moduleinstance'.

But you can use SQL to further refine what you want.

So far so good, but this just clears the data from the database, but not reduce disc space. To do this, start FEWS, press F12 and select under Database the option "repair and defrag local database". This will reduce the file size.

Please note that just deleting data from the timeSeriesTable could have adverse effects. Directly editing the database is always a last effort. So indeed make sure to make a backup first!