iMOD Forum

To report a problem/bug or request a new feature,
please go to its subcategory in 'iMOD-GUI' or 'iMODFLOW' below


First-time users: please have a look here for info on the iMOD forum-principles.

« Back to User Questions

Parameter Estimation

Combination View Flat View Tree View
Threads [ Previous | Next ]
toggle
Parameter Estimation
Answer (Unmark)
11/15/16 2:56 PM
Hi

I am currently learning how to use the PST module and have looked at all previous comments from this community about it. Like Alphoso, I ran the PEST and some new folders were generated inside the folder "pest", they are named "factor0", "factor1", that have some IDF files inside, but theIDF haVE no data values.

Also are generated some txt files named "log_pest_efficiency_mf2005!.txt", "log_pest_mf2005.txt", "log_pest_progress_mf2005.txt", "log_pest_residual_0mf2005.txt" I have opened these text files but i dont understand their meaning. Can someone help please
RE: Parameter Estimation
Answer (Unmark)
11/15/16 9:36 PMas a reply to Patrick Wakhungu.
Hi Patrick,

Nice hearing from you.

Regarding the files:

log_pest_efficiency_mf2005!txt - use this file to read a short summary of the optimization, objective function value and parameter adjustments, efficiency is expressed by the ratio of objective function reduction and parameter adjustment. If that number tend toward 1.0, optimization is finished

log_pest_mf2005.txt - total overview of the details of each iteration, including covraciences, correlation matrices and eigenvalues. Important in this files are the number of nodes associated for each zone, listed in the beginning of the file

log_pest_progress_mf2005.txt - another sumamry of the optimiation process, lists all individual objective function values per iteration cycle

log_pest_residual_0mf2005.txt - list the residual per measurment for each iteration cycle, cycle zero is the start, oftern I compare the initial version (0-version) with the final one to see what measurment s improved and those who didn't.

factor.idf - those files represent the optimization values per parameter saved in seperate IDF files. If nodata exists in those files, no gridcells were allocated to zones or something else went wrong.

To help you most efficiently, best if you send me your runfile first. After that, sending me your complete model (only the data specified in the runfile) would be necessary to assist you optimally.

Greetings,
Peter Vermeulen
RE: Parameter Estimation
Answer (Unmark)
11/18/16 3:22 PMas a reply to Peter Vermeulen.
Hi Peter

I realized that the problem was with my IDF Zone definition of my PST module. I tried to change it but to no avail. I have attached my runfile for you to look at. Please let me know where my problem is. Thanks
Attachments: IMODFLOW.RUN (8.2k)
RE: Parameter Estimation
Answer (Unmark)
11/18/16 3:30 PMas a reply to Patrick Wakhungu.
I forgot to attach my Zone IDF. Here it is
Attachments: PSTZones_L1.IDF (418.8k), log_pest_efficiency_mf2005.txt (0.2k), log_pest_mf2005.txt (1.8k), log_pest_sensitivity_mf2005.txt (0.1k)
RE: Parameter Estimation
Answer (Unmark)
11/18/16 4:11 PMas a reply to Patrick Wakhungu.
Hi Patrick,

You'd grouped all parameters together, see the adjusted PST section below. It is also better to apply a finite-difference step of 1.1 instead of 2.0, so I changed that too.

6,(PST)
10,10.00,0.05,0,0,1.00,0.00,0,0.05,0.01,1
1,RC,1,1,1.00,1.10,0.10,100.00,4.00,1
1,KV,1,1,1.00,1.10,0.10,100.00,4.00,2
1,KH,1,1,1.00,1.10,0.10,100.00,4.00,3
1,RI,1,1,1.00,1.10,0.10,100.00,4.00,4
1,VA,1,1,1.00,1.10,0.10,100.00,4.00,5
1,RE,1,1,1.00,1.10,0.10,100.00,4.00,6
1
E:\GW_MODEL_RIFTVALLEY_2016_PATRICK\INPUT_FILES\PARAMETER_ESTIMATION\VERSION_1\PSTZONES_L1.IDF


Can you send me the log_pest_residual_*.txt as well because I think the computation of the residual does go wrong.

Regards,
Peter Vermeulen
RE: Parameter Estimation
Answer (Unmark)
11/18/16 4:20 PMas a reply to Patrick Wakhungu.
Hi Peter

Here is the log_pest_residual_*.txt
Attachments: log_pest_residual_0mf2005.txt (0.2k)
RE: Parameter Estimation
Answer (Unmark)
11/18/16 4:31 PMas a reply to Patrick Wakhungu.
Hi Peter,

I have effected the suggestions you have made and these are the file outputs - Attached
Attachments: IMODFLOW.RUN (8.2k), log_pest_efficiency_mf2005.txt (0.2k), log_pest_mf2005.txt (1.8k), log_pest_progress_mf2005.txt (0.2k), log_pest_residual_0mf2005.txt (0.2k)
RE: Parameter Estimation
Answer (Unmark)
11/19/16 11:03 PMas a reply to Patrick Wakhungu.
Hi Patrick,

After examining your runfile, you didn't change it as I told you, you still have grouped all parameters together:

6,(PST)
10,10.00,0.05,0,0,1.00,0.00,0,0.05,0.01,1
1,RC,1,1,1.00,1.10,0.10,100.00,4.00,1,1
1,KV,1,1,1.00,1.10,0.10,100.00,4.00,1,2
1,KH,1,1,1.00,1.10,0.10,100.00,4.00,1,3
1,RI,1,1,1.00,1.10,0.10,100.00,4.00,1,4
1,VA,1,1,1.00,1.10,0.10,100.00,4.00,1,5
1,RE,1,1,1.00,1.10,0.10,100.00,4.00,1,6
1
E:\GW_MODEL_RIFTVALLEY_2016_PATRICK\INPUT_FILES\PARAMETER_ESTIMATION\VERSION_1\PSTZONES_L1.IDF


need to be:

6,(PST)
10,10.00,0.05,0,0,1.00,0.00,0,0.05,0.01,1
1,RC,1,1,1.00,1.10,0.10,100.00,4.00,1
1,KV,1,1,1.00,1.10,0.10,100.00,4.00,2
1,KH,1,1,1.00,1.10,0.10,100.00,4.00,3
1,RI,1,1,1.00,1.10,0.10,100.00,4.00,4
1,VA,1,1,1.00,1.10,0.10,100.00,4.00,5
1,RE,1,1,1.00,1.10,0.10,100.00,4.00,6
1
E:\GW_MODEL_RIFTVALLEY_2016_PATRICK\INPUT_FILES\PARAMETER_ESTIMATION\VERSION_1\PSTZONES_L1.IDF


Please have a close look ad the runfile description in the iMOD Manual, starting from page 513.

Finally, I didn't see any measurements from the IPF in the *residual* logfile. I think you made a typo in the definition of the IPF file in the runfile, below is yours and in bold ("1") you define that the layer is defined by the first column, the same column as you use for the x-coordinate of the measurement:

E:\GW_Model_RiftValley_2016_Patrick\Input_Files\Wells\Version_1\Lithology\LithologyWells2.ipf,1,1,2,1,4,3

I believe you need to change that.

Hopefully those changes will start you going.

Regards
Peter Vermeulen
RE: Parameter Estimation
Answer (Unmark)
11/20/16 7:47 PMas a reply to Peter Vermeulen.
Hi Peter

Thank you for that Correction. There has been some improvements now but the IDF still have no values.

I think the problem is with my IPF file columns as you have suggested

E:\GW_Model_RiftValley_2016_Patrick\Input_Files\Wells\Version_1\Lithology\LithologyWells21.ipf,1,1,2,1,4,3

My understanding about it was:
1 - X axis
2 - Y axis
1 - Layer number being optimized (Layer 1 in my case - not as a column in the ipf )
4 - Z axis
3 - Variance, I dont know how this is determined (I tried with 0 and 1 in column 3 of the ipf )


I have now made some other changes with the ipf file as follows

E:\GW_Model_RiftValley_2016_Patrick\Input_Files\Wells\Version_1\Lithology\LithologyWells21.ipf,1,1,2,5,4,3
1 - X axis
2 - Y axis
5 - Layer number being optimized (I have created a new column in the ipf with the value 1 all through )
4 - Z axis
3 - Variance, I dont know how this is determined (I have created a new column in the ipf with the value 1 all through )

The results are attached
Attachments: log_pest_efficiency_mf2005.txt (1.1k), log_pest_mf2005.txt (61.9k), log_pest_residual_10mf2005.txt (5.8k), log_pest_runfile_mf2005.txt (5.9k), log_pest_sensitivity_mf2005.txt (0.8k)
RE: Parameter Estimation
Answer (Unmark)
11/22/16 8:31 AMas a reply to Patrick Wakhungu.
Hi Patrick,

I've ran your model and the optimization went well. Now, you need to decide whether you are going to include more zones or use weight values for your measurements to improve your model more. Please have a look at the technical documentation of PST in the iMOD Manual to understand a bit more on the mathematical background of this optimization process.

Kind regards,
Peter Vermeulen
RE: Parameter Estimation
Answer (Unmark)
11/23/16 3:36 AMas a reply to Peter Vermeulen.
Hi Peter

Thank you for your previous comments. Have really been helpful. I have made many changes since and my PEST module runs but IDF's have no value still. I was wondering when you ran my model, did your IDF's have values. I am attaching my last ran PEST for your review and advice. The parameter zones IDF (Attached) is different from last time.

Thank you
Attachments: GeologyZones_L1.IDF (418.8k), IMODFLOW.RUN (8.2k), log_pest_efficiency_mf2005.txt (1.0k), log_pest_mf2005.txt (72.9k), log_pest_residual_9mf2005.txt (5.8k)
RE: Parameter Estimation
Answer (Unmark)
11/23/16 11:15 AMas a reply to Patrick Wakhungu.
Hi Patrick,

Well I think the you cannot use the IDF files, something is wrong, but nevertheless, you need to apply the optimized factors from your log-files to the zones yourself, for now. So, it I look at your log file I see the summary mentioned below:

Upgrade Vector Parameter History:
NO PT ITER010 ITER009 ITER008 ITER007 ITER006 ITER005 ITER004 ITER003 ITER002 ITER001 ITER000
1 KH 0.203 0.212 0.213 0.216 0.217 0.214 0.214 0.213 0.213 0.547 1.000
2 KH 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000
3 KH 0.100 0.125 0.126 0.133 0.135 0.404 0.415 0.448 0.453 0.901 1.000
4 KH 0.526 0.523 0.977 0.300 1.103 0.500 0.654 0.395 1.162 0.391 1.000
5 KV 0.448 0.469 0.470 0.477 0.479 0.908 0.922 0.964 0.969 1.000 1.000
6 KV 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000
0.034 0.454 0.678 0.803 0.788 0.155 0.264 0.767 0.953 0.766


It is up to you what optimization cycle (0-10) you pick for your variables. I checked your log file a bit more, but the parameters that you've chosen now, are very insensitive to your objective function, there is hardly any reduction, see below:

TJ SQRT(TJ) MEAN(TJ) MEAN(SQRT(TJ)) ADJUSTMENTS EFFICIENCY
(L2) (L) MEAN(L2) MEAN(L) (%) -
0.1900822E+07 0.1378703E+04 0.4420517E+05 0.3206286E+02 0.1161315E+03 0.1001684E+01
0.1899624E+07 0.1378268E+04 0.4417729E+05 0.3205275E+02 0.3110636E+03 0.1000631E+01
0.1898508E+07 0.1377863E+04 0.4415134E+05 0.3204333E+02 0.6779857E+02 0.1000588E+01
0.1898209E+07 0.1377755E+04 0.4414440E+05 0.3204082E+02 0.7760151E+02 0.1000157E+01
0.1898091E+07 0.1377712E+04 0.4414165E+05 0.3203982E+02 0.2779780E+02 0.1000062E+01
0.1897457E+07 0.1377482E+04 0.4412690E+05 0.3203447E+02 0.2356954E+03 0.1000334E+01
0.1897174E+07 0.1377379E+04 0.4412033E+05 0.3203208E+02 0.7486774E+02 0.1000149E+01
0.1897036E+07 0.1377329E+04 0.4411711E+05 0.3203091E+02 0.2340537E+03 0.1000073E+01
0.1896313E+07 0.1377067E+04 0.4410030E+05 0.3202481E+02 0.4802499E+02 0.1000381E+01


You can also see in the confidential intervals that are sky-high:

Confidence Limits (96%):

KH001003-001 0.0000000 0.2129785 Infinity
KH001004-002 0.0000000 1.0000005 Infinity
KH001005-003 0.0000000 0.1263526 Infinity
KH001006-004 0.0000000 0.9773086***************
KV001003-005 0.0000000 0.4702945 Infinity
KV001004-006 1.0000000 1.0000000 1.0000000


I remember that your previous set of parameters gave a more realistic optimization result, see below:

Confidence Limits (96%):

RC001001-001 100.0000076 100.0000076 100.0000076
KV001001-002 0.5385622 0.5385622 0.5385622
KH001001-003 0.0000000 0.3502407 Infinity
RI001001-004 0.0495009 0.1471716 0.4375574
VA001001-005 1.0000000 1.0000000 1.0000000
RE001001-006 1.0000000 1.0000000 1.0000000

Upgrade Vector Parameter:
NO PT NEW.FACTOR PREV.FACTOR BND
1 RC 100.0000076 100.0000076 1
2 KV 0.5385622 0.5385622 0
3 KH 1.0357687 0.3502407 0
4 RI 0.1489031 0.1471716 0
5 VA 1.0000000 1.0000000 0
6 RE 1.0000000 1.0000000 0

Upgrade Vector Parameter History:
NO PT ITER010 ITER009 ITER008 ITER007 ITER006 ITER005 ITER004 ITER003 ITER002 ITER001 ITER000
1 RC 100.000 100.000 100.000 100.000 100.000 100.000 76.862 22.360 12.673 3.236 1.000
2 KV 0.539 0.539 0.539 0.539 0.539 0.539 0.539 0.990 1.000 1.000 1.000
3 KH 1.036 0.350 0.668 0.213 0.726 0.366 0.363 0.578 0.990 0.999 1.000
4 RI 0.149 0.147 0.148 0.146 0.144 0.139 0.181 0.683 1.351 1.000 1.000
5 VA 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000
6 RE 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000
0.686 0.318 0.455 0.513 0.360 23.138 54.506 9.719 9.443 2.236


Sensitivity (%):
Iteration RC001001 KV001001 KH001001 RI001001 VA001001 RE001001
1 84.078 1.840 9.560 4.522 0.000 0.000
2 59.100 0.509 3.118 37.273 0.000 0.000
3 54.628 0.062 0.904 44.406 0.000 0.000
4 52.595 0.066 0.556 46.783 0.000 0.000
5 50.635 0.023 0.394 48.949 0.000 0.000
6 50.444 0.018 0.399 49.140 0.000 0.000
7 50.280 0.009 0.749 48.962 0.000 0.000
8 50.560 0.011 0.231 49.198 0.000 0.000
9 50.328 0.007 0.682 48.984 0.000 0.000
10 50.484 0.010 0.374 49.132 0.000 0.000


Finally, check the sensitivities:

Sensitivity (%):
Iteration RC001001 KV001001 KH001001 RI001001 VA001001 RE001001
1 84.078 1.840 9.560 4.522 0.000 0.000
2 59.100 0.509 3.118 37.273 0.000 0.000
3 54.628 0.062 0.904 44.406 0.000 0.000
4 52.595 0.066 0.556 46.783 0.000 0.000
5 50.635 0.023 0.394 48.949 0.000 0.000
6 50.444 0.018 0.399 49.140 0.000 0.000
7 50.280 0.009 0.749 48.962 0.000 0.000
8 50.560 0.011 0.231 49.198 0.000 0.000
9 50.328 0.007 0.682 48.984 0.000 0.000
10 50.484 0.010 0.374 49.132 0.000 0.000

Probably the most sensitive parameters are the RC and RI, better leave the rest out.

Kind regards,

Peter Vermeulen
RE: Parameter Estimation
Answer (Unmark)
11/24/16 3:29 AMas a reply to Peter Vermeulen.
Hi Peter

One last question. Is there any output Text file in iMOD that compares Observed Heads with Simulated Heads? Secondly, must calibration be done only via iPEST?

Thank you for continued support
RE: Parameter Estimation
Answer (Unmark)
11/24/16 1:41 PMas a reply to Patrick Wakhungu.
Hi Patrick,

You can use the log_pest_residual_*mf2005.txt files. Here, each observation is listed, see example below:

E:\GW_Model_RiftValley_2016_Patrick\Input_Files\Wells\Version_1\Lithology\LithologyWells21.ipf

X Y ILAY MSR MDL J WMDL WRESIDUAL WEIGH
159665.0000000 9983730.0000000 1 1807.0000000 1708.4429932 9713.4824219 1708.4428711 -98.5569992 0.9999999
219177.0000000 9923667.0000000 1 2129.0000000 1930.8149414 39277.3125000 1930.8148193 -198.1850433 0.9999999
177719.0000000 9969015.0000000 1 1856.0000000 1767.0909424 7904.8198242 1767.0908203 -88.9090500 0.9999999
170919.0000000 9987495.0000000 1 1714.0000000 1670.1409912 1923.6124268 1670.1408691 -43.8590050 0.9999999
215058.0000000 9920014.0000000 1 1955.0000000 1894.8590088 3616.9384766 1894.8588867 -60.1409874 0.9999999
215392.0000000 9919793.0000000 1 1955.0000000 1894.4229736 3669.5759277 1894.4228516 -60.5770226 0.9999999
215840.0000000 9916474.0000000 1 1995.0000000 1885.5219727 11985.4375000 1885.5218506 -109.4780197 0.9999999
215174.0000000 9912491.0000000 1 1966.0000000 1875.7480469 8145.4145508 1875.7479248 -90.2519455 0.9999999

No, you can optimize the model like you want. iPEST is an automatic optimization algorithm, but if you just want to optimize a few parameters, you could do it manually. In that ase you could use the iMOD Batch functions IDFTIMESERIE and IPFRESIDUAL to compare observed and computed heads.

Kind regards,

Peter Vermeulen