[ana6, com8]: Add ana6Optimisation Module, apply changes in com8MoTPSA#1245
[ana6, com8]: Add ana6Optimisation Module, apply changes in com8MoTPSA#1245RolandFischbacher wants to merge 15 commits intomasterfrom
Conversation
❌ 2 blocking issues (5 total)
@qltysh one-click actions:
|
avaframe/out3Plot/outAna6Plots.py
Outdated
| # 4) Plot | ||
| x = np.arange(len(df)) | ||
| fig, ax = plt.subplots(figsize=(12, 6)) | ||
| bars = ax.bar( |
f66f702 to
38e42ca
Compare
Squash of 20 commits from RF_com8MoTPSA branch including: - com8MoTPSA workflow improvements (chunked multiprocessing, path handling) - Bayesian optimisation integration (ana6Optimisation module) - Morris sensitivity analysis scripts - AIMEC runout reference implementation - probAna pickle saving and bounds - Plotting and config improvements
|
Coverage Impact ⬇️ Merging this pull request will decrease total coverage on Modified Components (1)
Modified Files with Diff Coverage (2)
🤖 Increase coverage with AI coding...🚦 See full report on Qlty Cloud » 🛟 Help
|
- Add bounds to paramValuesD in createSamplesWithVariation (StandardParameters) - Add writing of visualisation scenario and sampling method to com8MoTPSACfg.ini
| @@ -0,0 +1,886 @@ | |||
| import numpy as np | |||
There was a problem hiding this comment.
A general note for this file: use log.info instead of prints for user info
| return xBest, xBestDict, np.max(ei), np.max(lcb) | ||
|
|
||
|
|
||
| def runCom8MoTPSA(avalancheDir, xBestDict, cfgMain, i=0, optimisationType=None): |
There was a problem hiding this comment.
I think this could be solved differently, as it duplicates code that might be changed. Investigate if a direct call with updated cfgs would be better. Let me now if you need help
There was a problem hiding this comment.
Implemented a direct call of com8MoTPSAMain with updated cfgs (writeCfgFiles in optimisationUtils.py)
| if 'VISUALISATION' in config.sections(): | ||
| # config is inifile | ||
| index = config['VISUALISATION']['scenario'] | ||
|
|
||
| if 'VISUALISATION' in config.sections(): | ||
| # config is inifile | ||
| index = config['VISUALISATION']['scenario'] | ||
| if 'sampleMethod' in config['VISUALISATION']: | ||
| sampleMethod = config['VISUALISATION']['sampleMethod'] |
There was a problem hiding this comment.
if any .ini file in the config directory lacks a VISUALISATION section, the variables index and sampleMethod are never assigned, but they are unconditionally appended on lines 107-108. On the first such file, this raises UnboundLocalError. Even if subsequent iterations reuse a stale value from a previous file, the data would be silently wrong.
There was a problem hiding this comment.
Thank you for the info. I will initialize index and sampleMethod with np.nan to avoid UnboundLocalError and stale values.
| # config is inifile | ||
| index = config['VISUALISATION']['scenario'] | ||
|
|
||
| if 'VISUALISATION' in config.sections(): |
There was a problem hiding this comment.
Why the duplicate if? Copy/Paste error?
There was a problem hiding this comment.
Yes copy paste error.
avaframe/ana4Stats/probAna.py
Outdated
| if modName.lower() in ["com1dfa", "com5snowslide", "com6rockavalanche", 'com8motpsa']: | ||
| cfgStart["VISUALISATION"]["scenario"] = str(count1) | ||
| cfgStart["INPUT"]["thFromIni"] = paramValuesD["thFromIni"] | ||
| cfgStart["VISUALISATION"]["sampleMethod"] = cfg['PROBRUN']['sampleMethod'] |
There was a problem hiding this comment.
The new line cfgStart["VISUALISATION"]["sampleMethod"] = cfg['PROBRUN']['sampleMethod'] reads sampleMethod from cfg['PROBRUN'].
However, probAnaCfg.ini has sampleMethod under [PROBRUN] only when probAna is the caller. If createCfgFiles is called from a different path where cfg doesn't have PROBRUN.sampleMethod, this will raise KeyError. The code also assumes VISUALISATION section exists in cfgStart for com8MoTPSA — while the new com8MoTPSACfg.ini does add it, there's no sampleMethod default there, making the flow dependent on the caller always providing this key.
There was a problem hiding this comment.
Added check if 'VISUALISATION' exists in cfgStart, if not, it will be added to cfgStart.
And read sample method with fallback; meaning if 'PROBANA' or 'sampleMethod' is missing sample_method contains an empty string.
…i file, delete redundant code
…Files in optimisationUtils.py), add possibility to RUN writeCfgFiles with counter
…clean header Implement modName to make code general and remove adjustText package
…ror and stale values and remove copy paste error.
… added to cfgStart. And read sample method with fallback; meaning if 'PROBANA' or 'sampleMethod' is missing sample_method contains an empty string
| if modName.lower() in ["com1dfa", "com5snowslide", "com6rockavalanche"]: | ||
| if modName.lower() in ["com1dfa", "com5snowslide", "com6rockavalanche", 'com8motpsa']: | ||
| # Check if visualisation exists in cfgStart, if not add the section | ||
| if not cfgStart.has_section("VISUALISATION"): |
| index = config['VISUALISATION']['scenario'] | ||
| if 'sampleMethod' in config['VISUALISATION']: | ||
| sampleMethod = config['VISUALISATION']['sampleMethod'] | ||
| else: |
| nCPU = cfgUtils.getNumberOfProcesses(cfgMain, len(rcfFilesChunk)) | ||
|
|
||
| with Pool(processes=nCPU) as pool: | ||
| results = pool.map(com8MoTPSATask, rcfFilesChunk) |
… docstring, tidy up code, add if __name__='__main__': in runPlotMorrisConvergence.py and improve BOConvergencePlot.
awirb
left a comment
There was a problem hiding this comment.
first comments, more to come
| # check for allConfigurationsInfo to find computation info and add to info fetched from ini files | ||
| if latest == False and isinstance(simDF, pd.DataFrame): | ||
| # check if in allConfigurationsInfo also info for existing sims | ||
| simDFALL, _ = readAllConfigurationInfo(avaDir, specDir="", configCsvName="allConfigurations") |
There was a problem hiding this comment.
would you also require here to set the modName variable that it doesn't just read the allConfigurations.csv from the Outputs of com1DFA?
There was a problem hiding this comment.
allConfigurations.csv is currently not available for com8MoTPSA module
| @@ -0,0 +1,64 @@ | |||
| ### Config File - This file contains the main settings for the optimisation process | |||
|
|
|||
| # Sidenote: (1) when running runOptimisation.py the working directory needs to be in the ana6Optimisation folder | |||
There was a problem hiding this comment.
why is it actually located here and not in the runScripts directory?
|
|
||
|
|
||
| [GENERAL] | ||
| # USER input for running and plotting a comparison of simulation result to reference polygon |
There was a problem hiding this comment.
does this refer to the settings in runPlotAreaProfile?
There was a problem hiding this comment.
Yes this refers to the settings in runPlotAreaRefDiffs.py
| [PARAM_BOUNDS] | ||
| # 2 scenarios: choose 1 or 2 | ||
| scenario = 1 | ||
| #(1): morris is run prior, then dataframe of ranked input parameters is already saved by runMorris.py as pickle file, user only need to determine how much input paramters to use for optimisation |
There was a problem hiding this comment.
| #(1): morris is run prior, then dataframe of ranked input parameters is already saved by runMorris.py as pickle file, user only need to determine how much input paramters to use for optimisation | |
| #(1): morris is run prior, then dataframe of ranked input parameters is already saved by runMorris.py as pickle file, user only needs to determine how much input parameters to use for optimisation |
There was a problem hiding this comment.
Applied locally
| # 2 scenarios: choose 1 or 2 | ||
| scenario = 1 | ||
| #(1): morris is run prior, then dataframe of ranked input parameters is already saved by runMorris.py as pickle file, user only need to determine how much input paramters to use for optimisation | ||
| topN = 3 |
There was a problem hiding this comment.
what is this parameter? add description
There was a problem hiding this comment.
and move description of second scenario up
| Contains Cropshape and defines the maximal extent of runout area that is used for calculating areal indicators. | ||
|
|
||
| - **REFDATA** | ||
| Defines the runout area of the reference event. |
There was a problem hiding this comment.
add info that this needs to have the suffix _POLY.shp (if polygon is required)
| - **Digital Elevation Model (DEM)** | ||
| Must be placed directly in the `Inputs` directory and must cover the entire affected area. | ||
|
|
||
| More Details here: https://docs.avaframe.org/en/latest/moduleCom1DFA.html |
There was a problem hiding this comment.
Do you mean that in provided link in section Inputs?
| - Run `runMorrisSA.py` | ||
| - Configure settings via `runMorrisSACfg.ini` | ||
| - The `MORRIS_CONVERGENCE` setting can be ignored for standard sensitivity analysis | ||
|
|
There was a problem hiding this comment.
is it required to set in ana3AIMECCfg.ini or local copy the comMod and runoutLayer info? and flagMass=False and includeReference=True please add info that this requires aimec settings and these need to be set - OR add override section in runMorrisSACfg.ini
There was a problem hiding this comment.
Actually I think a section: ana3AIMEC_ana3AIMEC_override in the runMorrisSACfg.ini file would be the desired solution
|
|
||
| [GENERAL] | ||
| # USER input for running and plotting a comparison of simulation result to reference polygon | ||
| resType = ppr |
|
|
||
| [GENERAL] | ||
| # USER input for running and plotting a comparison of simulation result to reference polygon | ||
| resType = ppr |
awirb
left a comment
There was a problem hiding this comment.
comments for the optimisation part and plotting still missing
| - thresholdValueSimulation | ||
| - modName | ||
| avalancheDir : str | ||
| Directory containing the directory of the reference avalanche |
There was a problem hiding this comment.
do you mean just the path to the avalanche directory?
| ana3AIMEC : module | ||
| AIMEC analysis module providing `fullAimecAnalysis()`. |
There was a problem hiding this comment.
why does this need to be passed to the function?
| # ToDo: to reduce comp. cost: run AIMEC and calcArealIndicators only for new simulations | ||
| # Areal indicators | ||
| resType = cfgOpt["GENERAL"]["resType"] | ||
| thresholdValueSimulation = float(cfgOpt["GENERAL"]["thresholdValueSimulation"]) |
There was a problem hiding this comment.
use cfgOpt['GENERAL'].getfloat('....
| cfgAIMEC = cfgUtils.getModuleConfig(ana3AIMEC) | ||
| rasterTransfo, resAnalysisDF, plotDict, _, pathDict = ana3AIMEC.fullAimecAnalysis(avalancheDir, cfgAIMEC) |
There was a problem hiding this comment.
instead of using the passed module, consider loading the config already in the runScript before you call the function and just pass the config, then also the override is easier (using ana3AIMEC_ana3AIMEC_override) or is there a special reason for passing the module?
| Build the final merged DataFrame for a given avalanche. | ||
|
|
||
| Combines parameter sets, AIMEC results, and areal indicators into one DataFrame, | ||
| then computes evaluation metrics 'via addLossMetrics'. |
There was a problem hiding this comment.
consider adding info on how DF is organised, one row per simulation?
| # Create df with parameters and the loss function for summary statistics | ||
| paramLossDF, paramLossScaledDF = optimisationUtils.createDFParameterLoss(morrisDF, SiDFSort['Parameter']) | ||
|
|
||
| # Raise error if topN is bigger than available model runs |
There was a problem hiding this comment.
can you add a description of topN parameter?
| ) | ||
| raise ValueError(message) | ||
|
|
||
| paramLossSubsetDF = paramLossDF.sort_values(by='Loss', ascending=True)[:N] |
There was a problem hiding this comment.
why is the [:N] needed - is that from start to the end if len(DF) is N no?
|
|
||
| def createDFParameterLoss(df, paramSelected): | ||
| """ | ||
| Create DataFrames linking selected parameters with the loss function. |
There was a problem hiding this comment.
does selected mean - the ones that were used for the parameter variation using the morris sampling method?
| df : pandas.DataFrame | ||
| DataFrame that contains the selected parameters and their values as well as the loss function. | ||
| paramSelected : list of str | ||
| Subset of parameters to include in the output DataFrames. |
There was a problem hiding this comment.
same here, consider adding what kind of subset this is, based on what information?
| # beta gives penalty if sim. comes short compared to the ref. (FN) | ||
| tverskyAlpha = 2 | ||
| tverskyBeta = 1 | ||
| # Loss function is kombination of TverskyScore * weightTversky + RunoutNormalised * weight runout |
There was a problem hiding this comment.
| # Loss function is kombination of TverskyScore * weightTversky + RunoutNormalised * weight runout | |
| # Loss function is a combination of TverskyScore * weightTversky + RunoutNormalised * weight runout |
There was a problem hiding this comment.
Changed locally

This PR introduces a new optimisation module
ana6Optimisationforcom8MoTPSAand updates the simulation workflow.The module ana6Optimisation includes:
New files in ana6Opitmisaton:
runMorrisSA.py(configuration:runMorrisSACfg.ini)runPlotMorrisConvergence.py(usesrunMorrisSACfg.ini)runOptimisation.py(configuration:runOptimisationCfg.ini)optimisationUtils.pyREADME_ana6.md(contains usage instructions)New file in out3Plot:
outAna6Plots.pyChanged workflow of runing com8MoTPSA: