Mercurial > repos > siwaa > wine_quality_train_eval

annotate fidle__init__.py.original @ 13:2f6c77865180 draft

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
author siwaa
date Thu, 05 Dec 2024 18:28:28 +0000
parents
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
13
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
1 # ==================================================================
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
2 # ______ _ _ _ __ __ _
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
3 # | ____(_) | | | | \/ | | |
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
4 # | |__ _ __| | | ___ | \ / | ___ __| |
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
5 # | __| | |/ _` | |/ _ \ | |\/| |/ _ \ / _` |
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
6 # | | | | (_| | | __/ | | | | (_) | (_| |
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
7 # |_| |_|\__,_|_|\___| |_| |_|\___/ \__,_|
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
8 #
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
9 # ==================================================================
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
10 # A simple module to host usefull functions for Fidle practical work
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
11 # Jean-Luc Parouty CNRS/MIAI/SIMaP 2022
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
12 # Contributed by Achille Mbogol Touye MIAI/SIMAP 2023 (PyTorch support)
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
13
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
14 import os,sys,platform
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
15 import glob
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
16 import shutil
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
17 import pathlib
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
18
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
19 from random import randint
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
20 import matplotlib
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
21
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
22 from IPython.display import display,HTML
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
23
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
24 import fidle.config
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
25 import fidle.scrawler
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
26 import fidle.utils
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
27
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
28 from fidle.Chrono import Chrono
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
29
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
30
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
31 __version__ = fidle.config.VERSION
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
32
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
33 __run_id = None
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
34 __run_dir = None
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
35 __datasets_dir = None
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
36
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
37 utils = fidle.utils
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
38 plot = fidle.scrawler
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
39 config = fidle.config
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
40 chrono = Chrono()
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
41
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
42
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
43 # -------------------------------------------------------------
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
44 # init
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
45 # -------------------------------------------------------------
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
46 #
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
47 '''
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
48 Initialization with parameters run_id and run_dir.
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
49 These two parameters can be overridden via environment variables
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
50 FIDLE_OVERRIDE_run_id and FIDLE_OVERRIDE_run_dir.
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
51 datasets_dir is retrieved via the environment variable <datasets_env_var>.
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
52 params:
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
53 run_id : Run id for the notebook (random number if None) (None)
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
54 run_dir : Run directory (./run/{run_id} if None) (None)
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
55 datasets_env_var : Name of env. var. specifying the location of the datasets (FIDLE_DATASETS_DIR)
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
56 return:
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
57 run_id, run_dir, datasets_dir
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
58 '''
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
59 def init(run_id=None, run_dir=None, datasets_env_var='FIDLE_DATASETS_DIR'):
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
60 global __run_id
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
61 global __run_dir
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
62 global __datasets_dir
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
63
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
64 # ---- run_id
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
65 #
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
66 # If None, we choose
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
67 __run_id = str(randint(10000,99999)) if run_id is None else run_id
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
68
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
69 # Overrided ?
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
70 __run_id = __get_override_env('run_id', __run_id)
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
71
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
72 # ---- run_dir
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
73 #
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
74 # If None, we set it
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
75 __run_dir = f'./run/{__run_id}' if run_dir is None else run_dir
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
76
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
77 # Override run_dir ?
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
78 __run_dir = __get_override_env('run_dir', __run_dir)
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
79
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
80 # Create run_dir
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
81 utils.mkdir(__run_dir)
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
82
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
83 # ---- Parameters from config.py
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
84 #
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
85 mplstyle = config.FIDLE_MPLSTYLE
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
86 cssfile = config.FIDLE_CSSFILE
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
87
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
88 # ---- Load matplotlib style and css
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
89 #
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
90 module_dir = pathlib.Path(__file__).parent
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
91 matplotlib.style.use(f'{module_dir}/{mplstyle}')
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
92 __load_cssfile(f'{module_dir}/{cssfile}')
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
93
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
94 # ---- Get datasets location
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
95 # From env var or by looking for
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
96 # Raise an exception if cannot be found.
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
97 #
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
98 __datasets_dir=utils.get_datasets_dir(datasets_env_var)
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
99
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
100 # ---- Update Keras cache
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
101 #
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
102 updated_keras = __update_keras_cache()
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
103
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
104 # ---- Update torch cache
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
105 #
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
106 updated_torch = __update_torch_cache()
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
107
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
108 # ---- Tensorflow log level
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
109 #
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
110 log_level = int(os.getenv('TF_CPP_MIN_LOG_LEVEL', 0 ))
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
111 str_level = ['Info + Warning + Error','Warning + Error','Error only'][log_level]
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
112
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
113 # ---- Today, now and hostname
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
114 #
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
115 chrono.start('__global__')
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
116 h = platform.uname()
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
117
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
118 # ---- Save figs
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
119 #
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
120 save_figs = os.getenv('FIDLE_SAVE_FIGS', str(config.SAVE_FIGS) )
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
121 save_figs = (save_figs.lower() == 'true')
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
122 figs_dir = f'{__run_dir}/figs'
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
123
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
124 plot.set_save_fig( save = save_figs,
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
125 figs_dir = figs_dir,
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
126 figs_name = f'fig_{__run_id}_',
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
127 figs_id = 0 )
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
128
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
129 # ---- Hello world
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
130 #
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
131 utils.display_md('<br>**FIDLE - Environment initialization**')
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
132 print('Version :', config.VERSION)
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
133 print('Run id :', __run_id)
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
134 print('Run dir :', __run_dir)
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
135 print('Datasets dir :', __datasets_dir)
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
136 print('Start time :', chrono.get_start('__global__'))
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
137 print('Hostname :', f'{h[1]} ({h[0]})')
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
138 print('Tensorflow log level :', str_level,f' (={log_level})')
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
139 print('Update keras cache :', updated_keras)
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
140 print('Update torch cache :', updated_torch)
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
141 print('Save figs :', figs_dir, f'({save_figs})')
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
142
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
143 # ---- Modules versions
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
144 #
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
145 for m in config.USED_MODULES:
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
146 if m in sys.modules:
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
147 print(f'{m:21s}:', sys.modules[m].__version__)
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
148
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
149 # ---- Overrided ?
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
150 #
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
151 if run_id != __run_id:
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
152 print(f'\n** run_id has been overrided from {run_id} to {__run_id}')
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
153
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
154
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
155 return __run_id, __run_dir, __datasets_dir
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
156
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
157 # ------------------------------------------------------------------
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
158 # Update keras cache
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
159 # ------------------------------------------------------------------
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
160 # Try to sync ~/.keras/cache with datasets/keras_cache
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
161 # because sometime, we cannot access to internet... (IDRIS..)
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
162 #
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
163 def __update_keras_cache():
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
164 updated = False
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
165 if os.path.isdir(f'{__datasets_dir}/keras_cache'):
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
166 from_dir = f'{__datasets_dir}/keras_cache/*.*'
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
167 to_dir = os.path.expanduser('~/.keras/datasets')
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
168 utils.mkdir(to_dir)
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
169 for pathname in glob.glob(from_dir):
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
170 filename=os.path.basename(pathname)
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
171 destname=f'{to_dir}/{filename}'
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
172 if not os.path.isfile(destname):
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
173 shutil.copy(pathname, destname)
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
174 updated=True
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
175 return updated
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
176
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
177
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
178 # ------------------------------------------------------------------
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
179 # Update torch cache
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
180 # ------------------------------------------------------------------
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
181 # Try to sync ~/data/MNIST/raw with datasets/torch_cache
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
182 # because sometime, we cannot access to internet... (IDRIS..)
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
183 #
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
184
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
185 def __update_torch_cache():
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
186 updated = False
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
187 if os.path.isdir(f'{__datasets_dir}/torch_cache/MNIST/raw'):
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
188 from_dir = f'{__datasets_dir}/torch_cache/MNIST/raw/*'
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
189 to_dir = os.getcwd() + '/data/MNIST/raw'
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
190 utils.mkdir(to_dir)
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
191 for pathname in glob.glob(from_dir):
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
192 filename=os.path.basename(pathname)
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
193 destname=f'{to_dir}/{filename}'
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
194 if not os.path.isfile(destname):
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
195 shutil.copy(pathname, destname)
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
196 updated=True
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
197 return updated
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
198
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
199
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
200 # ------------------------------------------------------------------
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
201 # Override
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
202 # ------------------------------------------------------------------
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
203 #
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
204
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
205 def override(*names, module_name='__main__', verbose=True, return_attributes=False):
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
206 '''
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
207 Try to override attributes given par name with environment variables.
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
208 Environment variables name must be : FIDLE_OVERRIDE_<NAME>
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
209 If no env variable is available for a given name, nothing is change.
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
210 If type is str, substitution is done with 'notebook_id' and 'datasets_dir'
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
211 Example : override('image_size','nb_epochs')
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
212 params:
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
213 names : list of attributes names as a str list
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
214 if empty, all attributes can be override
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
215 return :
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
216 dict {name=new value}
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
217 '''
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
218 # ---- Where to override
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
219 #
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
220 module=sys.modules[module_name]
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
221
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
222 # ---- No names : mean all
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
223 #
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
224 if len(names)==0:
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
225 names=[]
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
226 for name in dir(module):
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
227 if name.startswith('_'): continue
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
228 v=getattr(module,name)
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
229 if type(v) not in [str, int, float, bool, tuple, list, dict]: continue
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
230 names.append(name)
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
231
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
232 # ---- Search for names
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
233 #
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
234 overrides={}
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
235 for name in names:
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
236
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
237 # ---- Variable doesn't exist
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
238 #
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
239 if not hasattr(module,name):
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
240 print(f'** Warning : You try to override an inexistant variable ({name})')
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
241 continue
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
242
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
243 # ---- Get override environment variable name
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
244 #
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
245 env_value = __get_override_env(name, None)
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
246
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
247 # ---- Environment variable doesn't exist
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
248 #
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
249 if env_value is None: continue
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
250
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
251 # ---- Environment variable and variable exist both
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
252 #
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
253 value_old = getattr(module,name)
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
254 value_type = type(value_old)
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
255
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
256 if value_type in [ str ] :
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
257 new_value = env_value.format(datasets_dir=__datasets_dir, run_id=__run_id)
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
258
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
259 if value_type in [ int, float, bool, tuple, list, dict, type(None)]:
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
260 new_value = eval(env_value)
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
261
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
262 # ---- Override value
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
263 #
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
264 setattr(module,name,new_value)
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
265 overrides[name]=new_value
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
266
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
267 if verbose and len(overrides)>0:
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
268 print('** Overrided parameters : **')
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
269 for name,value in overrides.items():
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
270 print(f'{name:20s} : {value}')
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
271
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
272 if return_attributes:
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
273 return overrides
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
274
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
275
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
276 def __get_override_env(name, default_value=None):
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
277 env_name = f'FIDLE_OVERRIDE_{name}'
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
278 env_value = os.environ.get(env_name, default_value)
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
279 return env_value
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
280
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
281
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
282 # def __has_override_env(name):
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
283 # env_name = f'FIDLE_OVERRIDE_{name}'
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
284 # return (env_name in os.environ)
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
285
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
286
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
287
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
288 def __load_cssfile(cssfile):
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
289 if cssfile is None: return
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
290 styles = open(cssfile, "r").read()
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
291 display(HTML(styles))
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
292
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
293
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
294
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
295
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
296
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
297 def end():
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
298 chrono.stop('__global__')
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
299 end_time = chrono.get_end('__global__')
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
300 duration = chrono.get_delay('__global__', format='human')
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
301 site_url = "https://fidle.cnrs.fr"
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
302 md = f'**End time :** {end_time} \n'
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
303 md+= f'**Duration :** {duration} \n'
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
304 md+= f'This notebook ends here :-) \n'
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
305 md+= f'[{site_url}]({site_url})'
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
306 utils.display_md(md)
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
307
2f6c77865180 planemo upload for repository https://forgemia.inra.fr/nathalie.rousse/use/-/tree/dnn/DNN/galaxy-tools/wine_quality_train_eval commit 1906137478520a9f71a4d8996c0a8827f27aed16-dirty
siwaa
parents:
diff changeset
308