| Geant4 Cross Reference |
1 This repository contains the set of scripts us 1 This repository contains the set of scripts used to train, generate and validate the generative model used
2 in this example. 2 in this example.
3 3
4 - root2h5.py: translation of ROOT file with sh 4 - root2h5.py: translation of ROOT file with showers to h5 files.
5 - core/constants.py: defines the set of common 5 - core/constants.py: defines the set of common variables.
6 - core/model.py: defines the VAE model class a 6 - core/model.py: defines the VAE model class and a handler to construct the model.
7 - utils/preprocess.py: defines the data loadin 7 - utils/preprocess.py: defines the data loading and preprocessing functions.
8 - utils/hyperparameter_tuner.py: defines the H 8 - utils/hyperparameter_tuner.py: defines the HyperparameterTuner class.
9 - utils/gpu_limiter.py: defines a logic respon 9 - utils/gpu_limiter.py: defines a logic responsible for GPU memory management.
10 - utils/observables.py: defines a set of obser 10 - utils/observables.py: defines a set of observable possibly calculated from a shower.
11 - utils/plotter.py: defines plotting classes r 11 - utils/plotter.py: defines plotting classes responsible for manufacturing various plots of observables.
12 - train.py: performs model training. 12 - train.py: performs model training.
13 - generate.py: generate showers using a saved 13 - generate.py: generate showers using a saved VAE model.
14 - observables.py: defines a set of shower obse 14 - observables.py: defines a set of shower observables.
15 - validate.py: creates validation plots using 15 - validate.py: creates validation plots using shower observables.
16 - convert.py: defines the conversion function 16 - convert.py: defines the conversion function to an ONNX file.
17 - tune_model.py: performs hyperparameters opti 17 - tune_model.py: performs hyperparameters optimization.
18 18
19 ## Getting Started 19 ## Getting Started
20 20
21 `setup.py` script creates necessary folders us 21 `setup.py` script creates necessary folders used to save model checkpoints, generate showers and validation plots.
22 22
23 ``` 23 ```
24 python3 setup.py 24 python3 setup.py
25 ``` 25 ```
26 26
27 ## Full simulation dataset 27 ## Full simulation dataset
28 28
29 The full simulation dataset can be downloaded 29 The full simulation dataset can be downloaded from/linked to [Zenodo](https://zenodo.org/record/6082201#.Ypo5UeDRaL4).
30 30
31 If custom simulation is used, the output of fu 31 If custom simulation is used, the output of full simulation must be translated to h5 files using `root2h5.py` script. Please see the header of that script to see what name of the root file is expected.
32 32
33 ## Training 33 ## Training
34 34
35 In order to launch the training: 35 In order to launch the training:
36 36
37 ``` 37 ```
38 python3 train.py 38 python3 train.py
39 ``` 39 ```
40 40
41 You may specify those three following flags. I 41 You may specify those three following flags. If you do not, then default values will be used.
42 42
43 ```--max-gpu-memory-allocation``` specifies a 43 ```--max-gpu-memory-allocation``` specifies a maximum memory allocation on a single, logic GPU unit. Should be given as
44 an integer. 44 an integer.
45 45
46 ```--gpu-ids``` specifies IDs of physical GPUs 46 ```--gpu-ids``` specifies IDs of physical GPUs. Should be given as a string, separated with comas, no spaces.
47 If you specify more than one GPU then automati 47 If you specify more than one GPU then automatically ```tf.distribute.MirroredStrategy``` will be applied to the
48 training. 48 training.
49 49
50 ```--study-name``` specifies a study name. Thi 50 ```--study-name``` specifies a study name. This name is used as an experiment name in W&B dashboard and as a name of
51 directory for saving models. 51 directory for saving models.
52 52
53 ## Hyperparameters tuning 53 ## Hyperparameters tuning
54 54
55 If you want to tune hyperparameters, specify i 55 If you want to tune hyperparameters, specify in `tune_model.py` parameters to be tuned. There are three types of
56 parameters: discrete, continuous and categoric 56 parameters: discrete, continuous and categorical. Discrete and continuous require range specification (low, high), while
57 the categorical parameter requires a list of p 57 the categorical parameter requires a list of possible values to be chosen. Then run it with:
58 58
59 ``` 59 ```
60 python3 tune_model.py 60 python3 tune_model.py
61 ``` 61 ```
62 62
63 If you want to parallelize tuning process you 63 If you want to parallelize tuning process you need to specify a common storage (preferable MySQL database) by
64 setting `--storage="URL_TO_MYSQL_DATABASE"`. T 64 setting `--storage="URL_TO_MYSQL_DATABASE"`. Then you can run multiple processes with the same command:
65 65
66 ``` 66 ```
67 python3 tune_model.py --storage="URL_TO_MYSQL_ 67 python3 tune_model.py --storage="URL_TO_MYSQL_DATABASE"
68 ``` 68 ```
69 69
70 Similarly to training procedure, you may speci 70 Similarly to training procedure, you may specify ```--max-gpu-memory-allocation```, ```--gpu-ids``` and
71 ```--study-name```. 71 ```--study-name```.
72 72
73 ## ML shower generation (MLFastSim) 73 ## ML shower generation (MLFastSim)
74 74
75 In order to generate showers using the ML mode 75 In order to generate showers using the ML model, use `generate.py` script and specify information of geometry, energy
76 and angle of the particle and the epoch of the 76 and angle of the particle and the epoch of the saved checkpoint model. The number of events to generate can also be
77 specified (by default is set to 10.000): 77 specified (by default is set to 10.000):
78 78
79 ``` 79 ```
80 python3 generate.py --geometry=SiW --energy=64 80 python3 generate.py --geometry=SiW --energy=64 --angle=90 --epoch=1000 --study-name=YOUR_STUDY_NAME
81 ``` 81 ```
82 82
83 If you do not specify an epoch number the base 83 If you do not specify an epoch number the based model (saved as ```VAEbest```) will be used for shower generation.
84 84
85 ## Validation 85 ## Validation
86 86
87 In order to validate the MLFastSim and the ful 87 In order to validate the MLFastSim and the full simulation, use `validate.py` script and specify information of
88 geometry, energy and angle of the particle: 88 geometry, energy and angle of the particle:
89 89
90 ``` 90 ```
91 python3 validate.py --geometry=SiW --energye=6 91 python3 validate.py --geometry=SiW --energye=64 --angle=90
92 ``` 92 ```
93 93
94 ## Conversion 94 ## Conversion
95 95
96 After training and validation, the model can b 96 After training and validation, the model can be converted into a format that can be used in C++, such as ONNX,
97 use `convert.py` script: 97 use `convert.py` script:
98 98
99 ``` 99 ```
100 python3 convert.py --epoch 1000 100 python3 convert.py --epoch 1000
101 ``` 101 ```