Miriam Schirru, Florence Véronneau-Veilleux, Fahima Nekka and Mauro Ursino
Feb. 2022
The same descriptions will be applicable both for 2 actions
and 4 actions, with appropriate adaptations. To avoid repetition, we will just
present the case with 2 actions.
Name: ‘Basal_Training_synapses_2_channels.m’
Description: Script to perform the first part of learning (basal learning). The program is used to train synapses of the two channels.
10 independent learning procedures (each with 200 epochs starting from the naïve network and different noise realizations, with different seeds of noise) are performed.
Each epoch consists of the two stimuli, S1 = [1 0.3] and S2 = [0.3 1].
In response to the first stimulus, S1, the first action is randomly rewarded with a probability P_high as high as 0.8 (otherwise the action is punished), while the second action is randomly rewarded with a probability P_small = 0.3 as low as 0.3 (otherwise punished). When a second stimulus S2 is given the reward probabilities are P_small for the first choice and P_high for the second one.
The user chooses the Hebb rule by entering the corresponding value. Once the user performed the choice, during training the program calls the corresponding function which simulates the dynamics of the basal ganglia.
As regards ‘BG_model_function_Ach_post_adapt.m’, the parameter m is indicated in line 185 of the present function.
To perform the sensitivity analysis, you can manually change the value of the dopaminergic input D, in line 94.
The results are saved in a .mat file which indicates:
1. The name of the rule used;
2. The number of the learning procedure, with ii procedure index.
(ex: Post Post Rule , ii = 1 'W_tot_post_post_1.mat')
The results can be shown by running the program ‘plot_training_synapses_2_ch.m’.
Name: ‘Reversal_Training_synapses_2_channels.m’.
Description: Script to perform the reversal learning. The program is used to train synapses of the two channels.
To be run after ‘Basal_Training_synapses_2_channels.m’, since the program loads simulations result obtained from ‘Basal_Training_synapses_2_channels.m’.
Each epoch consists of the two stimuli, S1 = [1 0.3] and S2 = [0.3 1].
Reversal consists in the same inputs S1 and S2, randomly permuted at each epoch, but with the opposite associated reward probabilities: action 1 is associated with P_small reward probability after the presentation of the stimulus S1 and with P_high reward probability after S2; action 2 is associated with P_high and P_small reward probabilities after S1 and S2, respectively.
Reversal learning was attempted at different moments: after 50 epochs and after 75 epochs of previous learning. The epoch has to be indicated in line 100.
The user chooses the Hebb rule by entering the corresponding value.
The user can manually change the value of the phasic dopamine, in line 108.
The results are saved in a .mat file which indicates:
1. The type of learning (Rev = Reversal Learning);
2. The name of the rule used;
3. The number of the learning procedure, with ii procedure index.
(ex: Reversal Learning, Post Post Rule , ii = 1 'Rev_W_tot_post_post_1.mat')
The results can be shown by running the program ‘plot_training_synapses_2_ch.m’.
Name: ‘Task_10trial_2actions_Matrice.m’
Description: Script to perform the testing procedure, two-choice experiment. To
be run after ‘Basal_Training_synapses_2_channels.m’ or ‘Reversal_Training_synapses_2_channels.m’.
The program calls the function ('BG_model_Response_Stimuli_Task1.m') used to simulate the task.
100 trials are executed, by using the inputs S1 and S2 50 times each. During the test, random Gaussian noise was applied not only to cortical neurons but also to the input stimuli (simulating a noisy environment), see lines 181-188.
This testing procedure has been performed both for basal and reversal procedures, starting from the network obtained after different epochs of learning indicated by vectors in lines 46-47.
The results are shown at the end of the simulation in a table organized as follow:
|
Epoch |
Mean 1st Action |
Std 1st Action |
Mean 2nd Action |
Std 2nd Action |
|
1 |
51.1 |
0.31623 |
47 |
0.4714 |
|
25 |
57.9 |
2.846 |
41.4 |
2.7162 |
|
50 |
66.5 |
3.6286 |
32.5 |
3.6286 |
|
75 |
74.2 |
3.4254 |
24.3 |
3.653 |
|
100 |
77.3 |
3.401 |
17.2 |
3.7947 |
|
150 |
83.5 |
1.7795 |
8.2 |
1.6193 |
|
200 |
84.9 |
1.4491 |
7.6 |
0.96609 |
Description: The script performs the plot of all synapses subject to training concerning one learning procedure. To be run after ‘Basal_Training_synapses_2_channels.m’ or ‘Reversal_Training_synapses_2_channels.m’.
The script loads simulations results that come from .mat files saved during training. The user can choose the simulation in line 23.