Detecting Muon Momentum in the CMS Experiment at CERN using Deep Learning
Posted on Thu 19 November 2020 in posts • 67 min read
In the following notebook, we will apply several Deep Learning approaches to properly predict muon momentum. We will use Monte-Carlo simulated data from the Cathode Strip Chambers (CSC) at the CMS experiment. These chambers detect muon particles in the outer layer of the CMS detector, allowing us to store information about the hit locations.
The dataset contains more than 3 million muon events generated using Pythia.
Importing Libraries¶
Let's import the libraries that we will need in this project:
First, let's import the libraries that we will need for this project:
- Numpy - for vectors manipulation and operations
- Scikit-Learn - for building and training predictive models
- Matplotlib and Seaborn - for visualizing various types of plots
- Pandas - for data cleaning and analysis
In [1]:
import numpy as np
import sklearn
import matplotlib.pyplot as plt
import seaborn as sns
import pandas as pd
np.random.seed(48) # set the random seed for reproducible results
For training Neural Networks, we will be using Tensorflow 2 which is a famous Deep Learning library used for developing, training and deploying deep learning models.
Let's import Tensorflow and check its version:
In [2]:
import tensorflow as tf
print(tf.__version__)
2.3.0
Retrieving the Dataset¶
It seems our data is in NPZ format, therefore will use numpy.load() to load the data files:
In [5]:
data=np.load("histos_tba.npz")
The data file contain two arrays: variables and parameters.
In [6]:
data["variables"].shape,data["parameters"].shape
Out[6]:
((3272341, 87), (3272341, 3))
We will extract the data from the variabes array for the muon hits detected at the CSC chambers only:
In [7]:
def delete_not_csc(data):
muon_hits=data['variables']
indices_to_del=[]
for i in range(muon_hits.shape[1]):
if(i%12>=5 and i<84):
indices_to_del.append(i)
return np.delete(muon_hits,indices_to_del,axis=1)
muon_hits=delete_not_csc(data)
muon_hits.shape
Out[7]:
(3272341, 38)
Next, let's prepare the columns names for each feature and combine the two datasets into a Pandas DataFrame for easier analysis:
In [8]:
original_columns_names=["Phi angle","Theta angle","Bend angle","Time", "Ring",
"Front/Rear","Mask"]
columns_names=[]
for element in enumerate(original_columns_names):
for i in range(5):
columns_names.append(str(element[1])+str(i))
columns_names.append("XRoad0")
columns_names.append("XRoad1")
columns_names.append("XRoad2")
muon_hits_df=pd.DataFrame(muon_hits,columns=columns_names)
muon_hits_df["q/pt"]=data["parameters"][:,0]
muon_hits_df["Phi_angle"]=data["parameters"][:,1]
muon_hits_df["Eta_angle"]=data["parameters"][:,2]
Exploratory Data Analysis (EDA)¶
Let's start our EDA by taking a look at the columns types:
In [9]:
muon_hits_df.info()
<class 'pandas.core.frame.DataFrame'> RangeIndex: 3272341 entries, 0 to 3272340 Data columns (total 41 columns): # Column Dtype --- ------ ----- 0 Phi angle0 float32 1 Phi angle1 float32 2 Phi angle2 float32 3 Phi angle3 float32 4 Phi angle4 float32 5 Theta angle0 float32 6 Theta angle1 float32 7 Theta angle2 float32 8 Theta angle3 float32 9 Theta angle4 float32 10 Bend angle0 float32 11 Bend angle1 float32 12 Bend angle2 float32 13 Bend angle3 float32 14 Bend angle4 float32 15 Time0 float32 16 Time1 float32 17 Time2 float32 18 Time3 float32 19 Time4 float32 20 Ring0 float32 21 Ring1 float32 22 Ring2 float32 23 Ring3 float32 24 Ring4 float32 25 Front/Rear0 float32 26 Front/Rear1 float32 27 Front/Rear2 float32 28 Front/Rear3 float32 29 Front/Rear4 float32 30 Mask0 float32 31 Mask1 float32 32 Mask2 float32 33 Mask3 float32 34 Mask4 float32 35 XRoad0 float32 36 XRoad1 float32 37 XRoad2 float32 38 q/pt float32 39 Phi_angle float32 40 Eta_angle float32 dtypes: float32(41) memory usage: 511.8 MB
We notice that all our variables have float data type, which we will prove helpful later on when preprocessing the data.
Next, let's generate some useful statistics about our features:
In [10]:
muon_hits_df.describe()
Out[10]:
| Phi angle0 | Phi angle1 | Phi angle2 | Phi angle3 | Phi angle4 | Theta angle0 | Theta angle1 | Theta angle2 | Theta angle3 | Theta angle4 | Bend angle0 | Bend angle1 | Bend angle2 | Bend angle3 | Bend angle4 | Time0 | Time1 | Time2 | Time3 | Time4 | Ring0 | Ring1 | Ring2 | Ring3 | Ring4 | Front/Rear0 | Front/Rear1 | Front/Rear2 | Front/Rear3 | Front/Rear4 | Mask0 | Mask1 | Mask2 | Mask3 | Mask4 | XRoad0 | XRoad1 | XRoad2 | q/pt | Phi_angle | Eta_angle | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| count | 2.300975e+06 | 870666.000000 | 2.789617e+06 | 2.605933e+06 | 2.439864e+06 | 2.300975e+06 | 870666.000000 | 2.789617e+06 | 2.605933e+06 | 2.439864e+06 | 2.300975e+06 | 870666.000000 | 2.789617e+06 | 2.605933e+06 | 2.439864e+06 | 2.300975e+06 | 870666.000000 | 2.789617e+06 | 2.605933e+06 | 2.439864e+06 | 2.300975e+06 | 870666.0 | 2.789617e+06 | 2.605933e+06 | 2.439864e+06 | 2.300975e+06 | 870666.000000 | 2.789617e+06 | 2.605933e+06 | 2.439864e+06 | 3.272341e+06 | 3.272341e+06 | 3.272341e+06 | 3.272341e+06 | 3.272341e+06 | 3.272341e+06 | 3.272341e+06 | 3.272341e+06 | 3.272341e+06 | 3.272341e+06 | 3.272341e+06 |
| mean | 2.898916e+03 | 2942.738281 | 2.884543e+03 | 2.895544e+03 | 2.901002e+03 | 2.513442e+01 | 62.888813 | 3.345796e+01 | 3.376106e+01 | 3.479657e+01 | 1.771088e-02 | -0.075262 | 2.404990e-03 | 7.820615e-04 | 1.634517e-03 | -1.554471e-02 | -0.029150 | -2.238408e-02 | -2.388741e-02 | -1.674315e-02 | 2.208553e+00 | 2.0 | 1.234539e+00 | 1.328101e+00 | 1.416067e+00 | 4.993205e-01 | 0.465420 | 5.033727e-01 | 4.964771e-01 | 5.009226e-01 | 2.968413e-01 | 7.339318e-01 | 1.475164e-01 | 2.036487e-01 | 2.543980e-01 | 3.977467e+00 | 2.312063e+00 | 9.032854e+01 | -1.427713e-04 | -4.566138e-04 | -1.287372e-03 |
| std | 1.063501e+03 | 1061.839233 | 1.087773e+03 | 1.087355e+03 | 1.084414e+03 | 1.279694e+01 | 10.498766 | 2.147677e+01 | 2.178618e+01 | 2.156608e+01 | 1.678599e+01 | 19.846676 | 3.044079e-01 | 3.709632e-01 | 3.650869e-01 | 1.228342e-01 | 0.167572 | 1.469481e-01 | 1.515253e-01 | 1.274973e-01 | 1.475620e+00 | 0.0 | 4.290860e-01 | 4.634236e-01 | 4.966435e-01 | 5.000001e-01 | 0.499028 | 4.999973e-01 | 4.999975e-01 | 5.000001e-01 | 4.557488e-01 | 4.387615e-01 | 3.566231e-01 | 3.976815e-01 | 4.353520e-01 | 1.956127e+00 | 1.881727e+00 | 3.388365e+01 | 2.625431e-01 | 1.811694e+00 | 1.889245e+00 |
| min | 6.620000e+02 | 690.000000 | 9.600000e+01 | 9.400000e+01 | 9.600000e+01 | 5.000000e+00 | 46.000000 | 5.000000e+00 | 5.000000e+00 | 6.000000e+00 | -1.329103e+02 | -81.000000 | -1.000000e+00 | -1.000000e+00 | -1.000000e+00 | -1.000000e+00 | -1.000000 | -1.000000e+00 | -1.000000e+00 | -1.000000e+00 | 1.000000e+00 | 2.0 | 1.000000e+00 | 1.000000e+00 | 1.000000e+00 | 0.000000e+00 | 0.000000 | 0.000000e+00 | 0.000000e+00 | 0.000000e+00 | 0.000000e+00 | 0.000000e+00 | 0.000000e+00 | 0.000000e+00 | 0.000000e+00 | 0.000000e+00 | 0.000000e+00 | 2.000000e+00 | -4.999985e-01 | -3.141591e+00 | -2.499856e+00 |
| 25% | 2.005000e+03 | 2032.000000 | 1.980000e+03 | 1.998000e+03 | 2.000000e+03 | 1.400000e+01 | 54.000000 | 1.600000e+01 | 1.600000e+01 | 1.600000e+01 | -1.200000e+01 | -14.000000 | 0.000000e+00 | 0.000000e+00 | 0.000000e+00 | 0.000000e+00 | 0.000000 | 0.000000e+00 | 0.000000e+00 | 0.000000e+00 | 1.000000e+00 | 2.0 | 1.000000e+00 | 1.000000e+00 | 1.000000e+00 | 0.000000e+00 | 0.000000 | 0.000000e+00 | 0.000000e+00 | 0.000000e+00 | 0.000000e+00 | 0.000000e+00 | 0.000000e+00 | 0.000000e+00 | 0.000000e+00 | 3.000000e+00 | 1.000000e+00 | 6.200000e+01 | -2.194096e-01 | -1.571401e+00 | -1.880253e+00 |
| 50% | 2.891000e+03 | 2937.000000 | 2.882000e+03 | 2.896000e+03 | 2.904000e+03 | 2.300000e+01 | 62.000000 | 2.800000e+01 | 2.800000e+01 | 2.900000e+01 | 0.000000e+00 | 0.000000 | 0.000000e+00 | 0.000000e+00 | 0.000000e+00 | 0.000000e+00 | 0.000000 | 0.000000e+00 | 0.000000e+00 | 0.000000e+00 | 1.000000e+00 | 2.0 | 1.000000e+00 | 1.000000e+00 | 1.000000e+00 | 0.000000e+00 | 0.000000 | 1.000000e+00 | 0.000000e+00 | 1.000000e+00 | 0.000000e+00 | 1.000000e+00 | 0.000000e+00 | 0.000000e+00 | 0.000000e+00 | 4.000000e+00 | 2.000000e+00 | 9.000000e+01 | -4.459424e-04 | 7.330791e-04 | -1.202088e+00 |
| 75% | 3.792000e+03 | 3853.000000 | 3.791000e+03 | 3.795000e+03 | 3.800000e+03 | 3.500000e+01 | 71.000000 | 4.700000e+01 | 5.200000e+01 | 5.200000e+01 | 1.200000e+01 | 14.000000 | 0.000000e+00 | 0.000000e+00 | 0.000000e+00 | 0.000000e+00 | 0.000000 | 0.000000e+00 | 0.000000e+00 | 0.000000e+00 | 4.000000e+00 | 2.0 | 1.000000e+00 | 2.000000e+00 | 2.000000e+00 | 1.000000e+00 | 1.000000 | 1.000000e+00 | 1.000000e+00 | 1.000000e+00 | 1.000000e+00 | 1.000000e+00 | 0.000000e+00 | 0.000000e+00 | 1.000000e+00 | 5.000000e+00 | 4.000000e+00 | 1.190000e+02 | 2.191571e-01 | 1.569422e+00 | 1.879843e+00 |
| max | 4.987000e+03 | 4950.000000 | 4.954000e+03 | 4.952000e+03 | 4.952000e+03 | 5.300000e+01 | 88.000000 | 8.800000e+01 | 8.800000e+01 | 8.800000e+01 | 1.421401e+02 | 70.000000 | 1.000000e+00 | 1.000000e+00 | 1.000000e+00 | 0.000000e+00 | 0.000000 | 0.000000e+00 | 0.000000e+00 | 0.000000e+00 | 4.000000e+00 | 2.0 | 2.000000e+00 | 2.000000e+00 | 2.000000e+00 | 1.000000e+00 | 1.000000 | 1.000000e+00 | 1.000000e+00 | 1.000000e+00 | 1.000000e+00 | 1.000000e+00 | 1.000000e+00 | 1.000000e+00 | 1.000000e+00 | 8.000000e+00 | 5.000000e+00 | 1.540000e+02 | 4.999996e-01 | 3.141592e+00 | 2.499996e+00 |
We notice that some features such as Phi angle1, Theta angle1 and Bend angle1 have some missing values.
Let's find the exact percentage of null values for each feature:
In [11]:
null_perc=((muon_hits_df.isnull().
sum()/muon_hits_df.
shape[0]
)*100).sort_values(ascending=False)
fig = plt.figure(figsize = (10,5))
ax = fig.gca()
ax.set_xlabel("features names")
ax.set_ylabel("missing values percentage")
null_perc.plot.bar(ax=ax)
plt.show()
We notice that Front/Rear1, Phi angle1, Theta angle1, Bend angle1, Time1 and Ring1 features have more than 70% missing values!
Next, let's take a look on the data distribution of each feature:
In [12]:
fig = plt.figure(figsize = (30,40))
ax = fig.gca()
muon_hits_df.hist(ax=ax)
plt.show()
/usr/local/lib/python3.6/dist-packages/ipykernel_launcher.py:3: UserWarning: To output multiple subplots, the figure containing the passed axes is being cleared This is separate from the ipykernel package so we can avoid doing imports until
Wow, those are a LOT of plots! It is clear from these plots that some features such as Theta Angle1,Theta Angle2, Theta Angle3 and XRoad1 can benefit from being transformed into Normal Distribution (Standardization).
Next, let's investigate the effect of each feature on the target feature q/pt:
In [13]:
corr=muon_hits_df.corr()
f, ax = plt.subplots(figsize=(11, 9))
cmap = sns.diverging_palette(230, 20, as_cmap=True)
mask = np.triu(np.ones_like(corr, dtype=bool))
sns.heatmap(corr, mask=mask, cmap=cmap, vmax=.3, center=0,
square=True, linewidths=.5, cbar_kws={"shrink": .5})
Out[13]:
<matplotlib.axes._subplots.AxesSubplot at 0x7f6a2cb0de48>
Framing the Problem as a Classification Task¶
Instead of trying to apply regression to predict the muon momenta values (q/pt), let's try to cluster the momenta into 4 classes: 0-10 GeV, 10-30 GeV, 30-100 GeV and >100 GeV, and instead frame the problem as a classification task.
Let's investigate the q/pt column and separate it from the data:
In [14]:
pt_inverse_labels=muon_hits_df["q/pt"]
muon_hits_df_x=muon_hits_df.drop("q/pt",axis=1)
muon_hits_df["q/pt"].describe()
Out[14]:
count 3.272341e+06 mean -1.427713e-04 std 2.625431e-01 min -4.999985e-01 25% -2.194096e-01 50% -4.459424e-04 75% 2.191571e-01 max 4.999996e-01 Name: q/pt, dtype: float64
We will use the absolute value of the reciprocal of the target feature (pt/q) to group the momenta into the previously mentioned classes:
In [15]:
pt_labels=abs((1/pt_inverse_labels))
pt_labels.describe()
Out[15]:
count 3.272341e+06 mean 1.859705e+01 std 1.257423e+02 min 2.000001e+00 25% 2.998728e+00 50% 4.560210e+00 75% 9.115835e+00 max 6.989345e+03 Name: q/pt, dtype: float64
Let's visualize the data distribution of this feature:
In [16]:
pt_labels.hist(bins=1000)
plt.xlim([0,150])
plt.title("Data Distribution of p_T labels")
plt.show()
That is a very interesting plot! We can clearly see that there are distinct groups of points of different magnitudes.
We can now group the values into 4 groups: 0-10 GeV, 10-30 GeV, 30-100 GeV and >100 GeV using the cut method from Pandas.
In [17]:
bins = pd.IntervalIndex.from_tuples([(0, 10), (10, 30), (30, 100),(100,pt_labels.max()+1)])
org_pt_labels_groups=pd.cut(pt_labels, bins)
Let's plot the grouped data now:
In [18]:
target_counts=org_pt_labels_groups.value_counts()
target_counts.plot.bar()
plt.title("Distribution of classes in target feature")
plt.show()
It is very clear that the classes are not balanced, therefore it is important to balance those classes whenever we start training our Neural Network to avoid any bias.
Splitting the Data¶
Next, let's seperate the data into train and test data, we will use a 90%-10% split.
As we already noticed, the data is highly imbalanced, therefore it is important to perform the following two tasks:
- Add class weights to balance out the classes
- Make the test set representative of the classes
To split the train and test data, we will use the StratifiedShuffleSplit method in Scikit-Learn to make both the train data and test data representative of the target classes distribution:
In [19]:
from sklearn.model_selection import StratifiedShuffleSplit
split = StratifiedShuffleSplit(n_splits=1, test_size=0.1, random_state=42)
for train_index, test_index in split.split(muon_hits_df_x, org_pt_labels_groups):
muon_hits_x_train_set = muon_hits_df_x.loc[train_index]
pt_labels_groups_y_train_set=org_pt_labels_groups[train_index]
muon_hits_x_test_set = muon_hits_df_x.loc[test_index]
pt_labels_groups_y_test_set=org_pt_labels_groups[test_index]
Let's verify if the data has the same proportions for the target label in the train and test sets:
In [20]:
def target_proportions(data):
return data.value_counts()/len(data)*100
proportions_df=pd.DataFrame({"Original": target_proportions(org_pt_labels_groups),
"Train":target_proportions(pt_labels_groups_y_train_set),
"Test":target_proportions(pt_labels_groups_y_test_set)},
)
proportions_df
Out[20]:
| Original | Train | Test | |
|---|---|---|---|
| (0.0, 10.0] | 77.201062 | 77.201058 | 77.201094 |
| (10.0, 30.0] | 15.207951 | 15.207942 | 15.208031 |
| (30.0, 100.0] | 5.331871 | 5.331862 | 5.331948 |
| (100.0, 6990.3447265625] | 2.259117 | 2.259138 | 2.258927 |
Neat! Now that we are sure that the train and test data are representative of the original dataset, we can now proceed to the preprocessing phase.
Preparing data for training¶
As we noticed earlier, we need delete the columns with percentage of missing values bigger than 70%:
In [21]:
def delete_columns(df,perc):
null_perc=(df.isnull().sum()/df.shape[0])*100
col_to_del= [col for col in df.columns if(((null_perc)>perc)[col])]
print("columns deleted:", col_to_del)
return df.drop(col_to_del,axis=1)
muon_hits_x_train_set_1=delete_columns(muon_hits_x_train_set,70)
columns deleted: ['Phi angle1', 'Theta angle1', 'Bend angle1', 'Time1', 'Ring1', 'Front/Rear1']
In [22]:
muon_hits_x_train_set_1.shape
Out[22]:
(2945106, 34)
For all other features with percentage of null values less than 70%, we will replace the null values with the mean value of their corresponding columns:
In [23]:
def replace_missing_with_mean(df):
from sklearn.impute import SimpleImputer
imputer = SimpleImputer(missing_values=np.nan, strategy='mean')
imputed=imputer.fit_transform(df)
return pd.DataFrame(imputed,columns=df.columns)
In [24]:
muon_hits_x_train_set_2=replace_missing_with_mean(muon_hits_x_train_set_1)
In [25]:
null_perc_2=(muon_hits_x_train_set_2.isnull().
sum()/muon_hits_x_train_set_2.
shape[0])*100
null_perc_2.sort_values(ascending=False)
Out[25]:
Eta_angle 0.0 Bend angle0 0.0 Time3 0.0 Time2 0.0 Time0 0.0 Bend angle4 0.0 Bend angle3 0.0 Bend angle2 0.0 Theta angle4 0.0 Phi_angle 0.0 Theta angle3 0.0 Theta angle2 0.0 Theta angle0 0.0 Phi angle4 0.0 Phi angle3 0.0 Phi angle2 0.0 Time4 0.0 Ring0 0.0 Ring2 0.0 Ring3 0.0 Ring4 0.0 Front/Rear0 0.0 Front/Rear2 0.0 Front/Rear3 0.0 Front/Rear4 0.0 Mask0 0.0 Mask1 0.0 Mask2 0.0 Mask3 0.0 Mask4 0.0 XRoad0 0.0 XRoad1 0.0 XRoad2 0.0 Phi angle0 0.0 dtype: float64
Now we are sure that we filled all missing values.
Let's next create the preprocessing pipeline and convert the target feature into one hot encoded variables.
In [26]:
def preprocess_pipeline(df_x,df_y):
#delete columns with missing values more than 70%
df_x=delete_columns(df_x,70)
#impute the other columns with missing values with their corresponding means
df_x=replace_missing_with_mean(df_x)
#standardize the data
from sklearn.preprocessing import StandardScaler
scaler=StandardScaler()
x=scaler.fit_transform(df_x)
#convert labels into dummy variables (one hot encoded)
from sklearn.preprocessing import LabelEncoder
encoder = LabelEncoder()
encoded_y = encoder.fit_transform(df_y)
print(encoder.classes_)
from sklearn.preprocessing import OneHotEncoder
ohe=OneHotEncoder(sparse=False)
ohe_y=ohe.fit_transform(encoded_y.reshape(-1,1))
print(ohe.categories_)
return x,ohe_y,encoded_y
In [27]:
muon_hits_x_train_values,pt_labels_ohe_y_train,y_encoded_train=preprocess_pipeline(muon_hits_x_train_set,pt_labels_groups_y_train_set)
columns deleted: ['Phi angle1', 'Theta angle1', 'Bend angle1', 'Time1', 'Ring1', 'Front/Rear1'] [Interval(0.0, 10.0, closed='right') Interval(10.0, 30.0, closed='right') Interval(30.0, 100.0, closed='right') Interval(100.0, 6990.3447265625, closed='right')] [array([0, 1, 2, 3])]
In [28]:
muon_hits_x_test_values,pt_labels_ohe_y_test,y_encoded_test=preprocess_pipeline(muon_hits_x_test_set,pt_labels_groups_y_test_set)
columns deleted: ['Phi angle1', 'Theta angle1', 'Bend angle1', 'Time1', 'Ring1', 'Front/Rear1'] [Interval(0.0, 10.0, closed='right') Interval(10.0, 30.0, closed='right') Interval(30.0, 100.0, closed='right') Interval(100.0, 6990.3447265625, closed='right')] [array([0, 1, 2, 3])]
Now our datasets are ready for training and testing!
One last task to do before training, which is generating the classes weights:
In [29]:
#option 1
from sklearn.utils import class_weight
class_weights = class_weight.compute_class_weight('balanced',
np.unique(y_encoded_train),
y_encoded_train)
In [30]:
class_weights
Out[30]:
array([ 0.32382976, 1.64387796, 4.68879315, 11.06616918])
In [31]:
#option 2
class_weights=len(y_encoded_train)/(4*(np.unique(y_encoded_train,return_counts=True)[1]))
classes_weights={i: class_weights[i] for i in range(len(class_weights))}
classes_weights
Out[31]:
{0: 0.3238297576631087,
1: 1.6438779611065217,
2: 4.688793152857116,
3: 11.066169176661557}
Building the Neural Network¶
We will use Tensorflow's Keras library to build the Neural Network
Building a Fully Connected Neural Network¶
Let's first build and compile the model:
In [ ]:
from tensorflow.keras import Sequential
from tensorflow.keras.layers import Dense,Dropout
model=Sequential()
model.add(Dense(512,input_dim=(muon_hits_x_train_values.shape[1]),activation='relu'))
model.add(Dense(256,activation='relu'))
model.add(Dropout(0.1))
model.add(Dense(128,activation='relu'))
model.add(Dense(64,activation='relu'))
model.add(Dropout(0.1))
model.add(Dense(32,activation='relu'))
model.add(Dropout(0.1))
model.add(Dense(4,activation='softmax'))
model.compile(optimizer='adam',loss='categorical_crossentropy',metrics=['accuracy'])
Let's take a closer look at the model architecture:
In [ ]:
model.summary()
Model: "sequential" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= dense (Dense) (None, 512) 17920 _________________________________________________________________ dense_1 (Dense) (None, 256) 131328 _________________________________________________________________ dropout (Dropout) (None, 256) 0 _________________________________________________________________ dense_2 (Dense) (None, 128) 32896 _________________________________________________________________ dense_3 (Dense) (None, 64) 8256 _________________________________________________________________ dropout_1 (Dropout) (None, 64) 0 _________________________________________________________________ dense_4 (Dense) (None, 32) 2080 _________________________________________________________________ dropout_2 (Dropout) (None, 32) 0 _________________________________________________________________ dense_5 (Dense) (None, 4) 132 ================================================================= Total params: 192,612 Trainable params: 192,612 Non-trainable params: 0 _________________________________________________________________
Next, we will set some callbacks for the model including a Checkpoint callback and Early Stopping callback that both monitor the model's validation accuracy:
In [ ]:
from tensorflow.keras.callbacks import ModelCheckpoint
from tensorflow.keras.callbacks import EarlyStopping
filepath="classifier_weights2-improvement-{epoch:02d}-{val_accuracy:.2f}.hdf5"
checkpoint1 = ModelCheckpoint(filepath, monitor='val_accuracy', verbose=1, save_best_only=True, mode='max')
checkpoint2= EarlyStopping( monitor='val_accuracy', patience=3)
callbacks_list = [checkpoint1,checkpoint1]
Now we can finally start training!
We will use a 25% validation split, batch size of 500 and will train for 100 epochs.
In [ ]:
history = model.fit(muon_hits_x_train_values, pt_labels_ohe_y_train,
validation_split=0.25,
epochs=200,
batch_size=500,
callbacks=callbacks_list,
class_weight=classes_weights)
Epoch 1/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.9673 - accuracy: 0.7593 Epoch 00001: val_accuracy improved from -inf to 0.76394, saving model to classifier_weights2-improvement-01-0.76.hdf5 Epoch 00001: val_accuracy did not improve from 0.76394 4418/4418 [==============================] - 91s 21ms/step - loss: 0.9673 - accuracy: 0.7593 - val_loss: 0.5584 - val_accuracy: 0.7639 Epoch 2/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.9354 - accuracy: 0.7702 Epoch 00002: val_accuracy improved from 0.76394 to 0.77115, saving model to classifier_weights2-improvement-02-0.77.hdf5 Epoch 00002: val_accuracy did not improve from 0.77115 4418/4418 [==============================] - 91s 21ms/step - loss: 0.9354 - accuracy: 0.7702 - val_loss: 0.5633 - val_accuracy: 0.7711 Epoch 3/200 4418/4418 [==============================] - ETA: 0s - loss: 0.9298 - accuracy: 0.7716 Epoch 00003: val_accuracy improved from 0.77115 to 0.77407, saving model to classifier_weights2-improvement-03-0.77.hdf5 Epoch 00003: val_accuracy did not improve from 0.77407 4418/4418 [==============================] - 95s 22ms/step - loss: 0.9298 - accuracy: 0.7716 - val_loss: 0.5344 - val_accuracy: 0.7741 Epoch 4/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.9159 - accuracy: 0.7738 Epoch 00004: val_accuracy improved from 0.77407 to 0.78281, saving model to classifier_weights2-improvement-04-0.78.hdf5 Epoch 00004: val_accuracy did not improve from 0.78281 4418/4418 [==============================] - 92s 21ms/step - loss: 0.9159 - accuracy: 0.7738 - val_loss: 0.5153 - val_accuracy: 0.7828 Epoch 5/200 4418/4418 [==============================] - ETA: 0s - loss: 0.8343 - accuracy: 0.7938 Epoch 00005: val_accuracy improved from 0.78281 to 0.80337, saving model to classifier_weights2-improvement-05-0.80.hdf5 Epoch 00005: val_accuracy did not improve from 0.80337 4418/4418 [==============================] - 90s 20ms/step - loss: 0.8343 - accuracy: 0.7938 - val_loss: 0.4637 - val_accuracy: 0.8034 Epoch 6/200 4418/4418 [==============================] - ETA: 0s - loss: 0.7896 - accuracy: 0.8026 Epoch 00006: val_accuracy improved from 0.80337 to 0.80959, saving model to classifier_weights2-improvement-06-0.81.hdf5 Epoch 00006: val_accuracy did not improve from 0.80959 4418/4418 [==============================] - 94s 21ms/step - loss: 0.7896 - accuracy: 0.8026 - val_loss: 0.4423 - val_accuracy: 0.8096 Epoch 7/200 4418/4418 [==============================] - ETA: 0s - loss: 0.7749 - accuracy: 0.8073 Epoch 00007: val_accuracy improved from 0.80959 to 0.81298, saving model to classifier_weights2-improvement-07-0.81.hdf5 Epoch 00007: val_accuracy did not improve from 0.81298 4418/4418 [==============================] - 88s 20ms/step - loss: 0.7749 - accuracy: 0.8073 - val_loss: 0.4559 - val_accuracy: 0.8130 Epoch 8/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.7646 - accuracy: 0.8099 Epoch 00008: val_accuracy did not improve from 0.81298 Epoch 00008: val_accuracy did not improve from 0.81298 4418/4418 [==============================] - 88s 20ms/step - loss: 0.7646 - accuracy: 0.8099 - val_loss: 0.4592 - val_accuracy: 0.8104 Epoch 9/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.7573 - accuracy: 0.8124 Epoch 00009: val_accuracy did not improve from 0.81298 Epoch 00009: val_accuracy did not improve from 0.81298 4418/4418 [==============================] - 89s 20ms/step - loss: 0.7573 - accuracy: 0.8124 - val_loss: 0.4624 - val_accuracy: 0.8085 Epoch 10/200 4418/4418 [==============================] - ETA: 0s - loss: 0.7539 - accuracy: 0.8128 Epoch 00010: val_accuracy did not improve from 0.81298 Epoch 00010: val_accuracy did not improve from 0.81298 4418/4418 [==============================] - 93s 21ms/step - loss: 0.7539 - accuracy: 0.8128 - val_loss: 0.4635 - val_accuracy: 0.8106 Epoch 11/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.7482 - accuracy: 0.8145 Epoch 00011: val_accuracy did not improve from 0.81298 Epoch 00011: val_accuracy did not improve from 0.81298 4418/4418 [==============================] - 92s 21ms/step - loss: 0.7482 - accuracy: 0.8145 - val_loss: 0.4657 - val_accuracy: 0.8091 Epoch 12/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.7413 - accuracy: 0.8146 Epoch 00012: val_accuracy improved from 0.81298 to 0.81855, saving model to classifier_weights2-improvement-12-0.82.hdf5 Epoch 00012: val_accuracy did not improve from 0.81855 4418/4418 [==============================] - 89s 20ms/step - loss: 0.7413 - accuracy: 0.8146 - val_loss: 0.4390 - val_accuracy: 0.8185 Epoch 13/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.7392 - accuracy: 0.8161 Epoch 00013: val_accuracy did not improve from 0.81855 Epoch 00013: val_accuracy did not improve from 0.81855 4418/4418 [==============================] - 92s 21ms/step - loss: 0.7392 - accuracy: 0.8161 - val_loss: 0.4667 - val_accuracy: 0.8143 Epoch 14/200 4416/4418 [============================>.] - ETA: 0s - loss: 0.7366 - accuracy: 0.8172 Epoch 00014: val_accuracy improved from 0.81855 to 0.82150, saving model to classifier_weights2-improvement-14-0.82.hdf5 Epoch 00014: val_accuracy did not improve from 0.82150 4418/4418 [==============================] - 91s 21ms/step - loss: 0.7366 - accuracy: 0.8172 - val_loss: 0.4311 - val_accuracy: 0.8215 Epoch 15/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.7353 - accuracy: 0.8164 Epoch 00015: val_accuracy did not improve from 0.82150 Epoch 00015: val_accuracy did not improve from 0.82150 4418/4418 [==============================] - 91s 21ms/step - loss: 0.7353 - accuracy: 0.8164 - val_loss: 0.4500 - val_accuracy: 0.8124 Epoch 16/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.7307 - accuracy: 0.8174 Epoch 00016: val_accuracy did not improve from 0.82150 Epoch 00016: val_accuracy did not improve from 0.82150 4418/4418 [==============================] - 91s 21ms/step - loss: 0.7306 - accuracy: 0.8174 - val_loss: 0.4410 - val_accuracy: 0.8155 Epoch 17/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.7278 - accuracy: 0.8189 Epoch 00017: val_accuracy did not improve from 0.82150 Epoch 00017: val_accuracy did not improve from 0.82150 4418/4418 [==============================] - 93s 21ms/step - loss: 0.7278 - accuracy: 0.8189 - val_loss: 0.4358 - val_accuracy: 0.8186 Epoch 18/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.7255 - accuracy: 0.8186 Epoch 00018: val_accuracy did not improve from 0.82150 Epoch 00018: val_accuracy did not improve from 0.82150 4418/4418 [==============================] - 94s 21ms/step - loss: 0.7255 - accuracy: 0.8186 - val_loss: 0.4360 - val_accuracy: 0.8170 Epoch 19/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.7242 - accuracy: 0.8177 Epoch 00019: val_accuracy did not improve from 0.82150 Epoch 00019: val_accuracy did not improve from 0.82150 4418/4418 [==============================] - 90s 20ms/step - loss: 0.7242 - accuracy: 0.8177 - val_loss: 0.4554 - val_accuracy: 0.8125 Epoch 20/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.7227 - accuracy: 0.8190 Epoch 00020: val_accuracy did not improve from 0.82150 Epoch 00020: val_accuracy did not improve from 0.82150 4418/4418 [==============================] - 90s 20ms/step - loss: 0.7227 - accuracy: 0.8190 - val_loss: 0.4464 - val_accuracy: 0.8158 Epoch 21/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.7190 - accuracy: 0.8190 Epoch 00021: val_accuracy did not improve from 0.82150 Epoch 00021: val_accuracy did not improve from 0.82150 4418/4418 [==============================] - 93s 21ms/step - loss: 0.7189 - accuracy: 0.8190 - val_loss: 0.4375 - val_accuracy: 0.8176 Epoch 22/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.7161 - accuracy: 0.8198 Epoch 00022: val_accuracy did not improve from 0.82150 Epoch 00022: val_accuracy did not improve from 0.82150 4418/4418 [==============================] - 91s 21ms/step - loss: 0.7161 - accuracy: 0.8198 - val_loss: 0.4517 - val_accuracy: 0.8116 Epoch 23/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.7152 - accuracy: 0.8203 Epoch 00023: val_accuracy improved from 0.82150 to 0.82385, saving model to classifier_weights2-improvement-23-0.82.hdf5 Epoch 00023: val_accuracy did not improve from 0.82385 4418/4418 [==============================] - 91s 21ms/step - loss: 0.7152 - accuracy: 0.8203 - val_loss: 0.4387 - val_accuracy: 0.8238 Epoch 24/200 4418/4418 [==============================] - ETA: 0s - loss: 0.7133 - accuracy: 0.8208 Epoch 00024: val_accuracy did not improve from 0.82385 Epoch 00024: val_accuracy did not improve from 0.82385 4418/4418 [==============================] - 96s 22ms/step - loss: 0.7133 - accuracy: 0.8208 - val_loss: 0.4476 - val_accuracy: 0.8203 Epoch 25/200 4416/4418 [============================>.] - ETA: 0s - loss: 0.7122 - accuracy: 0.8221 Epoch 00025: val_accuracy did not improve from 0.82385 Epoch 00025: val_accuracy did not improve from 0.82385 4418/4418 [==============================] - 92s 21ms/step - loss: 0.7122 - accuracy: 0.8221 - val_loss: 0.4301 - val_accuracy: 0.8223 Epoch 26/200 4416/4418 [============================>.] - ETA: 0s - loss: 0.7089 - accuracy: 0.8231 Epoch 00026: val_accuracy improved from 0.82385 to 0.82570, saving model to classifier_weights2-improvement-26-0.83.hdf5 Epoch 00026: val_accuracy did not improve from 0.82570 4418/4418 [==============================] - 91s 21ms/step - loss: 0.7089 - accuracy: 0.8231 - val_loss: 0.4222 - val_accuracy: 0.8257 Epoch 27/200 4418/4418 [==============================] - ETA: 0s - loss: 0.7094 - accuracy: 0.8213 Epoch 00027: val_accuracy did not improve from 0.82570 Epoch 00027: val_accuracy did not improve from 0.82570 4418/4418 [==============================] - 91s 21ms/step - loss: 0.7094 - accuracy: 0.8213 - val_loss: 0.4425 - val_accuracy: 0.8158 Epoch 28/200 4418/4418 [==============================] - ETA: 0s - loss: 0.7062 - accuracy: 0.8222 Epoch 00028: val_accuracy did not improve from 0.82570 Epoch 00028: val_accuracy did not improve from 0.82570 4418/4418 [==============================] - 93s 21ms/step - loss: 0.7062 - accuracy: 0.8222 - val_loss: 0.4274 - val_accuracy: 0.8257 Epoch 29/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.7058 - accuracy: 0.8225 Epoch 00029: val_accuracy did not improve from 0.82570 Epoch 00029: val_accuracy did not improve from 0.82570 4418/4418 [==============================] - 90s 20ms/step - loss: 0.7058 - accuracy: 0.8225 - val_loss: 0.4374 - val_accuracy: 0.8187 Epoch 30/200 4418/4418 [==============================] - ETA: 0s - loss: 0.7044 - accuracy: 0.8224 Epoch 00030: val_accuracy did not improve from 0.82570 Epoch 00030: val_accuracy did not improve from 0.82570 4418/4418 [==============================] - 90s 20ms/step - loss: 0.7044 - accuracy: 0.8224 - val_loss: 0.4268 - val_accuracy: 0.8248 Epoch 31/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.7017 - accuracy: 0.8249 Epoch 00031: val_accuracy improved from 0.82570 to 0.82720, saving model to classifier_weights2-improvement-31-0.83.hdf5 Epoch 00031: val_accuracy did not improve from 0.82720 4418/4418 [==============================] - 93s 21ms/step - loss: 0.7016 - accuracy: 0.8249 - val_loss: 0.4211 - val_accuracy: 0.8272 Epoch 32/200 4418/4418 [==============================] - ETA: 0s - loss: 0.6994 - accuracy: 0.8242 Epoch 00032: val_accuracy did not improve from 0.82720 Epoch 00032: val_accuracy did not improve from 0.82720 4418/4418 [==============================] - 90s 20ms/step - loss: 0.6994 - accuracy: 0.8242 - val_loss: 0.4412 - val_accuracy: 0.8167 Epoch 33/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6996 - accuracy: 0.8228 Epoch 00033: val_accuracy did not improve from 0.82720 Epoch 00033: val_accuracy did not improve from 0.82720 4418/4418 [==============================] - 90s 20ms/step - loss: 0.6996 - accuracy: 0.8228 - val_loss: 0.4265 - val_accuracy: 0.8225 Epoch 34/200 4418/4418 [==============================] - ETA: 0s - loss: 0.6986 - accuracy: 0.8244 Epoch 00034: val_accuracy did not improve from 0.82720 Epoch 00034: val_accuracy did not improve from 0.82720 4418/4418 [==============================] - 90s 20ms/step - loss: 0.6986 - accuracy: 0.8244 - val_loss: 0.4371 - val_accuracy: 0.8162 Epoch 35/200 4418/4418 [==============================] - ETA: 0s - loss: 0.6970 - accuracy: 0.8235 Epoch 00035: val_accuracy improved from 0.82720 to 0.83614, saving model to classifier_weights2-improvement-35-0.84.hdf5 Epoch 00035: val_accuracy did not improve from 0.83614 4418/4418 [==============================] - 90s 20ms/step - loss: 0.6970 - accuracy: 0.8235 - val_loss: 0.4109 - val_accuracy: 0.8361 Epoch 36/200 4418/4418 [==============================] - ETA: 0s - loss: 0.6961 - accuracy: 0.8236 Epoch 00036: val_accuracy did not improve from 0.83614 Epoch 00036: val_accuracy did not improve from 0.83614 4418/4418 [==============================] - 90s 20ms/step - loss: 0.6961 - accuracy: 0.8236 - val_loss: 0.4190 - val_accuracy: 0.8298 Epoch 37/200 4416/4418 [============================>.] - ETA: 0s - loss: 0.6948 - accuracy: 0.8239 Epoch 00037: val_accuracy did not improve from 0.83614 Epoch 00037: val_accuracy did not improve from 0.83614 4418/4418 [==============================] - 90s 20ms/step - loss: 0.6947 - accuracy: 0.8239 - val_loss: 0.4279 - val_accuracy: 0.8200 Epoch 38/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6929 - accuracy: 0.8248 Epoch 00038: val_accuracy did not improve from 0.83614 Epoch 00038: val_accuracy did not improve from 0.83614 4418/4418 [==============================] - 93s 21ms/step - loss: 0.6929 - accuracy: 0.8248 - val_loss: 0.4087 - val_accuracy: 0.8290 Epoch 39/200 4418/4418 [==============================] - ETA: 0s - loss: 0.6919 - accuracy: 0.8243 Epoch 00039: val_accuracy did not improve from 0.83614 Epoch 00039: val_accuracy did not improve from 0.83614 4418/4418 [==============================] - 90s 20ms/step - loss: 0.6919 - accuracy: 0.8243 - val_loss: 0.4278 - val_accuracy: 0.8223 Epoch 40/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6887 - accuracy: 0.8244 Epoch 00040: val_accuracy did not improve from 0.83614 Epoch 00040: val_accuracy did not improve from 0.83614 4418/4418 [==============================] - 89s 20ms/step - loss: 0.6887 - accuracy: 0.8244 - val_loss: 0.4057 - val_accuracy: 0.8279 Epoch 41/200 4416/4418 [============================>.] - ETA: 0s - loss: 0.6887 - accuracy: 0.8248 Epoch 00041: val_accuracy did not improve from 0.83614 Epoch 00041: val_accuracy did not improve from 0.83614 4418/4418 [==============================] - 89s 20ms/step - loss: 0.6887 - accuracy: 0.8248 - val_loss: 0.4429 - val_accuracy: 0.8188 Epoch 42/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6875 - accuracy: 0.8257 Epoch 00042: val_accuracy improved from 0.83614 to 0.83851, saving model to classifier_weights2-improvement-42-0.84.hdf5 Epoch 00042: val_accuracy did not improve from 0.83851 4418/4418 [==============================] - 90s 20ms/step - loss: 0.6875 - accuracy: 0.8257 - val_loss: 0.4032 - val_accuracy: 0.8385 Epoch 43/200 4418/4418 [==============================] - ETA: 0s - loss: 0.6856 - accuracy: 0.8250 Epoch 00043: val_accuracy did not improve from 0.83851 Epoch 00043: val_accuracy did not improve from 0.83851 4418/4418 [==============================] - 88s 20ms/step - loss: 0.6856 - accuracy: 0.8250 - val_loss: 0.4211 - val_accuracy: 0.8292 Epoch 44/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6848 - accuracy: 0.8254 Epoch 00044: val_accuracy did not improve from 0.83851 Epoch 00044: val_accuracy did not improve from 0.83851 4418/4418 [==============================] - 91s 21ms/step - loss: 0.6848 - accuracy: 0.8254 - val_loss: 0.4338 - val_accuracy: 0.8118 Epoch 45/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6830 - accuracy: 0.8260 Epoch 00045: val_accuracy did not improve from 0.83851 Epoch 00045: val_accuracy did not improve from 0.83851 4418/4418 [==============================] - 90s 20ms/step - loss: 0.6830 - accuracy: 0.8260 - val_loss: 0.4360 - val_accuracy: 0.8181 Epoch 46/200 4416/4418 [============================>.] - ETA: 0s - loss: 0.6836 - accuracy: 0.8251 Epoch 00046: val_accuracy did not improve from 0.83851 Epoch 00046: val_accuracy did not improve from 0.83851 4418/4418 [==============================] - 88s 20ms/step - loss: 0.6837 - accuracy: 0.8251 - val_loss: 0.4258 - val_accuracy: 0.8206 Epoch 47/200 4418/4418 [==============================] - ETA: 0s - loss: 0.6817 - accuracy: 0.8260 Epoch 00047: val_accuracy did not improve from 0.83851 Epoch 00047: val_accuracy did not improve from 0.83851 4418/4418 [==============================] - 88s 20ms/step - loss: 0.6817 - accuracy: 0.8260 - val_loss: 0.4144 - val_accuracy: 0.8268 Epoch 48/200 4418/4418 [==============================] - ETA: 0s - loss: 0.6802 - accuracy: 0.8273 Epoch 00048: val_accuracy did not improve from 0.83851 Epoch 00048: val_accuracy did not improve from 0.83851 4418/4418 [==============================] - 88s 20ms/step - loss: 0.6802 - accuracy: 0.8273 - val_loss: 0.4184 - val_accuracy: 0.8236 Epoch 49/200 4418/4418 [==============================] - ETA: 0s - loss: 0.6792 - accuracy: 0.8269 Epoch 00049: val_accuracy did not improve from 0.83851 Epoch 00049: val_accuracy did not improve from 0.83851 4418/4418 [==============================] - 89s 20ms/step - loss: 0.6792 - accuracy: 0.8269 - val_loss: 0.4236 - val_accuracy: 0.8174 Epoch 50/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6772 - accuracy: 0.8279 Epoch 00050: val_accuracy did not improve from 0.83851 Epoch 00050: val_accuracy did not improve from 0.83851 4418/4418 [==============================] - 88s 20ms/step - loss: 0.6772 - accuracy: 0.8279 - val_loss: 0.4175 - val_accuracy: 0.8240 Epoch 51/200 4418/4418 [==============================] - ETA: 0s - loss: 0.6770 - accuracy: 0.8283 Epoch 00051: val_accuracy did not improve from 0.83851 Epoch 00051: val_accuracy did not improve from 0.83851 4418/4418 [==============================] - 90s 20ms/step - loss: 0.6770 - accuracy: 0.8283 - val_loss: 0.4426 - val_accuracy: 0.8136 Epoch 52/200 4418/4418 [==============================] - ETA: 0s - loss: 0.6772 - accuracy: 0.8273 Epoch 00052: val_accuracy did not improve from 0.83851 Epoch 00052: val_accuracy did not improve from 0.83851 4418/4418 [==============================] - 88s 20ms/step - loss: 0.6772 - accuracy: 0.8273 - val_loss: 0.4204 - val_accuracy: 0.8353 Epoch 53/200 4416/4418 [============================>.] - ETA: 0s - loss: 0.6756 - accuracy: 0.8281 Epoch 00053: val_accuracy did not improve from 0.83851 Epoch 00053: val_accuracy did not improve from 0.83851 4418/4418 [==============================] - 89s 20ms/step - loss: 0.6756 - accuracy: 0.8281 - val_loss: 0.4271 - val_accuracy: 0.8215 Epoch 54/200 4416/4418 [============================>.] - ETA: 0s - loss: 0.6749 - accuracy: 0.8286 Epoch 00054: val_accuracy did not improve from 0.83851 Epoch 00054: val_accuracy did not improve from 0.83851 4418/4418 [==============================] - 88s 20ms/step - loss: 0.6749 - accuracy: 0.8286 - val_loss: 0.4306 - val_accuracy: 0.8179 Epoch 55/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6743 - accuracy: 0.8294 Epoch 00055: val_accuracy did not improve from 0.83851 Epoch 00055: val_accuracy did not improve from 0.83851 4418/4418 [==============================] - 87s 20ms/step - loss: 0.6743 - accuracy: 0.8294 - val_loss: 0.4061 - val_accuracy: 0.8287 Epoch 56/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6713 - accuracy: 0.8293 Epoch 00056: val_accuracy did not improve from 0.83851 Epoch 00056: val_accuracy did not improve from 0.83851 4418/4418 [==============================] - 89s 20ms/step - loss: 0.6713 - accuracy: 0.8293 - val_loss: 0.3953 - val_accuracy: 0.8326 Epoch 57/200 4418/4418 [==============================] - ETA: 0s - loss: 0.6714 - accuracy: 0.8291 Epoch 00057: val_accuracy did not improve from 0.83851 Epoch 00057: val_accuracy did not improve from 0.83851 4418/4418 [==============================] - 88s 20ms/step - loss: 0.6714 - accuracy: 0.8291 - val_loss: 0.4076 - val_accuracy: 0.8293 Epoch 58/200 4418/4418 [==============================] - ETA: 0s - loss: 0.6725 - accuracy: 0.8294 Epoch 00058: val_accuracy did not improve from 0.83851 Epoch 00058: val_accuracy did not improve from 0.83851 4418/4418 [==============================] - 90s 20ms/step - loss: 0.6725 - accuracy: 0.8294 - val_loss: 0.4043 - val_accuracy: 0.8316 Epoch 59/200 4416/4418 [============================>.] - ETA: 0s - loss: 0.6699 - accuracy: 0.8290 Epoch 00059: val_accuracy did not improve from 0.83851 Epoch 00059: val_accuracy did not improve from 0.83851 4418/4418 [==============================] - 88s 20ms/step - loss: 0.6699 - accuracy: 0.8290 - val_loss: 0.4302 - val_accuracy: 0.8239 Epoch 60/200 4418/4418 [==============================] - ETA: 0s - loss: 0.6706 - accuracy: 0.8290 Epoch 00060: val_accuracy did not improve from 0.83851 Epoch 00060: val_accuracy did not improve from 0.83851 4418/4418 [==============================] - 89s 20ms/step - loss: 0.6706 - accuracy: 0.8290 - val_loss: 0.4042 - val_accuracy: 0.8289 Epoch 61/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6694 - accuracy: 0.8299 Epoch 00061: val_accuracy did not improve from 0.83851 Epoch 00061: val_accuracy did not improve from 0.83851 4418/4418 [==============================] - 90s 20ms/step - loss: 0.6694 - accuracy: 0.8299 - val_loss: 0.4129 - val_accuracy: 0.8218 Epoch 62/200 4418/4418 [==============================] - ETA: 0s - loss: 0.6672 - accuracy: 0.8291 Epoch 00062: val_accuracy did not improve from 0.83851 Epoch 00062: val_accuracy did not improve from 0.83851 4418/4418 [==============================] - 88s 20ms/step - loss: 0.6672 - accuracy: 0.8291 - val_loss: 0.4064 - val_accuracy: 0.8276 Epoch 63/200 4418/4418 [==============================] - ETA: 0s - loss: 0.6678 - accuracy: 0.8291 Epoch 00063: val_accuracy did not improve from 0.83851 Epoch 00063: val_accuracy did not improve from 0.83851 4418/4418 [==============================] - 90s 20ms/step - loss: 0.6678 - accuracy: 0.8291 - val_loss: 0.4141 - val_accuracy: 0.8276 Epoch 64/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6662 - accuracy: 0.8302 Epoch 00064: val_accuracy did not improve from 0.83851 Epoch 00064: val_accuracy did not improve from 0.83851 4418/4418 [==============================] - 91s 21ms/step - loss: 0.6662 - accuracy: 0.8302 - val_loss: 0.4018 - val_accuracy: 0.8343 Epoch 65/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6658 - accuracy: 0.8296 Epoch 00065: val_accuracy did not improve from 0.83851 Epoch 00065: val_accuracy did not improve from 0.83851 4418/4418 [==============================] - 91s 21ms/step - loss: 0.6658 - accuracy: 0.8296 - val_loss: 0.4059 - val_accuracy: 0.8296 Epoch 66/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6660 - accuracy: 0.8308 Epoch 00066: val_accuracy did not improve from 0.83851 Epoch 00066: val_accuracy did not improve from 0.83851 4418/4418 [==============================] - 88s 20ms/step - loss: 0.6660 - accuracy: 0.8308 - val_loss: 0.4154 - val_accuracy: 0.8300 Epoch 67/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6646 - accuracy: 0.8303 Epoch 00067: val_accuracy did not improve from 0.83851 Epoch 00067: val_accuracy did not improve from 0.83851 4418/4418 [==============================] - 90s 20ms/step - loss: 0.6647 - accuracy: 0.8303 - val_loss: 0.4123 - val_accuracy: 0.8306 Epoch 68/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6642 - accuracy: 0.8297 Epoch 00068: val_accuracy did not improve from 0.83851 Epoch 00068: val_accuracy did not improve from 0.83851 4418/4418 [==============================] - 89s 20ms/step - loss: 0.6642 - accuracy: 0.8297 - val_loss: 0.4243 - val_accuracy: 0.8229 Epoch 69/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6637 - accuracy: 0.8293 Epoch 00069: val_accuracy improved from 0.83851 to 0.83934, saving model to classifier_weights2-improvement-69-0.84.hdf5 Epoch 00069: val_accuracy did not improve from 0.83934 4418/4418 [==============================] - 89s 20ms/step - loss: 0.6637 - accuracy: 0.8293 - val_loss: 0.4002 - val_accuracy: 0.8393 Epoch 70/200 4418/4418 [==============================] - ETA: 0s - loss: 0.6635 - accuracy: 0.8302 Epoch 00070: val_accuracy did not improve from 0.83934 Epoch 00070: val_accuracy did not improve from 0.83934 4418/4418 [==============================] - 88s 20ms/step - loss: 0.6635 - accuracy: 0.8302 - val_loss: 0.4440 - val_accuracy: 0.8170 Epoch 71/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6638 - accuracy: 0.8305 Epoch 00071: val_accuracy did not improve from 0.83934 Epoch 00071: val_accuracy did not improve from 0.83934 4418/4418 [==============================] - 90s 20ms/step - loss: 0.6638 - accuracy: 0.8305 - val_loss: 0.4385 - val_accuracy: 0.8223 Epoch 72/200 4416/4418 [============================>.] - ETA: 0s - loss: 0.6625 - accuracy: 0.8308 Epoch 00072: val_accuracy did not improve from 0.83934 Epoch 00072: val_accuracy did not improve from 0.83934 4418/4418 [==============================] - 90s 20ms/step - loss: 0.6625 - accuracy: 0.8308 - val_loss: 0.4020 - val_accuracy: 0.8372 Epoch 73/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6610 - accuracy: 0.8300 Epoch 00073: val_accuracy did not improve from 0.83934 Epoch 00073: val_accuracy did not improve from 0.83934 4418/4418 [==============================] - 88s 20ms/step - loss: 0.6610 - accuracy: 0.8300 - val_loss: 0.4192 - val_accuracy: 0.8242 Epoch 74/200 4416/4418 [============================>.] - ETA: 0s - loss: 0.6620 - accuracy: 0.8290 Epoch 00074: val_accuracy did not improve from 0.83934 Epoch 00074: val_accuracy did not improve from 0.83934 4418/4418 [==============================] - 90s 20ms/step - loss: 0.6620 - accuracy: 0.8290 - val_loss: 0.4140 - val_accuracy: 0.8226 Epoch 75/200 4418/4418 [==============================] - ETA: 0s - loss: 0.6613 - accuracy: 0.8281 Epoch 00075: val_accuracy did not improve from 0.83934 Epoch 00075: val_accuracy did not improve from 0.83934 4418/4418 [==============================] - 88s 20ms/step - loss: 0.6613 - accuracy: 0.8281 - val_loss: 0.4012 - val_accuracy: 0.8354 Epoch 76/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6596 - accuracy: 0.8306 Epoch 00076: val_accuracy did not improve from 0.83934 Epoch 00076: val_accuracy did not improve from 0.83934 4418/4418 [==============================] - 88s 20ms/step - loss: 0.6596 - accuracy: 0.8306 - val_loss: 0.4068 - val_accuracy: 0.8265 Epoch 77/200 4418/4418 [==============================] - ETA: 0s - loss: 0.6610 - accuracy: 0.8292 Epoch 00077: val_accuracy did not improve from 0.83934 Epoch 00077: val_accuracy did not improve from 0.83934 4418/4418 [==============================] - 88s 20ms/step - loss: 0.6610 - accuracy: 0.8292 - val_loss: 0.4488 - val_accuracy: 0.8171 Epoch 78/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6591 - accuracy: 0.8318 Epoch 00078: val_accuracy did not improve from 0.83934 Epoch 00078: val_accuracy did not improve from 0.83934 4418/4418 [==============================] - 90s 20ms/step - loss: 0.6591 - accuracy: 0.8318 - val_loss: 0.4243 - val_accuracy: 0.8215 Epoch 79/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6591 - accuracy: 0.8297 Epoch 00079: val_accuracy did not improve from 0.83934 Epoch 00079: val_accuracy did not improve from 0.83934 4418/4418 [==============================] - 90s 20ms/step - loss: 0.6591 - accuracy: 0.8297 - val_loss: 0.4175 - val_accuracy: 0.8261 Epoch 80/200 4416/4418 [============================>.] - ETA: 0s - loss: 0.6587 - accuracy: 0.8294 Epoch 00080: val_accuracy did not improve from 0.83934 Epoch 00080: val_accuracy did not improve from 0.83934 4418/4418 [==============================] - 88s 20ms/step - loss: 0.6587 - accuracy: 0.8294 - val_loss: 0.4109 - val_accuracy: 0.8247 Epoch 81/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6593 - accuracy: 0.8292 Epoch 00081: val_accuracy did not improve from 0.83934 Epoch 00081: val_accuracy did not improve from 0.83934 4418/4418 [==============================] - 88s 20ms/step - loss: 0.6593 - accuracy: 0.8292 - val_loss: 0.4011 - val_accuracy: 0.8279 Epoch 82/200 4416/4418 [============================>.] - ETA: 0s - loss: 0.6570 - accuracy: 0.8305 Epoch 00082: val_accuracy did not improve from 0.83934 Epoch 00082: val_accuracy did not improve from 0.83934 4418/4418 [==============================] - 89s 20ms/step - loss: 0.6570 - accuracy: 0.8305 - val_loss: 0.4349 - val_accuracy: 0.8269 Epoch 83/200 4418/4418 [==============================] - ETA: 0s - loss: 0.6573 - accuracy: 0.8315 Epoch 00083: val_accuracy did not improve from 0.83934 Epoch 00083: val_accuracy did not improve from 0.83934 4418/4418 [==============================] - 87s 20ms/step - loss: 0.6573 - accuracy: 0.8315 - val_loss: 0.3926 - val_accuracy: 0.8336 Epoch 84/200 4418/4418 [==============================] - ETA: 0s - loss: 0.6560 - accuracy: 0.8300 Epoch 00084: val_accuracy did not improve from 0.83934 Epoch 00084: val_accuracy did not improve from 0.83934 4418/4418 [==============================] - 88s 20ms/step - loss: 0.6560 - accuracy: 0.8300 - val_loss: 0.4181 - val_accuracy: 0.8258 Epoch 85/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6544 - accuracy: 0.8305 Epoch 00085: val_accuracy did not improve from 0.83934 Epoch 00085: val_accuracy did not improve from 0.83934 4418/4418 [==============================] - 90s 20ms/step - loss: 0.6544 - accuracy: 0.8305 - val_loss: 0.4146 - val_accuracy: 0.8294 Epoch 86/200 4416/4418 [============================>.] - ETA: 0s - loss: 0.6550 - accuracy: 0.8314 Epoch 00086: val_accuracy did not improve from 0.83934 Epoch 00086: val_accuracy did not improve from 0.83934 4418/4418 [==============================] - 89s 20ms/step - loss: 0.6549 - accuracy: 0.8314 - val_loss: 0.4198 - val_accuracy: 0.8211 Epoch 87/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6539 - accuracy: 0.8322 Epoch 00087: val_accuracy did not improve from 0.83934 Epoch 00087: val_accuracy did not improve from 0.83934 4418/4418 [==============================] - 87s 20ms/step - loss: 0.6539 - accuracy: 0.8322 - val_loss: 0.4175 - val_accuracy: 0.8256 Epoch 88/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6550 - accuracy: 0.8305 Epoch 00088: val_accuracy did not improve from 0.83934 Epoch 00088: val_accuracy did not improve from 0.83934 4418/4418 [==============================] - 89s 20ms/step - loss: 0.6550 - accuracy: 0.8305 - val_loss: 0.4165 - val_accuracy: 0.8260 Epoch 89/200 4418/4418 [==============================] - ETA: 0s - loss: 0.6543 - accuracy: 0.8310 Epoch 00089: val_accuracy did not improve from 0.83934 Epoch 00089: val_accuracy did not improve from 0.83934 4418/4418 [==============================] - 89s 20ms/step - loss: 0.6543 - accuracy: 0.8310 - val_loss: 0.4248 - val_accuracy: 0.8249 Epoch 90/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6544 - accuracy: 0.8316 Epoch 00090: val_accuracy did not improve from 0.83934 Epoch 00090: val_accuracy did not improve from 0.83934 4418/4418 [==============================] - 88s 20ms/step - loss: 0.6544 - accuracy: 0.8316 - val_loss: 0.4047 - val_accuracy: 0.8280 Epoch 91/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6525 - accuracy: 0.8319 Epoch 00091: val_accuracy did not improve from 0.83934 Epoch 00091: val_accuracy did not improve from 0.83934 4418/4418 [==============================] - 88s 20ms/step - loss: 0.6525 - accuracy: 0.8319 - val_loss: 0.4221 - val_accuracy: 0.8211 Epoch 92/200 4416/4418 [============================>.] - ETA: 0s - loss: 0.6555 - accuracy: 0.8318 Epoch 00092: val_accuracy did not improve from 0.83934 Epoch 00092: val_accuracy did not improve from 0.83934 4418/4418 [==============================] - 90s 20ms/step - loss: 0.6554 - accuracy: 0.8318 - val_loss: 0.4002 - val_accuracy: 0.8327 Epoch 93/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6519 - accuracy: 0.8318 Epoch 00093: val_accuracy did not improve from 0.83934 Epoch 00093: val_accuracy did not improve from 0.83934 4418/4418 [==============================] - 88s 20ms/step - loss: 0.6519 - accuracy: 0.8318 - val_loss: 0.4018 - val_accuracy: 0.8275 Epoch 94/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6513 - accuracy: 0.8318 Epoch 00094: val_accuracy improved from 0.83934 to 0.84072, saving model to classifier_weights2-improvement-94-0.84.hdf5 Epoch 00094: val_accuracy did not improve from 0.84072 4418/4418 [==============================] - 88s 20ms/step - loss: 0.6513 - accuracy: 0.8318 - val_loss: 0.3906 - val_accuracy: 0.8407 Epoch 95/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6518 - accuracy: 0.8316 Epoch 00095: val_accuracy did not improve from 0.84072 Epoch 00095: val_accuracy did not improve from 0.84072 4418/4418 [==============================] - 87s 20ms/step - loss: 0.6518 - accuracy: 0.8316 - val_loss: 0.4107 - val_accuracy: 0.8258 Epoch 96/200 4418/4418 [==============================] - ETA: 0s - loss: 0.6504 - accuracy: 0.8312 Epoch 00096: val_accuracy did not improve from 0.84072 Epoch 00096: val_accuracy did not improve from 0.84072 4418/4418 [==============================] - 89s 20ms/step - loss: 0.6504 - accuracy: 0.8312 - val_loss: 0.3982 - val_accuracy: 0.8349 Epoch 97/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6504 - accuracy: 0.8318 Epoch 00097: val_accuracy did not improve from 0.84072 Epoch 00097: val_accuracy did not improve from 0.84072 4418/4418 [==============================] - 88s 20ms/step - loss: 0.6504 - accuracy: 0.8318 - val_loss: 0.4091 - val_accuracy: 0.8281 Epoch 98/200 4418/4418 [==============================] - ETA: 0s - loss: 0.6489 - accuracy: 0.8310 Epoch 00098: val_accuracy did not improve from 0.84072 Epoch 00098: val_accuracy did not improve from 0.84072 4418/4418 [==============================] - 88s 20ms/step - loss: 0.6489 - accuracy: 0.8310 - val_loss: 0.3783 - val_accuracy: 0.8389 Epoch 99/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6486 - accuracy: 0.8321 Epoch 00099: val_accuracy did not improve from 0.84072 Epoch 00099: val_accuracy did not improve from 0.84072 4418/4418 [==============================] - 90s 20ms/step - loss: 0.6486 - accuracy: 0.8321 - val_loss: 0.3980 - val_accuracy: 0.8279 Epoch 100/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6488 - accuracy: 0.8319 Epoch 00100: val_accuracy did not improve from 0.84072 Epoch 00100: val_accuracy did not improve from 0.84072 4418/4418 [==============================] - 88s 20ms/step - loss: 0.6488 - accuracy: 0.8319 - val_loss: 0.4063 - val_accuracy: 0.8300 Epoch 101/200 4416/4418 [============================>.] - ETA: 0s - loss: 0.6475 - accuracy: 0.8314 Epoch 00101: val_accuracy did not improve from 0.84072 Epoch 00101: val_accuracy did not improve from 0.84072 4418/4418 [==============================] - 87s 20ms/step - loss: 0.6475 - accuracy: 0.8314 - val_loss: 0.3950 - val_accuracy: 0.8352 Epoch 102/200 4416/4418 [============================>.] - ETA: 0s - loss: 0.6484 - accuracy: 0.8315 Epoch 00102: val_accuracy did not improve from 0.84072 Epoch 00102: val_accuracy did not improve from 0.84072 4418/4418 [==============================] - 88s 20ms/step - loss: 0.6484 - accuracy: 0.8315 - val_loss: 0.4108 - val_accuracy: 0.8294 Epoch 103/200 4418/4418 [==============================] - ETA: 0s - loss: 0.6491 - accuracy: 0.8333 Epoch 00103: val_accuracy did not improve from 0.84072 Epoch 00103: val_accuracy did not improve from 0.84072 4418/4418 [==============================] - 88s 20ms/step - loss: 0.6491 - accuracy: 0.8333 - val_loss: 0.4283 - val_accuracy: 0.8248 Epoch 104/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6466 - accuracy: 0.8328 Epoch 00104: val_accuracy did not improve from 0.84072 Epoch 00104: val_accuracy did not improve from 0.84072 4418/4418 [==============================] - 87s 20ms/step - loss: 0.6466 - accuracy: 0.8328 - val_loss: 0.4319 - val_accuracy: 0.8187 Epoch 105/200 4416/4418 [============================>.] - ETA: 0s - loss: 0.6484 - accuracy: 0.8323 Epoch 00105: val_accuracy did not improve from 0.84072 Epoch 00105: val_accuracy did not improve from 0.84072 4418/4418 [==============================] - 88s 20ms/step - loss: 0.6484 - accuracy: 0.8323 - val_loss: 0.4338 - val_accuracy: 0.8339 Epoch 106/200 4418/4418 [==============================] - ETA: 0s - loss: 0.6457 - accuracy: 0.8320 Epoch 00106: val_accuracy did not improve from 0.84072 Epoch 00106: val_accuracy did not improve from 0.84072 4418/4418 [==============================] - 90s 20ms/step - loss: 0.6457 - accuracy: 0.8320 - val_loss: 0.4070 - val_accuracy: 0.8315 Epoch 107/200 4418/4418 [==============================] - ETA: 0s - loss: 0.6466 - accuracy: 0.8340 Epoch 00107: val_accuracy did not improve from 0.84072 Epoch 00107: val_accuracy did not improve from 0.84072 4418/4418 [==============================] - 89s 20ms/step - loss: 0.6466 - accuracy: 0.8340 - val_loss: 0.4105 - val_accuracy: 0.8319 Epoch 108/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6456 - accuracy: 0.8331 Epoch 00108: val_accuracy did not improve from 0.84072 Epoch 00108: val_accuracy did not improve from 0.84072 4418/4418 [==============================] - 87s 20ms/step - loss: 0.6456 - accuracy: 0.8331 - val_loss: 0.4021 - val_accuracy: 0.8291 Epoch 109/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6448 - accuracy: 0.8316 Epoch 00109: val_accuracy did not improve from 0.84072 Epoch 00109: val_accuracy did not improve from 0.84072 4418/4418 [==============================] - 87s 20ms/step - loss: 0.6448 - accuracy: 0.8316 - val_loss: 0.3888 - val_accuracy: 0.8335 Epoch 110/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6442 - accuracy: 0.8319 Epoch 00110: val_accuracy did not improve from 0.84072 Epoch 00110: val_accuracy did not improve from 0.84072 4418/4418 [==============================] - 89s 20ms/step - loss: 0.6442 - accuracy: 0.8319 - val_loss: 0.3990 - val_accuracy: 0.8302 Epoch 111/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6449 - accuracy: 0.8315 Epoch 00111: val_accuracy did not improve from 0.84072 Epoch 00111: val_accuracy did not improve from 0.84072 4418/4418 [==============================] - 88s 20ms/step - loss: 0.6449 - accuracy: 0.8315 - val_loss: 0.4081 - val_accuracy: 0.8290 Epoch 112/200 4418/4418 [==============================] - ETA: 0s - loss: 0.6453 - accuracy: 0.8319 Epoch 00112: val_accuracy did not improve from 0.84072 Epoch 00112: val_accuracy did not improve from 0.84072 4418/4418 [==============================] - 87s 20ms/step - loss: 0.6453 - accuracy: 0.8319 - val_loss: 0.4034 - val_accuracy: 0.8348 Epoch 113/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6436 - accuracy: 0.8326 Epoch 00113: val_accuracy improved from 0.84072 to 0.84206, saving model to classifier_weights2-improvement-113-0.84.hdf5 Epoch 00113: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 90s 20ms/step - loss: 0.6436 - accuracy: 0.8326 - val_loss: 0.4085 - val_accuracy: 0.8421 Epoch 114/200 4416/4418 [============================>.] - ETA: 0s - loss: 0.6431 - accuracy: 0.8335 Epoch 00114: val_accuracy did not improve from 0.84206 Epoch 00114: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 88s 20ms/step - loss: 0.6430 - accuracy: 0.8334 - val_loss: 0.4307 - val_accuracy: 0.8278 Epoch 115/200 4418/4418 [==============================] - ETA: 0s - loss: 0.6423 - accuracy: 0.8329 Epoch 00115: val_accuracy did not improve from 0.84206 Epoch 00115: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 88s 20ms/step - loss: 0.6423 - accuracy: 0.8329 - val_loss: 0.4052 - val_accuracy: 0.8355 Epoch 116/200 4418/4418 [==============================] - ETA: 0s - loss: 0.6426 - accuracy: 0.8318 Epoch 00116: val_accuracy did not improve from 0.84206 Epoch 00116: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 88s 20ms/step - loss: 0.6426 - accuracy: 0.8318 - val_loss: 0.4126 - val_accuracy: 0.8314 Epoch 117/200 4416/4418 [============================>.] - ETA: 0s - loss: 0.6427 - accuracy: 0.8331 Epoch 00117: val_accuracy did not improve from 0.84206 Epoch 00117: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 88s 20ms/step - loss: 0.6428 - accuracy: 0.8331 - val_loss: 0.4099 - val_accuracy: 0.8301 Epoch 118/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6424 - accuracy: 0.8329 Epoch 00118: val_accuracy did not improve from 0.84206 Epoch 00118: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 88s 20ms/step - loss: 0.6423 - accuracy: 0.8329 - val_loss: 0.4081 - val_accuracy: 0.8299 Epoch 119/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6425 - accuracy: 0.8333 Epoch 00119: val_accuracy did not improve from 0.84206 Epoch 00119: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 88s 20ms/step - loss: 0.6425 - accuracy: 0.8333 - val_loss: 0.4091 - val_accuracy: 0.8282 Epoch 120/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6410 - accuracy: 0.8337 Epoch 00120: val_accuracy did not improve from 0.84206 Epoch 00120: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 91s 21ms/step - loss: 0.6411 - accuracy: 0.8337 - val_loss: 0.4345 - val_accuracy: 0.8231 Epoch 121/200 4418/4418 [==============================] - ETA: 0s - loss: 0.6425 - accuracy: 0.8333 Epoch 00121: val_accuracy did not improve from 0.84206 Epoch 00121: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 89s 20ms/step - loss: 0.6425 - accuracy: 0.8333 - val_loss: 0.3969 - val_accuracy: 0.8344 Epoch 122/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6414 - accuracy: 0.8329 Epoch 00122: val_accuracy did not improve from 0.84206 Epoch 00122: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 88s 20ms/step - loss: 0.6414 - accuracy: 0.8329 - val_loss: 0.4022 - val_accuracy: 0.8352 Epoch 123/200 4418/4418 [==============================] - ETA: 0s - loss: 0.6410 - accuracy: 0.8329 Epoch 00123: val_accuracy did not improve from 0.84206 Epoch 00123: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 88s 20ms/step - loss: 0.6410 - accuracy: 0.8329 - val_loss: 0.3956 - val_accuracy: 0.8355 Epoch 124/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6401 - accuracy: 0.8337 Epoch 00124: val_accuracy did not improve from 0.84206 Epoch 00124: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 88s 20ms/step - loss: 0.6401 - accuracy: 0.8337 - val_loss: 0.4229 - val_accuracy: 0.8249 Epoch 125/200 4418/4418 [==============================] - ETA: 0s - loss: 0.6393 - accuracy: 0.8337 Epoch 00125: val_accuracy did not improve from 0.84206 Epoch 00125: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 90s 20ms/step - loss: 0.6393 - accuracy: 0.8337 - val_loss: 0.4066 - val_accuracy: 0.8295 Epoch 126/200 4416/4418 [============================>.] - ETA: 0s - loss: 0.6385 - accuracy: 0.8334 Epoch 00126: val_accuracy did not improve from 0.84206 Epoch 00126: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 88s 20ms/step - loss: 0.6385 - accuracy: 0.8334 - val_loss: 0.4069 - val_accuracy: 0.8319 Epoch 127/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6385 - accuracy: 0.8341 Epoch 00127: val_accuracy did not improve from 0.84206 Epoch 00127: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 90s 20ms/step - loss: 0.6385 - accuracy: 0.8341 - val_loss: 0.4034 - val_accuracy: 0.8292 Epoch 128/200 4416/4418 [============================>.] - ETA: 0s - loss: 0.6420 - accuracy: 0.8326 Epoch 00128: val_accuracy did not improve from 0.84206 Epoch 00128: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 89s 20ms/step - loss: 0.6421 - accuracy: 0.8326 - val_loss: 0.4038 - val_accuracy: 0.8346 Epoch 129/200 4418/4418 [==============================] - ETA: 0s - loss: 0.6392 - accuracy: 0.8335 Epoch 00129: val_accuracy did not improve from 0.84206 Epoch 00129: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 89s 20ms/step - loss: 0.6392 - accuracy: 0.8335 - val_loss: 0.4001 - val_accuracy: 0.8337 Epoch 130/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6384 - accuracy: 0.8343 Epoch 00130: val_accuracy did not improve from 0.84206 Epoch 00130: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 88s 20ms/step - loss: 0.6384 - accuracy: 0.8343 - val_loss: 0.3939 - val_accuracy: 0.8380 Epoch 131/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6379 - accuracy: 0.8329 Epoch 00131: val_accuracy did not improve from 0.84206 Epoch 00131: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 88s 20ms/step - loss: 0.6379 - accuracy: 0.8329 - val_loss: 0.4156 - val_accuracy: 0.8278 Epoch 132/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6391 - accuracy: 0.8318 Epoch 00132: val_accuracy did not improve from 0.84206 Epoch 00132: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 89s 20ms/step - loss: 0.6391 - accuracy: 0.8318 - val_loss: 0.4120 - val_accuracy: 0.8245 Epoch 133/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6396 - accuracy: 0.8327 Epoch 00133: val_accuracy did not improve from 0.84206 Epoch 00133: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 88s 20ms/step - loss: 0.6396 - accuracy: 0.8327 - val_loss: 0.4121 - val_accuracy: 0.8246 Epoch 134/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6383 - accuracy: 0.8336 Epoch 00134: val_accuracy did not improve from 0.84206 Epoch 00134: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 90s 20ms/step - loss: 0.6382 - accuracy: 0.8336 - val_loss: 0.4151 - val_accuracy: 0.8307 Epoch 135/200 4416/4418 [============================>.] - ETA: 0s - loss: 0.6374 - accuracy: 0.8336 Epoch 00135: val_accuracy did not improve from 0.84206 Epoch 00135: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 89s 20ms/step - loss: 0.6374 - accuracy: 0.8336 - val_loss: 0.3973 - val_accuracy: 0.8337 Epoch 136/200 4418/4418 [==============================] - ETA: 0s - loss: 0.6366 - accuracy: 0.8334 Epoch 00136: val_accuracy did not improve from 0.84206 Epoch 00136: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 89s 20ms/step - loss: 0.6366 - accuracy: 0.8334 - val_loss: 0.3977 - val_accuracy: 0.8313 Epoch 137/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6358 - accuracy: 0.8332 Epoch 00137: val_accuracy did not improve from 0.84206 Epoch 00137: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 89s 20ms/step - loss: 0.6359 - accuracy: 0.8332 - val_loss: 0.4163 - val_accuracy: 0.8269 Epoch 138/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6346 - accuracy: 0.8341 Epoch 00138: val_accuracy did not improve from 0.84206 Epoch 00138: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 88s 20ms/step - loss: 0.6346 - accuracy: 0.8341 - val_loss: 0.4398 - val_accuracy: 0.8233 Epoch 139/200 4418/4418 [==============================] - ETA: 0s - loss: 0.6362 - accuracy: 0.8343 Epoch 00139: val_accuracy did not improve from 0.84206 Epoch 00139: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 90s 20ms/step - loss: 0.6362 - accuracy: 0.8343 - val_loss: 0.3991 - val_accuracy: 0.8330 Epoch 140/200 4418/4418 [==============================] - ETA: 0s - loss: 0.6352 - accuracy: 0.8342 Epoch 00140: val_accuracy did not improve from 0.84206 Epoch 00140: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 89s 20ms/step - loss: 0.6352 - accuracy: 0.8342 - val_loss: 0.4091 - val_accuracy: 0.8315 Epoch 141/200 4418/4418 [==============================] - ETA: 0s - loss: 0.6365 - accuracy: 0.8335 Epoch 00141: val_accuracy did not improve from 0.84206 Epoch 00141: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 91s 21ms/step - loss: 0.6365 - accuracy: 0.8335 - val_loss: 0.4268 - val_accuracy: 0.8195 Epoch 142/200 4416/4418 [============================>.] - ETA: 0s - loss: 0.6349 - accuracy: 0.8330 Epoch 00142: val_accuracy did not improve from 0.84206 Epoch 00142: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 88s 20ms/step - loss: 0.6349 - accuracy: 0.8330 - val_loss: 0.4061 - val_accuracy: 0.8336 Epoch 143/200 4418/4418 [==============================] - ETA: 0s - loss: 0.6358 - accuracy: 0.8338 Epoch 00143: val_accuracy did not improve from 0.84206 Epoch 00143: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 90s 20ms/step - loss: 0.6358 - accuracy: 0.8338 - val_loss: 0.4094 - val_accuracy: 0.8246 Epoch 144/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6343 - accuracy: 0.8324 Epoch 00144: val_accuracy did not improve from 0.84206 Epoch 00144: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 89s 20ms/step - loss: 0.6343 - accuracy: 0.8324 - val_loss: 0.3982 - val_accuracy: 0.8297 Epoch 145/200 4416/4418 [============================>.] - ETA: 0s - loss: 0.6342 - accuracy: 0.8333 Epoch 00145: val_accuracy did not improve from 0.84206 Epoch 00145: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 88s 20ms/step - loss: 0.6342 - accuracy: 0.8333 - val_loss: 0.4118 - val_accuracy: 0.8356 Epoch 146/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6322 - accuracy: 0.8342 Epoch 00146: val_accuracy did not improve from 0.84206 Epoch 00146: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 89s 20ms/step - loss: 0.6322 - accuracy: 0.8342 - val_loss: 0.3949 - val_accuracy: 0.8355 Epoch 147/200 4416/4418 [============================>.] - ETA: 0s - loss: 0.6339 - accuracy: 0.8337 Epoch 00147: val_accuracy did not improve from 0.84206 Epoch 00147: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 88s 20ms/step - loss: 0.6339 - accuracy: 0.8337 - val_loss: 0.4012 - val_accuracy: 0.8316 Epoch 148/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6332 - accuracy: 0.8339 Epoch 00148: val_accuracy did not improve from 0.84206 Epoch 00148: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 90s 20ms/step - loss: 0.6332 - accuracy: 0.8339 - val_loss: 0.4049 - val_accuracy: 0.8331 Epoch 149/200 4416/4418 [============================>.] - ETA: 0s - loss: 0.6333 - accuracy: 0.8340 Epoch 00149: val_accuracy did not improve from 0.84206 Epoch 00149: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 88s 20ms/step - loss: 0.6333 - accuracy: 0.8340 - val_loss: 0.4300 - val_accuracy: 0.8176 Epoch 150/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6338 - accuracy: 0.8336 Epoch 00150: val_accuracy did not improve from 0.84206 Epoch 00150: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 90s 20ms/step - loss: 0.6338 - accuracy: 0.8336 - val_loss: 0.4131 - val_accuracy: 0.8282 Epoch 151/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6326 - accuracy: 0.8330 Epoch 00151: val_accuracy did not improve from 0.84206 Epoch 00151: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 89s 20ms/step - loss: 0.6326 - accuracy: 0.8330 - val_loss: 0.3834 - val_accuracy: 0.8375 Epoch 152/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6324 - accuracy: 0.8348 Epoch 00152: val_accuracy did not improve from 0.84206 Epoch 00152: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 88s 20ms/step - loss: 0.6324 - accuracy: 0.8349 - val_loss: 0.3945 - val_accuracy: 0.8353 Epoch 153/200 4416/4418 [============================>.] - ETA: 0s - loss: 0.6323 - accuracy: 0.8337 Epoch 00153: val_accuracy did not improve from 0.84206 Epoch 00153: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 88s 20ms/step - loss: 0.6323 - accuracy: 0.8337 - val_loss: 0.3882 - val_accuracy: 0.8377 Epoch 154/200 4416/4418 [============================>.] - ETA: 0s - loss: 0.6346 - accuracy: 0.8335 Epoch 00154: val_accuracy did not improve from 0.84206 Epoch 00154: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 90s 20ms/step - loss: 0.6345 - accuracy: 0.8336 - val_loss: 0.3931 - val_accuracy: 0.8340 Epoch 155/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6332 - accuracy: 0.8342 Epoch 00155: val_accuracy did not improve from 0.84206 Epoch 00155: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 89s 20ms/step - loss: 0.6333 - accuracy: 0.8342 - val_loss: 0.4102 - val_accuracy: 0.8275 Epoch 156/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6318 - accuracy: 0.8342 Epoch 00156: val_accuracy did not improve from 0.84206 Epoch 00156: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 88s 20ms/step - loss: 0.6318 - accuracy: 0.8342 - val_loss: 0.4147 - val_accuracy: 0.8304 Epoch 157/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6317 - accuracy: 0.8338 Epoch 00157: val_accuracy did not improve from 0.84206 Epoch 00157: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 89s 20ms/step - loss: 0.6317 - accuracy: 0.8338 - val_loss: 0.4074 - val_accuracy: 0.8311 Epoch 158/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6324 - accuracy: 0.8347 Epoch 00158: val_accuracy did not improve from 0.84206 Epoch 00158: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 88s 20ms/step - loss: 0.6324 - accuracy: 0.8347 - val_loss: 0.4078 - val_accuracy: 0.8322 Epoch 159/200 4416/4418 [============================>.] - ETA: 0s - loss: 0.6295 - accuracy: 0.8345 Epoch 00159: val_accuracy did not improve from 0.84206 Epoch 00159: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 88s 20ms/step - loss: 0.6295 - accuracy: 0.8345 - val_loss: 0.3967 - val_accuracy: 0.8353 Epoch 160/200 4418/4418 [==============================] - ETA: 0s - loss: 0.6310 - accuracy: 0.8341 Epoch 00160: val_accuracy did not improve from 0.84206 Epoch 00160: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 88s 20ms/step - loss: 0.6310 - accuracy: 0.8341 - val_loss: 0.4299 - val_accuracy: 0.8265 Epoch 161/200 4416/4418 [============================>.] - ETA: 0s - loss: 0.6295 - accuracy: 0.8342 Epoch 00161: val_accuracy did not improve from 0.84206 Epoch 00161: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 92s 21ms/step - loss: 0.6295 - accuracy: 0.8342 - val_loss: 0.3974 - val_accuracy: 0.8326 Epoch 162/200 4418/4418 [==============================] - ETA: 0s - loss: 0.6293 - accuracy: 0.8352 Epoch 00162: val_accuracy did not improve from 0.84206 Epoch 00162: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 89s 20ms/step - loss: 0.6293 - accuracy: 0.8352 - val_loss: 0.4082 - val_accuracy: 0.8274 Epoch 163/200 4418/4418 [==============================] - ETA: 0s - loss: 0.6289 - accuracy: 0.8352 Epoch 00163: val_accuracy did not improve from 0.84206 Epoch 00163: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 89s 20ms/step - loss: 0.6289 - accuracy: 0.8352 - val_loss: 0.4052 - val_accuracy: 0.8339 Epoch 164/200 4416/4418 [============================>.] - ETA: 0s - loss: 0.6285 - accuracy: 0.8348 Epoch 00164: val_accuracy did not improve from 0.84206 Epoch 00164: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 90s 20ms/step - loss: 0.6285 - accuracy: 0.8348 - val_loss: 0.4084 - val_accuracy: 0.8325 Epoch 165/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6283 - accuracy: 0.8343 Epoch 00165: val_accuracy did not improve from 0.84206 Epoch 00165: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 89s 20ms/step - loss: 0.6283 - accuracy: 0.8343 - val_loss: 0.4223 - val_accuracy: 0.8211 Epoch 166/200 4416/4418 [============================>.] - ETA: 0s - loss: 0.6288 - accuracy: 0.8343 Epoch 00166: val_accuracy did not improve from 0.84206 Epoch 00166: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 88s 20ms/step - loss: 0.6288 - accuracy: 0.8343 - val_loss: 0.4051 - val_accuracy: 0.8294 Epoch 167/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6294 - accuracy: 0.8346 Epoch 00167: val_accuracy did not improve from 0.84206 Epoch 00167: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 89s 20ms/step - loss: 0.6294 - accuracy: 0.8346 - val_loss: 0.3929 - val_accuracy: 0.8386 Epoch 168/200 4418/4418 [==============================] - ETA: 0s - loss: 0.6292 - accuracy: 0.8348 Epoch 00168: val_accuracy did not improve from 0.84206 Epoch 00168: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 92s 21ms/step - loss: 0.6292 - accuracy: 0.8348 - val_loss: 0.4150 - val_accuracy: 0.8284 Epoch 169/200 4418/4418 [==============================] - ETA: 0s - loss: 0.6285 - accuracy: 0.8352 Epoch 00169: val_accuracy did not improve from 0.84206 Epoch 00169: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 90s 20ms/step - loss: 0.6285 - accuracy: 0.8352 - val_loss: 0.4115 - val_accuracy: 0.8324 Epoch 170/200 4418/4418 [==============================] - ETA: 0s - loss: 0.6281 - accuracy: 0.8352 Epoch 00170: val_accuracy did not improve from 0.84206 Epoch 00170: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 89s 20ms/step - loss: 0.6281 - accuracy: 0.8352 - val_loss: 0.4089 - val_accuracy: 0.8342 Epoch 171/200 4418/4418 [==============================] - ETA: 0s - loss: 0.6285 - accuracy: 0.8349 Epoch 00171: val_accuracy did not improve from 0.84206 Epoch 00171: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 89s 20ms/step - loss: 0.6285 - accuracy: 0.8349 - val_loss: 0.4131 - val_accuracy: 0.8276 Epoch 172/200 4418/4418 [==============================] - ETA: 0s - loss: 0.6277 - accuracy: 0.8347 Epoch 00172: val_accuracy did not improve from 0.84206 Epoch 00172: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 91s 21ms/step - loss: 0.6277 - accuracy: 0.8347 - val_loss: 0.4234 - val_accuracy: 0.8279 Epoch 173/200 4418/4418 [==============================] - ETA: 0s - loss: 0.6262 - accuracy: 0.8351 Epoch 00173: val_accuracy did not improve from 0.84206 Epoch 00173: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 88s 20ms/step - loss: 0.6262 - accuracy: 0.8351 - val_loss: 0.4133 - val_accuracy: 0.8338 Epoch 174/200 4418/4418 [==============================] - ETA: 0s - loss: 0.6286 - accuracy: 0.8348 Epoch 00174: val_accuracy did not improve from 0.84206 Epoch 00174: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 89s 20ms/step - loss: 0.6286 - accuracy: 0.8348 - val_loss: 0.4096 - val_accuracy: 0.8311 Epoch 175/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6266 - accuracy: 0.8355 Epoch 00175: val_accuracy did not improve from 0.84206 Epoch 00175: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 92s 21ms/step - loss: 0.6267 - accuracy: 0.8355 - val_loss: 0.4054 - val_accuracy: 0.8329 Epoch 176/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6280 - accuracy: 0.8363 Epoch 00176: val_accuracy did not improve from 0.84206 Epoch 00176: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 88s 20ms/step - loss: 0.6280 - accuracy: 0.8363 - val_loss: 0.3961 - val_accuracy: 0.8377 Epoch 177/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6273 - accuracy: 0.8351 Epoch 00177: val_accuracy did not improve from 0.84206 Epoch 00177: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 88s 20ms/step - loss: 0.6273 - accuracy: 0.8351 - val_loss: 0.4056 - val_accuracy: 0.8316 Epoch 178/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6245 - accuracy: 0.8363 Epoch 00178: val_accuracy did not improve from 0.84206 Epoch 00178: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 88s 20ms/step - loss: 0.6244 - accuracy: 0.8363 - val_loss: 0.4252 - val_accuracy: 0.8256 Epoch 179/200 4416/4418 [============================>.] - ETA: 0s - loss: 0.6264 - accuracy: 0.8347 Epoch 00179: val_accuracy did not improve from 0.84206 Epoch 00179: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 90s 20ms/step - loss: 0.6264 - accuracy: 0.8347 - val_loss: 0.3888 - val_accuracy: 0.8366 Epoch 180/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6264 - accuracy: 0.8359 Epoch 00180: val_accuracy did not improve from 0.84206 Epoch 00180: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 87s 20ms/step - loss: 0.6263 - accuracy: 0.8359 - val_loss: 0.4097 - val_accuracy: 0.8328 Epoch 181/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6250 - accuracy: 0.8357 Epoch 00181: val_accuracy did not improve from 0.84206 Epoch 00181: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 88s 20ms/step - loss: 0.6250 - accuracy: 0.8357 - val_loss: 0.3998 - val_accuracy: 0.8350 Epoch 182/200 4416/4418 [============================>.] - ETA: 0s - loss: 0.6262 - accuracy: 0.8353 Epoch 00182: val_accuracy did not improve from 0.84206 Epoch 00182: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 94s 21ms/step - loss: 0.6262 - accuracy: 0.8353 - val_loss: 0.3992 - val_accuracy: 0.8330 Epoch 183/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6253 - accuracy: 0.8361 Epoch 00183: val_accuracy did not improve from 0.84206 Epoch 00183: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 89s 20ms/step - loss: 0.6253 - accuracy: 0.8361 - val_loss: 0.3984 - val_accuracy: 0.8338 Epoch 184/200 4416/4418 [============================>.] - ETA: 0s - loss: 0.6249 - accuracy: 0.8344 Epoch 00184: val_accuracy did not improve from 0.84206 Epoch 00184: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 88s 20ms/step - loss: 0.6248 - accuracy: 0.8344 - val_loss: 0.3816 - val_accuracy: 0.8398 Epoch 185/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6247 - accuracy: 0.8360 Epoch 00185: val_accuracy did not improve from 0.84206 Epoch 00185: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 89s 20ms/step - loss: 0.6247 - accuracy: 0.8360 - val_loss: 0.3983 - val_accuracy: 0.8358 Epoch 186/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6259 - accuracy: 0.8364 Epoch 00186: val_accuracy did not improve from 0.84206 Epoch 00186: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 92s 21ms/step - loss: 0.6259 - accuracy: 0.8364 - val_loss: 0.4101 - val_accuracy: 0.8341 Epoch 187/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6236 - accuracy: 0.8365 Epoch 00187: val_accuracy did not improve from 0.84206 Epoch 00187: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 88s 20ms/step - loss: 0.6236 - accuracy: 0.8365 - val_loss: 0.4022 - val_accuracy: 0.8339 Epoch 188/200 4418/4418 [==============================] - ETA: 0s - loss: 0.6238 - accuracy: 0.8356 Epoch 00188: val_accuracy did not improve from 0.84206 Epoch 00188: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 88s 20ms/step - loss: 0.6238 - accuracy: 0.8356 - val_loss: 0.4080 - val_accuracy: 0.8297 Epoch 189/200 4416/4418 [============================>.] - ETA: 0s - loss: 0.6242 - accuracy: 0.8358 Epoch 00189: val_accuracy did not improve from 0.84206 Epoch 00189: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 91s 21ms/step - loss: 0.6242 - accuracy: 0.8358 - val_loss: 0.4019 - val_accuracy: 0.8321 Epoch 190/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6256 - accuracy: 0.8360 Epoch 00190: val_accuracy did not improve from 0.84206 Epoch 00190: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 90s 20ms/step - loss: 0.6256 - accuracy: 0.8360 - val_loss: 0.4053 - val_accuracy: 0.8296 Epoch 191/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6257 - accuracy: 0.8359 Epoch 00191: val_accuracy did not improve from 0.84206 Epoch 00191: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 88s 20ms/step - loss: 0.6257 - accuracy: 0.8359 - val_loss: 0.4088 - val_accuracy: 0.8237 Epoch 192/200 4418/4418 [==============================] - ETA: 0s - loss: 0.6245 - accuracy: 0.8359 Epoch 00192: val_accuracy did not improve from 0.84206 Epoch 00192: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 89s 20ms/step - loss: 0.6245 - accuracy: 0.8359 - val_loss: 0.4046 - val_accuracy: 0.8297 Epoch 193/200 4418/4418 [==============================] - ETA: 0s - loss: 0.6225 - accuracy: 0.8365 Epoch 00193: val_accuracy did not improve from 0.84206 Epoch 00193: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 90s 20ms/step - loss: 0.6225 - accuracy: 0.8365 - val_loss: 0.4106 - val_accuracy: 0.8312 Epoch 194/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6205 - accuracy: 0.8367 Epoch 00194: val_accuracy did not improve from 0.84206 Epoch 00194: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 88s 20ms/step - loss: 0.6205 - accuracy: 0.8368 - val_loss: 0.4002 - val_accuracy: 0.8335 Epoch 195/200 4418/4418 [==============================] - ETA: 0s - loss: 0.6225 - accuracy: 0.8363 Epoch 00195: val_accuracy did not improve from 0.84206 Epoch 00195: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 88s 20ms/step - loss: 0.6225 - accuracy: 0.8363 - val_loss: 0.4062 - val_accuracy: 0.8319 Epoch 196/200 4416/4418 [============================>.] - ETA: 0s - loss: 0.6223 - accuracy: 0.8366 Epoch 00196: val_accuracy did not improve from 0.84206 Epoch 00196: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 90s 20ms/step - loss: 0.6223 - accuracy: 0.8366 - val_loss: 0.4042 - val_accuracy: 0.8300 Epoch 197/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6217 - accuracy: 0.8372 Epoch 00197: val_accuracy did not improve from 0.84206 Epoch 00197: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 89s 20ms/step - loss: 0.6216 - accuracy: 0.8372 - val_loss: 0.4034 - val_accuracy: 0.8300 Epoch 198/200 4416/4418 [============================>.] - ETA: 0s - loss: 0.6221 - accuracy: 0.8361 Epoch 00198: val_accuracy did not improve from 0.84206 Epoch 00198: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 87s 20ms/step - loss: 0.6220 - accuracy: 0.8361 - val_loss: 0.4139 - val_accuracy: 0.8303 Epoch 199/200 4416/4418 [============================>.] - ETA: 0s - loss: 0.6209 - accuracy: 0.8369 Epoch 00199: val_accuracy did not improve from 0.84206 Epoch 00199: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 87s 20ms/step - loss: 0.6210 - accuracy: 0.8369 - val_loss: 0.4161 - val_accuracy: 0.8290 Epoch 200/200 4417/4418 [============================>.] - ETA: 0s - loss: 0.6207 - accuracy: 0.8363 Epoch 00200: val_accuracy did not improve from 0.84206 Epoch 00200: val_accuracy did not improve from 0.84206 4418/4418 [==============================] - 89s 20ms/step - loss: 0.6207 - accuracy: 0.8363 - val_loss: 0.4041 - val_accuracy: 0.8359
Next, let's plot the evolution of the metrics over the epochs of training:
In [ ]:
# list all data in history
print(history.history.keys())
# summarize history for accuracy
plt.plot(history.history['accuracy'])
plt.plot(history.history['val_accuracy'])
plt.title('model accuracy')
plt.ylabel('accuracy')
plt.xlabel('epoch')
plt.legend(['train', 'test'], loc='upper left')
plt.savefig("accuracy2.png")
plt.show()
# summarize history for loss
plt.plot(history.history['loss'])
plt.title('model train loss')
plt.ylabel('loss')
plt.xlabel('epoch')
plt.savefig("loss2-1.png")
plt.show()
plt.plot(history.history['val_loss'])
plt.title('model validation loss')
plt.ylabel('loss')
plt.xlabel('epoch')
plt.savefig("loss2-2.png")
plt.show()
dict_keys(['loss', 'accuracy', 'val_loss', 'val_accuracy'])
Let's load the model with the best performance:
In [ ]:
model.load_weights("/content/classifier_weights-improvement-94-0.84.hdf5")
Testing the Model¶
We will examine the accuracy of the model on the train set and the test set:
Let's generate a classification report that will give us insights on the performance of the model on both train set and test set:
In [ ]:
from sklearn.metrics import classification_report
y_train_pred=model.predict(muon_hits_x_train_values)
y_train_pred_best=np.argmax(y_train_pred,axis=1)
train_report=classification_report(y_encoded_train,y_train_pred_best)
In [ ]:
y_test_pred=model.predict(muon_hits_x_test_values)
y_test_pred_best=np.argmax(y_test_pred,axis=1)
test_report=classification_report(y_encoded_test,y_test_pred_best)
In [ ]:
print("Train report \n",train_report)
print("Test report \n",test_report)
Train report
precision recall f1-score support
0 0.99 0.90 0.94 2273653
1 0.65 0.59 0.62 447890
2 0.33 0.73 0.46 157029
3 0.36 0.66 0.47 66534
accuracy 0.84 2945106
macro avg 0.58 0.72 0.62 2945106
weighted avg 0.89 0.84 0.86 2945106
Test report
precision recall f1-score support
0 0.99 0.90 0.94 252629
1 0.64 0.58 0.61 49766
2 0.32 0.70 0.44 17448
3 0.33 0.61 0.43 7392
accuracy 0.84 327235
macro avg 0.57 0.70 0.61 327235
weighted avg 0.89 0.84 0.85 327235
That's impressive! It seems that adding class weights definitely reduced bias and we reached a weighted f1-score of 0.85 on the test data!
Experimenting with Regression¶
Instead of classifying into four group, let us try to directly predict the q/pT value for each muon momentum:
Splitting the Data¶
Let us first split the dataset into train and test data:
In [32]:
from sklearn.model_selection import train_test_split
muon_hits_x_train,muon_hits_x_test,pt_inverse_labels_train, pt_inverse_labels_test = train_test_split(muon_hits_df_x,pt_inverse_labels,test_size=0.1, random_state=42)
Next, let's create the pipeline for preprocessing the data for regression:
In [33]:
def preprocess_pipeline_reg(df_x,df_y):
#delete columns with missing values more than 70%
df_x=delete_columns(df_x,70)
#impute the other columns with missing values with their corresponding means
df_x=replace_missing_with_mean(df_x)
#standardize the data
from sklearn.preprocessing import StandardScaler
scaler=StandardScaler()
x=scaler.fit_transform(df_x)
return x,df_y.values
In [34]:
muon_hits_x_train_values,pt_inverse_labels_train_values=preprocess_pipeline_reg(muon_hits_x_train, pt_inverse_labels_train)
muon_hits_x_test_values,pt_inverse_labels_test_values=preprocess_pipeline_reg(muon_hits_x_test, pt_inverse_labels_test)
columns deleted: ['Phi angle1', 'Theta angle1', 'Bend angle1', 'Time1', 'Ring1', 'Front/Rear1'] columns deleted: ['Phi angle1', 'Theta angle1', 'Bend angle1', 'Time1', 'Ring1', 'Front/Rear1']
Building the Model¶
Let's first build and compile the model:
In [39]:
from tensorflow.keras import Sequential
from tensorflow.keras.layers import Dense,Dropout
reg_model=Sequential()
reg_model.add(Dense(512,input_dim=(muon_hits_x_train_values.shape[1]),activation='relu'))
reg_model.add(Dense(256,activation='relu'))
reg_model.add(Dropout(0.1))
reg_model.add(Dense(128,activation='relu'))
reg_model.add(Dense(64,activation='relu'))
reg_model.add(Dropout(0.1))
reg_model.add(Dense(32,activation='relu'))
reg_model.add(Dropout(0.1))
reg_model.add(Dense(1,activation='linear'))
reg_model.compile(optimizer=tf.keras.optimizers.Adam(1e-3),loss='mean_squared_error')
In [36]:
reg_model.summary()
Model: "sequential" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= dense (Dense) (None, 512) 17920 _________________________________________________________________ dense_1 (Dense) (None, 256) 131328 _________________________________________________________________ dropout (Dropout) (None, 256) 0 _________________________________________________________________ dense_2 (Dense) (None, 128) 32896 _________________________________________________________________ dense_3 (Dense) (None, 64) 8256 _________________________________________________________________ dropout_1 (Dropout) (None, 64) 0 _________________________________________________________________ dense_4 (Dense) (None, 32) 2080 _________________________________________________________________ dropout_2 (Dropout) (None, 32) 0 _________________________________________________________________ dense_5 (Dense) (None, 1) 33 ================================================================= Total params: 192,513 Trainable params: 192,513 Non-trainable params: 0 _________________________________________________________________
Next, we will set some callbacks for the model including a Checkpoint callback and Early Stopping callback that will both monitor the model's validation mean square error:
In [40]:
from tensorflow.keras.callbacks import ModelCheckpoint
from tensorflow.keras.callbacks import EarlyStopping
filepath="reg_weights2-improvement-{epoch:02d}-{val_loss:.6f}.hdf5"
checkpoint1 = ModelCheckpoint(filepath, monitor='val_loss', verbose=1, save_best_only=True, mode='min')
checkpoint2= EarlyStopping( monitor='val_loss', patience=3)
callbacks_list = [checkpoint1,checkpoint1]
In [41]:
reg_history = reg_model.fit(muon_hits_x_train_values, pt_inverse_labels_train_values, validation_split=0.25, epochs=200, batch_size=2000,callbacks=callbacks_list)
Epoch 1/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0067 Epoch 00001: val_loss improved from inf to 0.00518, saving model to reg_weights2-improvement-01-0.005183.hdf5 Epoch 00001: val_loss did not improve from 0.00518 1105/1105 [==============================] - 56s 51ms/step - loss: 0.0067 - val_loss: 0.0052 Epoch 2/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0050 Epoch 00002: val_loss improved from 0.00518 to 0.00482, saving model to reg_weights2-improvement-02-0.004817.hdf5 Epoch 00002: val_loss did not improve from 0.00482 1105/1105 [==============================] - 58s 52ms/step - loss: 0.0050 - val_loss: 0.0048 Epoch 3/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0047 Epoch 00003: val_loss improved from 0.00482 to 0.00456, saving model to reg_weights2-improvement-03-0.004556.hdf5 Epoch 00003: val_loss did not improve from 0.00456 1105/1105 [==============================] - 56s 51ms/step - loss: 0.0047 - val_loss: 0.0046 Epoch 4/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0044 Epoch 00004: val_loss improved from 0.00456 to 0.00426, saving model to reg_weights2-improvement-04-0.004255.hdf5 Epoch 00004: val_loss did not improve from 0.00426 1105/1105 [==============================] - 57s 51ms/step - loss: 0.0044 - val_loss: 0.0043 Epoch 5/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0043 Epoch 00005: val_loss improved from 0.00426 to 0.00403, saving model to reg_weights2-improvement-05-0.004026.hdf5 Epoch 00005: val_loss did not improve from 0.00403 1105/1105 [==============================] - 56s 51ms/step - loss: 0.0043 - val_loss: 0.0040 Epoch 6/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0041 Epoch 00006: val_loss improved from 0.00403 to 0.00402, saving model to reg_weights2-improvement-06-0.004021.hdf5 Epoch 00006: val_loss did not improve from 0.00402 1105/1105 [==============================] - 56s 51ms/step - loss: 0.0041 - val_loss: 0.0040 Epoch 7/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0040 Epoch 00007: val_loss did not improve from 0.00402 Epoch 00007: val_loss did not improve from 0.00402 1105/1105 [==============================] - 56s 51ms/step - loss: 0.0040 - val_loss: 0.0041 Epoch 8/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0039 Epoch 00008: val_loss improved from 0.00402 to 0.00387, saving model to reg_weights2-improvement-08-0.003872.hdf5 Epoch 00008: val_loss did not improve from 0.00387 1105/1105 [==============================] - 57s 51ms/step - loss: 0.0039 - val_loss: 0.0039 Epoch 9/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0038 Epoch 00009: val_loss improved from 0.00387 to 0.00366, saving model to reg_weights2-improvement-09-0.003664.hdf5 Epoch 00009: val_loss did not improve from 0.00366 1105/1105 [==============================] - 56s 51ms/step - loss: 0.0038 - val_loss: 0.0037 Epoch 10/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0037 Epoch 00010: val_loss did not improve from 0.00366 Epoch 00010: val_loss did not improve from 0.00366 1105/1105 [==============================] - 57s 51ms/step - loss: 0.0037 - val_loss: 0.0037 Epoch 11/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0036 Epoch 00011: val_loss did not improve from 0.00366 Epoch 00011: val_loss did not improve from 0.00366 1105/1105 [==============================] - 56s 51ms/step - loss: 0.0036 - val_loss: 0.0038 Epoch 12/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0036 Epoch 00012: val_loss improved from 0.00366 to 0.00335, saving model to reg_weights2-improvement-12-0.003351.hdf5 Epoch 00012: val_loss did not improve from 0.00335 1105/1105 [==============================] - 57s 51ms/step - loss: 0.0036 - val_loss: 0.0034 Epoch 13/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0035 Epoch 00013: val_loss did not improve from 0.00335 Epoch 00013: val_loss did not improve from 0.00335 1105/1105 [==============================] - 57s 52ms/step - loss: 0.0035 - val_loss: 0.0039 Epoch 14/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0035 Epoch 00014: val_loss did not improve from 0.00335 Epoch 00014: val_loss did not improve from 0.00335 1105/1105 [==============================] - 56s 51ms/step - loss: 0.0035 - val_loss: 0.0036 Epoch 15/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0034 Epoch 00015: val_loss did not improve from 0.00335 Epoch 00015: val_loss did not improve from 0.00335 1105/1105 [==============================] - 56s 51ms/step - loss: 0.0034 - val_loss: 0.0039 Epoch 16/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0034 Epoch 00016: val_loss did not improve from 0.00335 Epoch 00016: val_loss did not improve from 0.00335 1105/1105 [==============================] - 56s 51ms/step - loss: 0.0034 - val_loss: 0.0035 Epoch 17/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0034 Epoch 00017: val_loss did not improve from 0.00335 Epoch 00017: val_loss did not improve from 0.00335 1105/1105 [==============================] - 56s 51ms/step - loss: 0.0034 - val_loss: 0.0036 Epoch 18/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0033 Epoch 00018: val_loss improved from 0.00335 to 0.00332, saving model to reg_weights2-improvement-18-0.003321.hdf5 Epoch 00018: val_loss did not improve from 0.00332 1105/1105 [==============================] - 56s 51ms/step - loss: 0.0033 - val_loss: 0.0033 Epoch 19/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0033 Epoch 00019: val_loss improved from 0.00332 to 0.00324, saving model to reg_weights2-improvement-19-0.003239.hdf5 Epoch 00019: val_loss did not improve from 0.00324 1105/1105 [==============================] - 57s 51ms/step - loss: 0.0033 - val_loss: 0.0032 Epoch 20/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0033 Epoch 00020: val_loss did not improve from 0.00324 Epoch 00020: val_loss did not improve from 0.00324 1105/1105 [==============================] - 56s 51ms/step - loss: 0.0033 - val_loss: 0.0034 Epoch 21/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0033 Epoch 00021: val_loss did not improve from 0.00324 Epoch 00021: val_loss did not improve from 0.00324 1105/1105 [==============================] - 57s 51ms/step - loss: 0.0033 - val_loss: 0.0035 Epoch 22/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0032 Epoch 00022: val_loss did not improve from 0.00324 Epoch 00022: val_loss did not improve from 0.00324 1105/1105 [==============================] - 56s 51ms/step - loss: 0.0032 - val_loss: 0.0034 Epoch 23/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0032 Epoch 00023: val_loss did not improve from 0.00324 Epoch 00023: val_loss did not improve from 0.00324 1105/1105 [==============================] - 56s 51ms/step - loss: 0.0032 - val_loss: 0.0033 Epoch 24/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0032 Epoch 00024: val_loss did not improve from 0.00324 Epoch 00024: val_loss did not improve from 0.00324 1105/1105 [==============================] - 57s 51ms/step - loss: 0.0032 - val_loss: 0.0034 Epoch 25/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0031 Epoch 00025: val_loss did not improve from 0.00324 Epoch 00025: val_loss did not improve from 0.00324 1105/1105 [==============================] - 57s 51ms/step - loss: 0.0031 - val_loss: 0.0035 Epoch 26/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0031 Epoch 00026: val_loss improved from 0.00324 to 0.00321, saving model to reg_weights2-improvement-26-0.003208.hdf5 Epoch 00026: val_loss did not improve from 0.00321 1105/1105 [==============================] - 57s 52ms/step - loss: 0.0031 - val_loss: 0.0032 Epoch 27/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0031 Epoch 00027: val_loss did not improve from 0.00321 Epoch 00027: val_loss did not improve from 0.00321 1105/1105 [==============================] - 57s 52ms/step - loss: 0.0031 - val_loss: 0.0035 Epoch 28/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0031 Epoch 00028: val_loss did not improve from 0.00321 Epoch 00028: val_loss did not improve from 0.00321 1105/1105 [==============================] - 57s 51ms/step - loss: 0.0031 - val_loss: 0.0032 Epoch 29/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0030 Epoch 00029: val_loss did not improve from 0.00321 Epoch 00029: val_loss did not improve from 0.00321 1105/1105 [==============================] - 57s 52ms/step - loss: 0.0030 - val_loss: 0.0033 Epoch 30/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0030 Epoch 00030: val_loss improved from 0.00321 to 0.00312, saving model to reg_weights2-improvement-30-0.003124.hdf5 Epoch 00030: val_loss did not improve from 0.00312 1105/1105 [==============================] - 57s 52ms/step - loss: 0.0030 - val_loss: 0.0031 Epoch 31/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0030 Epoch 00031: val_loss did not improve from 0.00312 Epoch 00031: val_loss did not improve from 0.00312 1105/1105 [==============================] - 56s 51ms/step - loss: 0.0030 - val_loss: 0.0032 Epoch 32/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0030 Epoch 00032: val_loss did not improve from 0.00312 Epoch 00032: val_loss did not improve from 0.00312 1105/1105 [==============================] - 57s 52ms/step - loss: 0.0030 - val_loss: 0.0031 Epoch 33/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0029 Epoch 00033: val_loss did not improve from 0.00312 Epoch 00033: val_loss did not improve from 0.00312 1105/1105 [==============================] - 57s 52ms/step - loss: 0.0029 - val_loss: 0.0034 Epoch 34/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0029 Epoch 00034: val_loss did not improve from 0.00312 Epoch 00034: val_loss did not improve from 0.00312 1105/1105 [==============================] - 57s 51ms/step - loss: 0.0029 - val_loss: 0.0032 Epoch 35/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0029 Epoch 00035: val_loss improved from 0.00312 to 0.00302, saving model to reg_weights2-improvement-35-0.003019.hdf5 Epoch 00035: val_loss did not improve from 0.00302 1105/1105 [==============================] - 58s 52ms/step - loss: 0.0029 - val_loss: 0.0030 Epoch 36/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0029 Epoch 00036: val_loss improved from 0.00302 to 0.00289, saving model to reg_weights2-improvement-36-0.002890.hdf5 Epoch 00036: val_loss did not improve from 0.00289 1105/1105 [==============================] - 57s 51ms/step - loss: 0.0029 - val_loss: 0.0029 Epoch 37/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0028 Epoch 00037: val_loss did not improve from 0.00289 Epoch 00037: val_loss did not improve from 0.00289 1105/1105 [==============================] - 57s 52ms/step - loss: 0.0028 - val_loss: 0.0030 Epoch 38/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0028 Epoch 00038: val_loss did not improve from 0.00289 Epoch 00038: val_loss did not improve from 0.00289 1105/1105 [==============================] - 57s 52ms/step - loss: 0.0028 - val_loss: 0.0030 Epoch 39/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0028 Epoch 00039: val_loss did not improve from 0.00289 Epoch 00039: val_loss did not improve from 0.00289 1105/1105 [==============================] - 57s 52ms/step - loss: 0.0028 - val_loss: 0.0030 Epoch 40/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0028 Epoch 00040: val_loss did not improve from 0.00289 Epoch 00040: val_loss did not improve from 0.00289 1105/1105 [==============================] - 57s 52ms/step - loss: 0.0028 - val_loss: 0.0030 Epoch 41/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0028 Epoch 00041: val_loss did not improve from 0.00289 Epoch 00041: val_loss did not improve from 0.00289 1105/1105 [==============================] - 58s 52ms/step - loss: 0.0028 - val_loss: 0.0033 Epoch 42/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0027 Epoch 00042: val_loss did not improve from 0.00289 Epoch 00042: val_loss did not improve from 0.00289 1105/1105 [==============================] - 57s 52ms/step - loss: 0.0027 - val_loss: 0.0031 Epoch 43/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0027 Epoch 00043: val_loss improved from 0.00289 to 0.00279, saving model to reg_weights2-improvement-43-0.002789.hdf5 Epoch 00043: val_loss did not improve from 0.00279 1105/1105 [==============================] - 57s 52ms/step - loss: 0.0027 - val_loss: 0.0028 Epoch 44/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0027 Epoch 00044: val_loss did not improve from 0.00279 Epoch 00044: val_loss did not improve from 0.00279 1105/1105 [==============================] - 57s 52ms/step - loss: 0.0027 - val_loss: 0.0031 Epoch 45/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0027 Epoch 00045: val_loss did not improve from 0.00279 Epoch 00045: val_loss did not improve from 0.00279 1105/1105 [==============================] - 57s 52ms/step - loss: 0.0027 - val_loss: 0.0030 Epoch 46/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0027 Epoch 00046: val_loss did not improve from 0.00279 Epoch 00046: val_loss did not improve from 0.00279 1105/1105 [==============================] - 58s 53ms/step - loss: 0.0027 - val_loss: 0.0030 Epoch 47/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0026 Epoch 00047: val_loss did not improve from 0.00279 Epoch 00047: val_loss did not improve from 0.00279 1105/1105 [==============================] - 58s 52ms/step - loss: 0.0026 - val_loss: 0.0031 Epoch 48/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0026 Epoch 00048: val_loss did not improve from 0.00279 Epoch 00048: val_loss did not improve from 0.00279 1105/1105 [==============================] - 58s 52ms/step - loss: 0.0026 - val_loss: 0.0032 Epoch 49/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0026 Epoch 00049: val_loss did not improve from 0.00279 Epoch 00049: val_loss did not improve from 0.00279 1105/1105 [==============================] - 57s 52ms/step - loss: 0.0026 - val_loss: 0.0031 Epoch 50/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0026 Epoch 00050: val_loss did not improve from 0.00279 Epoch 00050: val_loss did not improve from 0.00279 1105/1105 [==============================] - 57s 52ms/step - loss: 0.0026 - val_loss: 0.0030 Epoch 51/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0026 Epoch 00051: val_loss did not improve from 0.00279 Epoch 00051: val_loss did not improve from 0.00279 1105/1105 [==============================] - 57s 52ms/step - loss: 0.0026 - val_loss: 0.0030 Epoch 52/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0025 Epoch 00052: val_loss did not improve from 0.00279 Epoch 00052: val_loss did not improve from 0.00279 1105/1105 [==============================] - 57s 52ms/step - loss: 0.0025 - val_loss: 0.0030 Epoch 53/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0025 Epoch 00053: val_loss did not improve from 0.00279 Epoch 00053: val_loss did not improve from 0.00279 1105/1105 [==============================] - 58s 52ms/step - loss: 0.0025 - val_loss: 0.0028 Epoch 54/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0025 Epoch 00054: val_loss did not improve from 0.00279 Epoch 00054: val_loss did not improve from 0.00279 1105/1105 [==============================] - 58s 52ms/step - loss: 0.0025 - val_loss: 0.0028 Epoch 55/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0025 Epoch 00055: val_loss did not improve from 0.00279 Epoch 00055: val_loss did not improve from 0.00279 1105/1105 [==============================] - 57s 52ms/step - loss: 0.0025 - val_loss: 0.0030 Epoch 56/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0024 Epoch 00056: val_loss did not improve from 0.00279 Epoch 00056: val_loss did not improve from 0.00279 1105/1105 [==============================] - 57s 51ms/step - loss: 0.0024 - val_loss: 0.0028 Epoch 57/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0024 Epoch 00057: val_loss did not improve from 0.00279 Epoch 00057: val_loss did not improve from 0.00279 1105/1105 [==============================] - 58s 53ms/step - loss: 0.0024 - val_loss: 0.0030 Epoch 58/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0024 Epoch 00058: val_loss did not improve from 0.00279 Epoch 00058: val_loss did not improve from 0.00279 1105/1105 [==============================] - 58s 52ms/step - loss: 0.0024 - val_loss: 0.0028 Epoch 59/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0023 Epoch 00059: val_loss did not improve from 0.00279 Epoch 00059: val_loss did not improve from 0.00279 1105/1105 [==============================] - 57s 52ms/step - loss: 0.0023 - val_loss: 0.0029 Epoch 60/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0023 Epoch 00060: val_loss did not improve from 0.00279 Epoch 00060: val_loss did not improve from 0.00279 1105/1105 [==============================] - 57s 52ms/step - loss: 0.0023 - val_loss: 0.0031 Epoch 61/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0023 Epoch 00061: val_loss did not improve from 0.00279 Epoch 00061: val_loss did not improve from 0.00279 1105/1105 [==============================] - 58s 52ms/step - loss: 0.0023 - val_loss: 0.0028 Epoch 62/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0023 Epoch 00062: val_loss improved from 0.00279 to 0.00251, saving model to reg_weights2-improvement-62-0.002510.hdf5 Epoch 00062: val_loss did not improve from 0.00251 1105/1105 [==============================] - 58s 53ms/step - loss: 0.0023 - val_loss: 0.0025 Epoch 63/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0022 Epoch 00063: val_loss did not improve from 0.00251 Epoch 00063: val_loss did not improve from 0.00251 1105/1105 [==============================] - 57s 52ms/step - loss: 0.0022 - val_loss: 0.0028 Epoch 64/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0022 Epoch 00064: val_loss did not improve from 0.00251 Epoch 00064: val_loss did not improve from 0.00251 1105/1105 [==============================] - 58s 52ms/step - loss: 0.0022 - val_loss: 0.0029 Epoch 65/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0022 Epoch 00065: val_loss did not improve from 0.00251 Epoch 00065: val_loss did not improve from 0.00251 1105/1105 [==============================] - 57s 52ms/step - loss: 0.0022 - val_loss: 0.0028 Epoch 66/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0022 Epoch 00066: val_loss did not improve from 0.00251 Epoch 00066: val_loss did not improve from 0.00251 1105/1105 [==============================] - 57s 52ms/step - loss: 0.0022 - val_loss: 0.0026 Epoch 67/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0022 Epoch 00067: val_loss did not improve from 0.00251 Epoch 00067: val_loss did not improve from 0.00251 1105/1105 [==============================] - 58s 52ms/step - loss: 0.0022 - val_loss: 0.0027 Epoch 68/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0022 Epoch 00068: val_loss did not improve from 0.00251 Epoch 00068: val_loss did not improve from 0.00251 1105/1105 [==============================] - 58s 53ms/step - loss: 0.0022 - val_loss: 0.0026 Epoch 69/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0021 Epoch 00069: val_loss did not improve from 0.00251 Epoch 00069: val_loss did not improve from 0.00251 1105/1105 [==============================] - 58s 53ms/step - loss: 0.0021 - val_loss: 0.0029 Epoch 70/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0021 Epoch 00070: val_loss did not improve from 0.00251 Epoch 00070: val_loss did not improve from 0.00251 1105/1105 [==============================] - 58s 52ms/step - loss: 0.0021 - val_loss: 0.0028 Epoch 71/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0021 Epoch 00071: val_loss did not improve from 0.00251 Epoch 00071: val_loss did not improve from 0.00251 1105/1105 [==============================] - 57s 52ms/step - loss: 0.0021 - val_loss: 0.0026 Epoch 72/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0021 Epoch 00072: val_loss did not improve from 0.00251 Epoch 00072: val_loss did not improve from 0.00251 1105/1105 [==============================] - 58s 52ms/step - loss: 0.0021 - val_loss: 0.0026 Epoch 73/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0021 Epoch 00073: val_loss did not improve from 0.00251 Epoch 00073: val_loss did not improve from 0.00251 1105/1105 [==============================] - 58s 53ms/step - loss: 0.0021 - val_loss: 0.0026 Epoch 74/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0021 Epoch 00074: val_loss did not improve from 0.00251 Epoch 00074: val_loss did not improve from 0.00251 1105/1105 [==============================] - 58s 53ms/step - loss: 0.0021 - val_loss: 0.0026 Epoch 75/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0021 Epoch 00075: val_loss improved from 0.00251 to 0.00243, saving model to reg_weights2-improvement-75-0.002429.hdf5 Epoch 00075: val_loss did not improve from 0.00243 1105/1105 [==============================] - 58s 53ms/step - loss: 0.0021 - val_loss: 0.0024 Epoch 76/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0020 Epoch 00076: val_loss did not improve from 0.00243 Epoch 00076: val_loss did not improve from 0.00243 1105/1105 [==============================] - 59s 53ms/step - loss: 0.0020 - val_loss: 0.0026 Epoch 77/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0020 Epoch 00077: val_loss improved from 0.00243 to 0.00242, saving model to reg_weights2-improvement-77-0.002417.hdf5 Epoch 00077: val_loss did not improve from 0.00242 1105/1105 [==============================] - 59s 53ms/step - loss: 0.0020 - val_loss: 0.0024 Epoch 78/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0020 Epoch 00078: val_loss did not improve from 0.00242 Epoch 00078: val_loss did not improve from 0.00242 1105/1105 [==============================] - 60s 54ms/step - loss: 0.0020 - val_loss: 0.0025 Epoch 79/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0020 Epoch 00079: val_loss did not improve from 0.00242 Epoch 00079: val_loss did not improve from 0.00242 1105/1105 [==============================] - 59s 54ms/step - loss: 0.0020 - val_loss: 0.0026 Epoch 80/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0020 Epoch 00080: val_loss did not improve from 0.00242 Epoch 00080: val_loss did not improve from 0.00242 1105/1105 [==============================] - 59s 54ms/step - loss: 0.0020 - val_loss: 0.0026 Epoch 81/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0020 Epoch 00081: val_loss improved from 0.00242 to 0.00223, saving model to reg_weights2-improvement-81-0.002230.hdf5 Epoch 00081: val_loss did not improve from 0.00223 1105/1105 [==============================] - 59s 53ms/step - loss: 0.0020 - val_loss: 0.0022 Epoch 82/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0020 Epoch 00082: val_loss did not improve from 0.00223 Epoch 00082: val_loss did not improve from 0.00223 1105/1105 [==============================] - 59s 53ms/step - loss: 0.0020 - val_loss: 0.0025 Epoch 83/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0020 Epoch 00083: val_loss did not improve from 0.00223 Epoch 00083: val_loss did not improve from 0.00223 1105/1105 [==============================] - 59s 53ms/step - loss: 0.0020 - val_loss: 0.0025 Epoch 84/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0019 Epoch 00084: val_loss did not improve from 0.00223 Epoch 00084: val_loss did not improve from 0.00223 1105/1105 [==============================] - 60s 54ms/step - loss: 0.0019 - val_loss: 0.0026 Epoch 85/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0019 Epoch 00085: val_loss did not improve from 0.00223 Epoch 00085: val_loss did not improve from 0.00223 1105/1105 [==============================] - 60s 54ms/step - loss: 0.0019 - val_loss: 0.0024 Epoch 86/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0019 Epoch 00086: val_loss did not improve from 0.00223 Epoch 00086: val_loss did not improve from 0.00223 1105/1105 [==============================] - 60s 54ms/step - loss: 0.0019 - val_loss: 0.0026 Epoch 87/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0019 Epoch 00087: val_loss did not improve from 0.00223 Epoch 00087: val_loss did not improve from 0.00223 1105/1105 [==============================] - 60s 54ms/step - loss: 0.0019 - val_loss: 0.0027 Epoch 88/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0019 Epoch 00088: val_loss did not improve from 0.00223 Epoch 00088: val_loss did not improve from 0.00223 1105/1105 [==============================] - 60s 55ms/step - loss: 0.0019 - val_loss: 0.0025 Epoch 89/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0019 Epoch 00089: val_loss did not improve from 0.00223 Epoch 00089: val_loss did not improve from 0.00223 1105/1105 [==============================] - 61s 55ms/step - loss: 0.0019 - val_loss: 0.0025 Epoch 90/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0019 Epoch 00090: val_loss did not improve from 0.00223 Epoch 00090: val_loss did not improve from 0.00223 1105/1105 [==============================] - 58s 52ms/step - loss: 0.0019 - val_loss: 0.0028 Epoch 91/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0018 Epoch 00091: val_loss did not improve from 0.00223 Epoch 00091: val_loss did not improve from 0.00223 1105/1105 [==============================] - 59s 53ms/step - loss: 0.0018 - val_loss: 0.0027 Epoch 92/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0018 Epoch 00092: val_loss did not improve from 0.00223 Epoch 00092: val_loss did not improve from 0.00223 1105/1105 [==============================] - 58s 53ms/step - loss: 0.0018 - val_loss: 0.0026 Epoch 93/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0018 Epoch 00093: val_loss did not improve from 0.00223 Epoch 00093: val_loss did not improve from 0.00223 1105/1105 [==============================] - 58s 52ms/step - loss: 0.0018 - val_loss: 0.0026 Epoch 94/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0018 Epoch 00094: val_loss did not improve from 0.00223 Epoch 00094: val_loss did not improve from 0.00223 1105/1105 [==============================] - 58s 53ms/step - loss: 0.0018 - val_loss: 0.0025 Epoch 95/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0018 Epoch 00095: val_loss did not improve from 0.00223 Epoch 00095: val_loss did not improve from 0.00223 1105/1105 [==============================] - 58s 52ms/step - loss: 0.0018 - val_loss: 0.0026 Epoch 96/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0018 Epoch 00096: val_loss did not improve from 0.00223 Epoch 00096: val_loss did not improve from 0.00223 1105/1105 [==============================] - 57s 52ms/step - loss: 0.0018 - val_loss: 0.0026 Epoch 97/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0018 Epoch 00097: val_loss did not improve from 0.00223 Epoch 00097: val_loss did not improve from 0.00223 1105/1105 [==============================] - 58s 53ms/step - loss: 0.0018 - val_loss: 0.0025 Epoch 98/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0017 Epoch 00098: val_loss did not improve from 0.00223 Epoch 00098: val_loss did not improve from 0.00223 1105/1105 [==============================] - 60s 54ms/step - loss: 0.0017 - val_loss: 0.0024 Epoch 99/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0017 Epoch 00099: val_loss did not improve from 0.00223 Epoch 00099: val_loss did not improve from 0.00223 1105/1105 [==============================] - 59s 53ms/step - loss: 0.0017 - val_loss: 0.0022 Epoch 100/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0017 Epoch 00100: val_loss did not improve from 0.00223 Epoch 00100: val_loss did not improve from 0.00223 1105/1105 [==============================] - 59s 54ms/step - loss: 0.0017 - val_loss: 0.0026 Epoch 101/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0017 Epoch 00101: val_loss did not improve from 0.00223 Epoch 00101: val_loss did not improve from 0.00223 1105/1105 [==============================] - 58s 52ms/step - loss: 0.0017 - val_loss: 0.0025 Epoch 102/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0017 Epoch 00102: val_loss did not improve from 0.00223 Epoch 00102: val_loss did not improve from 0.00223 1105/1105 [==============================] - 57s 52ms/step - loss: 0.0017 - val_loss: 0.0028 Epoch 103/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0017 Epoch 00103: val_loss did not improve from 0.00223 Epoch 00103: val_loss did not improve from 0.00223 1105/1105 [==============================] - 57s 52ms/step - loss: 0.0017 - val_loss: 0.0028 Epoch 104/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0017 Epoch 00104: val_loss did not improve from 0.00223 Epoch 00104: val_loss did not improve from 0.00223 1105/1105 [==============================] - 58s 52ms/step - loss: 0.0017 - val_loss: 0.0026 Epoch 105/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0017 Epoch 00105: val_loss did not improve from 0.00223 Epoch 00105: val_loss did not improve from 0.00223 1105/1105 [==============================] - 59s 54ms/step - loss: 0.0017 - val_loss: 0.0027 Epoch 106/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0017 Epoch 00106: val_loss did not improve from 0.00223 Epoch 00106: val_loss did not improve from 0.00223 1105/1105 [==============================] - 58s 53ms/step - loss: 0.0017 - val_loss: 0.0025 Epoch 107/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0017 Epoch 00107: val_loss did not improve from 0.00223 Epoch 00107: val_loss did not improve from 0.00223 1105/1105 [==============================] - 57s 52ms/step - loss: 0.0017 - val_loss: 0.0023 Epoch 108/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0017 Epoch 00108: val_loss did not improve from 0.00223 Epoch 00108: val_loss did not improve from 0.00223 1105/1105 [==============================] - 58s 53ms/step - loss: 0.0017 - val_loss: 0.0024 Epoch 109/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0016 Epoch 00109: val_loss did not improve from 0.00223 Epoch 00109: val_loss did not improve from 0.00223 1105/1105 [==============================] - 57s 52ms/step - loss: 0.0016 - val_loss: 0.0027 Epoch 110/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0016 Epoch 00110: val_loss improved from 0.00223 to 0.00219, saving model to reg_weights2-improvement-110-0.002186.hdf5 Epoch 00110: val_loss did not improve from 0.00219 1105/1105 [==============================] - 59s 53ms/step - loss: 0.0016 - val_loss: 0.0022 Epoch 111/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0016 Epoch 00111: val_loss did not improve from 0.00219 Epoch 00111: val_loss did not improve from 0.00219 1105/1105 [==============================] - 58s 52ms/step - loss: 0.0016 - val_loss: 0.0022 Epoch 112/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0016 Epoch 00112: val_loss did not improve from 0.00219 Epoch 00112: val_loss did not improve from 0.00219 1105/1105 [==============================] - 57s 52ms/step - loss: 0.0016 - val_loss: 0.0026 Epoch 113/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0016 Epoch 00113: val_loss did not improve from 0.00219 Epoch 00113: val_loss did not improve from 0.00219 1105/1105 [==============================] - 56s 51ms/step - loss: 0.0016 - val_loss: 0.0026 Epoch 114/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0016 Epoch 00114: val_loss did not improve from 0.00219 Epoch 00114: val_loss did not improve from 0.00219 1105/1105 [==============================] - 56s 51ms/step - loss: 0.0016 - val_loss: 0.0024 Epoch 115/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0016 Epoch 00115: val_loss did not improve from 0.00219 Epoch 00115: val_loss did not improve from 0.00219 1105/1105 [==============================] - 57s 51ms/step - loss: 0.0016 - val_loss: 0.0022 Epoch 116/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0016 Epoch 00116: val_loss did not improve from 0.00219 Epoch 00116: val_loss did not improve from 0.00219 1105/1105 [==============================] - 57s 52ms/step - loss: 0.0016 - val_loss: 0.0025 Epoch 117/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0015 Epoch 00117: val_loss did not improve from 0.00219 Epoch 00117: val_loss did not improve from 0.00219 1105/1105 [==============================] - 57s 51ms/step - loss: 0.0015 - val_loss: 0.0026 Epoch 118/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0015 Epoch 00118: val_loss did not improve from 0.00219 Epoch 00118: val_loss did not improve from 0.00219 1105/1105 [==============================] - 57s 51ms/step - loss: 0.0015 - val_loss: 0.0024 Epoch 119/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0015 Epoch 00119: val_loss did not improve from 0.00219 Epoch 00119: val_loss did not improve from 0.00219 1105/1105 [==============================] - 57s 51ms/step - loss: 0.0015 - val_loss: 0.0026 Epoch 120/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0015 Epoch 00120: val_loss did not improve from 0.00219 Epoch 00120: val_loss did not improve from 0.00219 1105/1105 [==============================] - 57s 51ms/step - loss: 0.0015 - val_loss: 0.0024 Epoch 121/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0015 Epoch 00121: val_loss did not improve from 0.00219 Epoch 00121: val_loss did not improve from 0.00219 1105/1105 [==============================] - 57s 52ms/step - loss: 0.0015 - val_loss: 0.0026 Epoch 122/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0015 Epoch 00122: val_loss did not improve from 0.00219 Epoch 00122: val_loss did not improve from 0.00219 1105/1105 [==============================] - 57s 52ms/step - loss: 0.0015 - val_loss: 0.0024 Epoch 123/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0015 Epoch 00123: val_loss did not improve from 0.00219 Epoch 00123: val_loss did not improve from 0.00219 1105/1105 [==============================] - 57s 52ms/step - loss: 0.0015 - val_loss: 0.0026 Epoch 124/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0015 Epoch 00124: val_loss did not improve from 0.00219 Epoch 00124: val_loss did not improve from 0.00219 1105/1105 [==============================] - 57s 52ms/step - loss: 0.0015 - val_loss: 0.0023 Epoch 125/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0015 Epoch 00125: val_loss did not improve from 0.00219 Epoch 00125: val_loss did not improve from 0.00219 1105/1105 [==============================] - 57s 52ms/step - loss: 0.0015 - val_loss: 0.0025 Epoch 126/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0015 Epoch 00126: val_loss did not improve from 0.00219 Epoch 00126: val_loss did not improve from 0.00219 1105/1105 [==============================] - 57s 51ms/step - loss: 0.0015 - val_loss: 0.0028 Epoch 127/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0015 Epoch 00127: val_loss did not improve from 0.00219 Epoch 00127: val_loss did not improve from 0.00219 1105/1105 [==============================] - 58s 52ms/step - loss: 0.0015 - val_loss: 0.0024 Epoch 128/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0015 Epoch 00128: val_loss did not improve from 0.00219 Epoch 00128: val_loss did not improve from 0.00219 1105/1105 [==============================] - 56s 51ms/step - loss: 0.0015 - val_loss: 0.0025 Epoch 129/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0015 Epoch 00129: val_loss did not improve from 0.00219 Epoch 00129: val_loss did not improve from 0.00219 1105/1105 [==============================] - 57s 52ms/step - loss: 0.0015 - val_loss: 0.0025 Epoch 130/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0015 Epoch 00130: val_loss did not improve from 0.00219 Epoch 00130: val_loss did not improve from 0.00219 1105/1105 [==============================] - 58s 52ms/step - loss: 0.0015 - val_loss: 0.0023 Epoch 131/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0014 Epoch 00131: val_loss did not improve from 0.00219 Epoch 00131: val_loss did not improve from 0.00219 1105/1105 [==============================] - 57s 51ms/step - loss: 0.0014 - val_loss: 0.0025 Epoch 132/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0014 Epoch 00132: val_loss did not improve from 0.00219 Epoch 00132: val_loss did not improve from 0.00219 1105/1105 [==============================] - 56s 51ms/step - loss: 0.0014 - val_loss: 0.0024 Epoch 133/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0014 Epoch 00133: val_loss did not improve from 0.00219 Epoch 00133: val_loss did not improve from 0.00219 1105/1105 [==============================] - 56s 51ms/step - loss: 0.0014 - val_loss: 0.0024 Epoch 134/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0014 Epoch 00134: val_loss did not improve from 0.00219 Epoch 00134: val_loss did not improve from 0.00219 1105/1105 [==============================] - 56s 50ms/step - loss: 0.0014 - val_loss: 0.0025 Epoch 135/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0014 Epoch 00135: val_loss did not improve from 0.00219 Epoch 00135: val_loss did not improve from 0.00219 1105/1105 [==============================] - 56s 50ms/step - loss: 0.0014 - val_loss: 0.0028 Epoch 136/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0014 Epoch 00136: val_loss did not improve from 0.00219 Epoch 00136: val_loss did not improve from 0.00219 1105/1105 [==============================] - 57s 51ms/step - loss: 0.0014 - val_loss: 0.0022 Epoch 137/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0014 Epoch 00137: val_loss did not improve from 0.00219 Epoch 00137: val_loss did not improve from 0.00219 1105/1105 [==============================] - 57s 52ms/step - loss: 0.0014 - val_loss: 0.0023 Epoch 138/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0014 Epoch 00138: val_loss did not improve from 0.00219 Epoch 00138: val_loss did not improve from 0.00219 1105/1105 [==============================] - 61s 55ms/step - loss: 0.0014 - val_loss: 0.0024 Epoch 139/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0014 Epoch 00139: val_loss did not improve from 0.00219 Epoch 00139: val_loss did not improve from 0.00219 1105/1105 [==============================] - 59s 53ms/step - loss: 0.0014 - val_loss: 0.0024 Epoch 140/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0014 Epoch 00140: val_loss did not improve from 0.00219 Epoch 00140: val_loss did not improve from 0.00219 1105/1105 [==============================] - 61s 55ms/step - loss: 0.0014 - val_loss: 0.0022 Epoch 141/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0014 Epoch 00141: val_loss did not improve from 0.00219 Epoch 00141: val_loss did not improve from 0.00219 1105/1105 [==============================] - 60s 54ms/step - loss: 0.0014 - val_loss: 0.0025 Epoch 142/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0014 Epoch 00142: val_loss did not improve from 0.00219 Epoch 00142: val_loss did not improve from 0.00219 1105/1105 [==============================] - 58s 52ms/step - loss: 0.0014 - val_loss: 0.0022 Epoch 143/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0014 Epoch 00143: val_loss did not improve from 0.00219 Epoch 00143: val_loss did not improve from 0.00219 1105/1105 [==============================] - 60s 55ms/step - loss: 0.0014 - val_loss: 0.0026 Epoch 144/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0014 Epoch 00144: val_loss did not improve from 0.00219 Epoch 00144: val_loss did not improve from 0.00219 1105/1105 [==============================] - 58s 53ms/step - loss: 0.0014 - val_loss: 0.0027 Epoch 145/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0014 Epoch 00145: val_loss did not improve from 0.00219 Epoch 00145: val_loss did not improve from 0.00219 1105/1105 [==============================] - 56s 51ms/step - loss: 0.0014 - val_loss: 0.0027 Epoch 146/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0014 Epoch 00146: val_loss did not improve from 0.00219 Epoch 00146: val_loss did not improve from 0.00219 1105/1105 [==============================] - 56s 51ms/step - loss: 0.0014 - val_loss: 0.0024 Epoch 147/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0014 Epoch 00147: val_loss did not improve from 0.00219 Epoch 00147: val_loss did not improve from 0.00219 1105/1105 [==============================] - 56s 51ms/step - loss: 0.0014 - val_loss: 0.0025 Epoch 148/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0014 Epoch 00148: val_loss did not improve from 0.00219 Epoch 00148: val_loss did not improve from 0.00219 1105/1105 [==============================] - 57s 52ms/step - loss: 0.0014 - val_loss: 0.0024 Epoch 149/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0014 Epoch 00149: val_loss did not improve from 0.00219 Epoch 00149: val_loss did not improve from 0.00219 1105/1105 [==============================] - 57s 51ms/step - loss: 0.0014 - val_loss: 0.0025 Epoch 150/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0014 Epoch 00150: val_loss did not improve from 0.00219 Epoch 00150: val_loss did not improve from 0.00219 1105/1105 [==============================] - 56s 51ms/step - loss: 0.0014 - val_loss: 0.0023 Epoch 151/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0014 Epoch 00151: val_loss did not improve from 0.00219 Epoch 00151: val_loss did not improve from 0.00219 1105/1105 [==============================] - 58s 52ms/step - loss: 0.0014 - val_loss: 0.0023 Epoch 152/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0014 Epoch 00152: val_loss did not improve from 0.00219 Epoch 00152: val_loss did not improve from 0.00219 1105/1105 [==============================] - 56s 51ms/step - loss: 0.0014 - val_loss: 0.0024 Epoch 153/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0014 Epoch 00153: val_loss did not improve from 0.00219 Epoch 00153: val_loss did not improve from 0.00219 1105/1105 [==============================] - 58s 52ms/step - loss: 0.0014 - val_loss: 0.0025 Epoch 154/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0014 Epoch 00154: val_loss did not improve from 0.00219 Epoch 00154: val_loss did not improve from 0.00219 1105/1105 [==============================] - 58s 53ms/step - loss: 0.0014 - val_loss: 0.0025 Epoch 155/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0013 Epoch 00155: val_loss did not improve from 0.00219 Epoch 00155: val_loss did not improve from 0.00219 1105/1105 [==============================] - 57s 52ms/step - loss: 0.0013 - val_loss: 0.0024 Epoch 156/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0014 Epoch 00156: val_loss did not improve from 0.00219 Epoch 00156: val_loss did not improve from 0.00219 1105/1105 [==============================] - 58s 52ms/step - loss: 0.0014 - val_loss: 0.0022 Epoch 157/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0013 Epoch 00157: val_loss did not improve from 0.00219 Epoch 00157: val_loss did not improve from 0.00219 1105/1105 [==============================] - 57s 52ms/step - loss: 0.0013 - val_loss: 0.0026 Epoch 158/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0013 Epoch 00158: val_loss did not improve from 0.00219 Epoch 00158: val_loss did not improve from 0.00219 1105/1105 [==============================] - 57s 52ms/step - loss: 0.0013 - val_loss: 0.0024 Epoch 159/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0013 Epoch 00159: val_loss did not improve from 0.00219 Epoch 00159: val_loss did not improve from 0.00219 1105/1105 [==============================] - 56s 51ms/step - loss: 0.0013 - val_loss: 0.0025 Epoch 160/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0013 Epoch 00160: val_loss did not improve from 0.00219 Epoch 00160: val_loss did not improve from 0.00219 1105/1105 [==============================] - 56s 51ms/step - loss: 0.0013 - val_loss: 0.0024 Epoch 161/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0013 Epoch 00161: val_loss did not improve from 0.00219 Epoch 00161: val_loss did not improve from 0.00219 1105/1105 [==============================] - 56s 51ms/step - loss: 0.0013 - val_loss: 0.0023 Epoch 162/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0013 Epoch 00162: val_loss did not improve from 0.00219 Epoch 00162: val_loss did not improve from 0.00219 1105/1105 [==============================] - 56s 51ms/step - loss: 0.0013 - val_loss: 0.0025 Epoch 163/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0013 Epoch 00163: val_loss did not improve from 0.00219 Epoch 00163: val_loss did not improve from 0.00219 1105/1105 [==============================] - 56s 51ms/step - loss: 0.0013 - val_loss: 0.0024 Epoch 164/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0013 Epoch 00164: val_loss did not improve from 0.00219 Epoch 00164: val_loss did not improve from 0.00219 1105/1105 [==============================] - 56s 51ms/step - loss: 0.0013 - val_loss: 0.0029 Epoch 165/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0013 Epoch 00165: val_loss did not improve from 0.00219 Epoch 00165: val_loss did not improve from 0.00219 1105/1105 [==============================] - 59s 53ms/step - loss: 0.0013 - val_loss: 0.0024 Epoch 166/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0013 Epoch 00166: val_loss did not improve from 0.00219 Epoch 00166: val_loss did not improve from 0.00219 1105/1105 [==============================] - 58s 52ms/step - loss: 0.0013 - val_loss: 0.0026 Epoch 167/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0013 Epoch 00167: val_loss did not improve from 0.00219 Epoch 00167: val_loss did not improve from 0.00219 1105/1105 [==============================] - 59s 53ms/step - loss: 0.0013 - val_loss: 0.0025 Epoch 168/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0013 Epoch 00168: val_loss did not improve from 0.00219 Epoch 00168: val_loss did not improve from 0.00219 1105/1105 [==============================] - 62s 56ms/step - loss: 0.0013 - val_loss: 0.0024 Epoch 169/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0013 Epoch 00169: val_loss did not improve from 0.00219 Epoch 00169: val_loss did not improve from 0.00219 1105/1105 [==============================] - 62s 56ms/step - loss: 0.0013 - val_loss: 0.0027 Epoch 170/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0013 Epoch 00170: val_loss did not improve from 0.00219 Epoch 00170: val_loss did not improve from 0.00219 1105/1105 [==============================] - 63s 57ms/step - loss: 0.0013 - val_loss: 0.0023 Epoch 171/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0013 Epoch 00171: val_loss did not improve from 0.00219 Epoch 00171: val_loss did not improve from 0.00219 1105/1105 [==============================] - 61s 55ms/step - loss: 0.0013 - val_loss: 0.0024 Epoch 172/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0013 Epoch 00172: val_loss did not improve from 0.00219 Epoch 00172: val_loss did not improve from 0.00219 1105/1105 [==============================] - 58s 53ms/step - loss: 0.0013 - val_loss: 0.0026 Epoch 173/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0013 Epoch 00173: val_loss did not improve from 0.00219 Epoch 00173: val_loss did not improve from 0.00219 1105/1105 [==============================] - 61s 56ms/step - loss: 0.0013 - val_loss: 0.0024 Epoch 174/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0013 Epoch 00174: val_loss did not improve from 0.00219 Epoch 00174: val_loss did not improve from 0.00219 1105/1105 [==============================] - 60s 54ms/step - loss: 0.0013 - val_loss: 0.0027 Epoch 175/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0013 Epoch 00175: val_loss did not improve from 0.00219 Epoch 00175: val_loss did not improve from 0.00219 1105/1105 [==============================] - 62s 57ms/step - loss: 0.0013 - val_loss: 0.0023 Epoch 176/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0013 Epoch 00176: val_loss did not improve from 0.00219 Epoch 00176: val_loss did not improve from 0.00219 1105/1105 [==============================] - 60s 54ms/step - loss: 0.0013 - val_loss: 0.0024 Epoch 177/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0013 Epoch 00177: val_loss did not improve from 0.00219 Epoch 00177: val_loss did not improve from 0.00219 1105/1105 [==============================] - 59s 53ms/step - loss: 0.0013 - val_loss: 0.0027 Epoch 178/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0013 Epoch 00178: val_loss did not improve from 0.00219 Epoch 00178: val_loss did not improve from 0.00219 1105/1105 [==============================] - 57s 52ms/step - loss: 0.0013 - val_loss: 0.0026 Epoch 179/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0013 Epoch 00179: val_loss did not improve from 0.00219 Epoch 00179: val_loss did not improve from 0.00219 1105/1105 [==============================] - 57s 52ms/step - loss: 0.0013 - val_loss: 0.0025 Epoch 180/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0013 Epoch 00180: val_loss did not improve from 0.00219 Epoch 00180: val_loss did not improve from 0.00219 1105/1105 [==============================] - 57s 52ms/step - loss: 0.0013 - val_loss: 0.0026 Epoch 181/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0012 Epoch 00181: val_loss did not improve from 0.00219 Epoch 00181: val_loss did not improve from 0.00219 1105/1105 [==============================] - 57s 52ms/step - loss: 0.0012 - val_loss: 0.0026 Epoch 182/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0012 Epoch 00182: val_loss improved from 0.00219 to 0.00214, saving model to reg_weights2-improvement-182-0.002136.hdf5 Epoch 00182: val_loss did not improve from 0.00214 1105/1105 [==============================] - 57s 51ms/step - loss: 0.0012 - val_loss: 0.0021 Epoch 183/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0013 Epoch 00183: val_loss did not improve from 0.00214 Epoch 00183: val_loss did not improve from 0.00214 1105/1105 [==============================] - 57s 51ms/step - loss: 0.0013 - val_loss: 0.0025 Epoch 184/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0012 Epoch 00184: val_loss did not improve from 0.00214 Epoch 00184: val_loss did not improve from 0.00214 1105/1105 [==============================] - 56s 51ms/step - loss: 0.0012 - val_loss: 0.0026 Epoch 185/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0013 Epoch 00185: val_loss did not improve from 0.00214 Epoch 00185: val_loss did not improve from 0.00214 1105/1105 [==============================] - 56s 51ms/step - loss: 0.0013 - val_loss: 0.0026 Epoch 186/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0012 Epoch 00186: val_loss did not improve from 0.00214 Epoch 00186: val_loss did not improve from 0.00214 1105/1105 [==============================] - 57s 52ms/step - loss: 0.0012 - val_loss: 0.0025 Epoch 187/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0012 Epoch 00187: val_loss did not improve from 0.00214 Epoch 00187: val_loss did not improve from 0.00214 1105/1105 [==============================] - 59s 54ms/step - loss: 0.0012 - val_loss: 0.0023 Epoch 188/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0012 Epoch 00188: val_loss did not improve from 0.00214 Epoch 00188: val_loss did not improve from 0.00214 1105/1105 [==============================] - 60s 54ms/step - loss: 0.0012 - val_loss: 0.0025 Epoch 189/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0012 Epoch 00189: val_loss did not improve from 0.00214 Epoch 00189: val_loss did not improve from 0.00214 1105/1105 [==============================] - 59s 53ms/step - loss: 0.0012 - val_loss: 0.0028 Epoch 190/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0012 Epoch 00190: val_loss did not improve from 0.00214 Epoch 00190: val_loss did not improve from 0.00214 1105/1105 [==============================] - 59s 53ms/step - loss: 0.0012 - val_loss: 0.0027 Epoch 191/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0012 Epoch 00191: val_loss did not improve from 0.00214 Epoch 00191: val_loss did not improve from 0.00214 1105/1105 [==============================] - 57s 52ms/step - loss: 0.0012 - val_loss: 0.0024 Epoch 192/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0012 Epoch 00192: val_loss did not improve from 0.00214 Epoch 00192: val_loss did not improve from 0.00214 1105/1105 [==============================] - 58s 52ms/step - loss: 0.0012 - val_loss: 0.0027 Epoch 193/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0012 Epoch 00193: val_loss did not improve from 0.00214 Epoch 00193: val_loss did not improve from 0.00214 1105/1105 [==============================] - 60s 54ms/step - loss: 0.0012 - val_loss: 0.0026 Epoch 194/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0012 Epoch 00194: val_loss did not improve from 0.00214 Epoch 00194: val_loss did not improve from 0.00214 1105/1105 [==============================] - 59s 54ms/step - loss: 0.0012 - val_loss: 0.0024 Epoch 195/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0012 Epoch 00195: val_loss did not improve from 0.00214 Epoch 00195: val_loss did not improve from 0.00214 1105/1105 [==============================] - 60s 54ms/step - loss: 0.0012 - val_loss: 0.0023 Epoch 196/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0012 Epoch 00196: val_loss did not improve from 0.00214 Epoch 00196: val_loss did not improve from 0.00214 1105/1105 [==============================] - 60s 54ms/step - loss: 0.0012 - val_loss: 0.0023 Epoch 197/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0012 Epoch 00197: val_loss did not improve from 0.00214 Epoch 00197: val_loss did not improve from 0.00214 1105/1105 [==============================] - 57s 52ms/step - loss: 0.0012 - val_loss: 0.0023 Epoch 198/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0012 Epoch 00198: val_loss did not improve from 0.00214 Epoch 00198: val_loss did not improve from 0.00214 1105/1105 [==============================] - 56s 51ms/step - loss: 0.0012 - val_loss: 0.0024 Epoch 199/200 1105/1105 [==============================] - ETA: 0s - loss: 0.0012 Epoch 00199: val_loss did not improve from 0.00214 Epoch 00199: val_loss did not improve from 0.00214 1105/1105 [==============================] - 56s 51ms/step - loss: 0.0012 - val_loss: 0.0024 Epoch 200/200 1104/1105 [============================>.] - ETA: 0s - loss: 0.0012 Epoch 00200: val_loss did not improve from 0.00214 Epoch 00200: val_loss did not improve from 0.00214 1105/1105 [==============================] - 56s 51ms/step - loss: 0.0012 - val_loss: 0.0026
Next, let's plot the evolution of the model over the epochs of training:
In [42]:
# list all data in history
print(reg_history.history.keys())
# summarize history for loss
plt.plot(reg_history.history['loss'])
plt.plot(reg_history.history['val_loss'])
plt.title('model loss')
plt.ylabel('loss')
plt.xlabel('epoch')
plt.legend(['train', 'test'], loc='upper left')
plt.savefig("reg_loss.png")
plt.show()
dict_keys(['loss', 'val_loss'])
Model Testing¶
Let's evaluate the model performance over the train and test set:
In [44]:
reg_model.load_weights("/content/reg_weights2-improvement-81-0.002230.hdf5")
In [45]:
from sklearn.metrics import mean_squared_error
y_train_pred_reg=reg_model.predict(muon_hits_x_train_values)
print("train root mean squared error: ",mean_squared_error(pt_inverse_labels_train_values,y_train_pred_reg,squared=False))
y_test_pred_reg=reg_model.predict(muon_hits_x_test_values)
print("test root mean squared error: ",mean_squared_error(pt_inverse_labels_test_values,y_test_pred_reg,squared=False))
train root mean squared error: 0.046178654 test root mean squared error: 0.047247976
That is impressive! the root mean square error on the testing data is relatively small in comparison to the data!
Future Directions¶
- We discovered in this project that the classifier model was not very good in detecting minority classes. Therefore we need to look for more techniques to make the model less biased (such as up sampling minority classes or down sampling the majority class).
- We can further improve the performance of our regression model by hyperparameters tuning through grid search.