Inception network implementation
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
input_1 (InputLayer) (None, 74, 74, 3) 0
__________________________________________________________________________________________________
conv_1_7x7/2 (Conv2D) (None, 37, 37, 64) 9472 input_1[0][0]
__________________________________________________________________________________________________
max_pool_1_3x3/2 (MaxPooling2D) (None, 19, 19, 64) 0 conv_1_7x7/2[0][0]
__________________________________________________________________________________________________
conv_2a_3x3/1 (Conv2D) (None, 19, 19, 64) 4160 max_pool_1_3x3/2[0][0]
__________________________________________________________________________________________________
conv_2b_3x3/1 (Conv2D) (None, 19, 19, 192) 110784 conv_2a_3x3/1[0][0]
__________________________________________________________________________________________________
max_pool_2_3x3/2 (MaxPooling2D) (None, 10, 10, 192) 0 conv_2b_3x3/1[0][0]
__________________________________________________________________________________________________
conv2d_2 (Conv2D) (None, 10, 10, 96) 18528 max_pool_2_3x3/2[0][0]
__________________________________________________________________________________________________
conv2d_4 (Conv2D) (None, 10, 10, 16) 3088 max_pool_2_3x3/2[0][0]
__________________________________________________________________________________________________
max_pooling2d_1 (MaxPooling2D) (None, 10, 10, 192) 0 max_pool_2_3x3/2[0][0]
__________________________________________________________________________________________________
conv2d_1 (Conv2D) (None, 10, 10, 64) 12352 max_pool_2_3x3/2[0][0]
__________________________________________________________________________________________________
conv2d_3 (Conv2D) (None, 10, 10, 128) 110720 conv2d_2[0][0]
__________________________________________________________________________________________________
conv2d_5 (Conv2D) (None, 10, 10, 32) 544 conv2d_4[0][0]
__________________________________________________________________________________________________
conv2d_6 (Conv2D) (None, 10, 10, 32) 6176 max_pooling2d_1[0][0]
__________________________________________________________________________________________________
inception_3a (Concatenate) (None, 10, 10, 256) 0 conv2d_1[0][0]
conv2d_3[0][0]
conv2d_5[0][0]
conv2d_6[0][0]
__________________________________________________________________________________________________
conv2d_8 (Conv2D) (None, 10, 10, 128) 32896 inception_3a[0][0]
__________________________________________________________________________________________________
conv2d_10 (Conv2D) (None, 10, 10, 32) 8224 inception_3a[0][0]
__________________________________________________________________________________________________
max_pooling2d_2 (MaxPooling2D) (None, 10, 10, 256) 0 inception_3a[0][0]
__________________________________________________________________________________________________
conv2d_7 (Conv2D) (None, 10, 10, 128) 32896 inception_3a[0][0]
__________________________________________________________________________________________________
conv2d_9 (Conv2D) (None, 10, 10, 192) 221376 conv2d_8[0][0]
__________________________________________________________________________________________________
conv2d_11 (Conv2D) (None, 10, 10, 96) 3168 conv2d_10[0][0]
__________________________________________________________________________________________________
conv2d_12 (Conv2D) (None, 10, 10, 64) 16448 max_pooling2d_2[0][0]
__________________________________________________________________________________________________
inception_3b (Concatenate) (None, 10, 10, 480) 0 conv2d_7[0][0]
conv2d_9[0][0]
conv2d_11[0][0]
conv2d_12[0][0]
__________________________________________________________________________________________________
max_pool_3_3x3/2 (MaxPooling2D) (None, 5, 5, 480) 0 inception_3b[0][0]
__________________________________________________________________________________________________
conv2d_14 (Conv2D) (None, 5, 5, 96) 46176 max_pool_3_3x3/2[0][0]
__________________________________________________________________________________________________
conv2d_16 (Conv2D) (None, 5, 5, 16) 7696 max_pool_3_3x3/2[0][0]
__________________________________________________________________________________________________
max_pooling2d_3 (MaxPooling2D) (None, 5, 5, 480) 0 max_pool_3_3x3/2[0][0]
__________________________________________________________________________________________________
conv2d_13 (Conv2D) (None, 5, 5, 192) 92352 max_pool_3_3x3/2[0][0]
__________________________________________________________________________________________________
conv2d_15 (Conv2D) (None, 5, 5, 208) 179920 conv2d_14[0][0]
__________________________________________________________________________________________________
conv2d_17 (Conv2D) (None, 5, 5, 48) 816 conv2d_16[0][0]
__________________________________________________________________________________________________
conv2d_18 (Conv2D) (None, 5, 5, 64) 30784 max_pooling2d_3[0][0]
__________________________________________________________________________________________________
inception_4a (Concatenate) (None, 5, 5, 512) 0 conv2d_13[0][0]
conv2d_15[0][0]
conv2d_17[0][0]
conv2d_18[0][0]
__________________________________________________________________________________________________
conv2d_21 (Conv2D) (None, 5, 5, 112) 57456 inception_4a[0][0]
__________________________________________________________________________________________________
conv2d_23 (Conv2D) (None, 5, 5, 24) 12312 inception_4a[0][0]
__________________________________________________________________________________________________
max_pooling2d_4 (MaxPooling2D) (None, 5, 5, 512) 0 inception_4a[0][0]
__________________________________________________________________________________________________
conv2d_20 (Conv2D) (None, 5, 5, 160) 82080 inception_4a[0][0]
__________________________________________________________________________________________________
conv2d_22 (Conv2D) (None, 5, 5, 224) 226016 conv2d_21[0][0]
__________________________________________________________________________________________________
conv2d_24 (Conv2D) (None, 5, 5, 64) 1600 conv2d_23[0][0]
__________________________________________________________________________________________________
conv2d_25 (Conv2D) (None, 5, 5, 64) 32832 max_pooling2d_4[0][0]
__________________________________________________________________________________________________
inception_4b (Concatenate) (None, 5, 5, 512) 0 conv2d_20[0][0]
conv2d_22[0][0]
conv2d_24[0][0]
conv2d_25[0][0]
__________________________________________________________________________________________________
conv2d_27 (Conv2D) (None, 5, 5, 128) 65664 inception_4b[0][0]
__________________________________________________________________________________________________
conv2d_29 (Conv2D) (None, 5, 5, 24) 12312 inception_4b[0][0]
__________________________________________________________________________________________________
max_pooling2d_5 (MaxPooling2D) (None, 5, 5, 512) 0 inception_4b[0][0]
__________________________________________________________________________________________________
conv2d_26 (Conv2D) (None, 5, 5, 128) 65664 inception_4b[0][0]
__________________________________________________________________________________________________
conv2d_28 (Conv2D) (None, 5, 5, 256) 295168 conv2d_27[0][0]
__________________________________________________________________________________________________
conv2d_30 (Conv2D) (None, 5, 5, 64) 1600 conv2d_29[0][0]
__________________________________________________________________________________________________
conv2d_31 (Conv2D) (None, 5, 5, 64) 32832 max_pooling2d_5[0][0]
__________________________________________________________________________________________________
inception_4c (Concatenate) (None, 5, 5, 512) 0 conv2d_26[0][0]
conv2d_28[0][0]
conv2d_30[0][0]
conv2d_31[0][0]
__________________________________________________________________________________________________
conv2d_33 (Conv2D) (None, 5, 5, 144) 73872 inception_4c[0][0]
__________________________________________________________________________________________________
conv2d_35 (Conv2D) (None, 5, 5, 32) 16416 inception_4c[0][0]
__________________________________________________________________________________________________
max_pooling2d_6 (MaxPooling2D) (None, 5, 5, 512) 0 inception_4c[0][0]
__________________________________________________________________________________________________
conv2d_32 (Conv2D) (None, 5, 5, 112) 57456 inception_4c[0][0]
__________________________________________________________________________________________________
conv2d_34 (Conv2D) (None, 5, 5, 288) 373536 conv2d_33[0][0]
__________________________________________________________________________________________________
conv2d_36 (Conv2D) (None, 5, 5, 64) 2112 conv2d_35[0][0]
__________________________________________________________________________________________________
conv2d_37 (Conv2D) (None, 5, 5, 64) 32832 max_pooling2d_6[0][0]
__________________________________________________________________________________________________
inception_4d (Concatenate) (None, 5, 5, 528) 0 conv2d_32[0][0]
conv2d_34[0][0]
conv2d_36[0][0]
conv2d_37[0][0]
__________________________________________________________________________________________________
conv2d_40 (Conv2D) (None, 5, 5, 160) 84640 inception_4d[0][0]
__________________________________________________________________________________________________
conv2d_42 (Conv2D) (None, 5, 5, 32) 16928 inception_4d[0][0]
__________________________________________________________________________________________________
max_pooling2d_7 (MaxPooling2D) (None, 5, 5, 528) 0 inception_4d[0][0]
__________________________________________________________________________________________________
conv2d_39 (Conv2D) (None, 5, 5, 256) 135424 inception_4d[0][0]
__________________________________________________________________________________________________
conv2d_41 (Conv2D) (None, 5, 5, 320) 461120 conv2d_40[0][0]
__________________________________________________________________________________________________
conv2d_43 (Conv2D) (None, 5, 5, 128) 4224 conv2d_42[0][0]
__________________________________________________________________________________________________
conv2d_44 (Conv2D) (None, 5, 5, 128) 67712 max_pooling2d_7[0][0]
__________________________________________________________________________________________________
inception_4e (Concatenate) (None, 5, 5, 832) 0 conv2d_39[0][0]
conv2d_41[0][0]
conv2d_43[0][0]
conv2d_44[0][0]
__________________________________________________________________________________________________
max_pool_4_3x3/2 (MaxPooling2D) (None, 3, 3, 832) 0 inception_4e[0][0]
__________________________________________________________________________________________________
conv2d_46 (Conv2D) (None, 3, 3, 160) 133280 max_pool_4_3x3/2[0][0]
__________________________________________________________________________________________________
conv2d_48 (Conv2D) (None, 3, 3, 32) 26656 max_pool_4_3x3/2[0][0]
__________________________________________________________________________________________________
max_pooling2d_8 (MaxPooling2D) (None, 3, 3, 832) 0 max_pool_4_3x3/2[0][0]
__________________________________________________________________________________________________
conv2d_45 (Conv2D) (None, 3, 3, 256) 213248 max_pool_4_3x3/2[0][0]
__________________________________________________________________________________________________
conv2d_47 (Conv2D) (None, 3, 3, 320) 461120 conv2d_46[0][0]
__________________________________________________________________________________________________
conv2d_49 (Conv2D) (None, 3, 3, 128) 4224 conv2d_48[0][0]
__________________________________________________________________________________________________
conv2d_50 (Conv2D) (None, 3, 3, 128) 106624 max_pooling2d_8[0][0]
__________________________________________________________________________________________________
inception_5a (Concatenate) (None, 3, 3, 832) 0 conv2d_45[0][0]
conv2d_47[0][0]
conv2d_49[0][0]
conv2d_50[0][0]
__________________________________________________________________________________________________
conv2d_52 (Conv2D) (None, 3, 3, 192) 159936 inception_5a[0][0]
__________________________________________________________________________________________________
conv2d_54 (Conv2D) (None, 3, 3, 48) 39984 inception_5a[0][0]
__________________________________________________________________________________________________
max_pooling2d_9 (MaxPooling2D) (None, 3, 3, 832) 0 inception_5a[0][0]
__________________________________________________________________________________________________
average_pooling2d_1 (AveragePoo (None, 1, 1, 512) 0 inception_4a[0][0]
__________________________________________________________________________________________________
average_pooling2d_2 (AveragePoo (None, 1, 1, 528) 0 inception_4d[0][0]
__________________________________________________________________________________________________
conv2d_51 (Conv2D) (None, 3, 3, 384) 319872 inception_5a[0][0]
__________________________________________________________________________________________________
conv2d_53 (Conv2D) (None, 3, 3, 384) 663936 conv2d_52[0][0]
__________________________________________________________________________________________________
conv2d_55 (Conv2D) (None, 3, 3, 128) 6272 conv2d_54[0][0]
__________________________________________________________________________________________________
conv2d_56 (Conv2D) (None, 3, 3, 128) 106624 max_pooling2d_9[0][0]
__________________________________________________________________________________________________
conv2d_19 (Conv2D) (None, 1, 1, 128) 65664 average_pooling2d_1[0][0]
__________________________________________________________________________________________________
conv2d_38 (Conv2D) (None, 1, 1, 128) 67712 average_pooling2d_2[0][0]
__________________________________________________________________________________________________
inception_5b (Concatenate) (None, 3, 3, 1024) 0 conv2d_51[0][0]
conv2d_53[0][0]
conv2d_55[0][0]
conv2d_56[0][0]
__________________________________________________________________________________________________
flatten_1 (Flatten) (None, 128) 0 conv2d_19[0][0]
__________________________________________________________________________________________________
flatten_2 (Flatten) (None, 128) 0 conv2d_38[0][0]
__________________________________________________________________________________________________
avg_pool_5_3x3/1 (GlobalAverage (None, 1024) 0 inception_5b[0][0]
__________________________________________________________________________________________________
dense_1 (Dense) (None, 1024) 132096 flatten_1[0][0]
__________________________________________________________________________________________________
dense_2 (Dense) (None, 1024) 132096 flatten_2[0][0]
__________________________________________________________________________________________________
dropout_3 (Dropout) (None, 1024) 0 avg_pool_5_3x3/1[0][0]
__________________________________________________________________________________________________
dropout_1 (Dropout) (None, 1024) 0 dense_1[0][0]
__________________________________________________________________________________________________
dropout_2 (Dropout) (None, 1024) 0 dense_2[0][0]
__________________________________________________________________________________________________
output (Dense) (None, 10) 10250 dropout_3[0][0]
__________________________________________________________________________________________________
auxilliary_output_1 (Dense) (None, 10) 10250 dropout_1[0][0]
__________________________________________________________________________________________________
auxilliary_output_2 (Dense) (None, 10) 10250 dropout_2[0][0]
==================================================================================================
Total params: 5,830,478
Trainable params: 5,830,478
Non-trainable params: 0
__________________________________________________________________________________________________
Train on 50000 samples, validate on 10000 samples
Epoch 1/25
Epoch 00001: LearningRateScheduler setting learning rate to 0.01.
256/50000 [..............................] - ETA: 18:40 - loss: 4.2348 - output_loss: 2.8337 - auxilliary_output_1_loss: 2.3176 - auxilliary_output_2_loss: 2.3528 - output_acc: 0.0938 - auxilliary_output_1_acc: 0.1211 - auxilliary_output_2_acc: 0.0938
512/50000 [..............................] - ETA: 9:46 - loss: 4.0626 - output_loss: 2.6603 - auxilliary_output_1_loss: 2.3217 - auxilliary_output_2_loss: 2.3527 - output_acc: 0.1094 - auxilliary_output_1_acc: 0.1094 - auxilliary_output_2_acc: 0.0918
768/50000 [..............................] - ETA: 6:47 - loss: 4.0295 - output_loss: 2.6233 - auxilliary_output_1_loss: 2.3338 - auxilliary_output_2_loss: 2.3536 - output_acc: 0.1172 - auxilliary_output_1_acc: 0.0990 - auxilliary_output_2_acc: 0.0938
1024/50000 [..............................] - ETA: 5:18 - loss: 4.0102 - output_loss: 2.6069 - auxilliary_output_1_loss: 2.3309 - auxilliary_output_2_loss: 2.3467 - output_acc: 0.1172 - auxilliary_output_1_acc: 0.1006 - auxilliary_output_2_acc: 0.0996
1280/50000 [..............................] - ETA: 4:24 - loss: 3.9799 - output_loss: 2.5771 - auxilliary_output_1_loss: 2.3325 - auxilliary_output_2_loss: 2.3433 - output_acc: 0.1062 - auxilliary_output_1_acc: 0.1031 - auxilliary_output_2_acc: 0.0969
1536/50000 [..............................] - ETA: 3:48 - loss: 3.9568 - output_loss: 2.5552 - auxilliary_output_1_loss: 2.3304 - auxilliary_output_2_loss: 2.3414 - output_acc: 0.1074 - auxilliary_output_1_acc: 0.1087 - auxilliary_output_2_acc: 0.0977
1792/50000 [>.............................] - ETA: 3:22 - loss: 3.9452 - output_loss: 2.5458 - auxilliary_output_1_loss: 2.3292 - auxilliary_output_2_loss: 2.3354 - output_acc: 0.1094 - auxilliary_output_1_acc: 0.1099 - auxilliary_output_2_acc: 0.1038
2048/50000 [>.............................] - ETA: 3:03 - loss: 3.9324 - output_loss: 2.5327 - auxilliary_output_1_loss: 2.3306 - auxilliary_output_2_loss: 2.3353 - output_acc: 0.1079 - auxilliary_output_1_acc: 0.1099 - auxilliary_output_2_acc: 0.1001
2304/50000 [>.............................] - ETA: 2:48 - loss: 3.9177 - output_loss: 2.5179 - auxilliary_output_1_loss: 2.3306 - auxilliary_output_2_loss: 2.3355 - output_acc: 0.1081 - auxilliary_output_1_acc: 0.1072 - auxilliary_output_2_acc: 0.0981
2560/50000 [>.............................] - ETA: 2:35 - loss: 3.9135 - output_loss: 2.5151 - auxilliary_output_1_loss: 2.3289 - auxilliary_output_2_loss: 2.3327 - output_acc: 0.1059 - auxilliary_output_1_acc: 0.1070 - auxilliary_output_2_acc: 0.0996
2816/50000 [>.............................] - ETA: 2:25 - loss: 3.9011 - output_loss: 2.5036 - auxilliary_output_1_loss: 2.3282 - auxilliary_output_2_loss: 2.3301 - output_acc: 0.1058 - auxilliary_output_1_acc: 0.1076 - auxilliary_output_2_acc: 0.1005
3072/50000 [>.............................] - ETA: 2:17 - loss: 3.8927 - output_loss: 2.4953 - auxilliary_output_1_loss: 2.3286 - auxilliary_output_2_loss: 2.3296 - output_acc: 0.1087 - auxilliary_output_1_acc: 0.1042 - auxilliary_output_2_acc: 0.0990
3328/50000 [>.............................] - ETA: 2:10 - loss: 3.8834 - output_loss: 2.4864 - auxilliary_output_1_loss: 2.3283 - auxilliary_output_2_loss: 2.3284 - output_acc: 0.1085 - auxilliary_output_1_acc: 0.1025 - auxilliary_output_2_acc: 0.0998
3584/50000 [=>............................] - ETA: 2:04 - loss: 3.8731 - output_loss: 2.4763 - auxilliary_output_1_loss: 2.3287 - auxilliary_output_2_loss: 2.3272 - output_acc: 0.1099 - auxilliary_output_1_acc: 0.1004 - auxilliary_output_2_acc: 0.1010
3840/50000 [=>............................] - ETA: 1:58 - loss: 3.8657 - output_loss: 2.4692 - auxilliary_output_1_loss: 2.3275 - auxilliary_output_2_loss: 2.3272 - output_acc: 0.1099 - auxilliary_output_1_acc: 0.0982 - auxilliary_output_2_acc: 0.0997
4096/50000 [=>............................] - ETA: 1:53 - loss: 3.8581 - output_loss: 2.4623 - auxilliary_output_1_loss: 2.3257 - auxilliary_output_2_loss: 2.3267 - output_acc: 0.1096 - auxilliary_output_1_acc: 0.1003 - auxilliary_output_2_acc: 0.0974
4352/50000 [=>............................] - ETA: 1:49 - loss: 3.8513 - output_loss: 2.4558 - auxilliary_output_1_loss: 2.3261 - auxilliary_output_2_loss: 2.3257 - output_acc: 0.1101 - auxilliary_output_1_acc: 0.0990 - auxilliary_output_2_acc: 0.0988
4608/50000 [=>............................] - ETA: 1:45 - loss: 3.8469 - output_loss: 2.4515 - auxilliary_output_1_loss: 2.3252 - auxilliary_output_2_loss: 2.3261 - output_acc: 0.1109 - auxilliary_output_1_acc: 0.1003 - auxilliary_output_2_acc: 0.0970
4864/50000 [=>............................] - ETA: 1:42 - loss: 3.8402 - output_loss: 2.4455 - auxilliary_output_1_loss: 2.3240 - auxilliary_output_2_loss: 2.3250 - output_acc: 0.1120 - auxilliary_output_1_acc: 0.1007 - auxilliary_output_2_acc: 0.0987
5120/50000 [==>...........................] - ETA: 1:39 - loss: 3.8346 - output_loss: 2.4402 - auxilliary_output_1_loss: 2.3237 - auxilliary_output_2_loss: 2.3243 - output_acc: 0.1107 - auxilliary_output_1_acc: 0.0998 - auxilliary_output_2_acc: 0.0977
5376/50000 [==>...........................] - ETA: 1:36 - loss: 3.8283 - output_loss: 2.4344 - auxilliary_output_1_loss: 2.3229 - auxilliary_output_2_loss: 2.3237 - output_acc: 0.1114 - auxilliary_output_1_acc: 0.0995 - auxilliary_output_2_acc: 0.0977
5632/50000 [==>...........................] - ETA: 1:33 - loss: 3.8263 - output_loss: 2.4324 - auxilliary_output_1_loss: 2.3230 - auxilliary_output_2_loss: 2.3234 - output_acc: 0.1085 - auxilliary_output_1_acc: 0.0989 - auxilliary_output_2_acc: 0.0959
5888/50000 [==>...........................] - ETA: 1:31 - loss: 3.8219 - output_loss: 2.4283 - auxilliary_output_1_loss: 2.3225 - auxilliary_output_2_loss: 2.3226 - output_acc: 0.1082 - auxilliary_output_1_acc: 0.0990 - auxilliary_output_2_acc: 0.0960
6144/50000 [==>...........................] - ETA: 1:29 - loss: 3.8171 - output_loss: 2.4236 - auxilliary_output_1_loss: 2.3226 - auxilliary_output_2_loss: 2.3224 - output_acc: 0.1084 - auxilliary_output_1_acc: 0.0981 - auxilliary_output_2_acc: 0.0959
6400/50000 [==>...........................] - ETA: 1:27 - loss: 3.8130 - output_loss: 2.4197 -
49664/50000 [============================>.] - ETA: 0s - loss: 0.4584 - output_lo
......
ss: 0.2401 - auxilliary_output_1_loss: 0.4479 - auxilliary_output_2_loss: 0.2798 - output_acc: 0.9172 - auxilliary_output_1_acc: 0.8460 - auxilliary_output_2_acc: 0.9039
49920/50000 [============================>.] - ETA: 0s - loss: 0.4582 - output_loss: 0.2400 - auxilliary_output_1_loss: 0.4477 - auxilliary_output_2_loss: 0.2795 - output_acc: 0.9173 - auxilliary_output_1_acc: 0.8462 - auxilliary_output_2_acc: 0.9040
50000/50000 [==============================] - 61s 1ms/step - loss: 0.4587 - output_loss: 0.2404 - auxilliary_output_1_loss: 0.4479 - auxilliary_output_2_loss: 0.2799 - output_acc: 0.9171 - auxilliary_output_1_acc: 0.8461 - auxilliary_output_2_acc: 0.9038 - val_loss: 1.1348 - val_output_loss: 0.7098 - val_auxilliary_output_1_loss: 0.7043 - val_auxilliary_output_2_loss: 0.7124 - val_output_acc: 0.7922 - val_auxilliary_output_1_acc: 0.7670 - val_auxilliary_output_2_acc: 0.7879