Software Components in Deep Learning

 DenseNet

Introduction

Deep neural network design known as DenseNet (Dense Convolutional Network) was first presented in 2017 by Huang et al. The idea of densely connected layers—where each layer is coupled to every other layer in a feedforward manner—is the foundation of DenseNet. Traditional neural network architectures merely connect each layer to the one before it and the layer after it.

Compared to conventional topologies, DenseNet's dense connections have a number of benefits. In order to train the network, they first lower the amount of parameters needed, which could result in a quicker convergence and greater generalization. Second, they enhance the information flow in the network, enabling it to learn more intricate representations. 

The vanishing gradient problem, which can arise in deep networks and make training challenging, can also be helped by dense connections.

The DenseNet architecture is made up of a number of dense blocks, each of which has numerous layers and dense connections running through it. Each dense block generates a fresh set of feature maps using the output of the preceding block as input, which are then forwarded to the following block. The architecture also incorporates transition layers, which lessen the number of channels and the spatial dimensions of the feature maps before sending them to the following dense block.

On a variety of image classification benchmarks, including the ImageNet dataset, DenseNet has produced state-of-the-art results. With encouraging outcomes, it has also been used to other tasks like object detection and segmentation.

History

In their study "Densely Connected Convolutional Networks" from 2017, authors Gao Huang, Zhuang Liu, Laurens van der Maaten, and Kilian Q. Weinberger made the DenseNet suggestion. The scientists emphasized that it is challenging to train extremely deep networks using conventional convolutional neural networks (CNNs) because of the vanishing gradient problem, in which gradients get smaller as they move through the network.

The authors proposed the concept of densely connected layers to address this problem, where each layer is feedforward coupled to all preceding layers. This makes it possible for gradients to move across the network more directly, which helps ease the vanishing gradient problem and make it simpler.

On a number of benchmark datasets, including the massive image classification dataset ImageNet, DenseNet was shown to perform at the cutting edge. Object detection, semantic segmentation, and picture captioning are just a few of the computer vision tasks for which it has now become a popular design.

DenseNet expands on other well-known deep learning architectures like ResNet and Inception and has stimulated additional study on dense connection and associated ideas. 

Architecture

The DenseNet architecture consists of transition layers that decrease the spatial dimensions of the feature maps and the number of channels, as well as various dense blocks that are made up of numerous densely connected layers. A high-level summary of the architecture is given below:

• An image-representing 3D tensor serves as the network's input.

• The first layer of convolution: This layer applies a convolution operation on the input image to extract features.

• The network consists of a number of dense blocks, each of which is composed of numerous densely connected layers. A dense block of layers uses the feature maps created by all of the layers before it as input.

• Between each pair of dense blocks, the network has a transition layer that, by combining convolution and pooling processes, shrinks the spatial dimensions of the feature maps and the number of channels.

• The feature maps are then sent through a global average pooling layer, which creates a fixed-length feature vector for the entire image, after the last dense block.

• The final layer is a fully linked layer with softmax activation that generates a probability distribution across all conceivable classes as the output layer.

• Each dense block's layers are connected to one another in a feedforward fashion, meaning that each layer's output serves as an input to all levels after it. The network can learn more efficient and compact representations thanks to its dense connection, which also makes it easier to reuse features, which can improve generalization.

• DenseNet is an extremely flexible architecture that can be applied to a variety of computer vision problems and has been demonstrated to deliver state-of-the-art results on a number of benchmarks.

Working

The following succinct description of how DenseNet functions:

• A 3D tensor that represents an image serves as the network's input.

• Convolutional layers: To extract features, the input image is passed through several convolutional layers. Through the use of convolution, pooling, and activation functions, these features are learned.

• Multiple dense blocks are present in the network, each of which is composed of several densely connected layers. Each layer in a dense block creates a new set of feature maps as an output while utilizing the feature maps created by all earlier layers in the block as input. The network can learn more effective representations and promote feature reuse thanks to its dense connectivity.

• Each pair of dense blocks in the network has a transition layer in between them, which lowers the number of channels and the spatial dimensions of the feature maps. The network's computing cost is decreased, and overfitting is avoided, by performing this via a combination of convolution and pooling processes.

• Global average pooling layer: The feature maps are then subjected to a global average pooling layer after the last dense block, which results in a fixed-length feature vector for the entire image. As a result, it is easier to aggregate features throughout the entire image and the spatial dimensions of the feature maps are reduced to a size that is manageable.

• Layer that generates the output: The last layer is a fully linked layer with softmax activation, which creates a probability distribution across the possible classes.

• The network modifies the convolutional layer and fully connected layer weights during training to reduce the discrepancy between the predicted class probabilities and the actual class labels. Gradient descent optimization and backpropagation are used for this.

• In general, DenseNet is a strong and adaptable architecture that has been demonstrated to provide state-of-the-art performance on a variety of computer vision applications.

Applications

DenseNet has been utilized for a variety of computer vision tasks and has proven to perform at the cutting edge on a number of benchmarks. The following are some of the uses for DenseNet:

• DenseNet has been used for image classification tasks, including the categorization of images into several groups. On a number of benchmarks, including the ImageNet Large Scale Visual Recognition Challenge (ILSVRC) dataset, DenseNet has demonstrated state-of-the-art performance.

• DenseNet has been used for jobs involving object detection, where the objective is to find and identify items in an image. DenseNet-based object detectors have attained cutting-edge results on a number of benchmarks, including the COCO dataset.

• DenseNet has been used for semantic segmentation tasks, where the objective is to give each pixel in an image a class label. Modern performance has been attained by DenseNet-based semantic segmentation models on a number of benchmarks, including the Cityscapes dataset.

• Image captioning: Tasks requiring the creation of a natural language description of an image have been carried out using DenseNet. Modern performance has been attained by DenseNet-based image captioning models on a number of benchmarks, including the MSCOCO dataset.

• Medical image analysis: Tasks involving medical image analysis, such as locating and segmenting tumors in medical pictures, have also been handled using DenseNet. DenseNet-based models have attained cutting-edge performance on a number of benchmarks, including the ISIC 2017 challenge for the categorization of melanoma.

Overall, DenseNet is a very adaptable architecture that can be used for a variety of computer vision applications and has been demonstrated to perform at the cutting edge on a number of them.

Implementation

The DenseNet and its implementation with Keras are as follows below:

Source Code

# import the necessary libraries
from tensorflow.keras.models import Model
from tensorflow.keras.layers import Input, Dense, Conv2D, MaxPooling2D, AveragePooling2D, GlobalAveragePooling2D, concatenate, BatchNormalization, Activation
from tensorflow.keras.regularizers import l2
from tensorflow.keras import backend as K

def bn_relu(x):
    x = BatchNormalization()(x)
    x = Activation('relu')(x)
    return x

def conv(x, filters, kernel_size):
    x = Conv2D(filters, kernel_size, padding='same', use_bias=False, kernel_initializer='he_uniform', kernel_regularizer=l2(1e-4))(x)
    return x

def dense_block(x, blocks, growth_rate):
    for i in range(blocks):
        x1 = bn_relu(x)
        x1 = conv(x1, 4 * growth_rate, (1, 1))
        x1 = bn_relu(x1)
        x1 = conv(x1, growth_rate, (3, 3))
        x = concatenate([x, x1], axis=-1)
    return x

def transition_block(x, reduction):
    x = bn_relu(x)
    filters = K.int_shape(x)[-1]
    x = conv(x, filters * reduction, (1, 1))
    x = AveragePooling2D((2, 2), strides=(2, 2))(x)
    return x

def densenet(blocks, growth_rate, num_classes):
    input_shape = (32, 32, 3)
    img_input = Input(shape=input_shape)

    x = conv(img_input, 2 * growth_rate, (3, 3))
    x = dense_block(x, blocks[0], growth_rate)
    x = transition_block(x, 0.5)

    x = dense_block(x, blocks[1], growth_rate)
    x = transition_block(x, 0.5)

    x = dense_block(x, blocks[2], growth_rate)
    x = transition_block(x, 0.5)

    x = dense_block(x, blocks[3], growth_rate)

    x = bn_relu(x)
    x = GlobalAveragePooling2D()(x)
    x = Dense(num_classes, activation='softmax', kernel_initializer='he_normal')(x)

    model = Model(img_input, x, name='densenet')
    return model
# summary of the model
model = densenet([6, 12, 24, 16], 12, 10)
model.summary()

Obtained Output:

Model: "densenet" __________________________________________________________________________________________________ Layer (type) Output Shape Param # Connected to ================================================================================================== input_1 (InputLayer) [(None, 32, 32, 3)] 0 [] conv2d (Conv2D) (None, 32, 32, 24) 648 ['input_1[0][0]'] batch_normalization (BatchNorm (None, 32, 32, 24) 96 ['conv2d[0][0]'] alization) activation (Activation) (None, 32, 32, 24) 0 ['batch_normalization[0][0]'] conv2d_1 (Conv2D) (None, 32, 32, 48) 1152 ['activation[0][0]'] batch_normalization_1 (BatchNo (None, 32, 32, 48) 192 ['conv2d_1[0][0]'] rmalization) activation_1 (Activation) (None, 32, 32, 48) 0 ['batch_normalization_1[0][0]'] conv2d_2 (Conv2D) (None, 32, 32, 12) 5184 ['activation_1[0][0]'] concatenate (Concatenate) (None, 32, 32, 36) 0 ['conv2d[0][0]', 'conv2d_2[0][0]'] batch_normalization_2 (BatchNo (None, 32, 32, 36) 144 ['concatenate[0][0]'] rmalization) activation_2 (Activation) (None, 32, 32, 36) 0 ['batch_normalization_2[0][0]'] conv2d_3 (Conv2D) (None, 32, 32, 48) 1728 ['activation_2[0][0]'] batch_normalization_3 (BatchNo (None, 32, 32, 48) 192 ['conv2d_3[0][0]'] rmalization) activation_3 (Activation) (None, 32, 32, 48) 0 ['batch_normalization_3[0][0]'] conv2d_4 (Conv2D) (None, 32, 32, 12) 5184 ['activation_3[0][0]'] concatenate_1 (Concatenate) (None, 32, 32, 48) 0 ['concatenate[0][0]', 'conv2d_4[0][0]'] batch_normalization_4 (BatchNo (None, 32, 32, 48) 192 ['concatenate_1[0][0]'] rmalization) activation_4 (Activation) (None, 32, 32, 48) 0 ['batch_normalization_4[0][0]'] conv2d_5 (Conv2D) (None, 32, 32, 48) 2304 ['activation_4[0][0]'] batch_normalization_5 (BatchNo (None, 32, 32, 48) 192 ['conv2d_5[0][0]'] rmalization) activation_5 (Activation) (None, 32, 32, 48) 0 ['batch_normalization_5[0][0]'] conv2d_6 (Conv2D) (None, 32, 32, 12) 5184 ['activation_5[0][0]'] concatenate_2 (Concatenate) (None, 32, 32, 60) 0 ['concatenate_1[0][0]', 'conv2d_6[0][0]'] batch_normalization_6 (BatchNo (None, 32, 32, 60) 240 ['concatenate_2[0][0]'] rmalization) activation_6 (Activation) (None, 32, 32, 60) 0 ['batch_normalization_6[0][0]'] conv2d_7 (Conv2D) (None, 32, 32, 48) 2880 ['activation_6[0][0]'] batch_normalization_7 (BatchNo (None, 32, 32, 48) 192 ['conv2d_7[0][0]'] rmalization) activation_7 (Activation) (None, 32, 32, 48) 0 ['batch_normalization_7[0][0]'] conv2d_8 (Conv2D) (None, 32, 32, 12) 5184 ['activation_7[0][0]'] concatenate_3 (Concatenate) (None, 32, 32, 72) 0 ['concatenate_2[0][0]', 'conv2d_8[0][0]'] batch_normalization_8 (BatchNo (None, 32, 32, 72) 288 ['concatenate_3[0][0]'] rmalization) activation_8 (Activation) (None, 32, 32, 72) 0 ['batch_normalization_8[0][0]'] conv2d_9 (Conv2D) (None, 32, 32, 48) 3456 ['activation_8[0][0]'] batch_normalization_9 (BatchNo (None, 32, 32, 48) 192 ['conv2d_9[0][0]'] rmalization) activation_9 (Activation) (None, 32, 32, 48) 0 ['batch_normalization_9[0][0]'] conv2d_10 (Conv2D) (None, 32, 32, 12) 5184 ['activation_9[0][0]'] concatenate_4 (Concatenate) (None, 32, 32, 84) 0 ['concatenate_3[0][0]', 'conv2d_10[0][0]'] batch_normalization_10 (BatchN (None, 32, 32, 84) 336 ['concatenate_4[0][0]'] ormalization) activation_10 (Activation) (None, 32, 32, 84) 0 ['batch_normalization_10[0][0]'] conv2d_11 (Conv2D) (None, 32, 32, 48) 4032 ['activation_10[0][0]'] batch_normalization_11 (BatchN (None, 32, 32, 48) 192 ['conv2d_11[0][0]'] ormalization) activation_11 (Activation) (None, 32, 32, 48) 0 ['batch_normalization_11[0][0]'] conv2d_12 (Conv2D) (None, 32, 32, 12) 5184 ['activation_11[0][0]'] concatenate_5 (Concatenate) (None, 32, 32, 96) 0 ['concatenate_4[0][0]', 'conv2d_12[0][0]'] batch_normalization_12 (BatchN (None, 32, 32, 96) 384 ['concatenate_5[0][0]'] ormalization) activation_12 (Activation) (None, 32, 32, 96) 0 ['batch_normalization_12[0][0]'] conv2d_13 (Conv2D) (None, 32, 32, 48) 4608 ['activation_12[0][0]'] average_pooling2d (AveragePool (None, 16, 16, 48) 0 ['conv2d_13[0][0]'] ing2D) batch_normalization_13 (BatchN (None, 16, 16, 48) 192 ['average_pooling2d[0][0]'] ormalization) activation_13 (Activation) (None, 16, 16, 48) 0 ['batch_normalization_13[0][0]'] conv2d_14 (Conv2D) (None, 16, 16, 48) 2304 ['activation_13[0][0]'] batch_normalization_14 (BatchN (None, 16, 16, 48) 192 ['conv2d_14[0][0]'] ormalization) activation_14 (Activation) (None, 16, 16, 48) 0 ['batch_normalization_14[0][0]'] conv2d_15 (Conv2D) (None, 16, 16, 12) 5184 ['activation_14[0][0]'] concatenate_6 (Concatenate) (None, 16, 16, 60) 0 ['average_pooling2d[0][0]', 'conv2d_15[0][0]'] batch_normalization_15 (BatchN (None, 16, 16, 60) 240 ['concatenate_6[0][0]'] ormalization) activation_15 (Activation) (None, 16, 16, 60) 0 ['batch_normalization_15[0][0]'] conv2d_16 (Conv2D) (None, 16, 16, 48) 2880 ['activation_15[0][0]'] batch_normalization_16 (BatchN (None, 16, 16, 48) 192 ['conv2d_16[0][0]'] ormalization) activation_16 (Activation) (None, 16, 16, 48) 0 ['batch_normalization_16[0][0]'] conv2d_17 (Conv2D) (None, 16, 16, 12) 5184 ['activation_16[0][0]'] concatenate_7 (Concatenate) (None, 16, 16, 72) 0 ['concatenate_6[0][0]', 'conv2d_17[0][0]'] batch_normalization_17 (BatchN (None, 16, 16, 72) 288 ['concatenate_7[0][0]'] ormalization) activation_17 (Activation) (None, 16, 16, 72) 0 ['batch_normalization_17[0][0]'] conv2d_18 (Conv2D) (None, 16, 16, 48) 3456 ['activation_17[0][0]'] batch_normalization_18 (BatchN (None, 16, 16, 48) 192 ['conv2d_18[0][0]'] ormalization) activation_18 (Activation) (None, 16, 16, 48) 0 ['batch_normalization_18[0][0]'] conv2d_19 (Conv2D) (None, 16, 16, 12) 5184 ['activation_18[0][0]'] concatenate_8 (Concatenate) (None, 16, 16, 84) 0 ['concatenate_7[0][0]', 'conv2d_19[0][0]'] batch_normalization_19 (BatchN (None, 16, 16, 84) 336 ['concatenate_8[0][0]'] ormalization) activation_19 (Activation) (None, 16, 16, 84) 0 ['batch_normalization_19[0][0]'] conv2d_20 (Conv2D) (None, 16, 16, 48) 4032 ['activation_19[0][0]'] batch_normalization_20 (BatchN (None, 16, 16, 48) 192 ['conv2d_20[0][0]'] ormalization) activation_20 (Activation) (None, 16, 16, 48) 0 ['batch_normalization_20[0][0]'] conv2d_21 (Conv2D) (None, 16, 16, 12) 5184 ['activation_20[0][0]'] concatenate_9 (Concatenate) (None, 16, 16, 96) 0 ['concatenate_8[0][0]', 'conv2d_21[0][0]'] batch_normalization_21 (BatchN (None, 16, 16, 96) 384 ['concatenate_9[0][0]'] ormalization) activation_21 (Activation) (None, 16, 16, 96) 0 ['batch_normalization_21[0][0]'] conv2d_22 (Conv2D) (None, 16, 16, 48) 4608 ['activation_21[0][0]'] batch_normalization_22 (BatchN (None, 16, 16, 48) 192 ['conv2d_22[0][0]'] ormalization) activation_22 (Activation) (None, 16, 16, 48) 0 ['batch_normalization_22[0][0]'] conv2d_23 (Conv2D) (None, 16, 16, 12) 5184 ['activation_22[0][0]'] concatenate_10 (Concatenate) (None, 16, 16, 108) 0 ['concatenate_9[0][0]', 'conv2d_23[0][0]'] batch_normalization_23 (BatchN (None, 16, 16, 108) 432 ['concatenate_10[0][0]'] ormalization) activation_23 (Activation) (None, 16, 16, 108) 0 ['batch_normalization_23[0][0]'] conv2d_24 (Conv2D) (None, 16, 16, 48) 5184 ['activation_23[0][0]'] batch_normalization_24 (BatchN (None, 16, 16, 48) 192 ['conv2d_24[0][0]'] ormalization) activation_24 (Activation) (None, 16, 16, 48) 0 ['batch_normalization_24[0][0]'] conv2d_25 (Conv2D) (None, 16, 16, 12) 5184 ['activation_24[0][0]'] concatenate_11 (Concatenate) (None, 16, 16, 120) 0 ['concatenate_10[0][0]', 'conv2d_25[0][0]'] batch_normalization_25 (BatchN (None, 16, 16, 120) 480 ['concatenate_11[0][0]'] ormalization) activation_25 (Activation) (None, 16, 16, 120) 0 ['batch_normalization_25[0][0]'] conv2d_26 (Conv2D) (None, 16, 16, 48) 5760 ['activation_25[0][0]'] batch_normalization_26 (BatchN (None, 16, 16, 48) 192 ['conv2d_26[0][0]'] ormalization) activation_26 (Activation) (None, 16, 16, 48) 0 ['batch_normalization_26[0][0]'] conv2d_27 (Conv2D) (None, 16, 16, 12) 5184 ['activation_26[0][0]'] concatenate_12 (Concatenate) (None, 16, 16, 132) 0 ['concatenate_11[0][0]', 'conv2d_27[0][0]'] batch_normalization_27 (BatchN (None, 16, 16, 132) 528 ['concatenate_12[0][0]'] ormalization) activation_27 (Activation) (None, 16, 16, 132) 0 ['batch_normalization_27[0][0]'] conv2d_28 (Conv2D) (None, 16, 16, 48) 6336 ['activation_27[0][0]'] batch_normalization_28 (BatchN (None, 16, 16, 48) 192 ['conv2d_28[0][0]'] ormalization) activation_28 (Activation) (None, 16, 16, 48) 0 ['batch_normalization_28[0][0]'] conv2d_29 (Conv2D) (None, 16, 16, 12) 5184 ['activation_28[0][0]'] concatenate_13 (Concatenate) (None, 16, 16, 144) 0 ['concatenate_12[0][0]', 'conv2d_29[0][0]'] batch_normalization_29 (BatchN (None, 16, 16, 144) 576 ['concatenate_13[0][0]'] ormalization) activation_29 (Activation) (None, 16, 16, 144) 0 ['batch_normalization_29[0][0]'] conv2d_30 (Conv2D) (None, 16, 16, 48) 6912 ['activation_29[0][0]'] batch_normalization_30 (BatchN (None, 16, 16, 48) 192 ['conv2d_30[0][0]'] ormalization) activation_30 (Activation) (None, 16, 16, 48) 0 ['batch_normalization_30[0][0]'] conv2d_31 (Conv2D) (None, 16, 16, 12) 5184 ['activation_30[0][0]'] concatenate_14 (Concatenate) (None, 16, 16, 156) 0 ['concatenate_13[0][0]', 'conv2d_31[0][0]'] batch_normalization_31 (BatchN (None, 16, 16, 156) 624 ['concatenate_14[0][0]'] ormalization) activation_31 (Activation) (None, 16, 16, 156) 0 ['batch_normalization_31[0][0]'] conv2d_32 (Conv2D) (None, 16, 16, 48) 7488 ['activation_31[0][0]'] batch_normalization_32 (BatchN (None, 16, 16, 48) 192 ['conv2d_32[0][0]'] ormalization) activation_32 (Activation) (None, 16, 16, 48) 0 ['batch_normalization_32[0][0]'] conv2d_33 (Conv2D) (None, 16, 16, 12) 5184 ['activation_32[0][0]'] concatenate_15 (Concatenate) (None, 16, 16, 168) 0 ['concatenate_14[0][0]', 'conv2d_33[0][0]'] batch_normalization_33 (BatchN (None, 16, 16, 168) 672 ['concatenate_15[0][0]'] ormalization) activation_33 (Activation) (None, 16, 16, 168) 0 ['batch_normalization_33[0][0]'] conv2d_34 (Conv2D) (None, 16, 16, 48) 8064 ['activation_33[0][0]'] batch_normalization_34 (BatchN (None, 16, 16, 48) 192 ['conv2d_34[0][0]'] ormalization) activation_34 (Activation) (None, 16, 16, 48) 0 ['batch_normalization_34[0][0]'] conv2d_35 (Conv2D) (None, 16, 16, 12) 5184 ['activation_34[0][0]'] concatenate_16 (Concatenate) (None, 16, 16, 180) 0 ['concatenate_15[0][0]', 'conv2d_35[0][0]'] batch_normalization_35 (BatchN (None, 16, 16, 180) 720 ['concatenate_16[0][0]'] ormalization) activation_35 (Activation) (None, 16, 16, 180) 0 ['batch_normalization_35[0][0]'] conv2d_36 (Conv2D) (None, 16, 16, 48) 8640 ['activation_35[0][0]'] batch_normalization_36 (BatchN (None, 16, 16, 48) 192 ['conv2d_36[0][0]'] ormalization) activation_36 (Activation) (None, 16, 16, 48) 0 ['batch_normalization_36[0][0]'] conv2d_37 (Conv2D) (None, 16, 16, 12) 5184 ['activation_36[0][0]'] concatenate_17 (Concatenate) (None, 16, 16, 192) 0 ['concatenate_16[0][0]', 'conv2d_37[0][0]'] batch_normalization_37 (BatchN (None, 16, 16, 192) 768 ['concatenate_17[0][0]'] ormalization) activation_37 (Activation) (None, 16, 16, 192) 0 ['batch_normalization_37[0][0]'] conv2d_38 (Conv2D) (None, 16, 16, 96) 18432 ['activation_37[0][0]'] average_pooling2d_1 (AveragePo (None, 8, 8, 96) 0 ['conv2d_38[0][0]'] oling2D) batch_normalization_38 (BatchN (None, 8, 8, 96) 384 ['average_pooling2d_1[0][0]'] ormalization) activation_38 (Activation) (None, 8, 8, 96) 0 ['batch_normalization_38[0][0]'] conv2d_39 (Conv2D) (None, 8, 8, 48) 4608 ['activation_38[0][0]'] batch_normalization_39 (BatchN (None, 8, 8, 48) 192 ['conv2d_39[0][0]'] ormalization) activation_39 (Activation) (None, 8, 8, 48) 0 ['batch_normalization_39[0][0]'] conv2d_40 (Conv2D) (None, 8, 8, 12) 5184 ['activation_39[0][0]'] concatenate_18 (Concatenate) (None, 8, 8, 108) 0 ['average_pooling2d_1[0][0]', 'conv2d_40[0][0]'] batch_normalization_40 (BatchN (None, 8, 8, 108) 432 ['concatenate_18[0][0]'] ormalization) activation_40 (Activation) (None, 8, 8, 108) 0 ['batch_normalization_40[0][0]'] conv2d_41 (Conv2D) (None, 8, 8, 48) 5184 ['activation_40[0][0]'] batch_normalization_41 (BatchN (None, 8, 8, 48) 192 ['conv2d_41[0][0]'] ormalization) activation_41 (Activation) (None, 8, 8, 48) 0 ['batch_normalization_41[0][0]'] conv2d_42 (Conv2D) (None, 8, 8, 12) 5184 ['activation_41[0][0]'] concatenate_19 (Concatenate) (None, 8, 8, 120) 0 ['concatenate_18[0][0]', 'conv2d_42[0][0]'] batch_normalization_42 (BatchN (None, 8, 8, 120) 480 ['concatenate_19[0][0]'] ormalization) activation_42 (Activation) (None, 8, 8, 120) 0 ['batch_normalization_42[0][0]'] conv2d_43 (Conv2D) (None, 8, 8, 48) 5760 ['activation_42[0][0]'] batch_normalization_43 (BatchN (None, 8, 8, 48) 192 ['conv2d_43[0][0]'] ormalization) activation_43 (Activation) (None, 8, 8, 48) 0 ['batch_normalization_43[0][0]'] conv2d_44 (Conv2D) (None, 8, 8, 12) 5184 ['activation_43[0][0]'] concatenate_20 (Concatenate) (None, 8, 8, 132) 0 ['concatenate_19[0][0]', 'conv2d_44[0][0]'] batch_normalization_44 (BatchN (None, 8, 8, 132) 528 ['concatenate_20[0][0]'] ormalization) activation_44 (Activation) (None, 8, 8, 132) 0 ['batch_normalization_44[0][0]'] conv2d_45 (Conv2D) (None, 8, 8, 48) 6336 ['activation_44[0][0]'] batch_normalization_45 (BatchN (None, 8, 8, 48) 192 ['conv2d_45[0][0]'] ormalization) activation_45 (Activation) (None, 8, 8, 48) 0 ['batch_normalization_45[0][0]'] conv2d_46 (Conv2D) (None, 8, 8, 12) 5184 ['activation_45[0][0]'] concatenate_21 (Concatenate) (None, 8, 8, 144) 0 ['concatenate_20[0][0]', 'conv2d_46[0][0]'] batch_normalization_46 (BatchN (None, 8, 8, 144) 576 ['concatenate_21[0][0]'] ormalization) activation_46 (Activation) (None, 8, 8, 144) 0 ['batch_normalization_46[0][0]'] conv2d_47 (Conv2D) (None, 8, 8, 48) 6912 ['activation_46[0][0]'] batch_normalization_47 (BatchN (None, 8, 8, 48) 192 ['conv2d_47[0][0]'] ormalization) activation_47 (Activation) (None, 8, 8, 48) 0 ['batch_normalization_47[0][0]'] conv2d_48 (Conv2D) (None, 8, 8, 12) 5184 ['activation_47[0][0]'] concatenate_22 (Concatenate) (None, 8, 8, 156) 0 ['concatenate_21[0][0]', 'conv2d_48[0][0]'] batch_normalization_48 (BatchN (None, 8, 8, 156) 624 ['concatenate_22[0][0]'] ormalization) activation_48 (Activation) (None, 8, 8, 156) 0 ['batch_normalization_48[0][0]'] conv2d_49 (Conv2D) (None, 8, 8, 48) 7488 ['activation_48[0][0]'] batch_normalization_49 (BatchN (None, 8, 8, 48) 192 ['conv2d_49[0][0]'] ormalization) activation_49 (Activation) (None, 8, 8, 48) 0 ['batch_normalization_49[0][0]'] conv2d_50 (Conv2D) (None, 8, 8, 12) 5184 ['activation_49[0][0]'] concatenate_23 (Concatenate) (None, 8, 8, 168) 0 ['concatenate_22[0][0]', 'conv2d_50[0][0]'] batch_normalization_50 (BatchN (None, 8, 8, 168) 672 ['concatenate_23[0][0]'] ormalization) activation_50 (Activation) (None, 8, 8, 168) 0 ['batch_normalization_50[0][0]'] conv2d_51 (Conv2D) (None, 8, 8, 48) 8064 ['activation_50[0][0]'] batch_normalization_51 (BatchN (None, 8, 8, 48) 192 ['conv2d_51[0][0]'] ormalization) activation_51 (Activation) (None, 8, 8, 48) 0 ['batch_normalization_51[0][0]'] conv2d_52 (Conv2D) (None, 8, 8, 12) 5184 ['activation_51[0][0]'] concatenate_24 (Concatenate) (None, 8, 8, 180) 0 ['concatenate_23[0][0]', 'conv2d_52[0][0]'] batch_normalization_52 (BatchN (None, 8, 8, 180) 720 ['concatenate_24[0][0]'] ormalization) activation_52 (Activation) (None, 8, 8, 180) 0 ['batch_normalization_52[0][0]'] conv2d_53 (Conv2D) (None, 8, 8, 48) 8640 ['activation_52[0][0]'] batch_normalization_53 (BatchN (None, 8, 8, 48) 192 ['conv2d_53[0][0]'] ormalization) activation_53 (Activation) (None, 8, 8, 48) 0 ['batch_normalization_53[0][0]'] conv2d_54 (Conv2D) (None, 8, 8, 12) 5184 ['activation_53[0][0]'] concatenate_25 (Concatenate) (None, 8, 8, 192) 0 ['concatenate_24[0][0]', 'conv2d_54[0][0]'] batch_normalization_54 (BatchN (None, 8, 8, 192) 768 ['concatenate_25[0][0]'] ormalization) activation_54 (Activation) (None, 8, 8, 192) 0 ['batch_normalization_54[0][0]'] conv2d_55 (Conv2D) (None, 8, 8, 48) 9216 ['activation_54[0][0]'] batch_normalization_55 (BatchN (None, 8, 8, 48) 192 ['conv2d_55[0][0]'] ormalization) activation_55 (Activation) (None, 8, 8, 48) 0 ['batch_normalization_55[0][0]'] conv2d_56 (Conv2D) (None, 8, 8, 12) 5184 ['activation_55[0][0]'] concatenate_26 (Concatenate) (None, 8, 8, 204) 0 ['concatenate_25[0][0]', 'conv2d_56[0][0]'] batch_normalization_56 (BatchN (None, 8, 8, 204) 816 ['concatenate_26[0][0]'] ormalization) activation_56 (Activation) (None, 8, 8, 204) 0 ['batch_normalization_56[0][0]'] conv2d_57 (Conv2D) (None, 8, 8, 48) 9792 ['activation_56[0][0]'] batch_normalization_57 (BatchN (None, 8, 8, 48) 192 ['conv2d_57[0][0]'] ormalization) activation_57 (Activation) (None, 8, 8, 48) 0 ['batch_normalization_57[0][0]'] conv2d_58 (Conv2D) (None, 8, 8, 12) 5184 ['activation_57[0][0]'] concatenate_27 (Concatenate) (None, 8, 8, 216) 0 ['concatenate_26[0][0]', 'conv2d_58[0][0]'] batch_normalization_58 (BatchN (None, 8, 8, 216) 864 ['concatenate_27[0][0]'] ormalization) activation_58 (Activation) (None, 8, 8, 216) 0 ['batch_normalization_58[0][0]'] conv2d_59 (Conv2D) (None, 8, 8, 48) 10368 ['activation_58[0][0]'] batch_normalization_59 (BatchN (None, 8, 8, 48) 192 ['conv2d_59[0][0]'] ormalization) activation_59 (Activation) (None, 8, 8, 48) 0 ['batch_normalization_59[0][0]'] conv2d_60 (Conv2D) (None, 8, 8, 12) 5184 ['activation_59[0][0]'] concatenate_28 (Concatenate) (None, 8, 8, 228) 0 ['concatenate_27[0][0]', 'conv2d_60[0][0]'] batch_normalization_60 (BatchN (None, 8, 8, 228) 912 ['concatenate_28[0][0]'] ormalization) activation_60 (Activation) (None, 8, 8, 228) 0 ['batch_normalization_60[0][0]'] conv2d_61 (Conv2D) (None, 8, 8, 48) 10944 ['activation_60[0][0]'] batch_normalization_61 (BatchN (None, 8, 8, 48) 192 ['conv2d_61[0][0]'] ormalization) activation_61 (Activation) (None, 8, 8, 48) 0 ['batch_normalization_61[0][0]'] conv2d_62 (Conv2D) (None, 8, 8, 12) 5184 ['activation_61[0][0]'] concatenate_29 (Concatenate) (None, 8, 8, 240) 0 ['concatenate_28[0][0]', 'conv2d_62[0][0]'] batch_normalization_62 (BatchN (None, 8, 8, 240) 960 ['concatenate_29[0][0]'] ormalization) activation_62 (Activation) (None, 8, 8, 240) 0 ['batch_normalization_62[0][0]'] conv2d_63 (Conv2D) (None, 8, 8, 48) 11520 ['activation_62[0][0]'] batch_normalization_63 (BatchN (None, 8, 8, 48) 192 ['conv2d_63[0][0]'] ormalization) activation_63 (Activation) (None, 8, 8, 48) 0 ['batch_normalization_63[0][0]'] conv2d_64 (Conv2D) (None, 8, 8, 12) 5184 ['activation_63[0][0]'] concatenate_30 (Concatenate) (None, 8, 8, 252) 0 ['concatenate_29[0][0]', 'conv2d_64[0][0]'] batch_normalization_64 (BatchN (None, 8, 8, 252) 1008 ['concatenate_30[0][0]'] ormalization) activation_64 (Activation) (None, 8, 8, 252) 0 ['batch_normalization_64[0][0]'] conv2d_65 (Conv2D) (None, 8, 8, 48) 12096 ['activation_64[0][0]'] batch_normalization_65 (BatchN (None, 8, 8, 48) 192 ['conv2d_65[0][0]'] ormalization) activation_65 (Activation) (None, 8, 8, 48) 0 ['batch_normalization_65[0][0]'] conv2d_66 (Conv2D) (None, 8, 8, 12) 5184 ['activation_65[0][0]'] concatenate_31 (Concatenate) (None, 8, 8, 264) 0 ['concatenate_30[0][0]', 'conv2d_66[0][0]'] batch_normalization_66 (BatchN (None, 8, 8, 264) 1056 ['concatenate_31[0][0]'] ormalization) activation_66 (Activation) (None, 8, 8, 264) 0 ['batch_normalization_66[0][0]'] conv2d_67 (Conv2D) (None, 8, 8, 48) 12672 ['activation_66[0][0]'] batch_normalization_67 (BatchN (None, 8, 8, 48) 192 ['conv2d_67[0][0]'] ormalization) activation_67 (Activation) (None, 8, 8, 48) 0 ['batch_normalization_67[0][0]'] conv2d_68 (Conv2D) (None, 8, 8, 12) 5184 ['activation_67[0][0]'] concatenate_32 (Concatenate) (None, 8, 8, 276) 0 ['concatenate_31[0][0]', 'conv2d_68[0][0]'] batch_normalization_68 (BatchN (None, 8, 8, 276) 1104 ['concatenate_32[0][0]'] ormalization) activation_68 (Activation) (None, 8, 8, 276) 0 ['batch_normalization_68[0][0]'] conv2d_69 (Conv2D) (None, 8, 8, 48) 13248 ['activation_68[0][0]'] batch_normalization_69 (BatchN (None, 8, 8, 48) 192 ['conv2d_69[0][0]'] ormalization) activation_69 (Activation) (None, 8, 8, 48) 0 ['batch_normalization_69[0][0]'] conv2d_70 (Conv2D) (None, 8, 8, 12) 5184 ['activation_69[0][0]'] concatenate_33 (Concatenate) (None, 8, 8, 288) 0 ['concatenate_32[0][0]', 'conv2d_70[0][0]'] batch_normalization_70 (BatchN (None, 8, 8, 288) 1152 ['concatenate_33[0][0]'] ormalization) activation_70 (Activation) (None, 8, 8, 288) 0 ['batch_normalization_70[0][0]'] conv2d_71 (Conv2D) (None, 8, 8, 48) 13824 ['activation_70[0][0]'] batch_normalization_71 (BatchN (None, 8, 8, 48) 192 ['conv2d_71[0][0]'] ormalization) activation_71 (Activation) (None, 8, 8, 48) 0 ['batch_normalization_71[0][0]'] conv2d_72 (Conv2D) (None, 8, 8, 12) 5184 ['activation_71[0][0]'] concatenate_34 (Concatenate) (None, 8, 8, 300) 0 ['concatenate_33[0][0]', 'conv2d_72[0][0]'] batch_normalization_72 (BatchN (None, 8, 8, 300) 1200 ['concatenate_34[0][0]'] ormalization) activation_72 (Activation) (None, 8, 8, 300) 0 ['batch_normalization_72[0][0]'] conv2d_73 (Conv2D) (None, 8, 8, 48) 14400 ['activation_72[0][0]'] batch_normalization_73 (BatchN (None, 8, 8, 48) 192 ['conv2d_73[0][0]'] ormalization) activation_73 (Activation) (None, 8, 8, 48) 0 ['batch_normalization_73[0][0]'] conv2d_74 (Conv2D) (None, 8, 8, 12) 5184 ['activation_73[0][0]'] concatenate_35 (Concatenate) (None, 8, 8, 312) 0 ['concatenate_34[0][0]', 'conv2d_74[0][0]'] batch_normalization_74 (BatchN (None, 8, 8, 312) 1248 ['concatenate_35[0][0]'] ormalization) activation_74 (Activation) (None, 8, 8, 312) 0 ['batch_normalization_74[0][0]'] conv2d_75 (Conv2D) (None, 8, 8, 48) 14976 ['activation_74[0][0]'] batch_normalization_75 (BatchN (None, 8, 8, 48) 192 ['conv2d_75[0][0]'] ormalization) activation_75 (Activation) (None, 8, 8, 48) 0 ['batch_normalization_75[0][0]'] conv2d_76 (Conv2D) (None, 8, 8, 12) 5184 ['activation_75[0][0]'] concatenate_36 (Concatenate) (None, 8, 8, 324) 0 ['concatenate_35[0][0]', 'conv2d_76[0][0]'] batch_normalization_76 (BatchN (None, 8, 8, 324) 1296 ['concatenate_36[0][0]'] ormalization) activation_76 (Activation) (None, 8, 8, 324) 0 ['batch_normalization_76[0][0]'] conv2d_77 (Conv2D) (None, 8, 8, 48) 15552 ['activation_76[0][0]'] batch_normalization_77 (BatchN (None, 8, 8, 48) 192 ['conv2d_77[0][0]'] ormalization) activation_77 (Activation) (None, 8, 8, 48) 0 ['batch_normalization_77[0][0]'] conv2d_78 (Conv2D) (None, 8, 8, 12) 5184 ['activation_77[0][0]'] concatenate_37 (Concatenate) (None, 8, 8, 336) 0 ['concatenate_36[0][0]', 'conv2d_78[0][0]'] batch_normalization_78 (BatchN (None, 8, 8, 336) 1344 ['concatenate_37[0][0]'] ormalization) activation_78 (Activation) (None, 8, 8, 336) 0 ['batch_normalization_78[0][0]'] conv2d_79 (Conv2D) (None, 8, 8, 48) 16128 ['activation_78[0][0]'] batch_normalization_79 (BatchN (None, 8, 8, 48) 192 ['conv2d_79[0][0]'] ormalization) activation_79 (Activation) (None, 8, 8, 48) 0 ['batch_normalization_79[0][0]'] conv2d_80 (Conv2D) (None, 8, 8, 12) 5184 ['activation_79[0][0]'] concatenate_38 (Concatenate) (None, 8, 8, 348) 0 ['concatenate_37[0][0]', 'conv2d_80[0][0]'] batch_normalization_80 (BatchN (None, 8, 8, 348) 1392 ['concatenate_38[0][0]'] ormalization) activation_80 (Activation) (None, 8, 8, 348) 0 ['batch_normalization_80[0][0]'] conv2d_81 (Conv2D) (None, 8, 8, 48) 16704 ['activation_80[0][0]'] batch_normalization_81 (BatchN (None, 8, 8, 48) 192 ['conv2d_81[0][0]'] ormalization) activation_81 (Activation) (None, 8, 8, 48) 0 ['batch_normalization_81[0][0]'] conv2d_82 (Conv2D) (None, 8, 8, 12) 5184 ['activation_81[0][0]'] concatenate_39 (Concatenate) (None, 8, 8, 360) 0 ['concatenate_38[0][0]', 'conv2d_82[0][0]'] batch_normalization_82 (BatchN (None, 8, 8, 360) 1440 ['concatenate_39[0][0]'] ormalization) activation_82 (Activation) (None, 8, 8, 360) 0 ['batch_normalization_82[0][0]'] conv2d_83 (Conv2D) (None, 8, 8, 48) 17280 ['activation_82[0][0]'] batch_normalization_83 (BatchN (None, 8, 8, 48) 192 ['conv2d_83[0][0]'] ormalization) activation_83 (Activation) (None, 8, 8, 48) 0 ['batch_normalization_83[0][0]'] conv2d_84 (Conv2D) (None, 8, 8, 12) 5184 ['activation_83[0][0]'] concatenate_40 (Concatenate) (None, 8, 8, 372) 0 ['concatenate_39[0][0]', 'conv2d_84[0][0]'] batch_normalization_84 (BatchN (None, 8, 8, 372) 1488 ['concatenate_40[0][0]'] ormalization) activation_84 (Activation) (None, 8, 8, 372) 0 ['batch_normalization_84[0][0]'] conv2d_85 (Conv2D) (None, 8, 8, 48) 17856 ['activation_84[0][0]'] batch_normalization_85 (BatchN (None, 8, 8, 48) 192 ['conv2d_85[0][0]'] ormalization) activation_85 (Activation) (None, 8, 8, 48) 0 ['batch_normalization_85[0][0]'] conv2d_86 (Conv2D) (None, 8, 8, 12) 5184 ['activation_85[0][0]'] concatenate_41 (Concatenate) (None, 8, 8, 384) 0 ['concatenate_40[0][0]', 'conv2d_86[0][0]'] batch_normalization_86 (BatchN (None, 8, 8, 384) 1536 ['concatenate_41[0][0]'] ormalization) activation_86 (Activation) (None, 8, 8, 384) 0 ['batch_normalization_86[0][0]'] conv2d_87 (Conv2D) (None, 8, 8, 192) 73728 ['activation_86[0][0]'] average_pooling2d_2 (AveragePo (None, 4, 4, 192) 0 ['conv2d_87[0][0]'] oling2D) batch_normalization_87 (BatchN (None, 4, 4, 192) 768 ['average_pooling2d_2[0][0]'] ormalization) activation_87 (Activation) (None, 4, 4, 192) 0 ['batch_normalization_87[0][0]'] conv2d_88 (Conv2D) (None, 4, 4, 48) 9216 ['activation_87[0][0]'] batch_normalization_88 (BatchN (None, 4, 4, 48) 192 ['conv2d_88[0][0]'] ormalization) activation_88 (Activation) (None, 4, 4, 48) 0 ['batch_normalization_88[0][0]'] conv2d_89 (Conv2D) (None, 4, 4, 12) 5184 ['activation_88[0][0]'] concatenate_42 (Concatenate) (None, 4, 4, 204) 0 ['average_pooling2d_2[0][0]', 'conv2d_89[0][0]'] batch_normalization_89 (BatchN (None, 4, 4, 204) 816 ['concatenate_42[0][0]'] ormalization) activation_89 (Activation) (None, 4, 4, 204) 0 ['batch_normalization_89[0][0]'] conv2d_90 (Conv2D) (None, 4, 4, 48) 9792 ['activation_89[0][0]'] batch_normalization_90 (BatchN (None, 4, 4, 48) 192 ['conv2d_90[0][0]'] ormalization) activation_90 (Activation) (None, 4, 4, 48) 0 ['batch_normalization_90[0][0]'] conv2d_91 (Conv2D) (None, 4, 4, 12) 5184 ['activation_90[0][0]'] concatenate_43 (Concatenate) (None, 4, 4, 216) 0 ['concatenate_42[0][0]', 'conv2d_91[0][0]'] batch_normalization_91 (BatchN (None, 4, 4, 216) 864 ['concatenate_43[0][0]'] ormalization) activation_91 (Activation) (None, 4, 4, 216) 0 ['batch_normalization_91[0][0]'] conv2d_92 (Conv2D) (None, 4, 4, 48) 10368 ['activation_91[0][0]'] batch_normalization_92 (BatchN (None, 4, 4, 48) 192 ['conv2d_92[0][0]'] ormalization) activation_92 (Activation) (None, 4, 4, 48) 0 ['batch_normalization_92[0][0]'] conv2d_93 (Conv2D) (None, 4, 4, 12) 5184 ['activation_92[0][0]'] concatenate_44 (Concatenate) (None, 4, 4, 228) 0 ['concatenate_43[0][0]', 'conv2d_93[0][0]'] batch_normalization_93 (BatchN (None, 4, 4, 228) 912 ['concatenate_44[0][0]'] ormalization) activation_93 (Activation) (None, 4, 4, 228) 0 ['batch_normalization_93[0][0]'] conv2d_94 (Conv2D) (None, 4, 4, 48) 10944 ['activation_93[0][0]'] batch_normalization_94 (BatchN (None, 4, 4, 48) 192 ['conv2d_94[0][0]'] ormalization) activation_94 (Activation) (None, 4, 4, 48) 0 ['batch_normalization_94[0][0]'] conv2d_95 (Conv2D) (None, 4, 4, 12) 5184 ['activation_94[0][0]'] concatenate_45 (Concatenate) (None, 4, 4, 240) 0 ['concatenate_44[0][0]', 'conv2d_95[0][0]'] batch_normalization_95 (BatchN (None, 4, 4, 240) 960 ['concatenate_45[0][0]'] ormalization) activation_95 (Activation) (None, 4, 4, 240) 0 ['batch_normalization_95[0][0]'] conv2d_96 (Conv2D) (None, 4, 4, 48) 11520 ['activation_95[0][0]'] batch_normalization_96 (BatchN (None, 4, 4, 48) 192 ['conv2d_96[0][0]'] ormalization) activation_96 (Activation) (None, 4, 4, 48) 0 ['batch_normalization_96[0][0]'] conv2d_97 (Conv2D) (None, 4, 4, 12) 5184 ['activation_96[0][0]'] concatenate_46 (Concatenate) (None, 4, 4, 252) 0 ['concatenate_45[0][0]', 'conv2d_97[0][0]'] batch_normalization_97 (BatchN (None, 4, 4, 252) 1008 ['concatenate_46[0][0]'] ormalization) activation_97 (Activation) (None, 4, 4, 252) 0 ['batch_normalization_97[0][0]'] conv2d_98 (Conv2D) (None, 4, 4, 48) 12096 ['activation_97[0][0]'] batch_normalization_98 (BatchN (None, 4, 4, 48) 192 ['conv2d_98[0][0]'] ormalization) activation_98 (Activation) (None, 4, 4, 48) 0 ['batch_normalization_98[0][0]'] conv2d_99 (Conv2D) (None, 4, 4, 12) 5184 ['activation_98[0][0]'] concatenate_47 (Concatenate) (None, 4, 4, 264) 0 ['concatenate_46[0][0]', 'conv2d_99[0][0]'] batch_normalization_99 (BatchN (None, 4, 4, 264) 1056 ['concatenate_47[0][0]'] ormalization) activation_99 (Activation) (None, 4, 4, 264) 0 ['batch_normalization_99[0][0]'] conv2d_100 (Conv2D) (None, 4, 4, 48) 12672 ['activation_99[0][0]'] batch_normalization_100 (Batch (None, 4, 4, 48) 192 ['conv2d_100[0][0]'] Normalization) activation_100 (Activation) (None, 4, 4, 48) 0 ['batch_normalization_100[0][0]'] conv2d_101 (Conv2D) (None, 4, 4, 12) 5184 ['activation_100[0][0]'] concatenate_48 (Concatenate) (None, 4, 4, 276) 0 ['concatenate_47[0][0]', 'conv2d_101[0][0]'] batch_normalization_101 (Batch (None, 4, 4, 276) 1104 ['concatenate_48[0][0]'] Normalization) activation_101 (Activation) (None, 4, 4, 276) 0 ['batch_normalization_101[0][0]'] conv2d_102 (Conv2D) (None, 4, 4, 48) 13248 ['activation_101[0][0]'] batch_normalization_102 (Batch (None, 4, 4, 48) 192 ['conv2d_102[0][0]'] Normalization) activation_102 (Activation) (None, 4, 4, 48) 0 ['batch_normalization_102[0][0]'] conv2d_103 (Conv2D) (None, 4, 4, 12) 5184 ['activation_102[0][0]'] concatenate_49 (Concatenate) (None, 4, 4, 288) 0 ['concatenate_48[0][0]', 'conv2d_103[0][0]'] batch_normalization_103 (Batch (None, 4, 4, 288) 1152 ['concatenate_49[0][0]'] Normalization) activation_103 (Activation) (None, 4, 4, 288) 0 ['batch_normalization_103[0][0]'] conv2d_104 (Conv2D) (None, 4, 4, 48) 13824 ['activation_103[0][0]'] batch_normalization_104 (Batch (None, 4, 4, 48) 192 ['conv2d_104[0][0]'] Normalization) activation_104 (Activation) (None, 4, 4, 48) 0 ['batch_normalization_104[0][0]'] conv2d_105 (Conv2D) (None, 4, 4, 12) 5184 ['activation_104[0][0]'] concatenate_50 (Concatenate) (None, 4, 4, 300) 0 ['concatenate_49[0][0]', 'conv2d_105[0][0]'] batch_normalization_105 (Batch (None, 4, 4, 300) 1200 ['concatenate_50[0][0]'] Normalization) activation_105 (Activation) (None, 4, 4, 300) 0 ['batch_normalization_105[0][0]'] conv2d_106 (Conv2D) (None, 4, 4, 48) 14400 ['activation_105[0][0]'] batch_normalization_106 (Batch (None, 4, 4, 48) 192 ['conv2d_106[0][0]'] Normalization) activation_106 (Activation) (None, 4, 4, 48) 0 ['batch_normalization_106[0][0]'] conv2d_107 (Conv2D) (None, 4, 4, 12) 5184 ['activation_106[0][0]'] concatenate_51 (Concatenate) (None, 4, 4, 312) 0 ['concatenate_50[0][0]', 'conv2d_107[0][0]'] batch_normalization_107 (Batch (None, 4, 4, 312) 1248 ['concatenate_51[0][0]'] Normalization) activation_107 (Activation) (None, 4, 4, 312) 0 ['batch_normalization_107[0][0]'] conv2d_108 (Conv2D) (None, 4, 4, 48) 14976 ['activation_107[0][0]'] batch_normalization_108 (Batch (None, 4, 4, 48) 192 ['conv2d_108[0][0]'] Normalization) activation_108 (Activation) (None, 4, 4, 48) 0 ['batch_normalization_108[0][0]'] conv2d_109 (Conv2D) (None, 4, 4, 12) 5184 ['activation_108[0][0]'] concatenate_52 (Concatenate) (None, 4, 4, 324) 0 ['concatenate_51[0][0]', 'conv2d_109[0][0]'] batch_normalization_109 (Batch (None, 4, 4, 324) 1296 ['concatenate_52[0][0]'] Normalization) activation_109 (Activation) (None, 4, 4, 324) 0 ['batch_normalization_109[0][0]'] conv2d_110 (Conv2D) (None, 4, 4, 48) 15552 ['activation_109[0][0]'] batch_normalization_110 (Batch (None, 4, 4, 48) 192 ['conv2d_110[0][0]'] Normalization) activation_110 (Activation) (None, 4, 4, 48) 0 ['batch_normalization_110[0][0]'] conv2d_111 (Conv2D) (None, 4, 4, 12) 5184 ['activation_110[0][0]'] concatenate_53 (Concatenate) (None, 4, 4, 336) 0 ['concatenate_52[0][0]', 'conv2d_111[0][0]'] batch_normalization_111 (Batch (None, 4, 4, 336) 1344 ['concatenate_53[0][0]'] Normalization) activation_111 (Activation) (None, 4, 4, 336) 0 ['batch_normalization_111[0][0]'] conv2d_112 (Conv2D) (None, 4, 4, 48) 16128 ['activation_111[0][0]'] batch_normalization_112 (Batch (None, 4, 4, 48) 192 ['conv2d_112[0][0]'] Normalization) activation_112 (Activation) (None, 4, 4, 48) 0 ['batch_normalization_112[0][0]'] conv2d_113 (Conv2D) (None, 4, 4, 12) 5184 ['activation_112[0][0]'] concatenate_54 (Concatenate) (None, 4, 4, 348) 0 ['concatenate_53[0][0]', 'conv2d_113[0][0]'] batch_normalization_113 (Batch (None, 4, 4, 348) 1392 ['concatenate_54[0][0]'] Normalization) activation_113 (Activation) (None, 4, 4, 348) 0 ['batch_normalization_113[0][0]'] conv2d_114 (Conv2D) (None, 4, 4, 48) 16704 ['activation_113[0][0]'] batch_normalization_114 (Batch (None, 4, 4, 48) 192 ['conv2d_114[0][0]'] Normalization) activation_114 (Activation) (None, 4, 4, 48) 0 ['batch_normalization_114[0][0]'] conv2d_115 (Conv2D) (None, 4, 4, 12) 5184 ['activation_114[0][0]'] concatenate_55 (Concatenate) (None, 4, 4, 360) 0 ['concatenate_54[0][0]', 'conv2d_115[0][0]'] batch_normalization_115 (Batch (None, 4, 4, 360) 1440 ['concatenate_55[0][0]'] Normalization) activation_115 (Activation) (None, 4, 4, 360) 0 ['batch_normalization_115[0][0]'] conv2d_116 (Conv2D) (None, 4, 4, 48) 17280 ['activation_115[0][0]'] batch_normalization_116 (Batch (None, 4, 4, 48) 192 ['conv2d_116[0][0]'] Normalization) activation_116 (Activation) (None, 4, 4, 48) 0 ['batch_normalization_116[0][0]'] conv2d_117 (Conv2D) (None, 4, 4, 12) 5184 ['activation_116[0][0]'] concatenate_56 (Concatenate) (None, 4, 4, 372) 0 ['concatenate_55[0][0]', 'conv2d_117[0][0]'] batch_normalization_117 (Batch (None, 4, 4, 372) 1488 ['concatenate_56[0][0]'] Normalization) activation_117 (Activation) (None, 4, 4, 372) 0 ['batch_normalization_117[0][0]'] conv2d_118 (Conv2D) (None, 4, 4, 48) 17856 ['activation_117[0][0]'] batch_normalization_118 (Batch (None, 4, 4, 48) 192 ['conv2d_118[0][0]'] Normalization) activation_118 (Activation) (None, 4, 4, 48) 0 ['batch_normalization_118[0][0]'] conv2d_119 (Conv2D) (None, 4, 4, 12) 5184 ['activation_118[0][0]'] concatenate_57 (Concatenate) (None, 4, 4, 384) 0 ['concatenate_56[0][0]', 'conv2d_119[0][0]'] batch_normalization_119 (Batch (None, 4, 4, 384) 1536 ['concatenate_57[0][0]'] Normalization) activation_119 (Activation) (None, 4, 4, 384) 0 ['batch_normalization_119[0][0]'] global_average_pooling2d (Glob (None, 384) 0 ['activation_119[0][0]'] alAveragePooling2D) dense (Dense) (None, 10) 3850 ['global_average_pooling2d[0][0]' ] ================================================================================================== Total params: 1,031,938 Trainable params: 1,000,618 Non-trainable params: 31,320 __________________________________________________________________________________________________
 

Description

• Dense blocks and transition blocks serve as the building blocks for the DenseNet architecture.

• Each convolutional layer in a dense block receives as input the concatenation of the feature maps created by all of the layers that came before it. This method's goal is to provide direct information flow across all layers, which will enhance feature reuse and gradient propagation.

• A transition block applies a 1x1 convolution followed by average pooling to minimize the dimensionality of the feature maps. By doing this, the network's computational complexity and parameter count will be reduced.

• Batch normalization is applied before the ReLU activation function by the utility function bn_relu.

• A convolutional layer with a predetermined number of filters and kernel size is applied using the helper function conv.

• The primary function that defines the overall architecture is the densenet function. The number of output classes, the development rate of the network (i.e., the number of filters added to each layer), and the number of blocks in each dense block are all inputs.

• A 32x32x3 picture is used to define the input shape. The first dense block comes after the model's convolutional layer, which is followed by the first dense block, which is then followed by a transition block. For the stated number of dense blocks and transition blocks, this pattern is repeated.

• Following a dense layer with softmax activation for the classification task, the network's final layer is a global average pooling layer.

• Finally, the returned value from the dense net function is used to call the Model function, which will build a Keras model instance with the supplied input and output layers.

Conclusion

State-of-the-art performance in a range of computer vision tasks, such as picture classification, object recognition, and semantic segmentation, has been attained by the deep learning architecture known as DenseNet.

The utilization of dense blocks, which allow for direct information flow across all network levels and enhance feature reuse and gradient propagation, is the main novelty of DenseNet. A very parameter-efficient network that is less prone to overfitting and simpler to optimize is the outcome of this method.

All things considered, DenseNet has shown appreciable advancements over prior state-of-the-art designs and has gained popularity for a variety of computer vision applications.

References

[1]https://medium.com/analytics-vidhya/exploring-densenets-and-a-comparison-with-other-deep-architectures-85f02597400a

[2] https://becominghuman.ai/transfer-learning-part-7-3-densenet-architecture-in-keras-4cdec8b14b15