摘要
MobileNet网络一种针对移动端以及嵌入式视觉应用的轻量网络结构，作者来自Google。其贡献在于使用深度可分离卷积和1x1卷积代替传统的2d图像卷积，来构造轻型权重深度神经网络。在资源和准确率的权衡方面做了大量的实验并且相较于其他在ImageNet分类任务上著名的模型有很好的表现。
网络由来
网络小型化方法：
（1）卷积核分解，使用1×N和N×1的卷积核代替N×N的卷积核
（2）使用bottleneck结构，以SqueezeNet为代表
（3）以低精度浮点数保存，例如Deep Compression
（4）冗余卷积核剪枝及哈弗曼编码-
传统3D图像卷积
传统的3D图像卷积指的是一个多通道的图像(假设图像通道数为M，C>1) 和N个KxK的卷积核（这N个卷积核互不相同）做卷积，即深度神经网络中的卷积层所作的事情。卷积的流程是：
（1）对于其中一个卷积核，分别与图像的每个通道做2d图像卷积，M个通道得到M个卷积结果；
（2）对这M个卷积结果按元素求和，得到一张求和结果；
（3）对N个卷积核中的每个核，重复步骤(1)-(2)，即得到一个通道为N的卷积结果，即featuremap。
该过程的计算量为：((K x K x H x W) x M )x N
标准3D图像卷积 -
深度可分离卷积
深度可分离卷积(depthwith conv) 实际上就是传统3D卷积过程中的步骤(1)，将1个卷积核分别与每个通道进行卷积，得到M个卷积结果
深度可分离卷积 -
1x1 卷积
1x1卷积最初来自于Network in Network网络，主要用于通道压缩上，即改变特征图的通道数，后也用来代替全连接层以减少计算量。
对于一个通道数为M的图像，N个1x1卷积的计算量为：((1 x 1 x H x W) x M) x N
image.png mobilenet方案
对于一个标准的卷积操作，可用一个深度可分离卷积核一个1x1卷积替代，其计算量为：
(K x K x H x W) x M + ((1 x 1 x H x W) x M) x N = H x W x (K x K + N) x M
相比与标准的卷积操作，其计算量减少量为：
(H x W x M x (K x K + N) )/(H x W x K x K x M x N) = 1/N + 1/(K x K)
网络结构
Mobilenet的网络结构非常简单，第一层采用3x3标准的卷积层(stride=2)，其后采用深度可分离卷积和1x1卷积(即conv_dw + conv1x1)作为基础单元，若干个 这样的基础单元串联起来形成不同深度的网络，其中每个1x1卷积后都连接relu激活和batch norm，在conv_dw通过设置stride=2进行下采样。最后采用average pooling代替全连接，ImageNet分类任务上采用一层全连接(units=1000) 和softmax输出类别和置信概率。
mobile net structs
--------------------------------------------------
layer | kh x kw, out, s | out size
--------------------------------------------------
input image (224 x 224 x3)
--------------------------------------------------
conv | 3x3, 32, 2 | 112x112x32
--------------------------------------------------
conv_dw | 3x3, 32dw, 1 | 112x112x32
conv1x1 | 1x1, 64, 1 | 112x112x64
--------------------------------------------------
conv_dw | 3x3, 64dw, 2 | 56x56x64
conv1x1 | 1x1, 128, 1 | 56x56x128
--------------------------------------------------
conv_dw | 3x3, 128dw, 1 | 56x56x128
conv1x1 | 1x1, 128, 1 | 56x56x128
--------------------------------------------------
conv_dw | 3x3, 128dw, 2 | 28x28x128
conv1x1 | 1x1, 256, 1 | 28x28x128
--------------------------------------------------
conv_dw | 3x3, 256dw, 1 | 28x28x256
conv1x1 | 1x1, 256, 1 | 28x28x256
--------------------------------------------------
conv_dw | 3x3, 256dw, 2 | 14x14x256
conv1x1 | 1x1, 512, 1 | 14x14x512
--------------------------------------------------
5x
conv_dw | 3x3, 512dw, 1 | 14x14x512
conv1x1 | 1x1, 512, 1 | 14x14x512
--------------------------------------------------
conv_dw | 3x3, 512dw, 2 | 7x7x512
conv1x1 | 1x1, 1024, 1 | 7x7x1024
--------------------------------------------------
conv_dw | 3x3, 1024dw, 1 | 7x7x1024
conv1x1 | 1x1, 1024, 1 | 7x7x1024
--------------------------------------------------
Avg Pool | 7x7, 1 | 1x1x1024
FC | 1024, 1000 | 1x1x1000
Softmax | Classifier | 1x1x1000
--------------------------------------------------
代码实现
本文采用tensorflow.contrib.layers 模块来构建Mobilenet网络结构，关于tf.nn，tf.layers等api的构建方式参见VGG网络中的相关代码。
# --------------------------Method 1 --------------------------------------------
import tensorflow.contrib.layers as tcl
from tensorflow.contrib.framework import arg_scope
class Mobilenet:
def __init__(self, resolution_inp=224, channel=3, name='resnet50'):
self.name = name
self.channel = channel
self.resolution_inp = resolution_inp
def _depthwise_separable_conv(self, x, num_outputs, kernel_size=3, stride=1, scope=None):
with tf.variable_scope(scope, "dw_blk"):
dw_conv = tcl.separable_conv2d(x, num_outputs=None,
kernel_size=kernel_size,
stride=stride,
depth_multiplier=1)
conv_1x1 = tcl.conv2d(dw_conv, num_outputs=num_outputs, kernel_size=1, stride=1)
return conv_1x1
def __call__(self, x, dropout=0.5, is_training=True):
with tf.variable_scope(self.name) as scope:
with arg_scope([tcl.batch_norm], is_training=is_training, scale=True):
with arg_scope([tcl.conv2d, tcl.separable_conv2d],
activation_fn=tf.nn.relu,
normalizer_fn=tcl.batch_norm,
padding="SAME"):
conv1 = tcl.conv2d(x, 32, kernel_size=3, stride=2)
y = self._depthwise_separable_conv(conv1, 64, 3, stride=1)
y = self._depthwise_separable_conv(y, 128, 3, stride=2)
y = self._depthwise_separable_conv(y, 128, 3, stride=1)
y = self._depthwise_separable_conv(y, 256, 3, stride=2)
y = self._depthwise_separable_conv(y, 256, 3, stride=1)
y = self._depthwise_separable_conv(y, 512, 3, stride=2)
y = self._depthwise_separable_conv(y, 512, 3, stride=1)
y = self._depthwise_separable_conv(y, 512, 3, stride=1)
y = self._depthwise_separable_conv(y, 512, 3, stride=1)
y = self._depthwise_separable_conv(y, 512, 3, stride=1)
y = self._depthwise_separable_conv(y, 512, 3, stride=1)
print("y", y)
y = self._depthwise_separable_conv(y, 512, 3, stride=2)
y = self._depthwise_separable_conv(y, 512, 3, stride=1)
avg_pool = tcl.avg_pool2d(y, 7, stride=1)
flatten = tf.layers.flatten(avg_pool)
self.fc6 = tf.layers.dense(flatten, units=1000, activation=tf.nn.relu)
# dropout = tf.nn.dropout(fc6, keep_prob=0.5)
predictions = tf.nn.softmax(self.fc6)
return predictions
运行
该部分代码包含2部分：计时函数time_tensorflow_run接受一个tf.Session变量和待计算的tensor以及相应的参数字典和打印信息, 统计执行该tensor100次所需要的时间(平均值和方差)；主函数 run_benchmark中初始化了vgg16的3种调用方式，分别统计3中网络在推理(predict) 和梯度计算(后向传递)的时间消耗，详细代码如下：
# -------------------------- Demo and Test --------------------------------------------
batch_size = 16
num_batches = 100
import time
import math
from datetime import datetime
def time_tensorflow_run(session, target, feed, info_string):
"""
calculate time for each session run
:param session: tf.Session
:param target: opterator or tensor need to run with session
:param feed: feed dict for session
:param info_string: info message for print
:return:
"""
num_steps_burn_in = 10 # 预热轮数
total_duration = 0.0 # 总时间
total_duration_squared = 0.0 # 总时间的平方和用以计算方差
for i in range(num_batches + num_steps_burn_in):
start_time = time.time()
_ = session.run(target, feed_dict=feed)
duration = time.time() - start_time
if i >= num_steps_burn_in: # 只考虑预热轮数之后的时间
if not i % 10:
print('[%s] step %d, duration = %.3f' % (datetime.now(), i - num_steps_burn_in, duration))
total_duration += duration
total_duration_squared += duration * duration
mn = total_duration / num_batches # 平均每个batch的时间
vr = total_duration_squared / num_batches - mn * mn # 方差
sd = math.sqrt(vr) # 标准差
print('[%s] %s across %d steps, %.3f +/- %.3f sec/batch' % (datetime.now(), info_string, num_batches, mn, sd))
# test demo
def run_benchmark():
"""
main function for test or demo
:return:
"""
with tf.Graph().as_default():
image_size = 224 # 输入图像尺寸
images = tf.Variable(tf.random_normal([batch_size, image_size, image_size, 3], dtype=tf.float32, stddev=1e-1))
# method 0
# prediction, fc = resnet50(images, training=True)
model = Mobilenet(224, 3)
prediction = model(images, is_training=True)
fc = model.fc6
params = tf.trainable_variables()
for v in params:
print(v)
init = tf.global_variables_initializer()
print("out shape ", prediction)
sess = tf.Session()
print("init...")
sess.run(init)
print("predict..")
writer = tf.summary.FileWriter("./logs")
writer.add_graph(sess.graph)
time_tensorflow_run(sess, prediction, {}, "Forward")
# 用以模拟训练的过程
objective = tf.nn.l2_loss(fc) # 给一个loss
grad = tf.gradients(objective, params) # 相对于loss的 所有模型参数的梯度
print('grad backword')
time_tensorflow_run(sess, grad, {}, "Forward-backward")
writer.close()
if __name__ == '__main__':
run_benchmark()
注: 完整代码可参见个人github工程
参数量
时间效率
参考
https://blog.csdn.net/wfei101/article/details/78310226
https://blog.csdn.net/u013709270/article/details/78722985
1x1卷积
