使用飞桨PaddlePaddle复现用于图像光源处理的深度重照明网络（DRN）
- 一、效果展示
- 二、实现思路
- 冠军模型：Wavelet Decomposed RelightNet (WDRN)
- 经典模型：Norm-Relighting-U-Net (NRUNet)
- 本次项目：Deep Relighting Network (DRN)
- 三、模型结构复现
- 1.DRN的大架构
- 2.场景再转换网络
- 卷积块
- 反卷积块
- 下采样模块
- 残差块
- 上采样模块
- 将上述模块组装成场景再转换网络
- 查看场景再转换网络结构
- 3.阴影先验估计网络
- 查看阴影先验估计网络结构
- 4.阴影区鉴别器——重新渲染
- 5.将上述模块组装成DRN
- 6.查看DRN网络结构
- 四、数据准备
- 1.解压数据集
- 2.数据集定义
- 3.展示训练数据
- 五、模型训练
- 保存模型
- 六、总结与展望
- 个人简介
基于深度学习的图像重照明实践是“AI大陆”上实践类系列课程之一，项目实现任务为图像重照明，是人工智能计算机视觉图像处理领域的一个典型的图像增强任务。
本项目来源于“AIM 2020 Relighting Challenge Track 1 one-to-one relighting”，是一个图像处理类的国际挑战赛的一个子任务。AIM全称为Advances in Image Manipulation，由苏黎世联邦理工学院（ETH Zurich, Switzerland）主办，吸引了国内外80余支队伍参赛，受到了工业界中华为、高通、英伟达、谷歌等国际科技企业的资助。该竞赛今年共设有16个赛道，涵盖了图像、视频处理的多个热门研究方向。重光照技术广泛应用于增强现实、视觉特效和光照补偿中，该项技术的进步将极大促进计算机视觉与计算机图形学的发展。
本项目的主讲老师是中科院自动化研究所助研李成华老师，在成华老师的指导下，我基于飞桨PaddlePaddle复现深度重照明网络（DRN）
参考资料：
- AI-Talks：入门级竞赛全流程讲解
- b站：基于深度学习的图像重照明实践
- CSDN：基于深度学习的图像重照明实践学习笔记
- GitHub：Deep Relighting Network (DRN) for Image Light Source Manipulation
一、效果展示
图像重照明任务一般是给定一个输入图片，输出一个重照明后的图片，重照明就是说输出的图片中场景光照效果不同了。图像重照明相当于重新生成了一个图片，只是通过人工智能算法改变了输入图像中的像素值的大小，使得像素的亮度发生规律性的改变，整体看起来就是图像场景的光照发生了变化。
左边是输入，右边是模型输出，多看几张对比我们可以发现：
- 模型输出的图片清晰度相对输入图片来说较低，大部分图片都有一种模糊的感觉，特别是对应原图中的黑色区域会更加模糊，视觉体验极其不好；
- 从场景照明光源位置来看，基本发生了一致性的变化，比如输入图片场景的光源都是在正北方向（North），输出图片场景的光源都是在正东方向（East）；
- 从场景照明光线色温来看，基本发生了比较一致性的变化，比如输入图片场景的色温偏冷色，输出图片场景的色温偏暖色。
二、实现思路
物体和光源的相对位置不同，所呈现出的阴影不同，如下图所示：
我们的目标是将输入图像从一组预定义的照明设置转换为另一组预定义的照明设置（光源位置的改变）。
大体上来说就是将输入图片放入我们的“黑盒”中从而得到输出图片，而这个“黑盒”就是我们所使用的模型（Neural Network）,"黑盒"中则有着大量的参数需要我们去通过输入输出进行调整（ W 1 W_1 W1， W 2 W_2 W2等），这也就是我们所常说的模型训练过程。
冠军模型：Wavelet Decomposed RelightNet (WDRN)
对应论文：WDRN : A Wavelet Decomposed RelightNet for Image Relighting
经典模型：Norm-Relighting-U-Net (NRUNet)
对应论文：AIM 2020: Scene Relighting and Illumination Estimation Challenge
本次项目：Deep Relighting Network (DRN)
对应GitHub仓库：Deep Relighting Network (DRN) for Image Light Source Manipulation
三、模型结构复现
将代码与论文里的模型结构一一对应：
Deep Relighting Networks for Image Light Source Manipulation
- 论文地址：https://arxiv.org/abs/2008.08298
- 代码仓库：https://github.com/WangLiwen1994/DeepRelight
1.DRN的大架构
以上是DRN的作者开源的用pytorch实现的代码，将代码与图片结合，思路就变得非常清晰了：
- 将输入图片分别送入场景再转换网络和阴影先验估计网络
- 把以上两个网络的输出进行重新渲染并得到输出结果
所以其实只要弄清楚这三者分别是怎么做的，DRN就能很快地搭建出来，论文里也详细说明了场景再转换网络、阴影先验估计网络和重新渲染是怎么实现的，下面用飞桨PaddlePaddle进行复现。
2.场景再转换网络
场景重构的目的是从图像中提取图像的固有结构信息，从而消除光照效应。
这是一个类似于Pix2Pix的自动编码器结构，如上图所示，该网络采用具有跳接的自动编码器结构来将浅特征传输到端部。首先，对输入图像进行下采样（由图中的“DBP”执行），以此找到场景的区别特征。在每次下采样过程之后，通道将加倍，以尽可能地保留信息。这些特征具有较大的感受野，包含了大量的全局信息，有利于光照估计和操纵。接下来，向上四个块采样（由图中的“UBP”操作）将特征映射回原始大小。最后特征映射进一步与由卷积层作用的特征选择过程聚合，卷积层将通道从64减少到32。
下面用PaddlePaddle实现这一过程：
# 导入必要的库
import paddle
import paddle.nn as nn
卷积块
场景再转换网络中，无论是下采样还是上采样的过程，都使用了卷积的操作，这里对卷积的操作做了小小的优化。
# 卷积块
class ConvBlock(nn.Layer):def __init__(self, input_size, output_size, kernel_size, stride, padding, bias=True, isuseBN=True, groups=1):super(ConvBlock, self).__init__()self.isuseBN = isuseBNself.conv = nn.Conv2D(input_size, output_size, kernel_size, stride, padding, bias_attr=bias,groups=groups)if self.isuseBN:self.bn = nn.InstanceNorm2D(output_size)self.act = nn.ReLU()def forward(self, x):out = self.conv(x)if self.isuseBN:out = self.bn(out)out = self.act(out)return out
反卷积块
场景再转换网络中，无论是下采样还是上采样的过程，都使用了反卷积的操作，这里对反卷积的操作做了小小的优化。
# 反卷积块
class DeconvBlock(nn.Layer):def __init__(self, input_size, output_size, kernel_size, stride, padding, isuseBN=True, bias=True):super(DeconvBlock, self).__init__()self.isuseBN = isuseBNself.deconv = nn.Conv2DTranspose(input_size, output_size, kernel_size, stride, padding, bias_attr=bias)if self.isuseBN:self.bn = nn.InstanceNorm2D(output_size)self.act = nn.ReLU()def forward(self, x):out = self.deconv(x)if self.isuseBN:out = self.bn(out)return self.act(out)
下采样模块
下采样其实是一种信息降维，可以起到简化问题的作用。
# 下采样模块
class DownBlock(nn.Layer):def __init__(self, input_size):super(DownBlock, self).__init__()output_size = input_size*2self.conv1 = ConvBlock(input_size, output_size, kernel_size=3, stride=2, padding=1, bias=True)self.conv2 = DeconvBlock(output_size, output_size, kernel_size=4, stride=2, padding=1, bias=True)self.conv3 = ConvBlock(output_size, output_size, kernel_size=3, stride=2, padding=1, bias=True)self.local_weight1 = ConvBlock(input_size, output_size, kernel_size=1, stride=1, padding=0, bias=True)self.local_weight2 = ConvBlock(output_size, output_size, kernel_size=1, stride=1, padding=0, bias=True)def forward(self, x):lr = self.conv1(x)hr = self.conv2(lr)residue = self.local_weight1(x) - hrl_residue = self.conv3(residue)lr_weight = self.local_weight2(lr)return lr_weight + l_residue
残差块
在上采样之前，作者将下采样的结果送入残差块。
class ResnetBlock(nn.Layer):def __init__(self, dim, padding_type, norm_layer, activation=nn.ReLU(), use_dropout=False):super(ResnetBlock, self).__init__()self.conv_block = self.build_conv_block(dim, padding_type, norm_layer, activation, use_dropout)def build_conv_block(self, dim, padding_type, norm_layer, activation, use_dropout):conv_block = []p = 0if (padding_type == 'reflect') or (padding_type == 'replicate'):conv_block += [nn.Pad2D([1, 1, 1, 1], mode=padding_type)]elif padding_type == 'zero':p = 1else:raise NotImplementedError('padding [%s] is not implemented' % padding_type)conv_block += [nn.Conv2D(dim, dim, kernel_size=3, padding=p),norm_layer(dim),activation]if use_dropout:conv_block += [nn.Dropout(0.5)]p = 0if (padding_type == 'reflect') or (padding_type == 'replicate'):conv_block += [nn.Pad2D([1, 1, 1, 1], mode=padding_type)]elif padding_type == 'zero':p = 1else:raise NotImplementedError('padding [%s] is not implemented' % padding_type)conv_block += [nn.Conv2D(dim, dim, kernel_size=3, padding=p),norm_layer(dim)]return nn.Sequential(*conv_block)def forward(self, x):out = x + self.conv_block(x)return out
上采样模块
使用上采样将特征映射回原始大小。
# 上采样模块
class UpBlock(nn.Layer):def __init__(self, input_size):super(UpBlock, self).__init__()output_size = input_size//2self.conv1 = DeconvBlock(input_size, output_size, kernel_size=4, stride=2, padding=1, bias=True)self.conv2 = ConvBlock(output_size, output_size, kernel_size=3, stride=2, padding=1, bias=True)self.conv3 = DeconvBlock(output_size, output_size, kernel_size=4, stride=2, padding=1, bias=True)self.local_weight1 = ConvBlock(input_size, output_size, kernel_size=1, stride=1, padding=0, bias=True)self.local_weight2 = ConvBlock(output_size, output_size, kernel_size=1, stride=1, padding=0, bias=True)def forward(self, x):hr = self.conv1(x)lr = self.conv2(hr)residue = self.local_weight1(x) - lrh_residue = self.conv3(residue)hr_weight = self.local_weight2(hr)return hr_weight + h_residue将上述模块组装成场景再转换网络
import paddle.nn as nnclass BP_HDR(nn.Layer):def __init__(self, input_nc, output_nc, ngf=64, n_downsampling=4, n_blocks=9, norm_layer=nn.BatchNorm2D, padding_type='reflect'):assert (n_blocks >= 0)super(BP_HDR, self).__init__()activation = nn.ReLU()self.inlayer = nn.Sequential(nn.Pad2D([3, 3, 3, 3], mode=padding_type), nn.Conv2D(input_nc, ngf, kernel_size=7, padding=0), norm_layer(ngf), activation)### downsampleself.encoder = nn.Sequential()for i in range(n_downsampling):mult = 2 ** iself.encoder.add_sublayer(str(i), DownBlock(ngf * mult))### resnet blocksself.manipulate = nn.Sequential()mult = 2 ** n_downsamplingfor i in range(n_blocks):self.manipulate.add_sublayer(str(i), ResnetBlock(ngf * mult, padding_type=padding_type, activation=activation, norm_layer=norm_layer))### upsampleself.decoder = nn.Sequential()for i in range(n_downsampling):mult = 2 ** (n_downsampling - i)self.decoder.add_sublayer(str(i), UpBlock(ngf * mult))self.shortconect = nn.Sequential(nn.Pad2D([1, 1, 1, 1], mode=padding_type), nn.Conv2D(ngf*2, ngf, kernel_size=3, padding=0), nn.ReLU())def forward(self, input):m_in = self.inlayer(input)m = self.encoder(m_in)m = self.manipulate(m)m = self.decoder(m)m = paddle.concat([m_in,m], axis=1)m = self.shortconect(m)return m
查看场景再转换网络结构
model = BP_HDR(3, 32)
paddle.summary(model, (1, 3, 512, 512))
------------------------------------------------------------------------------Layer (type) Input Shape Output Shape Param #
==============================================================================Pad2D-121 [[1, 3, 512, 512]] [1, 3, 518, 518] 0 Conv2D-289 [[1, 3, 518, 518]] [1, 64, 512, 512] 9,472 BatchNorm2D-115 [[1, 64, 512, 512]] [1, 64, 512, 512] 256 ReLU-255 [[1, 1024, 32, 32]] [1, 1024, 32, 32] 0 Conv2D-290 [[1, 64, 512, 512]] [1, 128, 256, 256] 73,856
InstanceNorm2D-241 [[1, 128, 256, 256]] [1, 128, 256, 256] 256 ReLU-256 [[1, 128, 256, 256]] [1, 128, 256, 256] 0 ConvBlock-169 [[1, 64, 512, 512]] [1, 128, 256, 256] 0
Conv2DTranspose-73 [[1, 128, 256, 256]] [1, 128, 512, 512] 262,272
InstanceNorm2D-242 [[1, 128, 512, 512]] [1, 128, 512, 512] 256 ReLU-257 [[1, 128, 512, 512]] [1, 128, 512, 512] 0 DeconvBlock-73 [[1, 128, 256, 256]] [1, 128, 512, 512] 0 Conv2D-292 [[1, 64, 512, 512]] [1, 128, 512, 512] 8,320
InstanceNorm2D-244 [[1, 128, 512, 512]] [1, 128, 512, 512] 256 ReLU-259 [[1, 128, 512, 512]] [1, 128, 512, 512] 0 ConvBlock-171 [[1, 64, 512, 512]] [1, 128, 512, 512] 0 Conv2D-291 [[1, 128, 512, 512]] [1, 128, 256, 256] 147,584
InstanceNorm2D-243 [[1, 128, 256, 256]] [1, 128, 256, 256] 256 ReLU-258 [[1, 128, 256, 256]] [1, 128, 256, 256] 0 ConvBlock-170 [[1, 128, 512, 512]] [1, 128, 256, 256] 0 Conv2D-293 [[1, 128, 256, 256]] [1, 128, 256, 256] 16,512
InstanceNorm2D-245 [[1, 128, 256, 256]] [1, 128, 256, 256] 256 ReLU-260 [[1, 128, 256, 256]] [1, 128, 256, 256] 0 ConvBlock-172 [[1, 128, 256, 256]] [1, 128, 256, 256] 0 DownBlock-25 [[1, 64, 512, 512]] [1, 128, 256, 256] 0 Conv2D-294 [[1, 128, 256, 256]] [1, 256, 128, 128] 295,168
InstanceNorm2D-246 [[1, 256, 128, 128]] [1, 256, 128, 128] 512 ReLU-261 [[1, 256, 128, 128]] [1, 256, 128, 128] 0 ConvBlock-173 [[1, 128, 256, 256]] [1, 256, 128, 128] 0
Conv2DTranspose-74 [[1, 256, 128, 128]] [1, 256, 256, 256] 1,048,832
InstanceNorm2D-247 [[1, 256, 256, 256]] [1, 256, 256, 256] 512 ReLU-262 [[1, 256, 256, 256]] [1, 256, 256, 256] 0 DeconvBlock-74 [[1, 256, 128, 128]] [1, 256, 256, 256] 0 Conv2D-296 [[1, 128, 256, 256]] [1, 256, 256, 256] 33,024
InstanceNorm2D-249 [[1, 256, 256, 256]] [1, 256, 256, 256] 512 ReLU-264 [[1, 256, 256, 256]] [1, 256, 256, 256] 0 ConvBlock-175 [[1, 128, 256, 256]] [1, 256, 256, 256] 0 Conv2D-295 [[1, 256, 256, 256]] [1, 256, 128, 128] 590,080
InstanceNorm2D-248 [[1, 256, 128, 128]] [1, 256, 128, 128] 512 ReLU-263 [[1, 256, 128, 128]] [1, 256, 128, 128] 0 ConvBlock-174 [[1, 256, 256, 256]] [1, 256, 128, 128] 0 Conv2D-297 [[1, 256, 128, 128]] [1, 256, 128, 128] 65,792
InstanceNorm2D-250 [[1, 256, 128, 128]] [1, 256, 128, 128] 512 ReLU-265 [[1, 256, 128, 128]] [1, 256, 128, 128] 0 ConvBlock-176 [[1, 256, 128, 128]] [1, 256, 128, 128] 0 DownBlock-26 [[1, 128, 256, 256]] [1, 256, 128, 128] 0 Conv2D-298 [[1, 256, 128, 128]] [1, 512, 64, 64] 1,180,160
InstanceNorm2D-251 [[1, 512, 64, 64]] [1, 512, 64, 64] 1,024 ReLU-266 [[1, 512, 64, 64]] [1, 512, 64, 64] 0 ConvBlock-177 [[1, 256, 128, 128]] [1, 512, 64, 64] 0
Conv2DTranspose-75 [[1, 512, 64, 64]] [1, 512, 128, 128] 4,194,816
InstanceNorm2D-252 [[1, 512, 128, 128]] [1, 512, 128, 128] 1,024 ReLU-267 [[1, 512, 128, 128]] [1, 512, 128, 128] 0 DeconvBlock-75 [[1, 512, 64, 64]] [1, 512, 128, 128] 0 Conv2D-300 [[1, 256, 128, 128]] [1, 512, 128, 128] 131,584
InstanceNorm2D-254 [[1, 512, 128, 128]] [1, 512, 128, 128] 1,024 ReLU-269 [[1, 512, 128, 128]] [1, 512, 128, 128] 0 ConvBlock-179 [[1, 256, 128, 128]] [1, 512, 128, 128] 0 Conv2D-299 [[1, 512, 128, 128]] [1, 512, 64, 64] 2,359,808
InstanceNorm2D-253 [[1, 512, 64, 64]] [1, 512, 64, 64] 1,024 ReLU-268 [[1, 512, 64, 64]] [1, 512, 64, 64] 0 ConvBlock-178 [[1, 512, 128, 128]] [1, 512, 64, 64] 0 Conv2D-301 [[1, 512, 64, 64]] [1, 512, 64, 64] 262,656
InstanceNorm2D-255 [[1, 512, 64, 64]] [1, 512, 64, 64] 1,024 ReLU-270 [[1, 512, 64, 64]] [1, 512, 64, 64] 0 ConvBlock-180 [[1, 512, 64, 64]] [1, 512, 64, 64] 0 DownBlock-27 [[1, 256, 128, 128]] [1, 512, 64, 64] 0 Conv2D-302 [[1, 512, 64, 64]] [1, 1024, 32, 32] 4,719,616
InstanceNorm2D-256 [[1, 1024, 32, 32]] [1, 1024, 32, 32] 2,048 ReLU-271 [[1, 1024, 32, 32]] [1, 1024, 32, 32] 0 ConvBlock-181 [[1, 512, 64, 64]] [1, 1024, 32, 32] 0
Conv2DTranspose-76 [[1, 1024, 32, 32]] [1, 1024, 64, 64] 16,778,240
InstanceNorm2D-257 [[1, 1024, 64, 64]] [1, 1024, 64, 64] 2,048 ReLU-272 [[1, 1024, 64, 64]] [1, 1024, 64, 64] 0 DeconvBlock-76 [[1, 1024, 32, 32]] [1, 1024, 64, 64] 0 Conv2D-304 [[1, 512, 64, 64]] [1, 1024, 64, 64] 525,312
InstanceNorm2D-259 [[1, 1024, 64, 64]] [1, 1024, 64, 64] 2,048 ReLU-274 [[1, 1024, 64, 64]] [1, 1024, 64, 64] 0 ConvBlock-183 [[1, 512, 64, 64]] [1, 1024, 64, 64] 0 Conv2D-303 [[1, 1024, 64, 64]] [1, 1024, 32, 32] 9,438,208
InstanceNorm2D-258 [[1, 1024, 32, 32]] [1, 1024, 32, 32] 2,048 ReLU-273 [[1, 1024, 32, 32]] [1, 1024, 32, 32] 0 ConvBlock-182 [[1, 1024, 64, 64]] [1, 1024, 32, 32] 0 Conv2D-305 [[1, 1024, 32, 32]] [1, 1024, 32, 32] 1,049,600
InstanceNorm2D-260 [[1, 1024, 32, 32]] [1, 1024, 32, 32] 2,048 ReLU-275 [[1, 1024, 32, 32]] [1, 1024, 32, 32] 0 ConvBlock-184 [[1, 1024, 32, 32]] [1, 1024, 32, 32] 0 DownBlock-28 [[1, 512, 64, 64]] [1, 1024, 32, 32] 0 Pad2D-122 [[1, 1024, 32, 32]] [1, 1024, 34, 34] 0 Conv2D-306 [[1, 1024, 34, 34]] [1, 1024, 32, 32] 9,438,208 BatchNorm2D-116 [[1, 1024, 32, 32]] [1, 1024, 32, 32] 4,096 Pad2D-123 [[1, 1024, 32, 32]] [1, 1024, 34, 34] 0 Conv2D-307 [[1, 1024, 34, 34]] [1, 1024, 32, 32] 9,438,208 BatchNorm2D-117 [[1, 1024, 32, 32]] [1, 1024, 32, 32] 4,096 ResnetBlock-55 [[1, 1024, 32, 32]] [1, 1024, 32, 32] 0 Pad2D-124 [[1, 1024, 32, 32]] [1, 1024, 34, 34] 0 Conv2D-308 [[1, 1024, 34, 34]] [1, 1024, 32, 32] 9,438,208 BatchNorm2D-118 [[1, 1024, 32, 32]] [1, 1024, 32, 32] 4,096 Pad2D-125 [[1, 1024, 32, 32]] [1, 1024, 34, 34] 0 Conv2D-309 [[1, 1024, 34, 34]] [1, 1024, 32, 32] 9,438,208 BatchNorm2D-119 [[1, 1024, 32, 32]] [1, 1024, 32, 32] 4,096 ResnetBlock-56 [[1, 1024, 32, 32]] [1, 1024, 32, 32] 0 Pad2D-126 [[1, 1024, 32, 32]] [1, 1024, 34, 34] 0 Conv2D-310 [[1, 1024, 34, 34]] [1, 1024, 32, 32] 9,438,208 BatchNorm2D-120 [[1, 1024, 32, 32]] [1, 1024, 32, 32] 4,096 Pad2D-127 [[1, 1024, 32, 32]] [1, 1024, 34, 34] 0 Conv2D-311 [[1, 1024, 34, 34]] [1, 1024, 32, 32] 9,438,208 BatchNorm2D-121 [[1, 1024, 32, 32]] [1, 1024, 32, 32] 4,096 ResnetBlock-57 [[1, 1024, 32, 32]] [1, 1024, 32, 32] 0 Pad2D-128 [[1, 1024, 32, 32]] [1, 1024, 34, 34] 0 Conv2D-312 [[1, 1024, 34, 34]] [1, 1024, 32, 32] 9,438,208 BatchNorm2D-122 [[1, 1024, 32, 32]] [1, 1024, 32, 32] 4,096 Pad2D-129 [[1, 1024, 32, 32]] [1, 1024, 34, 34] 0 Conv2D-313 [[1, 1024, 34, 34]] [1, 1024, 32, 32] 9,438,208 BatchNorm2D-123 [[1, 1024, 32, 32]] [1, 1024, 32, 32] 4,096 ResnetBlock-58 [[1, 1024, 32, 32]] [1, 1024, 32, 32] 0 Pad2D-130 [[1, 1024, 32, 32]] [1, 1024, 34, 34] 0 Conv2D-314 [[1, 1024, 34, 34]] [1, 1024, 32, 32] 9,438,208 BatchNorm2D-124 [[1, 1024, 32, 32]] [1, 1024, 32, 32] 4,096 Pad2D-131 [[1, 1024, 32, 32]] [1, 1024, 34, 34] 0 Conv2D-315 [[1, 1024, 34, 34]] [1, 1024, 32, 32] 9,438,208 BatchNorm2D-125 [[1, 1024, 32, 32]] [1, 1024, 32, 32] 4,096 ResnetBlock-59 [[1, 1024, 32, 32]] [1, 1024, 32, 32] 0 Pad2D-132 [[1, 1024, 32, 32]] [1, 1024, 34, 34] 0 Conv2D-316 [[1, 1024, 34, 34]] [1, 1024, 32, 32] 9,438,208 BatchNorm2D-126 [[1, 1024, 32, 32]] [1, 1024, 32, 32] 4,096 Pad2D-133 [[1, 1024, 32, 32]] [1, 1024, 34, 34] 0 Conv2D-317 [[1, 1024, 34, 34]] [1, 1024, 32, 32] 9,438,208 BatchNorm2D-127 [[1, 1024, 32, 32]] [1, 1024, 32, 32] 4,096 ResnetBlock-60 [[1, 1024, 32, 32]] [1, 1024, 32, 32] 0 Pad2D-134 [[1, 1024, 32, 32]] [1, 1024, 34, 34] 0 Conv2D-318 [[1, 1024, 34, 34]] [1, 1024, 32, 32] 9,438,208 BatchNorm2D-128 [[1, 1024, 32, 32]] [1, 1024, 32, 32] 4,096 Pad2D-135 [[1, 1024, 32, 32]] [1, 1024, 34, 34] 0 Conv2D-319 [[1, 1024, 34, 34]] [1, 1024, 32, 32] 9,438,208 BatchNorm2D-129 [[1, 1024, 32, 32]] [1, 1024, 32, 32] 4,096 ResnetBlock-61 [[1, 1024, 32, 32]] [1, 1024, 32, 32] 0 Pad2D-136 [[1, 1024, 32, 32]] [1, 1024, 34, 34] 0 Conv2D-320 [[1, 1024, 34, 34]] [1, 1024, 32, 32] 9,438,208 BatchNorm2D-130 [[1, 1024, 32, 32]] [1, 1024, 32, 32] 4,096 Pad2D-137 [[1, 1024, 32, 32]] [1, 1024, 34, 34] 0 Conv2D-321 [[1, 1024, 34, 34]] [1, 1024, 32, 32] 9,438,208 BatchNorm2D-131 [[1, 1024, 32, 32]] [1, 1024, 32, 32] 4,096 ResnetBlock-62 [[1, 1024, 32, 32]] [1, 1024, 32, 32] 0 Pad2D-138 [[1, 1024, 32, 32]] [1, 1024, 34, 34] 0 Conv2D-322 [[1, 1024, 34, 34]] [1, 1024, 32, 32] 9,438,208 BatchNorm2D-132 [[1, 1024, 32, 32]] [1, 1024, 32, 32] 4,096 Pad2D-139 [[1, 1024, 32, 32]] [1, 1024, 34, 34] 0 Conv2D-323 [[1, 1024, 34, 34]] [1, 1024, 32, 32] 9,438,208 BatchNorm2D-133 [[1, 1024, 32, 32]] [1, 1024, 32, 32] 4,096 ResnetBlock-63 [[1, 1024, 32, 32]] [1, 1024, 32, 32] 0
Conv2DTranspose-77 [[1, 1024, 32, 32]] [1, 512, 64, 64] 8,389,120
InstanceNorm2D-261 [[1, 512, 64, 64]] [1, 512, 64, 64] 1,024 ReLU-276 [[1, 512, 64, 64]] [1, 512, 64, 64] 0 DeconvBlock-77 [[1, 1024, 32, 32]] [1, 512, 64, 64] 0 Conv2D-324 [[1, 512, 64, 64]] [1, 512, 32, 32] 2,359,808
InstanceNorm2D-262 [[1, 512, 32, 32]] [1, 512, 32, 32] 1,024 ReLU-277 [[1, 512, 32, 32]] [1, 512, 32, 32] 0 ConvBlock-185 [[1, 512, 64, 64]] [1, 512, 32, 32] 0 Conv2D-325 [[1, 1024, 32, 32]] [1, 512, 32, 32] 524,800
InstanceNorm2D-264 [[1, 512, 32, 32]] [1, 512, 32, 32] 1,024 ReLU-279 [[1, 512, 32, 32]] [1, 512, 32, 32] 0 ConvBlock-186 [[1, 1024, 32, 32]] [1, 512, 32, 32] 0
Conv2DTranspose-78 [[1, 512, 32, 32]] [1, 512, 64, 64] 4,194,816
InstanceNorm2D-263 [[1, 512, 64, 64]] [1, 512, 64, 64] 1,024 ReLU-278 [[1, 512, 64, 64]] [1, 512, 64, 64] 0 DeconvBlock-78 [[1, 512, 32, 32]] [1, 512, 64, 64] 0 Conv2D-326 [[1, 512, 64, 64]] [1, 512, 64, 64] 262,656
InstanceNorm2D-265 [[1, 512, 64, 64]] [1, 512, 64, 64] 1,024 ReLU-280 [[1, 512, 64, 64]] [1, 512, 64, 64] 0 ConvBlock-187 [[1, 512, 64, 64]] [1, 512, 64, 64] 0 UpBlock-25 [[1, 1024, 32, 32]] [1, 512, 64, 64] 0
Conv2DTranspose-79 [[1, 512, 64, 64]] [1, 256, 128, 128] 2,097,408
InstanceNorm2D-266 [[1, 256, 128, 128]] [1, 256, 128, 128] 512 ReLU-281 [[1, 256, 128, 128]] [1, 256, 128, 128] 0 DeconvBlock-79 [[1, 512, 64, 64]] [1, 256, 128, 128] 0 Conv2D-327 [[1, 256, 128, 128]] [1, 256, 64, 64] 590,080
InstanceNorm2D-267 [[1, 256, 64, 64]] [1, 256, 64, 64] 512 ReLU-282 [[1, 256, 64, 64]] [1, 256, 64, 64] 0 ConvBlock-188 [[1, 256, 128, 128]] [1, 256, 64, 64] 0 Conv2D-328 [[1, 512, 64, 64]] [1, 256, 64, 64] 131,328
InstanceNorm2D-269 [[1, 256, 64, 64]] [1, 256, 64, 64] 512 ReLU-284 [[1, 256, 64, 64]] [1, 256, 64, 64] 0 ConvBlock-189 [[1, 512, 64, 64]] [1, 256, 64, 64] 0
Conv2DTranspose-80 [[1, 256, 64, 64]] [1, 256, 128, 128] 1,048,832
InstanceNorm2D-268 [[1, 256, 128, 128]] [1, 256, 128, 128] 512 ReLU-283 [[1, 256, 128, 128]] [1, 256, 128, 128] 0 DeconvBlock-80 [[1, 256, 64, 64]] [1, 256, 128, 128] 0 Conv2D-329 [[1, 256, 128, 128]] [1, 256, 128, 128] 65,792
InstanceNorm2D-270 [[1, 256, 128, 128]] [1, 256, 128, 128] 512 ReLU-285 [[1, 256, 128, 128]] [1, 256, 128, 128] 0 ConvBlock-190 [[1, 256, 128, 128]] [1, 256, 128, 128] 0 UpBlock-26 [[1, 512, 64, 64]] [1, 256, 128, 128] 0
Conv2DTranspose-81 [[1, 256, 128, 128]] [1, 128, 256, 256] 524,416
InstanceNorm2D-271 [[1, 128, 256, 256]] [1, 128, 256, 256] 256 ReLU-286 [[1, 128, 256, 256]] [1, 128, 256, 256] 0 DeconvBlock-81 [[1, 256, 128, 128]] [1, 128, 256, 256] 0 Conv2D-330 [[1, 128, 256, 256]] [1, 128, 128, 128] 147,584
InstanceNorm2D-272 [[1, 128, 128, 128]] [1, 128, 128, 128] 256 ReLU-287 [[1, 128, 128, 128]] [1, 128, 128, 128] 0 ConvBlock-191 [[1, 128, 256, 256]] [1, 128, 128, 128] 0 Conv2D-331 [[1, 256, 128, 128]] [1, 128, 128, 128] 32,896
InstanceNorm2D-274 [[1, 128, 128, 128]] [1, 128, 128, 128] 256 ReLU-289 [[1, 128, 128, 128]] [1, 128, 128, 128] 0 ConvBlock-192 [[1, 256, 128, 128]] [1, 128, 128, 128] 0
Conv2DTranspose-82 [[1, 128, 128, 128]] [1, 128, 256, 256] 262,272
InstanceNorm2D-273 [[1, 128, 256, 256]] [1, 128, 256, 256] 256 ReLU-288 [[1, 128, 256, 256]] [1, 128, 256, 256] 0 DeconvBlock-82 [[1, 128, 128, 128]] [1, 128, 256, 256] 0 Conv2D-332 [[1, 128, 256, 256]] [1, 128, 256, 256] 16,512
InstanceNorm2D-275 [[1, 128, 256, 256]] [1, 128, 256, 256] 256 ReLU-290 [[1, 128, 256, 256]] [1, 128, 256, 256] 0 ConvBlock-193 [[1, 128, 256, 256]] [1, 128, 256, 256] 0 UpBlock-27 [[1, 256, 128, 128]] [1, 128, 256, 256] 0
Conv2DTranspose-83 [[1, 128, 256, 256]] [1, 64, 512, 512] 131,136
InstanceNorm2D-276 [[1, 64, 512, 512]] [1, 64, 512, 512] 128 ReLU-291 [[1, 64, 512, 512]] [1, 64, 512, 512] 0 DeconvBlock-83 [[1, 128, 256, 256]] [1, 64, 512, 512] 0 Conv2D-333 [[1, 64, 512, 512]] [1, 64, 256, 256] 36,928
InstanceNorm2D-277 [[1, 64, 256, 256]] [1, 64, 256, 256] 128 ReLU-292 [[1, 64, 256, 256]] [1, 64, 256, 256] 0 ConvBlock-194 [[1, 64, 512, 512]] [1, 64, 256, 256] 0 Conv2D-334 [[1, 128, 256, 256]] [1, 64, 256, 256] 8,256
InstanceNorm2D-279 [[1, 64, 256, 256]] [1, 64, 256, 256] 128 ReLU-294 [[1, 64, 256, 256]] [1, 64, 256, 256] 0 ConvBlock-195 [[1, 128, 256, 256]] [1, 64, 256, 256] 0
Conv2DTranspose-84 [[1, 64, 256, 256]] [1, 64, 512, 512] 65,600
InstanceNorm2D-278 [[1, 64, 512, 512]] [1, 64, 512, 512] 128 ReLU-293 [[1, 64, 512, 512]] [1, 64, 512, 512] 0 DeconvBlock-84 [[1, 64, 256, 256]] [1, 64, 512, 512] 0 Conv2D-335 [[1, 64, 512, 512]] [1, 64, 512, 512] 4,160
InstanceNorm2D-280 [[1, 64, 512, 512]] [1, 64, 512, 512] 128 ReLU-295 [[1, 64, 512, 512]] [1, 64, 512, 512] 0 ConvBlock-196 [[1, 64, 512, 512]] [1, 64, 512, 512] 0 UpBlock-28 [[1, 128, 256, 256]] [1, 64, 512, 512] 0 Pad2D-140 [[1, 128, 512, 512]] [1, 128, 514, 514] 0 Conv2D-336 [[1, 128, 514, 514]] [1, 64, 512, 512] 73,792 ReLU-296 [[1, 64, 512, 512]] [1, 64, 512, 512] 0
==============================================================================
Total params: 234,149,632
Trainable params: 234,075,648
Non-trainable params: 73,984
------------------------------------------------------------------------------
Input size (MB): 3.00
Forward/backward pass size (MB): 10306.71
Params size (MB): 893.21
Estimated Total Size (MB): 11202.92
------------------------------------------------------------------------------{'total_params': 234149632, 'trainable_params': 234075648}
3.阴影先验估计网络
不同的光源会产生不同的灯光效果，例如，产生不同的阴影和色温。为了产生目标光源的光效，作者设计了一种阴影先验估计网络，其结构如下图所示。
该网络采用与场景再转换网络类似的结构。
具体来说，有三大修改：
- 阴影先验估计网络放弃了跳跃连接，因为网络更加关注全局光效应。跳转连接直接将局部特征引入到输出中，使得网络对全局变化的学习变得迟钝。
- 它有另一个鉴别器，集中在阴影区域。
- 地面真目标是目标光源下的图像。
下面用PaddlePaddle实现这一过程：
import paddle.nn as nnclass BP_Light(nn.Layer):def __init__(self, input_nc, output_nc, ngf=64, n_downsampling=4, n_blocks=9, norm_layer=nn.BatchNorm2D, padding_type='reflect'):assert (n_blocks >= 0)super(BP_Light, self).__init__()activation = nn.ReLU()self.inlayer = nn.Sequential(nn.Pad2D([3, 3, 3, 3], mode=padding_type), nn.Conv2D(input_nc, ngf, kernel_size=7, padding=0), norm_layer(ngf), activation)### downsampleself.encoder = nn.Sequential()for i in range(n_downsampling):mult = 2 ** iself.encoder.add_sublayer(str(i), DownBlock(ngf * mult))### resnet blocksself.manipulate = nn.Sequential()mult = 2 ** n_downsamplingfor i in range(n_blocks):self.manipulate.add_sublayer(str(i), ResnetBlock(ngf * mult, padding_type=padding_type, activation=activation, norm_layer=norm_layer))### upsampleself.decoder = nn.Sequential()for i in range(n_downsampling):mult = 2 ** (n_downsampling - i)self.decoder.add_sublayer(str(i), UpBlock(ngf * mult))self.decoder.add_sublayer("dpad", nn.Pad2D([3, 3, 3, 3], mode=padding_type))self.decoder.add_sublayer("dconv", nn.Conv2D(ngf, output_nc, kernel_size=7, padding=0))self.decoder.add_sublayer("dtanh", nn.Tanh())def forward(self, input):m = self.inlayer(input)m = self.encoder(m)m = self.manipulate(m)for idx, layer in enumerate(self.decoder):m = layer(m)if idx == 3:breakreturn m
查看阴影先验估计网络结构
model = BP_Light(3, 32)
paddle.summary(model, (1, 3, 512, 512))
-------------------------------------------------------------------------------Layer (type) Input Shape Output Shape Param #
===============================================================================Pad2D-221 [[1, 3, 512, 512]] [1, 3, 518, 518] 0 Conv2D-518 [[1, 3, 518, 518]] [1, 64, 512, 512] 9,472 BatchNorm2D-210 [[1, 64, 512, 512]] [1, 64, 512, 512] 256 ReLU-447 [[1, 1024, 32, 32]] [1, 1024, 32, 32] 0 Conv2D-519 [[1, 64, 512, 512]] [1, 128, 256, 256] 73,856
InstanceNorm2D-426 [[1, 128, 256, 256]] [1, 128, 256, 256] 256 ReLU-448 [[1, 128, 256, 256]] [1, 128, 256, 256] 0 ConvBlock-300 [[1, 64, 512, 512]] [1, 128, 256, 256] 0
Conv2DTranspose-127 [[1, 128, 256, 256]] [1, 128, 512, 512] 262,272
InstanceNorm2D-427 [[1, 128, 512, 512]] [1, 128, 512, 512] 256 ReLU-449 [[1, 128, 512, 512]] [1, 128, 512, 512] 0 DeconvBlock-127 [[1, 128, 256, 256]] [1, 128, 512, 512] 0 Conv2D-521 [[1, 64, 512, 512]] [1, 128, 512, 512] 8,320
InstanceNorm2D-429 [[1, 128, 512, 512]] [1, 128, 512, 512] 256 ReLU-451 [[1, 128, 512, 512]] [1, 128, 512, 512] 0 ConvBlock-302 [[1, 64, 512, 512]] [1, 128, 512, 512] 0 Conv2D-520 [[1, 128, 512, 512]] [1, 128, 256, 256] 147,584
InstanceNorm2D-428 [[1, 128, 256, 256]] [1, 128, 256, 256] 256 ReLU-450 [[1, 128, 256, 256]] [1, 128, 256, 256] 0 ConvBlock-301 [[1, 128, 512, 512]] [1, 128, 256, 256] 0 Conv2D-522 [[1, 128, 256, 256]] [1, 128, 256, 256] 16,512
InstanceNorm2D-430 [[1, 128, 256, 256]] [1, 128, 256, 256] 256 ReLU-452 [[1, 128, 256, 256]] [1, 128, 256, 256] 0 ConvBlock-303 [[1, 128, 256, 256]] [1, 128, 256, 256] 0 DownBlock-45 [[1, 64, 512, 512]] [1, 128, 256, 256] 0 Conv2D-523 [[1, 128, 256, 256]] [1, 256, 128, 128] 295,168
InstanceNorm2D-431 [[1, 256, 128, 128]] [1, 256, 128, 128] 512 ReLU-453 [[1, 256, 128, 128]] [1, 256, 128, 128] 0 ConvBlock-304 [[1, 128, 256, 256]] [1, 256, 128, 128] 0
Conv2DTranspose-128 [[1, 256, 128, 128]] [1, 256, 256, 256] 1,048,832
InstanceNorm2D-432 [[1, 256, 256, 256]] [1, 256, 256, 256] 512 ReLU-454 [[1, 256, 256, 256]] [1, 256, 256, 256] 0 DeconvBlock-128 [[1, 256, 128, 128]] [1, 256, 256, 256] 0 Conv2D-525 [[1, 128, 256, 256]] [1, 256, 256, 256] 33,024
InstanceNorm2D-434 [[1, 256, 256, 256]] [1, 256, 256, 256] 512 ReLU-456 [[1, 256, 256, 256]] [1, 256, 256, 256] 0 ConvBlock-306 [[1, 128, 256, 256]] [1, 256, 256, 256] 0 Conv2D-524 [[1, 256, 256, 256]] [1, 256, 128, 128] 590,080
InstanceNorm2D-433 [[1, 256, 128, 128]] [1, 256, 128, 128] 512 ReLU-455 [[1, 256, 128, 128]] [1, 256, 128, 128] 0 ConvBlock-305 [[1, 256, 256, 256]] [1, 256, 128, 128] 0 Conv2D-526 [[1, 256, 128, 128]] [1, 256, 128, 128] 65,792
InstanceNorm2D-435 [[1, 256, 128, 128]] [1, 256, 128, 128] 512 ReLU-457 [[1, 256, 128, 128]] [1, 256, 128, 128] 0 ConvBlock-307 [[1, 256, 128, 128]] [1, 256, 128, 128] 0 DownBlock-46 [[1, 128, 256, 256]] [1, 256, 128, 128] 0 Conv2D-527 [[1, 256, 128, 128]] [1, 512, 64, 64] 1,180,160
InstanceNorm2D-436 [[1, 512, 64, 64]] [1, 512, 64, 64] 1,024 ReLU-458 [[1, 512, 64, 64]] [1, 512, 64, 64] 0 ConvBlock-308 [[1, 256, 128, 128]] [1, 512, 64, 64] 0
Conv2DTranspose-129 [[1, 512, 64, 64]] [1, 512, 128, 128] 4,194,816
InstanceNorm2D-437 [[1, 512, 128, 128]] [1, 512, 128, 128] 1,024 ReLU-459 [[1, 512, 128, 128]] [1, 512, 128, 128] 0 DeconvBlock-129 [[1, 512, 64, 64]] [1, 512, 128, 128] 0 Conv2D-529 [[1, 256, 128, 128]] [1, 512, 128, 128] 131,584
InstanceNorm2D-439 [[1, 512, 128, 128]] [1, 512, 128, 128] 1,024 ReLU-461 [[1, 512, 128, 128]] [1, 512, 128, 128] 0 ConvBlock-310 [[1, 256, 128, 128]] [1, 512, 128, 128] 0 Conv2D-528 [[1, 512, 128, 128]] [1, 512, 64, 64] 2,359,808
InstanceNorm2D-438 [[1, 512, 64, 64]] [1, 512, 64, 64] 1,024 ReLU-460 [[1, 512, 64, 64]] [1, 512, 64, 64] 0 ConvBlock-309 [[1, 512, 128, 128]] [1, 512, 64, 64] 0 Conv2D-530 [[1, 512, 64, 64]] [1, 512, 64, 64] 262,656
InstanceNorm2D-440 [[1, 512, 64, 64]] [1, 512, 64, 64] 1,024 ReLU-462 [[1, 512, 64, 64]] [1, 512, 64, 64] 0 ConvBlock-311 [[1, 512, 64, 64]] [1, 512, 64, 64] 0 DownBlock-47 [[1, 256, 128, 128]] [1, 512, 64, 64] 0 Conv2D-531 [[1, 512, 64, 64]] [1, 1024, 32, 32] 4,719,616
InstanceNorm2D-441 [[1, 1024, 32, 32]] [1, 1024, 32, 32] 2,048 ReLU-463 [[1, 1024, 32, 32]] [1, 1024, 32, 32] 0 ConvBlock-312 [[1, 512, 64, 64]] [1, 1024, 32, 32] 0
Conv2DTranspose-130 [[1, 1024, 32, 32]] [1, 1024, 64, 64] 16,778,240
InstanceNorm2D-442 [[1, 1024, 64, 64]] [1, 1024, 64, 64] 2,048 ReLU-464 [[1, 1024, 64, 64]] [1, 1024, 64, 64] 0 DeconvBlock-130 [[1, 1024, 32, 32]] [1, 1024, 64, 64] 0 Conv2D-533 [[1, 512, 64, 64]] [1, 1024, 64, 64] 525,312
InstanceNorm2D-444 [[1, 1024, 64, 64]] [1, 1024, 64, 64] 2,048 ReLU-466 [[1, 1024, 64, 64]] [1, 1024, 64, 64] 0 ConvBlock-314 [[1, 512, 64, 64]] [1, 1024, 64, 64] 0 Conv2D-532 [[1, 1024, 64, 64]] [1, 1024, 32, 32] 9,438,208
InstanceNorm2D-443 [[1, 1024, 32, 32]] [1, 1024, 32, 32] 2,048 ReLU-465 [[1, 1024, 32, 32]] [1, 1024, 32, 32] 0 ConvBlock-313 [[1, 1024, 64, 64]] [1, 1024, 32, 32] 0 Conv2D-534 [[1, 1024, 32, 32]] [1, 1024, 32, 32] 1,049,600
InstanceNorm2D-445 [[1, 1024, 32, 32]] [1, 1024, 32, 32] 2,048 ReLU-467 [[1, 1024, 32, 32]] [1, 1024, 32, 32] 0 ConvBlock-315 [[1, 1024, 32, 32]] [1, 1024, 32, 32] 0 DownBlock-48 [[1, 512, 64, 64]] [1, 1024, 32, 32] 0 Pad2D-222 [[1, 1024, 32, 32]] [1, 1024, 34, 34] 0 Conv2D-535 [[1, 1024, 34, 34]] [1, 1024, 32, 32] 9,438,208 BatchNorm2D-211 [[1, 1024, 32, 32]] [1, 1024, 32, 32] 4,096 Pad2D-223 [[1, 1024, 32, 32]] [1, 1024, 34, 34] 0 Conv2D-536 [[1, 1024, 34, 34]] [1, 1024, 32, 32] 9,438,208 BatchNorm2D-212 [[1, 1024, 32, 32]] [1, 1024, 32, 32] 4,096 ResnetBlock-100 [[1, 1024, 32, 32]] [1, 1024, 32, 32] 0 Pad2D-224 [[1, 1024, 32, 32]] [1, 1024, 34, 34] 0 Conv2D-537 [[1, 1024, 34, 34]] [1, 1024, 32, 32] 9,438,208 BatchNorm2D-213 [[1, 1024, 32, 32]] [1, 1024, 32, 32] 4,096 Pad2D-225 [[1, 1024, 32, 32]] [1, 1024, 34, 34] 0 Conv2D-538 [[1, 1024, 34, 34]] [1, 1024, 32, 32] 9,438,208 BatchNorm2D-214 [[1, 1024, 32, 32]] [1, 1024, 32, 32] 4,096 ResnetBlock-101 [[1, 1024, 32, 32]] [1, 1024, 32, 32] 0 Pad2D-226 [[1, 1024, 32, 32]] [1, 1024, 34, 34] 0 Conv2D-539 [[1, 1024, 34, 34]] [1, 1024, 32, 32] 9,438,208 BatchNorm2D-215 [[1, 1024, 32, 32]] [1, 1024, 32, 32] 4,096 Pad2D-227 [[1, 1024, 32, 32]] [1, 1024, 34, 34] 0 Conv2D-540 [[1, 1024, 34, 34]] [1, 1024, 32, 32] 9,438,208 BatchNorm2D-216 [[1, 1024, 32, 32]] [1, 1024, 32, 32] 4,096 ResnetBlock-102 [[1, 1024, 32, 32]] [1, 1024, 32, 32] 0 Pad2D-228 [[1, 1024, 32, 32]] [1, 1024, 34, 34] 0 Conv2D-541 [[1, 1024, 34, 34]] [1, 1024, 32, 32] 9,438,208 BatchNorm2D-217 [[1, 1024, 32, 32]] [1, 1024, 32, 32] 4,096 Pad2D-229 [[1, 1024, 32, 32]] [1, 1024, 34, 34] 0 Conv2D-542 [[1, 1024, 34, 34]] [1, 1024, 32, 32] 9,438,208 BatchNorm2D-218 [[1, 1024, 32, 32]] [1, 1024, 32, 32] 4,096 ResnetBlock-103 [[1, 1024, 32, 32]] [1, 1024, 32, 32] 0 Pad2D-230 [[1, 1024, 32, 32]] [1, 1024, 34, 34] 0 Conv2D-543 [[1, 1024, 34, 34]] [1, 1024, 32, 32] 9,438,208 BatchNorm2D-219 [[1, 1024, 32, 32]] [1, 1024, 32, 32] 4,096 Pad2D-231 [[1, 1024, 32, 32]] [1, 1024, 34, 34] 0 Conv2D-544 [[1, 1024, 34, 34]] [1, 1024, 32, 32] 9,438,208 BatchNorm2D-220 [[1, 1024, 32, 32]] [1, 1024, 32, 32] 4,096 ResnetBlock-104 [[1, 1024, 32, 32]] [1, 1024, 32, 32] 0 Pad2D-232 [[1, 1024, 32, 32]] [1, 1024, 34, 34] 0 Conv2D-545 [[1, 1024, 34, 34]] [1, 1024, 32, 32] 9,438,208 BatchNorm2D-221 [[1, 1024, 32, 32]] [1, 1024, 32, 32] 4,096 Pad2D-233 [[1, 1024, 32, 32]] [1, 1024, 34, 34] 0 Conv2D-546 [[1, 1024, 34, 34]] [1, 1024, 32, 32] 9,438,208 BatchNorm2D-222 [[1, 1024, 32, 32]] [1, 1024, 32, 32] 4,096 ResnetBlock-105 [[1, 1024, 32, 32]] [1, 1024, 32, 32] 0 Pad2D-234 [[1, 1024, 32, 32]] [1, 1024, 34, 34] 0 Conv2D-547 [[1, 1024, 34, 34]] [1, 1024, 32, 32] 9,438,208 BatchNorm2D-223 [[1, 1024, 32, 32]] [1, 1024, 32, 32] 4,096 Pad2D-235 [[1, 1024, 32, 32]] [1, 1024, 34, 34] 0 Conv2D-548 [[1, 1024, 34, 34]] [1, 1024, 32, 32] 9,438,208 BatchNorm2D-224 [[1, 1024, 32, 32]] [1, 1024, 32, 32] 4,096 ResnetBlock-106 [[1, 1024, 32, 32]] [1, 1024, 32, 32] 0 Pad2D-236 [[1, 1024, 32, 32]] [1, 1024, 34, 34] 0 Conv2D-549 [[1, 1024, 34, 34]] [1, 1024, 32, 32] 9,438,208 BatchNorm2D-225 [[1, 1024, 32, 32]] [1, 1024, 32, 32] 4,096 Pad2D-237 [[1, 1024, 32, 32]] [1, 1024, 34, 34] 0 Conv2D-550 [[1, 1024, 34, 34]] [1, 1024, 32, 32] 9,438,208 BatchNorm2D-226 [[1, 1024, 32, 32]] [1, 1024, 32, 32] 4,096 ResnetBlock-107 [[1, 1024, 32, 32]] [1, 1024, 32, 32] 0 Pad2D-238 [[1, 1024, 32, 32]] [1, 1024, 34, 34] 0 Conv2D-551 [[1, 1024, 34, 34]] [1, 1024, 32, 32] 9,438,208 BatchNorm2D-227 [[1, 1024, 32, 32]] [1, 1024, 32, 32] 4,096 Pad2D-239 [[1, 1024, 32, 32]] [1, 1024, 34, 34] 0 Conv2D-552 [[1, 1024, 34, 34]] [1, 1024, 32, 32] 9,438,208 BatchNorm2D-228 [[1, 1024, 32, 32]] [1, 1024, 32, 32] 4,096 ResnetBlock-108 [[1, 1024, 32, 32]] [1, 1024, 32, 32] 0
Conv2DTranspose-131 [[1, 1024, 32, 32]] [1, 512, 64, 64] 8,389,120
InstanceNorm2D-446 [[1, 512, 64, 64]] [1, 512, 64, 64] 1,024 ReLU-468 [[1, 512, 64, 64]] [1, 512, 64, 64] 0 DeconvBlock-131 [[1, 1024, 32, 32]] [1, 512, 64, 64] 0 Conv2D-553 [[1, 512, 64, 64]] [1, 512, 32, 32] 2,359,808
InstanceNorm2D-447 [[1, 512, 32, 32]] [1, 512, 32, 32] 1,024 ReLU-469 [[1, 512, 32, 32]] [1, 512, 32, 32] 0 ConvBlock-316 [[1, 512, 64, 64]] [1, 512, 32, 32] 0 Conv2D-554 [[1, 1024, 32, 32]] [1, 512, 32, 32] 524,800
InstanceNorm2D-449 [[1, 512, 32, 32]] [1, 512, 32, 32] 1,024 ReLU-471 [[1, 512, 32, 32]] [1, 512, 32, 32] 0 ConvBlock-317 [[1, 1024, 32, 32]] [1, 512, 32, 32] 0
Conv2DTranspose-132 [[1, 512, 32, 32]] [1, 512, 64, 64] 4,194,816
InstanceNorm2D-448 [[1, 512, 64, 64]] [1, 512, 64, 64] 1,024 ReLU-470 [[1, 512, 64, 64]] [1, 512, 64, 64] 0 DeconvBlock-132 [[1, 512, 32, 32]] [1, 512, 64, 64] 0 Conv2D-555 [[1, 512, 64, 64]] [1, 512, 64, 64] 262,656
InstanceNorm2D-450 [[1, 512, 64, 64]] [1, 512, 64, 64] 1,024 ReLU-472 [[1, 512, 64, 64]] [1, 512, 64, 64] 0 ConvBlock-318 [[1, 512, 64, 64]] [1, 512, 64, 64] 0 UpBlock-42 [[1, 1024, 32, 32]] [1, 512, 64, 64] 0
Conv2DTranspose-133 [[1, 512, 64, 64]] [1, 256, 128, 128] 2,097,408
InstanceNorm2D-451 [[1, 256, 128, 128]] [1, 256, 128, 128] 512 ReLU-473 [[1, 256, 128, 128]] [1, 256, 128, 128] 0 DeconvBlock-133 [[1, 512, 64, 64]] [1, 256, 128, 128] 0 Conv2D-556 [[1, 256, 128, 128]] [1, 256, 64, 64] 590,080
InstanceNorm2D-452 [[1, 256, 64, 64]] [1, 256, 64, 64] 512 ReLU-474 [[1, 256, 64, 64]] [1, 256, 64, 64] 0 ConvBlock-319 [[1, 256, 128, 128]] [1, 256, 64, 64] 0 Conv2D-557 [[1, 512, 64, 64]] [1, 256, 64, 64] 131,328
InstanceNorm2D-454 [[1, 256, 64, 64]] [1, 256, 64, 64] 512 ReLU-476 [[1, 256, 64, 64]] [1, 256, 64, 64] 0 ConvBlock-320 [[1, 512, 64, 64]] [1, 256, 64, 64] 0
Conv2DTranspose-134 [[1, 256, 64, 64]] [1, 256, 128, 128] 1,048,832
InstanceNorm2D-453 [[1, 256, 128, 128]] [1, 256, 128, 128] 512 ReLU-475 [[1, 256, 128, 128]] [1, 256, 128, 128] 0 DeconvBlock-134 [[1, 256, 64, 64]] [1, 256, 128, 128] 0 Conv2D-558 [[1, 256, 128, 128]] [1, 256, 128, 128] 65,792
InstanceNorm2D-455 [[1, 256, 128, 128]] [1, 256, 128, 128] 512 ReLU-477 [[1, 256, 128, 128]] [1, 256, 128, 128] 0 ConvBlock-321 [[1, 256, 128, 128]] [1, 256, 128, 128] 0 UpBlock-43 [[1, 512, 64, 64]] [1, 256, 128, 128] 0
Conv2DTranspose-135 [[1, 256, 128, 128]] [1, 128, 256, 256] 524,416
InstanceNorm2D-456 [[1, 128, 256, 256]] [1, 128, 256, 256] 256 ReLU-478 [[1, 128, 256, 256]] [1, 128, 256, 256] 0 DeconvBlock-135 [[1, 256, 128, 128]] [1, 128, 256, 256] 0 Conv2D-559 [[1, 128, 256, 256]] [1, 128, 128, 128] 147,584
InstanceNorm2D-457 [[1, 128, 128, 128]] [1, 128, 128, 128] 256 ReLU-479 [[1, 128, 128, 128]] [1, 128, 128, 128] 0 ConvBlock-322 [[1, 128, 256, 256]] [1, 128, 128, 128] 0 Conv2D-560 [[1, 256, 128, 128]] [1, 128, 128, 128] 32,896
InstanceNorm2D-459 [[1, 128, 128, 128]] [1, 128, 128, 128] 256 ReLU-481 [[1, 128, 128, 128]] [1, 128, 128, 128] 0 ConvBlock-323 [[1, 256, 128, 128]] [1, 128, 128, 128] 0
Conv2DTranspose-136 [[1, 128, 128, 128]] [1, 128, 256, 256] 262,272
InstanceNorm2D-458 [[1, 128, 256, 256]] [1, 128, 256, 256] 256 ReLU-480 [[1, 128, 256, 256]] [1, 128, 256, 256] 0 DeconvBlock-136 [[1, 128, 128, 128]] [1, 128, 256, 256] 0 Conv2D-561 [[1, 128, 256, 256]] [1, 128, 256, 256] 16,512
InstanceNorm2D-460 [[1, 128, 256, 256]] [1, 128, 256, 256] 256 ReLU-482 [[1, 128, 256, 256]] [1, 128, 256, 256] 0 ConvBlock-324 [[1, 128, 256, 256]] [1, 128, 256, 256] 0 UpBlock-44 [[1, 256, 128, 128]] [1, 128, 256, 256] 0
Conv2DTranspose-137 [[1, 128, 256, 256]] [1, 64, 512, 512] 131,136
InstanceNorm2D-461 [[1, 64, 512, 512]] [1, 64, 512, 512] 128 ReLU-483 [[1, 64, 512, 512]] [1, 64, 512, 512] 0 DeconvBlock-137 [[1, 128, 256, 256]] [1, 64, 512, 512] 0 Conv2D-562 [[1, 64, 512, 512]] [1, 64, 256, 256] 36,928
InstanceNorm2D-462 [[1, 64, 256, 256]] [1, 64, 256, 256] 128 ReLU-484 [[1, 64, 256, 256]] [1, 64, 256, 256] 0 ConvBlock-325 [[1, 64, 512, 512]] [1, 64, 256, 256] 0 Conv2D-563 [[1, 128, 256, 256]] [1, 64, 256, 256] 8,256
InstanceNorm2D-464 [[1, 64, 256, 256]] [1, 64, 256, 256] 128 ReLU-486 [[1, 64, 256, 256]] [1, 64, 256, 256] 0 ConvBlock-326 [[1, 128, 256, 256]] [1, 64, 256, 256] 0
Conv2DTranspose-138 [[1, 64, 256, 256]] [1, 64, 512, 512] 65,600
InstanceNorm2D-463 [[1, 64, 512, 512]] [1, 64, 512, 512] 128 ReLU-485 [[1, 64, 512, 512]] [1, 64, 512, 512] 0 DeconvBlock-138 [[1, 64, 256, 256]] [1, 64, 512, 512] 0 Conv2D-564 [[1, 64, 512, 512]] [1, 64, 512, 512] 4,160
InstanceNorm2D-465 [[1, 64, 512, 512]] [1, 64, 512, 512] 128 ReLU-487 [[1, 64, 512, 512]] [1, 64, 512, 512] 0 ConvBlock-327 [[1, 64, 512, 512]] [1, 64, 512, 512] 0 UpBlock-45 [[1, 128, 256, 256]] [1, 64, 512, 512] 0
===============================================================================
Total params: 234,075,840
Trainable params: 234,001,856
Non-trainable params: 73,984
-------------------------------------------------------------------------------
Input size (MB): 3.00
Forward/backward pass size (MB): 9792.70
Params size (MB): 892.93
Estimated Total Size (MB): 10688.63
-------------------------------------------------------------------------------{'total_params': 234075840, 'trainable_params': 234001856}
4.阴影区鉴别器——重新渲染
阴影区域鉴别器采用与文献Image-to-Image Translation with Conditional Adversarial Networks相同的结构，堆叠四个步幅卷积层，逐步提取全局特征表示。
为了聚焦阴影区域，首先通过 z = m i n （ α ， x ） z=min（α，x） z=min（α，x）校正估计以聚焦低强度（暗，通常是阴影）区域，其中符号x表示估计的像素强度。术语z表示将输入到鉴别器的校正值。术语α是阴影灵敏度的预定义阈值（根据经验，它设置为0.059=15/255）。
阴影区鉴别器的输出是重新渲染器的输入，分别来自场景再转换和阴影先验网络的两个特征映射。
import paddle.nn as nnclass ConvBlock2D(nn.Layer):def __init__(self, input_nc, output_nc, kernel_size, stride=1, padding=0, dilation=1, groups=1, bias=True,pad_type='zero', norm=None, acti='lrelu'):super(ConvBlock2D, self).__init__()self.use_bias = bias# initialize padding if pad_type == 'reflect':self.pad = nn.Pad2D([padding, padding, padding, padding], mode=pad_type)elif pad_type == 'zero':self.pad = nn.Pad2D([0, 0, 0, 0], mode='replicate')elif pad_type == 'replicate':self.pad = nn.Pad2D([padding, padding, padding, padding], mode=pad_type)else:assert 0, "Unsupported padding type: {}".format(pad_type)# initialize normalizationif norm == 'batch':self.norm = nn.BatchNorm2D(output_nc)elif norm == 'instance':self.norm = nn.InstanceNorm2D(output_nc)elif norm is None or norm == 'spectral':self.norm = Noneelse:assert 0, "Unsupported normalization: {}".format(norm)# initialize activationif acti == 'relu':self.acti = nn.ReLU()elif acti == 'lrelu':self.acti = nn.LeakyReLU()elif acti == 'prelu':self.acti = nn.PReLU()elif acti == 'elu':self.acti = nn.ELU()elif acti == 'tanh':self.acti = nn.Tanh()elif acti == 'sigmoid':self.acti = nn.Sigmoid()elif acti is None:self.acti = Noneelse:assert 0, "Unsupported activation: {}".format(acti)# initialize convolutionif norm == 'spectral':self.conv = nn.SpectralNorm(nn.Conv2d(input_nc, output_nc, kernel_size, stride, dilation=dilation, groups=groups,bias_attr=self.use_bias))else:self.conv = nn.Conv2D(input_nc, output_nc, kernel_size, stride, dilation=dilation, groups=groups,bias_attr=self.use_bias)def forward(self, x):x = self.conv(self.pad(x))if self.norm:x = self.norm(x)if self.acti:x = self.acti(x)return xclass FusionLayer(nn.Layer):def __init__(self, inchannel, outchannel, reduction=16):super(FusionLayer, self).__init__()self.mergeFeather = ConvBlock2D(inchannel, inchannel, kernel_size=3, stride=1, padding=1)self.avg_pool = nn.AdaptiveAvgPool2D(1)self.fc = nn.Sequential(nn.Linear(inchannel, inchannel // reduction, bias_attr=False),nn.ReLU(),nn.Linear(inchannel // reduction, inchannel, bias_attr=False),nn.Sigmoid())self.outlayer = ConvBlock2D(inchannel, outchannel, 1, 1, 0, bias=True)def forward(self, x):b, c = paddle.shape(x)[0], paddle.shape(x)[1]y = self.mergeFeather(x)y = self.avg_pool(y)y = paddle.reshape(y, [b, c])y = self.fc(y)y = paddle.reshape(y, [b, c, 1, 1])y = x * y.expand_as(x)y = y + xy = self.outlayer(y)return y
import paddle.nn as nnclass MultiDilationResnetBlock_attention(nn.Layer):def __init__(self, input_nc_each, kernel_size=3, stride=1, padding=1, dilation=1, groups=1, bias=True,pad_type='reflect', norm='instance', acti='relu', use_dropout=False):super(MultiDilationResnetBlock_attention, self).__init__()### hard code, 4 dilation levelsinput_nc = input_nc_each * 2self.branch1 = ConvBlock2D(input_nc, input_nc // 8, kernel_size=3, stride=1, padding=2, dilation=2, groups=1,bias=True, pad_type=pad_type, norm=norm, acti='relu')self.branch2 = ConvBlock2D(input_nc, input_nc // 8, kernel_size=3, stride=1, padding=3, dilation=3, groups=1,bias=True, pad_type=pad_type, norm=norm, acti='relu')self.branch3 = ConvBlock2D(input_nc, input_nc // 8, kernel_size=3, stride=1, padding=4, dilation=4, groups=1,bias=True, pad_type=pad_type, norm=norm, acti='relu')self.branch4 = ConvBlock2D(input_nc, input_nc // 8, kernel_size=3, stride=1, padding=5, dilation=5, groups=1,bias=True, pad_type=pad_type, norm=norm, acti='relu')self.branch5 = ConvBlock2D(input_nc, input_nc // 8, kernel_size=3, stride=1, padding=6, dilation=6, groups=1,bias=True, pad_type=pad_type, norm=norm, acti='relu')self.branch6 = ConvBlock2D(input_nc, input_nc // 8, kernel_size=3, stride=1, padding=8, dilation=8, groups=1,bias=True, pad_type=pad_type, norm=norm, acti='relu')self.branch7 = ConvBlock2D(input_nc, input_nc // 8, kernel_size=3, stride=1, padding=10, dilation=10, groups=1,bias=True, pad_type=pad_type, norm=norm, acti='relu')self.branch8 = ConvBlock2D(input_nc, input_nc // 8, kernel_size=3, stride=1, padding=12, dilation=12, groups=1,bias=True, pad_type=pad_type, norm=norm, acti='relu')self.fusion = FusionLayer(inchannel=input_nc, outchannel=input_nc_each, reduction=8)def forward(self, x_hdr, x_relight):x = paddle.concat([x_hdr, x_relight], axis=1)d1 = self.branch1(x)d2 = self.branch2(x)d3 = self.branch3(x)d4 = self.branch4(x)d5 = self.branch5(x)d6 = self.branch6(x)d7 = self.branch7(x)d8 = self.branch8(x)d9 = paddle.concat((d1, d2, d3, d4, d5, d6, d7, d8), axis=1)out = x_relight + self.fusion(d9)return out
5.将上述模块组装成DRN
import paddle.nn as nnclass DRN(nn.Layer):def __init__(self, input_nc, output_nc, ngf=64, n_downsampling=4, n_blocks=9, norm_layer=nn.BatchNorm2D,padding_type='reflect'):super(DRN, self).__init__()self.toHDR = BP_HDR(input_nc, output_nc, ngf=32, n_downsampling=n_downsampling, n_blocks=n_blocks, norm_layer=norm_layer,padding_type=padding_type)self.estLight = BP_Light(input_nc, output_nc, ngf=32, n_downsampling=n_downsampling, n_blocks=n_blocks, norm_layer=norm_layer,padding_type=padding_type)self.toHDR_channel = nn.Sequential(nn.Pad2D([1, 1, 1, 1], mode='reflect'), nn.Conv2D(32, 64, kernel_size=3, padding=0), nn.ReLU())self.estLight_channel = nn.Sequential(nn.Pad2D([1, 1, 1, 1], mode='reflect'), nn.Conv2D(32, 64, kernel_size=3, padding=0), nn.ReLU())self.comb =MultiDilationResnetBlock_attention(ngf)self.out = nn.Sequential(nn.Pad2D([3, 3, 3, 3], mode='reflect'), nn.Conv2D(ngf, output_nc, kernel_size=7, padding=0), nn.Tanh())def forward(self, input):# print(input.shape)m_hdr = self.toHDR(input)m_estLight = self.estLight(input)m_hdr = self.toHDR_channel(m_hdr)m_estLight = self.estLight_channel(m_estLight)m_fea = self.comb(x_hdr=m_hdr, x_relight=m_estLight)out = self.out(m_fea)return out
6.查看DRN网络结构
model = DRN(3, 3)
paddle.summary(model, (1, 3, 512, 512))
-------------------------------------------------------------------------------------------------Layer (type) Input Shape Output Shape Param #
=================================================================================================Pad2D-743 [[1, 3, 512, 512]] [1, 3, 518, 518] 0 Conv2D-1527 [[1, 3, 518, 518]] [1, 32, 512, 512] 4,736 BatchNorm2D-533 [[1, 32, 512, 512]] [1, 32, 512, 512] 128 ReLU-1318 [[1, 512, 32, 32]] [1, 512, 32, 32] 0 Conv2D-1528 [[1, 32, 512, 512]] [1, 64, 256, 256] 18,496 InstanceNorm2D-1233 [[1, 64, 256, 256]] [1, 64, 256, 256] 128 ReLU-1319 [[1, 64, 256, 256]] [1, 64, 256, 256] 0 ConvBlock-785 [[1, 32, 512, 512]] [1, 64, 256, 256] 0 Conv2DTranspose-337 [[1, 64, 256, 256]] [1, 64, 512, 512] 65,600 InstanceNorm2D-1234 [[1, 64, 512, 512]] [1, 64, 512, 512] 128 ReLU-1320 [[1, 64, 512, 512]] [1, 64, 512, 512] 0 DeconvBlock-337 [[1, 64, 256, 256]] [1, 64, 512, 512] 0 Conv2D-1530 [[1, 32, 512, 512]] [1, 64, 512, 512] 2,112 InstanceNorm2D-1236 [[1, 64, 512, 512]] [1, 64, 512, 512] 128 ReLU-1322 [[1, 64, 512, 512]] [1, 64, 512, 512] 0 ConvBlock-787 [[1, 32, 512, 512]] [1, 64, 512, 512] 0 Conv2D-1529 [[1, 64, 512, 512]] [1, 64, 256, 256] 36,928 InstanceNorm2D-1235 [[1, 64, 256, 256]] [1, 64, 256, 256] 128 ReLU-1321 [[1, 64, 256, 256]] [1, 64, 256, 256] 0 ConvBlock-786 [[1, 64, 512, 512]] [1, 64, 256, 256] 0 Conv2D-1531 [[1, 64, 256, 256]] [1, 64, 256, 256] 4,160 InstanceNorm2D-1237 [[1, 64, 256, 256]] [1, 64, 256, 256] 128 ReLU-1323 [[1, 64, 256, 256]] [1, 64, 256, 256] 0 ConvBlock-788 [[1, 64, 256, 256]] [1, 64, 256, 256] 0 DownBlock-113 [[1, 32, 512, 512]] [1, 64, 256, 256] 0 Conv2D-1532 [[1, 64, 256, 256]] [1, 128, 128, 128] 73,856 InstanceNorm2D-1238 [[1, 128, 128, 128]] [1, 128, 128, 128] 256 ReLU-1324 [[1, 128, 128, 128]] [1, 128, 128, 128] 0 ConvBlock-789 [[1, 64, 256, 256]] [1, 128, 128, 128] 0 Conv2DTranspose-338 [[1, 128, 128, 128]] [1, 128, 256, 256] 262,272 InstanceNorm2D-1239 [[1, 128, 256, 256]] [1, 128, 256, 256] 256 ReLU-1325 [[1, 128, 256, 256]] [1, 128, 256, 256] 0 DeconvBlock-338 [[1, 128, 128, 128]] [1, 128, 256, 256] 0 Conv2D-1534 [[1, 64, 256, 256]] [1, 128, 256, 256] 8,320 InstanceNorm2D-1241 [[1, 128, 256, 256]] [1, 128, 256, 256] 256 ReLU-1327 [[1, 128, 256, 256]] [1, 128, 256, 256] 0 ConvBlock-791 [[1, 64, 256, 256]] [1, 128, 256, 256] 0 Conv2D-1533 [[1, 128, 256, 256]] [1, 128, 128, 128] 147,584 InstanceNorm2D-1240 [[1, 128, 128, 128]] [1, 128, 128, 128] 256 ReLU-1326 [[1, 128, 128, 128]] [1, 128, 128, 128] 0 ConvBlock-790 [[1, 128, 256, 256]] [1, 128, 128, 128] 0 Conv2D-1535 [[1, 128, 128, 128]] [1, 128, 128, 128] 16,512 InstanceNorm2D-1242 [[1, 128, 128, 128]] [1, 128, 128, 128] 256 ReLU-1328 [[1, 128, 128, 128]] [1, 128, 128, 128] 0 ConvBlock-792 [[1, 128, 128, 128]] [1, 128, 128, 128] 0 DownBlock-114 [[1, 64, 256, 256]] [1, 128, 128, 128] 0 Conv2D-1536 [[1, 128, 128, 128]] [1, 256, 64, 64] 295,168 InstanceNorm2D-1243 [[1, 256, 64, 64]] [1, 256, 64, 64] 512 ReLU-1329 [[1, 256, 64, 64]] [1, 256, 64, 64] 0 ConvBlock-793 [[1, 128, 128, 128]] [1, 256, 64, 64] 0 Conv2DTranspose-339 [[1, 256, 64, 64]] [1, 256, 128, 128] 1,048,832 InstanceNorm2D-1244 [[1, 256, 128, 128]] [1, 256, 128, 128] 512 ReLU-1330 [[1, 256, 128, 128]] [1, 256, 128, 128] 0 DeconvBlock-339 [[1, 256, 64, 64]] [1, 256, 128, 128] 0 Conv2D-1538 [[1, 128, 128, 128]] [1, 256, 128, 128] 33,024 InstanceNorm2D-1246 [[1, 256, 128, 128]] [1, 256, 128, 128] 512 ReLU-1332 [[1, 256, 128, 128]] [1, 256, 128, 128] 0 ConvBlock-795 [[1, 128, 128, 128]] [1, 256, 128, 128] 0 Conv2D-1537 [[1, 256, 128, 128]] [1, 256, 64, 64] 590,080 InstanceNorm2D-1245 [[1, 256, 64, 64]] [1, 256, 64, 64] 512 ReLU-1331 [[1, 256, 64, 64]] [1, 256, 64, 64] 0 ConvBlock-794 [[1, 256, 128, 128]] [1, 256, 64, 64] 0 Conv2D-1539 [[1, 256, 64, 64]] [1, 256, 64, 64] 65,792 InstanceNorm2D-1247 [[1, 256, 64, 64]] [1, 256, 64, 64] 512 ReLU-1333 [[1, 256, 64, 64]] [1, 256, 64, 64] 0 ConvBlock-796 [[1, 256, 64, 64]] [1, 256, 64, 64] 0 DownBlock-115 [[1, 128, 128, 128]] [1, 256, 64, 64] 0 Conv2D-1540 [[1, 256, 64, 64]] [1, 512, 32, 32] 1,180,160 InstanceNorm2D-1248 [[1, 512, 32, 32]] [1, 512, 32, 32] 1,024 ReLU-1334 [[1, 512, 32, 32]] [1, 512, 32, 32] 0 ConvBlock-797 [[1, 256, 64, 64]] [1, 512, 32, 32] 0 Conv2DTranspose-340 [[1, 512, 32, 32]] [1, 512, 64, 64] 4,194,816 InstanceNorm2D-1249 [[1, 512, 64, 64]] [1, 512, 64, 64] 1,024 ReLU-1335 [[1, 512, 64, 64]] [1, 512, 64, 64] 0 DeconvBlock-340 [[1, 512, 32, 32]] [1, 512, 64, 64] 0 Conv2D-1542 [[1, 256, 64, 64]] [1, 512, 64, 64] 131,584 InstanceNorm2D-1251 [[1, 512, 64, 64]] [1, 512, 64, 64] 1,024 ReLU-1337 [[1, 512, 64, 64]] [1, 512, 64, 64] 0 ConvBlock-799 [[1, 256, 64, 64]] [1, 512, 64, 64] 0 Conv2D-1541 [[1, 512, 64, 64]] [1, 512, 32, 32] 2,359,808 InstanceNorm2D-1250 [[1, 512, 32, 32]] [1, 512, 32, 32] 1,024 ReLU-1336 [[1, 512, 32, 32]] [1, 512, 32, 32] 0 ConvBlock-798 [[1, 512, 64, 64]] [1, 512, 32, 32] 0 Conv2D-1543 [[1, 512, 32, 32]] [1, 512, 32, 32] 262,656 InstanceNorm2D-1252 [[1, 512, 32, 32]] [1, 512, 32, 32] 1,024 ReLU-1338 [[1, 512, 32, 32]] [1, 512, 32, 32] 0 ConvBlock-800 [[1, 512, 32, 32]] [1, 512, 32, 32] 0 DownBlock-116 [[1, 256, 64, 64]] [1, 512, 32, 32] 0 Pad2D-744 [[1, 512, 32, 32]] [1, 512, 34, 34] 0 Conv2D-1544 [[1, 512, 34, 34]] [1, 512, 32, 32] 2,359,808 BatchNorm2D-534 [[1, 512, 32, 32]] [1, 512, 32, 32] 2,048 Pad2D-745 [[1, 512, 32, 32]] [1, 512, 34, 34] 0 Conv2D-1545 [[1, 512, 34, 34]] [1, 512, 32, 32] 2,359,808 BatchNorm2D-535 [[1, 512, 32, 32]] [1, 512, 32, 32] 2,048 ResnetBlock-253 [[1, 512, 32, 32]] [1, 512, 32, 32] 0 Pad2D-746 [[1, 512, 32, 32]] [1, 512, 34, 34] 0 Conv2D-1546 [[1, 512, 34, 34]] [1, 512, 32, 32] 2,359,808 BatchNorm2D-536 [[1, 512, 32, 32]] [1, 512, 32, 32] 2,048 Pad2D-747 [[1, 512, 32, 32]] [1, 512, 34, 34] 0 Conv2D-1547 [[1, 512, 34, 34]] [1, 512, 32, 32] 2,359,808 BatchNorm2D-537 [[1, 512, 32, 32]] [1, 512, 32, 32] 2,048 ResnetBlock-254 [[1, 512, 32, 32]] [1, 512, 32, 32] 0 Pad2D-748 [[1, 512, 32, 32]] [1, 512, 34, 34] 0 Conv2D-1548 [[1, 512, 34, 34]] [1, 512, 32, 32] 2,359,808 BatchNorm2D-538 [[1, 512, 32, 32]] [1, 512, 32, 32] 2,048 Pad2D-749 [[1, 512, 32, 32]] [1, 512, 34, 34] 0 Conv2D-1549 [[1, 512, 34, 34]] [1, 512, 32, 32] 2,359,808 BatchNorm2D-539 [[1, 512, 32, 32]] [1, 512, 32, 32] 2,048 ResnetBlock-255 [[1, 512, 32, 32]] [1, 512, 32, 32] 0 Pad2D-750 [[1, 512, 32, 32]] [1, 512, 34, 34] 0 Conv2D-1550 [[1, 512, 34, 34]] [1, 512, 32, 32] 2,359,808 BatchNorm2D-540 [[1, 512, 32, 32]] [1, 512, 32, 32] 2,048 Pad2D-751 [[1, 512, 32, 32]] [1, 512, 34, 34] 0 Conv2D-1551 [[1, 512, 34, 34]] [1, 512, 32, 32] 2,359,808 BatchNorm2D-541 [[1, 512, 32, 32]] [1, 512, 32, 32] 2,048 ResnetBlock-256 [[1, 512, 32, 32]] [1, 512, 32, 32] 0 Pad2D-752 [[1, 512, 32, 32]] [1, 512, 34, 34] 0 Conv2D-1552 [[1, 512, 34, 34]] [1, 512, 32, 32] 2,359,808 BatchNorm2D-542 [[1, 512, 32, 32]] [1, 512, 32, 32] 2,048 Pad2D-753 [[1, 512, 32, 32]] [1, 512, 34, 34] 0 Conv2D-1553 [[1, 512, 34, 34]] [1, 512, 32, 32] 2,359,808 BatchNorm2D-543 [[1, 512, 32, 32]] [1, 512, 32, 32] 2,048 ResnetBlock-257 [[1, 512, 32, 32]] [1, 512, 32, 32] 0 Pad2D-754 [[1, 512, 32, 32]] [1, 512, 34, 34] 0 Conv2D-1554 [[1, 512, 34, 34]] [1, 512, 32, 32] 2,359,808 BatchNorm2D-544 [[1, 512, 32, 32]] [1, 512, 32, 32] 2,048 Pad2D-755 [[1, 512, 32, 32]] [1, 512, 34, 34] 0 Conv2D-1555 [[1, 512, 34, 34]] [1, 512, 32, 32] 2,359,808 BatchNorm2D-545 [[1, 512, 32, 32]] [1, 512, 32, 32] 2,048 ResnetBlock-258 [[1, 512, 32, 32]] [1, 512, 32, 32] 0 Pad2D-756 [[1, 512, 32, 32]] [1, 512, 34, 34] 0 Conv2D-1556 [[1, 512, 34, 34]] [1, 512, 32, 32] 2,359,808 BatchNorm2D-546 [[1, 512, 32, 32]] [1, 512, 32, 32] 2,048 Pad2D-757 [[1, 512, 32, 32]] [1, 512, 34, 34] 0 Conv2D-1557 [[1, 512, 34, 34]] [1, 512, 32, 32] 2,359,808 BatchNorm2D-547 [[1, 512, 32, 32]] [1, 512, 32, 32] 2,048 ResnetBlock-259 [[1, 512, 32, 32]] [1, 512, 32, 32] 0 Pad2D-758 [[1, 512, 32, 32]] [1, 512, 34, 34] 0 Conv2D-1558 [[1, 512, 34, 34]] [1, 512, 32, 32] 2,359,808 BatchNorm2D-548 [[1, 512, 32, 32]] [1, 512, 32, 32] 2,048 Pad2D-759 [[1, 512, 32, 32]] [1, 512, 34, 34] 0 Conv2D-1559 [[1, 512, 34, 34]] [1, 512, 32, 32] 2,359,808 BatchNorm2D-549 [[1, 512, 32, 32]] [1, 512, 32, 32] 2,048 ResnetBlock-260 [[1, 512, 32, 32]] [1, 512, 32, 32] 0 Pad2D-760 [[1, 512, 32, 32]] [1, 512, 34, 34] 0 Conv2D-1560 [[1, 512, 34, 34]] [1, 512, 32, 32] 2,359,808 BatchNorm2D-550 [[1, 512, 32, 32]] [1, 512, 32, 32] 2,048 Pad2D-761 [[1, 512, 32, 32]] [1, 512, 34, 34] 0 Conv2D-1561 [[1, 512, 34, 34]] [1, 512, 32, 32] 2,359,808 BatchNorm2D-551 [[1, 512, 32, 32]] [1, 512, 32, 32] 2,048 ResnetBlock-261 [[1, 512, 32, 32]] [1, 512, 32, 32] 0 Conv2DTranspose-341 [[1, 512, 32, 32]] [1, 256, 64, 64] 2,097,408 InstanceNorm2D-1253 [[1, 256, 64, 64]] [1, 256, 64, 64] 512 ReLU-1339 [[1, 256, 64, 64]] [1, 256, 64, 64] 0 DeconvBlock-341 [[1, 512, 32, 32]] [1, 256, 64, 64] 0 Conv2D-1562 [[1, 256, 64, 64]] [1, 256, 32, 32] 590,080 InstanceNorm2D-1254 [[1, 256, 32, 32]] [1, 256, 32, 32] 512 ReLU-1340 [[1, 256, 32, 32]] [1, 256, 32, 32] 0 ConvBlock-801 [[1, 256, 64, 64]] [1, 256, 32, 32] 0 Conv2D-1563 [[1, 512, 32, 32]] [1, 256, 32, 32] 131,328 InstanceNorm2D-1256 [[1, 256, 32, 32]] [1, 256, 32, 32] 512 ReLU-1342 [[1, 256, 32, 32]] [1, 256, 32, 32] 0 ConvBlock-802 [[1, 512, 32, 32]] [1, 256, 32, 32] 0 Conv2DTranspose-342 [[1, 256, 32, 32]] [1, 256, 64, 64] 1,048,832 InstanceNorm2D-1255 [[1, 256, 64, 64]] [1, 256, 64, 64] 512 ReLU-1341 [[1, 256, 64, 64]] [1, 256, 64, 64] 0 DeconvBlock-342 [[1, 256, 32, 32]] [1, 256, 64, 64] 0 Conv2D-1564 [[1, 256, 64, 64]] [1, 256, 64, 64] 65,792 InstanceNorm2D-1257 [[1, 256, 64, 64]] [1, 256, 64, 64] 512 ReLU-1343 [[1, 256, 64, 64]] [1, 256, 64, 64] 0 ConvBlock-803 [[1, 256, 64, 64]] [1, 256, 64, 64] 0 UpBlock-113 [[1, 512, 32, 32]] [1, 256, 64, 64] 0 Conv2DTranspose-343 [[1, 256, 64, 64]] [1, 128, 128, 128] 524,416 InstanceNorm2D-1258 [[1, 128, 128, 128]] [1, 128, 128, 128] 256 ReLU-1344 [[1, 128, 128, 128]] [1, 128, 128, 128] 0 DeconvBlock-343 [[1, 256, 64, 64]] [1, 128, 128, 128] 0 Conv2D-1565 [[1, 128, 128, 128]] [1, 128, 64, 64] 147,584 InstanceNorm2D-1259 [[1, 128, 64, 64]] [1, 128, 64, 64] 256 ReLU-1345 [[1, 128, 64, 64]] [1, 128, 64, 64] 0 ConvBlock-804 [[1, 128, 128, 128]] [1, 128, 64, 64] 0 Conv2D-1566 [[1, 256, 64, 64]] [1, 128, 64, 64] 32,896 InstanceNorm2D-1261 [[1, 128, 64, 64]] [1, 128, 64, 64] 256 ReLU-1347 [[1, 128, 64, 64]] [1, 128, 64, 64] 0 ConvBlock-805 [[1, 256, 64, 64]] [1, 128, 64, 64] 0 Conv2DTranspose-344 [[1, 128, 64, 64]] [1, 128, 128, 128] 262,272 InstanceNorm2D-1260 [[1, 128, 128, 128]] [1, 128, 128, 128] 256 ReLU-1346 [[1, 128, 128, 128]] [1, 128, 128, 128] 0 DeconvBlock-344 [[1, 128, 64, 64]] [1, 128, 128, 128] 0 Conv2D-1567 [[1, 128, 128, 128]] [1, 128, 128, 128] 16,512 InstanceNorm2D-1262 [[1, 128, 128, 128]] [1, 128, 128, 128] 256 ReLU-1348 [[1, 128, 128, 128]] [1, 128, 128, 128] 0 ConvBlock-806 [[1, 128, 128, 128]] [1, 128, 128, 128] 0 UpBlock-114 [[1, 256, 64, 64]] [1, 128, 128, 128] 0 Conv2DTranspose-345 [[1, 128, 128, 128]] [1, 64, 256, 256] 131,136 InstanceNorm2D-1263 [[1, 64, 256, 256]] [1, 64, 256, 256] 128 ReLU-1349 [[1, 64, 256, 256]] [1, 64, 256, 256] 0 DeconvBlock-345 [[1, 128, 128, 128]] [1, 64, 256, 256] 0 Conv2D-1568 [[1, 64, 256, 256]] [1, 64, 128, 128] 36,928 InstanceNorm2D-1264 [[1, 64, 128, 128]] [1, 64, 128, 128] 128 ReLU-1350 [[1, 64, 128, 128]] [1, 64, 128, 128] 0 ConvBlock-807 [[1, 64, 256, 256]] [1, 64, 128, 128] 0 Conv2D-1569 [[1, 128, 128, 128]] [1, 64, 128, 128] 8,256 InstanceNorm2D-1266 [[1, 64, 128, 128]] [1, 64, 128, 128] 128 ReLU-1352 [[1, 64, 128, 128]] [1, 64, 128, 128] 0 ConvBlock-808 [[1, 128, 128, 128]] [1, 64, 128, 128] 0 Conv2DTranspose-346 [[1, 64, 128, 128]] [1, 64, 256, 256] 65,600 InstanceNorm2D-1265 [[1, 64, 256, 256]] [1, 64, 256, 256] 128 ReLU-1351 [[1, 64, 256, 256]] [1, 64, 256, 256] 0 DeconvBlock-346 [[1, 64, 128, 128]] [1, 64, 256, 256] 0 Conv2D-1570 [[1, 64, 256, 256]] [1, 64, 256, 256] 4,160 InstanceNorm2D-1267 [[1, 64, 256, 256]] [1, 64, 256, 256] 128 ReLU-1353 [[1, 64, 256, 256]] [1, 64, 256, 256] 0 ConvBlock-809 [[1, 64, 256, 256]] [1, 64, 256, 256] 0 UpBlock-115 [[1, 128, 128, 128]] [1, 64, 256, 256] 0 Conv2DTranspose-347 [[1, 64, 256, 256]] [1, 32, 512, 512] 32,800 InstanceNorm2D-1268 [[1, 32, 512, 512]] [1, 32, 512, 512] 64 ReLU-1354 [[1, 32, 512, 512]] [1, 32, 512, 512] 0 DeconvBlock-347 [[1, 64, 256, 256]] [1, 32, 512, 512] 0 Conv2D-1571 [[1, 32, 512, 512]] [1, 32, 256, 256] 9,248 InstanceNorm2D-1269 [[1, 32, 256, 256]] [1, 32, 256, 256] 64 ReLU-1355 [[1, 32, 256, 256]] [1, 32, 256, 256] 0 ConvBlock-810 [[1, 32, 512, 512]] [1, 32, 256, 256] 0 Conv2D-1572 [[1, 64, 256, 256]] [1, 32, 256, 256] 2,080 InstanceNorm2D-1271 [[1, 32, 256, 256]] [1, 32, 256, 256] 64 ReLU-1357 [[1, 32, 256, 256]] [1, 32, 256, 256] 0 ConvBlock-811 [[1, 64, 256, 256]] [1, 32, 256, 256] 0 Conv2DTranspose-348 [[1, 32, 256, 256]] [1, 32, 512, 512] 16,416 InstanceNorm2D-1270 [[1, 32, 512, 512]] [1, 32, 512, 512] 64 ReLU-1356 [[1, 32, 512, 512]] [1, 32, 512, 512] 0 DeconvBlock-348 [[1, 32, 256, 256]] [1, 32, 512, 512] 0 Conv2D-1573 [[1, 32, 512, 512]] [1, 32, 512, 512] 1,056 InstanceNorm2D-1272 [[1, 32, 512, 512]] [1, 32, 512, 512] 64 ReLU-1358 [[1, 32, 512, 512]] [1, 32, 512, 512] 0 ConvBlock-812 [[1, 32, 512, 512]] [1, 32, 512, 512] 0 UpBlock-116 [[1, 64, 256, 256]] [1, 32, 512, 512] 0 Pad2D-762 [[1, 64, 512, 512]] [1, 64, 514, 514] 0 Conv2D-1574 [[1, 64, 514, 514]] [1, 32, 512, 512] 18,464 ReLU-1359 [[1, 32, 512, 512]] [1, 32, 512, 512] 0 BP_HDR-15 [[1, 3, 512, 512]] [1, 32, 512, 512] 0 Pad2D-763 [[1, 3, 512, 512]] [1, 3, 518, 518] 0 Conv2D-1575 [[1, 3, 518, 518]] [1, 32, 512, 512] 4,736 BatchNorm2D-552 [[1, 32, 512, 512]] [1, 32, 512, 512] 128 ReLU-1360 [[1, 512, 32, 32]] [1, 512, 32, 32] 0 Conv2D-1576 [[1, 32, 512, 512]] [1, 64, 256, 256] 18,496 InstanceNorm2D-1273 [[1, 64, 256, 256]] [1, 64, 256, 256] 128 ReLU-1361 [[1, 64, 256, 256]] [1, 64, 256, 256] 0 ConvBlock-813 [[1, 32, 512, 512]] [1, 64, 256, 256] 0 Conv2DTranspose-349 [[1, 64, 256, 256]] [1, 64, 512, 512] 65,600 InstanceNorm2D-1274 [[1, 64, 512, 512]] [1, 64, 512, 512] 128 ReLU-1362 [[1, 64, 512, 512]] [1, 64, 512, 512] 0 DeconvBlock-349 [[1, 64, 256, 256]] [1, 64, 512, 512] 0 Conv2D-1578 [[1, 32, 512, 512]] [1, 64, 512, 512] 2,112 InstanceNorm2D-1276 [[1, 64, 512, 512]] [1, 64, 512, 512] 128 ReLU-1364 [[1, 64, 512, 512]] [1, 64, 512, 512] 0 ConvBlock-815 [[1, 32, 512, 512]] [1, 64, 512, 512] 0 Conv2D-1577 [[1, 64, 512, 512]] [1, 64, 256, 256] 36,928 InstanceNorm2D-1275 [[1, 64, 256, 256]] [1, 64, 256, 256] 128 ReLU-1363 [[1, 64, 256, 256]] [1, 64, 256, 256] 0 ConvBlock-814 [[1, 64, 512, 512]] [1, 64, 256, 256] 0 Conv2D-1579 [[1, 64, 256, 256]] [1, 64, 256, 256] 4,160 InstanceNorm2D-1277 [[1, 64, 256, 256]] [1, 64, 256, 256] 128 ReLU-1365 [[1, 64, 256, 256]] [1, 64, 256, 256] 0 ConvBlock-816 [[1, 64, 256, 256]] [1, 64, 256, 256] 0 DownBlock-117 [[1, 32, 512, 512]] [1, 64, 256, 256] 0 Conv2D-1580 [[1, 64, 256, 256]] [1, 128, 128, 128] 73,856 InstanceNorm2D-1278 [[1, 128, 128, 128]] [1, 128, 128, 128] 256 ReLU-1366 [[1, 128, 128, 128]] [1, 128, 128, 128] 0 ConvBlock-817 [[1, 64, 256, 256]] [1, 128, 128, 128] 0 Conv2DTranspose-350 [[1, 128, 128, 128]] [1, 128, 256, 256] 262,272 InstanceNorm2D-1279 [[1, 128, 256, 256]] [1, 128, 256, 256] 256 ReLU-1367 [[1, 128, 256, 256]] [1, 128, 256, 256] 0 DeconvBlock-350 [[1, 128, 128, 128]] [1, 128, 256, 256] 0 Conv2D-1582 [[1, 64, 256, 256]] [1, 128, 256, 256] 8,320 InstanceNorm2D-1281 [[1, 128, 256, 256]] [1, 128, 256, 256] 256 ReLU-1369 [[1, 128, 256, 256]] [1, 128, 256, 256] 0 ConvBlock-819 [[1, 64, 256, 256]] [1, 128, 256, 256] 0 Conv2D-1581 [[1, 128, 256, 256]] [1, 128, 128, 128] 147,584 InstanceNorm2D-1280 [[1, 128, 128, 128]] [1, 128, 128, 128] 256 ReLU-1368 [[1, 128, 128, 128]] [1, 128, 128, 128] 0 ConvBlock-818 [[1, 128, 256, 256]] [1, 128, 128, 128] 0 Conv2D-1583 [[1, 128, 128, 128]] [1, 128, 128, 128] 16,512 InstanceNorm2D-1282 [[1, 128, 128, 128]] [1, 128, 128, 128] 256 ReLU-1370 [[1, 128, 128, 128]] [1, 128, 128, 128] 0 ConvBlock-820 [[1, 128, 128, 128]] [1, 128, 128, 128] 0 DownBlock-118 [[1, 64, 256, 256]] [1, 128, 128, 128] 0 Conv2D-1584 [[1, 128, 128, 128]] [1, 256, 64, 64] 295,168 InstanceNorm2D-1283 [[1, 256, 64, 64]] [1, 256, 64, 64] 512 ReLU-1371 [[1, 256, 64, 64]] [1, 256, 64, 64] 0 ConvBlock-821 [[1, 128, 128, 128]] [1, 256, 64, 64] 0 Conv2DTranspose-351 [[1, 256, 64, 64]] [1, 256, 128, 128] 1,048,832 InstanceNorm2D-1284 [[1, 256, 128, 128]] [1, 256, 128, 128] 512 ReLU-1372 [[1, 256, 128, 128]] [1, 256, 128, 128] 0 DeconvBlock-351 [[1, 256, 64, 64]] [1, 256, 128, 128] 0 Conv2D-1586 [[1, 128, 128, 128]] [1, 256, 128, 128] 33,024 InstanceNorm2D-1286 [[1, 256, 128, 128]] [1, 256, 128, 128] 512 ReLU-1374 [[1, 256, 128, 128]] [1, 256, 128, 128] 0 ConvBlock-823 [[1, 128, 128, 128]] [1, 256, 128, 128] 0 Conv2D-1585 [[1, 256, 128, 128]] [1, 256, 64, 64] 590,080 InstanceNorm2D-1285 [[1, 256, 64, 64]] [1, 256, 64, 64] 512 ReLU-1373 [[1, 256, 64, 64]] [1, 256, 64, 64] 0 ConvBlock-822 [[1, 256, 128, 128]] [1, 256, 64, 64] 0 Conv2D-1587 [[1, 256, 64, 64]] [1, 256, 64, 64] 65,792 InstanceNorm2D-1287 [[1, 256, 64, 64]] [1, 256, 64, 64] 512 ReLU-1375 [[1, 256, 64, 64]] [1, 256, 64, 64] 0 ConvBlock-824 [[1, 256, 64, 64]] [1, 256, 64, 64] 0 DownBlock-119 [[1, 128, 128, 128]] [1, 256, 64, 64] 0 Conv2D-1588 [[1, 256, 64, 64]] [1, 512, 32, 32] 1,180,160 InstanceNorm2D-1288 [[1, 512, 32, 32]] [1, 512, 32, 32] 1,024 ReLU-1376 [[1, 512, 32, 32]] [1, 512, 32, 32] 0 ConvBlock-825 [[1, 256, 64, 64]] [1, 512, 32, 32] 0 Conv2DTranspose-352 [[1, 512, 32, 32]] [1, 512, 64, 64] 4,194,816 InstanceNorm2D-1289 [[1, 512, 64, 64]] [1, 512, 64, 64] 1,024 ReLU-1377 [[1, 512, 64, 64]] [1, 512, 64, 64] 0 DeconvBlock-352 [[1, 512, 32, 32]] [1, 512, 64, 64] 0 Conv2D-1590 [[1, 256, 64, 64]] [1, 512, 64, 64] 131,584 InstanceNorm2D-1291 [[1, 512, 64, 64]] [1, 512, 64, 64] 1,024 ReLU-1379 [[1, 512, 64, 64]] [1, 512, 64, 64] 0 ConvBlock-827 [[1, 256, 64, 64]] [1, 512, 64, 64] 0 Conv2D-1589 [[1, 512, 64, 64]] [1, 512, 32, 32] 2,359,808 InstanceNorm2D-1290 [[1, 512, 32, 32]] [1, 512, 32, 32] 1,024 ReLU-1378 [[1, 512, 32, 32]] [1, 512, 32, 32] 0 ConvBlock-826 [[1, 512, 64, 64]] [1, 512, 32, 32] 0 Conv2D-1591 [[1, 512, 32, 32]] [1, 512, 32, 32] 262,656 InstanceNorm2D-1292 [[1, 512, 32, 32]] [1, 512, 32, 32] 1,024 ReLU-1380 [[1, 512, 32, 32]] [1, 512, 32, 32] 0 ConvBlock-828 [[1, 512, 32, 32]] [1, 512, 32, 32] 0 DownBlock-120 [[1, 256, 64, 64]] [1, 512, 32, 32] 0 Pad2D-764 [[1, 512, 32, 32]] [1, 512, 34, 34] 0 Conv2D-1592 [[1, 512, 34, 34]] [1, 512, 32, 32] 2,359,808 BatchNorm2D-553 [[1, 512, 32, 32]] [1, 512, 32, 32] 2,048 Pad2D-765 [[1, 512, 32, 32]] [1, 512, 34, 34] 0 Conv2D-1593 [[1, 512, 34, 34]] [1, 512, 32, 32] 2,359,808 BatchNorm2D-554 [[1, 512, 32, 32]] [1, 512, 32, 32] 2,048 ResnetBlock-262 [[1, 512, 32, 32]] [1, 512, 32, 32] 0 Pad2D-766 [[1, 512, 32, 32]] [1, 512, 34, 34] 0 Conv2D-1594 [[1, 512, 34, 34]] [1, 512, 32, 32] 2,359,808 BatchNorm2D-555 [[1, 512, 32, 32]] [1, 512, 32, 32] 2,048 Pad2D-767 [[1, 512, 32, 32]] [1, 512, 34, 34] 0 Conv2D-1595 [[1, 512, 34, 34]] [1, 512, 32, 32] 2,359,808 BatchNorm2D-556 [[1, 512, 32, 32]] [1, 512, 32, 32] 2,048 ResnetBlock-263 [[1, 512, 32, 32]] [1, 512, 32, 32] 0 Pad2D-768 [[1, 512, 32, 32]] [1, 512, 34, 34] 0 Conv2D-1596 [[1, 512, 34, 34]] [1, 512, 32, 32] 2,359,808 BatchNorm2D-557 [[1, 512, 32, 32]] [1, 512, 32, 32] 2,048 Pad2D-769 [[1, 512, 32, 32]] [1, 512, 34, 34] 0 Conv2D-1597 [[1, 512, 34, 34]] [1, 512, 32, 32] 2,359,808 BatchNorm2D-558 [[1, 512, 32, 32]] [1, 512, 32, 32] 2,048 ResnetBlock-264 [[1, 512, 32, 32]] [1, 512, 32, 32] 0 Pad2D-770 [[1, 512, 32, 32]] [1, 512, 34, 34] 0 Conv2D-1598 [[1, 512, 34, 34]] [1, 512, 32, 32] 2,359,808 BatchNorm2D-559 [[1, 512, 32, 32]] [1, 512, 32, 32] 2,048 Pad2D-771 [[1, 512, 32, 32]] [1, 512, 34, 34] 0 Conv2D-1599 [[1, 512, 34, 34]] [1, 512, 32, 32] 2,359,808 BatchNorm2D-560 [[1, 512, 32, 32]] [1, 512, 32, 32] 2,048 ResnetBlock-265 [[1, 512, 32, 32]] [1, 512, 32, 32] 0 Pad2D-772 [[1, 512, 32, 32]] [1, 512, 34, 34] 0 Conv2D-1600 [[1, 512, 34, 34]] [1, 512, 32, 32] 2,359,808 BatchNorm2D-561 [[1, 512, 32, 32]] [1, 512, 32, 32] 2,048 Pad2D-773 [[1, 512, 32, 32]] [1, 512, 34, 34] 0 Conv2D-1601 [[1, 512, 34, 34]] [1, 512, 32, 32] 2,359,808 BatchNorm2D-562 [[1, 512, 32, 32]] [1, 512, 32, 32] 2,048 ResnetBlock-266 [[1, 512, 32, 32]] [1, 512, 32, 32] 0 Pad2D-774 [[1, 512, 32, 32]] [1, 512, 34, 34] 0 Conv2D-1602 [[1, 512, 34, 34]] [1, 512, 32, 32] 2,359,808 BatchNorm2D-563 [[1, 512, 32, 32]] [1, 512, 32, 32] 2,048 Pad2D-775 [[1, 512, 32, 32]] [1, 512, 34, 34] 0 Conv2D-1603 [[1, 512, 34, 34]] [1, 512, 32, 32] 2,359,808 BatchNorm2D-564 [[1, 512, 32, 32]] [1, 512, 32, 32] 2,048 ResnetBlock-267 [[1, 512, 32, 32]] [1, 512, 32, 32] 0 Pad2D-776 [[1, 512, 32, 32]] [1, 512, 34, 34] 0 Conv2D-1604 [[1, 512, 34, 34]] [1, 512, 32, 32] 2,359,808 BatchNorm2D-565 [[1, 512, 32, 32]] [1, 512, 32, 32] 2,048 Pad2D-777 [[1, 512, 32, 32]] [1, 512, 34, 34] 0 Conv2D-1605 [[1, 512, 34, 34]] [1, 512, 32, 32] 2,359,808 BatchNorm2D-566 [[1, 512, 32, 32]] [1, 512, 32, 32] 2,048 ResnetBlock-268 [[1, 512, 32, 32]] [1, 512, 32, 32] 0 Pad2D-778 [[1, 512, 32, 32]] [1, 512, 34, 34] 0 Conv2D-1606 [[1, 512, 34, 34]] [1, 512, 32, 32] 2,359,808 BatchNorm2D-567 [[1, 512, 32, 32]] [1, 512, 32, 32] 2,048 Pad2D-779 [[1, 512, 32, 32]] [1, 512, 34, 34] 0 Conv2D-1607 [[1, 512, 34, 34]] [1, 512, 32, 32] 2,359,808 BatchNorm2D-568 [[1, 512, 32, 32]] [1, 512, 32, 32] 2,048 ResnetBlock-269 [[1, 512, 32, 32]] [1, 512, 32, 32] 0 Pad2D-780 [[1, 512, 32, 32]] [1, 512, 34, 34] 0 Conv2D-1608 [[1, 512, 34, 34]] [1, 512, 32, 32] 2,359,808 BatchNorm2D-569 [[1, 512, 32, 32]] [1, 512, 32, 32] 2,048 Pad2D-781 [[1, 512, 32, 32]] [1, 512, 34, 34] 0 Conv2D-1609 [[1, 512, 34, 34]] [1, 512, 32, 32] 2,359,808 BatchNorm2D-570 [[1, 512, 32, 32]] [1, 512, 32, 32] 2,048 ResnetBlock-270 [[1, 512, 32, 32]] [1, 512, 32, 32] 0 Conv2DTranspose-353 [[1, 512, 32, 32]] [1, 256, 64, 64] 2,097,408 InstanceNorm2D-1293 [[1, 256, 64, 64]] [1, 256, 64, 64] 512 ReLU-1381 [[1, 256, 64, 64]] [1, 256, 64, 64] 0 DeconvBlock-353 [[1, 512, 32, 32]] [1, 256, 64, 64] 0 Conv2D-1610 [[1, 256, 64, 64]] [1, 256, 32, 32] 590,080 InstanceNorm2D-1294 [[1, 256, 32, 32]] [1, 256, 32, 32] 512 ReLU-1382 [[1, 256, 32, 32]] [1, 256, 32, 32] 0 ConvBlock-829 [[1, 256, 64, 64]] [1, 256, 32, 32] 0 Conv2D-1611 [[1, 512, 32, 32]] [1, 256, 32, 32] 131,328 InstanceNorm2D-1296 [[1, 256, 32, 32]] [1, 256, 32, 32] 512 ReLU-1384 [[1, 256, 32, 32]] [1, 256, 32, 32] 0 ConvBlock-830 [[1, 512, 32, 32]] [1, 256, 32, 32] 0 Conv2DTranspose-354 [[1, 256, 32, 32]] [1, 256, 64, 64] 1,048,832 InstanceNorm2D-1295 [[1, 256, 64, 64]] [1, 256, 64, 64] 512 ReLU-1383 [[1, 256, 64, 64]] [1, 256, 64, 64] 0 DeconvBlock-354 [[1, 256, 32, 32]] [1, 256, 64, 64] 0 Conv2D-1612 [[1, 256, 64, 64]] [1, 256, 64, 64] 65,792 InstanceNorm2D-1297 [[1, 256, 64, 64]] [1, 256, 64, 64] 512 ReLU-1385 [[1, 256, 64, 64]] [1, 256, 64, 64] 0 ConvBlock-831 [[1, 256, 64, 64]] [1, 256, 64, 64] 0 UpBlock-117 [[1, 512, 32, 32]] [1, 256, 64, 64] 0 Conv2DTranspose-355 [[1, 256, 64, 64]] [1, 128, 128, 128] 524,416 InstanceNorm2D-1298 [[1, 128, 128, 128]] [1, 128, 128, 128] 256 ReLU-1386 [[1, 128, 128, 128]] [1, 128, 128, 128] 0 DeconvBlock-355 [[1, 256, 64, 64]] [1, 128, 128, 128] 0 Conv2D-1613 [[1, 128, 128, 128]] [1, 128, 64, 64] 147,584 InstanceNorm2D-1299 [[1, 128, 64, 64]] [1, 128, 64, 64] 256 ReLU-1387 [[1, 128, 64, 64]] [1, 128, 64, 64] 0 ConvBlock-832 [[1, 128, 128, 128]] [1, 128, 64, 64] 0 Conv2D-1614 [[1, 256, 64, 64]] [1, 128, 64, 64] 32,896 InstanceNorm2D-1301 [[1, 128, 64, 64]] [1, 128, 64, 64] 256 ReLU-1389 [[1, 128, 64, 64]] [1, 128, 64, 64] 0 ConvBlock-833 [[1, 256, 64, 64]] [1, 128, 64, 64] 0 Conv2DTranspose-356 [[1, 128, 64, 64]] [1, 128, 128, 128] 262,272 InstanceNorm2D-1300 [[1, 128, 128, 128]] [1, 128, 128, 128] 256 ReLU-1388 [[1, 128, 128, 128]] [1, 128, 128, 128] 0 DeconvBlock-356 [[1, 128, 64, 64]] [1, 128, 128, 128] 0 Conv2D-1615 [[1, 128, 128, 128]] [1, 128, 128, 128] 16,512 InstanceNorm2D-1302 [[1, 128, 128, 128]] [1, 128, 128, 128] 256 ReLU-1390 [[1, 128, 128, 128]] [1, 128, 128, 128] 0 ConvBlock-834 [[1, 128, 128, 128]] [1, 128, 128, 128] 0 UpBlock-118 [[1, 256, 64, 64]] [1, 128, 128, 128] 0 Conv2DTranspose-357 [[1, 128, 128, 128]] [1, 64, 256, 256] 131,136 InstanceNorm2D-1303 [[1, 64, 256, 256]] [1, 64, 256, 256] 128 ReLU-1391 [[1, 64, 256, 256]] [1, 64, 256, 256] 0 DeconvBlock-357 [[1, 128, 128, 128]] [1, 64, 256, 256] 0 Conv2D-1616 [[1, 64, 256, 256]] [1, 64, 128, 128] 36,928 InstanceNorm2D-1304 [[1, 64, 128, 128]] [1, 64, 128, 128] 128 ReLU-1392 [[1, 64, 128, 128]] [1, 64, 128, 128] 0 ConvBlock-835 [[1, 64, 256, 256]] [1, 64, 128, 128] 0 Conv2D-1617 [[1, 128, 128, 128]] [1, 64, 128, 128] 8,256 InstanceNorm2D-1306 [[1, 64, 128, 128]] [1, 64, 128, 128] 128 ReLU-1394 [[1, 64, 128, 128]] [1, 64, 128, 128] 0 ConvBlock-836 [[1, 128, 128, 128]] [1, 64, 128, 128] 0 Conv2DTranspose-358 [[1, 64, 128, 128]] [1, 64, 256, 256] 65,600 InstanceNorm2D-1305 [[1, 64, 256, 256]] [1, 64, 256, 256] 128 ReLU-1393 [[1, 64, 256, 256]] [1, 64, 256, 256] 0 DeconvBlock-358 [[1, 64, 128, 128]] [1, 64, 256, 256] 0 Conv2D-1618 [[1, 64, 256, 256]] [1, 64, 256, 256] 4,160 InstanceNorm2D-1307 [[1, 64, 256, 256]] [1, 64, 256, 256] 128 ReLU-1395 [[1, 64, 256, 256]] [1, 64, 256, 256] 0 ConvBlock-837 [[1, 64, 256, 256]] [1, 64, 256, 256] 0 UpBlock-119 [[1, 128, 128, 128]] [1, 64, 256, 256] 0 Conv2DTranspose-359 [[1, 64, 256, 256]] [1, 32, 512, 512] 32,800 InstanceNorm2D-1308 [[1, 32, 512, 512]] [1, 32, 512, 512] 64 ReLU-1396 [[1, 32, 512, 512]] [1, 32, 512, 512] 0 DeconvBlock-359 [[1, 64, 256, 256]] [1, 32, 512, 512] 0 Conv2D-1619 [[1, 32, 512, 512]] [1, 32, 256, 256] 9,248 InstanceNorm2D-1309 [[1, 32, 256, 256]] [1, 32, 256, 256] 64 ReLU-1397 [[1, 32, 256, 256]] [1, 32, 256, 256] 0 ConvBlock-838 [[1, 32, 512, 512]] [1, 32, 256, 256] 0 Conv2D-1620 [[1, 64, 256, 256]] [1, 32, 256, 256] 2,080 InstanceNorm2D-1311 [[1, 32, 256, 256]] [1, 32, 256, 256] 64 ReLU-1399 [[1, 32, 256, 256]] [1, 32, 256, 256] 0 ConvBlock-839 [[1, 64, 256, 256]] [1, 32, 256, 256] 0 Conv2DTranspose-360 [[1, 32, 256, 256]] [1, 32, 512, 512] 16,416 InstanceNorm2D-1310 [[1, 32, 512, 512]] [1, 32, 512, 512] 64 ReLU-1398 [[1, 32, 512, 512]] [1, 32, 512, 512] 0 DeconvBlock-360 [[1, 32, 256, 256]] [1, 32, 512, 512] 0 Conv2D-1621 [[1, 32, 512, 512]] [1, 32, 512, 512] 1,056 InstanceNorm2D-1312 [[1, 32, 512, 512]] [1, 32, 512, 512] 64 ReLU-1400 [[1, 32, 512, 512]] [1, 32, 512, 512] 0 ConvBlock-840 [[1, 32, 512, 512]] [1, 32, 512, 512] 0 UpBlock-120 [[1, 64, 256, 256]] [1, 32, 512, 512] 0 BP_Light-15 [[1, 3, 512, 512]] [1, 32, 512, 512] 0 Pad2D-783 [[1, 32, 512, 512]] [1, 32, 514, 514] 0 Conv2D-1623 [[1, 32, 514, 514]] [1, 64, 512, 512] 18,496 ReLU-1401 [[1, 64, 512, 512]] [1, 64, 512, 512] 0 Pad2D-784 [[1, 32, 512, 512]] [1, 32, 514, 514] 0 Conv2D-1624 [[1, 32, 514, 514]] [1, 64, 512, 512] 18,496 ReLU-1402 [[1, 64, 512, 512]] [1, 64, 512, 512] 0 Pad2D-785 [[1, 128, 512, 512]] [1, 128, 516, 516] 0 Conv2D-1625 [[1, 128, 516, 516]] [1, 16, 512, 512] 18,448 InstanceNorm2D-1313 [[1, 16, 512, 512]] [1, 16, 512, 512] 32 ReLU-1403 [[1, 16, 512, 512]] [1, 16, 512, 512] 0 ConvBlock2D-141 [[1, 128, 512, 512]] [1, 16, 512, 512] 0 Pad2D-786 [[1, 128, 512, 512]] [1, 128, 518, 518] 0 Conv2D-1626 [[1, 128, 518, 518]] [1, 16, 512, 512] 18,448 InstanceNorm2D-1314 [[1, 16, 512, 512]] [1, 16, 512, 512] 32 ReLU-1404 [[1, 16, 512, 512]] [1, 16, 512, 512] 0 ConvBlock2D-142 [[1, 128, 512, 512]] [1, 16, 512, 512] 0 Pad2D-787 [[1, 128, 512, 512]] [1, 128, 520, 520] 0 Conv2D-1627 [[1, 128, 520, 520]] [1, 16, 512, 512] 18,448 InstanceNorm2D-1315 [[1, 16, 512, 512]] [1, 16, 512, 512] 32 ReLU-1405 [[1, 16, 512, 512]] [1, 16, 512, 512] 0 ConvBlock2D-143 [[1, 128, 512, 512]] [1, 16, 512, 512] 0 Pad2D-788 [[1, 128, 512, 512]] [1, 128, 522, 522] 0 Conv2D-1628 [[1, 128, 522, 522]] [1, 16, 512, 512] 18,448 InstanceNorm2D-1316 [[1, 16, 512, 512]] [1, 16, 512, 512] 32 ReLU-1406 [[1, 16, 512, 512]] [1, 16, 512, 512] 0 ConvBlock2D-144 [[1, 128, 512, 512]] [1, 16, 512, 512] 0 Pad2D-789 [[1, 128, 512, 512]] [1, 128, 524, 524] 0 Conv2D-1629 [[1, 128, 524, 524]] [1, 16, 512, 512] 18,448 InstanceNorm2D-1317 [[1, 16, 512, 512]] [1, 16, 512, 512] 32 ReLU-1407 [[1, 16, 512, 512]] [1, 16, 512, 512] 0 ConvBlock2D-145 [[1, 128, 512, 512]] [1, 16, 512, 512] 0 Pad2D-790 [[1, 128, 512, 512]] [1, 128, 528, 528] 0 Conv2D-1630 [[1, 128, 528, 528]] [1, 16, 512, 512] 18,448 InstanceNorm2D-1318 [[1, 16, 512, 512]] [1, 16, 512, 512] 32 ReLU-1408 [[1, 16, 512, 512]] [1, 16, 512, 512] 0 ConvBlock2D-146 [[1, 128, 512, 512]] [1, 16, 512, 512] 0 Pad2D-791 [[1, 128, 512, 512]] [1, 128, 532, 532] 0 Conv2D-1631 [[1, 128, 532, 532]] [1, 16, 512, 512] 18,448 InstanceNorm2D-1319 [[1, 16, 512, 512]] [1, 16, 512, 512] 32 ReLU-1409 [[1, 16, 512, 512]] [1, 16, 512, 512] 0 ConvBlock2D-147 [[1, 128, 512, 512]] [1, 16, 512, 512] 0 Pad2D-792 [[1, 128, 512, 512]] [1, 128, 536, 536] 0 Conv2D-1632 [[1, 128, 536, 536]] [1, 16, 512, 512] 18,448 InstanceNorm2D-1320 [[1, 16, 512, 512]] [1, 16, 512, 512] 32 ReLU-1410 [[1, 16, 512, 512]] [1, 16, 512, 512] 0 ConvBlock2D-148 [[1, 128, 512, 512]] [1, 16, 512, 512] 0 Pad2D-793 [[1, 128, 512, 512]] [1, 128, 512, 512] 0 Conv2D-1633 [[1, 128, 512, 512]] [1, 128, 510, 510] 147,584 LeakyReLU-29 [[1, 128, 510, 510]] [1, 128, 510, 510] 0 ConvBlock2D-149 [[1, 128, 512, 512]] [1, 128, 510, 510] 0 AdaptiveAvgPool2D-15 [[1, 128, 510, 510]] [1, 128, 1, 1] 0 Linear-29 [[1, 128]] [1, 16] 2,048 ReLU-1411 [[1, 16]] [1, 16] 0 Linear-30 [[1, 16]] [1, 128] 2,048 Sigmoid-15 [[1, 128]] [1, 128] 0 Pad2D-794 [[1, 128, 512, 512]] [1, 128, 512, 512] 0 Conv2D-1634 [[1, 128, 512, 512]] [1, 64, 512, 512] 8,256 LeakyReLU-30 [[1, 64, 512, 512]] [1, 64, 512, 512] 0 ConvBlock2D-150 [[1, 128, 512, 512]] [1, 64, 512, 512] 0 FusionLayer-15 [[1, 128, 512, 512]] [1, 64, 512, 512] 0
MultiDilationResnetBlock_attention-15 [] [1, 64, 512, 512] 0 Pad2D-795 [[1, 64, 512, 512]] [1, 64, 518, 518] 0 Conv2D-1635 [[1, 64, 518, 518]] [1, 3, 512, 512] 9,411 Tanh-30 [[1, 3, 512, 512]] [1, 3, 512, 512] 0
=================================================================================================
Total params: 117,483,107
Trainable params: 117,409,123
Non-trainable params: 73,984
-------------------------------------------------------------------------------------------------
Input size (MB): 3.00
Forward/backward pass size (MB): 16055.44
Params size (MB): 448.16
Estimated Total Size (MB): 16506.60
-------------------------------------------------------------------------------------------------{'total_params': 117483107, 'trainable_params': 117409123}
四、数据准备
1.解压数据集
!unzip -oq /home/aistudio/data/data71049/data.zip -d work/
2.数据集定义
import paddle
from paddle.io import Dataset
from paddle.vision.transforms import Compose, Resize, ColorJitter, ToTensor, Normalize
from paddle.vision import transformsimport os
import os.path
from os import listdir
from os.path import join
import cv2
import numpy as np
from PIL import Image, ImageOps, ImageEnhance
import randomclass TraindataFromTrack1(Dataset):def __init__(self):super(TraindataFromTrack1, self).__init__()self.image_folder = "/home/aistudio/work/data/input"self.target_folder = "/home/aistudio/work/data/target"self.image_filenames_input = [join(self.image_folder, x) for x in listdir(self.image_folder) ifself.is_image_file(x)]self.image_filenames_target = [join(self.target_folder, x) for x in listdir(self.target_folder) ifself.is_image_file(x)]self.augmentation = ColorJitter(0.2, 0.2, 0.2, 0.2)print("data folder: %s; find %d images" % (self.image_folder, len(self.image_filenames_input)))print("data folder: %s; find %d images" % (self.target_folder, len(self.image_filenames_target)))def is_image_file(swlf, filename):return any(filename.endswith(extension) for extension in [".bmp", ".png", ".jpg", ".jpeg"])def get_img_flipAug(self, index):InputFile = self.image_filenames_input[index]InTargetFile = self.image_filenames_target[index]img_input = Image.open(InputFile).resize((512, 512))img_input = img_input.convert("RGB")img_target = Image.open(InTargetFile).resize((512, 512))img_target = img_target.convert("RGB")return img_input, img_targetdef __getitem__(self, index):img_input, img_target = self.get_img_flipAug(index)img_input = np.asarray(img_input,dtype='float32')img_target = np.asarray(img_target,dtype='float32')# print(img_input.shape)img_input = np.transpose(img_input, (2,0,1))img_target = np.transpose(img_target, (2,0,1))# print(img_input.shape)return img_input, img_targetdef __len__(self):return len(self.image_filenames_input)traindataFromTrack1 = TraindataFromTrack1()
data folder: /home/aistudio/work/data/input; find 168 images
data folder: /home/aistudio/work/data/target; find 168 images
3.展示训练数据
# 取出第一个数据
inputs, targer = traindataFromTrack1[0]# 将CHW转换成HWC
inputs = np.transpose(inputs, (1,2,0))
targer = np.transpose(targer, (1,2,0))
inputs = Image.fromarray(np.uint8(inputs))
targer = Image.fromarray(np.uint8(targer))# 展示输入数据与目标输出
display(inputs)
display(inputs.size)
display(targer)
display(targer.size)
(512, 512)
(512, 512)
五、模型训练
# 模型封装
model = paddle.Model(DRN(3, 3))
model.prepare(paddle.optimizer.Adam(learning_rate=0.0002,beta1=0.5,parameters=model.parameters()),paddle.nn.MSELoss())callback = paddle.callbacks.VisualDL(log_dir='visualdl_log_dir')# 启动模型训练，指定训练数据集，设置训练轮次，设置每次数据集计算的批次大小，设置日志格式
model.fit(traindataFromTrack1,epochs=10,batch_size=1,verbose=1,callbacks=callback,)
l.prepare(paddle.optimizer.Adam(learning_rate=0.0002,beta1=0.5,parameters=model.parameters()),paddle.nn.MSELoss())callback = paddle.callbacks.VisualDL(log_dir='visualdl_log_dir')# 启动模型训练，指定训练数据集，设置训练轮次，设置每次数据集计算的批次大小，设置日志格式
model.fit(traindataFromTrack1,epochs=10,batch_size=1,verbose=1,callbacks=callback,)The loss value printed in the log is the current step, and the metric is the average value of previous step.
Epoch 1/10
step 168/168 [==============================] - loss: 1308.8942 - 781ms/step
Epoch 2/10
step 168/168 [==============================] - loss: 773.7590 - 779ms/step
Epoch 3/10
step 168/168 [==============================] - loss: 997.8239 - 781ms/step
Epoch 4/10
step 168/168 [==============================] - loss: 2062.1763 - 779ms/step
Epoch 5/10
step 168/168 [==============================] - loss: 23303.9004 - 780ms/step
Epoch 6/10
step 168/168 [==============================] - loss: 2337.3169 - 780ms/step
Epoch 7/10
step 168/168 [==============================] - loss: 7284.5688 - 781ms/step
Epoch 8/10
step 168/168 [==============================] - loss: 5248.8242 - 780ms/step
Epoch 9/10
step 168/168 [==============================] - loss: 11754.9814 - 780ms/step
Epoch 10/10
step 168/168 [==============================] - loss: 1588.9448 - 780ms/step
保存模型
model.save('checkpoint/model_final')
六、总结与展望
讲讲复现的过程中遇到的一些困难以及解决思路：
- 作者是用Pytorch来实现DRN的，Pytorch与PaddlePaddle有相似之处，但是有一些细节需要注意，比如卷积的api，Pytorch里是小写的d，而PaddlePaddle里的卷积用的是大写的D；又比如pytorch里的cat与PaddlePaddle里的concat。这里需要细心地查阅两个框架的api文档，从不同中找到相同之处。
- 数据处理上，训练集的图片是4通道的，但论文上的输入是3通道的，因此舍去图片的第四通道，而图片有4个通道应该是虚幻引擎的问题；另一点就是注意区分NCHW格式与NHWC格式，以及两者的转换。
- 最后还是数据集上，比赛用的数据集有16G，比较大，这里为了简化问题，只选择了168张图片做训练，但Loss降不下去的原因应该不止这一个，还需要继续调参优化。
个人简介
北京联合大学 机器人学院 自动化专业 2018级 本科生 郑博培
百度飞桨开发者技术专家 PPDE
百度飞桨官方帮帮团、答疑团成员
深圳柴火创客空间 认证会员
百度大脑 智能对话训练师
我在AI Studio上获得至尊等级，点亮9个徽章，来互关呀！！！
https://aistudio.baidu.com/aistudio/personalcenter/thirdview/147378
![关于CSRF攻击及mvc中的解决方案 [ValidateAntiForgeryToken]](https://img-my.csdn.net/uploads/201212/05/1354701750_7294.jpg)
