源码：
这个版本的代码是最简洁易懂的，https://github.com/senliuy/Keras_HED_with_model
数据集：
链接：https://pan.baidu.com/s/13qStI9DP1mbt9JallQFpPg
提取码：wbfi
HED(Holistically-Nested Edge Detection) 网络
首先先看一下HED网络结构：搬运https://blog.csdn.net/sinat_26917383/article/details/73087831
HED 网络模型是在 VGG16 网络结构的基础上设计出来的，所以有必要先看看 VGG16。
HED 网络在 VGG 网络的基础上去除了后5层，vgg的后面的全连接层与 softmax 层主要用于分类，HED 网络只需要提取图片的特征，保留了前面的卷积层和池化层（注意：去掉最后一层池化层）。并将特征图合并然后1*1卷积，最后采用sigmoid激活，下面是 HED 网络的示意图：
特征图去除深度：
分别提取出 VGG 网络的 conv1_2, conv2_2, conv3_3, conv4_3, conv5_3 层，这些输出层的大小分别为 [224, 224, 64]，[112, 112, 128]，[56, 56, 256]，[28, 28, 512]，[14, 14, 512]，由于需要将这些层的数据和成一张图片，首先需要将深度降维到1，然后再按比例放大 1，2，4，8，16 倍使得每一层的数据大小都为 [224, 224]，最终相加就可以得到尺寸为 [224, 224] 的输出图片了。去深度使用输出深度为 1，卷积核为 1*1 的卷积操作，得到深度为 1 的输出，1*1 的卷积主要作用就是多通道融合。
在每个side-output layer,我们将每个输出都resize到原始尺寸和ground truth对比，在高层输出，由于经过了较大的stride,此时只剩下一些大的边缘，而精细的边缘已经检测不出来；
损失函数：
因为边界像素数目远小于其他数目，正负样本分布不均衡，所以不能采用普通的交叉熵损失，这里采用的是 pos_weight交叉熵。
模型输出图：
参考链接：
https://pqpo.me/2019/08/02/machine-learning-hed-smartcropper/
https://blog.csdn.net/qq_36187544/article/details/91447065
测试程序：opencv4.2 模型下载链接：https://download.csdn.net/download/qq_35054151/12275865
#include <iostream>
#include <opencv2/opencv.hpp>
#include<opencv2\dnn.hpp>
using namespace cv;
using namespace std;
void edgeDetection(cv::Mat &src, cv::Mat &dst, double threshold)
{//CV_DNN_REGISTER_LAYER_CLASS(Crop, CropLayer);cv::Mat img = src.clone();cv::Size reso(500, 500);cv::Mat blob = cv::dnn::blobFromImage(img, threshold, reso, cv::Scalar(104.00698793, 116.66876762, 122.67891434), false, false);cv::dnn::Net net = cv::dnn::readNet("deploy.prototxt", "hed_pretrained_bsds.caffemodel");net.setInput(blob);cv::Mat out = net.forward();cv::resize(out.reshape(1, reso.height), out, img.size());cv::Mat out2;out.convertTo(dst, CV_8UC3, 255);
}int main(void)
{cv::Mat src = cv::imread("1.jpg");cv::namedWindow("原图", 0);cv::imshow("原图", src);resize(src, src, cv::Size(500, 500));cv::Mat dst;edgeDetection(src, dst, 2.2);cv::namedWindow("HED", 0);cv::imshow("HED", dst);cv::waitKey(0);return 0;
}
import cv2 as cv
import argparseparser = argparse.ArgumentParser(description='This sample shows how to define custom OpenCV deep learning layers in Python. ''Holistically-Nested Edge Detection (https://arxiv.org/abs/1504.06375) neural network ''is used as an example model. Find a pre-trained model at https://github.com/s9xie/hed.')
parser.add_argument('--input', help='Path to image or video. Skip to capture frames from camera', default='1.jpg')
parser.add_argument('--prototxt', help='Path to deploy.prototxt', default='deploy.prototxt')
parser.add_argument('--caffemodel', help='Path to hed_pretrained_bsds.caffemodel', default='hed_pretrained_bsds.caffemodel')
parser.add_argument('--width', help='Resize input image to a specific width', default=500, type=int)
parser.add_argument('--height', help='Resize input image to a specific height', default=500, type=int)
args = parser.parse_args()#! [CropLayenr]
class CropLayer(object):def __init__(self, params, blobs):self.xstart = 0self.xend = 0self.ystart = 0self.yend = 0# Our layer receives two inputs. We need to crop the first input blob# to match a shape of the second one (keeping batch size and number of channels)def getMemoryShapes(self, inputs):inputShape, targetShape = inputs[0], inputs[1]batchSize, numChannels = inputShape[0], inputShape[1]height, width = targetShape[2], targetShape[3]#self.ystart = (inputShape[2] - targetShape[2]) / 2#self.xstart = (inputShape[3] - targetShape[3]) / 2self.ystart = int((inputShape[2] - targetShape[2]) / 2)self.xstart = int((inputShape[3] - targetShape[3]) / 2)self.yend = self.ystart + heightself.xend = self.xstart + widthreturn [[batchSize, numChannels, height, width]]def forward(self, inputs):return [inputs[0][:,:,self.ystart:self.yend,self.xstart:self.xend]]
#! [CropLayer]#! [Register]
cv.dnn_registerLayer('Crop', CropLayer)
#! [Register]# Load the model.
net = cv.dnn.readNet(cv.samples.findFile(args.prototxt), cv.samples.findFile(args.caffemodel))kWinName = 'Holistically-Nested Edge Detection'
cv.namedWindow('Input', cv.WINDOW_NORMAL)
cv.namedWindow(kWinName, cv.WINDOW_NORMAL)frame=cv.imread('1.jpg')cv.imshow('Input', frame)
#cv.waitKey(0)inp = cv.dnn.blobFromImage(frame, scalefactor=1.0, size=(args.width, args.height),mean=(104.00698793, 116.66876762, 122.67891434),swapRB=False, crop=False)
net.setInput(inp)out = net.forward()
out = out[0, 0]
out = cv.resize(out, (frame.shape[1], frame.shape[0]))
cv.imshow(kWinName, out)
cv.imwrite('result.png',out)
cv.waitKey(0)
测试图像：
openvino部署测试
opencino2020.4
python mo.py --input_model hed.pb --input_shape=[1,480,480,3] --data_type FP32 --model_name hed
