上一篇文档 https://blog.csdn.net/xingzhe2001/article/details/86316712 介绍了L1 L2正则化

本文介绍L1, L2 正则化的实现

L1正则化

代码

def L1Norm(l, theta):return  np.dot(np.abs(theta), np.ones(theta.size)) * ldef L1NormPartial(l, theta):return np.sign(theta) * l

以线性回归为例，J函数现在变成了

L2正则化

def L2Norm(l, theta):return  np.dot(theta, theta) * l def L2NormPartial(l, theta):return theta * l

以线性回归为例 J函数变为

    def __Jfunction(self):        sum = 0for i in range(0, self.m):err = self.__error_dist(self.x[i], self.y[i])sum += np.dot(err, err)sum += Regularization.L2Norm(0.8, self.theta)return 1/(2 * self.m) * sum

J函数导数变为

    def __partialderiv_J_func(self):sum = 0for i in range(0, self.m):err = self.__error_dist(self.x[i], self.y[i])sum += np.dot(self.x[i], err)sum += Regularization.L2NormPartial(0.8, self.theta)return 1/self.m * sum