介绍两种方法做多元非线性回归：lsqcurvefit、Adagrad 法。

lsqcurvefit是Matlab提供的内置函数，用于基于最小二乘的曲线拟合问题；Adagrad是一种基于梯度的迭代方法，特点在步长不固定，随着迭代次数变化。

clear; clc;
% 多元非线性回归模型
% z = w1 * exp(-x / w2) + w3 * y
% z = 10 * exp(-x / 5) + 2 * y
len = 20;
rng('default');
x = randi(len + 1, len, 1) / 5;
y = randi(len + 1, len, 1) / 5;
z = 10 * exp(-x / 2) + 0.5 * y;
ratio = 0.0;
z = z + ratio * max(z) * rand(len, 1);
X = [x, y];
fun = @(var, X)var(1) * exp(-X(:, 1) / var(2)) + var(3) * X(:, 2);
w = lsqcurvefit(fun, [1, 1, 1], X, z);
disp(['lsqcurvefit 计算结果：', num2str(w)]);
% 梯度下降法学习
% obj = 1 / (2 * len) * (w1 * exp(-x / w2) + w3 * y - z)' * (w1 * exp(-x / w2) + w3 * y - z)
alpha = 5;   % 学习率大收敛快，可能有震荡
iteMax = 10000;
w1 = 1;
w2 = 1;
w3 = 1;
initW = [w1; w2; w3];
err = 1e-6;
J = zeros(iteMax, 1);
G = zeros(3, 1);
e = 0.1;
for i = 1 : iteMaxgradW1 = 1 / len * (exp(-x / w2))' * (w1 * exp(-x / w2) + w3 * y - z);gradW2 = 1 / len * (w1 * x .* exp(-x / w2) / w2^2)' * (w1 * exp(-x / w2) + w3 * y - z);gradW3 = 1 / len * y' * (w1 * exp(-x / w2) + w3 * y - z);grad = [gradW1; gradW2; gradW3];% Adagrad 法    x = x + yita * inv(G) * grad;G = G + grad.^2;newW = initW - alpha * diag(1 ./ sqrt(G + e)) * grad;if norm(newW - initW) < errJ(i + 1 : end) = [];disp(['梯度下降法迭代次数：', num2str(i)]);disp(['梯度下降法迭代结果：', num2str(newW')]);break;elseinitW = newW;w1 = newW(1);w2 = newW(2);w3 = newW(3);J(i) = 1 / (2 * len) * (w1 * exp(-x / w2) + w3 * y - z)' * (w1 * exp(-x / w2) + w3 * y - z);end
end
% 绘图
subplot(1, 2, 1)
loglog(J, 'LineWidth', 2)
legend(['alpha = ', num2str(alpha)]);
xFit1 = linspace(min(x), max(x), 30);
yFit1 = linspace(min(y), max(y), 30);
[xFit2, yFit2] = meshgrid(xFit1, yFit1);
zFit = w1 * exp(-xFit2 / w2) + w3 * yFit2;
subplot(1, 2, 2)
scatter3(x, y, z, 'filled');
hold on
mesh(xFit2, yFit2, zFit);
hold off
legend('Points', 'Fitting', 'Location','NorthOutside');

结果打印

Local minimum found.Optimization completed because the size of the gradient is less than
the default value of the function tolerance.<stopping criteria details>lsqcurvefit 计算结果：10           2         0.5
梯度下降法迭代次数：1934
梯度下降法迭代结果：10            2          0.5