Weighted Linear Regression thoughts

wlr.md

Weighted Linear Regression $$ J(\theta)=\Sigma w_i (y_i - \theta^T x_i)^2 $$ $$ J'(\theta)=2\Sigma w_i x_i (\theta^T x_i - y_i) $$ $$ w_i = \exp(-\frac {(x_i - x) ^ 2} {2\tau^2}) $$

# x_query - точка для расчёта предсказания, x_query.shape - (9,)
# x.shape - (76, 9), y.shape - (76, 1)
def getWeightedLRPrediction(alpha, n_iterations, x_query, tau, x, y):
	# находим вектор весов w по формуле выше
	w = [np.exp(-((x[i] - x_query).T.dot(x[i] - x_query)) / (2 * tau ** 2) for i in range(len(x)]  # w.shape --> (76, 1)
	
	# преобразуем w в numpy.ndarray
	w = np.array(w)
	
	theta = np.zeros(shape=x_query.shape) # theta.shape --> (9,)
	for _ in range(n_iterations):
		pred = w * (np.dot(x, theta) - y) # pred.shape --> (76,)
		gradient = pred.dot(x) * 2 # gradient.shape --> (9,)
		theta -= alpha * gradient
		
	return np.dot(x_query, theta)