一、K 近邻算法
1、分类
考虑任意个（K 个）邻居，采用“投票法”来指定标签。
3 近邻分类模型
2、回归
多个近邻时候，预测结果为这些邻居的平均值
3 近邻回归模型.png
二、K 近邻算法分析
1、 决策边界
使用更少的邻居对应更高的模型复杂度，而使用更多的邻居对应更低的模型复杂度
不同 n_neighbors 值的 K 近邻模型的决策边界.png
2、模型复杂度和泛化能力之间的关系
随着邻居个数的增多，模型变得更简单，训练集精度也随之下降
image.png
3、优点、缺点和参数
- KNeighbors 分类器有 2 个重要参数：邻居个数与数据点之间距离的度量方法
- 构建最近邻模型的速度通常很快，但如果训练集很大，预测速度可能会比较慢。
- 如果数据集拥有很多特征（几百或更多），该算法效果不好
- 大于大多数特征的大多数取值都为 0 的数据集（稀疏数据集），算法的效果尤其不好。
三、代码展示
1、分类
import matplotlib.pyplot as plt
import mglearn
from sklearn.model_selection import train_test_split
from sklearn.neighbors import KNeighborsClassifier
from sklearn.datasets import load_breast_cancer
# 数据集展示
mglearn.plots.plot_knn_classification(n_neighbors=3)
plt.show()
# 获取数据集
X,y=mglearn.datasets.make_forge()
X_train,X_test,y_train,y_test=train_test_split(X,y,random_state=0)
# 构建模型
clf=KNeighborsClassifier(n_neighbors=3)
clf.fit(X_train,y_train)
print('Test set predictions:{}'.format(clf.predict(X_test)))
print('Test set accuracy:{:.2f}'.format(clf.score(X_test,y_test)))
# 决策边界展示
fig,axes=plt.subplots(1,3,figsize=(10,3))
for n_neighbors,ax in zip([1,3,9],axes):
clf=KNeighborsClassifier(n_neighbors=n_neighbors).fit(X,y)
mglearn.plots.plot_2d_separator(clf,X,fill=True,eps=0.5,ax=ax,alpha=.4)
mglearn.discrete_scatter(X[:,0],X[:,1],y,ax=ax)
ax.set_title('{} neighbor(s)'.format(n_neighbors))
ax.set_xlabel('feature 0')
ax.set_ylabel('feature 1')
axes[0].legend(loc=3)
plt.show()
# 真实数据集
cancer=load_breast_cancer()
X_train,X_test,y_train,y_test=train_test_split(cancer.data,cancer.target,stratify=cancer.target,random_state=66)
training_accuracy=[]
test_accuracy=[]
neighbors_settings=range(1,11)
# 模型复杂度和泛化能力
for n_neighbors in neighbors_settings:
clf=KNeighborsClassifier(n_neighbors=n_neighbors)
clf.fit(X_train,y_train)
training_accuracy.append(clf.score(X_train,y_train))
test_accuracy.append(clf.score(X_test,y_test))
plt.plot(neighbors_settings,training_accuracy,label='training accuracy')
plt.plot(neighbors_settings,test_accuracy,label='test accuracy')
plt.xlabel('n_neighbors')
plt.ylabel('Accuracy')
plt.legend()
plt.show()
2、回归
from sklearn.neighbors import KNeighborsRegressor
from sklearn.model_selection import train_test_split
import numpy as np
import matplotlib.pyplot as plt
import mglearn
# 取 K 个值的平均值
mglearn.plots.plot_knn_regression(n_neighbors=3)
# 划分数据集
X,y=mglearn.datasets.make_wave(n_samples=40)
X_train,X_test,y_train,y_test=train_test_split(X,y,random_state=0)
# 构建模型
reg=KNeighborsRegressor(n_neighbors=3)
reg.fit(X_train,y_train)
print('Test set predictions:\n{}'.format(reg.predict(X_test)))
print('Test set R^2:{:.2f}'.format(reg.score(X_test,y_test)))
fig,axes=plt.subplots(1,3,figsize=(15,4))
line=np.linspace(-3,3,1000).reshape(-1,1)
for n_neighbors,ax in zip([1,3,9],axes):
reg=KNeighborsRegressor(n_neighbors=n_neighbors)
reg.fit(X_train,y_train)
ax.plot(line,reg.predict(line))
ax.plot(X_train,y_train,'^',c=mglearn.cm2(0),markersize=8)
ax.plot(X_test,y_test,'v',c=mglearn.cm2(1),markersize=8)
ax.set_title('{} neighbors\n train score:{:.2f} test score:{:.2f}'.format(
n_neighbors,reg.score(X_train,y_train),
reg.score(X_test,y_test)
))
ax.set_xlabel('Feature')
ax.set_ylabel('Target')
axes[0].legend(['Model predictions','Training data/target','Test data/target'],loc='best')
plt.show()
