普通最小二乘法回归 – OLS (ordinary least square)

大家好，又见面了，我是你们的朋友全栈君。如果您正在找激活码,请点击查看最新教程,关注关注公众号 “全栈程序员社区” 获取激活教程,可能之前旧版本教程已经失效.最新Idea2022.1教程亲测有效,一键激活。

Jetbrains全系列IDE稳定放心使用

前言

这篇博客用来记录初学 普通最小二乘回归 遇到的相关知识点和解决问题的过程。

开发环境：Pycharm 2018.1.2
版本：Python 2.7.14 :: Anaconda, Inc.

普通最小二乘法回归

回归 – 已有数据

数据集：Cal_housing.csv
简 介：从 1990 年至今，美国加州所有街区人口普查的信息，关于 9 组变量，共 20640 个观测值。

用下面代码读入数据, 并弄清楚哪些是自变量哪个是因变量:

import pandas as pd
import numpy as np

data = pd.read_csv("cal_housing.csv")
name = data.columns

X = data[name[:8]]  # 第1-8列
y = data[name[8:9]]  # 第9列

print("X name :", name[:8])
print("y name :", name[8:9])
print(data.shape, X.shape, y.shape)  # 返回行列数

---------------------------------------------
('X name :', Index([u'longitude', u'latitude', u'housingMedianAge', u'totalRooms',
       u'totalBedrooms', u'population', u'households', u'medianIncome'],
      dtype='object'))
('y name :', Index([u'medianHouseValue'], dtype='object'))
((20640, 9), (20640, 8), (20640, 1))

把数据随机分成训练集和测试集

可自己决定随机种子(多少位数都可以)和测试集百分比(小于0.5即小于50%)

seed = 8888  # 随机种子
proportion = 0.1  # 测试集百分比

from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=proportion, random_state=seed)

print(X_train.shape,X_test.shape, y_train.shape, y_test.shape)
---------------------------------------------------
((18576, 8), (2064, 8), (18576, 1), (2064, 1))

做回归, 并求出 R^2 和 N M S E

reg = LinearRegression()  # 线性回归（Linear Regression）
res = reg.fit(X_train, y_train)  # 对训练集X_train, y_train进行训练
y_hat = res.predict(X_test)  # 使用训练得到的估计器对输入为X_test的集合进行预测,得到y_hat
e = y_test-y_hat  # 计算残差
SSE_cv = np.mean(e**2)  # 残差平方和
SSE_test = np.mean((y_test-np.mean(y_test))**2)  # 拍脑袋平方和
NMSE_cv = SSE_cv/SSE_test  # 标准化均方误差 NMSE_cv
R2_cv = 1 - NMSE_cv  # 可决系数R2_cv
print R2_cv
print NMSE_cv
-----------------------------------------------------------------
medianHouseValue    0.657186
dtype: float64
medianHouseValue    0.342814
dtype: float64

回归 – 模拟数据

自己决定样本量(n), 自变量个数(p)和系数值(B), 自己决定正态误差的均值m和标准差s

seed = 8888  # 随机种子
n = 100  # 样本量
p = 7  # 自变量个数
m = 0  # 误差项均值
s = 5  # 标准差
B = [2, 5, 16, 9, -3, -5, -2]  # beta值
C = [2, 2]

np.random.seed(seed)
X = np.random.normal(0, 1, (n, p))
y = X.dot(B)+np.random.normal(m, s, n)

print(X.shape, y.shape)
----------------------------------------------------------
((100L, 7L), (100L,))

实施回归

import statsmodels.api as sm
# 增加截距项
mod = sm.OLS(y, X)  # 普通最小二乘模型，ordinary least square model
res = mod.fit()
#输出R^2
print("R^2:",res2.rsquared,"\nNMSE:",1-res2.rsquared)
----------------------------------------------------------
R^2: 0.92564484308 
NMSE: 0.0743551569196

print (res2.summary())

---------------------------------
 OLS Regression Results                            
==============================================================================
Dep. Variable:                      y   R-squared:                       0.926
Model:                            OLS   Adj. R-squared:                  0.920
Method:                 Least Squares   F-statistic:                     165.4
Date:                Mon, 07 May 2018   Prob (F-statistic):           1.32e-49
Time:                        09:54:25   Log-Likelihood:                -304.71
No. Observations:                 100   AIC:                             623.4
Df Residuals:                      93   BIC:                             641.7
Df Model:                           7                                         
Covariance Type:            nonrobust                                         
==============================================================================
                 coef    std err          t      P>|t|      [0.025      0.975]
------------------------------------------------------------------------------
x1             2.3477      0.532      4.415      0.000       1.292       3.404
x2             5.5829      0.511     10.929      0.000       4.568       6.597
x3            15.7619      0.596     26.444      0.000      14.578      16.946
x4             8.9595      0.521     17.181      0.000       7.924       9.995
x5            -3.3048      0.530     -6.233      0.000      -4.358      -2.252
x6            -4.9932      0.491    -10.175      0.000      -5.968      -4.019
x7            -2.0126      0.536     -3.754      0.000      -3.077      -0.948
==============================================================================
Omnibus:                        0.577   Durbin-Watson:                   1.970
Prob(Omnibus):                  0.749   Jarque-Bera (JB):                0.227
Skew:                          -0.078   Prob(JB):                        0.893
Kurtosis:                       3.174   Cond. No.                         1.51
==============================================================================

Warnings:
[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.

附录

源码以及数据下载