Java教程

pandas的数据库操作-筛选数据

本文主要是介绍pandas的数据库操作-筛选数据,对大家解决编程问题具有一定的参考价值,需要的程序猿们随着小编来一起学习吧!

我们都知道SQL数据库的语句很简洁,python中的pandas库也很好用,但是如何将两者给关联起来???

本文将通过pandas来实现类似于SQL中的【过滤】、【排序】、【关联】、【合并】、【更新】、【删除】等操作。

用到的数据集

from sklearn.datasets import load_boston
import pandas as pd

boston = load_boston()
df = pd.DataFrame(boston.data,columns = boston.feature_names)
df['target'] =pd.Series(boston.target) 

df
CRIMZNINDUSCHASNOXRMAGEDISRADTAXPTRATIOBLSTATtarget
00.0063218.02.310.00.5386.57565.24.09001.0296.015.3396.904.9824.0
10.027310.07.070.00.4696.42178.94.96712.0242.017.8396.909.1421.6
20.027290.07.070.00.4697.18561.14.96712.0242.017.8392.834.0334.7
30.032370.02.180.00.4586.99845.86.06223.0222.018.7394.632.9433.4
40.069050.02.180.00.4587.14754.26.06223.0222.018.7396.905.3336.2
.............................................
5010.062630.011.930.00.5736.59369.12.47861.0273.021.0391.999.6722.4
5020.045270.011.930.00.5736.12076.72.28751.0273.021.0396.909.0820.6
5030.060760.011.930.00.5736.97691.02.16751.0273.021.0396.905.6423.9
5040.109590.011.930.00.5736.79489.32.38891.0273.021.0393.456.4822.0
5050.047410.011.930.00.5736.03080.82.50501.0273.021.0396.907.8811.9

506 rows × 14 columns

简单的字段查询

‘’‘SQL语句’’’

  • SELECT CRIM,ZN,CHAS,NOX,RM,RAD FROM boston LIMIT 2;

实现返回每行记录的CRIM,ZN,CHAS,NOX,RM,RAD字段,返回2行。

search = df[["CRIM","ZN","CHAS","NOX","RM","RAD"]].head(2)
search
CRIMZNCHASNOXRMRAD
00.0063218.00.00.5386.5751.0
10.027310.00.00.4696.4212.0

简单的条件过滤 WHERE

‘’‘SQL语句’’’

  • SELECT * FROM boston WHERE CHAS=1 LIMIT 2;
set(df['CHAS']) # 我们将CHAS特征作为分类标签
{0.0, 1.0}
search = df[df['CHAS']==1].head(2)
search
CRIMZNINDUSCHASNOXRMAGEDISRADTAXPTRATIOBLSTATtarget
1423.321050.019.581.00.8715.403100.01.32165.0403.014.7396.9026.8213.4
1521.126580.019.581.00.8715.01288.01.61025.0403.014.7343.2812.1215.3

多条件与或查询WHERE AND|OR

如上满足SQL实现查询同时满足CHAS=1和CRIM>1.0两个条件的记录,返回两行

与关系&

‘’‘SQL语句’’’

  • SELECT * FROM boston WHERE CHAS=1 AND CRIM>=1.0 LIMIT 2;
search = df[(df['CHAS']==1) & (df['CRIM'] >= 1.0)] .head(2) # 注意这里条件是用的圆括号!!!不是方括号
search
CRIMZNINDUSCHASNOXRMAGEDISRADTAXPTRATIOBLSTATtarget
1423.321050.019.581.00.8715.403100.01.32165.0403.014.7396.9026.8213.4
1521.126580.019.581.00.8715.01288.01.61025.0403.014.7343.2812.1215.3

或关系|

‘’‘SQL语句’’’

  • SELECT * FROM boston WHERE CHAS=1 OR CRIM>=1.0 LIMIT 2;
search = df[(df['CHAS']==1) | (df['CRIM'] >= 1.0)] .head(2) # 注意这里条件是用的圆括号!!!不是方括号
search
CRIMZNINDUSCHASNOXRMAGEDISRADTAXPTRATIOBLSTATtarget
161.053930.08.140.00.5385.93529.34.49864.0307.021.0386.856.5823.1
201.251790.08.140.00.5385.57098.13.79794.0307.021.0376.5721.0213.6

条件过滤 控制判断

空判断 查询null的记录

‘’‘SQL语句’’’

  • SELECT * FROM boston WHERE CHAS IS NULL;
search = df[df['CHAS'].isna()]# 注意这里条件是用的圆括号!!!不是方括号
search
CRIMZNINDUSCHASNOXRMAGEDISRADTAXPTRATIOBLSTATtarget

非空判断is not null

查询不为null的记录

‘’‘SQL语句’’’

  • SELECT * FROM boston WHERE CHAS IS NOT NULL;
search = df[df['CHAS'].notna()]# 注意这里条件是用的圆括号!!!不是方括号
search
CRIMZNINDUSCHASNOXRMAGEDISRADTAXPTRATIOBLSTATtarget
00.0063218.02.310.00.5386.57565.24.09001.0296.015.3396.904.9824.0
10.027310.07.070.00.4696.42178.94.96712.0242.017.8396.909.1421.6
20.027290.07.070.00.4697.18561.14.96712.0242.017.8392.834.0334.7
30.032370.02.180.00.4586.99845.86.06223.0222.018.7394.632.9433.4
40.069050.02.180.00.4587.14754.26.06223.0222.018.7396.905.3336.2
.............................................
5010.062630.011.930.00.5736.59369.12.47861.0273.021.0391.999.6722.4
5020.045270.011.930.00.5736.12076.72.28751.0273.021.0396.909.0820.6
5030.060760.011.930.00.5736.97691.02.16751.0273.021.0396.905.6423.9
5040.109590.011.930.00.5736.79489.32.38891.0273.021.0393.456.4822.0
5050.047410.011.930.00.5736.03080.82.50501.0273.021.0396.907.8811.9

506 rows × 14 columns

排序 ORDER BY ASC|DESC

‘’‘SQL语句’’’

  • SELECT * FROM boston WHERE INDUS>10 ORDER BY DESC CHAS;

满足INDUS>10的值按照CHAS进行降序排列

search = df[(df['INDUS']>10)].sort_values(by='CHAS',ascending=False)
search
CRIMZNINDUSCHASNOXRMAGEDISRADTAXPTRATIOBLSTATtarget
1621.833770.019.581.00.6057.80298.22.04075.0403.014.7389.611.9250.0
2120.217190.010.591.00.4895.80753.83.65264.0277.018.6390.9416.0322.4
3643.474280.018.101.00.7188.78082.91.904724.0666.020.2354.555.2921.9
3634.222390.018.101.00.7705.80389.01.904724.0666.020.2353.0414.6416.8
3573.849700.018.101.00.7706.39591.02.505224.0666.020.2391.3413.2721.7
.............................................
2930.082650.013.920.00.4376.12718.45.50274.0289.016.0396.908.5823.9
2940.081990.013.920.00.4376.00942.35.50274.0289.016.0396.9010.4021.7
2950.129320.013.920.00.4376.67831.15.96044.0289.016.0396.906.2728.6
2960.053720.013.920.00.4376.54951.05.96044.0289.016.0392.857.3927.1
5050.047410.011.930.00.5736.03080.82.50501.0273.021.0396.907.8811.9

236 rows × 14 columns

更新UPDATE

‘’‘SQL语句’’’

  • UPDATE boston SET CHAS = 1 WHERE INDUS >10 AND ZN = 0

将满足INDUS >10 和ZN > 0的CHAS字段更新为1

df.loc[(df['INDUS']>10) & (df['ZN']==0),'CHAS']=1
df[df['INDUS']>=10]
CRIMZNINDUSCHASNOXRMAGEDISRADTAXPTRATIOBLSTATtarget
700.088260.010.811.00.4136.4176.65.28734.0305.019.2383.736.7224.2
710.158760.010.811.00.4135.96117.55.28734.0305.019.2376.949.8821.7
720.091640.010.811.00.4136.0657.85.28734.0305.019.2390.915.5222.8
730.195390.010.811.00.4136.2456.25.28734.0305.019.2377.177.5423.4
740.078960.012.831.00.4376.2736.04.25155.0398.018.7394.926.7824.1
.............................................
5010.062630.011.931.00.5736.59369.12.47861.0273.021.0391.999.6722.4
5020.045270.011.931.00.5736.12076.72.28751.0273.021.0396.909.0820.6
5030.060760.011.931.00.5736.97691.02.16751.0273.021.0396.905.6423.9
5040.109590.011.931.00.5736.79489.32.38891.0273.021.0393.456.4822.0
5050.047410.011.931.00.5736.03080.82.50501.0273.021.0396.907.8811.9

236 rows × 14 columns

分组统计

根据CHAS进行分组,返回CHAS和每组的数量

‘’‘SQL语句’’’

  • SELECT CHAS,COUNT(*) FROM boston GROUP BY CHAS;
search = df.groupby('CHAS').size()
search
CHAS
0.0    261
1.0    245
dtype: int64

分组统计 聚合输出

根据CHAS进行分组,返回CHAS,每个组的INDUS和NOX的平均值、最大值

‘’‘SQL语句’’’

  • SELECT CHAS,avg(INDUS),max(INDUS),avg(NOX),max(NOX) FROM boston GROUP BY CHAS
import numpy as np
search = df.groupby('CHAS').agg({'INDUS':[np.mean,np.max],'NOX':[np.mean,np.max]})
search
INDUSNOX
meanamaxmeanamax
CHAS
0.05.48547915.040.4810910.647
1.017.15714327.740.6331060.871

删除

‘’‘SQL语句’’’

  • DELETE FROM boston WHERE WHERE TAX=305 AND RAD = 4;
drop = df.drop(df[(df['TAX']=305) & (df['RAD']=4)].index)
  File "<ipython-input-90-fff36bdbf4c7>", line 1
    drop = df.drop(df[(df['TAX']=305) & (df['RAD']=4)].index)
                                ^
SyntaxError: invalid syntax
这篇关于pandas的数据库操作-筛选数据的文章就介绍到这儿,希望我们推荐的文章对大家有所帮助,也希望大家多多支持为之网!