8個很少用，但卻很好用的 Python 庫，一起來看看吧
本文介紹一些我們可能很少聽過，但是對于特定問題或者特定任務(wù)來說，可能會非常有幫助的 Python 庫
為了配合上述 Python 庫的使用，我們先從 Kaggle 上下載一個數(shù)據(jù) -- Animal Care and Control Adopted Animals
https://www.kaggle.com/jinbonnie/animal-data
import pandas as pd
df = pd.read_csv('animal-data-1.csv')
print('Number of pets:', len(df))
print(df.head(3))
Number of pets: 10290
id intakedate intakereason istransfer sheltercode \
0 15801 2009-11-28 00:00:00 Moving 0 C09115463
1 15932 2009-12-08 00:00:00 Moving 0 D09125594
2 28859 2012-08-10 00:00:00 Abandoned 0 D12082309
identichipnumber animalname breedname basecolour speciesname \
0 0A115D7358 Jadzia Domestic Short Hair Tortie Cat
1 0A11675477 Gonzo German Shepherd Dog/Mix Tan Dog
2 0A13253C7B Maggie Shep Mix/Siberian Husky Various Dog
... movementdate movementtype istrial returndate returnedreason \
0 ... 2017-05-13 00:00:00 Adoption 0.0 NaN Stray
1 ... 2017-04-24 00:00:00 Adoption 0.0 NaN Stray
2 ... 2017-04-15 00:00:00 Adoption 0.0 NaN Stray
deceaseddate deceasedreason diedoffshelter puttosleep isdoa
0 NaN Died in care 0 0 0
1 NaN Died in care 0 0 0
2 NaN Died in care 0 0 0
[3 rows x 23 columns]
下面我們就進入這些 Python 庫的介紹
1. Missingno
Missingno 是用于在數(shù)據(jù)集當中顯示缺失值的，這對于我們的數(shù)據(jù)分析來說是非常有用的。而且還能做成熱力圖或者條形圖，來更加直觀的觀察缺失值
matrix - 類似于 seaborn 中的缺失值熱圖，可以最多展示數(shù)據(jù)集中50列的密度情況，也可以通過右側(cè)的迷你圖，來整體觀測數(shù)據(jù)集的缺失情況
bar - 案列顯示缺失值情況
heatmap - 展示缺失值之間的相關(guān)性，本質(zhì)上，揭示了變量的存在與否對另一個變量的存在的影響程度。而對于沒有缺失值的列或者全完沒有值的列，則不會出現(xiàn)在這里
dendrogram - 樹狀圖與熱圖類似，展示的是列之間缺失的相關(guān)性，而與熱圖不同的地方是通過一組列來揭示相關(guān)性
下面我們就來具體看看這些圖表
import missingno as msno
msno.matrix(df)
msno.bar(df)
msno.heatmap(df)
msno.dendrogram(df)
對于 missingno 圖表，我們還可以自定義一些參數(shù)
msno.matrix(
df,
figsize=(25,7),
fontsize=30,
sort='descending',
color=(0.494, 0.184, 0.556),
width_ratios=(10, 1)
)
最后我們還可以與 matplotlib 相結(jié)合，制作更加優(yōu)美的圖表
import matplotlib.pyplot as plt
msno.matrix(
df,
figsize=(25,7),
fontsize=30,
sort='descending',
color=(0.494, 0.184, 0.556),
width_ratios=(10, 1),
inline=False
)
plt.title('Missing Values Pet Dataset', fontsize=55)
plt.show()
2. Tabulate
這個庫可以在 Python 中打印出漂亮的表格，允許智能和可定制的列對齊、數(shù)字和文本格式、小數(shù)點對齊，也是一個數(shù)據(jù)分析過程中的好用工具。支持的數(shù)據(jù)類型包括 dataframe, list of lists or dictionaries, dictionary, NumPy array
from tabulate import tabulate
df_pretty_printed = df.iloc[:5, [1,2,4,6]]
print(tabulate(df_pretty_printed))
- ----------- ----------------------- ------ -----
0 Jadzia Domestic Short Hair Female Stray
1 Gonzo German Shepherd Dog/Mix Male Stray
2 Maggie Shep Mix/Siberian Husky Female Stray
3 Pretty Girl Domestic Short Hair Female Stray
4 Pretty Girl Domestic Short Hair Female Stray
- ----------- ----------------------- ------ -----
我們還可以自定義表格頭，使用參數(shù) headers
print(tabulate(
df_pretty_printed,
headers='keys',
tablefmt='fancy_grid',
stralign='center'
))
│ │ animalname │ breedname │ sexname │ returnedreason │
╞════╪══════════════╪═════════════════════════╪═══════════╪══════════════════╡
│ 0 │ Jadzia │ Domestic Short Hair │ Female │ Stray │
├────┼──────────────┼─────────────────────────┼───────────┼──────────────────┤
│ 1 │ Gonzo │ German Shepherd Dog/Mix │ Male │ Stray │
├────┼──────────────┼─────────────────────────┼───────────┼──────────────────┤
│ 2 │ Maggie │ Shep Mix/Siberian Husky │ Female │ Stray │
├────┼──────────────┼─────────────────────────┼───────────┼──────────────────┤
│ 3 │ Pretty Girl │ Domestic Short Hair │ Female │ Stray │
├────┼──────────────┼─────────────────────────┼───────────┼──────────────────┤
│ 4 │ Pretty Girl │ Domestic Short Hair │ Female │ Stray │
╘════╧══════════════╧═════════════════════════╧═══════════╧══════════════════╛
不過這個庫打印出的表格數(shù)據(jù)在手機屏幕上會有一定的兼容性問題，只有在PC機上才能有最佳的顯示效果
3. Wikipedia
維基百科庫，可以方便的訪問維基百科信息，以及獲取數(shù)據(jù)
該庫的幾個主要功能如下：
搜索維基百科 - search()
獲取文章摘要 - summary
獲取完整頁面內(nèi)容，包括圖像、鏈接等 - page()
選擇語言 - set_lang()
我們以上面數(shù)據(jù)集當中的 Siberian Husky 為關(guān)鍵詞，在維基百科中設(shè)置為俄語搜索一下，看看結(jié)果
import wikipedia
wikipedia.set_lang('ru')
print(wikipedia.search('Siberian Husky'))
['Сибирский хаски', 'Древние породы собак', 'Маккензи Ривер Хаски', 'Породы собак по классификации кинологических организаций', 'Ричардсон, Кевин Майкл']
我們獲取第一個搜索結(jié)果當中的第一段話
print(wikipedia.summary('Сибирский хаски', sentences=1))
Сибирский хаски — заводская специализированная порода собак, выведенная чукчами северо-восточной части Сибири и зарегистрированная американскими кинологами в 1930-х годах как ездовая собака, полученная от аборигенных собак Дальнего Востока России, в основном из Анадыря, Колымы, Камчатки у местных оседлых приморских племён — юкагиров, кереков, азиатских эскимосов и приморских чукчей — анкальын (приморские, поморы — от анкы (море)).
下面我們再來獲取圖片信息
print(wikipedia.page('Сибирский хаски').images[0])
就可以拿到圖片了
4. Wget
對于這個庫，熟悉 Linux 的同學應(yīng)該都知道，一個好用的 shell 命令也叫做 wget，是用來下載文件的，這個 Python 庫也有著同樣的功能
我們來試試下載上面哈士奇圖片吧
import wget
wget.download('https://upload.wikimedia.org/wikipedia/commons/a/a3/Black-Magic-Big-Boy.jpg')
'Black-Magic-Big-Boy.jpg'
當然使用該庫，我們還可以方便的下載 HTML 文件
wget.download('https://www.kaggle.com/jinbonnie/animal-data')
'animal-data'
下載好的文件內(nèi)容類似：
<!DOCTYPE html>
<html lang="en">
<head>
<title>Animal Care and Control Adopted Animals | Kaggle</title>
<meta charset="utf-8" />
<meta name="robots" content="index, follow" />
<meta name="description" content="animal situation in Bloomington Animal Shelter from 2017-2020" />
<meta name="turbolinks-cache-control" content="no-cache" />
5. Faker
這個庫是用來生成假數(shù)據(jù)的，這個在我們平時的程序測試當中還是非常好用的。它可以生成包括名字、郵件地址、電話號碼、工作、句子、顏色，貨幣等等眾多假數(shù)據(jù)，同時還支持本地化，也就是可以將當前工作語言環(huán)境作為參數(shù)，生成當前語言的假數(shù)據(jù)，實在是太貼心了
from faker import Faker
fake = Faker()
print(
'Fake color:', fake.color(), '\n'
'Fake job:', fake.job(), '\n'
'Fake email:', fake.email(), '\n'
)
# Printing a list of fake Korean and Portuguese addresses
fake = Faker(['ko_KR', 'pt_BR'])
for _ in range(5):
print(fake.unique.address()) # using the `.unique` property
print('\n')
# Assigning a seed number to print always the same value / data set
fake = Faker()
Faker.seed(3920)
print('This English fake name is always the same:', fake.name())
Fake color: #212591
Fake job: Occupational therapist
Fake email: [email protected]
Estrada Lavínia da Luz, 62
Oeste
85775858 Moura / SE
Residencial de Moreira, 57
Morro Dos Macacos
75273529 Farias / TO
??????? ??? ???? (??????)
???? ??? ????? (????)
???? ??? ??53?
This English fake name is always the same: Kim Lopez
我們再回到我們的動物數(shù)據(jù)集，我們發(fā)現(xiàn)有兩個動物的名字不是特別好
df_bad_names = df[df['animalname'].str.contains('Stink|Pooh')]
print(df_bad_names)
identichipnumber animalname breedname speciesname sexname \
1692 NaN Stinker Domestic Short Hair Cat Male
3336 981020023417175 Pooh German Shepherd Dog Dog Female
3337 981020023417175 Pooh German Shepherd Dog Dog Female
returndate returnedreason
1692 NaN Stray
3336 2018-05-14 00:00:00 Incompatible with owner lifestyle
3337 NaN Stray
下面我們分別為這兩只貓狗重新命名一個好聽的名字
# Defining a function to rename the unlucky pets
def rename_pets(name):
if name == 'Stinker':
fake = Faker()
Faker.seed(162)
name = fake.name()
if name == 'Pooh':
fake = Faker(['de_DE'])
Faker.seed(20387)
name = fake.name()
return name
# Renaming the pets
df['animalname'] = df['animalname'].apply(rename_pets)
# Checking the results
print(df.iloc[df_bad_names.index.tolist(), :] )
identichipnumber animalname breedname speciesname \
1692 NaN Steven Harris Domestic Short Hair Cat
3336 981020023417175 Helena Fliegner-Karz German Shepherd Dog Dog
3337 981020023417175 Helena Fliegner-Karz German Shepherd Dog Dog
sexname returndate returnedreason
1692 Male NaN Stray
3336 Female 2018-05-14 00:00:00 Incompatible with owner lifestyle
3337 Female NaN Stray
怎么樣，名字是不是好聽多了
6. Numerizer
該庫可以將自然語言轉(zhuǎn)化為數(shù)字，我們來看看吧
我們先來獲取名稱中包含數(shù)據(jù)的動物的信息
df_numerized_names = df[['identichipnumber', 'animalname', 'speciesname']]\
[df['animalname'].str.contains('Two|Seven|Fifty')]
df_numerized_names
下面我們就把名稱中的數(shù)字轉(zhuǎn)化成阿拉伯數(shù)字
from numerizer import numerize
df['animalname'] = df['animalname'].apply(lambda x: numerize(x))
df[['identichipnumber', 'animalname', 'speciesname']].iloc[df_numerized_names.index.tolist(), :]
7. Emoji
符號庫，我們可以根據(jù) Unicode Consortium 2 定義的表情符號代碼將字符串轉(zhuǎn)換為表情符號，emoji 庫只有兩個函數(shù)：emojize() 和 demojize()
import emoji
print(emoji.emojize(':koala:'))
print(emoji.demojize(''))
print(emoji.emojize(':rana:', language='it'))
??
:koala:
??
下面我們來符號化我們的動物吧
print(df['speciesname'].unique())
['Cat' 'Dog' 'House Rabbit' 'Rat' 'Bird' 'Opossum' 'Chicken' 'Wildlife'
'Ferret' 'Tortoise' 'Pig' 'Hamster' 'Guinea Pig' 'Gerbil' 'Lizard'
'Hedgehog' 'Chinchilla' 'Goat' 'Snake' 'Squirrel' 'Sugar Glider' 'Turtle'
'Tarantula' 'Mouse' 'Raccoon' 'Livestock' 'Fish']
我們要將字母全部轉(zhuǎn)化為小寫，然后在前后分別添加冒號
df['speciesname'] = df['speciesname'].apply(lambda x: emoji.emojize(f':{x.lower()}:',
use_aliases=True))
print(df['speciesname'].unique())
['' '' ':house rabbit:' '' '' ':opossum:' '' ':wildlife:' ':ferret:'
':tortoise:' '' '' ':guinea pig:' ':gerbil:' '' '' ':chinchilla:' ''
'' ':squirrel:' ':sugar glider:' '' ':tarantula:' '' '' ':livestock:'
'']
再進行名稱同義詞轉(zhuǎn)化
df['speciesname'] = df['speciesname'].str.replace(':house rabbit:', ':rabbit:')\
.replace(':tortoise:', ':turtle:')\
.replace(':squirrel:', ':chipmunk:')
df['speciesname'] = df['speciesname'].apply(lambda x: emoji.emojize(x, variant='emoji_type'))
print(df['speciesname'].unique())
['' '' '?' '' '' ':opossum:?' '' ':wildlife:?' ':ferret:?' '?' ''
'' ':guinea pig:' ':gerbil:?' '' '' ':chinchilla:?' '' '' ''
':sugar glider:' '' ':tarantula:?' '' '' ':livestock:?' '']
對于剩下的這些沒有對應(yīng)動物名稱的數(shù)據(jù)，我們再轉(zhuǎn)化會原來的數(shù)據(jù)形式
df['speciesname'] = df['speciesname'].str.replace(':', '').apply(lambda x: x.title())
print(df['speciesname'].unique())
df[['animalname', 'speciesname', 'breedname']].head(3)
['' '' '?' '' '' 'Opossum?' '' 'Wildlife?' 'Ferret?' '?' '' ''
'Guinea Pig' 'Gerbil?' '' '' 'Chinchilla?' '' '' '' 'Sugar Glider'
'' 'Tarantula?' '' '' 'Livestock?' '']
這樣，我們就完成了符號化動物名稱了
8. PyAztro
這個庫的創(chuàng)造可能僅僅是為了娛樂吧，該庫可以預測每一天不同星座的幸運數(shù)字、幸運時間、幸運顏色等等，感興趣的朋友可以玩一玩
import pyaztro
pyaztro.Aztro(sign='taurus').description
'You need to make a radical change in some aspect of your life - probably related to your home. It could be time to buy or sell or just to move on to some more promising location.'
我們再來看看我們的數(shù)據(jù)集，在我們的數(shù)據(jù)集中，有一只貓和一只狗叫 Aries（白羊座）
df[['animalname', 'speciesname']][(df['animalname'] == 'Aries')]
還有很多動物叫做 Leo （獅子座）
print('Leo:', df['animalname'][(df['animalname'] == 'Leo')].count())
Leo: 18
我們假設(shè)這就是動物們的星座，然后來使用該庫預測他們的運勢吧
aries = pyaztro.Aztro(sign='aries')
leo = pyaztro.Aztro(sign='leo')
print('ARIES: \n',
'Sign:', aries.sign, '\n',
'Current date:', aries.current_date, '\n',
'Date range:', aries.date_range, '\n',
'Sign description:', aries.description, '\n',
'Mood:', aries.mood, '\n',
'Compatibility:', aries.compatibility, '\n',
'Lucky number:', aries.lucky_number, '\n',
'Lucky time:', aries.lucky_time, '\n',
'Lucky color:', aries.color, 2*'\n',
'LEO: \n',
'Sign:', leo.sign, '\n',
'Current date:', leo.current_date, '\n',
'Date range:', leo.date_range, '\n',
'Sign description:', leo.description, '\n',
'Mood:', leo.mood, '\n',
'Compatibility:', leo.compatibility, '\n',
'Lucky number:', leo.lucky_number, '\n',
'Lucky time:', leo.lucky_time, '\n',
'Lucky color:', leo.color)
ARIES:
Sign: aries
Current date: 2021-02-06
Date range: [datetime.datetime(2021, 3, 21, 0, 0), datetime.datetime(2021, 4, 20, 0, 0)]
Sign description: It's a little harder to convince people your way is best today -- in part because it's much tougher to play on their emotions. Go for the intellectual arguments and you should do just fine.
Mood: Helpful
Compatibility: Leo
Lucky number: 18
Lucky time: 8am
Lucky color: Gold
LEO:
Sign: leo
Current date: 2021-02-06
Date range: [datetime.datetime(2021, 7, 23, 0, 0), datetime.datetime(2021, 8, 22, 0, 0)]
Sign description: Big problems need big solutions -- but none of the obvious ones seem to be working today! You need to stretch your mind as far as it will go in order to really make sense of today's issues.
Mood: Irritated
Compatibility: Libra
Lucky number: 44
Lucky time: 12am
Lucky color: Navy Blue
是不是還蠻有意思的呢
好了，今天的分享就到這里，我們下次見
點擊關(guān)注公眾號，閱讀更多精彩內(nèi)容
評論
圖片
表情