輕松學(xué)Pytorch-全卷積神經(jīng)網(wǎng)絡(luò)實(shí)現(xiàn)表情識(shí)別

我又又一周沒(méi)有更新這個(gè)系列文章了，但是我說(shuō)過(guò)我會(huì)繼續(xù)堅(jiān)持更新下去的，今天給大家更新了一篇如何使用殘差Block構(gòu)建全卷積神經(jīng)網(wǎng)絡(luò)實(shí)現(xiàn)圖像分類(lèi)，對(duì)的，你沒(méi)有看錯(cuò)就是基于全卷積神經(jīng)網(wǎng)絡(luò)實(shí)現(xiàn)人臉表情圖像的識(shí)別，其中數(shù)據(jù)集一部分來(lái)自CK+，更多的是我自己使用OpenVINO的表情識(shí)別模型來(lái)自動(dòng)標(biāo)注的，總數(shù)大致有5000張的表情圖像。

基于殘差Block，不過(guò)這個(gè)Block跟上一篇中不一樣地方是支持下采樣，它的代碼實(shí)現(xiàn)如下：

 1class?ResidualBlock(torch.nn.Module):
 2????def?__init__(self,?in_channels,?out_channels,?stride=1):
 3????????"""
 4????????Args:
 5??????????in_channels?(int):??Number?of?input?channels.
 6??????????out_channels?(int):?Number?of?output?channels.
 7??????????stride?(int):???????Controls?the?stride.
 8????????"""
 9????????super(ResidualBlock,?self).__init__()
10
11????????self.skip?=?torch.nn.Sequential()
12
13????????if?stride?!=?1?or?in_channels?!=?out_channels:
14????????????self.skip?=?torch.nn.Sequential(
15????????????????torch.nn.Conv2d(in_channels=in_channels,?out_channels=out_channels,?kernel_size=1,?stride=stride,?bias=False),
16????????????????torch.nn.BatchNorm2d(out_channels))
17
18????????self.block?=?torch.nn.Sequential(
19????????????torch.nn.Conv2d(in_channels=in_channels,?out_channels=out_channels,?kernel_size=3,?padding=1,?stride=stride,?bias=False),
20????????????torch.nn.BatchNorm2d(out_channels),
21????????????torch.nn.ReLU(),
22????????????torch.nn.Conv2d(in_channels=out_channels,?out_channels=out_channels,?kernel_size=3,?padding=1,?stride=1,?bias=False),
23????????????torch.nn.BatchNorm2d(out_channels))
24
25????def?forward(self,?x):
26????????out?=?self.block(x)
27????????identity?=?self.skip(x)
28????????out?+=?identity
29????????out?=?F.relu(out)
30????????return?out

其中stride參數(shù)為2的時(shí)候就會(huì)實(shí)現(xiàn)自動(dòng)下采樣；為1的時(shí)候表示跟前面大小保持一致。

模型結(jié)構(gòu)中包括多個(gè)殘差Block，最終的輸出Nx8x1x1, 表示8種表情，然后通過(guò)softmax完成分類(lèi)識(shí)別。模型的輸入：NCHW=Nx3x64x64。模型結(jié)構(gòu)參考了OpenVINO框架中的Caffe版本的表情識(shí)別模型。最終的模型實(shí)現(xiàn)代碼如下：

 1class?EmotionsResNet(torch.nn.Module):
 2????def?__init__(self):
 3????????super(EmotionsResNet,?self).__init__()
 4????????self.cnn_layers?=?torch.nn.Sequential(
 5????????????#?卷積層?(64x64x3的圖像)
 6????????????ResidualBlock(3,?32,?1),
 7????????????ResidualBlock(32,?64,?2),
 8????????????ResidualBlock(64,?64,?2),
 9????????????ResidualBlock(64,?128,?2),
10????????????ResidualBlock(128,?128,?2),
11????????????ResidualBlock(128,?256,?2),
12????????????ResidualBlock(256,?256,?2),
13????????????ResidualBlock(256,?8,?1),
14????????)
15
16????def?forward(self,?x):
17????????#?stack?convolution?layers
18????????x?=?self.cnn_layers(x)
19
20????????#?Nx5x1x1
21????????B,?C,?H,?W?=?x.size()
22????????out?=?x.view(B,?-1)
23????????return?out

基于交叉熵實(shí)現(xiàn)了模型訓(xùn)練，訓(xùn)練了15個(gè)epoch之后，保存模型。訓(xùn)練的代碼如下：

 1if?__name__?==?"__main__":
 2????#?create?a?complete?CNN
 3????model?=?EmotionsResNet()
 4????print(model)
 5
 6????#?使用GPU
 7????if?train_on_gpu:
 8????????model.cuda()
 9
10????ds?=?EmotionDataset("D:/facedb/emotion_dataset")
11????num_train_samples?=?ds.num_of_samples()
12????bs?=?16
13????dataloader?=?DataLoader(ds,?batch_size=bs,?shuffle=True)
14
15????#?訓(xùn)練模型的次數(shù)
16????num_epochs?=?15
17????#?optimizer?=?torch.optim.SGD(model.parameters(),?lr=0.001)
18????optimizer?=?torch.optim.Adam(model.parameters(),?lr=1e-2)
19????model.train()
20
21????#?損失函數(shù)
22????mse_loss?=?torch.nn.MSELoss()
23????cross_loss?=?torch.nn.CrossEntropyLoss()
24????index?=?0
25????for?epoch?in??range(num_epochs):
26????????train_loss?=?0.0
27????????for?i_batch,?sample_batched?in?enumerate(dataloader):
28????????????images_batch,?emotion_batch?=?\
29????????????????sample_batched['image'],?sample_batched['emotion']
30????????????if?train_on_gpu:
31????????????????images_batch,?emotion_batch=?images_batch.cuda(),?emotion_batch.cuda()
32????????????optimizer.zero_grad()
33
34????????????#?forward?pass:?compute?predicted?outputs?by?passing?inputs?to?the?model
35????????????m_emotion_out_?=?model(images_batch)
36????????????emotion_batch?=?emotion_batch.long()
37
38????????????#?calculate?the?batch?loss
39????????????loss?=?cross_loss(m_emotion_out_,?emotion_batch)
40
41????????????#?backward?pass:?compute?gradient?of?the?loss?with?respect?to?model?parameters
42????????????loss.backward()
43
44????????????#?perform?a?single?optimization?step?(parameter?update)
45????????????optimizer.step()
46
47????????????#?update?training?loss
48????????????train_loss?+=?loss.item()
49????????????if?index?%?100?==?0:
50????????????????print('step:?{}?\tTraining?Loss:?{:.6f}?'.format(index,?loss.item()))
51????????????index?+=?1
52
53????????????#?計(jì)算平均損失
54????????train_loss?=?train_loss?/?num_train_samples
55
56????????#?顯示訓(xùn)練集與驗(yàn)證集的損失函數(shù)
57????????print('Epoch:?{}?\tTraining?Loss:?{:.6f}?'.format(epoch,?train_loss))
58
59????#?save?model
60????model.eval()
61????torch.save(model,?'face_emotions_model.pt')

基于OpenCV人臉檢測(cè)得到的ROI區(qū)域，輸入到訓(xùn)練好的人臉表情識(shí)別模型中，就可以預(yù)測(cè)人臉表情，完成實(shí)時(shí)人臉表情識(shí)別，演示代碼如下：

 1cnn_model?=?torch.load("./face_emotions_model.pt")
 2print(cnn_model)
 3#?capture?=?cv.VideoCapture(0)
 4capture?=?cv.VideoCapture("D:/images/video/example_dsh.mp4")
 5
 6#?load?tensorflow?model
 7net?=?cv.dnn.readNetFromTensorflow(model_bin,?config=config_text)
 8while?True:
 9????ret,?frame?=?capture.read()
10????if?ret?is?not?True:
11????????break
12????frame?=?cv.flip(frame,?1)
13????h,?w,?c?=?frame.shape
14????blobImage?=?cv.dnn.blobFromImage(frame,?1.0,?(300,?300),?(104.0,?177.0,?123.0),?False,?False);
15????net.setInput(blobImage)
16????cvOut?=?net.forward()
17????#?繪制檢測(cè)矩形
18????for?detection?in?cvOut[0,0,:,:]:
19????????score?=?float(detection[2])
20????????if?score?>?0.5:
21????????????left?=?detection[3]*w
22????????????top?=?detection[4]*h
23????????????right?=?detection[5]*w
24????????????bottom?=?detection[6]*h
25
26????????????#?roi?and?detect?landmark
27????????????roi?=?frame[np.int32(top):np.int32(bottom),np.int32(left):np.int32(right),:]
28????????????rw?=?right?-?left
29????????????rh?=?bottom?-?top
30????????????img?=?cv.resize(roi,?(64,?64))
31????????????img?=?(np.float32(img)?/?255.0?-?0.5)?/?0.5
32????????????img?=?img.transpose((2,?0,?1))
33????????????x_input?=?torch.from_numpy(img).view(1,?3,?64,?64)
34????????????emotion_?=?cnn_model(x_input.cuda())
35????????????predict_?=?torch.max(emotion_,?1)[1].cpu().detach().numpy()[0]
36????????????emotion_txt?=?emotion_labels[predict_]
37????????????#?繪制
38????????????cv.putText(frame,?("%s"%(emotion_txt)),?(int(left),?int(top)-15),?cv.FONT_HERSHEY_SIMPLEX,?1,?(0,?0,?255),?2)
39????????????cv.rectangle(frame,?(int(left),?int(top)),?(int(right),?int(bottom)),?(255,?0,?0),?thickness=2)
40????????????c?=?cv.waitKey(10)
41????????????if?c?==?27:
42????????????????break
43????????????cv.imshow("face?detection?+?emotion",?frame)
44
45cv.waitKey(0)
46cv.destroyAllWindows()

運(yùn)行結(jié)果如下：

廢話(huà)就不多說(shuō)了，還是希望大家支持，我繼續(xù)寫(xiě)下去！

好消息！?
小白學(xué)視覺(jué)知識(shí)星球
開(kāi)始面向外開(kāi)放啦??????



下載1：OpenCV-Contrib擴(kuò)展模塊中文版教程
在「小白學(xué)視覺(jué)」公眾號(hào)后臺(tái)回復(fù)：擴(kuò)展模塊中文教程，即可下載全網(wǎng)第一份OpenCV擴(kuò)展模塊教程中文版，涵蓋擴(kuò)展模塊安裝、SFM算法、立體視覺(jué)、目標(biāo)跟蹤、生物視覺(jué)、超分辨率處理等二十多章內(nèi)容。

下載2：Python視覺(jué)實(shí)戰(zhàn)項(xiàng)目52講
在「小白學(xué)視覺(jué)」公眾號(hào)后臺(tái)回復(fù)：Python視覺(jué)實(shí)戰(zhàn)項(xiàng)目，即可下載包括圖像分割、口罩檢測(cè)、車(chē)道線(xiàn)檢測(cè)、車(chē)輛計(jì)數(shù)、添加眼線(xiàn)、車(chē)牌識(shí)別、字符識(shí)別、情緒檢測(cè)、文本內(nèi)容提取、面部識(shí)別等31個(gè)視覺(jué)實(shí)戰(zhàn)項(xiàng)目，助力快速學(xué)校計(jì)算機(jī)視覺(jué)。

下載3：OpenCV實(shí)戰(zhàn)項(xiàng)目20講
在「小白學(xué)視覺(jué)」公眾號(hào)后臺(tái)回復(fù)：OpenCV實(shí)戰(zhàn)項(xiàng)目20講，即可下載含有20個(gè)基于OpenCV實(shí)現(xiàn)20個(gè)實(shí)戰(zhàn)項(xiàng)目，實(shí)現(xiàn)OpenCV學(xué)習(xí)進(jìn)階。

交流群

歡迎加入公眾號(hào)讀者群一起和同行交流，目前有SLAM、三維視覺(jué)、傳感器、自動(dòng)駕駛、計(jì)算攝影、檢測(cè)、分割、識(shí)別、醫(yī)學(xué)影像、GAN、算法競(jìng)賽等微信群（以后會(huì)逐漸細(xì)分），請(qǐng)掃描下面微信號(hào)加群，備注：”昵稱(chēng)+學(xué)校/公司+研究方向“，例如：”張三?+?上海交大?+?視覺(jué)SLAM“。請(qǐng)按照格式備注，否則不予通過(guò)。添加成功后會(huì)根據(jù)研究方向邀請(qǐng)進(jìn)入相關(guān)微信群。請(qǐng)勿在群內(nèi)發(fā)送廣告，否則會(huì)請(qǐng)出群，謝謝理解~