OpenGL ES渲染播放視頻
PS: 學(xué)會合理安排，留足空間和時間。
MediaPlayer和 OpenGL ES 來實(shí)現(xiàn)基本視頻渲染以及視頻畫面的矯正，主要內(nèi)容如下：SurfaceTexture
渲染視頻
畫面矯正
SurfaceTexture
SurfaceTexture 從 Android 3.0 開始加入，其對圖像流的處理并不直接顯示，而是從圖像流中捕獲幀作為 OpenGL 的外部紋理，圖像流主要來自相機(jī)預(yù)覽和視頻解碼，可對圖像流進(jìn)行二次處理，如濾鏡以及特效等，可以理解為SurfaceTexture 是Surface 和 OpenGL ES 紋理的結(jié)合。SurfaceTexture 創(chuàng)建的 Surface 是數(shù)據(jù)的生產(chǎn)者，而 SurfaceTexture是對應(yīng)的消費(fèi)者，Surface接收媒體數(shù)據(jù)并將數(shù)據(jù)發(fā)送到SurfaceTexture，當(dāng)調(diào)用 updateTexImage 的時候，創(chuàng)建SurfaceTexture的紋理對象相應(yīng)的內(nèi)容將更新為最新圖像幀，也就是會將圖像幀轉(zhuǎn)換為 GL 紋理，并將該紋理綁定到GL_TEXTURE_EXTERNAL_OES紋理對象上，updateTexImage 僅在 OpenGL ES 上下文線程中調(diào)用，一般在onDrawFrame中進(jìn)行調(diào)用。渲染視頻
MediaPlayer如何播放視頻大家應(yīng)該非常熟悉，這里不在贅述，有了上面小節(jié)SurfaceTexture的介紹，使用 OpenGL ES 實(shí)現(xiàn)視頻渲染非常簡單，定義頂點(diǎn)坐標(biāo)和紋理坐標(biāo)如下：1// 頂點(diǎn)坐標(biāo)
2private val vertexCoordinates = floatArrayOf(
3 1.0f, 1.0f,
4 -1.0f, 1.0f,
5 -1.0f, -1.0f,
6 1.0f, -1.0f
7)
8// 紋理坐標(biāo)
9private val textureCoordinates = floatArrayOf(
10 1.0f, 0.0f,
11 0.0f, 0.0f,
12 0.0f, 1.0f,
13 1.0f, 1.0f
14)
1/**
2 * 生成紋理ID
3 */
4fun createTextureId(): Int {
5 val tex = IntArray(1)
6 GLES20.glGenTextures(1, tex, 0)
7 if (tex[0] == 0) {
8 throw RuntimeException("create OES texture failed, ${Thread.currentThread().name}")
9 }
10 return tex[0]
11}
12
13/**
14 * 創(chuàng)建OES紋理
15 * YUV格式到RGB的自動轉(zhuǎn)化
16 */
17fun activeBindOESTexture(textureId:Int) {
18 // 激活紋理單元
19 GLES20.glActiveTexture(GLES20.GL_TEXTURE0)
20 // 綁定紋理ID到紋理單元的紋理目標(biāo)上
21 GLES20.glBindTexture(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, textureId)
22 // 設(shè)置紋理參數(shù)
23 GLES20.glTexParameterf(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, GL10.GL_TEXTURE_MIN_FILTER, GL10.GL_NEAREST.toFloat())
24 GLES20.glTexParameterf(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, GL10.GL_TEXTURE_MAG_FILTER, GL10.GL_LINEAR.toFloat())
25 GLES20.glTexParameterf(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, GL10.GL_TEXTURE_WRAP_S, GL10.GL_CLAMP_TO_EDGE.toFloat())
26 GLES20.glTexParameterf(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, GL10.GL_TEXTURE_WRAP_T, GL10.GL_CLAMP_TO_EDGE.toFloat())
27 Log.d(TAG, "activeBindOESTexture: texture id $textureId")
28}
GL_TEXTURE_EXTERNAL_OES，可以自動完成 YUV 格式到 RGB 的自動轉(zhuǎn)換，下面來看下著色器，其中頂點(diǎn)著色器中接收紋理坐標(biāo)并保存到vTextureCoordinate供片段著色器使用，具體如下：1// 頂點(diǎn)著色器
2attribute vec4 aPosition; // 頂點(diǎn)坐標(biāo)
3attribute vec2 aCoordinate; // 紋理坐標(biāo)
4varying vec2 vTextureCoordinate;
5void main() {
6 gl_Position = aPosition;
7 vTextureCoordinate = aCoordinate;
8}
9
10// 片段著色器
11#extension GL_OES_EGL_image_external : require
12precision mediump float;
13varying vec2 vTextureCoordinate;
14uniform samplerExternalOES uTexture; // OES紋理
15void main() {
16 gl_FragColor=texture2D(uTexture, vTextureCoordinate);
17}
1class PlayRenderer(
2 private var context: Context,
3 private var glSurfaceView: GLSurfaceView
4) : GLSurfaceView.Renderer,
5 VideoRender.OnNotifyFrameUpdateListener, MediaPlayer.OnPreparedListener,
6 MediaPlayer.OnVideoSizeChangedListener, MediaPlayer.OnCompletionListener,
7 MediaPlayer.OnErrorListener {
8 companion object {
9 private const val TAG = "PlayRenderer"
10 }
11 private lateinit var videoRender: VideoRender
12 private lateinit var mediaPlayer: MediaPlayer
13 private val projectionMatrix = FloatArray(16)
14 private val viewMatrix = FloatArray(16)
15 private val vPMatrix = FloatArray(16)
16 // 用于視頻比例的計(jì)算，詳見下文
17 private var screenWidth: Int = -1
18 private var screenHeight: Int = -1
19 private var videoWidth: Int = -1
20 private var videoHeight: Int = -1
21
22 override fun onSurfaceCreated(gl: GL10?, config: EGLConfig?) {
23 L.i(TAG, "onSurfaceCreated")
24 GLES20.glClearColor(0f, 0f, 0f, 0f)
25 videoRender = VideoRender(context)
26 videoRender.setTextureID(TextureHelper.createTextureId())
27 videoRender.onNotifyFrameUpdateListener = this
28 initMediaPlayer()
29 }
30
31 override fun onSurfaceChanged(gl: GL10?, width: Int, height: Int) {
32 L.i(TAG, "onSurfaceChanged > width:$width,height:$height")
33 screenWidth = width
34 screenHeight = height
35 GLES20.glViewport(0, 0, width, height)
36 }
37
38 override fun onDrawFrame(gl: GL10) {
39 L.i(TAG, "onDrawFrame")
40 gl.glClear(GL10.GL_COLOR_BUFFER_BIT or GL10.GL_DEPTH_BUFFER_BIT)
41 videoRender.draw(vPMatrix)
42 }
43
44 override fun onPrepared(mp: MediaPlayer?) {
45 L.i(OpenGLActivity.TAG, "onPrepared")
46 mediaPlayer.start()
47 }
48
49 override fun onVideoSizeChanged(mp: MediaPlayer?, width: Int, height: Int) {
50 L.i(OpenGLActivity.TAG, "onVideoSizeChanged > width:$width ,height:$height")
51 this.videoWidth = width
52 this.videoHeight = height
53 }
54
55 override fun onCompletion(mp: MediaPlayer?) {
56 L.i(OpenGLActivity.TAG, "onCompletion")
57 }
58
59 override fun onError(mp: MediaPlayer?, what: Int, extra: Int): Boolean {
60 L.i(OpenGLActivity.TAG, "error > what:$what,extra:$extra")
61 return true
62 }
63
64 private fun initMediaPlayer() {
65 mediaPlayer = MediaPlayer()
66 mediaPlayer.setOnPreparedListener(this)
67 mediaPlayer.setOnVideoSizeChangedListener(this)
68 mediaPlayer.setOnCompletionListener(this)
69 mediaPlayer.setOnErrorListener(this)
70 mediaPlayer.setDataSource(Environment.getExternalStorageDirectory().absolutePath + "/video.mp4")
71 mediaPlayer.setSurface(videoRender.getSurface())
72 mediaPlayer.prepareAsync()
73 }
74 // 通知請求渲染
75 override fun onNotifyUpdate() {
76 glSurfaceView.requestRender()
77 }
78
79 fun destroy() {
80 mediaPlayer.stop()
81 mediaPlayer.release()
82 }
83}
VideoRender主要是渲染操作，這部分代碼和上篇文章中的大同小異，這里就不貼了。SurfaceTetre的updateTexImage方法更新圖像幀，該方法必須在 OpenGL ES 上下文中使用，可以設(shè)置GLSurfaceView的渲染模式為RENDERMODE_WHEN_DIRTY避免一直繪制，當(dāng)onFrameAvailable會調(diào)的時候，也就是有了可用的數(shù)據(jù)之后再進(jìn)行requestRender以減少必要的消耗。
畫面矯正
Shader的修改，主要是頂點(diǎn)頂點(diǎn)著色器的變化，如下：1attribute vec4 aPosition;
2attribute vec2 aCoordinate;
3uniform mat4 uMVPMatrix;
4varying vec2 vTextureCoordinate;
5void main() {
6 gl_Position = uMVPMatrix * aPosition;
7 vTextureCoordinate = aCoordinate;
8}
uMVPMatrix，而uMVPMatrix是投影矩陣和視圖矩陣的乘積，投影矩陣的計(jì)算，OpenGL ES 使用Matrix來進(jìn)行矩陣運(yùn)算，正交投影使用Matrix.orthoM來生成投影矩陣，計(jì)算方式如下：1// 計(jì)算視頻縮放比例（投影矩陣）
2val screenRatio = screenWidth / screenHeight.toFloat()
3val videoRatio = videoWidth / videoHeight.toFloat()
4val ratio: Float
5if (screenWidth > screenHeight) {
6 if (videoRatio >= screenRatio) {
7 ratio = videoRatio / screenRatio
8 Matrix.orthoM(
9 projectionMatrix, 0,
10 -1f, 1f, -ratio, ratio, 3f, 5f
11 )
12 } else {
13 ratio = screenRatio / videoRatio
14 Matrix.orthoM(
15 projectionMatrix, 0,
16 -ratio, ratio, -1f, 1f, 3f, 5f
17 )
18 }
19} else {
20 if (videoRatio >= screenRatio) {
21 ratio = videoRatio / screenRatio
22 Matrix.orthoM(
23 projectionMatrix, 0,
24 -1f, 1f, -ratio, ratio, 3f, 5f
25 )
26 } else {
27 ratio = screenRatio / videoRatio
28 Matrix.orthoM(
29 projectionMatrix, 0,
30 -ratio, ratio, -1f, 1f, 3f, 5f
31 )
32 }
33}
ratio就是就是正交投影視景體的邊界，以本人手機(jī)為例計(jì)算一下ratio，這里為了方便計(jì)算屏幕寬等于視頻寬，屏幕 1080 * 2260，視頻 1080 * 540，則ratio為 2260 / 540 約等于 4.18，顯然如果按照屏幕高度為基準(zhǔn)，當(dāng)視頻高度為 2260 的時候，視頻寬度為 4520 遠(yuǎn)遠(yuǎn)超出了屏幕寬度，故按照視頻寬度來進(jìn)行適配，下面看下相機(jī)位置設(shè)置：1// 設(shè)置相機(jī)位置（視圖矩陣）
2Matrix.setLookAtM(
3 viewMatrix, 0,
4 0.0f, 0.0f, 5.0f, // 相機(jī)位置
5 0.0f, 0.0f, 0.0f, // 目標(biāo)位置
6 0.0f, 1.0f, 0.0f // 相機(jī)正上方向量
7)
projectionMatrix和viewMatrix合并為vPMatrix:1// 計(jì)算投影和視圖變換
2Matrix.multiplyMM(vPMatrix, 0, projectionMatrix, 0, viewMatrix, 0)
MediaPlayer的onVideoSizeChanged回調(diào)中獲取視頻寬高并初始化矩陣數(shù)據(jù)，下面來看下畫面矯正后的效果：
評論
圖片
表情