還在用requests寫爬蟲嗎?這個(gè)庫效率提高一倍!
↑ 關(guān)注 + 星標(biāo) ,每天學(xué)Python新技能
后臺回復(fù)【大禮包】送你Python自學(xué)大禮包
什么是協(xié)程?
協(xié)程比多線程好在哪呢?
協(xié)程的適用 & 不適用場景
初探異步 http 框架 httpx
什么是 httpx
安裝
pip install httpx
最佳實(shí)踐
import asyncioimport httpximport threadingimport timedef sync_main(url, sign):response = httpx.get(url).status_codeprint(f'sync_main: {threading.current_thread()}: {sign}2 + 1{response}')sync_start = time.time()[sync_main(url='http://www.baidu.com', sign=i) for i in range(200)]sync_end = time.time()print(sync_end - sync_start)
sync_main: <_MainThread(MainThread, started 4471512512)>: 192: 200sync_main: <_MainThread(MainThread, started 4471512512)>: 193: 200sync_main: <_MainThread(MainThread, started 4471512512)>: 194: 200sync_main: <_MainThread(MainThread, started 4471512512)>: 195: 200sync_main: <_MainThread(MainThread, started 4471512512)>: 196: 200sync_main: <_MainThread(MainThread, started 4471512512)>: 197: 200sync_main: <_MainThread(MainThread, started 4471512512)>: 198: 200sync_main: <_MainThread(MainThread, started 4471512512)>: 199: 20016.56578803062439
import asyncioimport httpximport threadingimport timeclient = httpx.AsyncClient()async def async_main(url, sign):response = await client.get(url)status_code = response.status_codeprint(f'async_main: {threading.current_thread()}: {sign}:{status_code}')loop = asyncio.get_event_loop()tasks = [async_main(url='http://www.baidu.com', sign=i) for i in range(200)]async_start = time.time()loop.run_until_complete(asyncio.wait(tasks))async_end = time.time()loop.close()print(async_end - async_start)
async_main: <_MainThread(MainThread, started 4471512512)>: 56: 200async_main: <_MainThread(MainThread, started 4471512512)>: 99: 200async_main: <_MainThread(MainThread, started 4471512512)>: 67: 200async_main: <_MainThread(MainThread, started 4471512512)>: 93: 200async_main: <_MainThread(MainThread, started 4471512512)>: 125: 200async_main: <_MainThread(MainThread, started 4471512512)>: 193: 200async_main: <_MainThread(MainThread, started 4471512512)>: 100: 2004.518340110778809
如果你覺得本文對你有所幫助,給我來個(gè)點(diǎn)贊、在看呀!
評論
圖片
表情


