How could I use requests in asyncio?

I want to do parallel http request tasks in asyncio, but I find that python-requests would block the event loop of asyncio. I've found aiohttp but it couldn't provide the service of http request using a http proxy.

So I want to know if there's a way to do asynchronous http requests with the help of asyncio.

To use requests (or any other blocking libraries) with asyncio, you can use BaseEventLoop.run_in_executor to run a function in another thread and yield from it to get the result. For example:

    import asyncio
    import requests

    @asyncio.coroutine
    def main():
        loop = asyncio.get_event_loop()
        future1 = loop.run_in_executor(None, requests.get, 'http://www.google.com')
        future2 = loop.run_in_executor(None, requests.get, 'http://www.google.co.uk')
        response1 = yield from future1
        response2 = yield from future2
        print(response1.text)
        print(response2.text)

    loop = asyncio.get_event_loop()
    loop.run_until_complete(main())

This will get both responses in parallel.

With python 3.5 you can use the new await/async syntax:

    import asyncio
    import requests

    async def main():
        loop = asyncio.get_event_loop()
        future1 = loop.run_in_executor(None, requests.get, 'http://www.google.com')
        future2 = loop.run_in_executor(None, requests.get, 'http://www.google.co.uk')
        response1 = await future1
        response2 = await future2
        print(response1.text)
        print(response2.text)

    loop = asyncio.get_event_loop()
    loop.run_until_complete(main())

See PEP0492 for more.

From: stackoverflow.com/q/22190403