Performance Analysis: Goroutine & Python’s Coroutine

I made 1000 HTTP requests using Goroutines and Python’s Coroutines

Used Go 1.6.2 and Python 3.6
Implemented in Go using net/http package
Implemented in Python using aiohttp, requests and urllib3 libraries
Ran it over $10 DigitalOcean droplet

Scroll to bottom of this post to see results.

Go

Go implementation using goroutines:

package main

import (
    "fmt"
    "net/http"
)


func main() {
    url := "http://example.com"
    resc, errc := make(chan bool), make(chan error)

    for i := 0; i < 1000; i++ {
        go func(url string) {
            _, err := http.Head(url)
            if err != nil {
                errc <- err
                return
            }
            resc <- true
        }(url)
    }
    for i := 0; i < 1000; i++ {
        select {
        case <-resc:
        case err := <-errc:
            fmt.Println(err)
        }
    }
}

Python

Python implementation using asyncio with aiohttp, requests and urllib3

1. aiohttp

import asyncio

import aiohttp


async def main():
    async with aiohttp.ClientSession() as session:
        for i in range(1000):
            await session.head('http://example.com')


if __name__ == '__main__':
    loop = asyncio.get_event_loop()
    loop.run_until_complete(main())

2. requests

import asyncio

import requests


async def main():
    futures = [
        loop.run_in_executor(
            None,
            requests.get,
            'http://example.com'
        )
        for i in range(1000)
    ]
    return await asyncio.gather(*futures)


if __name__ == '__main__':
    loop = asyncio.get_event_loop()
    _ = loop.run_until_complete(main())

3. urllib3

import asyncio
import urllib3


async def main(http):
    futures = [
        loop.run_in_executor(
            None,
            http.request,
            'HEAD', 'http://example.com'
        )
        for i in range(1000)
    ]
    return await asyncio.gather(*futures)


if __name__ == '__main__':
    loop = asyncio.get_event_loop()
    http = urllib3.PoolManager()
    _ = loop.run_until_complete(main(http))

Analysis (Time)

1. Sync/Async HEAD requests (time in seconds)

2. Sync/Async GET requests (time in seconds)

Analysis (Memory)

1. Sync/Async HEAD requests (memory in MB)

2. Async/Sync GET requests (memory in MB)

Code for both synchronous and asynchronous implementations are at: https://github.com/vipul-sharma20/async-go-py

Vipul Sharma

Go