backend.ai

The most important thing in debugging is observability and reproducibility. Despite the steady improvement of the asyncio standard library, it's still a challenge to see what's going on inside a real production-level, complex asyncio application. Resource issues caused by silently swallowed cancalleation signals or arbitrarily created callbacks and coroutines inside some external code are very hard to debug, especially when you have a mix of 3rd-party libraries and frameworks running over which you have no control. Moreover, these issues tend to only occur in production environments with real workloads, not in development environments.

In this announcement, we present the aiomonitor-ng library, which is an enhancement to the previously released aiomonitor library. While the original library was based on a simple telnet server and REPL to help us see into the asyncio processes currently running, and can even help us in real production debugging, after using it for over a year, we realized what it lacked and took it upon ourselves to add a number of features, including the ability to directly create tasks and trace the cancel-terminate stack chain. I also added a terminal UI with autocomplete for ease of use.

I've been able to leverage aiomonitor and these improvements to aiomonitor-ng to discover and analyze a number of production issues in practice, and I hope you can use this experience to create more reliable asyncio applications.

About Speaker
He is currently the CTO of Lablup ("Lablup"), where he is developing Backend.AI, and has experience analyzing and implementing backend systems of various sizes. Through his open source activities, he has contributed to projects such as Textcube, iPuTTY, CPython, DPDK, pyzmq, aiodocker, and aiohttp.