backend.ai

The advent of personal computers has fundamentally transformed our lives over the past 40 years. Just to name a few, we witnessed the digitization of life by the internet and the smartphones. Now we're on the cusp of a new era, moving beyond the age of PCs to the age of personalized agents (PAs) or artificial intelligence. At the heart of this transformation is the rapid progress of large language models (LLMs) and multimodal AI. It's no longer about generating smart replies from somewhere in the cloud; with the advent of powerful consumer GPUs, it's now possible to run a personalized generative AI at home.

We'll introduce automated methods for running generative AI models on compact form factors like PCs or home servers, and for personalizing them via fine-tuning. We'll show how PA can be more closely integrated into our daily lives. Furthermore, we'll showcase the results obtained through these methods via a live demo, inviting you to contemplate the future of personalized AI with us.

Large language models (LLMs) are the pinnacle of generative AI. While cloud-based LLMs have enabled mass adoption, local on-premises LLMs are garnering attention in favor of personalization, security, and air-gapped setups. Ranging from personal hobbies to professional domains, both open-source foundational LLMs and fine-tuned models are utilized across diverse fields. We'll introduce the technology and use cases for fine-tuning and running LLMs on a small scale, like PC GPUs, to an expansive scale to serve mass users on data centers. We combine resource-saving and model compression techniques like quantization and QLoRA with vLLM and TensorRT-LLM.

 Additionally, we illustrate the scaling-up process of such genAI models by the fine-tuning pipeline with concrete and empirical examples. You'll gain a deep understanding of how to achieve the operation and expansion of personalized LLMs, and inspirations for the possibilities that this opens up.

As React becomes the standard for frontends, how should Python backends keep up? Learn how to support the GraphQL and Relay specifications to collaborate with the frontend.

Jungseok Kang

Jungseok Kang is a software engineer at Rableup. He started programming with Python and has been using it with love ever since.

Key Note : AI and open source: the dynamics of innovation, capital, and contributors
Jeonkyu Shin, CEO, Lablup
https://eostday.kr

#EOST2023 #ETRI #오픈소스테크데이

This talk presents the challenges of tracking and state management of time-consuming tasks in distributed systems implemented in Python, and some abstract ideas for solving them.
There will be code examples, but please keep in mind that this is a lighthearted talk with no detailed Python code implementations, only high-level abstractions.

About Speaker
3rd year Python developer. A junior with a lot of questions about what makes good software. TMI: Likes crossfit, meat, and bread.

The most important thing in debugging is observability and reproducibility. Despite the steady improvement of the asyncio standard library, it's still a challenge to see what's going on inside a real production-level, complex asyncio application. Resource issues caused by silently swallowed cancalleation signals or arbitrarily created callbacks and coroutines inside some external code are very hard to debug, especially when you have a mix of 3rd-party libraries and frameworks running over which you have no control. Moreover, these issues tend to only occur in production environments with real workloads, not in development environments.

In this announcement, we present the aiomonitor-ng library, which is an enhancement to the previously released aiomonitor library. While the original library was based on a simple telnet server and REPL to help us see into the asyncio processes currently running, and can even help us in real production debugging, after using it for over a year, we realized what it lacked and took it upon ourselves to add a number of features, including the ability to directly create tasks and trace the cancel-terminate stack chain. I also added a terminal UI with autocomplete for ease of use.

I've been able to leverage aiomonitor and these improvements to aiomonitor-ng to discover and analyze a number of production issues in practice, and I hope you can use this experience to create more reliable asyncio applications.

About Speaker
He is currently the CTO of Lablup ("Lablup"), where he is developing Backend.AI, and has experience analyzing and implementing backend systems of various sizes. Through his open source activities, he has contributed to projects such as Textcube, iPuTTY, CPython, DPDK, pyzmq, aiodocker, and aiohttp.

This talk will cover the technical details of the Python bindings that I've spent the most time thinking about since joining Lablup, and that I'm still working on.

More specifically, the talk will focus on the technical details of exposing traces and handling exceptions, as well as the challenges of abstracting reference types to overcome the memory management differences between Rust and Python.

Due to the technical details, prior knowledge of Rust or PyO3 may be helpful to understand the presentation, but even if you don't, you can still get a general idea of what to expect.

Gyubong Lee
DevOps / Developer at Lablup. A developer who is interested in various open source activities. Currently, I am working on various issues related to distributed systems at my company. In this talk, I will cover the technical details of the Python binding, which I have spent most of my time thinking about and working on since joining Lablup.

Sokovan is a Python-based container orchestrator that addresses the challenges of running resource-intensive batch workloads in a containerized environment. It offers acceleration-aware, multi-tenant, batch-oriented job scheduling and fully integrates multiple hardware acceleration technologies into various system layers. It consists of two layers of schedulers. The cluster-level scheduler allows users to customize job placement strategies and control the density and priority of workloads. The node-level scheduler optimizes per-container performance by automatically detecting and mapping underlying hardware accelerators to individual containers, improving the performance of AI workloads compared to Slurm and other existing tools. Sokovan has been deployed on a large scale in various industries for a range of GPU workloads, including AI training and services. It helps container-based MLOps platforms unleash the potential of the latest hardware technologies.

Speakers:
Jeongkyu Shin
Joongi Kim

Open Source with Automation

Jeongkyu Shin - EOST2022

Modernizing development workflow for a 7 year old 74K LoC Python project using Pantsbuild.

Joongi Kim - Pycon Japan 2022

Creating a Serverless Jupyter Notebook App for Python Education for Kids, Jeongkyu Shin - PyCon Korea 2022

SQLAlchemy with asyncio, From Core to ORM.

Sang Hun Lee - PyCon Korea 2022

Migrating Large-scale Python Projects to a Monorepo Using Pantsbuild.

Joongi Kim - Pycon Korea 2022

Practical usage of python ecosystem to build the AI model to fight.

Leksikov Sergey - PyCon Korea 2022

Presenting insights on open source utilization and enterprise-scale platform adoption and operation under the theme 'Successful Digital Transformation Found in the Open Source Ecosystem'

- Open Technet Summit 2022 Virtual Conference

Utilization of Artificial Intelligence Technology for Responding to New Infectious Diseases

• 2013 Ph.D. in Physics, POSTECH Nonlinear Statistical Physics (Characterization of anesthesia-induced consciousness transition in thalamo-cortical circuit)
• 2013-2017 Appointed Researcher, Brain Science Institute, KIST
• 2017-2021 Senior Researcher, Lablup Inc.
• 2022-Present Principal Researcher, Lablup Inc.

This talk was given at a TEDx event using the TED conference format but independently organized by a local community. Learn more at https://www.ted.com/tedx

Lablup CTO Joongi Kim participated in the joint webinar below to introduce Backend.AI

MLOps is key to accelerating AI deployments. In this session, you’ll hear interviews with three ISVs and their customers on how, together, they've found success implementing MLOps solutions to accelerate their respective AI deployments. We'll focus on addressing some of the most common deployment challenges that enterprise customers face and how the MLOPs partner ecosystem can help in addressing those challenges.

You’ll hear from Run.AI and Wayve on how they trailblazed the scaling of AI/ML in autonomous vehicles. You’ll also hear how Weights & Biases works with John Deere/Blue River Technology to achieve advancement of AI in agriculture. Finally, you’ll hear how Backend.AI has supported LG Electronics in making smart factories more efficient.

The session will highlight specific use cases and best practices across the MLOPs life cycle. Come learn about NVIDIA MLOPs partners, and how they've deployed with enterprise customers. This session will feature real solution examples that you won't want to miss.

We discuss what aspects to consider and be mindful of in the process of contributing to and managing open source projects, drawing from experiences on both sides: as a company directly creating and operating open source projects, and as an external contributor.

We introduced the release of SQLAlchemy v1.4, which provides basic support for asyncio-based Core API and ORM, and shared our engineering experiences with SQLAlchemy in the Backend.AI project. For reference, this was also presented at PyCon APAC 2021.

We introduce the process of creating deep learning-based functions in Python and attaching them to Excel, allowing various deep learning-based analyses and model training to be run by connecting Excel with Backend.AI. Through the experience of creating Excel functions and several examples, we can create Excel functions that train deep learning models and perform inference using trained models.

We cover the development and complementary relationship between public and private clouds, and share experiences in building container-based AI environments and workload pipelines on private clouds. Additionally, based on various experiences, we introduce the Backend.AI Reservoir project, a storage service for Python, R, and Linux packages that was started to provide a better user experience.

I share various stories experienced as a doctoral student, developer, and CTO of a startup.
We discuss the various elements required to build a developer career and talk about the strengths of developers from a company's perspective.

In this session, Lablup Inc. will present three solutions for achieving optimal performance when combining various GPUs as one AI/high performance computing cluster. Their solutions are based on Backend.AI, an open source container-based resource management platform specialized in AI and high performance computing. They'll include real-world examples that provide AI developers and researchers with an optimal scientific computing environment and massive cost savings.

It's commonly believed that hyperparameter tuning requires a large number of GPUs to get quick, optimal results. It's generally true that higher computation power delivers more accurate results quickly, but to what extent? We'll present our work and empirical results on finding a sweet spot to balance both costs and accuracy by exploiting partitioned GPUs with Backend.AI's container-level GPU virtualization. Our benchmark includes distributed MNIST, CIFAR-10 transfer learning, and TGS salt identification cases using AutoML with network morphism and ENAS tuner with NNI running on Backend.AI's NGC-optimized containers. Attendees will get a tangible guide to deploy their GPU infrastructure capacity in a more cost-effective way.