backend.ai

Hello, I'm Jinho Heo, a Technical Writer at Lablup.

Lablup participated in PyCon Korea 2024 as a platinum sponsor from October 26th to October 27th, at the Suwon Convention Center.

Lablup's founding philosophy is strongly tied to open source. It's not a stretch to say that open source is in Lablup's blood. There are a variety of open sources, but Lablup has a particularly strong relationship with Python. Lablup is an active contributor to open source, such as aiohttp, which was developed using asyncio, and also contributes to Python itself. Our deep ties go beyond Python to PyCon. Members of Lablup have delivered presentations on diverse topics at PyCons globally and have been a sponsor partner of PyCon Korea five times.

This year's PyCon was a big one for Lablup. This is because Lablup's CEO Jeonkyu Shin and CTO Joongi Kim were keynote speakers on both days of PyCon. Jeongkyu Shin gave a talk on 'Python, PyCon, the Dinosaur Age, and the Planet of Chickens', which is a confusing title at first glance, but he likened two qualities of Python to dinosaurs, the first evolutionary victors: Python's rapid growth and ascension to the number one language, and its status as a key language in the era of massive AI: which creates innovations with massive computation. This leads to 'chicken', the most consumed meat in the modern world, which he said of its accessibility and universality, making it easy for anyone to learn and utilize.Shin's presentation was well-received by the audience, as he entertained the audience with his witty titles, verbal skills, and various AI-generated illustrations.

Joongi Kim, the CTO, delivered a presentation entitled "10 Years of PyCon and Me." Spanning four chapters, he reflected on his PyCon presentations worldwide, beginning with asyncio's development, the company's expansion, managing code scaling, and motivating fellow Python enthusiasts. His clear explanation of the challenges developers encounter or may encounter garnered considerable applause from the audience.

Kyujin Cho, Senior Software Engineer | Sergey Leksikov, Researcher | Joongi Kim, Chief Technology Officer

In addition, Kyujin Cho, Senior Software Engineer, Sergey Leksikov, Researcher, and Joongi Kim, CTO, each presented a session. Kyujin presented “Automated Python Web Framework API Schema Creator: The long way around', where he shared with the audience the series of shovelfuls he went through to overcome the challenges of automatically generating API documentation using aiohttp. Sergey presented 'Automating CLI commands execution with LLM and LangGraph: A new frontier in Python automation', where he talked about how complex CLIs can be transformed into user-friendly tools using LLM and LangGraph frameworks to improve user experience and operational efficiency. Both talks were well attended and received great interest from the audience. Joongi presented 'Engineering Python for enterprise delivery', where he shared his experience in developing packaging/installers for Python app delivery.

👨‍🏫 Jinho: We'd love to hear your PyCon session recaps.

👨‍💻 Sergey: Although my presentation was in English, the PyCon Korea organizers offered a real-time translation service, enabling me to deliver my talk smoothly. The highlight of the session was the interaction with eager audience members who approached me with questions about my presentation topic afterwards.

'AI Score Reader' event (Drawing a duck and swan)

Lablup organized an 'AI Score Reader' event at PyCon Korea 2024. Attendees could join the event by scanning a QR code on their mobile devices, tablets, or laptops and were invited to draw "ducks and swans" for a chance to win prizes. Every participant had the opportunity to receive a Lablup Folding Pouch, while the grand prize for the best drawing was a Starbucks gift card.

The stickers Lablup carries to every exhibition

Why ducks and why swans? They are the ones in stickers we always bring to exhibitions. While they appear to glide smoothly across the water, they are paddling hard beneath the surface. This imagery serves as a metaphor for our desire to have our customers enjoy seamless AI services on the front end while Backend.AI handles the complex processes in the background.

The concept of the event is straightforward. If you've ever used generative AI tools such as Stable Diffusion or Dall-E for image creation, you're aware that the structure of generative AI is highly sensitive to its input. The output may vary with each attempt, even with identical inputs, or change drastically with minor input modifications. This method of instructing generative AI to yield specific results is known as 'prompt engineering.' All participants at our booth had the opportunity to engage in some prompt engineering themselves.

We asked Kyujin Cho, a senior software engineer who was responsible for developing the backend of our “AI Score Reader” event, to give us an overview.

👨‍🏫 Jinho: Can you explain how the “AI Picture Reader” event page was created?

👨‍💻 Kyujin: The "AI Score Reader" event page's backend was built on three microservices: the Web Application Server (WAS), the image generation pipeline, and the image similarity pipeline. The WAS includes a user database and an API to manage image generation requests. These microservices were all deployed on Backend.AI using the 'Model Service' feature. Similar to the Visutale demo by Sergey from our research team, this serves as a testament to the versatility of Backend.AI in developing AI services.

👨‍🏫 Jinho: Specifically, what process leads to the similarity between a user's drawing and a given image?

👨‍💻 Kyujin: When a user submits a prompt through a page accessed by scanning a QR code to generate an image, the Web Application Server (WAS) forwards the text to an image generation service and retrieves the created image. The WAS then sends this image to the similarity discriminator, receives a similarity score as a percentage, and delivers both the score and the generated image to the user.

👨‍🏫 Jinho: What criteria should we draw by to get a high score?

👨‍💻 Kyujin: I have no idea what the image similarity pipeline is determining similarity. Perhaps asking the AI directly would be a better way to get an answer.

'Massive' engagement

Our members perceived it as a "minor event." However, it took less than an hour for this perception to be completely overturned.

Our booth started to fill up, and the front of the booth became a scene of people staring hungrily at the leaderboard, eager to see where they stood and defend their top placement.

There were even reports of people sitting in lounges staring at their phone to draw swans.

Let's take a look at the stats.

On the 26th and 27th of October, the event saw the participation of 428 individuals. Throughout this period, we received 11,639 image creation requests, with the most surprising contribution from a single participant who submitted over 1,000 images, a detail we'll delve into later.

Unintended Side Effects Raised

Seated in the booth and observing the leaderboard, I noticed two familiar nicknames: "cloudshin" and "achimnol," which belong to CEO Shin Jeongkyu and CTO Kim Joongi, respectively. They were on a streak, achieving similarity scores above 90%. They kept drawing ducks non-stop, even while proceeding to lunch.

DevRel Lead WooYoung Yoo attempted to stop them, yet it is said that their enthusiasm was difficult to diminish. (P.S.1: Of course, we did forcefully erase their data when finalizing the scores).
(P.S.2: Indeed, the participants performed so well that they effortlessly exceeded the scores of the latter two, resulting in a dominant podium presence...)

In addition to the internal side effects, there were also external issues. While running a booth on Day 1, adeveloper screamed in the distance.

"I believe someone is executing a macro."

We discovered that duplicate submission requests were being sent to the same prompt every 1-5 seconds, due to a macro exploiting the AI's generative capabilities to yield varied results with identical prompts. However, addressing the issue was challenging at the moment of detection, because the backend developer was slated to present at PyCon on the second day. The situation was deteriorating.

“I can't submit” ‘The button doesn't work’ ”I get a gray blank space instead of an image”

An attendee opened our event website on their laptop, connected the developer tools, and pointed out that no requests were coming through. We found a developer tucked away in a corner preparing to present at a stage, and discovered that the earlier macro was bombarding our NVIDIA H100 with concurring requests.

Day 1 concluded with a trickle of image submissions. While tidying up the booth and reflecting on the day, we recognized the need to avert this issue on Day 2.

Poor developer pulled an all-nighter to prepare for Day 2, putting off his presentation and adding a couple new features to prevent the accident.

First, added same-prompt submission protection

We added the ability to validate submitted prompts on the server side. If a prompt has been submitted previously, the response has been altered to provide the initially generated image and its similarity score rather than creating a new image.

Second, added captcha when submitting images

To deter macros, we implemented a captcha for submitting responses. Although this may have been slightly inconvenient for participants, the introduction of the captcha successfully prevented random macros on the second day of the event.

Other minor improvements We were receiving data with quite a few decimal places to calculate participants' scores, but we were truncating them in the GUI to the first decimal place for user convenience. However, as the competition heated up, people started to show up with identical scores to the first decimal place, so we patched the GUI to show the second decimal place to reduce user confusion.

Thanks to Kyujin Cho, Senior Engineer, and Soojin Kim, Frontend Engineer, who worked tirelessly on the backend to make the event a success 🙂 .

Wrapping up PyCon Korea 2024

Numerous Python enthusiasts attended PyCon Korea 2024, where Lablup also engaged with many attendees. Lablup is committed to ongoing contributions and growth within the open source community. Echoing Chef Jeong Ji-sun's words from the hit show “Culinary Class War,” “Opening up recipes leads to more recipes. With many contributing ideas, it creates a larger tapestry.”

Here at Lablup, we will continue to think big and never lose sight of the open source spirit that has always been the cornerstone of our foundation.