23.09: September 2023 update

In the second half of 2023, we released 23.09, a major release of Backend.AI. In 23.09, we've significantly enhanced the development, fine-tuning, and operational automation of generative AI. We've automatically scaled and load-balanced AI models based on workload, expanded support for various GPUs/NPUs, and increased stability when managing a single node as well as 100-2000+ nodes. The team is working hard to squeeze every last bit out of it. Here are the main improvements since the last [23.03 July update] (/posts/2023/07/31/Backend.AI-23.03-update).

Backend.AI Core & UI

• The Backend.AI Model Service feature has been officially released. You can now use Backend.AI to more efficiently prepare environments for inference services as well as training of large models such as LLM. For more information, see the blog Backend.AI Model Service sneak peek.
• Added the ability to sign in to Backend.AI using OpenID single sign-on (SSO).
• If your kernel image supports it, you can enable the sudo command without a password in your compute session.

• Support for Redis Sentinel without HAProxy. To test this, we added the --configure-ha setting to the install-dev.sh file.
• Added the ability to use the RPC channel between Backend.AI Manager and Agent for authenticated and encrypted communication.
• Improved the CLI logging feature of Backend.AI Manager.
• Fixed an issue where Manager could not make an RPC connection when Backend.AI Agent was placed under a NAT environment.
• The Raft algorithm library, riteraft-py, will be renamed and developed as raftify.
• Support for the following new storage backends
  • VAST Data
  • KT Cloud NAS (Enterprise only)

Backend.AI FastTrack

• Improved UI for supporting various heterogeneous accelerators.
• Deleting a VFolder now uses an independent unique ID value instead of the storage name.
• Upgraded Django version to 4.2.5 and Node.js version to 20.
• Added pipeline template feature to create pipelines in a preset form.
• If a folder dedicated to a pipeline is deleted, it will be marked as disabled on the FastTrack UI.
• Improved the process of deleting pipelines.
• Added a per-task (session) accessible BACKENDAI_PIPELINE_TASK_ID environment variable.
• Actual execution time per task (session) is now displayed.

Contribution Academy

Especially in the past period, the following code contributions were made by junior developer mentees through the 2023 Open Source Contribution Academy organized by NIPA.

• Created a button to copy an SSH/SFTP connection example to the clipboard.
• Refactored several Lit elements of the existing WebUI to React.
• Wrote various test code.
• Found and fixed environment variable and message errors that were not working properly.

Backend.AI is constantly evolving to provide a more powerful and user-friendly experience while supporting various environments in the AI ecosystem. Stay tuned for more updates!
Make your AI accessible with Backend.AI!

This post is automatically translated from Korean