backend.ai

Hi, I'm Gyubong Lee, a software engineer at Lablup.

I've been working on maintaining Backend.AI core and working on the necessary tasks to bring Raft to our project.

I'd like to share my thoughts and impressions from OSSCA 2024.

What is Raftify?

Raftify is a framework that simplifies the application of the Raft algorithm to quickly build stable and reliable environments in distributed systems. It allows developers to easily handle core mechanisms such as leader election, log replication, and failover without the need for complex implementations.

The Raft algorithm is a consensus algorithm designed to maintain the consistency of data in distributed systems. While its predecessor, Paxos, had the disadvantage of being complex in structure and difficult to implement, Raft was designed to be easy to understand and simple to implement. These characteristics have made it widely used to ensure reliable behavior in a variety of fields, including distributed databases.

💡 If you're curious about Raftify, please read this article.

Reasons to participate in OSSCA

I've been working on Raftify as part of my work to bring a Raft-based high availability (HA) structure to Backend.AI. I wanted to share the various issues and concerns I've encountered along the way with others so that we can work together to find solutions and make Raftify a better project.

I became involved as a Lead Mentor because I saw OSSCA as an opportunity to not only realize these goals, but also provide a platform for growth for my mentees.

How OSSCA program works

As the lead mentor, my goal was to help mentees learn distributed systems theory, debug real-world implementations, reflect on what they learned by comparing and contrasting theory and implementation, and complete at least one contribution over the course of 13 weeks. This allowed mentees to experience real-world contributions, and helped them achieve personal growth and achievement.

To lower the barrier to entry for the project, we split the project into two phases: the OSSCA Challenge period was used to focus on background learning and study, and the actual contribution was done during the OSSCA Master period.

OSSCA Challenge - Study period

The first thing I did was study the Raft paper to get some background.

For mentees who are new to Raft, it's not easy to read and understand the papers straight away, so we had 'Mogakko' activity (Get together and code individually) to study together and ask questions about the parts that we didn't understand.

Since many of the participants were unable to join us online due to the distance between their residence and study location, we shared the lecture recordings on YouTube.

During the challenge, I asked mentees to summarize what they learned in their own words and organize it by keyword in markdown so that they could familiarize themselves with what they learned.

We used Discord to conduct Q&As online as well as in person, and we shared the questions and answers from the in-person meetings on Discord so that mentees who only participate online could follow along and participate.

After the basics, we moved on to learning the code: we had one big overview of the raft-rs structure and related terminology, and then we dived into the Raftify and raft-rs source code to learn the details together. Many of the mentees had a hard time when they first started, but they quickly absorbed what we were teaching them, and thanks to the mentees who followed along, we were able to prepare the material every week.

I think we were able to operate more efficiently during Masters period because we focused on contribution, as we spent time going through the theory and studying the actual code before I started contributing.

Everyone actively participated in the study and project and made quality contributions along the way, whether it was writing multiple PRs or reviewing code.

OSSCA Masters - Code Contributions

During the masters period, I assigned good first issues to help mentees get into the swing of contributing, and we had 1:1 coffee chats to help mentees pick one or more issues to contribute in areas they were interested in. For example, adding a Raft storage implementation, writing test cases, adding exception handling logic, etc.

We helped them choose a variety of issues that fit their interests, whether they wanted to add a simple feature to the CI or CLI, delve deeper into the algorithm implementation code they learned during their study period, or tackle a more difficult issue with further study. In total, more than 20 PRs were successfully merged.

Introduction to contributions

Throughout this section, we will showcase some of our favorite contributions.

(#118) Refactoring leader election integration test code

This contribution was made by a mentee who was very interested in implementing and testing the Raft algorithm.

Refactoring the leader election integration test code to allow testing of leader election on arbitrarily sized clusters has been improved, and the overall quality of the code has been greatly improved.

(#124) RocksDB Added support for log storage

Originally, Raftify was developed to use LMDB as log storage. However, from the beginning of the Contribution Academy program, we had a mentee who wanted to implement a new log storage based on RocksDB, and after a long period of learning and hard work, we were able to add that functionality.

This gives Raftify the flexibility to pick and choose from multiple storage implementations based on different requirements.

(#130) Add a 3-node cluster example docker-compose file

Added Dockerfile and docker-compose related configuration files so that you can spin up and test your Raftify cluster without having to build a separate development environment.

This made it much simpler to build and test a consistent development environment. You've also since followed up with follow-up PR on reducing image sizes with multi-stage builds.

Wrapping up

This is the first time I've participated in the Open Source Contribution Academy as a mentor. I was worried about being a mentor for the first time, but the mentees were very enthusiastic, and when I worked with them, I realized that having a group of people with the same goal was more efficient than I thought, and it was fun and rewarding.

Learning about complex topics in piecemeal fashion can often be confusing or difficult to understand, but it's only when you get to use or create an implementation that things become clear. By analyzing, using, and contributing to implementations, mentees in OSSCA are able to naturally connect theory to practice. In this sense, we believe that OSSCA can be a valuable opportunity for those who want to properly study a complex topic like distributed systems.

Raftify offers an intuitive and simple interface compared to other Raft libraries, and abstracts complex logic that requires background knowledge into a black box, giving beginners a great starting point to understand and build distributed systems. What's more, the Python bindings allow for hands-on practice with Python, which is a low barrier to entry, and the well-organized examples and test code help learners clarify concepts through hands-on practice.

If you're interested in decentralized systems, check out Raftify!