backend.ai

Lablup commemorated the 2024 GTC event by hosting a special event. Participants created images similar to the given image using the LLM model provided by Lablup, and among those who scored high, an NVIDIA RTX 4090 graphics card was awarded through lottery. 🫢
In this post, we aim to highlight the subscription feature of GraphQL, which was used in the leaderboard page of the event, allowing participants to monitor their scores in real time.

GTC24 event page

What is a Subscription?

It is a mechanism that allows the client to query data in response to a server side event stream. In cases where data changes in real time, for example when implementing real-time logs or chat applications, updates can be immediately reflected when pushed from the server.

Subscription sends data only when the required information changes on the server. Therefore, in the case where data changes are not frequent, Subscription can reduce data traffic, which can also lead to cost savings.

A similar concept is setting the fetchPolicy of GraphQL's Query to network-only to always get the latest results, but it’s different from the features of subscriptions. This ensures the latest data by always requesting the server whenever the client needs data. However, network costs accompany each request. Thus, while it is okay to set fetchPolicy to network-only to guarantee the latest results whenever a button is clicked, if it is used to retrieve data where updates are frequent like a stock trading window, network costs would be significant.

How to Use

Defining Subscription

The usage is similar to Query, just use the keyword subscription.

  const leaderboardSubscriptions = graphql`
    subscription Ranking_leaderboardSubscription {
      leaderboard {
        submissions {
          id
          name
          score
          imageUrl
        }
        lastUpdatedAt
      }
    }
  `;

When an event occurs in the leaderboard stream, a notification is sent to the application, and the client can get the updated result.

Then the following result can be obtained.

leaderboard: {
	submissions: [
		{
    	"id": "76293167-e369-4610-b7ac-4c0f6aa8f699",
	    "name": "test",
    	"score": 0.5910864472389221,
	    "imageUrl": "<IMAGE_URL>"
		},
    ],
	lastUpdatedAt: 1710176566.493705
}

subscribe

To display real-time rankings, when entering the relevant page, call subscribe, and when moving to other pages, call dispose to unsubscribe using useEffect.

import { useEffect } from 'react';
import { requestSubscription } from 'react-relay';

useEffect(() => {
  const subscriptionConfig = {
    subscription: leaderboardSubscriptions,
    variables: {},
    onNext: (response: any) => {
      setLeaderboard(response.leaderboard.submissions); // 미리 정의된 state
    },
    onError: (error: any) => {
      console.error('Leaderboard subscription error', error);
    },
  };
  const { dispose } = requestSubscription(
    RelayEnvironment, // refer 'How to Configure' below
    subscriptionConfig,
  );
  return () => {
    dispose();
  };
}, []); //  Executing this part only when the component is mounted or unmounted by setting an empty dependency array

requestSubscription

• Provides a Disposable object as a return value.
• This Disposable object includes a `dispose method to cancel the subscription.

onNext

• As data is updated through subscription, it updates the pre-defined state to display real-time rankings.
• In addition to onNext, onError, there are various configurations such as onCompleted called when the subscription ends and updater to update the in-memory relay storage based on server response. For detailed descriptions, refer to this link.

dispose

• A cleanup function is returned in the useEffect hook and the dispose method is called to end the subscription when the component is unmounted.

How to set up (+Relay)

According to the Relay documentation, GraphQL subscriptions communicate with WebSockets, and you can set up a network using graphql-ws. (There is also a way to use subscriptions-transport-ws, but it's deprecated, so we'll pass on that).

import { ExecutionResult, Sink, createClient } from 'graphql-ws';
import {
  Environment,
  Network,
  RecordSource,
  Store,
  SubscribeFunction,
  RelayFeatureFlags,
  FetchFunction,
  Observable,
  GraphQLResponse,
} from 'relay-runtime';
import { RelayObservable } from 'relay-runtime/lib/network/RelayObservable';
import { createClient } from 'graphql-ws';

const wsClient = createClient({
  url: GRAPHQL_SUBSCRIPTION_ENDPOINT,
  connectionParams: () => {
    return {
      mode: 'cors',
      credentials: 'include',
    };
  },
});

const subscribeFn: SubscribeFunction = (operation, variables) => {
  return Observable.create((sink: Sink<ExecutionResult<GraphQLResponse>>) => {
    if (!operation.text) {
      return sink.error(new Error('Operation text cannot be empty'));
    }
    return wsClient.subscribe(
      {
        operationName: operation.name,
        query: operation.text,
        variables,
      },
      sink,
    );
  }) as RelayObservable<GraphQLResponse>;
};

// Export a singleton instance of Relay Environment
// configured with our network function:
export const createRelayEnvironment = () => {
  return new Environment({
    network: Network.create(fetchFn, subscribeFn),
    store: new Store(new RecordSource()),
  });
};

export const RelayEnvironment = createRelayEnvironment();

wsClient

• For url, enter the websocket URL of the GraphQL server.
• credentials can be set via connectionParams.

subscribeFn

• Defines the subscription behavior of the Observable.
• Validate the query string in if (!operation.text) { ... } and if it is invalid, raise an error and abort the execution.
• Finally, the return wsClient.subscribe( ... ) code actually subscribes to the subscription using the WebSocket client and passes the payload of the GraphQL operation to the sink (i.e., the Observer).
• In short, this function is responsible for handling the GraphQL subscription request and pushing the result to the Observable stream whenever a subscription event occurs.

createRelayEnvironment

• Create and return a new Relay Environment.
• A Relay environment is a container that manages other high-level Relay objects, network layer, cache, etc.
• We have assigned functions to fetchFn to handle GraphQL query/mutation requests and subscribeFn to handle subscription requests.
• To create a Relay Store to store and manage cache data, we used the RecordSource store.

RelayEnvironment

• The createRelayEnvironment function is called to initialize the RelayEnvironment and export it for later import and use elsewhere.
• This configured RelayEnvironment is mainly used by QueryRenderer, useLazyLoadQuery, commitMutation, etc.

CORS error

Initially, I read the config.toml file used on the server side to set the websocket URL of the GraphQL server and set the address. However, I kept getting CORS errors and Unauthorized every time I sent a request. So I did a lot of shoveling around, and with the help of my colleague, I was able to solve it. (Thank you so much 🥹🙏)

The solution is to use http-proxy-middleware to set up setupProxy!

As you can see in the create-react-app manual, you can set up a setupProxy to proxy requests from your development server to a specific path on your real server, usually to prevent CORS issues in development environments where the frontend and backend are separated, or to proxy requests from your development server to a specific path on your real server.

The code looks like this

const { createProxyMiddleware } = require('http-proxy-middleware');

module.exports = function (app) {
  app.use(
    createProxyMiddleware('/graphql', {
      target: 'http://127.0.0.1:9220',
      changeOrigin: true,
      followRedirects: true,
      ws: true,
    }),
  );
};

createProxyMiddleware('/graphql', { ... })

• Sets the middleware to handle all HTTP requests originating from '/graphql'.

target: 'http://127.0.0.1:9220'

• Set the address of the server to which proxied requests will be forwarded. Here we set it to port 9220.

changeOrigin: true

• Change the host header of the request to the host of the target. Use this to work around CORS issues.

followRedirects: true

• This setting causes the proxy to follow redirects when the server sends a redirect response to a request.

ws: true

• This setting enables the WebSocket proxy. The websocket connection between the client and server is also passed through this proxy, which we set to true for subscribe.

Leaderboard page

After a lot of digging, we've finally finished the leaderboard page! 🎉 A big thank you to everyone who participated. 🙇🏻‍♀️

Conclusion

Using GraphQL subscriptions, we were able to implement features like real-time rankings. Although I struggled with how to set it up because of CORS, it was not difficult to use because it is not much different from writing a query.

I think the biggest advantages of subscriptions are real-time updates and efficiency. Because it receives data from the server in real time, users always see the latest status, and because it only gets updates when the data it needs changes, it can minimize server requests for data that doesn't change often.

However, it is complex as it requires an implementation of websockets or similar real-time protocols, as well as logic to manage the connection state between the client and server. Although not covered in this article, subscription requires additional work on the server side. And because it requires a real-time connection, it can consume server resources and client resources.

Therefore, which method is more cost or performance efficient depends on many factors, including the nature of your application, the frequency of data updates, and the number of concurrent users, so use your best judgment.

references

• https://relay.dev/docs/v10.1.3/subscriptions/
• https://relay.dev/docs/guided-tour/updating-data/graphql-subscriptions/#configuring-the-network-layer
• https://developer.mozilla.org/en-US/docs/Web/API/WebSockets_API
• https://github.com/enisdenjo/graphql-ws
• https://github.com/apollographql/subscriptions-transport-ws
• https://graphql.org/blog/subscriptions-in-graphql-and-relay
• https://create-react-app.dev/docs/proxying-api-requests-in-development

This post is automatically translated from Korean