Engineering
Mar 28, 2024
Engineering
2024 GTC Event Live Rankings: How to Utilize GraphQL Subscription
Sujin Kim
Software Engineer
Mar 28, 2024
Engineering
2024 GTC Event Live Rankings: How to Utilize GraphQL Subscription
Sujin Kim
Software Engineer
Lablup commemorated the 2024 GTC event by hosting a special event. Participants created images similar to the given image using the LLM model provided by Lablup, and among those who scored high, an NVIDIA RTX 4090 graphics card was awarded through lottery. 🫢
In this post, we aim to highlight the subscription feature of GraphQL, which was used in the leaderboard page of the event, allowing participants to monitor their scores in real time.
GTC24 event page
What is a Subscription?
It is a mechanism that allows the client to query data in response to a server side event stream. In cases where data changes in real time, for example when implementing real-time logs or chat applications, updates can be immediately reflected when pushed from the server.
Subscription sends data only when the required information changes on the server. Therefore, in the case where data changes are not frequent, Subscription can reduce data traffic, which can also lead to cost savings.
A similar concept is setting the fetchPolicy
of GraphQL's Query to network-only to always get the latest results, but it’s different from the features of subscriptions. This ensures the latest data by always requesting the server whenever the client needs data. However, network costs accompany each request. Thus, while it is okay to set fetchPolicy to network-only to guarantee the latest results whenever a button is clicked, if it is used to retrieve data where updates are frequent like a stock trading window, network costs would be significant.
How to Use
Defining Subscription
The usage is similar to Query, just use the keyword subscription
.
const leaderboardSubscriptions = graphql` subscription Ranking_leaderboardSubscription { leaderboard { submissions { id name score imageUrl } lastUpdatedAt } } `;
When an event occurs in the leaderboard
stream, a notification is sent to the application, and the client can get the updated result.
Then the following result can be obtained.
leaderboard: {
submissions: [
{
"id": "76293167-e369-4610-b7ac-4c0f6aa8f699",
"name": "test",
"score": 0.5910864472389221,
"imageUrl": "<IMAGE_URL>"
},
],
lastUpdatedAt: 1710176566.493705
}
subscribe
To display real-time rankings, when entering the relevant page, call subscribe, and when moving to other pages, call dispose to unsubscribe using useEffect
.
import { useEffect } from 'react';
import { requestSubscription } from 'react-relay';
useEffect(() => {
const subscriptionConfig = {
subscription: leaderboardSubscriptions,
variables: {},
onNext: (response: any) => {
setLeaderboard(response.leaderboard.submissions); // 미리 정의된 state
},
onError: (error: any) => {
console.error('Leaderboard subscription error', error);
},
};
const { dispose } = requestSubscription(
RelayEnvironment, // refer 'How to Configure' below
subscriptionConfig,
);
return () => {
dispose();
};
}, []); // Executing this part only when the component is mounted or unmounted by setting an empty dependency array
requestSubscription
- Provides a
Disposable
object as a return value. - This
Disposable
object includes a `dispose method to cancel the subscription.
onNext
- As data is updated through subscription, it updates the pre-defined state to display real-time rankings.
- In addition to
onNext
,onError
, there are various configurations such as onCompleted called when the subscription ends andupdater
to update the in-memory relay storage based on server response. For detailed descriptions, refer to this link.
dispose
- A cleanup function is returned in the
useEffect
hook and the dispose method is called to end the subscription when the component is unmounted.
How to set up (+Relay)
According to the Relay documentation, GraphQL subscriptions communicate with WebSockets, and you can set up a network using graphql-ws. (There is also a way to use subscriptions-transport-ws, but it's deprecated, so we'll pass on that).
import { ExecutionResult, Sink, createClient } from 'graphql-ws';
import {
Environment,
Network,
RecordSource,
Store,
SubscribeFunction,
RelayFeatureFlags,
FetchFunction,
Observable,
GraphQLResponse,
} from 'relay-runtime';
import { RelayObservable } from 'relay-runtime/lib/network/RelayObservable';
import { createClient } from 'graphql-ws';
const wsClient = createClient({
url: GRAPHQL_SUBSCRIPTION_ENDPOINT,
connectionParams: () => {
return {
mode: 'cors',
credentials: 'include',
};
},
});
const subscribeFn: SubscribeFunction = (operation, variables) => {
return Observable.create((sink: Sink<ExecutionResult<GraphQLResponse>>) => {
if (!operation.text) {
return sink.error(new Error('Operation text cannot be empty'));
}
return wsClient.subscribe(
{
operationName: operation.name,
query: operation.text,
variables,
},
sink,
);
}) as RelayObservable<GraphQLResponse>;
};
// Export a singleton instance of Relay Environment
// configured with our network function:
export const createRelayEnvironment = () => {
return new Environment({
network: Network.create(fetchFn, subscribeFn),
store: new Store(new RecordSource()),
});
};
export const RelayEnvironment = createRelayEnvironment();
wsClient
- For url, enter the websocket URL of the GraphQL server.
- credentials can be set via
connectionParams
.
subscribeFn
- Defines the subscription behavior of the Observable.
- Validate the query string in
if (!operation.text) { ... }
and if it is invalid, raise an error and abort the execution. - Finally, the
return wsClient.subscribe( ... )
code actually subscribes to the subscription using the WebSocket client and passes the payload of the GraphQL operation to the sink (i.e., the Observer). - In short, this function is responsible for handling the GraphQL subscription request and pushing the result to the Observable stream whenever a subscription event occurs.
createRelayEnvironment
- Create and return a new Relay Environment.
- A Relay environment is a container that manages other high-level Relay objects, network layer, cache, etc.
- We have assigned functions to
fetchFn
to handle GraphQL query/mutation requests andsubscribeFn
to handle subscription requests. - To create a Relay Store to store and manage cache data, we used the
RecordSource
store.
RelayEnvironment
- The
createRelayEnvironment
function is called to initialize the RelayEnvironment and export it for later import and use elsewhere. - This configured
RelayEnvironment
is mainly used byQueryRenderer
,useLazyLoadQuery
,commitMutation
, etc.
CORS error
Initially, I read the config.toml
file used on the server side to set the websocket URL of the GraphQL server and set the address. However, I kept getting CORS errors and Unauthorized every time I sent a request. So I did a lot of shoveling around, and with the help of my colleague, I was able to solve it. (Thank you so much 🥹🙏)
The solution is to use http-proxy-middleware
to set up setupProxy
!
As you can see in the create-react-app manual, you can set up a setupProxy
to proxy requests from your development server to a specific path on your real server, usually to prevent CORS issues in development environments where the frontend and backend are separated, or to proxy requests from your development server to a specific path on your real server.
The code looks like this
const { createProxyMiddleware } = require('http-proxy-middleware');
module.exports = function (app) {
app.use(
createProxyMiddleware('/graphql', {
target: 'http://127.0.0.1:9220',
changeOrigin: true,
followRedirects: true,
ws: true,
}),
);
};
createProxyMiddleware('/graphql', { ... })
- Sets the middleware to handle all HTTP requests originating from '/graphql'.
target: 'http://127.0.0.1:9220'
- Set the address of the server to which proxied requests will be forwarded. Here we set it to port 9220.
changeOrigin: true
- Change the host header of the request to the host of the target. Use this to work around CORS issues.
followRedirects: true
- This setting causes the proxy to follow redirects when the server sends a redirect response to a request.
ws: true
- This setting enables the WebSocket proxy. The websocket connection between the client and server is also passed through this proxy, which we set to
true
for subscribe.
Leaderboard page
After a lot of digging, we've finally finished the leaderboard page! 🎉 A big thank you to everyone who participated. 🙇🏻♀️
Conclusion
Using GraphQL subscriptions, we were able to implement features like real-time rankings. Although I struggled with how to set it up because of CORS, it was not difficult to use because it is not much different from writing a query.
I think the biggest advantages of subscriptions are real-time updates and efficiency. Because it receives data from the server in real time, users always see the latest status, and because it only gets updates when the data it needs changes, it can minimize server requests for data that doesn't change often.
However, it is complex as it requires an implementation of websockets or similar real-time protocols, as well as logic to manage the connection state between the client and server. Although not covered in this article, subscription requires additional work on the server side. And because it requires a real-time connection, it can consume server resources and client resources.
Therefore, which method is more cost or performance efficient depends on many factors, including the nature of your application, the frequency of data updates, and the number of concurrent users, so use your best judgment.
references
- https://relay.dev/docs/v10.1.3/subscriptions/
- https://relay.dev/docs/guided-tour/updating-data/graphql-subscriptions/#configuring-the-network-layer
- https://developer.mozilla.org/en-US/docs/Web/API/WebSockets_API
- https://github.com/enisdenjo/graphql-ws
- https://github.com/apollographql/subscriptions-transport-ws
- https://graphql.org/blog/subscriptions-in-graphql-and-relay
- https://create-react-app.dev/docs/proxying-api-requests-in-development
This post is automatically translated from Korean