Performance Testing GraphQL Resolvers

GraphQL, the API specification developed at Facebook, redefines how an API is constructed and used. It’s a powerful technology that is driving many mainstream API providers to replace their REST versions with new releases using GraphQL.

Unlike RESTful APIs, GraphQL supports declarative data structures and recursive queries. These two features alone make using GraphQL very attractive in terms of developer experience. However, using GraphQL comes with a price. There are some “gotchas” in GraphQL that can create unintended performance problems, as Netflix experienced recently. Most of the problems are centered around how to program resolvers.

A resolver is a function that does the actual work of getting the data for a particular query or mutation being executed. A resolver can work with a number of data sources, which makes performance testing a challenge. Performance testing a GraphQL API requires paying particular attention to the GraphQL resolver mechanism.

This article covers the basics of the relationship between a GraphQL query and resolver and provides suggestions for creating performance tests that ensure the resolvers you write are working to maximum efficiency.

A resolver is a function that backs a particular query or mutation within a GraphQL API. Figure 1 shows an example of a GraphQL query run in Apollo Server’s GraphQL Playground. The query shown in the left panel is written in the GraphQL query language. The JSON response from the query is shown in the right panel.

Figure 1: A simple GraphQL query run in the Apollo Server GraphQL Playground IDE

Every query or mutation exposed in a GraphQL API will be backed by a resolver, and a resolver is written in the language used to implement a given GraphQL specification. For example, if you’re running Apollo Server as your GraphQL API engine, as shown above, then the resolver is written in NodeJS. You’ll write your resolvers in C# if you’re using a .NET implementation of the spec. (Remember, GraphQL is a specification that only describes how a GraphQL API works. There are a variety of implementations of the spec available in a number of languages.)

As mentioned previously, there is a direct one-to-one relationship between a query (or mutation) and a resolver. Figure 2 below shows two queries, movies (which is illustrated in figure 1 above) and movie. The query movies returns an array of the type movie. The query movie returns a single movie type based on a unique identifier that is passed as a query parameter. Notice that each query is bound to a resolver.

Figure 2: There is a one-to-one relationship between a query and a resolver

The important thing to understand is that there are a variety of ways that a query can be bound to a given resolver. How this happens depends on the GraphQL implementation. For example, the GraphQL npm package allows developers to write a query and resolver within a single object, GraphQLObjectType, as shown in listing 1 below.

const schema = new GraphQLSchema({
query: new GraphQLObjectType({
name: 'RootQueryType',
fields: {
hello: {
type: GraphQLString,
resolve() {
return 'world';
}
}
}
})
})

Listing 1: Binding a query to a resolver using the NodeJS GraphQLObjectType

On the other hand, the Java implementation of the GraphQL spec takes a more long-winded approach to bind the query to the resolver, as shown in listing 2.

DataFetcher<Hello> helloDataFetcher = new DataFetcher<Hello>() {
@Override
public Hello get(DataFetchingEnvironment environment) {
Hello value = "world";
return value;
}
};

GraphQLObjectType objectType = newObject()
.name("ObjectType")
.field(newFieldDefinition()
.name("hello")
.type(GraphQLString)
)
.build();
GraphQLCodeRegistry codeRegistry = newCodeRegistry()
.dataFetcher(
coordinates("ObjectType", "hello"),
helloDataFetcher)
.build();

Listing 2: Binding a query to a resolver using the Java GraphQL framework

Some resolvers can become quite complex if they need to support recursive queries. A recursive query returns data in a hierarchical manner, depending on the value associated with parent data — think “drill down.” Writing the resolver to support a recursive query can be a challenge, both in terms of writing the resolver itself and in creating the performance test that verifies that the code is running within expected operating parameters.

Benefits and Challenges of Recursive Queries

A key benefit that a recursive query provides to GraphQL is that you can create an expression in GQL that returns an object graph. Using our movies example, a developer can easily write a query that returns not only the title and release date of each movie, but also actors in the movie by first name, last name and date of birth, as well as the first and last name of the director of the movie, as shown in figure 3 below.

Figure 3: GraphQL queries can return results that describe objects within a broad object graph

The query in Figure 3 requires only one network call out to the GraphQL API. Were we to use a RESTful API to get the information provided by the GraphQL query, we’d have to make a number of calls to the network.

First, we’d have to get all the movies:

http://www.imbob-example.com/api/movies 

Then, we’d need to make a call to the network for more details about each movie (one call per movie). The result that returned would most likely contain unique identifiers for the actors and directors:

http://www.imbob-example.com/api/movies/6fceee97-6b03-4758-a429-2d5b6746e24e

Once we have actors and directors identifiers, we’ll need to make other calls to the network for the details of each actor:

http://www.imbob-example.com/api/actors/5130acee-21fe-48bf-a06f-c8bdcb38ea60

And then yet another set of calls for the details of each director:

http://www.imbob-example.com/api/directors/e71b9207-6f27-4d87-bce0-86ef47c97302

As you can see, using REST incurs a lot of back-and-forth on the network between the client and the REST server. GraphQL requires but a single network call by the client. The server-side GraphQL API code does all the recursion work of getting the data about the movie, the actors and the director — in other words, the GraphQL API is performing a recursive query.

At first glance, this is a definite benefit. But just because the client is not making a kazillion trips over the network to satisfy the requirements of the query does not mean that a kazillion trips are not being made. An astounding number of network calls might be happening within the API server. In fact, it’s not uncommon for a single GraphQL resolver to make network calls to a variety of databases and other APIs, both internal and external to the data center hosting the API itself. This can be a problem. After all, a trip to the network is still a trip to the network, no matter where it’s being made.

Any trip to a network incurs an expense, at the very least in terms of latency, and maybe even more in terms of CPU usage and other operational parameters. Consequently, the true cost of queries must be known before the code makes its way to production.
In order to get a definite picture of what’s going on, we need to test. While testing the API directly using standard HTTP calls is useful in terms of understanding client-to-API performance, to get a crystal-clear idea of API performance overall, we also need to test the resolver for each of the API’s queries and mutations to get some insight into resolver performance. Otherwise, should a problem occur, we’re only guessing about the cause.

Performance Testing a GraphQL Resolver

There are a variety of ways to performance test a GraphQL resolver. The first, most apparent way is at the function entry point.
For example, in the case of the movies resolver that’s associated with the query movies, as shown above in figure 2, we’ll need to execute a performance test directly against the function getItemFromCollection(‘MOVIES’, args.movie.id). Running a test against the function in isolation can be of limited value because isolation shields testing activity from the impact and side effects of other functions running simultaneously. An isolated test is good for developers running unit tests, but in terms of the entirety of the API, more is needed. This is where application monitors come into play.

Many of the open-source implementations of the GraphQL specification ship with a tracing feature that reports resolver activity as a query makes its way along the execution path. One example of a framework with tracing built-in is Apollo Server 2.0.
To enable tracing in Apollo Server 2.0, you add the configuration property tracing:true to the configuration JSON when constructing a new server instance (as seen in red typeface in listing 3 below).

const schema = makeExecutableSchema({
typeDefs,
resolvers,
schemaDirectives: {
requiresPersonalScope: RequiresPersonalScope
}
});

const server = new ApolloServer({
subscriptions,schema, tracing: true, context: ({req, res}) => {
console.log(req)
return req
}
});

Listing 3: Enabling tracing on Apollo Server 2.0 on the server-side

Once tracing is turned on the server-side, each response that’s returned from a query or mutation will include trace information about each relevant resolver (see figure 4 below). Should you want to view trace information along with a query’s response in GraphQL Playground, you’ll need to change the value of tracing.hideTracingResponse to false in the Settings page.

Figure 4: Set tracing.hideTracingResponse to false in the Settings page of GraphQL Playground to view trace data in a query response

Built-in tracing gives developers and testers visibility into what’s happening within the actual API code. But once you go beyond the API server, things get tricky.

For example, as mentioned earlier, some resolver code might use intelligence that lives outside the API, such as in a database or another API altogether. In cases such as these, you might need to use a distributed tracing solution such as Zipkin or Jaeger. You also can put monitoring in place that encompasses the GraphQL host that observes egress and ingress activity to external databases and APIs, and then you can correlate resolver tracing data with trace data from the databases or external APIs on an ad hoc basis.

Putting It All Together

GraphQL is a powerful way to approach API implementation. Allowing developers to make one trip to the network to get the data they need, as they need it, by writing declarative queries brings a level of versatility and efficiency to data-driven programming that is missing from the RESTful API experience. However, this power comes with a price. What used to be multiple network trips between client and API now becomes multiple network trips within the GraphQL API.

Benefits that GraphQL realizes between client and server can easily be lost due to poor server-side activity. Thus, test practitioners need to performance test activities not only between clients and the GraphQL API, but also within the API itself. In order to conduct adequate performance testing, you need to see what’s going on in those resolvers.

The tracing and monitoring mechanisms built into the various GraphQL API implementations are an invaluable tool for observing resolver behavior within a GraphQL API. Distributed tracing will provide the information required to understand what’s going on between a GraphQL API and the external sources an API’s resolvers use.

Learn more about GraphQL bottlenecks and performance testing in this article from our sister company, TestRail.