API Gateways – An Evaluation of Zuul 2

https://www.novatec-gmbh.de/en/blog/api-gateways-an-evaluation-of-zuul-2/

API Gateways – also known as Edge Service – are a fundamental part of a cloud-native microservice architecture. They represent the central access point for all requests for the backend services. In this blog post we evaluate Zuul 2, the open source Gateway solution of Netflix. Using specific code examples, we show how to create your own Zuul 2-based Edge Service.

Netflix Zuul 1 and Zuul 2

Founded in the late 90s Netflix moved from being an online DVD-by-mail service to offer
a streaming application service starting by 2007 [NeStat]. By now almost everyone owns or at least
shares a Netflix account. The chart below shows the number of households subscribing to Netflix.

Figure 1 Number of households subscribing to Netflix in millions [NeStat]

To offer such a reliable experience Netflix maintains and operates a complex
intertwined system of software components. The Netflix API is the front door to that system,
supporting over 1,000 different device types and handling over 50,000 requests per second
during peak hours [NetZu]. In order to handle the complexity of this system Netflix developed
their own solution for particular problems. One of these problems is how to handle HTTP requests
from different devices. How do different clients communicate with the back-end services without
exploiting the entire back-end infrastructure? Thus, Netflix developed Zuul as a solution for these problems.

In 2013 Netflix announced Zuul 1 as their implementation of an API Gateway.
Within a microservice architecture, the API Gateway pattern suggests a wrapper around
your microservice landscape in order to expose a single API to the client thus providing
a single-entry point to your back-end infrastructure. The Gateway may also expose
different kinds of APIs respective to different clients connecting to your application.
Additionally, an API Gateway may implement cross-cutting concerns such as authorization,
analytics and so on. By now API Gateways are a common architectural pattern
to proxy requests to their origins and secure microservices within a cloud environment.
Zuul 1’s popularity increased after Spring integrated it into their Cloud Framework and
made it therefore easily accessible to the broad community.

In order to get a better understanding about Zuul 1, Figure 2 gives a brief overview of its
application within the Netflix Cloud.

Figure 2 Zuul 1 within the Netflix Cloud [NCldZ]

Zuul serves as an Edge Service executing different tasks such as request routing, analytics,
decline bad requests, authorization, and stress testing. Filters provide all these functionalities.

To get a better understanding of Filters the following Figure shows the request life cycle of HTTP requests.

Figure 3 Request Life cycle of incoming HTTP Requests handled by Zuul’s filters

Figure 3 shows different Filter types. Pre-Filters are executed before the HTTP request is routed to
the origin (Netflix’s synonym of back-end services). Routing-Filters are responsible for
routing HTTP requests to their origin and also handle outgoing responses from the origin.
Post-Filters perform tasks such as analytics or customization of HTTP headers.

Why did Netflix build Zuul 2?

Motivation

The main disadvantage of Zuul 1 is that calls to the API Gateway are all blocking.
If there is an HTTP request to the API Gateway, the calling thread will be blocked until
the client receives a response from the server. To take care of this Netflix needed to adjust
Zuul 1’s architecture in order to support non-blocking asynchronous request/response lifecycles.
Additionally, Netflix intended that Zuul 2 should support HTTP 2 as well as
WebSockets which Zuul 1 doesn’t. As a drawback there is no backwards compatibility
between Zuul 1 and Zuul 2 therefore you will have to rewrite your application
if you have integrated Zuul 1 and want to transition to Zuul 2.

In order to adjust Zuul 1 for their needs Netflix has made major architectural changes to
their Cloud Gateway which resulted in the newer Zuul 2. It still does the same as
his predecessor Zuul 1 such as request routing, authorization and analytics but with minor changes.

One of the primary advantages of Zuul 2 is that it now provides the capability
for devices and browsers to have a persistent connection to Netflix’s cloud services.
This emerged due to running Zuul 2 in production for several months. Additionally,
the execution of synchronous and asynchronous Filters is a huge benefit.
Understandably, Netflix’s intention was to improve performance of major tasks
such as connection scaling. More information may be found in this article.

What has changed

Unlike his predecessor, the Spring Cloud Framework does not integrate Zuul 2.
The implementation of Filters have slightly changed due to the transition from a synchronous blocking
system to an asynchronous non-blocking framework. In the following sections
we discuss the changes and how Zuul 1 has changed.

Netty

The Netty framework provides easy development of network applications such as
protocol servers and clients. It greatly simplifies and streamlines network programming
such as TCP and UDP socket server development. It is well documented and supported by a great community. Additionally, Netty enables bi-directional connections thus client and server can communicate without any interruptions. Otherwise the client (usually) needs to initiate a Two way-Handshake in order communicate with the server.

A Two way-Handshake protocol refers to a procedure where the client makes a request
to the server for enabling a connection. The server then responds with an acknowledgment
which the client accepts or declines. To take advantage of the previously mentioned benefits,
Netflix integrated Netty into Zuul 1’s architecture.

In Figure 4 you see a high-level architecture diagram of Netflix Zuul 2 with Netty.

Figure 4 High level architecture of Netflix Zuul 2 with Netty

The Netty handlers on the front and back of the Filters are responsible for handling the network protocol, web server, connection management and proxy work. With those inner workings abstracted away, the Filters do all of the heavy lifting [NzZuul].

Filters

Zuul 2 supports the following filter types:

Inbound Filters are the same as Pre-Filters and implement features such as authorization or manipulating the HTTP request headers.
Endpoint Filters either return a static response or proxy a request to the appropriate origin. Endpoint Filters are executed after Inbound Filters.
Outbound Filters implement the same logic as Post Filters and are executed after Endpoint Filters. Outbound Filters can be used to perform metrics or customizing the response.

Due to the changes Netflix made to their API Gateway each type of Filter may be implemented as a synchronous Filter or asynchronous Filter.

Zuul 1 had a major disadvantage respective to Zuul 2 being a blocking synchronous Gateway. Thus, being a choke point between a client and your back-end services. Netflix’s Zuul 2 is therefore more powerful respective to the possibility that you can decide if you want to implement asynchronous Filters or not.

Code Example

In the following chapter we will show an example how Zuul 2 is implemented and used as a reverse-proxy Gateway. All mentioned code examples can be found in GitHub.

Figure 5 shows a brief description of the following scenario. The Gateway (Zuul 2) will receive a HTTP request from a Client and copy the value of a Cookie to the Authorization header. If the HTTP request does not provide a Cookie, we route the request to a special Endpoint Filter which will return a Bad Request response. The Endpoint Filter is part of the API Gateway. Otherwise the Gateway proxies the request to one of the back-end services users, comments or images.

Figure 5 Scenario of given example

Dependencies

To use Zuul 2 you have to get the correct dependency which is available in the Netflix Zuul wiki. For Maven you may use the following dependency inside your pom.xml [DNet]

<groupId>com.netflix.zuul</groupId>

</dependency>

and for Gradle [DNet]

1	compile "com.netflix.zuul:zuul-core:2.1.2"

Setup Netty

Since Netflix uses the Netty framework to provide an asynchronous style of implementation you need to setup a Netty server on your own. Although Netflix provided a sample setup within their Git repository. You may decide to come up with your own implementation of a Netty server.

As it’s still a complex and extensive framework we will not cover to explain the setup of a Netty server instance in this post. You are free to visit the Netty page and work out your own solution, use Netflix’s example or have a look at our example solution which is accessible by a Git repository.

Authorization Filter

In Zuul 2 Filters are implemented in Groovy but Zuul does support any other JVM-based language.

package info.novatec.zuul2.filters.inbound

import com.netflix.zuul.filters.http.HttpInboundSyncFilter

import com.netflix.zuul.message.http.HttpRequestMessage

import info.novatec.zuul2.filters.endpoint.BadRequestEndpoint

import io.netty.handler.codec.http.cookie.Cookie

import org.apache.http.HttpHeaders

/**

* HttpInboundSyncFilter that extracts the 'customer-Id' cookie and adds its value to the

* request's Authorization header.

class AuthorizationFilter extends HttpInboundSyncFilter {

@Override

HttpRequestMessage apply(HttpRequestMessage request) {

Cookie cookie = request.parseCookies().getFirst("customer-Id")

if (!cookie?.value()?.trim()) {

request.getContext().setEndpoint(BadRequestEndpoint.class.getCanonicalName())

} else {

request.getHeaders().add(HttpHeaders.AUTHORIZATION, cookie.value())

return request

}

@Override

int filterOrder() {

return 1

}

@Override

boolean shouldFilter(HttpRequestMessage request) {

return true

}

As mentioned in section Netflix Zuul 1 and Zuul 2, Inbound-Filter are executed first within the Request Lifecycle. In case you have more than one Inbound-Filter that needs to be executed you can also define an execution order inside the filterOrder() method. The custom AuthorizationFilter extends the Zuul class HttpInboundSyncFilter. By doing this you have to overwrite three methods apply(), shouldFilter()and filterOrder().

filterOrder() returns an Integer value which defines the index of execution for this particular Filter.
There may be requests that do not need the application of specific Filters. Therefore, shouldFilter()implements predicates in order to verify the application of the Filter. If the method body is empty for every Filter then all Filters apply consecutively on every incoming requests. apply() implements the main functionality of a Filter.

In this example we apply a Filter on an HTTP request to copy the value of a Cookie and pass it into an Authorization Header. In line 17 we extract the Cookie. If there is no Cookie in the request we set our Endpoint to be BadRequestEndpointFilter in order to return a Bad Request response with status code 400.

Finally, in line 21 we add the retrieved customer-Id to the Authorization Header of the request.

The return type of the apply() method depends on the type of Filter you would like to implement. In case of an Outbound Filter which modifies responses from an origin you will need to change the return type to HttpResponseMessage and consecutively receive an HttpResponse object as a parameter instead of an HttpRequest object.

You may have noticed that we used the HttpInboundSyncFilter class to extend our AuthorizationFilterclass therefore implementing a synchronous Filter. Again, we emphazise that due to the new architecture of Zuul 2 you also have the possibility to extend from HttpInboundFilter class and therefore implement an asynchronous approach.

Bad Request Endpoint Filter

In this section we provide an Endpoint-Filter which returns a Bad Request as a response if no Cookie exists in the HTTP request.

package info.novatec.zuul2.filters.endpoint

import com.netflix.zuul.filters.http.HttpSyncEndpoint

import com.netflix.zuul.message.http.HttpRequestMessage

import com.netflix.zuul.message.http.HttpResponseMessage

import com.netflix.zuul.message.http.HttpResponseMessageImpl

import org.apache.http.HttpStatus

class BadRequestEndpoint extends HttpSyncEndpoint {

@Override

HttpResponseMessage apply(HttpRequestMessage request) {

HttpResponseMessage response = new HttpResponseMessageImpl(request.getContext(), request, HttpStatus.SC_BAD_REQUEST)

response.finishBufferedBodyIfIncomplete()

return response

}

Endpoint Filters are executed after Inbound Filters and are responsible for routing requests to their origin. In this example we have used an Endpoint Filter to return a Bad Request in case that no Cookie is present or empty. In order to implement our Filter as an Endpoint Filter we extend the class HttpSyncEndpoint.

The apply() method will receive an HttpRequestMessage object as a parameter and return a response message. Then we create a new HTTP response message and set the appropriate status code.

Routes Filter

The Routes-Filter is responsible for routing of incoming HTTP requests.

package info.novatec.zuul2.filters.inbound

import com.netflix.zuul.context.SessionContext

import com.netflix.zuul.filters.http.HttpInboundSyncFilter

import com.netflix.zuul.message.http.HttpRequestMessage

import com.netflix.zuul.netty.filter.ZuulEndPointRunner

import info.novatec.zuul2.filters.endpoint.NotFoundEndpoint

/**

* Routing filter on base of HttpInboundSyncFilter.<br/>

* URIs of the proxied back-end services are defined in {@code application.properties}

class Routes extends HttpInboundSyncFilter {

@Override

HttpRequestMessage apply(HttpRequestMessage request) {

SessionContext context = request.getContext()

switch (request.getPath()) {

case "/images":

context.setEndpoint(ZuulEndPointRunner.PROXY_ENDPOINT_FILTER_NAME)

context.setRouteVIP("images")

break

case "/comments":

context.setEndpoint(ZuulEndPointRunner.PROXY_ENDPOINT_FILTER_NAME)

context.setRouteVIP("comments")

break

case "/users":

context.setEndpoint(ZuulEndPointRunner.PROXY_ENDPOINT_FILTER_NAME)

context.setRouteVIP("users")

break

default:

context.setEndpoint(NotFoundEndpoint.class.getCanonicalName())

}

return request

}

@Override

int filterOrder() {

return 0

}

@Override

boolean shouldFilter(HttpRequestMessage request) {

return true

}

The Routes-Filter looks at the request path of an incoming HTTP request in order to decide which origin will be invoked. Thus, we use the built-in ProxyEndpoint Filter which is responsible for routing the request to the appropriate origin. If the requested path does not exist, we will route the request to the Endpoint Filter in order to return a Not Found status message.

Below is the implementation of the NotFoundEndpoint Filter.

package info.novatec.zuul2.filters.endpoint

import com.netflix.zuul.filters.http.HttpSyncEndpoint

import com.netflix.zuul.message.http.HttpRequestMessage

import com.netflix.zuul.message.http.HttpResponseMessage

import com.netflix.zuul.message.http.HttpResponseMessageImpl

import org.apache.http.HttpStatus

class NotFoundEndpoint extends HttpSyncEndpoint {

@Override

HttpResponseMessage apply(HttpRequestMessage request) {

HttpResponseMessage response = new HttpResponseMessageImpl(request.getContext(), request, HttpStatus.SC_NOT_FOUND)

response.finishBufferedBodyIfIncomplete()

return response

}

There already have been issues reported to Netflix’s Github respective to a filed-based routing possibility such as in Zuul 1. According to Netflix routing may also be done by a routing configuration file in the future but it’s not implemented yet. Thus, the Routes Inbound Filter may not be necessary in the future.

Application Properties

In the application.properties we declare specific properties in order to configure our API Gateway.

zuul.server.port.main=8887

# Configure filters

zuul.filters.root=src/main/groovy/info/novatec/zuul2/filters

zuul.filters.locations=${zuul.filters.root}/inbound,${zuul.filters.root}/outbound,${zuul.filters.root}/endpoint

zuul.filters.packages=com.netflix.zuul.filters.common

# Routing to proxied back-end services

users.ribbon.listOfServers=localhost:8081

users.ribbon.client.NIWSServerListClassName=com.netflix.loadbalancer.ConfigurationBasedServerList

comments.ribbon.listOfServers=localhost:8082

comments.ribbon.client.NIWSServerListClassName=com.netflix.loadbalancer.ConfigurationBasedServerList

images.ribbon.listOfServers=localhost:8083

images.ribbon.client.NIWSServerListClassName=com.netflix.loadbalancer.ConfigurationBasedServerList

# Deactivate Eureka

eureka.shouldFetchRegistry=false

eureka.validateInstanceId=false

@next=application-${@environment}.properties

In order to execute Filters, Zuuls’ FilterLoader has to know where to look for Filters. Therefore, the path to Filters must be declared. Thus, you need to set the root folder and accordingly the packages where the different Filters are implemented.

In Line 1 we state the port where our Gateway will be running, in this case the Gateway will be listening on port 8887. Line 9 to 14 define the routes where your back-end services are listening. Ribbon performs the back-end routing. To set a route to a back-end service you need to declare the following:

1 2	<name_of_service>.ribbon.listOfServers= <port> <name_of_service>.client.NIWSServerListClassName=com.netflix.loadbalancer.ConfigurationBasedServerLis

If you would like to disable Eureka as a Service Discovery you can set both eureka.shouldFetchRegistryand eureka.validateInstanceId to false.

For sake of simplicity we have used localhost to run the example on our local machines. In a real cloud-based environment you would usually integrated a service-discovery such as Eureka to fetch the URL of your back-end services instead of static URLs.

Outlook

The new version of Zuul has now been available for a couple of months. Implementing Zuul 2 as an API Gateway turned out to be a time-consuming task unlike Zuul 1 or Spring Cloud Gateway. Complexity increases by implementing your own Netty server. Additionally, this also adds more complexity to your application at the latest if you need to adjust your Netty server for additional needs.

Since the Spring Cloud Team integrated Zuul 1 into the Spring Cloud Framework one might assume that the Spring Cloud team considers to include Zuul 2 as well. According to a talk given by Spencer Gibbs on SpringOne in December 2017, the Spring Cloud team will not integrate Zuul 2 into the Spring ecosystem. However, the Spring Cloud Framework still supports Zuul 1 and thus, Zuul 1 remains as a viable solution. Particularly if you do not need your API Gateway to be asynchronous and non-blocking.

However, if you want your API Gateway to be asynchronous and non-blocking then you need to consider to either implement Zuul 2 or evaluate another solution. Though you want to try out Zuul 2 there a few aspects to consider.

There have been questions about missing, or precisely, desired features which are currently not available. One being the possibility to configure routes inside a configuration file. But this way of configuration only refers to Zuul 1. However, Zuul 2 is configured by a combination of an Inbound-Filter as well as a corresponding Endpoint-Filter as shown in the Routes-Filter example.

We already mentioned that Netflix intended to add file-based routing to Zuul 2 as a possibility to manage API routes, but it is yet unclear when they intended to add this feature. Further we explained that Zuul 2 comes with Netty as a network application framework. Thus, it is your responsibility as a developer to setup a Netty server in order to start your API Gateway. Despite the fact that you have to deal with Netty as an additional framework, this will certainly add more complexity to your application and implies more effort.

Lastly, as previously mentioned there is no backwards compatibility between both versions of Zuul. Thus, if you would like to use Zuul 2 in your application and already integrated Zuul 1 you have to consider rewriting your entire application.

Moreover, if you are already operating within a Spring Cloud environment but need an asynchronous and non-blocking Gateway there might be another possibility to consider: Spring Cloud Gateway is easy to integrate and to maintain.

Besides, Filters in Spring Cloud Gateway are route specific. As a consequence Filters apply merely on requests that match a defined route on the Filter. In Zuul 1 as well as in Zuul 2, Filters are global and thus executed consecutively to every request.

As a conclusion Zuul 2 remains a solid and viable solution as an API Gateway. If you would like to get more information about Zuul 1 and/or Zuul 2 you can follow this link.

Thanks for reading! If you have further questions feel free to leave a comment!