ETags for REST With Spring

By

As part of the REST with Spring Series, this post will cover the fundamentals of REST API ETags. We are going to focus on the ETags with the Spring.

 

1. What is Etag

Etag, also known as entity tag is an HTTP response header returned by an HTTP/1.1 compliant web server used to determine a change in content at a given URL. On a high level, there are two main uses of for Etags:

  1. Conditional GET request.
  2. Concurrency Control

Conditional GET helps in the caching and reduce the data traffic. Conditional GETs allow a client to ask a server if a resource has changed. If it has not changed, it can assume it’s current knowledge is up to date. If it has changed, the server will send the resource back to the client.

 

2. REST and Etags

To understand Etags work, let’s take a look at one of the example to understand the workflow. As part of the example, we are going to get the information on the JavaDevJournal GitHub repository. Let’s give a simple curl call to get information about the repository

$ curl -H "Accept: application/json" -i https://api.github.com/repos/umeshawasthi/javadevjournal

Here is the output of our curl request

HTTP/1.1 200 OK
Date: Sat, 27 Oct 2018 23:15:33 GMT
Content-Type: application/json; charset=utf-8
Content-Length: 5682
Server: GitHub.com
Status: 200 OK
X-RateLimit-Limit: 60
X-RateLimit-Remaining: 50
X-RateLimit-Reset: 1540684457
Cache-Control: public, max-age=60, s-maxage=60
Vary: Accept
ETag: "85143f9cd38933556c486dd386f28a93"
Last-Modified: Sat, 27 Oct 2018 23:15:30 GMT
.......
Vary: Accept-Encoding
X-GitHub-Request-Id: CC0C:3215:80DEE7:A284EC:5BD4F195

{
  "id": 128960270,
  "node_id": "MDEwOlJlcG9zaXRvcnkxMjg5NjAyNzA=",
  "name": "javadevjournal",
  "full_name": "umeshawasthi/javadevjournal",
  "private": false,
  "owner": {
    "login": "umeshawasthi",
    "id": 1564304,
    "node_id": "MDQ6VXNlcjE1NjQzMDQ="
     .....
}

Let’s break above example to understand the basic workflow.

  • The client made a REST call and got a response back. The response also includes the Etags header.

One of the basic questions is “How can a client use Etags header information?”.The simple answer is, REST client can use this Etags in future calls to see if there is any change happened in the repository or not. Our client is going to use the “If-None-Match” header with the ETag value from the last response ( 85143f9cd38933556c486dd386f28a93 ) in the next REST call.

  • With this value, we request GitHub REST API to send us data only if Etag does not match. This means that there is some change happened on the repo and we can not use the cache data.

Let’s see this in action by sending the following curl request to GitHub.

$ curl -H "Accept: application/json" -H 'If-None-Match: "85143f9cd38933556c486dd386f28a93"' -i https://api.github.com/repos/umeshawasthi/javadevjournal

If nothing changed on our repo, we may see similar results:

HTTP/1.1 304 Not Modified
Date: Sat, 27 Oct 2018 23:49:22 GMT
Content-Type: application/octet-stream
Server: GitHub.com
Status: 304 Not Modified
X-RateLimit-Limit: 60
X-RateLimit-Remaining: 48
X-RateLimit-Reset: 1540684457
Cache-Control: public, max-age=60, s-maxage=60
Vary: Accept
ETag: "85143f9cd38933556c486dd386f28a93"
Last-Modified: Sat, 27 Oct 2018 23:15:30 GMT
Access-Control-Expose-Headers: ETag, Link, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval
Access-Control-Allow-Origin: *
Strict-Transport-Security: max-age=31536000; includeSubdomains; preload
X-Frame-Options: deny
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
Referrer-Policy: origin-when-cross-origin, strict-origin-when-cross-origin
Content-Security-Policy: default-src 'none'
Vary: Accept-Encoding
X-GitHub-Request-Id: CEA3:3216:D33F04:10D9486:5BD4F982

This indicates that nothing changed on the repo. The REST client can continue using the cache snapshot of the data. Let’s make some changes to our Github repository and again sent the same curl request.

$ curl -H "Accept: application/json" -H 'If-None-Match: "85143f9cd38933556c486dd386f28a93"' -i https://api.github.com/repos/umeshawasthi/javadevjournal

Since we made changes to the repository before this call, REST API will send a response with new Etag indicating that repository status changed.API will also send the updated data to out REST client. Here is the sample response from GitHub REST API.

HTTP/1.1 200 OK
Date: Sat, 27 Oct 2018 23:57:03 GMT
Content-Type: application/json; charset=utf-8
Content-Length: 5682
Server: GitHub.com
Status: 200 OK
X-RateLimit-Limit: 60
X-RateLimit-Remaining: 59
X-RateLimit-Reset: 1540688223
Cache-Control: public, max-age=60, s-maxage=60
Vary: Accept
ETag: "6db3e94246ae8596f171092f7db9c1a6"
Last-Modified: Sat, 27 Oct 2018 23:56:54 GMT
X-GitHub-Media-Type: github.v3
Access-Control-Expose-Headers: ETag, Link, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval
Access-Control-Allow-Origin: *
Strict-Transport-Security: max-age=31536000; includeSubdomains; preload
X-Frame-Options: deny
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
Referrer-Policy: origin-when-cross-origin, strict-origin-when-cross-origin
Content-Security-Policy: default-src 'none'
Vary: Accept-Encoding
X-GitHub-Request-Id: CFA1:3215:825426:A461C6:5BD4FB4F

{
  "id": 128960270,
  "node_id": "MDEwOlJlcG9zaXRvcnkxMjg5NjAyNzA=",
  "name": "javadevjournal",
  "full_name": "umeshawasthi/javadevjournal",
  "private": false,
  "owner": {
    "login": "umeshawasthi",
    "id": 1564304,
    "node_id": "MDQ6VXNlcjE1NjQzMDQ="
   }
}

Just pay close attention to Etag as there is a new Etag in the response.

[pullquote align=”normal”]Etags is one of the common ways in a REST API to save bandwidth and pull data only when it is modified. [/pullquote]

 

3. Spring and Etags

Spring Framework provides transparent support to Etags using the ShallowEtagHeaderFilter. The ShallowEtagHeaderFilter filter creates a “shallow” ETag by caching the content written to the response and computing an MD5 hash from it. The next time a client sends, it does the same, but it also compares the computed value against the If-None-Match request header and, if the two are equal, returns a 304 (NOT_MODIFIED). Let’s see how we can add Etags support in our Spring MVC application.

<filter>
	<filter-name>etagFilter</filter-name>
	<filter-class>org.springframework.web.filter.ShallowEtagHeaderFilter</filter-class>
</filter>
<filter-mapping>
	<filter-name>etagFilter</filter-name>
	<url-pattern>/*</url-pattern>
</filter-mapping>
<filter-mapping>

If you are using Spring Boot, add ShallowEtagHeaderFilter as a bean in your configuration class. This filter will automatically add ETags for the responses.

@Configuration
public class WebConfig {

 @Bean
 public Filter shallowEtagHeaderFilter() {
  return new ShallowEtagHeaderFilter();
 }
}

[pullquote align=”normal”]Keep in mind that above works as long as the response does not contain any cache control headers. Also be aware that Spring filter only creates a “shallow” ETags calculated based on the response.[/pullquote]

 

4. Etags Workflow

Let’s put above workflow in a sequence diagram

5. Etags and Concurrency Control

Concurrency control is another important aspects while working on the REST API. This is really important in the modern enterprise applications. Concurrency control is an important aspect if we have multiple REST clients. These clients can update the same data at the same time leading to data inconsistencies. This topic deserves a separate article and we may cover in future posts. Here are my recommendations for a quick read. 

  1. ETags and Optimistic Concurrency Control
  2. Avoid Data Corruption in Your REST API with ETags

 

6. Video Tutorial

If you prefer video content, please refer to the following video to get all these details

[responsive_video type=’youtube’ hide_related=’1′ hide_logo=’1′ hide_controls=’0′ hide_title=’0′ hide_fullscreen=’0′ autoplay=’0′]https://www.youtube.com/watch?v=kDrbKzDOYYk[/responsive_video]

 

Summary

In this post, we discussed the fundamentals of REST API ETags and how to use Etags for conditional GET. We also talked about ETags with the Spring. In the last part of the article, we also talk about the concurrency control using Etags. 

You Might Also Like

Scroll to Top