Spring 5 - WebClient and WebTestClient Usage

Objective:

The objective of this tutorial is to use Spring 5’s reactive WebClient to consume the GitHub API's and to perform CRUD operations on user’s Github Repositories.

Github links:

The github link for this project is available here. Refer the project spring-webclient-demo in this link.

What is WebClient?

WebClient is an interface representing the main entry point for performing web requests.

It has been created as a part of the Spring Web Reactive module and will be replacing the classic RestTemplate in these scenarios. The new client is a reactive, non-blocking solution that works over the HTTP/1.1 protocol.

How to use WebClient

WebClient is part of Spring 5’s reactive web framework called Spring WebFlux. To use WebClient, you need to include the spring-webflux module and reactor in your project.

Dependencies:

If you have an existing Spring Boot project, you can add the spring-webflux module and reactor-spring by adding the following dependencies in the pom.xml file -

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-webflux</artifactId>
</dependency>
<dependency>
    <groupId>org.projectreactor</groupId>
    <artifactId>reactor-spring</artifactId>
    <version>1.0.1.RELEASE</version>

</dependency>

Note that, you need Spring Boot version 2.x.x for using the Spring WebFlux module.

Create a new project from Scratch

If you’re creating a project from scratch, then you can generate a starter project with spring-webflux module from the Spring Initializr website -

1. Go to http://start.spring.io.
2. Select Spring Boot version 2.x.x.
3. Add Spring Reactive Web dependency in the dependencies section.
4. Change Group and Artifact details if you want to, and click Generate Project to download the project.

Consuming Remote APIs using WebClient

Let’s make things interesting and use WebClient to consume a Real World API.

In this article, we’ll consume Github’s APIs using WebClient. We’ll use WebClient to perform CRUD operations on user’s Github Repositories.

You can read on and understand the bits and pieces of WebClient from scratch or download the entire demo project with all the examples are available here. Refer the project spring-webclient-demo in this link.

Creating an instance of WebClient

1. Creating WebClient using the create() method

You can create an instance of WebClient using the create() factory method -

WebClient webClient = WebClient.create();

If you’re consuming APIs from a specific service only, then you can initialize WebClient with the baseUrl of that service like so -

WebClient webClient = WebClient.create("https://api.github.com");

2. Creating WebClient using the WebClient builder

WebClient also comes with a builder that gives you a bunch of customization options including filters, default headers, cookies, client-connectors etc -

WebClient webClient = WebClient.builder()
        .baseUrl("https://api.github.com")
        .defaultHeader(HttpHeaders.CONTENT_TYPE, "application/vnd.github.v3+json")
        .defaultHeader(HttpHeaders.USER_AGENT, "Spring 5 WebClient")
        .build();

Making a Request using WebClient and retrieving the response

Here is how you can use WebClient to make a GET request to Github’s List Repositories API -

public Flux<GithubRepo> listGithubRepositories(String username, String token) {
     return webClient.get()
            .uri("/user/repos")
            .header("Authorization", "Basic " + Base64Utils
                    .encodeToString((username + ":" + token).getBytes(UTF_8)))
            .retrieve()
            .bodyToFlux(GithubRepo.class);
}

See how simple and concise the API calls are!

Assuming we have a class named GithubRepo that confirms to the Github’s API response, the above function will return a Flux of GithubRepo objects.

Note that, I’m using Github’s Basic Authentication mechanism for calling the APIs. It requires your github username and a personal access token that you can generate from https://github.com/settings/tokens. You can refer this link on how create personal access token for your Github account.

Using the exchange() method to retrieve the response

The retrieve() method is the simplest way to get the response body. However, If you want to have more control over the response, then you can use the exchange() method which has access to the entire ClientResponse including all the headers and the body -

public Flux<GithubRepo> listGithubRepositories(String username, String token) {
     return webClient.get()
            .uri("/user/repos")
            .header("Authorization", "Basic " + Base64Utils
                    .encodeToString((username + ":" + token).getBytes(UTF_8)))
            .exchange()
            .flatMapMany(clientResponse -> clientResponse.bodyToFlux(GithubRepo.class));
}

Using parameters in the request URI

You can use parameters in the request URI and pass their values separately in the uri() function. All the parameters are surrounded by curly braces. The parameters will automatically be replaced by WebClient before making the request -

public Flux<GithubRepo> listGithubRepositories(String username, String token) {
     return webClient.get()
            .uri("/user/repos?sort={sortField}&direction={sortDirection}", 
                     "updated", "desc")
            .header("Authorization", "Basic " + Base64Utils
                    .encodeToString((username + ":" + token).getBytes(UTF_8)))
            .retrieve()
            .bodyToFlux(GithubRepo.class);
}

Using the URIBuilder to construct the request URI

You can also gain full programmatic control over the request URI using a UriBuilder like so -

public Flux<GithubRepo> listGithubRepositories(String username, String token) {
     return webClient.get()
            .uri(uriBuilder -> uriBuilder.path("/user/repos")
                    .queryParam("sort", "updated")
                    .queryParam("direction", "desc")
                    .build())
            .header("Authorization", "Basic " + Base64Utils
                    .encodeToString((username + ":" + token).getBytes(UTF_8)))
            .retrieve()
            .bodyToFlux(GithubRepo.class);
}

Passing Request Body in WebClient requests

If you have the request body in the form of a Mono or a Flux, then you can directly pass it to the body() method in WebClient, otherwise you can just create a Mono/Flux from an object and pass it like so -

public Mono<GithubRepo> createGithubRepository(String username, String token, 
    RepoRequest createRepoRequest) {
    return webClient.post()
            .uri("/user/repos")
            .body(Mono.just(createRepoRequest), RepoRequest.class)
            .header("Authorization", "Basic " + Base64Utils
                    .encodeToString((username + ":" + token).getBytes(UTF_8)))
            .retrieve()
            .bodyToMono(GithubRepo.class);
}

If you have an actual value instead of a Publisher (Flux/Mono), you can use the syncBody() shortcut method to pass the request body -

public Mono<GithubRepo> createGithubRepository(String username, String token, 
    RepoRequest createRepoRequest) {
    return webClient.post()
            .uri("/user/repos")
            .syncBody(createRepoRequest)
            .header("Authorization", "Basic " + Base64Utils
                    .encodeToString((username + ":" + token).getBytes(UTF_8)))
            .retrieve()
            .bodyToMono(GithubRepo.class);
}

Finally, you can use various factory methods provided by BodyInserters class to construct a BodyInserter object and pass it in the body() method. The BodyInserters class contains methods to create a BodyInserter from an Object, Publisher, Resource, FormData, MultipartData etc -

public Mono<GithubRepo> createGithubRepository(String username, String token, 
    RepoRequest createRepoRequest) {
    return webClient.post()
            .uri("/user/repos")
            .body(BodyInserters.fromObject(createRepoRequest))
            .header("Authorization", "Basic " + Base64Utils
                    .encodeToString((username + ":" + token).getBytes(UTF_8)))
            .retrieve()
            .bodyToMono(GithubRepo.class);
}

Adding Filter Functions

WebClient supports request filtering using an ExchangeFilterFunction. You can use filter functions to intercept and modify the request in any way. For example, you can use a filter function to add an Authorization header to every request, or to log the details of every request.

The ExchangeFilterFunction takes two arguments -

1. The ClientRequest and

2. The next ExchangeFilterFunction in the filter chain.

It can modify the ClientRequest and call the next ExchangeFilterFucntion in the filter chain to proceed to the next filter or return the modified ClientRequest directly to block the filter chain.

1. Adding Basic Authentication using a filter function

In all the examples above, we are including an Authorization header for basic authentication with the Github API. Since this is something that is common to all the requests, you can add this logic in a filter function while creating the WebClient.

The ExchaneFilterFunctions API already provides a filter for basic authentication. You can use it like this -

WebClient webClient = WebClient.builder()
        .baseUrl(GITHUB_API_BASE_URL)
        .defaultHeader(HttpHeaders.CONTENT_TYPE, GITHUB_V3_MIME_TYPE)
        .filter(ExchangeFilterFunctions
                .basicAuthentication(username, token))
        .build()

Now, You don’t need to add the Authorization header in every request. The filter function will intercept every WebClient request and add this header.

2. Logging all the requests using a filter function

Let’s see an example of a custom ExchangeFilterFunction. We’ll write a filter function to intercept and log every request -

WebClient webClient = WebClient.builder()
        .baseUrl(GITHUB_API_BASE_URL)
        .defaultHeader(HttpHeaders.CONTENT_TYPE, GITHUB_V3_MIME_TYPE)
        .filter(ExchangeFilterFunctions
                .basicAuthentication(username, token))
        .filter(logRequest())
        .build();

Here is the implementation of the logRequest() filter function -

private ExchangeFilterFunction logRequest() {
    return (clientRequest, next) -> {
        logger.info("Request: {} {}", clientRequest.method(), clientRequest.url());
        clientRequest.headers()
                .forEach((name, values) -> values.forEach(value -> logger.info("{}={}", name, value)));
        return next.exchange(clientRequest);
    };
}

3. Using ofRequestProcessor() and ofResponseProcessor() factory methods to create filters

ExchangeFilterFunction API provides two factory methods named ofRequestProcessor() and ofResponseProcessor() for creating filter functions that intercepts the request and response respectively.

The logRequest() filter function that we created in the previous section can be created using ofRequestProcessor() factory method like this -

private ExchangeFilterFunction logRequest() {
    ExchangeFilterFunction.ofRequestProcessor(clientRequest -> {
        logger.info("Request: {} {}", clientRequest.method(), clientRequest.url());
        clientRequest.headers()
                .forEach((name, values) -> values.forEach(value -> logger.info("{}={}", name, value)));
        return Mono.just(clientRequest);
    });
}        

If you want to intercept the WebClient response, you can use the ofResponseProcessor() method to create a filter function like this -

private ExchangeFilterFunction logResposneStatus() {
    return ExchangeFilterFunction.ofResponseProcessor(clientResponse -> {
        logger.info("Response Status {}", clientResponse.statusCode());
        return Mono.just(clientResponse);
    });
}

Handling WebClient Errors

The retrieve() method in WebClient throws a WebClientResponseException whenever a response with status code 4xx or 5xx is received.

You can customize that using the onStatus() methods like so -

public Flux<GithubRepo> listGithubRepositories() {
     return webClient.get()
            .uri("/user/repos?sort={sortField}&direction={sortDirection}", 
                     "updated", "desc")
            .retrieve()
            .onStatus(HttpStatus::is4xxClientError, clientResponse ->
                Mono.error(new MyCustomClientException())
            )
            .onStatus(HttpStatus::is5xxServerError, clientResponse ->
                Mono.error(new MyCustomServerException())
            )
            .bodyToFlux(GithubRepo.class);

}

Note that Unlike retrieve() method, the exchange() method does not throw exceptions in case of 4xx or 5xx responses. You need to check the status codes yourself and handle them in the way you want to.

Handling WebClientResponseExceptions using an @ExceptionHandler inside the controller

You can use an @ExceptionHandler inside your controller to handle WebClientResponseException and return an appropriate response to the clients like this -

@ExceptionHandler(WebClientResponseException.class)
public ResponseEntity<String> handleWebClientResponseException(WebClientResponseException ex) {
    logger.error("Error from WebClient - Status {}, Body {}", ex.getRawStatusCode(), ex.getResponseBodyAsString(), ex);
    return ResponseEntity.status(ex.getRawStatusCode()).body(ex.getResponseBodyAsString());
}

Testing Rest APIs using Spring 5 WebTestClient

WebTestClient contains request methods that are similar to WebClient. In addition, it contains methods to check the response status, header and body. You can also use assertion libraries like AssertJ with WebTestClient.

Check out the following example for learning how to perform rest API tests using WebTestClient -

package com.example.webclientdemo;

import com.example.webclientdemo.payload.GithubRepo;
import com.example.webclientdemo.payload.RepoRequest;
import org.assertj.core.api.Assertions;
import org.junit.FixMethodOrder;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.junit.runners.MethodSorters;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.http.MediaType;
import org.springframework.test.context.junit4.SpringRunner;
import org.springframework.test.web.reactive.server.WebTestClient;
import reactor.core.publisher.Mono;


@RunWith(SpringRunner.class)
@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
@FixMethodOrder(MethodSorters.NAME_ASCENDING)
public class WebclientDemoApplicationTests {

    @Autowired
    private WebTestClient webTestClient;

    @Test
    public void test1CreateGithubRepository() {
        RepoRequest repoRequest = new RepoRequest("test-webclient-repository", "Repository created for testing WebClient");

        webTestClient.post().uri("/api/repos")
                .contentType(MediaType.APPLICATION_JSON_UTF8)
                .accept(MediaType.APPLICATION_JSON_UTF8)
                .body(Mono.just(repoRequest), RepoRequest.class)
                .exchange()
                .expectStatus().isOk()
                .expectHeader().contentType(MediaType.APPLICATION_JSON_UTF8)
                .expectBody()
                .jsonPath("$.name").isNotEmpty()
                .jsonPath("$.name").isEqualTo("test-webclient-repository");
    }

    @Test
    public void test2GetAllGithubRepositories() {
        webTestClient.get().uri("/api/repos")
                .accept(MediaType.APPLICATION_JSON_UTF8)
                .exchange()
                .expectStatus().isOk()
                .expectHeader().contentType(MediaType.APPLICATION_JSON_UTF8)
                .expectBodyList(GithubRepo.class);
    }

    @Test
    public void test3GetSingleGithubRepository() {
        webTestClient.get()
                .uri("/api/repos/{repo}", "test-webclient-repository")
                .exchange()
                .expectStatus().isOk()
                .expectBody()
                .consumeWith(response ->
                        Assertions.assertThat(response.getResponseBody()).isNotNull());
    }

    @Test
    public void test4EditGithubRepository() {
        RepoRequest newRepoDetails = new RepoRequest("updated-webclient-repository", "Updated name and description");
        webTestClient.patch()
                .uri("/api/repos/{repo}", "test-webclient-repository")
                .contentType(MediaType.APPLICATION_JSON_UTF8)
                .accept(MediaType.APPLICATION_JSON_UTF8)
                .body(Mono.just(newRepoDetails), RepoRequest.class)
                .exchange()
                .expectStatus().isOk()
                .expectHeader().contentType(MediaType.APPLICATION_JSON_UTF8)
                .expectBody()
                .jsonPath("$.name").isEqualTo("updated-webclient-repository");
    }

    @Test
    public void test5DeleteGithubRepository() {
        webTestClient.delete()
                .uri("/api/repos/{repo}", "updated-webclient-repository")
                .exchange()
                .expectStatus().isOk();
    }
}

Conclusion

Congratulations! In this article, you learned how to work with Spring 5’s reactive WebClient and WebTestClient APIs.

I hope you’ll find the examples presented in this article helpful. You can download the complete sample project with all the examples from here. Refer the project spring-webclient-demo in this link.

Thanks for reading. I really enjoyed writing this article for you and I hope to see you soon in my next article.