Using the Spring RESTTemplate with Jackson
About a week a go I wrote a blog post on gridshore about the NOS open data API. Most of the technical items are explained in that post. In this post I want to tell you about the small library I have created around the NOS open data REST based API. With this client it is easy to create your own application that connects to the NOS servers and obtains news items, videos or audio fragments.
In this post you can find information about the Spring REST template, jackson mapping of JSON to java beans, and some spring tricks to make an easy web application.
Source code
The sources for the project are on Github.
https://github.com/jettro/JavaNosApi
I am working on getting the binaries in the maven central repository. If that is finished you can obtain them with the following dependency.
<br>
<dependency><br>
<groupId>nl.gridshore.nosapi</groupId><br>
<artifactId>gridshore-nosapi</artifactId><br>
<version>0.1</version><br>
</dependency><br>
NOS open data overview
The NOS, “Nederlands Omroep Stichting”, is providing their news articles, videos and audio files through a REST based api. Before you can use it, you need to have an API key. You can easily request one via the website. The service returns xml, json and a php format. At the moment the possibilities are limited, but the API is ok. It is easy to request information, sometimes the return values are a bit strange. There seem to be to many arrays in the results. The biggest limitation for me is that you are not able to order search results. When looking for news you will usually get old articles.
The following urls are some examples that you can call. For an overview of return values, check the resources for a link to the documentation.
http://open.nos.nl/v1/latest/article/key/{APIKEY}/output/json/category/nieuws/
http://open.nos.nl/v1/search/query/key/{APIKEY}/output/json/q/{SEARCH}
http://open.nos.nl/v1/guide/tv/key/{APIKEY}/output/json/start/2010-10-06/end/2010-10-18/
Solution overview
The following image gives you an idea of the main components of the solution. Your application calls one of the obtain methods from the DataProvider, obtainVersion() for instance. The DataProvider creates the url and calls the RestTemplate with this url and the jackson bean that it would like in return. Then, the RestTemplate requests the url from the NOS API server and gets a JSON response. The RestTemplate uses the Jackson to create a Jackson java bean from the provided JSON. This bean is returned to the DataProvider. The DataProvider maps the Jackson bean to our own java bean, the data bean, and returns this to the calling application.
Let us move on to a more in depth look into the used technologies.
Jackson JSON mapper
Jackson is used a a JSON processor. The JSON that is obtained from the REST api using the spring RestTemplate is converted into java objects. The RestTemplate supports jackson out of the box. More on that later on. Let us focus on jackson for now.
There is not a lot that you need to do for jackson to do its work. You have to add the dependency to your maven pom. Then you have to create the java domain classes that are java representations of the JSON structure that you want to proces. If you use properties with the same name as you JSON properties and you use basic types, you just have to create the classes and that is it. Of course their are a few exceptions.
I am one of those persons that hates the java Date implementation. Therefore I try to use Jodatime where possible. There is no problem to use Jodatime with jackson. You do have to create custom deserializers that conform to the date and time format as provided by JSON. I have properties that are DateTime objects and I have some LocalDate objects. For both of these I have created deserializers that are configured in the java classes. The following code block shows you the class. The next code block an example of how to wire this custom deserializer in you java bean.
<br>
public class JsonDateTimeDeserializer extends JsonDeserializer<DateTime> {<br>
private final DateTimeFormatter formatter = DateTimeFormat.forPattern("yyyy-MM-dd HH:mm:ss");</p>
<p> @Override<br>
public DateTime deserialize(JsonParser jp, DeserializationContext ctxt) throws IOException, JsonProcessingException {<br>
return formatter.parseDateTime(jp.getText());<br>
}<br>
}<br>
<br>
@JsonDeserialize(using = JsonDateTimeDeserializer.class)<br>
@JsonProperty("last_update")<br>
public void setLastUpdate(DateTime lastUpdate) {<br>
this.lastUpdate = lastUpdate;<br>
}<br>
Notice that the annotation is placed on the setter. You cannot configure this on the property itself.
There is a second annotation in the previous code block. The JsonProperty annotion can be used to indicate the name of the JSON property is not the same as the property the java bean has.
Another thing I want to mention is the structure of some of the beans. Look at the VersionWrapper class. This class contains an ArrayList of Version objects. I would have preferred to have the version and build properties at the first level, but the NOS api returns them as {version:[{version:v1, build:0.0.1}]}. That is why I need the wrapper.
Spring RestTemplate
The RestTemplate is used to request the provided url and transform the obtained JSON object into a java bean. The transformation is done using Jackson. The following code block shows you how to do it. The example shows obtaining the version information of the API.
<br>
public Version obtainVersion() {<br>
VersionWrapper versionWrapper = restTemplate.getForObject(<br>
serverBaseUrl + "index/version/key/{apikey}/output/json", VersionWrapper.class, apiKey);<br>
return new Version(<br>
versionWrapper.getVersions().get(0).getBuild(),<br>
versionWrapper.getVersions().get(0).getVersion());<br>
}<br>
Notice that we need to provide the API key. The RestTemplate makes it easy to use placeholders in creating the url to request.
But wait, where do I tell the RestTemplate to use Jackson to do the transformation? You are right, we do not tell it at all. The RestTemplate has configured a number of MessageConversters out of the box. One of them is the MappingJacksonHttpMessageConverter. This converter uses the mediatype to determine whether it can and should convert the message. If the application/json mediatype is provided in the response, this converter picks up the response.
One thing I want to talk about in relation to the RestTemplate is error handling. The RestTemplate by default configures the DefaultResponseErrorHandler. In case of errors in the response from the server, this handler checks for client errors and server errors. Client errors are the html 400 series and server errors are the html 500 series. I wanted to have slightly different mechanism for error handling. I want to explicitly throw a custom ClientException that knows what went wrong because it analyzes the data from the response object it received from the server. The NOS API returns a JSON object for certain specific errors. Using this information you can take specific actions in your application.
The following code block shows the important part of my own error handler.
<br>
private void throwClientException(Charset charset, byte[] body) throws IOException {<br>
String jsonContentString = (charset != null)?new String(body, charset):new String(body);</p>
<p> ObjectMapper mapper = new ObjectMapper();<br>
JsonNode rootNode = mapper.readValue(jsonContentString, JsonNode.class);<br>
String type = rootNode.getFieldNames().next();</p>
<p> JsonNode errorNode = rootNode.getElements().next().get("error");<br>
int code = errorNode.get("code").getIntValue();<br>
String message = errorNode.get("message").getTextValue();<br>
throw new ClientException(code, type, message);<br>
}<br>
As you can see, the JSON content is parsed using jackson. Now we can read the actual error from the JSON object.
That is all. Now you are able to use the DataProviderImpl class to communicate with the NOS API. Be sure to provide your own key when initializing the DataProviderImpl. Time to have a look at the sample that comes with the java client API.
Integration tests
If you checkout the source code, you will not find a lot of unit tests. I decided to focus on integration tests instead. For all the DataProvider methods I have at least one test and there are some additional tests to check the error handling. You can find the integration tests in the class DataProviderImplIntegrationTest.
The sample
The sample is a basic spring mvc application. I try to use the new features of spring mvc 3.0 as much as possible. Some of the features in the sample application that I want to mention are mentioned below. For the applicition itself I urge you to have a look at the sourcecode. The following image gives you an idea of what the application does.
Using SpEL to configure the API Key
You need to specify the API key to be able to connect to the NOS API. So configuring the DataProviderImpl as a bean requires you to configure the key. Of course you do not want to store it in a property file and add it to a public source control repository. Therefore I used the SpEL, Spring Expression Langugae, to obtain it from an environment variable. The following code block shows the interesting code.
<br>
<bean id="dataProvider" class="nl.gridshore.nosapi.impl.DataProviderImpl"><br>
<constructor-arg value="#{systemProperties['nosApiKey']}"/><br>
</bean><br>
Error handling
You can configure spring mvc to handle exception in a certain way. It is very easy to direct all exceptions to one specific view. In my code I wanted to forward all errors to the error page. Not showing all the ugly stacktraces to your user if something goes wrong. This is done in the exceptionResolver. The following code block shows the configuration.
<br>
<bean id="exceptionResolver" class="org.springframework.web.servlet.handler.SimpleMappingExceptionResolver"><br>
<property name="exceptionMappings"><br>
<map><br>
<entry key="java.lang.Exception" value="error"/><br>
</map><br>
</property><br>
</bean><br>
Static resources
New in spring 3.0 is the configuration of static resources. You can specify which resources can be served by the default servlet. You can also set the right headers for client side caching. The following code block shows the config.
<br> <mvc:default-servlet-handler/><br> <mvc:resources mapping="/js/**" location="/js/" cache-period="31556926"/><br> <mvc:resources mapping="/style/**" location="/style/" cache-period="31556926"/><br> <mvc:resources mapping="/image/**" location="/image/" cache-period="31556926"/><br>
ParameterizableViewController
The final thing I want to mention from the spring mvc new stuff. You can easily map incoming requests to pages without the need to create a custom controller. The following config maps the root and the index.html request to the index.jsp page.
<br> <mvc:view-controller path="/" view-name="index"/><br> <mvc:view-controller path="/index.html" view-name="index"/><br>
The Future
I am not able to see what the future will bring. I am waiting for new releases of the API. Hope to have the ordering capabilities in the search soon and maybe some additional categories of news?
I am looking into out of the box support for caching. That way you can easily cache the data and you do not have to worry about reaching the allowed query limits of the API. Another thing I want to do is create an application in google app engine. Have to check if this is possible with the IP addresses that the NOS API wants to have before allowing access.
If you have questions or ideas for improvement, feel free to enter an issue at Github or post a comment on this post.
Resources
Other blogpost about the NOS API on gridshore
Official documentation of the NOS API
Documentation about the spring RestTemplate
