Introducing a Query tool as an Elasticsearch plugin (part 2)
In the first part of this series of blogs on Introducing a Query as an Elasticsearch plugin, I described the functionality of a query tool I also discussed the functionality of the tool as the structure of the project. In this part I want to take a deeper dive in interacting with Elasticsearch.
The post consists of four parts:
- Getting information about the Elasticsearch instance,
- Constructing the query using the JavaScript QueryDSL,
- Executing the query,
- Using the results.
If you need more background around AngularJS, which is used on the client side you can have a look at the following two posts:
Getting information about indexes, types and the available fields
The following image shows the first three items on the page. We obtain all the available indexes, the types and the fields.
These are all dynamic and obtained from Elasticsearch. The following code block shows how we obtaining the indexes.
http.get('/_status').success(function (data) {
var indices = [];
for (var index in data.indices) {
indices.push(index);
}
callback(indices);
});
When calling this function you need to provide a callback function that excepts a collection of indexes. For the indexes we use the special $http object from AngularJS that is injected into the AngularJS service. Check the services.js file for the complete code. You can also look at the output of this function by calling http://localhost:9200/_status. From the result we take the values for the indexes.
I have created similar functions for obtaining types and fields. They do however use another URL. For information about the types and fields we use the http://localhost:9200/_mapping url. The next code block shows obtaining the types from the mapping.
http.get('/_mapping').success(function (data) {
var myTypes = [];
for (var index in data) {
for (var type in data[index]) {
if (myTypes.indexOf(type) == -1) {
myTypes.push(type);
}
}
}
callback(myTypes);
});
And for the fun of it, the next code block shows obtaining the fields from the mapping.
http.get('/_mapping').success(function (data) {
var myTypes = [];
var myFields = [];
for (var index in data) {
for (var type in data[index]) {
if (myTypes.indexOf(type) == -1) {
myTypes.push(type);
var properties = data[index][type].properties;
for (var field in properties) {
if (myFields.indexOf(field) == -1) {
myFields.push(field);
}
}
}
}
}
callback(myFields);
});
Next step is about creating the query from the input fields.
Constructing the query using the JavaScript QueryDSL
It all starts with the Request object. This object is used to add everything you need to add to the query. You add the fields to return using the fields function, the induced using the indices function and yes indeed the types with the types function. The next code block shows these three configurations.
var request = ejs.Request();
request.indices($scope.chosenIndices);
request.types($scope.chosenTypes);
if ($scope.chosenFields.length > 0) {
request.fields($scope.chosenFields);
}
The next part is creating the query part based on the input field for terms, and the type of query (OR, AND or PHRASE). If nothing is entered in the search box we do the special match all query. In all other cases we create a Match query on the special _all field. Based on the chosen type of query we set the operator to OR or AND, or we set the type to be PHRASE.
if ($scope.search.term.length > 0) {
var matchQuery = ejs.MatchQuery("_all", $scope.search.term);
if ($scope.search.type === 'phrase') {
matchQuery.type('phrase');
} else {
matchQuery.operator($scope.search.type);
}
request.query(matchQuery);
} else {
request.query(ejs.MatchAllQuery());
}
The api also facilitates in adding facets to the query. At the moment we support only three facet types: Term, Range and DateHistogram. Of course there are a lot more facets. They all have their own configuration. The range facet needs a number of ranges. The DateHistogram facet needs a period and the term facet does not need a lot. Refer to the api documentation to figure out what all the facets need. Below you can find the code for the three different facets.
var termsFacet = ejs.TermsFacet(facet.field);
termsFacet.field(facet.field);
request.facet(termsFacet);
var rangeFacet = ejs.RangeFacet(facet.field);
for (var j = 0; j < facet.ranges.length; j++) {
var range = facet.ranges[j];
if (range[0] == undefined) {
rangeFacet.addUnboundedTo(range[1]);
} else if (range[1] == undefined) {
rangeFacet.addUnboundedFrom(range[0]);
} else {
rangeFacet.addRange(range[0], range[1]);
}
}
rangeFacet.field(facet.field);
request.facet(rangeFacet);
var dateHistogramFacet = ejs.DateHistogramFacet(facet.field + 'Facet');
dateHistogramFacet.field(facet.field);
dateHistogramFacet.interval(facet.interval);
request.facet(dateHistogramFacet);
Another option you can set on the request is the explain option. If you check this, you get detailed information about the scoring for the specific returned items.
request.explain($scope.search.explain);
The last item I discuss is highlighting. In my plugin I add the same fields you add to limit the returned fields for highlighting. So if you check highlighting, but no fields, nothing will be shown. The most basic options to set for highlighting are in the next code block. A lot of additional configurations are possible. Check the API documentation.
var highlight = ejs.Highlight(); highlight.fields($scope.chosenFields); request.highlight(highlight);
Executing the query
Time to show some results. Executing the query is straightforward. Notice that the hits and the facets are returned as different collections.
request.doSearch(function (results) {
$scope.queryResults = results.hits;
$scope.facetResults = results.facets;
});
Using the results of a query
Using the results with AngularJS is so easy that I do not want to show everything. You get back a json structure. The following code block shows the front-end for showing the facets result if the type is a Term facet.
<div><strong>{{key}} - {{value._type}}</strong></div>
<div ng-show="value._type === 'terms'">
<div>
<span>Total: {{value.total}}</span>
<span>Missing: {{value.missing}}</span>
<span>Other: {{value.other}}</span>
</div>
<div>
<span ng-repeat="term in value.terms"> {{term.term}} ({{term.count}})</span>
</div>
</div>
The last example I want to show is the selected field, the first block if the highlighting option is not selected. The second option if it is selected. Notice that we use other properties when highlighting is enabled.
<table class="table table-condensed" ng-hide="search.highlight">
<tr ng-repeat="(key,value) in doc.fields">
<th>{{key}}</th>
<td>{{value}}</td>
</tr>
</table>
<table class="table table-condensed" ng-show="search.highlight">
<tr ng-repeat="(key,value) in doc.highlight">
<th>{{key}}</th>
<td><p ng-repeat="item in value">{{item}}</p></td>
</tr>
</table>
Concluding
As you can see from the content. Using the elastic.js library is very straightforward. I was impressed by the level of documentation. I am planning to create additional features. The focus will be on visualization. There are some nice plugins to use when showing data coming from facets. I also want to create a more dynamic query tool. With this tool you will get more options for creating queries. Think about combining multiple query types, search by multiple fields, there are a host of opportunities available and I am looking forward to exploring them in the future further too.
