Elasticsearch is an open-source search and analytics engine where you can gather, process, store, analyze, and visualize significant volumes of real-time data.
Why use Elasticsearch?
- Search: The main advantage of using Elasticsearch is its rapid and accurate search functionality. For large datasets, relational databases take much more time for search queries because of the number of joins the query has to go through.
- Scaling: The distributed architecture of Elasticsearch allows you to scale many servers and data. We can scale the clusters to hundreds of nodes and replicate data to prevent data loss in case of a node failure.
- Analytical engine: Elasticsearch analytical use case has become more popular than the search use case. Elasticsearch is specifically used for log analysis.
To learn more about Elasticsearch, read our blog post here: What is Elasticsearch?
This blog post will follow a step-by-step guide to building Elasticsearch queries using Knowi’s query builder.
Setting up Elasticsearch Datasource in Knowi
In this section, we will set up the Elasticsearch data source using a few configuration steps. This can be used to build queries once it is set up.
Note: To proceed further, you must be signed up with Knowi. Start your free trial now by visiting Knowi.
Once you’re signed in to Knowi, click Queries on the left side panel.
On the Queries page, click New Datasource+.
In the next window, a list of datasources is displayed. Click on Elasticsearch in the NoSQL Datasources.
In the next window, the datasource config details can be entered. Knowi prepopulates with standard information that can be used.
Click Test Connection to make sure the connection is working. A successful toast message appears if the connection is successful. Click Save.
Once saved, a Start Querying button is enabled. Click on it to query the Elasticsearch datasource.
Querying the Elasticsearch Datasource
In this section, we can see how we can build Elasticsearch queries. Once the datasource is set up, you can directly start querying by clicking the Start Querying button. Or navigate to the queries page, click New Query+, and select the datasource from the datasource dropdown menu.
Once the datasource is set, Move on to the Settings tab.
Settings
In this tab, we will set up the Elasticsearch datasource for querying.
In the Indexes section, choose an index from the dropdown list.
In the Types, choose one from the dropdown to determine the fields. You can select all or specific fields.
In the Metrics, choose the metrics to display. You can also set up labels and operations for the metrics. Click Save.
Next, you can choose the dimensions to group by in the Group by dropdown.
You can sort the records in Ascending or descending order using the Sort option.
You can also set a record display limit in the Limits box.
You can set filters by choosing fields in the dataset to filter out in the Filters dropdown.
Preview the Results
Once the settings are in place, you can preview the results using the preview button.
This is what the preview looks like:
Data Strategy
In the data strategy, you can define how your query can run. There are three types to choose from:
- Direct Query
- Run Once
- Scheduled Intervals
Direct Query – This executes directly on the original datasource and bypasses the warehouse. When a widget is displayed, it will fetch data in real time from the underlying datastore.
Run Once – Query executes only once
Scheduled Intervals – Runs the query at specified intervals. The results are stored in Knowi, and the visualizations/alerts, etc., are driven by the Knowi datastore.
In the next Settings tab, specify the Dataset Name, category, and description.
Click Create & Run to create the query and run.
Now, you are redirected to the dataset page with further options for creating Visualization, AI Dashboard, and much more.
This is how we set up Elasticsearch and query it using Knowi’s query builder.
What Knowi Brings to the Table
Apart from the native support for Elasticsearch, Knowi additionally offers the below features to support Elasticsearch queries for better insights into your data.
Highlighting
Elasticsearch highlighting allows users to retrieve search results with portions of a text (usually called snippets) highlighted to indicate where the search query matched within the document or field. This is particularly useful in search applications where users need to quickly identify relevant information within large volumes of text-based data.
For example,
To get highlights for the content field in each search, hit using the default highlighter and include a highlight object in the request body that specifies the content field.
GET /_search
{
"query": {
"match": { "content": "kimchy" }
},
"highlight": {
"fields": {
"content": {}
}
}
}Native Integration
Knowi provides a native connection to Elasticsearch, allowing users to query Elasticsearch indices directly using Elasticsearch DSL (Domain Specific Language) queries or leveraging Looker’s SQL-like interface (LookML). As we saw in the example in the blog, it is easy to connect with Elasticsearch datasource and query it using the visual builder or c9QL.
Querying ES Templates
Query the Elasticsearch template in Knowi by allowing the end-user queries to pass as parameters into the search template. This prevents your query-building logic from being exposed to the end user. Learn How.
Indexing
Indexing in Elasticsearch refers to storing and organizing data within Elasticsearch indices to be efficiently searched, retrieved, and analyzed. It is fundamental to Elasticsearch’s ability to provide fast and efficient search capabilities across large volumes of structured and unstructured data. Properly configured indexing and mapping strategies ensure that Elasticsearch indices are optimized for the specific needs of the application or use case, supporting rapid data retrieval and analysis. Knowi supports seamless indexing of Elasticsearch indices.
At Knowi
Knowi, a Business Intelligence (BI) platform, unifies analytics across structured, unstructured, and semi-structured data. Knowi has firmly established itself as a leader in the Elasticsearch and OpenSearch domains. Knowi also provides Elasticsearch analytics; give it a try if you want to elevate your business data with insights, embedded analytics, and visualizations. Make sure to contact us for any of your database queries. Book a demo today with Knowi and embark on a transformative analytics journey.
