Search-Based Analytics / Search-Driven Analytics
You can think of search-based analytics, sometimes referred to as “search driven analytics”, as a search engine for your company data. Search-based analytics is the ability to ask questions like “what was our net revenue last quarter” or “show me how many people downloaded our app in the last 30 days sorted by week” and get back actionable data and charts.
Why is search-based analytics needed?
In the last decade, business intelligence tools have given business users the power to create their own dashboards and visualizations. This has aided in the ability of executives and department heads to make data-driven decisions to support company goals. But the ability to make use of these dashboards and visualizations is limited by issues like data governance, data silos and data sprawl, and ability of the individual users.
You can think of search-based analytics as a search engine for your company data
Not just that but in order to use a data analytics dashboard the dashboard must first be created. Data must be pulled in, visualizations must be selected and applied, and the dashboard must be designed in a way that answers specific business questions. This process can take hours or even days.
Search-based analytics, on the other hand, allows a business user to ask questions in plain English and get back actionable data and visualizations immediately.
But there are challenges to building an effective search-driven analytics platform. For example, in order for it to work effectively, search-based analytics platforms need to understand terms like “compare”, “how many”, “average”, and “total”. They also need to understand business-specific terms like “growth”, “retention”, “churn”, and “cost”. It is also crucially important that they be able to interpret a lot of different ways of saying dates or spans of time, such as “from Sept to Dec”, “last quarter”, “display by month”, and “last year”. This is done with natural language processing.
Natural Language Processing (NLP)
Natural Language Processing (NLP) first emerged in the 1950s but only recently has reached a point where the technology is broadly applicable. As you can imagine, getting a computer to understand human languages was a major hurdle — just think about the complexities and nuances of English.
The job of NLP is to convert that human language question into a query that the analytics engine can understand
In the past decade, this has changed significantly. Google search, Wolfram Alpha, Amazon Alexa, and Apple’s Siri have all taken great strides in using NLP to provide real services to people.
In the case of search-based analytics, this can be even more complicated. There are many ways to ask for a particular type of data or visualization. And the job of NLP is to convert that human language question into a query that the analytics engine can understand.
Customizing search-based analytics for specific use-cases
In addition to the general challenges presented by applying NLP to data to create search-based analytics, there are use-case specific challenges.
If you survey how 100 companies are using business intelligence tools, you are likely to get 100 different answers. That’s because all industries work with different data. Even individual companies within the same industry are likely tracking different metrics and KPIs (key performance indicators). Many industries also have different terminology and domain-specific knowledge that has to be — at least at a cursory level — understood when included in natural language queries.
This is why it is essential in such cases for search-based analytics deployments to be customized by the provider for the customer’s specific use-case. This process often involves adding custom terms and synonyms to the NLP library, mapping relationships, adjusting variable names, and customizing the visualization selection logic.
Example: What does this look like in practice?
Imagine a refrigerated trucking company that uses GPS and IoT (internet of things) temperature sensors to feed monitoring data into their business intelligence platform. They will want to know things like where their trucks are, how fast they are going, and what the temperature inside the refrigeration compartments.
That means a search-based analytics platform would need to understand questions like “show me the locations and temperatures of all trucks in Arizona”, “how cold is rig number 5”, and “are there any units above freezing?”. The first query may work off-the-shelf but the second and third examples would probably require some customization to be interpreted correctly.
Is it really as easy as Googling something?
Short answer: no. But a better answer might be “not yet.” If you remember the days of Web Crawler, Alta Vista, and Excite, you know that early search engines weren’t great at getting people exactly what they were looking for.
Building search engines for data is an even more difficult challenge, especially since — as mentioned above — it differs so much from company to company.
However, there is still hope. That easy Google-like performance and accuracy can be achieved when a search-based analytics solution is customized for a particular business or use-case. It requires a little bit of work upfront, but the results are potentially a game-changer.
