Natural Language Processing

Knowi's Natural Language capabilities is a powerful way to enable self-service analytics to non-technical users, by asking questions in plain English to drive insights and visualizations quickly. This empowers any member and individual the ability to make better, data-driven decisions, any time.

Natural Language capabilities are available for use across all datasets and widgets within a dashboard. It can be accessed using the Analyze icon on a widget. As you explore your data with natural language, Knowi recommends questions based on your input and the data source available. The technology works by translating your natural language query to Cloud9QL statement to resolve your request.

NLP Across Datasets

Using natural language processing at the dashboard queries across all datasets within your account. Auto-complete suggestions will be recommended as you type based on the associated dataset. For example, if you had multiple datasets powering a single sales and marketing dashboard, and you wanted answer on the latest marketing information, typing your query matches the requested fields and aggregation to the datasets the user has access to.

Natural Language Processing

When using NLP across datasets, you can explicitly request which datasets/query to use by incorporating "from" at the end of your query. For example,

what is the average revenue by sales_representation by region from product sales (prod)

From the example above, product sales (prod) is the name of the dataset/query containing the sales information.

NLP From

Enable NLP Usage Across Datasets

Using NLP across datasets is a customer user feature that must be enabled by an admin-role. To enable this feature for users:

  1. From the Knowi panel, click Settings and navigate to the User Settings page
  2. In the Account Settings tab, click the NLP ACCESS switch to ENABLE/DISABLE the feature

Enable NLP Dash

NLP Within Widgets

Unlike across datasets, for use within widgets, natural language queries are applied directly on the dataset powering that widget.

NLP is available on all widgets inside a dashboard from the widget menu and available to all users, regardless of user role/permission.

NLP Widget

Dataset Tags

You can tag your dataset fields with text strings making it easier to execute NLP queries. These tags do not change your data model but instead are used in Knowi services such as NLP.

For example, if your data model contains the following fields employeefirstname, employeelastname, sales, department. You can add one or more tags to synonymize select fields i.e. department can hold the following tags: team, group. While employeelastname and employeefirstname share the tags: associates, rep

To add tags:

  1. Select the dataset. On any widget, click More Settings -> Data Diagram
  2. Click NLP tab. Under Synonyms, select the fields, enter tags, then click Save.
  3. Decide how to query with NLP: on a widget, within a dashboard, or across all datasets 

NLP Widget

Examples

The following are examples of natural language query to get you started:

Sample field selection
"bounced, sent, customer"

"bounced and sent and customer"

"Show me all for Wells Fargo"

"all for customer Wells Fargo"

"all fields for customer like Wells Fargo"

"bounced and sent for Wells Fargo"
Aggregations
"Sum of sent sum of open by customer"

"Total sent by customer by week"

"Total sent, Total bounced for Wells Fargo"

"Average Sent, Total bounced by customer by campaign_name for newsletter"
Dates
"all for date after January this year"

"average close monthly"

"close by date for january this year"

"close by date between january and march this year"

"From 2022 to 2024"

"Between 2022 and 2024"

"Since 2024, From 2024, Until 2024" 

"In 2024"

"from June 1st to Jun 4th"

Note: If the year is not specified, it assumes the current year.

NLP Ranking

In Natural Language Processing, ranking often refers to ordering or prioritizing items based on their relevance to a particular query or context. In the Knowi NLP prompt, users can rank their data in ascending or descending orders, fetch the top five entries, etc. 

Here are a few examples: 

"Top 5 names for (total/average) opens"

"Rank name ascending/desc/asc/descending"

"Top 5 names since September last year"

"Arrange the names in ascending/ descending order"

"Rank by name"

NLP Ranking

Edit Search-Based Analytics

Using Knowi's Search-based Analytics available on the Dataset page, the settings related to natural language queries can be edited for the following:

NLP Indexer: Useful when you want to exclude a dataset from search-based analytics/NLP. By default, it's on. Turn off indexing for NLP.

NLP Slack Indexer: Turn off indexing for slack NLP. By default, it's on.

Synonyms: Specify one or more synonyms that can be used in NLP for the Dataset. Note: Users can add more than one set of synonyms and also remove them. To add, click +; to remove, click -.

Data Limit: Limit the number of records to process for NLP. Set to 0 for unlimited. Defaults to 200K when empty.

Skip NLP Processing: Specify one or more fields that will be skipped in NLP for the Dataset.

Exact Match Fields: Specify one or more text fields that will be exactly matched including punctuations and spaces. By default, we will default to fuzzy matching. Exact matches can also boost performance.

Default NLP Date: Specify the default NLP date to fetch in NLP processing for the Dataset. A random date is selected for NLP when empty.

Indexable Fields: Gives you control over the ability to set unique values on string fields. These fields are often used as a part of a condition.

Index values from another Dataset: Pointing to other datasets gives you fine-grained values that can be driven from another dataset. The values are determined for the first column of the dataset, or if the dataset has multiple columns, the column name matches the field to index. By default, the values are sampled from the dataset, if non-direct.

NLP Widget

Alternatively, you can navigate to the edit settings of the Search-Based Analytics by clicking on the edit button on the dataset within the dataset lineage diagram.

NLP Widget

Extended Support For Synonyms

Multiple keywords can be associated with a field. Instead of a field, you can also specify calculations or Cloud9QL functions. For example, if the bounced rate per million is defined as (bounced/sent)*1000000 set the definition as a field name and set bounced rate per million as the synonym.

NLP Widget

Configure Runtime Parameters for Synonyms

You can use runtime parameters into the synonyms where a user can specify/define one or more parameters with a value in the synonyms fields. It will help the users to return the results in an optimized and quicker way for a particular given NLP query.

It allows the users to change runtime param values upon on your NLP query but the idea of changing the runtime params is that the user can define params once in query settings and then use them in the widget Analyze screen (which appears when the user applies the NLP query on the widget) and then the user can fine-tune values of runtime params.

For example, you can pass the runtime parameters (bounced/$c9_someval$(100)$[Threshold]$) + 5 in the synonyms field and provide any related label to this param named as bouncedThresholdQuery as illustrated below.

"bounced" is a field name from source data. "$c9_sometoken$" is defined as the c9 runtime param. The default value and label are optional. The 100 is the default value and "Threshold" is a label in the sample above.

Note: After applying the NLP query synonyms, you can change the "Threshold" value. It will reset the data and instantly return the latest result on your screen based on your executed input values query.

NLP Widget

Embedding Search-based Analytics (NLP)

Using Knowi's Search Analytics can be embedded in your application through Single Sign-On. Try it out.

Similar to embedding with SSO and the Javascript API, the steps to embedding Search Analytics is almost identical and can be achieved in three steps:

1. User Token: Create a Knowi user token for a new or existing user

curl -i -X POST -d "user=email@domain.com&ssoCustomerToken=<SSO_TOKEN>" https://knowi.com/sso/user/create

2. Session Token: Create a Knowi session token with the user token from step 1

curl -i -X POST -d "user=email@domain.com&userToken=<USER_TOKEN>" https://knowi.com/sso/session/create

3. Embed: Write a Javascript file with the session token returned from step 2

function loadSearchAnalytics() {
    Knowi.render('#knowi-div', {
        type: 'nlp',
        token: "BYhXXCeu0Ego8zA1FqtLOOM4BDWot5AYBLI7PWrQTbsie",
        url: 'https://www.knowi.com',
        view: {
            nlpOptions: {
                icon: "off",
                css: {
                    background: 'transparent'
                },
                placeholder: "Try: total sales for canada this week",
            }
        }
    }, function () {
    });
}

Note: For embedding Search Analytics, type must be nlp

Search-Analytics Resources:

  • Javascript API Reference | link
  • Search Analytics code samples | link


Natural Language Processing Settings

You can edit the natural language processing settings given under the user settings to configure NLP Across Datasets, Index By Default, NLP Bot Integration, and more.

NLP Across Dataset: Enables natural language interface across datasets.

NLP Bot Integration: Enables natural language interface for slack/teams.

NLP Favorites: Configure list of NLP queries per category type displayed when a user types in /knowi. Typically, this would be the most commonly used questions. The format can be as follows:

Category1
Category2
Category3
/knowi yournlpquestion1
/knowi yournlpquestion2

NLP Synonyms: Configure list of NLP synonyms. The format can be as follows:

FieldName1=Synonym1,Synonym2,Synonym3
FieldName2=Synonym1,Synonym2,Synonym3

Note: Users can add more than one set of synonyms and also remove them. To add, click +; to remove, click -.

NLP Widget

Slack Integration

Slack is where the people you need, the information you share, and the tools you use come together to get things done. With Knowi's Slack integration, its powerful Search-based analytics (or Natural Language Processing) capabilities combine with Slack's intuitive messaging UI, allowing you to ask questions from your Knowi data directly from the Slack app and get answers back instantly. This allows you to obtain quick insights from your data, optimize by integrating it into your workflows, and enhance collaboration within your team.

For instructions on using Knowi's Slack integration as an Admin, click here.

For instructions on using Knowi's Slack integration as a User, click here.

MS Teams Integration

MS teams allow you and your team to stay informed, organized, and connected all the time, no matter where you are. Knowi integration with Microsoft Teams' combines Knowi's strong search-based analytics capabilities with Teams' user-friendly interface. With this, users are now able to query their data, receive responses, and view those responses within Teams.

For instructions on using Knowi's Slack integration as an Admin, click here.

For instructions on using Knowi's Slack integration as a User, click here.