What is a Concept?
A Concept may comprise a group of two or more words that have semantic meaning together or that represent a cluster or group of texts that share the same semantic meaning, e.g., the reason why the user has written a certain text or the core of that text.
Some examples of Concepts are those that are topic-related (e.g., bag-check), action related (e.g., Pay-bill, fix-please), sentiment related (e.g., great-service), emotion related (e.g., Hate-airline), characteristic-related (e.g., Gluten-free, battery-life), brand related (e.g., Trader Joe’s-quality), and product related (e.g., Iphone-x), but others may be utilized.
How are Concepts created? Based on frequency of co-occurrence?
The algorithm that does the Concept Induction is based on measures from Information Theory. Based on the relevance of semantic representations in the whole corpora, the algorithm is able to extract which are the most representative and meaningful contexts using entropy, information gain and divergence measures.
Are Concepts always bigrams?
Concepts don’t have to be bigrams but bigrams are usually a way of contexts disambiguation. Eg “customer” may be ambiguous but “customer+service” is not.
What Concepts are more specific?
Concepts that are more specific have different semantic variations than other Concepts. For example, the Concept password→incorrect is more specific since the dataset has a different semantic variation of password that includes the word incorrect.
Are Concepts that are more specific always unigrams?
Concepts that are more specific do not have to be unigrams. More than one word can be used to make a Concept more specific in some cases. For example, the Concept need+help uses send+email to make the Concept more specific.
How is the Concept name defined?
A Concept may comprise a group of two or more words that have semantic meaning together or that represent a cluster or group of texts that share the same semantic meaning. We define the name using that group of words.
Do the words in the Concept title have to be present in the texts?
Once a Concept, and therefore the semantic relationship between a pair of words or more, is understood by a system of the present invention, such a system can identify similar contexts automatically even if the group of words comprising the Concept are not included in the similar context. By way of example, an inquiry made as to whether a passenger may “take” a “briefcase” onboard may be related to a “bag-check” Concept.
Why are some Concepts more specific than other Concepts?
Some Concepts can be more specific than other Concepts if the dataset has a Concept with many semantic variations.