NLUToken
An individual token — i.e. a word, punctuation symbol, whitespace, etc.
Implements
Fields
Boolean)
A boolean value indicating whether a word vector is associated with the object.
String)
Verbatim text content.
String)
Text content, with trailing space character if present.
[Float])
A real-valued meaning representation.
Float)
The L2 norm of the document’s vector representation.
String)
Syntactic dependency relation.
String)
Named entity type
Boolean)
Does the token consist of alphabetic characters?
Boolean)
Does the token consist of ASCII characters?
Boolean)
Is the token a bracket?
Boolean)
Is the token a currency symbol?
Boolean)
Does the token consist of digits?
Boolean)
Is the token a left punctuation mark, e.g. "(" ?
Boolean)
Is the token in lowercase?
Boolean)
Is the token out-of-vocabulary (i.e. does it not have a word vector)?
Boolean)
Is the token punctuation?
Boolean)
Is the token a quotation mark?
Boolean)
Is the token a right punctuation mark, e.g. ")" ?
Boolean)
A boolean value indicating whether the token starts a sentence
Boolean)
Does the token consist of whitespace characters?
Boolean)
Is the token part of a “stop list”?
Boolean)
Is the token in titlecase?
Boolean)
Is the token in uppercase?
String)
Base form of the token, with no inflectional suffixes.
Boolean)
Does the token resemble an email address?
Boolean)
Does the token represent a number? e.g. "10.9", "10", "ten", etc.
Boolean)
Does the token resemble a URL?
Float)
Smoothed log probability estimate of token's word type (context-independent entry in the vocabulary).
String)
The token's norm, i.e. a normalized form of the token text
String)
Coarse-grained part-of-speech.
[NLUToken])
A sequence containing the token and all the token’s syntactic descendants.
String)
Fine-grained part-of-speech.