View source on GitHub |
Converts a text to a sequence of words (or tokens).
tf.keras.preprocessing.text.text_to_word_sequence(
input_text,
filters='!"#$%&()*+,-./:;<=>?@[\\]^_`{|}~\t\n',
lower=True,
split=' '
)
This function transforms a string of text into a list of words
while ignoring filters
which include punctuations by default.
sample_text = 'This is a sample sentence.'
tf.keras.preprocessing.text.text_to_word_sequence(sample_text)
['this', 'is', 'a', 'sample', 'sentence']
Returns | |
---|---|
A list of words (or tokens). |