День сообщества ML - 9 ноября! Присоединяйтесь к нам для обновления от TensorFlow, JAX, и многое другое Подробнее

Классификация текста с помощью RNN

Посмотреть на TensorFlow.org Запускаем в Google Colab Посмотреть исходный код на GitHub Скачать блокнот

Этот текст классификация учебник тренирует рекуррентную нейронную сеть на IMDB большого фильм набора данных обзора для анализа настроений.

Настраивать

import numpy as np

import tensorflow_datasets as tfds
import tensorflow as tf

tfds.disable_progress_bar()

Импорт matplotlib и создать вспомогательную функцию для построения графиков:

import matplotlib.pyplot as plt


def plot_graphs(history, metric):
  plt.plot(history.history[metric])
  plt.plot(history.history['val_'+metric], '')
  plt.xlabel("Epochs")
  plt.ylabel(metric)
  plt.legend([metric, 'val_'+metric])

Настройка входного конвейера

IMDB большой фильм обзор набора данных является бинарной классификации наборов данных-все отзывы имеют положительное или отрицательное чувство.

Загрузить набор данных с помощью TFDS . Смотрите загрузки текста учебника для получения подробной информации о том , как загрузить этот вид данных вручную.

dataset, info = tfds.load('imdb_reviews', with_info=True,
                          as_supervised=True)
train_dataset, test_dataset = dataset['train'], dataset['test']

train_dataset.element_spec
(TensorSpec(shape=(), dtype=tf.string, name=None),
 TensorSpec(shape=(), dtype=tf.int64, name=None))

Первоначально это возвращает набор данных (текст, пары меток):

for example, label in train_dataset.take(1):
  print('text: ', example.numpy())
  print('label: ', label.numpy())
text:  b"This was an absolutely terrible movie. Don't be lured in by Christopher Walken or Michael Ironside. Both are great actors, but this must simply be their worst role in history. Even their great acting could not redeem this movie's ridiculous storyline. This movie is an early nineties US propaganda piece. The most pathetic scenes were those when the Columbian rebels were making their cases for revolutions. Maria Conchita Alonso appeared phony, and her pseudo-love affair with Walken was nothing but a pathetic emotional plug in a movie that was devoid of any real meaning. I am disappointed that there are movies like this, ruining actor's like Christopher Walken's good name. I could barely sit through it."
label:  0

Затем перемешайте данные для подготовки и создание партии этого (text, label) пар:

BUFFER_SIZE = 10000
BATCH_SIZE = 64
train_dataset = train_dataset.shuffle(BUFFER_SIZE).batch(BATCH_SIZE).prefetch(tf.data.AUTOTUNE)
test_dataset = test_dataset.batch(BATCH_SIZE).prefetch(tf.data.AUTOTUNE)
for example, label in train_dataset.take(1):
  print('texts: ', example.numpy()[:3])
  print()
  print('labels: ', label.numpy()[:3])
texts:  [b'Scary Movie 3 (2003) was a bad idea to begin with. The last film was a mediocre effort. Put it next to this load, it\'s a comedy classic. Whilst part two was filled with a lot of dated humor and cheap shots, at least it was funny. There\'s nothing funny about forced humor. Jokes, pratfalls and sight gags are supposed to be naturally funny. Hitting the viewer over the head with tired jokes is not cool. The humor in this film was caters to juvenile imbeciles who\'ll laugh at anything. When they catered to the junior high school crowd, any sense of self respect was tossed out the window. Ring parodies are not funny. I have watched them in comedies since 1998. They\'re so dated. Michael Jackson jokes are not cool either. What\'s even worse is making fun of two broken down has been "performers" whose best days were NEVER.<br /><br />The death of American cinema has been a slow one. Films like this are the nails that are being pounded into it\'s coffin. Whatever happened to real humor? I haven\'t laughed out loud in a movie theater in a long time. Too many bad movies rot the brain. You want proof? Go to your local mega chain video rental store and see what\'s on the shelves. This movie is bad. Don\'t believe the hype. I would rather watch Scary Movie 2 in a continuous loop than to suffer through this poor excuse of a comedy ever again!<br /><br />Definitely not recommended (unless you have a handful of brain cells).'
 b"First of all, I don't understand why some people find this movie so anti-american. Sure, there are moments when the U.S. are accused directly, like at the segments of Youssef Chahine, Ken Loach and, to a certain extent, Mira Nair. But come on, they aren't naive accusations; instead, they are based on real and documented facts, and all the documents that the CIA released about Chile confirms this, for example.<br /><br />But returning to the film itself, what I enjoyed most on it is the variety of moods we find in it. We find children being educated for the respect of the all the people who died in the event; we find a unhappy couple that will be changed by the tragedy of that day; we find common people that have their feelings downgraded on the shadow of the events of September 11 and react differently to this, with dignity or frustration; we even find someone in the movie for who the fall of the towers grounds for a moment of real happiness.<br /><br />All these visions and others - as powerful as these or even more - make a consistent blend and help the spectator to have a glimpse about how different people spread across the world reacted to the events of September 11th. Thus, what we see is a panorama that is much more complex than whites and blacks, and this may make some people infuriated; but this is the world where we live, and in it there is no place for manicheistic ideologies, regardless of what presidents or priests may say us.<br /><br />Finally, I think it's a shame that there isn't even a release date for this movie in the United States of America. It's a shame because most of the american people is asking why this catastrophe happened, and this movie could give some clues to them. This film puts very clearly - differently of what some people of this forum think - that everything we do today will determine our future, and that the errors of the past will affect how we live today."
 b"I'll dispense with the usual comparisons to a certain legendary filmmaker known for his neurotic New Yorker persona, because quite frankly, to draw comparisons with bumbling loser Josh Kornbluth, is just an insult to any such director. I will also avoid mentioning the spot-on satire `Office Space' in the same breath as this celluloid catastrophe. I can, however, compare it to waking up during your own surgery \xc2\x96 it's painful to watch and you wonder whether the surgeons really know what they're doing. Haiku Tunnel is the kind of film you wish they'd pulled the plug on in its early stages of production. It was cruel to let it live and as a result, audiences around the world are being made to suffer.<br /><br />The film's premise \xc2\x96 if indeed it has one \xc2\x96 is not even worth discussing, but for the sake of caution I will. Josh Kornbluth, a temp worker with severe commitment-phobia, is offered a permanent job. His main duty is to mail out 17 high priority letters for his boss. But ludicrously, he is unable to perform this simple task. My reaction? Big deal! That's not a story\xc2\x85 it's a passing thought at best - one that should've passed any self-respecting filmmaker by. <br /><br />The leading actor \xc2\x96 if you can call him that \xc2\x96 is a clumsy buffoon of a man, with chubby features, a receding, untamed hairline, and a series of facial expressions that range from cringe-making to plain disturbing. Where o where did the director find this schmuck? What's that you say\xc2\x85\xc2\x85 he is the director? Oh, my mistake. Playing yourself in your own embarrassment of a screenplay is one thing, but I suspect that Mr Kornbluth isn't that convincing as a human being, let alone an actor. Rest assured, this is by no means an aimless character assassination, but never before have I been so riled up by an actor's on-screen presence! My frustration was further confounded by his incessant to-camera monologues in between scenes. I mean, as if the viewer needs an ounce of intelligence to comprehend this drivel, Kornbluth insults us further by `explaining' the action (first rule of filmmaking: `dramatize exposition'\xc2\x85 show, don't tell). Who does this guy think he is? He has no charisma, no charm, and judging by his Hawaiian shirts, no sense of style. His casting agent should be shot point blank!<br /><br />The supporting actors do nothing to relieve the intense boredom I felt, with but one exception. Patricia Scanlon puts in a very funny appearance as Helen the ex-secretary, who has been driven insane by her old boss, and makes harassing phone calls from her basement, while holding a flashlight under her face. This did make me chuckle to myself, but the moment soon passed and I was back to checking my watch for the remainder of the film.<br /><br />The film's title is also a misnomer. Haiku Tunnel has nothing to do with the ancient form of Japanese poetry. Don't be fooled into thinking this is an art house film because of its pretentious-sounding title or the fact that it only played in a handful of cinemas and made no money at the box office\xc2\x85\xc2\x85\xc2\x85 there's a very good reason for that!<br /><br />"]

labels:  [0 1 0]

Создайте кодировщик текста

Сырец текст загружен tfds должен быть обработан , прежде чем он может быть использован в качестве модели. Самый простой способ обработки текста для обучения с использованием TextVectorization слоя. Этот слой имеет много возможностей, но в этом руководстве используется поведение по умолчанию.

Создайте слой, и передать текст набора данных для слоя .adapt метода:

VOCAB_SIZE = 1000
encoder = tf.keras.layers.TextVectorization(
    max_tokens=VOCAB_SIZE)
encoder.adapt(train_dataset.map(lambda text, label: text))

.adapt метод устанавливает словарный запас слоя. Вот первые 20 жетонов. После заполнения и неизвестных токенов они сортируются по частоте:

vocab = np.array(encoder.get_vocabulary())
vocab[:20]
array(['', '[UNK]', 'the', 'and', 'a', 'of', 'to', 'is', 'in', 'it', 'i',
       'this', 'that', 'br', 'was', 'as', 'for', 'with', 'movie', 'but'],
      dtype='<U14')

Как только словарь установлен, слой может кодировать текст в индексы. Тензоры индексов 0-проложенный до самой длинной последовательности в пакете (если не установлен фиксированный output_sequence_length ):

encoded_example = encoder(example)[:3].numpy()
encoded_example
array([[633,  18, 442, ...,   0,   0,   0],
       [ 86,   5,  32, ...,   0,   0,   0],
       [527,   1,  17, ...,   0,   0,   0]])

При настройках по умолчанию процесс не является полностью обратимым. Для этого есть три основных причины:

  1. Значение по умолчанию для preprocessing.TextVectorization «s standardize аргумент "lower_and_strip_punctuation" .
  2. Ограниченный размер словарного запаса и отсутствие резервных символов на основе символов приводит к появлению некоторых неизвестных токенов.
for n in range(3):
  print("Original: ", example[n].numpy())
  print("Round-trip: ", " ".join(vocab[encoded_example[n]]))
  print()
Original:  b'Scary Movie 3 (2003) was a bad idea to begin with. The last film was a mediocre effort. Put it next to this load, it\'s a comedy classic. Whilst part two was filled with a lot of dated humor and cheap shots, at least it was funny. There\'s nothing funny about forced humor. Jokes, pratfalls and sight gags are supposed to be naturally funny. Hitting the viewer over the head with tired jokes is not cool. The humor in this film was caters to juvenile imbeciles who\'ll laugh at anything. When they catered to the junior high school crowd, any sense of self respect was tossed out the window. Ring parodies are not funny. I have watched them in comedies since 1998. They\'re so dated. Michael Jackson jokes are not cool either. What\'s even worse is making fun of two broken down has been "performers" whose best days were NEVER.<br /><br />The death of American cinema has been a slow one. Films like this are the nails that are being pounded into it\'s coffin. Whatever happened to real humor? I haven\'t laughed out loud in a movie theater in a long time. Too many bad movies rot the brain. You want proof? Go to your local mega chain video rental store and see what\'s on the shelves. This movie is bad. Don\'t believe the hype. I would rather watch Scary Movie 2 in a continuous loop than to suffer through this poor excuse of a comedy ever again!<br /><br />Definitely not recommended (unless you have a handful of brain cells).'
Round-trip:  scary movie 3 [UNK] was a bad idea to begin with the last film was a [UNK] effort put it next to this [UNK] its a comedy classic [UNK] part two was [UNK] with a lot of [UNK] humor and cheap shots at least it was funny theres nothing funny about forced humor jokes [UNK] and [UNK] [UNK] are supposed to be [UNK] funny [UNK] the viewer over the head with [UNK] jokes is not cool the humor in this film was [UNK] to [UNK] [UNK] [UNK] laugh at anything when they [UNK] to the [UNK] high school [UNK] any sense of [UNK] [UNK] was [UNK] out the [UNK] [UNK] [UNK] are not funny i have watched them in [UNK] since [UNK] theyre so [UNK] michael [UNK] jokes are not cool either whats even worse is making fun of two [UNK] down has been [UNK] whose best days were [UNK] br the death of american cinema has been a slow one films like this are the [UNK] that are being [UNK] into its [UNK] whatever happened to real humor i havent [UNK] out [UNK] in a movie theater in a long time too many bad movies [UNK] the [UNK] you want [UNK] go to your local [UNK] [UNK] video [UNK] [UNK] and see whats on the [UNK] this movie is bad dont believe the [UNK] i would rather watch scary movie 2 in a [UNK] [UNK] than to [UNK] through this poor [UNK] of a comedy ever [UNK] br definitely not [UNK] unless you have a [UNK] of [UNK] [UNK]                                                                                                                                                                                                                                                                                                                                                                                                                                                                              

Original:  b"First of all, I don't understand why some people find this movie so anti-american. Sure, there are moments when the U.S. are accused directly, like at the segments of Youssef Chahine, Ken Loach and, to a certain extent, Mira Nair. But come on, they aren't naive accusations; instead, they are based on real and documented facts, and all the documents that the CIA released about Chile confirms this, for example.<br /><br />But returning to the film itself, what I enjoyed most on it is the variety of moods we find in it. We find children being educated for the respect of the all the people who died in the event; we find a unhappy couple that will be changed by the tragedy of that day; we find common people that have their feelings downgraded on the shadow of the events of September 11 and react differently to this, with dignity or frustration; we even find someone in the movie for who the fall of the towers grounds for a moment of real happiness.<br /><br />All these visions and others - as powerful as these or even more - make a consistent blend and help the spectator to have a glimpse about how different people spread across the world reacted to the events of September 11th. Thus, what we see is a panorama that is much more complex than whites and blacks, and this may make some people infuriated; but this is the world where we live, and in it there is no place for manicheistic ideologies, regardless of what presidents or priests may say us.<br /><br />Finally, I think it's a shame that there isn't even a release date for this movie in the United States of America. It's a shame because most of the american people is asking why this catastrophe happened, and this movie could give some clues to them. This film puts very clearly - differently of what some people of this forum think - that everything we do today will determine our future, and that the errors of the past will affect how we live today."
Round-trip:  first of all i dont understand why some people find this movie so [UNK] sure there are moments when the us are [UNK] [UNK] like at the [UNK] of [UNK] [UNK] [UNK] [UNK] and to a certain [UNK] [UNK] [UNK] but come on they arent [UNK] [UNK] instead they are based on real and [UNK] [UNK] and all the [UNK] that the [UNK] released about [UNK] [UNK] this for [UNK] br but [UNK] to the film itself what i enjoyed most on it is the [UNK] of [UNK] we find in it we find children being [UNK] for the [UNK] of the all the people who [UNK] in the [UNK] we find a [UNK] couple that will be [UNK] by the [UNK] of that day we find [UNK] people that have their [UNK] [UNK] on the [UNK] of the events of [UNK] [UNK] and [UNK] [UNK] to this with [UNK] or [UNK] we even find someone in the movie for who the fall of the [UNK] [UNK] for a moment of real [UNK] br all these [UNK] and others as powerful as these or even more make a [UNK] [UNK] and help the [UNK] to have a [UNK] about how different people [UNK] across the world [UNK] to the events of [UNK] [UNK] [UNK] what we see is a [UNK] that is much more [UNK] than [UNK] and [UNK] and this may make some people [UNK] but this is the world where we live and in it there is no place for [UNK] [UNK] [UNK] of what [UNK] or [UNK] may say [UNK] br finally i think its a shame that there isnt even a release [UNK] for this movie in the [UNK] [UNK] of america its a shame because most of the american people is [UNK] why this [UNK] happened and this movie could give some [UNK] to them this film [UNK] very clearly [UNK] of what some people of this [UNK] think that everything we do today will [UNK] our future and that the [UNK] of the past will [UNK] how we live today                                                                                                                                                                                                                                                                                                                                                                                        

Original:  b"I'll dispense with the usual comparisons to a certain legendary filmmaker known for his neurotic New Yorker persona, because quite frankly, to draw comparisons with bumbling loser Josh Kornbluth, is just an insult to any such director. I will also avoid mentioning the spot-on satire `Office Space' in the same breath as this celluloid catastrophe. I can, however, compare it to waking up during your own surgery \xc2\x96 it's painful to watch and you wonder whether the surgeons really know what they're doing. Haiku Tunnel is the kind of film you wish they'd pulled the plug on in its early stages of production. It was cruel to let it live and as a result, audiences around the world are being made to suffer.<br /><br />The film's premise \xc2\x96 if indeed it has one \xc2\x96 is not even worth discussing, but for the sake of caution I will. Josh Kornbluth, a temp worker with severe commitment-phobia, is offered a permanent job. His main duty is to mail out 17 high priority letters for his boss. But ludicrously, he is unable to perform this simple task. My reaction? Big deal! That's not a story\xc2\x85 it's a passing thought at best - one that should've passed any self-respecting filmmaker by. <br /><br />The leading actor \xc2\x96 if you can call him that \xc2\x96 is a clumsy buffoon of a man, with chubby features, a receding, untamed hairline, and a series of facial expressions that range from cringe-making to plain disturbing. Where o where did the director find this schmuck? What's that you say\xc2\x85\xc2\x85 he is the director? Oh, my mistake. Playing yourself in your own embarrassment of a screenplay is one thing, but I suspect that Mr Kornbluth isn't that convincing as a human being, let alone an actor. Rest assured, this is by no means an aimless character assassination, but never before have I been so riled up by an actor's on-screen presence! My frustration was further confounded by his incessant to-camera monologues in between scenes. I mean, as if the viewer needs an ounce of intelligence to comprehend this drivel, Kornbluth insults us further by `explaining' the action (first rule of filmmaking: `dramatize exposition'\xc2\x85 show, don't tell). Who does this guy think he is? He has no charisma, no charm, and judging by his Hawaiian shirts, no sense of style. His casting agent should be shot point blank!<br /><br />The supporting actors do nothing to relieve the intense boredom I felt, with but one exception. Patricia Scanlon puts in a very funny appearance as Helen the ex-secretary, who has been driven insane by her old boss, and makes harassing phone calls from her basement, while holding a flashlight under her face. This did make me chuckle to myself, but the moment soon passed and I was back to checking my watch for the remainder of the film.<br /><br />The film's title is also a misnomer. Haiku Tunnel has nothing to do with the ancient form of Japanese poetry. Don't be fooled into thinking this is an art house film because of its pretentious-sounding title or the fact that it only played in a handful of cinemas and made no money at the box office\xc2\x85\xc2\x85\xc2\x85 there's a very good reason for that!<br /><br />"
Round-trip:  ill [UNK] with the usual [UNK] to a certain [UNK] [UNK] known for his [UNK] new [UNK] [UNK] because quite [UNK] to [UNK] [UNK] with [UNK] [UNK] [UNK] [UNK] is just an [UNK] to any such director i will also avoid [UNK] the [UNK] [UNK] [UNK] space in the same [UNK] as this [UNK] [UNK] i can however [UNK] it to [UNK] up during your own [UNK] – its [UNK] to watch and you wonder whether the [UNK] really know what theyre doing [UNK] [UNK] is the kind of film you wish [UNK] [UNK] the [UNK] on in its early [UNK] of production it was [UNK] to let it live and as a result [UNK] around the world are being made to [UNK] br the films premise – if indeed it has one – is not even worth [UNK] but for the [UNK] of [UNK] i will [UNK] [UNK] a [UNK] [UNK] with [UNK] [UNK] is [UNK] a [UNK] job his main [UNK] is to [UNK] out [UNK] high [UNK] [UNK] for his [UNK] but [UNK] he is [UNK] to [UNK] this simple [UNK] my [UNK] big deal thats not a [UNK] its a [UNK] thought at best one that [UNK] [UNK] any [UNK] [UNK] by br br the leading actor – if you can call him that – is a [UNK] [UNK] of a man with [UNK] features a [UNK] [UNK] [UNK] and a series of [UNK] [UNK] that [UNK] from [UNK] to [UNK] [UNK] where [UNK] where did the director find this [UNK] whats that you [UNK] he is the director oh my [UNK] playing yourself in your own [UNK] of a screenplay is one thing but i [UNK] that mr [UNK] isnt that [UNK] as a human being let alone an actor rest [UNK] this is by no means an [UNK] character [UNK] but never before have i been so [UNK] up by an actors [UNK] [UNK] my [UNK] was [UNK] [UNK] by his [UNK] [UNK] [UNK] in between scenes i mean as if the viewer needs an [UNK] of [UNK] to [UNK] this [UNK] [UNK] [UNK] us [UNK] by [UNK] the action first [UNK] of [UNK] [UNK] [UNK] show dont tell who does this guy think he is he has no [UNK] no [UNK] and [UNK] by his [UNK] [UNK] no sense of style his casting [UNK] should be shot point [UNK] br the supporting actors do nothing to [UNK] the [UNK] [UNK] i felt with but one [UNK] [UNK] [UNK] [UNK] in a very funny [UNK] as [UNK] the [UNK] who has been [UNK] [UNK] by her old [UNK] and makes [UNK] [UNK] [UNK] from her [UNK] while [UNK] a [UNK] under her face this did make me [UNK] to myself but the moment soon [UNK] and i was back to [UNK] my watch for the [UNK] of the filmbr br the films title is also a [UNK] [UNK] [UNK] has nothing to do with the [UNK] form of japanese [UNK] dont be [UNK] into thinking this is an art house film because of its [UNK] title or the fact that it only played in a [UNK] of [UNK] and made no money at the [UNK] [UNK] theres a very good reason for [UNK] br

Создать модель

Чертеж информационного потока в модели

Выше представлена ​​схема модели.

  1. Эта модель может быть построена как tf.keras.Sequential .

  2. Первый слой представляет собой encoder , который преобразует текст в виде последовательности лексем индексов.

  3. После кодировщика идет слой встраивания. Слой внедрения хранит один вектор на слово. При вызове он преобразует последовательности индексов слов в последовательности векторов. Эти векторы можно обучить. После обучения (на достаточном количестве данных) слова с похожими значениями часто имеют похожие векторы.

    Этот индекс-поиска является гораздо более эффективным , чем эквивалентные операции перехода в один горячий кодированный вектор через tf.keras.layers.Dense слой.

  4. Рекуррентная нейронная сеть (RNN) обрабатывает ввод последовательности путем итерации по элементам. RNN передают выходы от одного временного шага к их входу на следующем временном шаге.

    tf.keras.layers.Bidirectional обертка также может быть использована с слоем РНН. Это распространяет ввод вперед и назад через слой RNN, а затем объединяет окончательный вывод.

    • Основное преимущество двунаправленной RNN состоит в том, что сигнал с начала входа не нужно обрабатывать на всем протяжении каждого временного шага, чтобы повлиять на выход.

    • Основным недостатком двунаправленной RNN является то, что вы не можете эффективно передавать предсказания, поскольку слова добавляются в конец.

  5. После того , как РНН преобразовал последовательность в одном вектор два layers.Dense сделать некоторую окончательную обработку, и конвертирование из этого векторного представления к одному логиту в качестве выхода классификации.

Код для реализации этого приведен ниже:

model = tf.keras.Sequential([
    encoder,
    tf.keras.layers.Embedding(
        input_dim=len(encoder.get_vocabulary()),
        output_dim=64,
        # Use masking to handle the variable sequence lengths
        mask_zero=True),
    tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(64)),
    tf.keras.layers.Dense(64, activation='relu'),
    tf.keras.layers.Dense(1)
])

Обратите внимание, что здесь используется последовательная модель Keras, поскольку все слои в модели имеют только один вход и производят один выход. Если вы хотите использовать уровень RNN с отслеживанием состояния, вы можете построить свою модель с помощью функционального API Keras или подкласса модели, чтобы вы могли извлекать и повторно использовать состояния уровня RNN. Пожалуйста , проверьте Keras РНН руководство для получения более подробной информации.

Вложение слой использует маскирующие для обработки различной последовательности длин. Все слои после Embedding поддержки маскирования:

print([layer.supports_masking for layer in model.layers])
[False, True, True, True, True]

Чтобы убедиться, что это работает должным образом, оцените предложение дважды. Во-первых, в одиночку, поэтому нет заполнения для маскировки:

# predict on a sample text without padding.

sample_text = ('The movie was cool. The animation and the graphics '
               'were out of this world. I would recommend this movie.')
predictions = model.predict(np.array([sample_text]))
print(predictions[0])
[0.01226519]

Теперь оцените его еще раз, добавив более длинное предложение. Результат должен быть идентичным:

# predict on a sample text with padding

padding = "the " * 2000
predictions = model.predict(np.array([sample_text, padding]))
print(predictions[0])
[0.01226519]

Скомпилируйте модель Keras для настройки тренировочного процесса:

model.compile(loss=tf.keras.losses.BinaryCrossentropy(from_logits=True),
              optimizer=tf.keras.optimizers.Adam(1e-4),
              metrics=['accuracy'])

Обучите модель

history = model.fit(train_dataset, epochs=10,
                    validation_data=test_dataset,
                    validation_steps=30)
Epoch 1/10
391/391 [==============================] - 44s 89ms/step - loss: 0.6525 - accuracy: 0.5551 - val_loss: 0.5196 - val_accuracy: 0.7312
Epoch 2/10
391/391 [==============================] - 31s 78ms/step - loss: 0.4593 - accuracy: 0.7856 - val_loss: 0.4159 - val_accuracy: 0.8276
Epoch 3/10
391/391 [==============================] - 32s 79ms/step - loss: 0.3769 - accuracy: 0.8395 - val_loss: 0.3691 - val_accuracy: 0.8385
Epoch 4/10
391/391 [==============================] - 33s 82ms/step - loss: 0.3399 - accuracy: 0.8547 - val_loss: 0.3643 - val_accuracy: 0.8370
Epoch 5/10
391/391 [==============================] - 33s 83ms/step - loss: 0.3214 - accuracy: 0.8629 - val_loss: 0.3367 - val_accuracy: 0.8531
Epoch 6/10
391/391 [==============================] - 32s 81ms/step - loss: 0.3118 - accuracy: 0.8675 - val_loss: 0.3414 - val_accuracy: 0.8620
Epoch 7/10
391/391 [==============================] - 32s 80ms/step - loss: 0.3075 - accuracy: 0.8697 - val_loss: 0.3385 - val_accuracy: 0.8578
Epoch 8/10
391/391 [==============================] - 33s 83ms/step - loss: 0.3022 - accuracy: 0.8731 - val_loss: 0.3317 - val_accuracy: 0.8484
Epoch 9/10
391/391 [==============================] - 32s 81ms/step - loss: 0.3000 - accuracy: 0.8742 - val_loss: 0.3241 - val_accuracy: 0.8557
Epoch 10/10
391/391 [==============================] - 31s 77ms/step - loss: 0.2972 - accuracy: 0.8732 - val_loss: 0.3216 - val_accuracy: 0.8552
test_loss, test_acc = model.evaluate(test_dataset)

print('Test Loss:', test_loss)
print('Test Accuracy:', test_acc)
391/391 [==============================] - 15s 38ms/step - loss: 0.3170 - accuracy: 0.8618
Test Loss: 0.3169540762901306
Test Accuracy: 0.861840009689331
plt.figure(figsize=(16, 8))
plt.subplot(1, 2, 1)
plot_graphs(history, 'accuracy')
plt.ylim(None, 1)
plt.subplot(1, 2, 2)
plot_graphs(history, 'loss')
plt.ylim(0, None)
(0.0, 0.6703125551342964)

PNG

Сделайте прогноз на основе нового предложения:

Если прогноз> = 0,0, он положительный, иначе он отрицательный.

sample_text = ('The movie was cool. The animation and the graphics '
               'were out of this world. I would recommend this movie.')
predictions = model.predict(np.array([sample_text]))

Сложите два или более слоев LSTM

Keras рецидивирующий слои имеют два режима работы , которые управляются с помощью return_sequences аргумента конструктора:

  • Если False возвращает только последний выход для каждой входной последовательности (2D - тензор формы (batch_size, output_features)). Это значение по умолчанию, используемое в предыдущей модели.

  • Если True , полные последовательности последовательных выходов для каждого временного шага возвращается (3D - тензор формы (batch_size, timesteps, output_features) ).

Вот что поток информации выглядит как с return_sequences=True :

Layered_bidirectional

Интересная вещь об использовании RNN с return_sequences=True в том , что выход все же имеет 3-оси, как на входе, так что он может быть передан на другой слой РНН, как это:

model = tf.keras.Sequential([
    encoder,
    tf.keras.layers.Embedding(len(encoder.get_vocabulary()), 64, mask_zero=True),
    tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(64,  return_sequences=True)),
    tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(32)),
    tf.keras.layers.Dense(64, activation='relu'),
    tf.keras.layers.Dropout(0.5),
    tf.keras.layers.Dense(1)
])
model.compile(loss=tf.keras.losses.BinaryCrossentropy(from_logits=True),
              optimizer=tf.keras.optimizers.Adam(1e-4),
              metrics=['accuracy'])
history = model.fit(train_dataset, epochs=10,
                    validation_data=test_dataset,
                    validation_steps=30)
Epoch 1/10
391/391 [==============================] - 73s 152ms/step - loss: 0.6501 - accuracy: 0.5531 - val_loss: 0.4503 - val_accuracy: 0.7943
Epoch 2/10
391/391 [==============================] - 54s 137ms/step - loss: 0.3922 - accuracy: 0.8259 - val_loss: 0.3550 - val_accuracy: 0.8448
Epoch 3/10
391/391 [==============================] - 53s 135ms/step - loss: 0.3395 - accuracy: 0.8548 - val_loss: 0.3345 - val_accuracy: 0.8500
Epoch 4/10
391/391 [==============================] - 53s 135ms/step - loss: 0.3222 - accuracy: 0.8645 - val_loss: 0.3248 - val_accuracy: 0.8578
Epoch 5/10
391/391 [==============================] - 54s 137ms/step - loss: 0.3158 - accuracy: 0.8662 - val_loss: 0.3232 - val_accuracy: 0.8604
Epoch 6/10
391/391 [==============================] - 52s 131ms/step - loss: 0.3056 - accuracy: 0.8698 - val_loss: 0.3233 - val_accuracy: 0.8573
Epoch 7/10
391/391 [==============================] - 52s 130ms/step - loss: 0.3032 - accuracy: 0.8712 - val_loss: 0.3197 - val_accuracy: 0.8526
Epoch 8/10
391/391 [==============================] - 51s 130ms/step - loss: 0.3041 - accuracy: 0.8688 - val_loss: 0.3210 - val_accuracy: 0.8552
Epoch 9/10
391/391 [==============================] - 54s 136ms/step - loss: 0.3007 - accuracy: 0.8726 - val_loss: 0.3178 - val_accuracy: 0.8589
Epoch 10/10
391/391 [==============================] - 55s 138ms/step - loss: 0.2931 - accuracy: 0.8745 - val_loss: 0.3416 - val_accuracy: 0.8656
test_loss, test_acc = model.evaluate(test_dataset)

print('Test Loss:', test_loss)
print('Test Accuracy:', test_acc)
391/391 [==============================] - 26s 66ms/step - loss: 0.3409 - accuracy: 0.8628
Test Loss: 0.3409377634525299
Test Accuracy: 0.8628000020980835
# predict on a sample text without padding.

sample_text = ('The movie was not good. The animation and the graphics '
               'were terrible. I would not recommend this movie.')
predictions = model.predict(np.array([sample_text]))
print(predictions)
[[-1.9127669]]
plt.figure(figsize=(16, 6))
plt.subplot(1, 2, 1)
plot_graphs(history, 'accuracy')
plt.subplot(1, 2, 2)
plot_graphs(history, 'loss')

PNG

Проверьте другие существующие повторяющиеся слои , такие как ГРУ слоев .

Если вы interestied в создании пользовательской RNNs см РНН Руководство Keras .