Esta página foi traduzida pela API Cloud Translation.
Switch to English

Classificação de texto com um RNN

Ver no TensorFlow.org Executar no Google Colab Ver fonte no GitHub Baixar caderno

Este tutorial de classificação de texto treina uma rede neural recorrente no grande conjunto de dados de revisão de filmes IMDB para análise de sentimento.

Configuração

pip install -q tensorflow_datasets
import numpy as np

import tensorflow_datasets as tfds
import tensorflow as tf

tfds.disable_progress_bar()

Importe matplotlib e crie uma função auxiliar para traçar gráficos:

import matplotlib.pyplot as plt

def plot_graphs(history, metric):
  plt.plot(history.history[metric])
  plt.plot(history.history['val_'+metric], '')
  plt.xlabel("Epochs")
  plt.ylabel(metric)
  plt.legend([metric, 'val_'+metric])

Configurar pipeline de entrada

O grande conjunto de dados de resenhas de filmes do IMDB é um conjunto de dados de classificação binária - todas as resenhas têm um sentimento positivo ou negativo .

Baixe o conjunto de dados usando TFDS . Consulte o tutorial de carregamento de texto para obter detalhes sobre como carregar esse tipo de dados manualmente.

dataset, info = tfds.load('imdb_reviews', with_info=True,
                          as_supervised=True)
train_dataset, test_dataset = dataset['train'], dataset['test']

train_dataset.element_spec
(TensorSpec(shape=(), dtype=tf.string, name=None),
 TensorSpec(shape=(), dtype=tf.int64, name=None))

Inicialmente, isso retorna um conjunto de dados de (texto, pares de rótulos):

for example, label in train_dataset.take(1):
  print('text: ', example.numpy())
  print('label: ', label.numpy())
text:  b"This was an absolutely terrible movie. Don't be lured in by Christopher Walken or Michael Ironside. Both are great actors, but this must simply be their worst role in history. Even their great acting could not redeem this movie's ridiculous storyline. This movie is an early nineties US propaganda piece. The most pathetic scenes were those when the Columbian rebels were making their cases for revolutions. Maria Conchita Alonso appeared phony, and her pseudo-love affair with Walken was nothing but a pathetic emotional plug in a movie that was devoid of any real meaning. I am disappointed that there are movies like this, ruining actor's like Christopher Walken's good name. I could barely sit through it."
label:  0

Em seguida, embaralhe os dados para treinamento e crie lotes desses pares (text, label) :

BUFFER_SIZE = 10000
BATCH_SIZE = 64
train_dataset = train_dataset.shuffle(BUFFER_SIZE).batch(BATCH_SIZE).prefetch(tf.data.AUTOTUNE)
test_dataset = test_dataset.batch(BATCH_SIZE).prefetch(tf.data.AUTOTUNE)
for example, label in train_dataset.take(1):
  print('texts: ', example.numpy()[:3])
  print()
  print('labels: ', label.numpy()[:3])
texts:  [b'Cult of the Cobra is now available on DVD in a pristine print that does full justice to whatever merits it has as a movie. Unfortunately, that is not saying much.<br /><br />It has a competent cast of second-rankers that acquit themselves as well as could be expected under the circumstances. It is efficiently directed, entirely on sound stages and standing sets on the studio backlot. It looks OK, but is ponderously over-plotted and at a scant 80 minutes it is still heavily padded.<br /><br />For example, the double cobra attack on the first of the GIs was surely one attack too many.<br /><br />The business about Julia choosing to marry Pete rather than Tom never amounts to anything. Tom immediately falls in love with Lisa and she never has any reason to be jealous of Julia (nor is she).<br /><br />Julia\'s \'feminine intuition\' is introduced as if it is going to lead to an important plot development, but it doesn\'t. Similarly, Pete\'s investigation into cobra cults and the suspicion that briefly falls on Tom serve no purpose other than to fill up screen time.<br /><br />These are just symptoms of the underlying problem. The movie is structured like a mystery but it isn\'t. As soon as the curse is pronounced we know exactly where the story is heading, so the characters are left painstakingly uncovering what we already know.<br /><br />The ending is particularly lame. Julia is menaced purely by accident. Lisa has no reason to want to kill her - she just happens to be in the wrong place at the wrong time. When Tom turns up in the nick of time to save her, it is not even clear whether she was threatened at all. He then simply disposes of the cobra in the way any of the previous victims might have done.<br /><br />It is such an inconsequential little pipsqueak of a story that I found myself wondering how on earth it had been pitched to the studio heads. Then it occurred to me. Someone said: "Those Val Lewton movies were very successful over at RKO, so why don\'t we make one like that?"<br /><br />Cult of the Cobra is clearly modelled on Cat People: mysterious, troubled, shape-shifting woman falls in love with the hero, is apparently frigid, kills people, arouses the suspicions of the hero\'s woman friend and dies at the end. But \'modelled on\' doesn\'t mean \'as good as\' - by a wide margin. It copies, but doesn\'t understand what it is copying.<br /><br />It is obviously trying for the low-key, suggestive Lewton style, but this approach doesn\'t follow through into the story. Lisa is no Irene. She is meant to be strange and mysterious but there is no mystery about her. We get a glimpse of her after the first attack in Asia, so immediately recognise her when she turns up in New York. There is never any doubt about her purpose. Neither is there any ambiguity about whether of not she actually turns into a snake.<br /><br />Then again, during her nocturnal prowling we get, not one, but two attempts at \'buses\'. Neither come off, because the director doesn\'t understand what makes a \'bus\' work and, in any case, they happen to the stalker, not the person being stalked.<br /><br />These faint echoes of Cat People give Cult of the Cobra whatever small distinction it might have, but they only draw attention to the yawning gulf between the original and the imitation.<br /><br />Plagiarism may be the sincerest form of flattery, but I doubt if Lewton or Tourneur were particularly flattered when this tepid little time-passer came out.'
 b'as an actor I really like independent films but this one is amateur at best.<br /><br />The boys go to Vermont for a civil service yet when the plane lands it flies over a palm tree - were the directors aware that palm trees are not in Vermont? Pines yes - palms no. And the same for the wedding service - again nice grove of palm trees.<br /><br />When the boys are leaving VT they apparently could not get a ticket on any major airline since the plane that is filmed is Federal Express. Did they ship themselves Overnight in a crate? Come on guys little details like this separate an indi film from totally amateur.<br /><br />The Christian brother is far gayer than Arthur with his bleached hair and tribal band tattoo. The two should have switched roles.<br /><br />The minor characters are laughable and overact something terrible.<br /><br />Applause to the directors for making a gay film but pay some attention to your locations and casting next time'
 b'"Come Undone" appears to elicit a lot of opinions among the contributors to this forum. Granted, it\'s a film that promises a take on gay life, as most viewers expect and somehow, it gets away from that promise into an introspective view at a young man\'s soul. The film has a way of staying with us even when it has ended. It is a character study about how a young man gets involved into a love affair with someone so much different than him that, in the end, will leave Mathieu confused, hurt and depressed when things don\'t go according to what he hoped the relationship would be.<br /><br />If you haven\'t seen the film, perhaps you would like to stop reading.<br /><br />Sebastien Lifshitz, the director of the film, has told his story from Mathieu\'s viewpoint. Most viewers appear to be disoriented by the different times within the film, but there are hints that are not obvious, as one can see, in retrospect. The story is told in flashbacks that might add to the way some people will view the film. This is a story about the doomed the love Mathieu felt for Cedric and the ultimate breakdown of their life together.<br /><br />First of all, Cedric, the handsome young local, pursues Mathieu until he succeeds in convincing him he likes him. Mathieu feels the attraction for Cedric too. We realize how different both young men are by the way Cedric tells Mathieu\'s family how he feels school is not for him. On the other hand, Mathieu, who wants to be an architect, finds beauty in the abandoned place where Cedric has taken him. We watch as Mathieu, reading from the guide book, wants Cedric\'s attention.<br /><br />When Mathieu comes out to his mother, she wisely tells him about the importance of continuing his career. She also points out about what future both of them would have together, which proves to be true. Mathieu appears to have learned his lesson, the hard way. He goes on to an uncertain life with Cedric and attempts to take his own life. We watch him in the hospital speaking to a psychiatrist that has treated his wounded soul.<br /><br />The ending might be confusing for most viewers, but there is a moment in the film when Mathieu goes to work in a bar where we see him washing glasses and looking intently to Pierre, the young man who frequents the bar. That is why when Mathieu goes looking for Pierre at his house, appears to be hard to imagine. Yet, we have seen the way Mathieu is obviously interested in Pierre. The last scene at the beach, when Pierre and Mathieu are seen strolling in the sand, has a hopeful sign that things will be better between them as they watch a young boy, apparently lost, but then realizing the father is nearby.<br /><br />Jeremie Elkaim makes Mathieu one of the most complex characters in recent films. This is a young man who is hard to understand on a simple level. Mathieu has suffered a lot, first with the separation of his parents, then with his depressed mother and with losing Cedric. Stephan Rideau, who has been seen on other important French films, is equally good, as the shallow Cedric.<br /><br />While "Come Undone" will divide opinions, the film deserves a viewing because of the complexity and the care Sebastien Lifshitz gives to the story.']

labels:  [0 0 1]

Crie o codificador de texto

O texto bruto carregado pelo tfds precisa ser processado antes de ser usado em um modelo. A maneira mais simples de processar texto para treinamento é usando a camada experimental.preprocessing.TextVectorization . Esta camada tem muitos recursos, mas este tutorial mantém o comportamento padrão.

Crie a camada e passe o texto do conjunto de dados para o método .adapt da camada:

VOCAB_SIZE=1000
encoder = tf.keras.layers.experimental.preprocessing.TextVectorization(
    max_tokens=VOCAB_SIZE)
encoder.adapt(train_dataset.map(lambda text, label: text))

O método .adapt define o vocabulário da camada. Aqui estão os primeiros 20 tokens. Após o preenchimento e os tokens desconhecidos, eles são classificados por frequência:

vocab = np.array(encoder.get_vocabulary())
vocab[:20]
array(['', '[UNK]', 'the', 'and', 'a', 'of', 'to', 'is', 'in', 'it', 'i',
       'this', 'that', 'br', 'was', 'as', 'for', 'with', 'movie', 'but'],
      dtype='<U14')

Depois que o vocabulário é definido, a camada pode codificar o texto em índices. Os tensores de índices são preenchidos com 0 até a sequência mais longa do lote (a menos que você defina um output_sequence_length fixo):

encoded_example = encoder(example)[:3].numpy()
encoded_example
array([[  1,   5,   2, ...,   1, 366,  46],
       [ 15,  34, 288, ...,   0,   0,   0],
       [209,   1, 717, ...,   0,   0,   0]])

Com as configurações padrão, o processo não é totalmente reversível. Existem três razões principais para isso:

  1. O valor padrão para o argumento de standardize preprocessing.TextVectorization é "lower_and_strip_punctuation" .
  2. O tamanho limitado do vocabulário e a falta de fallback baseado em caracteres resultam em alguns tokens desconhecidos.
for n in range(3):
  print("Original: ", example[n].numpy())
  print("Round-trip: ", " ".join(vocab[encoded_example[n]]))
  print()
Original:  b'Cult of the Cobra is now available on DVD in a pristine print that does full justice to whatever merits it has as a movie. Unfortunately, that is not saying much.<br /><br />It has a competent cast of second-rankers that acquit themselves as well as could be expected under the circumstances. It is efficiently directed, entirely on sound stages and standing sets on the studio backlot. It looks OK, but is ponderously over-plotted and at a scant 80 minutes it is still heavily padded.<br /><br />For example, the double cobra attack on the first of the GIs was surely one attack too many.<br /><br />The business about Julia choosing to marry Pete rather than Tom never amounts to anything. Tom immediately falls in love with Lisa and she never has any reason to be jealous of Julia (nor is she).<br /><br />Julia\'s \'feminine intuition\' is introduced as if it is going to lead to an important plot development, but it doesn\'t. Similarly, Pete\'s investigation into cobra cults and the suspicion that briefly falls on Tom serve no purpose other than to fill up screen time.<br /><br />These are just symptoms of the underlying problem. The movie is structured like a mystery but it isn\'t. As soon as the curse is pronounced we know exactly where the story is heading, so the characters are left painstakingly uncovering what we already know.<br /><br />The ending is particularly lame. Julia is menaced purely by accident. Lisa has no reason to want to kill her - she just happens to be in the wrong place at the wrong time. When Tom turns up in the nick of time to save her, it is not even clear whether she was threatened at all. He then simply disposes of the cobra in the way any of the previous victims might have done.<br /><br />It is such an inconsequential little pipsqueak of a story that I found myself wondering how on earth it had been pitched to the studio heads. Then it occurred to me. Someone said: "Those Val Lewton movies were very successful over at RKO, so why don\'t we make one like that?"<br /><br />Cult of the Cobra is clearly modelled on Cat People: mysterious, troubled, shape-shifting woman falls in love with the hero, is apparently frigid, kills people, arouses the suspicions of the hero\'s woman friend and dies at the end. But \'modelled on\' doesn\'t mean \'as good as\' - by a wide margin. It copies, but doesn\'t understand what it is copying.<br /><br />It is obviously trying for the low-key, suggestive Lewton style, but this approach doesn\'t follow through into the story. Lisa is no Irene. She is meant to be strange and mysterious but there is no mystery about her. We get a glimpse of her after the first attack in Asia, so immediately recognise her when she turns up in New York. There is never any doubt about her purpose. Neither is there any ambiguity about whether of not she actually turns into a snake.<br /><br />Then again, during her nocturnal prowling we get, not one, but two attempts at \'buses\'. Neither come off, because the director doesn\'t understand what makes a \'bus\' work and, in any case, they happen to the stalker, not the person being stalked.<br /><br />These faint echoes of Cat People give Cult of the Cobra whatever small distinction it might have, but they only draw attention to the yawning gulf between the original and the imitation.<br /><br />Plagiarism may be the sincerest form of flattery, but I doubt if Lewton or Tourneur were particularly flattered when this tepid little time-passer came out.'
Round-trip:  [UNK] of the [UNK] is now [UNK] on dvd in a [UNK] [UNK] that does full [UNK] to whatever [UNK] it has as a movie unfortunately that is not saying [UNK] br it has a [UNK] cast of [UNK] that [UNK] themselves as well as could be expected under the [UNK] it is [UNK] directed [UNK] on sound [UNK] and [UNK] sets on the [UNK] [UNK] it looks ok but is [UNK] [UNK] and at a [UNK] [UNK] minutes it is still [UNK] [UNK] br for example the [UNK] [UNK] [UNK] on the first of the [UNK] was [UNK] one [UNK] too [UNK] br the business about [UNK] [UNK] to [UNK] [UNK] rather than tom never [UNK] to anything tom [UNK] falls in love with [UNK] and she never has any reason to be [UNK] of [UNK] nor is [UNK] br [UNK] [UNK] [UNK] is [UNK] as if it is going to lead to an important plot development but it doesnt [UNK] [UNK] [UNK] into [UNK] [UNK] and the [UNK] that [UNK] falls on tom [UNK] no [UNK] other than to [UNK] up screen [UNK] br these are just [UNK] of the [UNK] problem the movie is [UNK] like a mystery but it isnt as soon as the [UNK] is [UNK] we know exactly where the story is [UNK] so the characters are left [UNK] [UNK] what we already [UNK] br the ending is particularly lame [UNK] is [UNK] [UNK] by [UNK] [UNK] has no reason to want to kill her she just happens to be in the wrong place at the wrong time when tom turns up in the [UNK] of time to save her it is not even clear whether she was [UNK] at all he then simply [UNK] of the [UNK] in the way any of the previous [UNK] might have [UNK] br it is such an [UNK] little [UNK] of a story that i found myself [UNK] how on earth it had been [UNK] to the [UNK] [UNK] then it [UNK] to me someone said those [UNK] [UNK] movies were very [UNK] over at [UNK] so why dont we make one like [UNK] br [UNK] of the [UNK] is clearly [UNK] on [UNK] people [UNK] [UNK] [UNK] woman falls in love with the hero is apparently [UNK] [UNK] people [UNK] the [UNK] of the [UNK] woman friend and [UNK] at the end but [UNK] on doesnt mean as good as by a [UNK] [UNK] it [UNK] but doesnt understand what it is [UNK] br it is obviously trying for the [UNK] [UNK] [UNK] style but this [UNK] doesnt follow through into the story [UNK] is no [UNK] she is meant to be strange and [UNK] but there is no mystery about her we get a [UNK] of her after the first [UNK] in [UNK] so [UNK] [UNK] her when she turns up in new york there is never any doubt about her [UNK] [UNK] is there any [UNK] about whether of not she actually turns into a [UNK] br then again during her [UNK] [UNK] we get not one but two attempts at [UNK] [UNK] come off because the director doesnt understand what makes a [UNK] work and in any case they happen to the [UNK] not the person being [UNK] br these [UNK] [UNK] of [UNK] people give [UNK] of the [UNK] whatever small [UNK] it might have but they only [UNK] attention to the [UNK] [UNK] between the original and the [UNK] br [UNK] may be the [UNK] form of [UNK] but i doubt if [UNK] or [UNK] were particularly [UNK] when this [UNK] little [UNK] came out

Original:  b'as an actor I really like independent films but this one is amateur at best.<br /><br />The boys go to Vermont for a civil service yet when the plane lands it flies over a palm tree - were the directors aware that palm trees are not in Vermont? Pines yes - palms no. And the same for the wedding service - again nice grove of palm trees.<br /><br />When the boys are leaving VT they apparently could not get a ticket on any major airline since the plane that is filmed is Federal Express. Did they ship themselves Overnight in a crate? Come on guys little details like this separate an indi film from totally amateur.<br /><br />The Christian brother is far gayer than Arthur with his bleached hair and tribal band tattoo. The two should have switched roles.<br /><br />The minor characters are laughable and overact something terrible.<br /><br />Applause to the directors for making a gay film but pay some attention to your locations and casting next time'
Round-trip:  as an actor i really like [UNK] films but this one is [UNK] at [UNK] br the boys go to [UNK] for a [UNK] [UNK] yet when the [UNK] [UNK] it [UNK] over a [UNK] [UNK] were the directors [UNK] that [UNK] [UNK] are not in [UNK] [UNK] yes [UNK] no and the same for the [UNK] [UNK] again nice [UNK] of [UNK] [UNK] br when the boys are [UNK] [UNK] they apparently could not get a [UNK] on any major [UNK] since the [UNK] that is filmed is [UNK] [UNK] did they [UNK] themselves [UNK] in a [UNK] come on guys little [UNK] like this [UNK] an [UNK] film from totally [UNK] br the [UNK] brother is far [UNK] than [UNK] with his [UNK] [UNK] and [UNK] [UNK] [UNK] the two should have [UNK] [UNK] br the [UNK] characters are [UNK] and [UNK] something [UNK] br [UNK] to the directors for making a [UNK] film but pay some attention to your [UNK] and casting next time                                                                                                                                                                                                                                                                                                                                                                                                                                                     

Original:  b'"Come Undone" appears to elicit a lot of opinions among the contributors to this forum. Granted, it\'s a film that promises a take on gay life, as most viewers expect and somehow, it gets away from that promise into an introspective view at a young man\'s soul. The film has a way of staying with us even when it has ended. It is a character study about how a young man gets involved into a love affair with someone so much different than him that, in the end, will leave Mathieu confused, hurt and depressed when things don\'t go according to what he hoped the relationship would be.<br /><br />If you haven\'t seen the film, perhaps you would like to stop reading.<br /><br />Sebastien Lifshitz, the director of the film, has told his story from Mathieu\'s viewpoint. Most viewers appear to be disoriented by the different times within the film, but there are hints that are not obvious, as one can see, in retrospect. The story is told in flashbacks that might add to the way some people will view the film. This is a story about the doomed the love Mathieu felt for Cedric and the ultimate breakdown of their life together.<br /><br />First of all, Cedric, the handsome young local, pursues Mathieu until he succeeds in convincing him he likes him. Mathieu feels the attraction for Cedric too. We realize how different both young men are by the way Cedric tells Mathieu\'s family how he feels school is not for him. On the other hand, Mathieu, who wants to be an architect, finds beauty in the abandoned place where Cedric has taken him. We watch as Mathieu, reading from the guide book, wants Cedric\'s attention.<br /><br />When Mathieu comes out to his mother, she wisely tells him about the importance of continuing his career. She also points out about what future both of them would have together, which proves to be true. Mathieu appears to have learned his lesson, the hard way. He goes on to an uncertain life with Cedric and attempts to take his own life. We watch him in the hospital speaking to a psychiatrist that has treated his wounded soul.<br /><br />The ending might be confusing for most viewers, but there is a moment in the film when Mathieu goes to work in a bar where we see him washing glasses and looking intently to Pierre, the young man who frequents the bar. That is why when Mathieu goes looking for Pierre at his house, appears to be hard to imagine. Yet, we have seen the way Mathieu is obviously interested in Pierre. The last scene at the beach, when Pierre and Mathieu are seen strolling in the sand, has a hopeful sign that things will be better between them as they watch a young boy, apparently lost, but then realizing the father is nearby.<br /><br />Jeremie Elkaim makes Mathieu one of the most complex characters in recent films. This is a young man who is hard to understand on a simple level. Mathieu has suffered a lot, first with the separation of his parents, then with his depressed mother and with losing Cedric. Stephan Rideau, who has been seen on other important French films, is equally good, as the shallow Cedric.<br /><br />While "Come Undone" will divide opinions, the film deserves a viewing because of the complexity and the care Sebastien Lifshitz gives to the story.'
Round-trip:  come [UNK] appears to [UNK] a lot of [UNK] among the [UNK] to this [UNK] [UNK] its a film that [UNK] a take on [UNK] life as most viewers expect and somehow it gets away from that [UNK] into an [UNK] view at a young [UNK] [UNK] the film has a way of [UNK] with us even when it has [UNK] it is a character [UNK] about how a young man gets involved into a love [UNK] with someone so much different than him that in the end will leave [UNK] [UNK] [UNK] and [UNK] when things dont go [UNK] to what he [UNK] the relationship would [UNK] br if you havent seen the film perhaps you would like to stop [UNK] br [UNK] [UNK] the director of the film has told his story from [UNK] [UNK] most viewers appear to be [UNK] by the different times within the film but there are [UNK] that are not obvious as one can see in [UNK] the story is told in [UNK] that might add to the way some people will view the film this is a story about the [UNK] the love [UNK] felt for [UNK] and the [UNK] [UNK] of their life [UNK] br first of all [UNK] the [UNK] young local [UNK] [UNK] until he [UNK] in [UNK] him he [UNK] him [UNK] feels the [UNK] for [UNK] too we realize how different both young men are by the way [UNK] tells [UNK] family how he feels school is not for him on the other hand [UNK] who wants to be an [UNK] finds beauty in the [UNK] place where [UNK] has taken him we watch as [UNK] reading from the [UNK] book wants [UNK] [UNK] br when [UNK] comes out to his mother she [UNK] tells him about the [UNK] of [UNK] his career she also points out about what future both of them would have together which [UNK] to be true [UNK] appears to have [UNK] his [UNK] the hard way he goes on to an [UNK] life with [UNK] and attempts to take his own life we watch him in the [UNK] [UNK] to a [UNK] that has [UNK] his [UNK] [UNK] br the ending might be [UNK] for most viewers but there is a moment in the film when [UNK] goes to work in a [UNK] where we see him [UNK] [UNK] and looking [UNK] to [UNK] the young man who [UNK] the [UNK] that is why when [UNK] goes looking for [UNK] at his house appears to be hard to imagine yet we have seen the way [UNK] is obviously interested in [UNK] the last scene at the [UNK] when [UNK] and [UNK] are seen [UNK] in the [UNK] has a [UNK] [UNK] that things will be better between them as they watch a young boy apparently lost but then [UNK] the father is [UNK] br [UNK] [UNK] makes [UNK] one of the most [UNK] characters in [UNK] films this is a young man who is hard to understand on a simple level [UNK] has [UNK] a lot first with the [UNK] of his parents then with his [UNK] mother and with [UNK] [UNK] [UNK] [UNK] who has been seen on other important french films is [UNK] good as the [UNK] [UNK] br while come [UNK] will [UNK] [UNK] the film deserves a viewing because of the [UNK] and the care [UNK] [UNK] gives to the story                               


Crie o modelo

Um desenho do fluxo de informações no modelo

Acima está um diagrama do modelo.

  1. Este modelo pode ser construído como um tf.keras.Sequential .

  2. A primeira camada é o encoder , que converte o texto em uma sequência de índices de token.

  3. Depois que o codificador é uma camada de incorporação. Uma camada de incorporação armazena um vetor por palavra. Quando chamado, ele converte as sequências de índices de palavras em sequências de vetores. Esses vetores são treináveis. Após o treinamento (com dados suficientes), palavras com significados semelhantes geralmente têm vetores semelhantes.

    Essa pesquisa de índice é muito mais eficiente do que a operação equivalente de passar um vetor codificado one-hot por uma camada tf.keras.layers.Dense .

  4. Uma rede neural recorrente (RNN) processa a entrada de sequência iterando através dos elementos. Os RNNs passam as saídas de um passo de tempo para sua entrada no próximo passo de tempo.

    O wrapper tf.keras.layers.Bidirectional também pode ser usado com uma camada RNN. Isso propaga a entrada para a frente e para trás através da camada RNN e então concatena a saída final.

    • A principal vantagem de um RNN bidirecional é que o sinal do início da entrada não precisa ser processado em todos os passos de tempo para afetar a saída.

    • A principal desvantagem de um RNN bidirecional é que você não pode transmitir previsões de forma eficiente enquanto as palavras são adicionadas ao final.

  5. Depois que o RNN converteu a sequência em um único vetor, as duas layers.Dense . layers.Dense fazem algum processamento final e convertem essa representação vetorial em um único logit como a saída de classificação.

O código para implementar isso está abaixo:

model = tf.keras.Sequential([
    encoder,
    tf.keras.layers.Embedding(
        input_dim=len(encoder.get_vocabulary()),
        output_dim=64,
        # Use masking to handle the variable sequence lengths
        mask_zero=True),
    tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(64)),
    tf.keras.layers.Dense(64, activation='relu'),
    tf.keras.layers.Dense(1)
])

Observe que o modelo sequencial Keras é usado aqui, uma vez que todas as camadas no modelo têm apenas uma entrada e produzem uma saída única. Caso queira usar a camada RNN com estado, você pode querer construir seu modelo com a API funcional Keras ou subclasse de modelo para que possa recuperar e reutilizar os estados da camada RNN. Consulte o guia Keras RNN para obter mais detalhes.

A camada de incorporação usa o mascaramento para lidar com os comprimentos de sequência variáveis. Todas as camadas após o mascaramento de suporte de Embedding :

print([layer.supports_masking for layer in model.layers])
[False, True, True, True, True]

Para confirmar que isso funciona conforme o esperado, avalie uma frase duas vezes. Primeiro, sozinho, para que não haja preenchimento para mascarar:

# predict on a sample text without padding.

sample_text = ('The movie was cool. The animation and the graphics '
               'were out of this world. I would recommend this movie.')
predictions = model.predict(np.array([sample_text]))
print(predictions[0])
[-0.02237528]

Agora, avalie novamente em um lote com uma frase mais longa. O resultado deve ser idêntico:

# predict on a sample text with padding

padding = "the " * 2000
predictions = model.predict(np.array([sample_text, padding]))
print(predictions[0])
[-0.02237528]

Compile o modelo Keras para configurar o processo de treinamento:

model.compile(loss=tf.keras.losses.BinaryCrossentropy(from_logits=True),
              optimizer=tf.keras.optimizers.Adam(1e-4),
              metrics=['accuracy'])

Treine o modelo

history = model.fit(train_dataset, epochs=10,
                    validation_data=test_dataset, 
                    validation_steps=30)
Epoch 1/10
391/391 [==============================] - 44s 95ms/step - loss: 0.6858 - accuracy: 0.5052 - val_loss: 0.5634 - val_accuracy: 0.7193
Epoch 2/10
391/391 [==============================] - 34s 87ms/step - loss: 0.4815 - accuracy: 0.7763 - val_loss: 0.4102 - val_accuracy: 0.8328
Epoch 3/10
391/391 [==============================] - 34s 87ms/step - loss: 0.3718 - accuracy: 0.8415 - val_loss: 0.3441 - val_accuracy: 0.8542
Epoch 4/10
391/391 [==============================] - 34s 86ms/step - loss: 0.3363 - accuracy: 0.8604 - val_loss: 0.3365 - val_accuracy: 0.8594
Epoch 5/10
391/391 [==============================] - 34s 87ms/step - loss: 0.3221 - accuracy: 0.8638 - val_loss: 0.3395 - val_accuracy: 0.8583
Epoch 6/10
391/391 [==============================] - 34s 86ms/step - loss: 0.3133 - accuracy: 0.8692 - val_loss: 0.3307 - val_accuracy: 0.8583
Epoch 7/10
391/391 [==============================] - 34s 87ms/step - loss: 0.3106 - accuracy: 0.8687 - val_loss: 0.3240 - val_accuracy: 0.8599
Epoch 8/10
391/391 [==============================] - 34s 86ms/step - loss: 0.3014 - accuracy: 0.8759 - val_loss: 0.3178 - val_accuracy: 0.8630
Epoch 9/10
391/391 [==============================] - 33s 84ms/step - loss: 0.2971 - accuracy: 0.8747 - val_loss: 0.3278 - val_accuracy: 0.8562
Epoch 10/10
391/391 [==============================] - 33s 85ms/step - loss: 0.2982 - accuracy: 0.8731 - val_loss: 0.3194 - val_accuracy: 0.8656

test_loss, test_acc = model.evaluate(test_dataset)

print('Test Loss: {}'.format(test_loss))
print('Test Accuracy: {}'.format(test_acc))
391/391 [==============================] - 16s 42ms/step - loss: 0.3157 - accuracy: 0.8639
Test Loss: 0.315724641084671
Test Accuracy: 0.8639199733734131

plt.figure(figsize=(16,8))
plt.subplot(1,2,1)
plot_graphs(history, 'accuracy')
plt.ylim(None,1)
plt.subplot(1,2,2)
plot_graphs(history, 'loss')
plt.ylim(0,None)
(0.0, 0.6800670087337494)

png

Faça uma previsão em uma nova frase:

Se a previsão for> = 0,0, é positiva, caso contrário, é negativa.

sample_text = ('The movie was cool. The animation and the graphics '
               'were out of this world. I would recommend this movie.')
predictions = model.predict(np.array([sample_text]))

Empilhe duas ou mais camadas LSTM

As camadas recorrentes de Keras têm dois modos disponíveis que são controlados pelo argumento do construtor return_sequences :

  • Se False , retorna apenas a última saída para cada sequência de entrada (um tensor 2D de forma (batch_size, output_features)). Este é o padrão, usado no modelo anterior.

  • Se True as sequências completas de saídas sucessivas para cada passo de tempo são retornadas (um tensor 3D de forma (batch_size, timesteps, output_features) ).

Aqui está a aparência do fluxo de informações com return_sequences=True :

layered_bidirectional

O interessante de usar um RNN com return_sequences=True é que a saída ainda tem 3 eixos, como a entrada, então pode ser passada para outra camada RNN, assim:

model = tf.keras.Sequential([
    encoder,
    tf.keras.layers.Embedding(len(encoder.get_vocabulary()), 64, mask_zero=True),
    tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(64,  return_sequences=True)),
    tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(32)),
    tf.keras.layers.Dense(64, activation='relu'),
    tf.keras.layers.Dropout(0.5),
    tf.keras.layers.Dense(1)
])
model.compile(loss=tf.keras.losses.BinaryCrossentropy(from_logits=True),
              optimizer=tf.keras.optimizers.Adam(1e-4),
              metrics=['accuracy'])
history = model.fit(train_dataset, epochs=10,
                    validation_data=test_dataset,
                    validation_steps=30)
Epoch 1/10
391/391 [==============================] - 80s 164ms/step - loss: 0.6861 - accuracy: 0.5069 - val_loss: 0.4562 - val_accuracy: 0.7812
Epoch 2/10
391/391 [==============================] - 58s 149ms/step - loss: 0.4216 - accuracy: 0.8140 - val_loss: 0.3823 - val_accuracy: 0.8146
Epoch 3/10
391/391 [==============================] - 59s 150ms/step - loss: 0.3561 - accuracy: 0.8470 - val_loss: 0.3346 - val_accuracy: 0.8396
Epoch 4/10
391/391 [==============================] - 59s 150ms/step - loss: 0.3264 - accuracy: 0.8627 - val_loss: 0.3252 - val_accuracy: 0.8500
Epoch 5/10
391/391 [==============================] - 58s 149ms/step - loss: 0.3225 - accuracy: 0.8629 - val_loss: 0.3498 - val_accuracy: 0.8271
Epoch 6/10
391/391 [==============================] - 58s 149ms/step - loss: 0.3055 - accuracy: 0.8738 - val_loss: 0.3250 - val_accuracy: 0.8568
Epoch 7/10
391/391 [==============================] - 58s 149ms/step - loss: 0.3110 - accuracy: 0.8679 - val_loss: 0.3410 - val_accuracy: 0.8422
Epoch 8/10
391/391 [==============================] - 58s 149ms/step - loss: 0.2991 - accuracy: 0.8724 - val_loss: 0.3189 - val_accuracy: 0.8573
Epoch 9/10
391/391 [==============================] - 60s 154ms/step - loss: 0.2977 - accuracy: 0.8736 - val_loss: 0.3241 - val_accuracy: 0.8505
Epoch 10/10
391/391 [==============================] - 60s 153ms/step - loss: 0.2976 - accuracy: 0.8727 - val_loss: 0.3212 - val_accuracy: 0.8557

test_loss, test_acc = model.evaluate(test_dataset)

print('Test Loss: {}'.format(test_loss))
print('Test Accuracy: {}'.format(test_acc))
391/391 [==============================] - 28s 72ms/step - loss: 0.3196 - accuracy: 0.8563
Test Loss: 0.31963279843330383
Test Accuracy: 0.8562800288200378

# predict on a sample text without padding.

sample_text = ('The movie was not good. The animation and the graphics '
                    'were terrible. I would not recommend this movie.')
predictions = model.predict(np.array([sample_text]))
print(predictions)
[[-1.7240916]]

plt.figure(figsize=(16,6))
plt.subplot(1,2,1)
plot_graphs(history, 'accuracy')
plt.subplot(1,2,2)
plot_graphs(history, 'loss')

png

Verifique outras camadas recorrentes existentes, como camadas GRU .

Se você estiver interessado em construir RNNs personalizados, consulte o Guia Keras RNN .