 View on TensorFlow.org
|
 Run in Google Colab
|
 View source on GitHub
|
 Download notebook
|
Overview
This tutorial focuses on streaming data from a Kafka cluster into a tf.data.Dataset which is then used in conjunction with tf.keras for training and inference.
Kafka is primarily a distributed event-streaming platform which provides scalable and fault-tolerant streaming data across data pipelines. It is an essential technical component of a plethora of major enterprises where mission-critical data delivery is a primary requirement.
Setup
Install the required tensorflow-io and kafka packages
pip install tensorflow-iopip install kafka-python
Import packages
import os
from datetime import datetime
import time
import threading
import json
from kafka import KafkaProducer
from kafka.errors import KafkaError
from sklearn.model_selection import train_test_split
import pandas as pd
import tensorflow as tf
import tensorflow_io as tfio
Validate tf and tfio imports
print("tensorflow-io version: {}".format(tfio.__version__))
print("tensorflow version: {}".format(tf.__version__))
tensorflow-io version: 0.23.1 tensorflow version: 2.8.0-rc0
Download and setup Kafka and Zookeeper instances
For demo purposes, the following instances are setup locally:
- Kafka (Brokers: 127.0.0.1:9092)
- Zookeeper (Node: 127.0.0.1:2181)
curl -sSOL https://downloads.apache.org/kafka/2.7.2/kafka_2.13-2.7.2.tgztar -xzf kafka_2.13-2.7.2.tgz
Using the default configurations (provided by Apache Kafka) for spinning up the instances.
./kafka_2.13-2.7.2/bin/zookeeper-server-start.sh -daemon ./kafka_2.13-2.7.2/config/zookeeper.properties./kafka_2.13-2.7.2/bin/kafka-server-start.sh -daemon ./kafka_2.13-2.7.2/config/server.propertiesecho "Waiting for 10 secs until kafka and zookeeper services are up and running"sleep 10
Waiting for 10 secs until kafka and zookeeper services are up and running
Once the instances are started as daemon processes, grep for kafka in the processes list. The two java processes correspond to zookeeper and the kafka instances.
ps -ef | grep kafkakbuilder 27856 20044 4 20:28 ? 00:00:00 python /tmpfs/src/gfile/executor.py --input_notebook=/tmpfs/src/temp/docs/tutorials/kafka.ipynb --timeout=15000 kbuilder 28271 1 16 20:28 ? 00:00:01 java -Xmx512M -Xms512M -server -XX:+UseG1GC -XX:MaxGCPauseMillis=20 -XX:InitiatingHeapOccupancyPercent=35 -XX:+ExplicitGCInvokesConcurrent -XX:MaxInlineLevel=15 -Djava.awt.headless=true -Xlog:gc*:file=/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../logs/zookeeper-gc.log:time,tags:filecount=10,filesize=100M -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dkafka.logs.dir=/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../logs -Dlog4j.configuration=file:./kafka_2.13-2.7.2/bin/../config/log4j.properties -cp /tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/activation-1.1.1.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/aopalliance-repackaged-2.6.1.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/argparse4j-0.7.0.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/audience-annotations-0.5.0.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/commons-cli-1.4.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/commons-lang3-3.8.1.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/connect-api-2.7.2.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/connect-basic-auth-extension-2.7.2.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/connect-file-2.7.2.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/connect-json-2.7.2.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/connect-mirror-2.7.2.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/connect-mirror-client-2.7.2.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/connect-runtime-2.7.2.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/connect-transforms-2.7.2.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/hk2-api-2.6.1.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/hk2-locator-2.6.1.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/hk2-utils-2.6.1.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/jackson-annotations-2.10.5.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/jackson-core-2.10.5.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/jackson-databind-2.10.5.1.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/jackson-dataformat-csv-2.10.5.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/jackson-datatype-jdk8-2.10.5.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/jackson-jaxrs-base-2.10.5.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/jackson-jaxrs-json-provider-2.10.5.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/jackson-module-jaxb-annotations-2.10.5.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/jackson-module-paranamer-2.10.5.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/jackson-module-scala_2.13-2.10.5.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/jakarta.activation-api-1.2.1.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/jakarta.annotation-api-1.3.5.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/jakarta.inject-2.6.1.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/jakarta.validation-api-2.0.2.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/jakarta.ws.rs-api-2.1.6.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/jakarta.xml.bind-api-2.3.2.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/javassist-3.25.0-GA.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/javassist-3.26.0-GA.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/javax.servlet-api-3.1.0.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/javax.ws.rs-api-2.1.1.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/jaxb-api-2.3.0.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/jersey-client-2.34.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/jersey-common-2.34.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/jersey-container-servlet-2.34.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/jersey-container-servlet-core-2.34.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/jersey-hk2-2.34.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/jersey-server-2.34.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/jetty-client-9.4.43.v20210629.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/jetty-continuation-9.4.43.v20210629.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/jetty-http-9.4.43.v20210629.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/jetty-io-9.4.43.v20210629.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/jetty-security-9.4.43.v20210629.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/jetty-server-9.4.43.v20210629.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/jetty-servlet-9.4.43.v20210629.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/jetty-servlets-9.4.43.v20210629.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/jetty-util-9.4.43.v20210629.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/jetty-util-ajax-9.4.43.v20210629.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/jopt-simple-5.0.4.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/kafka-clients-2.7.2.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/kafka-log4j-appender-2.7.2.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/kafka-raft-2.7.2.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/kafka-streams-2.7.2.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/kafka-streams-examples-2.7.2.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/kafka-streams-scala_2.13-2.7.2.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/kafka-streams-test-utils-2.7.2.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/kafka-tools-2.7.2.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/kafka_2.13-2.7.2-sources.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/kafka_2.13-2.7.2.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/log4j-1.2.17.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/lz4-java-1.7.1.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/maven-artifact-3.8.1.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/metrics-core-2.2.0.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/netty-buffer-4.1.59.Final.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/netty-codec-4.1.59.Final.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/netty-common-4.1.59.Final.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/netty-handler-4.1.59.Final.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/netty-resolver-4.1.59.Final.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/netty-transport-4.1.59.Final.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/netty-transport-native-epoll-4.1.59.Final.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/netty-transport-native-unix-common-4.1.59.Final.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/osgi-resource-locator-1.0.3.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/paranamer-2.8.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/plexus-utils-3.2.1.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/reflections-0.9.12.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/rocksdbjni-5.18.4.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/scala-collection-compat_2.13-2.2.0.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/scala-java8-compat_2.13-0.9.1.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/scala-library-2.13.3.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/scala-logging_2.13-3.9.2.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/scala-reflect-2.13.3.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/slf4j-api-1.7.30.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/slf4j-log4j12-1.7.30.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/snappy-java-1.1.7.7.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/zookeeper-3.5.9.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/zookeeper-jute-3.5.9.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/zstd-jni-1.4.5-6.jar org.apache.zookeeper.server.quorum.QuorumPeerMain ./kafka_2.13-2.7.2/config/zookeeper.properties kbuilder 28635 1 57 20:28 ? 00:00:05 java -Xmx1G -Xms1G -server -XX:+UseG1GC -XX:MaxGCPauseMillis=20 -XX:InitiatingHeapOccupancyPercent=35 -XX:+ExplicitGCInvokesConcurrent -XX:MaxInlineLevel=15 -Djava.awt.headless=true -Xlog:gc*:file=/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../logs/kafkaServer-gc.log:time,tags:filecount=10,filesize=100M -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dkafka.logs.dir=/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../logs -Dlog4j.configuration=file:./kafka_2.13-2.7.2/bin/../config/log4j.properties -cp /tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/activation-1.1.1.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/aopalliance-repackaged-2.6.1.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/argparse4j-0.7.0.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/audience-annotations-0.5.0.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/commons-cli-1.4.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/commons-lang3-3.8.1.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/connect-api-2.7.2.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/connect-basic-auth-extension-2.7.2.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/connect-file-2.7.2.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/connect-json-2.7.2.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/connect-mirror-2.7.2.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/connect-mirror-client-2.7.2.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/connect-runtime-2.7.2.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/connect-transforms-2.7.2.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/hk2-api-2.6.1.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/hk2-locator-2.6.1.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/hk2-utils-2.6.1.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/jackson-annotations-2.10.5.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/jackson-core-2.10.5.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/jackson-databind-2.10.5.1.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/jackson-dataformat-csv-2.10.5.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/jackson-datatype-jdk8-2.10.5.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/jackson-jaxrs-base-2.10.5.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/jackson-jaxrs-json-provider-2.10.5.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/jackson-module-jaxb-annotations-2.10.5.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/jackson-module-paranamer-2.10.5.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/jackson-module-scala_2.13-2.10.5.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/jakarta.activation-api-1.2.1.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/jakarta.annotation-api-1.3.5.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/jakarta.inject-2.6.1.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/jakarta.validation-api-2.0.2.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/jakarta.ws.rs-api-2.1.6.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/jakarta.xml.bind-api-2.3.2.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/javassist-3.25.0-GA.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/javassist-3.26.0-GA.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/javax.servlet-api-3.1.0.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/javax.ws.rs-api-2.1.1.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/jaxb-api-2.3.0.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/jersey-client-2.34.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/jersey-common-2.34.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/jersey-container-servlet-2.34.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/jersey-container-servlet-core-2.34.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/jersey-hk2-2.34.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/jersey-server-2.34.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/jetty-client-9.4.43.v20210629.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/jetty-continuation-9.4.43.v20210629.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/jetty-http-9.4.43.v20210629.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/jetty-io-9.4.43.v20210629.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/jetty-security-9.4.43.v20210629.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/jetty-server-9.4.43.v20210629.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/jetty-servlet-9.4.43.v20210629.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/jetty-servlets-9.4.43.v20210629.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/jetty-util-9.4.43.v20210629.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/jetty-util-ajax-9.4.43.v20210629.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/jopt-simple-5.0.4.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/kafka-clients-2.7.2.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/kafka-log4j-appender-2.7.2.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/kafka-raft-2.7.2.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/kafka-streams-2.7.2.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/kafka-streams-examples-2.7.2.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/kafka-streams-scala_2.13-2.7.2.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/kafka-streams-test-utils-2.7.2.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/kafka-tools-2.7.2.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/kafka_2.13-2.7.2-sources.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/kafka_2.13-2.7.2.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/log4j-1.2.17.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/lz4-java-1.7.1.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/maven-artifact-3.8.1.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/metrics-core-2.2.0.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/netty-buffer-4.1.59.Final.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/netty-codec-4.1.59.Final.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/netty-common-4.1.59.Final.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/netty-handler-4.1.59.Final.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/netty-resolver-4.1.59.Final.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/netty-transport-4.1.59.Final.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/netty-transport-native-epoll-4.1.59.Final.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/netty-transport-native-unix-common-4.1.59.Final.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/osgi-resource-locator-1.0.3.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/paranamer-2.8.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/plexus-utils-3.2.1.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/reflections-0.9.12.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/rocksdbjni-5.18.4.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/scala-collection-compat_2.13-2.2.0.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/scala-java8-compat_2.13-0.9.1.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/scala-library-2.13.3.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/scala-logging_2.13-3.9.2.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/scala-reflect-2.13.3.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/slf4j-api-1.7.30.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/slf4j-log4j12-1.7.30.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/snappy-java-1.1.7.7.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/zookeeper-3.5.9.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/zookeeper-jute-3.5.9.jar:/tmpfs/src/temp/docs/tutorials/kafka_2.13-2.7.2/bin/../libs/zstd-jni-1.4.5-6.jar kafka.Kafka ./kafka_2.13-2.7.2/config/server.properties kbuilder 28821 27860 0 20:28 pts/0 00:00:00 /bin/bash -c ps -ef | grep kafka kbuilder 28823 28821 0 20:28 pts/0 00:00:00 grep kafka
Create the kafka topics with the following specs:
- susy-train: partitions=1, replication-factor=1
- susy-test: partitions=2, replication-factor=1
./kafka_2.13-2.7.2/bin/kafka-topics.sh --create --bootstrap-server 127.0.0.1:9092 --replication-factor 1 --partitions 1 --topic susy-train./kafka_2.13-2.7.2/bin/kafka-topics.sh --create --bootstrap-server 127.0.0.1:9092 --replication-factor 1 --partitions 2 --topic susy-test
Created topic susy-train. Created topic susy-test.
Describe the topic for details on the configuration
./kafka_2.13-2.7.2/bin/kafka-topics.sh --describe --bootstrap-server 127.0.0.1:9092 --topic susy-train./kafka_2.13-2.7.2/bin/kafka-topics.sh --describe --bootstrap-server 127.0.0.1:9092 --topic susy-test
Topic: susy-train PartitionCount: 1 ReplicationFactor: 1 Configs: segment.bytes=1073741824
Topic: susy-train Partition: 0 Leader: 0 Replicas: 0 Isr: 0
Topic: susy-test PartitionCount: 2 ReplicationFactor: 1 Configs: segment.bytes=1073741824
Topic: susy-test Partition: 0 Leader: 0 Replicas: 0 Isr: 0
Topic: susy-test Partition: 1 Leader: 0 Replicas: 0 Isr: 0
The replication factor 1 indicates that the data is not being replicated. This is due to the presence of a single broker in our kafka setup. In production systems, the number of bootstrap servers can be in the range of 100's of nodes. That is where the fault-tolerance using replication comes into picture.
Please refer to the docs for more details.
SUSY Dataset
Kafka being an event streaming platform, enables data from various sources to be written into it. For instance:
- Web traffic logs
- Astronomical measurements
- IoT sensor data
- Product reviews and many more.
For the purpose of this tutorial, lets download the SUSY dataset and feed the data into kafka manually. The goal of this classification problem is to distinguish between a signal process which produces supersymmetric particles and a background process which does not.
curl -sSOL https://archive.ics.uci.edu/ml/machine-learning-databases/00279/SUSY.csv.gzExplore the dataset
The first column is the class label (1 for signal, 0 for background), followed by the 18 features (8 low-level features then 10 high-level features). The first 8 features are kinematic properties measured by the particle detectors in the accelerator. The last 10 features are functions of the first 8 features. These are high-level features derived by physicists to help discriminate between the two classes.
COLUMNS = [
# labels
'class',
# low-level features
'lepton_1_pT',
'lepton_1_eta',
'lepton_1_phi',
'lepton_2_pT',
'lepton_2_eta',
'lepton_2_phi',
'missing_energy_magnitude',
'missing_energy_phi',
# high-level derived features
'MET_rel',
'axial_MET',
'M_R',
'M_TR_2',
'R',
'MT2',
'S_R',
'M_Delta_R',
'dPhi_r_b',
'cos(theta_r1)'
]
The entire dataset consists of 5 million rows. However, for the purpose of this tutorial, let's consider only a fraction of the dataset (100,000 rows) so that less time is spent on the moving the data and more time on understanding the functionality of the api.
susy_iterator = pd.read_csv('SUSY.csv.gz', header=None, names=COLUMNS, chunksize=100000)
susy_df = next(susy_iterator)
susy_df.head()
# Number of datapoints and columns
len(susy_df), len(susy_df.columns)
(100000, 19)
# Number of datapoints belonging to each class (0: background noise, 1: signal)
len(susy_df[susy_df["class"]==0]), len(susy_df[susy_df["class"]==1])
(54025, 45975)
Split the dataset
train_df, test_df = train_test_split(susy_df, test_size=0.4, shuffle=True)
print("Number of training samples: ",len(train_df))
print("Number of testing sample: ",len(test_df))
x_train_df = train_df.drop(["class"], axis=1)
y_train_df = train_df["class"]
x_test_df = test_df.drop(["class"], axis=1)
y_test_df = test_df["class"]
# The labels are set as the kafka message keys so as to store data
# in multiple-partitions. Thus, enabling efficient data retrieval
# using the consumer groups.
x_train = list(filter(None, x_train_df.to_csv(index=False).split("\n")[1:]))
y_train = list(filter(None, y_train_df.to_csv(index=False).split("\n")[1:]))
x_test = list(filter(None, x_test_df.to_csv(index=False).split("\n")[1:]))
y_test = list(filter(None, y_test_df.to_csv(index=False).split("\n")[1:]))
Number of training samples: 60000 Number of testing sample: 40000
NUM_COLUMNS = len(x_train_df.columns)
len(x_train), len(y_train), len(x_test), len(y_test)
(60000, 60000, 40000, 40000)
Store the train and test data in kafka
Storing the data in kafka simulates an environment for continuous remote data retrieval for training and inference purposes.
def error_callback(exc):
raise Exception('Error while sendig data to kafka: {0}'.format(str(exc)))
def write_to_kafka(topic_name, items):
count=0
producer = KafkaProducer(bootstrap_servers=['127.0.0.1:9092'])
for message, key in items:
producer.send(topic_name, key=key.encode('utf-8'), value=message.encode('utf-8')).add_errback(error_callback)
count+=1
producer.flush()
print("Wrote {0} messages into topic: {1}".format(count, topic_name))
write_to_kafka("susy-train", zip(x_train, y_train))
write_to_kafka("susy-test", zip(x_test, y_test))
Wrote 60000 messages into topic: susy-train Wrote 40000 messages into topic: susy-test
Define the tfio train dataset
The IODataset class is utilized for streaming data from kafka into tensorflow. The class inherits from tf.data.Dataset and thus has all the useful functionalities of tf.data.Dataset out of the box.
def decode_kafka_item(item):
message = tf.io.decode_csv(item.message, [[0.0] for i in range(NUM_COLUMNS)])
key = tf.strings.to_number(item.key)
return (message, key)
BATCH_SIZE=64
SHUFFLE_BUFFER_SIZE=64
train_ds = tfio.IODataset.from_kafka('susy-train', partition=0, offset=0)
train_ds = train_ds.shuffle(buffer_size=SHUFFLE_BUFFER_SIZE)
train_ds = train_ds.map(decode_kafka_item)
train_ds = train_ds.batch(BATCH_SIZE)
2022-01-07 20:29:21.602817: E tensorflow/stream_executor/cuda/cuda_driver.cc:271] failed call to cuInit: CUDA_ERROR_NO_DEVICE: no CUDA-capable device is detected
Build and train the model
# Set the parameters
OPTIMIZER="adam"
LOSS=tf.keras.losses.BinaryCrossentropy(from_logits=True)
METRICS=['accuracy']
EPOCHS=10
# design/build the model
model = tf.keras.Sequential([
tf.keras.layers.Input(shape=(NUM_COLUMNS,)),
tf.keras.layers.Dense(128, activation='relu'),
tf.keras.layers.Dropout(0.2),
tf.keras.layers.Dense(256, activation='relu'),
tf.keras.layers.Dropout(0.4),
tf.keras.layers.Dense(128, activation='relu'),
tf.keras.layers.Dropout(0.4),
tf.keras.layers.Dense(1, activation='sigmoid')
])
print(model.summary())
Model: "sequential" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= dense (Dense) (None, 128) 2432 dropout (Dropout) (None, 128) 0 dense_1 (Dense) (None, 256) 33024 dropout_1 (Dropout) (None, 256) 0 dense_2 (Dense) (None, 128) 32896 dropout_2 (Dropout) (None, 128) 0 dense_3 (Dense) (None, 1) 129 ================================================================= Total params: 68,481 Trainable params: 68,481 Non-trainable params: 0 _________________________________________________________________ None
# compile the model
model.compile(optimizer=OPTIMIZER, loss=LOSS, metrics=METRICS)
# fit the model
model.fit(train_ds, epochs=EPOCHS)
Epoch 1/10 /tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow/python/util/dispatch.py:1082: UserWarning: "`binary_crossentropy` received `from_logits=True`, but the `output` argument was produced by a sigmoid or softmax activation and thus does not represent logits. Was this intended?" return dispatch_target(*args, **kwargs) 1/Unknown - 1s 875ms/step - loss: 0.7451 - accuracy: 0.4688 16/Unknown - 1s 32ms/step - loss: 0.6742 - accuracy: 0.5586 32/Unknown - 2s 33ms/step - loss: 0.6469 - accuracy: 0.6035 48/Unknown - 2s 34ms/step - loss: 0.6246 - accuracy: 0.6364 64/Unknown - 3s 34ms/step - loss: 0.5936 - accuracy: 0.6655 80/Unknown - 4s 34ms/step - loss: 0.5817 - accuracy: 0.6783 96/Unknown - 4s 34ms/step - loss: 0.5709 - accuracy: 0.6893 112/Unknown - 5s 34ms/step - loss: 0.5598 - accuracy: 0.6988 128/Unknown - 5s 34ms/step - loss: 0.5558 - accuracy: 0.7041 144/Unknown - 6s 34ms/step - loss: 0.5487 - accuracy: 0.7128 160/Unknown - 6s 34ms/step - loss: 0.5435 - accuracy: 0.7172 176/Unknown - 7s 34ms/step - loss: 0.5386 - accuracy: 0.7222 192/Unknown - 7s 34ms/step - loss: 0.5341 - accuracy: 0.7252 208/Unknown - 8s 34ms/step - loss: 0.5294 - accuracy: 0.7298 224/Unknown - 8s 34ms/step - loss: 0.5263 - accuracy: 0.7327 240/Unknown - 9s 34ms/step - loss: 0.5228 - accuracy: 0.7364 256/Unknown - 9s 34ms/step - loss: 0.5189 - accuracy: 0.7393 272/Unknown - 10s 34ms/step - loss: 0.5150 - accuracy: 0.7424 288/Unknown - 10s 33ms/step - loss: 0.5130 - accuracy: 0.7443 304/Unknown - 11s 33ms/step - loss: 0.5110 - accuracy: 0.7456 320/Unknown - 12s 33ms/step - loss: 0.5084 - accuracy: 0.7477 336/Unknown - 12s 33ms/step - loss: 0.5077 - accuracy: 0.7487 352/Unknown - 13s 33ms/step - loss: 0.5054 - accuracy: 0.7507 368/Unknown - 13s 33ms/step - loss: 0.5036 - accuracy: 0.7518 384/Unknown - 14s 33ms/step - loss: 0.5025 - accuracy: 0.7533 400/Unknown - 14s 33ms/step - loss: 0.5010 - accuracy: 0.7544 416/Unknown - 15s 33ms/step - loss: 0.4993 - accuracy: 0.7553 432/Unknown - 15s 33ms/step - loss: 0.4990 - accuracy: 0.7560 448/Unknown - 16s 33ms/step - loss: 0.4981 - accuracy: 0.7571 464/Unknown - 16s 33ms/step - loss: 0.4973 - accuracy: 0.7577 480/Unknown - 17s 33ms/step - loss: 0.4965 - accuracy: 0.7582 496/Unknown - 17s 33ms/step - loss: 0.4956 - accuracy: 0.7589 512/Unknown - 18s 33ms/step - loss: 0.4951 - accuracy: 0.7596 528/Unknown - 18s 33ms/step - loss: 0.4944 - accuracy: 0.7601 544/Unknown - 19s 33ms/step - loss: 0.4936 - accuracy: 0.7608 560/Unknown - 19s 33ms/step - loss: 0.4918 - accuracy: 0.7620 576/Unknown - 20s 33ms/step - loss: 0.4907 - accuracy: 0.7628 592/Unknown - 20s 33ms/step - loss: 0.4897 - accuracy: 0.7635 608/Unknown - 21s 33ms/step - loss: 0.4895 - accuracy: 0.7638 624/Unknown - 21s 33ms/step - loss: 0.4891 - accuracy: 0.7642 640/Unknown - 22s 33ms/step - loss: 0.4888 - accuracy: 0.7643 656/Unknown - 22s 33ms/step - loss: 0.4886 - accuracy: 0.7644 672/Unknown - 23s 33ms/step - loss: 0.4889 - accuracy: 0.7640 688/Unknown - 23s 33ms/step - loss: 0.4877 - accuracy: 0.7647 704/Unknown - 24s 33ms/step - loss: 0.4867 - accuracy: 0.7655 720/Unknown - 24s 33ms/step - loss: 0.4867 - accuracy: 0.7656 736/Unknown - 25s 33ms/step - loss: 0.4863 - accuracy: 0.7659 752/Unknown - 26s 33ms/step - loss: 0.4856 - accuracy: 0.7664 768/Unknown - 26s 33ms/step - loss: 0.4855 - accuracy: 0.7668 784/Unknown - 27s 33ms/step - loss: 0.4849 - accuracy: 0.7671 800/Unknown - 27s 33ms/step - loss: 0.4846 - accuracy: 0.7673 816/Unknown - 28s 33ms/step - loss: 0.4847 - accuracy: 0.7674 832/Unknown - 28s 33ms/step - loss: 0.4845 - accuracy: 0.7675 848/Unknown - 29s 33ms/step - loss: 0.4839 - accuracy: 0.7678 864/Unknown - 29s 33ms/step - loss: 0.4833 - accuracy: 0.7680 880/Unknown - 30s 33ms/step - loss: 0.4830 - accuracy: 0.7683 896/Unknown - 30s 33ms/step - loss: 0.4828 - accuracy: 0.7685 912/Unknown - 31s 33ms/step - loss: 0.4822 - accuracy: 0.7689 928/Unknown - 31s 33ms/step - loss: 0.4816 - accuracy: 0.7693 938/938 [==============================] - 31s 33ms/step - loss: 0.4817 - accuracy: 0.7691 Epoch 2/10 1/938 [..............................] - ETA: 2:45 - loss: 0.5905 - accuracy: 0.6719 16/938 [..............................] - ETA: 33s - loss: 0.4763 - accuracy: 0.7627 32/938 [>.............................] - ETA: 31s - loss: 0.4660 - accuracy: 0.7817 48/938 [>.............................] - ETA: 30s - loss: 0.4733 - accuracy: 0.7822 64/938 [=>............................] - ETA: 29s - loss: 0.4647 - accuracy: 0.7876 80/938 [=>............................] - ETA: 29s - loss: 0.4702 - accuracy: 0.7836 96/938 [==>...........................] - ETA: 28s - loss: 0.4663 - accuracy: 0.7847 112/938 [==>...........................] - ETA: 27s - loss: 0.4629 - accuracy: 0.7857 128/938 [===>..........................] - ETA: 27s - loss: 0.4669 - accuracy: 0.7831 144/938 [===>..........................] - ETA: 26s - loss: 0.4632 - accuracy: 0.7848 160/938 [====>.........................] - ETA: 26s - loss: 0.4637 - accuracy: 0.7841 176/938 [====>.........................] - ETA: 25s - loss: 0.4612 - accuracy: 0.7832 192/938 [=====>........................] - ETA: 24s - loss: 0.4604 - accuracy: 0.7835 208/938 [=====>........................] - ETA: 24s - loss: 0.4594 - accuracy: 0.7849 224/938 [======>.......................] - ETA: 23s - loss: 0.4588 - accuracy: 0.7852 240/938 [======>.......................] - ETA: 23s - loss: 0.4581 - accuracy: 0.7852 256/938 [=======>......................] - ETA: 22s - loss: 0.4564 - accuracy: 0.7861 272/938 [=======>......................] - ETA: 22s - loss: 0.4549 - accuracy: 0.7875 288/938 [========>.....................] - ETA: 21s - loss: 0.4555 - accuracy: 0.7871 304/938 [========>.....................] - ETA: 21s - loss: 0.4552 - accuracy: 0.7877 320/938 [=========>....................] - ETA: 20s - loss: 0.4549 - accuracy: 0.7876 336/938 [=========>....................] - ETA: 20s - loss: 0.4550 - accuracy: 0.7877 352/938 [==========>...................] - ETA: 19s - loss: 0.4546 - accuracy: 0.7877 368/938 [==========>...................] - ETA: 18s - loss: 0.4550 - accuracy: 0.7874 384/938 [===========>..................] - ETA: 18s - loss: 0.4551 - accuracy: 0.7877 400/938 [===========>..................] - ETA: 17s - loss: 0.4549 - accuracy: 0.7878 416/938 [============>.................] - ETA: 17s - loss: 0.4548 - accuracy: 0.7880 432/938 [============>.................] - ETA: 16s - loss: 0.4556 - accuracy: 0.7877 448/938 [=============>................] - ETA: 16s - loss: 0.4557 - accuracy: 0.7878 464/938 [=============>................] - ETA: 15s - loss: 0.4551 - accuracy: 0.7879 480/938 [==============>...............] - ETA: 15s - loss: 0.4554 - accuracy: 0.7876 496/938 [==============>...............] - ETA: 14s - loss: 0.4556 - accuracy: 0.7876 512/938 [===============>..............] - ETA: 14s - loss: 0.4559 - accuracy: 0.7873 528/938 [===============>..............] - ETA: 13s - loss: 0.4558 - accuracy: 0.7874 544/938 [================>.............] - ETA: 12s - loss: 0.4558 - accuracy: 0.7875 560/938 [================>.............] - ETA: 12s - loss: 0.4546 - accuracy: 0.7883 576/938 [=================>............] - ETA: 11s - loss: 0.4542 - accuracy: 0.7885 592/938 [=================>............] - ETA: 11s - loss: 0.4540 - accuracy: 0.7886 608/938 [==================>...........] - ETA: 10s - loss: 0.4544 - accuracy: 0.7884 624/938 [==================>...........] - ETA: 10s - loss: 0.4545 - accuracy: 0.7883 640/938 [===================>..........] - ETA: 9s - loss: 0.4546 - accuracy: 0.7880 656/938 [===================>..........] - ETA: 9s - loss: 0.4548 - accuracy: 0.7878 672/938 [====================>.........] - ETA: 8s - loss: 0.4554 - accuracy: 0.7869 688/938 [=====================>........] - ETA: 8s - loss: 0.4547 - accuracy: 0.7874 704/938 [=====================>........] - ETA: 7s - loss: 0.4544 - accuracy: 0.7874 720/938 [======================>.......] - ETA: 7s - loss: 0.4551 - accuracy: 0.7870 736/938 [======================>.......] - ETA: 6s - loss: 0.4553 - accuracy: 0.7869 752/938 [=======================>......] - ETA: 6s - loss: 0.4547 - accuracy: 0.7874 768/938 [=======================>......] - ETA: 5s - loss: 0.4550 - accuracy: 0.7874 784/938 [========================>.....] - ETA: 5s - loss: 0.4549 - accuracy: 0.7875 800/938 [========================>.....] - ETA: 4s - loss: 0.4550 - accuracy: 0.7876 816/938 [=========================>....] - ETA: 3s - loss: 0.4555 - accuracy: 0.7872 832/938 [=========================>....] - ETA: 3s - loss: 0.4555 - accuracy: 0.7870 848/938 [==========================>...] - ETA: 2s - loss: 0.4557 - accuracy: 0.7870 864/938 [==========================>...] - ETA: 2s - loss: 0.4556 - accuracy: 0.7871 880/938 [===========================>..] - ETA: 1s - loss: 0.4552 - accuracy: 0.7873 896/938 [===========================>..] - ETA: 1s - loss: 0.4553 - accuracy: 0.7873 912/938 [============================>.] - ETA: 0s - loss: 0.4550 - accuracy: 0.7876 928/938 [============================>.] - ETA: 0s - loss: 0.4547 - accuracy: 0.7877 938/938 [==============================] - 30s 32ms/step - loss: 0.4550 - accuracy: 0.7875 Epoch 3/10 1/938 [..............................] - ETA: 7:05 - loss: 0.5060 - accuracy: 0.7344 16/938 [..............................] - ETA: 32s - loss: 0.4667 - accuracy: 0.7725 32/938 [>.............................] - ETA: 31s - loss: 0.4581 - accuracy: 0.7803 48/938 [>.............................] - ETA: 30s - loss: 0.4691 - accuracy: 0.7786 64/938 [=>............................] - ETA: 29s - loss: 0.4593 - accuracy: 0.7849 80/938 [=>............................] - ETA: 29s - loss: 0.4639 - accuracy: 0.7822 96/938 [==>...........................] - ETA: 28s - loss: 0.4597 - accuracy: 0.7840 112/938 [==>...........................] - ETA: 27s - loss: 0.4567 - accuracy: 0.7871 128/938 [===>..........................] - ETA: 27s - loss: 0.4600 - accuracy: 0.7844 144/938 [===>..........................] - ETA: 26s - loss: 0.4565 - accuracy: 0.7862 160/938 [====>.........................] - ETA: 26s - loss: 0.4574 - accuracy: 0.7858 176/938 [====>.........................] - ETA: 25s - loss: 0.4556 - accuracy: 0.7876 192/938 [=====>........................] - ETA: 24s - loss: 0.4554 - accuracy: 0.7873 208/938 [=====>........................] - ETA: 24s - loss: 0.4535 - accuracy: 0.7885 224/938 [======>.......................] - ETA: 23s - loss: 0.4527 - accuracy: 0.7896 240/938 [======>.......................] - ETA: 23s - loss: 0.4517 - accuracy: 0.7905 256/938 [=======>......................] - ETA: 22s - loss: 0.4503 - accuracy: 0.7910 272/938 [=======>......................] - ETA: 22s - loss: 0.4490 - accuracy: 0.7919 288/938 [========>.....................] - ETA: 21s - loss: 0.4498 - accuracy: 0.7920 304/938 [========>.....................] - ETA: 21s - loss: 0.4496 - accuracy: 0.7916 320/938 [=========>....................] - ETA: 20s - loss: 0.4495 - accuracy: 0.7915 336/938 [=========>....................] - ETA: 19s - loss: 0.4502 - accuracy: 0.7916 352/938 [==========>...................] - ETA: 19s - loss: 0.4502 - accuracy: 0.7910 368/938 [==========>...................] - ETA: 18s - loss: 0.4503 - accuracy: 0.7907 384/938 [===========>..................] - ETA: 18s - loss: 0.4500 - accuracy: 0.7912 400/938 [===========>..................] - ETA: 17s - loss: 0.4501 - accuracy: 0.7916 416/938 [============>.................] - ETA: 17s - loss: 0.4495 - accuracy: 0.7914 432/938 [============>.................] - ETA: 16s - loss: 0.4504 - accuracy: 0.7913 448/938 [=============>................] - ETA: 16s - loss: 0.4501 - accuracy: 0.7918 464/938 [=============>................] - ETA: 15s - loss: 0.4502 - accuracy: 0.7918 480/938 [==============>...............] - ETA: 15s - loss: 0.4503 - accuracy: 0.7920 496/938 [==============>...............] - ETA: 14s - loss: 0.4497 - accuracy: 0.7919 512/938 [===============>..............] - ETA: 14s - loss: 0.4505 - accuracy: 0.7914 528/938 [===============>..............] - ETA: 13s - loss: 0.4504 - accuracy: 0.7916 544/938 [================>.............] - ETA: 12s - loss: 0.4502 - accuracy: 0.7918 560/938 [================>.............] - ETA: 12s - loss: 0.4491 - accuracy: 0.7926 576/938 [=================>............] - ETA: 11s - loss: 0.4492 - accuracy: 0.7925 592/938 [=================>............] - ETA: 11s - loss: 0.4489 - accuracy: 0.7927 608/938 [==================>...........] - ETA: 10s - loss: 0.4493 - accuracy: 0.7926 624/938 [==================>...........] - ETA: 10s - loss: 0.4495 - accuracy: 0.7926 640/938 [===================>..........] - ETA: 9s - loss: 0.4498 - accuracy: 0.7923 656/938 [===================>..........] - ETA: 9s - loss: 0.4500 - accuracy: 0.7920 672/938 [====================>.........] - ETA: 8s - loss: 0.4507 - accuracy: 0.7912 688/938 [=====================>........] - ETA: 8s - loss: 0.4501 - accuracy: 0.7916 704/938 [=====================>........] - ETA: 7s - loss: 0.4498 - accuracy: 0.7919 720/938 [======================>.......] - ETA: 7s - loss: 0.4504 - accuracy: 0.7916 736/938 [======================>.......] - ETA: 6s - loss: 0.4507 - accuracy: 0.7915 752/938 [=======================>......] - ETA: 6s - loss: 0.4504 - accuracy: 0.7918 768/938 [=======================>......] - ETA: 5s - loss: 0.4505 - accuracy: 0.7917 784/938 [========================>.....] - ETA: 5s - loss: 0.4506 - accuracy: 0.7916 800/938 [========================>.....] - ETA: 4s - loss: 0.4508 - accuracy: 0.7915 816/938 [=========================>....] - ETA: 3s - loss: 0.4511 - accuracy: 0.7913 832/938 [=========================>....] - ETA: 3s - loss: 0.4513 - accuracy: 0.7913 848/938 [==========================>...] - ETA: 2s - loss: 0.4516 - accuracy: 0.7913 864/938 [==========================>...] - ETA: 2s - loss: 0.4514 - accuracy: 0.7915 880/938 [===========================>..] - ETA: 1s - loss: 0.4512 - accuracy: 0.7915 896/938 [===========================>..] - ETA: 1s - loss: 0.4514 - accuracy: 0.7914 912/938 [============================>.] - ETA: 0s - loss: 0.4510 - accuracy: 0.7916 928/938 [============================>.] - ETA: 0s - loss: 0.4507 - accuracy: 0.7916 938/938 [==============================] - 31s 32ms/step - loss: 0.4512 - accuracy: 0.7911 Epoch 4/10 1/938 [..............................] - ETA: 7:05 - loss: 0.5126 - accuracy: 0.6875 16/938 [..............................] - ETA: 32s - loss: 0.4622 - accuracy: 0.7754 32/938 [>.............................] - ETA: 31s - loss: 0.4543 - accuracy: 0.7891 48/938 [>.............................] - ETA: 30s - loss: 0.4650 - accuracy: 0.7852 64/938 [=>............................] - ETA: 29s - loss: 0.4558 - accuracy: 0.7883 80/938 [=>............................] - ETA: 29s - loss: 0.4589 - accuracy: 0.7842 96/938 [==>...........................] - ETA: 28s - loss: 0.4563 - accuracy: 0.7868 112/938 [==>...........................] - ETA: 27s - loss: 0.4542 - accuracy: 0.7892 128/938 [===>..........................] - ETA: 27s - loss: 0.4569 - accuracy: 0.7889 144/938 [===>..........................] - ETA: 26s - loss: 0.4538 - accuracy: 0.7909 160/938 [====>.........................] - ETA: 26s - loss: 0.4547 - accuracy: 0.7909 176/938 [====>.........................] - ETA: 25s - loss: 0.4519 - accuracy: 0.7931 192/938 [=====>........................] - ETA: 24s - loss: 0.4509 - accuracy: 0.7933 208/938 [=====>........................] - ETA: 24s - loss: 0.4492 - accuracy: 0.7942 224/938 [======>.......................] - ETA: 23s - loss: 0.4489 - accuracy: 0.7949 240/938 [======>.......................] - ETA: 23s - loss: 0.4483 - accuracy: 0.7956 256/938 [=======>......................] - ETA: 22s - loss: 0.4462 - accuracy: 0.7962 272/938 [=======>......................] - ETA: 22s - loss: 0.4454 - accuracy: 0.7967 288/938 [========>.....................] - ETA: 21s - loss: 0.4456 - accuracy: 0.7971 304/938 [========>.....................] - ETA: 21s - loss: 0.4459 - accuracy: 0.7965 320/938 [=========>....................] - ETA: 20s - loss: 0.4457 - accuracy: 0.7964 336/938 [=========>....................] - ETA: 19s - loss: 0.4461 - accuracy: 0.7965 352/938 [==========>...................] - ETA: 19s - loss: 0.4456 - accuracy: 0.7965 368/938 [==========>...................] - ETA: 18s - loss: 0.4461 - accuracy: 0.7958 384/938 [===========>..................] - ETA: 18s - loss: 0.4460 - accuracy: 0.7959 400/938 [===========>..................] - ETA: 17s - loss: 0.4463 - accuracy: 0.7955 416/938 [============>.................] - ETA: 17s - loss: 0.4459 - accuracy: 0.7960 432/938 [============>.................] - ETA: 16s - loss: 0.4470 - accuracy: 0.7954 448/938 [=============>................] - ETA: 16s - loss: 0.4471 - accuracy: 0.7960 464/938 [=============>................] - ETA: 15s - loss: 0.4468 - accuracy: 0.7959 480/938 [==============>...............] - ETA: 15s - loss: 0.4471 - accuracy: 0.7959 496/938 [==============>...............] - ETA: 14s - loss: 0.4467 - accuracy: 0.7960 512/938 [===============>..............] - ETA: 14s - loss: 0.4473 - accuracy: 0.7955 528/938 [===============>..............] - ETA: 13s - loss: 0.4472 - accuracy: 0.7955 544/938 [================>.............] - ETA: 12s - loss: 0.4475 - accuracy: 0.7952 560/938 [================>.............] - ETA: 12s - loss: 0.4465 - accuracy: 0.7956 576/938 [=================>............] - ETA: 11s - loss: 0.4462 - accuracy: 0.7956 592/938 [=================>............] - ETA: 11s - loss: 0.4459 - accuracy: 0.7960 608/938 [==================>...........] - ETA: 10s - loss: 0.4465 - accuracy: 0.7956 624/938 [==================>...........] - ETA: 10s - loss: 0.4467 - accuracy: 0.7954 640/938 [===================>..........] - ETA: 9s - loss: 0.4471 - accuracy: 0.7952 656/938 [===================>..........] - ETA: 9s - loss: 0.4473 - accuracy: 0.7952 672/938 [====================>.........] - ETA: 8s - loss: 0.4480 - accuracy: 0.7945 688/938 [=====================>........] - ETA: 8s - loss: 0.4475 - accuracy: 0.7947 704/938 [=====================>........] - ETA: 7s - loss: 0.4473 - accuracy: 0.7949 720/938 [======================>.......] - ETA: 7s - loss: 0.4479 - accuracy: 0.7947 736/938 [======================>.......] - ETA: 6s - loss: 0.4483 - accuracy: 0.7945 752/938 [=======================>......] - ETA: 6s - loss: 0.4479 - accuracy: 0.7948 768/938 [=======================>......] - ETA: 5s - loss: 0.4479 - accuracy: 0.7950 784/938 [========================>.....] - ETA: 5s - loss: 0.4480 - accuracy: 0.7950 800/938 [========================>.....] - ETA: 4s - loss: 0.4482 - accuracy: 0.7948 816/938 [=========================>....] - ETA: 3s - loss: 0.4484 - accuracy: 0.7947 832/938 [=========================>....] - ETA: 3s - loss: 0.4485 - accuracy: 0.7943 848/938 [==========================>...] - ETA: 2s - loss: 0.4488 - accuracy: 0.7941 864/938 [==========================>...] - ETA: 2s - loss: 0.4488 - accuracy: 0.7941 880/938 [===========================>..] - ETA: 1s - loss: 0.4485 - accuracy: 0.7942 896/938 [===========================>..] - ETA: 1s - loss: 0.4489 - accuracy: 0.7940 912/938 [============================>.] - ETA: 0s - loss: 0.4485 - accuracy: 0.7944 928/938 [============================>.] - ETA: 0s - loss: 0.4482 - accuracy: 0.7945 938/938 [==============================] - 31s 32ms/step - loss: 0.4487 - accuracy: 0.7940 Epoch 5/10 1/938 [..............................] - ETA: 7:06 - loss: 0.5000 - accuracy: 0.7656 16/938 [..............................] - ETA: 32s - loss: 0.4615 - accuracy: 0.7891 32/938 [>.............................] - ETA: 31s - loss: 0.4467 - accuracy: 0.7935 48/938 [>.............................] - ETA: 30s - loss: 0.4626 - accuracy: 0.7858 64/938 [=>............................] - ETA: 29s - loss: 0.4529 - accuracy: 0.7913 80/938 [=>............................] - ETA: 28s - loss: 0.4558 - accuracy: 0.7906 96/938 [==>...........................] - ETA: 28s - loss: 0.4540 - accuracy: 0.7910 112/938 [==>...........................] - ETA: 27s - loss: 0.4514 - accuracy: 0.7920 128/938 [===>..........................] - ETA: 27s - loss: 0.4555 - accuracy: 0.7891 144/938 [===>..........................] - ETA: 26s - loss: 0.4523 - accuracy: 0.7910 160/938 [====>.........................] - ETA: 26s - loss: 0.4524 - accuracy: 0.7910 176/938 [====>.........................] - ETA: 25s - loss: 0.4513 - accuracy: 0.7917 192/938 [=====>........................] - ETA: 24s - loss: 0.4501 - accuracy: 0.7926 208/938 [=====>........................] - ETA: 24s - loss: 0.4490 - accuracy: 0.7934 224/938 [======>.......................] - ETA: 23s - loss: 0.4478 - accuracy: 0.7941 240/938 [======>.......................] - ETA: 23s - loss: 0.4480 - accuracy: 0.7944 256/938 [=======>......................] - ETA: 22s - loss: 0.4461 - accuracy: 0.7952 272/938 [=======>......................] - ETA: 22s - loss: 0.4454 - accuracy: 0.7961 288/938 [========>.....................] - ETA: 21s - loss: 0.4453 - accuracy: 0.7964 304/938 [========>.....................] - ETA: 21s - loss: 0.4448 - accuracy: 0.7957 320/938 [=========>....................] - ETA: 20s - loss: 0.4441 - accuracy: 0.7958 336/938 [=========>....................] - ETA: 19s - loss: 0.4453 - accuracy: 0.7950 352/938 [==========>...................] - ETA: 19s - loss: 0.4452 - accuracy: 0.7945 368/938 [==========>...................] - ETA: 18s - loss: 0.4454 - accuracy: 0.7945 384/938 [===========>..................] - ETA: 18s - loss: 0.4455 - accuracy: 0.7947 400/938 [===========>..................] - ETA: 17s - loss: 0.4450 - accuracy: 0.7947 416/938 [============>.................] - ETA: 17s - loss: 0.4448 - accuracy: 0.7947 432/938 [============>.................] - ETA: 16s - loss: 0.4453 - accuracy: 0.7944 448/938 [=============>................] - ETA: 16s - loss: 0.4454 - accuracy: 0.7951 464/938 [=============>................] - ETA: 15s - loss: 0.4454 - accuracy: 0.7951 480/938 [==============>...............] - ETA: 15s - loss: 0.4454 - accuracy: 0.7952 496/938 [==============>...............] - ETA: 14s - loss: 0.4451 - accuracy: 0.7953 512/938 [===============>..............] - ETA: 14s - loss: 0.4457 - accuracy: 0.7948 528/938 [===============>..............] - ETA: 13s - loss: 0.4457 - accuracy: 0.7952 544/938 [================>.............] - ETA: 12s - loss: 0.4456 - accuracy: 0.7953 560/938 [================>.............] - ETA: 12s - loss: 0.4448 - accuracy: 0.7957 576/938 [=================>............] - ETA: 11s - loss: 0.4446 - accuracy: 0.7956 592/938 [=================>............] - ETA: 11s - loss: 0.4442 - accuracy: 0.7958 608/938 [==================>...........] - ETA: 10s - loss: 0.4448 - accuracy: 0.7957 624/938 [==================>...........] - ETA: 10s - loss: 0.4448 - accuracy: 0.7957 640/938 [===================>..........] - ETA: 9s - loss: 0.4452 - accuracy: 0.7954 656/938 [===================>..........] - ETA: 9s - loss: 0.4451 - accuracy: 0.7954 672/938 [====================>.........] - ETA: 8s - loss: 0.4459 - accuracy: 0.7945 688/938 [=====================>........] - ETA: 8s - loss: 0.4455 - accuracy: 0.7947 704/938 [=====================>........] - ETA: 7s - loss: 0.4453 - accuracy: 0.7951 720/938 [======================>.......] - ETA: 7s - loss: 0.4460 - accuracy: 0.7947 736/938 [======================>.......] - ETA: 6s - loss: 0.4464 - accuracy: 0.7945 752/938 [=======================>......] - ETA: 6s - loss: 0.4460 - accuracy: 0.7946 768/938 [=======================>......] - ETA: 5s - loss: 0.4460 - accuracy: 0.7945 784/938 [========================>.....] - ETA: 5s - loss: 0.4463 - accuracy: 0.7940 800/938 [========================>.....] - ETA: 4s - loss: 0.4464 - accuracy: 0.7940 816/938 [=========================>....] - ETA: 3s - loss: 0.4468 - accuracy: 0.7936 832/938 [=========================>....] - ETA: 3s - loss: 0.4469 - accuracy: 0.7934 848/938 [==========================>...] - ETA: 2s - loss: 0.4471 - accuracy: 0.7932 864/938 [==========================>...] - ETA: 2s - loss: 0.4468 - accuracy: 0.7936 880/938 [===========================>..] - ETA: 1s - loss: 0.4466 - accuracy: 0.7935 896/938 [===========================>..] - ETA: 1s - loss: 0.4468 - accuracy: 0.7934 912/938 [============================>.] - ETA: 0s - loss: 0.4463 - accuracy: 0.7937 928/938 [============================>.] - ETA: 0s - loss: 0.4464 - accuracy: 0.7937 938/938 [==============================] - 31s 32ms/step - loss: 0.4466 - accuracy: 0.7934 Epoch 6/10 1/938 [..............................] - ETA: 7:06 - loss: 0.5106 - accuracy: 0.7031 16/938 [..............................] - ETA: 32s - loss: 0.4582 - accuracy: 0.7822 32/938 [>.............................] - ETA: 31s - loss: 0.4492 - accuracy: 0.7915 48/938 [>.............................] - ETA: 30s - loss: 0.4640 - accuracy: 0.7816 64/938 [=>............................] - ETA: 29s - loss: 0.4526 - accuracy: 0.7891 80/938 [=>............................] - ETA: 29s - loss: 0.4571 - accuracy: 0.7867 96/938 [==>...........................] - ETA: 28s - loss: 0.4561 - accuracy: 0.7891 112/938 [==>...........................] - ETA: 27s - loss: 0.4520 - accuracy: 0.7906 128/938 [===>..........................] - ETA: 27s - loss: 0.4555 - accuracy: 0.7866 144/938 [===>..........................] - ETA: 26s - loss: 0.4520 - accuracy: 0.7895 160/938 [====>.........................] - ETA: 26s - loss: 0.4522 - accuracy: 0.7896 176/938 [====>.........................] - ETA: 25s - loss: 0.4500 - accuracy: 0.7911 192/938 [=====>........................] - ETA: 24s - loss: 0.4492 - accuracy: 0.7897 208/938 [=====>........................] - ETA: 24s - loss: 0.4476 - accuracy: 0.7909 224/938 [======>.......................] - ETA: 23s - loss: 0.4464 - accuracy: 0.7923 240/938 [======>.......................] - ETA: 23s - loss: 0.4466 - accuracy: 0.7939 256/938 [=======>......................] - ETA: 22s - loss: 0.4449 - accuracy: 0.7943 272/938 [=======>......................] - ETA: 22s - loss: 0.4441 - accuracy: 0.7949 288/938 [========>.....................] - ETA: 21s - loss: 0.4446 - accuracy: 0.7948 304/938 [========>.....................] - ETA: 21s - loss: 0.4442 - accuracy: 0.7948 320/938 [=========>....................] - ETA: 20s - loss: 0.4435 - accuracy: 0.7948 336/938 [=========>....................] - ETA: 19s - loss: 0.4443 - accuracy: 0.7944 352/938 [==========>...................] - ETA: 19s - loss: 0.4441 - accuracy: 0.7942 368/938 [==========>...................] - ETA: 18s - loss: 0.4445 - accuracy: 0.7942 384/938 [===========>..................] - ETA: 18s - loss: 0.4443 - accuracy: 0.7945 400/938 [===========>..................] - ETA: 17s - loss: 0.4449 - accuracy: 0.7942 416/938 [============>.................] - ETA: 17s - loss: 0.4442 - accuracy: 0.7944 432/938 [============>.................] - ETA: 16s - loss: 0.4446 - accuracy: 0.7942 448/938 [=============>................] - ETA: 16s - loss: 0.4450 - accuracy: 0.7943 464/938 [=============>................] - ETA: 15s - loss: 0.4454 - accuracy: 0.7938 480/938 [==============>...............] - ETA: 15s - loss: 0.4453 - accuracy: 0.7942 496/938 [==============>...............] - ETA: 14s - loss: 0.4450 - accuracy: 0.7946 512/938 [===============>..............] - ETA: 14s - loss: 0.4457 - accuracy: 0.7942 528/938 [===============>..............] - ETA: 13s - loss: 0.4455 - accuracy: 0.7942 544/938 [================>.............] - ETA: 12s - loss: 0.4455 - accuracy: 0.7943 560/938 [================>.............] - ETA: 12s - loss: 0.4444 - accuracy: 0.7951 576/938 [=================>............] - ETA: 11s - loss: 0.4438 - accuracy: 0.7953 592/938 [=================>............] - ETA: 11s - loss: 0.4440 - accuracy: 0.7955 608/938 [==================>...........] - ETA: 10s - loss: 0.4444 - accuracy: 0.7951 624/938 [==================>...........] - ETA: 10s - loss: 0.4442 - accuracy: 0.7952 640/938 [===================>..........] - ETA: 9s - loss: 0.4446 - accuracy: 0.7948 656/938 [===================>..........] - ETA: 9s - loss: 0.4446 - accuracy: 0.7945 672/938 [====================>.........] - ETA: 8s - loss: 0.4453 - accuracy: 0.7940 688/938 [=====================>........] - ETA: 8s - loss: 0.4448 - accuracy: 0.7941 704/938 [=====================>........] - ETA: 7s - loss: 0.4446 - accuracy: 0.7946 720/938 [======================>.......] - ETA: 7s - loss: 0.4453 - accuracy: 0.7940 736/938 [======================>.......] - ETA: 6s - loss: 0.4455 - accuracy: 0.7941 752/938 [=======================>......] - ETA: 6s - loss: 0.4453 - accuracy: 0.7941 768/938 [=======================>......] - ETA: 5s - loss: 0.4455 - accuracy: 0.7941 784/938 [========================>.....] - ETA: 5s - loss: 0.4456 - accuracy: 0.7938 800/938 [========================>.....] - ETA: 4s - loss: 0.4457 - accuracy: 0.7940 816/938 [=========================>....] - ETA: 3s - loss: 0.4461 - accuracy: 0.7936 832/938 [=========================>....] - ETA: 3s - loss: 0.4461 - accuracy: 0.7933 848/938 [==========================>...] - ETA: 2s - loss: 0.4463 - accuracy: 0.7931 864/938 [==========================>...] - ETA: 2s - loss: 0.4461 - accuracy: 0.7933 880/938 [===========================>..] - ETA: 1s - loss: 0.4458 - accuracy: 0.7935 896/938 [===========================>..] - ETA: 1s - loss: 0.4461 - accuracy: 0.7934 912/938 [============================>.] - ETA: 0s - loss: 0.4458 - accuracy: 0.7937 928/938 [============================>.] - ETA: 0s - loss: 0.4456 - accuracy: 0.7937 938/938 [==============================] - 31s 32ms/step - loss: 0.4459 - accuracy: 0.7933 Epoch 7/10 1/938 [..............................] - ETA: 7:05 - loss: 0.4353 - accuracy: 0.8281 16/938 [..............................] - ETA: 32s - loss: 0.4563 - accuracy: 0.7822 32/938 [>.............................] - ETA: 31s - loss: 0.4467 - accuracy: 0.7920 48/938 [>.............................] - ETA: 30s - loss: 0.4569 - accuracy: 0.7861 64/938 [=>............................] - ETA: 29s - loss: 0.4524 - accuracy: 0.7900 80/938 [=>............................] - ETA: 28s - loss: 0.4542 - accuracy: 0.7865 96/938 [==>...........................] - ETA: 28s - loss: 0.4513 - accuracy: 0.7884 112/938 [==>...........................] - ETA: 27s - loss: 0.4495 - accuracy: 0.7907 128/938 [===>..........................] - ETA: 27s - loss: 0.4532 - accuracy: 0.7885 144/938 [===>..........................] - ETA: 26s - loss: 0.4497 - accuracy: 0.7906 160/938 [====>.........................] - ETA: 26s - loss: 0.4510 - accuracy: 0.7899 176/938 [====>.........................] - ETA: 25s - loss: 0.4480 - accuracy: 0.7921 192/938 [=====>........................] - ETA: 24s - loss: 0.4476 - accuracy: 0.7921 208/938 [=====>........................] - ETA: 24s - loss: 0.4458 - accuracy: 0.7927 224/938 [======>.......................] - ETA: 23s - loss: 0.4451 - accuracy: 0.7935 240/938 [======>.......................] - ETA: 23s - loss: 0.4442 - accuracy: 0.7940 256/938 [=======>......................] - ETA: 22s - loss: 0.4419 - accuracy: 0.7950 272/938 [=======>......................] - ETA: 22s - loss: 0.4405 - accuracy: 0.7958 288/938 [========>.....................] - ETA: 21s - loss: 0.4412 - accuracy: 0.7957 304/938 [========>.....................] - ETA: 21s - loss: 0.4408 - accuracy: 0.7956 320/938 [=========>....................] - ETA: 20s - loss: 0.4406 - accuracy: 0.7957 336/938 [=========>....................] - ETA: 19s - loss: 0.4413 - accuracy: 0.7958 352/938 [==========>...................] - ETA: 19s - loss: 0.4415 - accuracy: 0.7952 368/938 [==========>...................] - ETA: 18s - loss: 0.4421 - accuracy: 0.7949 384/938 [===========>..................] - ETA: 18s - loss: 0.4417 - accuracy: 0.7952 400/938 [===========>..................] - ETA: 17s - loss: 0.4418 - accuracy: 0.7951 416/938 [============>.................] - ETA: 17s - loss: 0.4421 - accuracy: 0.7950 432/938 [============>.................] - ETA: 16s - loss: 0.4425 - accuracy: 0.7946 448/938 [=============>................] - ETA: 16s - loss: 0.4426 - accuracy: 0.7948 464/938 [=============>................] - ETA: 15s - loss: 0.4421 - accuracy: 0.7951 480/938 [==============>...............] - ETA: 15s - loss: 0.4419 - accuracy: 0.7954 496/938 [==============>...............] - ETA: 14s - loss: 0.4420 - accuracy: 0.7957 512/938 [===============>..............] - ETA: 14s - loss: 0.4426 - accuracy: 0.7951 528/938 [===============>..............] - ETA: 13s - loss: 0.4428 - accuracy: 0.7951 544/938 [================>.............] - ETA: 12s - loss: 0.4428 - accuracy: 0.7952 560/938 [================>.............] - ETA: 12s - loss: 0.4415 - accuracy: 0.7961 576/938 [=================>............] - ETA: 11s - loss: 0.4414 - accuracy: 0.7959 592/938 [=================>............] - ETA: 11s - loss: 0.4417 - accuracy: 0.7959 608/938 [==================>...........] - ETA: 10s - loss: 0.4420 - accuracy: 0.7956 624/938 [==================>...........] - ETA: 10s - loss: 0.4421 - accuracy: 0.7959 640/938 [===================>..........] - ETA: 9s - loss: 0.4425 - accuracy: 0.7957 656/938 [===================>..........] - ETA: 9s - loss: 0.4426 - accuracy: 0.7956 672/938 [====================>.........] - ETA: 8s - loss: 0.4434 - accuracy: 0.7948 688/938 [=====================>........] - ETA: 8s - loss: 0.4429 - accuracy: 0.7949 704/938 [=====================>........] - ETA: 7s - loss: 0.4427 - accuracy: 0.7954 720/938 [======================>.......] - ETA: 7s - loss: 0.4434 - accuracy: 0.7949 736/938 [======================>.......] - ETA: 6s - loss: 0.4437 - accuracy: 0.7949 752/938 [=======================>......] - ETA: 6s - loss: 0.4435 - accuracy: 0.7949 768/938 [=======================>......] - ETA: 5s - loss: 0.4438 - accuracy: 0.7948 784/938 [========================>.....] - ETA: 5s - loss: 0.4442 - accuracy: 0.7945 800/938 [========================>.....] - ETA: 4s - loss: 0.4443 - accuracy: 0.7944 816/938 [=========================>....] - ETA: 3s - loss: 0.4447 - accuracy: 0.7942 832/938 [=========================>....] - ETA: 3s - loss: 0.4448 - accuracy: 0.7939 848/938 [==========================>...] - ETA: 2s - loss: 0.4449 - accuracy: 0.7938 864/938 [==========================>...] - ETA: 2s - loss: 0.4448 - accuracy: 0.7939 880/938 [===========================>..] - ETA: 1s - loss: 0.4446 - accuracy: 0.7940 896/938 [===========================>..] - ETA: 1s - loss: 0.4449 - accuracy: 0.7939 912/938 [============================>.] - ETA: 0s - loss: 0.4444 - accuracy: 0.7941 928/938 [============================>.] - ETA: 0s - loss: 0.4443 - accuracy: 0.7940 938/938 [==============================] - 31s 32ms/step - loss: 0.4448 - accuracy: 0.7935 Epoch 8/10 1/938 [..............................] - ETA: 7:04 - loss: 0.4919 - accuracy: 0.7344 16/938 [..............................] - ETA: 32s - loss: 0.4506 - accuracy: 0.7822 32/938 [>.............................] - ETA: 31s - loss: 0.4459 - accuracy: 0.7871 48/938 [>.............................] - ETA: 30s - loss: 0.4571 - accuracy: 0.7835 64/938 [=>............................] - ETA: 29s - loss: 0.4509 - accuracy: 0.7864 80/938 [=>............................] - ETA: 28s - loss: 0.4539 - accuracy: 0.7844 96/938 [==>...........................] - ETA: 28s - loss: 0.4521 - accuracy: 0.7856 112/938 [==>...........................] - ETA: 27s - loss: 0.4489 - accuracy: 0.7888 128/938 [===>..........................] - ETA: 27s - loss: 0.4533 - accuracy: 0.7875 144/938 [===>..........................] - ETA: 26s - loss: 0.4494 - accuracy: 0.7910 160/938 [====>.........................] - ETA: 25s - loss: 0.4509 - accuracy: 0.7912 176/938 [====>.........................] - ETA: 25s - loss: 0.4495 - accuracy: 0.7919 192/938 [=====>........................] - ETA: 24s - loss: 0.4482 - accuracy: 0.7925 208/938 [=====>........................] - ETA: 24s - loss: 0.4462 - accuracy: 0.7936 224/938 [======>.......................] - ETA: 23s - loss: 0.4452 - accuracy: 0.7949 240/938 [======>.......................] - ETA: 23s - loss: 0.4446 - accuracy: 0.7960 256/938 [=======>......................] - ETA: 22s - loss: 0.4425 - accuracy: 0.7962 272/938 [=======>......................] - ETA: 22s - loss: 0.4414 - accuracy: 0.7968 288/938 [========>.....................] - ETA: 21s - loss: 0.4415 - accuracy: 0.7964 304/938 [========>.....................] - ETA: 21s - loss: 0.4411 - accuracy: 0.7959 320/938 [=========>....................] - ETA: 20s - loss: 0.4411 - accuracy: 0.7957 336/938 [=========>....................] - ETA: 19s - loss: 0.4417 - accuracy: 0.7955 352/938 [==========>...................] - ETA: 19s - loss: 0.4415 - accuracy: 0.7955 368/938 [==========>...................] - ETA: 18s - loss: 0.4415 - accuracy: 0.7953 384/938 [===========>..................] - ETA: 18s - loss: 0.4414 - accuracy: 0.7957 400/938 [===========>..................] - ETA: 17s - loss: 0.4417 - accuracy: 0.7957 416/938 [============>.................] - ETA: 17s - loss: 0.4410 - accuracy: 0.7961 432/938 [============>.................] - ETA: 16s - loss: 0.4419 - accuracy: 0.7958 448/938 [=============>................] - ETA: 16s - loss: 0.4417 - accuracy: 0.7961 464/938 [=============>................] - ETA: 15s - loss: 0.4422 - accuracy: 0.7955 480/938 [==============>...............] - ETA: 15s - loss: 0.4419 - accuracy: 0.7956 496/938 [==============>...............] - ETA: 14s - loss: 0.4416 - accuracy: 0.7957 512/938 [===============>..............] - ETA: 14s - loss: 0.4427 - accuracy: 0.7951 528/938 [===============>..............] - ETA: 13s - loss: 0.4424 - accuracy: 0.7955 544/938 [================>.............] - ETA: 12s - loss: 0.4426 - accuracy: 0.7954 560/938 [================>.............] - ETA: 12s - loss: 0.4414 - accuracy: 0.7963 576/938 [=================>............] - ETA: 11s - loss: 0.4411 - accuracy: 0.7963 592/938 [=================>............] - ETA: 11s - loss: 0.4411 - accuracy: 0.7965 608/938 [==================>...........] - ETA: 10s - loss: 0.4418 - accuracy: 0.7959 624/938 [==================>...........] - ETA: 10s - loss: 0.4417 - accuracy: 0.7961 640/938 [===================>..........] - ETA: 9s - loss: 0.4423 - accuracy: 0.7961 656/938 [===================>..........] - ETA: 9s - loss: 0.4422 - accuracy: 0.7960 672/938 [====================>.........] - ETA: 8s - loss: 0.4430 - accuracy: 0.7954 688/938 [=====================>........] - ETA: 8s - loss: 0.4425 - accuracy: 0.7958 704/938 [=====================>........] - ETA: 7s - loss: 0.4425 - accuracy: 0.7960 720/938 [======================>.......] - ETA: 7s - loss: 0.4430 - accuracy: 0.7957 736/938 [======================>.......] - ETA: 6s - loss: 0.4435 - accuracy: 0.7955 752/938 [=======================>......] - ETA: 6s - loss: 0.4430 - accuracy: 0.7959 768/938 [=======================>......] - ETA: 5s - loss: 0.4433 - accuracy: 0.7959 784/938 [========================>.....] - ETA: 5s - loss: 0.4435 - accuracy: 0.7958 800/938 [========================>.....] - ETA: 4s - loss: 0.4437 - accuracy: 0.7959 816/938 [=========================>....] - ETA: 3s - loss: 0.4440 - accuracy: 0.7956 832/938 [=========================>....] - ETA: 3s - loss: 0.4440 - accuracy: 0.7954 848/938 [==========================>...] - ETA: 2s - loss: 0.4442 - accuracy: 0.7952 864/938 [==========================>...] - ETA: 2s - loss: 0.4441 - accuracy: 0.7952 880/938 [===========================>..] - ETA: 1s - loss: 0.4438 - accuracy: 0.7953 896/938 [===========================>..] - ETA: 1s - loss: 0.4441 - accuracy: 0.7951 912/938 [============================>.] - ETA: 0s - loss: 0.4437 - accuracy: 0.7954 928/938 [============================>.] - ETA: 0s - loss: 0.4435 - accuracy: 0.7954 938/938 [==============================] - 31s 32ms/step - loss: 0.4439 - accuracy: 0.7950 Epoch 9/10 1/938 [..............................] - ETA: 7:03 - loss: 0.5275 - accuracy: 0.7656 16/938 [..............................] - ETA: 32s - loss: 0.4546 - accuracy: 0.7910 32/938 [>.............................] - ETA: 31s - loss: 0.4495 - accuracy: 0.7881 48/938 [>.............................] - ETA: 30s - loss: 0.4595 - accuracy: 0.7855 64/938 [=>............................] - ETA: 29s - loss: 0.4507 - accuracy: 0.7905 80/938 [=>............................] - ETA: 28s - loss: 0.4521 - accuracy: 0.7895 96/938 [==>...........................] - ETA: 28s - loss: 0.4510 - accuracy: 0.7917 112/938 [==>...........................] - ETA: 27s - loss: 0.4500 - accuracy: 0.7913 128/938 [===>..........................] - ETA: 27s - loss: 0.4516 - accuracy: 0.7904 144/938 [===>..........................] - ETA: 26s - loss: 0.4484 - accuracy: 0.7920 160/938 [====>.........................] - ETA: 26s - loss: 0.4502 - accuracy: 0.7908 176/938 [====>.........................] - ETA: 25s - loss: 0.4475 - accuracy: 0.7925 192/938 [=====>........................] - ETA: 24s - loss: 0.4470 - accuracy: 0.7922 208/938 [=====>........................] - ETA: 24s - loss: 0.4449 - accuracy: 0.7935 224/938 [======>.......................] - ETA: 23s - loss: 0.4443 - accuracy: 0.7942 240/938 [======>.......................] - ETA: 23s - loss: 0.4436 - accuracy: 0.7949 256/938 [=======>......................] - ETA: 22s - loss: 0.4410 - accuracy: 0.7961 272/938 [=======>......................] - ETA: 22s - loss: 0.4400 - accuracy: 0.7969 288/938 [========>.....................] - ETA: 21s - loss: 0.4405 - accuracy: 0.7969 304/938 [========>.....................] - ETA: 21s - loss: 0.4399 - accuracy: 0.7973 320/938 [=========>....................] - ETA: 20s - loss: 0.4396 - accuracy: 0.7973 336/938 [=========>....................] - ETA: 19s - loss: 0.4405 - accuracy: 0.7969 352/938 [==========>...................] - ETA: 19s - loss: 0.4408 - accuracy: 0.7968 368/938 [==========>...................] - ETA: 18s - loss: 0.4407 - accuracy: 0.7968 384/938 [===========>..................] - ETA: 18s - loss: 0.4403 - accuracy: 0.7964 400/938 [===========>..................] - ETA: 17s - loss: 0.4403 - accuracy: 0.7964 416/938 [============>.................] - ETA: 17s - loss: 0.4401 - accuracy: 0.7968 432/938 [============>.................] - ETA: 16s - loss: 0.4409 - accuracy: 0.7962 448/938 [=============>................] - ETA: 16s - loss: 0.4406 - accuracy: 0.7968 464/938 [=============>................] - ETA: 15s - loss: 0.4409 - accuracy: 0.7965 480/938 [==============>...............] - ETA: 15s - loss: 0.4408 - accuracy: 0.7967 496/938 [==============>...............] - ETA: 14s - loss: 0.4405 - accuracy: 0.7971 512/938 [===============>..............] - ETA: 14s - loss: 0.4413 - accuracy: 0.7967 528/938 [===============>..............] - ETA: 13s - loss: 0.4411 - accuracy: 0.7970 544/938 [================>.............] - ETA: 12s - loss: 0.4411 - accuracy: 0.7968 560/938 [================>.............] - ETA: 12s - loss: 0.4401 - accuracy: 0.7974 576/938 [=================>............] - ETA: 11s - loss: 0.4397 - accuracy: 0.7974 592/938 [=================>............] - ETA: 11s - loss: 0.4397 - accuracy: 0.7975 608/938 [==================>...........] - ETA: 10s - loss: 0.4402 - accuracy: 0.7972 624/938 [==================>...........] - ETA: 10s - loss: 0.4400 - accuracy: 0.7974 640/938 [===================>..........] - ETA: 9s - loss: 0.4407 - accuracy: 0.7969 656/938 [===================>..........] - ETA: 9s - loss: 0.4406 - accuracy: 0.7967 672/938 [====================>.........] - ETA: 8s - loss: 0.4413 - accuracy: 0.7962 688/938 [=====================>........] - ETA: 8s - loss: 0.4410 - accuracy: 0.7962 704/938 [=====================>........] - ETA: 7s - loss: 0.4407 - accuracy: 0.7966 720/938 [======================>.......] - ETA: 7s - loss: 0.4411 - accuracy: 0.7965 736/938 [======================>.......] - ETA: 6s - loss: 0.4414 - accuracy: 0.7964 752/938 [=======================>......] - ETA: 6s - loss: 0.4410 - accuracy: 0.7966 768/938 [=======================>......] - ETA: 5s - loss: 0.4411 - accuracy: 0.7966 784/938 [========================>.....] - ETA: 5s - loss: 0.4414 - accuracy: 0.7963 800/938 [========================>.....] - ETA: 4s - loss: 0.4417 - accuracy: 0.7964 816/938 [=========================>....] - ETA: 3s - loss: 0.4421 - accuracy: 0.7960 832/938 [=========================>....] - ETA: 3s - loss: 0.4420 - accuracy: 0.7959 848/938 [==========================>...] - ETA: 2s - loss: 0.4425 - accuracy: 0.7956 864/938 [==========================>...] - ETA: 2s - loss: 0.4423 - accuracy: 0.7958 880/938 [===========================>..] - ETA: 1s - loss: 0.4422 - accuracy: 0.7959 896/938 [===========================>..] - ETA: 1s - loss: 0.4423 - accuracy: 0.7959 912/938 [============================>.] - ETA: 0s - loss: 0.4420 - accuracy: 0.7960 928/938 [============================>.] - ETA: 0s - loss: 0.4419 - accuracy: 0.7960 938/938 [==============================] - 31s 32ms/step - loss: 0.4421 - accuracy: 0.7956 Epoch 10/10 1/938 [..............................] - ETA: 7:04 - loss: 0.3892 - accuracy: 0.7812 16/938 [..............................] - ETA: 32s - loss: 0.4551 - accuracy: 0.7803 32/938 [>.............................] - ETA: 31s - loss: 0.4466 - accuracy: 0.7891 48/938 [>.............................] - ETA: 30s - loss: 0.4600 - accuracy: 0.7812 64/938 [=>............................] - ETA: 29s - loss: 0.4496 - accuracy: 0.7891 80/938 [=>............................] - ETA: 28s - loss: 0.4531 - accuracy: 0.7881 96/938 [==>...........................] - ETA: 28s - loss: 0.4528 - accuracy: 0.7887 112/938 [==>...........................] - ETA: 27s - loss: 0.4499 - accuracy: 0.7909 128/938 [===>..........................] - ETA: 27s - loss: 0.4528 - accuracy: 0.7887 144/938 [===>..........................] - ETA: 26s - loss: 0.4497 - accuracy: 0.7908 160/938 [====>.........................] - ETA: 26s - loss: 0.4500 - accuracy: 0.7913 176/938 [====>.........................] - ETA: 25s - loss: 0.4473 - accuracy: 0.7928 192/938 [=====>........................] - ETA: 24s - loss: 0.4466 - accuracy: 0.7936 208/938 [=====>........................] - ETA: 24s - loss: 0.4450 - accuracy: 0.7936 224/938 [======>.......................] - ETA: 23s - loss: 0.4448 - accuracy: 0.7950 240/938 [======>.......................] - ETA: 23s - loss: 0.4431 - accuracy: 0.7964 256/938 [=======>......................] - ETA: 22s - loss: 0.4406 - accuracy: 0.7982 272/938 [=======>......................] - ETA: 22s - loss: 0.4405 - accuracy: 0.7984 288/938 [========>.....................] - ETA: 21s - loss: 0.4413 - accuracy: 0.7986 304/938 [========>.....................] - ETA: 21s - loss: 0.4410 - accuracy: 0.7985 320/938 [=========>....................] - ETA: 20s - loss: 0.4407 - accuracy: 0.7983 336/938 [=========>....................] - ETA: 19s - loss: 0.4412 - accuracy: 0.7983 352/938 [==========>...................] - ETA: 19s - loss: 0.4413 - accuracy: 0.7980 368/938 [==========>...................] - ETA: 18s - loss: 0.4409 - accuracy: 0.7981 384/938 [===========>..................] - ETA: 18s - loss: 0.4407 - accuracy: 0.7979 400/938 [===========>..................] - ETA: 17s - loss: 0.4409 - accuracy: 0.7976 416/938 [============>.................] - ETA: 17s - loss: 0.4406 - accuracy: 0.7979 432/938 [============>.................] - ETA: 16s - loss: 0.4411 - accuracy: 0.7975 448/938 [=============>................] - ETA: 16s - loss: 0.4410 - accuracy: 0.7979 464/938 [=============>................] - ETA: 15s - loss: 0.4413 - accuracy: 0.7975 480/938 [==============>...............] - ETA: 15s - loss: 0.4412 - accuracy: 0.7979 496/938 [==============>...............] - ETA: 14s - loss: 0.4407 - accuracy: 0.7980 512/938 [===============>..............] - ETA: 14s - loss: 0.4414 - accuracy: 0.7978 528/938 [===============>..............] - ETA: 13s - loss: 0.4414 - accuracy: 0.7977 544/938 [================>.............] - ETA: 12s - loss: 0.4414 - accuracy: 0.7977 560/938 [================>.............] - ETA: 12s - loss: 0.4402 - accuracy: 0.7983 576/938 [=================>............] - ETA: 11s - loss: 0.4401 - accuracy: 0.7979 592/938 [=================>............] - ETA: 11s - loss: 0.4399 - accuracy: 0.7980 608/938 [==================>...........] - ETA: 10s - loss: 0.4402 - accuracy: 0.7978 624/938 [==================>...........] - ETA: 10s - loss: 0.4404 - accuracy: 0.7980 640/938 [===================>..........] - ETA: 9s - loss: 0.4406 - accuracy: 0.7978 656/938 [===================>..........] - ETA: 9s - loss: 0.4408 - accuracy: 0.7976 672/938 [====================>.........] - ETA: 8s - loss: 0.4417 - accuracy: 0.7970 688/938 [=====================>........] - ETA: 8s - loss: 0.4411 - accuracy: 0.7974 704/938 [=====================>........] - ETA: 7s - loss: 0.4408 - accuracy: 0.7974 720/938 [======================>.......] - ETA: 7s - loss: 0.4414 - accuracy: 0.7969 736/938 [======================>.......] - ETA: 6s - loss: 0.4417 - accuracy: 0.7970 752/938 [=======================>......] - ETA: 6s - loss: 0.4415 - accuracy: 0.7971 768/938 [=======================>......] - ETA: 5s - loss: 0.4418 - accuracy: 0.7970 784/938 [========================>.....] - ETA: 5s - loss: 0.4418 - accuracy: 0.7970 800/938 [========================>.....] - ETA: 4s - loss: 0.4421 - accuracy: 0.7970 816/938 [=========================>....] - ETA: 3s - loss: 0.4425 - accuracy: 0.7965 832/938 [=========================>....] - ETA: 3s - loss: 0.4425 - accuracy: 0.7962 848/938 [==========================>...] - ETA: 2s - loss: 0.4426 - accuracy: 0.7962 864/938 [==========================>...] - ETA: 2s - loss: 0.4425 - accuracy: 0.7963 880/938 [===========================>..] - ETA: 1s - loss: 0.4423 - accuracy: 0.7964 896/938 [===========================>..] - ETA: 1s - loss: 0.4426 - accuracy: 0.7963 912/938 [============================>.] - ETA: 0s - loss: 0.4422 - accuracy: 0.7967 928/938 [============================>.] - ETA: 0s - loss: 0.4421 - accuracy: 0.7967 938/938 [==============================] - 31s 32ms/step - loss: 0.4425 - accuracy: 0.7962 <keras.callbacks.History at 0x7fb364fd2a90>
Since only a fraction of the dataset is being utilized, our accuracy is limited to ~78% during the training phase. However, please feel free to store additional data in kafka for a better model performance. Also, since the goal was to just demonstrate the functionality of the tfio kafka datasets, a smaller and less-complicated neural network was used. However, one can increase the complexity of the model, modify the learning strategy, tune hyper-parameters etc for exploration purposes. For a baseline approach, please refer to this article.
Infer on the test data
To infer on the test data by adhering to the 'exactly-once' semantics along with fault-tolerance, the streaming.KafkaGroupIODataset can be utilized.
Define the tfio test dataset
The stream_timeout parameter blocks for the given duration for new data points to be streamed into the topic. This removes the need for creating new datasets if the data is being streamed into the topic in an intermittent fashion.
test_ds = tfio.experimental.streaming.KafkaGroupIODataset(
topics=["susy-test"],
group_id="testcg",
servers="127.0.0.1:9092",
stream_timeout=10000,
configuration=[
"session.timeout.ms=7000",
"max.poll.interval.ms=8000",
"auto.offset.reset=earliest"
],
)
def decode_kafka_test_item(raw_message, raw_key):
message = tf.io.decode_csv(raw_message, [[0.0] for i in range(NUM_COLUMNS)])
key = tf.strings.to_number(raw_key)
return (message, key)
test_ds = test_ds.map(decode_kafka_test_item)
test_ds = test_ds.batch(BATCH_SIZE)
WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow_io/python/experimental/kafka_group_io_dataset_ops.py:188: take_while (from tensorflow.python.data.experimental.ops.take_while_ops) is deprecated and will be removed in a future version. Instructions for updating: Use `tf.data.Dataset.take_while(...)
Though this class can be used for training purposes, there are caveats which need to be addressed. Once all the messages are read from kafka and the latest offsets are committed using the streaming.KafkaGroupIODataset, the consumer doesn't restart reading the messages from the beginning. Thus, while training, it is possible only to train for a single epoch with the data continuously flowing in. This kind of a functionality has limited use cases during the training phase wherein, once a datapoint has been consumed by the model it is no longer required and can be discarded.
However, this functionality shines when it comes to robust inference with exactly-once semantics.
evaluate the performance on the test data
res = model.evaluate(test_ds)
print("test loss, test acc:", res)
1/Unknown - 0s 317ms/step - loss: 0.4462 - accuracy: 0.7500 34/Unknown - 0s 2ms/step - loss: 0.4434 - accuracy: 0.8194 2022-01-07 20:34:29.402707: E tensorflow_io/core/kernels/kafka_kernels.cc:774] REBALANCE: Local: Assign partitions 2022-01-07 20:34:29.406789: E tensorflow_io/core/kernels/kafka_kernels.cc:776] Retrieved committed offsets with status code: 0 68/Unknown - 0s 2ms/step - loss: 0.4420 - accuracy: 0.8199 102/Unknown - 0s 2ms/step - loss: 0.4472 - accuracy: 0.8078 137/Unknown - 1s 2ms/step - loss: 0.4550 - accuracy: 0.7935 172/Unknown - 1s 1ms/step - loss: 0.4551 - accuracy: 0.7850 210/Unknown - 1s 1ms/step - loss: 0.4521 - accuracy: 0.7864 249/Unknown - 1s 1ms/step - loss: 0.4506 - accuracy: 0.7914 287/Unknown - 1s 1ms/step - loss: 0.4476 - accuracy: 0.7940 325/Unknown - 1s 1ms/step - loss: 0.4478 - accuracy: 0.7912 361/Unknown - 1s 1ms/step - loss: 0.4485 - accuracy: 0.7868 398/Unknown - 1s 1ms/step - loss: 0.4469 - accuracy: 0.7890 433/Unknown - 1s 1ms/step - loss: 0.4438 - accuracy: 0.7926 469/Unknown - 1s 1ms/step - loss: 0.4435 - accuracy: 0.7931 508/Unknown - 1s 1ms/step - loss: 0.4441 - accuracy: 0.7906 544/Unknown - 1s 1ms/step - loss: 0.4442 - accuracy: 0.7891 582/Unknown - 1s 1ms/step - loss: 0.4438 - accuracy: 0.7902 621/Unknown - 1s 1ms/step - loss: 0.4435 - accuracy: 0.7916 625/625 [==============================] - 11s 17ms/step - loss: 0.4437 - accuracy: 0.7915 test loss, test acc: [0.4436523914337158, 0.7915250062942505] 2022-01-07 20:34:40.051954: E tensorflow_io/core/kernels/kafka_kernels.cc:1001] Local: Timed out
Since the inference is based on 'exactly-once' semantics, the evaluation on the test set can be run only once. In order to run the inference again on the test data, a new consumer group should be used.
Track the offset lag of the testcg consumer group
./kafka_2.13-2.7.2/bin/kafka-consumer-groups.sh --bootstrap-server 127.0.0.1:9092 --describe --group testcgGROUP TOPIC PARTITION CURRENT-OFFSET LOG-END-OFFSET LAG CONSUMER-ID HOST CLIENT-ID testcg susy-test 0 21626 21626 0 rdkafka-534f63d0-b91e-4976-a3ca-832b6c91210e /10.142.0.103 rdkafka testcg susy-test 1 18374 18374 0 rdkafka-534f63d0-b91e-4976-a3ca-832b6c91210e /10.142.0.103 rdkafka
Once the current-offset matches the log-end-offset for all the partitions, it indicates that the consumer(s) have completed fetching all the messages from the kafka topic.
Online learning
The online machine learning paradigm is a bit different from the traditional/conventional way of training machine learning models. In the former case, the model continues to incrementally learn/update it's parameters as soon as the new data points are available and this process is expected to continue indefinitely. This is unlike the latter approaches where the dataset is fixed and the model iterates over it n number of times. In online learning, the data once consumed by the model may not be available for training again.
By utilizing the streaming.KafkaBatchIODataset, it is now possible to train the models in this fashion. Let's continue to use our SUSY dataset for demonstrating this functionality.
The tfio training dataset for online learning
The streaming.KafkaBatchIODataset is similar to the streaming.KafkaGroupIODataset in it's API. Additionally, it is recommended to utilize the stream_timeout parameter to configure the duration for which the dataset will block for new messages before timing out. In the instance below, the dataset is configured with a stream_timeout of 10000 milliseconds. This implies that, after all the messages from the topic have been consumed, the dataset will wait for an additional 10 seconds before timing out and disconnecting from the kafka cluster. If new messages are streamed into the topic before timing out, the data consumption and model training resumes for those newly consumed data points. To block indefinitely, set it to -1.
online_train_ds = tfio.experimental.streaming.KafkaBatchIODataset(
topics=["susy-train"],
group_id="cgonline",
servers="127.0.0.1:9092",
stream_timeout=10000, # in milliseconds, to block indefinitely, set it to -1.
configuration=[
"session.timeout.ms=7000",
"max.poll.interval.ms=8000",
"auto.offset.reset=earliest"
],
)
Every item that the online_train_ds generates is a tf.data.Dataset in itself. Thus, all the standard transformations can be applied as usual.
def decode_kafka_online_item(raw_message, raw_key):
message = tf.io.decode_csv(raw_message, [[0.0] for i in range(NUM_COLUMNS)])
key = tf.strings.to_number(raw_key)
return (message, key)
for mini_ds in online_train_ds:
mini_ds = mini_ds.shuffle(buffer_size=32)
mini_ds = mini_ds.map(decode_kafka_online_item)
mini_ds = mini_ds.batch(32)
if len(mini_ds) > 0:
model.fit(mini_ds, epochs=3)
2022-01-07 20:34:42.024915: E tensorflow_io/core/kernels/kafka_kernels.cc:774] REBALANCE: Local: Assign partitions 2022-01-07 20:34:42.025797: E tensorflow_io/core/kernels/kafka_kernels.cc:776] Retrieved committed offsets with status code: 0 Epoch 1/3 1/313 [..............................] - ETA: 1s - loss: 0.4036 - accuracy: 0.8438 24/313 [=>............................] - ETA: 0s - loss: 0.4580 - accuracy: 0.7852 48/313 [===>..........................] - ETA: 0s - loss: 0.4605 - accuracy: 0.7878 72/313 [=====>........................] - ETA: 0s - loss: 0.4540 - accuracy: 0.7912 96/313 [========>.....................] - ETA: 0s - loss: 0.4633 - accuracy: 0.7858 120/313 [==========>...................] - ETA: 0s - loss: 0.4536 - accuracy: 0.7919 144/313 [============>.................] - ETA: 0s - loss: 0.4563 - accuracy: 0.7906 168/313 [===============>..............] - ETA: 0s - loss: 0.4545 - accuracy: 0.7913 192/313 [=================>............] - ETA: 0s - loss: 0.4560 - accuracy: 0.7913 216/313 [===================>..........] - ETA: 0s - loss: 0.4538 - accuracy: 0.7922 240/313 [======================>.......] - ETA: 0s - loss: 0.4543 - accuracy: 0.7918 264/313 [========================>.....] - ETA: 0s - loss: 0.4567 - accuracy: 0.7898 288/313 [==========================>...] - ETA: 0s - loss: 0.4537 - accuracy: 0.7912 312/313 [============================>.] - ETA: 0s - loss: 0.4554 - accuracy: 0.7912 313/313 [==============================] - 1s 2ms/step - loss: 0.4561 - accuracy: 0.7909 Epoch 2/3 1/313 [..............................] - ETA: 1s - loss: 0.3691 - accuracy: 0.8438 24/313 [=>............................] - ETA: 0s - loss: 0.4521 - accuracy: 0.7865 48/313 [===>..........................] - ETA: 0s - loss: 0.4597 - accuracy: 0.7897 72/313 [=====>........................] - ETA: 0s - loss: 0.4562 - accuracy: 0.7930 96/313 [========>.....................] - ETA: 0s - loss: 0.4629 - accuracy: 0.7887 120/313 [==========>...................] - ETA: 0s - loss: 0.4518 - accuracy: 0.7956 144/313 [============>.................] - ETA: 0s - loss: 0.4566 - accuracy: 0.7919 168/313 [===============>..............] - ETA: 0s - loss: 0.4537 - accuracy: 0.7915 192/313 [=================>............] - ETA: 0s - loss: 0.4559 - accuracy: 0.7909 216/313 [===================>..........] - ETA: 0s - loss: 0.4524 - accuracy: 0.7921 240/313 [======================>.......] - ETA: 0s - loss: 0.4525 - accuracy: 0.7923 264/313 [========================>.....] - ETA: 0s - loss: 0.4551 - accuracy: 0.7910 288/313 [==========================>...] - ETA: 0s - loss: 0.4524 - accuracy: 0.7920 312/313 [============================>.] - ETA: 0s - loss: 0.4535 - accuracy: 0.7911 313/313 [==============================] - 1s 2ms/step - loss: 0.4538 - accuracy: 0.7909 Epoch 3/3 1/313 [..............................] - ETA: 1s - loss: 0.4029 - accuracy: 0.7812 24/313 [=>............................] - ETA: 0s - loss: 0.4500 - accuracy: 0.7917 47/313 [===>..........................] - ETA: 0s - loss: 0.4627 - accuracy: 0.7899 70/313 [=====>........................] - ETA: 0s - loss: 0.4571 - accuracy: 0.7942 94/313 [========>.....................] - ETA: 0s - loss: 0.4632 - accuracy: 0.7906 118/313 [==========>...................] - ETA: 0s - loss: 0.4530 - accuracy: 0.7961 141/313 [============>.................] - ETA: 0s - loss: 0.4564 - accuracy: 0.7932 164/313 [==============>...............] - ETA: 0s - loss: 0.4524 - accuracy: 0.7961 187/313 [================>.............] - ETA: 0s - loss: 0.4500 - accuracy: 0.7960 210/313 [===================>..........] - ETA: 0s - loss: 0.4483 - accuracy: 0.7966 234/313 [=====================>........] - ETA: 0s - loss: 0.4496 - accuracy: 0.7955 257/313 [=======================>......] - ETA: 0s - loss: 0.4516 - accuracy: 0.7943 280/313 [=========================>....] - ETA: 0s - loss: 0.4478 - accuracy: 0.7959 303/313 [============================>.] - ETA: 0s - loss: 0.4501 - accuracy: 0.7945 313/313 [==============================] - 1s 2ms/step - loss: 0.4499 - accuracy: 0.7947 Epoch 1/3 1/313 [..............................] - ETA: 1s - loss: 0.5160 - accuracy: 0.8125 25/313 [=>............................] - ETA: 0s - loss: 0.4322 - accuracy: 0.8112 48/313 [===>..........................] - ETA: 0s - loss: 0.4178 - accuracy: 0.8112 72/313 [=====>........................] - ETA: 0s - loss: 0.4307 - accuracy: 0.8043 95/313 [========>.....................] - ETA: 0s - loss: 0.4329 - accuracy: 0.8036 118/313 [==========>...................] - ETA: 0s - loss: 0.4270 - accuracy: 0.8077 142/313 [============>.................] - ETA: 0s - loss: 0.4332 - accuracy: 0.8072 166/313 [==============>...............] - ETA: 0s - loss: 0.4335 - accuracy: 0.8065 189/313 [=================>............] - ETA: 0s - loss: 0.4319 - accuracy: 0.8059 213/313 [===================>..........] - ETA: 0s - loss: 0.4301 - accuracy: 0.8062 236/313 [=====================>........] - ETA: 0s - loss: 0.4309 - accuracy: 0.8053 260/313 [=======================>......] - ETA: 0s - loss: 0.4334 - accuracy: 0.8041 284/313 [==========================>...] - ETA: 0s - loss: 0.4343 - accuracy: 0.8029 307/313 [============================>.] - ETA: 0s - loss: 0.4351 - accuracy: 0.8014 313/313 [==============================] - 1s 2ms/step - loss: 0.4347 - accuracy: 0.8018 Epoch 2/3 1/313 [..............................] - ETA: 1s - loss: 0.4240 - accuracy: 0.8125 25/313 [=>............................] - ETA: 0s - loss: 0.4338 - accuracy: 0.8075 49/313 [===>..........................] - ETA: 0s - loss: 0.4216 - accuracy: 0.8125 74/313 [======>.......................] - ETA: 0s - loss: 0.4244 - accuracy: 0.8091 98/313 [========>.....................] - ETA: 0s - loss: 0.4287 - accuracy: 0.8084 122/313 [==========>...................] - ETA: 0s - loss: 0.4258 - accuracy: 0.8069 146/313 [============>.................] - ETA: 0s - loss: 0.4295 - accuracy: 0.8082 170/313 [===============>..............] - ETA: 0s - loss: 0.4280 - accuracy: 0.8092 194/313 [=================>............] - ETA: 0s - loss: 0.4264 - accuracy: 0.8080 218/313 [===================>..........] - ETA: 0s - loss: 0.4257 - accuracy: 0.8085 242/313 [======================>.......] - ETA: 0s - loss: 0.4287 - accuracy: 0.8060 266/313 [========================>.....] - ETA: 0s - loss: 0.4298 - accuracy: 0.8062 290/313 [==========================>...] - ETA: 0s - loss: 0.4298 - accuracy: 0.8051 313/313 [==============================] - 1s 2ms/step - loss: 0.4314 - accuracy: 0.8048 Epoch 3/3 1/313 [..............................] - ETA: 1s - loss: 0.4738 - accuracy: 0.7812 25/313 [=>............................] - ETA: 0s - loss: 0.4344 - accuracy: 0.8000 49/313 [===>..........................] - ETA: 0s - loss: 0.4190 - accuracy: 0.8093 73/313 [=====>........................] - ETA: 0s - loss: 0.4255 - accuracy: 0.8048 97/313 [========>.....................] - ETA: 0s - loss: 0.4281 - accuracy: 0.8048 120/313 [==========>...................] - ETA: 0s - loss: 0.4240 - accuracy: 0.8091 144/313 [============>.................] - ETA: 0s - loss: 0.4289 - accuracy: 0.8105 168/313 [===============>..............] - ETA: 0s - loss: 0.4261 - accuracy: 0.8106 192/313 [=================>............] - ETA: 0s - loss: 0.4255 - accuracy: 0.8101 216/313 [===================>..........] - ETA: 0s - loss: 0.4250 - accuracy: 0.8096 240/313 [======================>.......] - ETA: 0s - loss: 0.4270 - accuracy: 0.8086 264/313 [========================>.....] - ETA: 0s - loss: 0.4278 - accuracy: 0.8084 288/313 [==========================>...] - ETA: 0s - loss: 0.4277 - accuracy: 0.8069 312/313 [============================>.] - ETA: 0s - loss: 0.4288 - accuracy: 0.8062 313/313 [==============================] - 1s 2ms/step - loss: 0.4286 - accuracy: 0.8063 Epoch 1/3 1/313 [..............................] - ETA: 1s - loss: 0.6701 - accuracy: 0.6562 25/313 [=>............................] - ETA: 0s - loss: 0.4458 - accuracy: 0.8025 50/313 [===>..........................] - ETA: 0s - loss: 0.4505 - accuracy: 0.7950 74/313 [======>.......................] - ETA: 0s - loss: 0.4471 - accuracy: 0.7965 98/313 [========>.....................] - ETA: 0s - loss: 0.4517 - accuracy: 0.7921 122/313 [==========>...................] - ETA: 0s - loss: 0.4466 - accuracy: 0.7915 146/313 [============>.................] - ETA: 0s - loss: 0.4469 - accuracy: 0.7917 170/313 [===============>..............] - ETA: 0s - loss: 0.4491 - accuracy: 0.7906 194/313 [=================>............] - ETA: 0s - loss: 0.4455 - accuracy: 0.7925 218/313 [===================>..........] - ETA: 0s - loss: 0.4456 - accuracy: 0.7921 242/313 [======================>.......] - ETA: 0s - loss: 0.4490 - accuracy: 0.7896 266/313 [========================>.....] - ETA: 0s - loss: 0.4482 - accuracy: 0.7912 290/313 [==========================>...] - ETA: 0s - loss: 0.4476 - accuracy: 0.7914 313/313 [==============================] - 1s 2ms/step - loss: 0.4480 - accuracy: 0.7910 Epoch 2/3 1/313 [..............................] - ETA: 1s - loss: 0.3952 - accuracy: 0.8125 25/313 [=>............................] - ETA: 0s - loss: 0.4344 - accuracy: 0.7987 50/313 [===>..........................] - ETA: 0s - loss: 0.4469 - accuracy: 0.7956 74/313 [======>.......................] - ETA: 0s - loss: 0.4399 - accuracy: 0.7952 98/313 [========>.....................] - ETA: 0s - loss: 0.4439 - accuracy: 0.7921 121/313 [==========>...................] - ETA: 0s - loss: 0.4393 - accuracy: 0.7939 145/313 [============>.................] - ETA: 0s - loss: 0.4393 - accuracy: 0.7948 169/313 [===============>..............] - ETA: 0s - loss: 0.4411 - accuracy: 0.7936 193/313 [=================>............] - ETA: 0s - loss: 0.4400 - accuracy: 0.7936 217/313 [===================>..........] - ETA: 0s - loss: 0.4395 - accuracy: 0.7939 241/313 [======================>.......] - ETA: 0s - loss: 0.4429 - accuracy: 0.7930 265/313 [========================>.....] - ETA: 0s - loss: 0.4426 - accuracy: 0.7941 288/313 [==========================>...] - ETA: 0s - loss: 0.4416 - accuracy: 0.7948 312/313 [============================>.] - ETA: 0s - loss: 0.4425 - accuracy: 0.7946 313/313 [==============================] - 1s 2ms/step - loss: 0.4425 - accuracy: 0.7945 Epoch 3/3 1/313 [..............................] - ETA: 1s - loss: 0.4741 - accuracy: 0.7188 25/313 [=>............................] - ETA: 0s - loss: 0.4262 - accuracy: 0.8100 49/313 [===>..........................] - ETA: 0s - loss: 0.4403 - accuracy: 0.8029 73/313 [=====>........................] - ETA: 0s - loss: 0.4325 - accuracy: 0.8031 97/313 [========>.....................] - ETA: 0s - loss: 0.4392 - accuracy: 0.7986 121/313 [==========>...................] - ETA: 0s - loss: 0.4376 - accuracy: 0.7991 144/313 [============>.................] - ETA: 0s - loss: 0.4374 - accuracy: 0.7997 168/313 [===============>..............] - ETA: 0s - loss: 0.4405 - accuracy: 0.7969 192/313 [=================>............] - ETA: 0s - loss: 0.4379 - accuracy: 0.7980 216/313 [===================>..........] - ETA: 0s - loss: 0.4375 - accuracy: 0.7996 240/313 [======================>.......] - ETA: 0s - loss: 0.4396 - accuracy: 0.7979 264/313 [========================>.....] - ETA: 0s - loss: 0.4388 - accuracy: 0.7988 288/313 [==========================>...] - ETA: 0s - loss: 0.4382 - accuracy: 0.7984 312/313 [============================>.] - ETA: 0s - loss: 0.4389 - accuracy: 0.7971 313/313 [==============================] - 1s 2ms/step - loss: 0.4390 - accuracy: 0.7970 Epoch 1/3 1/313 [..............................] - ETA: 1s - loss: 0.5348 - accuracy: 0.7188 25/313 [=>............................] - ETA: 0s - loss: 0.4457 - accuracy: 0.8087 49/313 [===>..........................] - ETA: 0s - loss: 0.4390 - accuracy: 0.8068 73/313 [=====>........................] - ETA: 0s - loss: 0.4450 - accuracy: 0.8035 97/313 [========>.....................] - ETA: 0s - loss: 0.4500 - accuracy: 0.7951 121/313 [==========>...................] - ETA: 0s - loss: 0.4481 - accuracy: 0.7980 145/313 [============>.................] - ETA: 0s - loss: 0.4477 - accuracy: 0.7972 169/313 [===============>..............] - ETA: 0s - loss: 0.4459 - accuracy: 0.7960 193/313 [=================>............] - ETA: 0s - loss: 0.4409 - accuracy: 0.7995 217/313 [===================>..........] - ETA: 0s - loss: 0.4401 - accuracy: 0.7991 241/313 [======================>.......] - ETA: 0s - loss: 0.4399 - accuracy: 0.8003 265/313 [========================>.....] - ETA: 0s - loss: 0.4416 - accuracy: 0.7983 289/313 [==========================>...] - ETA: 0s - loss: 0.4429 - accuracy: 0.7961 313/313 [==============================] - ETA: 0s - loss: 0.4434 - accuracy: 0.7965 313/313 [==============================] - 1s 2ms/step - loss: 0.4434 - accuracy: 0.7965 Epoch 2/3 1/313 [..............................] - ETA: 1s - loss: 0.4956 - accuracy: 0.7812 25/313 [=>............................] - ETA: 0s - loss: 0.4351 - accuracy: 0.8100 49/313 [===>..........................] - ETA: 0s - loss: 0.4324 - accuracy: 0.8036 73/313 [=====>........................] - ETA: 0s - loss: 0.4395 - accuracy: 0.7971 97/313 [========>.....................] - ETA: 0s - loss: 0.4477 - accuracy: 0.7906 121/313 [==========>...................] - ETA: 0s - loss: 0.4428 - accuracy: 0.7967 145/313 [============>.................] - ETA: 0s - loss: 0.4413 - accuracy: 0.7983 169/313 [===============>..............] - ETA: 0s - loss: 0.4404 - accuracy: 0.7959 193/313 [=================>............] - ETA: 0s - loss: 0.4362 - accuracy: 0.7987 217/313 [===================>..........] - ETA: 0s - loss: 0.4336 - accuracy: 0.7982 241/313 [======================>.......] - ETA: 0s - loss: 0.4354 - accuracy: 0.7978 265/313 [========================>.....] - ETA: 0s - loss: 0.4367 - accuracy: 0.7980 289/313 [==========================>...] - ETA: 0s - loss: 0.4376 - accuracy: 0.7975 313/313 [==============================] - ETA: 0s - loss: 0.4380 - accuracy: 0.7974 313/313 [==============================] - 1s 2ms/step - loss: 0.4380 - accuracy: 0.7974 Epoch 3/3 1/313 [..............................] - ETA: 1s - loss: 0.3930 - accuracy: 0.8438 25/313 [=>............................] - ETA: 0s - loss: 0.4262 - accuracy: 0.8050 49/313 [===>..........................] - ETA: 0s - loss: 0.4302 - accuracy: 0.7985 73/313 [=====>........................] - ETA: 0s - loss: 0.4316 - accuracy: 0.8018 97/313 [========>.....................] - ETA: 0s - loss: 0.4411 - accuracy: 0.7922 121/313 [==========>...................] - ETA: 0s - loss: 0.4378 - accuracy: 0.7962 145/313 [============>.................] - ETA: 0s - loss: 0.4385 - accuracy: 0.7961 168/313 [===============>..............] - ETA: 0s - loss: 0.4364 - accuracy: 0.7958 192/313 [=================>............] - ETA: 0s - loss: 0.4307 - accuracy: 0.7990 216/313 [===================>..........] - ETA: 0s - loss: 0.4311 - accuracy: 0.7992 239/313 [=====================>........] - ETA: 0s - loss: 0.4325 - accuracy: 0.7993 262/313 [========================>.....] - ETA: 0s - loss: 0.4326 - accuracy: 0.7995 285/313 [==========================>...] - ETA: 0s - loss: 0.4343 - accuracy: 0.7991 309/313 [============================>.] - ETA: 0s - loss: 0.4354 - accuracy: 0.7994 313/313 [==============================] - 1s 2ms/step - loss: 0.4354 - accuracy: 0.7992 Epoch 1/3 1/313 [..............................] - ETA: 1s - loss: 0.4726 - accuracy: 0.7500 25/313 [=>............................] - ETA: 0s - loss: 0.4629 - accuracy: 0.7788 48/313 [===>..........................] - ETA: 0s - loss: 0.4507 - accuracy: 0.7845 71/313 [=====>........................] - ETA: 0s - loss: 0.4562 - accuracy: 0.7830 95/313 [========>.....................] - ETA: 0s - loss: 0.4629 - accuracy: 0.7822 119/313 [==========>...................] - ETA: 0s - loss: 0.4553 - accuracy: 0.7865 142/313 [============>.................] - ETA: 0s - loss: 0.4522 - accuracy: 0.7898 166/313 [==============>...............] - ETA: 0s - loss: 0.4516 - accuracy: 0.7893 189/313 [=================>............] - ETA: 0s - loss: 0.4543 - accuracy: 0.7894 213/313 [===================>..........] - ETA: 0s - loss: 0.4557 - accuracy: 0.7893 237/313 [=====================>........] - ETA: 0s - loss: 0.4541 - accuracy: 0.7915 261/313 [========================>.....] - ETA: 0s - loss: 0.4525 - accuracy: 0.7915 284/313 [==========================>...] - ETA: 0s - loss: 0.4514 - accuracy: 0.7920 308/313 [============================>.] - ETA: 0s - loss: 0.4535 - accuracy: 0.7901 313/313 [==============================] - 1s 2ms/step - loss: 0.4522 - accuracy: 0.7909 Epoch 2/3 1/313 [..............................] - ETA: 1s - loss: 0.5659 - accuracy: 0.6875 24/313 [=>............................] - ETA: 0s - loss: 0.4419 - accuracy: 0.7904 48/313 [===>..........................] - ETA: 0s - loss: 0.4496 - accuracy: 0.7832 72/313 [=====>........................] - ETA: 0s - loss: 0.4534 - accuracy: 0.7812 96/313 [========>.....................] - ETA: 0s - loss: 0.4555 - accuracy: 0.7799 120/313 [==========>...................] - ETA: 0s - loss: 0.4477 - accuracy: 0.7846 144/313 [============>.................] - ETA: 0s - loss: 0.4439 - accuracy: 0.7901 168/313 [===============>..............] - ETA: 0s - loss: 0.4412 - accuracy: 0.7906 192/313 [=================>............] - ETA: 0s - loss: 0.4467 - accuracy: 0.7884 216/313 [===================>..........] - ETA: 0s - loss: 0.4469 - accuracy: 0.7892 240/313 [======================>.......] - ETA: 0s - loss: 0.4459 - accuracy: 0.7902 263/313 [========================>.....] - ETA: 0s - loss: 0.4461 - accuracy: 0.7912 286/313 [==========================>...] - ETA: 0s - loss: 0.4461 - accuracy: 0.7914 310/313 [============================>.] - ETA: 0s - loss: 0.4485 - accuracy: 0.7903 313/313 [==============================] - 1s 2ms/step - loss: 0.4475 - accuracy: 0.7910 Epoch 3/3 1/313 [..............................] - ETA: 1s - loss: 0.4420 - accuracy: 0.7500 25/313 [=>............................] - ETA: 0s - loss: 0.4426 - accuracy: 0.8012 49/313 [===>..........................] - ETA: 0s - loss: 0.4497 - accuracy: 0.7997 72/313 [=====>........................] - ETA: 0s - loss: 0.4522 - accuracy: 0.7912 96/313 [========>.....................] - ETA: 0s - loss: 0.4570 - accuracy: 0.7887 120/313 [==========>...................] - ETA: 0s - loss: 0.4471 - accuracy: 0.7917 144/313 [============>.................] - ETA: 0s - loss: 0.4440 - accuracy: 0.7947 167/313 [===============>..............] - ETA: 0s - loss: 0.4433 - accuracy: 0.7947 190/313 [=================>............] - ETA: 0s - loss: 0.4457 - accuracy: 0.7931 214/313 [===================>..........] - ETA: 0s - loss: 0.4461 - accuracy: 0.7935 237/313 [=====================>........] - ETA: 0s - loss: 0.4448 - accuracy: 0.7943 261/313 [========================>.....] - ETA: 0s - loss: 0.4439 - accuracy: 0.7944 285/313 [==========================>...] - ETA: 0s - loss: 0.4429 - accuracy: 0.7953 309/313 [============================>.] - ETA: 0s - loss: 0.4450 - accuracy: 0.7936 313/313 [==============================] - 1s 2ms/step - loss: 0.4435 - accuracy: 0.7947 Epoch 1/3 1/313 [..............................] - ETA: 1s - loss: 0.3380 - accuracy: 0.8438 25/313 [=>............................] - ETA: 0s - loss: 0.4456 - accuracy: 0.7875 49/313 [===>..........................] - ETA: 0s - loss: 0.4561 - accuracy: 0.7876 73/313 [=====>........................] - ETA: 0s - loss: 0.4586 - accuracy: 0.7872 97/313 [========>.....................] - ETA: 0s - loss: 0.4546 - accuracy: 0.7874 121/313 [==========>...................] - ETA: 0s - loss: 0.4583 - accuracy: 0.7838 145/313 [============>.................] - ETA: 0s - loss: 0.4526 - accuracy: 0.7856 168/313 [===============>..............] - ETA: 0s - loss: 0.4504 - accuracy: 0.7887 192/313 [=================>............] - ETA: 0s - loss: 0.4475 - accuracy: 0.7913 216/313 [===================>..........] - ETA: 0s - loss: 0.4512 - accuracy: 0.7894 240/313 [======================>.......] - ETA: 0s - loss: 0.4464 - accuracy: 0.7918 264/313 [========================>.....] - ETA: 0s - loss: 0.4445 - accuracy: 0.7940 288/313 [==========================>...] - ETA: 0s - loss: 0.4441 - accuracy: 0.7928 312/313 [============================>.] - ETA: 0s - loss: 0.4464 - accuracy: 0.7904 313/313 [==============================] - 1s 2ms/step - loss: 0.4464 - accuracy: 0.7906 Epoch 2/3 1/313 [..............................] - ETA: 1s - loss: 0.3898 - accuracy: 0.8125 25/313 [=>............................] - ETA: 0s - loss: 0.4381 - accuracy: 0.7937 49/313 [===>..........................] - ETA: 0s - loss: 0.4484 - accuracy: 0.7889 73/313 [=====>........................] - ETA: 0s - loss: 0.4545 - accuracy: 0.7885 97/313 [========>.....................] - ETA: 0s - loss: 0.4503 - accuracy: 0.7867 121/313 [==========>...................] - ETA: 0s - loss: 0.4551 - accuracy: 0.7851 145/313 [============>.................] - ETA: 0s - loss: 0.4501 - accuracy: 0.7894 169/313 [===============>..............] - ETA: 0s - loss: 0.4482 - accuracy: 0.7914 193/313 [=================>............] - ETA: 0s - loss: 0.4466 - accuracy: 0.7936 217/313 [===================>..........] - ETA: 0s - loss: 0.4506 - accuracy: 0.7919 241/313 [======================>.......] - ETA: 0s - loss: 0.4473 - accuracy: 0.7937 265/313 [========================>.....] - ETA: 0s - loss: 0.4456 - accuracy: 0.7949 289/313 [==========================>...] - ETA: 0s - loss: 0.4440 - accuracy: 0.7951 313/313 [==============================] - ETA: 0s - loss: 0.4467 - accuracy: 0.7922 313/313 [==============================] - 1s 2ms/step - loss: 0.4467 - accuracy: 0.7922 Epoch 3/3 1/313 [..............................] - ETA: 1s - loss: 0.4130 - accuracy: 0.7812 25/313 [=>............................] - ETA: 0s - loss: 0.4473 - accuracy: 0.7900 49/313 [===>..........................] - ETA: 0s - loss: 0.4482 - accuracy: 0.7953 74/313 [======>.......................] - ETA: 0s - loss: 0.4483 - accuracy: 0.7935 98/313 [========>.....................] - ETA: 0s - loss: 0.4467 - accuracy: 0.7924 122/313 [==========>...................] - ETA: 0s - loss: 0.4496 - accuracy: 0.7902 146/313 [============>.................] - ETA: 0s - loss: 0.4456 - accuracy: 0.7930 170/313 [===============>..............] - ETA: 0s - loss: 0.4454 - accuracy: 0.7943 194/313 [=================>............] - ETA: 0s - loss: 0.4436 - accuracy: 0.7956 218/313 [===================>..........] - ETA: 0s - loss: 0.4482 - accuracy: 0.7917 242/313 [======================>.......] - ETA: 0s - loss: 0.4438 - accuracy: 0.7948 266/313 [========================>.....] - ETA: 0s - loss: 0.4406 - accuracy: 0.7964 290/313 [==========================>...] - ETA: 0s - loss: 0.4407 - accuracy: 0.7958 313/313 [==============================] - 1s 2ms/step - loss: 0.4424 - accuracy: 0.7933 2022-01-07 20:35:04.916208: E tensorflow_io/core/kernels/kafka_kernels.cc:1001] Local: Timed out
The incrementally trained model can be saved in a periodic fashion (based on use-cases) and can be utilized to infer on the test data in either online or offline modes.
References:
Baldi, P., P. Sadowski, and D. Whiteson. “Searching for Exotic Particles in High-energy Physics with Deep Learning.” Nature Communications 5 (July 2, 2014)
SUSY Dataset: https://archive.ics.uci.edu/ml/datasets/SUSY#