¡El Día de la Comunidad de ML es el 9 de noviembre! Únase a nosotros para recibir actualizaciones de TensorFlow, JAX, y más Más información

Cree, entrene y evalúe modelos con TensorFlow Decision Forests

Ver en TensorFlow.org Ejecutar en Google Colab Ver en GitHub Descargar cuaderno

Introducción

Los bosques de decisiones (DF) son una gran familia de algoritmos de aprendizaje automático para clasificación, regresión y clasificación supervisadas. Como sugiere el nombre, los DF utilizan árboles de decisión como un bloque de construcción. Hoy en día, los dos algoritmos más populares de formación DF son aleatorios bosques y árboles de decisión de gradiente potenciadas . Ambos algoritmos son técnicas de conjunto que utilizan múltiples árboles de decisión, pero difieren en cómo lo hacen.

TensorFlow Decision Forests (TF-DF) es una biblioteca para el entrenamiento, evaluación, interpretación e inferencia de modelos de Decision Forest.

En este tutorial, aprenderá a:

  1. Entrene un bosque aleatorio de clasificación binaria en un conjunto de datos que contenga características numéricas, categóricas y faltantes.
  2. Evalúe el modelo en un conjunto de datos de prueba.
  3. Preparar el modelo para TensorFlow porción .
  4. Examine la estructura general del modelo y la importancia de cada característica.
  5. Vuelva a entrenar el modelo con un algoritmo de aprendizaje diferente (árboles de decisión potenciados por gradientes).
  6. Utilice un conjunto diferente de funciones de entrada.
  7. Cambie los hiperparámetros del modelo.
  8. Procese previamente las funciones.
  9. Entrene un modelo de regresión.
  10. Entrene un modelo para la clasificación.

La documentación detallada está disponible en el manual de usuario . El directorio de ejemplo contiene otros ejemplos de extremo a extremo.

Instalación de bosques de decisiones de TensorFlow

Instale TF-DF ejecutando la siguiente celda.

pip install tensorflow_decision_forests

Instalar Wurlitzer para mostrar los registros detallados de la formación. Esto solo es necesario en colabs.

pip install wurlitzer

Importación de bibliotecas

import tensorflow_decision_forests as tfdf

import os
import numpy as np
import pandas as pd
import tensorflow as tf
import math

try:
  from wurlitzer import sys_pipes
except:
  from colabtools.googlelog import CaptureLog as sys_pipes

from IPython.core.magic import register_line_magic
from IPython.display import Javascript

La celda de código oculta limita la altura de salida en colab.

# Check the version of TensorFlow Decision Forests
print("Found TensorFlow Decision Forests v" + tfdf.__version__)
Found TensorFlow Decision Forests v0.1.9

Entrenamiento de un modelo de bosque aleatorio

En esta sección, entrenamos, evaluar, analizar y exportar una clasificación binaria aleatoria Bosque entrenó en el Pingüinos de Palmer conjunto de datos.

Cargue el conjunto de datos y conviértalo en un tf.

Este conjunto de datos es muy pequeño (300 ejemplos) y se almacena como un archivo tipo .csv. Por lo tanto, use Pandas para cargarlo.

Ensamblemos el conjunto de datos en un archivo csv (es decir, agreguemos el encabezado) y carguemos:

# Download the dataset
!wget -q https://storage.googleapis.com/download.tensorflow.org/data/palmer_penguins/penguins.csv -O /tmp/penguins.csv

# Load a dataset into a Pandas Dataframe.
dataset_df = pd.read_csv("/tmp/penguins.csv")

# Display the first 3 examples.
dataset_df.head(3)

El conjunto de datos contiene una mezcla de numérica (por ejemplo bill_depth_mm ), categóricas (por ejemplo, island ) y características que faltan. TF-DF soporta todos estos tipos de características de forma nativa (de manera diferente que NN modelos basados), por lo tanto no hay necesidad para el preprocesamiento en forma de codificación de una sola caliente, normalización o adicional is_present función.

Las etiquetas son un poco diferentes: las métricas de Keras esperan números enteros. La etiqueta ( species ) se almacena como una cadena, así que vamos a convertirlo en un entero.

# Encode the categorical label into an integer.
#
# Details:
# This stage is necessary if your classification label is represented as a
# string. Note: Keras expected classification labels to be integers.

# Name of the label column.
label = "species"

classes = dataset_df[label].unique().tolist()
print(f"Label classes: {classes}")

dataset_df[label] = dataset_df[label].map(classes.index)
Label classes: ['Adelie', 'Gentoo', 'Chinstrap']

A continuación, divida el conjunto de datos en entrenamiento y prueba:

# Split the dataset into a training and a testing dataset.

def split_dataset(dataset, test_ratio=0.30):
  """Splits a panda dataframe in two."""
  test_indices = np.random.rand(len(dataset)) < test_ratio
  return dataset[~test_indices], dataset[test_indices]


train_ds_pd, test_ds_pd = split_dataset(dataset_df)
print("{} examples in training, {} examples for testing.".format(
    len(train_ds_pd), len(test_ds_pd)))
245 examples in training, 99 examples for testing.

Y, por último, convertir los pandas trama de datos ( pd.Dataframe ) en conjuntos de datos tensorflow ( tf.data.Dataset ):

train_ds = tfdf.keras.pd_dataframe_to_tf_dataset(train_ds_pd, label=label)
test_ds = tfdf.keras.pd_dataframe_to_tf_dataset(test_ds_pd, label=label)
/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow_decision_forests/keras/core.py:1224: FutureWarning: In a future version of pandas all arguments of DataFrame.drop except for the argument 'labels' will be keyword-only
  (dict(dataframe.drop(label, 1)), dataframe[label].values))

Notas: pd_dataframe_to_tf_dataset podría haber convertido la etiqueta de número entero para usted.

Y, si se desea crear el tf.data.Dataset usted mismo, hay un par de cosas a tener en cuenta:

  • Los algoritmos de aprendizaje funcionan con un conjunto de datos de una época y sin mezclar.
  • El tamaño del lote no afecta el algoritmo de entrenamiento, pero un valor pequeño puede ralentizar la lectura del conjunto de datos.

Entrena el modelo

%set_cell_height 300

# Specify the model.
model_1 = tfdf.keras.RandomForestModel()

# Optionally, add evaluation metrics.
model_1.compile(
    metrics=["accuracy"])

# Train the model.
# "sys_pipes" is optional. It enables the display of the training logs.
with sys_pipes():
  model_1.fit(x=train_ds)
<IPython.core.display.Javascript object>
4/4 [==============================] - 3s 2ms/step
[INFO kernel.cc:746] Start Yggdrasil model training
[INFO kernel.cc:747] Collect training examples
[INFO kernel.cc:392] Number of batches: 4
[INFO kernel.cc:393] Number of examples: 245
[INFO kernel.cc:769] Dataset:
Number of records: 245
Number of columns: 8

Number of columns by type:
    NUMERICAL: 5 (62.5%)
    CATEGORICAL: 3 (37.5%)

Columns:

NUMERICAL: 5 (62.5%)
    0: "bill_depth_mm" NUMERICAL num-nas:1 (0.408163%) mean:17.1254 min:13.2 max:21.5 sd:2.00348
    1: "bill_length_mm" NUMERICAL num-nas:1 (0.408163%) mean:43.8016 min:32.1 max:59.6 sd:5.48753
    2: "body_mass_g" NUMERICAL num-nas:1 (0.408163%) mean:4185.96 min:2850 max:6300 sd:811.612
    3: "flipper_length_mm" NUMERICAL num-nas:1 (0.408163%) mean:201.061 min:172 max:231 sd:14.3124
    6: "year" NUMERICAL mean:2008.09 min:2007 max:2009 sd:0.828139

CATEGORICAL: 3 (37.5%)
    4: "island" CATEGORICAL has-dict vocab-size:4 zero-ood-items most-frequent:"Biscoe" 121 (49.3878%)
    5: "sex" CATEGORICAL num-nas:7 (2.85714%) has-dict vocab-size:3 zero-ood-items most-frequent:"male" 120 (50.4202%)
    7: "__LABEL" CATEGORICAL integerized vocab-size:4 no-ood-item

Terminology:
    nas: Number of non-available (i.e. missing) values.
    ood: Out of dictionary.
    manually-defined: Attribute which type is manually defined by the user i.e. the type was not automatically inferred.
    tokenized: The attribute value is obtained through tokenization.
    has-dict: The attribute is attached to a string dictionary e.g. a categorical attribute stored as a string.
    vocab-size: Number of unique values.

[INFO kernel.cc:772] Configure learner
[INFO kernel.cc:797] Training config:
learner: "RANDOM_FOREST"
features: "bill_depth_mm"
features: "bill_length_mm"
features: "body_mass_g"
features: "flipper_length_mm"
features: "island"
features: "sex"
features: "year"
label: "__LABEL"
task: CLASSIFICATION
[yggdrasil_decision_forests.model.random_forest.proto.random_forest_config] {
  num_trees: 300
  decision_tree {
    max_depth: 16
    min_examples: 5
    in_split_min_examples_check: true
    missing_value_policy: GLOBAL_IMPUTATION
    allow_na_conditions: false
    categorical_set_greedy_forward {
      sampling: 0.1
      max_num_items: -1
      min_item_frequency: 1
    }
    growing_strategy_local {
    }
    categorical {
      cart {
      }
    }
    num_candidate_attributes_ratio: -1
    axis_aligned_split {
    }
    internal {
      sorting_strategy: PRESORTED
    }
  }
  winner_take_all_inference: true
  compute_oob_performances: true
  compute_oob_variable_importances: false
  adapt_bootstrap_size_ratio_for_maximum_training_duration: false
}

[INFO kernel.cc:800] Deployment config:
num_threads: 6

[INFO kernel.cc:837] Train model
[INFO random_forest.cc:303] Training random forest on 245 example(s) and 7 feature(s).
[INFO random_forest.cc:578] Training of tree  1/300 (tree index:3) done accuracy:0.965116 logloss:1.25734
[INFO random_forest.cc:578] Training of tree  11/300 (tree index:12) done accuracy:0.963115 logloss:0.638779
[INFO random_forest.cc:578] Training of tree  21/300 (tree index:19) done accuracy:0.963265 logloss:0.351662
[INFO random_forest.cc:578] Training of tree  31/300 (tree index:29) done accuracy:0.97551 logloss:0.223883
[INFO random_forest.cc:578] Training of tree  41/300 (tree index:40) done accuracy:0.97551 logloss:0.226019
[INFO random_forest.cc:578] Training of tree  51/300 (tree index:51) done accuracy:0.97551 logloss:0.221203
[INFO random_forest.cc:578] Training of tree  61/300 (tree index:59) done accuracy:0.971429 logloss:0.221955
[INFO random_forest.cc:578] Training of tree  71/300 (tree index:69) done accuracy:0.971429 logloss:0.220639
[INFO random_forest.cc:578] Training of tree  81/300 (tree index:81) done accuracy:0.97551 logloss:0.217759
[INFO random_forest.cc:578] Training of tree  91/300 (tree index:90) done accuracy:0.97551 logloss:0.0837968
[INFO random_forest.cc:578] Training of tree  101/300 (tree index:100) done accuracy:0.979592 logloss:0.0838433
[INFO random_forest.cc:578] Training of tree  111/300 (tree index:111) done accuracy:0.97551 logloss:0.0837273
[INFO random_forest.cc:578] Training of tree  121/300 (tree index:120) done accuracy:0.97551 logloss:0.0845685
[INFO random_forest.cc:578] Training of tree  131/300 (tree index:129) done accuracy:0.979592 logloss:0.0821311
[INFO random_forest.cc:578] Training of tree  141/300 (tree index:141) done accuracy:0.979592 logloss:0.0831507
[INFO random_forest.cc:578] Training of tree  151/300 (tree index:151) done accuracy:0.979592 logloss:0.0843571
[INFO random_forest.cc:578] Training of tree  161/300 (tree index:162) done accuracy:0.979592 logloss:0.0799464
[INFO random_forest.cc:578] Training of tree  171/300 (tree index:171) done accuracy:0.979592 logloss:0.0784707
[INFO random_forest.cc:578] Training of tree  181/300 (tree index:181) done accuracy:0.979592 logloss:0.0786835
[INFO random_forest.cc:578] Training of tree  191/300 (tree index:190) done accuracy:0.979592 logloss:0.0785385
[INFO random_forest.cc:578] Training of tree  201/300 (tree index:200) done accuracy:0.983673 logloss:0.0787414
[INFO random_forest.cc:578] Training of tree  211/300 (tree index:209) done accuracy:0.983673 logloss:0.0780315
[INFO random_forest.cc:578] Training of tree  221/300 (tree index:220) done accuracy:0.983673 logloss:0.0780052
[INFO random_forest.cc:578] Training of tree  231/300 (tree index:230) done accuracy:0.983673 logloss:0.0769405
[INFO random_forest.cc:578] Training of tree  241/300 (tree index:239) done accuracy:0.983673 logloss:0.0763699
[INFO random_forest.cc:578] Training of tree  251/300 (tree index:246) done accuracy:0.983673 logloss:0.0746821
[INFO random_forest.cc:578] Training of tree  261/300 (tree index:260) done accuracy:0.983673 logloss:0.0724621
[INFO random_forest.cc:578] Training of tree  271/300 (tree index:268) done accuracy:0.983673 logloss:0.0712471
[INFO random_forest.cc:578] Training of tree  281/300 (tree index:280) done accuracy:0.983673 logloss:0.0713778
[INFO random_forest.cc:578] Training of tree  291/300 (tree index:290) done accuracy:0.983673 logloss:0.0715795
[INFO random_forest.cc:578] Training of tree  300/300 (tree index:297) done accuracy:0.983673 logloss:0.0716408
[INFO random_forest.cc:645] Final OOB metrics: accuracy:0.983673 logloss:0.0716408
[INFO kernel.cc:856] Export model in log directory: /tmp/tmp04y_06pr
[INFO kernel.cc:864] Save model in resources
[INFO kernel.cc:988] Loading model from path
[INFO decision_forest.cc:590] Model loaded with 300 root(s), 4080 node(s), and 7 input feature(s).
[INFO abstract_model.cc:993] Engine "RandomForestGeneric" built
[INFO kernel.cc:848] Use fast generic engine

Observaciones

  • No se especifican características de entrada. Por lo tanto, todas las columnas se utilizarán como características de entrada excepto la etiqueta. La función utilizada por el modelo se muestran en los registros de entrenamiento y en el model.summary() .
  • Los DF consumen características nativas numéricas, categóricas, de conjuntos categóricos y valores perdidos. No es necesario normalizar las características numéricas. Los valores de cadenas categóricas no necesitan estar codificados en un diccionario.
  • No se especifican hiperparámetros de entrenamiento. Por lo tanto, se utilizarán los hiperparámetros predeterminados. Los hiperparámetros predeterminados proporcionan resultados razonables en la mayoría de las situaciones.
  • Llamando compile en el modelo antes de que el fit es opcional. Compilar se puede utilizar para proporcionar métricas de evaluación adicionales.
  • Los algoritmos de entrenamiento no necesitan conjuntos de datos de validación. Si se proporciona un conjunto de datos de validación, solo se utilizará para mostrar métricas.

Evaluar el modelo

Evaluemos nuestro modelo en el conjunto de datos de prueba.

evaluation = model_1.evaluate(test_ds, return_dict=True)
print()

for name, value in evaluation.items():
  print(f"{name}: {value:.4f}")
2/2 [==============================] - 0s 4ms/step - loss: 0.0000e+00 - accuracy: 0.9798

loss: 0.0000
accuracy: 0.9798

Observación: La exactitud de la prueba (0,86514) está cerca de la exactitud fuera de la bolsa (0.8672) se muestra en los registros de entrenamiento.

Vea la sección de modelo de auto evaluación a continuación para más métodos de evaluación.

Prepara este modelo para la publicación de TensorFlow.

Exportar el modelo al formato SavedModel para su reutilización posterior por ejemplo TensorFlow Servir .

model_1.save("/tmp/my_saved_model")
2021-09-01 11:09:36.332158: W tensorflow/python/util/util.cc:348] Sets are not currently considered sequences, but this may change in the future, so consider avoiding using them.
INFO:tensorflow:Assets written to: /tmp/my_saved_model/assets
INFO:tensorflow:Assets written to: /tmp/my_saved_model/assets

Trazar el modelo

Trazar un árbol de decisiones y seguir las primeras ramas ayuda a aprender sobre los bosques de decisiones. En algunos casos, trazar un modelo incluso se puede utilizar para depurar.

Debido a la diferencia en la forma en que se entrenan, algunos modelos son más interesantes de planificar que otros. Debido al ruido inyectado durante el entrenamiento y la profundidad de los árboles, trazar Random Forest es menos informativo que trazar un CART o el primer árbol de un Gradient Boosted Tree.

No obstante, tracemos el primer árbol de nuestro modelo de bosque aleatorio:

tfdf.model_plotter.plot_model_in_colab(model_1, tree_idx=0, max_depth=3)

El nodo raíz de la izquierda contiene la primera condición ( bill_depth_mm >= 16.55 ), número de ejemplos (240) y de distribución de etiquetas (la barra de rojo-azul-verde).

Ejemplos que evalúa fiel a bill_depth_mm >= 16.55 están ramificados a la ruta verde. Los otros se ramifican al camino rojo.

Cuanto más profundo es el nodo, el más pure que se conviertan es decir, la distribución de etiquetas está sesgada hacia un subconjunto de las clases.

Importancia de la estructura y la característica del modelo

La estructura general del modelo es espectáculo con .summary() . Ya verás:

  • Tipo: El algoritmo de aprendizaje utilizado para entrenar el modelo ( Random Forest en nuestro caso).
  • Tarea: El problema resuelto por el modelo ( Classification en nuestro caso).
  • Características de entrada: La entrada de características del modelo.
  • Importancia de la variable: Diferentes medidas de la importancia de cada característica para el modelo.
  • Fuera de la bolsa de evaluación: La evaluación fuera de la bolsa del modelo. Esta es una alternativa económica y eficiente a la validación cruzada.
  • Número de árboles, nodos {} y otras métricas: Estadísticas sobre la estructura de los bosques de decisiones.

Observación: El contenido del resumen depende del algoritmo de aprendizaje (por ejemplo, fuera de la bolsa sólo está disponible para Random Bosque) y las hiper-parámetros (por ejemplo, la variable de importancia media-descenso-en-exactitud se puede desactivar en los hiper-parámetros) .

%set_cell_height 300
model_1.summary()
<IPython.core.display.Javascript object>
Model: "random_forest_model"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
Total params: 1
Trainable params: 0
Non-trainable params: 1
_________________________________________________________________
Type: "RANDOM_FOREST"
Task: CLASSIFICATION
Label: "__LABEL"

Input Features (7):
    bill_depth_mm
    bill_length_mm
    body_mass_g
    flipper_length_mm
    island
    sex
    year

No weights

Variable Importance: MEAN_MIN_DEPTH:

    1.           "__LABEL"  3.082386 ################
    2.              "year"  3.066715 ###############
    3.               "sex"  3.044466 ###############
    4.       "body_mass_g"  2.632450 ############
    5.     "bill_depth_mm"  2.331007 #########
    6.            "island"  2.058386 #######
    7.    "bill_length_mm"  1.283269 
    8. "flipper_length_mm"  1.188279 

Variable Importance: NUM_AS_ROOT:

    1. "flipper_length_mm" 166.000000 ################
    2.    "bill_length_mm" 88.000000 ########
    3.     "bill_depth_mm" 29.000000 ##
    4.            "island" 10.000000 
    5.       "body_mass_g"  7.000000 

Variable Importance: NUM_NODES:

    1.    "bill_length_mm" 645.000000 ################
    2. "flipper_length_mm" 343.000000 ########
    3.            "island" 341.000000 ########
    4.     "bill_depth_mm" 315.000000 #######
    5.       "body_mass_g" 204.000000 ####
    6.               "sex" 25.000000 
    7.              "year" 17.000000 

Variable Importance: SUM_SCORE:

    1.    "bill_length_mm" 26131.849414 ################
    2. "flipper_length_mm" 25237.745342 ###############
    3.            "island" 12402.388541 #######
    4.     "bill_depth_mm" 6274.353085 ###
    5.       "body_mass_g" 2771.279828 #
    6.               "sex" 157.155306 
    7.              "year" 44.959749 



Winner take all: true
Out-of-bag evaluation: accuracy:0.983673 logloss:0.0716408
Number of trees: 300
Total number of nodes: 4080

Number of nodes by tree:
Count: 300 Average: 13.6 StdDev: 2.75439
Min: 7 Max: 23 Ignored: 0
----------------------------------------------
[  7,  8)  1   0.33%   0.33%
[  8,  9)  0   0.00%   0.33%
[  9, 10) 22   7.33%   7.67% ###
[ 10, 11)  0   0.00%   7.67%
[ 11, 12) 66  22.00%  29.67% ########
[ 12, 13)  0   0.00%  29.67%
[ 13, 14) 88  29.33%  59.00% ##########
[ 14, 15)  0   0.00%  59.00%
[ 15, 16) 71  23.67%  82.67% ########
[ 16, 17)  0   0.00%  82.67%
[ 17, 18) 30  10.00%  92.67% ###
[ 18, 19)  0   0.00%  92.67%
[ 19, 20) 17   5.67%  98.33% ##
[ 20, 21)  0   0.00%  98.33%
[ 21, 22)  4   1.33%  99.67%
[ 22, 23)  0   0.00%  99.67%
[ 23, 23]  1   0.33% 100.00%

Depth by leafs:
Count: 2190 Average: 3.14155 StdDev: 0.926554
Min: 1 Max: 6 Ignored: 0
----------------------------------------------
[ 1, 2)  10   0.46%   0.46%
[ 2, 3) 606  27.67%  28.13% ########
[ 3, 4) 803  36.67%  64.79% ##########
[ 4, 5) 612  27.95%  92.74% ########
[ 5, 6) 153   6.99%  99.73% ##
[ 6, 6]   6   0.27% 100.00%

Number of training obs by leaf:
Count: 2190 Average: 33.5616 StdDev: 35.139
Min: 5 Max: 120 Ignored: 0
----------------------------------------------
[   5,  10) 1056  48.22%  48.22% ##########
[  10,  16)  107   4.89%  53.11% #
[  16,  22)   61   2.79%  55.89% #
[  22,  28)   82   3.74%  59.63% #
[  28,  34)  101   4.61%  64.25% #
[  34,  39)   76   3.47%  67.72% #
[  39,  45)   68   3.11%  70.82% #
[  45,  51)   39   1.78%  72.60%
[  51,  57)   16   0.73%  73.33%
[  57,  63)   17   0.78%  74.11%
[  63,  68)   23   1.05%  75.16%
[  68,  74)   46   2.10%  77.26%
[  74,  80)   76   3.47%  80.73% #
[  80,  86)   91   4.16%  84.89% #
[  86,  92)   92   4.20%  89.09% #
[  92,  97)   88   4.02%  93.11% #
[  97, 103)   64   2.92%  96.03% #
[ 103, 109)   54   2.47%  98.49% #
[ 109, 115)   28   1.28%  99.77%
[ 115, 120]    5   0.23% 100.00%

Attribute in nodes:
    645 : bill_length_mm [NUMERICAL]
    343 : flipper_length_mm [NUMERICAL]
    341 : island [CATEGORICAL]
    315 : bill_depth_mm [NUMERICAL]
    204 : body_mass_g [NUMERICAL]
    25 : sex [CATEGORICAL]
    17 : year [NUMERICAL]

Attribute in nodes with depth <= 0:
    166 : flipper_length_mm [NUMERICAL]
    88 : bill_length_mm [NUMERICAL]
    29 : bill_depth_mm [NUMERICAL]
    10 : island [CATEGORICAL]
    7 : body_mass_g [NUMERICAL]

Attribute in nodes with depth <= 1:
    252 : bill_length_mm [NUMERICAL]
    236 : flipper_length_mm [NUMERICAL]
    205 : island [CATEGORICAL]
    139 : bill_depth_mm [NUMERICAL]
    58 : body_mass_g [NUMERICAL]

Attribute in nodes with depth <= 2:
    461 : bill_length_mm [NUMERICAL]
    321 : island [CATEGORICAL]
    295 : flipper_length_mm [NUMERICAL]
    220 : bill_depth_mm [NUMERICAL]
    155 : body_mass_g [NUMERICAL]
    7 : sex [CATEGORICAL]
    5 : year [NUMERICAL]

Attribute in nodes with depth <= 3:
    607 : bill_length_mm [NUMERICAL]
    340 : island [CATEGORICAL]
    333 : flipper_length_mm [NUMERICAL]
    297 : bill_depth_mm [NUMERICAL]
    193 : body_mass_g [NUMERICAL]
    25 : sex [CATEGORICAL]
    14 : year [NUMERICAL]

Attribute in nodes with depth <= 5:
    645 : bill_length_mm [NUMERICAL]
    343 : flipper_length_mm [NUMERICAL]
    341 : island [CATEGORICAL]
    315 : bill_depth_mm [NUMERICAL]
    204 : body_mass_g [NUMERICAL]
    25 : sex [CATEGORICAL]
    17 : year [NUMERICAL]

Condition type in nodes:
    1524 : HigherCondition
    366 : ContainsBitmapCondition
Condition type in nodes with depth <= 0:
    290 : HigherCondition
    10 : ContainsBitmapCondition
Condition type in nodes with depth <= 1:
    685 : HigherCondition
    205 : ContainsBitmapCondition
Condition type in nodes with depth <= 2:
    1136 : HigherCondition
    328 : ContainsBitmapCondition
Condition type in nodes with depth <= 3:
    1444 : HigherCondition
    365 : ContainsBitmapCondition
Condition type in nodes with depth <= 5:
    1524 : HigherCondition
    366 : ContainsBitmapCondition
Node format: NOT_SET

Training OOB:
    trees: 1, Out-of-bag evaluation: accuracy:0.965116 logloss:1.25734
    trees: 11, Out-of-bag evaluation: accuracy:0.963115 logloss:0.638779
    trees: 21, Out-of-bag evaluation: accuracy:0.963265 logloss:0.351662
    trees: 31, Out-of-bag evaluation: accuracy:0.97551 logloss:0.223883
    trees: 41, Out-of-bag evaluation: accuracy:0.97551 logloss:0.226019
    trees: 51, Out-of-bag evaluation: accuracy:0.97551 logloss:0.221203
    trees: 61, Out-of-bag evaluation: accuracy:0.971429 logloss:0.221955
    trees: 71, Out-of-bag evaluation: accuracy:0.971429 logloss:0.220639
    trees: 81, Out-of-bag evaluation: accuracy:0.97551 logloss:0.217759
    trees: 91, Out-of-bag evaluation: accuracy:0.97551 logloss:0.0837968
    trees: 101, Out-of-bag evaluation: accuracy:0.979592 logloss:0.0838433
    trees: 111, Out-of-bag evaluation: accuracy:0.97551 logloss:0.0837273
    trees: 121, Out-of-bag evaluation: accuracy:0.97551 logloss:0.0845685
    trees: 131, Out-of-bag evaluation: accuracy:0.979592 logloss:0.0821311
    trees: 141, Out-of-bag evaluation: accuracy:0.979592 logloss:0.0831507
    trees: 151, Out-of-bag evaluation: accuracy:0.979592 logloss:0.0843571
    trees: 161, Out-of-bag evaluation: accuracy:0.979592 logloss:0.0799464
    trees: 171, Out-of-bag evaluation: accuracy:0.979592 logloss:0.0784707
    trees: 181, Out-of-bag evaluation: accuracy:0.979592 logloss:0.0786835
    trees: 191, Out-of-bag evaluation: accuracy:0.979592 logloss:0.0785385
    trees: 201, Out-of-bag evaluation: accuracy:0.983673 logloss:0.0787414
    trees: 211, Out-of-bag evaluation: accuracy:0.983673 logloss:0.0780315
    trees: 221, Out-of-bag evaluation: accuracy:0.983673 logloss:0.0780052
    trees: 231, Out-of-bag evaluation: accuracy:0.983673 logloss:0.0769405
    trees: 241, Out-of-bag evaluation: accuracy:0.983673 logloss:0.0763699
    trees: 251, Out-of-bag evaluation: accuracy:0.983673 logloss:0.0746821
    trees: 261, Out-of-bag evaluation: accuracy:0.983673 logloss:0.0724621
    trees: 271, Out-of-bag evaluation: accuracy:0.983673 logloss:0.0712471
    trees: 281, Out-of-bag evaluation: accuracy:0.983673 logloss:0.0713778
    trees: 291, Out-of-bag evaluation: accuracy:0.983673 logloss:0.0715795
    trees: 300, Out-of-bag evaluation: accuracy:0.983673 logloss:0.0716408

La información de summary están todos disponibles mediante programación utilizando el modelo inspector:

# The input features
model_1.make_inspector().features()
["bill_depth_mm" (1; #0),
 "bill_length_mm" (1; #1),
 "body_mass_g" (1; #2),
 "flipper_length_mm" (1; #3),
 "island" (4; #4),
 "sex" (4; #5),
 "year" (1; #6)]
# The feature importances
model_1.make_inspector().variable_importances()
{'NUM_AS_ROOT': [("flipper_length_mm" (1; #3), 166.0),
  ("bill_length_mm" (1; #1), 88.0),
  ("bill_depth_mm" (1; #0), 29.0),
  ("island" (4; #4), 10.0),
  ("body_mass_g" (1; #2), 7.0)],
 'MEAN_MIN_DEPTH': [("__LABEL" (4; #7), 3.0823862433862366),
  ("year" (1; #6), 3.066715367965361),
  ("sex" (4; #5), 3.0444656084656025),
  ("body_mass_g" (1; #2), 2.6324502164502137),
  ("bill_depth_mm" (1; #0), 2.3310072150072134),
  ("island" (4; #4), 2.058385762385763),
  ("bill_length_mm" (1; #1), 1.2832685185185186),
  ("flipper_length_mm" (1; #3), 1.1882787397787404)],
 'SUM_SCORE': [("bill_length_mm" (1; #1), 26131.84941410646),
  ("flipper_length_mm" (1; #3), 25237.745342441835),
  ("island" (4; #4), 12402.388540707529),
  ("bill_depth_mm" (1; #0), 6274.353085439652),
  ("body_mass_g" (1; #2), 2771.279828016646),
  ("sex" (4; #5), 157.15530564822257),
  ("year" (1; #6), 44.95974919386208)],
 'NUM_NODES': [("bill_length_mm" (1; #1), 645.0),
  ("flipper_length_mm" (1; #3), 343.0),
  ("island" (4; #4), 341.0),
  ("bill_depth_mm" (1; #0), 315.0),
  ("body_mass_g" (1; #2), 204.0),
  ("sex" (4; #5), 25.0),
  ("year" (1; #6), 17.0)]}

El contenido del resumen y el inspector depende del algoritmo de aprendizaje ( tfdf.keras.RandomForestModel en este caso) y sus hiper-parámetros (por ejemplo compute_oob_variable_importances=True disparará el cálculo de importancias variables Fuera de la bolsa para el alumno Random Forest ).

Autoevaluación del modelo

Durante el entrenamiento, los modelos TFDF puede autoevaluarse incluso si no hay validación de datos se proporciona al fit() método. La lógica exacta depende del modelo. Por ejemplo, Random Forest usará la evaluación fuera de la bolsa, mientras que los árboles mejorados con gradiente usarán la validación interna de trenes.

La evaluación del modelo de auto está disponible con el inspector de evaluation() :

model_1.make_inspector().evaluation()
Evaluation(num_examples=245, accuracy=0.9836734693877551, loss=0.07164076633409273, rmse=None, ndcg=None, aucs=None)

Trazar los registros de entrenamiento

Los registros de entrenamiento muestran la calidad del modelo (por ejemplo, la precisión evaluada en el conjunto de datos de validación o fuera de la bolsa) de acuerdo con el número de árboles en el modelo. Estos registros son útiles para estudiar el equilibrio entre el tamaño del modelo y la calidad del modelo.

Los registros están disponibles de varias formas:

  1. De preferencia durante el entrenamiento si fit() está envuelto en with sys_pipes(): (ver ejemplo anterior).
  2. Al final del resumen modelo es decir model.summary() (ver ejemplo anterior).
  3. Mediante programación, utilizando el modelo inspector es decir model.make_inspector().training_logs() .
  4. usando TensorBoard

Probemos las opciones 2 y 3:

%set_cell_height 150
model_1.make_inspector().training_logs()
<IPython.core.display.Javascript object>
[TrainLog(num_trees=1, evaluation=Evaluation(num_examples=86, accuracy=0.9651162790697675, loss=1.2573366830515307, rmse=None, ndcg=None, aucs=None)),
 TrainLog(num_trees=11, evaluation=Evaluation(num_examples=244, accuracy=0.9631147540983607, loss=0.638778987600178, rmse=None, ndcg=None, aucs=None)),
 TrainLog(num_trees=21, evaluation=Evaluation(num_examples=245, accuracy=0.963265306122449, loss=0.3516620799594996, rmse=None, ndcg=None, aucs=None)),
 TrainLog(num_trees=31, evaluation=Evaluation(num_examples=245, accuracy=0.9755102040816327, loss=0.2238831247298085, rmse=None, ndcg=None, aucs=None)),
 TrainLog(num_trees=41, evaluation=Evaluation(num_examples=245, accuracy=0.9755102040816327, loss=0.22601929421023448, rmse=None, ndcg=None, aucs=None)),
 TrainLog(num_trees=51, evaluation=Evaluation(num_examples=245, accuracy=0.9755102040816327, loss=0.22120297957135707, rmse=None, ndcg=None, aucs=None)),
 TrainLog(num_trees=61, evaluation=Evaluation(num_examples=245, accuracy=0.9714285714285714, loss=0.2219547504521146, rmse=None, ndcg=None, aucs=None)),
 TrainLog(num_trees=71, evaluation=Evaluation(num_examples=245, accuracy=0.9714285714285714, loss=0.22063938636256725, rmse=None, ndcg=None, aucs=None)),
 TrainLog(num_trees=81, evaluation=Evaluation(num_examples=245, accuracy=0.9755102040816327, loss=0.21775881735493943, rmse=None, ndcg=None, aucs=None)),
 TrainLog(num_trees=91, evaluation=Evaluation(num_examples=245, accuracy=0.9755102040816327, loss=0.08379678393961215, rmse=None, ndcg=None, aucs=None)),
 TrainLog(num_trees=101, evaluation=Evaluation(num_examples=245, accuracy=0.9795918367346939, loss=0.08384326424221603, rmse=None, ndcg=None, aucs=None)),
 TrainLog(num_trees=111, evaluation=Evaluation(num_examples=245, accuracy=0.9755102040816327, loss=0.0837273224199913, rmse=None, ndcg=None, aucs=None)),
 TrainLog(num_trees=121, evaluation=Evaluation(num_examples=245, accuracy=0.9755102040816327, loss=0.08456851608899175, rmse=None, ndcg=None, aucs=None)),
 TrainLog(num_trees=131, evaluation=Evaluation(num_examples=245, accuracy=0.9795918367346939, loss=0.08213109902429337, rmse=None, ndcg=None, aucs=None)),
 TrainLog(num_trees=141, evaluation=Evaluation(num_examples=245, accuracy=0.9795918367346939, loss=0.08315070383250714, rmse=None, ndcg=None, aucs=None)),
 TrainLog(num_trees=151, evaluation=Evaluation(num_examples=245, accuracy=0.9795918367346939, loss=0.08435713196439402, rmse=None, ndcg=None, aucs=None)),
 TrainLog(num_trees=161, evaluation=Evaluation(num_examples=245, accuracy=0.9795918367346939, loss=0.07994639682678543, rmse=None, ndcg=None, aucs=None)),
 TrainLog(num_trees=171, evaluation=Evaluation(num_examples=245, accuracy=0.9795918367346939, loss=0.07847065808897724, rmse=None, ndcg=None, aucs=None)),
 TrainLog(num_trees=181, evaluation=Evaluation(num_examples=245, accuracy=0.9795918367346939, loss=0.07868352722452611, rmse=None, ndcg=None, aucs=None)),
 TrainLog(num_trees=191, evaluation=Evaluation(num_examples=245, accuracy=0.9795918367346939, loss=0.07853846467879354, rmse=None, ndcg=None, aucs=None)),
 TrainLog(num_trees=201, evaluation=Evaluation(num_examples=245, accuracy=0.9836734693877551, loss=0.0787414179582681, rmse=None, ndcg=None, aucs=None)),
 TrainLog(num_trees=211, evaluation=Evaluation(num_examples=245, accuracy=0.9836734693877551, loss=0.07803151620818036, rmse=None, ndcg=None, aucs=None)),
 TrainLog(num_trees=221, evaluation=Evaluation(num_examples=245, accuracy=0.9836734693877551, loss=0.07800520779374912, rmse=None, ndcg=None, aucs=None)),
 TrainLog(num_trees=231, evaluation=Evaluation(num_examples=245, accuracy=0.9836734693877551, loss=0.07694047748358274, rmse=None, ndcg=None, aucs=None)),
 TrainLog(num_trees=241, evaluation=Evaluation(num_examples=245, accuracy=0.9836734693877551, loss=0.07636986605594961, rmse=None, ndcg=None, aucs=None)),
 TrainLog(num_trees=251, evaluation=Evaluation(num_examples=245, accuracy=0.9836734693877551, loss=0.07468211872465148, rmse=None, ndcg=None, aucs=None)),
 TrainLog(num_trees=261, evaluation=Evaluation(num_examples=245, accuracy=0.9836734693877551, loss=0.07246212887064535, rmse=None, ndcg=None, aucs=None)),
 TrainLog(num_trees=271, evaluation=Evaluation(num_examples=245, accuracy=0.9836734693877551, loss=0.07124707738735846, rmse=None, ndcg=None, aucs=None)),
 TrainLog(num_trees=281, evaluation=Evaluation(num_examples=245, accuracy=0.9836734693877551, loss=0.07137784018200272, rmse=None, ndcg=None, aucs=None)),
 TrainLog(num_trees=291, evaluation=Evaluation(num_examples=245, accuracy=0.9836734693877551, loss=0.07157952656247178, rmse=None, ndcg=None, aucs=None)),
 TrainLog(num_trees=300, evaluation=Evaluation(num_examples=245, accuracy=0.9836734693877551, loss=0.07164076633409273, rmse=None, ndcg=None, aucs=None))]

Vamos a trazarlo:

import matplotlib.pyplot as plt

logs = model_1.make_inspector().training_logs()

plt.figure(figsize=(12, 4))

plt.subplot(1, 2, 1)
plt.plot([log.num_trees for log in logs], [log.evaluation.accuracy for log in logs])
plt.xlabel("Number of trees")
plt.ylabel("Accuracy (out-of-bag)")

plt.subplot(1, 2, 2)
plt.plot([log.num_trees for log in logs], [log.evaluation.loss for log in logs])
plt.xlabel("Number of trees")
plt.ylabel("Logloss (out-of-bag)")

plt.show()

png

Este conjunto de datos es pequeño. Puede ver el modelo convergiendo casi de inmediato.

Usemos TensorBoard:

# This cell start TensorBoard that can be slow.
# Load the TensorBoard notebook extension
%load_ext tensorboard
# Google internal version
# %load_ext google3.learning.brain.tensorboard.notebook.extension
# Clear existing results (if any)
rm -fr "/tmp/tensorboard_logs"
# Export the meta-data to tensorboard.
model_1.make_inspector().export_to_tensorboard("/tmp/tensorboard_logs")
# Start a tensorboard instance.
%tensorboard --logdir "/tmp/tensorboard_logs"

Vuelva a entrenar el modelo con un algoritmo de aprendizaje diferente

El algoritmo de aprendizaje está definido por la clase modelo. Por ejemplo, tfdf.keras.RandomForestModel() entrena a un bosque aleatorio, mientras que tfdf.keras.GradientBoostedTreesModel() entrena un gradiente Impulsado árboles de decisión.

Los algoritmos de aprendizaje se enumeran llamando tfdf.keras.get_all_models() o en la lista de alumno .

tfdf.keras.get_all_models()
[tensorflow_decision_forests.keras.RandomForestModel,
 tensorflow_decision_forests.keras.GradientBoostedTreesModel,
 tensorflow_decision_forests.keras.CartModel]

La descripción de los algoritmos de aprendizaje y sus hiper-parámetros también están disponibles en la referencia de la API y la orden interna ayuda:

# help works anywhere.
help(tfdf.keras.RandomForestModel)

# ? only works in ipython or notebooks, it usually opens on a separate panel.
tfdf.keras.RandomForestModel?
Help on class RandomForestModel in module tensorflow_decision_forests.keras:

class RandomForestModel(tensorflow_decision_forests.keras.wrappers.RandomForestModel)
 |  RandomForestModel(*args, **kwargs)
 |  
 |  Random Forest learning algorithm.
 |  
 |  A Random Forest (https://www.stat.berkeley.edu/~breiman/randomforest2001.pdf)
 |  is a collection of deep CART decision trees trained independently and without
 |  pruning. Each tree is trained on a random subset of the original training 
 |  dataset (sampled with replacement).
 |  
 |  The algorithm is unique in that it is robust to overfitting, even in extreme
 |  cases e.g. when there is more features than training examples.
 |  
 |  It is probably the most well-known of the Decision Forest training
 |  algorithms.
 |  
 |  Usage example:
 |  
 |  ```python
 |  import tensorflow_decision_forests as tfdf
 |  import pandas as pd
 |  
 |  dataset = pd.read_csv("project/dataset.csv")
 |  tf_dataset = tfdf.keras.pd_dataframe_to_tf_dataset(dataset, label="my_label")
 |  
 |  model = tfdf.keras.RandomForestModel()
 |  model.fit(tf_dataset)
 |  
 |  print(model.summary())
 |  ```
 |  
 |  Attributes:
 |    task: Task to solve (e.g. Task.CLASSIFICATION, Task.REGRESSION,
 |      Task.RANKING).
 |    features: Specify the list and semantic of the input features of the model.
 |      If not specified, all the available features will be used. If specified
 |      and if `exclude_non_specified_features=True`, only the features in
 |      `features` will be used by the model. If "preprocessing" is used,
 |      `features` corresponds to the output of the preprocessing. In this case,
 |      it is recommended for the preprocessing to return a dictionary of tensors.
 |    exclude_non_specified_features: If true, only use the features specified in
 |      `features`.
 |    preprocessing: Functional keras model or @tf.function to apply on the input
 |      feature before the model to train. This preprocessing model can consume
 |      and return tensors, list of tensors or dictionary of tensors. If
 |      specified, the model only "sees" the output of the preprocessing (and not
 |      the raw input). Can be used to prepare the features or to stack multiple
 |      models on top of each other. Unlike preprocessing done in the tf.dataset,
 |      the operation in "preprocessing" are serialized with the model.
 |    postprocessing: Like "preprocessing" but applied on the model output.
 |    ranking_group: Only for `task=Task.RANKING`. Name of a tf.string feature that
 |      identifies queries in a query/document ranking task. The ranking group
 |      is not added automatically for the set of features if
 |      `exclude_non_specified_features=false`.
 |    temp_directory: Temporary directory used to store the model Assets after the
 |      training, and possibly as a work directory during the training. This
 |      temporary directory is necessary for the model to be exported after
 |      training e.g. `model.save(path)`. If not specified, `temp_directory` is
 |      set to a temporary directory using `tempfile.TemporaryDirectory`. This
 |      directory is deleted when the model python object is garbage-collected.
 |    verbose: If true, displays information about the training.
 |    hyperparameter_template: Override the default value of the hyper-parameters.
 |      If None (default) the default parameters of the library are used. If set,
 |      `default_hyperparameter_template` refers to one of the following
 |      preconfigured hyper-parameter sets. Those sets outperforms the default
 |      hyper-parameters (either generally or in specific scenarios).
 |      You can omit the version (e.g. remove "@v5") to use the last version of
 |      the template. In this case, the hyper-parameter can change in between
 |      releases (not recommended for training in production).
 |      - better_default@v1: A configuration that is generally better than the
 |        default parameters without being more expensive. The parameters are:
 |        winner_take_all=True.
 |      - benchmark_rank1@v1: Top ranking hyper-parameters on our benchmark
 |        slightly modified to run in reasonable time. The parameters are:
 |        winner_take_all=True, categorical_algorithm="RANDOM",
 |        split_axis="SPARSE_OBLIQUE", sparse_oblique_normalization="MIN_MAX",
 |        sparse_oblique_num_projections_exponent=1.0.
 |  
 |    advanced_arguments: Advanced control of the model that most users won't need
 |      to use. See `AdvancedArguments` for details.
 |    num_threads: Number of threads used to train the model. Different learning
 |      algorithms use multi-threading differently and with different degree of
 |      efficiency. If specified, `num_threads` field of the
 |      `advanced_arguments.yggdrasil_deployment_config` has priority.
 |    name: The name of the model.
 |    adapt_bootstrap_size_ratio_for_maximum_training_duration: Control how the
 |      maximum training duration (if set) is applied. If false, the training
 |      stop when the time is used. If true, adapts the size of the sampled
 |      dataset used to train each tree such that `num_trees` will train within
 |      `maximum_training_duration`. Has no effect if there is no maximum
 |      training duration specified. Default: False.
 |    allow_na_conditions: If true, the tree training evaluates conditions of the
 |      type `X is NA` i.e. `X is missing`. Default: False.
 |    categorical_algorithm: How to learn splits on categorical attributes.
 |      - `CART`: CART algorithm. Find categorical splits of the form "value \\in
 |        mask". The solution is exact for binary classification, regression and
 |        ranking. It is approximated for multi-class classification. This is a
 |        good first algorithm to use. In case of overfitting (very small
 |        dataset, large dictionary), the "random" algorithm is a good
 |        alternative.
 |      - `ONE_HOT`: One-hot encoding. Find the optimal categorical split of the
 |        form "attribute == param". This method is similar (but more efficient)
 |        than converting converting each possible categorical value into a
 |        boolean feature. This method is available for comparison purpose and
 |        generally performs worse than other alternatives.
 |      - `RANDOM`: Best splits among a set of random candidate. Find the a
 |        categorical split of the form "value \\in mask" using a random search.
 |        This solution can be seen as an approximation of the CART algorithm.
 |        This method is a strong alternative to CART. This algorithm is inspired
 |        from section "5.1 Categorical Variables" of "Random Forest", 2001.
 |        Default: "CART".
 |    categorical_set_split_greedy_sampling: For categorical set splits e.g.
 |      texts. Probability for a categorical value to be a candidate for the
 |      positive set. The sampling is applied once per node (i.e. not at every
 |      step of the greedy optimization). Default: 0.1.
 |    categorical_set_split_max_num_items: For categorical set splits e.g. texts.
 |      Maximum number of items (prior to the sampling). If more items are
 |      available, the least frequent items are ignored. Changing this value is
 |      similar to change the "max_vocab_count" before loading the dataset, with
 |      the following exception: With `max_vocab_count`, all the remaining items
 |      are grouped in a special Out-of-vocabulary item. With `max_num_items`,
 |      this is not the case. Default: -1.
 |    categorical_set_split_min_item_frequency: For categorical set splits e.g.
 |      texts. Minimum number of occurrences of an item to be considered.
 |      Default: 1.
 |    compute_oob_performances: If true, compute the Out-of-bag evaluation (then
 |      available in the summary and model inspector). This evaluation is a cheap
 |      alternative to cross-validation evaluation. Default: True.
 |    compute_oob_variable_importances: If true, compute the Out-of-bag feature
 |      importance (then available in the summary and model inspector). Note that
 |      the OOB feature importance can be expensive to compute. Default: False.
 |    growing_strategy: How to grow the tree.
 |      - `LOCAL`: Each node is split independently of the other nodes. In other
 |        words, as long as a node satisfy the splits "constraints (e.g. maximum
 |        depth, minimum number of observations), the node will be split. This is
 |        the "classical" way to grow decision trees.
 |      - `BEST_FIRST_GLOBAL`: The node with the best loss reduction among all
 |        the nodes of the tree is selected for splitting. This method is also
 |        called "best first" or "leaf-wise growth". See "Best-first decision
 |        tree learning", Shi and "Additive logistic regression : A statistical
 |        view of boosting", Friedman for more details. Default: "LOCAL".
 |    in_split_min_examples_check: Whether to check the `min_examples` constraint
 |      in the split search (i.e. splits leading to one child having less than
 |      `min_examples` examples are considered invalid) or before the split
 |      search (i.e. a node can be derived only if it contains more than
 |      `min_examples` examples). If false, there can be nodes with less than
 |      `min_examples` training examples. Default: True.
 |    max_depth: Maximum depth of the tree. `max_depth=1` means that all trees
 |      will be roots. Negative values are ignored. Default: 16.
 |    max_num_nodes: Maximum number of nodes in the tree. Set to -1 to disable
 |      this limit. Only available for `growing_strategy=BEST_FIRST_GLOBAL`.
 |      Default: None.
 |    maximum_training_duration_seconds: Maximum training duration of the model
 |      expressed in seconds. Each learning algorithm is free to use this
 |      parameter at it sees fit. Enabling maximum training duration makes the
 |      model training non-deterministic. Default: -1.0.
 |    min_examples: Minimum number of examples in a node. Default: 5.
 |    missing_value_policy: Method used to handle missing attribute values.
 |      - `GLOBAL_IMPUTATION`: Missing attribute values are imputed, with the
 |        mean (in case of numerical attribute) or the most-frequent-item (in
 |        case of categorical attribute) computed on the entire dataset (i.e. the
 |        information contained in the data spec).
 |      - `LOCAL_IMPUTATION`: Missing attribute values are imputed with the mean
 |        (numerical attribute) or most-frequent-item (in the case of categorical
 |        attribute) evaluated on the training examples in the current node.
 |      - `RANDOM_LOCAL_IMPUTATION`: Missing attribute values are imputed from
 |        randomly sampled values from the training examples in the current node.
 |        This method was proposed by Clinic et al. in "Random Survival Forests"
 |        (https://projecteuclid.org/download/pdfview_1/euclid.aoas/1223908043).
 |        Default: "GLOBAL_IMPUTATION".
 |    num_candidate_attributes: Number of unique valid attributes tested for each
 |      node. An attribute is valid if it has at least a valid split. If
 |      `num_candidate_attributes=0`, the value is set to the classical default
 |      value for Random Forest: `sqrt(number of input attributes)` in case of
 |      classification and `number_of_input_attributes / 3` in case of
 |      regression. If `num_candidate_attributes=-1`, all the attributes are
 |      tested. Default: 0.
 |    num_candidate_attributes_ratio: Ratio of attributes tested at each node. If
 |      set, it is equivalent to `num_candidate_attributes =
 |      number_of_input_features x num_candidate_attributes_ratio`. The possible
 |      values are between ]0, and 1] as well as -1. If not set or equal to -1,
 |      the `num_candidate_attributes` is used. Default: -1.0.
 |    num_trees: Number of individual decision trees. Increasing the number of
 |      trees can increase the quality of the model at the expense of size,
 |      training speed, and inference latency. Default: 300.
 |    sorting_strategy: How are sorted the numerical features in order to find
 |      the splits
 |      - PRESORT: The features are pre-sorted at the start of the training. This
 |        solution is faster but consumes much more memory than IN_NODE.
 |      - IN_NODE: The features are sorted just before being used in the node.
 |        This solution is slow but consumes little amount of memory.
 |      . Default: "PRESORT".
 |    sparse_oblique_normalization: For sparse oblique splits i.e.
 |      `split_axis=SPARSE_OBLIQUE`. Normalization applied on the features,
 |      before applying the sparse oblique projections.
 |      - `NONE`: No normalization.
 |      - `STANDARD_DEVIATION`: Normalize the feature by the estimated standard
 |        deviation on the entire train dataset. Also known as Z-Score
 |        normalization.
 |      - `MIN_MAX`: Normalize the feature by the range (i.e. max-min) estimated
 |        on the entire train dataset. Default: None.
 |    sparse_oblique_num_projections_exponent: For sparse oblique splits i.e.
 |      `split_axis=SPARSE_OBLIQUE`. Controls of the number of random projections
 |      to test at each node as `num_features^num_projections_exponent`. Default:
 |      None.
 |    sparse_oblique_projection_density_factor: For sparse oblique splits i.e.
 |      `split_axis=SPARSE_OBLIQUE`. Controls of the number of random projections
 |      to test at each node as `num_features^num_projections_exponent`. Default:
 |      None.
 |    split_axis: What structure of split to consider for numerical features.
 |      - `AXIS_ALIGNED`: Axis aligned splits (i.e. one condition at a time).
 |        This is the "classical" way to train a tree. Default value.
 |      - `SPARSE_OBLIQUE`: Sparse oblique splits (i.e. splits one a small number
 |        of features) from "Sparse Projection Oblique Random Forests", Tomita et
 |        al., 2020. Default: "AXIS_ALIGNED".
 |    winner_take_all: Control how classification trees vote. If true, each tree
 |      votes for one class. If false, each tree vote for a distribution of
 |      classes. winner_take_all_inference=false is often preferable. Default:
 |      True.
 |  
 |  Method resolution order:
 |      RandomForestModel
 |      tensorflow_decision_forests.keras.wrappers.RandomForestModel
 |      tensorflow_decision_forests.keras.core.CoreModel
 |      keras.engine.training.Model
 |      keras.engine.base_layer.Layer
 |      tensorflow.python.module.module.Module
 |      tensorflow.python.training.tracking.tracking.AutoTrackable
 |      tensorflow.python.training.tracking.base.Trackable
 |      keras.utils.version_utils.LayerVersionSelector
 |      keras.utils.version_utils.ModelVersionSelector
 |      builtins.object
 |  
 |  Methods inherited from tensorflow_decision_forests.keras.wrappers.RandomForestModel:
 |  
 |  __init__ = wrapper(*args, **kargs)
 |  
 |  ----------------------------------------------------------------------
 |  Static methods inherited from tensorflow_decision_forests.keras.wrappers.RandomForestModel:
 |  
 |  predefined_hyperparameters() -> List[tensorflow_decision_forests.keras.core.HyperParameterTemplate]
 |      Returns a better than default set of hyper-parameters.
 |      
 |      They can be used directly with the `hyperparameter_template` argument of the
 |      model constructor.
 |      
 |      These hyper-parameters outperforms the default hyper-parameters (either
 |      generally or in specific scenarios). Like default hyper-parameters, existing
 |      pre-defined hyper-parameters cannot change.
 |  
 |  ----------------------------------------------------------------------
 |  Methods inherited from tensorflow_decision_forests.keras.core.CoreModel:
 |  
 |  call(self, inputs, training=False)
 |      Inference of the model.
 |      
 |      This method is used for prediction and evaluation of a trained model.
 |      
 |      Args:
 |        inputs: Input tensors.
 |        training: Is the model being trained. Always False.
 |      
 |      Returns:
 |        Model predictions.
 |  
 |  compile(self, metrics=None)
 |      Configure the model for training.
 |      
 |      Unlike for most Keras model, calling "compile" is optional before calling
 |      "fit".
 |      
 |      Args:
 |        metrics: Metrics to report during training.
 |      
 |      Raises:
 |        ValueError: Invalid arguments.
 |  
 |  evaluate(self, *args, **kwargs)
 |      Returns the loss value & metrics values for the model.
 |      
 |      See details on `keras.Model.evaluate`.
 |      
 |      Args:
 |        *args: Passed to `keras.Model.evaluate`.
 |        **kwargs: Passed to `keras.Model.evaluate`.  Scalar test loss (if the
 |          model has a single output and no metrics) or list of scalars (if the
 |          model has multiple outputs and/or metrics). See details in
 |          `keras.Model.evaluate`.
 |  
 |  fit(self, x=None, y=None, callbacks=None, **kwargs) -> keras.callbacks.History
 |      Trains the model.
 |      
 |      The following dataset formats are supported:
 |      
 |        1. "x" is a tf.data.Dataset containing a tuple "(features, labels)".
 |           "features" can be a dictionary a tensor, a list of tensors or a
 |           dictionary of tensors (recommended). "labels" is a tensor.
 |      
 |        2. "x" is a tensor, list of tensors or dictionary of tensors containing
 |           the input features. "y" is a tensor.
 |      
 |        3. "x" is a numpy-array, list of numpy-arrays or dictionary of
 |           numpy-arrays containing the input features. "y" is a numpy-array.
 |      
 |      Pandas Dataframe can be consumed with "dataframe_to_tf_dataset":
 |        dataset = pandas.Dataframe(...)
 |        model.fit(pd_dataframe_to_tf_dataset(dataset, label="my_label"))
 |      
 |      Args:
 |        x: Training dataset (See details above for the supported formats).
 |        y: Label of the training dataset. Only used if "x" does not contains the
 |          labels.
 |        callbacks: Callbacks triggered during the training.
 |        **kwargs: Arguments passed to the core keras model's fit.
 |      
 |      Returns:
 |        A `History` object. Its `History.history` attribute is not yet
 |        implemented for decision forests algorithms, and will return empty.
 |        All other fields are filled as usual for `Keras.Mode.fit()`.
 |  
 |  make_inspector(self) -> tensorflow_decision_forests.component.inspector.inspector.AbstractInspector
 |      Creates an inspector to access the internal model structure.
 |      
 |      Usage example:
 |      
 |      ```python
 |      inspector = model.make_inspector()
 |      print(inspector.num_trees())
 |      print(inspector.variable_importances())
 |      ```
 |      
 |      Returns:
 |        A model inspector.
 |  
 |  make_predict_function(self)
 |      Prediction of the model (!= evaluation).
 |  
 |  make_test_function(self)
 |      Predictions for evaluation.
 |  
 |  save(self, filepath: str, overwrite: Union[bool, NoneType] = True, **kwargs)
 |      Saves the model as a TensorFlow SavedModel.
 |      
 |      The exported SavedModel contains a standalone Yggdrasil Decision Forests
 |      model in the "assets" sub-directory. The Yggdrasil model can be used
 |      directly using the Yggdrasil API. However, this model does not contain the
 |      "preprocessing" layer (if any).
 |      
 |      Args:
 |        filepath: Path to the output model.
 |        overwrite: If true, override an already existing model. If false, raise an
 |          error if a model already exist.
 |        **kwargs: Arguments passed to the core keras model's save.
 |  
 |  summary(self, line_length=None, positions=None, print_fn=None)
 |      Shows information about the model.
 |  
 |  train_step(self, data)
 |      Collects training examples.
 |  
 |  yggdrasil_model_path_tensor(self) -> Union[tensorflow.python.framework.ops.Tensor, NoneType]
 |      Gets the path to yggdrasil model, if available.
 |      
 |      The effective path can be obtained with:
 |      
 |      ```python
 |      yggdrasil_model_path_tensor().numpy().decode("utf-8")
 |      ```
 |      
 |      Returns:
 |        Path to the Yggdrasil model.
 |  
 |  ----------------------------------------------------------------------
 |  Methods inherited from keras.engine.training.Model:
 |  
 |  __setattr__(self, name, value)
 |      Support self.foo = trackable syntax.
 |  
 |  build(self, input_shape)
 |      Builds the model based on input shapes received.
 |      
 |      This is to be used for subclassed models, which do not know at instantiation
 |      time what their inputs look like.
 |      
 |      This method only exists for users who want to call `model.build()` in a
 |      standalone way (as a substitute for calling the model on real data to
 |      build it). It will never be called by the framework (and thus it will
 |      never throw unexpected errors in an unrelated workflow).
 |      
 |      Args:
 |       input_shape: Single tuple, TensorShape, or list/dict of shapes, where
 |           shapes are tuples, integers, or TensorShapes.
 |      
 |      Raises:
 |        ValueError:
 |          1. In case of invalid user-provided data (not of type tuple,
 |             list, TensorShape, or dict).
 |          2. If the model requires call arguments that are agnostic
 |             to the input shapes (positional or kwarg in call signature).
 |          3. If not all layers were properly built.
 |          4. If float type inputs are not supported within the layers.
 |      
 |        In each of these cases, the user should build their model by calling it
 |        on real tensor data.
 |  
 |  evaluate_generator(self, generator, steps=None, callbacks=None, max_queue_size=10, workers=1, use_multiprocessing=False, verbose=0)
 |      Evaluates the model on a data generator.
 |      
 |      DEPRECATED:
 |        `Model.evaluate` now supports generators, so there is no longer any need
 |        to use this endpoint.
 |  
 |  fit_generator(self, generator, steps_per_epoch=None, epochs=1, verbose=1, callbacks=None, validation_data=None, validation_steps=None, validation_freq=1, class_weight=None, max_queue_size=10, workers=1, use_multiprocessing=False, shuffle=True, initial_epoch=0)
 |      Fits the model on data yielded batch-by-batch by a Python generator.
 |      
 |      DEPRECATED:
 |        `Model.fit` now supports generators, so there is no longer any need to use
 |        this endpoint.
 |  
 |  get_config(self)
 |      Returns the config of the layer.
 |      
 |      A layer config is a Python dictionary (serializable)
 |      containing the configuration of a layer.
 |      The same layer can be reinstantiated later
 |      (without its trained weights) from this configuration.
 |      
 |      The config of a layer does not include connectivity
 |      information, nor the layer class name. These are handled
 |      by `Network` (one layer of abstraction above).
 |      
 |      Note that `get_config()` does not guarantee to return a fresh copy of dict
 |      every time it is called. The callers should make a copy of the returned dict
 |      if they want to modify it.
 |      
 |      Returns:
 |          Python dictionary.
 |  
 |  get_layer(self, name=None, index=None)
 |      Retrieves a layer based on either its name (unique) or index.
 |      
 |      If `name` and `index` are both provided, `index` will take precedence.
 |      Indices are based on order of horizontal graph traversal (bottom-up).
 |      
 |      Args:
 |          name: String, name of layer.
 |          index: Integer, index of layer.
 |      
 |      Returns:
 |          A layer instance.
 |      
 |      Raises:
 |          ValueError: In case of invalid layer name or index.
 |  
 |  get_weights(self)
 |      Retrieves the weights of the model.
 |      
 |      Returns:
 |          A flat list of Numpy arrays.
 |  
 |  load_weights(self, filepath, by_name=False, skip_mismatch=False, options=None)
 |      Loads all layer weights, either from a TensorFlow or an HDF5 weight file.
 |      
 |      If `by_name` is False weights are loaded based on the network's
 |      topology. This means the architecture should be the same as when the weights
 |      were saved.  Note that layers that don't have weights are not taken into
 |      account in the topological ordering, so adding or removing layers is fine as
 |      long as they don't have weights.
 |      
 |      If `by_name` is True, weights are loaded into layers only if they share the
 |      same name. This is useful for fine-tuning or transfer-learning models where
 |      some of the layers have changed.
 |      
 |      Only topological loading (`by_name=False`) is supported when loading weights
 |      from the TensorFlow format. Note that topological loading differs slightly
 |      between TensorFlow and HDF5 formats for user-defined classes inheriting from
 |      `tf.keras.Model`: HDF5 loads based on a flattened list of weights, while the
 |      TensorFlow format loads based on the object-local names of attributes to
 |      which layers are assigned in the `Model`'s constructor.
 |      
 |      Args:
 |          filepath: String, path to the weights file to load. For weight files in
 |              TensorFlow format, this is the file prefix (the same as was passed
 |              to `save_weights`). This can also be a path to a SavedModel
 |              saved from `model.save`.
 |          by_name: Boolean, whether to load weights by name or by topological
 |              order. Only topological loading is supported for weight files in
 |              TensorFlow format.
 |          skip_mismatch: Boolean, whether to skip loading of layers where there is
 |              a mismatch in the number of weights, or a mismatch in the shape of
 |              the weight (only valid when `by_name=True`).
 |          options: Optional `tf.train.CheckpointOptions` object that specifies
 |              options for loading weights.
 |      
 |      Returns:
 |          When loading a weight file in TensorFlow format, returns the same status
 |          object as `tf.train.Checkpoint.restore`. When graph building, restore
 |          ops are run automatically as soon as the network is built (on first call
 |          for user-defined classes inheriting from `Model`, immediately if it is
 |          already built).
 |      
 |          When loading weights in HDF5 format, returns `None`.
 |      
 |      Raises:
 |          ImportError: If h5py is not available and the weight file is in HDF5
 |              format.
 |          ValueError: If `skip_mismatch` is set to `True` when `by_name` is
 |            `False`.
 |  
 |  make_train_function(self, force=False)
 |      Creates a function that executes one step of training.
 |      
 |      This method can be overridden to support custom training logic.
 |      This method is called by `Model.fit` and `Model.train_on_batch`.
 |      
 |      Typically, this method directly controls `tf.function` and
 |      `tf.distribute.Strategy` settings, and delegates the actual training
 |      logic to `Model.train_step`.
 |      
 |      This function is cached the first time `Model.fit` or
 |      `Model.train_on_batch` is called. The cache is cleared whenever
 |      `Model.compile` is called. You can skip the cache and generate again the
 |      function with `force=True`.
 |      
 |      Args:
 |        force: Whether to regenerate the train function and skip the cached
 |          function if available.
 |      
 |      Returns:
 |        Function. The function created by this method should accept a
 |        `tf.data.Iterator`, and return a `dict` containing values that will
 |        be passed to `tf.keras.Callbacks.on_train_batch_end`, such as
 |        `{'loss': 0.2, 'accuracy': 0.7}`.
 |  
 |  predict(self, x, batch_size=None, verbose=0, steps=None, callbacks=None, max_queue_size=10, workers=1, use_multiprocessing=False)
 |      Generates output predictions for the input samples.
 |      
 |      Computation is done in batches. This method is designed for performance in
 |      large scale inputs. For small amount of inputs that fit in one batch,
 |      directly using `__call__` is recommended for faster execution, e.g.,
 |      `model(x)`, or `model(x, training=False)` if you have layers such as
 |      `tf.keras.layers.BatchNormalization` that behaves differently during
 |      inference. Also, note the fact that test loss is not affected by
 |      regularization layers like noise and dropout.
 |      
 |      Args:
 |          x: Input samples. It could be:
 |            - A Numpy array (or array-like), or a list of arrays
 |              (in case the model has multiple inputs).
 |            - A TensorFlow tensor, or a list of tensors
 |              (in case the model has multiple inputs).
 |            - A `tf.data` dataset.
 |            - A generator or `keras.utils.Sequence` instance.
 |            A more detailed description of unpacking behavior for iterator types
 |            (Dataset, generator, Sequence) is given in the `Unpacking behavior
 |            for iterator-like inputs` section of `Model.fit`.
 |          batch_size: Integer or `None`.
 |              Number of samples per batch.
 |              If unspecified, `batch_size` will default to 32.
 |              Do not specify the `batch_size` if your data is in the
 |              form of dataset, generators, or `keras.utils.Sequence` instances
 |              (since they generate batches).
 |          verbose: Verbosity mode, 0 or 1.
 |          steps: Total number of steps (batches of samples)
 |              before declaring the prediction round finished.
 |              Ignored with the default value of `None`. If x is a `tf.data`
 |              dataset and `steps` is None, `predict` will
 |              run until the input dataset is exhausted.
 |          callbacks: List of `keras.callbacks.Callback` instances.
 |              List of callbacks to apply during prediction.
 |              See [callbacks](/api_docs/python/tf/keras/callbacks).
 |          max_queue_size: Integer. Used for generator or `keras.utils.Sequence`
 |              input only. Maximum size for the generator queue.
 |              If unspecified, `max_queue_size` will default to 10.
 |          workers: Integer. Used for generator or `keras.utils.Sequence` input
 |              only. Maximum number of processes to spin up when using
 |              process-based threading. If unspecified, `workers` will default
 |              to 1.
 |          use_multiprocessing: Boolean. Used for generator or
 |              `keras.utils.Sequence` input only. If `True`, use process-based
 |              threading. If unspecified, `use_multiprocessing` will default to
 |              `False`. Note that because this implementation relies on
 |              multiprocessing, you should not pass non-picklable arguments to
 |              the generator as they can't be passed easily to children processes.
 |      
 |      See the discussion of `Unpacking behavior for iterator-like inputs` for
 |      `Model.fit`. Note that Model.predict uses the same interpretation rules as
 |      `Model.fit` and `Model.evaluate`, so inputs must be unambiguous for all
 |      three methods.
 |      
 |      Returns:
 |          Numpy array(s) of predictions.
 |      
 |      Raises:
 |          RuntimeError: If `model.predict` is wrapped in `tf.function`.
 |          ValueError: In case of mismatch between the provided
 |              input data and the model's expectations,
 |              or in case a stateful model receives a number of samples
 |              that is not a multiple of the batch size.
 |  
 |  predict_generator(self, generator, steps=None, callbacks=None, max_queue_size=10, workers=1, use_multiprocessing=False, verbose=0)
 |      Generates predictions for the input samples from a data generator.
 |      
 |      DEPRECATED:
 |        `Model.predict` now supports generators, so there is no longer any need
 |        to use this endpoint.
 |  
 |  predict_on_batch(self, x)
 |      Returns predictions for a single batch of samples.
 |      
 |      Args:
 |          x: Input data. It could be:
 |            - A Numpy array (or array-like), or a list of arrays (in case the
 |                model has multiple inputs).
 |            - A TensorFlow tensor, or a list of tensors (in case the model has
 |                multiple inputs).
 |      
 |      Returns:
 |          Numpy array(s) of predictions.
 |      
 |      Raises:
 |          RuntimeError: If `model.predict_on_batch` is wrapped in `tf.function`.
 |          ValueError: In case of mismatch between given number of inputs and
 |            expectations of the model.
 |  
 |  predict_step(self, data)
 |      The logic for one inference step.
 |      
 |      This method can be overridden to support custom inference logic.
 |      This method is called by `Model.make_predict_function`.
 |      
 |      This method should contain the mathematical logic for one step of inference.
 |      This typically includes the forward pass.
 |      
 |      Configuration details for *how* this logic is run (e.g. `tf.function` and
 |      `tf.distribute.Strategy` settings), should be left to
 |      `Model.make_predict_function`, which can also be overridden.
 |      
 |      Args:
 |        data: A nested structure of `Tensor`s.
 |      
 |      Returns:
 |        The result of one inference step, typically the output of calling the
 |        `Model` on data.
 |  
 |  reset_metrics(self)
 |      Resets the state of all the metrics in the model.
 |      
 |      Examples:
 |      
 |      >>> inputs = tf.keras.layers.Input(shape=(3,))
 |      >>> outputs = tf.keras.layers.Dense(2)(inputs)
 |      >>> model = tf.keras.models.Model(inputs=inputs, outputs=outputs)
 |      >>> model.compile(optimizer="Adam", loss="mse", metrics=["mae"])
 |      
 |      >>> x = np.random.random((2, 3))
 |      >>> y = np.random.randint(0, 2, (2, 2))
 |      >>> _ = model.fit(x, y, verbose=0)
 |      >>> assert all(float(m.result()) for m in model.metrics)
 |      
 |      >>> model.reset_metrics()
 |      >>> assert all(float(m.result()) == 0 for m in model.metrics)
 |  
 |  reset_states(self)
 |  
 |  save_spec(self, dynamic_batch=True)
 |      Returns the `tf.TensorSpec` of call inputs as a tuple `(args, kwargs)`.
 |      
 |      This value is automatically defined after calling the model for the first
 |      time. Afterwards, you can use it when exporting the model for serving:
 |      
 |      ```python
 |      model = tf.keras.Model(...)
 |      
 |      @tf.function
 |      def serve(*args, **kwargs):
 |        outputs = model(*args, **kwargs)
 |        # Apply postprocessing steps, or add additional outputs.
 |        ...
 |        return outputs
 |      
 |      # arg_specs is `[tf.TensorSpec(...), ...]`. kwarg_specs, in this example, is
 |      # an empty dict since functional models do not use keyword arguments.
 |      arg_specs, kwarg_specs = model.save_spec()
 |      
 |      model.save(path, signatures={
 |        'serving_default': serve.get_concrete_function(*arg_specs, **kwarg_specs)
 |      })
 |      ```
 |      
 |      Args:
 |        dynamic_batch: Whether to set the batch sizes of all the returned
 |          `tf.TensorSpec` to `None`. (Note that when defining functional or
 |          Sequential models with `tf.keras.Input([...], batch_size=X)`, the
 |          batch size will always be preserved). Defaults to `True`.
 |      Returns:
 |        If the model inputs are defined, returns a tuple `(args, kwargs)`. All
 |        elements in `args` and `kwargs` are `tf.TensorSpec`.
 |        If the model inputs are not defined, returns `None`.
 |        The model inputs are automatically set when calling the model,
 |        `model.fit`, `model.evaluate` or `model.predict`.
 |  
 |  save_weights(self, filepath, overwrite=True, save_format=None, options=None)
 |      Saves all layer weights.
 |      
 |      Either saves in HDF5 or in TensorFlow format based on the `save_format`
 |      argument.
 |      
 |      When saving in HDF5 format, the weight file has:
 |        - `layer_names` (attribute), a list of strings
 |            (ordered names of model layers).
 |        - For every layer, a `group` named `layer.name`
 |            - For every such layer group, a group attribute `weight_names`,
 |                a list of strings
 |                (ordered names of weights tensor of the layer).
 |            - For every weight in the layer, a dataset
 |                storing the weight value, named after the weight tensor.
 |      
 |      When saving in TensorFlow format, all objects referenced by the network are
 |      saved in the same format as `tf.train.Checkpoint`, including any `Layer`
 |      instances or `Optimizer` instances assigned to object attributes. For
 |      networks constructed from inputs and outputs using `tf.keras.Model(inputs,
 |      outputs)`, `Layer` instances used by the network are tracked/saved
 |      automatically. For user-defined classes which inherit from `tf.keras.Model`,
 |      `Layer` instances must be assigned to object attributes, typically in the
 |      constructor. See the documentation of `tf.train.Checkpoint` and
 |      `tf.keras.Model` for details.
 |      
 |      While the formats are the same, do not mix `save_weights` and
 |      `tf.train.Checkpoint`. Checkpoints saved by `Model.save_weights` should be
 |      loaded using `Model.load_weights`. Checkpoints saved using
 |      `tf.train.Checkpoint.save` should be restored using the corresponding
 |      `tf.train.Checkpoint.restore`. Prefer `tf.train.Checkpoint` over
 |      `save_weights` for training checkpoints.
 |      
 |      The TensorFlow format matches objects and variables by starting at a root
 |      object, `self` for `save_weights`, and greedily matching attribute
 |      names. For `Model.save` this is the `Model`, and for `Checkpoint.save` this
 |      is the `Checkpoint` even if the `Checkpoint` has a model attached. This
 |      means saving a `tf.keras.Model` using `save_weights` and loading into a
 |      `tf.train.Checkpoint` with a `Model` attached (or vice versa) will not match
 |      the `Model`'s variables. See the [guide to training
 |      checkpoints](https://www.tensorflow.org/guide/checkpoint) for details
 |      on the TensorFlow format.
 |      
 |      Args:
 |          filepath: String or PathLike, path to the file to save the weights to.
 |              When saving in TensorFlow format, this is the prefix used for
 |              checkpoint files (multiple files are generated). Note that the '.h5'
 |              suffix causes weights to be saved in HDF5 format.
 |          overwrite: Whether to silently overwrite any existing file at the
 |              target location, or provide the user with a manual prompt.
 |          save_format: Either 'tf' or 'h5'. A `filepath` ending in '.h5' or
 |              '.keras' will default to HDF5 if `save_format` is `None`. Otherwise
 |              `None` defaults to 'tf'.
 |          options: Optional `tf.train.CheckpointOptions` object that specifies
 |              options for saving weights.
 |      
 |      Raises:
 |          ImportError: If h5py is not available when attempting to save in HDF5
 |              format.
 |          ValueError: For invalid/unknown format arguments.
 |  
 |  test_on_batch(self, x, y=None, sample_weight=None, reset_metrics=True, return_dict=False)
 |      Test the model on a single batch of samples.
 |      
 |      Args:
 |          x: Input data. It could be:
 |            - A Numpy array (or array-like), or a list of arrays (in case the
 |                model has multiple inputs).
 |            - A TensorFlow tensor, or a list of tensors (in case the model has
 |                multiple inputs).
 |            - A dict mapping input names to the corresponding array/tensors, if
 |                the model has named inputs.
 |          y: Target data. Like the input data `x`, it could be either Numpy
 |            array(s) or TensorFlow tensor(s). It should be consistent with `x`
 |            (you cannot have Numpy inputs and tensor targets, or inversely).
 |          sample_weight: Optional array of the same length as x, containing
 |            weights to apply to the model's loss for each sample. In the case of
 |            temporal data, you can pass a 2D array with shape (samples,
 |            sequence_length), to apply a different weight to every timestep of
 |            every sample.
 |          reset_metrics: If `True`, the metrics returned will be only for this
 |            batch. If `False`, the metrics will be statefully accumulated across
 |            batches.
 |          return_dict: If `True`, loss and metric results are returned as a dict,
 |            with each key being the name of the metric. If `False`, they are
 |            returned as a list.
 |      
 |      Returns:
 |          Scalar test loss (if the model has a single output and no metrics)
 |          or list of scalars (if the model has multiple outputs
 |          and/or metrics). The attribute `model.metrics_names` will give you
 |          the display labels for the scalar outputs.
 |      
 |      Raises:
 |          RuntimeError: If `model.test_on_batch` is wrapped in `tf.function`.
 |          ValueError: In case of invalid user-provided arguments.
 |  
 |  test_step(self, data)
 |      The logic for one evaluation step.
 |      
 |      This method can be overridden to support custom evaluation logic.
 |      This method is called by `Model.make_test_function`.
 |      
 |      This function should contain the mathematical logic for one step of
 |      evaluation.
 |      This typically includes the forward pass, loss calculation, and metrics
 |      updates.
 |      
 |      Configuration details for *how* this logic is run (e.g. `tf.function` and
 |      `tf.distribute.Strategy` settings), should be left to
 |      `Model.make_test_function`, which can also be overridden.
 |      
 |      Args:
 |        data: A nested structure of `Tensor`s.
 |      
 |      Returns:
 |        A `dict` containing values that will be passed to
 |        `tf.keras.callbacks.CallbackList.on_train_batch_end`. Typically, the
 |        values of the `Model`'s metrics are returned.
 |  
 |  to_json(self, **kwargs)
 |      Returns a JSON string containing the network configuration.
 |      
 |      To load a network from a JSON save file, use
 |      `keras.models.model_from_json(json_string, custom_objects={})`.
 |      
 |      Args:
 |          **kwargs: Additional keyword arguments
 |              to be passed to `json.dumps()`.
 |      
 |      Returns:
 |          A JSON string.
 |  
 |  to_yaml(self, **kwargs)
 |      Returns a yaml string containing the network configuration.
 |      
 |      Note: Since TF 2.6, this method is no longer supported and will raise a
 |      RuntimeError.
 |      
 |      To load a network from a yaml save file, use
 |      `keras.models.model_from_yaml(yaml_string, custom_objects={})`.
 |      
 |      `custom_objects` should be a dictionary mapping
 |      the names of custom losses / layers / etc to the corresponding
 |      functions / classes.
 |      
 |      Args:
 |          **kwargs: Additional keyword arguments
 |              to be passed to `yaml.dump()`.
 |      
 |      Returns:
 |          A YAML string.
 |      
 |      Raises:
 |          RuntimeError: announces that the method poses a security risk
 |            (Use the safer `safe_load` function instead of `unsafe_load` when possible)
 |  
 |  train_on_batch(self, x, y=None, sample_weight=None, class_weight=None, reset_metrics=True, return_dict=False)
 |      Runs a single gradient update on a single batch of data.
 |      
 |      Args:
 |          x: Input data. It could be:
 |            - A Numpy array (or array-like), or a list of arrays
 |                (in case the model has multiple inputs).
 |            - A TensorFlow tensor, or a list of tensors
 |                (in case the model has multiple inputs).
 |            - A dict mapping input names to the corresponding array/tensors,
 |                if the model has named inputs.
 |          y: Target data. Like the input data `x`, it could be either Numpy
 |            array(s) or TensorFlow tensor(s). It should be consistent with `x`
 |            (you cannot have Numpy inputs and tensor targets, or inversely).
 |          sample_weight: Optional array of the same length as x, containing
 |            weights to apply to the model's loss for each sample. In the case of
 |            temporal data, you can pass a 2D array with shape (samples,
 |            sequence_length), to apply a different weight to every timestep of
 |            every sample.
 |          class_weight: Optional dictionary mapping class indices (integers) to a
 |            weight (float) to apply to the model's loss for the samples from this
 |            class during training. This can be useful to tell the model to "pay
 |            more attention" to samples from an under-represented class.
 |          reset_metrics: If `True`, the metrics returned will be only for this
 |            batch. If `False`, the metrics will be statefully accumulated across
 |            batches.
 |          return_dict: If `True`, loss and metric results are returned as a dict,
 |            with each key being the name of the metric. If `False`, they are
 |            returned as a list.
 |      
 |      Returns:
 |          Scalar training loss
 |          (if the model has a single output and no metrics)
 |          or list of scalars (if the model has multiple outputs
 |          and/or metrics). The attribute `model.metrics_names` will give you
 |          the display labels for the scalar outputs.
 |      
 |      Raises:
 |        RuntimeError: If `model.train_on_batch` is wrapped in `tf.function`.
 |        ValueError: In case of invalid user-provided arguments.
 |  
 |  ----------------------------------------------------------------------
 |  Class methods inherited from keras.engine.training.Model:
 |  
 |  from_config(config, custom_objects=None) from builtins.type
 |      Creates a layer from its config.
 |      
 |      This method is the reverse of `get_config`,
 |      capable of instantiating the same layer from the config
 |      dictionary. It does not handle layer connectivity
 |      (handled by Network), nor weights (handled by `set_weights`).
 |      
 |      Args:
 |          config: A Python dictionary, typically the
 |              output of get_config.
 |      
 |      Returns:
 |          A layer instance.
 |  
 |  ----------------------------------------------------------------------
 |  Static methods inherited from keras.engine.training.Model:
 |  
 |  __new__(cls, *args, **kwargs)
 |      Create and return a new object.  See help(type) for accurate signature.
 |  
 |  ----------------------------------------------------------------------
 |  Data descriptors inherited from keras.engine.training.Model:
 |  
 |  distribute_strategy
 |      The `tf.distribute.Strategy` this model was created under.
 |  
 |  layers
 |  
 |  metrics
 |      Returns the model's metrics added using `compile`, `add_metric` APIs.
 |      
 |      Note: Metrics passed to `compile()` are available only after a `keras.Model`
 |      has been trained/evaluated on actual data.
 |      
 |      Examples:
 |      
 |      >>> inputs = tf.keras.layers.Input(shape=(3,))
 |      >>> outputs = tf.keras.layers.Dense(2)(inputs)
 |      >>> model = tf.keras.models.Model(inputs=inputs, outputs=outputs)
 |      >>> model.compile(optimizer="Adam", loss="mse", metrics=["mae"])
 |      >>> [m.name for m in model.metrics]
 |      []
 |      
 |      >>> x = np.random.random((2, 3))
 |      >>> y = np.random.randint(0, 2, (2, 2))
 |      >>> model.fit(x, y)
 |      >>> [m.name for m in model.metrics]
 |      ['loss', 'mae']
 |      
 |      >>> inputs = tf.keras.layers.Input(shape=(3,))
 |      >>> d = tf.keras.layers.Dense(2, name='out')
 |      >>> output_1 = d(inputs)
 |      >>> output_2 = d(inputs)
 |      >>> model = tf.keras.models.Model(
 |      ...    inputs=inputs, outputs=[output_1, output_2])
 |      >>> model.add_metric(
 |      ...    tf.reduce_sum(output_2), name='mean', aggregation='mean')
 |      >>> model.compile(optimizer="Adam", loss="mse", metrics=["mae", "acc"])
 |      >>> model.fit(x, (y, y))
 |      >>> [m.name for m in model.metrics]
 |      ['loss', 'out_loss', 'out_1_loss', 'out_mae', 'out_acc', 'out_1_mae',
 |      'out_1_acc', 'mean']
 |  
 |  metrics_names
 |      Returns the model's display labels for all outputs.
 |      
 |      Note: `metrics_names` are available only after a `keras.Model` has been
 |      trained/evaluated on actual data.
 |      
 |      Examples:
 |      
 |      >>> inputs = tf.keras.layers.Input(shape=(3,))
 |      >>> outputs = tf.keras.layers.Dense(2)(inputs)
 |      >>> model = tf.keras.models.Model(inputs=inputs, outputs=outputs)
 |      >>> model.compile(optimizer="Adam", loss="mse", metrics=["mae"])
 |      >>> model.metrics_names
 |      []
 |      
 |      >>> x = np.random.random((2, 3))
 |      >>> y = np.random.randint(0, 2, (2, 2))
 |      >>> model.fit(x, y)
 |      >>> model.metrics_names
 |      ['loss', 'mae']
 |      
 |      >>> inputs = tf.keras.layers.Input(shape=(3,))
 |      >>> d = tf.keras.layers.Dense(2, name='out')
 |      >>> output_1 = d(inputs)
 |      >>> output_2 = d(inputs)
 |      >>> model = tf.keras.models.Model(
 |      ...    inputs=inputs, outputs=[output_1, output_2])
 |      >>> model.compile(optimizer="Adam", loss="mse", metrics=["mae", "acc"])
 |      >>> model.fit(x, (y, y))
 |      >>> model.metrics_names
 |      ['loss', 'out_loss', 'out_1_loss', 'out_mae', 'out_acc', 'out_1_mae',
 |      'out_1_acc']
 |  
 |  non_trainable_weights
 |      List of all non-trainable weights tracked by this layer.
 |      
 |      Non-trainable weights are *not* updated during training. They are expected
 |      to be updated manually in `call()`.
 |      
 |      Returns:
 |        A list of non-trainable variables.
 |  
 |  run_eagerly
 |      Settable attribute indicating whether the model should run eagerly.
 |      
 |      Running eagerly means that your model will be run step by step,
 |      like Python code. Your model might run slower, but it should become easier
 |      for you to debug it by stepping into individual layer calls.
 |      
 |      By default, we will attempt to compile your model to a static graph to
 |      deliver the best execution performance.
 |      
 |      Returns:
 |        Boolean, whether the model should run eagerly.
 |  
 |  state_updates
 |      Deprecated, do NOT use!
 |      
 |      Returns the `updates` from all layers that are stateful.
 |      
 |      This is useful for separating training updates and
 |      state updates, e.g. when we need to update a layer's internal state
 |      during prediction.
 |      
 |      Returns:
 |          A list of update ops.
 |  
 |  trainable_weights
 |      List of all trainable weights tracked by this layer.
 |      
 |      Trainable weights are updated via gradient descent during training.
 |      
 |      Returns:
 |        A list of trainable variables.
 |  
 |  weights
 |      Returns the list of all layer variables/weights.
 |      
 |      Note: This will not track the weights of nested `tf.Modules` that are not
 |      themselves Keras layers.
 |      
 |      Returns:
 |        A list of variables.
 |  
 |  ----------------------------------------------------------------------
 |  Methods inherited from keras.engine.base_layer.Layer:
 |  
 |  __call__(self, *args, **kwargs)
 |      Wraps `call`, applying pre- and post-processing steps.
 |      
 |      Args:
 |        *args: Positional arguments to be passed to `self.call`.
 |        **kwargs: Keyword arguments to be passed to `self.call`.
 |      
 |      Returns:
 |        Output tensor(s).
 |      
 |      Note:
 |        - The following optional keyword arguments are reserved for specific uses:
 |          * `training`: Boolean scalar tensor of Python boolean indicating
 |            whether the `call` is meant for training or inference.
 |          * `mask`: Boolean input mask.
 |        - If the layer's `call` method takes a `mask` argument (as some Keras
 |          layers do), its default value will be set to the mask generated
 |          for `inputs` by the previous layer (if `input` did come from
 |          a layer that generated a corresponding mask, i.e. if it came from
 |          a Keras layer with masking support.
 |        - If the layer is not built, the method will call `build`.
 |      
 |      Raises:
 |        ValueError: if the layer's `call` method returns None (an invalid value).
 |        RuntimeError: if `super().__init__()` was not called in the constructor.
 |  
 |  __delattr__(self, name)
 |      Implement delattr(self, name).
 |  
 |  __getstate__(self)
 |  
 |  __setstate__(self, state)
 |  
 |  add_loss(self, losses, **kwargs)
 |      Add loss tensor(s), potentially dependent on layer inputs.
 |      
 |      Some losses (for instance, activity regularization losses) may be dependent
 |      on the inputs passed when calling a layer. Hence, when reusing the same
 |      layer on different inputs `a` and `b`, some entries in `layer.losses` may
 |      be dependent on `a` and some on `b`. This method automatically keeps track
 |      of dependencies.
 |      
 |      This method can be used inside a subclassed layer or model's `call`
 |      function, in which case `losses` should be a Tensor or list of Tensors.
 |      
 |      Example:
 |      
 |      ```python
 |      class MyLayer(tf.keras.layers.Layer):
 |        def call(self, inputs):
 |          self.add_loss(tf.abs(tf.reduce_mean(inputs)))
 |          return inputs
 |      ```
 |      
 |      This method can also be called directly on a Functional Model during
 |      construction. In this case, any loss Tensors passed to this Model must
 |      be symbolic and be able to be traced back to the model's `Input`s. These
 |      losses become part of the model's topology and are tracked in `get_config`.
 |      
 |      Example:
 |      
 |      ```python
 |      inputs = tf.keras.Input(shape=(10,))
 |      x = tf.keras.layers.Dense(10)(inputs)
 |      outputs = tf.keras.layers.Dense(1)(x)
 |      model = tf.keras.Model(inputs, outputs)
 |      # Activity regularization.
 |      model.add_loss(tf.abs(tf.reduce_mean(x)))
 |      ```
 |      
 |      If this is not the case for your loss (if, for example, your loss references
 |      a `Variable` of one of the model's layers), you can wrap your loss in a
 |      zero-argument lambda. These losses are not tracked as part of the model's
 |      topology since they can't be serialized.
 |      
 |      Example:
 |      
 |      ```python
 |      inputs = tf.keras.Input(shape=(10,))
 |      d = tf.keras.layers.Dense(10)
 |      x = d(inputs)
 |      outputs = tf.keras.layers.Dense(1)(x)
 |      model = tf.keras.Model(inputs, outputs)
 |      # Weight regularization.
 |      model.add_loss(lambda: tf.reduce_mean(d.kernel))
 |      ```
 |      
 |      Args:
 |        losses: Loss tensor, or list/tuple of tensors. Rather than tensors, losses
 |          may also be zero-argument callables which create a loss tensor.
 |        **kwargs: Additional keyword arguments for backward compatibility.
 |          Accepted values:
 |            inputs - Deprecated, will be automatically inferred.
 |  
 |  add_metric(self, value, name=None, **kwargs)
 |      Adds metric tensor to the layer.
 |      
 |      This method can be used inside the `call()` method of a subclassed layer
 |      or model.
 |      
 |      ```python
 |      class MyMetricLayer(tf.keras.layers.Layer):
 |        def __init__(self):
 |          super(MyMetricLayer, self).__init__(name='my_metric_layer')
 |          self.mean = tf.keras.metrics.Mean(name='metric_1')
 |      
 |        def call(self, inputs):
 |          self.add_metric(self.mean(inputs))
 |          self.add_metric(tf.reduce_sum(inputs), name='metric_2')
 |          return inputs
 |      ```
 |      
 |      This method can also be called directly on a Functional Model during
 |      construction. In this case, any tensor passed to this Model must
 |      be symbolic and be able to be traced back to the model's `Input`s. These
 |      metrics become part of the model's topology and are tracked when you
 |      save the model via `save()`.
 |      
 |      ```python
 |      inputs = tf.keras.Input(shape=(10,))
 |      x = tf.keras.layers.Dense(10)(inputs)
 |      outputs = tf.keras.layers.Dense(1)(x)
 |      model = tf.keras.Model(inputs, outputs)
 |      model.add_metric(math_ops.reduce_sum(x), name='metric_1')
 |      ```
 |      
 |      Note: Calling `add_metric()` with the result of a metric object on a
 |      Functional Model, as shown in the example below, is not supported. This is
 |      because we cannot trace the metric result tensor back to the model's inputs.
 |      
 |      ```python
 |      inputs = tf.keras.Input(shape=(10,))
 |      x = tf.keras.layers.Dense(10)(inputs)
 |      outputs = tf.keras.layers.Dense(1)(x)
 |      model = tf.keras.Model(inputs, outputs)
 |      model.add_metric(tf.keras.metrics.Mean()(x), name='metric_1')
 |      ```
 |      
 |      Args:
 |        value: Metric tensor.
 |        name: String metric name.
 |        **kwargs: Additional keyword arguments for backward compatibility.
 |          Accepted values:
 |          `aggregation` - When the `value` tensor provided is not the result of
 |          calling a `keras.Metric` instance, it will be aggregated by default
 |          using a `keras.Metric.Mean`.
 |  
 |  add_update(self, updates, inputs=None)
 |      Add update op(s), potentially dependent on layer inputs.
 |      
 |      Weight updates (for instance, the updates of the moving mean and variance
 |      in a BatchNormalization layer) may be dependent on the inputs passed
 |      when calling a layer. Hence, when reusing the same layer on
 |      different inputs `a` and `b`, some entries in `layer.updates` may be
 |      dependent on `a` and some on `b`. This method automatically keeps track
 |      of dependencies.
 |      
 |      This call is ignored when eager execution is enabled (in that case, variable
 |      updates are run on the fly and thus do not need to be tracked for later
 |      execution).
 |      
 |      Args:
 |        updates: Update op, or list/tuple of update ops, or zero-arg callable
 |          that returns an update op. A zero-arg callable should be passed in
 |          order to disable running the updates by setting `trainable=False`
 |          on this Layer, when executing in Eager mode.
 |        inputs: Deprecated, will be automatically inferred.
 |  
 |  add_variable(self, *args, **kwargs)
 |      Deprecated, do NOT use! Alias for `add_weight`.
 |  
 |  add_weight(self, name=None, shape=None, dtype=None, initializer=None, regularizer=None, trainable=None, constraint=None, use_resource=None, synchronization=<VariableSynchronization.AUTO: 0>, aggregation=<VariableAggregationV2.NONE: 0>, **kwargs)
 |      Adds a new variable to the layer.
 |      
 |      Args:
 |        name: Variable name.
 |        shape: Variable shape. Defaults to scalar if unspecified.
 |        dtype: The type of the variable. Defaults to `self.dtype`.
 |        initializer: Initializer instance (callable).
 |        regularizer: Regularizer instance (callable).
 |        trainable: Boolean, whether the variable should be part of the layer's
 |          "trainable_variables" (e.g. variables, biases)
 |          or "non_trainable_variables" (e.g. BatchNorm mean and variance).
 |          Note that `trainable` cannot be `True` if `synchronization`
 |          is set to `ON_READ`.
 |        constraint: Constraint instance (callable).
 |        use_resource: Whether to use `ResourceVariable`.
 |        synchronization: Indicates when a distributed a variable will be
 |          aggregated. Accepted values are constants defined in the class
 |          `tf.VariableSynchronization`. By default the synchronization is set to
 |          `AUTO` and the current `DistributionStrategy` chooses
 |          when to synchronize. If `synchronization` is set to `ON_READ`,
 |          `trainable` must not be set to `True`.
 |        aggregation: Indicates how a distributed variable will be aggregated.
 |          Accepted values are constants defined in the class
 |          `tf.VariableAggregation`.
 |        **kwargs: Additional keyword arguments. Accepted values are `getter`,
 |          `collections`, `experimental_autocast` and `caching_device`.
 |      
 |      Returns:
 |        The variable created.
 |      
 |      Raises:
 |        ValueError: When giving unsupported dtype and no initializer or when
 |          trainable has been set to True with synchronization set as `ON_READ`.
 |  
 |  apply(self, inputs, *args, **kwargs)
 |      Deprecated, do NOT use!
 |      
 |      This is an alias of `self.__call__`.
 |      
 |      Args:
 |        inputs: Input tensor(s).
 |        *args: additional positional arguments to be passed to `self.call`.
 |        **kwargs: additional keyword arguments to be passed to `self.call`.
 |      
 |      Returns:
 |        Output tensor(s).
 |  
 |  compute_mask(self, inputs, mask=None)
 |      Computes an output mask tensor.
 |      
 |      Args:
 |          inputs: Tensor or list of tensors.
 |          mask: Tensor or list of tensors.
 |      
 |      Returns:
 |          None or a tensor (or list of tensors,
 |              one per output tensor of the layer).
 |  
 |  compute_output_shape(self, input_shape)
 |      Computes the output shape of the layer.
 |      
 |      If the layer has not been built, this method will call `build` on the
 |      layer. This assumes that the layer will later be used with inputs that
 |      match the input shape provided here.
 |      
 |      Args:
 |          input_shape: Shape tuple (tuple of integers)
 |              or list of shape tuples (one per output tensor of the layer).
 |              Shape tuples can include None for free dimensions,
 |              instead of an integer.
 |      
 |      Returns:
 |          An input shape tuple.
 |  
 |  compute_output_signature(self, input_signature)
 |      Compute the output tensor signature of the layer based on the inputs.
 |      
 |      Unlike a TensorShape object, a TensorSpec object contains both shape
 |      and dtype information for a tensor. This method allows layers to provide
 |      output dtype information if it is different from the input dtype.
 |      For any layer that doesn't implement this function,
 |      the framework will fall back to use `compute_output_shape`, and will
 |      assume that the output dtype matches the input dtype.
 |      
 |      Args:
 |        input_signature: Single TensorSpec or nested structure of TensorSpec
 |          objects, describing a candidate input for the layer.
 |      
 |      Returns:
 |        Single TensorSpec or nested structure of TensorSpec objects, describing
 |          how the layer would transform the provided input.
 |      
 |      Raises:
 |        TypeError: If input_signature contains a non-TensorSpec object.
 |  
 |  count_params(self)
 |      Count the total number of scalars composing the weights.
 |      
 |      Returns:
 |          An integer count.
 |      
 |      Raises:
 |          ValueError: if the layer isn't yet built
 |            (in which case its weights aren't yet defined).
 |  
 |  finalize_state(self)
 |      Finalizes the layers state after updating layer weights.
 |      
 |      This function can be subclassed in a layer and will be called after updating
 |      a layer weights. It can be overridden to finalize any additional layer state
 |      after a weight update.
 |  
 |  get_input_at(self, node_index)
 |      Retrieves the input tensor(s) of a layer at a given node.
 |      
 |      Args:
 |          node_index: Integer, index of the node
 |              from which to retrieve the attribute.
 |              E.g. `node_index=0` will correspond to the
 |              first input node of the layer.
 |      
 |      Returns:
 |          A tensor (or list of tensors if the layer has multiple inputs).
 |      
 |      Raises:
 |        RuntimeError: If called in Eager mode.
 |  
 |  get_input_mask_at(self, node_index)
 |      Retrieves the input mask tensor(s) of a layer at a given node.
 |      
 |      Args:
 |          node_index: Integer, index of the node
 |              from which to retrieve the attribute.
 |              E.g. `node_index=0` will correspond to the
 |              first time the layer was called.
 |      
 |      Returns:
 |          A mask tensor
 |          (or list of tensors if the layer has multiple inputs).
 |  
 |  get_input_shape_at(self, node_index)
 |      Retrieves the input shape(s) of a layer at a given node.
 |      
 |      Args:
 |          node_index: Integer, index of the node
 |              from which to retrieve the attribute.
 |              E.g. `node_index=0` will correspond to the
 |              first time the layer was called.
 |      
 |      Returns:
 |          A shape tuple
 |          (or list of shape tuples if the layer has multiple inputs).
 |      
 |      Raises:
 |        RuntimeError: If called in Eager mode.
 |  
 |  get_losses_for(self, inputs)
 |      Deprecated, do NOT use!
 |      
 |      Retrieves losses relevant to a specific set of inputs.
 |      
 |      Args:
 |        inputs: Input tensor or list/tuple of input tensors.
 |      
 |      Returns:
 |        List of loss tensors of the layer that depend on `inputs`.
 |  
 |  get_output_at(self, node_index)
 |      Retrieves the output tensor(s) of a layer at a given node.
 |      
 |      Args:
 |          node_index: Integer, index of the node
 |              from which to retrieve the attribute.
 |              E.g. `node_index=0` will correspond to the
 |              first output node of the layer.
 |      
 |      Returns:
 |          A tensor (or list of tensors if the layer has multiple outputs).
 |      
 |      Raises:
 |        RuntimeError: If called in Eager mode.
 |  
 |  get_output_mask_at(self, node_index)
 |      Retrieves the output mask tensor(s) of a layer at a given node.
 |      
 |      Args:
 |          node_index: Integer, index of the node
 |              from which to retrieve the attribute.
 |              E.g. `node_index=0` will correspond to the
 |              first time the layer was called.
 |      
 |      Returns:
 |          A mask tensor
 |          (or list of tensors if the layer has multiple outputs).
 |  
 |  get_output_shape_at(self, node_index)
 |      Retrieves the output shape(s) of a layer at a given node.
 |      
 |      Args:
 |          node_index: Integer, index of the node
 |              from which to retrieve the attribute.
 |              E.g. `node_index=0` will correspond to the
 |              first time the layer was called.
 |      
 |      Returns:
 |          A shape tuple
 |          (or list of shape tuples if the layer has multiple outputs).
 |      
 |      Raises:
 |        RuntimeError: If called in Eager mode.
 |  
 |  get_updates_for(self, inputs)
 |      Deprecated, do NOT use!
 |      
 |      Retrieves updates relevant to a specific set of inputs.
 |      
 |      Args:
 |        inputs: Input tensor or list/tuple of input tensors.
 |      
 |      Returns:
 |        List of update ops of the layer that depend on `inputs`.
 |  
 |  set_weights(self, weights)
 |      Sets the weights of the layer, from NumPy arrays.
 |      
 |      The weights of a layer represent the state of the layer. This function
 |      sets the weight values from numpy arrays. The weight values should be
 |      passed in the order they are created by the layer. Note that the layer's
 |      weights must be instantiated before calling this function, by calling
 |      the layer.
 |      
 |      For example, a `Dense` layer returns a list of two values: the kernel matrix
 |      and the bias vector. These can be used to set the weights of another
 |      `Dense` layer:
 |      
 |      >>> layer_a = tf.keras.layers.Dense(1,
 |      ...   kernel_initializer=tf.constant_initializer(1.))
 |      >>> a_out = layer_a(tf.convert_to_tensor([[1., 2., 3.]]))
 |      >>> layer_a.get_weights()
 |      [array([[1.],
 |             [1.],
 |             [1.]], dtype=float32), array([0.], dtype=float32)]
 |      >>> layer_b = tf.keras.layers.Dense(1,
 |      ...   kernel_initializer=tf.constant_initializer(2.))
 |      >>> b_out = layer_b(tf.convert_to_tensor([[10., 20., 30.]]))
 |      >>> layer_b.get_weights()
 |      [array([[2.],
 |             [2.],
 |             [2.]], dtype=float32), array([0.], dtype=float32)]
 |      >>> layer_b.set_weights(layer_a.get_weights())
 |      >>> layer_b.get_weights()
 |      [array([[1.],
 |             [1.],
 |             [1.]], dtype=float32), array([0.], dtype=float32)]
 |      
 |      Args:
 |        weights: a list of NumPy arrays. The number
 |          of arrays and their shape must match
 |          number of the dimensions of the weights
 |          of the layer (i.e. it should match the
 |          output of `get_weights`).
 |      
 |      Raises:
 |        ValueError: If the provided weights list does not match the
 |          layer's specifications.
 |  
 |  ----------------------------------------------------------------------
 |  Data descriptors inherited from keras.engine.base_layer.Layer:
 |  
 |  activity_regularizer
 |      Optional regularizer function for the output of this layer.
 |  
 |  compute_dtype
 |      The dtype of the layer's computations.
 |      
 |      This is equivalent to `Layer.dtype_policy.compute_dtype`. Unless
 |      mixed precision is used, this is the same as `Layer.dtype`, the dtype of
 |      the weights.
 |      
 |      Layers automatically cast their inputs to the compute dtype, which causes
 |      computations and the output to be in the compute dtype as well. This is done
 |      by the base Layer class in `Layer.__call__`, so you do not have to insert
 |      these casts if implementing your own layer.
 |      
 |      Layers often perform certain internal computations in higher precision when
 |      `compute_dtype` is float16 or bfloat16 for numeric stability. The output
 |      will still typically be float16 or bfloat16 in such cases.
 |      
 |      Returns:
 |        The layer's compute dtype.
 |  
 |  dtype
 |      The dtype of the layer weights.
 |      
 |      This is equivalent to `Layer.dtype_policy.variable_dtype`. Unless
 |      mixed precision is used, this is the same as `Layer.compute_dtype`, the
 |      dtype of the layer's computations.
 |  
 |  dtype_policy
 |      The dtype policy associated with this layer.
 |      
 |      This is an instance of a `tf.keras.mixed_precision.Policy`.
 |  
 |  dynamic
 |      Whether the layer is dynamic (eager-only); set in the constructor.
 |  
 |  inbound_nodes
 |      Deprecated, do NOT use! Only for compatibility with external Keras.
 |  
 |  input
 |      Retrieves the input tensor(s) of a layer.
 |      
 |      Only applicable if the layer has exactly one input,
 |      i.e. if it is connected to one incoming layer.
 |      
 |      Returns:
 |          Input tensor or list of input tensors.
 |      
 |      Raises:
 |        RuntimeError: If called in Eager mode.
 |        AttributeError: If no inbound nodes are found.
 |  
 |  input_mask
 |      Retrieves the input mask tensor(s) of a layer.
 |      
 |      Only applicable if the layer has exactly one inbound node,
 |      i.e. if it is connected to one incoming layer.
 |      
 |      Returns:
 |          Input mask tensor (potentially None) or list of input
 |          mask tensors.
 |      
 |      Raises:
 |          AttributeError: if the layer is connected to
 |          more than one incoming layers.
 |  
 |  input_shape
 |      Retrieves the input shape(s) of a layer.
 |      
 |      Only applicable if the layer has exactly one input,
 |      i.e. if it is connected to one incoming layer, or if all inputs
 |      have the same shape.
 |      
 |      Returns:
 |          Input shape, as an integer shape tuple
 |          (or list of shape tuples, one tuple per input tensor).
 |      
 |      Raises:
 |          AttributeError: if the layer has no defined input_shape.
 |          RuntimeError: if called in Eager mode.
 |  
 |  input_spec
 |      `InputSpec` instance(s) describing the input format for this layer.
 |      
 |      When you create a layer subclass, you can set `self.input_spec` to enable
 |      the layer to run input compatibility checks when it is called.
 |      Consider a `Conv2D` layer: it can only be called on a single input tensor
 |      of rank 4. As such, you can set, in `__init__()`:
 |      
 |      ```python
 |      self.input_spec = tf.keras.layers.InputSpec(ndim=4)
 |      ```
 |      
 |      Now, if you try to call the layer on an input that isn't rank 4
 |      (for instance, an input of shape `(2,)`, it will raise a nicely-formatted
 |      error:
 |      
 |      ```
 |      ValueError: Input 0 of layer conv2d is incompatible with the layer:
 |      expected ndim=4, found ndim=1. Full shape received: [2]
 |      ```
 |      
 |      Input checks that can be specified via `input_spec` include:
 |      - Structure (e.g. a single input, a list of 2 inputs, etc)
 |      - Shape
 |      - Rank (ndim)
 |      - Dtype
 |      
 |      For more information, see `tf.keras.layers.InputSpec`.
 |      
 |      Returns:
 |        A `tf.keras.layers.InputSpec` instance, or nested structure thereof.
 |  
 |  losses
 |      List of losses added using the `add_loss()` API.
 |      
 |      Variable regularization tensors are created when this property is accessed,
 |      so it is eager safe: accessing `losses` under a `tf.GradientTape` will
 |      propagate gradients back to the corresponding variables.
 |      
 |      Examples:
 |      
 |      >>> class MyLayer(tf.keras.layers.Layer):
 |      ...   def call(self, inputs):
 |      ...     self.add_loss(tf.abs(tf.reduce_mean(inputs)))
 |      ...     return inputs
 |      >>> l = MyLayer()
 |      >>> l(np.ones((10, 1)))
 |      >>> l.losses
 |      [1.0]
 |      
 |      >>> inputs = tf.keras.Input(shape=(10,))
 |      >>> x = tf.keras.layers.Dense(10)(inputs)
 |      >>> outputs = tf.keras.layers.Dense(1)(x)
 |      >>> model = tf.keras.Model(inputs, outputs)
 |      >>> # Activity regularization.
 |      >>> len(model.losses)
 |      0
 |      >>> model.add_loss(tf.abs(tf.reduce_mean(x)))
 |      >>> len(model.losses)
 |      1
 |      
 |      >>> inputs = tf.keras.Input(shape=(10,))
 |      >>> d = tf.keras.layers.Dense(10, kernel_initializer='ones')
 |      >>> x = d(inputs)
 |      >>> outputs = tf.keras.layers.Dense(1)(x)
 |      >>> model = tf.keras.Model(inputs, outputs)
 |      >>> # Weight regularization.
 |      >>> model.add_loss(lambda: tf.reduce_mean(d.kernel))
 |      >>> model.losses
 |      [<tf.Tensor: shape=(), dtype=float32, numpy=1.0>]
 |      
 |      Returns:
 |        A list of tensors.
 |  
 |  name
 |      Name of the layer (string), set in the constructor.
 |  
 |  non_trainable_variables
 |      Sequence of non-trainable variables owned by this module and its submodules.
 |      
 |      Note: this method uses reflection to find variables on the current instance
 |      and submodules. For performance reasons you may wish to cache the result
 |      of calling this method if you don't expect the return value to change.
 |      
 |      Returns:
 |        A sequence of variables for the current module (sorted by attribute
 |        name) followed by variables from all submodules recursively (breadth
 |        first).
 |  
 |  outbound_nodes
 |      Deprecated, do NOT use! Only for compatibility with external Keras.
 |  
 |  output
 |      Retrieves the output tensor(s) of a layer.
 |      
 |      Only applicable if the layer has exactly one output,
 |      i.e. if it is connected to one incoming layer.
 |      
 |      Returns:
 |        Output tensor or list of output tensors.
 |      
 |      Raises:
 |        AttributeError: if the layer is connected to more than one incoming
 |          layers.
 |        RuntimeError: if called in Eager mode.
 |  
 |  output_mask
 |      Retrieves the output mask tensor(s) of a layer.
 |      
 |      Only applicable if the layer has exactly one inbound node,
 |      i.e. if it is connected to one incoming layer.
 |      
 |      Returns:
 |          Output mask tensor (potentially None) or list of output
 |          mask tensors.
 |      
 |      Raises:
 |          AttributeError: if the layer is connected to
 |          more than one incoming layers.
 |  
 |  output_shape
 |      Retrieves the output shape(s) of a layer.
 |      
 |      Only applicable if the layer has one output,
 |      or if all outputs have the same shape.
 |      
 |      Returns:
 |          Output shape, as an integer shape tuple
 |          (or list of shape tuples, one tuple per output tensor).
 |      
 |      Raises:
 |          AttributeError: if the layer has no defined output shape.
 |          RuntimeError: if called in Eager mode.
 |  
 |  stateful
 |  
 |  supports_masking
 |      Whether this layer supports computing a mask using `compute_mask`.
 |  
 |  trainable
 |  
 |  trainable_variables
 |      Sequence of trainable variables owned by this module and its submodules.
 |      
 |      Note: this method uses reflection to find variables on the current instance
 |      and submodules. For performance reasons you may wish to cache the result
 |      of calling this method if you don't expect the return value to change.
 |      
 |      Returns:
 |        A sequence of variables for the current module (sorted by attribute
 |        name) followed by variables from all submodules recursively (breadth
 |        first).
 |  
 |  updates
 |  
 |  variable_dtype
 |      Alias of `Layer.dtype`, the dtype of the weights.
 |  
 |  variables
 |      Returns the list of all layer variables/weights.
 |      
 |      Alias of `self.weights`.
 |      
 |      Note: This will not track the weights of nested `tf.Modules` that are not
 |      themselves Keras layers.
 |      
 |      Returns:
 |        A list of variables.
 |  
 |  ----------------------------------------------------------------------
 |  Class methods inherited from tensorflow.python.module.module.Module:
 |  
 |  with_name_scope(method) from builtins.type
 |      Decorator to automatically enter the module name scope.
 |      
 |      >>> class MyModule(tf.Module):
 |      ...   @tf.Module.with_name_scope
 |      ...   def __call__(self, x):
 |      ...     if not hasattr(self, 'w'):
 |      ...       self.w = tf.Variable(tf.random.normal([x.shape[1], 3]))
 |      ...     return tf.matmul(x, self.w)
 |      
 |      Using the above module would produce `tf.Variable`s and `tf.Tensor`s whose
 |      names included the module name:
 |      
 |      >>> mod = MyModule()
 |      >>> mod(tf.ones([1, 2]))
 |      <tf.Tensor: shape=(1, 3), dtype=float32, numpy=..., dtype=float32)>
 |      >>> mod.w
 |      <tf.Variable 'my_module/Variable:0' shape=(2, 3) dtype=float32,
 |      numpy=..., dtype=float32)>
 |      
 |      Args:
 |        method: The method to wrap.
 |      
 |      Returns:
 |        The original method wrapped such that it enters the module's name scope.
 |  
 |  ----------------------------------------------------------------------
 |  Data descriptors inherited from tensorflow.python.module.module.Module:
 |  
 |  name_scope
 |      Returns a `tf.name_scope` instance for this class.
 |  
 |  submodules
 |      Sequence of all sub-modules.
 |      
 |      Submodules are modules which are properties of this module, or found as
 |      properties of modules which are properties of this module (and so on).
 |      
 |      >>> a = tf.Module()
 |      >>> b = tf.Module()
 |      >>> c = tf.Module()
 |      >>> a.b = b
 |      >>> b.c = c
 |      >>> list(a.submodules) == [b, c]
 |      True
 |      >>> list(b.submodules) == [c]
 |      True
 |      >>> list(c.submodules) == []
 |      True
 |      
 |      Returns:
 |        A sequence of all submodules.
 |  
 |  ----------------------------------------------------------------------
 |  Data descriptors inherited from tensorflow.python.training.tracking.base.Trackable:
 |  
 |  __dict__
 |      dictionary for instance variables (if defined)
 |  
 |  __weakref__
 |      list of weak references to the object (if defined)

Usar un subconjunto de funciones

El ejemplo anterior no especificó las características, por lo que todas las columnas se usaron como característica de entrada (excepto la etiqueta). El siguiente ejemplo muestra cómo especificar características de entrada.

feature_1 = tfdf.keras.FeatureUsage(name="bill_length_mm")
feature_2 = tfdf.keras.FeatureUsage(name="island")

all_features = [feature_1, feature_2]

# Note: This model is only trained with two features. It will not be as good as
# the one trained on all features.

model_2 = tfdf.keras.GradientBoostedTreesModel(
    features=all_features, exclude_non_specified_features=True)

model_2.compile(metrics=["accuracy"])
model_2.fit(x=train_ds, validation_data=test_ds)

print(model_2.evaluate(test_ds, return_dict=True))
1/4 [======>.......................] - ETA: 0s
[INFO kernel.cc:746] Start Yggdrasil model training
[INFO kernel.cc:747] Collect training examples
[INFO kernel.cc:392] Number of batches: 4
[INFO kernel.cc:393] Number of examples: 245
[INFO kernel.cc:769] Dataset:
Number of records: 245
Number of columns: 3

Number of columns by type:
    CATEGORICAL: 2 (66.6667%)
    NUMERICAL: 1 (33.3333%)

Columns:

CATEGORICAL: 2 (66.6667%)
    1: "island" CATEGORICAL has-dict vocab-size:4 zero-ood-items most-frequent:"Biscoe" 121 (49.3878%)
    2: "__LABEL" CATEGORICAL integerized vocab-size:4 no-ood-item

NUMERICAL: 1 (33.3333%)
    0: "bill_length_mm" NUMERICAL num-nas:1 (0.408163%) mean:43.8016 min:32.1 max:59.6 sd:5.48753

Terminology:
    nas: Number of non-available (i.e. missing) values.
    ood: Out of dictionary.
    manually-defined: Attribute which type is manually defined by the user i.e. the type was not automatically inferred.
    tokenized: The attribute value is obtained through tokenization.
    has-dict: The attribute is attached to a string dictionary e.g. a categorical attribute stored as a string.
    vocab-size: Number of unique values.

[INFO kernel.cc:772] Configure learner
[WARNING gradient_boosted_trees.cc:1692] Subsample hyperparameter given but sampling method does not match.
[WARNING gradient_boosted_trees.cc:1705] GOSS alpha hyperparameter given but GOSS is disabled.
[WARNING gradient_boosted_trees.cc:1714] GOSS beta hyperparameter given but GOSS is disabled.
[WARNING gradient_boosted_trees.cc:1726] SelGB ratio hyperparameter given but SelGB is disabled.
[INFO kernel.cc:797] Training config:
learner: "GRADIENT_BOOSTED_TREES"
features: "bill_length_mm"
features: "island"
label: "__LABEL"
task: CLASSIFICATION
[yggdrasil_decision_forests.model.gradient_boosted_trees.proto.gradient_boosted_trees_config] {
  num_trees: 300
  decision_tree {
    max_depth: 6
    min_examples: 5
    in_split_min_examples_check: true
    missing_value_policy: GLOBAL_IMPUTATION
    allow_na_conditions: false
    categorical_set_greedy_forward {
      sampling: 0.1
      max_num_items: -1
      min_item_frequency: 1
    }
    growing_strategy_local {
    }
    categorical {
      cart {
      }
    }
    num_candidate_attributes_ratio: -1
    axis_aligned_split {
    }
    internal {
      sorting_strategy: PRESORTED
    }
  }
  shrinkage: 0.1
  validation_set_ratio: 0.1
  early_stopping: VALIDATION_LOSS_INCREASE
  early_stopping_num_trees_look_ahead: 30
  l2_regularization: 0
  lambda_loss: 1
  mart {
  }
  adapt_subsample_for_maximum_training_duration: false
  l1_regularization: 0
  use_hessian_gain: false
  l2_regularization_categorical: 1
  apply_link_function: true
}

[INFO kernel.cc:800] Deployment config:
num_threads: 6

[INFO kernel.cc:837] Train model
[INFO gradient_boosted_trees.cc:503] Default loss set to MULTINOMIAL_LOG_LIKELIHOOD
[INFO gradient_boosted_trees.cc:1079] Training gradient boosted tree on 245 example(s) and 2 feature(s).
[INFO gradient_boosted_trees.cc:1493]     num-trees:1 train-loss:0.922904 train-accuracy:0.976852 valid-loss:0.919366 valid-accuracy:1.000000
[INFO gradient_boosted_trees.cc:1495]     num-trees:2 train-loss:0.783754 train-accuracy:0.976852 valid-loss:0.772764 valid-accuracy:1.000000
4/4 [==============================] - 1s 160ms/step - val_loss: 0.0000e+00 - val_accuracy: 0.9495
[INFO gradient_boosted_trees.cc:2750] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.00871497
[INFO gradient_boosted_trees.cc:336] Truncates the model to 249 tree(s) i.e. 83  iteration(s).
[INFO gradient_boosted_trees.cc:370] Final model num-trees:83 valid-loss:0.008715 valid-accuracy:1.000000
[INFO kernel.cc:856] Export model in log directory: /tmp/tmpubloiisb
[INFO kernel.cc:864] Save model in resources
[INFO kernel.cc:988] Loading model from path
[INFO decision_forest.cc:590] Model loaded with 249 root(s), 8475 node(s), and 2 input feature(s).
[INFO kernel.cc:848] Use fast generic engine
2/2 [==============================] - 0s 4ms/step - loss: 0.0000e+00 - accuracy: 0.9495
{'loss': 0.0, 'accuracy': 0.9494949579238892}

TF-DF concede una semántica a cada función. Esta semántica controla cómo el modelo utiliza la característica. Actualmente se admite la siguiente semántica:

  • Numérica: En general, para las cantidades o cuenta con plena pedido. Por ejemplo, la edad de una persona o la cantidad de artículos en una bolsa. Puede ser un número flotante o un número entero. Los valores que faltan se representan con float (Nan) o con un tensor disperso vacío.
  • Categórica: Generalmente para un tipo / clase en conjunto finito de valores posibles sin ordenar. Por ejemplo, el color ROJO en el conjunto {ROJO, AZUL, VERDE}. Puede ser una cadena o un número entero. Los valores faltantes se representan como "" (picadura vacía), valor -2 o con un tensor disperso vacío.
  • Categórica-Set: Un conjunto de valores categóricos. Genial para representar texto tokenizado. Puede ser una cadena o un número entero en un tensor escaso o desigual (recomendado). El orden / índice de cada artículo no importa.

Si no se especifica, la semántica se infiere del tipo de representación y se muestra en los registros de entrenamiento:

  • int, float (denso o disperso) → Semántica numérica.
  • str (denso o escaso) → Semántica categórica
  • int, str (irregular) → Semántica de conjuntos categóricos

En algunos casos, la semántica inferida es incorrecta. Por ejemplo: una enumeración almacenada como un número entero es semánticamente categórica, pero se detectará como numérica. En este caso, debe especificar el argumento semántico en la entrada. El education_num campo del conjunto de datos de adultos es ejemplo clásico.

Este conjunto de datos no contiene tal característica. Sin embargo, para la demostración, vamos a hacer el modelo de tratar los year como una característica categórica:

%set_cell_height 300

feature_1 = tfdf.keras.FeatureUsage(name="year", semantic=tfdf.keras.FeatureSemantic.CATEGORICAL)
feature_2 = tfdf.keras.FeatureUsage(name="bill_length_mm")
feature_3 = tfdf.keras.FeatureUsage(name="sex")
all_features = [feature_1, feature_2, feature_3]

model_3 = tfdf.keras.GradientBoostedTreesModel(features=all_features, exclude_non_specified_features=True)
model_3.compile( metrics=["accuracy"])

with sys_pipes():
  model_3.fit(x=train_ds, validation_data=test_ds)
<IPython.core.display.Javascript object>
1/4 [======>.......................] - ETA: 0s
[INFO kernel.cc:746] Start Yggdrasil model training
[INFO kernel.cc:747] Collect training examples
[INFO kernel.cc:392] Number of batches: 4
[INFO kernel.cc:393] Number of examples: 245
[INFO kernel.cc:769] Dataset:
Number of records: 245
Number of columns: 4

Number of columns by type:
    CATEGORICAL: 3 (75%)
    NUMERICAL: 1 (25%)

Columns:

CATEGORICAL: 3 (75%)
    1: "sex" CATEGORICAL num-nas:7 (2.85714%) has-dict vocab-size:3 zero-ood-items most-frequent:"male" 120 (50.4202%)
    2: "year" CATEGORICAL integerized vocab-size:2011 no-ood-item
    3: "__LABEL" CATEGORICAL integerized vocab-size:4 no-ood-item

NUMERICAL: 1 (25%)
    0: "bill_length_mm" NUMERICAL num-nas:1 (0.408163%) mean:43.8016 min:32.1 max:59.6 sd:5.48753

Terminology:
    nas: Number of non-available (i.e. missing) values.
    ood: Out of dictionary.
    manually-defined: Attribute which type is manually defined by the user i.e. the type was not automatically inferred.
    tokenized: The attribute value is obtained through tokenization.
    has-dict: The attribute is attached to a string dictionary e.g. a categorical attribute stored as a string.
    vocab-size: Number of unique values.

[INFO kernel.cc:772] Configure learner
[WARNING gradient_boosted_trees.cc:1692] Subsample hyperparameter given but sampling method does not match.
[WARNING gradient_boosted_trees.cc:1705] GOSS alpha hyperparameter given but GOSS is disabled.
[WARNING gradient_boosted_trees.cc:1714] GOSS beta hyperparameter given but GOSS is disabled.
[WARNING gradient_boosted_trees.cc:1726] SelGB ratio hyperparameter given but SelGB is disabled.
[INFO kernel.cc:797] Training config:
learner: "GRADIENT_BOOSTED_TREES"
features: "bill_length_mm"
features: "sex"
features: "year"
label: "__LABEL"
task: CLASSIFICATION
[yggdrasil_decision_forests.model.gradient_boosted_trees.proto.gradient_boosted_trees_config] {
  num_trees: 300
  decision_tree {
    max_depth: 6
    min_examples: 5
    in_split_min_examples_check: true
    missing_value_policy: GLOBAL_IMPUTATION
    allow_na_conditions: false
    categorical_set_greedy_forward {
      sampling: 0.1
      max_num_items: -1
      min_item_frequency: 1
    }
    growing_strategy_local {
    }
    categorical {
      cart {
      }
    }
    num_candidate_attributes_ratio: -1
    axis_aligned_split {
    }
    internal {
      sorting_strategy: PRESORTED
    }
  }
  shrinkage: 0.1
  validation_set_ratio: 0.1
  early_stopping: VALIDATION_LOSS_INCREASE
  early_stopping_num_trees_look_ahead: 30
  l2_regularization: 0
  lambda_loss: 1
  mart {
  }
  adapt_subsample_for_maximum_training_duration: false
  l1_regularization: 0
  use_hessian_gain: false
  l2_regularization_categorical: 1
  apply_link_function: true
}

[INFO kernel.cc:800] Deployment config:
num_threads: 6

[INFO kernel.cc:837] Train model
[INFO gradient_boosted_trees.cc:503] Default loss set to MULTINOMIAL_LOG_LIKELIHOOD
[INFO gradient_boosted_trees.cc:1079] Training gradient boosted tree on 245 example(s) and 3 feature(s).
[INFO gradient_boosted_trees.cc:1493]     num-trees:1 train-loss:0.971118 train-accuracy:0.847222 valid-loss:0.983760 valid-accuracy:0.896552
[INFO gradient_boosted_trees.cc:2750] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.410065
[INFO gradient_boosted_trees.cc:336] Truncates the model to 81 tree(s) i.e. 27  iteration(s).
[INFO gradient_boosted_trees.cc:370] Final model num-trees:27 valid-loss:0.410065 valid-accuracy:0.827586
[INFO kernel.cc:856] Export model in log directory: /tmp/tmpd8g4rfv2
[INFO kernel.cc:864] Save model in resources
[INFO kernel.cc:988] Loading model from path
[INFO decision_forest.cc:590] Model loaded with 81 root(s), 3491 node(s), and 3 input feature(s).
[INFO kernel.cc:848] Use fast generic engine
4/4 [==============================] - 0s 122ms/step - val_loss: 0.0000e+00 - val_accuracy: 0.7374

Tenga en cuenta que year está en la lista de características categóricas (a diferencia de la primera carrera).

Hiperparámetros

Hiper-parámetros son parámetros del algoritmo de entrenamiento que impactan la calidad del modelo final. Se especifican en el constructor de la clase del modelo. La lista de hiper-parámetros es visible con el comando colab signo de interrogación (por ejemplo ?tfdf.keras.GradientBoostedTreesModel ).

Alternativamente, se puede encontrar en la Decisión TensorFlow Bosque Github o la documentación de la Decisión Bosque Yggdrasil .

Los hiperparámetros predeterminados de cada algoritmo coinciden aproximadamente con el artículo de publicación inicial. Para garantizar la coherencia, las nuevas funciones y sus hiperparámetros coincidentes siempre están deshabilitados de forma predeterminada. Por eso es una buena idea ajustar sus hiperparámetros.

# A classical but slighly more complex model.
model_6 = tfdf.keras.GradientBoostedTreesModel(
    num_trees=500, growing_strategy="BEST_FIRST_GLOBAL", max_depth=8)
model_6.fit(x=train_ds)
4/4 [==============================] - 0s 2ms/step
[INFO kernel.cc:746] Start Yggdrasil model training
[INFO kernel.cc:747] Collect training examples
[INFO kernel.cc:392] Number of batches: 4
[INFO kernel.cc:393] Number of examples: 245
[INFO kernel.cc:769] Dataset:
Number of records: 245
Number of columns: 8

Number of columns by type:
    NUMERICAL: 5 (62.5%)
    CATEGORICAL: 3 (37.5%)

Columns:

NUMERICAL: 5 (62.5%)
    0: "bill_depth_mm" NUMERICAL num-nas:1 (0.408163%) mean:17.1254 min:13.2 max:21.5 sd:2.00348
    1: "bill_length_mm" NUMERICAL num-nas:1 (0.408163%) mean:43.8016 min:32.1 max:59.6 sd:5.48753
    2: "body_mass_g" NUMERICAL num-nas:1 (0.408163%) mean:4185.96 min:2850 max:6300 sd:811.612
    3: "flipper_length_mm" NUMERICAL num-nas:1 (0.408163%) mean:201.061 min:172 max:231 sd:14.3124
    6: "year" NUMERICAL mean:2008.09 min:2007 max:2009 sd:0.828139

CATEGORICAL: 3 (37.5%)
    4: "island" CATEGORICAL has-dict vocab-size:4 zero-ood-items most-frequent:"Biscoe" 121 (49.3878%)
    5: "sex" CATEGORICAL num-nas:7 (2.85714%) has-dict vocab-size:3 zero-ood-items most-frequent:"male" 120 (50.4202%)
    7: "__LABEL" CATEGORICAL integerized vocab-size:4 no-ood-item

Terminology:
    nas: Number of non-available (i.e. missing) values.
    ood: Out of dictionary.
    manually-defined: Attribute which type is manually defined by the user i.e. the type was not automatically inferred.
    tokenized: The attribute value is obtained through tokenization.
    has-dict: The attribute is attached to a string dictionary e.g. a categorical attribute stored as a string.
    vocab-size: Number of unique values.

[INFO kernel.cc:772] Configure learner
[WARNING gradient_boosted_trees.cc:1692] Subsample hyperparameter given but sampling method does not match.
[WARNING gradient_boosted_trees.cc:1705] GOSS alpha hyperparameter given but GOSS is disabled.
[WARNING gradient_boosted_trees.cc:1714] GOSS beta hyperparameter given but GOSS is disabled.
[WARNING gradient_boosted_trees.cc:1726] SelGB ratio hyperparameter given but SelGB is disabled.
[INFO kernel.cc:797] Training config:
learner: "GRADIENT_BOOSTED_TREES"
features: "bill_depth_mm"
features: "bill_length_mm"
features: "body_mass_g"
features: "flipper_length_mm"
features: "island"
features: "sex"
features: "year"
label: "__LABEL"
task: CLASSIFICATION
[yggdrasil_decision_forests.model.gradient_boosted_trees.proto.gradient_boosted_trees_config] {
  num_trees: 500
  decision_tree {
    max_depth: 8
    min_examples: 5
    in_split_min_examples_check: true
    missing_value_policy: GLOBAL_IMPUTATION
    allow_na_conditions: false
    categorical_set_greedy_forward {
      sampling: 0.1
      max_num_items: -1
      min_item_frequency: 1
    }
    growing_strategy_best_first_global {
    }
    categorical {
      cart {
      }
    }
    num_candidate_attributes_ratio: -1
    axis_aligned_split {
    }
    internal {
      sorting_strategy: PRESORTED
    }
  }
  shrinkage: 0.1
  validation_set_ratio: 0.1
  early_stopping: VALIDATION_LOSS_INCREASE
  early_stopping_num_trees_look_ahead: 30
  l2_regularization: 0
  lambda_loss: 1
  mart {
  }
  adapt_subsample_for_maximum_training_duration: false
  l1_regularization: 0
  use_hessian_gain: false
  l2_regularization_categorical: 1
  apply_link_function: true
}

[INFO kernel.cc:800] Deployment config:
num_threads: 6

[INFO kernel.cc:837] Train model
[INFO gradient_boosted_trees.cc:503] Default loss set to MULTINOMIAL_LOG_LIKELIHOOD
[INFO gradient_boosted_trees.cc:1079] Training gradient boosted tree on 245 example(s) and 7 feature(s).
[INFO gradient_boosted_trees.cc:1493]     num-trees:1 train-loss:0.917692 train-accuracy:1.000000 valid-loss:0.918721 valid-accuracy:1.000000
[INFO gradient_boosted_trees.cc:2750] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.000341958
[INFO gradient_boosted_trees.cc:336] Truncates the model to 513 tree(s) i.e. 171  iteration(s).
[INFO gradient_boosted_trees.cc:370] Final model num-trees:171 valid-loss:0.000342 valid-accuracy:1.000000
[INFO kernel.cc:856] Export model in log directory: /tmp/tmpkc34wq5q
[INFO kernel.cc:864] Save model in resources
[INFO kernel.cc:988] Loading model from path
[INFO decision_forest.cc:590] Model loaded with 513 root(s), 29733 node(s), and 7 input feature(s).
[INFO kernel.cc:848] Use fast generic engine
<keras.callbacks.History at 0x7fb2cc049d50>
# A more complex, but possibly, more accurate model.
model_7 = tfdf.keras.GradientBoostedTreesModel(
    num_trees=500,
    growing_strategy="BEST_FIRST_GLOBAL",
    max_depth=8,
    split_axis="SPARSE_OBLIQUE",
    categorical_algorithm="RANDOM",
    )
model_7.fit(x=train_ds)
4/4 [==============================] - 0s 2ms/step
[INFO kernel.cc:746] Start Yggdrasil model training
[INFO kernel.cc:747] Collect training examples
[INFO kernel.cc:392] Number of batches: 4
[INFO kernel.cc:393] Number of examples: 245
[INFO kernel.cc:769] Dataset:
Number of records: 245
Number of columns: 8

Number of columns by type:
    NUMERICAL: 5 (62.5%)
    CATEGORICAL: 3 (37.5%)

Columns:

NUMERICAL: 5 (62.5%)
    0: "bill_depth_mm" NUMERICAL num-nas:1 (0.408163%) mean:17.1254 min:13.2 max:21.5 sd:2.00348
    1: "bill_length_mm" NUMERICAL num-nas:1 (0.408163%) mean:43.8016 min:32.1 max:59.6 sd:5.48753
    2: "body_mass_g" NUMERICAL num-nas:1 (0.408163%) mean:4185.96 min:2850 max:6300 sd:811.612
    3: "flipper_length_mm" NUMERICAL num-nas:1 (0.408163%) mean:201.061 min:172 max:231 sd:14.3124
    6: "year" NUMERICAL mean:2008.09 min:2007 max:2009 sd:0.828139

CATEGORICAL: 3 (37.5%)
    4: "island" CATEGORICAL has-dict vocab-size:4 zero-ood-items most-frequent:"Biscoe" 121 (49.3878%)
    5: "sex" CATEGORICAL num-nas:7 (2.85714%) has-dict vocab-size:3 zero-ood-items most-frequent:"male" 120 (50.4202%)
    7: "__LABEL" CATEGORICAL integerized vocab-size:4 no-ood-item

Terminology:
    nas: Number of non-available (i.e. missing) values.
    ood: Out of dictionary.
    manually-defined: Attribute which type is manually defined by the user i.e. the type was not automatically inferred.
    tokenized: The attribute value is obtained through tokenization.
    has-dict: The attribute is attached to a string dictionary e.g. a categorical attribute stored as a string.
    vocab-size: Number of unique values.

[INFO kernel.cc:772] Configure learner
[WARNING gradient_boosted_trees.cc:1692] Subsample hyperparameter given but sampling method does not match.
[WARNING gradient_boosted_trees.cc:1705] GOSS alpha hyperparameter given but GOSS is disabled.
[WARNING gradient_boosted_trees.cc:1714] GOSS beta hyperparameter given but GOSS is disabled.
[WARNING gradient_boosted_trees.cc:1726] SelGB ratio hyperparameter given but SelGB is disabled.
[INFO kernel.cc:797] Training config:
learner: "GRADIENT_BOOSTED_TREES"
features: "bill_depth_mm"
features: "bill_length_mm"
features: "body_mass_g"
features: "flipper_length_mm"
features: "island"
features: "sex"
features: "year"
label: "__LABEL"
task: CLASSIFICATION
[yggdrasil_decision_forests.model.gradient_boosted_trees.proto.gradient_boosted_trees_config] {
  num_trees: 500
  decision_tree {
    max_depth: 8
    min_examples: 5
    in_split_min_examples_check: true
    missing_value_policy: GLOBAL_IMPUTATION
    allow_na_conditions: false
    categorical_set_greedy_forward {
      sampling: 0.1
      max_num_items: -1
      min_item_frequency: 1
    }
    growing_strategy_best_first_global {
    }
    categorical {
      random {
      }
    }
    num_candidate_attributes_ratio: -1
    sparse_oblique_split {
    }
    internal {
      sorting_strategy: PRESORTED
    }
  }
  shrinkage: 0.1
  validation_set_ratio: 0.1
  early_stopping: VALIDATION_LOSS_INCREASE
  early_stopping_num_trees_look_ahead: 30
  l2_regularization: 0
  lambda_loss: 1
  mart {
  }
  adapt_subsample_for_maximum_training_duration: false
  l1_regularization: 0
  use_hessian_gain: false
  l2_regularization_categorical: 1
  apply_link_function: true
}

[INFO kernel.cc:800] Deployment config:
num_threads: 6

[INFO kernel.cc:837] Train model
[INFO gradient_boosted_trees.cc:503] Default loss set to MULTINOMIAL_LOG_LIKELIHOOD
[INFO gradient_boosted_trees.cc:1079] Training gradient boosted tree on 245 example(s) and 7 feature(s).
[INFO gradient_boosted_trees.cc:1493]     num-trees:1 train-loss:0.914094 train-accuracy:1.000000 valid-loss:0.912344 valid-accuracy:0.965517
WARNING:tensorflow:5 out of the last 5 calls to <function CoreModel.make_predict_function.<locals>.predict_function_trained at 0x7faf06df3ef0> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has experimental_relax_shapes=True option that relaxes argument shapes that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.
[INFO gradient_boosted_trees.cc:2750] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.00679692
[INFO gradient_boosted_trees.cc:336] Truncates the model to 234 tree(s) i.e. 78  iteration(s).
[INFO gradient_boosted_trees.cc:370] Final model num-trees:78 valid-loss:0.006797 valid-accuracy:1.000000
[INFO kernel.cc:856] Export model in log directory: /tmp/tmppuf9_w6_
[INFO kernel.cc:864] Save model in resources
[INFO kernel.cc:988] Loading model from path
[INFO decision_forest.cc:590] Model loaded with 234 root(s), 11828 node(s), and 7 input feature(s).
[INFO kernel.cc:848] Use fast generic engine
WARNING:tensorflow:5 out of the last 5 calls to <function CoreModel.make_predict_function.<locals>.predict_function_trained at 0x7faf06df3ef0> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has experimental_relax_shapes=True option that relaxes argument shapes that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.
<keras.callbacks.History at 0x7faf06de86d0>

A medida que se publican e implementan nuevos métodos de entrenamiento, las combinaciones de hiperparámetros pueden surgir tan buenas o casi siempre mejores que los parámetros predeterminados. Para evitar cambiar los valores de hiperparámetros predeterminados, estas buenas combinaciones están indexadas y disponibles como plantillas de hiperparámetros.

Por ejemplo, el benchmark_rank1 plantilla es la mejor Combinaison en nuestros puntos de referencia internos. Esas plantillas están versionadas para permitir la formación estabilidad de configuración por ejemplo benchmark_rank1@v1 .

# A good template of hyper-parameters.
model_8 = tfdf.keras.GradientBoostedTreesModel(hyperparameter_template="benchmark_rank1")
model_8.fit(x=train_ds)
4/4 [==============================] - 0s 2ms/step
[INFO kernel.cc:746] Start Yggdrasil model training
[INFO kernel.cc:747] Collect training examples
[INFO kernel.cc:392] Number of batches: 4
[INFO kernel.cc:393] Number of examples: 245
[INFO kernel.cc:769] Dataset:
Number of records: 245
Number of columns: 8

Number of columns by type:
    NUMERICAL: 5 (62.5%)
    CATEGORICAL: 3 (37.5%)

Columns:

NUMERICAL: 5 (62.5%)
    0: "bill_depth_mm" NUMERICAL num-nas:1 (0.408163%) mean:17.1254 min:13.2 max:21.5 sd:2.00348
    1: "bill_length_mm" NUMERICAL num-nas:1 (0.408163%) mean:43.8016 min:32.1 max:59.6 sd:5.48753
    2: "body_mass_g" NUMERICAL num-nas:1 (0.408163%) mean:4185.96 min:2850 max:6300 sd:811.612
    3: "flipper_length_mm" NUMERICAL num-nas:1 (0.408163%) mean:201.061 min:172 max:231 sd:14.3124
    6: "year" NUMERICAL mean:2008.09 min:2007 max:2009 sd:0.828139

CATEGORICAL: 3 (37.5%)
    4: "island" CATEGORICAL has-dict vocab-size:4 zero-ood-items most-frequent:"Biscoe" 121 (49.3878%)
    5: "sex" CATEGORICAL num-nas:7 (2.85714%) has-dict vocab-size:3 zero-ood-items most-frequent:"male" 120 (50.4202%)
    7: "__LABEL" CATEGORICAL integerized vocab-size:4 no-ood-item

Terminology:
    nas: Number of non-available (i.e. missing) values.
    ood: Out of dictionary.
    manually-defined: Attribute which type is manually defined by the user i.e. the type was not automatically inferred.
    tokenized: The attribute value is obtained through tokenization.
    has-dict: The attribute is attached to a string dictionary e.g. a categorical attribute stored as a string.
    vocab-size: Number of unique values.

[INFO kernel.cc:772] Configure learner
[WARNING gradient_boosted_trees.cc:1692] Subsample hyperparameter given but sampling method does not match.
[WARNING gradient_boosted_trees.cc:1705] GOSS alpha hyperparameter given but GOSS is disabled.
[WARNING gradient_boosted_trees.cc:1714] GOSS beta hyperparameter given but GOSS is disabled.
[WARNING gradient_boosted_trees.cc:1726] SelGB ratio hyperparameter given but SelGB is disabled.
[INFO kernel.cc:797] Training config:
learner: "GRADIENT_BOOSTED_TREES"
features: "bill_depth_mm"
features: "bill_length_mm"
features: "body_mass_g"
features: "flipper_length_mm"
features: "island"
features: "sex"
features: "year"
label: "__LABEL"
task: CLASSIFICATION
[yggdrasil_decision_forests.model.gradient_boosted_trees.proto.gradient_boosted_trees_config] {
  num_trees: 300
  decision_tree {
    max_depth: 6
    min_examples: 5
    in_split_min_examples_check: true
    missing_value_policy: GLOBAL_IMPUTATION
    allow_na_conditions: false
    categorical_set_greedy_forward {
      sampling: 0.1
      max_num_items: -1
      min_item_frequency: 1
    }
    growing_strategy_best_first_global {
    }
    categorical {
      random {
      }
    }
    num_candidate_attributes_ratio: -1
    sparse_oblique_split {
      num_projections_exponent: 1
      normalization: MIN_MAX
    }
    internal {
      sorting_strategy: PRESORTED
    }
  }
  shrinkage: 0.1
  validation_set_ratio: 0.1
  early_stopping: VALIDATION_LOSS_INCREASE
  early_stopping_num_trees_look_ahead: 30
  l2_regularization: 0
  lambda_loss: 1
  mart {
  }
  adapt_subsample_for_maximum_training_duration: false
  l1_regularization: 0
  use_hessian_gain: false
  l2_regularization_categorical: 1
  apply_link_function: true
}

[INFO kernel.cc:800] Deployment config:
num_threads: 6

[INFO kernel.cc:837] Train model
[INFO gradient_boosted_trees.cc:503] Default loss set to MULTINOMIAL_LOG_LIKELIHOOD
[INFO gradient_boosted_trees.cc:1079] Training gradient boosted tree on 245 example(s) and 7 feature(s).
[INFO gradient_boosted_trees.cc:1493]     num-trees:1 train-loss:0.914621 train-accuracy:1.000000 valid-loss:0.919392 valid-accuracy:0.965517
[INFO gradient_boosted_trees.cc:2750] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.00130715
[INFO gradient_boosted_trees.cc:336] Truncates the model to 357 tree(s) i.e. 119  iteration(s).
[INFO gradient_boosted_trees.cc:370] Final model num-trees:119 valid-loss:0.001307 valid-accuracy:1.000000
[INFO kernel.cc:856] Export model in log directory: /tmp/tmpuwy03v7z
[INFO kernel.cc:864] Save model in resources
[INFO kernel.cc:988] Loading model from path
WARNING:tensorflow:6 out of the last 6 calls to <function CoreModel.make_predict_function.<locals>.predict_function_trained at 0x7faf06d0cc20> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has experimental_relax_shapes=True option that relaxes argument shapes that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.
[INFO decision_forest.cc:590] Model loaded with 357 root(s), 13329 node(s), and 7 input feature(s).
[INFO kernel.cc:848] Use fast generic engine
WARNING:tensorflow:6 out of the last 6 calls to <function CoreModel.make_predict_function.<locals>.predict_function_trained at 0x7faf06d0cc20> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has experimental_relax_shapes=True option that relaxes argument shapes that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.
<keras.callbacks.History at 0x7faf06d706d0>

Los Tempaltes disponibles están disponibles con predefined_hyperparameters . Tenga en cuenta que los diferentes algoritmos de aprendizaje tienen diferentes plantillas, incluso si el nombre es similar.

# The hyper-parameter templates of the Gradient Boosted Tree model.
print(tfdf.keras.GradientBoostedTreesModel.predefined_hyperparameters())
[HyperParameterTemplate(name='better_default', version=1, parameters={'growing_strategy': 'BEST_FIRST_GLOBAL'}, description='A configuration that is generally better than the default parameters without being more expensive.'), HyperParameterTemplate(name='benchmark_rank1', version=1, parameters={'growing_strategy': 'BEST_FIRST_GLOBAL', 'categorical_algorithm': 'RANDOM', 'split_axis': 'SPARSE_OBLIQUE', 'sparse_oblique_normalization': 'MIN_MAX', 'sparse_oblique_num_projections_exponent': 1.0}, description='Top ranking hyper-parameters on our benchmark slightly modified to run in reasonable time.')]

Preprocesamiento de funciones

A veces es necesario preprocesar características para consumir señales con estructuras complejas, regularizar el modelo o aplicar el aprendizaje por transferencia. El preprocesamiento se puede realizar de tres formas:

  1. Procesamiento previo en el marco de datos de Pandas. Esta solución es fácil de implementar y generalmente adecuada para la experimentación. Sin embargo, la lógica de tratamiento previo no se exportará en el modelo por model.save() .

  2. Keras Preprocesamiento : Aunque más compleja que la solución anterior, Keras preprocesamiento se empaqueta en el modelo.

  3. TensorFlow presentan columnas : Esta API es parte de la biblioteca del TF Estimador (= Keras!) Y previsto para desaprobación. Esta solución es interesante cuando se usa código de preprocesamiento existente.

En el siguiente ejemplo, pre-procesar la body_mass_g característica en body_mass_kg = body_mass_g / 1000 . El bill_length_mm se consume sin tratamiento previo. Tenga en cuenta que estas transformaciones monótonas generalmente no tienen impacto en los modelos de bosque de decisiones.

%set_cell_height 300

body_mass_g = tf.keras.layers.Input(shape=(1,), name="body_mass_g")
body_mass_kg = body_mass_g / 1000.0

bill_length_mm = tf.keras.layers.Input(shape=(1,), name="bill_length_mm")

raw_inputs = {"body_mass_g": body_mass_g, "bill_length_mm": bill_length_mm}
processed_inputs = {"body_mass_kg": body_mass_kg, "bill_length_mm": bill_length_mm}

# "preprocessor" contains the preprocessing logic.
preprocessor = tf.keras.Model(inputs=raw_inputs, outputs=processed_inputs)

# "model_4" contains both the pre-processing logic and the decision forest.
model_4 = tfdf.keras.RandomForestModel(preprocessing=preprocessor)
model_4.fit(x=train_ds)

model_4.summary()
<IPython.core.display.Javascript object>
/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/keras/engine/functional.py:585: UserWarning: Input dict contained keys ['island', 'bill_depth_mm', 'flipper_length_mm', 'sex', 'year'] which did not match any model input. They will be ignored by the model.
  [n for n in tensors.keys() if n not in ref_input_names])
4/4 [==============================] - 0s 2ms/step
Model: "random_forest_model_1"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
model (Functional)           {'body_mass_kg': (None, 1 0         
=================================================================
Total params: 1
Trainable params: 0
Non-trainable params: 1
_________________________________________________________________
Type: "RANDOM_FOREST"
Task: CLASSIFICATION
Label: "__LABEL"

Input Features (2):
    bill_length_mm
    body_mass_kg

No weights

Variable Importance: MEAN_MIN_DEPTH:

    1.        "__LABEL"  3.944941 ################
    2.   "body_mass_kg"  1.094376 ####
    3. "bill_length_mm"  0.012333 

Variable Importance: NUM_AS_ROOT:

    1. "bill_length_mm" 297.000000 ################
    2.   "body_mass_kg"  3.000000 

Variable Importance: NUM_NODES:

    1. "bill_length_mm" 1655.000000 ################
    2.   "body_mass_kg" 1317.000000 

Variable Importance: SUM_SCORE:

    1. "bill_length_mm" 45305.476804 ################
    2.   "body_mass_kg" 23757.287657 



Winner take all: true
Out-of-bag evaluation: accuracy:0.922449 logloss:0.366314
Number of trees: 300
Total number of nodes: 6244

Number of nodes by tree:
Count: 300 Average: 20.8133 StdDev: 3.10996
Min: 13 Max: 31 Ignored: 0
----------------------------------------------
[ 13, 14)  4   1.33%   1.33% #
[ 14, 15)  0   0.00%   1.33%
[ 15, 16) 11   3.67%   5.00% #
[ 16, 17)  0   0.00%   5.00%
[ 17, 18) 37  12.33%  17.33% #####
[ 18, 19)  0   0.00%  17.33%
[ 19, 20) 74  24.67%  42.00% ##########
[ 20, 21)  0   0.00%  42.00%
[ 21, 22) 68  22.67%  64.67% #########
[ 22, 23)  0   0.00%  64.67%
[ 23, 24) 62  20.67%  85.33% ########
[ 24, 25)  0   0.00%  85.33%
[ 25, 26) 29   9.67%  95.00% ####
[ 26, 27)  0   0.00%  95.00%
[ 27, 28) 12   4.00%  99.00% ##
[ 28, 29)  0   0.00%  99.00%
[ 29, 30)  2   0.67%  99.67%
[ 30, 31)  0   0.00%  99.67%
[ 31, 31]  1   0.33% 100.00%

Depth by leafs:
Count: 3272 Average: 3.98319 StdDev: 1.33255
Min: 1 Max: 9 Ignored: 0
----------------------------------------------
[ 1, 2)   30   0.92%   0.92%
[ 2, 3)  377  11.52%  12.44% ####
[ 3, 4)  819  25.03%  37.47% ########
[ 4, 5) 1032  31.54%  69.01% ##########
[ 5, 6)  589  18.00%  87.01% ######
[ 6, 7)  292   8.92%  95.94% ###
[ 7, 8)  101   3.09%  99.02% #
[ 8, 9)   28   0.86%  99.88%
[ 9, 9]    4   0.12% 100.00%

Number of training obs by leaf:
Count: 3272 Average: 22.4633 StdDev: 28.982
Min: 5 Max: 119 Ignored: 0
----------------------------------------------
[   5,  10) 2112  64.55%  64.55% ##########
[  10,  16)  214   6.54%  71.09% #
[  16,  22)   54   1.65%  72.74%
[  22,  28)   97   2.96%  75.70%
[  28,  33)   90   2.75%  78.45%
[  33,  39)   76   2.32%  80.78%
[  39,  45)   36   1.10%  81.88%
[  45,  51)   16   0.49%  82.37%
[  51,  56)   41   1.25%  83.62%
[  56,  62)   55   1.68%  85.30%
[  62,  68)   76   2.32%  87.62%
[  68,  74)   62   1.89%  89.52%
[  74,  79)   28   0.86%  90.37%
[  79,  85)   46   1.41%  91.78%
[  85,  91)   60   1.83%  93.61%
[  91,  97)   82   2.51%  96.12%
[  97, 102)   56   1.71%  97.83%
[ 102, 108)   50   1.53%  99.36%
[ 108, 114)   16   0.49%  99.85%
[ 114, 119]    5   0.15% 100.00%

Attribute in nodes:
    1655 : bill_length_mm [NUMERICAL]
    1317 : body_mass_kg [NUMERICAL]

Attribute in nodes with depth <= 0:
    297 : bill_length_mm [NUMERICAL]
    3 : body_mass_kg [NUMERICAL]

Attribute in nodes with depth <= 1:
    477 : body_mass_kg [NUMERICAL]
    393 : bill_length_mm [NUMERICAL]

Attribute in nodes with depth <= 2:
    864 : bill_length_mm [NUMERICAL]
    769 : body_mass_kg [NUMERICAL]

Attribute in nodes with depth <= 3:
    1278 : bill_length_mm [NUMERICAL]
    1062 : body_mass_kg [NUMERICAL]

Attribute in nodes with depth <= 5:
    1612 : bill_length_mm [NUMERICAL]
    1285 : body_mass_kg [NUMERICAL]

Condition type in nodes:
    2972 : HigherCondition
Condition type in nodes with depth <= 0:
    300 : HigherCondition
Condition type in nodes with depth <= 1:
    870 : HigherCondition
Condition type in nodes with depth <= 2:
    1633 : HigherCondition
Condition type in nodes with depth <= 3:
    2340 : HigherCondition
Condition type in nodes with depth <= 5:
    2897 : HigherCondition
Node format: NOT_SET

Training OOB:
    trees: 1, Out-of-bag evaluation: accuracy:0.858696 logloss:5.09312
    trees: 11, Out-of-bag evaluation: accuracy:0.90535 logloss:1.85944
    trees: 21, Out-of-bag evaluation: accuracy:0.914286 logloss:1.71015
    trees: 31, Out-of-bag evaluation: accuracy:0.910204 logloss:1.57257
    trees: 41, Out-of-bag evaluation: accuracy:0.926531 logloss:1.30585
    trees: 51, Out-of-bag evaluation: accuracy:0.910204 logloss:1.30729
    trees: 61, Out-of-bag evaluation: accuracy:0.926531 logloss:1.16995
    trees: 71, Out-of-bag evaluation: accuracy:0.926531 logloss:1.03329
    trees: 81, Out-of-bag evaluation: accuracy:0.914286 logloss:0.900096
    trees: 91, Out-of-bag evaluation: accuracy:0.918367 logloss:0.897564
    trees: 101, Out-of-bag evaluation: accuracy:0.922449 logloss:0.893559
    trees: 111, Out-of-bag evaluation: accuracy:0.926531 logloss:0.89369
    trees: 121, Out-of-bag evaluation: accuracy:0.922449 logloss:0.893663
    trees: 131, Out-of-bag evaluation: accuracy:0.930612 logloss:0.759198
    trees: 141, Out-of-bag evaluation: accuracy:0.926531 logloss:0.759931
    trees: 151, Out-of-bag evaluation: accuracy:0.922449 logloss:0.760019
    trees: 161, Out-of-bag evaluation: accuracy:0.922449 logloss:0.758033
    trees: 171, Out-of-bag evaluation: accuracy:0.922449 logloss:0.62668
    trees: 181, Out-of-bag evaluation: accuracy:0.922449 logloss:0.627142
    trees: 191, Out-of-bag evaluation: accuracy:0.922449 logloss:0.499365
    trees: 201, Out-of-bag evaluation: accuracy:0.926531 logloss:0.49864
    trees: 211, Out-of-bag evaluation: accuracy:0.922449 logloss:0.369094
    trees: 223, Out-of-bag evaluation: accuracy:0.922449 logloss:0.370484
    trees: 233, Out-of-bag evaluation: accuracy:0.918367 logloss:0.367249
    trees: 244, Out-of-bag evaluation: accuracy:0.918367 logloss:0.365513
    trees: 254, Out-of-bag evaluation: accuracy:0.918367 logloss:0.366475
    trees: 264, Out-of-bag evaluation: accuracy:0.918367 logloss:0.367287
    trees: 274, Out-of-bag evaluation: accuracy:0.918367 logloss:0.364562
    trees: 284, Out-of-bag evaluation: accuracy:0.918367 logloss:0.366077
    trees: 294, Out-of-bag evaluation: accuracy:0.926531 logloss:0.365515
    trees: 300, Out-of-bag evaluation: accuracy:0.922449 logloss:0.366314
[INFO kernel.cc:746] Start Yggdrasil model training
[INFO kernel.cc:747] Collect training examples
[INFO kernel.cc:392] Number of batches: 4
[INFO kernel.cc:393] Number of examples: 245
[INFO kernel.cc:769] Dataset:
Number of records: 245
Number of columns: 3

Number of columns by type:
    NUMERICAL: 2 (66.6667%)
    CATEGORICAL: 1 (33.3333%)

Columns:

NUMERICAL: 2 (66.6667%)
    0: "bill_length_mm" NUMERICAL num-nas:1 (0.408163%) mean:43.8016 min:32.1 max:59.6 sd:5.48753
    1: "body_mass_kg" NUMERICAL num-nas:1 (0.408163%) mean:4.18596 min:2.85 max:6.3 sd:0.811612

CATEGORICAL: 1 (33.3333%)
    2: "__LABEL" CATEGORICAL integerized vocab-size:4 no-ood-item

Terminology:
    nas: Number of non-available (i.e. missing) values.
    ood: Out of dictionary.
    manually-defined: Attribute which type is manually defined by the user i.e. the type was not automatically inferred.
    tokenized: The attribute value is obtained through tokenization.
    has-dict: The attribute is attached to a string dictionary e.g. a categorical attribute stored as a string.
    vocab-size: Number of unique values.

[INFO kernel.cc:772] Configure learner
[INFO kernel.cc:797] Training config:
learner: "RANDOM_FOREST"
features: "bill_length_mm"
features: "body_mass_kg"
label: "__LABEL"
task: CLASSIFICATION
[yggdrasil_decision_forests.model.random_forest.proto.random_forest_config] {
  num_trees: 300
  decision_tree {
    max_depth: 16
    min_examples: 5
    in_split_min_examples_check: true
    missing_value_policy: GLOBAL_IMPUTATION
    allow_na_conditions: false
    categorical_set_greedy_forward {
      sampling: 0.1
      max_num_items: -1
      min_item_frequency: 1
    }
    growing_strategy_local {
    }
    categorical {
      cart {
      }
    }
    num_candidate_attributes_ratio: -1
    axis_aligned_split {
    }
    internal {
      sorting_strategy: PRESORTED
    }
  }
  winner_take_all_inference: true
  compute_oob_performances: true
  compute_oob_variable_importances: false
  adapt_bootstrap_size_ratio_for_maximum_training_duration: false
}

[INFO kernel.cc:800] Deployment config:
num_threads: 6

[INFO kernel.cc:837] Train model
[INFO random_forest.cc:303] Training random forest on 245 example(s) and 2 feature(s).
[INFO random_forest.cc:578] Training of tree  1/300 (tree index:0) done accuracy:0.858696 logloss:5.09312
[INFO random_forest.cc:578] Training of tree  11/300 (tree index:10) done accuracy:0.90535 logloss:1.85944
[INFO random_forest.cc:578] Training of tree  21/300 (tree index:20) done accuracy:0.914286 logloss:1.71015
[INFO random_forest.cc:578] Training of tree  31/300 (tree index:29) done accuracy:0.910204 logloss:1.57257
[INFO random_forest.cc:578] Training of tree  41/300 (tree index:41) done accuracy:0.926531 logloss:1.30585
[INFO random_forest.cc:578] Training of tree  51/300 (tree index:52) done accuracy:0.910204 logloss:1.30729
[INFO random_forest.cc:578] Training of tree  61/300 (tree index:60) done accuracy:0.926531 logloss:1.16995
[INFO random_forest.cc:578] Training of tree  71/300 (tree index:72) done accuracy:0.926531 logloss:1.03329
[INFO random_forest.cc:578] Training of tree  81/300 (tree index:82) done accuracy:0.914286 logloss:0.900096
[INFO random_forest.cc:578] Training of tree  91/300 (tree index:89) done accuracy:0.918367 logloss:0.897564
[INFO random_forest.cc:578] Training of tree  101/300 (tree index:100) done accuracy:0.922449 logloss:0.893559
[INFO random_forest.cc:578] Training of tree  111/300 (tree index:113) done accuracy:0.926531 logloss:0.89369
[INFO random_forest.cc:578] Training of tree  121/300 (tree index:122) done accuracy:0.922449 logloss:0.893663
[INFO random_forest.cc:578] Training of tree  131/300 (tree index:130) done accuracy:0.930612 logloss:0.759198
[INFO random_forest.cc:578] Training of tree  141/300 (tree index:142) done accuracy:0.926531 logloss:0.759931
[INFO random_forest.cc:578] Training of tree  151/300 (tree index:152) done accuracy:0.922449 logloss:0.760019
[INFO random_forest.cc:578] Training of tree  161/300 (tree index:160) done accuracy:0.922449 logloss:0.758033
[INFO random_forest.cc:578] Training of tree  171/300 (tree index:170) done accuracy:0.922449 logloss:0.62668
[INFO random_forest.cc:578] Training of tree  181/300 (tree index:179) done accuracy:0.922449 logloss:0.627142
[INFO random_forest.cc:578] Training of tree  191/300 (tree index:189) done accuracy:0.922449 logloss:0.499365
[INFO random_forest.cc:578] Training of tree  201/300 (tree index:199) done accuracy:0.926531 logloss:0.49864
[INFO random_forest.cc:578] Training of tree  211/300 (tree index:212) done accuracy:0.922449 logloss:0.369094
[INFO random_forest.cc:578] Training of tree  223/300 (tree index:220) done accuracy:0.922449 logloss:0.370484
[INFO random_forest.cc:578] Training of tree  233/300 (tree index:231) done accuracy:0.918367 logloss:0.367249
[INFO random_forest.cc:578] Training of tree  244/300 (tree index:241) done accuracy:0.918367 logloss:0.365513
[INFO random_forest.cc:578] Training of tree  254/300 (tree index:252) done accuracy:0.918367 logloss:0.366475
[INFO random_forest.cc:578] Training of tree  264/300 (tree index:263) done accuracy:0.918367 logloss:0.367287
[INFO random_forest.cc:578] Training of tree  274/300 (tree index:273) done accuracy:0.918367 logloss:0.364562
[INFO random_forest.cc:578] Training of tree  284/300 (tree index:283) done accuracy:0.918367 logloss:0.366077
[INFO random_forest.cc:578] Training of tree  294/300 (tree index:293) done accuracy:0.926531 logloss:0.365515
[INFO random_forest.cc:578] Training of tree  300/300 (tree index:299) done accuracy:0.922449 logloss:0.366314
[INFO random_forest.cc:645] Final OOB metrics: accuracy:0.922449 logloss:0.366314
[INFO kernel.cc:856] Export model in log directory: /tmp/tmph2iz55u9
[INFO kernel.cc:864] Save model in resources
[INFO kernel.cc:988] Loading model from path
[INFO decision_forest.cc:590] Model loaded with 300 root(s), 6244 node(s), and 2 input feature(s).
[INFO kernel.cc:848] Use fast generic engine

El siguiente ejemplo vuelve a implementar la misma lógica con las columnas de funciones de TensorFlow.

def g_to_kg(x):
  return x / 1000

feature_columns = [
    tf.feature_column.numeric_column("body_mass_g", normalizer_fn=g_to_kg),
    tf.feature_column.numeric_column("bill_length_mm"),
]

preprocessing = tf.keras.layers.DenseFeatures(feature_columns)

model_5 = tfdf.keras.RandomForestModel(preprocessing=preprocessing)
model_5.compile(metrics=["accuracy"])
model_5.fit(x=train_ds)
4/4 [==============================] - 0s 2ms/step
[INFO kernel.cc:746] Start Yggdrasil model training
[INFO kernel.cc:747] Collect training examples
[INFO kernel.cc:392] Number of batches: 4
[INFO kernel.cc:393] Number of examples: 245
[INFO kernel.cc:769] Dataset:
Number of records: 245
Number of columns: 3

Number of columns by type:
    NUMERICAL: 2 (66.6667%)
    CATEGORICAL: 1 (33.3333%)

Columns:

NUMERICAL: 2 (66.6667%)
    0: "dense_features/concat:0.0" NUMERICAL num-nas:1 (0.408163%) mean:43.8016 min:32.1 max:59.6 sd:5.48753
    1: "dense_features/concat:0.1" NUMERICAL num-nas:1 (0.408163%) mean:4.18596 min:2.85 max:6.3 sd:0.811612

CATEGORICAL: 1 (33.3333%)
    2: "__LABEL" CATEGORICAL integerized vocab-size:4 no-ood-item

Terminology:
    nas: Number of non-available (i.e. missing) values.
    ood: Out of dictionary.
    manually-defined: Attribute which type is manually defined by the user i.e. the type was not automatically inferred.
    tokenized: The attribute value is obtained through tokenization.
    has-dict: The attribute is attached to a string dictionary e.g. a categorical attribute stored as a string.
    vocab-size: Number of unique values.

[INFO kernel.cc:772] Configure learner
[INFO kernel.cc:797] Training config:
learner: "RANDOM_FOREST"
features: "dense_features/concat:0\\.0"
features: "dense_features/concat:0\\.1"
label: "__LABEL"
task: CLASSIFICATION
[yggdrasil_decision_forests.model.random_forest.proto.random_forest_config] {
  num_trees: 300
  decision_tree {
    max_depth: 16
    min_examples: 5
    in_split_min_examples_check: true
    missing_value_policy: GLOBAL_IMPUTATION
    allow_na_conditions: false
    categorical_set_greedy_forward {
      sampling: 0.1
      max_num_items: -1
      min_item_frequency: 1
    }
    growing_strategy_local {
    }
    categorical {
      cart {
      }
    }
    num_candidate_attributes_ratio: -1
    axis_aligned_split {
    }
    internal {
      sorting_strategy: PRESORTED
    }
  }
  winner_take_all_inference: true
  compute_oob_performances: true
  compute_oob_variable_importances: false
  adapt_bootstrap_size_ratio_for_maximum_training_duration: false
}

[INFO kernel.cc:800] Deployment config:
num_threads: 6

[INFO kernel.cc:837] Train model
[INFO random_forest.cc:303] Training random forest on 245 example(s) and 2 feature(s).
[INFO random_forest.cc:578] Training of tree  1/300 (tree index:3) done accuracy:0.895349 logloss:3.77201
[INFO random_forest.cc:578] Training of tree  11/300 (tree index:10) done accuracy:0.905738 logloss:1.85772
[INFO random_forest.cc:578] Training of tree  21/300 (tree index:21) done accuracy:0.906122 logloss:1.71468
[INFO random_forest.cc:578] Training of tree  31/300 (tree index:30) done accuracy:0.914286 logloss:1.57234
[INFO random_forest.cc:578] Training of tree  41/300 (tree index:40) done accuracy:0.926531 logloss:1.30484
[INFO random_forest.cc:578] Training of tree  51/300 (tree index:51) done accuracy:0.918367 logloss:1.30661
[INFO random_forest.cc:578] Training of tree  61/300 (tree index:59) done accuracy:0.922449 logloss:1.16928
[INFO random_forest.cc:578] Training of tree  71/300 (tree index:72) done accuracy:0.926531 logloss:1.03292
[INFO random_forest.cc:578] Training of tree  81/300 (tree index:81) done accuracy:0.918367 logloss:0.899809
[INFO random_forest.cc:578] Training of tree  91/300 (tree index:89) done accuracy:0.918367 logloss:0.897564
[INFO random_forest.cc:578] Training of tree  102/300 (tree index:101) done accuracy:0.922449 logloss:0.893211
[INFO random_forest.cc:578] Training of tree  112/300 (tree index:111) done accuracy:0.926531 logloss:0.893353
[INFO random_forest.cc:578] Training of tree  122/300 (tree index:120) done accuracy:0.926531 logloss:0.893013
[INFO random_forest.cc:578] Training of tree  132/300 (tree index:131) done accuracy:0.930612 logloss:0.760904
[INFO random_forest.cc:578] Training of tree  142/300 (tree index:142) done accuracy:0.926531 logloss:0.758321
[INFO random_forest.cc:578] Training of tree  152/300 (tree index:151) done accuracy:0.926531 logloss:0.75838
[INFO random_forest.cc:578] Training of tree  162/300 (tree index:161) done accuracy:0.922449 logloss:0.757657
[INFO random_forest.cc:578] Training of tree  172/300 (tree index:171) done accuracy:0.922449 logloss:0.62637
[INFO random_forest.cc:578] Training of tree  182/300 (tree index:181) done accuracy:0.922449 logloss:0.627377
[INFO random_forest.cc:578] Training of tree  192/300 (tree index:193) done accuracy:0.922449 logloss:0.496634
[INFO random_forest.cc:578] Training of tree  202/300 (tree index:200) done accuracy:0.926531 logloss:0.498635
[INFO random_forest.cc:578] Training of tree  212/300 (tree index:211) done accuracy:0.922449 logloss:0.369699
[INFO random_forest.cc:578] Training of tree  222/300 (tree index:222) done accuracy:0.918367 logloss:0.369778
[INFO random_forest.cc:578] Training of tree  232/300 (tree index:231) done accuracy:0.918367 logloss:0.369071
[INFO random_forest.cc:578] Training of tree  242/300 (tree index:240) done accuracy:0.922449 logloss:0.365444
[INFO random_forest.cc:578] Training of tree  252/300 (tree index:250) done accuracy:0.918367 logloss:0.365968
[INFO random_forest.cc:578] Training of tree  262/300 (tree index:261) done accuracy:0.918367 logloss:0.366834
[INFO random_forest.cc:578] Training of tree  273/300 (tree index:270) done accuracy:0.918367 logloss:0.364275
[INFO random_forest.cc:578] Training of tree  283/300 (tree index:281) done accuracy:0.918367 logloss:0.365446
[INFO random_forest.cc:578] Training of tree  293/300 (tree index:292) done accuracy:0.926531 logloss:0.366609
[INFO random_forest.cc:578] Training of tree  300/300 (tree index:297) done accuracy:0.922449 logloss:0.366314
[INFO random_forest.cc:645] Final OOB metrics: accuracy:0.922449 logloss:0.366314
[INFO kernel.cc:856] Export model in log directory: /tmp/tmpgjvicb2p
[INFO kernel.cc:864] Save model in resources
[INFO kernel.cc:988] Loading model from path
[INFO decision_forest.cc:590] Model loaded with 300 root(s), 6244 node(s), and 2 input feature(s).
[INFO kernel.cc:848] Use fast generic engine
<keras.callbacks.History at 0x7fb2cc878f50>

Entrenamiento de un modelo de regresión

El ejemplo anterior entrena un modelo de clasificación (TF-DF no diferencia entre clasificación binaria y clasificación de clases múltiples). En el siguiente ejemplo, entrenar un modelo de regresión en el conjunto de datos del olmo . El objetivo de este conjunto de datos es predecir el número de anillos de concha de un abulón.

# Download the dataset.
!wget -q https://storage.googleapis.com/download.tensorflow.org/data/abalone_raw.csv -O /tmp/abalone.csv

dataset_df = pd.read_csv("/tmp/abalone.csv")
print(dataset_df.head(3))
Type  LongestShell  Diameter  Height  WholeWeight  ShuckedWeight  \
0    M         0.455     0.365   0.095       0.5140         0.2245   
1    M         0.350     0.265   0.090       0.2255         0.0995   
2    F         0.530     0.420   0.135       0.6770         0.2565   

   VisceraWeight  ShellWeight  Rings  
0         0.1010         0.15     15  
1         0.0485         0.07      7  
2         0.1415         0.21      9
# Split the dataset into a training and testing dataset.
train_ds_pd, test_ds_pd = split_dataset(dataset_df)
print("{} examples in training, {} examples for testing.".format(
    len(train_ds_pd), len(test_ds_pd)))

# Name of the label column.
label = "Rings"

train_ds = tfdf.keras.pd_dataframe_to_tf_dataset(train_ds_pd, label=label, task=tfdf.keras.Task.REGRESSION)
test_ds = tfdf.keras.pd_dataframe_to_tf_dataset(train_ds_pd, label=label, task=tfdf.keras.Task.REGRESSION)
2940 examples in training, 1237 examples for testing.
/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow_decision_forests/keras/core.py:1224: FutureWarning: In a future version of pandas all arguments of DataFrame.drop except for the argument 'labels' will be keyword-only
  (dict(dataframe.drop(label, 1)), dataframe[label].values))
%set_cell_height 300

# Configure the model.
model_7 = tfdf.keras.RandomForestModel(task = tfdf.keras.Task.REGRESSION)

# Optional.
model_7.compile(metrics=["mse"])

# Train the model.
with sys_pipes():
  model_7.fit(x=train_ds)
<IPython.core.display.Javascript object>
46/46 [==============================] - 0s 2ms/step
[INFO kernel.cc:746] Start Yggdrasil model training
[INFO kernel.cc:747] Collect training examples
[INFO kernel.cc:392] Number of batches: 46
[INFO kernel.cc:393] Number of examples: 2940
[INFO kernel.cc:769] Dataset:
Number of records: 2940
Number of columns: 9

Number of columns by type:
    NUMERICAL: 8 (88.8889%)
    CATEGORICAL: 1 (11.1111%)

Columns:

NUMERICAL: 8 (88.8889%)
    0: "Diameter" NUMERICAL mean:0.406861 min:0.09 max:0.65 sd:0.0996003
    1: "Height" NUMERICAL mean:0.139379 min:0 max:1.13 sd:0.0432488
    2: "LongestShell" NUMERICAL mean:0.522656 min:0.11 max:0.815 sd:0.120628
    3: "ShellWeight" NUMERICAL mean:0.237927 min:0.003 max:1.005 sd:0.140322
    4: "ShuckedWeight" NUMERICAL mean:0.356906 min:0.0025 max:1.351 sd:0.221252
    6: "VisceraWeight" NUMERICAL mean:0.179291 min:0.0005 max:0.6415 sd:0.109435
    7: "WholeWeight" NUMERICAL mean:0.824617 min:0.008 max:2.8255 sd:0.490996
    8: "__LABEL" NUMERICAL mean:9.89286 min:3 max:29 sd:3.22634

CATEGORICAL: 1 (11.1111%)
    5: "Type" CATEGORICAL has-dict vocab-size:4 zero-ood-items most-frequent:"M" 1064 (36.1905%)

Terminology:
    nas: Number of non-available (i.e. missing) values.
    ood: Out of dictionary.
    manually-defined: Attribute which type is manually defined by the user i.e. the type was not automatically inferred.
    tokenized: The attribute value is obtained through tokenization.
    has-dict: The attribute is attached to a string dictionary e.g. a categorical attribute stored as a string.
    vocab-size: Number of unique values.

[INFO kernel.cc:772] Configure learner
[INFO kernel.cc:797] Training config:
learner: "RANDOM_FOREST"
features: "Diameter"
features: "Height"
features: "LongestShell"
features: "ShellWeight"
features: "ShuckedWeight"
features: "Type"
features: "VisceraWeight"
features: "WholeWeight"
label: "__LABEL"
task: REGRESSION
[yggdrasil_decision_forests.model.random_forest.proto.random_forest_config] {
  num_trees: 300
  decision_tree {
    max_depth: 16
    min_examples: 5
    in_split_min_examples_check: true
    missing_value_policy: GLOBAL_IMPUTATION
    allow_na_conditions: false
    categorical_set_greedy_forward {
      sampling: 0.1
      max_num_items: -1
      min_item_frequency: 1
    }
    growing_strategy_local {
    }
    categorical {
      cart {
      }
    }
    num_candidate_attributes_ratio: -1
    axis_aligned_split {
    }
    internal {
      sorting_strategy: PRESORTED
    }
  }
  winner_take_all_inference: true
  compute_oob_performances: true
  compute_oob_variable_importances: false
  adapt_bootstrap_size_ratio_for_maximum_training_duration: false
}

[INFO kernel.cc:800] Deployment config:
num_threads: 6

[INFO kernel.cc:837] Train model
[INFO random_forest.cc:303] Training random forest on 2940 example(s) and 8 feature(s).
[INFO random_forest.cc:578] Training of tree  1/300 (tree index:1) done rmse:2.64473
[INFO random_forest.cc:578] Training of tree  11/300 (tree index:11) done rmse:2.31793
[INFO random_forest.cc:578] Training of tree  21/300 (tree index:18) done rmse:2.25077
[INFO random_forest.cc:578] Training of tree  31/300 (tree index:29) done rmse:2.22234
[INFO random_forest.cc:578] Training of tree  41/300 (tree index:43) done rmse:2.2023
[INFO random_forest.cc:578] Training of tree  51/300 (tree index:52) done rmse:2.18601
[INFO random_forest.cc:578] Training of tree  61/300 (tree index:63) done rmse:2.17663
[INFO random_forest.cc:578] Training of tree  71/300 (tree index:69) done rmse:2.17069
[INFO random_forest.cc:578] Training of tree  81/300 (tree index:84) done rmse:2.16817
[INFO random_forest.cc:578] Training of tree  91/300 (tree index:92) done rmse:2.16932
[INFO random_forest.cc:578] Training of tree  101/300 (tree index:98) done rmse:2.16462
[INFO random_forest.cc:578] Training of tree  111/300 (tree index:108) done rmse:2.16608
[INFO random_forest.cc:578] Training of tree  121/300 (tree index:120) done rmse:2.16779
[INFO random_forest.cc:578] Training of tree  131/300 (tree index:128) done rmse:2.16945
[INFO random_forest.cc:578] Training of tree  141/300 (tree index:138) done rmse:2.16776
[INFO random_forest.cc:578] Training of tree  151/300 (tree index:148) done rmse:2.16542
[INFO random_forest.cc:578] Training of tree  161/300 (tree index:160) done rmse:2.16544
[INFO random_forest.cc:578] Training of tree  171/300 (tree index:171) done rmse:2.16462
[INFO random_forest.cc:578] Training of tree  181/300 (tree index:182) done rmse:2.16335
[INFO random_forest.cc:578] Training of tree  191/300 (tree index:191) done rmse:2.16359
[INFO random_forest.cc:578] Training of tree  201/300 (tree index:199) done rmse:2.1628
[INFO random_forest.cc:578] Training of tree  211/300 (tree index:209) done rmse:2.16344
[INFO random_forest.cc:578] Training of tree  221/300 (tree index:222) done rmse:2.16181
[INFO random_forest.cc:578] Training of tree  231/300 (tree index:231) done rmse:2.16298
[INFO random_forest.cc:578] Training of tree  241/300 (tree index:238) done rmse:2.16298
[INFO random_forest.cc:578] Training of tree  251/300 (tree index:248) done rmse:2.16248
[INFO random_forest.cc:578] Training of tree  261/300 (tree index:260) done rmse:2.16154
[INFO random_forest.cc:578] Training of tree  271/300 (tree index:272) done rmse:2.16215
[INFO random_forest.cc:578] Training of tree  281/300 (tree index:280) done rmse:2.16294
[INFO random_forest.cc:578] Training of tree  291/300 (tree index:289) done rmse:2.16169
[INFO random_forest.cc:578] Training of tree  300/300 (tree index:299) done rmse:2.16092
[INFO random_forest.cc:645] Final OOB metrics: rmse:2.16092
[INFO kernel.cc:856] Export model in log directory: /tmp/tmpq2juxn_c
[INFO kernel.cc:864] Save model in resources
[INFO kernel.cc:988] Loading model from path
[INFO decision_forest.cc:590] Model loaded with 300 root(s), 263226 node(s), and 8 input feature(s).
[INFO abstract_model.cc:993] Engine "RandomForestOptPred" built
[INFO kernel.cc:848] Use fast generic engine
# Evaluate the model on the test dataset.
evaluation = model_7.evaluate(test_ds, return_dict=True)

print(evaluation)
print()
print(f"MSE: {evaluation['mse']}")
print(f"RMSE: {math.sqrt(evaluation['mse'])}")
WARNING:tensorflow:5 out of the last 9 calls to <function CoreModel.make_test_function.<locals>.test_function_trained at 0x7faf06cae8c0> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has experimental_relax_shapes=True option that relaxes argument shapes that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.
WARNING:tensorflow:5 out of the last 9 calls to <function CoreModel.make_test_function.<locals>.test_function_trained at 0x7faf06cae8c0> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has experimental_relax_shapes=True option that relaxes argument shapes that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.
46/46 [==============================] - 0s 4ms/step - loss: 0.0000e+00 - mse: 1.9425
{'loss': 0.0, 'mse': 1.942549228668213}

MSE: 1.942549228668213
RMSE: 1.3937536470510894

Entrenando un modelo de ranking

Finaly, después de haber formado una clasificación y un modelos de regresión, entrenar a un ranking de modelo.

El objetivo de una clasificación es para artículos de la orden de importancia. El "valor" de la relevancia no importa directamente. Clasificación de un conjunto de documentos con respecto a una consulta del usuario es un ejemplo de ordenamiento de problemas: Sólo es importante para conseguir el orden correcto, cuando los documentos superiores son más importantes.

TF-DF espera que los conjuntos de datos de clasificación se presenten en un formato "plano". Un conjunto de datos de documento + consulta podría verse así:

consulta document_id feature_1 feature_2 relevancia / etiqueta
gato 1 0,1 azul 4
gato 2 0,5 verde 1
gato 3 0,2 rojo 2
perro 4 N / A rojo 0
perro 5 0,2 rojo 1
perro 6 0,6 verde 1

La relevancia / etiqueta es un valor numérico de coma flotante entre 0 y 5 (en general, entre 0 y 4) donde 0 significa "completamente no relacionadas", 4 significa "muy relevante" y 5 significa "lo mismo que la consulta".

Curiosamente, los bosques de decisión suelen ser buenos clasificadores, y muchos modelos de clasificación de última generación son bosques de decisión.

En este ejemplo, usar una muestra de la LETOR3 conjunto de datos. Más precisamente, queremos descargar el OHSUMED.zip desde el repositorio LETOR3 . Este conjunto de datos se almacena en formato libsvm, por lo que tendremos que convertirlo a csv.

%set_cell_height 200

archive_path = tf.keras.utils.get_file("letor.zip",
  "https://download.microsoft.com/download/E/7/E/E7EABEF1-4C7B-4E31-ACE5-73927950ED5E/Letor.zip",
  extract=True)

# Path to the train and test dataset using libsvm format.
raw_dataset_path = os.path.join(os.path.dirname(archive_path),"OHSUMED/Data/All/OHSUMED.txt")
<IPython.core.display.Javascript object>
Downloading data from https://download.microsoft.com/download/E/7/E/E7EABEF1-4C7B-4E31-ACE5-73927950ED5E/Letor.zip
61825024/61824018 [==============================] - 7s 0us/step
61833216/61824018 [==============================] - 7s 0us/step

El conjunto de datos se almacena como un archivo .txt en un formato específico, así que primero conviértalo en un archivo csv.

def convert_libsvm_to_csv(src_path, dst_path):
  """Converts a libsvm ranking dataset into a flat csv file.

  Note: This code is specific to the LETOR3 dataset.
  """
  dst_handle = open(dst_path, "w")
  first_line = True
  for src_line in open(src_path,"r"):
    # Note: The last 3 items are comments.
    items = src_line.split(" ")[:-3]
    relevance = items[0]
    group = items[1].split(":")[1]
    features = [ item.split(":") for item in items[2:]]

    if first_line:
      # Csv header
      dst_handle.write("relevance,group," + ",".join(["f_" + feature[0] for feature in features]) + "\n")
      first_line = False
    dst_handle.write(relevance + ",g_" + group + "," + (",".join([feature[1] for feature in features])) + "\n")
  dst_handle.close()

# Convert the dataset.
csv_dataset_path="/tmp/ohsumed.csv"
convert_libsvm_to_csv(raw_dataset_path, csv_dataset_path)

# Load a dataset into a Pandas Dataframe.
dataset_df = pd.read_csv(csv_dataset_path)

# Display the first 3 examples.
dataset_df.head(3)
train_ds_pd, test_ds_pd = split_dataset(dataset_df)
print("{} examples in training, {} examples for testing.".format(
    len(train_ds_pd), len(test_ds_pd)))

# Display the first 3 examples of the training dataset.
train_ds_pd.head(3)
11259 examples in training, 4881 examples for testing.

En este conjunto de datos, la relevance define el rango realidad sobre el terreno entre las filas del mismo group .

# Name of the relevance and grouping columns.
relevance = "relevance"

ranking_train_ds = tfdf.keras.pd_dataframe_to_tf_dataset(train_ds_pd, label=relevance, task=tfdf.keras.Task.RANKING)
ranking_test_ds = tfdf.keras.pd_dataframe_to_tf_dataset(train_ds_pd, label=relevance, task=tfdf.keras.Task.RANKING)
/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow_decision_forests/keras/core.py:1224: FutureWarning: In a future version of pandas all arguments of DataFrame.drop except for the argument 'labels' will be keyword-only
  (dict(dataframe.drop(label, 1)), dataframe[label].values))
%set_cell_height 400

model_8 = tfdf.keras.GradientBoostedTreesModel(
    task=tfdf.keras.Task.RANKING,
    ranking_group="group",
    num_trees=50)

with sys_pipes():
  model_8.fit(x=ranking_train_ds)
<IPython.core.display.Javascript object>
176/176 [==============================] - 1s 4ms/step
[INFO kernel.cc:746] Start Yggdrasil model training
[INFO kernel.cc:747] Collect training examples
[INFO kernel.cc:392] Number of batches: 176
[INFO kernel.cc:393] Number of examples: 11259
[INFO kernel.cc:769] Dataset:
Number of records: 11259
Number of columns: 27

Number of columns by type:
    NUMERICAL: 26 (96.2963%)
    HASH: 1 (3.7037%)

Columns:

NUMERICAL: 26 (96.2963%)
    1: "f_1" NUMERICAL mean:1.17213 min:0 max:9 sd:1.121
    2: "f_10" NUMERICAL mean:3.96545 min:0 max:21.0369 sd:3.73706
    3: "f_11" NUMERICAL mean:4.5798 min:0 max:46 sd:4.75749
    4: "f_12" NUMERICAL mean:2.02252 min:0 max:9.75731 sd:1.64969
    5: "f_13" NUMERICAL mean:0.0483011 min:0 max:0.384615 sd:0.0478282
    6: "f_14" NUMERICAL mean:0.0472603 min:0 max:0.361682 sd:0.0463024
    7: "f_15" NUMERICAL mean:21.7652 min:7.51456 max:40.0616 sd:6.98535
    8: "f_16" NUMERICAL mean:6.91992 min:2.01684 max:13.5772 sd:2.22126
    9: "f_17" NUMERICAL mean:19.7844 min:9.0472 max:40.1808 sd:6.56356
    10: "f_18" NUMERICAL mean:0.200189 min:0 max:1.51088 sd:0.189003
    11: "f_19" NUMERICAL mean:20.6679 min:0 max:178.097 sd:20.9729
    12: "f_2" NUMERICAL mean:0.798267 min:0 max:4.56435 sd:0.749238
    13: "f_20" NUMERICAL mean:1.82174 min:0 max:13.4423 sd:1.71117
    14: "f_21" NUMERICAL mean:12.2433 min:3.18098 max:43.3771 sd:6.90342
    15: "f_22" NUMERICAL mean:2.31003 min:1.15719 max:3.76993 sd:0.673792
    16: "f_23" NUMERICAL mean:-6.11841 min:-9.49097 max:-1.85651 sd:2.20745
    17: "f_24" NUMERICAL mean:-5.80871 min:-9.22971 max:-1.02685 sd:1.99262
    18: "f_25" NUMERICAL mean:-5.97799 min:-9.60073 max:-1.02685 sd:2.20452
    19: "f_3" NUMERICAL mean:0.158338 min:0 max:1 sd:0.163356
    20: "f_4" NUMERICAL mean:0.146345 min:0 max:0.892574 sd:0.147729
    21: "f_5" NUMERICAL mean:27.2029 min:6.3511 max:55.3932 sd:9.34721
    22: "f_6" NUMERICAL mean:8.15474 min:2.03154 max:16.8986 sd:2.73277
    23: "f_7" NUMERICAL mean:27.6877 min:14.2035 max:55.1926 sd:9.40568
    24: "f_8" NUMERICAL mean:0.622094 min:0 max:3.80599 sd:0.600034
    25: "f_9" NUMERICAL mean:6.45924 min:0 max:47.7046 sd:6.07158
    26: "__LABEL" NUMERICAL mean:0.441513 min:0 max:2 sd:0.72746

HASH: 1 (3.7037%)
    0: "__RANK_GROUP" HASH

Terminology:
    nas: Number of non-available (i.e. missing) values.
    ood: Out of dictionary.
    manually-defined: Attribute which type is manually defined by the user i.e. the type was not automatically inferred.
    tokenized: The attribute value is obtained through tokenization.
    has-dict: The attribute is attached to a string dictionary e.g. a categorical attribute stored as a string.
    vocab-size: Number of unique values.

[INFO kernel.cc:772] Configure learner
[WARNING gradient_boosted_trees.cc:1692] Subsample hyperparameter given but sampling method does not match.
[WARNING gradient_boosted_trees.cc:1705] GOSS alpha hyperparameter given but GOSS is disabled.
[WARNING gradient_boosted_trees.cc:1714] GOSS beta hyperparameter given but GOSS is disabled.
[WARNING gradient_boosted_trees.cc:1726] SelGB ratio hyperparameter given but SelGB is disabled.
[INFO kernel.cc:797] Training config:
learner: "GRADIENT_BOOSTED_TREES"
features: "__RANK_GROUP"
features: "f_1"
features: "f_10"
features: "f_11"
features: "f_12"
features: "f_13"
features: "f_14"
features: "f_15"
features: "f_16"
features: "f_17"
features: "f_18"
features: "f_19"
features: "f_2"
features: "f_20"
features: "f_21"
features: "f_22"
features: "f_23"
features: "f_24"
features: "f_25"
features: "f_3"
features: "f_4"
features: "f_5"
features: "f_6"
features: "f_7"
features: "f_8"
features: "f_9"
label: "__LABEL"
task: RANKING
ranking_group: "__RANK_GROUP"
[yggdrasil_decision_forests.model.gradient_boosted_trees.proto.gradient_boosted_trees_config] {
  num_trees: 50
  decision_tree {
    max_depth: 6
    min_examples: 5
    in_split_min_examples_check: true
    missing_value_policy: GLOBAL_IMPUTATION
    allow_na_conditions: false
    categorical_set_greedy_forward {
      sampling: 0.1
      max_num_items: -1
      min_item_frequency: 1
    }
    growing_strategy_local {
    }
    categorical {
      cart {
      }
    }
    num_candidate_attributes_ratio: -1
    axis_aligned_split {
    }
    internal {
      sorting_strategy: PRESORTED
    }
  }
  shrinkage: 0.1
  validation_set_ratio: 0.1
  early_stopping: VALIDATION_LOSS_INCREASE
  early_stopping_num_trees_look_ahead: 30
  l2_regularization: 0
  lambda_loss: 1
  mart {
  }
  adapt_subsample_for_maximum_training_duration: false
  l1_regularization: 0
  use_hessian_gain: false
  l2_regularization_categorical: 1
  apply_link_function: true
}

[INFO kernel.cc:800] Deployment config:
num_threads: 6

[INFO kernel.cc:837] Train model
[INFO abstract_learner.cc:125] The ranking_group "__RANK_GROUP" was removed from the input feature set.
[INFO gradient_boosted_trees.cc:503] Default loss set to LAMBDA_MART_NDCG5
[INFO gradient_boosted_trees.cc:1079] Training gradient boosted tree on 11259 example(s) and 25 feature(s).
[INFO gradient_boosted_trees.cc:2306] Split training/validation dataset by "__RANK_GROUP". 106 groups found in 11259 examples i.e. 106.217 examples/groups.
[INFO gradient_boosted_trees_loss.cc:1231] Found 97 groups in 10225 examples.
[INFO gradient_boosted_trees_loss.cc:1231] Found 9 groups in 1034 examples.
[INFO gradient_boosted_trees.cc:1493]     num-trees:1 train-loss:-0.358436 train-NDCG@5:0.358436 valid-loss:-0.505667 valid-NDCG@5:0.505667
[INFO gradient_boosted_trees.cc:1493]     num-trees:50 train-loss:-0.600826 train-NDCG@5:0.600826 valid-loss:-0.563146 valid-NDCG@5:0.563146
[INFO gradient_boosted_trees.cc:336] Truncates the model to 35 tree(s) i.e. 35  iteration(s).
[INFO gradient_boosted_trees.cc:370] Final model num-trees:35 valid-loss:-0.591633 valid-NDCG@5:0.591633
[INFO kernel.cc:856] Export model in log directory: /tmp/tmptnyhr2k4
[INFO kernel.cc:864] Save model in resources
[INFO kernel.cc:988] Loading model from path
[INFO abstract_model.cc:993] Engine "GradientBoostedTreesQuickScorerExtended" built
[INFO kernel.cc:848] Use fast generic engine

En este punto, keras no propone ninguna métrica de clasificación. En cambio, el entrenamiento y la validación (un GBDT usa un conjunto de datos de validación) se muestran en los registros de entrenamiento. En este caso la pérdida es LAMBDA_MART_NDCG5 , y el final (es decir, al final de la formación) NDCG (normalizó ganancia acumulada descontado) es 0.510136 (véase la línea Final model valid-loss: -0.510136 ).

Tenga en cuenta que el NDCG es un valor entre 0 y 1. Cuanto mayor sea el NDCG, mejor será el modelo. Por esta razón, la pérdida será -NDCG.

Como antes, el modelo se puede analizar:

%set_cell_height 400

model_8.summary()
<IPython.core.display.Javascript object>
Model: "gradient_boosted_trees_model_5"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
Total params: 1
Trainable params: 0
Non-trainable params: 1
_________________________________________________________________
Type: "GRADIENT_BOOSTED_TREES"
Task: RANKING
Label: "__LABEL"
Rank group: "__RANK_GROUP"

Input Features (25):
    f_1
    f_10
    f_11
    f_12
    f_13
    f_14
    f_15
    f_16
    f_17
    f_18
    f_19
    f_2
    f_20
    f_21
    f_22
    f_23
    f_24
    f_25
    f_3
    f_4
    f_5
    f_6
    f_7
    f_8
    f_9

No weights

Variable Importance: MEAN_MIN_DEPTH:

    1. "__RANK_GROUP"  4.530347 ################
    2.      "__LABEL"  4.530347 ################
    3.         "f_13"  4.496583 ###############
    4.          "f_2"  4.468529 ###############
    5.         "f_14"  4.430297 ##############
    6.         "f_11"  4.411654 ##############
    7.         "f_19"  4.403542 ##############
    8.         "f_18"  4.395156 ##############
    9.         "f_17"  4.378767 ##############
   10.          "f_1"  4.364304 ##############
   11.          "f_7"  4.341317 ##############
   12.         "f_16"  4.320280 #############
   13.          "f_6"  4.306558 #############
   14.          "f_5"  4.305171 #############
   15.         "f_20"  4.264664 #############
   16.         "f_15"  4.111839 ###########
   17.         "f_24"  3.998693 ##########
   18.         "f_25"  3.973909 ##########
   19.         "f_12"  3.959334 ##########
   20.         "f_22"  3.889284 #########
   21.          "f_3"  3.865117 #########
   22.         "f_21"  3.708907 #######
   23.         "f_10"  3.670751 #######
   24.          "f_9"  3.580533 ######
   25.         "f_23"  3.343641 ###
   26.          "f_4"  2.972983 
   27.          "f_8"  2.969561 

Variable Importance: NUM_AS_ROOT:

    1.  "f_4"  9.000000 ################
    2.  "f_8"  7.000000 ############
    3. "f_23"  6.000000 ##########
    4. "f_10"  4.000000 ######
    5. "f_22"  2.000000 ##
    6. "f_25"  2.000000 ##
    7.  "f_9"  2.000000 ##
    8.  "f_1"  1.000000 
    9. "f_24"  1.000000 
   10.  "f_3"  1.000000 

Variable Importance: NUM_NODES:

    1.  "f_8" 45.000000 ################
    2.  "f_9" 44.000000 ###############
    3. "f_21" 43.000000 ###############
    4. "f_12" 42.000000 ##############
    5.  "f_4" 39.000000 #############
    6. "f_23" 35.000000 ############
    7. "f_10" 34.000000 ###########
    8. "f_22" 34.000000 ###########
    9. "f_25" 28.000000 #########
   10. "f_24" 26.000000 ########
   11.  "f_5" 23.000000 #######
   12. "f_20" 22.000000 #######
   13.  "f_3" 22.000000 #######
   14. "f_15" 21.000000 ######
   15.  "f_6" 21.000000 ######
   16. "f_17" 20.000000 ######
   17. "f_16" 19.000000 #####
   18. "f_18" 17.000000 #####
   19. "f_19" 16.000000 ####
   20. "f_14" 15.000000 ####
   21.  "f_7" 14.000000 ###
   22. "f_11" 12.000000 ###
   23. "f_13"  6.000000 
   24.  "f_2"  6.000000 
   25.  "f_1"  4.000000 

Variable Importance: SUM_SCORE:

    1. "f_23" 4302.791448 ################
    2.  "f_8" 3518.019818 #############
    3.  "f_3" 2220.676527 ########
    4.  "f_9" 2032.929265 #######
    5. "f_21" 1538.908728 #####
    6.  "f_4" 1377.808135 ####
    7. "f_12" 1330.282088 ####
    8. "f_17" 1316.601042 ####
    9. "f_10" 1281.559223 ####
   10. "f_15" 1269.617477 ####
   11. "f_16" 1250.207300 ####
   12.  "f_7" 1169.581349 ####
   13. "f_22" 1090.183782 ###
   14. "f_24" 1072.937149 ###
   15.  "f_5" 971.754396 ###
   16.  "f_6" 657.819007 ##
   17. "f_20" 624.759689 ##
   18. "f_18" 608.597656 ##
   19. "f_25" 569.072304 #
   20. "f_19" 499.554447 #
   21. "f_11" 386.626719 #
   22. "f_14" 359.273525 #
   23. "f_13" 87.172405 
   24.  "f_1" 79.791417 
   25.  "f_2" 52.463786 



Loss: LAMBDA_MART_NDCG5
Validation loss value: -0.591633
Number of trees per iteration: 1
Node format: NOT_SET
Number of trees: 35
Total number of nodes: 1251

Number of nodes by tree:
Count: 35 Average: 35.7429 StdDev: 6.62611
Min: 23 Max: 47 Ignored: 0
----------------------------------------------
[ 23, 24) 2   5.71%   5.71% #####
[ 24, 25) 0   0.00%   5.71%
[ 25, 26) 1   2.86%   8.57% ###
[ 26, 28) 2   5.71%  14.29% #####
[ 28, 29) 0   0.00%  14.29%
[ 29, 30) 2   5.71%  20.00% #####
[ 30, 31) 0   0.00%  20.00%
[ 31, 33) 4  11.43%  31.43% ##########
[ 33, 34) 3   8.57%  40.00% ########
[ 34, 35) 0   0.00%  40.00%
[ 35, 36) 4  11.43%  51.43% ##########
[ 36, 38) 4  11.43%  62.86% ##########
[ 38, 39) 0   0.00%  62.86%
[ 39, 40) 2   5.71%  68.57% #####
[ 40, 41) 0   0.00%  68.57%
[ 41, 43) 3   8.57%  77.14% ########
[ 43, 44) 3   8.57%  85.71% ########
[ 44, 45) 0   0.00%  85.71%
[ 45, 46) 4  11.43%  97.14% ##########
[ 46, 47] 1   2.86% 100.00% ###

Depth by leafs:
Count: 643 Average: 4.5521 StdDev: 0.833778
Min: 1 Max: 5 Ignored: 0
----------------------------------------------
[ 1, 2)   5   0.78%   0.78%
[ 2, 3)  20   3.11%   3.89%
[ 3, 4)  54   8.40%  12.29% #
[ 4, 5) 100  15.55%  27.84% ##
[ 5, 5] 464  72.16% 100.00% ##########

Number of training obs by leaf:
Count: 643 Average: 556.571 StdDev: 1918.54
Min: 5 Max: 10064 Ignored: 0
----------------------------------------------
[     5,   508) 572  88.96%  88.96% ##########
[   508,  1011)  14   2.18%  91.14%
[  1011,  1514)   9   1.40%  92.53%
[  1514,  2017)   8   1.24%  93.78%
[  2017,  2520)   2   0.31%  94.09%
[  2520,  3023)   0   0.00%  94.09%
[  3023,  3526)   1   0.16%  94.25%
[  3526,  4029)   3   0.47%  94.71%
[  4029,  4532)   1   0.16%  94.87%
[  4532,  5035)   1   0.16%  95.02%
[  5035,  5538)   2   0.31%  95.33%
[  5538,  6041)   0   0.00%  95.33%
[  6041,  6544)   1   0.16%  95.49%
[  6544,  7047)   1   0.16%  95.65%
[  7047,  7550)   3   0.47%  96.11%
[  7550,  8053)   1   0.16%  96.27%
[  8053,  8556)   4   0.62%  96.89%
[  8556,  9059)   6   0.93%  97.82%
[  9059,  9562)   5   0.78%  98.60%
[  9562, 10064]   9   1.40% 100.00%

Attribute in nodes:
    45 : f_8 [NUMERICAL]
    44 : f_9 [NUMERICAL]
    43 : f_21 [NUMERICAL]
    42 : f_12 [NUMERICAL]
    39 : f_4 [NUMERICAL]
    35 : f_23 [NUMERICAL]
    34 : f_22 [NUMERICAL]
    34 : f_10 [NUMERICAL]
    28 : f_25 [NUMERICAL]
    26 : f_24 [NUMERICAL]
    23 : f_5 [NUMERICAL]
    22 : f_3 [NUMERICAL]
    22 : f_20 [NUMERICAL]
    21 : f_6 [NUMERICAL]
    21 : f_15 [NUMERICAL]
    20 : f_17 [NUMERICAL]
    19 : f_16 [NUMERICAL]
    17 : f_18 [NUMERICAL]
    16 : f_19 [NUMERICAL]
    15 : f_14 [NUMERICAL]
    14 : f_7 [NUMERICAL]
    12 : f_11 [NUMERICAL]
    6 : f_2 [NUMERICAL]
    6 : f_13 [NUMERICAL]
    4 : f_1 [NUMERICAL]

Attribute in nodes with depth <= 0:
    9 : f_4 [NUMERICAL]
    7 : f_8 [NUMERICAL]
    6 : f_23 [NUMERICAL]
    4 : f_10 [NUMERICAL]
    2 : f_9 [NUMERICAL]
    2 : f_25 [NUMERICAL]
    2 : f_22 [NUMERICAL]
    1 : f_3 [NUMERICAL]
    1 : f_24 [NUMERICAL]
    1 : f_1 [NUMERICAL]

Attribute in nodes with depth <= 1:
    14 : f_23 [NUMERICAL]
    13 : f_4 [NUMERICAL]
    11 : f_8 [NUMERICAL]
    9 : f_9 [NUMERICAL]
    8 : f_21 [NUMERICAL]
    7 : f_3 [NUMERICAL]
    7 : f_12 [NUMERICAL]
    6 : f_10 [NUMERICAL]
    5 : f_15 [NUMERICAL]
    4 : f_24 [NUMERICAL]
    4 : f_22 [NUMERICAL]
    3 : f_25 [NUMERICAL]
    2 : f_7 [NUMERICAL]
    2 : f_5 [NUMERICAL]
    2 : f_20 [NUMERICAL]
    1 : f_6 [NUMERICAL]
    1 : f_18 [NUMERICAL]
    1 : f_1 [NUMERICAL]

Attribute in nodes with depth <= 2:
    19 : f_8 [NUMERICAL]
    19 : f_4 [NUMERICAL]
    18 : f_23 [NUMERICAL]
    18 : f_21 [NUMERICAL]
    16 : f_9 [NUMERICAL]
    13 : f_10 [NUMERICAL]
    12 : f_22 [NUMERICAL]
    12 : f_12 [NUMERICAL]
    11 : f_25 [NUMERICAL]
    10 : f_24 [NUMERICAL]
    9 : f_3 [NUMERICAL]
    8 : f_15 [NUMERICAL]
    5 : f_6 [NUMERICAL]
    5 : f_20 [NUMERICAL]
    4 : f_7 [NUMERICAL]
    4 : f_5 [NUMERICAL]
    4 : f_18 [NUMERICAL]
    4 : f_16 [NUMERICAL]
    4 : f_14 [NUMERICAL]
    4 : f_11 [NUMERICAL]
    3 : f_2 [NUMERICAL]
    3 : f_17 [NUMERICAL]
    2 : f_19 [NUMERICAL]
    2 : f_1 [NUMERICAL]
    1 : f_13 [NUMERICAL]

Attribute in nodes with depth <= 3:
    34 : f_21 [NUMERICAL]
    32 : f_9 [NUMERICAL]
    30 : f_8 [NUMERICAL]
    25 : f_10 [NUMERICAL]
    24 : f_4 [NUMERICAL]
    23 : f_12 [NUMERICAL]
    22 : f_23 [NUMERICAL]
    20 : f_22 [NUMERICAL]
    18 : f_25 [NUMERICAL]
    17 : f_3 [NUMERICAL]
    17 : f_24 [NUMERICAL]
    14 : f_15 [NUMERICAL]
    12 : f_18 [NUMERICAL]
    12 : f_17 [NUMERICAL]
    11 : f_7 [NUMERICAL]
    11 : f_6 [NUMERICAL]
    10 : f_16 [NUMERICAL]
    9 : f_5 [NUMERICAL]
    8 : f_20 [NUMERICAL]
    8 : f_11 [NUMERICAL]
    6 : f_19 [NUMERICAL]
    6 : f_14 [NUMERICAL]
    4 : f_2 [NUMERICAL]
    2 : f_1 [NUMERICAL]
    1 : f_13 [NUMERICAL]

Attribute in nodes with depth <= 5:
    45 : f_8 [NUMERICAL]
    44 : f_9 [NUMERICAL]
    43 : f_21 [NUMERICAL]
    42 : f_12 [NUMERICAL]
    39 : f_4 [NUMERICAL]
    35 : f_23 [NUMERICAL]
    34 : f_22 [NUMERICAL]
    34 : f_10 [NUMERICAL]
    28 : f_25 [NUMERICAL]
    26 : f_24 [NUMERICAL]
    23 : f_5 [NUMERICAL]
    22 : f_3 [NUMERICAL]
    22 : f_20 [NUMERICAL]
    21 : f_6 [NUMERICAL]
    21 : f_15 [NUMERICAL]
    20 : f_17 [NUMERICAL]
    19 : f_16 [NUMERICAL]
    17 : f_18 [NUMERICAL]
    16 : f_19 [NUMERICAL]
    15 : f_14 [NUMERICAL]
    14 : f_7 [NUMERICAL]
    12 : f_11 [NUMERICAL]
    6 : f_2 [NUMERICAL]
    6 : f_13 [NUMERICAL]
    4 : f_1 [NUMERICAL]

Condition type in nodes:
    608 : HigherCondition
Condition type in nodes with depth <= 0:
    35 : HigherCondition
Condition type in nodes with depth <= 1:
    100 : HigherCondition
Condition type in nodes with depth <= 2:
    210 : HigherCondition
Condition type in nodes with depth <= 3:
    376 : HigherCondition
Condition type in nodes with depth <= 5:
    608 : HigherCondition