Automated hyper-parameter tuning

Stay organized with collections Save and categorize content based on your preferences.

View on TensorFlow.org Run in Google Colab View on GitHub Download notebook

Welcome to the Automated hyper-parameter tuning tutorial. In this colab, you will learn how to improve your models using automated hyper-parameter tuning with TensorFlow Decision Forests.

Introduction

A learning algorithm trains a machine learning model on a training dataset. The parameters of a learning algorithm–called "hyper-parameters"–control how the model is trained and impact its quality. Therefore, finding the best hyper-parameters is an important stage of modeling.

Some hyper-parameters are simple to configure. For example, increasing the number of trees (num_trees) in a random forest increases the quality of the model until a plateau. Therefore, setting the largest value compatible with the serving constraints (more trees means a larger model) is a valid rule of thumb. However, other hyper-parameters have a more complex interaction with the model and cannot be chosen with such a simple rule. For example, increasing the maximum tree depth (max_depth) of a gradient boosted tree model can both increase or decrease the quality of the model. Furthermore, hyper-parameters can interact between each others, and the optimal value of a hyper-parameter cannot be found in isolation.

There are three main approaches to select the hyper-parameter values:

  1. The default approach: Learning algorithms come with default values. While not ideal in all cases, those values produce reasonable results in most situations. This approach is recommended as the first approach to use in any modeling. This page lists the default value of TF Decision Forests.

  2. The template hyper-parameter approach: In addition to the default values, TF Decision Forests also exposes the hyper-parameter templates. Those are benchmark-tuned hyper-parameter values with excellent performance but high training cost (e.g. hyperparameter_template="benchmark_rank1").

  3. The manual tuning approach: You can manually test different hyper-parameter values and select the one that performs best. The advanced users guide give some advice.

  4. The automated tuning approach: A tuning algorithm can be used to find automatically the best hyper-parameter values. This approach gives often the best results and does not require expertise. The main downside of this approach is the time it takes for large datasets.

In this colab, we shows the default and automated tuning approaches with the TensorFlow Decision Forests library.

Hyper-parameter tuning algorithms

Automated tuning algorithms work by generating and evaluating a large number of hyper-parameter values. Each of those iterations is called a "trial". The evaluation of a trial is expensive as it requires to train a new model each time. At the end of the tuning, the hyper-parameter with the best evaluation is used.

Tuning algorithm are configured as follow:

The search space

The search space is the list of hyper-parameters to optimize and the values they can take. For example, the maximum depth of a tree could be optimized for values in between 1 and 32. Exploring more hyper-parameters and more possible values often leads to better models but also takes more time. The hyper-parameters listed in the user manual are the most impactful ones to tune. The other hyper-parameters are listed in the documentation.

When the possible value of one hyper-parameter depends on the value of another hyper-parameter, the search space is said to be conditional.

The number of trials

The number of trials defines how many models will be trained and evaluated. Larger number of trials generally leads to better models, but takes more time.

The optimizer

The optimizer selects the next hyper-parameter to evaluate the past trial evaluations. The simplest and often reasonable optimizer is the one that selects the hyper-parameter at random.

The objective / trial score

The objective is the metric optimized by the tuner. Often, this metric is a measure of quality (e.g. accuracy, log loss) of the model evaluated on a validation dataset.

Train-valid-test

The validation dataset should be different from the training datasets: If the training and validation datasets are the same, the selected hyper-parameters will be irrelevant. The validation dataset should also be different from the testing dataset (also called holdout dataset): Because hyper-parameter tuning is a form of training, if the testing and validation datasets are the same, you are effectively training on the test dataset. In this case, you might overfit on your test dataset without a way to measure it.

Cross-validation

In the case of a small dataset, for example a dataset with less than 100k examples, hyper-parameter tuning can be coupled with cross-validation: Instead of being evaluated from a single training-test round, the objective/trial score is evaluated as the average of the metric over multiple cross-validation rounds.

Similarly as to the train-valid-and-test datasets, the cross-validation used to evaluate the objective/score during hyper-parameter tuning should be different from the cross-validation used to evaluate the quality of the model.

Out-of-bag evaluation

Some models, like Random Forests, can be evaluated on the training datasets using the "out-of-bag evaluation" method. While not as accurate as cross-validation, the "out-of-bag evaluation" is much faster than cross-validation and does not require a separate validation datasets.

In tensorflow decision forests

In TF-DF, the model "self" evaluation is always a fair way to evaluate a model. For example, an out-of-bag evaluation is used for Random Forest models while a validation dataset is used for Gradient Boosted models.

Hyper-parameter tuning with TF Decision Forests

TF-DF supports automatic hyper-parameter tuning with minimal configuration. In the next example, we will train and compare two models: One trained with default hyper-parameters, and one trained with hyper-parameter tuning.

Setup

# Install TensorFlow Dececision Forests
pip install tensorflow_decision_forests -U -qq

Install Wurlitzer. Wurlitzer is required to show the detailed training logs in colabs (with verbose=2).

pip install wurlitzer -U -qq

Import the necessary libraries.

import tensorflow_decision_forests as tfdf
import matplotlib.pyplot as plt
import pandas as pd
import tensorflow as tf
import numpy as np
2022-09-19 11:09:55.311417: E tensorflow/stream_executor/cuda/cuda_blas.cc:2981] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2022-09-19 11:09:56.012376: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvrtc.so.11.1: cannot open shared object file: No such file or directory
2022-09-19 11:09:56.012625: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvrtc.so.11.1: cannot open shared object file: No such file or directory
2022-09-19 11:09:56.012638: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.

The hidden code cell limits the output height in colab.

Define "set_cell_height".

Training a model without Automated hyper-parameter tuning

We will train a model on the Multiple Features Data Set multi-class classification dataset available on the UCI. Let's download the dataset.

!wget -q https://archive.ics.uci.edu/ml/machine-learning-databases/mfeat/mfeat-fou -O /tmp/mfeat-fou.csv
all_df = pd.read_csv("/tmp/mfeat-fou.csv", header=None, names=[f"coef_{i}" for i in range(76) ], sep="  ")
# The label values are not directly recorded in the dataset. Instead, the first 200 examples have class 0,
# the next 200 examples have class 1, and so on.
label="label"
all_df[label] = [i // 200 for i in range(2000)]
print(all_df)
coef_0    coef_1    coef_2    coef_3    coef_4    coef_5    coef_6  \
0     0.065882  0.197312  0.103826  0.270362  0.616078  0.035856  0.424572   
1     0.049142  0.175971  0.105515  0.227095  0.599280  0.041217  0.431078   
2     0.034172  0.227649  0.108766  0.127697  0.612494  0.056554  0.470639   
3     0.062336  0.217979  0.080243  0.289592  0.546316  0.045779  0.425545   
4     0.061970  0.198358  0.111239  0.253460  0.608455  0.023631  0.415246   
...        ...       ...       ...       ...       ...       ...       ...   
1995  0.171874  0.221525  0.426470  0.077220  0.264413  0.220274  0.452574   
1996  0.152025  0.347439  0.323534  0.087706  0.427077  0.313656  0.371549   
1997  0.335606  0.318426  0.257948  0.225381  0.417769  0.123067  0.333455   
1998  0.253910  0.171202  0.302622  0.181066  0.326383  0.167960  0.417409   
1999  0.271575  0.149037  0.232753  0.197720  0.236424  0.187395  0.467213   

        coef_7    coef_8    coef_9  ...   coef_67   coef_68   coef_69  \
0     0.089701  0.367773  0.037065  ...  0.113374  0.074343  0.281666   
1     0.096801  0.326739  0.059661  ...  0.069337  0.045386  0.254264   
2     0.041903  0.324267  0.044569  ...  0.218842  0.041087  0.360464   
3     0.022841  0.331454  0.119052  ...  0.085479  0.087658  0.151880   
4     0.091866  0.309310  0.049142  ...  0.106960  0.032283  0.248565   
...        ...       ...       ...  ...       ...       ...       ...   
1995  0.161113  0.356788  0.081889  ...  0.197440  0.197778  0.205866   
1996  0.262768  0.396422  0.124391  ...  0.161606  0.033621  0.173155   
1997  0.342044  0.398151  0.079550  ...  0.121659  0.316333  0.244846   
1998  0.399065  0.266153  0.097131  ...  0.168154  0.147167  0.268675   
1999  0.385117  0.191292  0.149162  ...  0.225634  0.061724  0.267944   

       coef_70   coef_71   coef_72   coef_73   coef_74   coef_75  label  
0     0.067795  0.344182  0.038963  0.394366  0.049971  0.344871      0  
1     0.045447  0.335659  0.021719  0.445277  0.083978  0.354092      0  
2     0.047154  0.377408  0.052099  0.445029  0.071234  0.261465      0  
3     0.075950  0.293462  0.022675  0.408291  0.063010  0.401376      0  
4     0.015674  0.386276  0.039481  0.434701  0.069218  0.405403      0  
...        ...       ...       ...       ...       ...       ...    ...  
1995  0.138887  0.160848  0.126190  0.247345  0.239117  0.136807      9  
1996  0.295227  0.259417  0.345691  0.068303  0.158352  0.282327      9  
1997  0.132893  0.042988  0.250740  0.141576  0.038776  0.114591      9  
1998  0.199125  0.128708  0.285220  0.230322  0.103664  0.130003      9  
1999  0.251182  0.170772  0.322067  0.275769  0.021074  0.085241      9  

[2000 rows x 77 columns]
/tmpfs/tmp/ipykernel_13574/163055519.py:2: ParserWarning: Falling back to the 'python' engine because the 'c' engine does not support regex separators (separators > 1 char and different from '\s+' are interpreted as regex); you can avoid this warning by specifying engine='python'.
  all_df = pd.read_csv("/tmp/mfeat-fou.csv", header=None, names=[f"coef_{i}" for i in range(76) ], sep="  ")

Split the dataset into a training and a testing dataset.

def split_dataset(dataset, test_ratio=0.30):
  """Splits a panda dataframe in two."""
  test_indices = np.random.rand(len(dataset)) < test_ratio
  return dataset[~test_indices], dataset[test_indices]

train_df, test_df = split_dataset(all_df)
print("{} examples in training, {} examples for testing.".format(len(train_df), len(test_df)))
label="label"
train_ds = tfdf.keras.pd_dataframe_to_tf_dataset(train_df, label=label)
test_ds = tfdf.keras.pd_dataframe_to_tf_dataset(test_df, label=label)
/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow_decision_forests/keras/core_inference.py:873: FutureWarning: In a future version of pandas all arguments of DataFrame.drop except for the argument 'labels' will be keyword-only.
  features_dataframe = dataframe.drop(label, 1)
1363 examples in training, 637 examples for testing.
/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow_decision_forests/keras/core_inference.py:873: FutureWarning: In a future version of pandas all arguments of DataFrame.drop except for the argument 'labels' will be keyword-only.
  features_dataframe = dataframe.drop(label, 1)

First, we train and evaluate the quality of a Gradient Boosted Trees model trained with the default hyper-parameters.

%%time
# Train a model with default hyper-parameters
model = tfdf.keras.GradientBoostedTreesModel()
model.fit(train_ds)
Warning: The `num_threads` constructor argument is not set and the number of CPU is os.cpu_count()=32 > 32. Setting num_threads to 32. Set num_threads manually to use more than 32 cpus.
WARNING:absl:The `num_threads` constructor argument is not set and the number of CPU is os.cpu_count()=32 > 32. Setting num_threads to 32. Set num_threads manually to use more than 32 cpus.
Use /tmpfs/tmp/tmpbfb82qvm as temporary training directory
Reading training dataset...
Training dataset read in 0:00:03.620528. Found 1363 examples.
Training model...
Model trained in 0:00:02.701696
Compiling model...
[INFO kernel.cc:1176] Loading model from path /tmpfs/tmp/tmpbfb82qvm/model/ with prefix 0eff9f84e8ca4625
[INFO abstract_model.cc:1248] Engine "GradientBoostedTreesGeneric" built
[INFO kernel.cc:1022] Use fast generic engine
WARNING:tensorflow:AutoGraph could not transform <function simple_ml_inference_op_with_handle at 0x7ff4d0786940> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: could not get source code
To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convert
WARNING:tensorflow:AutoGraph could not transform <function simple_ml_inference_op_with_handle at 0x7ff4d0786940> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: could not get source code
To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convert
WARNING: AutoGraph could not transform <function simple_ml_inference_op_with_handle at 0x7ff4d0786940> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: could not get source code
To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convert
Model compiled.
CPU times: user 24.3 s, sys: 6.28 s, total: 30.6 s
Wall time: 7.88 s
<keras.callbacks.History at 0x7ff57de93a60>
# Evaluate the model
model.compile(["accuracy"])
test_accuracy = model.evaluate(test_ds, return_dict=True, verbose=0)["accuracy"]
print(f"Test accuracy without hyper-parameter tuning: {test_accuracy:.4f}")
Test accuracy without hyper-parameter tuning: 0.7959

The default hyper-parameters of the model are available with the learner_params function. The definition of those parameters is available in the documentation.

print("Default hyper-parameters of the model:\n", model.learner_params)
Default hyper-parameters of the model:
 {'adapt_subsample_for_maximum_training_duration': False, 'allow_na_conditions': False, 'apply_link_function': True, 'categorical_algorithm': 'CART', 'categorical_set_split_greedy_sampling': 0.1, 'categorical_set_split_max_num_items': -1, 'categorical_set_split_min_item_frequency': 1, 'compute_permutation_variable_importance': False, 'dart_dropout': 0.01, 'early_stopping': 'LOSS_INCREASE', 'early_stopping_num_trees_look_ahead': 30, 'focal_loss_alpha': 0.5, 'focal_loss_gamma': 2.0, 'forest_extraction': 'MART', 'goss_alpha': 0.2, 'goss_beta': 0.1, 'growing_strategy': 'LOCAL', 'honest': False, 'honest_fixed_separation': False, 'honest_ratio_leaf_examples': 0.5, 'in_split_min_examples_check': True, 'keep_non_leaf_label_distribution': True, 'l1_regularization': 0.0, 'l2_categorical_regularization': 1.0, 'l2_regularization': 0.0, 'lambda_loss': 1.0, 'loss': 'DEFAULT', 'max_depth': 6, 'max_num_nodes': None, 'maximum_model_size_in_memory_in_bytes': -1.0, 'maximum_training_duration_seconds': -1.0, 'min_examples': 5, 'missing_value_policy': 'GLOBAL_IMPUTATION', 'num_candidate_attributes': -1, 'num_candidate_attributes_ratio': -1.0, 'num_trees': 300, 'pure_serving_model': False, 'random_seed': 123456, 'sampling_method': 'NONE', 'selective_gradient_boosting_ratio': 0.01, 'shrinkage': 0.1, 'sorting_strategy': 'PRESORT', 'sparse_oblique_normalization': None, 'sparse_oblique_num_projections_exponent': None, 'sparse_oblique_projection_density_factor': None, 'sparse_oblique_weights': None, 'split_axis': 'AXIS_ALIGNED', 'subsample': 1.0, 'uplift_min_examples_in_treatment': 5, 'uplift_split_score': 'KULLBACK_LEIBLER', 'use_hessian_gain': False, 'validation_interval_in_trees': 1, 'validation_ratio': 0.1}

Training a model with automated hyper-parameter tuning

Hyper-parameter tuning is enabled by specifying the tuner constructor argument of the model. The tuner object contains all the configuration of the tuner (search space, optimizer, trial and objective).

# Configure the tuner.

# Create a Random Search tuner with 50 trials.
tuner = tfdf.tuner.RandomSearch(num_trials=50)

# Define the search space.
#
# Adding more parameters generaly improve the quality of the model, but make
# the tuning last longer.

tuner.choice("min_examples", [2, 5, 7, 10])
tuner.choice("categorical_algorithm", ["CART", "RANDOM"])

# Some hyper-parameters are only valid for specific values of other
# hyper-parameters. For example, the "max_depth" parameter is mostly useful when
# "growing_strategy=LOCAL" while "max_num_nodes" is better suited when
# "growing_strategy=BEST_FIRST_GLOBAL".

local_search_space = tuner.choice("growing_strategy", ["LOCAL"])
local_search_space.choice("max_depth", [3, 4, 5, 6, 8])

# merge=True indicates that the parameter (here "growing_strategy") is already
# defined, and that new values are added to it.
global_search_space = tuner.choice("growing_strategy", ["BEST_FIRST_GLOBAL"], merge=True)
global_search_space.choice("max_num_nodes", [16, 32, 64, 128, 256])

tuner.choice("use_hessian_gain", [True, False])
tuner.choice("shrinkage", [0.02, 0.05, 0.10, 0.15])
tuner.choice("num_candidate_attributes_ratio", [0.2, 0.5, 0.9, 1.0])

# Uncomment some (or all) of the following hyper-parameters to increase the
# quality of the search. The number of trial should be increased accordingly.

# tuner.choice("split_axis", ["AXIS_ALIGNED"])
# oblique_space = tuner.choice("split_axis", ["SPARSE_OBLIQUE"], merge=True)
# oblique_space.choice("sparse_oblique_normalization",
#                      ["NONE", "STANDARD_DEVIATION", "MIN_MAX"])
# oblique_space.choice("sparse_oblique_weights", ["BINARY", "CONTINUOUS"])
# oblique_space.choice("sparse_oblique_num_projections_exponent", [1.0, 1.5])
<tensorflow_decision_forests.component.tuner.tuner.SearchSpace at 0x7ff500e4f070>
%%time
%set_cell_height 300

# Tune the model. Notice the `tuner=tuner`.
tuned_model = tfdf.keras.GradientBoostedTreesModel(tuner=tuner)
tuned_model.fit(train_ds, verbose=2)

# The `num_threads` model constructor argument (not specified in the example
# above) controls how many trials are run in parallel (one per thread). If
# `num_threads` is not specified (like in the example above), one thread is
# allocated for each available CPU core.
#
# If the training is interrupted (for example, by pressing on the "stop" button
# on the top-left of the colab cell), the best model so-far will be returned.

# In the training logs, you can see lines such as `[10/50] Score: -0.45 / -0.40
# HParams: ...`. This indicates that 10 of the 50 trials have been completed.
# And that the last trial returned a score of "-0.45" and that the best trial so
# far has a score of "-0.40". In this example, the model is optimized by
# logloss. Since scores are maximized and log loss should be minimized, the
# score is effectively minus the log loss.
<IPython.core.display.Javascript object>
Warning: The `num_threads` constructor argument is not set and the number of CPU is os.cpu_count()=32 > 32. Setting num_threads to 32. Set num_threads manually to use more than 32 cpus.
WARNING:absl:The `num_threads` constructor argument is not set and the number of CPU is os.cpu_count()=32 > 32. Setting num_threads to 32. Set num_threads manually to use more than 32 cpus.
Use /tmpfs/tmp/tmpp2l5nm7g as temporary training directory
Reading training dataset...
Training tensor examples:
Features: {'coef_0': <tf.Tensor 'data:0' shape=(None,) dtype=float64>, 'coef_1': <tf.Tensor 'data_1:0' shape=(None,) dtype=float64>, 'coef_2': <tf.Tensor 'data_12:0' shape=(None,) dtype=float64>, 'coef_3': <tf.Tensor 'data_23:0' shape=(None,) dtype=float64>, 'coef_4': <tf.Tensor 'data_34:0' shape=(None,) dtype=float64>, 'coef_5': <tf.Tensor 'data_45:0' shape=(None,) dtype=float64>, 'coef_6': <tf.Tensor 'data_56:0' shape=(None,) dtype=float64>, 'coef_7': <tf.Tensor 'data_67:0' shape=(None,) dtype=float64>, 'coef_8': <tf.Tensor 'data_74:0' shape=(None,) dtype=float64>, 'coef_9': <tf.Tensor 'data_75:0' shape=(None,) dtype=float64>, 'coef_10': <tf.Tensor 'data_2:0' shape=(None,) dtype=float64>, 'coef_11': <tf.Tensor 'data_3:0' shape=(None,) dtype=float64>, 'coef_12': <tf.Tensor 'data_4:0' shape=(None,) dtype=float64>, 'coef_13': <tf.Tensor 'data_5:0' shape=(None,) dtype=float64>, 'coef_14': <tf.Tensor 'data_6:0' shape=(None,) dtype=float64>, 'coef_15': <tf.Tensor 'data_7:0' shape=(None,) dtype=float64>, 'coef_16': <tf.Tensor 'data_8:0' shape=(None,) dtype=float64>, 'coef_17': <tf.Tensor 'data_9:0' shape=(None,) dtype=float64>, 'coef_18': <tf.Tensor 'data_10:0' shape=(None,) dtype=float64>, 'coef_19': <tf.Tensor 'data_11:0' shape=(None,) dtype=float64>, 'coef_20': <tf.Tensor 'data_13:0' shape=(None,) dtype=float64>, 'coef_21': <tf.Tensor 'data_14:0' shape=(None,) dtype=float64>, 'coef_22': <tf.Tensor 'data_15:0' shape=(None,) dtype=float64>, 'coef_23': <tf.Tensor 'data_16:0' shape=(None,) dtype=float64>, 'coef_24': <tf.Tensor 'data_17:0' shape=(None,) dtype=float64>, 'coef_25': <tf.Tensor 'data_18:0' shape=(None,) dtype=float64>, 'coef_26': <tf.Tensor 'data_19:0' shape=(None,) dtype=float64>, 'coef_27': <tf.Tensor 'data_20:0' shape=(None,) dtype=float64>, 'coef_28': <tf.Tensor 'data_21:0' shape=(None,) dtype=float64>, 'coef_29': <tf.Tensor 'data_22:0' shape=(None,) dtype=float64>, 'coef_30': <tf.Tensor 'data_24:0' shape=(None,) dtype=float64>, 'coef_31': <tf.Tensor 'data_25:0' shape=(None,) dtype=float64>, 'coef_32': <tf.Tensor 'data_26:0' shape=(None,) dtype=float64>, 'coef_33': <tf.Tensor 'data_27:0' shape=(None,) dtype=float64>, 'coef_34': <tf.Tensor 'data_28:0' shape=(None,) dtype=float64>, 'coef_35': <tf.Tensor 'data_29:0' shape=(None,) dtype=float64>, 'coef_36': <tf.Tensor 'data_30:0' shape=(None,) dtype=float64>, 'coef_37': <tf.Tensor 'data_31:0' shape=(None,) dtype=float64>, 'coef_38': <tf.Tensor 'data_32:0' shape=(None,) dtype=float64>, 'coef_39': <tf.Tensor 'data_33:0' shape=(None,) dtype=float64>, 'coef_40': <tf.Tensor 'data_35:0' shape=(None,) dtype=float64>, 'coef_41': <tf.Tensor 'data_36:0' shape=(None,) dtype=float64>, 'coef_42': <tf.Tensor 'data_37:0' shape=(None,) dtype=float64>, 'coef_43': <tf.Tensor 'data_38:0' shape=(None,) dtype=float64>, 'coef_44': <tf.Tensor 'data_39:0' shape=(None,) dtype=float64>, 'coef_45': <tf.Tensor 'data_40:0' shape=(None,) dtype=float64>, 'coef_46': <tf.Tensor 'data_41:0' shape=(None,) dtype=float64>, 'coef_47': <tf.Tensor 'data_42:0' shape=(None,) dtype=float64>, 'coef_48': <tf.Tensor 'data_43:0' shape=(None,) dtype=float64>, 'coef_49': <tf.Tensor 'data_44:0' shape=(None,) dtype=float64>, 'coef_50': <tf.Tensor 'data_46:0' shape=(None,) dtype=float64>, 'coef_51': <tf.Tensor 'data_47:0' shape=(None,) dtype=float64>, 'coef_52': <tf.Tensor 'data_48:0' shape=(None,) dtype=float64>, 'coef_53': <tf.Tensor 'data_49:0' shape=(None,) dtype=float64>, 'coef_54': <tf.Tensor 'data_50:0' shape=(None,) dtype=float64>, 'coef_55': <tf.Tensor 'data_51:0' shape=(None,) dtype=float64>, 'coef_56': <tf.Tensor 'data_52:0' shape=(None,) dtype=float64>, 'coef_57': <tf.Tensor 'data_53:0' shape=(None,) dtype=float64>, 'coef_58': <tf.Tensor 'data_54:0' shape=(None,) dtype=float64>, 'coef_59': <tf.Tensor 'data_55:0' shape=(None,) dtype=float64>, 'coef_60': <tf.Tensor 'data_57:0' shape=(None,) dtype=float64>, 'coef_61': <tf.Tensor 'data_58:0' shape=(None,) dtype=float64>, 'coef_62': <tf.Tensor 'data_59:0' shape=(None,) dtype=float64>, 'coef_63': <tf.Tensor 'data_60:0' shape=(None,) dtype=float64>, 'coef_64': <tf.Tensor 'data_61:0' shape=(None,) dtype=float64>, 'coef_65': <tf.Tensor 'data_62:0' shape=(None,) dtype=float64>, 'coef_66': <tf.Tensor 'data_63:0' shape=(None,) dtype=float64>, 'coef_67': <tf.Tensor 'data_64:0' shape=(None,) dtype=float64>, 'coef_68': <tf.Tensor 'data_65:0' shape=(None,) dtype=float64>, 'coef_69': <tf.Tensor 'data_66:0' shape=(None,) dtype=float64>, 'coef_70': <tf.Tensor 'data_68:0' shape=(None,) dtype=float64>, 'coef_71': <tf.Tensor 'data_69:0' shape=(None,) dtype=float64>, 'coef_72': <tf.Tensor 'data_70:0' shape=(None,) dtype=float64>, 'coef_73': <tf.Tensor 'data_71:0' shape=(None,) dtype=float64>, 'coef_74': <tf.Tensor 'data_72:0' shape=(None,) dtype=float64>, 'coef_75': <tf.Tensor 'data_73:0' shape=(None,) dtype=float64>}
Label: Tensor("data_76:0", shape=(None,), dtype=int64)
Weights: None
Normalized tensor features:
 {'coef_0': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast:0' shape=(None,) dtype=float32>), 'coef_1': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_1:0' shape=(None,) dtype=float32>), 'coef_2': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_2:0' shape=(None,) dtype=float32>), 'coef_3': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_3:0' shape=(None,) dtype=float32>), 'coef_4': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_4:0' shape=(None,) dtype=float32>), 'coef_5': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_5:0' shape=(None,) dtype=float32>), 'coef_6': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_6:0' shape=(None,) dtype=float32>), 'coef_7': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_7:0' shape=(None,) dtype=float32>), 'coef_8': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_8:0' shape=(None,) dtype=float32>), 'coef_9': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_9:0' shape=(None,) dtype=float32>), 'coef_10': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_10:0' shape=(None,) dtype=float32>), 'coef_11': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_11:0' shape=(None,) dtype=float32>), 'coef_12': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_12:0' shape=(None,) dtype=float32>), 'coef_13': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_13:0' shape=(None,) dtype=float32>), 'coef_14': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_14:0' shape=(None,) dtype=float32>), 'coef_15': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_15:0' shape=(None,) dtype=float32>), 'coef_16': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_16:0' shape=(None,) dtype=float32>), 'coef_17': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_17:0' shape=(None,) dtype=float32>), 'coef_18': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_18:0' shape=(None,) dtype=float32>), 'coef_19': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_19:0' shape=(None,) dtype=float32>), 'coef_20': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_20:0' shape=(None,) dtype=float32>), 'coef_21': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_21:0' shape=(None,) dtype=float32>), 'coef_22': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_22:0' shape=(None,) dtype=float32>), 'coef_23': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_23:0' shape=(None,) dtype=float32>), 'coef_24': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_24:0' shape=(None,) dtype=float32>), 'coef_25': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_25:0' shape=(None,) dtype=float32>), 'coef_26': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_26:0' shape=(None,) dtype=float32>), 'coef_27': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_27:0' shape=(None,) dtype=float32>), 'coef_28': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_28:0' shape=(None,) dtype=float32>), 'coef_29': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_29:0' shape=(None,) dtype=float32>), 'coef_30': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_30:0' shape=(None,) dtype=float32>), 'coef_31': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_31:0' shape=(None,) dtype=float32>), 'coef_32': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_32:0' shape=(None,) dtype=float32>), 'coef_33': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_33:0' shape=(None,) dtype=float32>), 'coef_34': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_34:0' shape=(None,) dtype=float32>), 'coef_35': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_35:0' shape=(None,) dtype=float32>), 'coef_36': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_36:0' shape=(None,) dtype=float32>), 'coef_37': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_37:0' shape=(None,) dtype=float32>), 'coef_38': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_38:0' shape=(None,) dtype=float32>), 'coef_39': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_39:0' shape=(None,) dtype=float32>), 'coef_40': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_40:0' shape=(None,) dtype=float32>), 'coef_41': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_41:0' shape=(None,) dtype=float32>), 'coef_42': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_42:0' shape=(None,) dtype=float32>), 'coef_43': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_43:0' shape=(None,) dtype=float32>), 'coef_44': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_44:0' shape=(None,) dtype=float32>), 'coef_45': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_45:0' shape=(None,) dtype=float32>), 'coef_46': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_46:0' shape=(None,) dtype=float32>), 'coef_47': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_47:0' shape=(None,) dtype=float32>), 'coef_48': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_48:0' shape=(None,) dtype=float32>), 'coef_49': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_49:0' shape=(None,) dtype=float32>), 'coef_50': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_50:0' shape=(None,) dtype=float32>), 'coef_51': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_51:0' shape=(None,) dtype=float32>), 'coef_52': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_52:0' shape=(None,) dtype=float32>), 'coef_53': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_53:0' shape=(None,) dtype=float32>), 'coef_54': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_54:0' shape=(None,) dtype=float32>), 'coef_55': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_55:0' shape=(None,) dtype=float32>), 'coef_56': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_56:0' shape=(None,) dtype=float32>), 'coef_57': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_57:0' shape=(None,) dtype=float32>), 'coef_58': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_58:0' shape=(None,) dtype=float32>), 'coef_59': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_59:0' shape=(None,) dtype=float32>), 'coef_60': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_60:0' shape=(None,) dtype=float32>), 'coef_61': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_61:0' shape=(None,) dtype=float32>), 'coef_62': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_62:0' shape=(None,) dtype=float32>), 'coef_63': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_63:0' shape=(None,) dtype=float32>), 'coef_64': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_64:0' shape=(None,) dtype=float32>), 'coef_65': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_65:0' shape=(None,) dtype=float32>), 'coef_66': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_66:0' shape=(None,) dtype=float32>), 'coef_67': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_67:0' shape=(None,) dtype=float32>), 'coef_68': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_68:0' shape=(None,) dtype=float32>), 'coef_69': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_69:0' shape=(None,) dtype=float32>), 'coef_70': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_70:0' shape=(None,) dtype=float32>), 'coef_71': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_71:0' shape=(None,) dtype=float32>), 'coef_72': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_72:0' shape=(None,) dtype=float32>), 'coef_73': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_73:0' shape=(None,) dtype=float32>), 'coef_74': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_74:0' shape=(None,) dtype=float32>), 'coef_75': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_75:0' shape=(None,) dtype=float32>)}
Training dataset read in 0:00:00.827423. Found 1363 examples.
Training model...
Standard output detected as not visible to the user e.g. running in a notebook. Creating a training log redirection. If training get stuck, try calling tfdf.keras.set_training_logs_redirection(False).
[INFO kernel.cc:813] Start Yggdrasil model training
[INFO kernel.cc:814] Collect training examples
[INFO kernel.cc:422] Number of batches: 2
[INFO kernel.cc:423] Number of examples: 1363
[INFO kernel.cc:836] Training dataset:
Number of records: 1363
Number of columns: 77

Number of columns by type:
    NUMERICAL: 76 (98.7013%)
    CATEGORICAL: 1 (1.2987%)

Columns:

NUMERICAL: 76 (98.7013%)
    0: "coef_0" NUMERICAL mean:0.183378 min:0.00492771 max:0.395772 sd:0.0922694
    1: "coef_1" NUMERICAL mean:0.375241 min:0.0140609 max:0.771281 sd:0.174751
    2: "coef_10" NUMERICAL mean:0.155655 min:0.00329301 max:0.420803 sd:0.085883
    3: "coef_11" NUMERICAL mean:0.164945 min:0.00409906 max:0.488068 sd:0.0820937
    4: "coef_12" NUMERICAL mean:0.163995 min:0.00142364 max:0.471881 sd:0.0910029
    5: "coef_13" NUMERICAL mean:0.192368 min:0.00590215 max:0.56481 sd:0.110566
    6: "coef_14" NUMERICAL mean:0.142261 min:0.0105102 max:0.332034 sd:0.0634995
    7: "coef_15" NUMERICAL mean:0.1153 min:0.00290088 max:0.366277 sd:0.058216
    8: "coef_16" NUMERICAL mean:0.127758 min:0.00287311 max:0.462718 sd:0.0671196
    9: "coef_17" NUMERICAL mean:0.130601 min:0.00208767 max:0.396381 sd:0.0784973
    10: "coef_18" NUMERICAL mean:0.118898 min:0.00230379 max:0.440113 sd:0.064675
    11: "coef_19" NUMERICAL mean:0.116073 min:0.00783665 max:0.345249 sd:0.0527643
    12: "coef_2" NUMERICAL mean:0.269396 min:0.00562382 max:0.665372 sd:0.138844
    13: "coef_20" NUMERICAL mean:0.107888 min:0.00243648 max:0.315663 sd:0.0520058
    14: "coef_21" NUMERICAL mean:0.108169 min:0.00260839 max:0.337783 sd:0.0528892
    15: "coef_22" NUMERICAL mean:0.0979448 min:0.0026094 max:0.332622 sd:0.057335
    16: "coef_23" NUMERICAL mean:0.113556 min:0.00158288 max:0.34269 sd:0.0627629
    17: "coef_24" NUMERICAL mean:0.105542 min:0.00182609 max:0.280131 sd:0.0463991
    18: "coef_25" NUMERICAL mean:0.0982893 min:0.00110629 max:0.312352 sd:0.0483484
    19: "coef_26" NUMERICAL mean:0.0951503 min:0.00527843 max:0.320455 sd:0.0476453
    20: "coef_27" NUMERICAL mean:0.0918396 min:0.00356535 max:0.32458 sd:0.0503912
    21: "coef_28" NUMERICAL mean:0.091518 min:0.00017629 max:0.312097 sd:0.0510217
    22: "coef_29" NUMERICAL mean:0.0946162 min:0.0044099 max:0.262548 sd:0.0452071
    23: "coef_3" NUMERICAL mean:0.264062 min:0.019079 max:0.748012 sd:0.109897
    24: "coef_30" NUMERICAL mean:0.0937955 min:0.00282674 max:0.247583 sd:0.044382
    25: "coef_31" NUMERICAL mean:0.0864024 min:0.00132908 max:0.289284 sd:0.04254
    26: "coef_32" NUMERICAL mean:0.0819615 min:0.00025893 max:0.25609 sd:0.0463068
    27: "coef_33" NUMERICAL mean:0.0831697 min:0.00256918 max:0.314669 sd:0.0447715
    28: "coef_34" NUMERICAL mean:0.0912174 min:0.0043182 max:0.278892 sd:0.0430243
    29: "coef_35" NUMERICAL mean:0.0856046 min:0.00158334 max:0.251377 sd:0.0414026
    30: "coef_36" NUMERICAL mean:0.0810899 min:0.00228121 max:0.257029 sd:0.0406632
    31: "coef_37" NUMERICAL mean:0.0773527 min:0.00046895 max:0.291041 sd:0.0417818
    32: "coef_38" NUMERICAL mean:0.0771318 min:0.00434419 max:0.272558 sd:0.0414008
    33: "coef_39" NUMERICAL mean:0.0851039 min:0.00480153 max:0.247084 sd:0.0411211
    34: "coef_4" NUMERICAL mean:0.300226 min:0.00209035 max:0.796487 sd:0.155972
    35: "coef_40" NUMERICAL mean:0.0819136 min:0.00087734 max:0.243434 sd:0.0394354
    36: "coef_41" NUMERICAL mean:0.0772351 min:0.00317318 max:0.22198 sd:0.0375504
    37: "coef_42" NUMERICAL mean:0.072984 min:0.00149386 max:0.235539 sd:0.0391918
    38: "coef_43" NUMERICAL mean:0.0716651 min:0.00113918 max:0.245737 sd:0.0389648
    39: "coef_44" NUMERICAL mean:0.0842747 min:0.00308002 max:0.230327 sd:0.0410869
    40: "coef_45" NUMERICAL mean:0.0782796 min:0.00519238 max:0.231811 sd:0.037137
    41: "coef_46" NUMERICAL mean:0.0739795 min:0.00223798 max:0.222058 sd:0.0386209
    42: "coef_47" NUMERICAL mean:0.0726674 min:0.00325095 max:0.227016 sd:0.0379209
    43: "coef_48" NUMERICAL mean:0.0861999 min:0.00440658 max:0.266345 sd:0.042239
    44: "coef_49" NUMERICAL mean:0.0825943 min:0.00115433 max:0.242921 sd:0.0408022
    45: "coef_5" NUMERICAL mean:0.158264 min:0.00267619 max:0.483342 sd:0.0890157
    46: "coef_50" NUMERICAL mean:0.0795964 min:0.00187021 max:0.247757 sd:0.0417273
    47: "coef_51" NUMERICAL mean:0.0783834 min:0.00227356 max:0.233935 sd:0.0418267
    48: "coef_52" NUMERICAL mean:0.0929331 min:0.00345921 max:0.250377 sd:0.0446751
    49: "coef_53" NUMERICAL mean:0.0909221 min:0.00157472 max:0.257 sd:0.044437
    50: "coef_54" NUMERICAL mean:0.0862077 min:0.00255567 max:0.253345 sd:0.0474379
    51: "coef_55" NUMERICAL mean:0.0851051 min:0.00156102 max:0.241507 sd:0.0435042
    52: "coef_56" NUMERICAL mean:0.101834 min:0.00166723 max:0.275385 sd:0.0498542
    53: "coef_57" NUMERICAL mean:0.100743 min:0.00266166 max:0.287096 sd:0.0499937
    54: "coef_58" NUMERICAL mean:0.0894441 min:0.00173879 max:0.249129 sd:0.0484976
    55: "coef_59" NUMERICAL mean:0.0946268 min:0.00189643 max:0.284695 sd:0.0529451
    56: "coef_6" NUMERICAL mean:0.367482 min:0.0119186 max:0.712186 sd:0.146432
    57: "coef_60" NUMERICAL mean:0.117082 min:0.00347996 max:0.300376 sd:0.0589656
    58: "coef_61" NUMERICAL mean:0.108374 min:0.00661185 max:0.319926 sd:0.0532069
    59: "coef_62" NUMERICAL mean:0.104477 min:0.00227164 max:0.31959 sd:0.0610118
    60: "coef_63" NUMERICAL mean:0.109393 min:0.00495736 max:0.351732 sd:0.0561484
    61: "coef_64" NUMERICAL mean:0.118121 min:0.00188382 max:0.427654 sd:0.0634669
    62: "coef_65" NUMERICAL mean:0.131464 min:0.00458132 max:0.369027 sd:0.0666763
    63: "coef_66" NUMERICAL mean:0.120773 min:0.00523224 max:0.397127 sd:0.0686975
    64: "coef_67" NUMERICAL mean:0.128344 min:0.00400679 max:0.418783 sd:0.0642292
    65: "coef_68" NUMERICAL mean:0.158361 min:0.00600492 max:0.47947 sd:0.091666
    66: "coef_69" NUMERICAL mean:0.180439 min:0.00813938 max:0.490737 sd:0.0859493
    67: "coef_7" NUMERICAL mean:0.247872 min:0.0092282 max:0.542303 sd:0.124564
    68: "coef_70" NUMERICAL mean:0.147855 min:0.00478888 max:0.456282 sd:0.0869718
    69: "coef_71" NUMERICAL mean:0.166876 min:0.00778369 max:0.457965 sd:0.091042
    70: "coef_72" NUMERICAL mean:0.229906 min:0.00356929 max:0.60137 sd:0.12371
    71: "coef_73" NUMERICAL mean:0.264569 min:0.0114335 max:0.620857 sd:0.131169
    72: "coef_74" NUMERICAL mean:0.143232 min:0.00113625 max:0.510432 sd:0.0852575
    73: "coef_75" NUMERICAL mean:0.215316 min:0.00457179 max:0.562397 sd:0.123783
    74: "coef_8" NUMERICAL mean:0.291335 min:0.0133491 max:0.702383 sd:0.117268
    75: "coef_9" NUMERICAL mean:0.153057 min:0.00234154 max:0.426099 sd:0.0874617

CATEGORICAL: 1 (1.2987%)
    76: "__LABEL" CATEGORICAL integerized vocab-size:11 no-ood-item

Terminology:
    nas: Number of non-available (i.e. missing) values.
    ood: Out of dictionary.
    manually-defined: Attribute which type is manually defined by the user i.e. the type was not automatically inferred.
    tokenized: The attribute value is obtained through tokenization.
    has-dict: The attribute is attached to a string dictionary e.g. a categorical attribute stored as a string.
    vocab-size: Number of unique values.

[INFO kernel.cc:882] Configure learner
[WARNING gradient_boosted_trees.cc:1691] Subsample hyperparameter given but sampling method does not match.
[WARNING gradient_boosted_trees.cc:1704] GOSS alpha hyperparameter given but GOSS is disabled.
[WARNING gradient_boosted_trees.cc:1713] GOSS beta hyperparameter given but GOSS is disabled.
[WARNING gradient_boosted_trees.cc:1725] SelGB ratio hyperparameter given but SelGB is disabled.
[INFO kernel.cc:912] Training config:
learner: "HYPERPARAMETER_OPTIMIZER"
features: "coef_0"
features: "coef_1"
features: "coef_10"
features: "coef_11"
features: "coef_12"
features: "coef_13"
features: "coef_14"
features: "coef_15"
features: "coef_16"
features: "coef_17"
features: "coef_18"
features: "coef_19"
features: "coef_2"
features: "coef_20"
features: "coef_21"
features: "coef_22"
features: "coef_23"
features: "coef_24"
features: "coef_25"
features: "coef_26"
features: "coef_27"
features: "coef_28"
features: "coef_29"
features: "coef_3"
features: "coef_30"
features: "coef_31"
features: "coef_32"
features: "coef_33"
features: "coef_34"
features: "coef_35"
features: "coef_36"
features: "coef_37"
features: "coef_38"
features: "coef_39"
features: "coef_4"
features: "coef_40"
features: "coef_41"
features: "coef_42"
features: "coef_43"
features: "coef_44"
features: "coef_45"
features: "coef_46"
features: "coef_47"
features: "coef_48"
features: "coef_49"
features: "coef_5"
features: "coef_50"
features: "coef_51"
features: "coef_52"
features: "coef_53"
features: "coef_54"
features: "coef_55"
features: "coef_56"
features: "coef_57"
features: "coef_58"
features: "coef_59"
features: "coef_6"
features: "coef_60"
features: "coef_61"
features: "coef_62"
features: "coef_63"
features: "coef_64"
features: "coef_65"
features: "coef_66"
features: "coef_67"
features: "coef_68"
features: "coef_69"
features: "coef_7"
features: "coef_70"
features: "coef_71"
features: "coef_72"
features: "coef_73"
features: "coef_74"
features: "coef_75"
features: "coef_8"
features: "coef_9"
label: "__LABEL"
task: CLASSIFICATION
metadata {
  framework: "TF Keras"
}
[yggdrasil_decision_forests.model.hyperparameters_optimizer_v2.proto.hyperparameters_optimizer_config] {
  base_learner {
    learner: "GRADIENT_BOOSTED_TREES"
    features: "coef_0"
    features: "coef_1"
    features: "coef_10"
    features: "coef_11"
    features: "coef_12"
    features: "coef_13"
    features: "coef_14"
    features: "coef_15"
    features: "coef_16"
    features: "coef_17"
    features: "coef_18"
    features: "coef_19"
    features: "coef_2"
    features: "coef_20"
    features: "coef_21"
    features: "coef_22"
    features: "coef_23"
    features: "coef_24"
    features: "coef_25"
    features: "coef_26"
    features: "coef_27"
    features: "coef_28"
    features: "coef_29"
    features: "coef_3"
    features: "coef_30"
    features: "coef_31"
    features: "coef_32"
    features: "coef_33"
    features: "coef_34"
    features: "coef_35"
    features: "coef_36"
    features: "coef_37"
    features: "coef_38"
    features: "coef_39"
    features: "coef_4"
    features: "coef_40"
    features: "coef_41"
    features: "coef_42"
    features: "coef_43"
    features: "coef_44"
    features: "coef_45"
    features: "coef_46"
    features: "coef_47"
    features: "coef_48"
    features: "coef_49"
    features: "coef_5"
    features: "coef_50"
    features: "coef_51"
    features: "coef_52"
    features: "coef_53"
    features: "coef_54"
    features: "coef_55"
    features: "coef_56"
    features: "coef_57"
    features: "coef_58"
    features: "coef_59"
    features: "coef_6"
    features: "coef_60"
    features: "coef_61"
    features: "coef_62"
    features: "coef_63"
    features: "coef_64"
    features: "coef_65"
    features: "coef_66"
    features: "coef_67"
    features: "coef_68"
    features: "coef_69"
    features: "coef_7"
    features: "coef_70"
    features: "coef_71"
    features: "coef_72"
    features: "coef_73"
    features: "coef_74"
    features: "coef_75"
    features: "coef_8"
    features: "coef_9"
    label: "__LABEL"
    task: CLASSIFICATION
    random_seed: 123456
    pure_serving_model: false
    [yggdrasil_decision_forests.model.gradient_boosted_trees.proto.gradient_boosted_trees_config] {
      num_trees: 300
      decision_tree {
        max_depth: 6
        min_examples: 5
        in_split_min_examples_check: true
        keep_non_leaf_label_distribution: true
        num_candidate_attributes: -1
        missing_value_policy: GLOBAL_IMPUTATION
        allow_na_conditions: false
        categorical_set_greedy_forward {
          sampling: 0.1
          max_num_items: -1
          min_item_frequency: 1
        }
        growing_strategy_local {
        }
        categorical {
          cart {
          }
        }
        axis_aligned_split {
        }
        internal {
          sorting_strategy: PRESORTED
        }
        uplift {
          min_examples_in_treatment: 5
          split_score: KULLBACK_LEIBLER
        }
      }
      shrinkage: 0.1
      loss: DEFAULT
      validation_set_ratio: 0.1
      validation_interval_in_trees: 1
      early_stopping: VALIDATION_LOSS_INCREASE
      early_stopping_num_trees_look_ahead: 30
      l2_regularization: 0
      lambda_loss: 1
      mart {
      }
      adapt_subsample_for_maximum_training_duration: false
      l1_regularization: 0
      use_hessian_gain: false
      l2_regularization_categorical: 1
      apply_link_function: true
      compute_permutation_variable_importance: false
      binary_focal_loss_options {
        misprediction_exponent: 2
        positive_sample_coefficient: 0.5
      }
    }
  }
  optimizer {
    optimizer_key: "RANDOM"
    [yggdrasil_decision_forests.model.hyperparameters_optimizer_v2.proto.random] {
      num_trials: 50
    }
  }
  search_space {
    fields {
      name: "min_examples"
      discrete_candidates {
        possible_values {
          integer: 2
        }
        possible_values {
          integer: 5
        }
        possible_values {
          integer: 7
        }
        possible_values {
          integer: 10
        }
      }
    }
    fields {
      name: "categorical_algorithm"
      discrete_candidates {
        possible_values {
          categorical: "CART"
        }
        possible_values {
          categorical: "RANDOM"
        }
      }
    }
    fields {
      name: "growing_strategy"
      discrete_candidates {
        possible_values {
          categorical: "LOCAL"
        }
        possible_values {
          categorical: "BEST_FIRST_GLOBAL"
        }
      }
      children {
        name: "max_depth"
        discrete_candidates {
          possible_values {
            integer: 3
          }
          possible_values {
            integer: 4
          }
          possible_values {
            integer: 5
          }
          possible_values {
            integer: 6
          }
          possible_values {
            integer: 8
          }
        }
        parent_discrete_values {
          possible_values {
            categorical: "LOCAL"
          }
        }
      }
      children {
        name: "max_num_nodes"
        discrete_candidates {
          possible_values {
            integer: 16
          }
          possible_values {
            integer: 32
          }
          possible_values {
            integer: 64
          }
          possible_values {
            integer: 128
          }
          possible_values {
            integer: 256
          }
        }
        parent_discrete_values {
          possible_values {
            categorical: "BEST_FIRST_GLOBAL"
          }
        }
      }
    }
    fields {
      name: "use_hessian_gain"
      discrete_candidates {
        possible_values {
          categorical: "true"
        }
        possible_values {
          categorical: "false"
        }
      }
    }
    fields {
      name: "shrinkage"
      discrete_candidates {
        possible_values {
          real: 0.02
        }
        possible_values {
          real: 0.05
        }
        possible_values {
          real: 0.1
        }
        possible_values {
          real: 0.15
        }
      }
    }
    fields {
      name: "num_candidate_attributes_ratio"
      discrete_candidates {
        possible_values {
          real: 0.2
        }
        possible_values {
          real: 0.5
        }
        possible_values {
          real: 0.9
        }
        possible_values {
          real: 1
        }
      }
    }
  }
}

[INFO kernel.cc:915] Deployment config:
cache_path: "/tmpfs/tmp/tmpp2l5nm7g/working_cache"
num_threads: 32
try_resume_training: true

[INFO kernel.cc:944] Train model
[INFO hyperparameters_optimizer.cc:207] Hyperparameter search space:
fields {
  name: "min_examples"
  discrete_candidates {
    possible_values {
      integer: 2
    }
    possible_values {
      integer: 5
    }
    possible_values {
      integer: 7
    }
    possible_values {
      integer: 10
    }
  }
}
fields {
  name: "categorical_algorithm"
  discrete_candidates {
    possible_values {
      categorical: "CART"
    }
    possible_values {
      categorical: "RANDOM"
    }
  }
}
fields {
  name: "growing_strategy"
  discrete_candidates {
    possible_values {
      categorical: "LOCAL"
    }
    possible_values {
      categorical: "BEST_FIRST_GLOBAL"
    }
  }
  children {
    name: "max_depth"
    discrete_candidates {
      possible_values {
        integer: 3
      }
      possible_values {
        integer: 4
      }
      possible_values {
        integer: 5
      }
      possible_values {
        integer: 6
      }
      possible_values {
        integer: 8
      }
    }
    parent_discrete_values {
      possible_values {
        categorical: "LOCAL"
      }
    }
  }
  children {
    name: "max_num_nodes"
    discrete_candidates {
      possible_values {
        integer: 16
      }
      possible_values {
        integer: 32
      }
      possible_values {
        integer: 64
      }
      possible_values {
        integer: 128
      }
      possible_values {
        integer: 256
      }
    }
    parent_discrete_values {
      possible_values {
        categorical: "BEST_FIRST_GLOBAL"
      }
    }
  }
}
fields {
  name: "use_hessian_gain"
  discrete_candidates {
    possible_values {
      categorical: "true"
    }
    possible_values {
      categorical: "false"
    }
  }
}
fields {
  name: "shrinkage"
  discrete_candidates {
    possible_values {
      real: 0.02
    }
    possible_values {
      real: 0.05
    }
    possible_values {
      real: 0.1
    }
    possible_values {
      real: 0.15
    }
  }
}
fields {
  name: "num_candidate_attributes_ratio"
  discrete_candidates {
    possible_values {
      real: 0.2
    }
    possible_values {
      real: 0.5
    }
    possible_values {
      real: 0.9
    }
    possible_values {
      real: 1
    }
  }
}

[INFO hyperparameters_optimizer.cc:480] Start local tuner with 32 thread(s)
[INFO gradient_boosted_trees.cc:408] Default loss set to MULTINOMIAL_LOG_LIKELIHOOD
[INFO gradient_boosted_trees.cc:1011] Training gradient boosted tree on 1363 example(s) and 76 feature(s).
[INFO gradient_boosted_trees.cc:408] Default loss set to MULTINOMIAL_LOG_LIKELIHOOD
[INFO gradient_boosted_trees.cc:1011] Training gradient boosted tree on 1363 example(s) and 76 feature(s).
[INFO [INFO gradient_boosted_trees.cc:[gradient_boosted_trees.cc:[408INFO] Default loss set to MULTINOMIAL_LOG_LIKELIHOOD
[INFO [INFO [gradient_boosted_trees.cc:408] Default loss set to INFO gradient_boosted_trees.cc:408]  [gradient_boosted_trees.cc:408] Default loss set to MULTINOMIAL_LOG_LIKELIHOODMULTINOMIAL_LOG_LIKELIHOOD408[INFOINFO[gradient_boosted_trees.cc
[ INFO[Default loss set to gradient_boosted_trees.ccINFO gradient_boosted_trees.ccMULTINOMIAL_LOG_LIKELIHOOD] [::INFOINFO408  
gradient_boosted_trees.cc:408] Default loss set to  
gradient_boosted_trees.cc:408] [] Default loss set to  Default loss set to Default loss set to gradient_boosted_trees.cc1011MULTINOMIAL_LOG_LIKELIHOOD] Training gradient boosted tree on INFO
MULTINOMIAL_LOG_LIKELIHOOD[
:gradient_boosted_trees.ccMULTINOMIAL_LOG_LIKELIHOOD[INFOMULTINOMIAL_LOG_LIKELIHOOD:[
408INFO[ INFO   :gradient_boosted_trees.cc4081011] Training gradient boosted tree on :INFOgradient_boosted_trees.cc1011] Training gradient boosted tree on :
] gradient_boosted_trees.ccDefault loss set to :[MULTINOMIAL_LOG_LIKELIHOOD
INFO1011 INFOgradient_boosted_trees.cc [1363gradient_boosted_trees.cc example(s) and 76 feature(s).
:10111011] ] ] Training gradient boosted tree on gradient_boosted_trees.cc:[ 1011] ] gradient_boosted_trees.cc13631363Training gradient boosted tree on 1363:INFOTraining gradient boosted tree on 1011:1363] 408 example(s) and Training gradient boosted tree on 1363 example(s) and Default loss set to  example(s) and INFO example(s) and  76gradient_boosted_trees.cc:408 feature(s).] Default loss set to 76 example(s) and [76 feature(s).
Training gradient boosted tree on INFO 76] 1363 example(s) and Default loss set to  MULTINOMIAL_LOG_LIKELIHOODMULTINOMIAL_LOG_LIKELIHOOD


 feature(s).1363MULTINOMIAL_LOG_LIKELIHOODgradient_boosted_trees.cc
: feature(s).[40876] Default loss set to MULTINOMIAL_LOG_LIKELIHOOD feature(s).

gradient_boosted_trees.cc[76INFO gradient_boosted_trees.cc[[
INFO[  feature(s).
[INFO:INFO
1011] INFOTraining gradient boosted tree on [INFO gradient_boosted_trees.cc:408] Default loss set to MULTINOMIAL_LOG_LIKELIHOOD
:[ INFO gradient_boosted_trees.cc:1011] INFOTraining gradient boosted tree on 1363 example(s) and 76 feature(s).
 gradient_boosted_trees.ccgradient_boosted_trees.cc:10111011] Training gradient boosted tree on ] 13631363Training gradient boosted tree on gradient_boosted_trees.cc example(s) and ::761011 example(s) and  ] gradient_boosted_trees.cc example(s) and 1363:1011 example(s) and [] Training gradient boosted tree on 76INFO 408 76] Default loss set to  feature(s). feature(s).MULTINOMIAL_LOG_LIKELIHOOD
76
1363 example(s) and 
76 feature(s).Training gradient boosted tree on 
[1363 feature(s).INFO gradient_boosted_trees.cc: example(s) and gradient_boosted_trees.ccgradient_boosted_trees.cc
::101176408] Training gradient boosted tree on ]  feature(s). feature(s).408
] 1363Default loss set to  example(s) and 
[76INFOMULTINOMIAL_LOG_LIKELIHOOD 
gradient_boosted_trees.ccDefault loss set to  feature(s).MULTINOMIAL_LOG_LIKELIHOOD
:[[INFO408[] Default loss set to [ INFOINFO gradient_boosted_trees.cc INFO:408] Default loss set to [gradient_boosted_trees.ccMULTINOMIAL_LOG_LIKELIHOODgradient_boosted_trees.cc[MULTINOMIAL_LOG_LIKELIHOOD
INFO gradient_boosted_trees.cc :408INFO: gradient_boosted_trees.cc:408] :[gradient_boosted_trees.cc[
Default loss set to INFO1011
 ] Training gradient boosted tree on 1363INFOMULTINOMIAL_LOG_LIKELIHOOD ] [gradient_boosted_trees.cc[:408INFO [[INFO INFO[ example(s) and INFO gradient_boosted_trees.cc76INFO feature(s).Default loss set to  [[
INFO ]  gradient_boosted_trees.ccgradient_boosted_trees.cc:4081054
] [1251[MULTINOMIAL_LOG_LIKELIHOODgradient_boosted_trees.ccINFO
: gradient_boosted_trees.ccgradient_boosted_trees.cc:1011Default loss set to :] ] 408MULTINOMIAL_LOG_LIKELIHOODINFOgradient_boosted_trees.cc408 :[gradient_boosted_trees.ccgradient_boosted_trees.ccINFO:[ examples used for training and ] 
:::INFO408]  :Training gradient boosted tree on Default loss set to 10114081363Default loss set to 112] 408 examples used for validation408Default loss set to 1011 ] MULTINOMIAL_LOG_LIKELIHOODMULTINOMIAL_LOG_LIKELIHOODTraining gradient boosted tree on 
INFOMULTINOMIAL_LOG_LIKELIHOOD ] gradient_boosted_trees.ccDefault loss set to 
:MULTINOMIAL_LOG_LIKELIHOODgradient_boosted_trees.cc
Default loss set to :4081363MULTINOMIAL_LOG_LIKELIHOOD] Default loss set to [] MULTINOMIAL_LOG_LIKELIHOODINFODefault loss set to gradient_boosted_trees.cc MULTINOMIAL_LOG_LIKELIHOOD] ] 
[Default loss set to 
 example(s) and [[INFO
 :1011gradient_boosted_trees.ccTraining gradient boosted tree on  example(s) and INFO1363MULTINOMIAL_LOG_LIKELIHOOD7676gradient_boosted_trees.cc feature(s).[:10111011:
 
INFOgradient_boosted_trees.cc:1011
 feature(s).] Training gradient boosted tree on 1363 example(s) and 
INFO76]  feature(s).] Training gradient boosted tree on 1011 example(s) and 1363 example(s) and 76[ INFOTraining gradient boosted tree on  1363 example(s) and 

]  feature(s).Training gradient boosted tree on 1363gradient_boosted_trees.cc
 example(s) and [7676INFO]   gradient_boosted_trees.cc: feature(s). feature(s).
Training gradient boosted tree on 1363 example(s) and 7676 feature(s).gradient_boosted_trees.cc feature(s).
:
1011:gradient_boosted_trees.cc1011] 
[Training gradient boosted tree on ] 1363:Training gradient boosted tree on  example(s) and INFO[ 1363INFO 101176] 1011 feature(s).gradient_boosted_trees.ccgradient_boosted_trees.cc] ::
 example(s) and 1011] Training gradient boosted tree on Training gradient boosted tree on 1363 example(s) and 101176] Training gradient boosted tree on Training gradient boosted tree on 761363 feature(s). example(s) and 1363761363
 feature(s). feature(s).

 example(s) and 76 feature(s).
 example(s) and 76 feature(s).
[INFO gradient_boosted_trees.cc:1054] 1251 examples used for training and 112 examples used for validation
[INFO gradient_boosted_trees.cc:1054] 1251 examples used for training and 112 examples used for validation
[INFO gradient_boosted_trees.cc:1054] 1251 examples used for training and 112 examples used for validation
[INFO gradient_boosted_trees.cc:1054] 1251 examples used for training and 112 examples used for validation
[INFO gradient_boosted_trees.cc:1054] 1251 examples used for training and 112 examples used for validation
[INFO gradient_boosted_trees.cc:1054] 1251 examples used for training and 112 examples used for validation
[INFO gradient_boosted_trees.cc:1054] 1251 examples used for training and 112 examples used for validation
[INFO gradient_boosted_trees.cc:1054] 1251 examples used for training and 112 examples used for validation
[INFO gradient_boosted_trees.cc:1054] 1251 examples used for training and 112 examples used for validation
[INFO gradient_boosted_trees.cc:1054] 1251 examples used for training and 112 examples used for validation
[INFO gradient_boosted_trees.cc:1054] 1251 examples used for training and 112 examples used for validation
[INFO gradient_boosted_trees.cc:1054] 1251 examples used for training and 112 examples used for validation
[INFO gradient_boosted_trees.cc:1054] 1251 examples used for training and 112 examples used for validation
[INFO gradient_boosted_trees.cc:1054] 1251 examples used for training and 112 examples used for validation
[INFO gradient_boosted_trees.cc:1054] 1251 examples used for training and 112 examples used for validation
[INFO gradient_boosted_trees.cc:1054] 1251 examples used for training and 112 examples used for validation
[INFO gradient_boosted_trees.cc:1054] 1251 examples used for training and 112 examples used for validation
[INFO gradient_boosted_trees.cc:1054] 1251 examples used for training and 112 examples used for validation
[INFO gradient_boosted_trees.cc:1054] 1251 examples used for training and 112 examples used for validation
[INFO gradient_boosted_trees.cc:1054] 1251 examples used for training and 112 examples used for validation
[INFO gradient_boosted_trees.cc:1054] 1251 examples used for training and 112 examples used for validation
[INFO gradient_boosted_trees.cc:1054] 1251 examples used for training and 112 examples used for validation
[INFO gradient_boosted_trees.cc:1054] 1251 examples used for training and 112 examples used for validation
[INFO gradient_boosted_trees.cc:1054] 1251 examples used for training and 112 examples used for validation
[INFO gradient_boosted_trees.cc:1054] 1251 examples used for training and 112 examples used for validation
[INFO gradient_boosted_trees.cc:1054] 1251 examples used for training and 112 examples used for validation
[INFO gradient_boosted_trees.cc:1054] 1251 examples used for training and 112 examples used for validation
[INFO gradient_boosted_trees.cc:1054] 1251 examples used for training and 112 examples used for validation
[INFO gradient_boosted_trees.cc:1054] 1251 examples used for training and 112 examples used for validation
[INFO gradient_boosted_trees.cc:1054] 1251 examples used for training and 112 examples used for validation
[INFO gradient_boosted_trees.cc:1054] 1251 examples used for training and 112 examples used for validation
[INFO gradient_boosted_trees.cc:1444]     num-trees:1 train-loss:1.653413 train-accuracy:0.888090 valid-loss:1.834304 valid-accuracy:0.669643
[INFO gradient_boosted_trees.cc:1444]     num-trees:1 train-loss:1.750658 train-accuracy:0.667466 valid-loss:1.796293 valid-accuracy:0.660714
[INFO gradient_boosted_trees.cc:1444]     num-trees:1 train-loss:1.737728 train-accuracy:0.832934 valid-loss:1.834889 valid-accuracy:0.723214
[INFO gradient_boosted_trees.cc:1444]     num-trees:1 train-loss:1.791635 train-accuracy:0.632294 valid-loss:1.829796 valid-accuracy:0.562500
[INFO gradient_boosted_trees.cc:1444]     num-trees:1 train-loss:1.509444 train-accuracy:0.827338 valid-loss:1.614802 valid-accuracy:0.750000
[INFO gradient_boosted_trees.cc:1444]     num-trees:1 train-loss:1.624682 train-accuracy:0.742606 valid-loss:1.699273 valid-accuracy:0.669643
[INFO gradient_boosted_trees.cc:1444]     num-trees:1 train-loss:1.834921 train-accuracy:0.752998 valid-loss:1.896991 valid-accuracy:0.758929
[INFO gradient_boosted_trees.cc:1444]     num-trees:1 train-loss:1.520203 train-accuracy:0.820144 valid-loss:1.601876 valid-accuracy:0.758929
[INFO gradient_boosted_trees.cc:1444]     num-trees:1 train-loss:1.707267 train-accuracy:0.865707 valid-loss:1.804670 valid-accuracy:0.741071
[INFO gradient_boosted_trees.cc:1444]     num-trees:1 train-loss:1.414888 train-accuracy:0.873701 valid-loss:1.553746 valid-accuracy:0.767857
[INFO gradient_boosted_trees.cc:1444]     num-trees:1 train-loss:1.993931 train-accuracy:0.864908 valid-loss:2.051592 valid-accuracy:0.696429
[INFO gradient_boosted_trees.cc:1444]     num-trees:1 train-loss:1.367150 train-accuracy:0.924860 valid-loss:1.548600 valid-accuracy:0.705357
[INFO gradient_boosted_trees.cc:1444]     num-trees:1 train-loss:2.175236 train-accuracy:0.884093 valid-loss:2.195420 valid-accuracy:0.669643
[INFO gradient_boosted_trees.cc:1444]     num-trees:1 train-loss:1.981728 train-accuracy:0.879297 valid-loss:2.029289 valid-accuracy:0.750000
[INFO gradient_boosted_trees.cc:1444]     num-trees:1 train-loss:1.983707 train-accuracy:0.884093 valid-loss:2.035470 valid-accuracy:0.776786
[INFO gradient_boosted_trees.cc:1444]     num-trees:1 train-loss:2.166944 train-accuracy:0.900879 valid-loss:2.187509 valid-accuracy:0.767857
[INFO gradient_boosted_trees.cc:1444]     num-trees:1 train-loss:2.165487 train-accuracy:0.916866 valid-loss:2.203004 valid-accuracy:0.696429
[INFO gradient_boosted_trees.cc:1444]     num-trees:1 train-loss:1.689289 train-accuracy:0.876099 valid-loss:1.808787 valid-accuracy:0.723214
[INFO gradient_boosted_trees.cc:1444]     num-trees:1 train-loss:1.659706 train-accuracy:0.900879 valid-loss:1.760347 valid-accuracy:0.767857
[INFO gradient_boosted_trees.cc:1444]     num-trees:1 train-loss:1.977811 train-accuracy:0.887290 valid-loss:2.052473 valid-accuracy:0.687500
[INFO gradient_boosted_trees.cc:1444]     num-trees:1 train-loss:1.963744 train-accuracy:0.902478 valid-loss:2.019070 valid-accuracy:0.723214
[INFO gradient_boosted_trees.cc:1444]     num-trees:1 train-loss:1.958318 train-accuracy:0.918465 valid-loss:2.025226 valid-accuracy:0.696429
[INFO gradient_boosted_trees.cc:1444]     num-trees:1 train-loss:1.672690 train-accuracy:0.892086 valid-loss:1.739787 valid-accuracy:0.803571
[INFO gradient_boosted_trees.cc:1444]     num-trees:1 train-loss:1.940455 train-accuracy:0.948841 valid-loss:2.041302 valid-accuracy:0.660714
[INFO gradient_boosted_trees.cc:1444]     num-trees:1 train-loss:2.157651 train-accuracy:0.908074 valid-loss:2.191730 valid-accuracy:0.750000
[INFO gradient_boosted_trees.cc:1444]     num-trees:1 train-loss:2.168729 train-accuracy:0.898481 valid-loss:2.192214 valid-accuracy:0.767857
[INFO gradient_boosted_trees.cc:1444]     num-trees:1 train-loss:2.152745 train-accuracy:0.946443 valid-loss:2.187933 valid-accuracy:0.714286
[INFO gradient_boosted_trees.cc:1444]     num-trees:1 train-loss:1.961435 train-accuracy:0.916866 valid-loss:2.014613 valid-accuracy:0.732143
[INFO gradient_boosted_trees.cc:1444]     num-trees:1 train-loss:2.162730 train-accuracy:0.908873 valid-loss:2.183622 valid-accuracy:0.758929
[INFO gradient_boosted_trees.cc:1444]     num-trees:1 train-loss:1.365464 train-accuracy:0.906475 valid-loss:1.521096 valid-accuracy:0.732143
[INFO gradient_boosted_trees.cc:1444]     num-trees:1 train-loss:2.152681 train-accuracy:0.946443 valid-loss:2.184971 valid-accuracy:0.723214
[INFO gradient_boosted_trees.cc:1444]     num-trees:1 train-loss:1.632447 train-accuracy:0.920863 valid-loss:1.735372 valid-accuracy:0.741071
[INFO gradient_boosted_trees.cc:2891] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.502119
[INFO gradient_boosted_trees.cc:233] Truncates the model to 230 tree(s) i.e. 23  iteration(s).
[INFO gradient_boosted_trees.cc:267] Final model num-trees:23 valid-loss:0.502119 valid-accuracy:0.812500
[INFO hyperparameters_optimizer.cc:562] [1/50] Score: -0.502119 / -0.502119 HParams: fields { name: "min_examples" value { integer: 7 } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 4 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "shrinkage" value { real: 0.15 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.9 } }
[INFO gradient_boosted_trees.cc:408] Default loss set to MULTINOMIAL_LOG_LIKELIHOOD
[INFO gradient_boosted_trees.cc:1011] Training gradient boosted tree on 1363 example(s) and 76 feature(s).
[INFO gradient_boosted_trees.cc:1054] 1251 examples used for training and 112 examples used for validation
[INFO gradient_boosted_trees.cc:2891] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.521248
[INFO gradient_boosted_trees.cc:233] Truncates the model to 330 tree(s) i.e. 33  iteration(s).
[INFO gradient_boosted_trees.cc:267] Final model num-trees:33 valid-loss:0.521248 valid-accuracy:0.785714
[INFO hyperparameters_optimizer.cc:562] [2/50] Score: -0.521248 / -0.502119 HParams: fields { name: "min_examples" value { integer: 10 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 3 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "shrinkage" value { real: 0.15 } } fields { name: "num_candidate_attributes_ratio" value { real: 1 } }
[INFO gradient_boosted_trees.cc:408] Default loss set to MULTINOMIAL_LOG_LIKELIHOOD
[INFO gradient_boosted_trees.cc:1011] Training gradient boosted tree on 1363 example(s) and 76 feature(s).
[INFO gradient_boosted_trees.cc:1054] 1251 examples used for training and 112 examples used for validation
[INFO gradient_boosted_trees.cc:2891] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.528781
[INFO gradient_boosted_trees.cc:233] Truncates the model to 240 tree(s) i.e. 24  iteration(s).
[INFO gradient_boosted_trees.cc:267] Final model num-trees:24 valid-loss:0.528781 valid-accuracy:0.794643
[INFO hyperparameters_optimizer.cc:562] [3/50] Score: -0.528781 / -0.502119 HParams: fields { name: "min_examples" value { integer: 2 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 4 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "shrinkage" value { real: 0.15 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.9 } }
[INFO gradient_boosted_trees.cc:408] Default loss set to MULTINOMIAL_LOG_LIKELIHOOD
[INFO gradient_boosted_trees.cc:1011] Training gradient boosted tree on 1363 example(s) and 76 feature(s).
[INFO gradient_boosted_trees.cc:1054] 1251 examples used for training and 112 examples used for validation
[INFO gradient_boosted_trees.cc:2891] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.525314
[INFO gradient_boosted_trees.cc:233] Truncates the model to 420 tree(s) i.e. 42  iteration(s).
[INFO gradient_boosted_trees.cc:267] Final model num-trees:42 valid-loss:0.525314 valid-accuracy:0.794643
[INFO gradient_boosted_trees.cc:408] Default loss set to MULTINOMIAL_LOG_LIKELIHOOD
[INFO gradient_boosted_trees.cc:1011] Training gradient boosted tree on 1363 example(s) and 76 feature(s).
[INFO hyperparameters_optimizer.cc:562] [4/50] Score: -0.525314 / -0.502119 HParams: fields { name: "min_examples" value { integer: 7 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 3 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "shrinkage" value { real: 0.15 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.2 } }
[INFO gradient_boosted_trees.cc:1054] 1251 examples used for training and 112 examples used for validation
[INFO gradient_boosted_trees.cc:1444]     num-trees:1 train-loss:2.163671 train-accuracy:0.916067 valid-loss:2.193245 valid-accuracy:0.687500
[INFO gradient_boosted_trees.cc:1444]     num-trees:1 train-loss:2.004538 train-accuracy:0.840927 valid-loss:2.066230 valid-accuracy:0.660714
[INFO gradient_boosted_trees.cc:1444]     num-trees:1 train-loss:1.600906 train-accuracy:0.948841 valid-loss:1.807883 valid-accuracy:0.678571
[INFO gradient_boosted_trees.cc:1444]     num-trees:1 train-loss:1.554608 train-accuracy:0.965627 valid-loss:1.791488 valid-accuracy:0.669643
[INFO gradient_boosted_trees.cc:2891] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.496028
[INFO gradient_boosted_trees.cc:233] Truncates the model to 410 tree(s) i.e. 41  iteration(s).
[INFO gradient_boosted_trees.cc:267] Final model num-trees:41 valid-loss:0.496028 valid-accuracy:0.812500
[INFO hyperparameters_optimizer.cc:562] [5/50] Score: -0.496028 / -0.496028 HParams: fields { name: "min_examples" value { integer: 10 } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 3 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "shrinkage" value { real: 0.15 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.5 } }
[INFO gradient_boosted_trees.cc:408] Default loss set to MULTINOMIAL_LOG_LIKELIHOOD
[INFO gradient_boosted_trees.cc:1011] Training gradient boosted tree on 1363 example(s) and 76 feature(s).
[INFO gradient_boosted_trees.cc:1054] 1251 examples used for training and 112 examples used for validation
[INFO gradient_boosted_trees.cc:1444]     num-trees:1 train-loss:1.947790 train-accuracy:0.631495 valid-loss:1.974304 valid-accuracy:0.562500
[INFO gradient_boosted_trees.cc:2891] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.5064
[INFO gradient_boosted_trees.cc:233] Truncates the model to 270 tree(s) i.e. 27  iteration(s).
[INFO gradient_boosted_trees.cc:267] Final model num-trees:27 valid-loss:0.506400 valid-accuracy:0.821429
[INFO hyperparameters_optimizer.cc:562] [6/50] Score: -0.5064 / -0.496028 HParams: fields { name: "min_examples" value { integer: 2 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 5 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "shrinkage" value { real: 0.15 } } fields { name: "num_candidate_attributes_ratio" value { real: 1 } }
[INFO gradient_boosted_trees.cc:408] Default loss set to MULTINOMIAL_LOG_LIKELIHOOD
[INFO gradient_boosted_trees.cc:1011] Training gradient boosted tree on 1363 example(s) and 76 feature(s).
[INFO gradient_boosted_trees.cc:1054] 1251 examples used for training and 112 examples used for validation
[INFO gradient_boosted_trees.cc:2891] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.491424
[INFO gradient_boosted_trees.cc:233] Truncates the model to 330 tree(s) i.e. 33  iteration(s).
[INFO gradient_boosted_trees.cc:267] Final model num-trees:33 valid-loss:0.491424 valid-accuracy:0.812500
[INFO hyperparameters_optimizer.cc:562] [7/50] Score: -0.491424 / -0.491424 HParams: fields { name: "min_examples" value { integer: 7 } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 5 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "shrinkage" value { real: 0.1 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.5 } }
[INFO gradient_boosted_trees.cc:408] Default loss set to MULTINOMIAL_LOG_LIKELIHOOD
[INFO gradient_boosted_trees.cc:1011] Training gradient boosted tree on 1363 example(s) and 76 feature(s).
[INFO gradient_boosted_trees.cc:1054] 1251 examples used for training and 112 examples used for validation
[INFO gradient_boosted_trees.cc:1444]     num-trees:1 train-loss:1.416606 train-accuracy:0.865707 valid-loss:1.544478 valid-accuracy:0.714286
[INFO gradient_boosted_trees.cc:1444]     num-trees:1 train-loss:2.164281 train-accuracy:0.912070 valid-loss:2.208599 valid-accuracy:0.633929
[INFO gradient_boosted_trees.cc:2891] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.44606
[INFO gradient_boosted_trees.cc:233] Truncates the model to 360 tree(s) i.e. 36  iteration(s).
[INFO gradient_boosted_trees.cc:267] Final model num-trees:36 valid-loss:0.446060 valid-accuracy:0.803571
[INFO gradient_boosted_trees.cc:408] Default loss set to MULTINOMIAL_LOG_LIKELIHOOD
[INFO gradient_boosted_trees.cc:1011] Training gradient boosted tree on 1363 example(s) and 76 feature(s).
[INFO gradient_boosted_trees.cc:1054] 1251 examples used for training and 112 examples used for validation
[INFO hyperparameters_optimizer.cc:562] [8/50] Score: -0.44606 / -0.44606 HParams: fields { name: "min_examples" value { integer: 10 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 5 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "shrinkage" value { real: 0.1 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.5 } }
[INFO gradient_boosted_trees.cc:1444]     num-trees:1 train-loss:2.187537 train-accuracy:0.822542 valid-loss:2.201325 valid-accuracy:0.750000
[INFO gradient_boosted_trees.cc:2891] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.499614
[INFO gradient_boosted_trees.cc:233] Truncates the model to 340 tree(s) i.e. 34  iteration(s).
[INFO gradient_boosted_trees.cc:267] Final model num-trees:34 valid-loss:0.499614 valid-accuracy:0.785714
[INFO hyperparameters_optimizer.cc:562] [9/50] Score: -0.499614 / -0.44606 HParams: fields { name: "min_examples" value { integer: 2 } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 32 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "shrinkage" value { real: 0.1 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.2 } }
[INFO gradient_boosted_trees.cc:408] Default loss set to MULTINOMIAL_LOG_LIKELIHOOD
[INFO gradient_boosted_trees.cc:1011] Training gradient boosted tree on 1363 example(s) and 76 feature(s).
[INFO gradient_boosted_trees.cc:1054] 1251 examples used for training and 112 examples used for validation
[INFO gradient_boosted_trees.cc:1444]     num-trees:1 train-loss:1.450488 train-accuracy:0.858513 valid-loss:1.568415 valid-accuracy:0.714286
[INFO gradient_boosted_trees.cc:2891] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.450929
[INFO gradient_boosted_trees.cc:233] Truncates the model to 240 tree(s) i.e. 24  iteration(s).
[INFO gradient_boosted_trees.cc:267] Final model num-trees:24 valid-loss:0.450929 valid-accuracy:0.821429
[INFO hyperparameters_optimizer.cc:562] [10/50] Score: -0.450929 / -0.44606 HParams: fields { name: "min_examples" value { integer: 5 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 8 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "shrinkage" value { real: 0.15 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.2 } }
[INFO gradient_boosted_trees.cc:408] Default loss set to MULTINOMIAL_LOG_LIKELIHOOD
[INFO gradient_boosted_trees.cc:1011] Training gradient boosted tree on 1363 example(s) and 76 feature(s).
[INFO gradient_boosted_trees.cc:1054] 1251 examples used for training and 112 examples used for validation
[INFO gradient_boosted_trees.cc:1444]     num-trees:1 train-loss:1.647848 train-accuracy:0.916866 valid-loss:1.787233 valid-accuracy:0.714286
[INFO gradient_boosted_trees.cc:2891] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.467523
[INFO gradient_boosted_trees.cc:233] Truncates the model to 230 tree(s) i.e. 23  iteration(s).
[INFO gradient_boosted_trees.cc:267] Final model num-trees:23 valid-loss:0.467523 valid-accuracy:0.821429
[INFO hyperparameters_optimizer.cc:562] [11/50] Score: -0.467523 / -0.44606 HParams: fields { name: "min_examples" value { integer: 7 } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 16 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "shrinkage" value { real: 0.15 } } fields { name: "num_candidate_attributes_ratio" value { real: 1 } }
[INFO gradient_boosted_trees.cc:408] Default loss set to MULTINOMIAL_LOG_LIKELIHOOD
[INFO gradient_boosted_trees.cc:1011] Training gradient boosted tree on 1363 example(s) and 76 feature(s).
[INFO gradient_boosted_trees.cc:1054] 1251 examples used for training and 112 examples used for validation
[INFO gradient_boosted_trees.cc:1444]     num-trees:1 train-loss:1.969352 train-accuracy:0.898481 valid-loss:2.029026 valid-accuracy:0.750000
[INFO gradient_boosted_trees.cc:2891] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.507174
[INFO gradient_boosted_trees.cc:233] Truncates the model to 650 tree(s) i.e. 65  iteration(s).
[INFO gradient_boosted_trees.cc:267] Final model num-trees:65 valid-loss:0.507174 valid-accuracy:0.767857
[INFO hyperparameters_optimizer.cc:562] [12/50] Score: -0.507174 / -0.44606 HParams: fields { name: "min_examples" value { integer: 2 } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 3 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "shrinkage" value { real: 0.1 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.9 } }
[INFO gradient_boosted_trees.cc:408] Default loss set to MULTINOMIAL_LOG_LIKELIHOOD
[INFO gradient_boosted_trees.cc:1011] Training gradient boosted tree on 1363 example(s) and 76 feature(s).
[INFO gradient_boosted_trees.cc:1054] 1251 examples used for training and 112 examples used for validation
[INFO gradient_boosted_trees.cc:1444]     num-trees:1 train-loss:2.171837 train-accuracy:0.879297 valid-loss:2.206239 valid-accuracy:0.660714
[INFO gradient_boosted_trees.cc:2891] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.470876
[INFO gradient_boosted_trees.cc:233] Truncates the model to 670 tree(s) i.e. 67  iteration(s).
[INFO gradient_boosted_trees.cc:267] Final model num-trees:67 valid-loss:0.470876 valid-accuracy:0.803571
[INFO hyperparameters_optimizer.cc:562] [13/50] Score: -0.470876 / -0.44606 HParams: fields { name: "min_examples" value { integer: 2 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 3 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "shrinkage" value { real: 0.1 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.2 } }
[INFO gradient_boosted_trees.cc:408] Default loss set to MULTINOMIAL_LOG_LIKELIHOOD
[INFO gradient_boosted_trees.cc:1011] Training gradient boosted tree on 1363 example(s) and 76 feature(s).
[INFO gradient_boosted_trees.cc:1054] 1251 examples used for training and 112 examples used for validation
[INFO gradient_boosted_trees.cc:1444]     num-trees:1 train-loss:1.633314 train-accuracy:0.924860 valid-loss:1.752720 valid-accuracy:0.767857
[INFO gradient_boosted_trees.cc:2891] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.478369
[INFO gradient_boosted_trees.cc:233] Truncates the model to 250 tree(s) i.e. 25  iteration(s).
[INFO gradient_boosted_trees.cc:267] Final model num-trees:25 valid-loss:0.478369 valid-accuracy:0.812500
[INFO hyperparameters_optimizer.cc:562] [14/50] Score: -0.478369 / -0.44606 HParams: fields { name: "min_examples" value { integer: 5 } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 5 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "shrinkage" value { real: 0.15 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.5 } }
[INFO gradient_boosted_trees.cc:408] Default loss set to MULTINOMIAL_LOG_LIKELIHOOD
[INFO gradient_boosted_trees.cc:1011] Training gradient boosted tree on 1363 example(s) and 76 feature(s).
[INFO gradient_boosted_trees.cc:1054] 1251 examples used for training and 112 examples used for validation
[INFO gradient_boosted_trees.cc:2891] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.519242
[INFO gradient_boosted_trees.cc:233] Truncates the model to 320 tree(s) i.e. 32  iteration(s).
[INFO gradient_boosted_trees.cc:267] Final model num-trees:32 valid-loss:0.519242 valid-accuracy:0.812500
[INFO hyperparameters_optimizer.cc:562] [15/50] Score: -0.519242 / -0.44606 HParams: fields { name: "min_examples" value { integer: 5 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 6 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "shrinkage" value { real: 0.1 } } fields { name: "num_candidate_attributes_ratio" value { real: 1 } }
[INFO gradient_boosted_trees.cc:408] Default loss set to MULTINOMIAL_LOG_LIKELIHOOD
[INFO gradient_boosted_trees.cc:1011] Training gradient boosted tree on 1363 example(s) and 76 feature(s).
[INFO gradient_boosted_trees.cc:1054] 1251 examples used for training and 112 examples used for validation
[INFO gradient_boosted_trees.cc:1444]     num-trees:1 train-loss:1.630183 train-accuracy:0.928058 valid-loss:1.766456 valid-accuracy:0.741071
[INFO gradient_boosted_trees.cc:1444]     num-trees:1 train-loss:1.588608 train-accuracy:0.949640 valid-loss:1.828821 valid-accuracy:0.633929
[INFO gradient_boosted_trees.cc:2891] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.475435
[INFO gradient_boosted_trees.cc:233] Truncates the model to 480 tree(s) i.e. 48  iteration(s).
[INFO gradient_boosted_trees.cc:267] Final model num-trees:48 valid-loss:0.475435 valid-accuracy:0.785714
[INFO hyperparameters_optimizer.cc:562] [16/50] Score: -0.475435 / -0.44606 HParams: fields { name: "min_examples" value { integer: 7 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 64 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "shrinkage" value { real: 0.1 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.2 } }
[INFO gradient_boosted_trees.cc:408] Default loss set to MULTINOMIAL_LOG_LIKELIHOOD
[INFO gradient_boosted_trees.cc:1011] Training gradient boosted tree on 1363 example(s) and 76 feature(s).
[INFO gradient_boosted_trees.cc:1054] 1251 examples used for training and 112 examples used for validation
[INFO gradient_boosted_trees.cc:1444]     num-trees:1 train-loss:2.161376 train-accuracy:0.924860 valid-loss:2.185787 valid-accuracy:0.767857
[INFO gradient_boosted_trees.cc:2891] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.558749
[INFO gradient_boosted_trees.cc:233] Truncates the model to 300 tree(s) i.e. 30  iteration(s).
[INFO gradient_boosted_trees.cc:267] Final model num-trees:30 valid-loss:0.558749 valid-accuracy:0.830357
[INFO hyperparameters_optimizer.cc:562] [17/50] Score: -0.558749 / -0.44606 HParams: fields { name: "min_examples" value { integer: 5 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 128 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "shrinkage" value { real: 0.1 } } fields { name: "num_candidate_attributes_ratio" value { real: 1 } }
[INFO gradient_boosted_trees.cc:408] Default loss set to MULTINOMIAL_LOG_LIKELIHOOD
[INFO gradient_boosted_trees.cc:1011] Training gradient boosted tree on 1363 example(s) and 76 feature(s).
[INFO gradient_boosted_trees.cc:1054] 1251 examples used for training and 112 examples used for validation
[INFO gradient_boosted_trees.cc:1444]     num-trees:1 train-loss:1.955647 train-accuracy:0.908873 valid-loss:2.023832 valid-accuracy:0.767857
[INFO gradient_boosted_trees.cc:2891] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.492647
[INFO gradient_boosted_trees.cc:233] Truncates the model to 560 tree(s) i.e. 56  iteration(s).
[INFO gradient_boosted_trees.cc:267] Final model num-trees:56 valid-loss:0.492647 valid-accuracy:0.812500
[INFO hyperparameters_optimizer.cc:562] [18/50] Score: -0.492647 / -0.44606 HParams: fields { name: "min_examples" value { integer: 10 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 16 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "shrinkage" value { real: 0.05 } } fields { name: "num_candidate_attributes_ratio" value { real: 1 } }
[INFO gradient_boosted_trees.cc:408] Default loss set to MULTINOMIAL_LOG_LIKELIHOOD
[INFO gradient_boosted_trees.cc:1011] Training gradient boosted tree on 1363 example(s) and 76 feature(s).
[INFO gradient_boosted_trees.cc:1054] 1251 examples used for training and 112 examples used for validation
[INFO gradient_boosted_trees.cc:2891] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.475416
[INFO gradient_boosted_trees.cc:233] Truncates the model to 360 tree(s) i.e. 36  iteration(s).
[INFO gradient_boosted_trees.cc:267] Final model num-trees:36 valid-loss:0.475416 valid-accuracy:0.803571
[INFO hyperparameters_optimizer.cc:562] [19/50] Score: -0.475416 / -0.44606 HParams: fields { name: "min_examples" value { integer: 10 } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 16 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "shrinkage" value { real: 0.15 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.5 } }
[INFO gradient_boosted_trees.cc:1444]     num-trees:1 train-loss:1.593897 train-accuracy:0.942446 valid-loss:1.784653 valid-accuracy:0.714286
[INFO gradient_boosted_trees.cc:1446]     num-trees:67 train-loss:0.024280 train-accuracy:1.000000 valid-loss:0.468924 valid-accuracy:0.794643
[INFO gradient_boosted_trees.cc:2891] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.538939
[INFO gradient_boosted_trees.cc:233] Truncates the model to 290 tree(s) i.e. 29  iteration(s).
[INFO gradient_boosted_trees.cc:267] Final model num-trees:29 valid-loss:0.538939 valid-accuracy:0.812500
[INFO hyperparameters_optimizer.cc:562] [20/50] Score: -0.538939 / -0.44606 HParams: fields { name: "min_examples" value { integer: 2 } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 8 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "shrinkage" value { real: 0.1 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.5 } }
[INFO gradient_boosted_trees.cc:2891] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.445973
[INFO gradient_boosted_trees.cc:233] Truncates the model to 460 tree(s) i.e. 46  iteration(s).
[INFO gradient_boosted_trees.cc:267] Final model num-trees:46 valid-loss:0.445973 valid-accuracy:0.830357
[INFO hyperparameters_optimizer.cc:562] [21/50] Score: -0.445973 / -0.445973 HParams: fields { name: "min_examples" value { integer: 10 } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 16 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "shrinkage" value { real: 0.1 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.9 } }
[INFO gradient_boosted_trees.cc:2891] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.503971
[INFO gradient_boosted_trees.cc:233] Truncates the model to 400 tree(s) i.e. 40  iteration(s).
[INFO gradient_boosted_trees.cc:267] Final model num-trees:40 valid-loss:0.503971 valid-accuracy:0.803571
[INFO hyperparameters_optimizer.cc:562] [22/50] Score: -0.503971 / -0.445973 HParams: fields { name: "min_examples" value { integer: 2 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 8 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "shrinkage" value { real: 0.1 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.2 } }
[INFO gradient_boosted_trees.cc:2891] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.462804
[INFO gradient_boosted_trees.cc:233] Truncates the model to 730 tree(s) i.e. 73  iteration(s).
[INFO gradient_boosted_trees.cc:267] Final model num-trees:73 valid-loss:0.462804 valid-accuracy:0.794643
[INFO hyperparameters_optimizer.cc:562] [23/50] Score: -0.462804 / -0.445973 HParams: fields { name: "min_examples" value { integer: 5 } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 256 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "shrinkage" value { real: 0.05 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.2 } }
[INFO gradient_boosted_trees.cc:2891] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.472479
[INFO gradient_boosted_trees.cc:233] Truncates the model to 320 tree(s) i.e. 32  iteration(s).
[INFO gradient_boosted_trees.cc:267] Final model num-trees:32 valid-loss:0.472479 valid-accuracy:0.821429
[INFO hyperparameters_optimizer.cc:562] [24/50] Score: -0.472479 / -0.445973 HParams: fields { name: "min_examples" value { integer: 7 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 32 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "shrinkage" value { real: 0.1 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.9 } }
[INFO gradient_boosted_trees.cc:2891] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.412312
[INFO gradient_boosted_trees.cc:233] Truncates the model to 1010 tree(s) i.e. 101  iteration(s).
[INFO gradient_boosted_trees.cc:267] Final model num-trees:101 valid-loss:0.412312 valid-accuracy:0.848214
[INFO hyperparameters_optimizer.cc:562] [25/50] Score: -0.412312 / -0.412312 HParams: fields { name: "min_examples" value { integer: 7 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 16 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "shrinkage" value { real: 0.05 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.2 } }
[INFO gradient_boosted_trees.cc:2891] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.449361
[INFO gradient_boosted_trees.cc:233] Truncates the model to 820 tree(s) i.e. 82  iteration(s).
[INFO gradient_boosted_trees.cc:267] Final model num-trees:82 valid-loss:0.449361 valid-accuracy:0.794643
[INFO hyperparameters_optimizer.cc:562] [26/50] Score: -0.449361 / -0.412312 HParams: fields { name: "min_examples" value { integer: 10 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 16 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "shrinkage" value { real: 0.05 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.2 } }
[INFO gradient_boosted_trees.cc:2891] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.441124
[INFO gradient_boosted_trees.cc:233] Truncates the model to 650 tree(s) i.e. 65  iteration(s).
[INFO gradient_boosted_trees.cc:267] Final model num-trees:65 valid-loss:0.441124 valid-accuracy:0.821429
[INFO hyperparameters_optimizer.cc:562] [27/50] Score: -0.441124 / -0.412312 HParams: fields { name: "min_examples" value { integer: 5 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 32 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "shrinkage" value { real: 0.05 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.5 } }
[INFO gradient_boosted_trees.cc:2891] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.47638
[INFO gradient_boosted_trees.cc:233] Truncates the model to 340 tree(s) i.e. 34  iteration(s).
[INFO gradient_boosted_trees.cc:267] Final model num-trees:34 valid-loss:0.476380 valid-accuracy:0.794643
[INFO hyperparameters_optimizer.cc:562] [28/50] Score: -0.47638 / -0.412312 HParams: fields { name: "min_examples" value { integer: 7 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 8 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "shrinkage" value { real: 0.1 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.5 } }
[INFO gradient_boosted_trees.cc:2891] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.478586
[INFO gradient_boosted_trees.cc:233] Truncates the model to 310 tree(s) i.e. 31  iteration(s).
[INFO gradient_boosted_trees.cc:267] Final model num-trees:31 valid-loss:0.478586 valid-accuracy:0.812500
[INFO hyperparameters_optimizer.cc:562] [29/50] Score: -0.478586 / -0.412312 HParams: fields { name: "min_examples" value { integer: 5 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 256 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "shrinkage" value { real: 0.1 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.9 } }
[INFO gradient_boosted_trees.cc:2891] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.489521
[INFO gradient_boosted_trees.cc:233] Truncates the model to 630 tree(s) i.e. 63  iteration(s).
[INFO gradient_boosted_trees.cc:267] Final model num-trees:63 valid-loss:0.489521 valid-accuracy:0.839286
[INFO hyperparameters_optimizer.cc:562] [30/50] Score: -0.489521 / -0.412312 HParams: fields { name: "min_examples" value { integer: 5 } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 8 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "shrinkage" value { real: 0.05 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.5 } }
[INFO gradient_boosted_trees.cc:2891] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.477423
[INFO gradient_boosted_trees.cc:233] Truncates the model to 720 tree(s) i.e. 72  iteration(s).
[INFO gradient_boosted_trees.cc:267] Final model num-trees:72 valid-loss:0.477423 valid-accuracy:0.830357
[INFO hyperparameters_optimizer.cc:562] [31/50] Score: -0.477423 / -0.412312 HParams: fields { name: "min_examples" value { integer: 7 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 16 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "shrinkage" value { real: 0.05 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.9 } }
[INFO gradient_boosted_trees.cc:2891] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.525921
[INFO gradient_boosted_trees.cc:233] Truncates the model to 380 tree(s) i.e. 38  iteration(s).
[INFO gradient_boosted_trees.cc:267] Final model num-trees:38 valid-loss:0.525921 valid-accuracy:0.794643
[INFO hyperparameters_optimizer.cc:562] [32/50] Score: -0.525921 / -0.412312 HParams: fields { name: "min_examples" value { integer: 2 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 8 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "shrinkage" value { real: 0.1 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.2 } }
[INFO gradient_boosted_trees.cc:2891] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.506496
[INFO gradient_boosted_trees.cc:233] Truncates the model to 610 tree(s) i.e. 61  iteration(s).
[INFO gradient_boosted_trees.cc:267] Final model num-trees:61 valid-loss:0.506496 valid-accuracy:0.848214
[INFO hyperparameters_optimizer.cc:562] [33/50] Score: -0.506496 / -0.412312 HParams: fields { name: "min_examples" value { integer: 7 } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 64 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "shrinkage" value { real: 0.05 } } fields { name: "num_candidate_attributes_ratio" value { real: 1 } }
[INFO gradient_boosted_trees.cc:2891] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.488399
[INFO gradient_boosted_trees.cc:233] Truncates the model to 730 tree(s) i.e. 73  iteration(s).
[INFO gradient_boosted_trees.cc:267] Final model num-trees:73 valid-loss:0.488399 valid-accuracy:0.812500
[INFO hyperparameters_optimizer.cc:562] [34/50] Score: -0.488399 / -0.412312 HParams: fields { name: "min_examples" value { integer: 5 } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 6 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "shrinkage" value { real: 0.05 } } fields { name: "num_candidate_attributes_ratio" value { real: 1 } }
[INFO gradient_boosted_trees.cc:2891] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.420508
[INFO gradient_boosted_trees.cc:233] Truncates the model to 890 tree(s) i.e. 89  iteration(s).
[INFO gradient_boosted_trees.cc:267] Final model num-trees:89 valid-loss:0.420508 valid-accuracy:0.830357
[INFO hyperparameters_optimizer.cc:562] [35/50] Score: -0.420508 / -0.412312 HParams: fields { name: "min_examples" value { integer: 10 } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 6 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "shrinkage" value { real: 0.05 } } fields { name: "num_candidate_attributes_ratio" value { real: 1 } }
[INFO gradient_boosted_trees.cc:2891] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.541539
[INFO gradient_boosted_trees.cc:233] Truncates the model to 340 tree(s) i.e. 34  iteration(s).
[INFO gradient_boosted_trees.cc:267] Final model num-trees:34 valid-loss:0.541539 valid-accuracy:0.794643
[INFO hyperparameters_optimizer.cc:562] [36/50] Score: -0.541539 / -0.412312 HParams: fields { name: "min_examples" value { integer: 2 } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 64 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "shrinkage" value { real: 0.1 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.9 } }
[INFO gradient_boosted_trees.cc:2891] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.474173
[INFO gradient_boosted_trees.cc:233] Truncates the model to 2110 tree(s) i.e. 211  iteration(s).
[INFO gradient_boosted_trees.cc:267] Final model num-trees:211 valid-loss:0.474173 valid-accuracy:0.794643
[INFO hyperparameters_optimizer.cc:562] [37/50] Score: -0.474173 / -0.412312 HParams: fields { name: "min_examples" value { integer: 5 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 4 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "shrinkage" value { real: 0.02 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.9 } }
[INFO gradient_boosted_trees.cc:2891] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.470248
[INFO gradient_boosted_trees.cc:233] Truncates the model to 1480 tree(s) i.e. 148  iteration(s).
[INFO gradient_boosted_trees.cc:267] Final model num-trees:148 valid-loss:0.470248 valid-accuracy:0.821429
[INFO hyperparameters_optimizer.cc:562] [38/50] Score: -0.470248 / -0.412312 HParams: fields { name: "min_examples" value { integer: 7 } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 128 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "shrinkage" value { real: 0.02 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.2 } }
[INFO gradient_boosted_trees.cc:2891] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.508392
[INFO gradient_boosted_trees.cc:233] Truncates the model to 840 tree(s) i.e. 84  iteration(s).
[INFO gradient_boosted_trees.cc:267] Final model num-trees:84 valid-loss:0.508392 valid-accuracy:0.803571
[INFO hyperparameters_optimizer.cc:562] [39/50] Score: -0.508392 / -0.412312 HParams: fields { name: "min_examples" value { integer: 2 } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 6 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "shrinkage" value { real: 0.05 } } fields { name: "num_candidate_attributes_ratio" value { real: 1 } }
[INFO gradient_boosted_trees.cc:1446]     num-trees:150 train-loss:0.055142 train-accuracy:1.000000 valid-loss:0.512609 valid-accuracy:0.821429
[INFO gradient_boosted_trees.cc:2891] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.429937
[INFO gradient_boosted_trees.cc:233] Truncates the model to 2070 tree(s) i.e. 207  iteration(s).
[INFO gradient_boosted_trees.cc:267] Final model num-trees:207 valid-loss:0.429937 valid-accuracy:0.785714
[INFO hyperparameters_optimizer.cc:562] [40/50] Score: -0.429937 / -0.412312 HParams: fields { name: "min_examples" value { integer: 10 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 64 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "shrinkage" value { real: 0.02 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.2 } }
[INFO gradient_boosted_trees.cc:2891] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.441083
[INFO gradient_boosted_trees.cc:233] Truncates the model to 1720 tree(s) i.e. 172  iteration(s).
[INFO gradient_boosted_trees.cc:267] Final model num-trees:172 valid-loss:0.441083 valid-accuracy:0.830357
[INFO hyperparameters_optimizer.cc:562] [41/50] Score: -0.441083 / -0.412312 HParams: fields { name: "min_examples" value { integer: 7 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 128 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "shrinkage" value { real: 0.02 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.5 } }
[INFO gradient_boosted_trees.cc:2891] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.50387
[INFO gradient_boosted_trees.cc:233] Truncates the model to 1660 tree(s) i.e. 166  iteration(s).
[INFO gradient_boosted_trees.cc:267] Final model num-trees:166 valid-loss:0.503870 valid-accuracy:0.803571
[INFO hyperparameters_optimizer.cc:562] [42/50] Score: -0.50387 / -0.412312 HParams: fields { name: "min_examples" value { integer: 5 } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 6 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "shrinkage" value { real: 0.02 } } fields { name: "num_candidate_attributes_ratio" value { real: 1 } }
[INFO gradient_boosted_trees.cc:2891] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.448066
[INFO gradient_boosted_trees.cc:233] Truncates the model to 2080 tree(s) i.e. 208  iteration(s).
[INFO gradient_boosted_trees.cc:267] Final model num-trees:208 valid-loss:0.448066 valid-accuracy:0.812500
[INFO hyperparameters_optimizer.cc:562] [43/50] Score: -0.448066 / -0.412312 HParams: fields { name: "min_examples" value { integer: 2 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 32 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "shrinkage" value { real: 0.02 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.2 } }
[INFO gradient_boosted_trees.cc:2891] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.599439
[INFO gradient_boosted_trees.cc:233] Truncates the model to 1220 tree(s) i.e. 122  iteration(s).
[INFO gradient_boosted_trees.cc:267] Final model num-trees:122 valid-loss:0.599439 valid-accuracy:0.803571
[INFO hyperparameters_optimizer.cc:562] [44/50] Score: -0.599439 / -0.412312 HParams: fields { name: "min_examples" value { integer: 2 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 32 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "shrinkage" value { real: 0.02 } } fields { name: "num_candidate_attributes_ratio" value { real: 1 } }
[INFO gradient_boosted_trees.cc:2891] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.524993
[INFO gradient_boosted_trees.cc:233] Truncates the model to 1760 tree(s) i.e. 176  iteration(s).
[INFO gradient_boosted_trees.cc:267] Final model num-trees:176 valid-loss:0.524993 valid-accuracy:0.785714
[INFO hyperparameters_optimizer.cc:562] [45/50] Score: -0.524993 / -0.412312 HParams: fields { name: "min_examples" value { integer: 2 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 16 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "shrinkage" value { real: 0.02 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.9 } }
[INFO gradient_boosted_trees.cc:2891] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.444747
[INFO gradient_boosted_trees.cc:233] Truncates the model to 1870 tree(s) i.e. 187  iteration(s).
[INFO gradient_boosted_trees.cc:267] Final model num-trees:187 valid-loss:0.444747 valid-accuracy:0.785714
[INFO hyperparameters_optimizer.cc:562] [46/50] Score: -0.444747 / -0.412312 HParams: fields { name: "min_examples" value { integer: 5 } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 8 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "shrinkage" value { real: 0.02 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.2 } }
[INFO gradient_boosted_trees.cc:2891] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.502136
[INFO gradient_boosted_trees.cc:233] Truncates the model to 1870 tree(s) i.e. 187  iteration(s).
[INFO gradient_boosted_trees.cc:267] Final model num-trees:187 valid-loss:0.502136 valid-accuracy:0.812500
[INFO hyperparameters_optimizer.cc:562] [47/50] Score: -0.502136 / -0.412312 HParams: fields { name: "min_examples" value { integer: 5 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 16 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "shrinkage" value { real: 0.02 } } fields { name: "num_candidate_attributes_ratio" value { real: 1 } }
[INFO gradient_boosted_trees.cc:2891] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.517424
[INFO gradient_boosted_trees.cc:233] Truncates the model to 1660 tree(s) i.e. 166  iteration(s).
[INFO gradient_boosted_trees.cc:267] Final model num-trees:166 valid-loss:0.517424 valid-accuracy:0.812500
[INFO hyperparameters_optimizer.cc:562] [48/50] Score: -0.517424 / -0.412312 HParams: fields { name: "min_examples" value { integer: 2 } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 32 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "shrinkage" value { real: 0.02 } } fields { name: "num_candidate_attributes_ratio" value { real: 1 } }
[INFO gradient_boosted_trees.cc:2891] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.461539
[INFO gradient_boosted_trees.cc:233] Truncates the model to 1440 tree(s) i.e. 144  iteration(s).
[INFO gradient_boosted_trees.cc:267] Final model num-trees:144 valid-loss:0.461539 valid-accuracy:0.803571
[INFO hyperparameters_optimizer.cc:562] [49/50] Score: -0.461539 / -0.412312 HParams: fields { name: "min_examples" value { integer: 7 } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 8 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "shrinkage" value { real: 0.02 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.5 } }
[INFO gradient_boosted_trees.cc:2891] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.456647
[INFO gradient_boosted_trees.cc:233] Truncates the model to 1940 tree(s) i.e. 194  iteration(s).
[INFO gradient_boosted_trees.cc:267] Final model num-trees:194 valid-loss:0.456647 valid-accuracy:0.830357
[INFO hyperparameters_optimizer.cc:562] [50/50] Score: -0.456647 / -0.412312 HParams: fields { name: "min_examples" value { integer: 10 } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 128 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "shrinkage" value { real: 0.02 } } fields { name: "num_candidate_attributes_ratio" value { real: 1 } }
[INFO hyperparameters_optimizer.cc:216] Best hyperparameters:
fields {
  name: "min_examples"
  value {
    integer: 7
  }
}
fields {
  name: "categorical_algorithm"
  value {
    categorical: "CART"
  }
}
fields {
  name: "growing_strategy"
  value {
    categorical: "BEST_FIRST_GLOBAL"
  }
}
fields {
  name: "max_num_nodes"
  value {
    integer: 16
  }
}
fields {
  name: "use_hessian_gain"
  value {
    categorical: "false"
  }
}
fields {
  name: "shrinkage"
  value {
    real: 0.05
  }
}
fields {
  name: "num_candidate_attributes_ratio"
  value {
    real: 0.2
  }
}

[INFO kernel.cc:961] Export model in log directory: /tmpfs/tmp/tmpp2l5nm7g with prefix 26da4adf53374195
[INFO kernel.cc:978] Save model in resources
[INFO kernel.cc:1176] Loading model from path /tmpfs/tmp/tmpp2l5nm7g/model/ with prefix 26da4adf53374195
[INFO decision_forest.cc:639] Model loaded with 1010 root(s), 31238 node(s), and 76 input feature(s).
[INFO abstract_model.cc:1248] Engine "GradientBoostedTreesGeneric" built
[INFO kernel.cc:1022] Use fast generic engine
Model trained in 0:01:08.348700
Compiling model...
Model compiled.
CPU times: user 22min 54s, sys: 8min 8s, total: 31min 3s
Wall time: 1min 9s
<keras.callbacks.History at 0x7ff500e4ffa0>
# Evaluate the model
tuned_model.compile(["accuracy"])
tuned_test_accuracy = tuned_model.evaluate(test_ds, return_dict=True, verbose=0)["accuracy"]
print(f"Test accuracy with the TF-DF hyper-parameter tuner: {tuned_test_accuracy:.4f}")
Test accuracy with the TF-DF hyper-parameter tuner: 0.8163

The hyper-parameters and objective scores of the trials are available in the model inspector. The score value is always maximized. In this example, the score is the negative log loss on the validation dataset (selected automatically).

# Display the tuning logs.
tuning_logs = tuned_model.make_inspector().tuning_logs()
tuning_logs.head()

The single rows with best=True is the one used in the final model.

# Best hyper-parameters.
tuning_logs[tuning_logs.best].iloc[0]
score                                     -0.412312
evaluation_time                           29.209859
best                                           True
min_examples                                      7
categorical_algorithm                          CART
growing_strategy                  BEST_FIRST_GLOBAL
max_depth                                       NaN
use_hessian_gain                              false
shrinkage                                      0.05
num_candidate_attributes_ratio                  0.2
max_num_nodes                                  16.0
Name: 24, dtype: object

Next, we plot the evaluation of the best score during the tuning.

plt.figure(figsize=(10, 5))
plt.plot(tuning_logs["score"], label="current trial")
plt.plot(tuning_logs["score"].cummax(), label="best trial")
plt.xlabel("Tuning step")
plt.ylabel("Tuning score")
plt.legend()
plt.show()

png

Training a model with Keras Tuner (Alternative approach)

TensorFlow Decision Forests is based on the Keras framework, and it is compatible with the Keras tuner.

Currently, the TF-DF Tuner and the Keras Tuner are complementary.

TF-DF Tuner

  • Automatic configuration of the objective.
  • Automatic extraction of validation dataset (if needed).
  • Support model self evaluation (e.g. out-of-bag evaluation).
  • Distributed hyper-parameter tuning.
  • Shared dataset access in between the trials: The tensorflow dataset is read only once, speeding-up tuning significantly on small datasets.

Keras Tuner

  • Support tuning of the pre-processing parameters.
  • Support hyper-band optimizer.
  • Support custom objectives.

Let's tune a TF-DF model using the Keras tuner.

# Install the Keras tuner
!pip install keras-tuner -U -qq
import keras_tuner as kt
%%time

def build_model(hp):
  """Creates a model."""

  model = tfdf.keras.GradientBoostedTreesModel(
      min_examples=hp.Choice("min_examples", [2, 5, 7, 10]),
      categorical_algorithm=hp.Choice("categorical_algorithm", ["CART", "RANDOM"]),
      max_depth=hp.Choice("max_depth", [4, 5, 6, 7]),
      # The keras tuner convert automaticall boolean parameters to integers.
      use_hessian_gain=bool(hp.Choice("use_hessian_gain", [True, False])),
      shrinkage=hp.Choice("shrinkage", [0.02, 0.05, 0.10, 0.15]),
      num_candidate_attributes_ratio=hp.Choice("num_candidate_attributes_ratio", [0.2, 0.5, 0.9, 1.0]),
  )

  # Optimize the model accuracy as computed on the validation dataset.
  model.compile(metrics=["accuracy"])
  return model

keras_tuner = kt.RandomSearch(
    build_model,
    objective="val_accuracy",
    max_trials=50,
    overwrite=True,
    directory="/tmp/keras_tuning")

# Important: The tuning should not be done on the test dataset.

# Extract a validation dataset from the training dataset. The new training
# dataset is called the "sub-training-dataset".
sub_train_df, sub_valid_df = split_dataset(train_df)
sub_train_ds = tfdf.keras.pd_dataframe_to_tf_dataset(sub_train_df, label=label)
sub_valid_ds = tfdf.keras.pd_dataframe_to_tf_dataset(sub_valid_df, label=label)

# Tune the model
keras_tuner.search(sub_train_ds, validation_data=sub_valid_ds)
Trial 50 Complete [00h 00m 09s]
val_accuracy: 0.8030690550804138

Best val_accuracy So Far: 0.8542199730873108
Total elapsed time: 00h 06m 32s
INFO:tensorflow:Oracle triggered exit
INFO:tensorflow:Oracle triggered exit
CPU times: user 26min 26s, sys: 10min 42s, total: 37min 9s
Wall time: 6min 32s

The best hyper-parameter are available with get_best_hyperparameters:

# Tune the model
best_hyper_parameters = keras_tuner.get_best_hyperparameters()[0].values
print("Best hyper-parameters:", keras_tuner.get_best_hyperparameters()[0].values)
Best hyper-parameters: {'min_examples': 2, 'categorical_algorithm': 'RANDOM', 'max_depth': 5, 'use_hessian_gain': 1, 'shrinkage': 0.02, 'num_candidate_attributes_ratio': 0.2}

The model should be re-trained with the best hyper-parameters:

%set_cell_height 300
# Train the model
# The keras tuner convert automaticall boolean parameters to integers.
best_hyper_parameters["use_hessian_gain"] = bool(best_hyper_parameters["use_hessian_gain"])
best_model = tfdf.keras.GradientBoostedTreesModel(**best_hyper_parameters)
best_model.fit(train_ds, verbose=2)
<IPython.core.display.Javascript object>
Warning: The `num_threads` constructor argument is not set and the number of CPU is os.cpu_count()=32 > 32. Setting num_threads to 32. Set num_threads manually to use more than 32 cpus.
WARNING:absl:The `num_threads` constructor argument is not set and the number of CPU is os.cpu_count()=32 > 32. Setting num_threads to 32. Set num_threads manually to use more than 32 cpus.
Use /tmpfs/tmp/tmplsa9hl0b as temporary training directory
Reading training dataset...
Training tensor examples:
Features: {'coef_0': <tf.Tensor 'data:0' shape=(None,) dtype=float64>, 'coef_1': <tf.Tensor 'data_1:0' shape=(None,) dtype=float64>, 'coef_2': <tf.Tensor 'data_12:0' shape=(None,) dtype=float64>, 'coef_3': <tf.Tensor 'data_23:0' shape=(None,) dtype=float64>, 'coef_4': <tf.Tensor 'data_34:0' shape=(None,) dtype=float64>, 'coef_5': <tf.Tensor 'data_45:0' shape=(None,) dtype=float64>, 'coef_6': <tf.Tensor 'data_56:0' shape=(None,) dtype=float64>, 'coef_7': <tf.Tensor 'data_67:0' shape=(None,) dtype=float64>, 'coef_8': <tf.Tensor 'data_74:0' shape=(None,) dtype=float64>, 'coef_9': <tf.Tensor 'data_75:0' shape=(None,) dtype=float64>, 'coef_10': <tf.Tensor 'data_2:0' shape=(None,) dtype=float64>, 'coef_11': <tf.Tensor 'data_3:0' shape=(None,) dtype=float64>, 'coef_12': <tf.Tensor 'data_4:0' shape=(None,) dtype=float64>, 'coef_13': <tf.Tensor 'data_5:0' shape=(None,) dtype=float64>, 'coef_14': <tf.Tensor 'data_6:0' shape=(None,) dtype=float64>, 'coef_15': <tf.Tensor 'data_7:0' shape=(None,) dtype=float64>, 'coef_16': <tf.Tensor 'data_8:0' shape=(None,) dtype=float64>, 'coef_17': <tf.Tensor 'data_9:0' shape=(None,) dtype=float64>, 'coef_18': <tf.Tensor 'data_10:0' shape=(None,) dtype=float64>, 'coef_19': <tf.Tensor 'data_11:0' shape=(None,) dtype=float64>, 'coef_20': <tf.Tensor 'data_13:0' shape=(None,) dtype=float64>, 'coef_21': <tf.Tensor 'data_14:0' shape=(None,) dtype=float64>, 'coef_22': <tf.Tensor 'data_15:0' shape=(None,) dtype=float64>, 'coef_23': <tf.Tensor 'data_16:0' shape=(None,) dtype=float64>, 'coef_24': <tf.Tensor 'data_17:0' shape=(None,) dtype=float64>, 'coef_25': <tf.Tensor 'data_18:0' shape=(None,) dtype=float64>, 'coef_26': <tf.Tensor 'data_19:0' shape=(None,) dtype=float64>, 'coef_27': <tf.Tensor 'data_20:0' shape=(None,) dtype=float64>, 'coef_28': <tf.Tensor 'data_21:0' shape=(None,) dtype=float64>, 'coef_29': <tf.Tensor 'data_22:0' shape=(None,) dtype=float64>, 'coef_30': <tf.Tensor 'data_24:0' shape=(None,) dtype=float64>, 'coef_31': <tf.Tensor 'data_25:0' shape=(None,) dtype=float64>, 'coef_32': <tf.Tensor 'data_26:0' shape=(None,) dtype=float64>, 'coef_33': <tf.Tensor 'data_27:0' shape=(None,) dtype=float64>, 'coef_34': <tf.Tensor 'data_28:0' shape=(None,) dtype=float64>, 'coef_35': <tf.Tensor 'data_29:0' shape=(None,) dtype=float64>, 'coef_36': <tf.Tensor 'data_30:0' shape=(None,) dtype=float64>, 'coef_37': <tf.Tensor 'data_31:0' shape=(None,) dtype=float64>, 'coef_38': <tf.Tensor 'data_32:0' shape=(None,) dtype=float64>, 'coef_39': <tf.Tensor 'data_33:0' shape=(None,) dtype=float64>, 'coef_40': <tf.Tensor 'data_35:0' shape=(None,) dtype=float64>, 'coef_41': <tf.Tensor 'data_36:0' shape=(None,) dtype=float64>, 'coef_42': <tf.Tensor 'data_37:0' shape=(None,) dtype=float64>, 'coef_43': <tf.Tensor 'data_38:0' shape=(None,) dtype=float64>, 'coef_44': <tf.Tensor 'data_39:0' shape=(None,) dtype=float64>, 'coef_45': <tf.Tensor 'data_40:0' shape=(None,) dtype=float64>, 'coef_46': <tf.Tensor 'data_41:0' shape=(None,) dtype=float64>, 'coef_47': <tf.Tensor 'data_42:0' shape=(None,) dtype=float64>, 'coef_48': <tf.Tensor 'data_43:0' shape=(None,) dtype=float64>, 'coef_49': <tf.Tensor 'data_44:0' shape=(None,) dtype=float64>, 'coef_50': <tf.Tensor 'data_46:0' shape=(None,) dtype=float64>, 'coef_51': <tf.Tensor 'data_47:0' shape=(None,) dtype=float64>, 'coef_52': <tf.Tensor 'data_48:0' shape=(None,) dtype=float64>, 'coef_53': <tf.Tensor 'data_49:0' shape=(None,) dtype=float64>, 'coef_54': <tf.Tensor 'data_50:0' shape=(None,) dtype=float64>, 'coef_55': <tf.Tensor 'data_51:0' shape=(None,) dtype=float64>, 'coef_56': <tf.Tensor 'data_52:0' shape=(None,) dtype=float64>, 'coef_57': <tf.Tensor 'data_53:0' shape=(None,) dtype=float64>, 'coef_58': <tf.Tensor 'data_54:0' shape=(None,) dtype=float64>, 'coef_59': <tf.Tensor 'data_55:0' shape=(None,) dtype=float64>, 'coef_60': <tf.Tensor 'data_57:0' shape=(None,) dtype=float64>, 'coef_61': <tf.Tensor 'data_58:0' shape=(None,) dtype=float64>, 'coef_62': <tf.Tensor 'data_59:0' shape=(None,) dtype=float64>, 'coef_63': <tf.Tensor 'data_60:0' shape=(None,) dtype=float64>, 'coef_64': <tf.Tensor 'data_61:0' shape=(None,) dtype=float64>, 'coef_65': <tf.Tensor 'data_62:0' shape=(None,) dtype=float64>, 'coef_66': <tf.Tensor 'data_63:0' shape=(None,) dtype=float64>, 'coef_67': <tf.Tensor 'data_64:0' shape=(None,) dtype=float64>, 'coef_68': <tf.Tensor 'data_65:0' shape=(None,) dtype=float64>, 'coef_69': <tf.Tensor 'data_66:0' shape=(None,) dtype=float64>, 'coef_70': <tf.Tensor 'data_68:0' shape=(None,) dtype=float64>, 'coef_71': <tf.Tensor 'data_69:0' shape=(None,) dtype=float64>, 'coef_72': <tf.Tensor 'data_70:0' shape=(None,) dtype=float64>, 'coef_73': <tf.Tensor 'data_71:0' shape=(None,) dtype=float64>, 'coef_74': <tf.Tensor 'data_72:0' shape=(None,) dtype=float64>, 'coef_75': <tf.Tensor 'data_73:0' shape=(None,) dtype=float64>}
Label: Tensor("data_76:0", shape=(None,), dtype=int64)
Weights: None
Normalized tensor features:
 {'coef_0': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast:0' shape=(None,) dtype=float32>), 'coef_1': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_1:0' shape=(None,) dtype=float32>), 'coef_2': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_2:0' shape=(None,) dtype=float32>), 'coef_3': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_3:0' shape=(None,) dtype=float32>), 'coef_4': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_4:0' shape=(None,) dtype=float32>), 'coef_5': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_5:0' shape=(None,) dtype=float32>), 'coef_6': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_6:0' shape=(None,) dtype=float32>), 'coef_7': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_7:0' shape=(None,) dtype=float32>), 'coef_8': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_8:0' shape=(None,) dtype=float32>), 'coef_9': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_9:0' shape=(None,) dtype=float32>), 'coef_10': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_10:0' shape=(None,) dtype=float32>), 'coef_11': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_11:0' shape=(None,) dtype=float32>), 'coef_12': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_12:0' shape=(None,) dtype=float32>), 'coef_13': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_13:0' shape=(None,) dtype=float32>), 'coef_14': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_14:0' shape=(None,) dtype=float32>), 'coef_15': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_15:0' shape=(None,) dtype=float32>), 'coef_16': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_16:0' shape=(None,) dtype=float32>), 'coef_17': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_17:0' shape=(None,) dtype=float32>), 'coef_18': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_18:0' shape=(None,) dtype=float32>), 'coef_19': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_19:0' shape=(None,) dtype=float32>), 'coef_20': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_20:0' shape=(None,) dtype=float32>), 'coef_21': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_21:0' shape=(None,) dtype=float32>), 'coef_22': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_22:0' shape=(None,) dtype=float32>), 'coef_23': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_23:0' shape=(None,) dtype=float32>), 'coef_24': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_24:0' shape=(None,) dtype=float32>), 'coef_25': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_25:0' shape=(None,) dtype=float32>), 'coef_26': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_26:0' shape=(None,) dtype=float32>), 'coef_27': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_27:0' shape=(None,) dtype=float32>), 'coef_28': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_28:0' shape=(None,) dtype=float32>), 'coef_29': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_29:0' shape=(None,) dtype=float32>), 'coef_30': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_30:0' shape=(None,) dtype=float32>), 'coef_31': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_31:0' shape=(None,) dtype=float32>), 'coef_32': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_32:0' shape=(None,) dtype=float32>), 'coef_33': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_33:0' shape=(None,) dtype=float32>), 'coef_34': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_34:0' shape=(None,) dtype=float32>), 'coef_35': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_35:0' shape=(None,) dtype=float32>), 'coef_36': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_36:0' shape=(None,) dtype=float32>), 'coef_37': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_37:0' shape=(None,) dtype=float32>), 'coef_38': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_38:0' shape=(None,) dtype=float32>), 'coef_39': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_39:0' shape=(None,) dtype=float32>), 'coef_40': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_40:0' shape=(None,) dtype=float32>), 'coef_41': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_41:0' shape=(None,) dtype=float32>), 'coef_42': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_42:0' shape=(None,) dtype=float32>), 'coef_43': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_43:0' shape=(None,) dtype=float32>), 'coef_44': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_44:0' shape=(None,) dtype=float32>), 'coef_45': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_45:0' shape=(None,) dtype=float32>), 'coef_46': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_46:0' shape=(None,) dtype=float32>), 'coef_47': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_47:0' shape=(None,) dtype=float32>), 'coef_48': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_48:0' shape=(None,) dtype=float32>), 'coef_49': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_49:0' shape=(None,) dtype=float32>), 'coef_50': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_50:0' shape=(None,) dtype=float32>), 'coef_51': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_51:0' shape=(None,) dtype=float32>), 'coef_52': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_52:0' shape=(None,) dtype=float32>), 'coef_53': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_53:0' shape=(None,) dtype=float32>), 'coef_54': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_54:0' shape=(None,) dtype=float32>), 'coef_55': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_55:0' shape=(None,) dtype=float32>), 'coef_56': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_56:0' shape=(None,) dtype=float32>), 'coef_57': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_57:0' shape=(None,) dtype=float32>), 'coef_58': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_58:0' shape=(None,) dtype=float32>), 'coef_59': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_59:0' shape=(None,) dtype=float32>), 'coef_60': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_60:0' shape=(None,) dtype=float32>), 'coef_61': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_61:0' shape=(None,) dtype=float32>), 'coef_62': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_62:0' shape=(None,) dtype=float32>), 'coef_63': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_63:0' shape=(None,) dtype=float32>), 'coef_64': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_64:0' shape=(None,) dtype=float32>), 'coef_65': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_65:0' shape=(None,) dtype=float32>), 'coef_66': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_66:0' shape=(None,) dtype=float32>), 'coef_67': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_67:0' shape=(None,) dtype=float32>), 'coef_68': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_68:0' shape=(None,) dtype=float32>), 'coef_69': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_69:0' shape=(None,) dtype=float32>), 'coef_70': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_70:0' shape=(None,) dtype=float32>), 'coef_71': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_71:0' shape=(None,) dtype=float32>), 'coef_72': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_72:0' shape=(None,) dtype=float32>), 'coef_73': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_73:0' shape=(None,) dtype=float32>), 'coef_74': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_74:0' shape=(None,) dtype=float32>), 'coef_75': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_75:0' shape=(None,) dtype=float32>)}
Training dataset read in 0:00:00.824575. Found 1363 examples.
Training model...
[INFO kernel.cc:813] Start Yggdrasil model training
[INFO kernel.cc:814] Collect training examples
[INFO kernel.cc:422] Number of batches: 2
[INFO kernel.cc:423] Number of examples: 1363
[INFO kernel.cc:836] Training dataset:
Number of records: 1363
Number of columns: 77

Number of columns by type:
    NUMERICAL: 76 (98.7013%)
    CATEGORICAL: 1 (1.2987%)

Columns:

NUMERICAL: 76 (98.7013%)
    0: "coef_0" NUMERICAL mean:0.183378 min:0.00492771 max:0.395772 sd:0.0922694
    1: "coef_1" NUMERICAL mean:0.375241 min:0.0140609 max:0.771281 sd:0.174751
    2: "coef_10" NUMERICAL mean:0.155655 min:0.00329301 max:0.420803 sd:0.085883
    3: "coef_11" NUMERICAL mean:0.164945 min:0.00409906 max:0.488068 sd:0.0820937
    4: "coef_12" NUMERICAL mean:0.163995 min:0.00142364 max:0.471881 sd:0.0910029
    5: "coef_13" NUMERICAL mean:0.192368 min:0.00590215 max:0.56481 sd:0.110566
    6: "coef_14" NUMERICAL mean:0.142261 min:0.0105102 max:0.332034 sd:0.0634995
    7: "coef_15" NUMERICAL mean:0.1153 min:0.00290088 max:0.366277 sd:0.058216
    8: "coef_16" NUMERICAL mean:0.127758 min:0.00287311 max:0.462718 sd:0.0671196
    9: "coef_17" NUMERICAL mean:0.130601 min:0.00208767 max:0.396381 sd:0.0784973
    10: "coef_18" NUMERICAL mean:0.118898 min:0.00230379 max:0.440113 sd:0.064675
    11: "coef_19" NUMERICAL mean:0.116073 min:0.00783665 max:0.345249 sd:0.0527643
    12: "coef_2" NUMERICAL mean:0.269396 min:0.00562382 max:0.665372 sd:0.138844
    13: "coef_20" NUMERICAL mean:0.107888 min:0.00243648 max:0.315663 sd:0.0520058
    14: "coef_21" NUMERICAL mean:0.108169 min:0.00260839 max:0.337783 sd:0.0528892
    15: "coef_22" NUMERICAL mean:0.0979448 min:0.0026094 max:0.332622 sd:0.057335
    16: "coef_23" NUMERICAL mean:0.113556 min:0.00158288 max:0.34269 sd:0.0627629
    17: "coef_24" NUMERICAL mean:0.105542 min:0.00182609 max:0.280131 sd:0.0463991
    18: "coef_25" NUMERICAL mean:0.0982893 min:0.00110629 max:0.312352 sd:0.0483484
    19: "coef_26" NUMERICAL mean:0.0951503 min:0.00527843 max:0.320455 sd:0.0476453
    20: "coef_27" NUMERICAL mean:0.0918396 min:0.00356535 max:0.32458 sd:0.0503912
    21: "coef_28" NUMERICAL mean:0.091518 min:0.00017629 max:0.312097 sd:0.0510217
    22: "coef_29" NUMERICAL mean:0.0946162 min:0.0044099 max:0.262548 sd:0.0452071
    23: "coef_3" NUMERICAL mean:0.264062 min:0.019079 max:0.748012 sd:0.109897
    24: "coef_30" NUMERICAL mean:0.0937955 min:0.00282674 max:0.247583 sd:0.044382
    25: "coef_31" NUMERICAL mean:0.0864024 min:0.00132908 max:0.289284 sd:0.04254
    26: "coef_32" NUMERICAL mean:0.0819615 min:0.00025893 max:0.25609 sd:0.0463068
    27: "coef_33" NUMERICAL mean:0.0831697 min:0.00256918 max:0.314669 sd:0.0447715
    28: "coef_34" NUMERICAL mean:0.0912174 min:0.0043182 max:0.278892 sd:0.0430243
    29: "coef_35" NUMERICAL mean:0.0856046 min:0.00158334 max:0.251377 sd:0.0414026
    30: "coef_36" NUMERICAL mean:0.0810899 min:0.00228121 max:0.257029 sd:0.0406632
    31: "coef_37" NUMERICAL mean:0.0773527 min:0.00046895 max:0.291041 sd:0.0417818
    32: "coef_38" NUMERICAL mean:0.0771318 min:0.00434419 max:0.272558 sd:0.0414008
    33: "coef_39" NUMERICAL mean:0.0851039 min:0.00480153 max:0.247084 sd:0.0411211
    34: "coef_4" NUMERICAL mean:0.300226 min:0.00209035 max:0.796487 sd:0.155972
    35: "coef_40" NUMERICAL mean:0.0819136 min:0.00087734 max:0.243434 sd:0.0394354
    36: "coef_41" NUMERICAL mean:0.0772351 min:0.00317318 max:0.22198 sd:0.0375504
    37: "coef_42" NUMERICAL mean:0.072984 min:0.00149386 max:0.235539 sd:0.0391918
    38: "coef_43" NUMERICAL mean:0.0716651 min:0.00113918 max:0.245737 sd:0.0389648
    39: "coef_44" NUMERICAL mean:0.0842747 min:0.00308002 max:0.230327 sd:0.0410869
    40: "coef_45" NUMERICAL mean:0.0782796 min:0.00519238 max:0.231811 sd:0.037137
    41: "coef_46" NUMERICAL mean:0.0739795 min:0.00223798 max:0.222058 sd:0.0386209
    42: "coef_47" NUMERICAL mean:0.0726674 min:0.00325095 max:0.227016 sd:0.0379209
    43: "coef_48" NUMERICAL mean:0.0861999 min:0.00440658 max:0.266345 sd:0.042239
    44: "coef_49" NUMERICAL mean:0.0825943 min:0.00115433 max:0.242921 sd:0.0408022
    45: "coef_5" NUMERICAL mean:0.158264 min:0.00267619 max:0.483342 sd:0.0890157
    46: "coef_50" NUMERICAL mean:0.0795964 min:0.00187021 max:0.247757 sd:0.0417273
    47: "coef_51" NUMERICAL mean:0.0783834 min:0.00227356 max:0.233935 sd:0.0418267
    48: "coef_52" NUMERICAL mean:0.0929331 min:0.00345921 max:0.250377 sd:0.0446751
    49: "coef_53" NUMERICAL mean:0.0909221 min:0.00157472 max:0.257 sd:0.044437
    50: "coef_54" NUMERICAL mean:0.0862077 min:0.00255567 max:0.253345 sd:0.0474379
    51: "coef_55" NUMERICAL mean:0.0851051 min:0.00156102 max:0.241507 sd:0.0435042
    52: "coef_56" NUMERICAL mean:0.101834 min:0.00166723 max:0.275385 sd:0.0498542
    53: "coef_57" NUMERICAL mean:0.100743 min:0.00266166 max:0.287096 sd:0.0499937
    54: "coef_58" NUMERICAL mean:0.0894441 min:0.00173879 max:0.249129 sd:0.0484976
    55: "coef_59" NUMERICAL mean:0.0946268 min:0.00189643 max:0.284695 sd:0.0529451
    56: "coef_6" NUMERICAL mean:0.367482 min:0.0119186 max:0.712186 sd:0.146432
    57: "coef_60" NUMERICAL mean:0.117082 min:0.00347996 max:0.300376 sd:0.0589656
    58: "coef_61" NUMERICAL mean:0.108374 min:0.00661185 max:0.319926 sd:0.0532069
    59: "coef_62" NUMERICAL mean:0.104477 min:0.00227164 max:0.31959 sd:0.0610118
    60: "coef_63" NUMERICAL mean:0.109393 min:0.00495736 max:0.351732 sd:0.0561484
    61: "coef_64" NUMERICAL mean:0.118121 min:0.00188382 max:0.427654 sd:0.0634669
    62: "coef_65" NUMERICAL mean:0.131464 min:0.00458132 max:0.369027 sd:0.0666763
    63: "coef_66" NUMERICAL mean:0.120773 min:0.00523224 max:0.397127 sd:0.0686975
    64: "coef_67" NUMERICAL mean:0.128344 min:0.00400679 max:0.418783 sd:0.0642292
    65: "coef_68" NUMERICAL mean:0.158361 min:0.00600492 max:0.47947 sd:0.091666
    66: "coef_69" NUMERICAL mean:0.180439 min:0.00813938 max:0.490737 sd:0.0859493
    67: "coef_7" NUMERICAL mean:0.247872 min:0.0092282 max:0.542303 sd:0.124564
    68: "coef_70" NUMERICAL mean:0.147855 min:0.00478888 max:0.456282 sd:0.0869718
    69: "coef_71" NUMERICAL mean:0.166876 min:0.00778369 max:0.457965 sd:0.091042
    70: "coef_72" NUMERICAL mean:0.229906 min:0.00356929 max:0.60137 sd:0.12371
    71: "coef_73" NUMERICAL mean:0.264569 min:0.0114335 max:0.620857 sd:0.131169
    72: "coef_74" NUMERICAL mean:0.143232 min:0.00113625 max:0.510432 sd:0.0852575
    73: "coef_75" NUMERICAL mean:0.215316 min:0.00457179 max:0.562397 sd:0.123783
    74: "coef_8" NUMERICAL mean:0.291335 min:0.0133491 max:0.702383 sd:0.117268
    75: "coef_9" NUMERICAL mean:0.153057 min:0.00234154 max:0.426099 sd:0.0874617

CATEGORICAL: 1 (1.2987%)
    76: "__LABEL" CATEGORICAL integerized vocab-size:11 no-ood-item

Terminology:
    nas: Number of non-available (i.e. missing) values.
    ood: Out of dictionary.
    manually-defined: Attribute which type is manually defined by the user i.e. the type was not automatically inferred.
    tokenized: The attribute value is obtained through tokenization.
    has-dict: The attribute is attached to a string dictionary e.g. a categorical attribute stored as a string.
    vocab-size: Number of unique values.

[INFO kernel.cc:882] Configure learner
[WARNING gradient_boosted_trees.cc:1691] Subsample hyperparameter given but sampling method does not match.
[WARNING gradient_boosted_trees.cc:1704] GOSS alpha hyperparameter given but GOSS is disabled.
[WARNING gradient_boosted_trees.cc:1713] GOSS beta hyperparameter given but GOSS is disabled.
[WARNING gradient_boosted_trees.cc:1725] SelGB ratio hyperparameter given but SelGB is disabled.
[INFO kernel.cc:912] Training config:
learner: "GRADIENT_BOOSTED_TREES"
features: "coef_0"
features: "coef_1"
features: "coef_10"
features: "coef_11"
features: "coef_12"
features: "coef_13"
features: "coef_14"
features: "coef_15"
features: "coef_16"
features: "coef_17"
features: "coef_18"
features: "coef_19"
features: "coef_2"
features: "coef_20"
features: "coef_21"
features: "coef_22"
features: "coef_23"
features: "coef_24"
features: "coef_25"
features: "coef_26"
features: "coef_27"
features: "coef_28"
features: "coef_29"
features: "coef_3"
features: "coef_30"
features: "coef_31"
features: "coef_32"
features: "coef_33"
features: "coef_34"
features: "coef_35"
features: "coef_36"
features: "coef_37"
features: "coef_38"
features: "coef_39"
features: "coef_4"
features: "coef_40"
features: "coef_41"
features: "coef_42"
features: "coef_43"
features: "coef_44"
features: "coef_45"
features: "coef_46"
features: "coef_47"
features: "coef_48"
features: "coef_49"
features: "coef_5"
features: "coef_50"
features: "coef_51"
features: "coef_52"
features: "coef_53"
features: "coef_54"
features: "coef_55"
features: "coef_56"
features: "coef_57"
features: "coef_58"
features: "coef_59"
features: "coef_6"
features: "coef_60"
features: "coef_61"
features: "coef_62"
features: "coef_63"
features: "coef_64"
features: "coef_65"
features: "coef_66"
features: "coef_67"
features: "coef_68"
features: "coef_69"
features: "coef_7"
features: "coef_70"
features: "coef_71"
features: "coef_72"
features: "coef_73"
features: "coef_74"
features: "coef_75"
features: "coef_8"
features: "coef_9"
label: "__LABEL"
task: CLASSIFICATION
random_seed: 123456
metadata {
  framework: "TF Keras"
}
pure_serving_model: false
[yggdrasil_decision_forests.model.gradient_boosted_trees.proto.gradient_boosted_trees_config] {
  num_trees: 300
  decision_tree {
    max_depth: 5
    min_examples: 2
    in_split_min_examples_check: true
    keep_non_leaf_label_distribution: true
    missing_value_policy: GLOBAL_IMPUTATION
    allow_na_conditions: false
    categorical_set_greedy_forward {
      sampling: 0.1
      max_num_items: -1
      min_item_frequency: 1
    }
    growing_strategy_local {
    }
    categorical {
      random {
      }
    }
    num_candidate_attributes_ratio: 0.2
    axis_aligned_split {
    }
    internal {
      sorting_strategy: PRESORTED
    }
    uplift {
      min_examples_in_treatment: 5
      split_score: KULLBACK_LEIBLER
    }
  }
  shrinkage: 0.02
  loss: DEFAULT
  validation_set_ratio: 0.1
  validation_interval_in_trees: 1
  early_stopping: VALIDATION_LOSS_INCREASE
  early_stopping_num_trees_look_ahead: 30
  l2_regularization: 0
  lambda_loss: 1
  mart {
  }
  adapt_subsample_for_maximum_training_duration: false
  l1_regularization: 0
  use_hessian_gain: true
  l2_regularization_categorical: 1
  apply_link_function: true
  compute_permutation_variable_importance: false
  binary_focal_loss_options {
    misprediction_exponent: 2
    positive_sample_coefficient: 0.5
  }
}

[INFO kernel.cc:915] Deployment config:
cache_path: "/tmpfs/tmp/tmplsa9hl0b/working_cache"
num_threads: 32
try_resume_training: true

[INFO kernel.cc:944] Train model
[INFO gradient_boosted_trees.cc:408] Default loss set to MULTINOMIAL_LOG_LIKELIHOOD
[INFO gradient_boosted_trees.cc:1011] Training gradient boosted tree on 1363 example(s) and 76 feature(s).
[INFO gradient_boosted_trees.cc:1054] 1251 examples used for training and 112 examples used for validation
[INFO gradient_boosted_trees.cc:1444]     num-trees:1 train-loss:2.185436 train-accuracy:0.824141 valid-loss:2.207731 valid-accuracy:0.642857
[INFO gradient_boosted_trees.cc:2891] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.483995
[INFO gradient_boosted_trees.cc:1494] Create final snapshot of the model at iteration 175
[INFO gradient_boosted_trees.cc:233] Truncates the model to 1730 tree(s) i.e. 173  iteration(s).
[INFO gradient_boosted_trees.cc:267] Final model num-trees:173 valid-loss:0.483995 valid-accuracy:0.803571
[INFO kernel.cc:961] Export model in log directory: /tmpfs/tmp/tmplsa9hl0b with prefix 6280d9a2786c4833
[INFO kernel.cc:978] Save model in resources
[INFO kernel.cc:1176] Loading model from path /tmpfs/tmp/tmplsa9hl0b/model/ with prefix 6280d9a2786c4833
[INFO decision_forest.cc:639] Model loaded with 1730 root(s), 49466 node(s), and 76 input feature(s).
[INFO abstract_model.cc:1248] Engine "GradientBoostedTreesGeneric" built
[INFO kernel.cc:1022] Use fast generic engine
Model trained in 0:00:07.277813
Compiling model...
Model compiled.
<keras.callbacks.History at 0x7ff3fc2f5760>

We can then evaluate the tuned model:

# Evaluate the model
best_model.compile(["accuracy"])
tuned_test_accuracy = best_model.evaluate(test_ds, return_dict=True, verbose=0)["accuracy"]
print(f"Test accuracy with the Keras Tuner: {tuned_test_accuracy:.4f}")
Test accuracy with the Keras Tuner: 0.8006