Automated hyper-parameter tuning

View on TensorFlow.org Run in Google Colab View on GitHub Download notebook

Welcome to the Automated hyper-parameter tuning tutorial. In this colab, you will learn how to improve your models using automated hyper-parameter tuning with TensorFlow Decision Forests.

More precicely we will:

  1. Train a model without hyper-parameter tuning. This model will be used to measure the quality improvement of hyper-parameter tuning.
  2. Train a model with hyper-parameter tuning using TF-DF's tuner. The hyper-parameters to optimize will be defined manually.
  3. Train another model with hyper-parameter tuning using TF-DF's tuner. But this time, the hyper-parameters to optimize will be set automatically. This is the recommanded first approach to try when using hyper-parameter tuning.
  4. Finally, we will train a model with hyper-parameter tuning using Keras's tuner.

Introduction

A learning algorithm trains a machine learning model on a training dataset. The parameters of a learning algorithm–called "hyper-parameters"–control how the model is trained and impact its quality. Therefore, finding the best hyper-parameters is an important stage of modeling.

Some hyper-parameters are simple to configure. For example, increasing the number of trees (num_trees) in a random forest increases the quality of the model until a plateau. Therefore, setting the largest value compatible with the serving constraints (more trees means a larger model) is a valid rule of thumb. However, other hyper-parameters have a more complex interaction with the model and cannot be chosen with such a simple rule. For example, increasing the maximum tree depth (max_depth) of a gradient boosted tree model can both increase or decrease the quality of the model. Furthermore, hyper-parameters can interact between each others, and the optimal value of a hyper-parameter cannot be found in isolation.

There are three main approaches to select the hyper-parameter values:

  1. The default approach: Learning algorithms come with default values. While not ideal in all cases, those values produce reasonable results in most situations. This approach is recommended as the first approach to use in any modeling. This page lists the default values of TF Decision Forests.

  2. The template hyper-parameter approach: In addition to the default values, TF Decision Forests also exposes the hyper-parameter templates. Those are benchmark-tuned hyper-parameter values with excellent performance but high training cost (e.g. hyperparameter_template="benchmark_rank1").

  3. The manual tuning approach: You can manually test different hyper-parameter values and select the one that performs best. This guide give some advice.

  4. The automated tuning approach: A tuning algorithm can be used to find automatically the best hyper-parameter values. This approach gives often the best results and does not require expertise. The main downside of this approach is the time it takes for large datasets.

In this colab, we shows the default and automated tuning approaches with the TensorFlow Decision Forests library.

Hyper-parameter tuning algorithms

Automated tuning algorithms work by generating and evaluating a large number of hyper-parameter values. Each of those iterations is called a "trial". The evaluation of a trial is expensive as it requires to train a new model each time. At the end of the tuning, the hyper-parameter with the best evaluation is used.

Tuning algorithm are configured as follow:

The search space

The search space is the list of hyper-parameters to optimize and the values they can take. For example, the maximum depth of a tree could be optimized for values in between 1 and 32. Exploring more hyper-parameters and more possible values often leads to better models but also takes more time. The hyper-parameters hyper-parameters are listed in the documentation.

When the possible value of one hyper-parameter depends on the value of another hyper-parameter, the search space is said to be conditional.

The number of trials

The number of trials defines how many models will be trained and evaluated. Larger number of trials generally leads to better models, but takes more time.

The optimizer

The optimizer selects the next hyper-parameter to evaluate the past trial evaluations. The simplest and often reasonable optimizer is the one that selects the hyper-parameter at random.

The objective / trial score

The objective is the metric optimized by the tuner. Often, this metric is a measure of quality (e.g. accuracy, log loss) of the model evaluated on a validation dataset.

Train-valid-test

The validation dataset should be different from the training datasets: If the training and validation datasets are the same, the selected hyper-parameters will be irrelevant. The validation dataset should also be different from the testing dataset (also called holdout dataset): Because hyper-parameter tuning is a form of training, if the testing and validation datasets are the same, you are effectively training on the test dataset. In this case, you might overfit on your test dataset without a way to measure it.

Cross-validation

In the case of a small dataset, for example a dataset with less than 100k examples, hyper-parameter tuning can be coupled with cross-validation: Instead of being evaluated from a single training-test round, the objective/trial score is evaluated as the average of the metric over multiple cross-validation rounds.

Similarly as to the train-valid-and-test datasets, the cross-validation used to evaluate the objective/score during hyper-parameter tuning should be different from the cross-validation used to evaluate the quality of the model.

Out-of-bag evaluation

Some models, like Random Forests, can be evaluated on the training datasets using the "out-of-bag evaluation" method. While not as accurate as cross-validation, the "out-of-bag evaluation" is much faster than cross-validation and does not require a separate validation datasets.

In tensorflow decision forests

In TF-DF, the model "self" evaluation is always a fair way to evaluate a model. For example, an out-of-bag evaluation is used for Random Forest models while a validation dataset is used for Gradient Boosted models.

Hyper-parameter tuning with TF Decision Forests

TF-DF supports automatic hyper-parameter tuning with minimal configuration. In the next example, we will train and compare two models: One trained with default hyper-parameters, and one trained with hyper-parameter tuning.

Setup

# Install TensorFlow Dececision Forests
pip install tensorflow_decision_forests -U -qq

Install Wurlitzer. Wurlitzer is required to show the detailed training logs in colabs (with verbose=2).

pip install wurlitzer -U -qq

Import the necessary libraries.

import tensorflow_decision_forests as tfdf
import matplotlib.pyplot as plt
import pandas as pd
import tensorflow as tf
import numpy as np

The hidden code cell limits the output height in colab.

Define "set_cell_height".

Training a model without Automated hyper-parameter tuning

We will train a model on the Adult dataset available on the UCI. Let's download the dataset.

# Download a copy of the adult dataset.
wget -q https://raw.githubusercontent.com/google/yggdrasil-decision-forests/main/yggdrasil_decision_forests/test_data/dataset/adult_train.csv -O /tmp/adult_train.csv
wget -q https://raw.githubusercontent.com/google/yggdrasil-decision-forests/main/yggdrasil_decision_forests/test_data/dataset/adult_test.csv -O /tmp/adult_test.csv

Split the dataset into a training and a testing dataset.

# Load the dataset in memory
train_df = pd.read_csv("/tmp/adult_train.csv")
test_df = pd.read_csv("/tmp/adult_test.csv")

# , and convert it into a TensorFlow dataset.
train_ds = tfdf.keras.pd_dataframe_to_tf_dataset(train_df, label="income")
test_ds = tfdf.keras.pd_dataframe_to_tf_dataset(test_df, label="income")

First, we train and evaluate the quality of a Gradient Boosted Trees model trained with the default hyper-parameters.

%%time
# Train a model with default hyper-parameters
model = tfdf.keras.GradientBoostedTreesModel()
model.fit(train_ds)
Warning: The `num_threads` constructor argument is not set and the number of CPU is os.cpu_count()=32 > 32. Setting num_threads to 32. Set num_threads manually to use more than 32 cpus.
WARNING:absl:The `num_threads` constructor argument is not set and the number of CPU is os.cpu_count()=32 > 32. Setting num_threads to 32. Set num_threads manually to use more than 32 cpus.
Use /tmpfs/tmp/tmp8vxzd_gw as temporary training directory
Reading training dataset...
[WARNING 23-08-16 11:07:53.6383 UTC gradient_boosted_trees.cc:1818] "goss_alpha" set but "sampling_method" not equal to "GOSS".
[WARNING 23-08-16 11:07:53.6384 UTC gradient_boosted_trees.cc:1829] "goss_beta" set but "sampling_method" not equal to "GOSS".
[WARNING 23-08-16 11:07:53.6384 UTC gradient_boosted_trees.cc:1843] "selective_gradient_boosting_ratio" set but "sampling_method" not equal to "SELGB".
Training dataset read in 0:00:03.854321. Found 22792 examples.
Training model...
Model trained in 0:00:03.313284
Compiling model...
[INFO 23-08-16 11:08:00.8007 UTC kernel.cc:1243] Loading model from path /tmpfs/tmp/tmp8vxzd_gw/model/ with prefix 672884dfed9c4c02
[INFO 23-08-16 11:08:00.8244 UTC abstract_model.cc:1311] Engine "GradientBoostedTreesQuickScorerExtended" built
[INFO 23-08-16 11:08:00.8244 UTC kernel.cc:1075] Use fast generic engine
WARNING:tensorflow:AutoGraph could not transform <function simple_ml_inference_op_with_handle at 0x7f23da2a7ee0> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: could not get source code
To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convert
WARNING:tensorflow:AutoGraph could not transform <function simple_ml_inference_op_with_handle at 0x7f23da2a7ee0> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: could not get source code
To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convert
WARNING: AutoGraph could not transform <function simple_ml_inference_op_with_handle at 0x7f23da2a7ee0> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: could not get source code
To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convert
Model compiled.
CPU times: user 12.7 s, sys: 1.1 s, total: 13.8 s
Wall time: 8.9 s
<keras.src.callbacks.History at 0x7f24cdc1f9a0>
# Evaluate the model
model.compile(["accuracy"])
test_accuracy = model.evaluate(test_ds, return_dict=True, verbose=0)["accuracy"]
print(f"Test accuracy without hyper-parameter tuning: {test_accuracy:.4f}")
Test accuracy without hyper-parameter tuning: 0.8744

The default hyper-parameters of the model are available with the learner_params function. The definition of those parameters is available in the documentation.

print("Default hyper-parameters of the model:\n", model.learner_params)
Default hyper-parameters of the model:
 {'adapt_subsample_for_maximum_training_duration': False, 'allow_na_conditions': False, 'apply_link_function': True, 'categorical_algorithm': 'CART', 'categorical_set_split_greedy_sampling': 0.1, 'categorical_set_split_max_num_items': -1, 'categorical_set_split_min_item_frequency': 1, 'compute_permutation_variable_importance': False, 'dart_dropout': 0.01, 'early_stopping': 'LOSS_INCREASE', 'early_stopping_initial_iteration': 10, 'early_stopping_num_trees_look_ahead': 30, 'focal_loss_alpha': 0.5, 'focal_loss_gamma': 2.0, 'forest_extraction': 'MART', 'goss_alpha': 0.2, 'goss_beta': 0.1, 'growing_strategy': 'LOCAL', 'honest': False, 'honest_fixed_separation': False, 'honest_ratio_leaf_examples': 0.5, 'in_split_min_examples_check': True, 'keep_non_leaf_label_distribution': True, 'l1_regularization': 0.0, 'l2_categorical_regularization': 1.0, 'l2_regularization': 0.0, 'lambda_loss': 1.0, 'loss': 'DEFAULT', 'max_depth': 6, 'max_num_nodes': None, 'maximum_model_size_in_memory_in_bytes': -1.0, 'maximum_training_duration_seconds': -1.0, 'min_examples': 5, 'missing_value_policy': 'GLOBAL_IMPUTATION', 'num_candidate_attributes': -1, 'num_candidate_attributes_ratio': -1.0, 'num_trees': 300, 'pure_serving_model': False, 'random_seed': 123456, 'sampling_method': 'RANDOM', 'selective_gradient_boosting_ratio': 0.01, 'shrinkage': 0.1, 'sorting_strategy': 'PRESORT', 'sparse_oblique_normalization': None, 'sparse_oblique_num_projections_exponent': None, 'sparse_oblique_projection_density_factor': None, 'sparse_oblique_weights': None, 'split_axis': 'AXIS_ALIGNED', 'subsample': 1.0, 'uplift_min_examples_in_treatment': 5, 'uplift_split_score': 'KULLBACK_LEIBLER', 'use_hessian_gain': False, 'validation_interval_in_trees': 1, 'validation_ratio': 0.1}

Training a model with automated hyper-parameter tuning and manual definition of the hyper-parameters

Hyper-parameter tuning is enabled by specifying the tuner constructor argument of the model. The tuner object contains all the configuration of the tuner (search space, optimizer, trial and objective).

# Configure the tuner.

# Create a Random Search tuner with 50 trials.
tuner = tfdf.tuner.RandomSearch(num_trials=50)

# Define the search space.
#
# Adding more parameters generaly improve the quality of the model, but make
# the tuning last longer.

tuner.choice("min_examples", [2, 5, 7, 10])
tuner.choice("categorical_algorithm", ["CART", "RANDOM"])

# Some hyper-parameters are only valid for specific values of other
# hyper-parameters. For example, the "max_depth" parameter is mostly useful when
# "growing_strategy=LOCAL" while "max_num_nodes" is better suited when
# "growing_strategy=BEST_FIRST_GLOBAL".

local_search_space = tuner.choice("growing_strategy", ["LOCAL"])
local_search_space.choice("max_depth", [3, 4, 5, 6, 8])

# merge=True indicates that the parameter (here "growing_strategy") is already
# defined, and that new values are added to it.
global_search_space = tuner.choice("growing_strategy", ["BEST_FIRST_GLOBAL"], merge=True)
global_search_space.choice("max_num_nodes", [16, 32, 64, 128, 256])

tuner.choice("use_hessian_gain", [True, False])
tuner.choice("shrinkage", [0.02, 0.05, 0.10, 0.15])
tuner.choice("num_candidate_attributes_ratio", [0.2, 0.5, 0.9, 1.0])

# Uncomment some (or all) of the following hyper-parameters to increase the
# quality of the search. The number of trial should be increased accordingly.

# tuner.choice("split_axis", ["AXIS_ALIGNED"])
# oblique_space = tuner.choice("split_axis", ["SPARSE_OBLIQUE"], merge=True)
# oblique_space.choice("sparse_oblique_normalization",
#                      ["NONE", "STANDARD_DEVIATION", "MIN_MAX"])
# oblique_space.choice("sparse_oblique_weights", ["BINARY", "CONTINUOUS"])
# oblique_space.choice("sparse_oblique_num_projections_exponent", [1.0, 1.5])
<tensorflow_decision_forests.component.tuner.tuner.SearchSpace at 0x7f240c3b32b0>
%%time
%set_cell_height 300

# Tune the model. Notice the `tuner=tuner`.
tuned_model = tfdf.keras.GradientBoostedTreesModel(tuner=tuner)
tuned_model.fit(train_ds, verbose=2)

# The `num_threads` model constructor argument (not specified in the example
# above) controls how many trials are run in parallel (one per thread). If
# `num_threads` is not specified (like in the example above), one thread is
# allocated for each available CPU core.
#
# If the training is interrupted (for example, by pressing on the "stop" button
# on the top-left of the colab cell), the best model so-far will be returned.

# In the training logs, you can see lines such as `[10/50] Score: -0.45 / -0.40
# HParams: ...`. This indicates that 10 of the 50 trials have been completed.
# And that the last trial returned a score of "-0.45" and that the best trial so
# far has a score of "-0.40". In this example, the model is optimized by
# logloss. Since scores are maximized and log loss should be minimized, the
# score is effectively minus the log loss.
<IPython.core.display.Javascript object>
Warning: The `num_threads` constructor argument is not set and the number of CPU is os.cpu_count()=32 > 32. Setting num_threads to 32. Set num_threads manually to use more than 32 cpus.
WARNING:absl:The `num_threads` constructor argument is not set and the number of CPU is os.cpu_count()=32 > 32. Setting num_threads to 32. Set num_threads manually to use more than 32 cpus.
Use /tmpfs/tmp/tmpzdzgno07 as temporary training directory
Reading training dataset...
Training tensor examples:
Features: {'age': <tf.Tensor 'data:0' shape=(None,) dtype=int64>, 'workclass': <tf.Tensor 'data_1:0' shape=(None,) dtype=string>, 'fnlwgt': <tf.Tensor 'data_2:0' shape=(None,) dtype=int64>, 'education': <tf.Tensor 'data_3:0' shape=(None,) dtype=string>, 'education_num': <tf.Tensor 'data_4:0' shape=(None,) dtype=int64>, 'marital_status': <tf.Tensor 'data_5:0' shape=(None,) dtype=string>, 'occupation': <tf.Tensor 'data_6:0' shape=(None,) dtype=string>, 'relationship': <tf.Tensor 'data_7:0' shape=(None,) dtype=string>, 'race': <tf.Tensor 'data_8:0' shape=(None,) dtype=string>, 'sex': <tf.Tensor 'data_9:0' shape=(None,) dtype=string>, 'capital_gain': <tf.Tensor 'data_10:0' shape=(None,) dtype=int64>, 'capital_loss': <tf.Tensor 'data_11:0' shape=(None,) dtype=int64>, 'hours_per_week': <tf.Tensor 'data_12:0' shape=(None,) dtype=int64>, 'native_country': <tf.Tensor 'data_13:0' shape=(None,) dtype=string>}
Label: Tensor("data_14:0", shape=(None,), dtype=int64)
Weights: None
Normalized tensor features:
 {'age': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast:0' shape=(None,) dtype=float32>), 'workclass': SemanticTensor(semantic=<Semantic.CATEGORICAL: 2>, tensor=<tf.Tensor 'data_1:0' shape=(None,) dtype=string>), 'fnlwgt': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_1:0' shape=(None,) dtype=float32>), 'education': SemanticTensor(semantic=<Semantic.CATEGORICAL: 2>, tensor=<tf.Tensor 'data_3:0' shape=(None,) dtype=string>), 'education_num': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_2:0' shape=(None,) dtype=float32>), 'marital_status': SemanticTensor(semantic=<Semantic.CATEGORICAL: 2>, tensor=<tf.Tensor 'data_5:0' shape=(None,) dtype=string>), 'occupation': SemanticTensor(semantic=<Semantic.CATEGORICAL: 2>, tensor=<tf.Tensor 'data_6:0' shape=(None,) dtype=string>), 'relationship': SemanticTensor(semantic=<Semantic.CATEGORICAL: 2>, tensor=<tf.Tensor 'data_7:0' shape=(None,) dtype=string>), 'race': SemanticTensor(semantic=<Semantic.CATEGORICAL: 2>, tensor=<tf.Tensor 'data_8:0' shape=(None,) dtype=string>), 'sex': SemanticTensor(semantic=<Semantic.CATEGORICAL: 2>, tensor=<tf.Tensor 'data_9:0' shape=(None,) dtype=string>), 'capital_gain': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_3:0' shape=(None,) dtype=float32>), 'capital_loss': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_4:0' shape=(None,) dtype=float32>), 'hours_per_week': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_5:0' shape=(None,) dtype=float32>), 'native_country': SemanticTensor(semantic=<Semantic.CATEGORICAL: 2>, tensor=<tf.Tensor 'data_13:0' shape=(None,) dtype=string>)}
[WARNING 23-08-16 11:08:02.9532 UTC gradient_boosted_trees.cc:1818] "goss_alpha" set but "sampling_method" not equal to "GOSS".
[WARNING 23-08-16 11:08:02.9533 UTC gradient_boosted_trees.cc:1829] "goss_beta" set but "sampling_method" not equal to "GOSS".
[WARNING 23-08-16 11:08:02.9533 UTC gradient_boosted_trees.cc:1843] "selective_gradient_boosting_ratio" set but "sampling_method" not equal to "SELGB".
Training dataset read in 0:00:00.389683. Found 22792 examples.
Training model...
Standard output detected as not visible to the user e.g. running in a notebook. Creating a training log redirection. If training gets stuck, try calling tfdf.keras.set_training_logs_redirection(False).
[INFO 23-08-16 11:08:03.3555 UTC kernel.cc:773] Start Yggdrasil model training
[INFO 23-08-16 11:08:03.3555 UTC kernel.cc:774] Collect training examples
[INFO 23-08-16 11:08:03.3555 UTC kernel.cc:787] Dataspec guide:
column_guides {
  column_name_pattern: "^__LABEL$"
  type: CATEGORICAL
  categorial {
    min_vocab_frequency: 0
    max_vocab_count: -1
  }
}
default_column_guide {
  categorial {
    max_vocab_count: 2000
  }
  discretized_numerical {
    maximum_num_bins: 255
  }
}
ignore_columns_without_guides: false
detect_numerical_as_discretized_numerical: false

[INFO 23-08-16 11:08:03.3556 UTC kernel.cc:393] Number of batches: 23
[INFO 23-08-16 11:08:03.3556 UTC kernel.cc:394] Number of examples: 22792
[INFO 23-08-16 11:08:03.3630 UTC data_spec_inference.cc:305] 1 item(s) have been pruned (i.e. they are considered out of dictionary) for the column native_country (40 item(s) left) because min_value_count=5 and max_number_of_unique_values=2000
[INFO 23-08-16 11:08:03.3630 UTC data_spec_inference.cc:305] 1 item(s) have been pruned (i.e. they are considered out of dictionary) for the column occupation (13 item(s) left) because min_value_count=5 and max_number_of_unique_values=2000
[INFO 23-08-16 11:08:03.3631 UTC data_spec_inference.cc:305] 1 item(s) have been pruned (i.e. they are considered out of dictionary) for the column workclass (7 item(s) left) because min_value_count=5 and max_number_of_unique_values=2000
[INFO 23-08-16 11:08:03.3698 UTC kernel.cc:794] Training dataset:
Number of records: 22792
Number of columns: 15

Number of columns by type:
    CATEGORICAL: 9 (60%)
    NUMERICAL: 6 (40%)

Columns:

CATEGORICAL: 9 (60%)
    0: "__LABEL" CATEGORICAL integerized vocab-size:3 no-ood-item
    4: "education" CATEGORICAL has-dict vocab-size:17 zero-ood-items most-frequent:"HS-grad" 7340 (32.2043%)
    8: "marital_status" CATEGORICAL has-dict vocab-size:8 zero-ood-items most-frequent:"Married-civ-spouse" 10431 (45.7661%)
    9: "native_country" CATEGORICAL num-nas:407 (1.78571%) has-dict vocab-size:41 num-oods:1 (0.00446728%) most-frequent:"United-States" 20436 (91.2933%)
    10: "occupation" CATEGORICAL num-nas:1260 (5.52826%) has-dict vocab-size:14 num-oods:1 (0.00464425%) most-frequent:"Prof-specialty" 2870 (13.329%)
    11: "race" CATEGORICAL has-dict vocab-size:6 zero-ood-items most-frequent:"White" 19467 (85.4115%)
    12: "relationship" CATEGORICAL has-dict vocab-size:7 zero-ood-items most-frequent:"Husband" 9191 (40.3256%)
    13: "sex" CATEGORICAL has-dict vocab-size:3 zero-ood-items most-frequent:"Male" 15165 (66.5365%)
    14: "workclass" CATEGORICAL num-nas:1257 (5.51509%) has-dict vocab-size:8 num-oods:1 (0.0046436%) most-frequent:"Private" 15879 (73.7358%)

NUMERICAL: 6 (40%)
    1: "age" NUMERICAL mean:38.6153 min:17 max:90 sd:13.661
    2: "capital_gain" NUMERICAL mean:1081.9 min:0 max:99999 sd:7509.48
    3: "capital_loss" NUMERICAL mean:87.2806 min:0 max:4356 sd:403.01
    5: "education_num" NUMERICAL mean:10.0927 min:1 max:16 sd:2.56427
    6: "fnlwgt" NUMERICAL mean:189879 min:12285 max:1.4847e+06 sd:106423
    7: "hours_per_week" NUMERICAL mean:40.3955 min:1 max:99 sd:12.249

Terminology:
    nas: Number of non-available (i.e. missing) values.
    ood: Out of dictionary.
    manually-defined: Attribute which type is manually defined by the user i.e. the type was not automatically inferred.
    tokenized: The attribute value is obtained through tokenization.
    has-dict: The attribute is attached to a string dictionary e.g. a categorical attribute stored as a string.
    vocab-size: Number of unique values.

[INFO 23-08-16 11:08:03.3699 UTC kernel.cc:810] Configure learner
[WARNING 23-08-16 11:08:03.3702 UTC gradient_boosted_trees.cc:1818] "goss_alpha" set but "sampling_method" not equal to "GOSS".
[WARNING 23-08-16 11:08:03.3702 UTC gradient_boosted_trees.cc:1829] "goss_beta" set but "sampling_method" not equal to "GOSS".
[WARNING 23-08-16 11:08:03.3702 UTC gradient_boosted_trees.cc:1843] "selective_gradient_boosting_ratio" set but "sampling_method" not equal to "SELGB".
[INFO 23-08-16 11:08:03.3703 UTC kernel.cc:824] Training config:
learner: "HYPERPARAMETER_OPTIMIZER"
features: "^age$"
features: "^capital_gain$"
features: "^capital_loss$"
features: "^education$"
features: "^education_num$"
features: "^fnlwgt$"
features: "^hours_per_week$"
features: "^marital_status$"
features: "^native_country$"
features: "^occupation$"
features: "^race$"
features: "^relationship$"
features: "^sex$"
features: "^workclass$"
label: "^__LABEL$"
task: CLASSIFICATION
metadata {
  framework: "TF Keras"
}
[yggdrasil_decision_forests.model.hyperparameters_optimizer_v2.proto.hyperparameters_optimizer_config] {
  base_learner {
    learner: "GRADIENT_BOOSTED_TREES"
    features: "^age$"
    features: "^capital_gain$"
    features: "^capital_loss$"
    features: "^education$"
    features: "^education_num$"
    features: "^fnlwgt$"
    features: "^hours_per_week$"
    features: "^marital_status$"
    features: "^native_country$"
    features: "^occupation$"
    features: "^race$"
    features: "^relationship$"
    features: "^sex$"
    features: "^workclass$"
    label: "^__LABEL$"
    task: CLASSIFICATION
    random_seed: 123456
    pure_serving_model: false
    [yggdrasil_decision_forests.model.gradient_boosted_trees.proto.gradient_boosted_trees_config] {
      num_trees: 300
      decision_tree {
        max_depth: 6
        min_examples: 5
        in_split_min_examples_check: true
        keep_non_leaf_label_distribution: true
        num_candidate_attributes: -1
        missing_value_policy: GLOBAL_IMPUTATION
        allow_na_conditions: false
        categorical_set_greedy_forward {
          sampling: 0.1
          max_num_items: -1
          min_item_frequency: 1
        }
        growing_strategy_local {
        }
        categorical {
          cart {
          }
        }
        axis_aligned_split {
        }
        internal {
          sorting_strategy: PRESORTED
        }
        uplift {
          min_examples_in_treatment: 5
          split_score: KULLBACK_LEIBLER
        }
      }
      shrinkage: 0.1
      loss: DEFAULT
      validation_set_ratio: 0.1
      validation_interval_in_trees: 1
      early_stopping: VALIDATION_LOSS_INCREASE
      early_stopping_num_trees_look_ahead: 30
      l2_regularization: 0
      lambda_loss: 1
      mart {
      }
      adapt_subsample_for_maximum_training_duration: false
      l1_regularization: 0
      use_hessian_gain: false
      l2_regularization_categorical: 1
      stochastic_gradient_boosting {
        ratio: 1
      }
      apply_link_function: true
      compute_permutation_variable_importance: false
      binary_focal_loss_options {
        misprediction_exponent: 2
        positive_sample_coefficient: 0.5
      }
      early_stopping_initial_iteration: 10
    }
  }
  optimizer {
    optimizer_key: "RANDOM"
    [yggdrasil_decision_forests.model.hyperparameters_optimizer_v2.proto.random] {
      num_trials: 50
    }
  }
  search_space {
    fields {
      name: "min_examples"
      discrete_candidates {
        possible_values {
          integer: 2
        }
        possible_values {
          integer: 5
        }
        possible_values {
          integer: 7
        }
        possible_values {
          integer: 10
        }
      }
    }
    fields {
      name: "categorical_algorithm"
      discrete_candidates {
        possible_values {
          categorical: "CART"
        }
        possible_values {
          categorical: "RANDOM"
        }
      }
    }
    fields {
      name: "growing_strategy"
      discrete_candidates {
        possible_values {
          categorical: "LOCAL"
        }
        possible_values {
          categorical: "BEST_FIRST_GLOBAL"
        }
      }
      children {
        name: "max_depth"
        discrete_candidates {
          possible_values {
            integer: 3
          }
          possible_values {
            integer: 4
          }
          possible_values {
            integer: 5
          }
          possible_values {
            integer: 6
          }
          possible_values {
            integer: 8
          }
        }
        parent_discrete_values {
          possible_values {
            categorical: "LOCAL"
          }
        }
      }
      children {
        name: "max_num_nodes"
        discrete_candidates {
          possible_values {
            integer: 16
          }
          possible_values {
            integer: 32
          }
          possible_values {
            integer: 64
          }
          possible_values {
            integer: 128
          }
          possible_values {
            integer: 256
          }
        }
        parent_discrete_values {
          possible_values {
            categorical: "BEST_FIRST_GLOBAL"
          }
        }
      }
    }
    fields {
      name: "use_hessian_gain"
      discrete_candidates {
        possible_values {
          categorical: "true"
        }
        possible_values {
          categorical: "false"
        }
      }
    }
    fields {
      name: "shrinkage"
      discrete_candidates {
        possible_values {
          real: 0.02
        }
        possible_values {
          real: 0.05
        }
        possible_values {
          real: 0.1
        }
        possible_values {
          real: 0.15
        }
      }
    }
    fields {
      name: "num_candidate_attributes_ratio"
      discrete_candidates {
        possible_values {
          real: 0.2
        }
        possible_values {
          real: 0.5
        }
        possible_values {
          real: 0.9
        }
        possible_values {
          real: 1
        }
      }
    }
  }
  base_learner_deployment {
    num_threads: 1
  }
}

[INFO 23-08-16 11:08:03.3707 UTC kernel.cc:827] Deployment config:
cache_path: "/tmpfs/tmp/tmpzdzgno07/working_cache"
num_threads: 32
try_resume_training: true

[INFO 23-08-16 11:08:03.3709 UTC kernel.cc:889] Train model
[INFO 23-08-16 11:08:03.3711 UTC hyperparameters_optimizer.cc:209] Hyperparameter search space:
fields {
  name: "min_examples"
  discrete_candidates {
    possible_values {
      integer: 2
    }
    possible_values {
      integer: 5
    }
    possible_values {
      integer: 7
    }
    possible_values {
      integer: 10
    }
  }
}
fields {
  name: "categorical_algorithm"
  discrete_candidates {
    possible_values {
      categorical: "CART"
    }
    possible_values {
      categorical: "RANDOM"
    }
  }
}
fields {
  name: "growing_strategy"
  discrete_candidates {
    possible_values {
      categorical: "LOCAL"
    }
    possible_values {
      categorical: "BEST_FIRST_GLOBAL"
    }
  }
  children {
    name: "max_depth"
    discrete_candidates {
      possible_values {
        integer: 3
      }
      possible_values {
        integer: 4
      }
      possible_values {
        integer: 5
      }
      possible_values {
        integer: 6
      }
      possible_values {
        integer: 8
      }
    }
    parent_discrete_values {
      possible_values {
        categorical: "LOCAL"
      }
    }
  }
  children {
    name: "max_num_nodes"
    discrete_candidates {
      possible_values {
        integer: 16
      }
      possible_values {
        integer: 32
      }
      possible_values {
        integer: 64
      }
      possible_values {
        integer: 128
      }
      possible_values {
        integer: 256
      }
    }
    parent_discrete_values {
      possible_values {
        categorical: "BEST_FIRST_GLOBAL"
      }
    }
  }
}
fields {
  name: "use_hessian_gain"
  discrete_candidates {
    possible_values {
      categorical: "true"
    }
    possible_values {
      categorical: "false"
    }
  }
}
fields {
  name: "shrinkage"
  discrete_candidates {
    possible_values {
      real: 0.02
    }
    possible_values {
      real: 0.05
    }
    possible_values {
      real: 0.1
    }
    possible_values {
      real: 0.15
    }
  }
}
fields {
  name: "num_candidate_attributes_ratio"
  discrete_candidates {
    possible_values {
      real: 0.2
    }
    possible_values {
      real: 0.5
    }
    possible_values {
      real: 0.9
    }
    possible_values {
      real: 1
    }
  }
}

[INFO 23-08-16 11:08:03.3713 UTC hyperparameters_optimizer.cc:500] Start local tuner with 32 thread(s)
[INFO 23-08-16 11:08:03.3728 UTC gradient_boosted_trees.cc:459] Default loss set to BINOMIAL_LOG_LIKELIHOOD
[INFO 23-08-16 11:08:03.3728 UTC gradient_boosted_trees.cc:1085] Training gradient boosted tree on 22792 example(s) and 14 feature(s).
[INFO 23-08-16 11:08:03.3729 UTC gradient_boosted_trees.cc:459] Default loss set to BINOMIAL_LOG_LIKELIHOOD
[INFO 23-08-16 11:08:03.3729 UTC gradient_boosted_trees.cc:1085] Training gradient boosted tree on 22792 example(s) and 14 feature(s).
[INFO 23-08-16 11:08:03.3729 UTC gradient_boosted_trees.cc:459] Default loss set to BINOMIAL_LOG_LIKELIHOOD
[INFO 23-08-16 11:08:03.3730 UTC gradient_boosted_trees.cc:1085] Training gradient boosted tree on 22792 example(s) and [INFO 23-08-16 11:08:03.3730 UTC gradient_boosted_trees.cc:14 feature(s).
459] Default loss set to BINOMIAL_LOG_LIKELIHOOD
[INFO 23-08-16 11:08:03.3731 UTC gradient_boosted_trees.cc:1085] Training gradient boosted tree on 22792 example(s) and 14 feature(s).
[INFO[INFO 23-08-16 11:08:03.3731 UTC  23-08-16 11:08:03.3732 UTC gradient_boosted_trees.cc:459] Default loss set to BINOMIAL_LOG_LIKELIHOOD
gradient_boosted_trees.cc:459] Default loss set to BINOMIAL_LOG_LIKELIHOOD
[INFO 23-08-16 11:08:03.3732 UTC gradient_boosted_trees.cc[INFO 23-08-16 11:08:03.3732 UTC gradient_boosted_trees.cc:1085:1085] Training gradient boosted tree on 22792 example(s) and 14 feature(s).
] Training gradient boosted tree on 22792 example(s) and 14 feature(s).
[INFO 23-08-16 11:08:03.3733 UTC gradient_boosted_trees.cc:459] Default loss set to BINOMIAL_LOG_LIKELIHOOD
[INFO 23-08-16 11:08:03.3733 UTC gradient_boosted_trees.cc:459[[INFO 23-08-16 11:08:03.3733 UTC [INFOgradient_boosted_trees.cc:1085] Training gradient boosted tree on 22792 example(s) and 14 feature(s).[ 23-08-16 11:08:03.3734 UTC 
INFO] Default loss set to BINOMIAL_LOG_LIKELIHOOD[INFO 23-08-16 11:08:03.3734 UTC INFOgradient_boosted_trees.cc
 23-08-16 11:08:03.3734 UTC [ 23-08-16 11:08:03.3734 UTC :[INFO[[INFOINFO 23-08-16 11:08:03.3735 UTC gradient_boosted_trees.cc 23-08-16 11:08:03.3735 UTC gradient_boosted_trees.ccgradient_boosted_trees.ccgradient_boosted_trees.cc:459] 459Default loss set to BINOMIAL_LOG_LIKELIHOOD
 23-08-16 11:08:03.3735 UTC INFOgradient_boosted_trees.cc:459] Default loss set to BINOMIAL_LOG_LIKELIHOOD:gradient_boosted_trees.cc
:] 459:] Default loss set to 459BINOMIAL_LOG_LIKELIHOOD
[459INFO] Default loss set to  23-08-16 11:08:03.3736 UTC  23-08-16 11:08:03.3736 UTC BINOMIAL_LOG_LIKELIHOODgradient_boosted_trees.cc:Default loss set to [] [INFO:4591085 23-08-16 11:08:03.3737 UTC gradient_boosted_trees.cc
:gradient_boosted_trees.cc:INFOBINOMIAL_LOG_LIKELIHOOD459]  23-08-16 11:08:03.3737 UTC ] 
Default loss set to [BINOMIAL_LOG_LIKELIHOODINFO[Training gradient boosted tree on gradient_boosted_trees.ccINFO 23-08-16 11:08:03.3738 UTC :1085[] ] INFO22792 23-08-16 11:08:03.3738 UTC  example(s) and gradient_boosted_trees.cc14Default loss set to  feature(s).BINOMIAL_LOG_LIKELIHOODgradient_boosted_trees.cc
 23-08-16 11:08:03.3738 UTC :[INFO1085gradient_boosted_trees.ccTraining gradient boosted tree on ] 
:Default loss set to :227921085 23-08-16 11:08:03.3738 UTC [BINOMIAL_LOG_LIKELIHOOD10851085Training gradient boosted tree on ] [Training gradient boosted tree on INFOINFO22792
 example(s) and ]  23-08-16 11:08:03.3739 UTC gradient_boosted_trees.cc[ example(s) and INFO1422792 23-08-16 11:08:03.3739 UTC gradient_boosted_trees.cc feature(s).
:459] Default loss set to BINOMIAL_LOG_LIKELIHOOD
 23-08-16 11:08:03.3739 UTC gradient_boosted_trees.cc::10851085] ] Training gradient boosted tree on 14 example(s) and gradient_boosted_trees.cc feature(s).14 feature(s).Training gradient boosted tree on 

22792[ example(s) and :Training gradient boosted tree on 2279222792 example(s) and  example(s) and 1414INFO
[ 23-08-16 11:08:03.3741 UTC  feature(s).gradient_boosted_trees.cc45914] Default loss set to ]  feature(s).Training gradient boosted tree on 

 feature(s).
INFO 23-08-16 11:08:03.3742 UTC gradient_boosted_trees.cc:BINOMIAL_LOG_LIKELIHOOD1085
] [INFOTraining gradient boosted tree on 22792 example(s) and :[22792 23-08-16 11:08:03.3742 UTC 108514gradient_boosted_trees.cc feature(s).
:1085INFO]  example(s) and 14]  feature(s).Training gradient boosted tree on 22792
 23-08-16 11:08:03.3743 UTC gradient_boosted_trees.cc:459] Default loss set to BINOMIAL_LOG_LIKELIHOOD[ example(s) and INFO14 feature(s).Training gradient boosted tree on 
22792 example(s) and 14
 feature(s).[ 23-08-16 11:08:03.3743 UTC INFOgradient_boosted_trees.cc[ 23-08-16 11:08:03.3744 UTC [
INFO: 23-08-16 11:08:03.3744 UTC gradient_boosted_trees.cc459:] Default loss set to INFOBINOMIAL_LOG_LIKELIHOOD[459INFO] Default loss set to BINOMIAL_LOG_LIKELIHOOD 23-08-16 11:08:03.3745 UTC gradient_boosted_trees.ccgradient_boosted_trees.cc:459
] 
Default loss set to BINOMIAL_LOG_LIKELIHOOD:[459] INFO
Default loss set to [[INFO 23-08-16 11:08:03.3745 UTC  23-08-16 11:08:03.3745 UTC gradient_boosted_trees.ccBINOMIAL_LOG_LIKELIHOODgradient_boosted_trees.cc
:1085[[INFOINFO 23-08-16 11:08:03.3746 UTC  23-08-16 11:08:03.3746 UTC gradient_boosted_trees.cc: 23-08-16 11:08:03.3746 UTC [:INFO1085gradient_boosted_trees.cc1085]  23-08-16 11:08:03.3746 UTC gradient_boosted_trees.ccgradient_boosted_trees.cc:] Training gradient boosted tree on :] 1085Training gradient boosted tree on ] INFO22792: example(s) and 14Training gradient boosted tree on  feature(s).
22792 example(s) and 141085 feature(s).22792
 23-08-16 11:08:03.3747 UTC  example(s) and gradient_boosted_trees.cc:14459459] ] Training gradient boosted tree on  feature(s).22792 example(s) and 
Training gradient boosted tree on 14] 22792 example(s) and 14 feature(s).
Default loss set to BINOMIAL_LOG_LIKELIHOOD feature(s).Default loss set to 
BINOMIAL_LOG_LIKELIHOOD

[INFO 23-08-16 11:08:03.3748 UTC gradient_boosted_trees.cc:1085] Training gradient boosted tree on 22792 example(s) and 14[INFO 23-08-16 11:08:03.3749 UTC gradient_boosted_trees.cc:1085]  feature(s).
Training gradient boosted tree on 22792 example(s) and 14 feature(s).
[INFO 23-08-16 11:08:03.3752 UTC gradient_boosted_trees.cc:459] Default loss set to BINOMIAL_LOG_LIKELIHOOD
[INFO 23-08-16 11:08:03.3752 UTC gradient_boosted_trees.cc:1085] Training gradient boosted tree on 22792 example(s) and 14 feature(s).
[INFO 23-08-16 11:08:03.3754 UTC gradient_boosted_trees.cc:459] Default loss set to BINOMIAL_LOG_LIKELIHOOD
[INFO 23-08-16 11:08:03.3754 UTC gradient_boosted_trees.cc:1085] Training gradient boosted tree on 22792 example(s) and 14 feature(s).
[INFO 23-08-16 11:08:03.3761 UTC gradient_boosted_trees.cc:459] Default loss set to [INFOBINOMIAL_LOG_LIKELIHOOD
 23-08-16 11:08:03.3762 UTC gradient_boosted_trees.cc:459] Default loss set to BINOMIAL_LOG_LIKELIHOOD
[INFO[INFO 23-08-16 11:08:03.3762 UTC gradient_boosted_trees.cc:1085] Training gradient boosted tree on 22792 example(s) and  23-08-16 11:08:03.3762 UTC gradient_boosted_trees.cc:1085] Training gradient boosted tree on 22792 example(s) and 1414 feature(s).
 feature(s).
[INFO 23-08-16 11:08:03.3768 UTC gradient_boosted_trees.cc:459] Default loss set to BINOMIAL_LOG_LIKELIHOOD
[INFO 23-08-16 11:08:03.3768 UTC gradient_boosted_trees.cc:1085] Training gradient boosted tree on 22792 example(s) and 14 feature(s).
[INFO 23-08-16 11:08:03.3772 UTC gradient_boosted_trees.cc:459] Default loss set to BINOMIAL_LOG_LIKELIHOOD
[INFO 23-08-16 11:08:03.3772 UTC gradient_boosted_trees.cc:1085] Training gradient boosted tree on 22792 example(s) and 14 feature(s).
[INFO 23-08-16 11:08:03.3774 UTC gradient_boosted_trees.cc:459] Default loss set to BINOMIAL_LOG_LIKELIHOOD
[INFO 23-08-16 11:08:03.3775 UTC gradient_boosted_trees.cc:1085] Training gradient boosted tree on 22792 example(s) and 14 feature(s).
[INFO 23-08-16 11:08:03.3914 UTC gradient_boosted_trees.cc:1128] 20533 examples used for training and 2259 examples used for validation
[INFO 23-08-16 11:08:03.4337 UTC gradient_boosted_trees.cc:1128] 20533 examples used for training and 2259 examples used for validation
[INFO 23-08-16 11:08:03.4344 UTC gradient_boosted_trees.cc:1128] 20533 examples used for training and 2259 examples used for validation
[INFO 23-08-16 11:08:03.4345 UTC gradient_boosted_trees.cc:1128] 20533 examples used for training and 2259 examples used for validation
[INFO 23-08-16 11:08:03.4347 UTC gradient_boosted_trees.cc:1128] 20533 examples used for training and 2259 examples used for validation
[INFO 23-08-16 11:08:03.4356 UTC gradient_boosted_trees.cc:1128] 20533 examples used for training and 2259 examples used for validation
[INFO 23-08-16 11:08:03.4363 UTC gradient_boosted_trees.cc:1128] 20533 examples used for training and 2259 examples used for validation
[INFO[INFO 23-08-16 11:08:03.4369 UTC gradient_boosted_trees.cc: 23-08-16 11:08:03.4369 UTC gradient_boosted_trees.cc:1128] 20533 examples used for training and 2259 examples used for validation
1128] 20533 examples used for training and 2259 examples used for validation
[INFO 23-08-16 11:08:03.4370 UTC gradient_boosted_trees.cc:1128] 20533 examples used for training and 2259 examples used for validation
[INFO 23-08-16 11:08:03.4374 UTC gradient_boosted_trees.cc:1128] 20533 examples used for training and 2259 examples used for validation
[INFO[[[INFOINFO 23-08-16 11:08:03.4382 UTC gradient_boosted_trees.ccINFO 23-08-16 11:08:03.4382 UTC gradient_boosted_trees.cc:1128] 20533 examples used for training and [2259:INFO1128 23-08-16 11:08:03.4382 UTC [ 23-08-16 11:08:03.4382 UTC INFOgradient_boosted_trees.ccgradient_boosted_trees.cc: examples used for validation:] 
205331128 23-08-16 11:08:03.4382 UTC ] 20533gradient_boosted_trees.cc examples used for training and 2259:1128 examples used for validation examples used for training and ]  23-08-16 11:08:03.4382 UTC 205332259gradient_boosted_trees.cc examples used for training and 2259 examples used for validation:
1128] 205331128 examples used for training and  examples used for validation2259 examples used for validation
] 20533

 examples used for training and 2259 examples used for validation
[INFO[INFO 23-08-16 11:08:03.4385 UTC gradient_boosted_trees.cc 23-08-16 11:08:03.4385 UTC gradient_boosted_trees.cc::1128[INFO] 20533 examples used for training and 22591128] 20533 examples used for training and 2259 examples used for validation examples used for validation
 23-08-16 11:08:03.4386 UTC gradient_boosted_trees.cc:1128] 20533 examples used for training and 2259 examples used for validation

[INFO[INFO 23-08-16 11:08:03.4403 UTC gradient_boosted_trees.cc:1128[[[INFO] 20533INFO examples used for training and 2259INFO examples used for validation
 23-08-16 11:08:03.4404 UTC gradient_boosted_trees.cc 23-08-16 11:08:03.4404 UTC :gradient_boosted_trees.cc 23-08-16 11:08:03.4403 UTC :1128] 20533 examples used for training and gradient_boosted_trees.cc 23-08-16 11:08:03.4404 UTC 2259 examples used for validationgradient_boosted_trees.cc::1128] 20533 examples used for training and 11282259 examples used for validation1128
] 
20533] 20533 examples used for training and 2259 examples used for validation
 examples used for training and 2259 examples used for validation
[INFO 23-08-16 11:08:03.4409 UTC gradient_boosted_trees.cc:1128] 20533 examples used for training and 2259 examples used for validation
[[INFOINFO 23-08-16 11:08:03.4416 UTC gradient_boosted_trees.cc:1128] 20533 examples used for training and 2259[ 23-08-16 11:08:03.4416 UTC gradient_boosted_trees.ccINFO examples used for validation
: 23-08-16 11:08:03.4416 UTC gradient_boosted_trees.cc:1128] 20533 examples used for training and 2259 examples used for validation
1128] 20533 examples used for training and 2259 examples used for validation
[INFO 23-08-16 11:08:03.4447 UTC gradient_boosted_trees.cc:1128] 20533 examples used for training and 2259 examples used for validation
[INFO 23-08-16 11:08:03.4462 UTC gradient_boosted_trees.cc:1128] 20533 examples used for training and 2259 examples used for validation
[INFO 23-08-16 11:08:03.4490 UTC gradient_boosted_trees.cc:1128] 20533 examples used for training and 2259 examples used for validation
[INFO 23-08-16 11:08:03.4535 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.012352 train-accuracy:0.761895 valid-loss:1.067086 valid-accuracy:0.736609
[INFO 23-08-16 11:08:03.4640 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.033944 train-accuracy:0.761895 valid-loss:1.087890 valid-accuracy:0.736609
[INFO 23-08-16 11:08:03.4689 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.007318 train-accuracy:0.761895 valid-loss:1.063819 valid-accuracy:0.736609
[INFO 23-08-16 11:08:03.4717 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.015585 train-accuracy:0.761895 valid-loss:1.068358 valid-accuracy:0.736609
[INFO 23-08-16 11:08:03.4747 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:0.992466 train-accuracy:0.761895 valid-loss:1.048658 valid-accuracy:0.736609
[INFO 23-08-16 11:08:03.4758 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.080310 train-accuracy:0.761895 valid-loss:1.138544 valid-accuracy:0.736609
[INFO 23-08-16 11:08:03.4762 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.024983 train-accuracy:0.761895 valid-loss:1.080660 valid-accuracy:0.736609
[INFO 23-08-16 11:08:03.4800 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.013950 train-accuracy:0.761895 valid-loss:1.069965 valid-accuracy:0.736609
[INFO 23-08-16 11:08:03.4803 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.035081 train-accuracy:0.761895 valid-loss:1.091865 valid-accuracy:0.736609
[INFO 23-08-16 11:08:03.4826 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:0.974501 train-accuracy:0.761895 valid-loss:1.024211 valid-accuracy:0.736609
[INFO 23-08-16 11:08:03.4855 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:0.992049 train-accuracy:0.761895 valid-loss:1.047210 valid-accuracy:0.736609
[INFO 23-08-16 11:08:03.4868 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.021242 train-accuracy:0.761895 valid-loss:1.076859 valid-accuracy:0.736609
[INFO 23-08-16 11:08:03.4882 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.056437 train-accuracy:0.761895 valid-loss:1.113420 valid-accuracy:0.736609
[INFO 23-08-16 11:08:03.4903 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.057450 train-accuracy:0.761895 valid-loss:1.114456 valid-accuracy:0.736609
[INFO 23-08-16 11:08:03.4920 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.054434 train-accuracy:0.761895 valid-loss:1.110703 valid-accuracy:0.736609
[INFO 23-08-16 11:08:03.4927 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.022126 train-accuracy:0.761895 valid-loss:1.077863 valid-accuracy:0.736609
[INFO 23-08-16 11:08:03.4975 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:0.985785 train-accuracy:0.761895 valid-loss:1.041083 valid-accuracy:0.736609
[INFO 23-08-16 11:08:03.5011 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.015975 train-accuracy:0.761895 valid-loss:1.071430 valid-accuracy:0.736609
[INFO 23-08-16 11:08:03.5043 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.056455 train-accuracy:0.761895 valid-loss:1.113410 valid-accuracy:0.736609
[INFO 23-08-16 11:08:03.5052 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.080606 train-accuracy:0.761895 valid-loss:1.138615 valid-accuracy:0.736609
[INFO 23-08-16 11:08:03.5098 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.055526 train-accuracy:0.761895 valid-loss:1.112339 valid-accuracy:0.736609
[INFO 23-08-16 11:08:03.5126 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.080606 train-accuracy:0.761895 valid-loss:1.138615 valid-accuracy:0.736609
[INFO 23-08-16 11:08:03.5132 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.080079 train-accuracy:0.761895 valid-loss:1.138475 valid-accuracy:0.736609
[INFO 23-08-16 11:08:03.5158 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.080017 train-accuracy:0.761895 valid-loss:1.137988 valid-accuracy:0.736609
[INFO 23-08-16 11:08:03.5282 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.052474 train-accuracy:0.761895 valid-loss:1.109417 valid-accuracy:0.736609
[INFO 23-08-16 11:08:03.5328 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:0.978408 train-accuracy:0.761895 valid-loss:1.031947 valid-accuracy:0.736609
[INFO 23-08-16 11:08:03.5335 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.055966 train-accuracy:0.761895 valid-loss:1.113004 valid-accuracy:0.736609
[INFO 23-08-16 11:08:03.5340 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.080559 train-accuracy:0.761895 valid-loss:1.138519 valid-accuracy:0.736609
[INFO 23-08-16 11:08:03.5397 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.080851 train-accuracy:0.761895 valid-loss:1.138916 valid-accuracy:0.736609
[INFO 23-08-16 11:08:03.5398 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.015861 train-accuracy:0.761895 valid-loss:1.071101 valid-accuracy:0.736609
[INFO 23-08-16 11:08:03.5503 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.054509 train-accuracy:0.761895 valid-loss:1.111318 valid-accuracy:0.736609
[INFO 23-08-16 11:08:03.5527 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.080203 train-accuracy:0.761895 valid-loss:1.138223 valid-accuracy:0.736609
[INFO 23-08-16 11:08:05.4509 UTC gradient_boosted_trees.cc:1542]  num-trees:300 train-loss:0.553261 train-accuracy:0.875566 valid-loss:0.590388 valid-accuracy:0.865870
[INFO 23-08-16 11:08:05.4509 UTC gradient_boosted_trees.cc:247] Truncates the model to 299 tree(s) i.e. 299  iteration(s).
[INFO 23-08-16 11:08:05.4509 UTC gradient_boosted_trees.cc:310] Final model num-trees:299 valid-loss:0.590370 valid-accuracy:0.866313
[INFO 23-08-16 11:08:05.4520 UTC hyperparameters_optimizer.cc:582] [1/50] Score: -0.59037 / -0.59037 HParams: fields { name: "min_examples" value { integer: 7 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 3 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "shrinkage" value { real: 0.15 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.2 } }
[INFO 23-08-16 11:08:05.4525 UTC gradient_boosted_trees.cc:459] Default loss set to BINOMIAL_LOG_LIKELIHOOD
[INFO 23-08-16 11:08:05.4526 UTC gradient_boosted_trees.cc:1085] Training gradient boosted tree on 22792 example(s) and 14 feature(s).
[INFO 23-08-16 11:08:05.4580 UTC gradient_boosted_trees.cc:1128] 20533 examples used for training and 2259 examples used for validation
[INFO 23-08-16 11:08:05.4741 UTC early_stopping.cc:53] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.583674
[INFO 23-08-16 11:08:05.4742 UTC gradient_boosted_trees.cc:247] Truncates the model to 61 tree(s) i.e. 61  iteration(s).
[INFO 23-08-16 11:08:05.4753 UTC gradient_boosted_trees.cc:310] Final model num-trees:61 valid-loss:0.583674 valid-accuracy:0.866755
[INFO 23-08-16 11:08:05.4799 UTC hyperparameters_optimizer.cc:582] [2/50] Score: -0.583674 / -0.583674 HParams: fields { name: "min_examples" value { integer: 5 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 8 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "shrinkage" value { real: 0.15 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.2 } }
[INFO 23-08-16 11:08:05.4807 UTC gradient_boosted_trees.cc:459] Default loss set to BINOMIAL_LOG_LIKELIHOOD
[INFO 23-08-16 11:08:05.4807 UTC gradient_boosted_trees.cc:1085] Training gradient boosted tree on 22792 example(s) and 14 feature(s).
[INFO 23-08-16 11:08:05.4861 UTC gradient_boosted_trees.cc:1128] 20533 examples used for training and 2259 examples used for validation
[INFO 23-08-16 11:08:05.5125 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.080487 train-accuracy:0.761895 valid-loss:1.138629 valid-accuracy:0.736609
[INFO 23-08-16 11:08:05.5642 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.010715 train-accuracy:0.761895 valid-loss:1.065719 valid-accuracy:0.736609
[INFO 23-08-16 11:08:06.6744 UTC gradient_boosted_trees.cc:1542]  num-trees:300 train-loss:0.540563 train-accuracy:0.877271 valid-loss:0.581734 valid-accuracy:0.869854
[INFO 23-08-16 11:08:06.6744 UTC gradient_boosted_trees.cc:247] Truncates the model to 300 tree(s) i.e. 300  iteration(s).
[INFO 23-08-16 11:08:06.6745 UTC gradient_boosted_trees.cc:310] Final model num-trees:300 valid-loss:0.581734 valid-accuracy:0.869854
[INFO 23-08-16 11:08:06.6779 UTC hyperparameters_optimizer.cc:582] [3/50] Score: -0.581734 / -0.581734 HParams: fields { name: "min_examples" value { integer: 10 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 3 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "shrinkage" value { real: 0.15 } } fields { name: "num_candidate_attributes_ratio" value { real: 1 } }
[INFO 23-08-16 11:08:06.6786 UTC gradient_boosted_trees.cc:459] Default loss set to BINOMIAL_LOG_LIKELIHOOD
[INFO 23-08-16 11:08:06.6787 UTC gradient_boosted_trees.cc:1085] Training gradient boosted tree on 22792 example(s) and 14 feature(s).
[INFO 23-08-16 11:08:06.6844 UTC gradient_boosted_trees.cc:1128] 20533 examples used for training and 2259 examples used for validation
[INFO 23-08-16 11:08:06.7151 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.057450 train-accuracy:0.761895 valid-loss:1.114456 valid-accuracy:0.736609
[INFO 23-08-16 11:08:06.8117 UTC gradient_boosted_trees.cc:1542]  num-trees:300 train-loss:0.537262 train-accuracy:0.878780 valid-loss:0.585214 valid-accuracy:0.869854
[INFO 23-08-16 11:08:06.8117 UTC gradient_boosted_trees.cc:247] Truncates the model to 300 tree(s) i.e. 300  iteration(s).
[INFO 23-08-16 11:08:06.8117 UTC gradient_boosted_trees.cc:310] Final model num-trees:300 valid-loss:0.585214 valid-accuracy:0.869854
[INFO 23-08-16 11:08:06.8126 UTC hyperparameters_optimizer.cc:582] [4/50] Score: -0.585214 / -0.581734 HParams: fields { name: "min_examples" value { integer: 10 } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 3 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "shrinkage" value { real: 0.15 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.5 } }
[INFO 23-08-16 11:08:06.8139 UTC gradient_boosted_trees.cc:459] Default loss set to BINOMIAL_LOG_LIKELIHOOD
[INFO 23-08-16 11:08:06.8140 UTC gradient_boosted_trees.cc:1085] Training gradient boosted tree on 22792 example(s) and 14 feature(s).
[INFO 23-08-16 11:08:06.8191 UTC gradient_boosted_trees.cc:1128] 20533 examples used for training and 2259 examples used for validation
[INFO 23-08-16 11:08:06.8574 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.016525 train-accuracy:0.761895 valid-loss:1.069784 valid-accuracy:0.736609
[INFO 23-08-16 11:08:07.2475 UTC early_stopping.cc:53] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.588227
[INFO 23-08-16 11:08:07.2476 UTC gradient_boosted_trees.cc:247] Truncates the model to 113 tree(s) i.e. 113  iteration(s).
[INFO 23-08-16 11:08:07.2487 UTC gradient_boosted_trees.cc:310] Final model num-trees:113 valid-loss:0.588227 valid-accuracy:0.868969
[INFO 23-08-16 11:08:07.2525 UTC hyperparameters_optimizer.cc:582] [5/50] Score: -0.588227 / -0.581734 HParams: fields { name: "min_examples" value { integer: 7 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 64 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "shrinkage" value { real: 0.1 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.2 } }
[INFO 23-08-16 11:08:07.2582 UTC gradient_boosted_trees.cc:459] Default loss set to BINOMIAL_LOG_LIKELIHOOD
[INFO 23-08-16 11:08:07.2583 UTC gradient_boosted_trees.cc:1085] Training gradient boosted tree on 22792 example(s) and 14 feature(s).
[INFO 23-08-16 11:08:07.2630 UTC gradient_boosted_trees.cc:1128] 20533 examples used for training and 2259 examples used for validation
[INFO 23-08-16 11:08:07.2844 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.053989 train-accuracy:0.761895 valid-loss:1.109535 valid-accuracy:0.736609
[INFO 23-08-16 11:08:07.6031 UTC early_stopping.cc:53] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.569154
[INFO 23-08-16 11:08:07.6031 UTC gradient_boosted_trees.cc:247] Truncates the model to 161 tree(s) i.e. 161  iteration(s).
[INFO 23-08-16 11:08:07.6034 UTC gradient_boosted_trees.cc:310] Final model num-trees:161 valid-loss:0.569154 valid-accuracy:0.873838
[INFO 23-08-16 11:08:07.6057 UTC gradient_boosted_trees.cc:459] Default loss set to BINOMIAL_LOG_LIKELIHOOD
[INFO 23-08-16 11:08:07.6058 UTC gradient_boosted_trees.cc:1085] Training gradient boosted tree on 22792 example(s) and 14 feature(s).
[INFO 23-08-16 11:08:07.6059 UTC hyperparameters_optimizer.cc:582] [6/50] Score: -0.569154 / -0.569154 HParams: fields { name: "min_examples" value { integer: 2 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 5 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "shrinkage" value { real: 0.15 } } fields { name: "num_candidate_attributes_ratio" value { real: 1 } }
[INFO 23-08-16 11:08:07.6114 UTC gradient_boosted_trees.cc:1128] 20533 examples used for training and 2259 examples used for validation
[INFO 23-08-16 11:08:07.6632 UTC early_stopping.cc:53] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.578871
[INFO 23-08-16 11:08:07.6632 UTC gradient_boosted_trees.cc:247] Truncates the model to 130 tree(s) i.e. 130  iteration(s).
[INFO 23-08-16 11:08:07.6638 UTC gradient_boosted_trees.cc:310] Final model num-trees:130 valid-loss:0.578871 valid-accuracy:0.869854
[INFO 23-08-16 11:08:07.6667 UTC hyperparameters_optimizer.cc:582] [7/50] Score: -0.578871 / -0.569154 HParams: fields { name: "min_examples" value { integer: 2 } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 32 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "shrinkage" value { real: 0.1 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.2 } }
[INFO 23-08-16 11:08:07.6677 UTC gradient_boosted_trees.cc:459] Default loss set to BINOMIAL_LOG_LIKELIHOOD
[INFO 23-08-16 11:08:07.6677 UTC gradient_boosted_trees.cc:1085] Training gradient boosted tree on 22792 example(s) and 14 feature(s).
[INFO 23-08-16 11:08:07.6714 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:0.981052 train-accuracy:0.761895 valid-loss:1.035441 valid-accuracy:0.736609
[INFO 23-08-16 11:08:07.6733 UTC gradient_boosted_trees.cc:1128] 20533 examples used for training and 2259 examples used for validation
[INFO 23-08-16 11:08:07.7146 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.080688 train-accuracy:0.761895 valid-loss:1.138783 valid-accuracy:0.736609
[INFO 23-08-16 11:08:07.7908 UTC early_stopping.cc:53] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.574698
[INFO 23-08-16 11:08:07.7909 UTC gradient_boosted_trees.cc:247] Truncates the model to 242 tree(s) i.e. 242  iteration(s).
[INFO 23-08-16 11:08:07.7910 UTC gradient_boosted_trees.cc:310] Final model num-trees:242 valid-loss:0.574698 valid-accuracy:0.871625
[INFO 23-08-16 11:08:07.7922 UTC hyperparameters_optimizer.cc:582] [8/50] Score: -0.574698 / -0.569154 HParams: fields { name: "min_examples" value { integer: 2 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 4 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "shrinkage" value { real: 0.15 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.9 } }
[INFO 23-08-16 11:08:07.7931 UTC gradient_boosted_trees.cc:459] Default loss set to BINOMIAL_LOG_LIKELIHOOD
[INFO 23-08-16 11:08:07.7932 UTC gradient_boosted_trees.cc:1085] Training gradient boosted tree on 22792 example(s) and 14 feature(s).
[INFO 23-08-16 11:08:07.7991 UTC gradient_boosted_trees.cc:1128] 20533 examples used for training and 2259 examples used for validation
[INFO 23-08-16 11:08:07.8445 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.082622 train-accuracy:0.761895 valid-loss:1.140940 valid-accuracy:0.736609
[INFO 23-08-16 11:08:08.2101 UTC gradient_boosted_trees.cc:1542]  num-trees:300 train-loss:0.488794 train-accuracy:0.890031 valid-loss:0.571949 valid-accuracy:0.873395
[INFO 23-08-16 11:08:08.2101 UTC gradient_boosted_trees.cc:247] Truncates the model to 284 tree(s) i.e. 284  iteration(s).
[INFO 23-08-16 11:08:08.2102 UTC gradient_boosted_trees.cc:310] Final model num-trees:284 valid-loss:0.571257 valid-accuracy:0.872953
[INFO 23-08-16 11:08:08.2127 UTC hyperparameters_optimizer.cc:582] [9/50] Score: -0.571257 / -0.569154 HParams: fields { name: "min_examples" value { integer: 10 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 5 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "shrinkage" value { real: 0.1 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.5 } }
[INFO 23-08-16 11:08:08.2158 UTC gradient_boosted_trees.cc:459] Default loss set to BINOMIAL_LOG_LIKELIHOOD
[INFO 23-08-16 11:08:08.2158 UTC gradient_boosted_trees.cc:1085] Training gradient boosted tree on 22792 example(s) and 14 feature(s).
[INFO 23-08-16 11:08:08.2204 UTC gradient_boosted_trees.cc:1128] 20533 examples used for training and 2259 examples used for validation
[INFO 23-08-16 11:08:08.2651 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:0.991671 train-accuracy:0.761895 valid-loss:1.045193 valid-accuracy:0.736609
[INFO 23-08-16 11:08:09.2840 UTC gradient_boosted_trees.cc:1542]  num-trees:300 train-loss:0.567582 train-accuracy:0.871475 valid-loss:0.596684 valid-accuracy:0.865870
[INFO 23-08-16 11:08:09.2840 UTC gradient_boosted_trees.cc:247] Truncates the model to 300 tree(s) i.e. 300  iteration(s).
[INFO 23-08-16 11:08:09.2840 UTC gradient_boosted_trees.cc:310] Final model num-trees:300 valid-loss:0.596684 valid-accuracy:0.865870
[INFO 23-08-16 11:08:09.2850 UTC hyperparameters_optimizer.cc:582] [10/50] Score: -0.596684 / -0.569154 HParams: fields { name: "min_examples" value { integer: 2 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 3 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "shrinkage" value { real: 0.1 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.2 } }
[INFO 23-08-16 11:08:09.2866 UTC gradient_boosted_trees.cc:459] Default loss set to BINOMIAL_LOG_LIKELIHOOD
[INFO 23-08-16 11:08:09.2867 UTC gradient_boosted_trees.cc:1085] Training gradient boosted tree on 22792 example(s) and 14 feature(s).
[INFO 23-08-16 11:08:09.2915 UTC gradient_boosted_trees.cc:1128] 20533 examples used for training and 2259 examples used for validation
[INFO 23-08-16 11:08:09.3449 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.015325 train-accuracy:0.761895 valid-loss:1.070753 valid-accuracy:0.736609
[INFO 23-08-16 11:08:09.6511 UTC gradient_boosted_trees.cc:1542]  num-trees:300 train-loss:0.499704 train-accuracy:0.890712 valid-loss:0.584889 valid-accuracy:0.869854
[INFO 23-08-16 11:08:09.6511 UTC gradient_boosted_trees.cc:247] Truncates the model to 298 tree(s) i.e. 298  iteration(s).
[INFO 23-08-16 11:08:09.6512 UTC gradient_boosted_trees.cc:310] Final model num-trees:298 valid-loss:0.584790 valid-accuracy:0.869411
[INFO 23-08-16 11:08:09.6544 UTC hyperparameters_optimizer.cc:582] [11/50] Score: -0.58479 / -0.569154 HParams: fields { name: "min_examples" value { integer: 7 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 16 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "shrinkage" value { real: 0.05 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.2 } }
[INFO 23-08-16 11:08:09.6593 UTC gradient_boosted_trees.cc:459] Default loss set to BINOMIAL_LOG_LIKELIHOOD
[INFO 23-08-16 11:08:09.6594 UTC gradient_boosted_trees.cc:1085] Training gradient boosted tree on 22792 example(s) and 14 feature(s).
[INFO 23-08-16 11:08:09.6638 UTC gradient_boosted_trees.cc:1128] 20533 examples used for training and 2259 examples used for validation
[INFO 23-08-16 11:08:09.7372 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.056130 train-accuracy:0.761895 valid-loss:1.113107 valid-accuracy:0.736609
[INFO 23-08-16 11:08:09.7959 UTC early_stopping.cc:53] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.578549
[INFO 23-08-16 11:08:09.7959 UTC gradient_boosted_trees.cc:247] Truncates the model to 105 tree(s) i.e. 105  iteration(s).
[INFO 23-08-16 11:08:09.7971 UTC gradient_boosted_trees.cc:310] Final model num-trees:105 valid-loss:0.578549 valid-accuracy:0.871625
[INFO 23-08-16 11:08:09.8037 UTC hyperparameters_optimizer.cc:582] [12/50] Score: -0.578549 / -0.569154 HParams: fields { name: "min_examples" value { integer: 2 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 8 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "shrinkage" value { real: 0.1 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.2 } }
[INFO 23-08-16 11:08:09.8107 UTC gradient_boosted_trees.cc:459] Default loss set to BINOMIAL_LOG_LIKELIHOOD
[INFO 23-08-16 11:08:09.8107 UTC gradient_boosted_trees.cc:1085] Training gradient boosted tree on 22792 example(s) and 14 feature(s).
[INFO 23-08-16 11:08:09.8152 UTC gradient_boosted_trees.cc:1128] 20533 examples used for training and 2259 examples used for validation
[INFO 23-08-16 11:08:09.8803 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.081456 train-accuracy:0.761895 valid-loss:1.139474 valid-accuracy:0.736609
[INFO 23-08-16 11:08:10.0550 UTC early_stopping.cc:53] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.575113
[INFO 23-08-16 11:08:10.0551 UTC gradient_boosted_trees.cc:247] Truncates the model to 242 tree(s) i.e. 242  iteration(s).
[INFO 23-08-16 11:08:10.0553 UTC gradient_boosted_trees.cc:310] Final model num-trees:242 valid-loss:0.575113 valid-accuracy:0.870297
[INFO 23-08-16 11:08:10.0575 UTC hyperparameters_optimizer.cc:582] [13/50] Score: -0.575113 / -0.569154 HParams: fields { name: "min_examples" value { integer: 7 } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 5 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "shrinkage" value { real: 0.1 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.5 } }
[INFO 23-08-16 11:08:10.0586 UTC gradient_boosted_trees.cc:459] Default loss set to BINOMIAL_LOG_LIKELIHOOD
[INFO 23-08-16 11:08:10.0586 UTC gradient_boosted_trees.cc:1085] Training gradient boosted tree on 22792 example(s) and 14 feature(s).
[INFO 23-08-16 11:08:10.0638 UTC gradient_boosted_trees.cc:1128] 20533 examples used for training and 2259 examples used for validation
[INFO 23-08-16 11:08:10.1122 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.010652 train-accuracy:0.761895 valid-loss:1.064824 valid-accuracy:0.736609
[INFO 23-08-16 11:08:10.3474 UTC early_stopping.cc:53] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.574784
[INFO 23-08-16 11:08:10.3474 UTC gradient_boosted_trees.cc:247] Truncates the model to 249 tree(s) i.e. 249  iteration(s).
[INFO 23-08-16 11:08:10.3476 UTC gradient_boosted_trees.cc:310] Final model num-trees:249 valid-loss:0.574784 valid-accuracy:0.867641
[INFO 23-08-16 11:08:10.3491 UTC hyperparameters_optimizer.cc:582] [14/50] Score: -0.574784 / -0.569154 HParams: fields { name: "min_examples" value { integer: 7 } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 4 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "shrinkage" value { real: 0.15 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.9 } }
[INFO 23-08-16 11:08:10.3509 UTC gradient_boosted_trees.cc:459] Default loss set to BINOMIAL_LOG_LIKELIHOOD
[INFO 23-08-16 11:08:10.3510 UTC gradient_boosted_trees.cc:1085] Training gradient boosted tree on 22792 example(s) and 14 feature(s).
[INFO 23-08-16 11:08:10.3566 UTC gradient_boosted_trees.cc:1128] 20533 examples used for training and 2259 examples used for validation
[INFO 23-08-16 11:08:10.4110 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.013729 train-accuracy:0.761895 valid-loss:1.069266 valid-accuracy:0.736609
[INFO 23-08-16 11:08:10.8116 UTC early_stopping.cc:53] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.577748
[INFO 23-08-16 11:08:10.8117 UTC gradient_boosted_trees.cc:247] Truncates the model to 89 tree(s) i.e. 89  iteration(s).
[INFO 23-08-16 11:08:10.8120 UTC gradient_boosted_trees.cc:310] Final model num-trees:89 valid-loss:0.577748 valid-accuracy:0.871625
[INFO 23-08-16 11:08:10.8133 UTC hyperparameters_optimizer.cc:582] [15/50] Score: -0.577748 / -0.569154 HParams: fields { name: "min_examples" value { integer: 7 } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 16 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "shrinkage" value { real: 0.15 } } fields { name: "num_candidate_attributes_ratio" value { real: 1 } }
[INFO 23-08-16 11:08:10.8143 UTC gradient_boosted_trees.cc:459] Default loss set to BINOMIAL_LOG_LIKELIHOOD
[INFO 23-08-16 11:08:10.8143 UTC gradient_boosted_trees.cc:1085] Training gradient boosted tree on 22792 example(s) and 14 feature(s).
[INFO 23-08-16 11:08:10.8194 UTC gradient_boosted_trees.cc:1128] 20533 examples used for training and 2259 examples used for validation
[INFO 23-08-16 11:08:10.8644 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.009908 train-accuracy:0.761895 valid-loss:1.065147 valid-accuracy:0.736609
[INFO 23-08-16 11:08:11.2479 UTC early_stopping.cc:53] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.578216
[INFO 23-08-16 11:08:11.2480 UTC gradient_boosted_trees.cc:247] Truncates the model to 195 tree(s) i.e. 195  iteration(s).
[INFO 23-08-16 11:08:11.2489 UTC gradient_boosted_trees.cc:310] Final model num-trees:195 valid-loss:0.578216 valid-accuracy:0.869854
[INFO 23-08-16 11:08:11.2556 UTC hyperparameters_optimizer.cc:582] [16/50] Score: -0.578216 / -0.569154 HParams: fields { name: "min_examples" value { integer: 5 } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 256 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "shrinkage" value { real: 0.05 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.2 } }
[INFO 23-08-16 11:08:11.2566 UTC gradient_boosted_trees.cc:459] Default loss set to BINOMIAL_LOG_LIKELIHOOD
[INFO 23-08-16 11:08:11.2566 UTC gradient_boosted_trees.cc:1085] Training gradient boosted tree on 22792 example(s) and 14 feature(s).
[INFO 23-08-16 11:08:11.2647 UTC gradient_boosted_trees.cc:1128] 20533 examples used for training and 2259 examples used for validation
[INFO 23-08-16 11:08:11.3506 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.079317 train-accuracy:0.761895 valid-loss:1.137114 valid-accuracy:0.736609
[INFO 23-08-16 11:08:11.3940 UTC gradient_boosted_trees.cc:1542]  num-trees:300 train-loss:0.552215 train-accuracy:0.876248 valid-loss:0.594582 valid-accuracy:0.869411
[INFO 23-08-16 11:08:11.3940 UTC gradient_boosted_trees.cc:247] Truncates the model to 294 tree(s) i.e. 294  iteration(s).
[INFO 23-08-16 11:08:11.3941 UTC gradient_boosted_trees.cc:310] Final model num-trees:294 valid-loss:0.594392 valid-accuracy:0.868969
[INFO 23-08-16 11:08:11.3949 UTC hyperparameters_optimizer.cc:582] [17/50] Score: -0.594392 / -0.569154 HParams: fields { name: "min_examples" value { integer: 2 } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 3 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "shrinkage" value { real: 0.1 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.9 } }
[INFO 23-08-16 11:08:11.3962 UTC gradient_boosted_trees.cc:459] Default loss set to BINOMIAL_LOG_LIKELIHOOD
[INFO 23-08-16 11:08:11.3963 UTC gradient_boosted_trees.cc:1085] Training gradient boosted tree on 22792 example(s) and 14 feature(s).
[INFO 23-08-16 11:08:11.4011 UTC gradient_boosted_trees.cc:1128] 20533 examples used for training and 2259 examples used for validation
[INFO 23-08-16 11:08:11.4157 UTC gradient_boosted_trees.cc:1542]  num-trees:300 train-loss:0.515704 train-accuracy:0.886719 valid-loss:0.592440 valid-accuracy:0.868083
[INFO 23-08-16 11:08:11.4158 UTC gradient_boosted_trees.cc:247] Truncates the model to 300 tree(s) i.e. 300  iteration(s).
[INFO 23-08-16 11:08:11.4158 UTC gradient_boosted_trees.cc:310] Final model num-trees:300 valid-loss:0.592440 valid-accuracy:0.868083
[INFO 23-08-16 11:08:11.4257 UTC hyperparameters_optimizer.cc:582] [18/50] Score: -0.59244 / -0.569154 HParams: fields { name: "min_examples" value { integer: 10 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 64 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "shrinkage" value { real: 0.02 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.2 } }
[INFO 23-08-16 11:08:11.4423 UTC gradient_boosted_trees.cc:459] Default loss set to BINOMIAL_LOG_LIKELIHOOD
[INFO 23-08-16 11:08:11.4423 UTC gradient_boosted_trees.cc:1085] Training gradient boosted tree on 22792 example(s) and 14 feature(s).
[INFO 23-08-16 11:08:11.4468 UTC gradient_boosted_trees.cc:1128] 20533 examples used for training and 2259 examples used for validation
[INFO 23-08-16 11:08:11.4613 UTC early_stopping.cc:53] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.579733
[INFO 23-08-16 11:08:11.4613 UTC gradient_boosted_trees.cc:247] Truncates the model to 78 tree(s) i.e. 78  iteration(s).
[INFO 23-08-16 11:08:11.4627 UTC gradient_boosted_trees.cc:310] Final model num-trees:78 valid-loss:0.579733 valid-accuracy:0.867641
[INFO 23-08-16 11:08:11.4684 UTC hyperparameters_optimizer.cc:582] [19/50] Score: -0.579733 / -0.569154 HParams: fields { name: "min_examples" value { integer: 2 } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 8 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "shrinkage" value { real: 0.1 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.5 } }
[INFO 23-08-16 11:08:11.4914 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.055888 train-accuracy:0.761895 valid-loss:1.113012 valid-accuracy:0.736609
[INFO 23-08-16 11:08:11.4956 UTC early_stopping.cc:53] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.586668
[INFO 23-08-16 11:08:11.4956 UTC gradient_boosted_trees.cc:247] Truncates the model to 61 tree(s) i.e. 61  iteration(s).
[INFO 23-08-16 11:08:11.4962 UTC gradient_boosted_trees.cc:310] Final model num-trees:61 valid-loss:0.586668 valid-accuracy:0.868969
[INFO 23-08-16 11:08:11.4974 UTC hyperparameters_optimizer.cc:582] [20/50] Score: -0.586668 / -0.569154 HParams: fields { name: "min_examples" value { integer: 10 } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 16 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "shrinkage" value { real: 0.15 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.5 } }
[INFO 23-08-16 11:08:11.5397 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.012875 train-accuracy:0.761895 valid-loss:1.067941 valid-accuracy:0.736609
[INFO 23-08-16 11:08:12.4754 UTC early_stopping.cc:53] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.576467
[INFO 23-08-16 11:08:12.4754 UTC gradient_boosted_trees.cc:247] Truncates the model to 147 tree(s) i.e. 147  iteration(s).
[INFO 23-08-16 11:08:12.4757 UTC gradient_boosted_trees.cc:310] Final model num-trees:147 valid-loss:0.576467 valid-accuracy:0.870739
[INFO 23-08-16 11:08:12.4773 UTC hyperparameters_optimizer.cc:582] [21/50] Score: -0.576467 / -0.569154 HParams: fields { name: "min_examples" value { integer: 5 } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 5 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "shrinkage" value { real: 0.15 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.5 } }
[INFO 23-08-16 11:08:12.8409 UTC gradient_boosted_trees.cc:1542]  num-trees:300 train-loss:0.498302 train-accuracy:0.892222 valid-loss:0.585352 valid-accuracy:0.870297
[INFO 23-08-16 11:08:12.8409 UTC gradient_boosted_trees.cc:247] Truncates the model to 296 tree(s) i.e. 296  iteration(s).
[INFO 23-08-16 11:08:12.8410 UTC gradient_boosted_trees.cc:310] Final model num-trees:296 valid-loss:0.585279 valid-accuracy:0.870297
[INFO 23-08-16 11:08:12.8441 UTC hyperparameters_optimizer.cc:582] [22/50] Score: -0.585279 / -0.569154 HParams: fields { name: "min_examples" value { integer: 10 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 16 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "shrinkage" value { real: 0.05 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.2 } }
[INFO 23-08-16 11:08:13.1007 UTC early_stopping.cc:53] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.579464
[INFO 23-08-16 11:08:13.1008 UTC gradient_boosted_trees.cc:247] Truncates the model to 129 tree(s) i.e. 129  iteration(s).
[INFO 23-08-16 11:08:13.1018 UTC gradient_boosted_trees.cc:310] Final model num-trees:129 valid-loss:0.579464 valid-accuracy:0.870297
[INFO 23-08-16 11:08:13.1084 UTC hyperparameters_optimizer.cc:582] [23/50] Score: -0.579464 / -0.569154 HParams: fields { name: "min_examples" value { integer: 5 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 128 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "shrinkage" value { real: 0.1 } } fields { name: "num_candidate_attributes_ratio" value { real: 1 } }
[INFO 23-08-16 11:08:13.1165 UTC gradient_boosted_trees.cc:1542]  num-trees:300 train-loss:0.511416 train-accuracy:0.884381 valid-loss:0.572223 valid-accuracy:0.874723
[INFO 23-08-16 11:08:13.1166 UTC gradient_boosted_trees.cc:247] Truncates the model to 291 tree(s) i.e. 291  iteration(s).
[INFO 23-08-16 11:08:13.1166 UTC gradient_boosted_trees.cc:310] Final model num-trees:291 valid-loss:0.572029 valid-accuracy:0.874723
[INFO 23-08-16 11:08:13.1187 UTC hyperparameters_optimizer.cc:582] [24/50] Score: -0.572029 / -0.569154 HParams: fields { name: "min_examples" value { integer: 10 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 16 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "shrinkage" value { real: 0.05 } } fields { name: "num_candidate_attributes_ratio" value { real: 1 } }
[INFO 23-08-16 11:08:13.2108 UTC early_stopping.cc:53] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.578696
[INFO 23-08-16 11:08:13.2108 UTC gradient_boosted_trees.cc:247] Truncates the model to 228 tree(s) i.e. 228  iteration(s).
[INFO 23-08-16 11:08:13.2115 UTC gradient_boosted_trees.cc:310] Final model num-trees:228 valid-loss:0.578696 valid-accuracy:0.870739
[INFO 23-08-16 11:08:13.2163 UTC hyperparameters_optimizer.cc:582] [25/50] Score: -0.578696 / -0.569154 HParams: fields { name: "min_examples" value { integer: 5 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 32 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "shrinkage" value { real: 0.05 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.5 } }
[INFO 23-08-16 11:08:13.4696 UTC early_stopping.cc:53] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.574801
[INFO 23-08-16 11:08:13.4696 UTC gradient_boosted_trees.cc:247] Truncates the model to 83 tree(s) i.e. 83  iteration(s).
[INFO 23-08-16 11:08:13.4702 UTC gradient_boosted_trees.cc:310] Final model num-trees:83 valid-loss:0.574801 valid-accuracy:0.870297
[INFO 23-08-16 11:08:13.4733 UTC hyperparameters_optimizer.cc:582] [26/50] Score: -0.574801 / -0.569154 HParams: fields { name: "min_examples" value { integer: 7 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 8 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "shrinkage" value { real: 0.1 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.5 } }
[INFO 23-08-16 11:08:13.8788 UTC gradient_boosted_trees.cc:1542]  num-trees:300 train-loss:0.523350 train-accuracy:0.883992 valid-loss:0.583351 valid-accuracy:0.868526
[INFO 23-08-16 11:08:13.8788 UTC gradient_boosted_trees.cc:247] Truncates the model to 300 tree(s) i.e. 300  iteration(s).
[INFO 23-08-16 11:08:13.8789 UTC gradient_boosted_trees.cc:310] Final model num-trees:300 valid-loss:0.583351 valid-accuracy:0.868526
[INFO 23-08-16 11:08:13.8882 UTC hyperparameters_optimizer.cc:582] [27/50] Score: -0.583351 / -0.569154 HParams: fields { name: "min_examples" value { integer: 7 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 128 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "shrinkage" value { real: 0.02 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.5 } }
[INFO 23-08-16 11:08:13.9364 UTC early_stopping.cc:53] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.587919
[INFO 23-08-16 11:08:13.9365 UTC gradient_boosted_trees.cc:247] Truncates the model to 86 tree(s) i.e. 86  iteration(s).
[INFO 23-08-16 11:08:13.9380 UTC gradient_boosted_trees.cc:310] Final model num-trees:86 valid-loss:0.587919 valid-accuracy:0.866755
[INFO 23-08-16 11:08:13.9434 UTC hyperparameters_optimizer.cc:582] [28/50] Score: -0.587919 / -0.569154 HParams: fields { name: "min_examples" value { integer: 2 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 8 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "shrinkage" value { real: 0.1 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.2 } }
[INFO 23-08-16 11:08:14.2934 UTC early_stopping.cc:53] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.575133
[INFO 23-08-16 11:08:14.2934 UTC gradient_boosted_trees.cc:247] Truncates the model to 174 tree(s) i.e. 174  iteration(s).
[INFO 23-08-16 11:08:14.2941 UTC gradient_boosted_trees.cc:310] Final model num-trees:174 valid-loss:0.575133 valid-accuracy:0.872067
[INFO 23-08-16 11:08:14.3012 UTC hyperparameters_optimizer.cc:582] [29/50] Score: -0.575133 / -0.569154 HParams: fields { name: "min_examples" value { integer: 5 } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 8 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "shrinkage" value { real: 0.05 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.5 } }
[INFO 23-08-16 11:08:14.4387 UTC early_stopping.cc:53] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.578764
[INFO 23-08-16 11:08:14.4388 UTC gradient_boosted_trees.cc:247] Truncates the model to 186 tree(s) i.e. 186  iteration(s).
[INFO 23-08-16 11:08:14.4393 UTC gradient_boosted_trees.cc:310] Final model num-trees:186 valid-loss:0.578764 valid-accuracy:0.873395
[INFO 23-08-16 11:08:14.4425 UTC hyperparameters_optimizer.cc:582] [30/50] Score: -0.578764 / -0.569154 HParams: fields { name: "min_examples" value { integer: 5 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 6 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "shrinkage" value { real: 0.1 } } fields { name: "num_candidate_attributes_ratio" value { real: 1 } }
[INFO 23-08-16 11:08:14.5569 UTC gradient_boosted_trees.cc:1542]  num-trees:300 train-loss:0.523150 train-accuracy:0.886135 valid-loss:0.593369 valid-accuracy:0.869411
[INFO 23-08-16 11:08:14.5569 UTC gradient_boosted_trees.cc:247] Truncates the model to 300 tree(s) i.e. 300  iteration(s).
[INFO 23-08-16 11:08:14.5570 UTC gradient_boosted_trees.cc:310] Final model num-trees:300 valid-loss:0.593369 valid-accuracy:0.869411
[INFO 23-08-16 11:08:14.5615 UTC hyperparameters_optimizer.cc:582] [31/50] Score: -0.593369 / -0.569154 HParams: fields { name: "min_examples" value { integer: 2 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 32 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "shrinkage" value { real: 0.02 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.2 } }
[INFO 23-08-16 11:08:14.6584 UTC early_stopping.cc:53] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.57629
[INFO 23-08-16 11:08:14.6584 UTC gradient_boosted_trees.cc:247] Truncates the model to 151 tree(s) i.e. 151  iteration(s).
[INFO 23-08-16 11:08:14.6587 UTC gradient_boosted_trees.cc:310] Final model num-trees:151 valid-loss:0.576290 valid-accuracy:0.869411
[INFO 23-08-16 11:08:14.6600 UTC hyperparameters_optimizer.cc:582] [32/50] Score: -0.57629 / -0.569154 HParams: fields { name: "min_examples" value { integer: 10 } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 16 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "shrinkage" value { real: 0.1 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.9 } }
[INFO 23-08-16 11:08:15.0826 UTC early_stopping.cc:53] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.568287
[INFO 23-08-16 11:08:15.0827 UTC gradient_boosted_trees.cc:247] Truncates the model to 117 tree(s) i.e. 117  iteration(s).
[INFO 23-08-16 11:08:15.0832 UTC gradient_boosted_trees.cc:310] Final model num-trees:117 valid-loss:0.568287 valid-accuracy:0.873395
[INFO 23-08-16 11:08:15.0872 UTC hyperparameters_optimizer.cc:582] [33/50] Score: -0.568287 / -0.568287 HParams: fields { name: "min_examples" value { integer: 5 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 256 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "shrinkage" value { real: 0.1 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.9 } }
[INFO 23-08-16 11:08:15.3037 UTC early_stopping.cc:53] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.571936
[INFO 23-08-16 11:08:15.3038 UTC gradient_boosted_trees.cc:247] Truncates the model to 159 tree(s) i.e. 159  iteration(s).
[INFO 23-08-16 11:08:15.3041 UTC gradient_boosted_trees.cc:310] Final model num-trees:159 valid-loss:0.571936 valid-accuracy:0.872067
[INFO 23-08-16 11:08:15.3064 UTC hyperparameters_optimizer.cc:582] [34/50] Score: -0.571936 / -0.568287 HParams: fields { name: "min_examples" value { integer: 7 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 32 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "shrinkage" value { real: 0.1 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.9 } }
[INFO 23-08-16 11:08:15.6233 UTC gradient_boosted_trees.cc:1542]  num-trees:300 train-loss:0.481432 train-accuracy:0.895339 valid-loss:0.584104 valid-accuracy:0.869854
[INFO 23-08-16 11:08:15.6234 UTC gradient_boosted_trees.cc:247] Truncates the model to 300 tree(s) i.e. 300  iteration(s).
[INFO 23-08-16 11:08:15.6234 UTC gradient_boosted_trees.cc:310] Final model num-trees:300 valid-loss:0.584104 valid-accuracy:0.869854
[INFO 23-08-16 11:08:15.6356 UTC hyperparameters_optimizer.cc:582] [35/50] Score: -0.584104 / -0.568287 HParams: fields { name: "min_examples" value { integer: 5 } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 8 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "shrinkage" value { real: 0.02 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.2 } }
[INFO 23-08-16 11:08:15.9841 UTC early_stopping.cc:53] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.581114
[INFO 23-08-16 11:08:15.9842 UTC gradient_boosted_trees.cc:247] Truncates the model to 242 tree(s) i.e. 242  iteration(s).
[INFO 23-08-16 11:08:15.9845 UTC gradient_boosted_trees.cc:310] Final model num-trees:242 valid-loss:0.581114 valid-accuracy:0.867198
[INFO 23-08-16 11:08:15.9866 UTC hyperparameters_optimizer.cc:582] [36/50] Score: -0.581114 / -0.568287 HParams: fields { name: "min_examples" value { integer: 7 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 16 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "shrinkage" value { real: 0.05 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.9 } }
[INFO 23-08-16 11:08:16.2511 UTC gradient_boosted_trees.cc:1542]  num-trees:300 train-loss:0.542830 train-accuracy:0.881654 valid-loss:0.593285 valid-accuracy:0.867198
[INFO 23-08-16 11:08:16.2511 UTC gradient_boosted_trees.cc:247] Truncates the model to 300 tree(s) i.e. 300  iteration(s).
[INFO 23-08-16 11:08:16.2511 UTC gradient_boosted_trees.cc:310] Final model num-trees:300 valid-loss:0.593285 valid-accuracy:0.867198
[INFO 23-08-16 11:08:16.2534 UTC hyperparameters_optimizer.cc:582] [37/50] Score: -0.593285 / -0.568287 HParams: fields { name: "min_examples" value { integer: 5 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 16 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "shrinkage" value { real: 0.02 } } fields { name: "num_candidate_attributes_ratio" value { real: 1 } }
[INFO 23-08-16 11:08:16.5678 UTC gradient_boosted_trees.cc:1542]  num-trees:300 train-loss:0.588445 train-accuracy:0.867433 valid-loss:0.620650 valid-accuracy:0.862328
[INFO 23-08-16 11:08:16.5678 UTC gradient_boosted_trees.cc:247] Truncates the model to 300 tree(s) i.e. 300  iteration(s).
[INFO 23-08-16 11:08:16.5678 UTC gradient_boosted_trees.cc:310] Final model num-trees:300 valid-loss:0.620650 valid-accuracy:0.862328
[INFO 23-08-16 11:08:16.5688 UTC hyperparameters_optimizer.cc:582] [38/50] Score: -0.62065 / -0.568287 HParams: fields { name: "min_examples" value { integer: 5 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 4 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "shrinkage" value { real: 0.02 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.9 } }
[INFO 23-08-16 11:08:17.2407 UTC gradient_boosted_trees.cc:1542]  num-trees:300 train-loss:0.542858 train-accuracy:0.881800 valid-loss:0.595354 valid-accuracy:0.866755
[INFO 23-08-16 11:08:17.2407 UTC gradient_boosted_trees.cc:247] Truncates the model to 300 tree(s) i.e. 300  iteration(s).
[INFO 23-08-16 11:08:17.2408 UTC gradient_boosted_trees.cc:310] Final model num-trees:300 valid-loss:0.595354 valid-accuracy:0.866755
[INFO 23-08-16 11:08:17.2429 UTC hyperparameters_optimizer.cc:582] [39/50] Score: -0.595354 / -0.568287 HParams: fields { name: "min_examples" value { integer: 2 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 16 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "shrinkage" value { real: 0.02 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.9 } }
[INFO 23-08-16 11:08:17.3207 UTC gradient_boosted_trees.cc:1542]  num-trees:300 train-loss:0.519893 train-accuracy:0.882725 valid-loss:0.589835 valid-accuracy:0.868083
[INFO 23-08-16 11:08:17.3207 UTC gradient_boosted_trees.cc:247] Truncates the model to 300 tree(s) i.e. 300  iteration(s).
[INFO 23-08-16 11:08:17.3207 UTC gradient_boosted_trees.cc:310] Final model num-trees:300 valid-loss:0.589835 valid-accuracy:0.868083
[INFO 23-08-16 11:08:17.3268 UTC hyperparameters_optimizer.cc:582] [40/50] Score: -0.589835 / -0.568287 HParams: fields { name: "min_examples" value { integer: 7 } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 128 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "shrinkage" value { real: 0.02 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.2 } }
[INFO 23-08-16 11:08:18.2298 UTC gradient_boosted_trees.cc:1542]  num-trees:300 train-loss:0.510029 train-accuracy:0.890566 valid-loss:0.588633 valid-accuracy:0.866755
[INFO 23-08-16 11:08:18.2299 UTC gradient_boosted_trees.cc:247] Truncates the model to 299 tree(s) i.e. 299  iteration(s).
[INFO 23-08-16 11:08:18.2299 UTC gradient_boosted_trees.cc:310] Final model num-trees:299 valid-loss:0.588569 valid-accuracy:0.865870
[INFO 23-08-16 11:08:18.2339 UTC hyperparameters_optimizer.cc:582] [41/50] Score: -0.588569 / -0.568287 HParams: fields { name: "min_examples" value { integer: 2 } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 32 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "shrinkage" value { real: 0.02 } } fields { name: "num_candidate_attributes_ratio" value { real: 1 } }
[INFO 23-08-16 11:08:18.7000 UTC early_stopping.cc:53] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.566897
[INFO 23-08-16 11:08:18.7000 UTC gradient_boosted_trees.cc:247] Truncates the model to 112 tree(s) i.e. 112  iteration(s).
[INFO 23-08-16 11:08:18.7005 UTC gradient_boosted_trees.cc:310] Final model num-trees:112 valid-loss:0.566897 valid-accuracy:0.873395
[INFO 23-08-16 11:08:18.7053 UTC hyperparameters_optimizer.cc:582] [42/50] Score: -0.566897 / -0.566897 HParams: fields { name: "min_examples" value { integer: 2 } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 64 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "shrinkage" value { real: 0.1 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.9 } }
[INFO 23-08-16 11:08:20.3548 UTC gradient_boosted_trees.cc:1542]  num-trees:300 train-loss:0.466037 train-accuracy:0.896557 valid-loss:0.571031 valid-accuracy:0.869854
[INFO 23-08-16 11:08:20.3548 UTC gradient_boosted_trees.cc:247] Truncates the model to 294 tree(s) i.e. 294  iteration(s).
[INFO 23-08-16 11:08:20.3549 UTC gradient_boosted_trees.cc:310] Final model num-trees:294 valid-loss:0.570658 valid-accuracy:0.871625
[INFO 23-08-16 11:08:20.3583 UTC hyperparameters_optimizer.cc:582] [43/50] Score: -0.570658 / -0.566897 HParams: fields { name: "min_examples" value { integer: 5 } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 6 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "shrinkage" value { real: 0.05 } } fields { name: "num_candidate_attributes_ratio" value { real: 1 } }
[INFO 23-08-16 11:08:20.5117 UTC gradient_boosted_trees.cc:1542]  num-trees:300 train-loss:0.466786 train-accuracy:0.898115 valid-loss:0.574857 valid-accuracy:0.870739
[INFO 23-08-16 11:08:20.5118 UTC gradient_boosted_trees.cc:247] Truncates the model to 293 tree(s) i.e. 293  iteration(s).
[INFO 23-08-16 11:08:20.5119 UTC gradient_boosted_trees.cc:310] Final model num-trees:293 valid-loss:0.574461 valid-accuracy:0.870739
[INFO 23-08-16 11:08:20.5151 UTC hyperparameters_optimizer.cc:582] [44/50] Score: -0.574461 / -0.566897 HParams: fields { name: "min_examples" value { integer: 10 } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 6 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "shrinkage" value { real: 0.05 } } fields { name: "num_candidate_attributes_ratio" value { real: 1 } }
[INFO 23-08-16 11:08:20.7569 UTC early_stopping.cc:53] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.572158
[INFO 23-08-16 11:08:20.7569 UTC gradient_boosted_trees.cc:247] Truncates the model to 209 tree(s) i.e. 209  iteration(s).
[INFO 23-08-16 11:08:20.7574 UTC gradient_boosted_trees.cc:310] Final model num-trees:209 valid-loss:0.572158 valid-accuracy:0.872953
[INFO 23-08-16 11:08:20.7623 UTC hyperparameters_optimizer.cc:582] [45/50] Score: -0.572158 / -0.566897 HParams: fields { name: "min_examples" value { integer: 7 } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 64 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "shrinkage" value { real: 0.05 } } fields { name: "num_candidate_attributes_ratio" value { real: 1 } }
[INFO 23-08-16 11:08:21.3810 UTC gradient_boosted_trees.cc:1542]  num-trees:300 train-loss:0.524785 train-accuracy:0.882871 valid-loss:0.586497 valid-accuracy:0.869411
[INFO 23-08-16 11:08:21.3810 UTC gradient_boosted_trees.cc:247] Truncates the model to 300 tree(s) i.e. 300  iteration(s).
[INFO 23-08-16 11:08:21.3810 UTC gradient_boosted_trees.cc:310] Final model num-trees:300 valid-loss:0.586497 valid-accuracy:0.869411
[INFO 23-08-16 11:08:21.3853 UTC hyperparameters_optimizer.cc:582] [46/50] Score: -0.586497 / -0.566897 HParams: fields { name: "min_examples" value { integer: 2 } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 32 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "shrinkage" value { real: 0.02 } } fields { name: "num_candidate_attributes_ratio" value { real: 1 } }
[INFO 23-08-16 11:08:21.8747 UTC gradient_boosted_trees.cc:1542]  num-trees:300 train-loss:0.531623 train-accuracy:0.882871 valid-loss:0.586301 valid-accuracy:0.871625
[INFO 23-08-16 11:08:21.8748 UTC gradient_boosted_trees.cc:247] Truncates the model to 300 tree(s) i.e. 300  iteration(s).
[INFO 23-08-16 11:08:21.8748 UTC gradient_boosted_trees.cc:310] Final model num-trees:300 valid-loss:0.586301 valid-accuracy:0.871625
[INFO 23-08-16 11:08:21.8784 UTC hyperparameters_optimizer.cc:582] [47/50] Score: -0.586301 / -0.566897 HParams: fields { name: "min_examples" value { integer: 5 } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 6 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "shrinkage" value { real: 0.02 } } fields { name: "num_candidate_attributes_ratio" value { real: 1 } }
[INFO 23-08-16 11:08:24.0003 UTC gradient_boosted_trees.cc:1542]  num-trees:300 train-loss:0.479808 train-accuracy:0.892174 valid-loss:0.580708 valid-accuracy:0.871182
[INFO 23-08-16 11:08:24.0004 UTC gradient_boosted_trees.cc:247] Truncates the model to 299 tree(s) i.e. 299  iteration(s).
[INFO 23-08-16 11:08:24.0004 UTC gradient_boosted_trees.cc:310] Final model num-trees:299 valid-loss:0.580696 valid-accuracy:0.871625
[INFO 23-08-16 11:08:24.0105 UTC hyperparameters_optimizer.cc:582] [48/50] Score: -0.580696 / -0.566897 HParams: fields { name: "min_examples" value { integer: 7 } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 8 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "shrinkage" value { real: 0.02 } } fields { name: "num_candidate_attributes_ratio" value { real: 0.5 } }
[INFO 23-08-16 11:08:24.6673 UTC early_stopping.cc:53] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.573524
[INFO 23-08-16 11:08:24.6673 UTC gradient_boosted_trees.cc:247] Truncates the model to 252 tree(s) i.e. 252  iteration(s).
[INFO 23-08-16 11:08:24.6678 UTC gradient_boosted_trees.cc:310] Final model num-trees:252 valid-loss:0.573524 valid-accuracy:0.868083
[INFO 23-08-16 11:08:24.6718 UTC hyperparameters_optimizer.cc:582] [49/50] Score: -0.573524 / -0.566897 HParams: fields { name: "min_examples" value { integer: 2 } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 6 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "shrinkage" value { real: 0.05 } } fields { name: "num_candidate_attributes_ratio" value { real: 1 } }
[INFO 23-08-16 11:08:25.0729 UTC gradient_boosted_trees.cc:1542]  num-trees:300 train-loss:0.503526 train-accuracy:0.890420 valid-loss:0.580347 valid-accuracy:0.870297
[INFO 23-08-16 11:08:25.0729 UTC gradient_boosted_trees.cc:247] Truncates the model to 300 tree(s) i.e. 300  iteration(s).
[INFO 23-08-16 11:08:25.0730 UTC gradient_boosted_trees.cc:310] Final model num-trees:300 valid-loss:0.580347 valid-accuracy:0.870297
[INFO 23-08-16 11:08:25.0785 UTC hyperparameters_optimizer.cc:582] [50/50] Score: -0.580347 / -0.566897 HParams: fields { name: "min_examples" value { integer: 10 } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 128 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "shrinkage" value { real: 0.02 } } fields { name: "num_candidate_attributes_ratio" value { real: 1 } }
[INFO 23-08-16 11:08:25.1012 UTC hyperparameters_optimizer.cc:219] Best hyperparameters:
fields {
  name: "min_examples"
  value {
    integer: 2
  }
}
fields {
  name: "categorical_algorithm"
  value {
    categorical: "RANDOM"
  }
}
fields {
  name: "growing_strategy"
  value {
    categorical: "BEST_FIRST_GLOBAL"
  }
}
fields {
  name: "max_num_nodes"
  value {
    integer: 64
  }
}
fields {
  name: "use_hessian_gain"
  value {
    categorical: "true"
  }
}
fields {
  name: "shrinkage"
  value {
    real: 0.1
  }
}
fields {
  name: "num_candidate_attributes_ratio"
  value {
    real: 0.9
  }
}

[INFO 23-08-16 11:08:25.1016 UTC kernel.cc:926] Export model in log directory: /tmpfs/tmp/tmpzdzgno07 with prefix 752bc47fe3694f88
[INFO 23-08-16 11:08:25.1108 UTC kernel.cc:944] Save model in resources
[INFO 23-08-16 11:08:25.1135 UTC abstract_model.cc:849] Model self evaluation:
Task: CLASSIFICATION
Label: __LABEL
Loss (BINOMIAL_LOG_LIKELIHOOD): 0.566897

Accuracy: 0.873395  CI95[W][0 1]
ErrorRate: : 0.126605


Confusion Table:
truth\prediction
   0     1    2
0  0     0    0
1  0  1572   92
2  0   194  401
Total: 2259

One vs other classes:

[INFO 23-08-16 11:08:25.1327 UTC kernel.cc:1243] Loading model from path /tmpfs/tmp/tmpzdzgno07/model/ with prefix 752bc47fe3694f88
[INFO 23-08-16 11:08:25.1690 UTC abstract_model.cc:1311] Engine "GradientBoostedTreesQuickScorerExtended" built
[INFO 23-08-16 11:08:25.1691 UTC kernel.cc:1075] Use fast generic engine
Model trained in 0:00:21.820976
Compiling model...
Model compiled.
CPU times: user 7min 2s, sys: 576 ms, total: 7min 3s
Wall time: 22.4 s
<keras.src.callbacks.History at 0x7f2410cdc040>
# Evaluate the model
tuned_model.compile(["accuracy"])
tuned_test_accuracy = tuned_model.evaluate(test_ds, return_dict=True, verbose=0)["accuracy"]
print(f"Test accuracy with the TF-DF hyper-parameter tuner: {tuned_test_accuracy:.4f}")
Test accuracy with the TF-DF hyper-parameter tuner: 0.8744

The hyper-parameters and objective scores of the trials are available in the model inspector. The score value is always maximized. In this example, the score is the negative log loss on the validation dataset (selected automatically).

# Display the tuning logs.
tuning_logs = tuned_model.make_inspector().tuning_logs()
tuning_logs.head()

The single rows with best=True is the one used in the final model.

# Best hyper-parameters.
tuning_logs[tuning_logs.best].iloc[0]
score                                     -0.566897
evaluation_time                            15.33206
best                                           True
min_examples                                      2
categorical_algorithm                        RANDOM
growing_strategy                  BEST_FIRST_GLOBAL
max_depth                                       NaN
use_hessian_gain                               true
shrinkage                                       0.1
num_candidate_attributes_ratio                  0.9
max_num_nodes                                  64.0
Name: 41, dtype: object

Next, we plot the evaluation of the best score during the tuning.

plt.figure(figsize=(10, 5))
plt.plot(tuning_logs["score"], label="current trial")
plt.plot(tuning_logs["score"].cummax(), label="best trial")
plt.xlabel("Tuning step")
plt.ylabel("Tuning score")
plt.legend()
plt.show()

png

As before, hyper-parameter tuning is enabled by specifying the tuner constructor argument of the model. Set use_predefined_hps=True to automatically configure the search space for the hyper-parameters.

%%time
%set_cell_height 300

# Create a Random Search tuner with 50 trials and automatic hp configuration.
tuner = tfdf.tuner.RandomSearch(num_trials=50, use_predefined_hps=True)

# Define and train the model.
tuned_model = tfdf.keras.GradientBoostedTreesModel(tuner=tuner)
tuned_model.fit(train_ds, verbose=2)
<IPython.core.display.Javascript object>
Warning: The `num_threads` constructor argument is not set and the number of CPU is os.cpu_count()=32 > 32. Setting num_threads to 32. Set num_threads manually to use more than 32 cpus.
WARNING:absl:The `num_threads` constructor argument is not set and the number of CPU is os.cpu_count()=32 > 32. Setting num_threads to 32. Set num_threads manually to use more than 32 cpus.
Use /tmpfs/tmp/tmpfhjg70bi as temporary training directory
Reading training dataset...
Training tensor examples:
Features: {'age': <tf.Tensor 'data:0' shape=(None,) dtype=int64>, 'workclass': <tf.Tensor 'data_1:0' shape=(None,) dtype=string>, 'fnlwgt': <tf.Tensor 'data_2:0' shape=(None,) dtype=int64>, 'education': <tf.Tensor 'data_3:0' shape=(None,) dtype=string>, 'education_num': <tf.Tensor 'data_4:0' shape=(None,) dtype=int64>, 'marital_status': <tf.Tensor 'data_5:0' shape=(None,) dtype=string>, 'occupation': <tf.Tensor 'data_6:0' shape=(None,) dtype=string>, 'relationship': <tf.Tensor 'data_7:0' shape=(None,) dtype=string>, 'race': <tf.Tensor 'data_8:0' shape=(None,) dtype=string>, 'sex': <tf.Tensor 'data_9:0' shape=(None,) dtype=string>, 'capital_gain': <tf.Tensor 'data_10:0' shape=(None,) dtype=int64>, 'capital_loss': <tf.Tensor 'data_11:0' shape=(None,) dtype=int64>, 'hours_per_week': <tf.Tensor 'data_12:0' shape=(None,) dtype=int64>, 'native_country': <tf.Tensor 'data_13:0' shape=(None,) dtype=string>}
Label: Tensor("data_14:0", shape=(None,), dtype=int64)
Weights: None
Normalized tensor features:
 {'age': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast:0' shape=(None,) dtype=float32>), 'workclass': SemanticTensor(semantic=<Semantic.CATEGORICAL: 2>, tensor=<tf.Tensor 'data_1:0' shape=(None,) dtype=string>), 'fnlwgt': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_1:0' shape=(None,) dtype=float32>), 'education': SemanticTensor(semantic=<Semantic.CATEGORICAL: 2>, tensor=<tf.Tensor 'data_3:0' shape=(None,) dtype=string>), 'education_num': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_2:0' shape=(None,) dtype=float32>), 'marital_status': SemanticTensor(semantic=<Semantic.CATEGORICAL: 2>, tensor=<tf.Tensor 'data_5:0' shape=(None,) dtype=string>), 'occupation': SemanticTensor(semantic=<Semantic.CATEGORICAL: 2>, tensor=<tf.Tensor 'data_6:0' shape=(None,) dtype=string>), 'relationship': SemanticTensor(semantic=<Semantic.CATEGORICAL: 2>, tensor=<tf.Tensor 'data_7:0' shape=(None,) dtype=string>), 'race': SemanticTensor(semantic=<Semantic.CATEGORICAL: 2>, tensor=<tf.Tensor 'data_8:0' shape=(None,) dtype=string>), 'sex': SemanticTensor(semantic=<Semantic.CATEGORICAL: 2>, tensor=<tf.Tensor 'data_9:0' shape=(None,) dtype=string>), 'capital_gain': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_3:0' shape=(None,) dtype=float32>), 'capital_loss': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_4:0' shape=(None,) dtype=float32>), 'hours_per_week': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_5:0' shape=(None,) dtype=float32>), 'native_country': SemanticTensor(semantic=<Semantic.CATEGORICAL: 2>, tensor=<tf.Tensor 'data_13:0' shape=(None,) dtype=string>)}
[WARNING 23-08-16 11:08:25.8988 UTC gradient_boosted_trees.cc:1818] "goss_alpha" set but "sampling_method" not equal to "GOSS".
[WARNING 23-08-16 11:08:25.8988 UTC gradient_boosted_trees.cc:1829] "goss_beta" set but "sampling_method" not equal to "GOSS".
[WARNING 23-08-16 11:08:25.8988 UTC gradient_boosted_trees.cc:1843] "selective_gradient_boosting_ratio" set but "sampling_method" not equal to "SELGB".
Training dataset read in 0:00:00.378785. Found 22792 examples.
Training model...
[INFO 23-08-16 11:08:26.2894 UTC kernel.cc:773] Start Yggdrasil model training
[INFO 23-08-16 11:08:26.2895 UTC kernel.cc:774] Collect training examples
[INFO 23-08-16 11:08:26.2895 UTC kernel.cc:787] Dataspec guide:
column_guides {
  column_name_pattern: "^__LABEL$"
  type: CATEGORICAL
  categorial {
    min_vocab_frequency: 0
    max_vocab_count: -1
  }
}
default_column_guide {
  categorial {
    max_vocab_count: 2000
  }
  discretized_numerical {
    maximum_num_bins: 255
  }
}
ignore_columns_without_guides: false
detect_numerical_as_discretized_numerical: false

[INFO 23-08-16 11:08:26.2896 UTC kernel.cc:393] Number of batches: 23
[INFO 23-08-16 11:08:26.2896 UTC kernel.cc:394] Number of examples: 22792
[INFO 23-08-16 11:08:26.2970 UTC data_spec_inference.cc:305] 1 item(s) have been pruned (i.e. they are considered out of dictionary) for the column native_country (40 item(s) left) because min_value_count=5 and max_number_of_unique_values=2000
[INFO 23-08-16 11:08:26.2970 UTC data_spec_inference.cc:305] 1 item(s) have been pruned (i.e. they are considered out of dictionary) for the column occupation (13 item(s) left) because min_value_count=5 and max_number_of_unique_values=2000
[INFO 23-08-16 11:08:26.2971 UTC data_spec_inference.cc:305] 1 item(s) have been pruned (i.e. they are considered out of dictionary) for the column workclass (7 item(s) left) because min_value_count=5 and max_number_of_unique_values=2000
[INFO 23-08-16 11:08:26.3035 UTC kernel.cc:794] Training dataset:
Number of records: 22792
Number of columns: 15

Number of columns by type:
    CATEGORICAL: 9 (60%)
    NUMERICAL: 6 (40%)

Columns:

CATEGORICAL: 9 (60%)
    0: "__LABEL" CATEGORICAL integerized vocab-size:3 no-ood-item
    4: "education" CATEGORICAL has-dict vocab-size:17 zero-ood-items most-frequent:"HS-grad" 7340 (32.2043%)
    8: "marital_status" CATEGORICAL has-dict vocab-size:8 zero-ood-items most-frequent:"Married-civ-spouse" 10431 (45.7661%)
    9: "native_country" CATEGORICAL num-nas:407 (1.78571%) has-dict vocab-size:41 num-oods:1 (0.00446728%) most-frequent:"United-States" 20436 (91.2933%)
    10: "occupation" CATEGORICAL num-nas:1260 (5.52826%) has-dict vocab-size:14 num-oods:1 (0.00464425%) most-frequent:"Prof-specialty" 2870 (13.329%)
    11: "race" CATEGORICAL has-dict vocab-size:6 zero-ood-items most-frequent:"White" 19467 (85.4115%)
    12: "relationship" CATEGORICAL has-dict vocab-size:7 zero-ood-items most-frequent:"Husband" 9191 (40.3256%)
    13: "sex" CATEGORICAL has-dict vocab-size:3 zero-ood-items most-frequent:"Male" 15165 (66.5365%)
    14: "workclass" CATEGORICAL num-nas:1257 (5.51509%) has-dict vocab-size:8 num-oods:1 (0.0046436%) most-frequent:"Private" 15879 (73.7358%)

NUMERICAL: 6 (40%)
    1: "age" NUMERICAL mean:38.6153 min:17 max:90 sd:13.661
    2: "capital_gain" NUMERICAL mean:1081.9 min:0 max:99999 sd:7509.48
    3: "capital_loss" NUMERICAL mean:87.2806 min:0 max:4356 sd:403.01
    5: "education_num" NUMERICAL mean:10.0927 min:1 max:16 sd:2.56427
    6: "fnlwgt" NUMERICAL mean:189879 min:12285 max:1.4847e+06 sd:106423
    7: "hours_per_week" NUMERICAL mean:40.3955 min:1 max:99 sd:12.249

Terminology:
    nas: Number of non-available (i.e. missing) values.
    ood: Out of dictionary.
    manually-defined: Attribute which type is manually defined by the user i.e. the type was not automatically inferred.
    tokenized: The attribute value is obtained through tokenization.
    has-dict: The attribute is attached to a string dictionary e.g. a categorical attribute stored as a string.
    vocab-size: Number of unique values.

[INFO 23-08-16 11:08:26.3035 UTC kernel.cc:810] Configure learner
[WARNING 23-08-16 11:08:26.3038 UTC gradient_boosted_trees.cc:1818] "goss_alpha" set but "sampling_method" not equal to "GOSS".
[WARNING 23-08-16 11:08:26.3038 UTC gradient_boosted_trees.cc:1829] "goss_beta" set but "sampling_method" not equal to "GOSS".
[WARNING 23-08-16 11:08:26.3038 UTC gradient_boosted_trees.cc:1843] "selective_gradient_boosting_ratio" set but "sampling_method" not equal to "SELGB".
[INFO 23-08-16 11:08:26.3039 UTC kernel.cc:824] Training config:
learner: "HYPERPARAMETER_OPTIMIZER"
features: "^age$"
features: "^capital_gain$"
features: "^capital_loss$"
features: "^education$"
features: "^education_num$"
features: "^fnlwgt$"
features: "^hours_per_week$"
features: "^marital_status$"
features: "^native_country$"
features: "^occupation$"
features: "^race$"
features: "^relationship$"
features: "^sex$"
features: "^workclass$"
label: "^__LABEL$"
task: CLASSIFICATION
metadata {
  framework: "TF Keras"
}
[yggdrasil_decision_forests.model.hyperparameters_optimizer_v2.proto.hyperparameters_optimizer_config] {
  base_learner {
    learner: "GRADIENT_BOOSTED_TREES"
    features: "^age$"
    features: "^capital_gain$"
    features: "^capital_loss$"
    features: "^education$"
    features: "^education_num$"
    features: "^fnlwgt$"
    features: "^hours_per_week$"
    features: "^marital_status$"
    features: "^native_country$"
    features: "^occupation$"
    features: "^race$"
    features: "^relationship$"
    features: "^sex$"
    features: "^workclass$"
    label: "^__LABEL$"
    task: CLASSIFICATION
    random_seed: 123456
    pure_serving_model: false
    [yggdrasil_decision_forests.model.gradient_boosted_trees.proto.gradient_boosted_trees_config] {
      num_trees: 300
      decision_tree {
        max_depth: 6
        min_examples: 5
        in_split_min_examples_check: true
        keep_non_leaf_label_distribution: true
        num_candidate_attributes: -1
        missing_value_policy: GLOBAL_IMPUTATION
        allow_na_conditions: false
        categorical_set_greedy_forward {
          sampling: 0.1
          max_num_items: -1
          min_item_frequency: 1
        }
        growing_strategy_local {
        }
        categorical {
          cart {
          }
        }
        axis_aligned_split {
        }
        internal {
          sorting_strategy: PRESORTED
        }
        uplift {
          min_examples_in_treatment: 5
          split_score: KULLBACK_LEIBLER
        }
      }
      shrinkage: 0.1
      loss: DEFAULT
      validation_set_ratio: 0.1
      validation_interval_in_trees: 1
      early_stopping: VALIDATION_LOSS_INCREASE
      early_stopping_num_trees_look_ahead: 30
      l2_regularization: 0
      lambda_loss: 1
      mart {
      }
      adapt_subsample_for_maximum_training_duration: false
      l1_regularization: 0
      use_hessian_gain: false
      l2_regularization_categorical: 1
      stochastic_gradient_boosting {
        ratio: 1
      }
      apply_link_function: true
      compute_permutation_variable_importance: false
      binary_focal_loss_options {
        misprediction_exponent: 2
        positive_sample_coefficient: 0.5
      }
      early_stopping_initial_iteration: 10
    }
  }
  optimizer {
    optimizer_key: "RANDOM"
    [yggdrasil_decision_forests.model.hyperparameters_optimizer_v2.proto.random] {
      num_trials: 50
    }
  }
  base_learner_deployment {
    num_threads: 1
  }
  predefined_search_space {
  }
}

[INFO 23-08-16 11:08:26.3040 UTC kernel.cc:827] Deployment config:
cache_path: "/tmpfs/tmp/tmpfhjg70bi/working_cache"
num_threads: 32
try_resume_training: true

[INFO 23-08-16 11:08:26.3042 UTC kernel.cc:889] Train model
[INFO 23-08-16 11:08:26.3045 UTC hyperparameters_optimizer.cc:209] Hyperparameter search space:
fields {
  name: "split_axis"
  discrete_candidates {
    possible_values {
      categorical: "AXIS_ALIGNED"
    }
    possible_values {
      categorical: "SPARSE_OBLIQUE"
    }
  }
  children {
    name: "sparse_oblique_projection_density_factor"
    discrete_candidates {
      possible_values {
        real: 1
      }
      possible_values {
        real: 2
      }
      possible_values {
        real: 3
      }
      possible_values {
        real: 4
      }
      possible_values {
        real: 5
      }
    }
    parent_discrete_values {
      possible_values {
        categorical: "SPARSE_OBLIQUE"
      }
    }
  }
  children {
    name: "sparse_oblique_normalization"
    discrete_candidates {
      possible_values {
        categorical: "NONE"
      }
      possible_values {
        categorical: "STANDARD_DEVIATION"
      }
      possible_values {
        categorical: "MIN_MAX"
      }
    }
    parent_discrete_values {
      possible_values {
        categorical: "SPARSE_OBLIQUE"
      }
    }
  }
  children {
    name: "sparse_oblique_weights"
    discrete_candidates {
      possible_values {
        categorical: "BINARY"
      }
      possible_values {
        categorical: "CONTINUOUS"
      }
    }
    parent_discrete_values {
      possible_values {
        categorical: "SPARSE_OBLIQUE"
      }
    }
  }
}
fields {
  name: "categorical_algorithm"
  discrete_candidates {
    possible_values {
      categorical: "CART"
    }
    possible_values {
      categorical: "RANDOM"
    }
  }
}
fields {
  name: "growing_strategy"
  discrete_candidates {
    possible_values {
      categorical: "LOCAL"
    }
    possible_values {
      categorical: "BEST_FIRST_GLOBAL"
    }
  }
  children {
    name: "max_num_nodes"
    discrete_candidates {
      possible_values {
        integer: 16
      }
      possible_values {
        integer: 32
      }
      possible_values {
        integer: 64
      }
      possible_values {
        integer: 128
      }
      possible_values {
        integer: 256
      }
      possible_values {
        integer: 512
      }
    }
    parent_discrete_values {
      possible_values {
        categorical: "BEST_FIRST_GLOBAL"
      }
    }
  }
  children {
    name: "max_depth"
    discrete_candidates {
      possible_values {
        integer: 3
      }
      possible_values {
        integer: 4
      }
      possible_values {
        integer: 6
      }
      possible_values {
        integer: 8
      }
    }
    parent_discrete_values {
      possible_values {
        categorical: "LOCAL"
      }
    }
  }
}
fields {
  name: "sampling_method"
  discrete_candidates {
    possible_values {
      categorical: "RANDOM"
    }
  }
  children {
    name: "subsample"
    discrete_candidates {
      possible_values {
        real: 0.6
      }
      possible_values {
        real: 0.8
      }
      possible_values {
        real: 0.9
      }
      possible_values {
        real: 1
      }
    }
    parent_discrete_values {
      possible_values {
        categorical: "RANDOM"
      }
    }
  }
}
fields {
  name: "shrinkage"
  discrete_candidates {
    possible_values {
      real: 0.02
    }
    possible_values {
      real: 0.05
    }
    possible_values {
      real: 0.1
    }
  }
}
fields {
  name: "min_examples"
  discrete_candidates {
    possible_values {
      integer: 5
    }
    possible_values {
      integer: 7
    }
    possible_values {
      integer: 10
    }
    possible_values {
      integer: 20
    }
  }
}
fields {
  name: "use_hessian_gain"
  discrete_candidates {
    possible_values {
      categorical: "true"
    }
    possible_values {
      categorical: "false"
    }
  }
}
fields {
  name: "num_candidate_attributes_ratio"
  discrete_candidates {
    possible_values {
      real: 0.2
    }
    possible_values {
      real: 0.5
    }
    possible_values {
      real: 0.9
    }
    possible_values {
      real: 1
    }
  }
}

[INFO 23-08-16 11:08:26.3046 UTC hyperparameters_optimizer.cc:500] Start local tuner with 32 thread(s)
[INFO[INFO 23-08-16 11:08:26.3062 UTC  23-08-16 11:08:26.3062 UTC gradient_boosted_trees.cc:459] Default loss set to BINOMIAL_LOG_LIKELIHOOD
gradient_boosted_trees.cc:[INFO 23-08-16 11:08:26.3062 UTC gradient_boosted_trees.cc459] Default loss set to BINOMIAL_LOG_LIKELIHOOD
:1085] Training gradient boosted tree on 22792 example(s) and 14 feature(s).[INFO[INFO 23-08-16 11:08:26.3063 UTC gradient_boosted_trees.cc:459] Default loss set to BINOMIAL_LOG_LIKELIHOOD
[INFO 23-08-16 11:08:26.3063 UTC gradient_boosted_trees.cc:1085] 
[INFO 23-08-16 11:08:26.3063 UTC gradient_boosted_trees.cc:459 23-08-16 11:08:26.3063 UTC gradient_boosted_trees.cc:1085] Training gradient boosted tree on 22792 example(s) and 14 feature(s).
Training gradient boosted tree on 22792 example(s) and 14 feature(s).
] Default loss set to BINOMIAL_LOG_LIKELIHOOD
[INFO 23-08-16 11:08:26.3064 UTC gradient_boosted_trees.cc:1085] Training gradient boosted tree on 22792 example(s) and 14 feature(s).
[INFO 23-08-16 11:08:26.3065 UTC gradient_boosted_trees.cc:459] Default loss set to BINOMIAL_LOG_LIKELIHOOD
[INFO 23-08-16 11:08:26.3065 UTC gradient_boosted_trees.cc:1085] Training gradient boosted tree on 22792 example(s) and 14 feature(s).
[INFO 23-08-16 11:08:26.3065 UTC gradient_boosted_trees.cc:459] Default loss set to BINOMIAL_LOG_LIKELIHOOD
[INFO 23-08-16 11:08:26.3066 UTC gradient_boosted_trees.cc:1085] Training gradient boosted tree on 22792 example(s) and 14 feature(s).
[INFO 23-08-16 11:08:26.3066 UTC gradient_boosted_trees.cc:459] Default loss set to BINOMIAL_LOG_LIKELIHOOD
[INFO 23-08-16 11:08:26.3067 UTC gradient_boosted_trees.cc:1085] Training gradient boosted tree on 22792 example(s) and 14 feature(s).
[INFO 23-08-16 11:08:26.3067 UTC gradient_boosted_trees.cc:459] Default loss set to BINOMIAL_LOG_LIKELIHOOD
[INFO 23-08-16 11:08:26.3068 UTC gradient_boosted_trees.cc:1085] Training gradient boosted tree on 22792 example(s) and 14 feature(s).
[INFO[INFO 23-08-16 11:08:26.3070 UTC  23-08-16 11:08:26.3069 UTC gradient_boosted_trees.cc:459gradient_boosted_trees.cc:459] ] Default loss set to BINOMIAL_LOG_LIKELIHOOD
Default loss set to [INFOBINOMIAL_LOG_LIKELIHOOD
[INFO 23-08-16 11:08:26.3070 UTC gradient_boosted_trees.cc:1085] Training gradient boosted tree on 22792 example(s) and  23-08-16 11:08:26.3070 UTC gradient_boosted_trees.cc:108514 feature(s).
] Training gradient boosted tree on 22792 example(s) and 14 feature(s).
[[INFOINFO 23-08-16 11:08:26.3072 UTC gradient_boosted_trees.cc:459] Default loss set to BINOMIAL_LOG_LIKELIHOOD
 23-08-16 11:08:26.3072 UTC gradient_boosted_trees.cc:459] Default loss set to BINOMIAL_LOG_LIKELIHOOD[INFO 23-08-16 11:08:26.3072 UTC gradient_boosted_trees.cc:1085] Training gradient boosted tree on 22792 example(s) and 
14[INFO 23-08-16 11:08:26.3073 UTC gradient_boosted_trees.cc: feature(s).
1085] Training gradient boosted tree on 22792 example(s) and 14 feature(s).
[INFO 23-08-16 11:08:26.3074 UTC gradient_boosted_trees.cc:459] Default loss set to BINOMIAL_LOG_LIKELIHOOD
[INFO 23-08-16 11:08:26.3074 UTC gradient_boosted_trees.cc:1085] Training gradient boosted tree on 22792 example(s) and 14 feature(s).
[INFO 23-08-16 11:08:26.3075 UTC gradient_boosted_trees.cc:459] Default loss set to BINOMIAL_LOG_LIKELIHOOD
[INFO 23-08-16 11:08:26.3076 UTC gradient_boosted_trees.cc:1085] Training gradient boosted tree on 22792 example(s) and 14 feature(s).
[INFO 23-08-16 11:08:26.3077 UTC gradient_boosted_trees.cc:459] Default loss set to BINOMIAL_LOG_LIKELIHOOD
[INFO 23-08-16 11:08:26.3077 UTC gradient_boosted_trees.cc:1085] Training gradient boosted tree on 22792 example(s) and 14 feature(s).
[INFO 23-08-16 11:08:26.3082 UTC gradient_boosted_trees.cc:459] Default loss set to BINOMIAL_LOG_LIKELIHOOD
[INFO 23-08-16 11:08:26.3082 UTC gradient_boosted_trees.cc:1085] Training gradient boosted tree on 22792[INFO example(s) and 14 feature(s).
 23-08-16 11:08:26.3082 UTC gradient_boosted_trees.cc:459] Default loss set to BINOMIAL_LOG_LIKELIHOOD
[INFO 23-08-16 11:08:26.3083 UTC gradient_boosted_trees.cc:1085] Training gradient boosted tree on 22792 example(s) and 14 feature(s).
[INFO 23-08-16 11:08:26.3090 UTC gradient_boosted_trees.cc:459] Default loss set to BINOMIAL_LOG_LIKELIHOOD
[INFO 23-08-16 11:08:26.3090 UTC gradient_boosted_trees.cc:1085] Training gradient boosted tree on 22792 example(s) and 14 feature(s).
[INFO 23-08-16 11:08:26.3091 UTC gradient_boosted_trees.cc:459] Default loss set to BINOMIAL_LOG_LIKELIHOOD
[INFO 23-08-16 11:08:26.3091 UTC gradient_boosted_trees.cc:1085] Training gradient boosted tree on 22792 example(s) and 14 feature(s).
[INFO[ 23-08-16 11:08:26.3099 UTC gradient_boosted_trees.cc:459INFO 23-08-16 11:08:26.3099 UTC gradient_boosted_trees.cc:] Default loss set to BINOMIAL_LOG_LIKELIHOOD
459] Default loss set to [INFO 23-08-16 11:08:26.3099 UTC gradient_boosted_trees.cc:1085BINOMIAL_LOG_LIKELIHOOD
] [Training gradient boosted tree on 22792 example(s) and 14 feature(s).
INFO 23-08-16 11:08:26.3100 UTC gradient_boosted_trees.cc:1085] Training gradient boosted tree on 22792 example(s) and 14 feature(s).
[INFO[INFO 23-08-16 11:08:26.3102 UTC gradient_boosted_trees.cc 23-08-16 11:08:26.3102 UTC gradient_boosted_trees.cc:459] Default loss set to BINOMIAL_LOG_LIKELIHOOD
[INFO 23-08-16 11:08:26.3102 UTC gradient_boosted_trees.cc:1085:459] Default loss set to BINOMIAL_LOG_LIKELIHOOD
] Training gradient boosted tree on 22792 example(s) and 14 feature(s).[INFO[ 23-08-16 11:08:26.3103 UTC INFOgradient_boosted_trees.cc:459]  23-08-16 11:08:26.3103 UTC [INFOgradient_boosted_trees.ccDefault loss set to BINOMIAL_LOG_LIKELIHOOD
:[INFO459 23-08-16 11:08:26.3104 UTC gradient_boosted_trees.cc:1085] Training gradient boosted tree on 22792 example(s) and 14 feature(s).

] Default loss set to BINOMIAL_LOG_LIKELIHOOD
[INFO 23-08-16 11:08:26.3105 UTC gradient_boosted_trees.cc:1085] Training gradient boosted tree on 22792 example(s) and 14 23-08-16 11:08:26.3104 UTC gradient_boosted_trees.cc:1085] Training gradient boosted tree on 22792 example(s) and 14 feature(s).
 feature(s).
[INFO 23-08-16 11:08:26.3107 UTC gradient_boosted_trees.cc:459] Default loss set to BINOMIAL_LOG_LIKELIHOOD
[INFO 23-08-16 11:08:26.3107 UTC gradient_boosted_trees.cc:1085] Training gradient boosted tree on 22792 example(s) and 14 feature(s).
[INFO 23-08-16 11:08:26.3109 UTC gradient_boosted_trees.cc:459] Default loss set to BINOMIAL_LOG_LIKELIHOOD
[INFO 23-08-16 11:08:26.3109 UTC gradient_boosted_trees.cc[INFO:459] Default loss set to BINOMIAL_LOG_LIKELIHOOD
[ 23-08-16 11:08:26.3110 UTC gradient_boosted_trees.cc:1085] Training gradient boosted tree on 22792 example(s) and 14 feature(s).
INFO 23-08-16 11:08:26.3110 UTC gradient_boosted_trees.cc:1085] Training gradient boosted tree on 22792 example(s) and 14 feature(s).
[INFO 23-08-16 11:08:26.3116 UTC gradient_boosted_trees.cc:459] Default loss set to BINOMIAL_LOG_LIKELIHOOD
[INFO 23-08-16 11:08:26.3116 UTC gradient_boosted_trees.cc:1085] Training gradient boosted tree on 22792 example(s) and 14 feature(s).
[INFO 23-08-16 11:08:26.3117 UTC gradient_boosted_trees.cc:459] Default loss set to BINOMIAL_LOG_LIKELIHOOD
[INFO 23-08-16 11:08:26.3118 UTC gradient_boosted_trees.cc:1085] Training gradient boosted tree on 22792 example(s) and 14 feature(s).
[INFO 23-08-16 11:08:26.3123 UTC gradient_boosted_trees.cc:459] Default loss set to BINOMIAL_LOG_LIKELIHOOD
[INFO 23-08-16 11:08:26.3124 UTC gradient_boosted_trees.cc:1085] Training gradient boosted tree on 22792 example(s) and 14 feature(s).
[INFO 23-08-16 11:08:26.3125 UTC gradient_boosted_trees.cc:459] Default loss set to BINOMIAL_LOG_LIKELIHOOD
[INFO 23-08-16 11:08:26.3125 UTC gradient_boosted_trees.cc:1085] Training gradient boosted tree on 22792 example(s) and 14 feature(s).
[INFO 23-08-16 11:08:26.3150 UTC gradient_boosted_trees.cc:1128] 20533 examples used for training and 2259 examples used for validation
[INFO 23-08-16 11:08:26.3160 UTC gradient_boosted_trees.cc:1128] 20533 examples used for training and 2259 examples used for validation
[INFO[INFO 23-08-16 11:08:26.3404 UTC gradient_boosted_trees.cc: 23-08-16 11:08:26.3404 UTC 1128] 20533 examples used for training and 2259 examples used for validation
gradient_boosted_trees.cc:1128] 20533 examples used for training and 2259 examples used for validation
[INFO 23-08-16 11:08:26.3589 UTC gradient_boosted_trees.cc:1128] 20533 examples used for training and 2259 examples used for validation
[INFO 23-08-16 11:08:26.3593 UTC gradient_boosted_trees.cc:1128] 20533 examples used for training and 2259 examples used for validation
[INFO 23-08-16 11:08:26.3597 UTC gradient_boosted_trees.cc:[INFO1128] 20533 examples used for training and 2259 examples used for validation
 23-08-16 11:08:26.3597 UTC gradient_boosted_trees.cc:1128] 20533 examples used for training and 2259 examples used for validation
[INFO 23-08-16 11:08:26.3607 UTC gradient_boosted_trees.cc:1128] 20533 examples used for training and 2259 examples used for validation
[INFO 23-08-16 11:08:26.3614 UTC gradient_boosted_trees.cc:1128] 20533 examples used for training and 2259 examples used for validation
[INFO 23-08-16 11:08:26.3617 UTC gradient_boosted_trees.cc:1128] 20533 examples used for training and 2259 examples used for validation
[INFO 23-08-16 11:08:26.3618 UTC gradient_boosted_trees.cc:1128] 20533 examples used for training and 2259 examples used for validation
[INFO 23-08-16 11:08:26.3628 UTC gradient_boosted_trees.cc:1128] 20533 examples used for training and 2259 examples used for validation
[INFO 23-08-16 11:08:26.3629 UTC gradient_boosted_trees.cc:1128] 20533 examples used for training and 2259 examples used for validation[INFO
 23-08-16 11:08:26.3629 UTC gradient_boosted_trees.cc[[INFOINFO:1128]  23-08-16 11:08:26.3630 UTC gradient_boosted_trees.cc20533 examples used for training and 2259 examples used for validation
:1128] 20533 examples used for training and 2259 examples used for validation 23-08-16 11:08:26.3630 UTC gradient_boosted_trees.cc:1128] 20533 examples used for training and 2259 examples used for validation

[INFO[ 23-08-16 11:08:26.3631 UTC gradient_boosted_trees.ccINFO[INFO: 23-08-16 11:08:26.3632 UTC gradient_boosted_trees.cc:11281128] 20533 examples used for training and 2259 examples used for validation
 23-08-16 11:08:26.3632 UTC gradient_boosted_trees.cc:] 20533 examples used for training and 2259 examples used for validation
1128] 20533 examples used for training and 2259 examples used for validation
[INFO 23-08-16 11:08:26.3633 UTC gradient_boosted_trees.cc:1128] 20533 examples used for training and 2259 examples used for validation
[INFO 23-08-16 11:08:26.3640 UTC gradient_boosted_trees.cc:1128] 20533 examples used for training and 2259 examples used for validation
[INFO 23-08-16 11:08:26.3644 UTC gradient_boosted_trees.cc:1128] 20533 examples used for training and 2259 examples used for validation
[INFO 23-08-16 11:08:26.3653 UTC gradient_boosted_trees.cc:1128] 20533 examples used for training and 2259 examples used for validation
[INFO 23-08-16 11:08:26.3659 UTC gradient_boosted_trees.cc:1128] 20533 examples used for training and 2259 examples used for validation[INFO 23-08-16 11:08:26.3660 UTC gradient_boosted_trees.cc:1128] 
20533 examples used for training and 2259 examples used for validation
[INFO 23-08-16 11:08:26.3685 UTC gradient_boosted_trees.cc:1128] 20533[INFO examples used for training and 2259 examples used for validation 23-08-16 11:08:26.3685 UTC gradient_boosted_trees.cc:1128
] 20533 examples used for training and 2259 examples used for validation
[INFO 23-08-16 11:08:26.3805 UTC gradient_boosted_trees.cc:1128] 20533 examples used for training and 2259[[INFO examples used for validationINFO 23-08-16 11:08:26.3805 UTC gradient_boosted_trees.cc
 23-08-16 11:08:26.3805 UTC gradient_boosted_trees.cc::1128] 205331128] 20533 examples used for training and 2259 examples used for validation
 examples used for training and 2259 examples used for validation
[INFO 23-08-16 11:08:26.3807 UTC gradient_boosted_trees.cc:1128] 20533 examples used for training and 2259 examples used for validation
[INFO 23-08-16 11:08:26.4823 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.060520 train-accuracy:0.761895 valid-loss:1.117708 valid-accuracy:0.736609
[INFO 23-08-16 11:08:26.5109 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.034401 train-accuracy:0.761895 valid-loss:1.090277 valid-accuracy:0.736609
[INFO 23-08-16 11:08:26.5224 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.082692 train-accuracy:0.761895 valid-loss:1.140741 valid-accuracy:0.736609
[INFO 23-08-16 11:08:26.5375 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.024398 train-accuracy:0.761895 valid-loss:1.080875 valid-accuracy:0.736609
[INFO 23-08-16 11:08:26.5471 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.080233 train-accuracy:0.761895 valid-loss:1.138164 valid-accuracy:0.736609
[INFO 23-08-16 11:08:26.5489 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.060070 train-accuracy:0.761895 valid-loss:1.117365 valid-accuracy:0.736609
[INFO 23-08-16 11:08:26.5523 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.016328 train-accuracy:0.761895 valid-loss:1.070658 valid-accuracy:0.736609
[INFO 23-08-16 11:08:26.5628 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.085297 train-accuracy:0.761895 valid-loss:1.143266 valid-accuracy:0.736609
[INFO 23-08-16 11:08:26.6032 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.053581 train-accuracy:0.761895 valid-loss:1.110675 valid-accuracy:0.736609
[INFO 23-08-16 11:08:26.6160 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.009800 train-accuracy:0.761895 valid-loss:1.063156 valid-accuracy:0.736609
[INFO 23-08-16 11:08:26.6367 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.053989 train-accuracy:0.761895 valid-loss:1.111597 valid-accuracy:0.736609
[INFO 23-08-16 11:08:26.6621 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.079152 train-accuracy:0.761895 valid-loss:1.137115 valid-accuracy:0.736609
[INFO 23-08-16 11:08:26.6675 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.080746 train-accuracy:0.761895 valid-loss:1.138830 valid-accuracy:0.736609
[INFO 23-08-16 11:08:26.6724 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.016576 train-accuracy:0.761895 valid-loss:1.072904 valid-accuracy:0.736609
[INFO 23-08-16 11:08:26.6797 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.017781 train-accuracy:0.761895 valid-loss:1.072645 valid-accuracy:0.736609
[INFO 23-08-16 11:08:26.6858 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.079625 train-accuracy:0.761895 valid-loss:1.137371 valid-accuracy:0.736609
[INFO 23-08-16 11:08:26.7092 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.014103 train-accuracy:0.761895 valid-loss:1.069569 valid-accuracy:0.736609
[INFO 23-08-16 11:08:26.7182 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.019006 train-accuracy:0.761895 valid-loss:1.073190 valid-accuracy:0.736609
[INFO 23-08-16 11:08:26.7315 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.079718 train-accuracy:0.761895 valid-loss:1.137516 valid-accuracy:0.736609
[INFO 23-08-16 11:08:26.7553 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.052418 train-accuracy:0.761895 valid-loss:1.109157 valid-accuracy:0.736609
[INFO 23-08-16 11:08:26.7691 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.050215 train-accuracy:0.761895 valid-loss:1.106337 valid-accuracy:0.736609
[INFO 23-08-16 11:08:26.7820 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.052939 train-accuracy:0.761895 valid-loss:1.109668 valid-accuracy:0.736609
[INFO 23-08-16 11:08:26.7999 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.080744 train-accuracy:0.761895 valid-loss:1.138851 valid-accuracy:0.736609
[INFO 23-08-16 11:08:26.8126 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.017080 train-accuracy:0.761895 valid-loss:1.072045 valid-accuracy:0.736609
[INFO 23-08-16 11:08:26.8227 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.080979 train-accuracy:0.761895 valid-loss:1.138389 valid-accuracy:0.736609
[INFO 23-08-16 11:08:26.8270 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.078470 train-accuracy:0.761895 valid-loss:1.135989 valid-accuracy:0.736609
[INFO 23-08-16 11:08:26.8476 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.079121 train-accuracy:0.761895 valid-loss:1.136935 valid-accuracy:0.736609
[INFO 23-08-16 11:08:26.8502 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.011469 train-accuracy:0.761895 valid-loss:1.065462 valid-accuracy:0.736609
[INFO 23-08-16 11:08:26.8557 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.054467 train-accuracy:0.761895 valid-loss:1.111421 valid-accuracy:0.736609
[INFO 23-08-16 11:08:26.8624 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.007886 train-accuracy:0.761895 valid-loss:1.061560 valid-accuracy:0.736609
[INFO 23-08-16 11:08:26.8796 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.079504 train-accuracy:0.761895 valid-loss:1.137702 valid-accuracy:0.736609
[INFO 23-08-16 11:08:26.9287 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.052843 train-accuracy:0.761895 valid-loss:1.111456 valid-accuracy:0.736609
[INFO 23-08-16 11:08:27.5955 UTC gradient_boosted_trees.cc:1544]  num-trees:3 train-loss:0.978275 train-accuracy:0.761895 valid-loss:1.030907 valid-accuracy:0.736609
[INFO 23-08-16 11:08:57.6001 UTC gradient_boosted_trees.cc:1544]  num-trees:71 train-loss:0.645856 train-accuracy:0.864511 valid-loss:0.712521 valid-accuracy:0.838424
[INFO 23-08-16 11:08:59.8972 UTC early_stopping.cc:53] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.621505
[INFO 23-08-16 11:08:59.8972 UTC gradient_boosted_trees.cc:247] Truncates the model to 72 tree(s) i.e. 72  iteration(s).
[INFO 23-08-16 11:08:59.8981 UTC gradient_boosted_trees.cc:310] Final model num-trees:72 valid-loss:0.621505 valid-accuracy:0.858344
[INFO 23-08-16 11:08:59.9002 UTC hyperparameters_optimizer.cc:582] [1/50] Score: -0.621505 / -0.621505 HParams: fields { name: "split_axis" value { categorical: "SPARSE_OBLIQUE" } } fields { name: "sparse_oblique_projection_density_factor" value { real: 5 } } fields { name: "sparse_oblique_normalization" value { categorical: "STANDARD_DEVIATION" } } fields { name: "sparse_oblique_weights" value { categorical: "CONTINUOUS" } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 32 } } fields { name: "sampling_method" value { categorical: "RANDOM" } } fields { name: "subsample" value { real: 0.6 } } fields { name: "shrinkage" value { real: 0.1 } } fields { name: "min_examples" value { integer: 5 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "num_candidate_attributes_ratio" value { real: 1 } }
[INFO 23-08-16 11:08:59.9008 UTC gradient_boosted_trees.cc:459] Default loss set to BINOMIAL_LOG_LIKELIHOOD
[INFO 23-08-16 11:08:59.9009 UTC gradient_boosted_trees.cc:1085] Training gradient boosted tree on 22792 example(s) and 14 feature(s).
[INFO 23-08-16 11:08:59.9064 UTC gradient_boosted_trees.cc:1128] 20533 examples used for training and 2259 examples used for validation
[INFO 23-08-16 11:09:00.2983 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.082344 train-accuracy:0.761895 valid-loss:1.140383 valid-accuracy:0.736609
[INFO 23-08-16 11:09:00.5775 UTC early_stopping.cc:53] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.587795
[INFO 23-08-16 11:09:00.5775 UTC gradient_boosted_trees.cc:247] Truncates the model to 150 tree(s) i.e. 150  iteration(s).
[INFO 23-08-16 11:09:00.5779 UTC gradient_boosted_trees.cc:310] Final model num-trees:150 valid-loss:0.587795 valid-accuracy:0.864099
[INFO 23-08-16 11:09:00.5802 UTC gradient_boosted_trees.cc:459] Default loss set to BINOMIAL_LOG_LIKELIHOOD
[INFO 23-08-16 11:09:00.5802 UTC gradient_boosted_trees.cc:1085] Training gradient boosted tree on 22792 example(s) and 14 feature(s).
[INFO 23-08-16 11:09:00.5813 UTC hyperparameters_optimizer.cc:582] [2/50] Score: -0.587795 / -0.587795 HParams: fields { name: "split_axis" value { categorical: "SPARSE_OBLIQUE" } } fields { name: "sparse_oblique_projection_density_factor" value { real: 1 } } fields { name: "sparse_oblique_normalization" value { categorical: "MIN_MAX" } } fields { name: "sparse_oblique_weights" value { categorical: "BINARY" } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 16 } } fields { name: "sampling_method" value { categorical: "RANDOM" } } fields { name: "subsample" value { real: 0.6 } } fields { name: "shrinkage" value { real: 0.1 } } fields { name: "min_examples" value { integer: 10 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "num_candidate_attributes_ratio" value { real: 0.2 } }
[INFO 23-08-16 11:09:00.5852 UTC gradient_boosted_trees.cc:1128] 20533 examples used for training and 2259 examples used for validation
[INFO 23-08-16 11:09:01.0792 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.056010 train-accuracy:0.761895 valid-loss:1.115039 valid-accuracy:0.736609
[INFO 23-08-16 11:09:03.1408 UTC gradient_boosted_trees.cc:1542]  num-trees:300 train-loss:0.540863 train-accuracy:0.879268 valid-loss:0.593033 valid-accuracy:0.868969
[INFO 23-08-16 11:09:03.1408 UTC gradient_boosted_trees.cc:247] Truncates the model to 294 tree(s) i.e. 294  iteration(s).
[INFO 23-08-16 11:09:03.1409 UTC gradient_boosted_trees.cc:310] Final model num-trees:294 valid-loss:0.592873 valid-accuracy:0.868526
[INFO 23-08-16 11:09:03.1425 UTC hyperparameters_optimizer.cc:582] [3/50] Score: -0.592873 / -0.587795 HParams: fields { name: "split_axis" value { categorical: "SPARSE_OBLIQUE" } } fields { name: "sparse_oblique_projection_density_factor" value { real: 2 } } fields { name: "sparse_oblique_normalization" value { categorical: "NONE" } } fields { name: "sparse_oblique_weights" value { categorical: "BINARY" } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 4 } } fields { name: "sampling_method" value { categorical: "RANDOM" } } fields { name: "subsample" value { real: 0.6 } } fields { name: "shrinkage" value { real: 0.05 } } fields { name: "min_examples" value { integer: 20 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "num_candidate_attributes_ratio" value { real: 1 } }
[INFO 23-08-16 11:09:03.1457 UTC gradient_boosted_trees.cc:459] Default loss set to BINOMIAL_LOG_LIKELIHOOD
[INFO 23-08-16 11:09:03.1457 UTC gradient_boosted_trees.cc:1085] Training gradient boosted tree on 22792 example(s) and 14 feature(s).
[INFO 23-08-16 11:09:03.1505 UTC gradient_boosted_trees.cc:1128] 20533 examples used for training and 2259 examples used for validation
[INFO 23-08-16 11:09:03.5161 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.080643 train-accuracy:0.761895 valid-loss:1.138458 valid-accuracy:0.736609
[INFO 23-08-16 11:09:04.0248 UTC early_stopping.cc:53] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.579144
[INFO 23-08-16 11:09:04.0248 UTC gradient_boosted_trees.cc:247] Truncates the model to 87 tree(s) i.e. 87  iteration(s).
[INFO 23-08-16 11:09:04.0259 UTC gradient_boosted_trees.cc:310] Final model num-trees:87 valid-loss:0.579144 valid-accuracy:0.868969
[INFO 23-08-16 11:09:04.0310 UTC hyperparameters_optimizer.cc:582] [4/50] Score: -0.579144 / -0.579144 HParams: fields { name: "split_axis" value { categorical: "SPARSE_OBLIQUE" } } fields { name: "sparse_oblique_projection_density_factor" value { real: 1 } } fields { name: "sparse_oblique_normalization" value { categorical: "MIN_MAX" } } fields { name: "sparse_oblique_weights" value { categorical: "CONTINUOUS" } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 512 } } fields { name: "sampling_method" value { categorical: "RANDOM" } } fields { name: "subsample" value { real: 1 } } fields { name: "shrinkage" value { real: 0.1 } } fields { name: "min_examples" value { integer: 5 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "num_candidate_attributes_ratio" value { real: 0.5 } }
[INFO 23-08-16 11:09:04.0316 UTC gradient_boosted_trees.cc:459] Default loss set to BINOMIAL_LOG_LIKELIHOOD
[INFO 23-08-16 11:09:04.0317 UTC gradient_boosted_trees.cc:1085] Training gradient boosted tree on 22792 example(s) and 14 feature(s).
[INFO 23-08-16 11:09:04.0367 UTC gradient_boosted_trees.cc:1128] 20533 examples used for training and 2259 examples used for validation
[INFO 23-08-16 11:09:04.4008 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.008621 train-accuracy:0.761895 valid-loss:1.061219 valid-accuracy:0.736609
[INFO 23-08-16 11:09:07.4914 UTC early_stopping.cc:53] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.604322
[INFO 23-08-16 11:09:07.4914 UTC gradient_boosted_trees.cc:247] Truncates the model to 97 tree(s) i.e. 97  iteration(s).
[INFO 23-08-16 11:09:07.4921 UTC gradient_boosted_trees.cc:310] Final model num-trees:97 valid-loss:0.604322 valid-accuracy:0.860115
[INFO 23-08-16 11:09:07.4942 UTC hyperparameters_optimizer.cc:582] [5/50] Score: -0.604322 / -0.579144 HParams: fields { name: "split_axis" value { categorical: "SPARSE_OBLIQUE" } } fields { name: "sparse_oblique_projection_density_factor" value { real: 4 } } fields { name: "sparse_oblique_normalization" value { categorical: "STANDARD_DEVIATION" } } fields { name: "sparse_oblique_weights" value { categorical: "CONTINUOUS" } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 6 } } fields { name: "sampling_method" value { categorical: "RANDOM" } } fields { name: "subsample" value { real: 0.8 } } fields { name: "shrinkage" value { real: 0.1 } } fields { name: "min_examples" value { integer: 5 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "num_candidate_attributes_ratio" value { real: 0.5 } }
[INFO 23-08-16 11:09:07.4955 UTC gradient_boosted_trees.cc:459] Default loss set to BINOMIAL_LOG_LIKELIHOOD
[INFO 23-08-16 11:09:07.4955 UTC gradient_boosted_trees.cc:1085] Training gradient boosted tree on 22792 example(s) and 14 feature(s).
[INFO 23-08-16 11:09:07.5006 UTC gradient_boosted_trees.cc:1128] 20533 examples used for training and 2259 examples used for validation
[INFO 23-08-16 11:09:07.9368 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.022495 train-accuracy:0.761895 valid-loss:1.078056 valid-accuracy:0.736609
[INFO 23-08-16 11:09:11.4269 UTC gradient_boosted_trees.cc:1542]  num-trees:300 train-loss:0.558511 train-accuracy:0.874446 valid-loss:0.616054 valid-accuracy:0.861000
[INFO 23-08-16 11:09:11.4270 UTC gradient_boosted_trees.cc:247] Truncates the model to 299 tree(s) i.e. 299  iteration(s).
[INFO 23-08-16 11:09:11.4270 UTC gradient_boosted_trees.cc:310] Final model num-trees:299 valid-loss:0.616025 valid-accuracy:0.861000
[INFO 23-08-16 11:09:11.4279 UTC hyperparameters_optimizer.cc:582] [6/50] Score: -0.616025 / -0.579144 HParams: fields { name: "split_axis" value { categorical: "SPARSE_OBLIQUE" } } fields { name: "sparse_oblique_projection_density_factor" value { real: 4 } } fields { name: "sparse_oblique_normalization" value { categorical: "STANDARD_DEVIATION" } } fields { name: "sparse_oblique_weights" value { categorical: "BINARY" } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 3 } } fields { name: "sampling_method" value { categorical: "RANDOM" } } fields { name: "subsample" value { real: 0.9 } } fields { name: "shrinkage" value { real: 0.1 } } fields { name: "min_examples" value { integer: 20 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "num_candidate_attributes_ratio" value { real: 0.2 } }
[INFO 23-08-16 11:09:11.4297 UTC gradient_boosted_trees.cc:459] Default loss set to BINOMIAL_LOG_LIKELIHOOD
[INFO 23-08-16 11:09:11.4297 UTC gradient_boosted_trees.cc:1085] Training gradient boosted tree on 22792 example(s) and 14 feature(s).
[INFO 23-08-16 11:09:11.4346 UTC gradient_boosted_trees.cc:1128] 20533 examples used for training and 2259 examples used for validation
[INFO 23-08-16 11:09:11.5118 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.080831 train-accuracy:0.761895 valid-loss:1.138862 valid-accuracy:0.736609
[INFO 23-08-16 11:09:13.3635 UTC gradient_boosted_trees.cc:1542]  num-trees:300 train-loss:0.588981 train-accuracy:0.865972 valid-loss:0.618730 valid-accuracy:0.857459
[INFO 23-08-16 11:09:13.3636 UTC gradient_boosted_trees.cc:247] Truncates the model to 300 tree(s) i.e. 300  iteration(s).
[INFO 23-08-16 11:09:13.3636 UTC gradient_boosted_trees.cc:310] Final model num-trees:300 valid-loss:0.618730 valid-accuracy:0.857459
[INFO 23-08-16 11:09:13.3652 UTC hyperparameters_optimizer.cc:582] [7/50] Score: -0.61873 / -0.579144 HParams: fields { name: "split_axis" value { categorical: "SPARSE_OBLIQUE" } } fields { name: "sparse_oblique_projection_density_factor" value { real: 5 } } fields { name: "sparse_oblique_normalization" value { categorical: "NONE" } } fields { name: "sparse_oblique_weights" value { categorical: "CONTINUOUS" } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 4 } } fields { name: "sampling_method" value { categorical: "RANDOM" } } fields { name: "subsample" value { real: 0.6 } } fields { name: "shrinkage" value { real: 0.02 } } fields { name: "min_examples" value { integer: 10 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "num_candidate_attributes_ratio" value { real: 1 } }
[INFO 23-08-16 11:09:13.3679 UTC gradient_boosted_trees.cc:459] Default loss set to BINOMIAL_LOG_LIKELIHOOD
[INFO 23-08-16 11:09:13.3679 UTC gradient_boosted_trees.cc:1085] Training gradient boosted tree on 22792 example(s) and 14 feature(s).
[INFO 23-08-16 11:09:13.3727 UTC gradient_boosted_trees.cc:1128] 20533 examples used for training and 2259 examples used for validation
[INFO 23-08-16 11:09:13.5048 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.035720 train-accuracy:0.761895 valid-loss:1.091776 valid-accuracy:0.736609
[INFO 23-08-16 11:09:14.7979 UTC early_stopping.cc:53] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.594984
[INFO 23-08-16 11:09:14.7979 UTC gradient_boosted_trees.cc:247] Truncates the model to 67 tree(s) i.e. 67  iteration(s).
[INFO 23-08-16 11:09:14.7988 UTC gradient_boosted_trees.cc:310] Final model num-trees:67 valid-loss:0.594984 valid-accuracy:0.862328
[INFO 23-08-16 11:09:14.8013 UTC hyperparameters_optimizer.cc:582] [8/50] Score: -0.594984 / -0.579144 HParams: fields { name: "split_axis" value { categorical: "SPARSE_OBLIQUE" } } fields { name: "sparse_oblique_projection_density_factor" value { real: 4 } } fields { name: "sparse_oblique_normalization" value { categorical: "STANDARD_DEVIATION" } } fields { name: "sparse_oblique_weights" value { categorical: "CONTINUOUS" } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 256 } } fields { name: "sampling_method" value { categorical: "RANDOM" } } fields { name: "subsample" value { real: 1 } } fields { name: "shrinkage" value { real: 0.1 } } fields { name: "min_examples" value { integer: 20 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "num_candidate_attributes_ratio" value { real: 0.5 } }
[INFO 23-08-16 11:09:14.8046 UTC gradient_boosted_trees.cc:459] Default loss set to BINOMIAL_LOG_LIKELIHOOD
[INFO 23-08-16 11:09:14.8046 UTC gradient_boosted_trees.cc:1085] Training gradient boosted tree on 22792 example(s) and 14 feature(s).
[INFO 23-08-16 11:09:14.8094 UTC gradient_boosted_trees.cc:1128] 20533 examples used for training and 2259 examples used for validation
[INFO 23-08-16 11:09:14.8846 UTC early_stopping.cc:53] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.601315
[INFO 23-08-16 11:09:14.8847 UTC gradient_boosted_trees.cc:247] Truncates the model to 240 tree(s) i.e. 240  iteration(s).
[INFO 23-08-16 11:09:14.8849 UTC gradient_boosted_trees.cc:310] Final model num-trees:240 valid-loss:0.601315 valid-accuracy:0.865427
[INFO 23-08-16 11:09:14.8861 UTC hyperparameters_optimizer.cc:582] [9/50] Score: -0.601315 / -0.579144 HParams: fields { name: "split_axis" value { categorical: "SPARSE_OBLIQUE" } } fields { name: "sparse_oblique_projection_density_factor" value { real: 3 } } fields { name: "sparse_oblique_normalization" value { categorical: "STANDARD_DEVIATION" } } fields { name: "sparse_oblique_weights" value { categorical: "CONTINUOUS" } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 4 } } fields { name: "sampling_method" value { categorical: "RANDOM" } } fields { name: "subsample" value { real: 0.8 } } fields { name: "shrinkage" value { real: 0.1 } } fields { name: "min_examples" value { integer: 7 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "num_candidate_attributes_ratio" value { real: 0.9 } }
[INFO 23-08-16 11:09:14.8885 UTC gradient_boosted_trees.cc:459] Default loss set to BINOMIAL_LOG_LIKELIHOOD
[INFO 23-08-16 11:09:14.8886 UTC gradient_boosted_trees.cc:1085] Training gradient boosted tree on 22792 example(s) and 14 feature(s).
[INFO 23-08-16 11:09:14.8931 UTC gradient_boosted_trees.cc:1128] 20533 examples used for training and 2259 examples used for validation
[INFO 23-08-16 11:09:15.0170 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.033852 train-accuracy:0.761895 valid-loss:1.089140 valid-accuracy:0.736609
[INFO 23-08-16 11:09:15.2405 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.080070 train-accuracy:0.761895 valid-loss:1.138312 valid-accuracy:0.736609
[INFO 23-08-16 11:09:17.3329 UTC early_stopping.cc:53] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.609047
[INFO 23-08-16 11:09:17.3330 UTC gradient_boosted_trees.cc:247] Truncates the model to 151 tree(s) i.e. 151  iteration(s).
[INFO 23-08-16 11:09:17.3341 UTC gradient_boosted_trees.cc:310] Final model num-trees:151 valid-loss:0.609047 valid-accuracy:0.864542
[INFO 23-08-16 11:09:17.3392 UTC hyperparameters_optimizer.cc:582] [10/50] Score: -0.609047 / -0.579144 HParams: fields { name: "split_axis" value { categorical: "SPARSE_OBLIQUE" } } fields { name: "sparse_oblique_projection_density_factor" value { real: 4 } } fields { name: "sparse_oblique_normalization" value { categorical: "STANDARD_DEVIATION" } } fields { name: "sparse_oblique_weights" value { categorical: "CONTINUOUS" } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 256 } } fields { name: "sampling_method" value { categorical: "RANDOM" } } fields { name: "subsample" value { real: 0.6 } } fields { name: "shrinkage" value { real: 0.05 } } fields { name: "min_examples" value { integer: 5 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "num_candidate_attributes_ratio" value { real: 0.2 } }
[INFO 23-08-16 11:09:17.3474 UTC gradient_boosted_trees.cc:459] Default loss set to BINOMIAL_LOG_LIKELIHOOD
[INFO 23-08-16 11:09:17.3474 UTC gradient_boosted_trees.cc:1085] Training gradient boosted tree on 22792 example(s) and 14 feature(s).
[INFO 23-08-16 11:09:17.3521 UTC gradient_boosted_trees.cc:1128] 20533 examples used for training and 2259 examples used for validation
[INFO 23-08-16 11:09:17.6767 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.055057 train-accuracy:0.761895 valid-loss:1.112117 valid-accuracy:0.736609
[INFO 23-08-16 11:09:18.1050 UTC early_stopping.cc:53] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.585247
[INFO 23-08-16 11:09:18.1051 UTC gradient_boosted_trees.cc:247] Truncates the model to 180 tree(s) i.e. 180  iteration(s).
[INFO 23-08-16 11:09:18.1061 UTC gradient_boosted_trees.cc:310] Final model num-trees:180 valid-loss:0.585247 valid-accuracy:0.865870
[INFO 23-08-16 11:09:18.1116 UTC hyperparameters_optimizer.cc:582] [11/50] Score: -0.585247 / -0.579144 HParams: fields { name: "split_axis" value { categorical: "SPARSE_OBLIQUE" } } fields { name: "sparse_oblique_projection_density_factor" value { real: 3 } } fields { name: "sparse_oblique_normalization" value { categorical: "NONE" } } fields { name: "sparse_oblique_weights" value { categorical: "BINARY" } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 256 } } fields { name: "sampling_method" value { categorical: "RANDOM" } } fields { name: "subsample" value { real: 0.6 } } fields { name: "shrinkage" value { real: 0.05 } } fields { name: "min_examples" value { integer: 7 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "num_candidate_attributes_ratio" value { real: 0.5 } }
[INFO 23-08-16 11:09:18.1204 UTC gradient_boosted_trees.cc:459] Default loss set to BINOMIAL_LOG_LIKELIHOOD
[INFO 23-08-16 11:09:18.1205 UTC gradient_boosted_trees.cc:1085] Training gradient boosted tree on 22792 example(s) and 14 feature(s).
[INFO 23-08-16 11:09:18.1250 UTC gradient_boosted_trees.cc:1128] 20533 examples used for training and 2259 examples used for validation
[INFO 23-08-16 11:09:18.2802 UTC early_stopping.cc:53] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.576761
[INFO 23-08-16 11:09:18.2802 UTC gradient_boosted_trees.cc:247] Truncates the model to 104 tree(s) i.e. 104  iteration(s).
[INFO 23-08-16 11:09:18.2819 UTC gradient_boosted_trees.cc:310] Final model num-trees:104 valid-loss:0.576761 valid-accuracy:0.871182
[INFO 23-08-16 11:09:18.2890 UTC gradient_boosted_trees.cc:459] Default loss set to BINOMIAL_LOG_LIKELIHOOD
[INFO 23-08-16 11:09:18.2890 UTC gradient_boosted_trees.cc:1085] Training gradient boosted tree on 22792 example(s) and 14 feature(s).
[INFO 23-08-16 11:09:18.2920 UTC hyperparameters_optimizer.cc:582] [12/50] Score: -0.576761 / -0.576761 HParams: fields { name: "split_axis" value { categorical: "SPARSE_OBLIQUE" } } fields { name: "sparse_oblique_projection_density_factor" value { real: 2 } } fields { name: "sparse_oblique_normalization" value { categorical: "NONE" } } fields { name: "sparse_oblique_weights" value { categorical: "CONTINUOUS" } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 8 } } fields { name: "sampling_method" value { categorical: "RANDOM" } } fields { name: "subsample" value { real: 0.8 } } fields { name: "shrinkage" value { real: 0.05 } } fields { name: "min_examples" value { integer: 7 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "num_candidate_attributes_ratio" value { real: 1 } }
[INFO 23-08-16 11:09:18.2944 UTC gradient_boosted_trees.cc:1128] 20533 examples used for training and 2259 examples used for validation
[INFO 23-08-16 11:09:18.4524 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.011378 train-accuracy:0.761895 valid-loss:1.065565 valid-accuracy:0.736609
[INFO 23-08-16 11:09:18.7125 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.007702 train-accuracy:0.761895 valid-loss:1.061728 valid-accuracy:0.736609
[INFO 23-08-16 11:09:19.1543 UTC early_stopping.cc:53] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.574235
[INFO 23-08-16 11:09:19.1543 UTC gradient_boosted_trees.cc:247] Truncates the model to 135 tree(s) i.e. 135  iteration(s).
[INFO 23-08-16 11:09:19.1551 UTC gradient_boosted_trees.cc:310] Final model num-trees:135 valid-loss:0.574235 valid-accuracy:0.868969
[INFO 23-08-16 11:09:19.1589 UTC gradient_boosted_trees.cc:459] Default loss set to BINOMIAL_LOG_LIKELIHOOD
[INFO 23-08-16 11:09:19.1590 UTC gradient_boosted_trees.cc:1085] Training gradient boosted tree on 22792 example(s) and 14 feature(s).
[INFO 23-08-16 11:09:19.1635 UTC gradient_boosted_trees.cc:1128] 20533 examples used for training and 2259 examples used for validation
[INFO 23-08-16 11:09:19.1649 UTC hyperparameters_optimizer.cc:582] [13/50] Score: -0.574235 / -0.574235 HParams: fields { name: "split_axis" value { categorical: "SPARSE_OBLIQUE" } } fields { name: "sparse_oblique_projection_density_factor" value { real: 1 } } fields { name: "sparse_oblique_normalization" value { categorical: "NONE" } } fields { name: "sparse_oblique_weights" value { categorical: "BINARY" } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 64 } } fields { name: "sampling_method" value { categorical: "RANDOM" } } fields { name: "subsample" value { real: 1 } } fields { name: "shrinkage" value { real: 0.1 } } fields { name: "min_examples" value { integer: 20 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "num_candidate_attributes_ratio" value { real: 0.2 } }
[INFO 23-08-16 11:09:19.6725 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.054670 train-accuracy:0.761895 valid-loss:1.111134 valid-accuracy:0.736609
[INFO 23-08-16 11:09:19.8336 UTC gradient_boosted_trees.cc:1542]  num-trees:300 train-loss:0.616186 train-accuracy:0.859787 valid-loss:0.644742 valid-accuracy:0.851262
[INFO 23-08-16 11:09:19.8336 UTC gradient_boosted_trees.cc:247] Truncates the model to 300 tree(s) i.e. 300  iteration(s).
[INFO 23-08-16 11:09:19.8336 UTC gradient_boosted_trees.cc:310] Final model num-trees:300 valid-loss:0.644742 valid-accuracy:0.851262
[INFO 23-08-16 11:09:19.8346 UTC hyperparameters_optimizer.cc:582] [14/50] Score: -0.644742 / -0.574235 HParams: fields { name: "split_axis" value { categorical: "SPARSE_OBLIQUE" } } fields { name: "sparse_oblique_projection_density_factor" value { real: 5 } } fields { name: "sparse_oblique_normalization" value { categorical: "NONE" } } fields { name: "sparse_oblique_weights" value { categorical: "CONTINUOUS" } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 3 } } fields { name: "sampling_method" value { categorical: "RANDOM" } } fields { name: "subsample" value { real: 1 } } fields { name: "shrinkage" value { real: 0.02 } } fields { name: "min_examples" value { integer: 10 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "num_candidate_attributes_ratio" value { real: 0.2 } }
[INFO 23-08-16 11:09:19.8361 UTC gradient_boosted_trees.cc:459] Default loss set to BINOMIAL_LOG_LIKELIHOOD
[INFO 23-08-16 11:09:19.8361 UTC gradient_boosted_trees.cc:1085] Training gradient boosted tree on 22792 example(s) and 14 feature(s).
[INFO 23-08-16 11:09:19.8411 UTC gradient_boosted_trees.cc:1128] 20533 examples used for training and 2259 examples used for validation
[INFO 23-08-16 11:09:19.9495 UTC early_stopping.cc:53] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.575853
[INFO 23-08-16 11:09:19.9496 UTC gradient_boosted_trees.cc:247] Truncates the model to 75 tree(s) i.e. 75  iteration(s).
[INFO 23-08-16 11:09:19.9511 UTC gradient_boosted_trees.cc:310] Final model num-trees:75 valid-loss:0.575853 valid-accuracy:0.868083
[INFO 23-08-16 11:09:19.9560 UTC hyperparameters_optimizer.cc:582] [15/50] Score: -0.575853 / -0.574235 HParams: fields { name: "split_axis" value { categorical: "SPARSE_OBLIQUE" } } fields { name: "sparse_oblique_projection_density_factor" value { real: 3 } } fields { name: "sparse_oblique_normalization" value { categorical: "NONE" } } fields { name: "sparse_oblique_weights" value { categorical: "BINARY" } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 8 } } fields { name: "sampling_method" value { categorical: "RANDOM" } } fields { name: "subsample" value { real: 1 } } fields { name: "shrinkage" value { real: 0.1 } } fields { name: "min_examples" value { integer: 5 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "num_candidate_attributes_ratio" value { real: 0.5 } }
[INFO 23-08-16 11:09:19.9631 UTC gradient_boosted_trees.cc:459] Default loss set to BINOMIAL_LOG_LIKELIHOOD
[INFO 23-08-16 11:09:19.9631 UTC gradient_boosted_trees.cc:1085] Training gradient boosted tree on 22792 example(s) and 14 feature(s).
[INFO 23-08-16 11:09:19.9678 UTC gradient_boosted_trees.cc:1128] 20533 examples used for training and 2259 examples used for validation
[INFO 23-08-16 11:09:20.0111 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.034381 train-accuracy:0.761895 valid-loss:1.090479 valid-accuracy:0.736609
[INFO 23-08-16 11:09:20.1262 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.084782 train-accuracy:0.761895 valid-loss:1.143113 valid-accuracy:0.736609
[INFO 23-08-16 11:09:24.1281 UTC early_stopping.cc:53] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.634505
[INFO 23-08-16 11:09:24.1282 UTC gradient_boosted_trees.cc:247] Truncates the model to 91 tree(s) i.e. 91  iteration(s).
[INFO 23-08-16 11:09:24.1289 UTC gradient_boosted_trees.cc:310] Final model num-trees:91 valid-loss:0.634505 valid-accuracy:0.852147
[INFO 23-08-16 11:09:24.1312 UTC hyperparameters_optimizer.cc:582] [16/50] Score: -0.634505 / -0.574235 HParams: fields { name: "split_axis" value { categorical: "SPARSE_OBLIQUE" } } fields { name: "sparse_oblique_projection_density_factor" value { real: 5 } } fields { name: "sparse_oblique_normalization" value { categorical: "MIN_MAX" } } fields { name: "sparse_oblique_weights" value { categorical: "BINARY" } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 32 } } fields { name: "sampling_method" value { categorical: "RANDOM" } } fields { name: "subsample" value { real: 1 } } fields { name: "shrinkage" value { real: 0.1 } } fields { name: "min_examples" value { integer: 10 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "num_candidate_attributes_ratio" value { real: 0.2 } }
[INFO 23-08-16 11:09:24.1345 UTC gradient_boosted_trees.cc:459] Default loss set to BINOMIAL_LOG_LIKELIHOOD
[INFO 23-08-16 11:09:24.1346 UTC gradient_boosted_trees.cc:1085] Training gradient boosted tree on 22792 example(s) and 14 feature(s).
[INFO 23-08-16 11:09:24.1391 UTC gradient_boosted_trees.cc:1128] 20533 examples used for training and 2259 examples used for validation
[INFO 23-08-16 11:09:24.4314 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.078871 train-accuracy:0.761895 valid-loss:1.137038 valid-accuracy:0.736609
[INFO 23-08-16 11:09:27.6264 UTC gradient_boosted_trees.cc:1544]  num-trees:110 train-loss:0.590722 train-accuracy:0.866848 valid-loss:0.629717 valid-accuracy:0.856131
[INFO 23-08-16 11:09:29.2415 UTC gradient_boosted_trees.cc:1542]  num-trees:300 train-loss:0.511615 train-accuracy:0.890712 valid-loss:0.592446 valid-accuracy:0.867641
[INFO 23-08-16 11:09:29.2416 UTC gradient_boosted_trees.cc:247] Truncates the model to 289 tree(s) i.e. 289  iteration(s).
[INFO 23-08-16 11:09:29.2420 UTC gradient_boosted_trees.cc:310] Final model num-trees:289 valid-loss:0.592294 valid-accuracy:0.867641
[INFO 23-08-16 11:09:29.2483 UTC hyperparameters_optimizer.cc:582] [17/50] Score: -0.592294 / -0.574235 HParams: fields { name: "split_axis" value { categorical: "SPARSE_OBLIQUE" } } fields { name: "sparse_oblique_projection_density_factor" value { real: 1 } } fields { name: "sparse_oblique_normalization" value { categorical: "MIN_MAX" } } fields { name: "sparse_oblique_weights" value { categorical: "BINARY" } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 32 } } fields { name: "sampling_method" value { categorical: "RANDOM" } } fields { name: "subsample" value { real: 0.6 } } fields { name: "shrinkage" value { real: 0.02 } } fields { name: "min_examples" value { integer: 20 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "num_candidate_attributes_ratio" value { real: 1 } }
[INFO 23-08-16 11:09:29.2589 UTC gradient_boosted_trees.cc:459] Default loss set to BINOMIAL_LOG_LIKELIHOOD
[INFO 23-08-16 11:09:29.2589 UTC gradient_boosted_trees.cc:1085] Training gradient boosted tree on 22792 example(s) and 14 feature(s).
[INFO 23-08-16 11:09:29.2645 UTC gradient_boosted_trees.cc:1128] 20533 examples used for training and 2259 examples used for validation
[INFO 23-08-16 11:09:29.6504 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.059373 train-accuracy:0.761895 valid-loss:1.116517 valid-accuracy:0.736609
[INFO 23-08-16 11:09:30.1993 UTC gradient_boosted_trees.cc:1542]  num-trees:300 train-loss:0.530666 train-accuracy:0.884235 valid-loss:0.586935 valid-accuracy:0.869411
[INFO 23-08-16 11:09:30.1994 UTC gradient_boosted_trees.cc:247] Truncates the model to 300 tree(s) i.e. 300  iteration(s).
[INFO 23-08-16 11:09:30.1994 UTC gradient_boosted_trees.cc:310] Final model num-trees:300 valid-loss:0.586935 valid-accuracy:0.869411
[INFO 23-08-16 11:09:30.2058 UTC hyperparameters_optimizer.cc:582] [18/50] Score: -0.586935 / -0.574235 HParams: fields { name: "split_axis" value { categorical: "AXIS_ALIGNED" } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 6 } } fields { name: "sampling_method" value { categorical: "RANDOM" } } fields { name: "subsample" value { real: 0.8 } } fields { name: "shrinkage" value { real: 0.02 } } fields { name: "min_examples" value { integer: 5 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "num_candidate_attributes_ratio" value { real: 0.9 } }
[INFO 23-08-16 11:09:30.2064 UTC gradient_boosted_trees.cc:459] Default loss set to BINOMIAL_LOG_LIKELIHOOD
[INFO 23-08-16 11:09:30.2064 UTC gradient_boosted_trees.cc:1085] Training gradient boosted tree on 22792 example(s) and 14 feature(s).
[INFO 23-08-16 11:09:30.2130 UTC gradient_boosted_trees.cc:1128] 20533 examples used for training and 2259 examples used for validation
[INFO 23-08-16 11:09:30.4667 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.055494 train-accuracy:0.761895 valid-loss:1.112262 valid-accuracy:0.736609
[INFO 23-08-16 11:09:39.1713 UTC early_stopping.cc:53] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.57462
[INFO 23-08-16 11:09:39.1713 UTC gradient_boosted_trees.cc:247] Truncates the model to 68 tree(s) i.e. 68  iteration(s).
[INFO 23-08-16 11:09:39.1726 UTC gradient_boosted_trees.cc:310] Final model num-trees:68 valid-loss:0.574620 valid-accuracy:0.869411
[INFO 23-08-16 11:09:39.1758 UTC hyperparameters_optimizer.cc:582] [19/50] Score: -0.57462 / -0.574235 HParams: fields { name: "split_axis" value { categorical: "SPARSE_OBLIQUE" } } fields { name: "sparse_oblique_projection_density_factor" value { real: 2 } } fields { name: "sparse_oblique_normalization" value { categorical: "NONE" } } fields { name: "sparse_oblique_weights" value { categorical: "BINARY" } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 64 } } fields { name: "sampling_method" value { categorical: "RANDOM" } } fields { name: "subsample" value { real: 0.9 } } fields { name: "shrinkage" value { real: 0.1 } } fields { name: "min_examples" value { integer: 7 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "num_candidate_attributes_ratio" value { real: 1 } }
[INFO 23-08-16 11:09:42.3542 UTC gradient_boosted_trees.cc:1542]  num-trees:300 train-loss:0.534859 train-accuracy:0.880047 valid-loss:0.592023 valid-accuracy:0.870297
[INFO 23-08-16 11:09:42.3542 UTC gradient_boosted_trees.cc:247] Truncates the model to 297 tree(s) i.e. 297  iteration(s).
[INFO 23-08-16 11:09:42.3543 UTC gradient_boosted_trees.cc:310] Final model num-trees:297 valid-loss:0.591875 valid-accuracy:0.871182
[INFO 23-08-16 11:09:42.3557 UTC hyperparameters_optimizer.cc:582] [20/50] Score: -0.591875 / -0.574235 HParams: fields { name: "split_axis" value { categorical: "SPARSE_OBLIQUE" } } fields { name: "sparse_oblique_projection_density_factor" value { real: 4 } } fields { name: "sparse_oblique_normalization" value { categorical: "NONE" } } fields { name: "sparse_oblique_weights" value { categorical: "CONTINUOUS" } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 4 } } fields { name: "sampling_method" value { categorical: "RANDOM" } } fields { name: "subsample" value { real: 1 } } fields { name: "shrinkage" value { real: 0.05 } } fields { name: "min_examples" value { integer: 10 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "num_candidate_attributes_ratio" value { real: 0.5 } }
[INFO 23-08-16 11:09:45.1013 UTC early_stopping.cc:53] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.575868
[INFO 23-08-16 11:09:45.1013 UTC gradient_boosted_trees.cc:247] Truncates the model to 156 tree(s) i.e. 156  iteration(s).
[INFO 23-08-16 11:09:45.1024 UTC gradient_boosted_trees.cc:310] Final model num-trees:156 valid-loss:0.575868 valid-accuracy:0.870297
[INFO 23-08-16 11:09:45.1083 UTC hyperparameters_optimizer.cc:582] [21/50] Score: -0.575868 / -0.574235 HParams: fields { name: "split_axis" value { categorical: "SPARSE_OBLIQUE" } } fields { name: "sparse_oblique_projection_density_factor" value { real: 5 } } fields { name: "sparse_oblique_normalization" value { categorical: "NONE" } } fields { name: "sparse_oblique_weights" value { categorical: "CONTINUOUS" } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 256 } } fields { name: "sampling_method" value { categorical: "RANDOM" } } fields { name: "subsample" value { real: 0.8 } } fields { name: "shrinkage" value { real: 0.05 } } fields { name: "min_examples" value { integer: 5 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "num_candidate_attributes_ratio" value { real: 0.5 } }
[INFO 23-08-16 11:09:45.5594 UTC early_stopping.cc:53] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.585791
[INFO 23-08-16 11:09:45.5594 UTC gradient_boosted_trees.cc:247] Truncates the model to 158 tree(s) i.e. 158  iteration(s).
[INFO 23-08-16 11:09:45.5602 UTC gradient_boosted_trees.cc:310] Final model num-trees:158 valid-loss:0.585791 valid-accuracy:0.869854
[INFO 23-08-16 11:09:45.5651 UTC hyperparameters_optimizer.cc:582] [22/50] Score: -0.585791 / -0.574235 HParams: fields { name: "split_axis" value { categorical: "SPARSE_OBLIQUE" } } fields { name: "sparse_oblique_projection_density_factor" value { real: 3 } } fields { name: "sparse_oblique_normalization" value { categorical: "STANDARD_DEVIATION" } } fields { name: "sparse_oblique_weights" value { categorical: "CONTINUOUS" } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 256 } } fields { name: "sampling_method" value { categorical: "RANDOM" } } fields { name: "subsample" value { real: 0.9 } } fields { name: "shrinkage" value { real: 0.05 } } fields { name: "min_examples" value { integer: 10 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "num_candidate_attributes_ratio" value { real: 0.9 } }
[INFO 23-08-16 11:09:48.8690 UTC early_stopping.cc:53] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.575102
[INFO 23-08-16 11:09:48.8690 UTC gradient_boosted_trees.cc:247] Truncates the model to 182 tree(s) i.e. 182  iteration(s).
[INFO 23-08-16 11:09:48.8694 UTC gradient_boosted_trees.cc:310] Final model num-trees:182 valid-loss:0.575102 valid-accuracy:0.870739
[INFO 23-08-16 11:09:48.8715 UTC hyperparameters_optimizer.cc:582] [23/50] Score: -0.575102 / -0.574235 HParams: fields { name: "split_axis" value { categorical: "SPARSE_OBLIQUE" } } fields { name: "sparse_oblique_projection_density_factor" value { real: 2 } } fields { name: "sparse_oblique_normalization" value { categorical: "MIN_MAX" } } fields { name: "sparse_oblique_weights" value { categorical: "BINARY" } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 16 } } fields { name: "sampling_method" value { categorical: "RANDOM" } } fields { name: "subsample" value { real: 1 } } fields { name: "shrinkage" value { real: 0.1 } } fields { name: "min_examples" value { integer: 5 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "num_candidate_attributes_ratio" value { real: 0.5 } }
[INFO 23-08-16 11:09:49.2709 UTC gradient_boosted_trees.cc:1542]  num-trees:300 train-loss:0.522460 train-accuracy:0.884040 valid-loss:0.588174 valid-accuracy:0.871625
[INFO 23-08-16 11:09:49.2709 UTC gradient_boosted_trees.cc:247] Truncates the model to 300 tree(s) i.e. 300  iteration(s).
[INFO 23-08-16 11:09:49.2709 UTC gradient_boosted_trees.cc:310] Final model num-trees:300 valid-loss:0.588174 valid-accuracy:0.871625
[INFO 23-08-16 11:09:49.2758 UTC hyperparameters_optimizer.cc:582] [24/50] Score: -0.588174 / -0.574235 HParams: fields { name: "split_axis" value { categorical: "SPARSE_OBLIQUE" } } fields { name: "sparse_oblique_projection_density_factor" value { real: 1 } } fields { name: "sparse_oblique_normalization" value { categorical: "STANDARD_DEVIATION" } } fields { name: "sparse_oblique_weights" value { categorical: "BINARY" } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 6 } } fields { name: "sampling_method" value { categorical: "RANDOM" } } fields { name: "subsample" value { real: 0.9 } } fields { name: "shrinkage" value { real: 0.02 } } fields { name: "min_examples" value { integer: 7 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "num_candidate_attributes_ratio" value { real: 1 } }
[INFO 23-08-16 11:09:52.4145 UTC gradient_boosted_trees.cc:1542]  num-trees:300 train-loss:0.549486 train-accuracy:0.876881 valid-loss:0.608020 valid-accuracy:0.867198
[INFO 23-08-16 11:09:52.4145 UTC gradient_boosted_trees.cc:247] Truncates the model to 296 tree(s) i.e. 296  iteration(s).
[INFO 23-08-16 11:09:52.4146 UTC gradient_boosted_trees.cc:310] Final model num-trees:296 valid-loss:0.607491 valid-accuracy:0.867198
[INFO 23-08-16 11:09:52.4154 UTC hyperparameters_optimizer.cc:582] [25/50] Score: -0.607491 / -0.574235 HParams: fields { name: "split_axis" value { categorical: "SPARSE_OBLIQUE" } } fields { name: "sparse_oblique_projection_density_factor" value { real: 4 } } fields { name: "sparse_oblique_normalization" value { categorical: "NONE" } } fields { name: "sparse_oblique_weights" value { categorical: "BINARY" } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 3 } } fields { name: "sampling_method" value { categorical: "RANDOM" } } fields { name: "subsample" value { real: 0.8 } } fields { name: "shrinkage" value { real: 0.1 } } fields { name: "min_examples" value { integer: 7 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "num_candidate_attributes_ratio" value { real: 0.5 } }
[INFO 23-08-16 11:09:52.6914 UTC gradient_boosted_trees.cc:1542]  num-trees:300 train-loss:0.547628 train-accuracy:0.878001 valid-loss:0.597969 valid-accuracy:0.867198
[INFO 23-08-16 11:09:52.6914 UTC gradient_boosted_trees.cc:247] Truncates the model to 296 tree(s) i.e. 296  iteration(s).
[INFO 23-08-16 11:09:52.6915 UTC gradient_boosted_trees.cc:310] Final model num-trees:296 valid-loss:0.597909 valid-accuracy:0.867641
[INFO 23-08-16 11:09:52.6923 UTC hyperparameters_optimizer.cc:582] [26/50] Score: -0.597909 / -0.574235 HParams: fields { name: "split_axis" value { categorical: "SPARSE_OBLIQUE" } } fields { name: "sparse_oblique_projection_density_factor" value { real: 3 } } fields { name: "sparse_oblique_normalization" value { categorical: "NONE" } } fields { name: "sparse_oblique_weights" value { categorical: "CONTINUOUS" } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 3 } } fields { name: "sampling_method" value { categorical: "RANDOM" } } fields { name: "subsample" value { real: 0.8 } } fields { name: "shrinkage" value { real: 0.1 } } fields { name: "min_examples" value { integer: 7 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "num_candidate_attributes_ratio" value { real: 1 } }
[INFO 23-08-16 11:09:57.1849 UTC gradient_boosted_trees.cc:1542]  num-trees:300 train-loss:0.465381 train-accuracy:0.898115 valid-loss:0.597106 valid-accuracy:0.864542
[INFO 23-08-16 11:09:57.1850 UTC gradient_boosted_trees.cc:247] Truncates the model to 292 tree(s) i.e. 292  iteration(s).
[INFO 23-08-16 11:09:57.1853 UTC gradient_boosted_trees.cc:310] Final model num-trees:292 valid-loss:0.596803 valid-accuracy:0.864985
[INFO 23-08-16 11:09:57.1977 UTC hyperparameters_optimizer.cc:582] [27/50] Score: -0.596803 / -0.574235 HParams: fields { name: "split_axis" value { categorical: "SPARSE_OBLIQUE" } } fields { name: "sparse_oblique_projection_density_factor" value { real: 3 } } fields { name: "sparse_oblique_normalization" value { categorical: "STANDARD_DEVIATION" } } fields { name: "sparse_oblique_weights" value { categorical: "BINARY" } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 8 } } fields { name: "sampling_method" value { categorical: "RANDOM" } } fields { name: "subsample" value { real: 0.6 } } fields { name: "shrinkage" value { real: 0.02 } } fields { name: "min_examples" value { integer: 20 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "num_candidate_attributes_ratio" value { real: 0.2 } }
[INFO 23-08-16 11:09:57.6505 UTC gradient_boosted_trees.cc:1544]  num-trees:199 train-loss:0.489932 train-accuracy:0.891833 valid-loss:0.590156 valid-accuracy:0.865427
[INFO 23-08-16 11:10:00.7773 UTC gradient_boosted_trees.cc:1542]  num-trees:300 train-loss:0.626160 train-accuracy:0.856962 valid-loss:0.657957 valid-accuracy:0.841080
[INFO 23-08-16 11:10:00.7773 UTC gradient_boosted_trees.cc:247] Truncates the model to 300 tree(s) i.e. 300  iteration(s).
[INFO 23-08-16 11:10:00.7773 UTC gradient_boosted_trees.cc:310] Final model num-trees:300 valid-loss:0.657957 valid-accuracy:0.841080
[INFO 23-08-16 11:10:00.7779 UTC hyperparameters_optimizer.cc:582] [28/50] Score: -0.657957 / -0.574235 HParams: fields { name: "split_axis" value { categorical: "SPARSE_OBLIQUE" } } fields { name: "sparse_oblique_projection_density_factor" value { real: 4 } } fields { name: "sparse_oblique_normalization" value { categorical: "NONE" } } fields { name: "sparse_oblique_weights" value { categorical: "BINARY" } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 3 } } fields { name: "sampling_method" value { categorical: "RANDOM" } } fields { name: "subsample" value { real: 1 } } fields { name: "shrinkage" value { real: 0.02 } } fields { name: "min_examples" value { integer: 5 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "num_candidate_attributes_ratio" value { real: 0.5 } }
[INFO 23-08-16 11:10:01.8466 UTC early_stopping.cc:53] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.582612
[INFO 23-08-16 11:10:01.8467 UTC gradient_boosted_trees.cc:247] Truncates the model to 119 tree(s) i.e. 119  iteration(s).
[INFO 23-08-16 11:10:01.8472 UTC gradient_boosted_trees.cc:310] Final model num-trees:119 valid-loss:0.582612 valid-accuracy:0.865870
[INFO 23-08-16 11:10:01.8493 UTC hyperparameters_optimizer.cc:582] [29/50] Score: -0.582612 / -0.574235 HParams: fields { name: "split_axis" value { categorical: "SPARSE_OBLIQUE" } } fields { name: "sparse_oblique_projection_density_factor" value { real: 2 } } fields { name: "sparse_oblique_normalization" value { categorical: "MIN_MAX" } } fields { name: "sparse_oblique_weights" value { categorical: "BINARY" } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 32 } } fields { name: "sampling_method" value { categorical: "RANDOM" } } fields { name: "subsample" value { real: 0.9 } } fields { name: "shrinkage" value { real: 0.1 } } fields { name: "min_examples" value { integer: 10 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "num_candidate_attributes_ratio" value { real: 0.2 } }
[INFO 23-08-16 11:10:02.3571 UTC gradient_boosted_trees.cc:1542]  num-trees:300 train-loss:0.493647 train-accuracy:0.890761 valid-loss:0.580171 valid-accuracy:0.870297
[INFO 23-08-16 11:10:02.3571 UTC gradient_boosted_trees.cc:247] Truncates the model to 300 tree(s) i.e. 300  iteration(s).
[INFO 23-08-16 11:10:02.3572 UTC gradient_boosted_trees.cc:310] Final model num-trees:300 valid-loss:0.580171 valid-accuracy:0.870297
[INFO 23-08-16 11:10:02.3632 UTC hyperparameters_optimizer.cc:582] [30/50] Score: -0.580171 / -0.574235 HParams: fields { name: "split_axis" value { categorical: "SPARSE_OBLIQUE" } } fields { name: "sparse_oblique_projection_density_factor" value { real: 2 } } fields { name: "sparse_oblique_normalization" value { categorical: "NONE" } } fields { name: "sparse_oblique_weights" value { categorical: "BINARY" } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 128 } } fields { name: "sampling_method" value { categorical: "RANDOM" } } fields { name: "subsample" value { real: 0.8 } } fields { name: "shrinkage" value { real: 0.02 } } fields { name: "min_examples" value { integer: 10 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "num_candidate_attributes_ratio" value { real: 0.9 } }
[INFO 23-08-16 11:10:05.8787 UTC early_stopping.cc:53] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.580436
[INFO 23-08-16 11:10:05.8788 UTC gradient_boosted_trees.cc:247] Truncates the model to 87 tree(s) i.e. 87  iteration(s).
[INFO 23-08-16 11:10:05.8794 UTC gradient_boosted_trees.cc:310] Final model num-trees:87 valid-loss:0.580436 valid-accuracy:0.861443
[INFO 23-08-16 11:10:05.8818 UTC hyperparameters_optimizer.cc:582] [31/50] Score: -0.580436 / -0.574235 HParams: fields { name: "split_axis" value { categorical: "SPARSE_OBLIQUE" } } fields { name: "sparse_oblique_projection_density_factor" value { real: 4 } } fields { name: "sparse_oblique_normalization" value { categorical: "NONE" } } fields { name: "sparse_oblique_weights" value { categorical: "BINARY" } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 64 } } fields { name: "sampling_method" value { categorical: "RANDOM" } } fields { name: "subsample" value { real: 0.9 } } fields { name: "shrinkage" value { real: 0.1 } } fields { name: "min_examples" value { integer: 5 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "num_candidate_attributes_ratio" value { real: 0.9 } }
[INFO 23-08-16 11:10:07.7309 UTC gradient_boosted_trees.cc:1542]  num-trees:300 train-loss:0.552857 train-accuracy:0.875225 valid-loss:0.611411 valid-accuracy:0.861443
[INFO 23-08-16 11:10:07.7309 UTC gradient_boosted_trees.cc:247] Truncates the model to 299 tree(s) i.e. 299  iteration(s).
[INFO 23-08-16 11:10:07.7309 UTC gradient_boosted_trees.cc:310] Final model num-trees:299 valid-loss:0.611399 valid-accuracy:0.861443
[INFO 23-08-16 11:10:07.7315 UTC hyperparameters_optimizer.cc:582] [32/50] Score: -0.611399 / -0.574235 HParams: fields { name: "split_axis" value { categorical: "SPARSE_OBLIQUE" } } fields { name: "sparse_oblique_projection_density_factor" value { real: 4 } } fields { name: "sparse_oblique_normalization" value { categorical: "MIN_MAX" } } fields { name: "sparse_oblique_weights" value { categorical: "CONTINUOUS" } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 3 } } fields { name: "sampling_method" value { categorical: "RANDOM" } } fields { name: "subsample" value { real: 1 } } fields { name: "shrinkage" value { real: 0.1 } } fields { name: "min_examples" value { integer: 5 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "num_candidate_attributes_ratio" value { real: 0.5 } }
[INFO 23-08-16 11:10:10.5852 UTC early_stopping.cc:53] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.593538
[INFO 23-08-16 11:10:10.5853 UTC gradient_boosted_trees.cc:247] Truncates the model to 215 tree(s) i.e. 215  iteration(s).
[INFO 23-08-16 11:10:10.5859 UTC gradient_boosted_trees.cc:310] Final model num-trees:215 valid-loss:0.593538 valid-accuracy:0.860558
[INFO 23-08-16 11:10:10.5908 UTC hyperparameters_optimizer.cc:582] [33/50] Score: -0.593538 / -0.574235 HParams: fields { name: "split_axis" value { categorical: "SPARSE_OBLIQUE" } } fields { name: "sparse_oblique_projection_density_factor" value { real: 4 } } fields { name: "sparse_oblique_normalization" value { categorical: "STANDARD_DEVIATION" } } fields { name: "sparse_oblique_weights" value { categorical: "BINARY" } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 512 } } fields { name: "sampling_method" value { categorical: "RANDOM" } } fields { name: "subsample" value { real: 0.9 } } fields { name: "shrinkage" value { real: 0.05 } } fields { name: "min_examples" value { integer: 5 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "num_candidate_attributes_ratio" value { real: 0.5 } }
[INFO 23-08-16 11:10:12.5319 UTC early_stopping.cc:53] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.623461
[INFO 23-08-16 11:10:12.5320 UTC gradient_boosted_trees.cc:247] Truncates the model to 126 tree(s) i.e. 126  iteration(s).
[INFO 23-08-16 11:10:12.5323 UTC gradient_boosted_trees.cc:310] Final model num-trees:126 valid-loss:0.623461 valid-accuracy:0.852147
[INFO 23-08-16 11:10:12.5342 UTC hyperparameters_optimizer.cc:582] [34/50] Score: -0.623461 / -0.574235 HParams: fields { name: "split_axis" value { categorical: "SPARSE_OBLIQUE" } } fields { name: "sparse_oblique_projection_density_factor" value { real: 5 } } fields { name: "sparse_oblique_normalization" value { categorical: "MIN_MAX" } } fields { name: "sparse_oblique_weights" value { categorical: "BINARY" } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 16 } } fields { name: "sampling_method" value { categorical: "RANDOM" } } fields { name: "subsample" value { real: 1 } } fields { name: "shrinkage" value { real: 0.1 } } fields { name: "min_examples" value { integer: 20 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "num_candidate_attributes_ratio" value { real: 1 } }
[INFO 23-08-16 11:10:13.0852 UTC early_stopping.cc:53] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.642896
[INFO 23-08-16 11:10:13.0853 UTC gradient_boosted_trees.cc:247] Truncates the model to 143 tree(s) i.e. 143  iteration(s).
[INFO 23-08-16 11:10:13.0859 UTC gradient_boosted_trees.cc:310] Final model num-trees:143 valid-loss:0.642896 valid-accuracy:0.849048
[INFO 23-08-16 11:10:13.0893 UTC hyperparameters_optimizer.cc:582] [35/50] Score: -0.642896 / -0.574235 HParams: fields { name: "split_axis" value { categorical: "SPARSE_OBLIQUE" } } fields { name: "sparse_oblique_projection_density_factor" value { real: 5 } } fields { name: "sparse_oblique_normalization" value { categorical: "STANDARD_DEVIATION" } } fields { name: "sparse_oblique_weights" value { categorical: "BINARY" } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 512 } } fields { name: "sampling_method" value { categorical: "RANDOM" } } fields { name: "subsample" value { real: 0.6 } } fields { name: "shrinkage" value { real: 0.05 } } fields { name: "min_examples" value { integer: 10 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "num_candidate_attributes_ratio" value { real: 0.9 } }
[INFO 23-08-16 11:10:13.7497 UTC gradient_boosted_trees.cc:1542]  num-trees:300 train-loss:0.514136 train-accuracy:0.886719 valid-loss:0.582222 valid-accuracy:0.868969
[INFO 23-08-16 11:10:13.7498 UTC gradient_boosted_trees.cc:247] Truncates the model to 300 tree(s) i.e. 300  iteration(s).
[INFO 23-08-16 11:10:13.7498 UTC gradient_boosted_trees.cc:310] Final model num-trees:300 valid-loss:0.582222 valid-accuracy:0.868969
[INFO 23-08-16 11:10:13.7551 UTC hyperparameters_optimizer.cc:582] [36/50] Score: -0.582222 / -0.574235 HParams: fields { name: "split_axis" value { categorical: "SPARSE_OBLIQUE" } } fields { name: "sparse_oblique_projection_density_factor" value { real: 5 } } fields { name: "sparse_oblique_normalization" value { categorical: "NONE" } } fields { name: "sparse_oblique_weights" value { categorical: "BINARY" } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 256 } } fields { name: "sampling_method" value { categorical: "RANDOM" } } fields { name: "subsample" value { real: 0.8 } } fields { name: "shrinkage" value { real: 0.02 } } fields { name: "min_examples" value { integer: 20 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "num_candidate_attributes_ratio" value { real: 1 } }
[INFO 23-08-16 11:10:14.7675 UTC early_stopping.cc:53] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.625355
[INFO 23-08-16 11:10:14.7676 UTC gradient_boosted_trees.cc:247] Truncates the model to 182 tree(s) i.e. 182  iteration(s).
[INFO 23-08-16 11:10:14.7683 UTC gradient_boosted_trees.cc:310] Final model num-trees:182 valid-loss:0.625355 valid-accuracy:0.853032
[INFO 23-08-16 11:10:14.7741 UTC hyperparameters_optimizer.cc:582] [37/50] Score: -0.625355 / -0.574235 HParams: fields { name: "split_axis" value { categorical: "SPARSE_OBLIQUE" } } fields { name: "sparse_oblique_projection_density_factor" value { real: 5 } } fields { name: "sparse_oblique_normalization" value { categorical: "STANDARD_DEVIATION" } } fields { name: "sparse_oblique_weights" value { categorical: "BINARY" } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 8 } } fields { name: "sampling_method" value { categorical: "RANDOM" } } fields { name: "subsample" value { real: 0.9 } } fields { name: "shrinkage" value { real: 0.05 } } fields { name: "min_examples" value { integer: 20 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "num_candidate_attributes_ratio" value { real: 0.9 } }
[INFO 23-08-16 11:10:14.9745 UTC gradient_boosted_trees.cc:1542]  num-trees:300 train-loss:0.500521 train-accuracy:0.890128 valid-loss:0.587961 valid-accuracy:0.861886
[INFO 23-08-16 11:10:14.9745 UTC gradient_boosted_trees.cc:247] Truncates the model to 299 tree(s) i.e. 299  iteration(s).
[INFO 23-08-16 11:10:14.9745 UTC gradient_boosted_trees.cc:310] Final model num-trees:299 valid-loss:0.587948 valid-accuracy:0.861886
[INFO 23-08-16 11:10:14.9808 UTC hyperparameters_optimizer.cc:582] [38/50] Score: -0.587948 / -0.574235 HParams: fields { name: "split_axis" value { categorical: "SPARSE_OBLIQUE" } } fields { name: "sparse_oblique_projection_density_factor" value { real: 2 } } fields { name: "sparse_oblique_normalization" value { categorical: "MIN_MAX" } } fields { name: "sparse_oblique_weights" value { categorical: "BINARY" } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 256 } } fields { name: "sampling_method" value { categorical: "RANDOM" } } fields { name: "subsample" value { real: 1 } } fields { name: "shrinkage" value { real: 0.02 } } fields { name: "min_examples" value { integer: 5 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "num_candidate_attributes_ratio" value { real: 0.9 } }
[INFO 23-08-16 11:10:24.0655 UTC gradient_boosted_trees.cc:1542]  num-trees:300 train-loss:0.481010 train-accuracy:0.894414 valid-loss:0.628041 valid-accuracy:0.853475
[INFO 23-08-16 11:10:24.0655 UTC gradient_boosted_trees.cc:247] Truncates the model to 300 tree(s) i.e. 300  iteration(s).
[INFO 23-08-16 11:10:24.0656 UTC gradient_boosted_trees.cc:310] Final model num-trees:300 valid-loss:0.628041 valid-accuracy:0.853475
[INFO 23-08-16 11:10:24.0721 UTC hyperparameters_optimizer.cc:582] [39/50] Score: -0.628041 / -0.574235 HParams: fields { name: "split_axis" value { categorical: "SPARSE_OBLIQUE" } } fields { name: "sparse_oblique_projection_density_factor" value { real: 5 } } fields { name: "sparse_oblique_normalization" value { categorical: "STANDARD_DEVIATION" } } fields { name: "sparse_oblique_weights" value { categorical: "BINARY" } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 64 } } fields { name: "sampling_method" value { categorical: "RANDOM" } } fields { name: "subsample" value { real: 0.8 } } fields { name: "shrinkage" value { real: 0.02 } } fields { name: "min_examples" value { integer: 7 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "num_candidate_attributes_ratio" value { real: 1 } }
[INFO 23-08-16 11:10:27.6515 UTC gradient_boosted_trees.cc:1544]  num-trees:281 train-loss:0.528716 train-accuracy:0.881995 valid-loss:0.589776 valid-accuracy:0.867641
[INFO 23-08-16 11:10:30.0631 UTC gradient_boosted_trees.cc:1542]  num-trees:300 train-loss:0.492458 train-accuracy:0.890469 valid-loss:0.577581 valid-accuracy:0.868083
[INFO 23-08-16 11:10:30.0631 UTC gradient_boosted_trees.cc:247] Truncates the model to 287 tree(s) i.e. 287  iteration(s).
[INFO 23-08-16 11:10:30.0632 UTC gradient_boosted_trees.cc:310] Final model num-trees:287 valid-loss:0.576924 valid-accuracy:0.868969
[INFO 23-08-16 11:10:30.0655 UTC hyperparameters_optimizer.cc:582] [40/50] Score: -0.576924 / -0.574235 HParams: fields { name: "split_axis" value { categorical: "SPARSE_OBLIQUE" } } fields { name: "sparse_oblique_projection_density_factor" value { real: 1 } } fields { name: "sparse_oblique_normalization" value { categorical: "STANDARD_DEVIATION" } } fields { name: "sparse_oblique_weights" value { categorical: "BINARY" } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 16 } } fields { name: "sampling_method" value { categorical: "RANDOM" } } fields { name: "subsample" value { real: 0.8 } } fields { name: "shrinkage" value { real: 0.05 } } fields { name: "min_examples" value { integer: 5 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "num_candidate_attributes_ratio" value { real: 0.9 } }
[INFO 23-08-16 11:10:32.0797 UTC gradient_boosted_trees.cc:1542]  num-trees:300 train-loss:0.435926 train-accuracy:0.912775 valid-loss:0.599424 valid-accuracy:0.863656
[INFO 23-08-16 11:10:32.0797 UTC gradient_boosted_trees.cc:247] Truncates the model to 299 tree(s) i.e. 299  iteration(s).
[INFO 23-08-16 11:10:32.0798 UTC gradient_boosted_trees.cc:310] Final model num-trees:299 valid-loss:0.599401 valid-accuracy:0.863656
[INFO 23-08-16 11:10:32.0900 UTC hyperparameters_optimizer.cc:582] [41/50] Score: -0.599401 / -0.574235 HParams: fields { name: "split_axis" value { categorical: "SPARSE_OBLIQUE" } } fields { name: "sparse_oblique_projection_density_factor" value { real: 3 } } fields { name: "sparse_oblique_normalization" value { categorical: "MIN_MAX" } } fields { name: "sparse_oblique_weights" value { categorical: "BINARY" } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 8 } } fields { name: "sampling_method" value { categorical: "RANDOM" } } fields { name: "subsample" value { real: 0.6 } } fields { name: "shrinkage" value { real: 0.02 } } fields { name: "min_examples" value { integer: 5 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "num_candidate_attributes_ratio" value { real: 1 } }
[INFO 23-08-16 11:10:32.3847 UTC gradient_boosted_trees.cc:1542]  num-trees:300 train-loss:0.523727 train-accuracy:0.883456 valid-loss:0.587662 valid-accuracy:0.867198
[INFO 23-08-16 11:10:32.3848 UTC gradient_boosted_trees.cc:247] Truncates the model to 300 tree(s) i.e. 300  iteration(s).
[INFO 23-08-16 11:10:32.3848 UTC gradient_boosted_trees.cc:310] Final model num-trees:300 valid-loss:0.587662 valid-accuracy:0.867198
[INFO 23-08-16 11:10:32.3886 UTC hyperparameters_optimizer.cc:582] [42/50] Score: -0.587662 / -0.574235 HParams: fields { name: "split_axis" value { categorical: "SPARSE_OBLIQUE" } } fields { name: "sparse_oblique_projection_density_factor" value { real: 4 } } fields { name: "sparse_oblique_normalization" value { categorical: "NONE" } } fields { name: "sparse_oblique_weights" value { categorical: "BINARY" } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 6 } } fields { name: "sampling_method" value { categorical: "RANDOM" } } fields { name: "subsample" value { real: 0.9 } } fields { name: "shrinkage" value { real: 0.02 } } fields { name: "min_examples" value { integer: 7 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "num_candidate_attributes_ratio" value { real: 0.5 } }
[INFO 23-08-16 11:10:33.0908 UTC gradient_boosted_trees.cc:1542]  num-trees:300 train-loss:0.448174 train-accuracy:0.901719 valid-loss:0.584032 valid-accuracy:0.870297
[INFO 23-08-16 11:10:33.0908 UTC gradient_boosted_trees.cc:247] Truncates the model to 283 tree(s) i.e. 283  iteration(s).
[INFO 23-08-16 11:10:33.0914 UTC gradient_boosted_trees.cc:310] Final model num-trees:283 valid-loss:0.583289 valid-accuracy:0.870297
[INFO 23-08-16 11:10:33.1016 UTC hyperparameters_optimizer.cc:582] [43/50] Score: -0.583289 / -0.574235 HParams: fields { name: "split_axis" value { categorical: "SPARSE_OBLIQUE" } } fields { name: "sparse_oblique_projection_density_factor" value { real: 2 } } fields { name: "sparse_oblique_normalization" value { categorical: "STANDARD_DEVIATION" } } fields { name: "sparse_oblique_weights" value { categorical: "BINARY" } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 8 } } fields { name: "sampling_method" value { categorical: "RANDOM" } } fields { name: "subsample" value { real: 1 } } fields { name: "shrinkage" value { real: 0.02 } } fields { name: "min_examples" value { integer: 5 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "num_candidate_attributes_ratio" value { real: 0.5 } }
[INFO 23-08-16 11:10:34.3194 UTC early_stopping.cc:53] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.636921
[INFO 23-08-16 11:10:34.3195 UTC gradient_boosted_trees.cc:247] Truncates the model to 181 tree(s) i.e. 181  iteration(s).
[INFO 23-08-16 11:10:34.3200 UTC gradient_boosted_trees.cc:310] Final model num-trees:181 valid-loss:0.636921 valid-accuracy:0.853032
[INFO 23-08-16 11:10:34.3236 UTC hyperparameters_optimizer.cc:582] [44/50] Score: -0.636921 / -0.574235 HParams: fields { name: "split_axis" value { categorical: "SPARSE_OBLIQUE" } } fields { name: "sparse_oblique_projection_density_factor" value { real: 5 } } fields { name: "sparse_oblique_normalization" value { categorical: "MIN_MAX" } } fields { name: "sparse_oblique_weights" value { categorical: "BINARY" } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 64 } } fields { name: "sampling_method" value { categorical: "RANDOM" } } fields { name: "subsample" value { real: 0.9 } } fields { name: "shrinkage" value { real: 0.05 } } fields { name: "min_examples" value { integer: 20 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "num_candidate_attributes_ratio" value { real: 1 } }
[INFO 23-08-16 11:10:36.1570 UTC gradient_boosted_trees.cc:1542]  num-trees:300 train-loss:0.435564 train-accuracy:0.914236 valid-loss:0.600726 valid-accuracy:0.862771
[INFO 23-08-16 11:10:36.1571 UTC gradient_boosted_trees.cc:247] Truncates the model to 299 tree(s) i.e. 299  iteration(s).
[INFO 23-08-16 11:10:36.1571 UTC gradient_boosted_trees.cc:310] Final model num-trees:299 valid-loss:0.600701 valid-accuracy:0.862328
[INFO 23-08-16 11:10:36.1679 UTC hyperparameters_optimizer.cc:582] [45/50] Score: -0.600701 / -0.574235 HParams: fields { name: "split_axis" value { categorical: "SPARSE_OBLIQUE" } } fields { name: "sparse_oblique_projection_density_factor" value { real: 4 } } fields { name: "sparse_oblique_normalization" value { categorical: "MIN_MAX" } } fields { name: "sparse_oblique_weights" value { categorical: "BINARY" } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 8 } } fields { name: "sampling_method" value { categorical: "RANDOM" } } fields { name: "subsample" value { real: 0.9 } } fields { name: "shrinkage" value { real: 0.02 } } fields { name: "min_examples" value { integer: 10 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "num_candidate_attributes_ratio" value { real: 0.5 } }
[INFO 23-08-16 11:10:37.1114 UTC gradient_boosted_trees.cc:1542]  num-trees:300 train-loss:0.456672 train-accuracy:0.898067 valid-loss:0.573701 valid-accuracy:0.867641
[INFO 23-08-16 11:10:37.1115 UTC gradient_boosted_trees.cc:247] Truncates the model to 284 tree(s) i.e. 284  iteration(s).
[INFO 23-08-16 11:10:37.1120 UTC gradient_boosted_trees.cc:310] Final model num-trees:284 valid-loss:0.573333 valid-accuracy:0.867198
[INFO 23-08-16 11:10:37.1261 UTC hyperparameters_optimizer.cc:582] [46/50] Score: -0.573333 / -0.573333 HParams: fields { name: "split_axis" value { categorical: "SPARSE_OBLIQUE" } } fields { name: "sparse_oblique_projection_density_factor" value { real: 4 } } fields { name: "sparse_oblique_normalization" value { categorical: "NONE" } } fields { name: "sparse_oblique_weights" value { categorical: "CONTINUOUS" } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "LOCAL" } } fields { name: "max_depth" value { integer: 8 } } fields { name: "sampling_method" value { categorical: "RANDOM" } } fields { name: "subsample" value { real: 0.9 } } fields { name: "shrinkage" value { real: 0.02 } } fields { name: "min_examples" value { integer: 5 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "num_candidate_attributes_ratio" value { real: 0.2 } }
[INFO 23-08-16 11:10:41.5204 UTC gradient_boosted_trees.cc:1542]  num-trees:300 train-loss:0.562199 train-accuracy:0.873326 valid-loss:0.617909 valid-accuracy:0.855246
[INFO 23-08-16 11:10:41.5204 UTC gradient_boosted_trees.cc:247] Truncates the model to 300 tree(s) i.e. 300  iteration(s).
[INFO 23-08-16 11:10:41.5205 UTC gradient_boosted_trees.cc:310] Final model num-trees:300 valid-loss:0.617909 valid-accuracy:0.855246
[INFO 23-08-16 11:10:41.5229 UTC hyperparameters_optimizer.cc:582] [47/50] Score: -0.617909 / -0.573333 HParams: fields { name: "split_axis" value { categorical: "SPARSE_OBLIQUE" } } fields { name: "sparse_oblique_projection_density_factor" value { real: 4 } } fields { name: "sparse_oblique_normalization" value { categorical: "MIN_MAX" } } fields { name: "sparse_oblique_weights" value { categorical: "BINARY" } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 16 } } fields { name: "sampling_method" value { categorical: "RANDOM" } } fields { name: "subsample" value { real: 0.9 } } fields { name: "shrinkage" value { real: 0.02 } } fields { name: "min_examples" value { integer: 7 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "num_candidate_attributes_ratio" value { real: 0.2 } }
[INFO 23-08-16 11:10:50.2502 UTC early_stopping.cc:53] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.575538
[INFO 23-08-16 11:10:50.2503 UTC gradient_boosted_trees.cc:247] Truncates the model to 193 tree(s) i.e. 193  iteration(s).
[INFO 23-08-16 11:10:50.2507 UTC gradient_boosted_trees.cc:310] Final model num-trees:193 valid-loss:0.575538 valid-accuracy:0.866313
[INFO 23-08-16 11:10:50.2539 UTC hyperparameters_optimizer.cc:582] [48/50] Score: -0.575538 / -0.573333 HParams: fields { name: "split_axis" value { categorical: "SPARSE_OBLIQUE" } } fields { name: "sparse_oblique_projection_density_factor" value { real: 5 } } fields { name: "sparse_oblique_normalization" value { categorical: "NONE" } } fields { name: "sparse_oblique_weights" value { categorical: "BINARY" } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 32 } } fields { name: "sampling_method" value { categorical: "RANDOM" } } fields { name: "subsample" value { real: 1 } } fields { name: "shrinkage" value { real: 0.05 } } fields { name: "min_examples" value { integer: 7 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "num_candidate_attributes_ratio" value { real: 1 } }
[INFO 23-08-16 11:10:57.7539 UTC gradient_boosted_trees.cc:1544]  num-trees:275 train-loss:0.494402 train-accuracy:0.890907 valid-loss:0.583064 valid-accuracy:0.868083
[INFO 23-08-16 11:11:00.3725 UTC gradient_boosted_trees.cc:1542]  num-trees:300 train-loss:0.490348 train-accuracy:0.894998 valid-loss:0.577486 valid-accuracy:0.872510
[INFO 23-08-16 11:11:00.3726 UTC gradient_boosted_trees.cc:247] Truncates the model to 300 tree(s) i.e. 300  iteration(s).
[INFO 23-08-16 11:11:00.3726 UTC gradient_boosted_trees.cc:310] Final model num-trees:300 valid-loss:0.577486 valid-accuracy:0.872510
[INFO 23-08-16 11:11:00.3779 UTC hyperparameters_optimizer.cc:582] [49/50] Score: -0.577486 / -0.573333 HParams: fields { name: "split_axis" value { categorical: "SPARSE_OBLIQUE" } } fields { name: "sparse_oblique_projection_density_factor" value { real: 5 } } fields { name: "sparse_oblique_normalization" value { categorical: "NONE" } } fields { name: "sparse_oblique_weights" value { categorical: "CONTINUOUS" } } fields { name: "categorical_algorithm" value { categorical: "RANDOM" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 256 } } fields { name: "sampling_method" value { categorical: "RANDOM" } } fields { name: "subsample" value { real: 0.8 } } fields { name: "shrinkage" value { real: 0.02 } } fields { name: "min_examples" value { integer: 20 } } fields { name: "use_hessian_gain" value { categorical: "false" } } fields { name: "num_candidate_attributes_ratio" value { real: 1 } }
[INFO 23-08-16 11:11:04.5224 UTC gradient_boosted_trees.cc:1542]  num-trees:300 train-loss:0.487159 train-accuracy:0.892320 valid-loss:0.581956 valid-accuracy:0.868083
[INFO 23-08-16 11:11:04.5224 UTC gradient_boosted_trees.cc:247] Truncates the model to 299 tree(s) i.e. 299  iteration(s).
[INFO 23-08-16 11:11:04.5225 UTC gradient_boosted_trees.cc:310] Final model num-trees:299 valid-loss:0.581786 valid-accuracy:0.868083
[INFO 23-08-16 11:11:04.5248 UTC hyperparameters_optimizer.cc:582] [50/50] Score: -0.581786 / -0.573333 HParams: fields { name: "split_axis" value { categorical: "SPARSE_OBLIQUE" } } fields { name: "sparse_oblique_projection_density_factor" value { real: 3 } } fields { name: "sparse_oblique_normalization" value { categorical: "STANDARD_DEVIATION" } } fields { name: "sparse_oblique_weights" value { categorical: "CONTINUOUS" } } fields { name: "categorical_algorithm" value { categorical: "CART" } } fields { name: "growing_strategy" value { categorical: "BEST_FIRST_GLOBAL" } } fields { name: "max_num_nodes" value { integer: 16 } } fields { name: "sampling_method" value { categorical: "RANDOM" } } fields { name: "subsample" value { real: 1 } } fields { name: "shrinkage" value { real: 0.05 } } fields { name: "min_examples" value { integer: 10 } } fields { name: "use_hessian_gain" value { categorical: "true" } } fields { name: "num_candidate_attributes_ratio" value { real: 1 } }
[INFO 23-08-16 11:11:04.5502 UTC hyperparameters_optimizer.cc:219] Best hyperparameters:
fields {
  name: "split_axis"
  value {
    categorical: "SPARSE_OBLIQUE"
  }
}
fields {
  name: "sparse_oblique_projection_density_factor"
  value {
    real: 4
  }
}
fields {
  name: "sparse_oblique_normalization"
  value {
    categorical: "NONE"
  }
}
fields {
  name: "sparse_oblique_weights"
  value {
    categorical: "CONTINUOUS"
  }
}
fields {
  name: "categorical_algorithm"
  value {
    categorical: "CART"
  }
}
fields {
  name: "growing_strategy"
  value {
    categorical: "LOCAL"
  }
}
fields {
  name: "max_depth"
  value {
    integer: 8
  }
}
fields {
  name: "sampling_method"
  value {
    categorical: "RANDOM"
  }
}
fields {
  name: "subsample"
  value {
    real: 0.9
  }
}
fields {
  name: "shrinkage"
  value {
    real: 0.02
  }
}
fields {
  name: "min_examples"
  value {
    integer: 5
  }
}
fields {
  name: "use_hessian_gain"
  value {
    categorical: "true"
  }
}
fields {
  name: "num_candidate_attributes_ratio"
  value {
    real: 0.2
  }
}

[INFO 23-08-16 11:11:04.5509 UTC kernel.cc:926] Export model in log directory: /tmpfs/tmp/tmpfhjg70bi with prefix 2362b151e27349f1
[INFO 23-08-16 11:11:04.5900 UTC kernel.cc:944] Save model in resources
[INFO 23-08-16 11:11:04.5945 UTC abstract_model.cc:849] Model self evaluation:
Task: CLASSIFICATION
Label: __LABEL
Loss (BINOMIAL_LOG_LIKELIHOOD): 0.573333

Accuracy: 0.867198  CI95[W][0 1]
ErrorRate: : 0.132802


Confusion Table:
truth\prediction
   0     1    2
0  0     0    0
1  0  1578   86
2  0   214  381
Total: 2259

One vs other classes:

[INFO 23-08-16 11:11:04.6200 UTC kernel.cc:1243] Loading model from path /tmpfs/tmp/tmpfhjg70bi/model/ with prefix 2362b151e27349f1
[INFO 23-08-16 11:11:04.7821 UTC decision_forest.cc:660] Model loaded with 284 root(s), 48262 node(s), and 14 input feature(s).
[INFO 23-08-16 11:11:04.7822 UTC abstract_model.cc:1311] Engine "GradientBoostedTreesGeneric" built
[INFO 23-08-16 11:11:04.7822 UTC kernel.cc:1075] Use fast generic engine
Model trained in 0:02:38.504656
Compiling model...
Model compiled.
CPU times: user 57min 43s, sys: 1.03 s, total: 57min 44s
Wall time: 2min 39s
<keras.src.callbacks.History at 0x7f240c3141c0>
# Evaluate the model
tuned_model.compile(["accuracy"])
tuned_test_accuracy = tuned_model.evaluate(test_ds, return_dict=True, verbose=0)["accuracy"]
print(f"Test accuracy with the TF-DF hyper-parameter tuner: {tuned_test_accuracy:.4f}")
Test accuracy with the TF-DF hyper-parameter tuner: 0.8741

Same as before, display the tuning logs.

# Display the tuning logs.
tuning_logs = tuned_model.make_inspector().tuning_logs()
tuning_logs.head()

Same as before, shows the best hyper-parameters.

# Best hyper-parameters.
tuning_logs[tuning_logs.best].iloc[0]
score                                            -0.573333
evaluation_time                                 130.817537
best                                                  True
split_axis                                  SPARSE_OBLIQUE
sparse_oblique_projection_density_factor               4.0
sparse_oblique_normalization                          NONE
sparse_oblique_weights                          CONTINUOUS
categorical_algorithm                                 CART
growing_strategy                                     LOCAL
max_num_nodes                                          NaN
sampling_method                                     RANDOM
subsample                                              0.9
shrinkage                                             0.02
min_examples                                             5
use_hessian_gain                                      true
num_candidate_attributes_ratio                         0.2
max_depth                                              8.0
Name: 45, dtype: object

Finally, plots the evolution of the quality of the model during tuning:

plt.figure(figsize=(10, 5))
plt.plot(tuning_logs["score"], label="current trial")
plt.plot(tuning_logs["score"].cummax(), label="best trial")
plt.xlabel("Tuning step")
plt.ylabel("Tuning score")
plt.legend()
plt.show()

png

Training a model with Keras Tuner (Alternative approach)

TensorFlow Decision Forests is based on the Keras framework, and it is compatible with the Keras tuner.

Currently, the TF-DF Tuner and the Keras Tuner are complementary.

TF-DF Tuner

  • Automatic configuration of the objective.
  • Automatic extraction of validation dataset (if needed).
  • Support model self evaluation (e.g. out-of-bag evaluation).
  • Distributed hyper-parameter tuning.
  • Shared dataset access in between the trials: The tensorflow dataset is read only once, speeding-up tuning significantly on small datasets.

Keras Tuner

  • Support tuning of the pre-processing parameters.
  • Support hyper-band optimizer.
  • Support custom objectives.

Let's tune a TF-DF model using the Keras tuner.

# Install the Keras tuner
!pip install keras-tuner -U -qq
import keras_tuner as kt
%%time

def build_model(hp):
  """Creates a model."""

  model = tfdf.keras.GradientBoostedTreesModel(
      min_examples=hp.Choice("min_examples", [2, 5, 7, 10]),
      categorical_algorithm=hp.Choice("categorical_algorithm", ["CART", "RANDOM"]),
      max_depth=hp.Choice("max_depth", [4, 5, 6, 7]),
      # The keras tuner convert automaticall boolean parameters to integers.
      use_hessian_gain=bool(hp.Choice("use_hessian_gain", [True, False])),
      shrinkage=hp.Choice("shrinkage", [0.02, 0.05, 0.10, 0.15]),
      num_candidate_attributes_ratio=hp.Choice("num_candidate_attributes_ratio", [0.2, 0.5, 0.9, 1.0]),
  )

  # Optimize the model accuracy as computed on the validation dataset.
  model.compile(metrics=["accuracy"])
  return model

keras_tuner = kt.RandomSearch(
    build_model,
    objective="val_accuracy",
    max_trials=50,
    overwrite=True,
    directory="/tmp/keras_tuning")

# Important: The tuning should not be done on the test dataset.

# Extract a validation dataset from the training dataset. The new training
# dataset is called the "sub-training-dataset".

def split_dataset(dataset, test_ratio=0.30):
  """Splits a panda dataframe in two."""
  test_indices = np.random.rand(len(dataset)) < test_ratio
  return dataset[~test_indices], dataset[test_indices]

sub_train_df, sub_valid_df = split_dataset(train_df)
sub_train_ds = tfdf.keras.pd_dataframe_to_tf_dataset(sub_train_df, label="income")
sub_valid_ds = tfdf.keras.pd_dataframe_to_tf_dataset(sub_valid_df, label="income")

# Tune the model
keras_tuner.search(sub_train_ds, validation_data=sub_valid_ds)
Trial 50 Complete [00h 00m 09s]
val_accuracy: 0.8768961429595947

Best val_accuracy So Far: 0.8815636038780212
Total elapsed time: 00h 03m 58s
INFO:tensorflow:Oracle triggered exit
INFO:tensorflow:Oracle triggered exit
CPU times: user 6min 39s, sys: 1min 8s, total: 7min 47s
Wall time: 3min 57s

The best hyper-parameter are available with get_best_hyperparameters:

# Tune the model
best_hyper_parameters = keras_tuner.get_best_hyperparameters()[0].values
print("Best hyper-parameters:", keras_tuner.get_best_hyperparameters()[0].values)
Best hyper-parameters: {'min_examples': 10, 'categorical_algorithm': 'CART', 'max_depth': 6, 'use_hessian_gain': 1, 'shrinkage': 0.1, 'num_candidate_attributes_ratio': 0.9}

The model should be re-trained with the best hyper-parameters:

%set_cell_height 300
# Train the model
# The keras tuner convert automaticall boolean parameters to integers.
best_hyper_parameters["use_hessian_gain"] = bool(best_hyper_parameters["use_hessian_gain"])
best_model = tfdf.keras.GradientBoostedTreesModel(**best_hyper_parameters)
best_model.fit(train_ds, verbose=2)
<IPython.core.display.Javascript object>
Warning: The `num_threads` constructor argument is not set and the number of CPU is os.cpu_count()=32 > 32. Setting num_threads to 32. Set num_threads manually to use more than 32 cpus.
WARNING:absl:The `num_threads` constructor argument is not set and the number of CPU is os.cpu_count()=32 > 32. Setting num_threads to 32. Set num_threads manually to use more than 32 cpus.
Use /tmpfs/tmp/tmpewzhl309 as temporary training directory
Reading training dataset...
Training tensor examples:
Features: {'age': <tf.Tensor 'data:0' shape=(None,) dtype=int64>, 'workclass': <tf.Tensor 'data_1:0' shape=(None,) dtype=string>, 'fnlwgt': <tf.Tensor 'data_2:0' shape=(None,) dtype=int64>, 'education': <tf.Tensor 'data_3:0' shape=(None,) dtype=string>, 'education_num': <tf.Tensor 'data_4:0' shape=(None,) dtype=int64>, 'marital_status': <tf.Tensor 'data_5:0' shape=(None,) dtype=string>, 'occupation': <tf.Tensor 'data_6:0' shape=(None,) dtype=string>, 'relationship': <tf.Tensor 'data_7:0' shape=(None,) dtype=string>, 'race': <tf.Tensor 'data_8:0' shape=(None,) dtype=string>, 'sex': <tf.Tensor 'data_9:0' shape=(None,) dtype=string>, 'capital_gain': <tf.Tensor 'data_10:0' shape=(None,) dtype=int64>, 'capital_loss': <tf.Tensor 'data_11:0' shape=(None,) dtype=int64>, 'hours_per_week': <tf.Tensor 'data_12:0' shape=(None,) dtype=int64>, 'native_country': <tf.Tensor 'data_13:0' shape=(None,) dtype=string>}
Label: Tensor("data_14:0", shape=(None,), dtype=int64)
Weights: None
Normalized tensor features:
 {'age': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast:0' shape=(None,) dtype=float32>), 'workclass': SemanticTensor(semantic=<Semantic.CATEGORICAL: 2>, tensor=<tf.Tensor 'data_1:0' shape=(None,) dtype=string>), 'fnlwgt': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_1:0' shape=(None,) dtype=float32>), 'education': SemanticTensor(semantic=<Semantic.CATEGORICAL: 2>, tensor=<tf.Tensor 'data_3:0' shape=(None,) dtype=string>), 'education_num': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_2:0' shape=(None,) dtype=float32>), 'marital_status': SemanticTensor(semantic=<Semantic.CATEGORICAL: 2>, tensor=<tf.Tensor 'data_5:0' shape=(None,) dtype=string>), 'occupation': SemanticTensor(semantic=<Semantic.CATEGORICAL: 2>, tensor=<tf.Tensor 'data_6:0' shape=(None,) dtype=string>), 'relationship': SemanticTensor(semantic=<Semantic.CATEGORICAL: 2>, tensor=<tf.Tensor 'data_7:0' shape=(None,) dtype=string>), 'race': SemanticTensor(semantic=<Semantic.CATEGORICAL: 2>, tensor=<tf.Tensor 'data_8:0' shape=(None,) dtype=string>), 'sex': SemanticTensor(semantic=<Semantic.CATEGORICAL: 2>, tensor=<tf.Tensor 'data_9:0' shape=(None,) dtype=string>), 'capital_gain': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_3:0' shape=(None,) dtype=float32>), 'capital_loss': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_4:0' shape=(None,) dtype=float32>), 'hours_per_week': SemanticTensor(semantic=<Semantic.NUMERICAL: 1>, tensor=<tf.Tensor 'Cast_5:0' shape=(None,) dtype=float32>), 'native_country': SemanticTensor(semantic=<Semantic.CATEGORICAL: 2>, tensor=<tf.Tensor 'data_13:0' shape=(None,) dtype=string>)}
[WARNING 23-08-16 11:15:06.4338 UTC gradient_boosted_trees.cc:1818] "goss_alpha" set but "sampling_method" not equal to "GOSS".
[WARNING 23-08-16 11:15:06.4338 UTC gradient_boosted_trees.cc:1829] "goss_beta" set but "sampling_method" not equal to "GOSS".
[WARNING 23-08-16 11:15:06.4338 UTC gradient_boosted_trees.cc:1843] "selective_gradient_boosting_ratio" set but "sampling_method" not equal to "SELGB".
Training dataset read in 0:00:00.389690. Found 22792 examples.
Training model...
[INFO 23-08-16 11:15:06.8353 UTC kernel.cc:773] Start Yggdrasil model training
[INFO 23-08-16 11:15:06.8353 UTC kernel.cc:774] Collect training examples
[INFO 23-08-16 11:15:06.8354 UTC kernel.cc:787] Dataspec guide:
column_guides {
  column_name_pattern: "^__LABEL$"
  type: CATEGORICAL
  categorial {
    min_vocab_frequency: 0
    max_vocab_count: -1
  }
}
default_column_guide {
  categorial {
    max_vocab_count: 2000
  }
  discretized_numerical {
    maximum_num_bins: 255
  }
}
ignore_columns_without_guides: false
detect_numerical_as_discretized_numerical: false

[INFO 23-08-16 11:15:06.8355 UTC kernel.cc:393] Number of batches: 23
[INFO 23-08-16 11:15:06.8355 UTC kernel.cc:394] Number of examples: 22792
[INFO 23-08-16 11:15:06.8429 UTC data_spec_inference.cc:305] 1 item(s) have been pruned (i.e. they are considered out of dictionary) for the column native_country (40 item(s) left) because min_value_count=5 and max_number_of_unique_values=2000
[INFO 23-08-16 11:15:06.8429 UTC data_spec_inference.cc:305] 1 item(s) have been pruned (i.e. they are considered out of dictionary) for the column occupation (13 item(s) left) because min_value_count=5 and max_number_of_unique_values=2000
[INFO 23-08-16 11:15:06.8430 UTC data_spec_inference.cc:305] 1 item(s) have been pruned (i.e. they are considered out of dictionary) for the column workclass (7 item(s) left) because min_value_count=5 and max_number_of_unique_values=2000
[INFO 23-08-16 11:15:06.8491 UTC kernel.cc:794] Training dataset:
Number of records: 22792
Number of columns: 15

Number of columns by type:
    CATEGORICAL: 9 (60%)
    NUMERICAL: 6 (40%)

Columns:

CATEGORICAL: 9 (60%)
    0: "__LABEL" CATEGORICAL integerized vocab-size:3 no-ood-item
    4: "education" CATEGORICAL has-dict vocab-size:17 zero-ood-items most-frequent:"HS-grad" 7340 (32.2043%)
    8: "marital_status" CATEGORICAL has-dict vocab-size:8 zero-ood-items most-frequent:"Married-civ-spouse" 10431 (45.7661%)
    9: "native_country" CATEGORICAL num-nas:407 (1.78571%) has-dict vocab-size:41 num-oods:1 (0.00446728%) most-frequent:"United-States" 20436 (91.2933%)
    10: "occupation" CATEGORICAL num-nas:1260 (5.52826%) has-dict vocab-size:14 num-oods:1 (0.00464425%) most-frequent:"Prof-specialty" 2870 (13.329%)
    11: "race" CATEGORICAL has-dict vocab-size:6 zero-ood-items most-frequent:"White" 19467 (85.4115%)
    12: "relationship" CATEGORICAL has-dict vocab-size:7 zero-ood-items most-frequent:"Husband" 9191 (40.3256%)
    13: "sex" CATEGORICAL has-dict vocab-size:3 zero-ood-items most-frequent:"Male" 15165 (66.5365%)
    14: "workclass" CATEGORICAL num-nas:1257 (5.51509%) has-dict vocab-size:8 num-oods:1 (0.0046436%) most-frequent:"Private" 15879 (73.7358%)

NUMERICAL: 6 (40%)
    1: "age" NUMERICAL mean:38.6153 min:17 max:90 sd:13.661
    2: "capital_gain" NUMERICAL mean:1081.9 min:0 max:99999 sd:7509.48
    3: "capital_loss" NUMERICAL mean:87.2806 min:0 max:4356 sd:403.01
    5: "education_num" NUMERICAL mean:10.0927 min:1 max:16 sd:2.56427
    6: "fnlwgt" NUMERICAL mean:189879 min:12285 max:1.4847e+06 sd:106423
    7: "hours_per_week" NUMERICAL mean:40.3955 min:1 max:99 sd:12.249

Terminology:
    nas: Number of non-available (i.e. missing) values.
    ood: Out of dictionary.
    manually-defined: Attribute which type is manually defined by the user i.e. the type was not automatically inferred.
    tokenized: The attribute value is obtained through tokenization.
    has-dict: The attribute is attached to a string dictionary e.g. a categorical attribute stored as a string.
    vocab-size: Number of unique values.

[INFO 23-08-16 11:15:06.8492 UTC kernel.cc:810] Configure learner
[WARNING 23-08-16 11:15:06.8494 UTC gradient_boosted_trees.cc:1818] "goss_alpha" set but "sampling_method" not equal to "GOSS".
[WARNING 23-08-16 11:15:06.8494 UTC gradient_boosted_trees.cc:1829] "goss_beta" set but "sampling_method" not equal to "GOSS".
[WARNING 23-08-16 11:15:06.8494 UTC gradient_boosted_trees.cc:1843] "selective_gradient_boosting_ratio" set but "sampling_method" not equal to "SELGB".
[INFO 23-08-16 11:15:06.8495 UTC kernel.cc:824] Training config:
learner: "GRADIENT_BOOSTED_TREES"
features: "^age$"
features: "^capital_gain$"
features: "^capital_loss$"
features: "^education$"
features: "^education_num$"
features: "^fnlwgt$"
features: "^hours_per_week$"
features: "^marital_status$"
features: "^native_country$"
features: "^occupation$"
features: "^race$"
features: "^relationship$"
features: "^sex$"
features: "^workclass$"
label: "^__LABEL$"
task: CLASSIFICATION
random_seed: 123456
metadata {
  framework: "TF Keras"
}
pure_serving_model: false
[yggdrasil_decision_forests.model.gradient_boosted_trees.proto.gradient_boosted_trees_config] {
  num_trees: 300
  decision_tree {
    max_depth: 6
    min_examples: 10
    in_split_min_examples_check: true
    keep_non_leaf_label_distribution: true
    missing_value_policy: GLOBAL_IMPUTATION
    allow_na_conditions: false
    categorical_set_greedy_forward {
      sampling: 0.1
      max_num_items: -1
      min_item_frequency: 1
    }
    growing_strategy_local {
    }
    categorical {
      cart {
      }
    }
    num_candidate_attributes_ratio: 0.9
    axis_aligned_split {
    }
    internal {
      sorting_strategy: PRESORTED
    }
    uplift {
      min_examples_in_treatment: 5
      split_score: KULLBACK_LEIBLER
    }
  }
  shrinkage: 0.1
  loss: DEFAULT
  validation_set_ratio: 0.1
  validation_interval_in_trees: 1
  early_stopping: VALIDATION_LOSS_INCREASE
  early_stopping_num_trees_look_ahead: 30
  l2_regularization: 0
  lambda_loss: 1
  mart {
  }
  adapt_subsample_for_maximum_training_duration: false
  l1_regularization: 0
  use_hessian_gain: true
  l2_regularization_categorical: 1
  stochastic_gradient_boosting {
    ratio: 1
  }
  apply_link_function: true
  compute_permutation_variable_importance: false
  binary_focal_loss_options {
    misprediction_exponent: 2
    positive_sample_coefficient: 0.5
  }
  early_stopping_initial_iteration: 10
}

[INFO 23-08-16 11:15:06.8496 UTC kernel.cc:827] Deployment config:
cache_path: "/tmpfs/tmp/tmpewzhl309/working_cache"
num_threads: 32
try_resume_training: true

[INFO 23-08-16 11:15:06.8497 UTC kernel.cc:889] Train model
[INFO 23-08-16 11:15:06.8498 UTC gradient_boosted_trees.cc:459] Default loss set to BINOMIAL_LOG_LIKELIHOOD
[INFO 23-08-16 11:15:06.8499 UTC gradient_boosted_trees.cc:1085] Training gradient boosted tree on 22792 example(s) and 14 feature(s).
[INFO 23-08-16 11:15:06.8529 UTC gradient_boosted_trees.cc:1128] 20533 examples used for training and 2259 examples used for validation
[INFO 23-08-16 11:15:06.8732 UTC gradient_boosted_trees.cc:1542]  num-trees:1 train-loss:1.017085 train-accuracy:0.761895 valid-loss:1.072747 valid-accuracy:0.736609
[INFO 23-08-16 11:15:09.3786 UTC early_stopping.cc:53] Early stop of the training because the validation loss does not decrease anymore. Best valid-loss: 0.568219
[INFO 23-08-16 11:15:09.3789 UTC gradient_boosted_trees.cc:1594] Create final snapshot of the model at iteration 201
[INFO 23-08-16 11:15:09.3854 UTC gradient_boosted_trees.cc:247] Truncates the model to 172 tree(s) i.e. 172  iteration(s).
[INFO 23-08-16 11:15:09.3856 UTC gradient_boosted_trees.cc:310] Final model num-trees:172 valid-loss:0.568219 valid-accuracy:0.871182
[INFO 23-08-16 11:15:09.3873 UTC kernel.cc:926] Export model in log directory: /tmpfs/tmp/tmpewzhl309 with prefix 955b81bc89424569
[INFO 23-08-16 11:15:09.3925 UTC kernel.cc:944] Save model in resources
[INFO 23-08-16 11:15:09.3951 UTC abstract_model.cc:849] Model self evaluation:
Task: CLASSIFICATION
Label: __LABEL
Loss (BINOMIAL_LOG_LIKELIHOOD): 0.568219

Accuracy: 0.871182  CI95[W][0 1]
ErrorRate: : 0.128818


Confusion Table:
truth\prediction
   0     1    2
0  0     0    0
1  0  1573   91
2  0   200  395
Total: 2259

One vs other classes:

[INFO 23-08-16 11:15:09.4133 UTC kernel.cc:1243] Loading model from path /tmpfs/tmp/tmpewzhl309/model/ with prefix 955b81bc89424569
[INFO 23-08-16 11:15:09.4369 UTC kernel.cc:1075] Use fast generic engine
Model trained in 0:00:02.607547
Compiling model...
Model compiled.
<keras.src.callbacks.History at 0x7f240c4c63a0>

We can then evaluate the tuned model:

# Evaluate the model
best_model.compile(["accuracy"])
tuned_test_accuracy = best_model.evaluate(test_ds, return_dict=True, verbose=0)["accuracy"]
print(f"Test accuracy with the Keras Tuner: {tuned_test_accuracy:.4f}")
Test accuracy with the Keras Tuner: 0.8722