Shortcuts

Hyperparameter tuning with Ray Tune

Hyperparameter tuning can make the difference between an average model and a highly accurate one. Often simple things like choosing a different learning rate or changing a network layer size can have a dramatic impact on your model performance.

Fortunately, there are tools that help with finding the best combination of parameters. Ray Tune is an industry standard tool for distributed hyperparameter tuning. Ray Tune includes the latest hyperparameter search algorithms, integrates with TensorBoard and other analysis libraries, and natively supports distributed training through Ray’s distributed machine learning engine.

In this tutorial, we will show you how to integrate Ray Tune into your PyTorch training workflow. We will extend this tutorial from the PyTorch documentation for training a CIFAR10 image classifier.

As you will see, we only need to add some slight modifications. In particular, we need to

  1. wrap data loading and training in functions,
  2. make some network parameters configurable,
  3. add checkpointing (optional),
  4. and define the search space for the model tuning

To run this tutorial, please make sure the following packages are installed:

  • ray[tune]: Distributed hyperparameter tuning library
  • torchvision: For the data transformers

Setup / Imports

Let’s start with the imports:

from functools import partial
import numpy as np
import os
import torch
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim
from torch.utils.data import random_split
import torchvision
import torchvision.transforms as transforms
from ray import tune
from ray.tune import CLIReporter
from ray.tune.schedulers import ASHAScheduler

Most of the imports are needed for building the PyTorch model. Only the last three imports are for Ray Tune.

Data loaders

We wrap the data loaders in their own function and pass a global data directory. This way we can share a data directory between different trials.

def load_data(data_dir="./data"):
    transform = transforms.Compose([
        transforms.ToTensor(),
        transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))
    ])

    trainset = torchvision.datasets.CIFAR10(
        root=data_dir, train=True, download=True, transform=transform)

    testset = torchvision.datasets.CIFAR10(
        root=data_dir, train=False, download=True, transform=transform)

    return trainset, testset

Configurable neural network

We can only tune those parameters that are configurable. In this example, we can specify the layer sizes of the fully connected layers:

class Net(nn.Module):
    def __init__(self, l1=120, l2=84):
        super(Net, self).__init__()
        self.conv1 = nn.Conv2d(3, 6, 5)
        self.pool = nn.MaxPool2d(2, 2)
        self.conv2 = nn.Conv2d(6, 16, 5)
        self.fc1 = nn.Linear(16 * 5 * 5, l1)
        self.fc2 = nn.Linear(l1, l2)
        self.fc3 = nn.Linear(l2, 10)

    def forward(self, x):
        x = self.pool(F.relu(self.conv1(x)))
        x = self.pool(F.relu(self.conv2(x)))
        x = x.view(-1, 16 * 5 * 5)
        x = F.relu(self.fc1(x))
        x = F.relu(self.fc2(x))
        x = self.fc3(x)
        return x

The train function

Now it gets interesting, because we introduce some changes to the example from the PyTorch documentation.

We wrap the training script in a function train_cifar(config, checkpoint_dir=None, data_dir=None). As you can guess, the config parameter will receive the hyperparameters we would like to train with. The checkpoint_dir parameter is used to restore checkpoints. The data_dir specifies the directory where we load and store the data, so multiple runs can share the same data source.

net = Net(config["l1"], config["l2"])

if checkpoint_dir:
    model_state, optimizer_state = torch.load(
        os.path.join(checkpoint_dir, "checkpoint"))
    net.load_state_dict(model_state)
    optimizer.load_state_dict(optimizer_state)

The learning rate of the optimizer is made configurable, too:

optimizer = optim.SGD(net.parameters(), lr=config["lr"], momentum=0.9)

We also split the training data into a training and validation subset. We thus train on 80% of the data and calculate the validation loss on the remaining 20%. The batch sizes with which we iterate through the training and test sets are configurable as well.

Adding (multi) GPU support with DataParallel

Image classification benefits largely from GPUs. Luckily, we can continue to use PyTorch’s abstractions in Ray Tune. Thus, we can wrap our model in nn.DataParallel to support data parallel training on multiple GPUs:

device = "cpu"
if torch.cuda.is_available():
    device = "cuda:0"
    if torch.cuda.device_count() > 1:
        net = nn.DataParallel(net)
net.to(device)

By using a device variable we make sure that training also works when we have no GPUs available. PyTorch requires us to send our data to the GPU memory explicitly, like this:

for i, data in enumerate(trainloader, 0):
    inputs, labels = data
    inputs, labels = inputs.to(device), labels.to(device)

The code now supports training on CPUs, on a single GPU, and on multiple GPUs. Notably, Ray also supports fractional GPUs so we can share GPUs among trials, as long as the model still fits on the GPU memory. We’ll come back to that later.

Communicating with Ray Tune

The most interesting part is the communication with Ray Tune:

with tune.checkpoint_dir(epoch) as checkpoint_dir:
    path = os.path.join(checkpoint_dir, "checkpoint")
    torch.save((net.state_dict(), optimizer.state_dict()), path)

tune.report(loss=(val_loss / val_steps), accuracy=correct / total)

Here we first save a checkpoint and then report some metrics back to Ray Tune. Specifically, we send the validation loss and accuracy back to Ray Tune. Ray Tune can then use these metrics to decide which hyperparameter configuration lead to the best results. These metrics can also be used to stop bad performing trials early in order to avoid wasting resources on those trials.

The checkpoint saving is optional, however, it is necessary if we wanted to use advanced schedulers like Population Based Training. Also, by saving the checkpoint we can later load the trained models and validate them on a test set.

Full training function

The full code example looks like this:

def train_cifar(config, checkpoint_dir=None, data_dir=None):
    net = Net(config["l1"], config["l2"])

    device = "cpu"
    if torch.cuda.is_available():
        device = "cuda:0"
        if torch.cuda.device_count() > 1:
            net = nn.DataParallel(net)
    net.to(device)

    criterion = nn.CrossEntropyLoss()
    optimizer = optim.SGD(net.parameters(), lr=config["lr"], momentum=0.9)

    if checkpoint_dir:
        model_state, optimizer_state = torch.load(
            os.path.join(checkpoint_dir, "checkpoint"))
        net.load_state_dict(model_state)
        optimizer.load_state_dict(optimizer_state)

    trainset, testset = load_data(data_dir)

    test_abs = int(len(trainset) * 0.8)
    train_subset, val_subset = random_split(
        trainset, [test_abs, len(trainset) - test_abs])

    trainloader = torch.utils.data.DataLoader(
        train_subset,
        batch_size=int(config["batch_size"]),
        shuffle=True,
        num_workers=8)
    valloader = torch.utils.data.DataLoader(
        val_subset,
        batch_size=int(config["batch_size"]),
        shuffle=True,
        num_workers=8)

    for epoch in range(10):  # loop over the dataset multiple times
        running_loss = 0.0
        epoch_steps = 0
        for i, data in enumerate(trainloader, 0):
            # get the inputs; data is a list of [inputs, labels]
            inputs, labels = data
            inputs, labels = inputs.to(device), labels.to(device)

            # zero the parameter gradients
            optimizer.zero_grad()

            # forward + backward + optimize
            outputs = net(inputs)
            loss = criterion(outputs, labels)
            loss.backward()
            optimizer.step()

            # print statistics
            running_loss += loss.item()
            epoch_steps += 1
            if i % 2000 == 1999:  # print every 2000 mini-batches
                print("[%d, %5d] loss: %.3f" % (epoch + 1, i + 1,
                                                running_loss / epoch_steps))
                running_loss = 0.0

        # Validation loss
        val_loss = 0.0
        val_steps = 0
        total = 0
        correct = 0
        for i, data in enumerate(valloader, 0):
            with torch.no_grad():
                inputs, labels = data
                inputs, labels = inputs.to(device), labels.to(device)

                outputs = net(inputs)
                _, predicted = torch.max(outputs.data, 1)
                total += labels.size(0)
                correct += (predicted == labels).sum().item()

                loss = criterion(outputs, labels)
                val_loss += loss.cpu().numpy()
                val_steps += 1

        with tune.checkpoint_dir(epoch) as checkpoint_dir:
            path = os.path.join(checkpoint_dir, "checkpoint")
            torch.save((net.state_dict(), optimizer.state_dict()), path)

        tune.report(loss=(val_loss / val_steps), accuracy=correct / total)
    print("Finished Training")

As you can see, most of the code is adapted directly from the original example.

Test set accuracy

Commonly the performance of a machine learning model is tested on a hold-out test set with data that has not been used for training the model. We also wrap this in a function:

def test_accuracy(net, device="cpu"):
    trainset, testset = load_data()

    testloader = torch.utils.data.DataLoader(
        testset, batch_size=4, shuffle=False, num_workers=2)

    correct = 0
    total = 0
    with torch.no_grad():
        for data in testloader:
            images, labels = data
            images, labels = images.to(device), labels.to(device)
            outputs = net(images)
            _, predicted = torch.max(outputs.data, 1)
            total += labels.size(0)
            correct += (predicted == labels).sum().item()

    return correct / total

The function also expects a device parameter, so we can do the test set validation on a GPU.

Configuring the search space

Lastly, we need to define Ray Tune’s search space. Here is an example:

config = {
    "l1": tune.sample_from(lambda _: 2**np.random.randint(2, 9)),
    "l2": tune.sample_from(lambda _: 2**np.random.randint(2, 9)),
    "lr": tune.loguniform(1e-4, 1e-1),
    "batch_size": tune.choice([2, 4, 8, 16])
}

The tune.sample_from() function makes it possible to define your own sample methods to obtain hyperparameters. In this example, the l1 and l2 parameters should be powers of 2 between 4 and 256, so either 4, 8, 16, 32, 64, 128, or 256. The lr (learning rate) should be uniformly sampled between 0.0001 and 0.1. Lastly, the batch size is a choice between 2, 4, 8, and 16.

At each trial, Ray Tune will now randomly sample a combination of parameters from these search spaces. It will then train a number of models in parallel and find the best performing one among these. We also use the ASHAScheduler which will terminate bad performing trials early.

We wrap the train_cifar function with functools.partial to set the constant data_dir parameter. We can also tell Ray Tune what resources should be available for each trial:

gpus_per_trial = 2
# ...
result = tune.run(
    partial(train_cifar, data_dir=data_dir),
    resources_per_trial={"cpu": 8, "gpu": gpus_per_trial},
    config=config,
    num_samples=num_samples,
    scheduler=scheduler,
    progress_reporter=reporter,
    checkpoint_at_end=True)

You can specify the number of CPUs, which are then available e.g. to increase the num_workers of the PyTorch DataLoader instances. The selected number of GPUs are made visible to PyTorch in each trial. Trials do not have access to GPUs that haven’t been requested for them - so you don’t have to care about two trials using the same set of resources.

Here we can also specify fractional GPUs, so something like gpus_per_trial=0.5 is completely valid. The trials will then share GPUs among each other. You just have to make sure that the models still fit in the GPU memory.

After training the models, we will find the best performing one and load the trained network from the checkpoint file. We then obtain the test set accuracy and report everything by printing.

The full main function looks like this:

def main(num_samples=10, max_num_epochs=10, gpus_per_trial=2):
    data_dir = os.path.abspath("./data")
    load_data(data_dir)
    config = {
        "l1": tune.sample_from(lambda _: 2 ** np.random.randint(2, 9)),
        "l2": tune.sample_from(lambda _: 2 ** np.random.randint(2, 9)),
        "lr": tune.loguniform(1e-4, 1e-1),
        "batch_size": tune.choice([2, 4, 8, 16])
    }
    scheduler = ASHAScheduler(
        metric="loss",
        mode="min",
        max_t=max_num_epochs,
        grace_period=1,
        reduction_factor=2)
    reporter = CLIReporter(
        # parameter_columns=["l1", "l2", "lr", "batch_size"],
        metric_columns=["loss", "accuracy", "training_iteration"])
    result = tune.run(
        partial(train_cifar, data_dir=data_dir),
        resources_per_trial={"cpu": 2, "gpu": gpus_per_trial},
        config=config,
        num_samples=num_samples,
        scheduler=scheduler,
        progress_reporter=reporter)

    best_trial = result.get_best_trial("loss", "min", "last")
    print("Best trial config: {}".format(best_trial.config))
    print("Best trial final validation loss: {}".format(
        best_trial.last_result["loss"]))
    print("Best trial final validation accuracy: {}".format(
        best_trial.last_result["accuracy"]))

    best_trained_model = Net(best_trial.config["l1"], best_trial.config["l2"])
    device = "cpu"
    if torch.cuda.is_available():
        device = "cuda:0"
        if gpus_per_trial > 1:
            best_trained_model = nn.DataParallel(best_trained_model)
    best_trained_model.to(device)

    best_checkpoint_dir = best_trial.checkpoint.value
    model_state, optimizer_state = torch.load(os.path.join(
        best_checkpoint_dir, "checkpoint"))
    best_trained_model.load_state_dict(model_state)

    test_acc = test_accuracy(best_trained_model, device)
    print("Best trial test set accuracy: {}".format(test_acc))


if __name__ == "__main__":
    # You can change the number of GPUs per trial here:
    main(num_samples=10, max_num_epochs=10, gpus_per_trial=0)

Out:

Downloading https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz to /var/lib/jenkins/workspace/beginner_source/data/cifar-10-python.tar.gz
Extracting /var/lib/jenkins/workspace/beginner_source/data/cifar-10-python.tar.gz to /var/lib/jenkins/workspace/beginner_source/data
Files already downloaded and verified
== Status ==
Memory usage on this node: 4.1/240.1 GiB
Using AsyncHyperBand: num_stopped=0
Bracket: Iter 8.000: None | Iter 4.000: None | Iter 2.000: None | Iter 1.000: None
Resources requested: 2/32 CPUs, 0/2 GPUs, 0.0/157.76 GiB heap, 0.0/49.41 GiB objects
Result logdir: /var/lib/jenkins/ray_results/DEFAULT
Number of trials: 10 (9 PENDING, 1 RUNNING)
+---------------------+----------+-------+--------------+------+------+-------------+
| Trial name          | status   | loc   |   batch_size |   l1 |   l2 |          lr |
|---------------------+----------+-------+--------------+------+------+-------------|
| DEFAULT_1dde1_00000 | RUNNING  |       |            4 |  128 |    4 | 0.00648265  |
| DEFAULT_1dde1_00001 | PENDING  |       |            2 |   16 |    4 | 0.000419419 |
| DEFAULT_1dde1_00002 | PENDING  |       |           16 |   64 |  128 | 0.0486304   |
| DEFAULT_1dde1_00003 | PENDING  |       |            2 |   32 |    4 | 0.0290972   |
| DEFAULT_1dde1_00004 | PENDING  |       |           16 |   16 |  128 | 0.0422261   |
| DEFAULT_1dde1_00005 | PENDING  |       |            8 |  256 |  128 | 0.000362061 |
| DEFAULT_1dde1_00006 | PENDING  |       |            4 |  256 |   16 | 0.00696066  |
| DEFAULT_1dde1_00007 | PENDING  |       |            4 |  128 |   32 | 0.00139994  |
| DEFAULT_1dde1_00008 | PENDING  |       |            8 |    8 |  128 | 0.000178276 |
| DEFAULT_1dde1_00009 | PENDING  |       |            2 |   32 |   32 | 0.000438826 |
+---------------------+----------+-------+--------------+------+------+-------------+


(pid=1122) Files already downloaded and verified
(pid=1168) Files already downloaded and verified
(pid=1112) Files already downloaded and verified
(pid=1084) Files already downloaded and verified
(pid=1090) Files already downloaded and verified
(pid=1118) Files already downloaded and verified
(pid=1104) Files already downloaded and verified
(pid=1097) Files already downloaded and verified
(pid=1108) Files already downloaded and verified
(pid=1085) Files already downloaded and verified
(pid=1122) Files already downloaded and verified
(pid=1168) Files already downloaded and verified
(pid=1112) Files already downloaded and verified
(pid=1084) Files already downloaded and verified
(pid=1090) Files already downloaded and verified
(pid=1118) Files already downloaded and verified
(pid=1104) Files already downloaded and verified
(pid=1097) Files already downloaded and verified
(pid=1108) Files already downloaded and verified
(pid=1085) Files already downloaded and verified
(pid=1122) [1,  2000] loss: 2.310
(pid=1112) [1,  2000] loss: 2.339
(pid=1097) [1,  2000] loss: 2.300
(pid=1118) [1,  2000] loss: 2.215
(pid=1085) [1,  2000] loss: 2.174
(pid=1104) [1,  2000] loss: 2.049
(pid=1084) [1,  2000] loss: 2.303
(pid=1108) [1,  2000] loss: 2.290
(pid=1090) [1,  2000] loss: 2.080
(pid=1168) [1,  2000] loss: 2.066
(pid=1122) [1,  4000] loss: 1.124
(pid=1097) [1,  4000] loss: 1.107
(pid=1112) [1,  4000] loss: 1.167
(pid=1118) [1,  4000] loss: 1.000
(pid=1085) [1,  4000] loss: 0.936
Result for DEFAULT_1dde1_00004:
  accuracy: 0.2354
  date: 2020-09-22_01-01-47
  done: false
  experiment_id: 0e63a81fef9141bdb0635e2bd7085e0e
  experiment_tag: 4_batch_size=16,l1=16,l2=128,lr=0.042226
  hostname: fed1025d76e5
  iterations_since_restore: 1
  loss: 1.9921342937469482
  node_ip: 172.17.0.2
  pid: 1090
  should_checkpoint: true
  time_since_restore: 30.549424409866333
  time_this_iter_s: 30.549424409866333
  time_total_s: 30.549424409866333
  timestamp: 1600736507
  timesteps_since_restore: 0
  training_iteration: 1
  trial_id: 1dde1_00004

== Status ==
Memory usage on this node: 8.9/240.1 GiB
Using AsyncHyperBand: num_stopped=0
Bracket: Iter 8.000: None | Iter 4.000: None | Iter 2.000: None | Iter 1.000: -1.9921342937469482
Resources requested: 20/32 CPUs, 0/2 GPUs, 0.0/157.76 GiB heap, 0.0/49.41 GiB objects
Result logdir: /var/lib/jenkins/ray_results/DEFAULT
Number of trials: 10 (10 RUNNING)
+---------------------+----------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name          | status   | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|---------------------+----------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| DEFAULT_1dde1_00000 | RUNNING  |                 |            4 |  128 |    4 | 0.00648265  |         |            |                      |
| DEFAULT_1dde1_00001 | RUNNING  |                 |            2 |   16 |    4 | 0.000419419 |         |            |                      |
| DEFAULT_1dde1_00002 | RUNNING  |                 |           16 |   64 |  128 | 0.0486304   |         |            |                      |
| DEFAULT_1dde1_00003 | RUNNING  |                 |            2 |   32 |    4 | 0.0290972   |         |            |                      |
| DEFAULT_1dde1_00004 | RUNNING  | 172.17.0.2:1090 |           16 |   16 |  128 | 0.0422261   | 1.99213 |     0.2354 |                    1 |
| DEFAULT_1dde1_00005 | RUNNING  |                 |            8 |  256 |  128 | 0.000362061 |         |            |                      |
| DEFAULT_1dde1_00006 | RUNNING  |                 |            4 |  256 |   16 | 0.00696066  |         |            |                      |
| DEFAULT_1dde1_00007 | RUNNING  |                 |            4 |  128 |   32 | 0.00139994  |         |            |                      |
| DEFAULT_1dde1_00008 | RUNNING  |                 |            8 |    8 |  128 | 0.000178276 |         |            |                      |
| DEFAULT_1dde1_00009 | RUNNING  |                 |            2 |   32 |   32 | 0.000438826 |         |            |                      |
+---------------------+----------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


(pid=1104) [1,  4000] loss: 0.927
Result for DEFAULT_1dde1_00002:
  accuracy: 0.2616
  date: 2020-09-22_01-01-48
  done: true
  experiment_id: 28d45c999f63400bba7dbc305fc5a1be
  experiment_tag: 2_batch_size=16,l1=64,l2=128,lr=0.04863
  hostname: fed1025d76e5
  iterations_since_restore: 1
  loss: 2.0254285234451292
  node_ip: 172.17.0.2
  pid: 1168
  should_checkpoint: true
  time_since_restore: 31.062172651290894
  time_this_iter_s: 31.062172651290894
  time_total_s: 31.062172651290894
  timestamp: 1600736508
  timesteps_since_restore: 0
  training_iteration: 1
  trial_id: 1dde1_00002

(pid=1084) [1,  4000] loss: 1.150
(pid=1108) [1,  4000] loss: 1.045
(pid=1122) [1,  6000] loss: 0.700
(pid=1097) [1,  6000] loss: 0.668
(pid=1112) [1,  6000] loss: 0.778
(pid=1118) [1,  6000] loss: 0.657
(pid=1085) [1,  6000] loss: 0.557
(pid=1104) [1,  6000] loss: 0.597
Result for DEFAULT_1dde1_00008:
  accuracy: 0.1467
  date: 2020-09-22_01-02-00
  done: true
  experiment_id: 07b7e6afe93442f599e1fa458d9d45c7
  experiment_tag: 8_batch_size=8,l1=8,l2=128,lr=0.00017828
  hostname: fed1025d76e5
  iterations_since_restore: 1
  loss: 2.2940939085006713
  node_ip: 172.17.0.2
  pid: 1084
  should_checkpoint: true
  time_since_restore: 43.29920506477356
  time_this_iter_s: 43.29920506477356
  time_total_s: 43.29920506477356
  timestamp: 1600736520
  timesteps_since_restore: 0
  training_iteration: 1
  trial_id: 1dde1_00008

== Status ==
Memory usage on this node: 8.4/240.1 GiB
Using AsyncHyperBand: num_stopped=2
Bracket: Iter 8.000: None | Iter 4.000: None | Iter 2.000: None | Iter 1.000: -2.0254285234451292
Resources requested: 18/32 CPUs, 0/2 GPUs, 0.0/157.76 GiB heap, 0.0/49.41 GiB objects
Result logdir: /var/lib/jenkins/ray_results/DEFAULT
Number of trials: 10 (9 RUNNING, 1 TERMINATED)
+---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name          | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| DEFAULT_1dde1_00000 | RUNNING    |                 |            4 |  128 |    4 | 0.00648265  |         |            |                      |
| DEFAULT_1dde1_00001 | RUNNING    |                 |            2 |   16 |    4 | 0.000419419 |         |            |                      |
| DEFAULT_1dde1_00002 | TERMINATED |                 |           16 |   64 |  128 | 0.0486304   | 2.02543 |     0.2616 |                    1 |
| DEFAULT_1dde1_00003 | RUNNING    |                 |            2 |   32 |    4 | 0.0290972   |         |            |                      |
| DEFAULT_1dde1_00004 | RUNNING    | 172.17.0.2:1090 |           16 |   16 |  128 | 0.0422261   | 1.99213 |     0.2354 |                    1 |
| DEFAULT_1dde1_00005 | RUNNING    |                 |            8 |  256 |  128 | 0.000362061 |         |            |                      |
| DEFAULT_1dde1_00006 | RUNNING    |                 |            4 |  256 |   16 | 0.00696066  |         |            |                      |
| DEFAULT_1dde1_00007 | RUNNING    |                 |            4 |  128 |   32 | 0.00139994  |         |            |                      |
| DEFAULT_1dde1_00008 | RUNNING    | 172.17.0.2:1084 |            8 |    8 |  128 | 0.000178276 | 2.29409 |     0.1467 |                    1 |
| DEFAULT_1dde1_00009 | RUNNING    |                 |            2 |   32 |   32 | 0.000438826 |         |            |                      |
+---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


(pid=1090) [2,  2000] loss: 2.075
Result for DEFAULT_1dde1_00005:
  accuracy: 0.3182
  date: 2020-09-22_01-02-03
  done: false
  experiment_id: 6a1270e5c59840d8acaa81bba30ebb03
  experiment_tag: 5_batch_size=8,l1=256,l2=128,lr=0.00036206
  hostname: fed1025d76e5
  iterations_since_restore: 1
  loss: 1.8762480222702027
  node_ip: 172.17.0.2
  pid: 1108
  should_checkpoint: true
  time_since_restore: 45.70140099525452
  time_this_iter_s: 45.70140099525452
  time_total_s: 45.70140099525452
  timestamp: 1600736523
  timesteps_since_restore: 0
  training_iteration: 1
  trial_id: 1dde1_00005

(pid=1122) [1,  8000] loss: 0.491
(pid=1097) [1,  8000] loss: 0.463
(pid=1112) [1,  8000] loss: 0.583
(pid=1118) [1,  8000] loss: 0.484
(pid=1085) [1,  8000] loss: 0.396
(pid=1104) [1,  8000] loss: 0.446
Result for DEFAULT_1dde1_00004:
  accuracy: 0.1864
  date: 2020-09-22_01-02-09
  done: false
  experiment_id: 0e63a81fef9141bdb0635e2bd7085e0e
  experiment_tag: 4_batch_size=16,l1=16,l2=128,lr=0.042226
  hostname: fed1025d76e5
  iterations_since_restore: 2
  loss: 2.11986173248291
  node_ip: 172.17.0.2
  pid: 1090
  should_checkpoint: true
  time_since_restore: 52.03898477554321
  time_this_iter_s: 21.48956036567688
  time_total_s: 52.03898477554321
  timestamp: 1600736529
  timesteps_since_restore: 0
  training_iteration: 2
  trial_id: 1dde1_00004

== Status ==
Memory usage on this node: 7.8/240.1 GiB
Using AsyncHyperBand: num_stopped=2
Bracket: Iter 8.000: None | Iter 4.000: None | Iter 2.000: -2.11986173248291 | Iter 1.000: -2.0087814085960387
Resources requested: 16/32 CPUs, 0/2 GPUs, 0.0/157.76 GiB heap, 0.0/49.41 GiB objects
Result logdir: /var/lib/jenkins/ray_results/DEFAULT
Number of trials: 10 (8 RUNNING, 2 TERMINATED)
+---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name          | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| DEFAULT_1dde1_00000 | RUNNING    |                 |            4 |  128 |    4 | 0.00648265  |         |            |                      |
| DEFAULT_1dde1_00001 | RUNNING    |                 |            2 |   16 |    4 | 0.000419419 |         |            |                      |
| DEFAULT_1dde1_00002 | TERMINATED |                 |           16 |   64 |  128 | 0.0486304   | 2.02543 |     0.2616 |                    1 |
| DEFAULT_1dde1_00003 | RUNNING    |                 |            2 |   32 |    4 | 0.0290972   |         |            |                      |
| DEFAULT_1dde1_00004 | RUNNING    | 172.17.0.2:1090 |           16 |   16 |  128 | 0.0422261   | 2.11986 |     0.1864 |                    2 |
| DEFAULT_1dde1_00005 | RUNNING    | 172.17.0.2:1108 |            8 |  256 |  128 | 0.000362061 | 1.87625 |     0.3182 |                    1 |
| DEFAULT_1dde1_00006 | RUNNING    |                 |            4 |  256 |   16 | 0.00696066  |         |            |                      |
| DEFAULT_1dde1_00007 | RUNNING    |                 |            4 |  128 |   32 | 0.00139994  |         |            |                      |
| DEFAULT_1dde1_00008 | TERMINATED |                 |            8 |    8 |  128 | 0.000178276 | 2.29409 |     0.1467 |                    1 |
| DEFAULT_1dde1_00009 | RUNNING    |                 |            2 |   32 |   32 | 0.000438826 |         |            |                      |
+---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


(pid=1122) [1, 10000] loss: 0.378
(pid=1097) [1, 10000] loss: 0.351
(pid=1112) [1, 10000] loss: 0.468
(pid=1108) [2,  2000] loss: 1.810
(pid=1118) [1, 10000] loss: 0.382
(pid=1085) [1, 10000] loss: 0.305
(pid=1104) [1, 10000] loss: 0.356
(pid=1122) [1, 12000] loss: 0.303
(pid=1097) [1, 12000] loss: 0.282
(pid=1112) [1, 12000] loss: 0.390
(pid=1090) [3,  2000] loss: 2.039
Result for DEFAULT_1dde1_00000:
  accuracy: 0.2675
  date: 2020-09-22_01-02-24
  done: true
  experiment_id: 29a6426cadfc4173891a788f70eb2ee4
  experiment_tag: 0_batch_size=4,l1=128,l2=4,lr=0.0064826
  hostname: fed1025d76e5
  iterations_since_restore: 1
  loss: 2.025877732205391
  node_ip: 172.17.0.2
  pid: 1118
  should_checkpoint: true
  time_since_restore: 67.19454288482666
  time_this_iter_s: 67.19454288482666
  time_total_s: 67.19454288482666
  timestamp: 1600736544
  timesteps_since_restore: 0
  training_iteration: 1
  trial_id: 1dde1_00000

== Status ==
Memory usage on this node: 7.9/240.1 GiB
Using AsyncHyperBand: num_stopped=3
Bracket: Iter 8.000: None | Iter 4.000: None | Iter 2.000: -2.11986173248291 | Iter 1.000: -2.0254285234451292
Resources requested: 16/32 CPUs, 0/2 GPUs, 0.0/157.76 GiB heap, 0.0/49.41 GiB objects
Result logdir: /var/lib/jenkins/ray_results/DEFAULT
Number of trials: 10 (8 RUNNING, 2 TERMINATED)
+---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name          | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| DEFAULT_1dde1_00000 | RUNNING    | 172.17.0.2:1118 |            4 |  128 |    4 | 0.00648265  | 2.02588 |     0.2675 |                    1 |
| DEFAULT_1dde1_00001 | RUNNING    |                 |            2 |   16 |    4 | 0.000419419 |         |            |                      |
| DEFAULT_1dde1_00002 | TERMINATED |                 |           16 |   64 |  128 | 0.0486304   | 2.02543 |     0.2616 |                    1 |
| DEFAULT_1dde1_00003 | RUNNING    |                 |            2 |   32 |    4 | 0.0290972   |         |            |                      |
| DEFAULT_1dde1_00004 | RUNNING    | 172.17.0.2:1090 |           16 |   16 |  128 | 0.0422261   | 2.11986 |     0.1864 |                    2 |
| DEFAULT_1dde1_00005 | RUNNING    | 172.17.0.2:1108 |            8 |  256 |  128 | 0.000362061 | 1.87625 |     0.3182 |                    1 |
| DEFAULT_1dde1_00006 | RUNNING    |                 |            4 |  256 |   16 | 0.00696066  |         |            |                      |
| DEFAULT_1dde1_00007 | RUNNING    |                 |            4 |  128 |   32 | 0.00139994  |         |            |                      |
| DEFAULT_1dde1_00008 | TERMINATED |                 |            8 |    8 |  128 | 0.000178276 | 2.29409 |     0.1467 |                    1 |
| DEFAULT_1dde1_00009 | RUNNING    |                 |            2 |   32 |   32 | 0.000438826 |         |            |                      |
+---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


Result for DEFAULT_1dde1_00007:
  accuracy: 0.4737
  date: 2020-09-22_01-02-25
  done: false
  experiment_id: d130aa97ad4d43d1a015b0a98a769c9b
  experiment_tag: 7_batch_size=4,l1=128,l2=32,lr=0.0013999
  hostname: fed1025d76e5
  iterations_since_restore: 1
  loss: 1.46903585190773
  node_ip: 172.17.0.2
  pid: 1085
  should_checkpoint: true
  time_since_restore: 67.89717030525208
  time_this_iter_s: 67.89717030525208
  time_total_s: 67.89717030525208
  timestamp: 1600736545
  timesteps_since_restore: 0
  training_iteration: 1
  trial_id: 1dde1_00007

Result for DEFAULT_1dde1_00006:
  accuracy: 0.3355
  date: 2020-09-22_01-02-27
  done: false
  experiment_id: c90e171237b14250992a8ac860d4bf3d
  experiment_tag: 6_batch_size=4,l1=256,l2=16,lr=0.0069607
  hostname: fed1025d76e5
  iterations_since_restore: 1
  loss: 1.746452518939972
  node_ip: 172.17.0.2
  pid: 1104
  should_checkpoint: true
  time_since_restore: 69.79255509376526
  time_this_iter_s: 69.79255509376526
  time_total_s: 69.79255509376526
  timestamp: 1600736547
  timesteps_since_restore: 0
  training_iteration: 1
  trial_id: 1dde1_00006

(pid=1108) [2,  4000] loss: 0.840
(pid=1122) [1, 14000] loss: 0.253
Result for DEFAULT_1dde1_00004:
  accuracy: 0.2282
  date: 2020-09-22_01-02-30
  done: false
  experiment_id: 0e63a81fef9141bdb0635e2bd7085e0e
  experiment_tag: 4_batch_size=16,l1=16,l2=128,lr=0.042226
  hostname: fed1025d76e5
  iterations_since_restore: 3
  loss: 1.9991267137527466
  node_ip: 172.17.0.2
  pid: 1090
  should_checkpoint: true
  time_since_restore: 72.84821891784668
  time_this_iter_s: 20.809234142303467
  time_total_s: 72.84821891784668
  timestamp: 1600736550
  timesteps_since_restore: 0
  training_iteration: 3
  trial_id: 1dde1_00004

== Status ==
Memory usage on this node: 7.4/240.1 GiB
Using AsyncHyperBand: num_stopped=3
Bracket: Iter 8.000: None | Iter 4.000: None | Iter 2.000: -2.11986173248291 | Iter 1.000: -1.9921342937469482
Resources requested: 14/32 CPUs, 0/2 GPUs, 0.0/157.76 GiB heap, 0.0/49.41 GiB objects
Result logdir: /var/lib/jenkins/ray_results/DEFAULT
Number of trials: 10 (7 RUNNING, 3 TERMINATED)
+---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name          | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| DEFAULT_1dde1_00000 | TERMINATED |                 |            4 |  128 |    4 | 0.00648265  | 2.02588 |     0.2675 |                    1 |
| DEFAULT_1dde1_00001 | RUNNING    |                 |            2 |   16 |    4 | 0.000419419 |         |            |                      |
| DEFAULT_1dde1_00002 | TERMINATED |                 |           16 |   64 |  128 | 0.0486304   | 2.02543 |     0.2616 |                    1 |
| DEFAULT_1dde1_00003 | RUNNING    |                 |            2 |   32 |    4 | 0.0290972   |         |            |                      |
| DEFAULT_1dde1_00004 | RUNNING    | 172.17.0.2:1090 |           16 |   16 |  128 | 0.0422261   | 1.99913 |     0.2282 |                    3 |
| DEFAULT_1dde1_00005 | RUNNING    | 172.17.0.2:1108 |            8 |  256 |  128 | 0.000362061 | 1.87625 |     0.3182 |                    1 |
| DEFAULT_1dde1_00006 | RUNNING    | 172.17.0.2:1104 |            4 |  256 |   16 | 0.00696066  | 1.74645 |     0.3355 |                    1 |
| DEFAULT_1dde1_00007 | RUNNING    | 172.17.0.2:1085 |            4 |  128 |   32 | 0.00139994  | 1.46904 |     0.4737 |                    1 |
| DEFAULT_1dde1_00008 | TERMINATED |                 |            8 |    8 |  128 | 0.000178276 | 2.29409 |     0.1467 |                    1 |
| DEFAULT_1dde1_00009 | RUNNING    |                 |            2 |   32 |   32 | 0.000438826 |         |            |                      |
+---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


(pid=1097) [1, 14000] loss: 0.234
(pid=1112) [1, 14000] loss: 0.334
(pid=1085) [2,  2000] loss: 1.454
(pid=1104) [2,  2000] loss: 1.762
Result for DEFAULT_1dde1_00005:
  accuracy: 0.4157
  date: 2020-09-22_01-02-38
  done: false
  experiment_id: 6a1270e5c59840d8acaa81bba30ebb03
  experiment_tag: 5_batch_size=8,l1=256,l2=128,lr=0.00036206
  hostname: fed1025d76e5
  iterations_since_restore: 2
  loss: 1.6115445574760436
  node_ip: 172.17.0.2
  pid: 1108
  should_checkpoint: true
  time_since_restore: 80.84446930885315
  time_this_iter_s: 35.14306831359863
  time_total_s: 80.84446930885315
  timestamp: 1600736558
  timesteps_since_restore: 0
  training_iteration: 2
  trial_id: 1dde1_00005

== Status ==
Memory usage on this node: 7.4/240.1 GiB
Using AsyncHyperBand: num_stopped=3
Bracket: Iter 8.000: None | Iter 4.000: None | Iter 2.000: -1.8657031449794768 | Iter 1.000: -1.9921342937469482
Resources requested: 14/32 CPUs, 0/2 GPUs, 0.0/157.76 GiB heap, 0.0/49.41 GiB objects
Result logdir: /var/lib/jenkins/ray_results/DEFAULT
Number of trials: 10 (7 RUNNING, 3 TERMINATED)
+---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name          | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| DEFAULT_1dde1_00000 | TERMINATED |                 |            4 |  128 |    4 | 0.00648265  | 2.02588 |     0.2675 |                    1 |
| DEFAULT_1dde1_00001 | RUNNING    |                 |            2 |   16 |    4 | 0.000419419 |         |            |                      |
| DEFAULT_1dde1_00002 | TERMINATED |                 |           16 |   64 |  128 | 0.0486304   | 2.02543 |     0.2616 |                    1 |
| DEFAULT_1dde1_00003 | RUNNING    |                 |            2 |   32 |    4 | 0.0290972   |         |            |                      |
| DEFAULT_1dde1_00004 | RUNNING    | 172.17.0.2:1090 |           16 |   16 |  128 | 0.0422261   | 1.99913 |     0.2282 |                    3 |
| DEFAULT_1dde1_00005 | RUNNING    | 172.17.0.2:1108 |            8 |  256 |  128 | 0.000362061 | 1.61154 |     0.4157 |                    2 |
| DEFAULT_1dde1_00006 | RUNNING    | 172.17.0.2:1104 |            4 |  256 |   16 | 0.00696066  | 1.74645 |     0.3355 |                    1 |
| DEFAULT_1dde1_00007 | RUNNING    | 172.17.0.2:1085 |            4 |  128 |   32 | 0.00139994  | 1.46904 |     0.4737 |                    1 |
| DEFAULT_1dde1_00008 | TERMINATED |                 |            8 |    8 |  128 | 0.000178276 | 2.29409 |     0.1467 |                    1 |
| DEFAULT_1dde1_00009 | RUNNING    |                 |            2 |   32 |   32 | 0.000438826 |         |            |                      |
+---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


(pid=1122) [1, 16000] loss: 0.218
(pid=1097) [1, 16000] loss: 0.200
(pid=1112) [1, 16000] loss: 0.292
(pid=1090) [4,  2000] loss: 2.210
(pid=1085) [2,  4000] loss: 0.723
(pid=1122) [1, 18000] loss: 0.192
(pid=1104) [2,  4000] loss: 0.883
(pid=1097) [1, 18000] loss: 0.176
(pid=1112) [1, 18000] loss: 0.260
(pid=1108) [3,  2000] loss: 1.574
Result for DEFAULT_1dde1_00004:
  accuracy: 0.2056
  date: 2020-09-22_01-02-50
  done: false
  experiment_id: 0e63a81fef9141bdb0635e2bd7085e0e
  experiment_tag: 4_batch_size=16,l1=16,l2=128,lr=0.042226
  hostname: fed1025d76e5
  iterations_since_restore: 4
  loss: 2.0474100847244263
  node_ip: 172.17.0.2
  pid: 1090
  should_checkpoint: true
  time_since_restore: 93.15838694572449
  time_this_iter_s: 20.310168027877808
  time_total_s: 93.15838694572449
  timestamp: 1600736570
  timesteps_since_restore: 0
  training_iteration: 4
  trial_id: 1dde1_00004

== Status ==
Memory usage on this node: 7.4/240.1 GiB
Using AsyncHyperBand: num_stopped=3
Bracket: Iter 8.000: None | Iter 4.000: -2.0474100847244263 | Iter 2.000: -1.8657031449794768 | Iter 1.000: -1.9921342937469482
Resources requested: 14/32 CPUs, 0/2 GPUs, 0.0/157.76 GiB heap, 0.0/49.41 GiB objects
Result logdir: /var/lib/jenkins/ray_results/DEFAULT
Number of trials: 10 (7 RUNNING, 3 TERMINATED)
+---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name          | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| DEFAULT_1dde1_00000 | TERMINATED |                 |            4 |  128 |    4 | 0.00648265  | 2.02588 |     0.2675 |                    1 |
| DEFAULT_1dde1_00001 | RUNNING    |                 |            2 |   16 |    4 | 0.000419419 |         |            |                      |
| DEFAULT_1dde1_00002 | TERMINATED |                 |           16 |   64 |  128 | 0.0486304   | 2.02543 |     0.2616 |                    1 |
| DEFAULT_1dde1_00003 | RUNNING    |                 |            2 |   32 |    4 | 0.0290972   |         |            |                      |
| DEFAULT_1dde1_00004 | RUNNING    | 172.17.0.2:1090 |           16 |   16 |  128 | 0.0422261   | 2.04741 |     0.2056 |                    4 |
| DEFAULT_1dde1_00005 | RUNNING    | 172.17.0.2:1108 |            8 |  256 |  128 | 0.000362061 | 1.61154 |     0.4157 |                    2 |
| DEFAULT_1dde1_00006 | RUNNING    | 172.17.0.2:1104 |            4 |  256 |   16 | 0.00696066  | 1.74645 |     0.3355 |                    1 |
| DEFAULT_1dde1_00007 | RUNNING    | 172.17.0.2:1085 |            4 |  128 |   32 | 0.00139994  | 1.46904 |     0.4737 |                    1 |
| DEFAULT_1dde1_00008 | TERMINATED |                 |            8 |    8 |  128 | 0.000178276 | 2.29409 |     0.1467 |                    1 |
| DEFAULT_1dde1_00009 | RUNNING    |                 |            2 |   32 |   32 | 0.000438826 |         |            |                      |
+---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


(pid=1085) [2,  6000] loss: 0.467
(pid=1122) [1, 20000] loss: 0.171
(pid=1097) [1, 20000] loss: 0.154
(pid=1104) [2,  6000] loss: 0.581
(pid=1112) [1, 20000] loss: 0.233
(pid=1108) [3,  4000] loss: 0.757
(pid=1085) [2,  8000] loss: 0.349
(pid=1090) [5,  2000] loss: 2.049
(pid=1104) [2,  8000] loss: 0.437
Result for DEFAULT_1dde1_00001:
  accuracy: 0.3427
  date: 2020-09-22_01-03-08
  done: false
  experiment_id: f8daf9cd06ad4a4b9c3df3513effdf31
  experiment_tag: 1_batch_size=2,l1=16,l2=4,lr=0.00041942
  hostname: fed1025d76e5
  iterations_since_restore: 1
  loss: 1.690081286263466
  node_ip: 172.17.0.2
  pid: 1122
  should_checkpoint: true
  time_since_restore: 111.34616017341614
  time_this_iter_s: 111.34616017341614
  time_total_s: 111.34616017341614
  timestamp: 1600736588
  timesteps_since_restore: 0
  training_iteration: 1
  trial_id: 1dde1_00001

== Status ==
Memory usage on this node: 7.4/240.1 GiB
Using AsyncHyperBand: num_stopped=3
Bracket: Iter 8.000: None | Iter 4.000: -2.0474100847244263 | Iter 2.000: -1.8657031449794768 | Iter 1.000: -1.9341911580085753
Resources requested: 14/32 CPUs, 0/2 GPUs, 0.0/157.76 GiB heap, 0.0/49.41 GiB objects
Result logdir: /var/lib/jenkins/ray_results/DEFAULT
Number of trials: 10 (7 RUNNING, 3 TERMINATED)
+---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name          | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| DEFAULT_1dde1_00000 | TERMINATED |                 |            4 |  128 |    4 | 0.00648265  | 2.02588 |     0.2675 |                    1 |
| DEFAULT_1dde1_00001 | RUNNING    | 172.17.0.2:1122 |            2 |   16 |    4 | 0.000419419 | 1.69008 |     0.3427 |                    1 |
| DEFAULT_1dde1_00002 | TERMINATED |                 |           16 |   64 |  128 | 0.0486304   | 2.02543 |     0.2616 |                    1 |
| DEFAULT_1dde1_00003 | RUNNING    |                 |            2 |   32 |    4 | 0.0290972   |         |            |                      |
| DEFAULT_1dde1_00004 | RUNNING    | 172.17.0.2:1090 |           16 |   16 |  128 | 0.0422261   | 2.04741 |     0.2056 |                    4 |
| DEFAULT_1dde1_00005 | RUNNING    | 172.17.0.2:1108 |            8 |  256 |  128 | 0.000362061 | 1.61154 |     0.4157 |                    2 |
| DEFAULT_1dde1_00006 | RUNNING    | 172.17.0.2:1104 |            4 |  256 |   16 | 0.00696066  | 1.74645 |     0.3355 |                    1 |
| DEFAULT_1dde1_00007 | RUNNING    | 172.17.0.2:1085 |            4 |  128 |   32 | 0.00139994  | 1.46904 |     0.4737 |                    1 |
| DEFAULT_1dde1_00008 | TERMINATED |                 |            8 |    8 |  128 | 0.000178276 | 2.29409 |     0.1467 |                    1 |
| DEFAULT_1dde1_00009 | RUNNING    |                 |            2 |   32 |   32 | 0.000438826 |         |            |                      |
+---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


Result for DEFAULT_1dde1_00009:
  accuracy: 0.464
  date: 2020-09-22_01-03-09
  done: false
  experiment_id: cfed644d520f435688772253831a176f
  experiment_tag: 9_batch_size=2,l1=32,l2=32,lr=0.00043883
  hostname: fed1025d76e5
  iterations_since_restore: 1
  loss: 1.466922292420268
  node_ip: 172.17.0.2
  pid: 1097
  should_checkpoint: true
  time_since_restore: 112.3949773311615
  time_this_iter_s: 112.3949773311615
  time_total_s: 112.3949773311615
  timestamp: 1600736589
  timesteps_since_restore: 0
  training_iteration: 1
  trial_id: 1dde1_00009

Result for DEFAULT_1dde1_00004:
  accuracy: 0.208
  date: 2020-09-22_01-03-11
  done: false
  experiment_id: 0e63a81fef9141bdb0635e2bd7085e0e
  experiment_tag: 4_batch_size=16,l1=16,l2=128,lr=0.042226
  hostname: fed1025d76e5
  iterations_since_restore: 5
  loss: 2.071832389259338
  node_ip: 172.17.0.2
  pid: 1090
  should_checkpoint: true
  time_since_restore: 113.85181093215942
  time_this_iter_s: 20.693423986434937
  time_total_s: 113.85181093215942
  timestamp: 1600736591
  timesteps_since_restore: 0
  training_iteration: 5
  trial_id: 1dde1_00004

Result for DEFAULT_1dde1_00003:
  accuracy: 0.0979
  date: 2020-09-22_01-03-12
  done: true
  experiment_id: ab36e11e8e554eb1a9350acb371675f6
  experiment_tag: 3_batch_size=2,l1=32,l2=4,lr=0.029097
  hostname: fed1025d76e5
  iterations_since_restore: 1
  loss: 2.3260342679977417
  node_ip: 172.17.0.2
  pid: 1112
  should_checkpoint: true
  time_since_restore: 114.85109782218933
  time_this_iter_s: 114.85109782218933
  time_total_s: 114.85109782218933
  timestamp: 1600736592
  timesteps_since_restore: 0
  training_iteration: 1
  trial_id: 1dde1_00003

Result for DEFAULT_1dde1_00005:
  accuracy: 0.4589
  date: 2020-09-22_01-03-12
  done: false
  experiment_id: 6a1270e5c59840d8acaa81bba30ebb03
  experiment_tag: 5_batch_size=8,l1=256,l2=128,lr=0.00036206
  hostname: fed1025d76e5
  iterations_since_restore: 3
  loss: 1.4968988493680955
  node_ip: 172.17.0.2
  pid: 1108
  should_checkpoint: true
  time_since_restore: 115.53939604759216
  time_this_iter_s: 34.694926738739014
  time_total_s: 115.53939604759216
  timestamp: 1600736592
  timesteps_since_restore: 0
  training_iteration: 3
  trial_id: 1dde1_00005

(pid=1085) [2, 10000] loss: 0.274
(pid=1104) [2, 10000] loss: 0.353
(pid=1122) [2,  2000] loss: 1.672
(pid=1097) [2,  2000] loss: 1.524
Result for DEFAULT_1dde1_00007:
  accuracy: 0.5112
  date: 2020-09-22_01-03-20
  done: false
  experiment_id: d130aa97ad4d43d1a015b0a98a769c9b
  experiment_tag: 7_batch_size=4,l1=128,l2=32,lr=0.0013999
  hostname: fed1025d76e5
  iterations_since_restore: 2
  loss: 1.37119730758667
  node_ip: 172.17.0.2
  pid: 1085
  should_checkpoint: true
  time_since_restore: 123.12365698814392
  time_this_iter_s: 55.226486682891846
  time_total_s: 123.12365698814392
  timestamp: 1600736600
  timesteps_since_restore: 0
  training_iteration: 2
  trial_id: 1dde1_00007

== Status ==
Memory usage on this node: 6.9/240.1 GiB
Using AsyncHyperBand: num_stopped=4
Bracket: Iter 8.000: None | Iter 4.000: -2.0474100847244263 | Iter 2.000: -1.6115445574760436 | Iter 1.000: -1.9341911580085753
Resources requested: 12/32 CPUs, 0/2 GPUs, 0.0/157.76 GiB heap, 0.0/49.41 GiB objects
Result logdir: /var/lib/jenkins/ray_results/DEFAULT
Number of trials: 10 (6 RUNNING, 4 TERMINATED)
+---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name          | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| DEFAULT_1dde1_00000 | TERMINATED |                 |            4 |  128 |    4 | 0.00648265  | 2.02588 |     0.2675 |                    1 |
| DEFAULT_1dde1_00001 | RUNNING    | 172.17.0.2:1122 |            2 |   16 |    4 | 0.000419419 | 1.69008 |     0.3427 |                    1 |
| DEFAULT_1dde1_00002 | TERMINATED |                 |           16 |   64 |  128 | 0.0486304   | 2.02543 |     0.2616 |                    1 |
| DEFAULT_1dde1_00003 | TERMINATED |                 |            2 |   32 |    4 | 0.0290972   | 2.32603 |     0.0979 |                    1 |
| DEFAULT_1dde1_00004 | RUNNING    | 172.17.0.2:1090 |           16 |   16 |  128 | 0.0422261   | 2.07183 |     0.208  |                    5 |
| DEFAULT_1dde1_00005 | RUNNING    | 172.17.0.2:1108 |            8 |  256 |  128 | 0.000362061 | 1.4969  |     0.4589 |                    3 |
| DEFAULT_1dde1_00006 | RUNNING    | 172.17.0.2:1104 |            4 |  256 |   16 | 0.00696066  | 1.74645 |     0.3355 |                    1 |
| DEFAULT_1dde1_00007 | RUNNING    | 172.17.0.2:1085 |            4 |  128 |   32 | 0.00139994  | 1.3712  |     0.5112 |                    2 |
| DEFAULT_1dde1_00008 | TERMINATED |                 |            8 |    8 |  128 | 0.000178276 | 2.29409 |     0.1467 |                    1 |
| DEFAULT_1dde1_00009 | RUNNING    | 172.17.0.2:1097 |            2 |   32 |   32 | 0.000438826 | 1.46692 |     0.464  |                    1 |
+---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


Result for DEFAULT_1dde1_00006:
  accuracy: 0.3378
  date: 2020-09-22_01-03-23
  done: true
  experiment_id: c90e171237b14250992a8ac860d4bf3d
  experiment_tag: 6_batch_size=4,l1=256,l2=16,lr=0.0069607
  hostname: fed1025d76e5
  iterations_since_restore: 2
  loss: 1.7610837140202522
  node_ip: 172.17.0.2
  pid: 1104
  should_checkpoint: true
  time_since_restore: 126.36016178131104
  time_this_iter_s: 56.567606687545776
  time_total_s: 126.36016178131104
  timestamp: 1600736603
  timesteps_since_restore: 0
  training_iteration: 2
  trial_id: 1dde1_00006

(pid=1108) [4,  2000] loss: 1.424
(pid=1090) [6,  2000] loss: 2.023
(pid=1122) [2,  4000] loss: 0.823
(pid=1097) [2,  4000] loss: 0.736
(pid=1085) [3,  2000] loss: 1.296
Result for DEFAULT_1dde1_00004:
  accuracy: 0.2284
  date: 2020-09-22_01-03-31
  done: false
  experiment_id: 0e63a81fef9141bdb0635e2bd7085e0e
  experiment_tag: 4_batch_size=16,l1=16,l2=128,lr=0.042226
  hostname: fed1025d76e5
  iterations_since_restore: 6
  loss: 2.0067037870407103
  node_ip: 172.17.0.2
  pid: 1090
  should_checkpoint: true
  time_since_restore: 133.78824996948242
  time_this_iter_s: 19.936439037322998
  time_total_s: 133.78824996948242
  timestamp: 1600736611
  timesteps_since_restore: 0
  training_iteration: 6
  trial_id: 1dde1_00004

== Status ==
Memory usage on this node: 6.3/240.1 GiB
Using AsyncHyperBand: num_stopped=5
Bracket: Iter 8.000: None | Iter 4.000: -2.0474100847244263 | Iter 2.000: -1.686314135748148 | Iter 1.000: -1.9341911580085753
Resources requested: 10/32 CPUs, 0/2 GPUs, 0.0/157.76 GiB heap, 0.0/49.41 GiB objects
Result logdir: /var/lib/jenkins/ray_results/DEFAULT
Number of trials: 10 (5 RUNNING, 5 TERMINATED)
+---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name          | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| DEFAULT_1dde1_00000 | TERMINATED |                 |            4 |  128 |    4 | 0.00648265  | 2.02588 |     0.2675 |                    1 |
| DEFAULT_1dde1_00001 | RUNNING    | 172.17.0.2:1122 |            2 |   16 |    4 | 0.000419419 | 1.69008 |     0.3427 |                    1 |
| DEFAULT_1dde1_00002 | TERMINATED |                 |           16 |   64 |  128 | 0.0486304   | 2.02543 |     0.2616 |                    1 |
| DEFAULT_1dde1_00003 | TERMINATED |                 |            2 |   32 |    4 | 0.0290972   | 2.32603 |     0.0979 |                    1 |
| DEFAULT_1dde1_00004 | RUNNING    | 172.17.0.2:1090 |           16 |   16 |  128 | 0.0422261   | 2.0067  |     0.2284 |                    6 |
| DEFAULT_1dde1_00005 | RUNNING    | 172.17.0.2:1108 |            8 |  256 |  128 | 0.000362061 | 1.4969  |     0.4589 |                    3 |
| DEFAULT_1dde1_00006 | TERMINATED |                 |            4 |  256 |   16 | 0.00696066  | 1.76108 |     0.3378 |                    2 |
| DEFAULT_1dde1_00007 | RUNNING    | 172.17.0.2:1085 |            4 |  128 |   32 | 0.00139994  | 1.3712  |     0.5112 |                    2 |
| DEFAULT_1dde1_00008 | TERMINATED |                 |            8 |    8 |  128 | 0.000178276 | 2.29409 |     0.1467 |                    1 |
| DEFAULT_1dde1_00009 | RUNNING    | 172.17.0.2:1097 |            2 |   32 |   32 | 0.000438826 | 1.46692 |     0.464  |                    1 |
+---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


(pid=1122) [2,  6000] loss: 0.540
(pid=1097) [2,  6000] loss: 0.483
(pid=1108) [4,  4000] loss: 0.694
(pid=1085) [3,  4000] loss: 0.636
(pid=1122) [2,  8000] loss: 0.405
(pid=1097) [2,  8000] loss: 0.363
(pid=1090) [7,  2000] loss: 2.042
Result for DEFAULT_1dde1_00005:
  accuracy: 0.5079
  date: 2020-09-22_01-03-46
  done: false
  experiment_id: 6a1270e5c59840d8acaa81bba30ebb03
  experiment_tag: 5_batch_size=8,l1=256,l2=128,lr=0.00036206
  hostname: fed1025d76e5
  iterations_since_restore: 4
  loss: 1.3617212006807327
  node_ip: 172.17.0.2
  pid: 1108
  should_checkpoint: true
  time_since_restore: 148.9325716495514
  time_this_iter_s: 33.39317560195923
  time_total_s: 148.9325716495514
  timestamp: 1600736626
  timesteps_since_restore: 0
  training_iteration: 4
  trial_id: 1dde1_00005

== Status ==
Memory usage on this node: 6.3/240.1 GiB
Using AsyncHyperBand: num_stopped=5
Bracket: Iter 8.000: None | Iter 4.000: -1.7045656427025795 | Iter 2.000: -1.686314135748148 | Iter 1.000: -1.9341911580085753
Resources requested: 10/32 CPUs, 0/2 GPUs, 0.0/157.76 GiB heap, 0.0/49.41 GiB objects
Result logdir: /var/lib/jenkins/ray_results/DEFAULT
Number of trials: 10 (5 RUNNING, 5 TERMINATED)
+---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name          | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| DEFAULT_1dde1_00000 | TERMINATED |                 |            4 |  128 |    4 | 0.00648265  | 2.02588 |     0.2675 |                    1 |
| DEFAULT_1dde1_00001 | RUNNING    | 172.17.0.2:1122 |            2 |   16 |    4 | 0.000419419 | 1.69008 |     0.3427 |                    1 |
| DEFAULT_1dde1_00002 | TERMINATED |                 |           16 |   64 |  128 | 0.0486304   | 2.02543 |     0.2616 |                    1 |
| DEFAULT_1dde1_00003 | TERMINATED |                 |            2 |   32 |    4 | 0.0290972   | 2.32603 |     0.0979 |                    1 |
| DEFAULT_1dde1_00004 | RUNNING    | 172.17.0.2:1090 |           16 |   16 |  128 | 0.0422261   | 2.0067  |     0.2284 |                    6 |
| DEFAULT_1dde1_00005 | RUNNING    | 172.17.0.2:1108 |            8 |  256 |  128 | 0.000362061 | 1.36172 |     0.5079 |                    4 |
| DEFAULT_1dde1_00006 | TERMINATED |                 |            4 |  256 |   16 | 0.00696066  | 1.76108 |     0.3378 |                    2 |
| DEFAULT_1dde1_00007 | RUNNING    | 172.17.0.2:1085 |            4 |  128 |   32 | 0.00139994  | 1.3712  |     0.5112 |                    2 |
| DEFAULT_1dde1_00008 | TERMINATED |                 |            8 |    8 |  128 | 0.000178276 | 2.29409 |     0.1467 |                    1 |
| DEFAULT_1dde1_00009 | RUNNING    | 172.17.0.2:1097 |            2 |   32 |   32 | 0.000438826 | 1.46692 |     0.464  |                    1 |
+---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


(pid=1085) [3,  6000] loss: 0.425
(pid=1122) [2, 10000] loss: 0.323
(pid=1097) [2, 10000] loss: 0.288
Result for DEFAULT_1dde1_00004:
  accuracy: 0.1851
  date: 2020-09-22_01-03-50
  done: false
  experiment_id: 0e63a81fef9141bdb0635e2bd7085e0e
  experiment_tag: 4_batch_size=16,l1=16,l2=128,lr=0.042226
  hostname: fed1025d76e5
  iterations_since_restore: 7
  loss: 2.0709309223175048
  node_ip: 172.17.0.2
  pid: 1090
  should_checkpoint: true
  time_since_restore: 153.19460248947144
  time_this_iter_s: 19.406352519989014
  time_total_s: 153.19460248947144
  timestamp: 1600736630
  timesteps_since_restore: 0
  training_iteration: 7
  trial_id: 1dde1_00004

(pid=1085) [3,  8000] loss: 0.317
(pid=1122) [2, 12000] loss: 0.265
(pid=1108) [5,  2000] loss: 1.321
(pid=1097) [2, 12000] loss: 0.234
(pid=1090) [8,  2000] loss: 2.059
(pid=1122) [2, 14000] loss: 0.228
(pid=1085) [3, 10000] loss: 0.255
(pid=1097) [2, 14000] loss: 0.198
(pid=1108) [5,  4000] loss: 0.643
Result for DEFAULT_1dde1_00004:
  accuracy: 0.2073
  date: 2020-09-22_01-04-10
  done: false
  experiment_id: 0e63a81fef9141bdb0635e2bd7085e0e
  experiment_tag: 4_batch_size=16,l1=16,l2=128,lr=0.042226
  hostname: fed1025d76e5
  iterations_since_restore: 8
  loss: 2.0788925355911254
  node_ip: 172.17.0.2
  pid: 1090
  should_checkpoint: true
  time_since_restore: 172.85381960868835
  time_this_iter_s: 19.65921711921692
  time_total_s: 172.85381960868835
  timestamp: 1600736650
  timesteps_since_restore: 0
  training_iteration: 8
  trial_id: 1dde1_00004

== Status ==
Memory usage on this node: 6.3/240.1 GiB
Using AsyncHyperBand: num_stopped=5
Bracket: Iter 8.000: -2.0788925355911254 | Iter 4.000: -1.7045656427025795 | Iter 2.000: -1.686314135748148 | Iter 1.000: -1.9341911580085753
Resources requested: 10/32 CPUs, 0/2 GPUs, 0.0/157.76 GiB heap, 0.0/49.41 GiB objects
Result logdir: /var/lib/jenkins/ray_results/DEFAULT
Number of trials: 10 (5 RUNNING, 5 TERMINATED)
+---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name          | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| DEFAULT_1dde1_00000 | TERMINATED |                 |            4 |  128 |    4 | 0.00648265  | 2.02588 |     0.2675 |                    1 |
| DEFAULT_1dde1_00001 | RUNNING    | 172.17.0.2:1122 |            2 |   16 |    4 | 0.000419419 | 1.69008 |     0.3427 |                    1 |
| DEFAULT_1dde1_00002 | TERMINATED |                 |           16 |   64 |  128 | 0.0486304   | 2.02543 |     0.2616 |                    1 |
| DEFAULT_1dde1_00003 | TERMINATED |                 |            2 |   32 |    4 | 0.0290972   | 2.32603 |     0.0979 |                    1 |
| DEFAULT_1dde1_00004 | RUNNING    | 172.17.0.2:1090 |           16 |   16 |  128 | 0.0422261   | 2.07889 |     0.2073 |                    8 |
| DEFAULT_1dde1_00005 | RUNNING    | 172.17.0.2:1108 |            8 |  256 |  128 | 0.000362061 | 1.36172 |     0.5079 |                    4 |
| DEFAULT_1dde1_00006 | TERMINATED |                 |            4 |  256 |   16 | 0.00696066  | 1.76108 |     0.3378 |                    2 |
| DEFAULT_1dde1_00007 | RUNNING    | 172.17.0.2:1085 |            4 |  128 |   32 | 0.00139994  | 1.3712  |     0.5112 |                    2 |
| DEFAULT_1dde1_00008 | TERMINATED |                 |            8 |    8 |  128 | 0.000178276 | 2.29409 |     0.1467 |                    1 |
| DEFAULT_1dde1_00009 | RUNNING    | 172.17.0.2:1097 |            2 |   32 |   32 | 0.000438826 | 1.46692 |     0.464  |                    1 |
+---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


Result for DEFAULT_1dde1_00007:
  accuracy: 0.5556
  date: 2020-09-22_01-04-12
  done: false
  experiment_id: d130aa97ad4d43d1a015b0a98a769c9b
  experiment_tag: 7_batch_size=4,l1=128,l2=32,lr=0.0013999
  hostname: fed1025d76e5
  iterations_since_restore: 3
  loss: 1.2556879827991128
  node_ip: 172.17.0.2
  pid: 1085
  should_checkpoint: true
  time_since_restore: 174.86849665641785
  time_this_iter_s: 51.744839668273926
  time_total_s: 174.86849665641785
  timestamp: 1600736652
  timesteps_since_restore: 0
  training_iteration: 3
  trial_id: 1dde1_00007

(pid=1122) [2, 16000] loss: 0.195
(pid=1097) [2, 16000] loss: 0.173
Result for DEFAULT_1dde1_00005:
  accuracy: 0.5441
  date: 2020-09-22_01-04-19
  done: false
  experiment_id: 6a1270e5c59840d8acaa81bba30ebb03
  experiment_tag: 5_batch_size=8,l1=256,l2=128,lr=0.00036206
  hostname: fed1025d76e5
  iterations_since_restore: 5
  loss: 1.2748901859998703
  node_ip: 172.17.0.2
  pid: 1108
  should_checkpoint: true
  time_since_restore: 181.70323157310486
  time_this_iter_s: 32.77065992355347
  time_total_s: 181.70323157310486
  timestamp: 1600736659
  timesteps_since_restore: 0
  training_iteration: 5
  trial_id: 1dde1_00005

== Status ==
Memory usage on this node: 6.3/240.1 GiB
Using AsyncHyperBand: num_stopped=5
Bracket: Iter 8.000: -2.0788925355911254 | Iter 4.000: -1.7045656427025795 | Iter 2.000: -1.686314135748148 | Iter 1.000: -1.9341911580085753
Resources requested: 10/32 CPUs, 0/2 GPUs, 0.0/157.76 GiB heap, 0.0/49.41 GiB objects
Result logdir: /var/lib/jenkins/ray_results/DEFAULT
Number of trials: 10 (5 RUNNING, 5 TERMINATED)
+---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name          | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| DEFAULT_1dde1_00000 | TERMINATED |                 |            4 |  128 |    4 | 0.00648265  | 2.02588 |     0.2675 |                    1 |
| DEFAULT_1dde1_00001 | RUNNING    | 172.17.0.2:1122 |            2 |   16 |    4 | 0.000419419 | 1.69008 |     0.3427 |                    1 |
| DEFAULT_1dde1_00002 | TERMINATED |                 |           16 |   64 |  128 | 0.0486304   | 2.02543 |     0.2616 |                    1 |
| DEFAULT_1dde1_00003 | TERMINATED |                 |            2 |   32 |    4 | 0.0290972   | 2.32603 |     0.0979 |                    1 |
| DEFAULT_1dde1_00004 | RUNNING    | 172.17.0.2:1090 |           16 |   16 |  128 | 0.0422261   | 2.07889 |     0.2073 |                    8 |
| DEFAULT_1dde1_00005 | RUNNING    | 172.17.0.2:1108 |            8 |  256 |  128 | 0.000362061 | 1.27489 |     0.5441 |                    5 |
| DEFAULT_1dde1_00006 | TERMINATED |                 |            4 |  256 |   16 | 0.00696066  | 1.76108 |     0.3378 |                    2 |
| DEFAULT_1dde1_00007 | RUNNING    | 172.17.0.2:1085 |            4 |  128 |   32 | 0.00139994  | 1.25569 |     0.5556 |                    3 |
| DEFAULT_1dde1_00008 | TERMINATED |                 |            8 |    8 |  128 | 0.000178276 | 2.29409 |     0.1467 |                    1 |
| DEFAULT_1dde1_00009 | RUNNING    | 172.17.0.2:1097 |            2 |   32 |   32 | 0.000438826 | 1.46692 |     0.464  |                    1 |
+---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


(pid=1122) [2, 18000] loss: 0.175
(pid=1085) [4,  2000] loss: 1.175
(pid=1097) [2, 18000] loss: 0.152
(pid=1090) [9,  2000] loss: 2.048
(pid=1122) [2, 20000] loss: 0.154
Result for DEFAULT_1dde1_00004:
  accuracy: 0.2331
  date: 2020-09-22_01-04-29
  done: false
  experiment_id: 0e63a81fef9141bdb0635e2bd7085e0e
  experiment_tag: 4_batch_size=16,l1=16,l2=128,lr=0.042226
  hostname: fed1025d76e5
  iterations_since_restore: 9
  loss: 2.026749889564514
  node_ip: 172.17.0.2
  pid: 1090
  should_checkpoint: true
  time_since_restore: 192.57573747634888
  time_this_iter_s: 19.721917867660522
  time_total_s: 192.57573747634888
  timestamp: 1600736669
  timesteps_since_restore: 0
  training_iteration: 9
  trial_id: 1dde1_00004

== Status ==
Memory usage on this node: 6.3/240.1 GiB
Using AsyncHyperBand: num_stopped=5
Bracket: Iter 8.000: -2.0788925355911254 | Iter 4.000: -1.7045656427025795 | Iter 2.000: -1.686314135748148 | Iter 1.000: -1.9341911580085753
Resources requested: 10/32 CPUs, 0/2 GPUs, 0.0/157.76 GiB heap, 0.0/49.41 GiB objects
Result logdir: /var/lib/jenkins/ray_results/DEFAULT
Number of trials: 10 (5 RUNNING, 5 TERMINATED)
+---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name          | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| DEFAULT_1dde1_00000 | TERMINATED |                 |            4 |  128 |    4 | 0.00648265  | 2.02588 |     0.2675 |                    1 |
| DEFAULT_1dde1_00001 | RUNNING    | 172.17.0.2:1122 |            2 |   16 |    4 | 0.000419419 | 1.69008 |     0.3427 |                    1 |
| DEFAULT_1dde1_00002 | TERMINATED |                 |           16 |   64 |  128 | 0.0486304   | 2.02543 |     0.2616 |                    1 |
| DEFAULT_1dde1_00003 | TERMINATED |                 |            2 |   32 |    4 | 0.0290972   | 2.32603 |     0.0979 |                    1 |
| DEFAULT_1dde1_00004 | RUNNING    | 172.17.0.2:1090 |           16 |   16 |  128 | 0.0422261   | 2.02675 |     0.2331 |                    9 |
| DEFAULT_1dde1_00005 | RUNNING    | 172.17.0.2:1108 |            8 |  256 |  128 | 0.000362061 | 1.27489 |     0.5441 |                    5 |
| DEFAULT_1dde1_00006 | TERMINATED |                 |            4 |  256 |   16 | 0.00696066  | 1.76108 |     0.3378 |                    2 |
| DEFAULT_1dde1_00007 | RUNNING    | 172.17.0.2:1085 |            4 |  128 |   32 | 0.00139994  | 1.25569 |     0.5556 |                    3 |
| DEFAULT_1dde1_00008 | TERMINATED |                 |            8 |    8 |  128 | 0.000178276 | 2.29409 |     0.1467 |                    1 |
| DEFAULT_1dde1_00009 | RUNNING    | 172.17.0.2:1097 |            2 |   32 |   32 | 0.000438826 | 1.46692 |     0.464  |                    1 |
+---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


(pid=1085) [4,  4000] loss: 0.593
(pid=1097) [2, 20000] loss: 0.136
(pid=1108) [6,  2000] loss: 1.229
(pid=1085) [4,  6000] loss: 0.394
Result for DEFAULT_1dde1_00001:
  accuracy: 0.4297
  date: 2020-09-22_01-04-41
  done: false
  experiment_id: f8daf9cd06ad4a4b9c3df3513effdf31
  experiment_tag: 1_batch_size=2,l1=16,l2=4,lr=0.00041942
  hostname: fed1025d76e5
  iterations_since_restore: 2
  loss: 1.5426445162922144
  node_ip: 172.17.0.2
  pid: 1122
  should_checkpoint: true
  time_since_restore: 204.10034322738647
  time_this_iter_s: 92.75418305397034
  time_total_s: 204.10034322738647
  timestamp: 1600736681
  timesteps_since_restore: 0
  training_iteration: 2
  trial_id: 1dde1_00001

== Status ==
Memory usage on this node: 6.3/240.1 GiB
Using AsyncHyperBand: num_stopped=5
Bracket: Iter 8.000: -2.0788925355911254 | Iter 4.000: -1.7045656427025795 | Iter 2.000: -1.6115445574760436 | Iter 1.000: -1.9341911580085753
Resources requested: 10/32 CPUs, 0/2 GPUs, 0.0/157.76 GiB heap, 0.0/49.41 GiB objects
Result logdir: /var/lib/jenkins/ray_results/DEFAULT
Number of trials: 10 (5 RUNNING, 5 TERMINATED)
+---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name          | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| DEFAULT_1dde1_00000 | TERMINATED |                 |            4 |  128 |    4 | 0.00648265  | 2.02588 |     0.2675 |                    1 |
| DEFAULT_1dde1_00001 | RUNNING    | 172.17.0.2:1122 |            2 |   16 |    4 | 0.000419419 | 1.54264 |     0.4297 |                    2 |
| DEFAULT_1dde1_00002 | TERMINATED |                 |           16 |   64 |  128 | 0.0486304   | 2.02543 |     0.2616 |                    1 |
| DEFAULT_1dde1_00003 | TERMINATED |                 |            2 |   32 |    4 | 0.0290972   | 2.32603 |     0.0979 |                    1 |
| DEFAULT_1dde1_00004 | RUNNING    | 172.17.0.2:1090 |           16 |   16 |  128 | 0.0422261   | 2.02675 |     0.2331 |                    9 |
| DEFAULT_1dde1_00005 | RUNNING    | 172.17.0.2:1108 |            8 |  256 |  128 | 0.000362061 | 1.27489 |     0.5441 |                    5 |
| DEFAULT_1dde1_00006 | TERMINATED |                 |            4 |  256 |   16 | 0.00696066  | 1.76108 |     0.3378 |                    2 |
| DEFAULT_1dde1_00007 | RUNNING    | 172.17.0.2:1085 |            4 |  128 |   32 | 0.00139994  | 1.25569 |     0.5556 |                    3 |
| DEFAULT_1dde1_00008 | TERMINATED |                 |            8 |    8 |  128 | 0.000178276 | 2.29409 |     0.1467 |                    1 |
| DEFAULT_1dde1_00009 | RUNNING    | 172.17.0.2:1097 |            2 |   32 |   32 | 0.000438826 | 1.46692 |     0.464  |                    1 |
+---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


(pid=1108) [6,  4000] loss: 0.612
Result for DEFAULT_1dde1_00009:
  accuracy: 0.4776
  date: 2020-09-22_01-04-42
  done: false
  experiment_id: cfed644d520f435688772253831a176f
  experiment_tag: 9_batch_size=2,l1=32,l2=32,lr=0.00043883
  hostname: fed1025d76e5
  iterations_since_restore: 2
  loss: 1.4519525066554546
  node_ip: 172.17.0.2
  pid: 1097
  should_checkpoint: true
  time_since_restore: 205.36001992225647
  time_this_iter_s: 92.96504259109497
  time_total_s: 205.36001992225647
  timestamp: 1600736682
  timesteps_since_restore: 0
  training_iteration: 2
  trial_id: 1dde1_00009

(pid=1090) [10,  2000] loss: 2.016
(pid=1085) [4,  8000] loss: 0.304
(pid=1122) [3,  2000] loss: 1.510
Result for DEFAULT_1dde1_00004:
  accuracy: 0.1939
  date: 2020-09-22_01-04-49
  done: true
  experiment_id: 0e63a81fef9141bdb0635e2bd7085e0e
  experiment_tag: 4_batch_size=16,l1=16,l2=128,lr=0.042226
  hostname: fed1025d76e5
  iterations_since_restore: 10
  loss: 2.030654595375061
  node_ip: 172.17.0.2
  pid: 1090
  should_checkpoint: true
  time_since_restore: 212.41506910324097
  time_this_iter_s: 19.83933162689209
  time_total_s: 212.41506910324097
  timestamp: 1600736689
  timesteps_since_restore: 0
  training_iteration: 10
  trial_id: 1dde1_00004

== Status ==
Memory usage on this node: 6.3/240.1 GiB
Using AsyncHyperBand: num_stopped=6
Bracket: Iter 8.000: -2.0788925355911254 | Iter 4.000: -1.7045656427025795 | Iter 2.000: -1.577094536884129 | Iter 1.000: -1.9341911580085753
Resources requested: 10/32 CPUs, 0/2 GPUs, 0.0/157.76 GiB heap, 0.0/49.41 GiB objects
Result logdir: /var/lib/jenkins/ray_results/DEFAULT
Number of trials: 10 (5 RUNNING, 5 TERMINATED)
+---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name          | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| DEFAULT_1dde1_00000 | TERMINATED |                 |            4 |  128 |    4 | 0.00648265  | 2.02588 |     0.2675 |                    1 |
| DEFAULT_1dde1_00001 | RUNNING    | 172.17.0.2:1122 |            2 |   16 |    4 | 0.000419419 | 1.54264 |     0.4297 |                    2 |
| DEFAULT_1dde1_00002 | TERMINATED |                 |           16 |   64 |  128 | 0.0486304   | 2.02543 |     0.2616 |                    1 |
| DEFAULT_1dde1_00003 | TERMINATED |                 |            2 |   32 |    4 | 0.0290972   | 2.32603 |     0.0979 |                    1 |
| DEFAULT_1dde1_00004 | RUNNING    | 172.17.0.2:1090 |           16 |   16 |  128 | 0.0422261   | 2.03065 |     0.1939 |                   10 |
| DEFAULT_1dde1_00005 | RUNNING    | 172.17.0.2:1108 |            8 |  256 |  128 | 0.000362061 | 1.27489 |     0.5441 |                    5 |
| DEFAULT_1dde1_00006 | TERMINATED |                 |            4 |  256 |   16 | 0.00696066  | 1.76108 |     0.3378 |                    2 |
| DEFAULT_1dde1_00007 | RUNNING    | 172.17.0.2:1085 |            4 |  128 |   32 | 0.00139994  | 1.25569 |     0.5556 |                    3 |
| DEFAULT_1dde1_00008 | TERMINATED |                 |            8 |    8 |  128 | 0.000178276 | 2.29409 |     0.1467 |                    1 |
| DEFAULT_1dde1_00009 | RUNNING    | 172.17.0.2:1097 |            2 |   32 |   32 | 0.000438826 | 1.45195 |     0.4776 |                    2 |
+---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


(pid=1097) [3,  2000] loss: 1.312
Result for DEFAULT_1dde1_00005:
  accuracy: 0.5663
  date: 2020-09-22_01-04-51
  done: false
  experiment_id: 6a1270e5c59840d8acaa81bba30ebb03
  experiment_tag: 5_batch_size=8,l1=256,l2=128,lr=0.00036206
  hostname: fed1025d76e5
  iterations_since_restore: 6
  loss: 1.223778771853447
  node_ip: 172.17.0.2
  pid: 1108
  should_checkpoint: true
  time_since_restore: 214.51789498329163
  time_this_iter_s: 32.81466341018677
  time_total_s: 214.51789498329163
  timestamp: 1600736691
  timesteps_since_restore: 0
  training_iteration: 6
  trial_id: 1dde1_00005

(pid=1122) [3,  4000] loss: 0.750
(pid=1085) [4, 10000] loss: 0.233
(pid=1097) [3,  4000] loss: 0.655
(pid=1108) [7,  2000] loss: 1.171
Result for DEFAULT_1dde1_00007:
  accuracy: 0.5539
  date: 2020-09-22_01-05-03
  done: false
  experiment_id: d130aa97ad4d43d1a015b0a98a769c9b
  experiment_tag: 7_batch_size=4,l1=128,l2=32,lr=0.0013999
  hostname: fed1025d76e5
  iterations_since_restore: 4
  loss: 1.267463112605363
  node_ip: 172.17.0.2
  pid: 1085
  should_checkpoint: true
  time_since_restore: 226.45621824264526
  time_this_iter_s: 51.58772158622742
  time_total_s: 226.45621824264526
  timestamp: 1600736703
  timesteps_since_restore: 0
  training_iteration: 4
  trial_id: 1dde1_00007

== Status ==
Memory usage on this node: 5.8/240.1 GiB
Using AsyncHyperBand: num_stopped=6
Bracket: Iter 8.000: -2.0788925355911254 | Iter 4.000: -1.3617212006807327 | Iter 2.000: -1.577094536884129 | Iter 1.000: -1.9341911580085753
Resources requested: 8/32 CPUs, 0/2 GPUs, 0.0/157.76 GiB heap, 0.0/49.41 GiB objects
Result logdir: /var/lib/jenkins/ray_results/DEFAULT
Number of trials: 10 (4 RUNNING, 6 TERMINATED)
+---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name          | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| DEFAULT_1dde1_00000 | TERMINATED |                 |            4 |  128 |    4 | 0.00648265  | 2.02588 |     0.2675 |                    1 |
| DEFAULT_1dde1_00001 | RUNNING    | 172.17.0.2:1122 |            2 |   16 |    4 | 0.000419419 | 1.54264 |     0.4297 |                    2 |
| DEFAULT_1dde1_00002 | TERMINATED |                 |           16 |   64 |  128 | 0.0486304   | 2.02543 |     0.2616 |                    1 |
| DEFAULT_1dde1_00003 | TERMINATED |                 |            2 |   32 |    4 | 0.0290972   | 2.32603 |     0.0979 |                    1 |
| DEFAULT_1dde1_00004 | TERMINATED |                 |           16 |   16 |  128 | 0.0422261   | 2.03065 |     0.1939 |                   10 |
| DEFAULT_1dde1_00005 | RUNNING    | 172.17.0.2:1108 |            8 |  256 |  128 | 0.000362061 | 1.22378 |     0.5663 |                    6 |
| DEFAULT_1dde1_00006 | TERMINATED |                 |            4 |  256 |   16 | 0.00696066  | 1.76108 |     0.3378 |                    2 |
| DEFAULT_1dde1_00007 | RUNNING    | 172.17.0.2:1085 |            4 |  128 |   32 | 0.00139994  | 1.26746 |     0.5539 |                    4 |
| DEFAULT_1dde1_00008 | TERMINATED |                 |            8 |    8 |  128 | 0.000178276 | 2.29409 |     0.1467 |                    1 |
| DEFAULT_1dde1_00009 | RUNNING    | 172.17.0.2:1097 |            2 |   32 |   32 | 0.000438826 | 1.45195 |     0.4776 |                    2 |
+---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


(pid=1122) [3,  6000] loss: 0.496
(pid=1097) [3,  6000] loss: 0.441
(pid=1122) [3,  8000] loss: 0.375
(pid=1085) [5,  2000] loss: 1.102
(pid=1108) [7,  4000] loss: 0.578
(pid=1097) [3,  8000] loss: 0.331
(pid=1122) [3, 10000] loss: 0.297
(pid=1085) [5,  4000] loss: 0.554
(pid=1097) [3, 10000] loss: 0.263
Result for DEFAULT_1dde1_00005:
  accuracy: 0.569
  date: 2020-09-22_01-05-23
  done: false
  experiment_id: 6a1270e5c59840d8acaa81bba30ebb03
  experiment_tag: 5_batch_size=8,l1=256,l2=128,lr=0.00036206
  hostname: fed1025d76e5
  iterations_since_restore: 7
  loss: 1.2081069868803025
  node_ip: 172.17.0.2
  pid: 1108
  should_checkpoint: true
  time_since_restore: 246.3141324520111
  time_this_iter_s: 31.796237468719482
  time_total_s: 246.3141324520111
  timestamp: 1600736723
  timesteps_since_restore: 0
  training_iteration: 7
  trial_id: 1dde1_00005

== Status ==
Memory usage on this node: 5.7/240.1 GiB
Using AsyncHyperBand: num_stopped=6
Bracket: Iter 8.000: -2.0788925355911254 | Iter 4.000: -1.3617212006807327 | Iter 2.000: -1.577094536884129 | Iter 1.000: -1.9341911580085753
Resources requested: 8/32 CPUs, 0/2 GPUs, 0.0/157.76 GiB heap, 0.0/49.41 GiB objects
Result logdir: /var/lib/jenkins/ray_results/DEFAULT
Number of trials: 10 (4 RUNNING, 6 TERMINATED)
+---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name          | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| DEFAULT_1dde1_00000 | TERMINATED |                 |            4 |  128 |    4 | 0.00648265  | 2.02588 |     0.2675 |                    1 |
| DEFAULT_1dde1_00001 | RUNNING    | 172.17.0.2:1122 |            2 |   16 |    4 | 0.000419419 | 1.54264 |     0.4297 |                    2 |
| DEFAULT_1dde1_00002 | TERMINATED |                 |           16 |   64 |  128 | 0.0486304   | 2.02543 |     0.2616 |                    1 |
| DEFAULT_1dde1_00003 | TERMINATED |                 |            2 |   32 |    4 | 0.0290972   | 2.32603 |     0.0979 |                    1 |
| DEFAULT_1dde1_00004 | TERMINATED |                 |           16 |   16 |  128 | 0.0422261   | 2.03065 |     0.1939 |                   10 |
| DEFAULT_1dde1_00005 | RUNNING    | 172.17.0.2:1108 |            8 |  256 |  128 | 0.000362061 | 1.20811 |     0.569  |                    7 |
| DEFAULT_1dde1_00006 | TERMINATED |                 |            4 |  256 |   16 | 0.00696066  | 1.76108 |     0.3378 |                    2 |
| DEFAULT_1dde1_00007 | RUNNING    | 172.17.0.2:1085 |            4 |  128 |   32 | 0.00139994  | 1.26746 |     0.5539 |                    4 |
| DEFAULT_1dde1_00008 | TERMINATED |                 |            8 |    8 |  128 | 0.000178276 | 2.29409 |     0.1467 |                    1 |
| DEFAULT_1dde1_00009 | RUNNING    | 172.17.0.2:1097 |            2 |   32 |   32 | 0.000438826 | 1.45195 |     0.4776 |                    2 |
+---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


(pid=1122) [3, 12000] loss: 0.249
(pid=1097) [3, 12000] loss: 0.217
(pid=1085) [5,  6000] loss: 0.377
(pid=1108) [8,  2000] loss: 1.100
(pid=1122) [3, 14000] loss: 0.210
(pid=1097) [3, 14000] loss: 0.183
(pid=1085) [5,  8000] loss: 0.280
(pid=1122) [3, 16000] loss: 0.183
(pid=1097) [3, 16000] loss: 0.159
(pid=1108) [8,  4000] loss: 0.556
(pid=1085) [5, 10000] loss: 0.226
(pid=1122) [3, 18000] loss: 0.165
(pid=1097) [3, 18000] loss: 0.140
Result for DEFAULT_1dde1_00007:
  accuracy: 0.5792
  date: 2020-09-22_01-05-54
  done: false
  experiment_id: d130aa97ad4d43d1a015b0a98a769c9b
  experiment_tag: 7_batch_size=4,l1=128,l2=32,lr=0.0013999
  hostname: fed1025d76e5
  iterations_since_restore: 5
  loss: 1.2025769818842411
  node_ip: 172.17.0.2
  pid: 1085
  should_checkpoint: true
  time_since_restore: 276.8607568740845
  time_this_iter_s: 50.40453863143921
  time_total_s: 276.8607568740845
  timestamp: 1600736754
  timesteps_since_restore: 0
  training_iteration: 5
  trial_id: 1dde1_00007

== Status ==
Memory usage on this node: 5.7/240.1 GiB
Using AsyncHyperBand: num_stopped=6
Bracket: Iter 8.000: -2.0788925355911254 | Iter 4.000: -1.3617212006807327 | Iter 2.000: -1.577094536884129 | Iter 1.000: -1.9341911580085753
Resources requested: 8/32 CPUs, 0/2 GPUs, 0.0/157.76 GiB heap, 0.0/49.41 GiB objects
Result logdir: /var/lib/jenkins/ray_results/DEFAULT
Number of trials: 10 (4 RUNNING, 6 TERMINATED)
+---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name          | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| DEFAULT_1dde1_00000 | TERMINATED |                 |            4 |  128 |    4 | 0.00648265  | 2.02588 |     0.2675 |                    1 |
| DEFAULT_1dde1_00001 | RUNNING    | 172.17.0.2:1122 |            2 |   16 |    4 | 0.000419419 | 1.54264 |     0.4297 |                    2 |
| DEFAULT_1dde1_00002 | TERMINATED |                 |           16 |   64 |  128 | 0.0486304   | 2.02543 |     0.2616 |                    1 |
| DEFAULT_1dde1_00003 | TERMINATED |                 |            2 |   32 |    4 | 0.0290972   | 2.32603 |     0.0979 |                    1 |
| DEFAULT_1dde1_00004 | TERMINATED |                 |           16 |   16 |  128 | 0.0422261   | 2.03065 |     0.1939 |                   10 |
| DEFAULT_1dde1_00005 | RUNNING    | 172.17.0.2:1108 |            8 |  256 |  128 | 0.000362061 | 1.20811 |     0.569  |                    7 |
| DEFAULT_1dde1_00006 | TERMINATED |                 |            4 |  256 |   16 | 0.00696066  | 1.76108 |     0.3378 |                    2 |
| DEFAULT_1dde1_00007 | RUNNING    | 172.17.0.2:1085 |            4 |  128 |   32 | 0.00139994  | 1.20258 |     0.5792 |                    5 |
| DEFAULT_1dde1_00008 | TERMINATED |                 |            8 |    8 |  128 | 0.000178276 | 2.29409 |     0.1467 |                    1 |
| DEFAULT_1dde1_00009 | RUNNING    | 172.17.0.2:1097 |            2 |   32 |   32 | 0.000438826 | 1.45195 |     0.4776 |                    2 |
+---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


Result for DEFAULT_1dde1_00005:
  accuracy: 0.5998
  date: 2020-09-22_01-05-55
  done: false
  experiment_id: 6a1270e5c59840d8acaa81bba30ebb03
  experiment_tag: 5_batch_size=8,l1=256,l2=128,lr=0.00036206
  hostname: fed1025d76e5
  iterations_since_restore: 8
  loss: 1.133019412434101
  node_ip: 172.17.0.2
  pid: 1108
  should_checkpoint: true
  time_since_restore: 277.9527311325073
  time_this_iter_s: 31.638598680496216
  time_total_s: 277.9527311325073
  timestamp: 1600736755
  timesteps_since_restore: 0
  training_iteration: 8
  trial_id: 1dde1_00005

(pid=1122) [3, 20000] loss: 0.146
(pid=1097) [3, 20000] loss: 0.128
(pid=1085) [6,  2000] loss: 1.014
(pid=1108) [9,  2000] loss: 1.057
Result for DEFAULT_1dde1_00001:
  accuracy: 0.4713
  date: 2020-09-22_01-06-10
  done: false
  experiment_id: f8daf9cd06ad4a4b9c3df3513effdf31
  experiment_tag: 1_batch_size=2,l1=16,l2=4,lr=0.00041942
  hostname: fed1025d76e5
  iterations_since_restore: 3
  loss: 1.4906232760388405
  node_ip: 172.17.0.2
  pid: 1122
  should_checkpoint: true
  time_since_restore: 293.61701011657715
  time_this_iter_s: 89.51666688919067
  time_total_s: 293.61701011657715
  timestamp: 1600736770
  timesteps_since_restore: 0
  training_iteration: 3
  trial_id: 1dde1_00001

== Status ==
Memory usage on this node: 5.8/240.1 GiB
Using AsyncHyperBand: num_stopped=6
Bracket: Iter 8.000: -1.6059559740126133 | Iter 4.000: -1.3617212006807327 | Iter 2.000: -1.577094536884129 | Iter 1.000: -1.9341911580085753
Resources requested: 8/32 CPUs, 0/2 GPUs, 0.0/157.76 GiB heap, 0.0/49.41 GiB objects
Result logdir: /var/lib/jenkins/ray_results/DEFAULT
Number of trials: 10 (4 RUNNING, 6 TERMINATED)
+---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name          | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| DEFAULT_1dde1_00000 | TERMINATED |                 |            4 |  128 |    4 | 0.00648265  | 2.02588 |     0.2675 |                    1 |
| DEFAULT_1dde1_00001 | RUNNING    | 172.17.0.2:1122 |            2 |   16 |    4 | 0.000419419 | 1.49062 |     0.4713 |                    3 |
| DEFAULT_1dde1_00002 | TERMINATED |                 |           16 |   64 |  128 | 0.0486304   | 2.02543 |     0.2616 |                    1 |
| DEFAULT_1dde1_00003 | TERMINATED |                 |            2 |   32 |    4 | 0.0290972   | 2.32603 |     0.0979 |                    1 |
| DEFAULT_1dde1_00004 | TERMINATED |                 |           16 |   16 |  128 | 0.0422261   | 2.03065 |     0.1939 |                   10 |
| DEFAULT_1dde1_00005 | RUNNING    | 172.17.0.2:1108 |            8 |  256 |  128 | 0.000362061 | 1.13302 |     0.5998 |                    8 |
| DEFAULT_1dde1_00006 | TERMINATED |                 |            4 |  256 |   16 | 0.00696066  | 1.76108 |     0.3378 |                    2 |
| DEFAULT_1dde1_00007 | RUNNING    | 172.17.0.2:1085 |            4 |  128 |   32 | 0.00139994  | 1.20258 |     0.5792 |                    5 |
| DEFAULT_1dde1_00008 | TERMINATED |                 |            8 |    8 |  128 | 0.000178276 | 2.29409 |     0.1467 |                    1 |
| DEFAULT_1dde1_00009 | RUNNING    | 172.17.0.2:1097 |            2 |   32 |   32 | 0.000438826 | 1.45195 |     0.4776 |                    2 |
+---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


(pid=1085) [6,  4000] loss: 0.529
Result for DEFAULT_1dde1_00009:
  accuracy: 0.5465
  date: 2020-09-22_01-06-12
  done: false
  experiment_id: cfed644d520f435688772253831a176f
  experiment_tag: 9_batch_size=2,l1=32,l2=32,lr=0.00043883
  hostname: fed1025d76e5
  iterations_since_restore: 3
  loss: 1.2607018883639947
  node_ip: 172.17.0.2
  pid: 1097
  should_checkpoint: true
  time_since_restore: 295.5224437713623
  time_this_iter_s: 90.16242384910583
  time_total_s: 295.5224437713623
  timestamp: 1600736772
  timesteps_since_restore: 0
  training_iteration: 3
  trial_id: 1dde1_00009

(pid=1108) [9,  4000] loss: 0.527
(pid=1122) [4,  2000] loss: 1.435
(pid=1097) [4,  2000] loss: 1.238
(pid=1085) [6,  6000] loss: 0.357
(pid=1122) [4,  4000] loss: 0.704
Result for DEFAULT_1dde1_00005:
  accuracy: 0.603
  date: 2020-09-22_01-06-27
  done: false
  experiment_id: 6a1270e5c59840d8acaa81bba30ebb03
  experiment_tag: 5_batch_size=8,l1=256,l2=128,lr=0.00036206
  hostname: fed1025d76e5
  iterations_since_restore: 9
  loss: 1.121426535463333
  node_ip: 172.17.0.2
  pid: 1108
  should_checkpoint: true
  time_since_restore: 309.83672976493835
  time_this_iter_s: 31.88399863243103
  time_total_s: 309.83672976493835
  timestamp: 1600736787
  timesteps_since_restore: 0
  training_iteration: 9
  trial_id: 1dde1_00005

== Status ==
Memory usage on this node: 5.7/240.1 GiB
Using AsyncHyperBand: num_stopped=6
Bracket: Iter 8.000: -1.6059559740126133 | Iter 4.000: -1.3617212006807327 | Iter 2.000: -1.577094536884129 | Iter 1.000: -1.9341911580085753
Resources requested: 8/32 CPUs, 0/2 GPUs, 0.0/157.76 GiB heap, 0.0/49.41 GiB objects
Result logdir: /var/lib/jenkins/ray_results/DEFAULT
Number of trials: 10 (4 RUNNING, 6 TERMINATED)
+---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name          | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| DEFAULT_1dde1_00000 | TERMINATED |                 |            4 |  128 |    4 | 0.00648265  | 2.02588 |     0.2675 |                    1 |
| DEFAULT_1dde1_00001 | RUNNING    | 172.17.0.2:1122 |            2 |   16 |    4 | 0.000419419 | 1.49062 |     0.4713 |                    3 |
| DEFAULT_1dde1_00002 | TERMINATED |                 |           16 |   64 |  128 | 0.0486304   | 2.02543 |     0.2616 |                    1 |
| DEFAULT_1dde1_00003 | TERMINATED |                 |            2 |   32 |    4 | 0.0290972   | 2.32603 |     0.0979 |                    1 |
| DEFAULT_1dde1_00004 | TERMINATED |                 |           16 |   16 |  128 | 0.0422261   | 2.03065 |     0.1939 |                   10 |
| DEFAULT_1dde1_00005 | RUNNING    | 172.17.0.2:1108 |            8 |  256 |  128 | 0.000362061 | 1.12143 |     0.603  |                    9 |
| DEFAULT_1dde1_00006 | TERMINATED |                 |            4 |  256 |   16 | 0.00696066  | 1.76108 |     0.3378 |                    2 |
| DEFAULT_1dde1_00007 | RUNNING    | 172.17.0.2:1085 |            4 |  128 |   32 | 0.00139994  | 1.20258 |     0.5792 |                    5 |
| DEFAULT_1dde1_00008 | TERMINATED |                 |            8 |    8 |  128 | 0.000178276 | 2.29409 |     0.1467 |                    1 |
| DEFAULT_1dde1_00009 | RUNNING    | 172.17.0.2:1097 |            2 |   32 |   32 | 0.000438826 | 1.2607  |     0.5465 |                    3 |
+---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


(pid=1097) [4,  4000] loss: 0.595
(pid=1085) [6,  8000] loss: 0.268
(pid=1122) [4,  6000] loss: 0.466
(pid=1097) [4,  6000] loss: 0.413
(pid=1108) [10,  2000] loss: 0.999
(pid=1085) [6, 10000] loss: 0.218
(pid=1122) [4,  8000] loss: 0.359
(pid=1097) [4,  8000] loss: 0.301
Result for DEFAULT_1dde1_00007:
  accuracy: 0.5726
  date: 2020-09-22_01-06-45
  done: false
  experiment_id: d130aa97ad4d43d1a015b0a98a769c9b
  experiment_tag: 7_batch_size=4,l1=128,l2=32,lr=0.0013999
  hostname: fed1025d76e5
  iterations_since_restore: 6
  loss: 1.2504841807976366
  node_ip: 172.17.0.2
  pid: 1085
  should_checkpoint: true
  time_since_restore: 327.60362005233765
  time_this_iter_s: 50.742863178253174
  time_total_s: 327.60362005233765
  timestamp: 1600736805
  timesteps_since_restore: 0
  training_iteration: 6
  trial_id: 1dde1_00007

== Status ==
Memory usage on this node: 5.8/240.1 GiB
Using AsyncHyperBand: num_stopped=6
Bracket: Iter 8.000: -1.6059559740126133 | Iter 4.000: -1.3617212006807327 | Iter 2.000: -1.577094536884129 | Iter 1.000: -1.9341911580085753
Resources requested: 8/32 CPUs, 0/2 GPUs, 0.0/157.76 GiB heap, 0.0/49.41 GiB objects
Result logdir: /var/lib/jenkins/ray_results/DEFAULT
Number of trials: 10 (4 RUNNING, 6 TERMINATED)
+---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name          | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| DEFAULT_1dde1_00000 | TERMINATED |                 |            4 |  128 |    4 | 0.00648265  | 2.02588 |     0.2675 |                    1 |
| DEFAULT_1dde1_00001 | RUNNING    | 172.17.0.2:1122 |            2 |   16 |    4 | 0.000419419 | 1.49062 |     0.4713 |                    3 |
| DEFAULT_1dde1_00002 | TERMINATED |                 |           16 |   64 |  128 | 0.0486304   | 2.02543 |     0.2616 |                    1 |
| DEFAULT_1dde1_00003 | TERMINATED |                 |            2 |   32 |    4 | 0.0290972   | 2.32603 |     0.0979 |                    1 |
| DEFAULT_1dde1_00004 | TERMINATED |                 |           16 |   16 |  128 | 0.0422261   | 2.03065 |     0.1939 |                   10 |
| DEFAULT_1dde1_00005 | RUNNING    | 172.17.0.2:1108 |            8 |  256 |  128 | 0.000362061 | 1.12143 |     0.603  |                    9 |
| DEFAULT_1dde1_00006 | TERMINATED |                 |            4 |  256 |   16 | 0.00696066  | 1.76108 |     0.3378 |                    2 |
| DEFAULT_1dde1_00007 | RUNNING    | 172.17.0.2:1085 |            4 |  128 |   32 | 0.00139994  | 1.25048 |     0.5726 |                    6 |
| DEFAULT_1dde1_00008 | TERMINATED |                 |            8 |    8 |  128 | 0.000178276 | 2.29409 |     0.1467 |                    1 |
| DEFAULT_1dde1_00009 | RUNNING    | 172.17.0.2:1097 |            2 |   32 |   32 | 0.000438826 | 1.2607  |     0.5465 |                    3 |
+---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


(pid=1108) [10,  4000] loss: 0.502
(pid=1122) [4, 10000] loss: 0.280
(pid=1097) [4, 10000] loss: 0.244
(pid=1085) [7,  2000] loss: 0.989
(pid=1122) [4, 12000] loss: 0.234
Result for DEFAULT_1dde1_00005:
  accuracy: 0.6115
  date: 2020-09-22_01-06-58
  done: true
  experiment_id: 6a1270e5c59840d8acaa81bba30ebb03
  experiment_tag: 5_batch_size=8,l1=256,l2=128,lr=0.00036206
  hostname: fed1025d76e5
  iterations_since_restore: 10
  loss: 1.1189751214504242
  node_ip: 172.17.0.2
  pid: 1108
  should_checkpoint: true
  time_since_restore: 341.492472410202
  time_this_iter_s: 31.655742645263672
  time_total_s: 341.492472410202
  timestamp: 1600736818
  timesteps_since_restore: 0
  training_iteration: 10
  trial_id: 1dde1_00005

== Status ==
Memory usage on this node: 5.7/240.1 GiB
Using AsyncHyperBand: num_stopped=7
Bracket: Iter 8.000: -1.6059559740126133 | Iter 4.000: -1.3617212006807327 | Iter 2.000: -1.577094536884129 | Iter 1.000: -1.9341911580085753
Resources requested: 8/32 CPUs, 0/2 GPUs, 0.0/157.76 GiB heap, 0.0/49.41 GiB objects
Result logdir: /var/lib/jenkins/ray_results/DEFAULT
Number of trials: 10 (4 RUNNING, 6 TERMINATED)
+---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name          | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| DEFAULT_1dde1_00000 | TERMINATED |                 |            4 |  128 |    4 | 0.00648265  | 2.02588 |     0.2675 |                    1 |
| DEFAULT_1dde1_00001 | RUNNING    | 172.17.0.2:1122 |            2 |   16 |    4 | 0.000419419 | 1.49062 |     0.4713 |                    3 |
| DEFAULT_1dde1_00002 | TERMINATED |                 |           16 |   64 |  128 | 0.0486304   | 2.02543 |     0.2616 |                    1 |
| DEFAULT_1dde1_00003 | TERMINATED |                 |            2 |   32 |    4 | 0.0290972   | 2.32603 |     0.0979 |                    1 |
| DEFAULT_1dde1_00004 | TERMINATED |                 |           16 |   16 |  128 | 0.0422261   | 2.03065 |     0.1939 |                   10 |
| DEFAULT_1dde1_00005 | RUNNING    | 172.17.0.2:1108 |            8 |  256 |  128 | 0.000362061 | 1.11898 |     0.6115 |                   10 |
| DEFAULT_1dde1_00006 | TERMINATED |                 |            4 |  256 |   16 | 0.00696066  | 1.76108 |     0.3378 |                    2 |
| DEFAULT_1dde1_00007 | RUNNING    | 172.17.0.2:1085 |            4 |  128 |   32 | 0.00139994  | 1.25048 |     0.5726 |                    6 |
| DEFAULT_1dde1_00008 | TERMINATED |                 |            8 |    8 |  128 | 0.000178276 | 2.29409 |     0.1467 |                    1 |
| DEFAULT_1dde1_00009 | RUNNING    | 172.17.0.2:1097 |            2 |   32 |   32 | 0.000438826 | 1.2607  |     0.5465 |                    3 |
+---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


(pid=1097) [4, 12000] loss: 0.203
(pid=1085) [7,  4000] loss: 0.508
(pid=1122) [4, 14000] loss: 0.195
(pid=1097) [4, 14000] loss: 0.176
(pid=1085) [7,  6000] loss: 0.338
(pid=1122) [4, 16000] loss: 0.177
(pid=1097) [4, 16000] loss: 0.154
(pid=1085) [7,  8000] loss: 0.260
(pid=1122) [4, 18000] loss: 0.153
(pid=1097) [4, 18000] loss: 0.130
(pid=1122) [4, 20000] loss: 0.136
(pid=1085) [7, 10000] loss: 0.204
(pid=1097) [4, 20000] loss: 0.119
Result for DEFAULT_1dde1_00007:
  accuracy: 0.5953
  date: 2020-09-22_01-07-34
  done: false
  experiment_id: d130aa97ad4d43d1a015b0a98a769c9b
  experiment_tag: 7_batch_size=4,l1=128,l2=32,lr=0.0013999
  hostname: fed1025d76e5
  iterations_since_restore: 7
  loss: 1.1804789266631006
  node_ip: 172.17.0.2
  pid: 1085
  should_checkpoint: true
  time_since_restore: 376.9931342601776
  time_this_iter_s: 49.389514207839966
  time_total_s: 376.9931342601776
  timestamp: 1600736854
  timesteps_since_restore: 0
  training_iteration: 7
  trial_id: 1dde1_00007

== Status ==
Memory usage on this node: 5.1/240.1 GiB
Using AsyncHyperBand: num_stopped=7
Bracket: Iter 8.000: -1.6059559740126133 | Iter 4.000: -1.3617212006807327 | Iter 2.000: -1.577094536884129 | Iter 1.000: -1.9341911580085753
Resources requested: 6/32 CPUs, 0/2 GPUs, 0.0/157.76 GiB heap, 0.0/49.41 GiB objects
Result logdir: /var/lib/jenkins/ray_results/DEFAULT
Number of trials: 10 (3 RUNNING, 7 TERMINATED)
+---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name          | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| DEFAULT_1dde1_00000 | TERMINATED |                 |            4 |  128 |    4 | 0.00648265  | 2.02588 |     0.2675 |                    1 |
| DEFAULT_1dde1_00001 | RUNNING    | 172.17.0.2:1122 |            2 |   16 |    4 | 0.000419419 | 1.49062 |     0.4713 |                    3 |
| DEFAULT_1dde1_00002 | TERMINATED |                 |           16 |   64 |  128 | 0.0486304   | 2.02543 |     0.2616 |                    1 |
| DEFAULT_1dde1_00003 | TERMINATED |                 |            2 |   32 |    4 | 0.0290972   | 2.32603 |     0.0979 |                    1 |
| DEFAULT_1dde1_00004 | TERMINATED |                 |           16 |   16 |  128 | 0.0422261   | 2.03065 |     0.1939 |                   10 |
| DEFAULT_1dde1_00005 | TERMINATED |                 |            8 |  256 |  128 | 0.000362061 | 1.11898 |     0.6115 |                   10 |
| DEFAULT_1dde1_00006 | TERMINATED |                 |            4 |  256 |   16 | 0.00696066  | 1.76108 |     0.3378 |                    2 |
| DEFAULT_1dde1_00007 | RUNNING    | 172.17.0.2:1085 |            4 |  128 |   32 | 0.00139994  | 1.18048 |     0.5953 |                    7 |
| DEFAULT_1dde1_00008 | TERMINATED |                 |            8 |    8 |  128 | 0.000178276 | 2.29409 |     0.1467 |                    1 |
| DEFAULT_1dde1_00009 | RUNNING    | 172.17.0.2:1097 |            2 |   32 |   32 | 0.000438826 | 1.2607  |     0.5465 |                    3 |
+---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


Result for DEFAULT_1dde1_00001:
  accuracy: 0.5005
  date: 2020-09-22_01-07-38
  done: false
  experiment_id: f8daf9cd06ad4a4b9c3df3513effdf31
  experiment_tag: 1_batch_size=2,l1=16,l2=4,lr=0.00041942
  hostname: fed1025d76e5
  iterations_since_restore: 4
  loss: 1.35337228445746
  node_ip: 172.17.0.2
  pid: 1122
  should_checkpoint: true
  time_since_restore: 381.63718152046204
  time_this_iter_s: 88.02017140388489
  time_total_s: 381.63718152046204
  timestamp: 1600736858
  timesteps_since_restore: 0
  training_iteration: 4
  trial_id: 1dde1_00001

Result for DEFAULT_1dde1_00009:
  accuracy: 0.5718
  date: 2020-09-22_01-07-41
  done: false
  experiment_id: cfed644d520f435688772253831a176f
  experiment_tag: 9_batch_size=2,l1=32,l2=32,lr=0.00043883
  hostname: fed1025d76e5
  iterations_since_restore: 4
  loss: 1.1846606592386961
  node_ip: 172.17.0.2
  pid: 1097
  should_checkpoint: true
  time_since_restore: 383.6928722858429
  time_this_iter_s: 88.17042851448059
  time_total_s: 383.6928722858429
  timestamp: 1600736861
  timesteps_since_restore: 0
  training_iteration: 4
  trial_id: 1dde1_00009

== Status ==
Memory usage on this node: 5.1/240.1 GiB
Using AsyncHyperBand: num_stopped=7
Bracket: Iter 8.000: -1.6059559740126133 | Iter 4.000: -1.35337228445746 | Iter 2.000: -1.577094536884129 | Iter 1.000: -1.9341911580085753
Resources requested: 6/32 CPUs, 0/2 GPUs, 0.0/157.76 GiB heap, 0.0/49.41 GiB objects
Result logdir: /var/lib/jenkins/ray_results/DEFAULT
Number of trials: 10 (3 RUNNING, 7 TERMINATED)
+---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name          | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| DEFAULT_1dde1_00000 | TERMINATED |                 |            4 |  128 |    4 | 0.00648265  | 2.02588 |     0.2675 |                    1 |
| DEFAULT_1dde1_00001 | RUNNING    | 172.17.0.2:1122 |            2 |   16 |    4 | 0.000419419 | 1.35337 |     0.5005 |                    4 |
| DEFAULT_1dde1_00002 | TERMINATED |                 |           16 |   64 |  128 | 0.0486304   | 2.02543 |     0.2616 |                    1 |
| DEFAULT_1dde1_00003 | TERMINATED |                 |            2 |   32 |    4 | 0.0290972   | 2.32603 |     0.0979 |                    1 |
| DEFAULT_1dde1_00004 | TERMINATED |                 |           16 |   16 |  128 | 0.0422261   | 2.03065 |     0.1939 |                   10 |
| DEFAULT_1dde1_00005 | TERMINATED |                 |            8 |  256 |  128 | 0.000362061 | 1.11898 |     0.6115 |                   10 |
| DEFAULT_1dde1_00006 | TERMINATED |                 |            4 |  256 |   16 | 0.00696066  | 1.76108 |     0.3378 |                    2 |
| DEFAULT_1dde1_00007 | RUNNING    | 172.17.0.2:1085 |            4 |  128 |   32 | 0.00139994  | 1.18048 |     0.5953 |                    7 |
| DEFAULT_1dde1_00008 | TERMINATED |                 |            8 |    8 |  128 | 0.000178276 | 2.29409 |     0.1467 |                    1 |
| DEFAULT_1dde1_00009 | RUNNING    | 172.17.0.2:1097 |            2 |   32 |   32 | 0.000438826 | 1.18466 |     0.5718 |                    4 |
+---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


(pid=1085) [8,  2000] loss: 0.930
(pid=1122) [5,  2000] loss: 1.349
(pid=1097) [5,  2000] loss: 1.148
(pid=1085) [8,  4000] loss: 0.481
(pid=1122) [5,  4000] loss: 0.656
(pid=1097) [5,  4000] loss: 0.573
(pid=1085) [8,  6000] loss: 0.325
(pid=1122) [5,  6000] loss: 0.442
(pid=1097) [5,  6000] loss: 0.393
(pid=1085) [8,  8000] loss: 0.250
(pid=1122) [5,  8000] loss: 0.339
(pid=1097) [5,  8000] loss: 0.287
(pid=1122) [5, 10000] loss: 0.264
(pid=1085) [8, 10000] loss: 0.200
(pid=1097) [5, 10000] loss: 0.237
Result for DEFAULT_1dde1_00007:
  accuracy: 0.6036
  date: 2020-09-22_01-08-23
  done: false
  experiment_id: d130aa97ad4d43d1a015b0a98a769c9b
  experiment_tag: 7_batch_size=4,l1=128,l2=32,lr=0.0013999
  hostname: fed1025d76e5
  iterations_since_restore: 8
  loss: 1.1812849724005907
  node_ip: 172.17.0.2
  pid: 1085
  should_checkpoint: true
  time_since_restore: 426.14055013656616
  time_this_iter_s: 49.14741587638855
  time_total_s: 426.14055013656616
  timestamp: 1600736903
  timesteps_since_restore: 0
  training_iteration: 8
  trial_id: 1dde1_00007

== Status ==
Memory usage on this node: 5.1/240.1 GiB
Using AsyncHyperBand: num_stopped=7
Bracket: Iter 8.000: -1.1812849724005907 | Iter 4.000: -1.35337228445746 | Iter 2.000: -1.577094536884129 | Iter 1.000: -1.9341911580085753
Resources requested: 6/32 CPUs, 0/2 GPUs, 0.0/157.76 GiB heap, 0.0/49.41 GiB objects
Result logdir: /var/lib/jenkins/ray_results/DEFAULT
Number of trials: 10 (3 RUNNING, 7 TERMINATED)
+---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name          | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| DEFAULT_1dde1_00000 | TERMINATED |                 |            4 |  128 |    4 | 0.00648265  | 2.02588 |     0.2675 |                    1 |
| DEFAULT_1dde1_00001 | RUNNING    | 172.17.0.2:1122 |            2 |   16 |    4 | 0.000419419 | 1.35337 |     0.5005 |                    4 |
| DEFAULT_1dde1_00002 | TERMINATED |                 |           16 |   64 |  128 | 0.0486304   | 2.02543 |     0.2616 |                    1 |
| DEFAULT_1dde1_00003 | TERMINATED |                 |            2 |   32 |    4 | 0.0290972   | 2.32603 |     0.0979 |                    1 |
| DEFAULT_1dde1_00004 | TERMINATED |                 |           16 |   16 |  128 | 0.0422261   | 2.03065 |     0.1939 |                   10 |
| DEFAULT_1dde1_00005 | TERMINATED |                 |            8 |  256 |  128 | 0.000362061 | 1.11898 |     0.6115 |                   10 |
| DEFAULT_1dde1_00006 | TERMINATED |                 |            4 |  256 |   16 | 0.00696066  | 1.76108 |     0.3378 |                    2 |
| DEFAULT_1dde1_00007 | RUNNING    | 172.17.0.2:1085 |            4 |  128 |   32 | 0.00139994  | 1.18128 |     0.6036 |                    8 |
| DEFAULT_1dde1_00008 | TERMINATED |                 |            8 |    8 |  128 | 0.000178276 | 2.29409 |     0.1467 |                    1 |
| DEFAULT_1dde1_00009 | RUNNING    | 172.17.0.2:1097 |            2 |   32 |   32 | 0.000438826 | 1.18466 |     0.5718 |                    4 |
+---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


(pid=1122) [5, 12000] loss: 0.223
(pid=1097) [5, 12000] loss: 0.191
(pid=1122) [5, 14000] loss: 0.185
(pid=1085) [9,  2000] loss: 0.902
(pid=1097) [5, 14000] loss: 0.166
(pid=1122) [5, 16000] loss: 0.165
(pid=1085) [9,  4000] loss: 0.470
(pid=1097) [5, 16000] loss: 0.144
(pid=1122) [5, 18000] loss: 0.150
(pid=1097) [5, 18000] loss: 0.129
(pid=1085) [9,  6000] loss: 0.311
(pid=1122) [5, 20000] loss: 0.129
(pid=1097) [5, 20000] loss: 0.117
(pid=1085) [9,  8000] loss: 0.240
Result for DEFAULT_1dde1_00001:
  accuracy: 0.525
  date: 2020-09-22_01-09-05
  done: false
  experiment_id: f8daf9cd06ad4a4b9c3df3513effdf31
  experiment_tag: 1_batch_size=2,l1=16,l2=4,lr=0.00041942
  hostname: fed1025d76e5
  iterations_since_restore: 5
  loss: 1.3273668685505167
  node_ip: 172.17.0.2
  pid: 1122
  should_checkpoint: true
  time_since_restore: 468.13718366622925
  time_this_iter_s: 86.50000214576721
  time_total_s: 468.13718366622925
  timestamp: 1600736945
  timesteps_since_restore: 0
  training_iteration: 5
  trial_id: 1dde1_00001

== Status ==
Memory usage on this node: 5.1/240.1 GiB
Using AsyncHyperBand: num_stopped=7
Bracket: Iter 8.000: -1.1812849724005907 | Iter 4.000: -1.35337228445746 | Iter 2.000: -1.577094536884129 | Iter 1.000: -1.9341911580085753
Resources requested: 6/32 CPUs, 0/2 GPUs, 0.0/157.76 GiB heap, 0.0/49.41 GiB objects
Result logdir: /var/lib/jenkins/ray_results/DEFAULT
Number of trials: 10 (3 RUNNING, 7 TERMINATED)
+---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name          | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| DEFAULT_1dde1_00000 | TERMINATED |                 |            4 |  128 |    4 | 0.00648265  | 2.02588 |     0.2675 |                    1 |
| DEFAULT_1dde1_00001 | RUNNING    | 172.17.0.2:1122 |            2 |   16 |    4 | 0.000419419 | 1.32737 |     0.525  |                    5 |
| DEFAULT_1dde1_00002 | TERMINATED |                 |           16 |   64 |  128 | 0.0486304   | 2.02543 |     0.2616 |                    1 |
| DEFAULT_1dde1_00003 | TERMINATED |                 |            2 |   32 |    4 | 0.0290972   | 2.32603 |     0.0979 |                    1 |
| DEFAULT_1dde1_00004 | TERMINATED |                 |           16 |   16 |  128 | 0.0422261   | 2.03065 |     0.1939 |                   10 |
| DEFAULT_1dde1_00005 | TERMINATED |                 |            8 |  256 |  128 | 0.000362061 | 1.11898 |     0.6115 |                   10 |
| DEFAULT_1dde1_00006 | TERMINATED |                 |            4 |  256 |   16 | 0.00696066  | 1.76108 |     0.3378 |                    2 |
| DEFAULT_1dde1_00007 | RUNNING    | 172.17.0.2:1085 |            4 |  128 |   32 | 0.00139994  | 1.18128 |     0.6036 |                    8 |
| DEFAULT_1dde1_00008 | TERMINATED |                 |            8 |    8 |  128 | 0.000178276 | 2.29409 |     0.1467 |                    1 |
| DEFAULT_1dde1_00009 | RUNNING    | 172.17.0.2:1097 |            2 |   32 |   32 | 0.000438826 | 1.18466 |     0.5718 |                    4 |
+---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


(pid=1085) [9, 10000] loss: 0.191
Result for DEFAULT_1dde1_00009:
  accuracy: 0.5693
  date: 2020-09-22_01-09-08
  done: false
  experiment_id: cfed644d520f435688772253831a176f
  experiment_tag: 9_batch_size=2,l1=32,l2=32,lr=0.00043883
  hostname: fed1025d76e5
  iterations_since_restore: 5
  loss: 1.2223929223241286
  node_ip: 172.17.0.2
  pid: 1097
  should_checkpoint: true
  time_since_restore: 470.7961926460266
  time_this_iter_s: 87.10332036018372
  time_total_s: 470.7961926460266
  timestamp: 1600736948
  timesteps_since_restore: 0
  training_iteration: 5
  trial_id: 1dde1_00009

(pid=1122) [6,  2000] loss: 1.258
Result for DEFAULT_1dde1_00007:
  accuracy: 0.6008
  date: 2020-09-22_01-09-13
  done: false
  experiment_id: d130aa97ad4d43d1a015b0a98a769c9b
  experiment_tag: 7_batch_size=4,l1=128,l2=32,lr=0.0013999
  hostname: fed1025d76e5
  iterations_since_restore: 9
  loss: 1.2257308665644377
  node_ip: 172.17.0.2
  pid: 1085
  should_checkpoint: true
  time_since_restore: 475.85610842704773
  time_this_iter_s: 49.71555829048157
  time_total_s: 475.85610842704773
  timestamp: 1600736953
  timesteps_since_restore: 0
  training_iteration: 9
  trial_id: 1dde1_00007

== Status ==
Memory usage on this node: 5.1/240.1 GiB
Using AsyncHyperBand: num_stopped=7
Bracket: Iter 8.000: -1.1812849724005907 | Iter 4.000: -1.35337228445746 | Iter 2.000: -1.577094536884129 | Iter 1.000: -1.9341911580085753
Resources requested: 6/32 CPUs, 0/2 GPUs, 0.0/157.76 GiB heap, 0.0/49.41 GiB objects
Result logdir: /var/lib/jenkins/ray_results/DEFAULT
Number of trials: 10 (3 RUNNING, 7 TERMINATED)
+---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name          | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| DEFAULT_1dde1_00000 | TERMINATED |                 |            4 |  128 |    4 | 0.00648265  | 2.02588 |     0.2675 |                    1 |
| DEFAULT_1dde1_00001 | RUNNING    | 172.17.0.2:1122 |            2 |   16 |    4 | 0.000419419 | 1.32737 |     0.525  |                    5 |
| DEFAULT_1dde1_00002 | TERMINATED |                 |           16 |   64 |  128 | 0.0486304   | 2.02543 |     0.2616 |                    1 |
| DEFAULT_1dde1_00003 | TERMINATED |                 |            2 |   32 |    4 | 0.0290972   | 2.32603 |     0.0979 |                    1 |
| DEFAULT_1dde1_00004 | TERMINATED |                 |           16 |   16 |  128 | 0.0422261   | 2.03065 |     0.1939 |                   10 |
| DEFAULT_1dde1_00005 | TERMINATED |                 |            8 |  256 |  128 | 0.000362061 | 1.11898 |     0.6115 |                   10 |
| DEFAULT_1dde1_00006 | TERMINATED |                 |            4 |  256 |   16 | 0.00696066  | 1.76108 |     0.3378 |                    2 |
| DEFAULT_1dde1_00007 | RUNNING    | 172.17.0.2:1085 |            4 |  128 |   32 | 0.00139994  | 1.22573 |     0.6008 |                    9 |
| DEFAULT_1dde1_00008 | TERMINATED |                 |            8 |    8 |  128 | 0.000178276 | 2.29409 |     0.1467 |                    1 |
| DEFAULT_1dde1_00009 | RUNNING    | 172.17.0.2:1097 |            2 |   32 |   32 | 0.000438826 | 1.22239 |     0.5693 |                    5 |
+---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


(pid=1097) [6,  2000] loss: 1.087
(pid=1122) [6,  4000] loss: 0.644
(pid=1085) [10,  2000] loss: 0.860
(pid=1097) [6,  4000] loss: 0.550
(pid=1122) [6,  6000] loss: 0.426
(pid=1085) [10,  4000] loss: 0.443
(pid=1097) [6,  6000] loss: 0.361
(pid=1122) [6,  8000] loss: 0.321
(pid=1097) [6,  8000] loss: 0.281
(pid=1085) [10,  6000] loss: 0.306
(pid=1122) [6, 10000] loss: 0.257
(pid=1097) [6, 10000] loss: 0.228
(pid=1085) [10,  8000] loss: 0.232
(pid=1122) [6, 12000] loss: 0.222
(pid=1097) [6, 12000] loss: 0.190
(pid=1085) [10, 10000] loss: 0.191
(pid=1122) [6, 14000] loss: 0.183
(pid=1097) [6, 14000] loss: 0.161
Result for DEFAULT_1dde1_00007:
  accuracy: 0.6077
  date: 2020-09-22_01-10-02
  done: true
  experiment_id: d130aa97ad4d43d1a015b0a98a769c9b
  experiment_tag: 7_batch_size=4,l1=128,l2=32,lr=0.0013999
  hostname: fed1025d76e5
  iterations_since_restore: 10
  loss: 1.2082138083937577
  node_ip: 172.17.0.2
  pid: 1085
  should_checkpoint: true
  time_since_restore: 524.955450296402
  time_this_iter_s: 49.09934186935425
  time_total_s: 524.955450296402
  timestamp: 1600737002
  timesteps_since_restore: 0
  training_iteration: 10
  trial_id: 1dde1_00007

== Status ==
Memory usage on this node: 5.1/240.1 GiB
Using AsyncHyperBand: num_stopped=8
Bracket: Iter 8.000: -1.1812849724005907 | Iter 4.000: -1.35337228445746 | Iter 2.000: -1.577094536884129 | Iter 1.000: -1.9341911580085753
Resources requested: 6/32 CPUs, 0/2 GPUs, 0.0/157.76 GiB heap, 0.0/49.41 GiB objects
Result logdir: /var/lib/jenkins/ray_results/DEFAULT
Number of trials: 10 (3 RUNNING, 7 TERMINATED)
+---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name          | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| DEFAULT_1dde1_00000 | TERMINATED |                 |            4 |  128 |    4 | 0.00648265  | 2.02588 |     0.2675 |                    1 |
| DEFAULT_1dde1_00001 | RUNNING    | 172.17.0.2:1122 |            2 |   16 |    4 | 0.000419419 | 1.32737 |     0.525  |                    5 |
| DEFAULT_1dde1_00002 | TERMINATED |                 |           16 |   64 |  128 | 0.0486304   | 2.02543 |     0.2616 |                    1 |
| DEFAULT_1dde1_00003 | TERMINATED |                 |            2 |   32 |    4 | 0.0290972   | 2.32603 |     0.0979 |                    1 |
| DEFAULT_1dde1_00004 | TERMINATED |                 |           16 |   16 |  128 | 0.0422261   | 2.03065 |     0.1939 |                   10 |
| DEFAULT_1dde1_00005 | TERMINATED |                 |            8 |  256 |  128 | 0.000362061 | 1.11898 |     0.6115 |                   10 |
| DEFAULT_1dde1_00006 | TERMINATED |                 |            4 |  256 |   16 | 0.00696066  | 1.76108 |     0.3378 |                    2 |
| DEFAULT_1dde1_00007 | RUNNING    | 172.17.0.2:1085 |            4 |  128 |   32 | 0.00139994  | 1.20821 |     0.6077 |                   10 |
| DEFAULT_1dde1_00008 | TERMINATED |                 |            8 |    8 |  128 | 0.000178276 | 2.29409 |     0.1467 |                    1 |
| DEFAULT_1dde1_00009 | RUNNING    | 172.17.0.2:1097 |            2 |   32 |   32 | 0.000438826 | 1.22239 |     0.5693 |                    5 |
+---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


(pid=1122) [6, 16000] loss: 0.163
(pid=1097) [6, 16000] loss: 0.139
(pid=1122) [6, 18000] loss: 0.141
(pid=1097) [6, 18000] loss: 0.123
(pid=1122) [6, 20000] loss: 0.125
(pid=1097) [6, 20000] loss: 0.114
Result for DEFAULT_1dde1_00001:
  accuracy: 0.5372
  date: 2020-09-22_01-10-31
  done: false
  experiment_id: f8daf9cd06ad4a4b9c3df3513effdf31
  experiment_tag: 1_batch_size=2,l1=16,l2=4,lr=0.00041942
  hostname: fed1025d76e5
  iterations_since_restore: 6
  loss: 1.306188284834847
  node_ip: 172.17.0.2
  pid: 1122
  should_checkpoint: true
  time_since_restore: 553.9988505840302
  time_this_iter_s: 85.8616669178009
  time_total_s: 553.9988505840302
  timestamp: 1600737031
  timesteps_since_restore: 0
  training_iteration: 6
  trial_id: 1dde1_00001

== Status ==
Memory usage on this node: 4.6/240.1 GiB
Using AsyncHyperBand: num_stopped=8
Bracket: Iter 8.000: -1.1812849724005907 | Iter 4.000: -1.35337228445746 | Iter 2.000: -1.577094536884129 | Iter 1.000: -1.9341911580085753
Resources requested: 4/32 CPUs, 0/2 GPUs, 0.0/157.76 GiB heap, 0.0/49.41 GiB objects
Result logdir: /var/lib/jenkins/ray_results/DEFAULT
Number of trials: 10 (2 RUNNING, 8 TERMINATED)
+---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name          | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| DEFAULT_1dde1_00000 | TERMINATED |                 |            4 |  128 |    4 | 0.00648265  | 2.02588 |     0.2675 |                    1 |
| DEFAULT_1dde1_00001 | RUNNING    | 172.17.0.2:1122 |            2 |   16 |    4 | 0.000419419 | 1.30619 |     0.5372 |                    6 |
| DEFAULT_1dde1_00002 | TERMINATED |                 |           16 |   64 |  128 | 0.0486304   | 2.02543 |     0.2616 |                    1 |
| DEFAULT_1dde1_00003 | TERMINATED |                 |            2 |   32 |    4 | 0.0290972   | 2.32603 |     0.0979 |                    1 |
| DEFAULT_1dde1_00004 | TERMINATED |                 |           16 |   16 |  128 | 0.0422261   | 2.03065 |     0.1939 |                   10 |
| DEFAULT_1dde1_00005 | TERMINATED |                 |            8 |  256 |  128 | 0.000362061 | 1.11898 |     0.6115 |                   10 |
| DEFAULT_1dde1_00006 | TERMINATED |                 |            4 |  256 |   16 | 0.00696066  | 1.76108 |     0.3378 |                    2 |
| DEFAULT_1dde1_00007 | TERMINATED |                 |            4 |  128 |   32 | 0.00139994  | 1.20821 |     0.6077 |                   10 |
| DEFAULT_1dde1_00008 | TERMINATED |                 |            8 |    8 |  128 | 0.000178276 | 2.29409 |     0.1467 |                    1 |
| DEFAULT_1dde1_00009 | RUNNING    | 172.17.0.2:1097 |            2 |   32 |   32 | 0.000438826 | 1.22239 |     0.5693 |                    5 |
+---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


Result for DEFAULT_1dde1_00009:
  accuracy: 0.5829
  date: 2020-09-22_01-10-34
  done: false
  experiment_id: cfed644d520f435688772253831a176f
  experiment_tag: 9_batch_size=2,l1=32,l2=32,lr=0.00043883
  hostname: fed1025d76e5
  iterations_since_restore: 6
  loss: 1.1963300617878792
  node_ip: 172.17.0.2
  pid: 1097
  should_checkpoint: true
  time_since_restore: 557.4718091487885
  time_this_iter_s: 86.67561650276184
  time_total_s: 557.4718091487885
  timestamp: 1600737034
  timesteps_since_restore: 0
  training_iteration: 6
  trial_id: 1dde1_00009

(pid=1122) [7,  2000] loss: 1.218
(pid=1097) [7,  2000] loss: 1.064
(pid=1122) [7,  4000] loss: 0.608
(pid=1097) [7,  4000] loss: 0.540
(pid=1122) [7,  6000] loss: 0.414
(pid=1097) [7,  6000] loss: 0.365
(pid=1122) [7,  8000] loss: 0.313
(pid=1097) [7,  8000] loss: 0.265
(pid=1122) [7, 10000] loss: 0.254
(pid=1097) [7, 10000] loss: 0.221
(pid=1122) [7, 12000] loss: 0.210
(pid=1097) [7, 12000] loss: 0.180
(pid=1122) [7, 14000] loss: 0.183
(pid=1097) [7, 14000] loss: 0.153
(pid=1122) [7, 16000] loss: 0.159
(pid=1097) [7, 16000] loss: 0.136
(pid=1122) [7, 18000] loss: 0.139
(pid=1097) [7, 18000] loss: 0.123
(pid=1122) [7, 20000] loss: 0.125
(pid=1097) [7, 20000] loss: 0.111
Result for DEFAULT_1dde1_00001:
  accuracy: 0.5418
  date: 2020-09-22_01-11-56
  done: false
  experiment_id: f8daf9cd06ad4a4b9c3df3513effdf31
  experiment_tag: 1_batch_size=2,l1=16,l2=4,lr=0.00041942
  hostname: fed1025d76e5
  iterations_since_restore: 7
  loss: 1.3043943683009596
  node_ip: 172.17.0.2
  pid: 1122
  should_checkpoint: true
  time_since_restore: 638.82790184021
  time_this_iter_s: 84.82905125617981
  time_total_s: 638.82790184021
  timestamp: 1600737116
  timesteps_since_restore: 0
  training_iteration: 7
  trial_id: 1dde1_00001

== Status ==
Memory usage on this node: 4.6/240.1 GiB
Using AsyncHyperBand: num_stopped=8
Bracket: Iter 8.000: -1.1812849724005907 | Iter 4.000: -1.35337228445746 | Iter 2.000: -1.577094536884129 | Iter 1.000: -1.9341911580085753
Resources requested: 4/32 CPUs, 0/2 GPUs, 0.0/157.76 GiB heap, 0.0/49.41 GiB objects
Result logdir: /var/lib/jenkins/ray_results/DEFAULT
Number of trials: 10 (2 RUNNING, 8 TERMINATED)
+---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name          | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| DEFAULT_1dde1_00000 | TERMINATED |                 |            4 |  128 |    4 | 0.00648265  | 2.02588 |     0.2675 |                    1 |
| DEFAULT_1dde1_00001 | RUNNING    | 172.17.0.2:1122 |            2 |   16 |    4 | 0.000419419 | 1.30439 |     0.5418 |                    7 |
| DEFAULT_1dde1_00002 | TERMINATED |                 |           16 |   64 |  128 | 0.0486304   | 2.02543 |     0.2616 |                    1 |
| DEFAULT_1dde1_00003 | TERMINATED |                 |            2 |   32 |    4 | 0.0290972   | 2.32603 |     0.0979 |                    1 |
| DEFAULT_1dde1_00004 | TERMINATED |                 |           16 |   16 |  128 | 0.0422261   | 2.03065 |     0.1939 |                   10 |
| DEFAULT_1dde1_00005 | TERMINATED |                 |            8 |  256 |  128 | 0.000362061 | 1.11898 |     0.6115 |                   10 |
| DEFAULT_1dde1_00006 | TERMINATED |                 |            4 |  256 |   16 | 0.00696066  | 1.76108 |     0.3378 |                    2 |
| DEFAULT_1dde1_00007 | TERMINATED |                 |            4 |  128 |   32 | 0.00139994  | 1.20821 |     0.6077 |                   10 |
| DEFAULT_1dde1_00008 | TERMINATED |                 |            8 |    8 |  128 | 0.000178276 | 2.29409 |     0.1467 |                    1 |
| DEFAULT_1dde1_00009 | RUNNING    | 172.17.0.2:1097 |            2 |   32 |   32 | 0.000438826 | 1.19633 |     0.5829 |                    6 |
+---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


Result for DEFAULT_1dde1_00009:
  accuracy: 0.5828
  date: 2020-09-22_01-12-00
  done: false
  experiment_id: cfed644d520f435688772253831a176f
  experiment_tag: 9_batch_size=2,l1=32,l2=32,lr=0.00043883
  hostname: fed1025d76e5
  iterations_since_restore: 7
  loss: 1.1821620257275178
  node_ip: 172.17.0.2
  pid: 1097
  should_checkpoint: true
  time_since_restore: 643.4907233715057
  time_this_iter_s: 86.01891422271729
  time_total_s: 643.4907233715057
  timestamp: 1600737120
  timesteps_since_restore: 0
  training_iteration: 7
  trial_id: 1dde1_00009

(pid=1122) [8,  2000] loss: 1.203
(pid=1097) [8,  2000] loss: 1.043
(pid=1122) [8,  4000] loss: 0.625
(pid=1097) [8,  4000] loss: 0.529
(pid=1122) [8,  6000] loss: 0.410
(pid=1097) [8,  6000] loss: 0.359
(pid=1122) [8,  8000] loss: 0.305
(pid=1097) [8,  8000] loss: 0.265
(pid=1122) [8, 10000] loss: 0.247
(pid=1097) [8, 10000] loss: 0.208
(pid=1122) [8, 12000] loss: 0.201
(pid=1097) [8, 12000] loss: 0.182
(pid=1122) [8, 14000] loss: 0.181
(pid=1097) [8, 14000] loss: 0.151
(pid=1122) [8, 16000] loss: 0.153
(pid=1097) [8, 16000] loss: 0.136
(pid=1122) [8, 18000] loss: 0.137
(pid=1097) [8, 18000] loss: 0.117
(pid=1122) [8, 20000] loss: 0.121
(pid=1097) [8, 20000] loss: 0.103
Result for DEFAULT_1dde1_00001:
  accuracy: 0.5532
  date: 2020-09-22_01-13-21
  done: true
  experiment_id: f8daf9cd06ad4a4b9c3df3513effdf31
  experiment_tag: 1_batch_size=2,l1=16,l2=4,lr=0.00041942
  hostname: fed1025d76e5
  iterations_since_restore: 8
  loss: 1.2664230019509326
  node_ip: 172.17.0.2
  pid: 1122
  should_checkpoint: true
  time_since_restore: 723.7728803157806
  time_this_iter_s: 84.94497847557068
  time_total_s: 723.7728803157806
  timestamp: 1600737201
  timesteps_since_restore: 0
  training_iteration: 8
  trial_id: 1dde1_00001

== Status ==
Memory usage on this node: 4.6/240.1 GiB
Using AsyncHyperBand: num_stopped=9
Bracket: Iter 8.000: -1.2238539871757617 | Iter 4.000: -1.35337228445746 | Iter 2.000: -1.577094536884129 | Iter 1.000: -1.9341911580085753
Resources requested: 4/32 CPUs, 0/2 GPUs, 0.0/157.76 GiB heap, 0.0/49.41 GiB objects
Result logdir: /var/lib/jenkins/ray_results/DEFAULT
Number of trials: 10 (2 RUNNING, 8 TERMINATED)
+---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name          | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| DEFAULT_1dde1_00000 | TERMINATED |                 |            4 |  128 |    4 | 0.00648265  | 2.02588 |     0.2675 |                    1 |
| DEFAULT_1dde1_00001 | RUNNING    | 172.17.0.2:1122 |            2 |   16 |    4 | 0.000419419 | 1.26642 |     0.5532 |                    8 |
| DEFAULT_1dde1_00002 | TERMINATED |                 |           16 |   64 |  128 | 0.0486304   | 2.02543 |     0.2616 |                    1 |
| DEFAULT_1dde1_00003 | TERMINATED |                 |            2 |   32 |    4 | 0.0290972   | 2.32603 |     0.0979 |                    1 |
| DEFAULT_1dde1_00004 | TERMINATED |                 |           16 |   16 |  128 | 0.0422261   | 2.03065 |     0.1939 |                   10 |
| DEFAULT_1dde1_00005 | TERMINATED |                 |            8 |  256 |  128 | 0.000362061 | 1.11898 |     0.6115 |                   10 |
| DEFAULT_1dde1_00006 | TERMINATED |                 |            4 |  256 |   16 | 0.00696066  | 1.76108 |     0.3378 |                    2 |
| DEFAULT_1dde1_00007 | TERMINATED |                 |            4 |  128 |   32 | 0.00139994  | 1.20821 |     0.6077 |                   10 |
| DEFAULT_1dde1_00008 | TERMINATED |                 |            8 |    8 |  128 | 0.000178276 | 2.29409 |     0.1467 |                    1 |
| DEFAULT_1dde1_00009 | RUNNING    | 172.17.0.2:1097 |            2 |   32 |   32 | 0.000438826 | 1.18216 |     0.5828 |                    7 |
+---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


Result for DEFAULT_1dde1_00009:
  accuracy: 0.5975
  date: 2020-09-22_01-13-26
  done: false
  experiment_id: cfed644d520f435688772253831a176f
  experiment_tag: 9_batch_size=2,l1=32,l2=32,lr=0.00043883
  hostname: fed1025d76e5
  iterations_since_restore: 8
  loss: 1.139856807322579
  node_ip: 172.17.0.2
  pid: 1097
  should_checkpoint: true
  time_since_restore: 728.941769361496
  time_this_iter_s: 85.45104598999023
  time_total_s: 728.941769361496
  timestamp: 1600737206
  timesteps_since_restore: 0
  training_iteration: 8
  trial_id: 1dde1_00009

== Status ==
Memory usage on this node: 4.1/240.1 GiB
Using AsyncHyperBand: num_stopped=9
Bracket: Iter 8.000: -1.1812849724005907 | Iter 4.000: -1.35337228445746 | Iter 2.000: -1.577094536884129 | Iter 1.000: -1.9341911580085753
Resources requested: 2/32 CPUs, 0/2 GPUs, 0.0/157.76 GiB heap, 0.0/49.41 GiB objects
Result logdir: /var/lib/jenkins/ray_results/DEFAULT
Number of trials: 10 (1 RUNNING, 9 TERMINATED)
+---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name          | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| DEFAULT_1dde1_00000 | TERMINATED |                 |            4 |  128 |    4 | 0.00648265  | 2.02588 |     0.2675 |                    1 |
| DEFAULT_1dde1_00001 | TERMINATED |                 |            2 |   16 |    4 | 0.000419419 | 1.26642 |     0.5532 |                    8 |
| DEFAULT_1dde1_00002 | TERMINATED |                 |           16 |   64 |  128 | 0.0486304   | 2.02543 |     0.2616 |                    1 |
| DEFAULT_1dde1_00003 | TERMINATED |                 |            2 |   32 |    4 | 0.0290972   | 2.32603 |     0.0979 |                    1 |
| DEFAULT_1dde1_00004 | TERMINATED |                 |           16 |   16 |  128 | 0.0422261   | 2.03065 |     0.1939 |                   10 |
| DEFAULT_1dde1_00005 | TERMINATED |                 |            8 |  256 |  128 | 0.000362061 | 1.11898 |     0.6115 |                   10 |
| DEFAULT_1dde1_00006 | TERMINATED |                 |            4 |  256 |   16 | 0.00696066  | 1.76108 |     0.3378 |                    2 |
| DEFAULT_1dde1_00007 | TERMINATED |                 |            4 |  128 |   32 | 0.00139994  | 1.20821 |     0.6077 |                   10 |
| DEFAULT_1dde1_00008 | TERMINATED |                 |            8 |    8 |  128 | 0.000178276 | 2.29409 |     0.1467 |                    1 |
| DEFAULT_1dde1_00009 | RUNNING    | 172.17.0.2:1097 |            2 |   32 |   32 | 0.000438826 | 1.13986 |     0.5975 |                    8 |
+---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


(pid=1097) [9,  2000] loss: 1.029
(pid=1097) [9,  4000] loss: 0.509
(pid=1097) [9,  6000] loss: 0.339
(pid=1097) [9,  8000] loss: 0.257
(pid=1097) [9, 10000] loss: 0.207
(pid=1097) [9, 12000] loss: 0.175
(pid=1097) [9, 14000] loss: 0.150
(pid=1097) [9, 16000] loss: 0.129
(pid=1097) [9, 18000] loss: 0.117
(pid=1097) [9, 20000] loss: 0.109
Result for DEFAULT_1dde1_00009:
  accuracy: 0.5924
  date: 2020-09-22_01-14-50
  done: false
  experiment_id: cfed644d520f435688772253831a176f
  experiment_tag: 9_batch_size=2,l1=32,l2=32,lr=0.00043883
  hostname: fed1025d76e5
  iterations_since_restore: 9
  loss: 1.1540529461038997
  node_ip: 172.17.0.2
  pid: 1097
  should_checkpoint: true
  time_since_restore: 813.1155605316162
  time_this_iter_s: 84.17379117012024
  time_total_s: 813.1155605316162
  timestamp: 1600737290
  timesteps_since_restore: 0
  training_iteration: 9
  trial_id: 1dde1_00009

== Status ==
Memory usage on this node: 4.1/240.1 GiB
Using AsyncHyperBand: num_stopped=9
Bracket: Iter 8.000: -1.1812849724005907 | Iter 4.000: -1.35337228445746 | Iter 2.000: -1.577094536884129 | Iter 1.000: -1.9341911580085753
Resources requested: 2/32 CPUs, 0/2 GPUs, 0.0/157.76 GiB heap, 0.0/49.41 GiB objects
Result logdir: /var/lib/jenkins/ray_results/DEFAULT
Number of trials: 10 (1 RUNNING, 9 TERMINATED)
+---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name          | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| DEFAULT_1dde1_00000 | TERMINATED |                 |            4 |  128 |    4 | 0.00648265  | 2.02588 |     0.2675 |                    1 |
| DEFAULT_1dde1_00001 | TERMINATED |                 |            2 |   16 |    4 | 0.000419419 | 1.26642 |     0.5532 |                    8 |
| DEFAULT_1dde1_00002 | TERMINATED |                 |           16 |   64 |  128 | 0.0486304   | 2.02543 |     0.2616 |                    1 |
| DEFAULT_1dde1_00003 | TERMINATED |                 |            2 |   32 |    4 | 0.0290972   | 2.32603 |     0.0979 |                    1 |
| DEFAULT_1dde1_00004 | TERMINATED |                 |           16 |   16 |  128 | 0.0422261   | 2.03065 |     0.1939 |                   10 |
| DEFAULT_1dde1_00005 | TERMINATED |                 |            8 |  256 |  128 | 0.000362061 | 1.11898 |     0.6115 |                   10 |
| DEFAULT_1dde1_00006 | TERMINATED |                 |            4 |  256 |   16 | 0.00696066  | 1.76108 |     0.3378 |                    2 |
| DEFAULT_1dde1_00007 | TERMINATED |                 |            4 |  128 |   32 | 0.00139994  | 1.20821 |     0.6077 |                   10 |
| DEFAULT_1dde1_00008 | TERMINATED |                 |            8 |    8 |  128 | 0.000178276 | 2.29409 |     0.1467 |                    1 |
| DEFAULT_1dde1_00009 | RUNNING    | 172.17.0.2:1097 |            2 |   32 |   32 | 0.000438826 | 1.15405 |     0.5924 |                    9 |
+---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


(pid=1097) [10,  2000] loss: 0.984
(pid=1097) [10,  4000] loss: 0.490
(pid=1097) [10,  6000] loss: 0.339
(pid=1097) [10,  8000] loss: 0.264
(pid=1097) [10, 10000] loss: 0.206
(pid=1097) [10, 12000] loss: 0.167
(pid=1097) [10, 14000] loss: 0.150
(pid=1097) [10, 16000] loss: 0.127
(pid=1097) [10, 18000] loss: 0.116
(pid=1097) [10, 20000] loss: 0.107
Result for DEFAULT_1dde1_00009:
  accuracy: 0.5724
  date: 2020-09-22_01-16-15
  done: true
  experiment_id: cfed644d520f435688772253831a176f
  experiment_tag: 9_batch_size=2,l1=32,l2=32,lr=0.00043883
  hostname: fed1025d76e5
  iterations_since_restore: 10
  loss: 1.220944674178597
  node_ip: 172.17.0.2
  pid: 1097
  should_checkpoint: true
  time_since_restore: 897.8286879062653
  time_this_iter_s: 84.71312737464905
  time_total_s: 897.8286879062653
  timestamp: 1600737375
  timesteps_since_restore: 0
  training_iteration: 10
  trial_id: 1dde1_00009

== Status ==
Memory usage on this node: 4.1/240.1 GiB
Using AsyncHyperBand: num_stopped=10
Bracket: Iter 8.000: -1.1812849724005907 | Iter 4.000: -1.35337228445746 | Iter 2.000: -1.577094536884129 | Iter 1.000: -1.9341911580085753
Resources requested: 2/32 CPUs, 0/2 GPUs, 0.0/157.76 GiB heap, 0.0/49.41 GiB objects
Result logdir: /var/lib/jenkins/ray_results/DEFAULT
Number of trials: 10 (1 RUNNING, 9 TERMINATED)
+---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name          | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| DEFAULT_1dde1_00000 | TERMINATED |                 |            4 |  128 |    4 | 0.00648265  | 2.02588 |     0.2675 |                    1 |
| DEFAULT_1dde1_00001 | TERMINATED |                 |            2 |   16 |    4 | 0.000419419 | 1.26642 |     0.5532 |                    8 |
| DEFAULT_1dde1_00002 | TERMINATED |                 |           16 |   64 |  128 | 0.0486304   | 2.02543 |     0.2616 |                    1 |
| DEFAULT_1dde1_00003 | TERMINATED |                 |            2 |   32 |    4 | 0.0290972   | 2.32603 |     0.0979 |                    1 |
| DEFAULT_1dde1_00004 | TERMINATED |                 |           16 |   16 |  128 | 0.0422261   | 2.03065 |     0.1939 |                   10 |
| DEFAULT_1dde1_00005 | TERMINATED |                 |            8 |  256 |  128 | 0.000362061 | 1.11898 |     0.6115 |                   10 |
| DEFAULT_1dde1_00006 | TERMINATED |                 |            4 |  256 |   16 | 0.00696066  | 1.76108 |     0.3378 |                    2 |
| DEFAULT_1dde1_00007 | TERMINATED |                 |            4 |  128 |   32 | 0.00139994  | 1.20821 |     0.6077 |                   10 |
| DEFAULT_1dde1_00008 | TERMINATED |                 |            8 |    8 |  128 | 0.000178276 | 2.29409 |     0.1467 |                    1 |
| DEFAULT_1dde1_00009 | RUNNING    | 172.17.0.2:1097 |            2 |   32 |   32 | 0.000438826 | 1.22094 |     0.5724 |                   10 |
+---------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


== Status ==
Memory usage on this node: 4.1/240.1 GiB
Using AsyncHyperBand: num_stopped=10
Bracket: Iter 8.000: -1.1812849724005907 | Iter 4.000: -1.35337228445746 | Iter 2.000: -1.577094536884129 | Iter 1.000: -1.9341911580085753
Resources requested: 0/32 CPUs, 0/2 GPUs, 0.0/157.76 GiB heap, 0.0/49.41 GiB objects
Result logdir: /var/lib/jenkins/ray_results/DEFAULT
Number of trials: 10 (10 TERMINATED)
+---------------------+------------+-------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name          | status     | loc   |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|---------------------+------------+-------+--------------+------+------+-------------+---------+------------+----------------------|
| DEFAULT_1dde1_00000 | TERMINATED |       |            4 |  128 |    4 | 0.00648265  | 2.02588 |     0.2675 |                    1 |
| DEFAULT_1dde1_00001 | TERMINATED |       |            2 |   16 |    4 | 0.000419419 | 1.26642 |     0.5532 |                    8 |
| DEFAULT_1dde1_00002 | TERMINATED |       |           16 |   64 |  128 | 0.0486304   | 2.02543 |     0.2616 |                    1 |
| DEFAULT_1dde1_00003 | TERMINATED |       |            2 |   32 |    4 | 0.0290972   | 2.32603 |     0.0979 |                    1 |
| DEFAULT_1dde1_00004 | TERMINATED |       |           16 |   16 |  128 | 0.0422261   | 2.03065 |     0.1939 |                   10 |
| DEFAULT_1dde1_00005 | TERMINATED |       |            8 |  256 |  128 | 0.000362061 | 1.11898 |     0.6115 |                   10 |
| DEFAULT_1dde1_00006 | TERMINATED |       |            4 |  256 |   16 | 0.00696066  | 1.76108 |     0.3378 |                    2 |
| DEFAULT_1dde1_00007 | TERMINATED |       |            4 |  128 |   32 | 0.00139994  | 1.20821 |     0.6077 |                   10 |
| DEFAULT_1dde1_00008 | TERMINATED |       |            8 |    8 |  128 | 0.000178276 | 2.29409 |     0.1467 |                    1 |
| DEFAULT_1dde1_00009 | TERMINATED |       |            2 |   32 |   32 | 0.000438826 | 1.22094 |     0.5724 |                   10 |
+---------------------+------------+-------+--------------+------+------+-------------+---------+------------+----------------------+


Best trial config: {'l1': 256, 'l2': 128, 'lr': 0.00036206118636519547, 'batch_size': 8}
Best trial final validation loss: 1.1189751214504242
Best trial final validation accuracy: 0.6115
Files already downloaded and verified
Files already downloaded and verified
Best trial test set accuracy: 0.6028

If you run the code, an example output could look like this:

Most trials have been stopped early in order to avoid wasting resources. The best performing trial achieved a validation accuracy of about 58%, which could be confirmed on the test set.

So that’s it! You can now tune the parameters of your PyTorch models.

Total running time of the script: ( 15 minutes 20.830 seconds)

Gallery generated by Sphinx-Gallery

Docs

Access comprehensive developer documentation for PyTorch

View Docs

Tutorials

Get in-depth tutorials for beginners and advanced developers

View Tutorials

Resources

Find development resources and get your questions answered

View Resources