Distributed hyperparameter optimization framework, inspired by Optuna.

Overview

Goptuna

Software License GoDoc Go Report Card tests

Distributed hyperparameter optimization framework, inspired by Optuna [1]. This library is particularly designed for machine learning, but everything will be able to optimize if you can define the objective function (e.g. Optimizing the number of goroutines of your server and the memory buffer size of the caching systems).

Supported algorithms:

Goptuna supports various state-of-the-art Bayesian optimization, Evolution strategy and Multi-armed bandit algorithms. These algorithms are implemented in pure Go and continuously benchmarked on GitHub Actions.

  • Random search
  • TPE: Tree-structured Parzen Estimators [2]
  • CMA-ES: Covariance Matrix Adaptation Evolution Strategy [3]
  • IPOP-CMA-ES: CMA-ES with increasing population size [4]
  • BIPOP-CMA-ES: BI-population CMA-ES [5]
  • Median Stopping Rule [6]
  • ASHA: Asynchronous Successive Halving Algorithm (Optuna flavored version) [1,7,8]

Built-in dashboard:

Manage optimization results Interactive live-updating graphs
state-of-the-art-algorithms visualization

Projects using Goptuna:

Installation

You can integrate Goptuna in wide variety of Go projects because of its portability of pure Go.

$ go get -u github.com/c-bata/goptuna

Usage

Goptuna supports Define-by-Run style API like Optuna. You can dynamically construct the search spaces.

Basic usage

package main

import (
    "log"
    "math"

    "github.com/c-bata/goptuna"
    "github.com/c-bata/goptuna/tpe"
)

// ① Define an objective function which returns a value you want to minimize.
func objective(trial goptuna.Trial) (float64, error) {
    // ② Define the search space via Suggest APIs.
    x1, _ := trial.SuggestFloat("x1", -10, 10)
    x2, _ := trial.SuggestFloat("x2", -10, 10)
    return math.Pow(x1-2, 2) + math.Pow(x2+5, 2), nil
}

func main() {
    // ③ Create a study which manages each experiment.
    study, err := goptuna.CreateStudy(
        "goptuna-example",
        goptuna.StudyOptionSampler(tpe.NewSampler()))
    if err != nil { ... }

    // ④ Evaluate your objective function.
    err = study.Optimize(objective, 100)
    if err != nil { ... }

    // ⑤ Print the best evaluation parameters.
    v, _ := study.GetBestValue()
    p, _ := study.GetBestParams()
    log.Printf("Best value=%f (x1=%f, x2=%f)",
        v, p["x1"].(float64), p["x2"].(float64))
}

Link: Go Playground

Furthermore, I recommend you to use RDB storage backend for following purposes.

  • Continue from where we stopped in the previous optimizations.
  • Scale studies to tens of workers that connecting to the same RDB storage.
  • Check optimization results via built-in dashboard.

Advanced usage

Distributed optimization using MySQL

There is no complicated setup to use RDB storage backend. First, setup MySQL server like following to share the optimization result.

$ docker pull mysql:8.0
$ docker run \
  -d \
  --rm \
  -p 3306:3306 \
  --mount type=volume,src=mysql,dst=/etc/mysql/conf.d \
  -e MYSQL_USER=goptuna \
  -e MYSQL_DATABASE=goptuna \
  -e MYSQL_PASSWORD=password \
  -e MYSQL_ALLOW_EMPTY_PASSWORD=yes \
  --name goptuna-mysql \
  mysql:8.0

Then, create a study object using goptuna CLI.

$ goptuna create-study --storage mysql://goptuna:[email protected]:3306/yourdb --study yourstudy
yourstudy
$ mysql --host 127.0.0.1 --port 3306 --user goptuna -ppassword -e "SELECT * FROM studies;"
+----------+------------+-----------+
| study_id | study_name | direction |
+----------+------------+-----------+
|        1 | yourstudy  | MINIMIZE  |
+----------+------------+-----------+
1 row in set (0.00 sec)

Finally, run the Goptuna workers which contains following code. You can execute distributed optimization by just executing this script from multiple server instances.

package main

import ...

func main() {
    db, _ := gorm.Open(mysql.Open("goptuna:[email protected](localhost:3306)/yourdb?parseTime=true"), &gorm.Config{
        Logger: logger.Default.LogMode(logger.Silent),
    })
    storage := rdb.NewStorage(db)
    defer db.Close()

    study, _ := goptuna.LoadStudy(
        "yourstudy",
        goptuna.StudyOptionStorage(storage),
        ...,
    )
    _ = study.Optimize(objective, 50)
    ...
}

Full source code is available here.

Built-in Realtime Web Dashboard

You can check optimization results by built-in web dashboard.

SQLite3:

$ goptuna dashboard --storage sqlite:///example.db

MySQL:

$ goptuna dashboard --storage mysql://goptuna:[email protected]:3306/yourdb

goptuna dashboard

Shell script to reproduce this (SQLite3 version is here).

Links

References:

Presentations:

Blog posts:

Status:

License

This software is licensed under the MIT license, see LICENSE for more information.

Issues
  • Benchmarking tool like sile/kurobako.

    Benchmarking tool like sile/kurobako.

    It seems to be useful to check the efficiency of optimization algorithm like sile/kurobako.

    • nasbench
      • https://arxiv.org/abs/1902.09635
      • https://github.com/automl/nas_benchmarks
    • sigopt/evalset
      • https://github.com/sigopt/evalset
    • hpobench
      • https://arxiv.org/abs/1905.04970
    opened by c-bata 10
  • goptuna is not compiling using gollvm. asm level bugs caught

    goptuna is not compiling using gollvm. asm level bugs caught

    Hi. I have built and is able to use gollvm.

    When I tried to build your package - I caught such bugs:

    $ go get -u github.com/c-bata/goptuna/tpe

    gonum.org/v1/gonum/internal/asm/f64

    go/src/gonum.org/v1/gonum/internal/asm/f64/abssum_amd64.s: Assembler messages: go/src/gonum.org/v1/gonum/internal/asm/f64/abssum_amd64.s:10: Error: no such instruction: text ·L1Norm(SB),NOSPLIT,$0' go/src/gonum.org/v1/gonum/internal/asm/f64/abssum_amd64.s:11: Error: junk(FP)' after expression go/src/gonum.org/v1/gonum/internal/asm/f64/abssum_amd64.s:11: Error: invalid character '=' in operand 2 go/src/gonum.org/v1/gonum/internal/asm/f64/abssum_amd64.s:12: Error: junk (FP)' after expression go/src/gonum.org/v1/gonum/internal/asm/f64/abssum_amd64.s:12: Error: invalid character '=' in operand 2 go/src/gonum.org/v1/gonum/internal/asm/f64/abssum_amd64.s:13: Error: invalid character '=' in operand 2 go/src/gonum.org/v1/gonum/internal/asm/f64/abssum_amd64.s:14: Error: invalid character '=' in operand 2 go/src/gonum.org/v1/gonum/internal/asm/f64/abssum_amd64.s:15: Error: too many memory references forpxor' go/src/gonum.org/v1/gonum/internal/asm/f64/abssum_amd64.s:16: Error: too many memory references for pxor' go/src/gonum.org/v1/gonum/internal/asm/f64/abssum_amd64.s:17: Error: too many memory references forpxor' go/src/gonum.org/v1/gonum/internal/asm/f64/abssum_amd64.s:18: Error: too many memory references for pxor' go/src/gonum.org/v1/gonum/internal/asm/f64/abssum_amd64.s:19: Error: too many memory references forpxor' go/src/gonum.org/v1/gonum/internal/asm/f64/abssum_amd64.s:20: Error: too many memory references for pxor' go/src/gonum.org/v1/gonum/internal/asm/f64/abssum_amd64.s:21: Error: too many memory references forpxor' go/src/gonum.org/v1/gonum/internal/asm/f64/abssum_amd64.s:22: Error: invalid character '=' in operand 2 go/src/gonum.org/v1/gonum/internal/asm/f64/abssum_amd64.s:24: Error: too many memory references for movq' go/src/gonum.org/v1/gonum/internal/asm/f64/abssum_amd64.s:25: Error: invalid character '=' in operand 2 go/src/gonum.org/v1/gonum/internal/asm/f64/abssum_amd64.s:26: Error: invalid character '=' in operand 2 go/src/gonum.org/v1/gonum/internal/asm/f64/abssum_amd64.s:27: Error: invalid character '=' in operand 1 go/src/gonum.org/v1/gonum/internal/asm/f64/abssum_amd64.s:31: Error: junk(AX8)' after expression go/src/gonum.org/v1/gonum/internal/asm/f64/abssum_amd64.s:31: Error: invalid character '=' in operand 2 go/src/gonum.org/v1/gonum/internal/asm/f64/abssum_amd64.s:32: Error: junk (SI)(AX*8)' after expression go/src/gonum.org/v1/gonum/internal/asm/f64/abssum_amd64.s:32: Error: too many memory references formovups' go/src/gonum.org/v1/gonum/internal/asm/f64/abssum_amd64.s:33: Error: junk (SI)(AX*8)' after expression go/src/gonum.org/v1/gonum/internal/asm/f64/abssum_amd64.s:33: Error: too many memory references formovups' go/src/gonum.org/v1/gonum/internal/asm/f64/abssum_amd64.s:34: Error: junk (SI)(AX*8)' after expression go/src/gonum.org/v1/gonum/internal/asm/f64/abssum_amd64.s:34: Error: too many memory references formovups' go/src/gonum.org/v1/gonum/internal/asm/f64/abssum_amd64.s:35: Error: invalid character '=' in operand 2 go/src/gonum.org/v1/gonum/internal/asm/f64/abssum_amd64.s:36: Error: too many memory references for addpd' go/src/gonum.org/v1/gonum/internal/asm/f64/abssum_amd64.s:37: Error: too many memory references foraddpd' go/src/gonum.org/v1/gonum/internal/asm/f64/abssum_amd64.s:38: Error: too many memory references for addpd' go/src/gonum.org/v1/gonum/internal/asm/f64/abssum_amd64.s:39: Error: invalid character '=' in operand 2 go/src/gonum.org/v1/gonum/internal/asm/f64/abssum_amd64.s:40: Error: too many memory references forsubpd' go/src/gonum.org/v1/gonum/internal/asm/f64/abssum_amd64.s:41: Error: too many memory references for subpd' go/src/gonum.org/v1/gonum/internal/asm/f64/abssum_amd64.s:42: Error: too many memory references forsubpd' go/src/gonum.org/v1/gonum/internal/asm/f64/abssum_amd64.s:43: Error: invalid character '=' in operand 2 go/src/gonum.org/v1/gonum/internal/asm/f64/abssum_amd64.s:44: Error: too many memory references for maxpd' go/src/gonum.org/v1/gonum/internal/asm/f64/abssum_amd64.s:45: Error: too many memory references formaxpd' go/src/gonum.org/v1/gonum/internal/asm/f64/abssum_amd64.s:46: Error: too many memory references for maxpd' go/src/gonum.org/v1/gonum/internal/asm/f64/abssum_amd64.s:47: Error: invalid character '=' in operand 2 go/src/gonum.org/v1/gonum/internal/asm/f64/abssum_amd64.s:48: Error: too many memory references formovaps' go/src/gonum.org/v1/gonum/internal/asm/f64/abssum_amd64.s:49: Error: too many memory references for movaps' go/src/gonum.org/v1/gonum/internal/asm/f64/abssum_amd64.s:50: Error: too many memory references formovaps' go/src/gonum.org/v1/gonum/internal/asm/f64/abssum_amd64.s:51: Error: invalid character '=' in operand 2 go/src/gonum.org/v1/gonum/internal/asm/f64/abssum_amd64.s:52: Error: bad expression go/src/gonum.org/v1/gonum/internal/asm/f64/abssum_amd64.s:52: Error: junk } while --CX>0' after expression go/src/gonum.org/v1/gonum/internal/asm/f64/abssum_amd64.s:55: Error: too many memory references foraddpd' go/src/gonum.org/v1/gonum/internal/asm/f64/abssum_amd64.s:56: Error: too many memory references for addpd' go/src/gonum.org/v1/gonum/internal/asm/f64/abssum_amd64.s:57: Error: too many memory references foraddpd' go/src/gonum.org/v1/gonum/internal/asm/f64/abssum_amd64.s:60: Error: too many memory references for movaps' go/src/gonum.org/v1/gonum/internal/asm/f64/abssum_amd64.s:61: Error: invalid character '=' in operand 3 go/src/gonum.org/v1/gonum/internal/asm/f64/abssum_amd64.s:62: Error: too many memory references foraddsd' go/src/gonum.org/v1/gonum/internal/asm/f64/abssum_amd64.s:63: Error: operand size mismatch for cmp' go/src/gonum.org/v1/gonum/internal/asm/f64/abssum_amd64.s:64: Error: invalid character '=' in operand 1 go/src/gonum.org/v1/gonum/internal/asm/f64/abssum_amd64.s:67: Error: invalid character '=' in operand 2 go/src/gonum.org/v1/gonum/internal/asm/f64/abssum_amd64.s:68: Error: invalid character '=' in operand 2 go/src/gonum.org/v1/gonum/internal/asm/f64/abssum_amd64.s:72: Error: junk(AX8)' after expression go/src/gonum.org/v1/gonum/internal/asm/f64/abssum_amd64.s:72: Error: invalid character '=' in operand 2 go/src/gonum.org/v1/gonum/internal/asm/f64/abssum_amd64.s:73: Error: invalid character '=' in operand 2 go/src/gonum.org/v1/gonum/internal/asm/f64/abssum_amd64.s:74: Error: invalid character '=' in operand 2 go/src/gonum.org/v1/gonum/internal/asm/f64/abssum_amd64.s:75: Error: invalid character '=' in operand 2 go/src/gonum.org/v1/gonum/internal/asm/f64/abssum_amd64.s:76: Error: invalid character '=' in operand 2 go/src/gonum.org/v1/gonum/internal/asm/f64/abssum_amd64.s:77: Error: bad expression go/src/gonum.org/v1/gonum/internal/asm/f64/abssum_amd64.s:77: Error: junk i++' after expression go/src/gonum.org/v1/gonum/internal/asm/f64/abssum_amd64.s:78: Error: bad expression go/src/gonum.org/v1/gonum/internal/asm/f64/abssum_amd64.s:78: Error: junk} while --CX>0' after expression go/src/gonum.org/v1/gonum/internal/asm/f64/abssum_amd64.s:81: Warning: X0' is not valid here (expected(%rsi)') go/src/gonum.org/v1/gonum/internal/asm/f64/abssum_amd64.s:81: Error: junk (FP)' after expression go/src/gonum.org/v1/gonum/internal/asm/f64/abssum_amd64.s:81: Warning:sum+24(FP)' is not valid here (expected (%rdi)') go/src/gonum.org/v1/gonum/internal/asm/f64/abssum_amd64.s:52: Error: invalid operands (.text and *ABS* sections) for/' go/src/gonum.org/v1/gonum/internal/asm/f64/abssum_amd64.s:52: Error: division by zero go/src/gonum.org/v1/gonum/internal/asm/f64/abssum_amd64.s:77: Error: invalid operands (UND and ABS sections) for /' go/src/gonum.org/v1/gonum/internal/asm/f64/abssum_amd64.s:77: Error: division by zero go/src/gonum.org/v1/gonum/internal/asm/f64/abssum_amd64.s:78: Error: invalid operands (.text and *ABS* sections) for/' go/src/gonum.org/v1/gonum/internal/asm/f64/abssum_amd64.s:78: Error: division by zero

    It seems that your asm mix-ins are not suitable to LLVM oriented back-end. Or I am wrong? In any case: I suspect that it should be re-designed, to work with gollvm.

    Ivan

    opened by advancedwebdeveloper 5
  • Update c-bata/github-actions-kurobako action to v4 - autoclosed

    Update c-bata/github-actions-kurobako action to v4 - autoclosed

    WhiteSource Renovate

    This PR contains the following updates:

    | Package | Type | Update | Change | |---|---|---|---| | c-bata/github-actions-kurobako | action | major | v2 -> v4 | | c-bata/github-actions-kurobako | action | major | v3 -> v4 |


    Release Notes

    c-bata/github-actions-kurobako

    v4

    Compare Source

    v3

    Compare Source


    Renovate configuration

    :date: Schedule: At any time (no schedule defined).

    :vertical_traffic_light: Automerge: Disabled by config. Please merge this manually once you are satisfied.

    :recycle: Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

    :no_bell: Ignore: Close this PR and you won't be reminded about these updates again.


    • [ ] If you want to rebase/retry this PR, check this box

    This PR has been generated by WhiteSource Renovate. View repository job log here.

    opened by renovate[bot] 3
  • Faster TPE categorical sampling

    Faster TPE categorical sampling

    Refs https://github.com/optuna/optuna/pull/1603

    • Motivation: The cardinality of categorical parameters tends to be lower than NEICandidates (default: 24).
    • Description of the changes: Use [0, 1, 2, 3, ...] instead of sample from l(x) distribution.
    opened by c-bata 3
  • Support IPOP-CMA-ES algorithm.

    Support IPOP-CMA-ES algorithm.

    opened by c-bata 3
  • Fix a bug of successive halving pruner and alter pruner API more Go-ish.

    Fix a bug of successive halving pruner and alter pruner API more Go-ish.

    Summary

    In this PR, I added an example which uses successive halving pruner, then I found a lot of issues like pruners haven't worked ever. This PR contains a few breaking changes around pruner API. But I merge this without backward compatibility because I'm sure that nobody uses pruning functionalities.

    • [x] Use successive halving pruner on gorgonia_iris.
    • [x] Alter pruner API more Go-ish.
    • [x] Fix a typo of successive halving pruner option.
    • [x] Fix a bug of successive halving pruner.

    Checklist

    • [x] I confirmed that Kubeflow/Katib does not uses pruner API.
    • [x] I confirmed that sile/kurobako-go does not uses pruner API.

    Example

    $ go run _examples/gorgonia_iris/main.go -dataset ./_examples/gorgonia_iris/iris.csv 
    2020/07/29 04:13:59 [INFO] Trial finished: trialID=0 state=Complete evaluation=0.000000
    2020/07/29 04:14:01 [INFO] Trial finished: trialID=1 state=Complete evaluation=0.666667
    2020/07/29 04:14:03 [INFO] Trial finished: trialID=2 state=Complete evaluation=0.960000
    2020/07/29 04:14:03 [INFO] Trial finished: trialID=3 state=Pruned evaluation=0.000000
    2020/07/29 04:14:03 [INFO] Trial finished: trialID=4 state=Pruned evaluation=0.093333
    2020/07/29 04:14:03 [INFO] Trial finished: trialID=5 state=Pruned evaluation=0.700000
    2020/07/29 04:14:03 [INFO] Trial finished: trialID=6 state=Pruned evaluation=0.306667
    2020/07/29 04:14:03 [INFO] Trial finished: trialID=7 state=Pruned evaluation=0.260000
    2020/07/29 04:14:03 [INFO] Trial finished: trialID=8 state=Pruned evaluation=0.253333
    2020/07/29 04:14:03 [INFO] Trial finished: trialID=9 state=Pruned evaluation=0.953333
    2020/07/29 04:14:03 [INFO] Trial finished: trialID=10 state=Pruned evaluation=0.000000
    2020/07/29 04:14:03 [INFO] Trial finished: trialID=11 state=Pruned evaluation=0.833333
    2020/07/29 04:14:03 [INFO] Trial finished: trialID=12 state=Pruned evaluation=0.000000
    2020/07/29 04:14:03 [INFO] Trial finished: trialID=13 state=Pruned evaluation=0.233333
    2020/07/29 04:14:03 [INFO] Trial finished: trialID=14 state=Pruned evaluation=0.266667
    2020/07/29 04:14:03 [INFO] Trial finished: trialID=15 state=Pruned evaluation=0.580000
    2020/07/29 04:14:05 [INFO] Trial finished: trialID=16 state=Complete evaluation=0.966667
    2020/07/29 04:14:05 [INFO] Trial finished: trialID=17 state=Pruned evaluation=0.693333
    2020/07/29 04:14:05 [INFO] Trial finished: trialID=18 state=Pruned evaluation=0.253333
    2020/07/29 04:14:05 [INFO] Trial finished: trialID=19 state=Pruned evaluation=0.746667
    2020/07/29 04:14:05 [INFO] Trial finished: trialID=20 state=Pruned evaluation=0.433333
    2020/07/29 04:14:05 [INFO] Trial finished: trialID=21 state=Pruned evaluation=0.553333
    2020/07/29 04:14:07 [INFO] Trial finished: trialID=22 state=Complete evaluation=0.966667
    2020/07/29 04:14:07 [INFO] Trial finished: trialID=23 state=Pruned evaluation=0.306667
    2020/07/29 04:14:07 [INFO] Trial finished: trialID=24 state=Pruned evaluation=0.000000
    2020/07/29 04:14:07 [INFO] Trial finished: trialID=25 state=Pruned evaluation=0.000000
    2020/07/29 04:14:07 [INFO] Trial finished: trialID=26 state=Pruned evaluation=0.313333
    2020/07/29 04:14:07 [INFO] Trial finished: trialID=27 state=Pruned evaluation=0.153333
    2020/07/29 04:14:07 [INFO] Trial finished: trialID=28 state=Pruned evaluation=0.000000
    2020/07/29 04:14:07 [INFO] Trial finished: trialID=29 state=Pruned evaluation=0.233333
    2020/07/29 04:14:07 [INFO] Trial finished: trialID=30 state=Pruned evaluation=0.000000
    2020/07/29 04:14:07 [INFO] Trial finished: trialID=31 state=Pruned evaluation=0.706667
    2020/07/29 04:14:07 [INFO] Trial finished: trialID=32 state=Pruned evaluation=0.333333
    2020/07/29 04:14:07 [INFO] Trial finished: trialID=33 state=Pruned evaluation=0.000000
    2020/07/29 04:14:08 [INFO] Trial finished: trialID=34 state=Pruned evaluation=0.733333
    2020/07/29 04:14:08 [INFO] Trial finished: trialID=35 state=Pruned evaluation=0.793333
    2020/07/29 04:14:08 [INFO] Trial finished: trialID=36 state=Pruned evaluation=0.960000
    2020/07/29 04:14:08 [INFO] Trial finished: trialID=37 state=Pruned evaluation=0.506667
    2020/07/29 04:14:08 [INFO] Trial finished: trialID=38 state=Pruned evaluation=0.873333
    2020/07/29 04:14:08 [INFO] Trial finished: trialID=39 state=Pruned evaluation=0.726667
    2020/07/29 04:14:08 [INFO] Trial finished: trialID=40 state=Pruned evaluation=0.100000
    2020/07/29 04:14:08 [INFO] Trial finished: trialID=41 state=Pruned evaluation=0.000000
    2020/07/29 04:14:08 [INFO] Trial finished: trialID=42 state=Pruned evaluation=0.073333
    2020/07/29 04:14:08 [INFO] Trial finished: trialID=43 state=Pruned evaluation=0.000000
    2020/07/29 04:14:08 [INFO] Trial finished: trialID=44 state=Pruned evaluation=0.926667
    2020/07/29 04:14:08 [INFO] Trial finished: trialID=45 state=Pruned evaluation=0.033333
    2020/07/29 04:14:08 [INFO] Trial finished: trialID=46 state=Pruned evaluation=0.000000
    2020/07/29 04:14:08 [INFO] Trial finished: trialID=47 state=Pruned evaluation=0.000000
    2020/07/29 04:14:08 [INFO] Trial finished: trialID=48 state=Pruned evaluation=0.000000
    2020/07/29 04:14:08 [INFO] Trial finished: trialID=49 state=Pruned evaluation=0.000000
    2020/07/29 04:14:10 [INFO] Trial finished: trialID=50 state=Complete evaluation=0.966667
    2020/07/29 04:14:10 [INFO] Trial finished: trialID=51 state=Pruned evaluation=0.426667
    2020/07/29 04:14:10 [INFO] Trial finished: trialID=52 state=Pruned evaluation=0.000000
    2020/07/29 04:14:10 [INFO] Trial finished: trialID=53 state=Pruned evaluation=0.000000
    2020/07/29 04:14:10 [INFO] Trial finished: trialID=54 state=Pruned evaluation=0.426667
    2020/07/29 04:14:10 [INFO] Trial finished: trialID=55 state=Pruned evaluation=0.406667
    2020/07/29 04:14:10 [INFO] Trial finished: trialID=56 state=Pruned evaluation=0.313333
    2020/07/29 04:14:10 [INFO] Trial finished: trialID=57 state=Pruned evaluation=0.946667
    2020/07/29 04:14:10 [INFO] Trial finished: trialID=58 state=Pruned evaluation=0.000000
    2020/07/29 04:14:10 [INFO] Trial finished: trialID=59 state=Pruned evaluation=0.000000
    2020/07/29 04:14:10 [INFO] Trial finished: trialID=60 state=Pruned evaluation=0.220000
    2020/07/29 04:14:10 [INFO] Trial finished: trialID=61 state=Pruned evaluation=0.393333
    2020/07/29 04:14:10 [INFO] Trial finished: trialID=62 state=Pruned evaluation=0.646667
    2020/07/29 04:14:10 [INFO] Trial finished: trialID=63 state=Pruned evaluation=0.000000
    2020/07/29 04:14:10 [INFO] Trial finished: trialID=64 state=Pruned evaluation=0.853333
    2020/07/29 04:14:10 [INFO] Trial finished: trialID=65 state=Pruned evaluation=0.000000
    2020/07/29 04:14:10 [INFO] Trial finished: trialID=66 state=Pruned evaluation=0.313333
    2020/07/29 04:14:10 [INFO] Trial finished: trialID=67 state=Pruned evaluation=0.693333
    2020/07/29 04:14:10 [INFO] Trial finished: trialID=68 state=Pruned evaluation=0.000000
    2020/07/29 04:14:11 [INFO] Trial finished: trialID=69 state=Pruned evaluation=0.900000
    2020/07/29 04:14:11 [INFO] Trial finished: trialID=70 state=Pruned evaluation=0.280000
    2020/07/29 04:14:11 [INFO] Trial finished: trialID=71 state=Pruned evaluation=0.000000
    2020/07/29 04:14:11 [INFO] Trial finished: trialID=72 state=Pruned evaluation=0.000000
    2020/07/29 04:14:11 [INFO] Trial finished: trialID=73 state=Pruned evaluation=0.560000
    2020/07/29 04:14:11 [INFO] Trial finished: trialID=74 state=Pruned evaluation=0.400000
    2020/07/29 04:14:11 [INFO] Trial finished: trialID=75 state=Pruned evaluation=0.000000
    2020/07/29 04:14:11 [INFO] Trial finished: trialID=76 state=Pruned evaluation=0.686667
    2020/07/29 04:14:11 [INFO] Trial finished: trialID=77 state=Pruned evaluation=0.746667
    2020/07/29 04:14:11 [INFO] Trial finished: trialID=78 state=Pruned evaluation=0.306667
    2020/07/29 04:14:11 [INFO] Trial finished: trialID=79 state=Pruned evaluation=0.000000
    2020/07/29 04:14:11 [INFO] Trial finished: trialID=80 state=Pruned evaluation=0.000000
    2020/07/29 04:14:11 [INFO] Trial finished: trialID=81 state=Pruned evaluation=0.746667
    2020/07/29 04:14:11 [INFO] Trial finished: trialID=82 state=Pruned evaluation=0.346667
    2020/07/29 04:14:11 [INFO] Trial finished: trialID=83 state=Pruned evaluation=0.160000
    2020/07/29 04:14:11 [INFO] Trial finished: trialID=84 state=Pruned evaluation=0.780000
    2020/07/29 04:14:11 [INFO] Trial finished: trialID=85 state=Pruned evaluation=0.000000
    2020/07/29 04:14:11 [INFO] Trial finished: trialID=86 state=Pruned evaluation=0.306667
    2020/07/29 04:14:11 [INFO] Trial finished: trialID=87 state=Pruned evaluation=0.000000
    2020/07/29 04:14:11 [INFO] Trial finished: trialID=88 state=Pruned evaluation=0.060000
    2020/07/29 04:14:11 [INFO] Trial finished: trialID=89 state=Pruned evaluation=0.906667
    2020/07/29 04:14:11 [INFO] Trial finished: trialID=90 state=Pruned evaluation=0.333333
    2020/07/29 04:14:11 [INFO] Trial finished: trialID=91 state=Pruned evaluation=0.333333
    2020/07/29 04:14:11 [INFO] Trial finished: trialID=92 state=Pruned evaluation=0.960000
    2020/07/29 04:14:11 [INFO] Trial finished: trialID=93 state=Pruned evaluation=0.000000
    2020/07/29 04:14:11 [INFO] Trial finished: trialID=94 state=Pruned evaluation=0.000000
    2020/07/29 04:14:14 [INFO] Trial finished: trialID=95 state=Complete evaluation=0.973333
    2020/07/29 04:14:14 [INFO] Trial finished: trialID=96 state=Pruned evaluation=0.000000
    2020/07/29 04:14:14 [INFO] Trial finished: trialID=97 state=Pruned evaluation=0.713333
    2020/07/29 04:14:14 [INFO] Trial finished: trialID=98 state=Pruned evaluation=0.000000
    2020/07/29 04:14:14 [INFO] Trial finished: trialID=99 state=Pruned evaluation=0.400000
    2020/07/29 04:14:14 [INFO] Trial finished: trialID=100 state=Pruned evaluation=0.846667
    2020/07/29 04:14:14 [INFO] Trial finished: trialID=101 state=Pruned evaluation=0.946667
    2020/07/29 04:14:14 [INFO] Trial finished: trialID=102 state=Pruned evaluation=0.013333
    2020/07/29 04:14:14 [INFO] Trial finished: trialID=103 state=Pruned evaluation=0.586667
    2020/07/29 04:14:14 [INFO] Trial finished: trialID=104 state=Pruned evaluation=0.173333
    2020/07/29 04:14:14 [INFO] Trial finished: trialID=105 state=Pruned evaluation=0.713333
    2020/07/29 04:14:14 [INFO] Trial finished: trialID=106 state=Pruned evaluation=0.520000
    2020/07/29 04:14:14 [INFO] Trial finished: trialID=107 state=Pruned evaluation=0.173333
    2020/07/29 04:14:14 [INFO] Trial finished: trialID=108 state=Pruned evaluation=0.000000
    2020/07/29 04:14:14 [INFO] Trial finished: trialID=109 state=Pruned evaluation=0.806667
    2020/07/29 04:14:14 [INFO] Trial finished: trialID=110 state=Pruned evaluation=0.820000
    2020/07/29 04:14:14 [INFO] Trial finished: trialID=111 state=Pruned evaluation=0.000000
    2020/07/29 04:14:14 [INFO] Trial finished: trialID=112 state=Pruned evaluation=0.000000
    2020/07/29 04:14:14 [INFO] Trial finished: trialID=113 state=Pruned evaluation=0.000000
    2020/07/29 04:14:14 [INFO] Trial finished: trialID=114 state=Pruned evaluation=0.526667
    2020/07/29 04:14:14 [INFO] Trial finished: trialID=115 state=Pruned evaluation=0.260000
    2020/07/29 04:14:14 [INFO] Trial finished: trialID=116 state=Pruned evaluation=0.000000
    2020/07/29 04:14:14 [INFO] Trial finished: trialID=117 state=Pruned evaluation=0.946667
    2020/07/29 04:14:14 [INFO] Trial finished: trialID=118 state=Pruned evaluation=0.273333
    2020/07/29 04:14:14 [INFO] Trial finished: trialID=119 state=Pruned evaluation=0.860000
    2020/07/29 04:14:14 [INFO] Trial finished: trialID=120 state=Pruned evaluation=0.000000
    2020/07/29 04:14:14 [INFO] Trial finished: trialID=121 state=Pruned evaluation=0.000000
    2020/07/29 04:14:14 [INFO] Trial finished: trialID=122 state=Pruned evaluation=0.506667
    2020/07/29 04:14:14 [INFO] Trial finished: trialID=123 state=Pruned evaluation=0.326667
    2020/07/29 04:14:14 [INFO] Trial finished: trialID=124 state=Pruned evaluation=0.720000
    2020/07/29 04:14:14 [INFO] Trial finished: trialID=125 state=Pruned evaluation=0.280000
    2020/07/29 04:14:14 [INFO] Trial finished: trialID=126 state=Pruned evaluation=0.266667
    2020/07/29 04:14:14 [INFO] Trial finished: trialID=127 state=Pruned evaluation=0.000000
    2020/07/29 04:14:14 [INFO] Trial finished: trialID=128 state=Pruned evaluation=0.080000
    2020/07/29 04:14:14 [INFO] Trial finished: trialID=129 state=Pruned evaluation=0.426667
    2020/07/29 04:14:14 [INFO] Trial finished: trialID=130 state=Pruned evaluation=0.793333
    2020/07/29 04:14:14 [INFO] Trial finished: trialID=131 state=Pruned evaluation=0.026667
    2020/07/29 04:14:14 [INFO] Trial finished: trialID=132 state=Pruned evaluation=0.946667
    2020/07/29 04:14:14 [INFO] Trial finished: trialID=133 state=Pruned evaluation=0.793333
    2020/07/29 04:14:14 [INFO] Trial finished: trialID=134 state=Pruned evaluation=0.926667
    2020/07/29 04:14:14 [INFO] Trial finished: trialID=135 state=Pruned evaluation=0.800000
    2020/07/29 04:14:14 [INFO] Trial finished: trialID=136 state=Pruned evaluation=0.766667
    2020/07/29 04:14:15 [INFO] Trial finished: trialID=137 state=Pruned evaluation=0.726667
    2020/07/29 04:14:15 [INFO] Trial finished: trialID=138 state=Pruned evaluation=0.900000
    2020/07/29 04:14:15 [INFO] Trial finished: trialID=139 state=Pruned evaluation=0.153333
    2020/07/29 04:14:15 [INFO] Trial finished: trialID=140 state=Pruned evaluation=0.900000
    2020/07/29 04:14:15 [INFO] Trial finished: trialID=141 state=Pruned evaluation=0.893333
    2020/07/29 04:14:15 [INFO] Trial finished: trialID=142 state=Pruned evaluation=0.766667
    2020/07/29 04:14:15 [INFO] Trial finished: trialID=143 state=Pruned evaluation=0.373333
    2020/07/29 04:14:15 [INFO] Trial finished: trialID=144 state=Pruned evaluation=0.513333
    2020/07/29 04:14:15 [INFO] Trial finished: trialID=145 state=Pruned evaluation=0.580000
    2020/07/29 04:14:15 [INFO] Trial finished: trialID=146 state=Pruned evaluation=0.733333
    2020/07/29 04:14:15 [INFO] Trial finished: trialID=147 state=Pruned evaluation=0.866667
    2020/07/29 04:14:15 [INFO] Trial finished: trialID=148 state=Pruned evaluation=0.000000
    2020/07/29 04:14:15 [INFO] Trial finished: trialID=149 state=Pruned evaluation=0.960000
    2020/07/29 04:14:18 [INFO] Trial finished: trialID=150 state=Complete evaluation=0.973333
    2020/07/29 04:14:18 [INFO] Trial finished: trialID=151 state=Pruned evaluation=0.120000
    2020/07/29 04:14:18 [INFO] Trial finished: trialID=152 state=Pruned evaluation=0.833333
    2020/07/29 04:14:18 [INFO] Trial finished: trialID=153 state=Pruned evaluation=0.860000
    2020/07/29 04:14:18 [INFO] Trial finished: trialID=154 state=Pruned evaluation=0.266667
    2020/07/29 04:14:18 [INFO] Trial finished: trialID=155 state=Pruned evaluation=0.420000
    2020/07/29 04:14:18 [INFO] Trial finished: trialID=156 state=Pruned evaluation=0.366667
    2020/07/29 04:14:18 [INFO] Trial finished: trialID=157 state=Pruned evaluation=0.600000
    2020/07/29 04:14:18 [INFO] Trial finished: trialID=158 state=Pruned evaluation=0.960000
    2020/07/29 04:14:18 [INFO] Trial finished: trialID=159 state=Pruned evaluation=0.280000
    2020/07/29 04:14:18 [INFO] Trial finished: trialID=160 state=Pruned evaluation=0.886667
    2020/07/29 04:14:18 [INFO] Trial finished: trialID=161 state=Pruned evaluation=0.393333
    2020/07/29 04:14:18 [INFO] Trial finished: trialID=162 state=Pruned evaluation=0.826667
    2020/07/29 04:14:18 [INFO] Trial finished: trialID=163 state=Pruned evaluation=0.840000
    2020/07/29 04:14:19 [INFO] Trial finished: trialID=164 state=Pruned evaluation=0.940000
    2020/07/29 04:14:19 [INFO] Trial finished: trialID=165 state=Pruned evaluation=0.000000
    2020/07/29 04:14:19 [INFO] Trial finished: trialID=166 state=Pruned evaluation=0.313333
    2020/07/29 04:14:19 [INFO] Trial finished: trialID=167 state=Pruned evaluation=0.120000
    2020/07/29 04:14:19 [INFO] Trial finished: trialID=168 state=Pruned evaluation=0.000000
    2020/07/29 04:14:19 [INFO] Trial finished: trialID=169 state=Pruned evaluation=0.000000
    2020/07/29 04:14:19 [INFO] Trial finished: trialID=170 state=Pruned evaluation=0.686667
    2020/07/29 04:14:19 [INFO] Trial finished: trialID=171 state=Pruned evaluation=0.353333
    2020/07/29 04:14:19 [INFO] Trial finished: trialID=172 state=Pruned evaluation=0.113333
    2020/07/29 04:14:19 [INFO] Trial finished: trialID=173 state=Pruned evaluation=0.233333
    2020/07/29 04:14:19 [INFO] Trial finished: trialID=174 state=Pruned evaluation=0.440000
    2020/07/29 04:14:19 [INFO] Trial finished: trialID=175 state=Pruned evaluation=0.873333
    2020/07/29 04:14:19 [INFO] Trial finished: trialID=176 state=Pruned evaluation=0.906667
    2020/07/29 04:14:19 [INFO] Trial finished: trialID=177 state=Pruned evaluation=0.806667
    2020/07/29 04:14:19 [INFO] Trial finished: trialID=178 state=Pruned evaluation=0.173333
    2020/07/29 04:14:19 [INFO] Trial finished: trialID=179 state=Pruned evaluation=0.960000
    2020/07/29 04:14:19 [INFO] Trial finished: trialID=180 state=Pruned evaluation=0.680000
    2020/07/29 04:14:19 [INFO] Trial finished: trialID=181 state=Pruned evaluation=0.000000
    2020/07/29 04:14:19 [INFO] Trial finished: trialID=182 state=Pruned evaluation=0.593333
    2020/07/29 04:14:20 [INFO] Trial finished: trialID=183 state=Pruned evaluation=0.960000
    2020/07/29 04:14:20 [INFO] Trial finished: trialID=184 state=Pruned evaluation=0.620000
    2020/07/29 04:14:20 [INFO] Trial finished: trialID=185 state=Pruned evaluation=0.333333
    2020/07/29 04:14:20 [INFO] Trial finished: trialID=186 state=Pruned evaluation=0.953333
    2020/07/29 04:14:22 [INFO] Trial finished: trialID=187 state=Complete evaluation=0.973333
    2020/07/29 04:14:22 [INFO] Trial finished: trialID=188 state=Pruned evaluation=0.373333
    2020/07/29 04:14:22 [INFO] Trial finished: trialID=189 state=Pruned evaluation=0.533333
    2020/07/29 04:14:22 [INFO] Trial finished: trialID=190 state=Pruned evaluation=0.886667
    2020/07/29 04:14:22 [INFO] Trial finished: trialID=191 state=Pruned evaluation=0.826667
    2020/07/29 04:14:22 [INFO] Trial finished: trialID=192 state=Pruned evaluation=0.686667
    2020/07/29 04:14:22 [INFO] Trial finished: trialID=193 state=Pruned evaluation=0.806667
    2020/07/29 04:14:23 [INFO] Trial finished: trialID=194 state=Pruned evaluation=0.960000
    2020/07/29 04:14:23 [INFO] Trial finished: trialID=195 state=Pruned evaluation=0.000000
    2020/07/29 04:14:23 [INFO] Trial finished: trialID=196 state=Pruned evaluation=0.500000
    2020/07/29 04:14:23 [INFO] Trial finished: trialID=197 state=Pruned evaluation=0.866667
    2020/07/29 04:14:23 [INFO] Trial finished: trialID=198 state=Pruned evaluation=0.940000
    2020/07/29 04:14:23 [INFO] Trial finished: trialID=199 state=Pruned evaluation=0.000000
    2020/07/29 04:14:23 Best evaluation=0.973333
    2020/07/29 04:14:23 Solver: Vanilla
    2020/07/29 04:14:23 Learning rate (vanilla): 0.001816
    
    
    bug examples 
    opened by c-bata 3
  • Add BlackHoleStorage towards 100k+ evaluations

    Add BlackHoleStorage towards 100k+ evaluations

    Example

    package main
    
    import (
    	"log"
    	"math"
    
    	"github.com/c-bata/goptuna"
    	"github.com/c-bata/goptuna/cmaes"
    )
    
    func objective(trial goptuna.Trial) (float64, error) {
    	x1, _ := trial.SuggestFloat("x1", -10, 10)
    	x2, _ := trial.SuggestFloat("x2", -10, 10)
    	return math.Pow(x1-2, 2) + math.Pow(x2+5, 2), nil
    }
    
    func main() {
    	relativeSampler := cmaes.NewSampler(
    		cmaes.SamplerOptionNStartupTrials(5))
    	study, err := goptuna.CreateStudy(
    		"goptuna-example",
    		goptuna.StudyOptionStorage(goptuna.NewBlackholeStorage(20)),
    		goptuna.StudyOptionRelativeSampler(relativeSampler),
    	)
    	if err != nil {
    		log.Fatal("failed to create study:", err)
    	}
    
    	if err = study.Optimize(objective, 200); err != nil {
    		log.Fatal("failed to optimize:", err)
    	}
    
    	v, err := study.GetBestValue()
    	if err != nil {
    		log.Fatal("failed to get best value:", err)
    	}
    	params, err := study.GetBestParams()
    	if err != nil {
    		log.Fatal("failed to get best params:", err)
    	}
    	log.Printf("Best evaluation=%f (x1=%f, x2=%f)",
    		v, params["x1"].(float64), params["x2"].(float64))
    }
    
    (venv) $ go run _examples/cmaes/main.go 
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=0 state=Complete evaluation=47.674293
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=1 state=Complete evaluation=16.564975
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=2 state=Complete evaluation=22.229764
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=3 state=Complete evaluation=132.160176
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=4 state=Complete evaluation=38.056051
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=5 state=Complete evaluation=46.057644
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=6 state=Complete evaluation=22.140480
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=7 state=Complete evaluation=62.531182
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=8 state=Complete evaluation=103.705104
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=9 state=Complete evaluation=27.084488
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=10 state=Complete evaluation=33.224279
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=11 state=Complete evaluation=32.438836
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=12 state=Complete evaluation=62.485697
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=13 state=Complete evaluation=10.835063
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=14 state=Complete evaluation=61.231569
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=15 state=Complete evaluation=18.656672
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=16 state=Complete evaluation=20.223998
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=17 state=Complete evaluation=14.679579
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=18 state=Complete evaluation=2.867698
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=19 state=Complete evaluation=22.586033
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=20 state=Complete evaluation=44.850342
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=21 state=Complete evaluation=12.376161
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=22 state=Complete evaluation=14.530727
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=23 state=Complete evaluation=0.252565
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=24 state=Complete evaluation=23.930059
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=25 state=Complete evaluation=18.707611
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=26 state=Complete evaluation=23.723537
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=27 state=Complete evaluation=60.388024
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=28 state=Complete evaluation=49.107782
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=29 state=Complete evaluation=20.578979
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=30 state=Complete evaluation=17.343054
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=31 state=Complete evaluation=15.230447
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=32 state=Complete evaluation=0.688558
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=33 state=Complete evaluation=29.037456
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=34 state=Complete evaluation=30.733836
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=35 state=Complete evaluation=3.108564
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=36 state=Complete evaluation=15.524595
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=37 state=Complete evaluation=54.670613
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=38 state=Complete evaluation=5.800003
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=39 state=Complete evaluation=8.017986
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=40 state=Complete evaluation=12.064258
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=41 state=Complete evaluation=20.230681
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=42 state=Complete evaluation=1.552530
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=43 state=Complete evaluation=12.454509
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=44 state=Complete evaluation=14.703087
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=45 state=Complete evaluation=44.842796
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=46 state=Complete evaluation=28.649740
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=47 state=Complete evaluation=7.217258
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=48 state=Complete evaluation=0.151439
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=49 state=Complete evaluation=2.177288
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=50 state=Complete evaluation=1.167271
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=51 state=Complete evaluation=5.765259
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=52 state=Complete evaluation=0.662816
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=53 state=Complete evaluation=7.699746
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=54 state=Complete evaluation=9.101934
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=55 state=Complete evaluation=5.414616
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=56 state=Complete evaluation=4.842222
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=57 state=Complete evaluation=0.617716
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=58 state=Complete evaluation=17.855878
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=59 state=Complete evaluation=1.285338
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=60 state=Complete evaluation=0.894557
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=61 state=Complete evaluation=1.662849
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=62 state=Complete evaluation=2.419421
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=63 state=Complete evaluation=7.075800
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=64 state=Complete evaluation=0.977344
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=65 state=Complete evaluation=2.240027
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=66 state=Complete evaluation=4.759263
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=67 state=Complete evaluation=0.153983
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=68 state=Complete evaluation=0.658265
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=69 state=Complete evaluation=1.020960
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=70 state=Complete evaluation=0.569522
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=71 state=Complete evaluation=0.539054
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=72 state=Complete evaluation=2.391731
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=73 state=Complete evaluation=1.262207
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=74 state=Complete evaluation=1.410689
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=75 state=Complete evaluation=0.490438
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=76 state=Complete evaluation=1.446373
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=77 state=Complete evaluation=0.214436
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=78 state=Complete evaluation=0.707316
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=79 state=Complete evaluation=0.888024
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=80 state=Complete evaluation=1.500627
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=81 state=Complete evaluation=0.128145
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=82 state=Complete evaluation=0.170781
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=83 state=Complete evaluation=0.141563
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=84 state=Complete evaluation=0.153672
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=85 state=Complete evaluation=0.532152
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=86 state=Complete evaluation=1.286757
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=87 state=Complete evaluation=0.359358
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=88 state=Complete evaluation=0.152523
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=89 state=Complete evaluation=0.296147
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=90 state=Complete evaluation=0.261162
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=91 state=Complete evaluation=0.301497
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=92 state=Complete evaluation=0.064521
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=93 state=Complete evaluation=0.691118
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=94 state=Complete evaluation=0.070931
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=95 state=Complete evaluation=0.301691
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=96 state=Complete evaluation=0.078933
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=97 state=Complete evaluation=0.563108
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=98 state=Complete evaluation=0.045807
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=99 state=Complete evaluation=0.327774
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=100 state=Complete evaluation=0.009999
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=101 state=Complete evaluation=0.148794
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=102 state=Complete evaluation=0.018099
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=103 state=Complete evaluation=0.136851
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=104 state=Complete evaluation=0.042859
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=105 state=Complete evaluation=0.282536
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=106 state=Complete evaluation=0.125910
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=107 state=Complete evaluation=0.003086
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=108 state=Complete evaluation=0.124410
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=109 state=Complete evaluation=0.072585
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=110 state=Complete evaluation=0.000829
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=111 state=Complete evaluation=0.008112
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=112 state=Complete evaluation=0.058168
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=113 state=Complete evaluation=0.005940
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=114 state=Complete evaluation=0.030128
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=115 state=Complete evaluation=0.029122
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=116 state=Complete evaluation=0.125034
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=117 state=Complete evaluation=0.034411
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=118 state=Complete evaluation=0.077104
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=119 state=Complete evaluation=0.013749
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=120 state=Complete evaluation=0.000399
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=121 state=Complete evaluation=0.002622
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=122 state=Complete evaluation=0.003346
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=123 state=Complete evaluation=0.021465
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=124 state=Complete evaluation=0.016046
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=125 state=Complete evaluation=0.001959
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=126 state=Complete evaluation=0.022310
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=127 state=Complete evaluation=0.008421
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=128 state=Complete evaluation=0.000264
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=129 state=Complete evaluation=0.001306
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=130 state=Complete evaluation=0.007589
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=131 state=Complete evaluation=0.010711
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=132 state=Complete evaluation=0.000137
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=133 state=Complete evaluation=0.013548
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=134 state=Complete evaluation=0.007672
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=135 state=Complete evaluation=0.003877
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=136 state=Complete evaluation=0.007667
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=137 state=Complete evaluation=0.001077
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=138 state=Complete evaluation=0.000195
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=139 state=Complete evaluation=0.004136
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=140 state=Complete evaluation=0.008115
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=141 state=Complete evaluation=0.003044
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=142 state=Complete evaluation=0.006253
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=143 state=Complete evaluation=0.002411
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=144 state=Complete evaluation=0.001576
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=145 state=Complete evaluation=0.001547
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=146 state=Complete evaluation=0.002558
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=147 state=Complete evaluation=0.000411
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=148 state=Complete evaluation=0.000084
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=149 state=Complete evaluation=0.001228
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=150 state=Complete evaluation=0.000052
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=151 state=Complete evaluation=0.002752
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=152 state=Complete evaluation=0.003859
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=153 state=Complete evaluation=0.000480
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=154 state=Complete evaluation=0.001038
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=155 state=Complete evaluation=0.000385
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=156 state=Complete evaluation=0.001061
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=157 state=Complete evaluation=0.000026
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=158 state=Complete evaluation=0.000357
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=159 state=Complete evaluation=0.000382
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=160 state=Complete evaluation=0.000301
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=161 state=Complete evaluation=0.001494
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=162 state=Complete evaluation=0.001063
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=163 state=Complete evaluation=0.000044
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=164 state=Complete evaluation=0.000445
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=165 state=Complete evaluation=0.000032
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=166 state=Complete evaluation=0.000349
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=167 state=Complete evaluation=0.000036
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=168 state=Complete evaluation=0.000382
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=169 state=Complete evaluation=0.000045
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=170 state=Complete evaluation=0.000028
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=171 state=Complete evaluation=0.000061
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=172 state=Complete evaluation=0.000109
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=173 state=Complete evaluation=0.000014
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=174 state=Complete evaluation=0.000067
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=175 state=Complete evaluation=0.000191
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=176 state=Complete evaluation=0.000165
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=177 state=Complete evaluation=0.000002
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=178 state=Complete evaluation=0.000125
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=179 state=Complete evaluation=0.000056
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=180 state=Complete evaluation=0.000019
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=181 state=Complete evaluation=0.000028
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=182 state=Complete evaluation=0.000052
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=183 state=Complete evaluation=0.000025
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=184 state=Complete evaluation=0.000046
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=185 state=Complete evaluation=0.000006
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=186 state=Complete evaluation=0.000011
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=187 state=Complete evaluation=0.000075
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=188 state=Complete evaluation=0.000009
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=189 state=Complete evaluation=0.000026
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=190 state=Complete evaluation=0.000059
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=191 state=Complete evaluation=0.000028
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=192 state=Complete evaluation=0.000004
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=193 state=Complete evaluation=0.000015
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=194 state=Complete evaluation=0.000057
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=195 state=Complete evaluation=0.000011
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=196 state=Complete evaluation=0.000016
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=197 state=Complete evaluation=0.000012
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=198 state=Complete evaluation=0.000019
    2020/04/13 02:39:31 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:39:31 [INFO] Trial finished: trialID=199 state=Complete evaluation=0.000002
    2020/04/13 02:39:31 Best evaluation=0.000002 (x1=1.998966, x2=-5.000825)
    

    Time: 10000 trials

    $ time go run _examples/cmaes/main.go 
    ...
    2020/04/13 02:44:33 [INFO] Trial finished: trialID=9999 state=Complete evaluation=NaN
    2020/04/13 02:44:33 Best evaluation=0.000000 (x1=2.000000, x2=-5.000000)
    
    real    0m2.370s
    user    0m2.287s
    sys     0m0.496s
    

    Time: 100000 trials

     $ time go run _examples/cmaes/main.go
    ...
    2020/04/13 02:41:47 [INFO] Trial finished: trialID=99998 state=Complete evaluation=NaN
    2020/04/13 02:41:47 [WARN] IntersectionSearchSpace: GetTrials return some trials are already deleted:
    2020/04/13 02:41:47 [INFO] Trial finished: trialID=99999 state=Complete evaluation=NaN
    2020/04/13 02:41:47 Best evaluation=0.000000 (x1=2.000000, x2=-5.000000)
    
    real    0m20.989s
    user    0m20.829s
    sys     0m1.421s
    

    v.s. InMemoryStorage

    | storage | InMemoryStorage | BlackHoleStorage | | --- | --- | --- | | 10000 trials | 1m1.88s | 0m2.370s | | 100000 trials | 96m26.527s | 0m20.989s |

    opened by c-bata 3
  • Kurobako benchmark on GitHub Actions

    Kurobako benchmark on GitHub Actions

    I'm thinking about to run kurobako-go on GitHub Actions (refs #64)

    1. Trigger this action when created a PR which includes the code changes of algorithms.
    2. Run kurobako benchmark script (in _benchmarks directory)
    3. Post benchmark results on Pull Request as the comment like codecov.
    opened by c-bata 3
  • HyperbandPruner

    HyperbandPruner

    https://arxiv.org/abs/1603.06560

    Hyperband requires two inputs:

    • R: the maximum amount of resource that can be allocated to a single configuration
    • η: an input that controls the proportion of configurations discarded in each round of SuccessiveHalving

    Functions:

    • run_then_return_val_loss(t, r) – a function that takes a hyperparameter configuration t and resource allocation r as input and returns the validation loss after training the configuration for the allocated resources.
    • top_k(configs, losses, k) – a function that takes a set of configurations as well as their associated losses and returns the top k performing configurations.
    opened by c-bata 3
  • Use actions/cache for kurobako CLI

    Use actions/cache for kurobako CLI

    • Use actions/cache for kurobako CLI.
    • Use kurobako v0.2.0.
      • https://github.com/sile/kurobako/releases/tag/0.2.0
    • Run CMA-ES only on Rastrigin functions to make CI faster.
    • Use kurobako built-in Optuna solver.
    opened by c-bata 2
  • List key features on README

    List key features on README

    I guess my presentation on GoConference Japan is helpful. https://www.slideshare.net/c-bata/goptuna-distributed-bayesian-optimization-framework-at-go-conference-2019-autumn-187538495

    Key features:

    | State-of-the-art algorithms | Optuna compatible RDB backend | | --------------------------- | ----------------------------- | | state-of-the-art-algorithms | optuna-compatibility |

    documentation 
    opened by c-bata 2
  • Diff between goptuna Sobol sequence and scipy.stats.qmc.Sobol

    Diff between goptuna Sobol sequence and scipy.stats.qmc.Sobol

    Hi,

    Thank you so much on your work for Go optuna.

    There are differences between the Sobol sequence generated using goptuna Sobol Engine Draw versus scipy.stats.qmc.Sobol. Tracing the problem points to the wrong set of direction numbers being used.

    In func initDirectionNumbers from https://github.com/c-bata/goptuna/blob/main/sobol/engine.go, instead of

    dn := directionNumbers[j]
    

    it should be

    dn := directionNumbers[j-1]
    
    opened by chinsoon12 0
  • Bayesian Optimization over Rest API

    Bayesian Optimization over Rest API

    Hello I was wondering if someone could publish an example showing how to use the library to accept inputs over something like a gin REST API call.

    My set up is I have another service that does some processing in real time and I want to try using Bayesian optimization to increase performance of it on the fly. I have 3 values that effect the processing application which I want the BO to give better values for. My "reward" or the evaluation happens on the processing application side.

    How can I set up goptuna to accept inputs from gin context and then respond with the next set of parameters to try.

    Thank you very much!

    opened by jgoldverg 0
  • Bump lodash from 4.17.20 to 4.17.21 in /dashboard

    Bump lodash from 4.17.20 to 4.17.21 in /dashboard

    Bumps lodash from 4.17.20 to 4.17.21.

    Commits
    • f299b52 Bump to v4.17.21
    • c4847eb Improve performance of toNumber, trim and trimEnd on large input strings
    • 3469357 Prevent command injection through _.template's variable option
    • See full diff in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 0
  • Bump browserslist from 4.14.5 to 4.19.1 in /dashboard

    Bump browserslist from 4.14.5 to 4.19.1 in /dashboard

    Bumps browserslist from 4.14.5 to 4.19.1.

    Changelog

    Sourced from browserslist's changelog.

    4.19.1

    • Fixed throwOnMissing types (by Øyvind Saltvik).

    4.19

    • Added queries grammar definition (by Pig Fang).
    • Added throwOnMissing option (by Øyvind Saltvik).
    • Fixed null data ignoring in < 50% in CN (byPig Fang).
    • Fixed data parsing in in my stats (by Sun Xiaoran).
    • Fixed yarn.lock support with integrity (by Alexey Berezin).
    • Fixed Yarn Berry error message in --update-db.

    4.18.1

    • Fixed case inventiveness for cover queries (by Pig Fang).
    • Fixed since 1970 query for null in release date (by Pig Fang).

    4.18

    • Added --ignore-unknown-versions CLI option (by Pig Fang).

    4.17.6

    • Fixed sharable config resolution (by Adaline Valentina Simonian).

    4.17.5

    • Fixed --update-db for some pnpm cases.

    4.17.4

    • Reduced package size.

    4.17.3

    • Use picocolors for color output in --update-db.

    4.17.2

    • Reduced package size.

    4.17.1

    • Use Nano Colors for color output in --update-db.

    4.17

    • Added yarn 3 support to --update-db (by Himank Pathak).

    4.16.8

    • Updated Firefox ESR.

    4.16.7

    • Fixed oudated caniuse-lite warning text (by Paul Verest).
    • Fixed docs (by Alexander Belov).

    4.16.6

    • Fixed npm-shrinkwrap.json support in --update-db (by Geoff Newman).

    4.16.5

    ... (truncated)

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 0
  • Bump axios from 0.21.1 to 0.21.2 in /dashboard

    Bump axios from 0.21.1 to 0.21.2 in /dashboard

    Bumps axios from 0.21.1 to 0.21.2.

    Release notes

    Sourced from axios's releases.

    v0.21.2

    0.21.2 (September 4, 2021)

    Fixes and Functionality:

    • Updating axios requests to be delayed by pre-emptive promise creation (#2702)
    • Adding "synchronous" and "runWhen" options to interceptors api (#2702)
    • Updating of transformResponse (#3377)
    • Adding ability to omit User-Agent header (#3703)
    • Adding multiple JSON improvements (#3688, #3763)
    • Fixing quadratic runtime and extra memory usage when setting a maxContentLength (#3738)
    • Adding parseInt to config.timeout (#3781)
    • Adding custom return type support to interceptor (#3783)
    • Adding security fix for ReDoS vulnerability (#3980)

    Internal and Tests:

    • Updating build dev dependancies (#3401)
    • Fixing builds running on Travis CI (#3538)
    • Updating follow rediect version (#3694, #3771)
    • Updating karma sauce launcher to fix failing sauce tests (#3712, #3717)
    • Updating content-type header for application/json to not contain charset field, according do RFC 8259 (#2154)
    • Fixing tests by bumping karma-sauce-launcher version (#3813)
    • Changing testing process from Travis CI to GitHub Actions (#3938)

    Documentation:

    • Updating documentation around the use of AUTH_TOKEN with multiple domain endpoints (#3539)
    • Remove duplication of item in changelog (#3523)
    • Fixing gramatical errors (#2642)
    • Fixing spelling error (#3567)
    • Moving gitpod metion (#2637)
    • Adding new axios documentation website link (#3681, #3707)
    • Updating documentation around dispatching requests (#3772)
    • Adding documentation for the type guard isAxiosError (#3767)
    • Adding explanation of cancel token (#3803)
    • Updating CI status badge (#3953)
    • Fixing errors with JSON documentation (#3936)
    • Fixing README typo under Request Config (#3825)
    • Adding axios-multi-api to the ecosystem file (#3817)
    • Adding SECURITY.md to properly disclose security vulnerabilities (#3981)

    Huge thanks to everyone who contributed to this release via code (authors listed below) or via reviews and triaging on GitHub:

    ... (truncated)

    Changelog

    Sourced from axios's changelog.

    0.21.2 (September 4, 2021)

    Fixes and Functionality:

    • Updating axios requests to be delayed by pre-emptive promise creation (#2702)
    • Adding "synchronous" and "runWhen" options to interceptors api (#2702)
    • Updating of transformResponse (#3377)
    • Adding ability to omit User-Agent header (#3703)
    • Adding multiple JSON improvements (#3688, #3763)
    • Fixing quadratic runtime and extra memory usage when setting a maxContentLength (#3738)
    • Adding parseInt to config.timeout (#3781)
    • Adding custom return type support to interceptor (#3783)
    • Adding security fix for ReDoS vulnerability (#3980)

    Internal and Tests:

    • Updating build dev dependancies (#3401)
    • Fixing builds running on Travis CI (#3538)
    • Updating follow rediect version (#3694, #3771)
    • Updating karma sauce launcher to fix failing sauce tests (#3712, #3717)
    • Updating content-type header for application/json to not contain charset field, according do RFC 8259 (#2154)
    • Fixing tests by bumping karma-sauce-launcher version (#3813)
    • Changing testing process from Travis CI to GitHub Actions (#3938)

    Documentation:

    • Updating documentation around the use of AUTH_TOKEN with multiple domain endpoints (#3539)
    • Remove duplication of item in changelog (#3523)
    • Fixing gramatical errors (#2642)
    • Fixing spelling error (#3567)
    • Moving gitpod metion (#2637)
    • Adding new axios documentation website link (#3681, #3707)
    • Updating documentation around dispatching requests (#3772)
    • Adding documentation for the type guard isAxiosError (#3767)
    • Adding explanation of cancel token (#3803)
    • Updating CI status badge (#3953)
    • Fixing errors with JSON documentation (#3936)
    • Fixing README typo under Request Config (#3825)
    • Adding axios-multi-api to the ecosystem file (#3817)
    • Adding SECURITY.md to properly disclose security vulnerabilities (#3981)

    Huge thanks to everyone who contributed to this release via code (authors listed below) or via reviews and triaging on GitHub:

    ... (truncated)

    Commits
    Maintainer changes

    This version was pushed to npm by jasonsaayman, a new releaser for axios since your current version.


    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 0
  • Bump path-parse from 1.0.6 to 1.0.7 in /dashboard

    Bump path-parse from 1.0.6 to 1.0.7 in /dashboard

    Bumps path-parse from 1.0.6 to 1.0.7.

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 0
Releases(v0.8.1)
  • v0.8.1(Apr 23, 2021)

    Changelog

    CHANGES

    Minor improvements

    • Add options to skip an initial portion of Sobol' sequence (#208)

    Minor bug fixes

    • Fix less comparator of DataGrid component on Web Dashboard (#225).

    API Changes (not breaking changes)

    • Rename options for Asynchronous Successive Halving Pruner (56bbce2)
    Source code(tar.gz)
    Source code(zip)
    goptuna_0.8.1_checksums.txt(453 bytes)
    goptuna_darwin_arm64.zip(5.03 MB)
    goptuna_darwin_x86_64.zip(5.13 MB)
    goptuna_linux_arm64.zip(4.61 MB)
    goptuna_linux_i386.zip(4.76 MB)
    goptuna_linux_x86_64.zip(5.58 MB)
  • v0.8.0(Feb 9, 2021)

    CHANGES

    Support a new sampler based on Sobol's Quasi-random Sequence

    Goptuna now supports Sobol based sampler. See the paper S. Joe and F. Y. Kuo, Remark on Algorithm 659: Implementing Sobol's quasirandom sequence generator, ACM Trans. Math. Softw. 29, 49-57 (2003). for details.

    | np.random.rand() (seed=0) | Sobol (Goptuna) | | --- | --- | | random-seed0 | sobol |

    Usage is like this:

    import (
    	"github.com/c-bata/goptuna"
    	"github.com/c-bata/goptuna/sobol"
    )
    
    func main() {
    	study, err := goptuna.CreateStudy(
    		"goptuna-example",
    		goptuna.StudyOptionRelativeSampler(sobol.NewSampler()),  // Set Sobol based sampler.
    	)
    	...
    }
    

    Add github.com/c-bata/gorm/rdb.v2

    Add rdb.v2 package built on gorm v2. So github.com/c-bata/goptuna/rdb package is now deprecated.

    • Use gorm v2 package (#124).
    • Fix duplicate trial numbers are assigned on parallel execution (#130).
    • Add CASCADE ON DELETE constraints (#170).
    • Drop schema compatibility with Optuna (#166).

    Dashboard

    • Add datetime_start column on top page of Dashboard (#169).
    • Put select box to change the reload interval time on AppBar (#176).

    Minor compatibility changes

    • Include the upper bound of uniform/loguniform distributions (#205)
    Source code(tar.gz)
    Source code(zip)
    goptuna_0.8.0_checksums.txt(362 bytes)
    goptuna_darwin_x86_64.zip(5.38 MB)
    goptuna_linux_arm64.zip(4.82 MB)
    goptuna_linux_i386.zip(4.95 MB)
    goptuna_linux_x86_64.zip(5.80 MB)
  • v0.7.1(Oct 28, 2020)

  • v0.7.0(Oct 21, 2020)

    New feature

    Built-in Web Dashboard (#157)

    goptuna-dashboard

    You can download standalone binaries (JavaScript files are also embedded in these binaries) via GitHub release, then you can open Web dashboard like following command.

    $ goptuna --version
    goptuna version 0.7.0 (rev: e8db5dc)
    $ goptuna dashboard --storage sqlite:///example.db --host 127.0.0.1 --port 8000
    

    Goptuna's RDB storage backend is compatible with Optuna. So this CLI is able to visualize Optuna's optimization results.

    Improvements

    • Faster TPE categorical sampling (#152)

    Bug fixes

    • Fix log message of pruned trials (#150)
    • Calculate logLikeLihoodsAbove from samplesBelow (#151)
    Source code(tar.gz)
    Source code(zip)
    goptuna_0.7.0_checksums.txt(272 bytes)
    goptuna_darwin_x86_64.zip(4.30 MB)
    goptuna_linux_i386.zip(3.99 MB)
    goptuna_linux_x86_64.zip(4.73 MB)
  • v0.6.0(Jul 28, 2020)

    New feature

    API changes

    Pruning (Early-stopping) APIs are changed without backward compatibility.

    While I added an example which uses successive halving pruner at #120, I found a lot of issues. I'm sure that nobody uses pruning functionalities (I also confirmed that Kubeflow/Katib and sile/kurobako-go does not use pruning functionalities). So I fixes those issues. And I merge it without backward compatibility.

    • Fix typo of OptionSetReductionFactor
    • Alter the argument of trial.ShouldPrune() and remove trial.Report().

    Some APIs are renamed without breaking changes.

    These changes are added with backward compatibilities. Older functions are deprecated.

    • Rename RandomSearchSampler to RandomSampler (#137).
    • Rename StudyOptionSetDirection to StudyOptionDirection (#138).
    • Rename StudyOptionSetTrialNotifyChannel to StudyOptionTrialNotifyChannel (#138).

    Minor bug fixes

    • Fix IntUniformDistribution.ToExternalRepr() (#102)
    • Fix IntUniformDistributions.Contains() (#103)
    • Fix a bug when set attrs multiple times (#121)
    • CMA-ES: fix numerical overflow errors (#126)
    • Fix a bug of successive halving pruner (#120)

    Example

    • Add an example of Gorgonia hyperparameter optimization with early stopping (ASHA) (#118, #119 and #120)
    Source code(tar.gz)
    Source code(zip)
  • v0.5.1(Apr 10, 2020)

    In this release, some internal methods made public for Kubeflow/Katib.

    New feature

    • Add study option to define search space (#99)

    For Developers

    • Public API to call relative sampler for Kubeflow/Katib (#100)

    Others

    • Generate stringer code for trial state (#101)
    Source code(tar.gz)
    Source code(zip)
  • v0.5.0(Apr 10, 2020)

    From this release, new suggest API SuggestStepInt for step-interval integer parameter is available. SuggestUniform, SuggestLogUniform and SuggestDiscreteUniform are deprecated. These APIs are renamed to Suggest...Float API.

    New feature

    • Suggest int parameter with step (#98)

    API changes

    • Rename SuggestUniform to SuggestFloat (#96)
    Source code(tar.gz)
    Source code(zip)
  • v0.4.0(Mar 24, 2020)

    This release supports CMA-ES sampler and compatible with the RDB schema of Optuna v1.3.0.

    New feature

    • CMA-ES sampler (#93)

    Internal chages

    • Make StudyAttrs/UserAttrs more readable (#89)
    • Add number field in trials table (#88)

    Bug fixes

    • Fix concurrency problems of EnqueuTrial (#92)

    Compatibility with Optuna RDB storage

    This release has compatibility with Optuna v1.3.0.

    Source code(tar.gz)
    Source code(zip)
  • v0.3.0(Mar 11, 2020)

    This release supports enqueue_trial and has compatibility with the RDB schema of Optuna v1.2.0.

    New feature

    • Experimental support of EnqueueTrial (#86)

    Bug fixes

    • Fix trial number on RDB storage (#87)

    Compatibility with Optuna RDB storage

    This release has compatibility with Optuna v1.2.0.

    Source code(tar.gz)
    Source code(zip)
  • v0.2.0(Mar 10, 2020)

    This release contains ASHA support, experimental GP-BO support and a serious bug fix of the TPE optimization on DiscreteUniformDistribution.

    New feature

    • Published an integration sampler using go-bayesopt ( https://github.com/c-bata/goptuna-bayesopt ).
    • Support an Asynchronous Successive Halving Algorithm (#52, #62)
    • Add RelativeSampler interface for Gaussian Process and CMA-ES (#75, #76)
    • Add a subcommand to delete study (#56)
    • Add a function to delete study (#57)
    • Add study option for loading if exists (#61)

    API changes

    • Remove ID suffix from storage.CreateNew{Study,Trial}ID methods (#53).
    • Avoid to update value when set intermediate values (#84)

    Bug fixes

    • Set a seed number to the inner random sampler of TPE sampler (#66)
    • Fix an optimization of DiscreteUniform on TPE sampler (#79)

    For Developers

    • Use Go v1.13 on GitHub Actions (#59)
    • Parse SQLAlchemy engine format using regexp (#67, #68, #69)
    • Add kurobako-go benchmark for TPE sampler (#71)

    Compatibility with Optuna RDB storage

    This release has the compatibility with Optuna v1.1.0 or lower.

    Source code(tar.gz)
    Source code(zip)
  • v0.1.0(Aug 22, 2019)

    New feature

    • Add logger interface (#46)
    • Add validation for trial.SuggestXXX methods (#41)

    Bug fixes

    • Fix inconsistent state of trials (#50)
    • Add transaction for creating trial (#36)
    • Fix 'Too many SQL variables' error by separate queries to get all trials (#33)
    • Encode attr values to the format which is loadable from Python's json module. (#35)

    API changes

    • New pruner interface (#47)
    • Remove FrozenTrial.ParamsInIR field (#20)
    Source code(tar.gz)
    Source code(zip)
  • v0.0.4(Aug 14, 2019)

    New feature

    • Support Optuna compatible RDB storage backend (#18)
    • Add missing SuggestLogUniform and SuggestDiscreteUniform methods in Trial (#23)
    • Provide SetUserAttr, GetUserAttrs, SetSystemAttr and GetSystemAttrs methods in Trial and Study (#24)
    • Provide GetContext method in Trial for handling signals (#29)

    Others

    • Print more detail error logs (#28).
    Source code(tar.gz)
    Source code(zip)
  • v0.0.3(Aug 5, 2019)

    New feature

    • Add MedianPruner and PercentilePruner which implements median stopping rule of Vizier (Google) (#11)
    • Support DiscreteUniformDistribution (#14)
    • Support LogUniformDistribution (#16)

    Internal change

    • Make TPE 30% faster (#9)
    Source code(tar.gz)
    Source code(zip)
  • v0.0.2(Jul 29, 2019)

    New feature

    • Accept a go's context via study.WithContext (#4)
    • Support CategoricalDistribution (#8)

    Internal change

    • Redesign the distribution interface. (#6)
    • Add about 20 methods in storage interface and InMemoryStorage. (#7)

    For developers

    • Improve test coverages.
    • Fix all golint errors.
    Source code(tar.gz)
    Source code(zip)
  • v0.0.1(Jul 26, 2019)

    ʕ◔ϖ◔ʔ Initial release :tada:

    Source code(tar.gz)
    Source code(zip)
Owner
Masashi SHIBATA
Creator of go-prompt. Optuna committer. Kubeflow/Katib reviewer.
Masashi SHIBATA
SGPM: A coroutine scheduling model for wound-wait concurrency control optimization

Environment set up off the mod go env -w GO111MODULE=off change to GPATH to project directory go env -w GOPATH=$HOME/sgpm Usage This project serves th

Jack Wang 0 Dec 12, 2021
Vald. A Highly Scalable Distributed Vector Search Engine

Vald is a highly scalable distributed fast approximate nearest neighbor dense vector search engine.

Vector Data as a Service 946 Jun 28, 2022
PaddleDTX is a solution that focused on distributed machine learning technology based on decentralized storage.

中文 | English PaddleDTX PaddleDTX is a solution that focused on distributed machine learning technology based on decentralized storage. It solves the d

null 58 Jun 23, 2022
onnx-go gives the ability to import a pre-trained neural network within Go without being linked to a framework or library.

This is a Go Interface to Open Neural Network Exchange (ONNX). Overview onnx-go contains primitives to decode a onnx binary model into a computation b

Olivier Wulveryck 399 Jun 22, 2022
The Go App Boot Framework

The Go App Boot Framework good is a http framework that makes developers write go applications much easier. Download and Install go get -u github.com/

null 0 Jan 11, 2022
Evolutionary optimization library for Go (genetic algorithm, partical swarm optimization, differential evolution)

eaopt is an evolutionary optimization library Table of Contents Changelog Example Background Features Usage General advice Genetic algorithms Overview

Max Halford 791 Jun 15, 2022
Genetic Algorithm and Particle Swarm Optimization

evoli Genetic Algorithm and Particle Swarm Optimization written in Go Example Problem Given f(x,y) = cos(x^2 * y^2) * 1/(x^2 * y^2 + 1) Find (x,y) suc

Guillaume Simonneau 23 Jun 12, 2022
Go package for convex optimization

cvx Package cvx is a Go package for solving convex optimization problems. It is a straightforward translation of parts of the CVXOPT python package fo

Harri Rautila 20 Apr 5, 2022
Gonum is a set of numeric libraries for the Go programming language. It contains libraries for matrices, statistics, optimization, and more

Gonum Installation The core packages of the Gonum suite are written in pure Go with some assembly. Installation is done using go get. go get -u gonum.

null 5.9k Jun 30, 2022
Gonum is a set of numeric libraries for the Go programming language. It contains libraries for matrices, statistics, optimization, and more

Gonum Installation The core packages of the Gonum suite are written in pure Go with some assembly. Installation is done using go get. go get -u gonum.

null 5.8k Jun 26, 2022
Opsani Ignite for Kubernetes: Evaluate Applications for Optimization

Opsani Ignite for Kubernetes Opsani Ignite analyzes applications running on a Kubernetes cluster in order to identify performance and reliability risk

Opsani 16 Mar 1, 2022
SGPM: A coroutine scheduling model for wound-wait concurrency control optimization

Environment set up off the mod go env -w GO111MODULE=off change to GPATH to project directory go env -w GOPATH=$HOME/sgpm Usage This project serves th

Jack Wang 0 Dec 12, 2021
Distributed-Services - Distributed Systems with Golang to consequently build a fully-fletched distributed service

Distributed-Services This project is essentially a result of my attempt to under

Hamza Yusuff 6 Jun 1, 2022
7 days golang programs from scratch (web framework Gee, distributed cache GeeCache, object relational mapping ORM framework GeeORM, rpc framework GeeRPC etc) 7天用Go动手写/从零实现系列

7 days golang programs from scratch README 中文版本 7天用Go从零实现系列 7天能写什么呢?类似 gin 的 web 框架?类似 groupcache 的分布式缓存?或者一个简单的 Python 解释器?希望这个仓库能给你答案

Dai Jie 10.7k Jun 26, 2022
A distributed unique ID generator inspired by Twitter's Snowflake

Sonyflake is a distributed unique ID generator inspired by Twitter's Snowflake.

Sony 2.7k Jun 23, 2022
Kitten is a distributed file system optimized for small file storage, inspired by Facebook's Haystack.

Kitten is a distributed file system optimized for small file storage, inspired by Facebook's Haystack.

Jack Lee 9 Jun 29, 2022
⚡️ Express inspired web framework written in Go

Fiber is an Express inspired web framework built on top of Fasthttp, the fastest HTTP engine for Go. Designed to ease things up for fast development w

Fiber 20.7k Jun 27, 2022
Mango is a modular web-application framework for Go, inspired by Rack, and PEP333.

Mango Mango is a modular web-application framework for Go, inspired by Rack and PEP333. Note: Not actively maintained. Overview Mango is most of all a

Paul Bellamy 366 Jun 3, 2022
A Go framework for building JSON web services inspired by Dropwizard

Tiger Tonic A Go framework for building JSON web services inspired by Dropwizard. If HTML is your game, this will hurt a little. Like the Go language

Richard Crowley 1k May 29, 2022
Simple and lightweight Go web framework inspired by koa

VOX A golang web framework for humans, inspired by Koa heavily. Getting started Installation Using the go get power: $ go get -u github.com/aisk/vox B

An Long 78 Jun 14, 2022
API testing framework inspired by frisby-js

frisby REST API testing framework inspired by frisby-js, written in Go Proposals I'm starting to work on frisby again with the following ideas: Read s

_Hofstadter 272 Jun 13, 2022
Mango is a modular web-application framework for Go, inspired by Rack, and PEP333.

Mango Mango is a modular web-application framework for Go, inspired by Rack and PEP333. Note: Not actively maintained. Overview Mango is most of all a

Paul Bellamy 366 Jun 3, 2022
A Go framework for building JSON web services inspired by Dropwizard

Tiger Tonic A Go framework for building JSON web services inspired by Dropwizard. If HTML is your game, this will hurt a little. Like the Go language

Richard Crowley 1k May 29, 2022
Extensible network application framework inspired by netty

GO-NETTY 中文介绍 Introduction go-netty is heavily inspired by netty Feature Extensible transport support, default support TCP, UDP, QUIC, KCP, Websocket

The Go-Netty Project 306 Jun 23, 2022
Rocinante is a gin inspired web framework built on top of net/http.

Rocinante Rocinante is a gin inspired web framework built on top of net/http. ⚙️ Installation $ go get -u github.com/fskanokano/rocinante-go ⚡️ Quicks

null 3 Jul 27, 2021
📁 Examples for 🚀 Fiber - Express inspired web framework written in Go

?? Examples for ?? Fiber - Express inspired web framework written in Go

Fiber 1.6k Jun 28, 2022
WebAssembly Lightweight Javascript Framework in Go (AngularJS Inspired)

Tango Lightweight WASM HTML / Javascript Framework Intro WebAssembly is nice, Go on the web is nice, so I ported Tangu to Go and WebAssembly. Tangu is

enimatek 4 Jun 8, 2022
An easy-to-use Map Reduce Go parallel-computing framework inspired by 2021 6.824 lab1. It supports multiple workers on a single machine right now.

MapReduce This is an easy-to-use Map Reduce Go framework inspired by 2021 6.824 lab1. Feature Multiple workers on single machine right now. Easy to pa

Bo-Wei Chen 193 Jun 9, 2022
Gouweb: A web framework for go inspired by laravel

gouweb is a web framework for go inspired by laravel Installation go get -u gith

null 2 Feb 17, 2022