v0.9.0

Highlights

An Experimental New UI

An experimental new UI is available as an opt-in feature - simply launch the dashboard as usual and click the link to the new UI.

Please try it out and share your thoughts with us via the GitHub Discussion 👇
Feedback Survey: The New UI for Optuna Dashboard #332 - Optuna Dashboard

What's Changed

• Upgrade to React 18 by @c-bata in #309
• Use skip_table_creation when using Optuna v3 or later by @c-bata in #320
• Remove typing_extensions from the dependencies by @c-bata in #321
• Use type hinting generics. by @c-bata in #322
• Update js dependencies by @c-bata in #323
• Run npm audit fix by @c-bata in #324
• Fix warning for the use of ReactDOM.render() by @c-bata in #325
• Remove e.preventDefault by @c-bata in #327
• New Dashboard UI. by @c-bata in #328
• Make a lot of improvements in the new Dashboard UI by @c-bata in #329
• Add follow-up changes for #329 by @c-bata in #330
• Fix bug of markdown editor and add mathjax support. by @c-bata in #331
• Link to the GitHub Discussion for the feedback survey by @c-bata in #333
• Bump the version up to 0.9.0b1 by @c-bata in #334
• Add Python API to save the note by @c-bata in #337
• Add API to set objective names by @c-bata in #338
• Handle exceptions in optuna-fast-fanova by @c-bata in #339
• Fix the link to feedback survey by @c-bata in #340
• Use type instead of interface by @c-bata in #342
• Change Chip's colors and show durations by @c-bata in #343
• Customize TrialTable for new ui by @c-bata in #344
• Support trial user attributes on GraphHistory by @c-bata in #341
• Add button to rename study by @c-bata in #345
• Fix target arg for importance by @c-bata in #346
• Select multiple trials by Shift + Click by @c-bata in #347
• Make plot components faster and robust. by @c-bata in #348
• Fix seed visual regression tests by @c-bata in #349
• Simplify Python API to save notes by @c-bata in #350
• Improve graph components by @c-bata in #351
• Fix the link to trial detail page by @c-bata in #352
• Support fixed_params by @c-bata in #353
• Filter TrialState for GraphIntermediateValues by @c-bata in #310
• Fix bug of multiple select trials by @c-bata in #356
• Plot the hyperparameter importances for each objective value on a single bar chart. by @c-bata in #357
• Add global state filters for History and IntermediateValue plots by @c-bata in #358
• Use theme.typography.fontWeightBold instead of 600 by @c-bata in #359
• Remove paddingTop from CardAction on StudyList by @c-bata in #361
• Bump the version up to v0.9.0b2 by @c-bata in #360
• Add state filters in TrialList by @c-bata in #363
• Hide user_attrs on GraphParallelCoordinate by default by @c-bata in #365
• Update TrialList component by @c-bata in #364
• Support user attributes on GraphSlice by @c-bata in #366
• Remove TODOs in CachedExtraStudyProperty by @c-bata in #367
• Set key attributes in TrialList by @c-bata in #368
• Avoid to calculate importance when only available single trial by @c-bata in #369
• Add trial tell feature by @keisuke-umezawa in #355
• Introduce Artifact by @c-bata in #314
• Introducing Custom User Widget by @c-bata in #370
• Save reload interval on localStorage by @c-bata in #372
• Bump the version up to v0.9.0b3 by @c-bata in #373
• Fix bug of custom user widgets and enhance artifact support. by @c-bata in #375
• Update best trials when state is updated by @c-bata in #376
• Bump the version up to v0.9.0b4 by @c-bata in #377
• Fix artifact import issue by @c-bata in #381
• Bump the version up to v0.9.0b5 by @c-bata in #382
• Display objective forms even for completed trials by @c-bata in #387
• Remove redundant json dumps by @c-bata in #390
• Sort studies by study_id by @c-bata in #391
• Fix bug of setStateValues by @c-bata in #392
• Add an initial Audio support by @c-bata in #395
• Keep backward compatibility of widget definition by @c-bata in #393
• Introduce --storage-class CLi argument to support journal storage by @c-bata in #394
• Bump the version up to v0.9.0b6 by @c-bata in #396
• Add Sphinx documentation by @c-bata in #399
• Render an empty graph at plot_contour() when the length of trials less than two. by @c-bata in #402
• Change yAxis on history plot when user attribute is selected by @c-bata in #403
• Update release procedure in CONTRIBUTING.md by @keisuke-umezawa in #408
• Simplify the logic of selecting trials in TrialList by @keisuke-umezawa in #405
• Handle RuntimeError while calculating importance by @c-bata in #409
• Separate tell api into complete api and fail api by @keisuke-umezawa in #407
• Fix getDominatedTrials by @knshnb in #413
• Minor improvements on human-in-the-loop UI by @c-bata in #414
• Bump the version up to v0.9.0 by @c-bata in #415

New Contributors

• @knshnb made their first contribution in #413

Full Changelog: v0.8.1...v0.9.0

v0.9.0b6

What's Changed

• Display objective forms even for completed trials by @c-bata in #387
• Remove redundant json dumps by @c-bata in #390
• Sort studies by study_id by @c-bata in #391
• Fix bug of setStateValues by @c-bata in #392
• Add an initial Audio support by @c-bata in #395
• Keep backward compatibility of widget definition by @c-bata in #393
• Introduce --storage-class CLi argument to support journal storage by @c-bata in #394
• Bump the version up to v0.9.0b6 by @c-bata in #396

Full Changelog: v0.9.0b5...v0.9.0b6

v0.9.0b5

What's Changed

• Fix artifact import issue by @c-bata in #381
• Bump the version up to v0.9.0b5 by @c-bata in #382

Full Changelog: v0.9.0b4...v0.9.0b5

v0.9.0b4

Human-in-the-loop Optimization

$ pip install Pillow optuna==3.1.0 optuna-dashboard==0.9.0b4

from __future__ import annotations

import os
import textwrap
import threading
import time
from typing import NoReturn
from wsgiref.simple_server import make_server

import optuna
from PIL import Image
from optuna.trial import TrialState

from optuna_dashboard import ObjectiveChoiceWidget, save_note, ObjectiveSliderWidget
from optuna_dashboard import register_objective_form_widgets
from optuna_dashboard import set_objective_names
from optuna_dashboard import wsgi
from optuna_dashboard.artifact import upload_artifact
from optuna_dashboard.artifact.file_system import FileSystemBackend


storage = optuna.storages.InMemoryStorage()
base_path = os.path.join(os.path.dirname(__file__), "artifact")
artifact_backend = FileSystemBackend(base_path=base_path)


def suggest_and_generate_image(study: optuna.Study) -> None:
    # Ask new parameters
    trial = study.ask()
    r = trial.suggest_int("r", 0, 255)
    g = trial.suggest_int("g", 0, 255)
    b = trial.suggest_int("b", 0, 255)

    # Generate image
    image_path = f"tmp/sample-{trial.number}.png"
    image = Image.new("RGB", (320, 240), color=(r,g,b))
    image.save(image_path)

    # Upload Artifact
    artifact_id = upload_artifact(artifact_backend, trial, image_path)

    # Save Note
    note = textwrap.dedent(f'''\
    ## Trial {trial.number}
    
    ![generated-image](/artifacts/{study._study_id}/{trial._trial_id}/{artifact_id})
    ''')
    save_note(trial, note)


def start_preferential_optimization() -> NoReturn:
    # Create Study
    study = optuna.create_study(
        study_name="Human-in-the-loop Optimization",
        storage=storage,
        directions=["minimize", "maximize"]
    )
    set_objective_names(study, ["Human Perception Score", "Looks Yellow?"])
    register_objective_form_widgets(study, widgets=[
        ObjectiveChoiceWidget(
            choices=["Good 👍", "Bad 👎"],
            values=[-1, 1],
            description="Please input your score!",
        ),
        ObjectiveSliderWidget(
            min=1,
            max=10,
            step=1,
            description="Higher is better.",
        ),
    ])

    # Start Human-in-the-loop Optimization
    n_batch = 3
    while True:
        running_trials = study.get_trials(deepcopy=False, states=(TrialState.RUNNING,))
        if len(running_trials) >= n_batch:
            time.sleep(5)
            continue
        suggest_and_generate_image(study)


def main() -> None:
    if not os.path.exists(base_path):
        os.mkdir(base_path)

    # Start Dashboard server on background
    app = wsgi(storage, artifact_backend=artifact_backend)
    httpd = make_server("127.0.0.1", 8080, app)
    thread = threading.Thread(target=httpd.serve_forever)
    thread.start()

    # Run optimize loop
    try:
        start_preferential_optimization()
    except KeyboardInterrupt:
        httpd.shutdown()
        httpd.server_close()
        thread.join()


if __name__ == "__main__":
    main()

What's Changed

• Fix bug of custom user widgets and enhance artifact support. by @c-bata in #375
• Update best trials when state is updated by @c-bata in #376
• Bump the version up to v0.9.0b4 by @c-bata in #377

Full Changelog: v0.9.0b3...v0.9.0b4

v0.9.0b3

What's Changed

• Add state filters in TrialList by @c-bata in #363
• Hide user_attrs on GraphParallelCoordinate by default by @c-bata in #365
• Update TrialList component by @c-bata in #364
• Support user attributes on GraphSlice by @c-bata in #366
• Remove TODOs in CachedExtraStudyProperty by @c-bata in #367
• Set key attributes in TrialList by @c-bata in #368
• Avoid to calculate importance when only available single trial by @c-bata in #369
• Add trial tell feature by @keisuke-umezawa in #355
• Introduce Artifact by @c-bata in #314
• Introducing Custom User Widget by @c-bata in #370
• Save reload interval on localStorage by @c-bata in #372
• Bump the version up to v0.9.0b3 by @c-bata in #373

Full Changelog: v0.9.0b2...v0.9.0b3

v0.9.0b2

What's Changed

• Add Python API to save the note by @c-bata in #337
• Add API to set objective names by @c-bata in #338
• Handle exceptions in optuna-fast-fanova by @c-bata in #339
• Fix the link to feedback survey by @c-bata in #340
• Use type instead of interface by @c-bata in #342
• Change Chip's colors and show durations by @c-bata in #343
• Cutomize TrialTable for new ui by @c-bata in #344
• Support trial user attributes on GraphHistory by @c-bata in #341
• Add button to rename study by @c-bata in #345
• Fix target arg for importance by @c-bata in #346
• Select multiple trials by Shift + Click by @c-bata in #347
• Make plot components faster and robust. by @c-bata in #348
• Fix seed visual regression tests by @c-bata in #349
• Simplify Python API to save notes by @c-bata in #350
• Improve graph components by @c-bata in #351
• Fix the link to trial detail page by @c-bata in #352
• Support fixed_params by @c-bata in #353
• Filter TrialState for GraphIntermediateValues by @c-bata in #310
• Fix bug of multiple select trials by @c-bata in #356
• Plot the hyperparameter importances for each objective value on a single bar chart. by @c-bata in #357
• Add global state filters for History and IntermediateValue plots by @c-bata in #358
• Use theme.typography.fontWeightBold instead of 600 by @c-bata in #359
• Remove paddingTop from CardAction on StudyList by @c-bata in #361
• Bump the version up to v0.9.0b2 by @c-bata in #360

Full Changelog: v0.9.0b1...v0.9.0b2

v0.9.0b1

The new UI for Optuna Dashboard

TOP PAGE (NEW UI)	STUDY DETAIL PAGE (NEW UI)

We are considering replacing the current UI of Optuna Dashboard with the new UI that is available from this release. Please try it out and share your thoughts with us via this post 👇

What's Changed

• Upgrade to React 18 by @c-bata in #309
• Use skip_table_creation when using Optuna v3 or later by @c-bata in #320
• Remove typing_extensions from the dependencies by @c-bata in #321
• Use type hinting generics. by @c-bata in #322
• Update js dependencies by @c-bata in #323
• Run npm audit fix by @c-bata in #324
• Fix warning for the use of ReactDOM.render() by @c-bata in #325
• Remove e.preventDefault by @c-bata in #327
• New Dashboard UI. by @c-bata in #328
• Make a lot of improvements in the new Dashboard UI by @c-bata in #329
• Add follow-up changes for #329 by @c-bata in #330
• Fix bug of markdown editor and add mathjax support. by @c-bata in #331
• Link to the GitHub Discussion for the feedback survey by @c-bata in #333
• Bump the version up to 0.9.0b1 by @c-bata in #334

Full Changelog: v0.8.1...v0.9.0b1

v0.8.1

What's Changed

• Fix a bug of is_sortable inference for trial user attrs by @c-bata in #308
• Avoid to call deprecated system_attrs property by @c-bata in #311
• Fix typo by @Alnusjaponica in #312
• Update the getting started section of README by @c-bata in #313
• Interface change for set study directions by @c-bata in #315
• Interface change for set_study_directions by @gen740 in #292
• Fix a pypi publish action by @c-bata in #317
• Bump the version up to v0.8.1 by @c-bata in #318

New Contributors

• @Alnusjaponica made their first contribution in #312
• @gen740 made their first contribution in #292

Full Changelog: v0.8.0...v0.8.1

v0.8.0

Highlights

Contour Plot Support by @RE-yura

optuna.visualization.plot_contour() function is ported to Optuna Dashboard. It enables us to show the parameter relationship.

List user attributes on TrialTable by @msakai

The trial’s user attributes are listed on the TrialTable.

What's Changed

• Fix pypi-publish action by @c-bata in #271
• Align graph history between optuna and optuna-dashboard by @keisuke-umezawa in #272
• Use Python 3.10 in Docker image by @c-bata in #273
• Align graph slice between optuna and optuna-dashboard by @keisuke-umezawa in #274
• Drop Python 3.6 support by @c-bata in #278
• Fix mypy failure by @c-bata in #279
• Run actions with Python 3.11 by @c-bata in #277
• Use ubuntu-20.04 for integration tests by @c-bata in #283
• Align pram importance graph between optuna and optuna-dashboard by @keisuke-umezawa in #275
• Set an empty string as API_ENDPOINT at jest.config.js. by @c-bata in #285
• Align parallel coordinate graph between optuna and optuna-dashboard by @keisuke-umezawa in #280
• List user attributes on TrialTable by @msakai in #284
• Remove RedisStorage by @c-bata in #290
• Bump loader-utils from 2.0.0 to 2.0.4 by @dependabot in #289
• Align intermediate values graph between optuna and optuna-dashboard by @keisuke-umezawa in #291
• Align pareto front graph between optuna and optuna-dashboard by @keisuke-umezawa in #282
• Fix a less comparator of param value by @c-bata in #286
• ✨ feat: Contour plot by @RE-yura in #287
• Remove console.log in contor plot component by @keisuke-umezawa in #294
• Add preference panel for contour plot by @keisuke-umezawa in #296
• Refactor PreferenceDialog to simplify StudyDetail. by @c-bata in #297
• Split ReloadIntervalSelect and TrialTable from StudyDetail.tsx by @c-bata in #298
• Use functools.lru_cache to cache os.path.exists call by @c-bata in #299
• Run npm audit fix by @c-bata in #300
• Update GIF animation for v0.8.0 release by @c-bata in #301
• Fix union_user_attrs property of _CachedExtraStudyProperty. by @c-bata in #302
• Make user_attrs non-filterable if the value is number by @c-bata in #303
• Remove inner trial user attr table by @c-bata in #304
• Update the version to 0.8.0 by @keisuke-umezawa in #305

New Contributors

• @msakai made their first contribution in #284
• @RE-yura made their first contribution in #287

Full Changelog: v0.7.3...v0.8.0

v0.7.3

What's Changed

• Fix pypi-publish action by @c-bata in #261
• Define the project metadata in pyproject.toml instead of setup.cfg by @c-bata in #266
• Catch ValueError while importing optuna-fast-fanova by @c-bata in #267
• Bump the version up to v0.7.3 by @c-bata in #268
• Show warning messages when failed to import optuna-fast-fanova by @c-bata in #269

Full Changelog: v0.7.2...v0.7.3