Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pre-defined task improvements #662

Open
4 of 6 tasks
JulienVig opened this issue Apr 29, 2024 · 0 comments
Open
4 of 6 tasks

Pre-defined task improvements #662

JulienVig opened this issue Apr 29, 2024 · 0 comments
Labels
discojs Related to Disco.js feature New feature or request web client Related to the browser environment

Comments

@JulienVig
Copy link
Collaborator

JulienVig commented Apr 29, 2024

  • Training when a saved model already exists reuses it by default, while it should create a new model overwrite it.
  • The training graphs should show the training and validation loss too, sometimes the accuracies stay constant while the loss still changes.
  • The test accuracy graph doesn't update and stays at 0 -> Create a Skin Condition classification task #678 removed the test line plot (because it was only plotting a single scalar value)
  • The testing an image model (e.g. lus_covid) shows what are the predictions for each image and where the model made classification mistakes (groundtruth vs prediction). However errors are sometimes not displayed (only prediction).
  • Validation set sometimes contains only one class
    Image tasks support choosing a train-validation split in which a percentage of the training set is set apart as a validation set (discojs-core/src/dataset/data_loader/image_loader.ts). In some cases however, e.g., when using imbalanced or small datasets, this can result in some classes not being included in the validation set at all.
    Unfortunately, it seems that tensorflow.js doesn't offer any function for stratified splits.
  • Cached model is used by default while it is said in the task that it would be overwritten. Moreover the toggle button to disable this isn't displayed.
@JulienVig JulienVig added feature New feature or request discojs Related to Disco.js labels Apr 29, 2024
@JulienVig JulienVig changed the title Validation set sometimes contains only one class Pre-defined task improvements Apr 29, 2024
@JulienVig JulienVig added the web client Related to the browser environment label May 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
discojs Related to Disco.js feature New feature or request web client Related to the browser environment
Projects
None yet
Development

No branches or pull requests

1 participant