Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ENH: Add experiment tracking with mlflow as an extra #778

Open
2 tasks
NickleDave opened this issue Sep 28, 2024 · 1 comment
Open
2 tasks

ENH: Add experiment tracking with mlflow as an extra #778

NickleDave opened this issue Sep 28, 2024 · 1 comment

Comments

@NickleDave
Copy link
Collaborator

  • pip install vak[mlflow] should add this optional dependency and then allow experiment tracking
  • we should include at a bare minimum the path to metadata json files from datasets so that one can programatically relate metadata to experiments / runs

I don't want to invest a lot right now in supporting every feature of mlflow but it's clear to me we need some sort of experiment tracking (as @achabotl wisely pointed out many moons ago).

Prefer going with mlflow for now since it's the one I'm most familiar with and I would rather just adopt something that's widely used in industry, although sth more lightweight and application-agnostic like sacred or sumatra might be worth thinking about

@NickleDave
Copy link
Collaborator Author

Just realized lightning has an mlflow logger
https://lightning.ai/docs/pytorch/stable/api/lightning.pytorch.loggers.mlflow.html#mlflow-logger
so we can perhaps somewhat painlessly integrate that way -- pass in experiment name, run name, etc

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant