Runs generated by the "evaluate" function are part of the "evaluators" project.

[balvisio]

I have been using the evaluate() function and I have noticed that all the runs go to the "evaluators" project. AFAICT there is no way to configure it. In fact, I see that in _runner.py the evaluators project name is hard coded.

1. I was wondering if this is by design, or there are plans to support specifying the project name when running evaluations?
2. Does it make sense to have different projects? I am thinking that I might have different datasets or teams and I might want to logically separate the runs into different projects. Is this the right way to think about projects?

Thanks!

[hinthornw]

Hi @balvisio great question. It is indeed by design and we aren't currently planning to make it configurable. We do try to add metadata to the runs (in python at least; and may try to improve the specificity of that) so you could filter runs in the project by source experiment, etc.