Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Get multiple runs at the same time to call getOMLRunParList #377

Closed
jakob-r opened this issue Jul 5, 2017 · 3 comments
Closed

Get multiple runs at the same time to call getOMLRunParList #377

jakob-r opened this issue Jul 5, 2017 · 3 comments

Comments

@jakob-r
Copy link
Collaborator

jakob-r commented Jul 5, 2017

I am especially interested to get all the runs of the OpenML_Bot for a specific data.set and for a specific flow like in the following example:

library(data.table)
res = listOMLRuns(uploader.id = 2702, task.id = 9914, limit = 1000, offset = 0, flow.id = 5890)
res = as.data.table(res)
res[, {
  oml.run = getOMLRun(run.id = run.id, only.xml = TRUE)
  run.par = getOMLRunParList(oml.run)
  as.data.table(t(BBmisc::extractSubList(run.par, "value")))
  }, by = .(run.id)]
      run.id  k
  1: 1875250  5
  2: 1875252 12
  3: 1875253 27
  4: 1875254 29
  5: 1875255  3
 ---           
965: 2014491 27
966: 2014492 14
967: 2014493 10
968: 2014494 28
969: 2014495 26

This would be merged with listOMLRunEvaluations() to get the full picture. The question is if there is a more efficient way to get all these parameter configurations of the API even allows this.

@giuseppec
Copy link
Member

It looks like you want something like this #348 ?
For now, your solution is the most efficient way. However, it seems that @janvanrijn extended the API on the test server so that you can get all setups (and hyperparameters) that are connected to a specific flow (see also his comment in openml/OpenML#411).
Maybe you can try out if this extension will give you what you need and make suggestions if not?
And of course feel free to make a PR for this new API call.

@jakob-r
Copy link
Collaborator Author

jakob-r commented Jul 5, 2017

@berndbischl and me looked into it and yes there is this new API Call that get's the desired information for at least for one run. Could you implement it since you are the expert in the package. Otherwise let me know and I will work on this soon.

@berndbischl
Copy link
Contributor

this a duplicate of #348

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants