Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[experiments] filter on experiment runs #5681

Open
Tracked by #3398
mikeldking opened this issue Dec 10, 2024 · 2 comments · May be fixed by #5738
Open
Tracked by #3398

[experiments] filter on experiment runs #5681

mikeldking opened this issue Dec 10, 2024 · 2 comments · May be fixed by #5738
Assignees
@axiomofjoy

Comments

@mikeldking
Copy link
Contributor

mikeldking commented Dec 10, 2024 •
edited
Loading

As a user that has just run an experiment with a set of evaluators configured, I want the ability to quickly narrow down to the problematic "runs"

for example might have "has_errors" and "valid_sql" as validators. I want to quickly narrow down the rows to the runs where "has_errors=1" or "valid_sql=0".
@mikeldking mikeldking mentioned this issue Dec 10, 2024
Open
22 tasks
@github-project-automation github-project-automation bot moved this to 📘 Todo in phoenix Dec 10, 2024
@axiomofjoy axiomofjoy mentioned this issue Dec 11, 2024
Closed
@axiomofjoy
Copy link
Contributor

Also see #5015 for additional context

@axiomofjoy axiomofjoy moved this from 📘 Todo to 👨‍💻 In progress in phoenix Dec 11, 2024
@axiomofjoy
Copy link
Contributor

axiomofjoy commented Dec 12, 2024 •
edited
Loading

Some values we may wish to filter on:

  • eval
    • score (equality and inequality)
    • label (equality)
  • latency (greater than or eq)
  • status (success or error)
  • value for input key
  • value for output key

@axiomofjoy axiomofjoy linked a pull request Dec 14, 2024 that will close this issue
Open
@mikeldking mikeldking moved this from 👨‍💻 In progress to 🔍. Needs Review in phoenix Dec 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@axiomofjoy axiomofjoy

Labels
None yet
Projects
phoenix
Status: 🔍. Needs Review
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

feat(experiments): add experiment run filter to compare experiments page Arize-ai/phoenix
2 participants
@mikeldking @axiomofjoy