Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cross-validation ,plots, and eval metrics error julia 1.8.3 and julia 1.9.0 #18

Open
MrBenzWorld opened this issue Nov 23, 2022 · 2 comments

Comments

@MrBenzWorld
Copy link

not able to do Cross-validation, plots, and eval metrics error.

Please add example using these functions

@WilliamZimmerman83
Copy link

WilliamZimmerman83 commented Jan 24, 2023

I was able to get eval metrics and cv working - but plots don't work.

#CROSS VALIDATION
cat_features = ["x", "x", "x", "x", "x"]
cv_dataset = Pool(; data=pytable(test_data), label = test_label, cat_features = cat_features)
params = PyDict(Dict("iterations" => 200, 
                    "depth" => 5, 
                    "loss_function" => "Logloss",
                     "verbose" => false
                )

And here is the eval set:

#Creating Evaluation Set
eval_dataset = Pool(; data=test_data, label=test_label, cat_features = cat_features)

#INITIAL FIT MODEL
fit!(model, train_data, train_label,
    cat_features = cat_features,
    eval_set = eval_dataset,
    verbose = true,
    plot = true
)

# GET PREDICTIONS & BEST SCORE
best_iter = model.get_best_iteration()
preds = CatBoost.predict(model, test_data)
best_score = model.get_best_score()

When I run these with "plot = true" it will run successfully but no plot is generated. I tested in jupyter notebooks and in VScode.

I did the following to get plots to stop giving me errors:

using Conda
Conda.add("traitlets")
Conda.add("IPython")
Conda.add("ipywidgets")

Not sure what else needs to be done.

@tylerjthomas9
Copy link
Collaborator

tylerjthomas9 commented Jan 24, 2023

I not sure how to get the plots working inside Julia kernels. I think the best method, for now, would just be to visualize using python. Let me know if you manage to get the plots to show within a Julia kernel, but my guess is that it'll take some work because it isn't just a matplotlib plot.

import catboost

w = catboost.MetricVisualizer('/crossentropy/')
w.start()

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants