-
Notifications
You must be signed in to change notification settings - Fork 58
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Feature request] Add a possibility to persist artifacts besides the model itself #46
Comments
Another way to deal with this is to move the normalization into a model wrapper (or "meta-estimator" in scikit-learn). A |
Yes, for this specific case, that would work. For other cases, that could be an awkward solution. I could imagine that a more general solution would have a "cache" that is just stored together with the model, so that there is no need for handling separate files. |
There's this utility called To stick something in you would call |
Okay, so you would suggest to use this if extra data needs to be saved? |
Hmm, just had another look and it seems that at least Which leaves us with what you already did I assume, which is sticking attributes on the model object. Not too nice, but probably nicer than having to worry about storing extra data somewhere else and having to support that in all persisters. If you prefer to use something like |
But isn't the model just a blob? Instead of persisting the model, could we not persist something like |
At the moment, only the model can be persisted and loaded. However, there are scenarios that necessitate saving and loading additional data.
E.g., assume that we have a regression problem. We want to normalize the targets to a certain range during training but when calling the predict service, data should be mapped back to the original range. Touching the targets is not part of an
sklearn
pipeline, so we may do it during data loading. However, when we start the prediction service, we need to have access to the mapping. Currently, we would have to load the data again to generate the mapping, or try to save the mapping as an attribute of the model.Ideally, we would be able to just save and load the mapping using palladium tools. The solution should not be too specific to the example above, but be a more general solution to how to persist additional artifacts.
The text was updated successfully, but these errors were encountered: