Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Imbalance doesn't work with categorical data which has Textual type #98

Open
sylvaticus opened this issue Mar 28, 2024 · 4 comments
Open
Labels

Comments

@sylvaticus
Copy link
Contributor

Take this example:

a = DataFrame(a =["a","b","c"], b=[1,2,3])
b = ["a","a","b"]
Xover, yover = random_oversample(a, b)   

This fails because ScientificTypes.schema(X).scitypes fails, but the algorithms emploied doesn't really care about scientific types.

@EssamWisam
Copy link
Collaborator

EssamWisam commented Apr 1, 2024

For consistency, as in the documentation of each method, all methods at most assume that

X: A matrix of real numbers or a table with element scitypes that subtype Union{Finite, Infinite}.

Here the column a has scientific type Textual and should be coerced to MultiClass first. I see that the error does not seem to signal that directly and will consider making a PR that throws a better one.

We can also add support for the Textual type but it may not be exactly straightforward given the current implementation.

@ablaom
Copy link
Member

ablaom commented Apr 2, 2024

So, for example, this works:

using CategoricalArrays
a = DataFrame(a =categorical(["a","b","c"]), b=[1,2,3]);

@ablaom
Copy link
Member

ablaom commented Apr 2, 2024

Or:

using ScientificTypes
coerce!(a, a: => Multiclass)

@ablaom
Copy link
Member

ablaom commented Apr 7, 2024

But I agree with @sylvaticus that it's worth special casing this algorithm which can deal with arbitrary column types.

@EssamWisam EssamWisam changed the title Imbalance doesn't work with categorical data not encoded as categorical arrays Imbalance doesn't work with categorical data which has Textual type Jun 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants