Allow fitting arbitrary @formulas

[rikhuijzer]

Currently, there are only a few models available via this interface. I suggest implementing also adding a FormulaRegressor for arbitrary formulas, via StatsModels.@formula(...).

@ablaom, what do you think? Would this make sense to add this to this interface?

[ablaom]

Something along these lines would be useful and might help us win over some R users 🙏🏾 . I would support this.

However, I think that even more useful would be separate MLJ formula-based transformer that can be inserted anywhere in an MLJ pipeline (or other composite model). Here "formula" means "one-side formula"; I don't think two-sided formulas make much sense in the MLJ context because the target and features are treated separately, like in sklearn.

This transformer would probably be a Static model with a one-sided StatsModels formula as parameter. Ideally, and for consistency, it would perform a table-to-table transformation, rather than a table-to-matrix transformation, which is what StatsModels does. This does cause problems for very-high cardinality categorical features (which get one-hot encoded when you apply StatsBase formula??) but does have the advantage that new columns would come with informative names for interpretation downstream of the transformer. Actually, it probably makes sense not to force one-hot encoding anyway, as not all supervised models need this and we already have transformers to do one-hot encoding which generate the new column names.

I recall slack discussions with @kleinschmidt about this (now lost to the ether). Perhaps he would care to chime in.

See also JuliaAI/MLJModels.jl#314.

[ablaom]

Okay I've created a new issue here specific to the suggestion not immediately addressing the initial comment. So further comment on that should go there, thanks.