meaning of min_freq parameter

[marctorsoc]

This question is not really related to your R code, but maybe you know the answer so let's try :)

was wondering what's the meaning of min_freq param, as in the documentation says it's a float. So I was always convinced it was a number in the range [0,1] (a percentage) but then I see you use 5.0.

Is that then the absolute frequency of a feature? (e.g. the number of times a feature appears in the training data)

Is it a requirement for the entire training set or per document?

Thanks!

[jwijffels]

It's the number of times a feature (e.g. a word / a bigram / a pos tag / a suffix / ...) should occur before it will be included in the model.
I think it's a float because all these hyperparameters in CRFsuite are stored as floats. While it would be more intuitive to have it as an integer.
Look at crfsuite::crf_options(method = "lbfgs")$params to get a list of the hyperparameters of the model which you need to tune in order to get good results.
I was working yesterday on showing how to tune a model using caret functionalities. You can find example code here.
It's my understanding that this is at the level of the entire training set. If you want to be 100% sure on this, inspect the model itself as in stats <- summary(model, "modeldetails.txt") and look at the modeldetails.txt file. Hopefully the feature is not appearing if it occurs less than the number you set in feature.minfreq

[marctorsoc]

Thank you very much, awesome answer!

[marctorsoc]

then I understand that value 1 and 0 for min_freq is the same right?

[jwijffels]

please do not crosspost

• meaning of min_freq chokkan/crfsuite#107
• Question about active features chokkan/crfsuite#91