fix: unify token_limit property

[Wendong-Fan]

Description

if string value is passed to model_type when calling commercial model platforms like Mistral then the token_limit value would missing

also update OPENAI_COMPATIBLE_MODEL example and doc

Motivation and Context

• I have raised an issue to propose this change (required for new features and bug fixes)

Types of changes

What types of changes does your code introduce? Put an x in all the boxes that apply:

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds core functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Documentation (update in the documentation)
• Example (update in the folder of example)

Implemented Tasks

• Subtask 1
• Subtask 2
• Subtask 3

Checklist

Go over all the following points, and put an x in all the boxes that apply.
If you are unsure about any of these, don't hesitate to ask. We are here to help!

• I have read the CONTRIBUTION guide. (required)
• My change requires a change to the documentation.
• I have updated the tests accordingly. (required for a bug fix or a new feature)
• I have updated the documentation accordingly.

[WHALEEYE]

Do you think it's better to directly move the default value of token limit (4096) to the token_limit property of UnifiedModelType?

[Wendong-Fan]

Do you think it's better to directly move the default value of token limit (4096) to the token_limit property of UnifiedModelType?

If we move this under UnifiedModelType then it would be hard for us to give warming to user, I feel it would be better to put under BaseModel

[WHALEEYE]

I feel like it's okay to give warning in the UnifiedModelType, as the token_limit inside is a pre-set default value, and it will only be called when a user tries to get token limit for an unrecognized str

[Wendong-Fan]

I feel like it's okay to give warning in the UnifiedModelType, as the token_limit inside is a pre-set default value, and it will only be called when a user tries to get token limit for an unrecognized str

what is user passed an unrecognized str but set the token value in model_config_dict?

[WHALEEYE]

Then In our code I think the token_limit property will not be accessed given that the value in config dict is not None?

[Wendong-Fan]

@WHALEEYE updated

[lightaime]

Is 4096 too small for most of the model now? Should we just change it to large?

[lightaime]

Should we set it to a very large number instead?

[Wendong-Fan]

Should we set it to a very large number instead?

great point, updated