🚔 feat: OpenAI Moderation update

[berry-13]

Summary

This pull request introduces significant improvements to the moderation functionality and configuration management within the codebase. The key changes include adding a new violation type, enhancing the moderation middleware, and refactoring the configuration loading process for better caching and error handling

Moderation Enhancements:

• api/server/middleware/moderateText.js: Replaced axios with OpenAI for moderation requests, added new functions for formatting violations, and implemented a more robust moderation flow with configurable actions and thresholds

Configuration Management:

• api/server/services/Config/loadCustomConfig.js: Refactored the configuration loading process to include caching, error handling, and validation improvements. [1] [2]
• packages/data-provider/src/config.ts: Added a new moderationSchema to the configuration schema and included a new ViolationTypes.MODERATION enumeration. [1] [2] [3]

Middleware Adjustments:

• api/server/routes/ask/gptPlugins.js, api/server/routes/ask/openAI.js, api/server/routes/edit/gptPlugins.js, api/server/routes/edit/openAI.js: Moved the moderateText middleware to ensure it is applied correctly across routes. [1] [2] [3] [4]

Cache Updates:

• api/cache/getLogStores.js: Added a new violation type to the namespaces object

Change Type

• Refactor
• New feature (non-breaking change which adds functionality)

Testing

Tested with the OpenAI's endpoint by sending explicit messages containing self-harm/intent, self-harm, hate, harassment and sexual words

Checklist

• My code adheres to this project's style guidelines
• I have performed a self-review of my own code
• I have commented in any complex areas of my code
• I have made pertinent documentation changes
• My changes do not introduce new warnings
• I have written tests demonstrating that my changes are effective or that my feature works
• Local unit tests pass with my changes
• Any changes dependent on mine have been merged and published in downstream modules.
• A pull request for updating the documentation has been submitted.