Skip to content

Commit

Permalink
Merge pull request #717 from 10up/feature/dalle3
Browse files Browse the repository at this point in the history 
Upgrade from DALL·E 2 to DALL·E 3
  • Loading branch information
@dkotter
dkotter authored Feb 21, 2024
2 parents 75bc776 + af9c669 commit 8c140f5
Show file tree
Hide file tree
Showing 7 changed files with 159 additions and 58 deletions.
4 changes: 2 additions & 2 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ Tap into leading cloud-based services like [OpenAI](https://openai.com/), [Micro
* Generate a summary of post content and store it as an excerpt using [OpenAI's ChatGPT API](https://platform.openai.com/docs/guides/chat), [Microsoft Azure's OpenAI service](https://azure.microsoft.com/en-us/products/ai-services/openai-service) or [Google's Gemini API](https://ai.google.dev/docs/gemini_api_overview)
* Generate titles from post content using [OpenAI's ChatGPT API](https://platform.openai.com/docs/guides/chat), [Microsoft Azure's OpenAI service](https://azure.microsoft.com/en-us/products/ai-services/openai-service) or [Google's Gemini API](https://ai.google.dev/docs/gemini_api_overview)
* Expand or condense text content using [OpenAI's ChatGPT API](https://platform.openai.com/docs/guides/chat), [Microsoft Azure's OpenAI service](https://azure.microsoft.com/en-us/products/ai-services/openai-service) or [Google's Gemini API](https://ai.google.dev/docs/gemini_api_overview)
* Generate new images on demand to use in-content or as a featured image using [OpenAI's DALL·E API](https://platform.openai.com/docs/guides/images)
* Generate new images on demand to use in-content or as a featured image using [OpenAI's DALL·E 3 API](https://platform.openai.com/docs/guides/images)
* Generate transcripts of audio files using [OpenAI's Whisper API](https://platform.openai.com/docs/guides/speech-to-text)
* Moderate incoming comments for sensitive content using [OpenAI's Moderation API](https://platform.openai.com/docs/guides/moderation)
* Convert text content into audio and output a "read-to-me" feature on the front-end to play this audio using [Microsoft Azure's Text to Speech API](https://learn.microsoft.com/en-us/azure/cognitive-services/speech-service/text-to-speech)
Expand Down Expand Up @@ -438,7 +438,7 @@ Note that [Azure AI Vision](https://docs.microsoft.com/en-us/azure/cognitive-ser

### 2. Configure OpenAI API Keys under Tools > ClassifAI > Image Processing > Image Generation

* Select **OpenAI DALL-E** in the provider dropdown.
* Select **OpenAI DALL·E 3** in the provider dropdown.
* Enter your API Key copied from the above step into the `API Key` field.

### 3. Enable specific Image Processing features
Expand Down
4 changes: 3 additions & 1 deletion includes/Classifai/Features/Feature.php
View file
Original file line number Diff line number Diff line change
Expand Up @@ -1179,11 +1179,13 @@ function ( $role ) {
__( 'Provider', 'classifai' ) => $feature_settings['provider'],
];

if ( method_exists( $provider, 'get_debug_information' ) ) {
if ( $provider && method_exists( $provider, 'get_debug_information' ) ) {
$all_debug_info = array_merge(
$common_debug_info,
$provider->get_debug_information()
);
} else {
$all_debug_info = $common_debug_info;
}

/**
Expand Down
2 changes: 1 addition & 1 deletion includes/Classifai/Features/ImageGeneration.php
View file
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ public function __construct() {

// Contains just the providers this feature supports.
$this->supported_providers = [
DallE::ID => __( 'OpenAI Dall-E', 'classifai' ),
DallE::ID => __( 'OpenAI DALL·E 3', 'classifai' ),
];
}

Expand Down
185 changes: 138 additions & 47 deletions includes/Classifai/Providers/OpenAI/DallE.php
View file
Original file line number Diff line number Diff line change
Expand Up @@ -18,18 +18,18 @@ class DallE extends Provider {
const ID = 'openai_dalle';

/**
* OpenAI DALL·E URL
* OpenAI DALL·E URL.
*
* @var string
*/
protected $dalle_url = 'https://api.openai.com/v1/images/generations';

/**
* Maximum number of characters a prompt can have
* Maximum number of characters a prompt can have.
*
* @var int
*/
public $max_prompt_chars = 1000;
public $max_prompt_chars = 4000;

/**
* OpenAI DALL·E constructor.
Expand Down Expand Up @@ -64,33 +64,53 @@ public function register_endpoints() {
'methods' => WP_REST_Server::READABLE,
'callback' => [ $this->feature_instance, 'rest_endpoint_callback' ],
'args' => [
'prompt' => [
'prompt' => [
'required' => true,
'type' => 'string',
'sanitize_callback' => 'sanitize_text_field',
'validate_callback' => 'rest_validate_request_arg',
'description' => esc_html__( 'Prompt used to generate an image', 'classifai' ),
],
'n' => [
'n' => [
'type' => 'integer',
'minimum' => 1,
'maximum' => 10,
'sanitize_callback' => 'absint',
'validate_callback' => 'rest_validate_request_arg',
'description' => esc_html__( 'Number of images to generate', 'classifai' ),
],
'size' => [
'quality' => [
'type' => 'string',
'enum' => [
'standard',
'hd',
],
'sanitize_callback' => 'sanitize_text_field',
'validate_callback' => 'rest_validate_request_arg',
'description' => esc_html__( 'Quality of generated image', 'classifai' ),
],
'size' => [
'type' => 'string',
'enum' => [
'256x256',
'512x512',
'1024x1024',
'1792x1024',
'1024x1792',
],
'sanitize_callback' => 'sanitize_text_field',
'validate_callback' => 'rest_validate_request_arg',
'description' => esc_html__( 'Size of generated image', 'classifai' ),
],
'format' => [
'style' => [
'type' => 'string',
'enum' => [
'vivid',
'natural',
],
'sanitize_callback' => 'sanitize_text_field',
'validate_callback' => 'rest_validate_request_arg',
'description' => esc_html__( 'Style of generated image', 'classifai' ),
],
'format' => [
'type' => 'string',
'enum' => [
'url',
Expand Down Expand Up @@ -123,7 +143,7 @@ public function render_provider_fields() {
'label_for' => 'api_key',
'input_type' => 'password',
'default_value' => $settings['api_key'],
'class' => 'classifai-provider-field hidden provider-scope-' . static::ID, // Important to add this.
'class' => 'classifai-provider-field hidden provider-scope-' . static::ID,
'description' => sprintf(
wp_kses(
/* translators: %1$s is replaced with the OpenAI sign up URL */
Expand All @@ -141,7 +161,7 @@ public function render_provider_fields() {
);

add_settings_field(
static::ID . 'number_of_images',
static::ID . '_number_of_images',
esc_html__( 'Number of images', 'classifai' ),
[ $this->feature_instance, 'render_select' ],
$this->feature_instance->get_option_name(),
Expand All @@ -152,12 +172,31 @@ public function render_provider_fields() {
'options' => array_combine( range( 1, 10 ), range( 1, 10 ) ),
'default_value' => $settings['number_of_images'],
'description' => __( 'Number of images that will be generated in one request. Note that each image will incur separate costs.', 'classifai' ),
'class' => 'classifai-provider-field hidden provider-scope-' . static::ID, // Important to add this.
'class' => 'classifai-provider-field hidden provider-scope-' . static::ID,
]
);

add_settings_field(
static::ID . 'image_size',
static::ID . '_quality',
esc_html__( 'Image quality', 'classifai' ),
[ $this->feature_instance, 'render_select' ],
$this->feature_instance->get_option_name(),
$this->feature_instance->get_option_name() . '_section',
[
'option_index' => static::ID,
'label_for' => 'quality',
'options' => [
'standard' => __( 'Standard', 'classifai' ),
'hd' => __( 'High Definition', 'classifai' ),
],
'default_value' => $settings['quality'],
'description' => __( 'The quality of the image that will be generated. High Definition creates images with finer details and greater consistency across the image but costs more.', 'classifai' ),
'class' => 'classifai-provider-field hidden provider-scope-' . static::ID,
]
);

add_settings_field(
static::ID . '_image_size',
esc_html__( 'Image size', 'classifai' ),
[ $this->feature_instance, 'render_select' ],
$this->feature_instance->get_option_name(),
Expand All @@ -166,13 +205,32 @@ public function render_provider_fields() {
'option_index' => static::ID,
'label_for' => 'image_size',
'options' => [
'256x256' => '256x256',
'512x512' => '512x512',
'1024x1024' => '1024x1024',
'1024x1024' => '1024x1024 (square)',
'1792x1024' => '1792x1024 (landscape)',
'1024x1792' => '1024x1792 (portrait)',
],
'default_value' => $settings['image_size'],
'description' => __( 'Size of generated images.', 'classifai' ),
'class' => 'classifai-provider-field hidden provider-scope-' . static::ID, // Important to add this.
'description' => __( 'Size of generated images. Larger sizes cost more.', 'classifai' ),
'class' => 'classifai-provider-field hidden provider-scope-' . static::ID,
]
);

add_settings_field(
static::ID . '_style',
esc_html__( 'Image style', 'classifai' ),
[ $this->feature_instance, 'render_select' ],
$this->feature_instance->get_option_name(),
$this->feature_instance->get_option_name() . '_section',
[
'option_index' => static::ID,
'label_for' => 'style',
'options' => [
'vivid' => __( 'Vivid', 'classifai' ),
'natural' => __( 'Natural', 'classifai' ),
],
'default_value' => $settings['style'],
'description' => __( 'The style of the generated images. Vivid causes more hyper-real and dramatic images. Natural causes more natural, less hyper-real looking images.', 'classifai' ),
'class' => 'classifai-provider-field hidden provider-scope-' . static::ID,
]
);
}
Expand All @@ -194,7 +252,9 @@ public function get_default_provider_settings(): array {
$common_settings,
[
'number_of_images' => 1,
'image_size' => '256x256',
'quality' => 'standard',
'image_size' => '1024x1024',
'style' => 'vivid',
]
);
}
Expand All @@ -217,8 +277,22 @@ public function sanitize_settings( array $new_settings ): array {
if ( $this->feature_instance instanceof ImageGeneration ) {
$new_settings[ static::ID ]['number_of_images'] = absint( $new_settings[ static::ID ]['number_of_images'] ?? $settings[ static::ID ]['number_of_images'] );

if ( in_array( $new_settings[ static::ID ]['image_size'], [ '256x256', '512x512', '1024x1024' ], true ) ) {
$new_settings[ static::ID ]['image_size'] = sanitize_text_field( $new_settings[ static::ID ]['image_size'] ?? $settings[ static::ID ]['image_size'] );
if ( in_array( $new_settings[ static::ID ]['quality'], [ 'standard', 'hd' ], true ) ) {
$new_settings[ static::ID ]['quality'] = sanitize_text_field( $new_settings[ static::ID ]['quality'] );
} else {
$new_settings[ static::ID ]['quality'] = $settings[ static::ID ]['quality'];
}

if ( in_array( $new_settings[ static::ID ]['image_size'], [ '1024x1024', '1792x1024', '1024x1792' ], true ) ) {
$new_settings[ static::ID ]['image_size'] = sanitize_text_field( $new_settings[ static::ID ]['image_size'] );
} else {
$new_settings[ static::ID ]['image_size'] = $settings[ static::ID ]['image_size'];
}

if ( in_array( $new_settings[ static::ID ]['style'], [ 'vivid', 'natural' ], true ) ) {
$new_settings[ static::ID ]['style'] = sanitize_text_field( $new_settings[ static::ID ]['style'] );
} else {
$new_settings[ static::ID ]['style'] = $settings[ static::ID ]['style'];
}
}

Expand Down Expand Up @@ -264,12 +338,19 @@ public function generate_image( string $prompt = '', array $args = [] ) {
$args = wp_parse_args(
array_filter( $args ),
[
'num' => $settings['number_of_images'] ?? 1,
'size' => $settings['image_size'] ?? '1024x1024',
'format' => 'url',
'num' => $settings['number_of_images'] ?? 1,
'quality' => $settings['quality'] ?? 'standard',
'size' => $settings['image_size'] ?? '1024x1024',
'style' => $settings['style'] ?? 'vivid',
'format' => 'url',
]
);

// Force proper image size for those that had been using DALL·E 2 and haven't updated settings.
if ( ! in_array( $args['size'], [ '1024x1024', '1792x1024', '1024x1792' ], true ) ) {
$args['size'] = '1024x1024';
}

if ( ! $image_generation->is_feature_enabled() ) {
return new WP_Error( 'not_enabled', esc_html__( 'Image generation is disabled or OpenAI authentication failed. Please check your settings.', 'classifai' ) );
}
Expand Down Expand Up @@ -307,40 +388,47 @@ public function generate_image( string $prompt = '', array $args = [] ) {
'classifai_dalle_request_body',
[
'prompt' => sanitize_text_field( $prompt ),
'n' => absint( $args['num'] ),
'size' => sanitize_text_field( $args['size'] ),
'model' => 'dall-e-3',
'n' => 1,
'quality' => sanitize_text_field( $args['quality'] ),
'response_format' => sanitize_text_field( $args['format'] ),
'size' => sanitize_text_field( $args['size'] ),
'style' => sanitize_text_field( $args['style'] ),
]
);

// Make our API request.
$response = $request->post(
$this->dalle_url,
[
'body' => wp_json_encode( $body ),
]
);
$responses = [];

set_transient( 'classifai_openai_dalle_latest_response', $response, DAY_IN_SECONDS * 30 );
// DALL·E 3 doesn't support multiple images in a single request so make one request per image.
for ( $i = 0; $i < $args['num']; $i++ ) {
$responses[] = $request->post(
$this->dalle_url,
[
'body' => wp_json_encode( $body ),
]
);
}

set_transient( 'classifai_openai_dalle_latest_response', $responses[ array_key_last( $responses ) ], DAY_IN_SECONDS * 30 );

// Extract out the image response, if it exists.
if ( ! is_wp_error( $response ) && ! empty( $response['data'] ) ) {
$cleaned_response = [];
$cleaned_responses = [];

foreach ( $response['data'] as $data ) {
if ( ! empty( $data[ $args['format'] ] ) ) {
if ( 'url' === $args['format'] ) {
$cleaned_response[] = [ 'url' => esc_url_raw( $data[ $args['format'] ] ) ];
} else {
$cleaned_response[] = [ 'url' => $data[ $args['format'] ] ];
foreach ( $responses as $response ) {
// Extract out the image response, if it exists.
if ( ! is_wp_error( $response ) && ! empty( $response['data'] ) ) {
foreach ( $response['data'] as $data ) {
if ( ! empty( $data[ $args['format'] ] ) ) {
if ( 'url' === $args['format'] ) {
$cleaned_responses[] = [ 'url' => esc_url_raw( $data[ $args['format'] ] ) ];
} else {
$cleaned_responses[] = [ 'url' => $data[ $args['format'] ] ];
}
}
}
}

$response = $cleaned_response;
}

return $response;
return $cleaned_responses;
}

/**
Expand All @@ -354,7 +442,10 @@ public function get_debug_information(): array {
$debug_info = [];

if ( $this->feature_instance instanceof ImageGeneration ) {
$debug_info[ __( 'Number of images', 'classifai' ) ] = $provider_settings['number_of_images'];
$debug_info[ __( 'Number of images', 'classifai' ) ] = $provider_settings['number_of_images'] ?? 1;
$debug_info[ __( 'Quality', 'classifai' ) ] = $provider_settings['quality'] ?? 'standard';
$debug_info[ __( 'Size', 'classifai' ) ] = $provider_settings['image_size'] ?? '1024x1024';
$debug_info[ __( 'Style', 'classifai' ) ] = $provider_settings['style'] ?? 'vivid';
$debug_info[ __( 'Latest response:', 'classifai' ) ] = $this->get_formatted_latest_response( get_transient( 'classifai_openai_dalle_latest_response' ) );
}

Expand Down
2 changes: 1 addition & 1 deletion readme.txt
View file
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ Tap into leading cloud-based services like [OpenAI](https://openai.com/), [Micro
* Generate a summary of post content and store it as an excerpt using [OpenAI's ChatGPT API](https://platform.openai.com/docs/guides/chat), [Microsoft Azure's OpenAI service](https://azure.microsoft.com/en-us/products/ai-services/openai-service) or [Google's Gemini API](https://ai.google.dev/docs/gemini_api_overview)
* Generate titles from post content using [OpenAI's ChatGPT API](https://platform.openai.com/docs/guides/chat), [Microsoft Azure's OpenAI service](https://azure.microsoft.com/en-us/products/ai-services/openai-service) or [Google's Gemini API](https://ai.google.dev/docs/gemini_api_overview)
* Expand or condense text content using [OpenAI's ChatGPT API](https://platform.openai.com/docs/guides/chat), [Microsoft Azure's OpenAI service](https://azure.microsoft.com/en-us/products/ai-services/openai-service) or [Google's Gemini API](https://ai.google.dev/docs/gemini_api_overview)
* Generate new images on demand to use in-content or as a featured image using [OpenAI's DALL·E API](https://platform.openai.com/docs/guides/images)
* Generate new images on demand to use in-content or as a featured image using [OpenAI's DALL·E 3 API](https://platform.openai.com/docs/guides/images)
* Generate transcripts of audio files using [OpenAI's Whisper API](https://platform.openai.com/docs/guides/speech-to-text)
* Convert text content into audio and output a "read-to-me" feature on the front-end to play this audio using [Microsoft Azure's Text to Speech API](https://learn.microsoft.com/en-us/azure/cognitive-services/speech-service/text-to-speech)
* Classify post content using [IBM Watson's Natural Language Understanding API](https://www.ibm.com/watson/services/natural-language-understanding/) and [OpenAI's Embedding API](https://platform.openai.com/docs/guides/embeddings)
Expand Down
17 changes: 14 additions & 3 deletions tests/cypress/integration/image-processing/image-generation-openai-dalle.test.js
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ describe( 'Image Generation (OpenAI DALL·E) Tests', () => {
'/wp-admin/tools.php?page=classifai&tab=image_processing&feature=feature_image_generation'
);
cy.get( '#status' ).check();
cy.get( '#provider' ).select( 'openai_dalle' );
cy.get( '#submit' ).click();
cy.optInAllFeatures();
} );
Expand All @@ -19,13 +20,23 @@ describe( 'Image Generation (OpenAI DALL·E) Tests', () => {
);

cy.get( '#api_key' ).clear().type( 'password' );
cy.get(
'select[name="classifai_feature_image_generation[openai_dalle][number_of_images]"]'
).select( '2' );
cy.get(
'select[name="classifai_feature_image_generation[openai_dalle][quality]"]'
).select( 'hd' );
cy.get(
'select[name="classifai_feature_image_generation[openai_dalle][image_size]"]'
).select( '1024x1792' );
cy.get(
'select[name="classifai_feature_image_generation[openai_dalle][style]"]'
).select( 'natural' );

cy.get( '#status' ).check();
cy.get(
'#classifai_feature_image_generation_roles_administrator'
).check();
cy.get( '#number_of_images' ).select( '2' );
cy.get( '#image_size' ).select( '512x512' );

cy.get( '#submit' ).click();
} );

Expand Down
Loading

0 comments on commit 8c140f5

Please sign in to comment.