Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[rcore] Draft: IME support #2809

Open
wants to merge 9 commits into
base: master
Choose a base branch
from

Conversation

daipom
Copy link
Contributor

@daipom daipom commented Nov 25, 2022

sample

This is a fix for issue #1945.
We are mainly assuming Japanese input, but this feature can be used for other languages using IME as well.

We are implementing IME support features in GLFW now:

This fix adds the corresponding APIs to raylib.

It may take a while to be merged in GLFW, so we have created this PR as a draft ahead of time.

APIs to be added

  • void SetPreeditCallback(PreeditCallback callback)
    • PreeditCallback: void (*PreeditCallback)(int preeditLength, int *preeditString, int blockCount, int *blockSizes, int focusedBlock, int caret)
    • Preedit version of CharCallback.
    • An app gets preedit info (preedit text, blocks, and caret position) by this callback and draws preedit.
    • The reason why we have an app register the callback directly, which is a different from CharCallback, is that this is not like CharCallback, where the input is resolved for each character, but rather requires managing the entire current preedit info at all times. Therefore, it is inefficient for an app to ask for preedit info every frame, so we use a callback to notify the change of preedit info.
  • void SetPreeditCursorRectangle(int x, int y, int w, int h)
    • Set the area of the preedit cursor. This is used to decide the candidate window position that displays a list of conversion candidates. (For example, converting hiragana into kanji in Japanese).
    • In general, the position of this window should be managed by an app side by using this API.
  • void GetPreeditCursorRectangle(int *x, int *y, int *w, int *h)
    • The getter version of SetPreeditCursorRectangle.
  • void ResetPreedit(void)
    • Clear the current preedit.
  • bool IsImeOn(void)
    • Get if the IME is ON.
  • void SetImeStatus(bool on)
    • Set IME ON/OFF.

Feature to take candidate list (Win32 ONLY)

For a detailed explanation, please see below.

  • void SetPreeditCandidateCallback(PreeditCandidateCallback callback)
    • PreeditCandidateCallback: void (*PreeditCandidateCallback)(int candidatesCount, int selectedIndex, int pageStart, int pageSize)
    • The app takes the current candidate status by this callback.
    • To get the candidate text for each index, use GetPreeditCandidate.
  • int *GetPreeditCandidate(int index, int *textCount)
    • Get the current candiadate text of the specified index.

Specifications for each platform

Details are in the GLFW PR glfw/glfw#2130.

Win32

  • All APIs work fine.

macOS

  • All APIs work fine.

X11

  • Use over-the-spot style, so an app side doesn't need to draw preedit.
    • There are 2 main styles: on-the-spot and over-the-spot.
    • In on-the-spot style, an app side draws preedit, and this allows suitable drawing for each app. The other platforms use this style.
    • In over-the-spot style, the IME side draws preedit. This reduces what an app has to do, but it does not allow flexible display.
    • On X11, the behavior of on-the-spot style is not stable, so we use over-the-spot style.
  • Only SetPreeditWindowPosition and GetPreeditWindowPosition work, and other APIs don't work.
  • This is OK for this style because the IME side manages the drawing of preedit, so all an app side needs to do is managing the candidate window position.

Wayland

  • Currently, we support the minimum required APIs: SetPreeditCallback, SetPreeditWindowPosition, and GetPreeditWindowPosition.

Sample Application

We are implementing the sample application RaylibIMEInputSampleApp.

We can check all newly added APIs in this application.

Known issues

Additional planned fixes

Since the fix of GLFW is large, we will submit separate PRs to GLFW for the following features.
For this reason, we have also separated the fixes for raylib.

Feature to take candidate list (Win32 ONLY)

Usually, the IME displays the candidate window, so the app side doesn't need to get canidate list.

However, sometimes, we need to display it on the app side.
For example, it could be for displaying it correctly on exclusive fullscreen, or for using a custom design.

This is especially common in games for Windows.

This feature is OFF by default.
We can use this by setting the FLAG_MANAGE_PREEDIT_CANDIDATE config flag.

Currently, this feature is available only on Win32, since this feature is especially common on Windows.
We'd like to make it available on every platform if possible, but it may be difficult because of platform specific issues.

This feature is also involved in the issue with fullscreen and IME.
If fullscreen is strongly exclusive, then the candidate window of IME can't be displayed correctly.
This problem is especially serious in Windows, so this feature is one workaround for fullscreen on Windows.

We also have another fix that reduces fullscreen exclusivity for the IME, but the fix of GLFW for Windows is tricky...
(Please see Additional planned fixes)

We can check this behavior in the sample application with MANAGE_PREEDIT_CANDIDATE cmake option.

@daipom
Copy link
Contributor Author

daipom commented Nov 25, 2022

It has not yet been merged into GLFW, so the fails of the tests are as expected.

Please see the wiki of RaylibIMEInputSampleApp about how to check this feature.

@raysan5
Copy link
Owner

raysan5 commented Nov 28, 2022

@daipom Thank you very much for all the hard work on this new feature! I couldn't review it carefully yet, probably some function could be renamed but the example looks fantastic! Thanks! 😄

@daipom
Copy link
Contributor Author

daipom commented Nov 29, 2022

Thanks!
If you have any questions about this feature or see something to be fixed, please let me know!

@daipom
Copy link
Contributor Author

daipom commented Nov 29, 2022

I have just rebased this branch to the latest master.

@daipom
Copy link
Contributor Author

daipom commented Nov 29, 2022

@raysan5

In connection with #2814, many characters can be entered at one time when using an IME.
So the current MAX_CHAR_PRESSED_QUEUE value is small for input using IME.

raylib/src/rcore.c

Lines 335 to 337 in 50a716c

#ifndef MAX_CHAR_PRESSED_QUEUE
#define MAX_CHAR_PRESSED_QUEUE 16 // Maximum number of characters in the char input queue
#endif

It would be great if this could be made larger, such as 64, or if it could be changed on the application side.
What would you think about this?

@daipom
Copy link
Contributor Author

daipom commented Nov 30, 2022

I have fixed some APIs: SetPreeditCursorRectangle, GetPreeditCursorRectangle.

@daipom
Copy link
Contributor Author

daipom commented Dec 15, 2022

I rebased this branch to the latest master and added the feature to take candidate list (Win32 ONLY).
I have updated the initial comment on this PR, so please see it about this feature.

If you would like this feature to be separated into another PR, please let me know.
As for this feature, it is summarized in the last commit and can be easily removed.

@raysan5
Copy link
Owner

raysan5 commented Sep 7, 2023

@daipom Hi! It's been quite long since this amazing improvement was proposed, what is the current state? Is there some workaround to allow this feature to be added in raylib?

@Peter0x44
Copy link
Contributor

Peter0x44 commented Oct 16, 2023

This will definitely require some rebasing and modifications now the platform split is merged, to move the relevant code to rcore_desktop.c

@DreepyYunky
Copy link

DreepyYunky commented Nov 25, 2023

I think It'll be better to change SetImeStatus() to TurnIMEOn() or InitIME().

@raysan5
Copy link
Owner

raysan5 commented Dec 15, 2023

I just saw there was some update on this PR. I'm keeping it open for a bit longer, it seems lately GLFW is receiving several updates...

@raysan5 raysan5 changed the title Draft: IME support [rcore] Draft: IME support Feb 4, 2024
@ashie
Copy link

ashie commented Mar 11, 2024

Hi, I'm a collaborator of this work.
I've rebased this branch on top of current master (1fad827).
The previous one is backed up as https://github.com/clear-code/raylib/tree/4.2-2022-12-14%2Bbetter-ime-support
Since bundled GLFW isn't support IME yet, CI will fail as well as before.

I just saw there was some update on this PR. I'm keeping it open for a bit longer, it seems lately GLFW is receiving several updates...

IME support of GLFW isn't merged yet, it's planned as a part of v3.5: https://github.com/glfw/glfw/milestone/25
So we need to wait next minor update of GLFW (current is v3.4 released 2 weeks ago).

@ashie
Copy link

ashie commented Mar 11, 2024

A commit is added automatically by CI:
ccf2ddf

I'm not sure we can leave it as is, or we should do something more or not.

daipom and others added 8 commits August 19, 2024 16:43
This is for GLFW3 Preedit Callback: glfwSetPreeditCallback
This is for GLFW3: glfwSetPreeditCursorRectangle
This is for GLFW3: glfwGetPreeditCursorRectangle
This is for GLFW3: glfwGetInputMode (mode: GLFW_IME)
This is for GLFW3: glfwSetInputMode (mode: GLFW_IME)
This is for GLFW3: glfwResetPreeditText
Support the GLFW preedit candidate feature.
This feature supports only Win32 currently.
We can use this feature by enabling `FLAG_MANAGE_PREEDIT_CANDIDATE` flag
on Win32.
@raysan5
Copy link
Owner

raysan5 commented Aug 19, 2024

@daipom @ashie Thanks for the update! Glad to see that this PR is still active! Very nice! 😄

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants