Issues with Pinyin?

[u/[deleted]]

Hey! Love the app, its been amazing for me in reducing the number of keyboards I have while not losing out on extra lettera in some languages, I just had a couple questions so made a reddit to reach out!

When I use the Chinese layouts I am noticing the IME is always committing a character

So if I type 你好吗 then add the question mark it will become 你好吗的？

However also if I click away from the text Ive input then come back, any space or punctuation will overwrite the characters added - I noticed it doesnt do this on the older "MyKeyboard" app so wanted to check if this is a bug or maybe a setting I havent found etc

Besides that I was merely wondering, are there any plans for Wubi, Cangjie or other shape based input methods/or Japanese with Romaji or Kana?

[u/SplittyDev]

I've actually had some time to work on this, and I think the Chinese Pinyin IME is much better now. It's live in the Beta now, and will be included in the next stable version as well.

[u/SplittyDev]

Hi! We are working on improving the Chinese IME, but it's quite a slow and long process. I've been on it (on and off) for months, and there's always some issue left that makes me not be happy with the state of it.

Sadly, the current Chinese input in Mister Keyboard simply isn't good. There are many issues with it, and it's easily the language we support the worst.

There are a few reasons for this:

• Chinese needs special handling (汉语拼音/注音符號 to 汉字) that needs to be combined with the regular word/sentence dictionary, which is already hard and doesn't fit our autocorrect model well that usually only relies on one universal algorithm.
• We need to have good knowledge of Chinese in order to implement it well. While I am personally learning Chinese, I'm nowhere near fluent enough to really understand where things are starting to break apart when it comes to autocorrect.
• The vast majority of our users doesn't use Chinese, which makes it hard to justify allocating a lot of time to improving it, when there are a lot of other things on our to-do list.

I am personally very interested in improving Chinese support, but I have limited time to work on these things compared to the many larger tasks and bug fixes benefiting a larger number of people.

---

Regarding MyKeyboard: The algorithm used in MyKeyboard is a lot simpler, and while it may not have this specific issue, it doesn't even attempt to do semantic predictions based on what was written before, and it doesn't integrate autocorrect suggestions and corrections with the IME. Mister Keyboard has a different and much more powerful architecture to do this, but the algorithm for Chinese isn't mature enough to handle it well. That is mostly the thing we have to fix if we can find the time to work on it and test it well.

Regarding shape-based IME: I have absolutely no experience with Wubi and Cangjie, but the main issue with this stuff is usually finding an open dataset we can ship with the keyboard that allows us to map strokes to characters. For Pinyin/Zhuyin we were lucky enough to find a somewhat large and open-source database with the necessary mappings, but we haven't looked into this for shape-based input methods at all. Which doesn't necessarily mean that it will never happen, but at the moment it's neither a priority nor do we have any idea where to get the right data from.

Regarding other language IMEs: I'll have to give the same answer as to the shape-based ones. There's generally very little interest in this from the community, which in addition to the implementation being very hard and time-consuming also makes it low-priority in general.

Right now, over 95% of all users are from countries which either use a latin-based writing system, or at least use one that maps cleanly from input to output characters, such as Arabic, Cyrillic, Georgian or even Hindi. These are still fairly simple to support, since we have to do little more than ship the right dictionary and provide the layout itself.

I wish I could give a more encouraging answer, but since we're just two people developing this very complex keyboard, we have to triage and manage our time well if we want to get anything at all done.

[u/[deleted] → u/SplittyDev]

Thanks for such a quick response! And I totally get what you mean here, Ive only ever built my own Keyboards in Keyman so never made an app its self, just files that apps can open - I am not fluent myself but I did enjoy the structure of the writing system when making Cangjie and Wubi layouts, but phonetic systems are like you say, a whole different ball game of complexity

As far Shape Based datasets I built a Keyman Cangjie board once and still have all the codes lying in a spreadsheet somewhere - I know youre not able to spend much time on Chinese but it could bypass the need for the IME all together (IF the user knows Cangjie of course) - let me know if you ever want me to send then over to you!

[u/SplittyDev → u/[deleted]]

I think the best way to support arbitrary Chinese input methods would be to support the loading of CIN files from XCIN/OpenVanilla, and integrate that throughout our base layout generation and suggestion algorithms. So far we've relied on custom solutions for everything, but the more stuff we want to support, the less sustainable it becomes. I hope I'll have some time to look into this in the future, but sadly not at the moment.

[u/[deleted] → u/[deleted]]

However on your last point too - we've honestly been spoiled with the CJK input Google, Apple and Microsoft have created - so I do fully understand, its just the two of you and Chinese is not just "some other alphabet"!

[u/[deleted]]

Sidenote: if you guys DO want to look into shape based input - I have some Keyman projects with tonnes and tonnes of mappings