It’s a post about some discoveries/ideas related to custom keyboard layouts on macos based on my very recent experience fixing them.

I created a layout bundle with custom keyboard layouts based on the standard US English and Russian layouts a few years ago. It worked fine until macos 12.

But on 12.4, one of my macbook pros had no issues with it, whereas another, very similar MBP had weird problems. By that I mean that the custom layouts worked in most applications, but in Mail.app, TextEdit.app, Console.app and maybe others, the custom US layout refused to work, I just couldn’t switch to it at all, the system would enable the system US layout instead; meanwhile my custom Russian one worked fine everywhere. And both macs had exactly the same bundle. Another bizarre thing is that I couldn’t remove the custom layouts from the “Keyboard > Input Sources” preference pane in System Preferences at all; that is, I would remove them, reopen the dialog or restart the system, and see the system US layout, my custom US layout and three Russian layouts still in place! Console.app is completely unusable and useless since the rewrite a few years ago and I couldn’t find any errors about layouts there. Upgrade to macos 12.6.1 didn’t change a thing. I also tried using the bundle by a fresh user, which had the same issues.

Removing stuck layouts

First, to actually remove the old layouts, I’ve found that they are stored in ~/Library/Preferences/com.apple.inputsources.plist and the file can be viewed using:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

$ plutil -convert xml1 -o - ~/Library/Preferences/com.apple.inputsources.plist
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
        <key>AppleEnabledThirdPartyInputSources</key>
        <array>
                <dict>
                        <key>InputSourceKind</key>
                        <string>Keyboard Layout</string>
                        <key>KeyboardLayout ID</key>
                        <integer>-28676</integer>
                        <key>KeyboardLayout Name</key>
                        <string>US_Deutsch</string>
                </dict>
                <dict>
                        <key>InputSourceKind</key>
                        <string>Keyboard Layout</string>
                        <key>KeyboardLayout ID</key>
                        <integer>-14185</integer>
                        <key>KeyboardLayout Name</key>
                        <string>Russian_ё</string>
                </dict>
        </array>
</dict>
</plist>

By default the plist is stored using a binary format, so to edit it, I had to convert it to XML with plutil -convert xml1 ~/Library/Preferences/com.apple.inputsources.plist. Then I removed everything from inside the <array> tag and restarted. This did the trick.

Fixing the layout

Apparently the main issue was that the custom English layout’s id was 1. I had done that on purpose in order to replace the system one and be able to switch layouts with Caps Lock; of course, it’s very hard to do that now in macos 12 (if at all possible), so I changed the identifier in Ukelele (or you can edit the .keylayout file in a text editor). The Caps Lock switch is an official flag in Ukelele now: check the “Caps Lock Switch” checkbox for non-English layouts.

In Ukelele, if you open a layout so that you see the keyboard representation, you can click the Info toolbar button to open the Inspector. Changing the Script from and to Unicode changes the layout id, which is very useful to find out whether the system does use the updated bundle: use the plutil command from above and check the values for KeyboardLayout ID.

A more obvious way is to rename the layouts; I did it for both of mine, but the Russian one continued to have the old name even though Ukelele showed the new one. It turned out I had to right-click on it in the list, select “Localise Keyboard Name…” and rename it for all languages in the dialog too; for some reason, I didn’t have to do that for the English layout.

Fixing the Caps Lock switch

Switching keyboard layouts with Caps Lock is a nice and quick action, which was originally enabled by Karabiner and then added to OS X. Unfortunately it didn’t always work, and even now, while working on one MBP, it failed on this other MBP.

The Keyboard preferences have the “Use the Caps Lock key to switch to and from last used Latin input source” checkbox checked, but it never worked on this MBP with layout issues. After fixing the layouts, it continued not working. I went to the Keyboard tab, clicked “Modifier Keys…”, switched to “Apple Internal Keyboard / Trackpad” (I have no idea why it was set to “Bluetooth USB Host Controller” by default), toggled the Caps Lock key to “No Action”, confirmed my choice, then switched back to “Caps Lock”. And magic, it started working!

By the way, those two original layouts are still present in the picker even though the old bundle is gone from ~/Library/Keyboard Layouts/ and I tried restarting as well?!

My keyboard layouts for reference: custom_kbd.bundle.zip.