performance problems with X server freeze by inserting symbols like ä about xdotool HOT 42 OPEN

I just wanted to say that this issue still happens when multiple keyboard layouts are used. It seems to only affect the layout which is not the default one, as changing the order in "Input sources" (in gnome), the layout on top will run flawlessly but the second one will have that delay.

from xdotool.

I see two tests here:

I just wanted to say that this issue still happens when multiple keyboard layouts are used. It seems to only affect the layout which is not the default one, as changing the order in "Input sources" (in gnome), the layout on top will run flawlessly but the second one will have that delay.

and btw it is freezing when try do xdotool type "iiiiiiiiii" with russian keyboard layout and it is freezing when I do xdotool type "шшшшшшшшшш" with english layout. With correct layout it works without freezing. Very easy reproducible issue.

I could replicate both of the explained behaviors on Ubuntu 20.04.

from xdotool.

+1 for this issue

and btw it is freezing when try do xdotool type "iiiiiiiiii" with russian keyboard layout and it is freezing when I do xdotool type "шшшшшшшшшш" with english layout. With correct layout it works without freezing. Very easy reproducible issue.

from xdotool.

+1

from xdotool.

@Drek282 the man pages of xdotool is written:

... It does this using X11's XTEST extension and other Xlib functions.

the source of file https://github.com/jordansissel/xdotool/blob/master/xdo.h#L12-L13 has:

#include <X11/Xlib.h>
#include <X11/X.h>

Here are the links for such libs:

https://packages.debian.org/sid/libxdo-dev
https://packages.debian.org/sid/libxdo3
https://gitlab.freedesktop.org/xorg/lib/libx11
https://www.x.org/releases/current/doc/libX11/libX11/libX11.html

you may also want to check this xdo function references:

https://libxdo-d.dpldocs.info/xdo.xdo_send_keysequence_window.html

In order to fix this issue I think we need to address where this bug occurs in X11 lib.

from xdotool.

So I'm not really sure why, but on a Ubuntu 22.04 (Xorg) VM, any xdotool typing takes at least one second, freezing the entire display for that duration. I don't see the problem limited to special chars like ä. But maybe it's something else.

Same with python-evdev, something different entirely, not even depending on x11 (!): https://python-evdev.readthedocs.io/en/latest/tutorial.html#injecting-input this example also freezes up briefly.

AutoKey, on the other hand, is able to type instantaneously! No idea why, it doesn't seem to do anything special either: https://github.com/autokey/autokey/blob/master/lib/autokey/interface.py#L1105

from xdotool.

Confirmed, producing 200 characters (200 bytes for a, 400 bytes for ä) even prevents my keyboard from working for some seconds, but the KDE clock keeps updated and the mouse still works.

Running gprof on the code shows that the xdotool code is not really spending more time. The 1.5 seconds (200 times a) versus 33.8 seconds (200 times ä) difference seems to be caused by the X server. (400 times a takes 3.0 seconds.)

Debugging tool that can be used: http://cgit.freedesktop.org/xorg/app/xscope/

Steps to test protocol differences:

1. Start xscope > a.txt
2. Give yourself xauth with xauth add localhost:1 . $(xauth -n list localhost:0)
3. time DISPLAY=localhost:1 xdotool type ääääääääää
4. ^C xscope and type Q followed by Enter to quit (or killall xscope)

Grab https://lekensteyn.nl/files/qemu-sdl-debug/xscope-filter and compare two files with:

./xscope-filter a.txt b.txt | less -R

(needs colordiff to be installed, see also http://unix.stackexchange.com/q/79135/8250)

from xdotool.

...and the following shows the possible source of slowness:

$ time DEBUG=1 xdotool type ääääääääää
command: type
Exec arg[0]: ääääääääää
Mapping sym 228 to 8
Reverting scratch keycode (sym 8553216 to 8)
äMapping sym 228 to 8
Reverting scratch keycode (sym 8553216 to 8)
Mapping sym 228 to 8
^@Reverting scratch keycode (sym 8553216 to 8)
Mapping sym 228 to 8
ääReverting scratch keycode (sym 8553216 to 8)
Mapping sym 228 to 8
äReverting scratch keycode (sym 8553216 to 8)
Mapping sym 228 to 8
äReverting scratch keycode (sym 8553216 to 8)
Mapping sym 228 to 8
äReverting scratch keycode (sym 8553216 to 8)
Mapping sym 228 to 8
äReverting scratch keycode (sym 8553216 to 8)
Mapping sym 228 to 8
äReverting scratch keycode (sym 8553216 to 8)
Mapping sym 228 to 8
äReverting scratch keycode (sym 8553216 to 8)

real    0m1.973s
user    0m0.004s
sys     0m0.012s
$ time DEBUG=1 xdotool type aaaaaaaaaa
command: type
Exec arg[0]: aaaaaaaaaa
aaaaaaaaaa
real    0m0.172s
user    0m0.008s
sys     0m0.004s

from xdotool.

xdotool is only able to type letters that are mapped on your keyboard. If it detects situations where, for example, ä has no key mapping, it finds an empty key map and puts it there ("Mapping sym 228 to 8") and types it, then it reverts the keymap change ("Reverting scratch keycode") to restore the keymapping to the original setting.

Your time output is... weird. I cannot reproduce this on Ubuntu 13.10 under Openbox.

% DEBUG=1 xdotool type "ääääääääää" | uniq
command: type
Exec arg[0]: ääääääääää
Mapping sym 228 to 8
äReverting scratch keycode (sym 4374148392801768192 to 8)
Mapping sym 228 to 8
....

% time xdotool type "ääääääääää"        
ääääääääääxdotool type "ääääääääää"  0.00s user 0.01s system 5% cpu 0.115 total
% time xdotool type "aaaaaaaaaa"
aaaaaaaaaaxdotool type "aaaaaaaaaa"  0.00s user 0.01s system 7% cpu 0.113 total

It's possible we can optimize this by finding a scratch keycode, using it for every unmapped key, and only doing the 'revert' step after we finish? I do wonder if your X server or window manager is buggy and causing this problem. I'm open to working around this inside xdotool if we can, but for now I cannot reproduce this delay or freeze.

from xdotool.

lol printf bug 'sym 4374148392801768192' but unreltaed to this problem ;)

from xdotool.

I'm testing with KDE 4.12.0, Xorg 1.14.5 on Arch Linux. My keyboard mapping is set to US International - AltGr dead keys. When I run xdotool, I run the latest code from the clang-fixes branch (based on master). Each command is ran from the Konsole program.

A test with DISPLAY=localhost:1 xterm (under xscope). I press AltGr+Shift+", release keys, then manually press a to get ä. This generates four KeyPress and KeyRelease events (seen in xscope). With xdotool type. I get a bunch of XKEYBOARD-Event events after a ChangeKeyboardMapping request.

xdotool type ääääääääää acts really weird, sometimes I get a NUL byte (^@) too (as can be seen above). This is without modifiers like shift pressed, the chance of getting that behavior increases as more characters are typed:

$ time xdotool type $(yes a | tr -d '\n' | head -c 200)
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
real    0m1.524s
user    0m0.016s
sys     0m0.072s
$ time xdotool type $(yes ä | tr -d '\n' | head -c 400)
äääääääää^@ääääääääääää^@ääääääääääää^@äääääää^@ää^@ä^@äääää^@äää^@ä^@^@^@ä^@ä^@äää^@^@ää^@ää^@^@ää^@^@^@äääääääääääääää^@^@^@äääää^@^@^@ääää^@^@^@ä^@^@ää^@ääääääääääääääää^@äää^@äääääää^@^@ää^@ääääää^@äääääääääääää^@äää^@äääääääääääääääääää
real    0m33.840s
user    0m0.040s
sys     0m0.060s

from xdotool.

Sounds like something is interfering with xdotool's setting of the keyboard mapping. The slowness combined with the null bytes sound like two symptoms of the same problem; very weird. I'd be interested in seeing which parts of the code were being slow; I'd bet it's just the XChangeKeyboardMapping calls.

The fact that you're getting nulls is very troubling. xdotool sends XChangeKeyboardMapping and then XSync to make sure everything is aligned and sent to the X server, if the next SendKey attempt results in a null, it means that something bad happened to that XChangeKeyboardMapping call.

I do notice that xdotool doesn't check the return value of XChangeKeyboardMapping (which can return BadAlloc and BadValue, according to the manpage); a failure here could explain the null byte typing, but it wouldn't explain the delays.

XSync call might be what delays here; I'm not sure what the impact on performance is if there are lots of things waiting on MappingNotify events (perhaps KDE does this?).

I'm not familiar with KDE nor do I have easy access to it at this time (I'd have to install it and try to debug, but don't have time today)

Any debugging you can do here will help greatly :)

from xdotool.

Could this be a hint? (valgrind output is the same when I get ^@, only the first warning is shown when DEBUG=1 is not set)

$ time DEBUG=1 valgrind --leak-check=full /tmp/xbin/xdotool type ë
==19866== Memcheck, a memory error detector
==19866== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et al.
==19866== Using Valgrind-3.9.0 and LibVEX; rerun with -h for copyright info
==19866== Command: /tmp/xbin/xdotool type ë
==19866== 
command: type
Exec arg[0]: ë
==19866== Conditional jump or move depends on uninitialised value(s)
==19866==    at 0x406346: xdo_enter_text_window (xdo.c:1014)
==19866==    by 0x40BB78: cmd_type (cmd_type.c:197)
==19866==    by 0x403AD7: context_execute (xdotool.c:547)
==19866==    by 0x4039DF: args_main (xdotool.c:520)
==19866==    by 0x4033A9: xdotool_main (xdotool.c:316)
==19866==    by 0x403295: main (xdotool.c:287)
==19866== 
Mapping sym 235 to 8
ë==19866== Use of uninitialised value of size 8
==19866==    at 0x5AC742B: _itoa_word (in /usr/lib/libc-2.18.so)
==19866==    by 0x5ACB4FC: vfprintf (in /usr/lib/libc-2.18.so)
==19866==    by 0x5ACC4D0: buffered_vfprintf (in /usr/lib/libc-2.18.so)
==19866==    by 0x5AC770D: vfprintf (in /usr/lib/libc-2.18.so)
==19866==    by 0x408C9C: _xdo_debug (xdo.c:2059)
==19866==    by 0x40680B: xdo_send_keysequence_window_list_do (xdo.c:1142)
==19866==    by 0x4063C4: xdo_enter_text_window (xdo.c:1040)
==19866==    by 0x40BB78: cmd_type (cmd_type.c:197)
==19866==    by 0x403AD7: context_execute (xdotool.c:547)
==19866==    by 0x4039DF: args_main (xdotool.c:520)
==19866==    by 0x4033A9: xdotool_main (xdotool.c:316)
==19866==    by 0x403295: main (xdotool.c:287)
==19866== 
==19866== Conditional jump or move depends on uninitialised value(s)
==19866==    at 0x5AC7435: _itoa_word (in /usr/lib/libc-2.18.so)
==19866==    by 0x5ACB4FC: vfprintf (in /usr/lib/libc-2.18.so)
==19866==    by 0x5ACC4D0: buffered_vfprintf (in /usr/lib/libc-2.18.so)
==19866==    by 0x5AC770D: vfprintf (in /usr/lib/libc-2.18.so)
==19866==    by 0x408C9C: _xdo_debug (xdo.c:2059)
==19866==    by 0x40680B: xdo_send_keysequence_window_list_do (xdo.c:1142)
==19866==    by 0x4063C4: xdo_enter_text_window (xdo.c:1040)
==19866==    by 0x40BB78: cmd_type (cmd_type.c:197)
==19866==    by 0x403AD7: context_execute (xdotool.c:547)
==19866==    by 0x4039DF: args_main (xdotool.c:520)
==19866==    by 0x4033A9: xdotool_main (xdotool.c:316)
==19866==    by 0x403295: main (xdotool.c:287)
==19866== 
==19866== Conditional jump or move depends on uninitialised value(s)
==19866==    at 0x5ACB548: vfprintf (in /usr/lib/libc-2.18.so)
==19866==    by 0x5ACC4D0: buffered_vfprintf (in /usr/lib/libc-2.18.so)
==19866==    by 0x5AC770D: vfprintf (in /usr/lib/libc-2.18.so)
==19866==    by 0x408C9C: _xdo_debug (xdo.c:2059)
==19866==    by 0x40680B: xdo_send_keysequence_window_list_do (xdo.c:1142)
==19866==    by 0x4063C4: xdo_enter_text_window (xdo.c:1040)
==19866==    by 0x40BB78: cmd_type (cmd_type.c:197)
==19866==    by 0x403AD7: context_execute (xdotool.c:547)
==19866==    by 0x4039DF: args_main (xdotool.c:520)
==19866==    by 0x4033A9: xdotool_main (xdotool.c:316)
==19866==    by 0x403295: main (xdotool.c:287)
==19866== 
==19866== Conditional jump or move depends on uninitialised value(s)
==19866==    at 0x5AC7E5F: vfprintf (in /usr/lib/libc-2.18.so)
==19866==    by 0x5ACC4D0: buffered_vfprintf (in /usr/lib/libc-2.18.so)
==19866==    by 0x5AC770D: vfprintf (in /usr/lib/libc-2.18.so)
==19866==    by 0x408C9C: _xdo_debug (xdo.c:2059)
==19866==    by 0x40680B: xdo_send_keysequence_window_list_do (xdo.c:1142)
==19866==    by 0x4063C4: xdo_enter_text_window (xdo.c:1040)
==19866==    by 0x40BB78: cmd_type (cmd_type.c:197)
==19866==    by 0x403AD7: context_execute (xdotool.c:547)
==19866==    by 0x4039DF: args_main (xdotool.c:520)
==19866==    by 0x4033A9: xdotool_main (xdotool.c:316)
==19866==    by 0x403295: main (xdotool.c:287)
==19866== 
==19866== Conditional jump or move depends on uninitialised value(s)
==19866==    at 0x5AC7EDF: vfprintf (in /usr/lib/libc-2.18.so)
==19866==    by 0x5ACC4D0: buffered_vfprintf (in /usr/lib/libc-2.18.so)
==19866==    by 0x5AC770D: vfprintf (in /usr/lib/libc-2.18.so)
==19866==    by 0x408C9C: _xdo_debug (xdo.c:2059)
==19866==    by 0x40680B: xdo_send_keysequence_window_list_do (xdo.c:1142)
==19866==    by 0x4063C4: xdo_enter_text_window (xdo.c:1040)
==19866==    by 0x40BB78: cmd_type (cmd_type.c:197)
==19866==    by 0x403AD7: context_execute (xdotool.c:547)
==19866==    by 0x4039DF: args_main (xdotool.c:520)
==19866==    by 0x4033A9: xdotool_main (xdotool.c:316)
==19866==    by 0x403295: main (xdotool.c:287)
==19866== 
Reverting scratch keycode (sym 114256432 to 8)
==19866== 
==19866== HEAP SUMMARY:
==19866==     in use at exit: 103,622 bytes in 1 blocks
==19866==   total heap usage: 2,687 allocs, 2,686 frees, 504,912 bytes allocated
==19866== 
==19866== LEAK SUMMARY:
==19866==    definitely lost: 0 bytes in 0 blocks
==19866==    indirectly lost: 0 bytes in 0 blocks
==19866==      possibly lost: 0 bytes in 0 blocks
==19866==    still reachable: 103,622 bytes in 1 blocks
==19866==         suppressed: 0 bytes in 0 blocks
==19866== Reachable blocks (those to which a pointer was found) are not shown.
==19866== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==19866== 
==19866== For counts of detected and suppressed errors, rerun with: -v
==19866== Use --track-origins=yes to see where uninitialised values come from
==19866== ERROR SUMMARY: 22 errors from 6 contexts (suppressed: 2 from 2)
Profiling timer expired

real    0m1.055s
user    0m0.828s
sys     0m0.040s

The program at http://git.yoctoproject.org/cgit/cgit.cgi/libfakekey/tree/src/libfakekey.c is also affected. That program does, however, not revert the remapping. In a loop of 10 executions of the program, the execution time increases by about 400 ms (5xx ms vs 1xx ms) if the first run needs to remap a key. I do not experience issues with ^@ being typed though (even if I alternative typing ë and ä).

The following command causes my Xorg progress to use over 80% CPU (with ä substituted for a, Xorg consumes about 22% CPU):

xdotool type --terminator END " echo 'ä' | xxd | grep -vw 'c3a4' | grep --color=always ." END key Return type --terminator END ' !!' END key Return

(the code from version 2.20110530.1 segfaulted at the END keyword, if I use the clang-fixes branch it does not crash in __mbsrtowcs via xdo_type)

from xdotool.

If Xorg is using high cpu, then I believe this to be a bug in Xorg; I do not experience this problem under Xorg 1.14.3 (Ubuntu 13.10 which includes an 18000-line patch on top of Xorg 1.14.3).

Can you reproduce this delay-and-high-xorg-cpu under another windowing environment (Say, openbox, gnome, etc?)

from xdotool.

This is the run-test script I use for reproduction (openbox 3.5.2-6, Xorg 1.14.5, xdotool from clang-fixes branch, xf86-video-vesa 2.3.2-3, qemu 1.7.0, guest kernel: Linux 3.9.4, host kernel: 3.13-rc5):

#!/bin/sh
str=ääääääääää
#str=aaaaaaaaaa
f() {
time \
DEBUG=1 \
./xdotool type --terminator END \
" echo '$str' | xxd | grep -vw 'c3a4' | grep --color=always ." END \
key Return
# \
#type --terminator END $0 END \
#key Return
}

for i in {1..100}; do
    f
    ps auxww --sort %cpu | tail -n3
done

1. Arch Linux image started with qemu-system-x86_64 -enable-kvm -m 4G -smp threads=2 -vga std -vnc :0 -hda arch.qcow2
2. Start openbox (via startx)
3. start xterm, run script -c ./run-test test.log
4. Uncomment str=aaaaaaaaaa and retry 2 with a different log file

Run-time (time xdotool):
ä (accent): 470.87 (sd. 3.77)
a (no accent): 471.94 (sd. 21.1)

I think ps cannot be used to measure CPU usage, it stayed constant for some time during the runs. Nevertheless, for N=200, it appears that the CPU usage for openbox is about 12% in both ä and a cases and 8% for X.

Conclusion: openbox (or the driver combination I used) is not affected by the performance problem. All keys are correctly processed, no events were lost. Digging further if time allows...

from xdotool.

The ^@ keys are reproducable with KDE in QEMU too, as is the delay (57 vs 51 seconds for 100 executions of ä and a respectively). After the xdotool stuff with ä ends, X is still at high CPU usage.

(.xinitrc now contains exec startkde instead of exec openbox-session)

kde top CPU users:

peter     2164  4.2  0.7 547620 28576 ?        Sl   15:24   0:04 kdeinit4: konsole [kdeinit]                    
peter     2068  9.6  0.4 449292 20152 ?        S    15:23   0:15 kdeinit4: kglobalaccel [kdeinit]               
root      2010 37.8  1.1 181932 46744 tty2     Rsl+ 15:23   1:00 /usr/bin/X -nolisten tcp :0 -auth ...

openbox top CPU users:

peter     7467  2.5  0.3  78572 12296 pts/2    S    12:12   0:42 xterm
root      7296  8.6  0.5 155044 21804 tty4     Ssl+ 12:06   2:55 /usr/bin/X -nolisten tcp :0 -auth ...
peter     7301 13.6  0.2 166784 10440 pts/2    R    12:06   4:38 /usr/bin/openbox --startup /usr/lib/openbox/openbox-autostart OPENBOX

from xdotool.

I have the same problem with russian keyboard layout:

$ time xdotool type "iiiiiiiiii"
real    0m0.085s
user    0m0.000s
sys 0m0.000s
$ time xdotool type "шшшшшшшшшш"
real    0m0.539s
user    0m0.000s
sys 0m0.004s

from xdotool.

confirm: fine on first keyboard layout, slow on others
(Ubuntu 18.10 with Gnome)

from xdotool.

Any news on this issue?

from xdotool.

"xdotool key super" takes about 2 seconds on the Russian keyboard.

from xdotool.

On GNOME, I can solve this by running setxkbmap jp (jp keyboard layout). GNOME overrides this after some time and the problem starts again, so I have to rerun. Related? keepassxreboot/keepassxc#3422 and https://gitlab.gnome.org/GNOME/gnome-shell/issues/856

from xdotool.

It is not limited to keys that don't appear on the keyboard. See for example:

xdotool type "[[[[[[[[[[[[[["

from xdotool.

This issue is still bad. It's also present with apostrophes which for me will write 'èèèèèèèèèèè' instead of ''''''''''''' . Please someone we need a way to save the key code in a virtual keyboard in a way it doesn't have to be remade every time I think ? I'm not sure how to do this...

from xdotool.

For me, just xdotool keydown --delay 0 space makes my

Description:	Ubuntu 20.04.2 LTS
Release:	20.04
Codename:	focal

to freeze about 3 seconds.Any one knows how to solve it programatically?

from xdotool.

This is still a problem on Debian Bullseye 9 years after the initial post in this thread...

from xdotool.

I just tried to reproduce this on both Windows 11 w/ WSLg (XWayland/weston) and on Xephyr with no window manager. I can't seem to reproduce any slowness.

# US key map
% (setxkbmap us; time xdotool search --class "firefox" type "hello")
xdotool search --class "firefox" type "hello"  0.01s user 0.00s system 18% cpu 0.042 total

# Russian keymap
% (setxkbmap ru; time xdotool search --class "firefox" type "hello")
xdotool search --class "firefox" type "hello"  0.01s user 0.00s system 18% cpu 0.041 total

# US keymap
% (setxkbmap us; time xdotool search --class "firefox" type "Привет")
xdotool search --class "firefox" type "Привет"  0.01s user 0.00s system 15% cpu 0.047 total

# Russian keymap

% (setxkbmap ru; time xdotool search --class "firefox" type "Привет")
xdotool search --class "firefox" type "Привет"  0.01s user 0.00s system 17% cpu 0.050 total

I believe there is a problem somewhere, but at this time, I"m not quite sure what is causing it. I'll keep thinking about it. I know this is an old issue without any known solutions yet. :(

from xdotool.

Ahh, with KDE running, I do see noticeable performance differences:

Tested: US English layout, xdotool typing Russian.

Xephyr with just Firefox running:

% (export DISPLAY=:1; setxkbmap -layout us; time DEBUG=1 xdotool search --class --onlyvisible "firefox" type 'Привет')
DEBUG=1 xdotool search --class --onlyvisible "firefox" type 'Привет'  0.01s user 0.00s system 15% cpu 0.047 total

Xephyr with startkde and then firefox:

% (export DISPLAY=:1; setxkbmap -layout us; time DEBUG=1 xdotool search --class --onlyvisible "firefox" type 'Привет')
...
DEBUG=1 xdotool search --class --onlyvisible "firefox" type 'Привет'  0.01s user 0.00s system 7% cpu 0.110 total

• In testing, "Xephyr + firefox" with no window manager runs fairly around 47ms.
• With Xephyr+startkde+firefox, it takes between 110ms and 180ms.

Trying a longer string, ПППППППППППППППППППППППППППППП:

• Xephyr + firefox: Consistently around 220ms (+/- 4ms)
• Xephyr + startkde + firefox: between 450ms and 750ms.
• Xephyr + openbox + firefox: consistently around 240ms (+/- 4ms)

Seems pretty well related to having a more complex desktop environment, like KDE, running.

I can guess as, others in this issue have, that this has mostly to do with xdotool having to map symbols to keycodes vs not having to map anything (for example, on a US keyboard, there is no П key by default).

I don't really know why it happens yet and will need to research and explore a bit more.

from xdotool.

Had a hypothesis that delays were related to the number of X11 windows, and tested: 1 xterm, 30 xterms, and 100 xterms.

Test: xdotool type 'П'

Results:

• 1 xterm: 10ms
• 30 xterms: 39ms
• 100 xterms: 72ms

Might be relevant, though the delays measured aren't close to those reported (multiple seconds delay for single keystrokes).

I also noticed that xev receives MappingNotify 4 times for the same exact event when the keyboard mapping changes:

MappingNotify event, serial 31, synthetic NO, window 0x0,
    request MappingKeyboard, first_keycode 8, count 1

MappingNotify event, serial 31, synthetic NO, window 0x0,
    request MappingKeyboard, first_keycode 8, count 1

MappingNotify event, serial 31, synthetic NO, window 0x0,
    request MappingKeyboard, first_keycode 8, count 1

MappingNotify event, serial 31, synthetic NO, window 0x0,
    request MappingKeyboard, first_keycode 8, count 1

from xdotool.

After a bunch of testing tonight, I have a hypothesis that needs testing:

When xdotool changes the keyboard mapping, all X11 clients receive a MappingNotify event where they will likely invoke XRefreshKeyboardMapping. It's possible this could incur a thundering herd situation where every client is trying to respond by updating their keyboard maps.

This could especially become a problem for two reasons:

1. XChangeKeyboardMapping(dpy, scratch_keycode, 1, keysym_list, 1); causes four (4) MappingNotify events (with the same serial!) to be sent to xev and possibly to every other client as well.
2. Xdotool changing the keymap many times per second could exacerbate this issue.

If this hypothesis is correct, it could be what causes the whole X11 system to freeze while typing.

from xdotool.

For posterity, here's some code I was using to test this outside of xdotool:

#include <stdio.h>
#include <stdlib.h>
#include <X11/X.h>
#include <X11/extensions/XTest.h>
#include <X11/keysym.h>
#include <unistd.h>

int main() {
    Display *dpy = NULL;

    dpy = XOpenDisplay(NULL);
    if (dpy == NULL) {
        printf("Failed to open display\n");
        exit(1);
    }

    int scratch_keycode = 8; 

    KeySym keysym_list[] = { XK_Cyrillic_pe }; // Cyrillic_pe
    //KeySym keysym_list[] = { XK_plus };
    printf("Keymap change\n");
    XChangeKeyboardMapping(dpy, scratch_keycode, 1, keysym_list, 1);
    XFlush(dpy);

    sleep(1);

    printf("Sending key down\n");
    XTestFakeKeyEvent(dpy, scratch_keycode, True, CurrentTime);
    XFlush(dpy);

    printf("Sending key up\n");
    XTestFakeKeyEvent(dpy, scratch_keycode, False, CurrentTime);
    XFlush(dpy);

    sleep(1);

    // Revert the key map
    keysym_list[0] = 0;
    printf("Reverting key map change\n");
    XChangeKeyboardMapping(dpy, scratch_keycode, 1, keysym_list, 1);
    XFlush(dpy);
}

from xdotool.

I've reread this issue and I see a few different symptoms reported, I'll try to summarize:

• It freezes or is slow while typing normal 🗡️ keys
• It freezes or is slow while typing non-mapped 🕷️ keys
• It freezes or is slow and types the wrong keys

🗡️ "normal" meaning keys already in your keymap, like "a" for US keymap or "П" for RU keymap.
🕷️ "non-mapped" Meaning keys not available in your keymap, like "П" for US keymap.

So far, I've been able to reproduce /some/ slowdown:

• when typing non-mapped keys, especially when many windows/clients are active.

I haven't yet reproduced, and am not quite sure what causes:

• issue where it the "types the wrong key"
• issue where it freezes the X server for a few seconds
• issue where xdotool type takes multiple seconds for short text like "hello" or "[[[[["

from xdotool.

I haven't been very helpful in this whole discussion. I am having the issue where the slowdown occurs with "non-mapped" keys.

I am also running https://github.com/vividnightmare/g510s, which may or may not factor into it.

That repo hasn't been updated in 5 years and perhaps was compiled using old libraries, which might explain why I am experiencing issues that were first reported 9 years ago.

I love my Logitech peripherals but they aren't well supported under Linux and I am in the process of switching to Razer peripherals.

Anyway @jordansissel, thank you for the effort on this, and if this is because I'm using old software, my apologies. I might be able to help myself by compiling the software myself with newer libraries.

from xdotool.

Actually, it has been so long since I managed to get my keyboard working almost perfectly that I've forgotten what I did to get it working. I am using the following repo:

http://ppa.launchpad.net/vivnet/g510s/ubuntu focal

So it should be compiled against relatively up to date libraries.

from xdotool.

I've reread this issue and I see a few different symptoms reported, <snip>

Thank you for pursuing this as I know quite a few people are having this issue.

from xdotool.

I did @admirabilis suggestion to run setxkbmap jp (pt in my case) and it ran without freezing this time, but it still took some ~100 microseconds to finish the command (which in my opinion is too long).

Another thing I realized is that the xdotool do not freeze running under a docker container. (but has the ~100ms latency).
I also noticed that sometimes, not so often, the window do not receive the xdotool command (maybe make --delay higher? also making latency higher), so I cant genuinely trust xdotool to send commands to my app. Does anyone know another alternative?

from xdotool.

I would like to share with you guys, Today I was testing a C++ script that opens /dev/uinput and writes into it, so it is not using xdotool.
Unfortunately the script also hangs/freezes/halt my system for a second. Is it wise to conclude that the problem is NOT related to xdotool? Would like to hear your opinions.

Below is the code I am using. The problem arises when the method report runs.

#include <fcntl.h>
#include <linux/uinput.h>
#include <string.h>
#include <unistd.h>

#include "KeyPresser.hxx"

KeyPresser::KeyPresser()
{
    struct uinput_setup usetup;

    fd = open("/dev/uinput", O_WRONLY | O_NONBLOCK);

    /*
    * The ioctls below will enable the device that is about to be
    * created, to pass key events, in this case the mapping keys.
    */
    ioctl(fd, UI_SET_EVBIT, EV_KEY);

    ioctl(fd, UI_SET_KEYBIT, KEY_UP);
    ioctl(fd, UI_SET_KEYBIT, KEY_DOWN);
    ioctl(fd, UI_SET_KEYBIT, KEY_LEFT);
    ioctl(fd, UI_SET_KEYBIT, KEY_RIGHT);
    ioctl(fd, UI_SET_KEYBIT, KEY_A);
    ioctl(fd, UI_SET_KEYBIT, KEY_S);
    ioctl(fd, UI_SET_KEYBIT, KEY_D);
    ioctl(fd, UI_SET_KEYBIT, KEY_Z);
    ioctl(fd, UI_SET_KEYBIT, KEY_X);
    ioctl(fd, UI_SET_KEYBIT, KEY_C);
    ioctl(fd, UI_SET_KEYBIT, KEY_Q);
    ioctl(fd, UI_SET_KEYBIT, KEY_W);
    ioctl(fd, UI_SET_KEYBIT, KEY_K);
    ioctl(fd, UI_SET_KEYBIT, KEY_L);
    ioctl(fd, UI_SET_KEYBIT, KEY_ENTER);
    ioctl(fd, UI_SET_KEYBIT, KEY_SPACE);

    memset(&usetup, 0, sizeof(usetup));
    usetup.id.bustype = BUS_USB;
    usetup.id.vendor = 0x1234; /* sample vendor */
    usetup.id.product = 0x5678; /* sample product */
    strcpy(usetup.name, "Virtual C++ KeyPresser(v2)");

    ioctl(fd, UI_DEV_SETUP, &usetup);
    ioctl(fd, UI_DEV_CREATE);
}

KeyPresser& KeyPresser::hold(int key)
{
    return emit(EV_KEY, key, 1);
}

KeyPresser& KeyPresser::release(int key)
{
    return emit(EV_KEY, key, 0);
}

KeyPresser& KeyPresser::key(int key)
{
    return hold(key).release(key);
}

KeyPresser& KeyPresser::report()
{
    return emit(EV_SYN, SYN_REPORT, 0);
}

KeyPresser& KeyPresser::emit(int type, int code, int val)
{
   struct input_event ie;

   ie.type = type;
   ie.code = code;
   ie.value = val;
   /* timestamp values below are ignored */
   ie.time.tv_sec = 0;
   ie.time.tv_usec = 0;

   write(fd, &ie, sizeof(ie));
   return *this;
}

void KeyPresser::destroy()
{
    ioctl(fd, UI_DEV_DESTROY);
    close(fd);
}

from xdotool.

@mowses You might have hit on the real issue. Looking at your code I'm wondering if this is an issue located somewhere in the base Linux usb subsystem and drivers. I know I've had similar issues in ratbagd where certain actions take much longer than it seems they reasonably should, and then other issues with peripherals such as usb headsets that might be related.

The question is, where to report this issue. I have no idea who is developing and maintaining that code, or even what it is called. That's pretty deep in the Linux OS, and close to the kernel.

from xdotool.

The different subsystem mailing lists and maintainers are here: https://www.kernel.org/doc/html/latest/process/maintainers.html

I'm not sure which this would fit into.

from xdotool.

@Drek282 maybe it is a uinput module issue. Here's the link:
https://www.kernel.org/doc/html/v4.12/input/uinput.html#introduction

the doc says:
"libevdev is a wrapper library for evdev devices that provides interfaces to create uinput devices and send events. libevdev is less error-prone than accessing uinput directly, and should be considered for new software."
maybe we should consider use libevdev... I gonna take a look

from xdotool.

@mowses Which library/method does xdotool use? Or ratbagd for that matter? The long hang times I am seeing with ratbagd involve loading profiles to my usb mouse. Is it using uinput as well?

from xdotool.

@Drek282 I believe that the issue is like @jordansissel said:

When xdotool changes the keyboard mapping, all X11 clients receive a MappingNotify event where they will likely invoke XRefreshKeyboardMapping. It's possible this could incur a thundering herd situation where every client is trying to respond by updating their keyboard maps.
This could especially become a problem for two reasons:
1. XChangeKeyboardMapping(dpy, scratch_keycode, 1, keysym_list, 1); causes four (4) MappingNotify events (with the same serial!) to be sent to xev and possibly to every other client as well.
2. Xdotool changing the keymap many times per second could exacerbate this issue.

Also I am facing more 2 weird behaviors that may corroborates to the @jordansissel hypothesis, but I would like to make sure it also happens to other people as well.

1. when I run setxkbmap -model abnt2 -layout br -variant abnt2 this command have the same "freeze" effect as xdotool. My Ubuntu freezes for almost 3 seconds, even the clock seconds stops refreshing. I dont know if xdotool or any of its dependencies calls for such command or alike, but it may confirm the XChangeKeyboardMapping issue pointed by jordan.
2. Every time I run xdotool or any script that writes to /dev/uinput I noticed an odd behavior: My OS freezes in 2 distinct moments: when the script/command is initializing: ex: xdotool key a[enter] (freezes ~1.5sec) . The second moment is when I interact with my real keyboard (even after the xdotool completed the above command) - freezes for more 1.5sec. I believe that every time the input is switched from real or virtual one, a "keyboard remapping" or something like that is triggered causing the hang.

A curious thing is that time xdotool key a, timeis not accounting the hang time, just outputting

real	0m0,071s
user	0m0,001s
sys	0m0,013s

I am just concluding that the reported issue is not related with xdotool itself but with some other linux kernel bug.

from xdotool.

As a workaround, one can copy the text to clipboard and paste it instead of typing:

# Hangs with long text and text with special characters:
xdotool type "$1"

# Does not hang for me:
echo "$1" | xclip -sel primary    # for use with some shared clipboards (e.g. Remmina) 
echo "$1" | xclip -sel clipboard  # for use with local OS clipboard
xdotool key Shift+Insert

from xdotool.