Xen: Workaround for VNC Mouse Pointer Problem with Linux PV VMs

The framebuffer driver for Xen has an annoying bug: If you want to connect to the console of your virtual machine using VNC, and you have the resolution set to something other than the default 800x600, then your mouse behaves strangely. The mouse position will be correct when the mouse is at the upper left corner of the virtual console. However, as the mouse moves from the upper left corner, the mouse quickly becomes misaligned. The problem is worse the greater the difference between your virtual display resolution, and the default 800x600 resolution.

The reason this problem exists is that, as incredible as it may seem, the xen Linux kernel drivers (xen-kbdfront.c) hard code the display size! Not only is the display size hardcoded, it is done in a keyboard/mouse driver! And with absolutely no capability to override the values... brings back memories of having to hack hardcoded values in the Linux kernel back in 1996. So much for almost 20 years of progress... ok, /end rant

For the curious, the values are defined in /usr/include/xen/io/fbif.h:

...
#define XENFB_WIDTH 800
#define XENFB_HEIGHT 600
#define XENFB_DEPTH 32
...

and used in <your kernel path>/drivers/input/misc/xen-kbdfront.c:

 ...
/* pointing device */
ptr = input_allocate_device();
if (!ptr)
goto error_nomem;
ptr->name = "Xen Virtual Pointer";
ptr->phys = info->phys;
ptr->id.bustype = BUS_PCI;
ptr->id.vendor = 0x5853;
ptr->id.product = 0xfffe;

if (abs) {
__set_bit(EV_ABS, ptr->evbit);
input_set_abs_params(ptr, ABS_X, 0, XENFB_WIDTH, 0, 0);
input_set_abs_params(ptr, ABS_Y, 0, XENFB_HEIGHT, 0, 0);
} else {
input_set_capability(ptr, EV_REL, REL_X);
input_set_capability(ptr, EV_REL, REL_Y);
}
input_set_capability(ptr, EV_REL, REL_WHEEL);
...

While the kernel hackers amongst you might fault me for not writing a patch for this, I'd like to point out that it's already been done.  There have been numerous proposals to fix this issue by folk who already play in the kernel space on a daily basis.  I considered throwing together a quick hack to allow setting these values via kernel command-line (not a bad workaround since you typically set xen video size there anyway).  However, this hack would never get accepted into the mainstream kernel, and that means I would have to kernel patch every VM - not cool.  The real way to fix this would be to have the input driver read the display size from the video driver via xenbus, and perhaps one day I'll get around to figuring out how to do that (if you're a xen kernel hacker with some free time, feel free to use that idea!).

Fortunately, there is a much easier way to workaround this issue - in user space.  The problem is that the regards of your real display size, the mouse driver is always limited to [0,800] in the x direction, and [0,600] in the y direction.  So it scales the motion of the virtual pointer across the virtual display such that 0,0 puts the cursor at the upper left corner of the display (as usual), but 800,600 puts the cursor at the lower right corner of the display.  Try it and you can verify that this is indeed the behavior.

We can use xinput to work around this issue (assuming you are using Xorg to drive the display in your VM).  First, get the name of the mouse device for your VM:

xinput
⎡ Virtual core pointer                        id=2    [master pointer  (3)]
⎜ ↳ Virtual core XTEST pointer id=4 [slave pointer (2)]
⎜ ↳ Mouse0 id=7 [slave pointer (2)]
⎣ Virtual core keyboard id=3 [master keyboard (2)]
↳ Virtual core XTEST keyboard id=5 [slave keyboard (3)]
↳ Keyboard0 id=6 [slave keyboard (3)]

In the above example (Ubuntu 14.04 LTS) the pointer is named Mouse0.  Next, verify that the required property exists:

xinput list-props Mouse0
Device 'Mouse0':
Device Enabled (115): 1
Coordinate Transformation Matrix (117): 1.000000, 0.000000, 0.000000, 0.
000000, 1.000000, 0.000000, 0.000000, 0.000000, 1.000000
Device Accel Profile (245): 0
Device Accel Constant Deceleration (246): 1.000000
Device Accel Adaptive Deceleration (247): 1.000000
Device Accel Velocity Scaling (248): 10.000000
Device Product ID (234): 22611, 65534
Device Node (235): "/dev/input/event1"
Evdev Axis Inversion (249): 0, 0
Evdev Axis Calibration (250): <no items>
...

The parameter "Evdev Axis Calibration" is the parameter we are looking for, and it can be used to tune the mouse behavior.  Simply set the value of this parameter to your display size:

xinput set-prop Mouse0 'Evdev Axis Calibration' 0 1280 0 1024

The values of the arguments are:

  • minimum x value
  • maximum x value
  • minimum y value
  • maximum y value

So in the above example my display size was 1280x1024, so the corresponding values are: 0 1280 0 1024

Once applied, viola - the cursor position of the VM instantly begins matching local cursor position!

Now, of course, you're going to want to know where to use this command.  If you can manage to get logged int to your VM and access a terminal, you can type the command there.  Or, if you're more clever, perhaps put it in your login script.  However, it only needs to be run once, and only if the X-server is running.

For my Ubuntu VM, I chose to have it run after the display manager started.  Ubuntu 14.04 LTS was using lightdm as the display manager.  So I created a file /etc/lightdm/FixMouse.sh and put the command in there:

#!/bin/sh

exec xinput set-prop Mouse0 'Evdev Axis Calibration' 0 1280 0 1024

and then ensured that the script gets called by created the configuration file for lightdm, /etc/lightdm/lightdm.conf:

[SeatDefaults]
display-setup-script=/etc/lightdm/FixMouse.sh

You'll need to restart lightdm (or reboot) for this to take effect.  Now every time Ubuntu starts, the mouse fix will be in effect.

The FixMouse.sh script can be modified to read the display size and automatically set the values.  If I get time I'll enhance this article with those improments, as well as ways to call it for other distributions.