animcur: Fix crash when removing a master device

Reproducer:

$ Xvfb -ac -noreset :1 &
$ DISPLAY=:1 xinput create-master touch1
$ DISPLAY=:1 xinput remove-master "touch1 pointer"

Bugzilla: https://bugs.freedesktop.org/105761


Signed-off-by: Adam Jackson <ajax@redhat.com>
Reviewed-by: Peter Hutterer <peter.hutterer@who-t.net>