Turns out, inotify performance depends on the path.

Why is /tmp so much slower? The filesystem is the same (ext4). My only guess is that it has so many events that inotify_rm_watch has to do a lot of work to clear them.