UPDATE Solution is here: Fix CAps LOck delay in Ubuntu 21.04
Ubuntu 21.04 uses Wayland as its default display server, not Xorg. Xorg-specific configuration and tweak tools (which xkbcomp
is) will not work on Wayland.
As a temporary measure, I would recommend to just not use Wayland until it's ready and has a considerable user base. Without an active user base, there is always way too many bugs and no one to fix them.
To switch to Xorg:
Log out.
Click on your profile.
On the bottom right corner a display server selection button will appear. Click on it and select Ubuntu on Xorg.
Now log back in and everything should work.
UPDATE Solution is here: Fix CAps LOck delay in Ubuntu 21.04