You can now set your public ssh key(s) at http://rimuhosting.com/cp/sshkeys.jsp . These keys will automatically be added to new server setups. And the keys can be used when you enable a server's console-over-vps feature.
SSH keys are great when you want to avoid re-entering passwords every time you log into a server. The Ubuntu team wrote a page about setting up and using SSH keys. And SSH keys can be used with Windows SSH clients (Putty/Pageant)