GUI Client
Besides the command line interface, I like to be able to manage the database using a remote GUI client.
pgAdmin 3 from www.pgadmin.org
pgAdmin 3 is one of the most popular GUI clients. It is cross platform so I happen to run it on my Windows desktop.
Note: for the nit picky users like me, in the Windows environment I personally hate installers, so I install this in a test machine, copy out the directory and then just use on my other systems without having to install and have strange things in my Windows registry.
From my Windows desktop I SSH into the Ubuntu server and tunnel port of 5432 to connect. See, Secure TCP/IP Connections with SSH Tunnels for more details.
I find using SSH to be the most secure method. In some cases you may need to allow remote administration.