Install the PHP packages,
sudo apt-get install php5
As of Ubuntu 12 (an maybe even earlier), the installer will automatically restart Apache2 for you.
Quickly verify that everything works by creating a php info file with your favourite editor,
sudo vi /var/www/info.php # On public site do not use such an obvious file name
Put in the following contents,
<?php phpinfo(); ?>
Save the file and browser to file using a browser. You can use either the IP Address or the valid Domain Name. For example, http://173.194.75.94/info.php or http://www.krypton.com/info.php which should show a purple and grey PHP informational screen.
FAQ
Why do some of the php5 installations say to use install libapache2-mod-php5?
No need, it is included with the php5 package.
What is the difference between the php5 and libapache2-mod-php5 packages?
Nothing I can see. It just looks like php5 is an overarching package name.
References
Ubuntu Server Documentation - https://help.ubuntu.com/12.04/serverguide/php5.html
Another Ubuntu Help Page - https://help.ubuntu.com/community/ApacheMySQLPHP#Installing_MYSQL_with_PHP_5