Howtoforge posted a tutorial about upgrading from VMware ESX/ESXi 4.x to ESXi 5.0.

In the original ESX architecture, the virtualization kernel (referred to as the vmkernel) is augmented with a management partition known as the console operating system (also known as COS or service console). The primary purpose of the Console OS is to provide a management interface into the host. Various VMware management agents are deployed in the Console OS, along with other infrastructure service agents (e.g. name service, time service, logging, etc). In this architecture, many customers deploy other agents from 3rd parties to provide particular functionality, such as hardware monitoring and system management. Furthermore, individual admin users log into the Console OS to run configuration and diagnostic commands and scripts.

  Upgrade/Migrate VMware ESX / ESXi 4.x To ESXi 5.0