It seems there is some problems between Vagrant 1.6.2 and VirtualBox 4.3.12 (the latest at the time or writing this),
switching back to VirutalBox 4.3.6 or VirtualBox 4.3.8 seems to eliminate the problem.
update 1: Try both Vagrant 1.6.2 and VirtualBox 4.3.12 on Mac and they seem to work fine!
update 2: You need to enable Virtual Machine option in your BIOS to make VirtualBox work as expected. Saw someone mention about this in Laravel forum about this and it is true.
Also, Vagrant version should be 1.6.2.