First and most important classification, we have HDDs and SSDs.
HDD: Hard Disk Drive: This is a bunch of magnetic discs spinning very fast, with a head reading and writing data on the magnetic surface of the disks. These are larger and cheaper, but also slower, since the head has to reposition itself every time you need data from a different location.
SSD: Solid State Drives: You can consider these as bigger version of pen drives. They store data in memory (flash) chips, and this means they don't have any moving parts. This allows higher speeds, but also makes them expensive. Also, they don't come in the capacities HDDs come in.
For the rest of the answer, I will refer to HDDs as magnetic disks and SSDs as SSDs or flash. If I say hard drives, it encompasses both.
Now that we know what types of drives are there, let's start with internal drives:
Computer use two main types of connectors for hard drives:
SATA: This is the most common connector used in laptops. These come in SATA 2 and SATA 3 versions, supporting 3.0 and 6.0 Gbps respectively.
SATA drives come in two form factors. 2.5" which goes into laptops and looks like this when it is magnetic:
and like this when it is an SSD:
Then we have 3.5" which is bigger and thicker, a little bit faster, and looks like this when it is magnetic:
and like this when it is an SSD:
3.5" SSDs are actually quite rare. 3.5" drives are faster on average because they have more power to play with.
PCIe: This is a very recent technology which comes in select Ultrabooks like the Macbook Air. You can buy these for desktops, but they are quite expensive. However, they are insanely quick. I know of some that can hit 24 Gbps, and there are even faster versions available. For laptops, they look like this:
and for desktops, like this:
These are exclusively SSDs.
Apart from the size, you will hear about RPM rating. 5400, 7200 and 1000 RPM are the most common ones. This is the speed at which the magnetic disks inside the HDD spin in revolutions per minute A higher RPM HDD is faster, but it consumes more power and is more expensive. The extra power consumption also translates to more heat, both of which are issues in laptops, but not desktops.
Now we come to external drives.
External drives are of two main types. They are both based on SATA drives, and as a result, come in two form factors.
2.5": These contain 2.5" magnetic drives with a chip that converts SATA to any one of the connectors available.
These are small, and are powered by the port they are connected to, so they are commonly called portable hard drives (for being small and portable, not because they use one port, although they are portable also because they only use one port...okay enough puns :D). They mostly connect using USB 2.0 (480 Mbps) or USB 3.0 (5 Gbps)
(USB 3 is blue inside, USB 2 may be white/black/grey. It's white here)
but you also have some Firewire 400(400 Mbps)
(4 pin variant on the left and 6 pin variant on the right)
Firewire 800
and Thunderbolt (10 Gbps)
versions available. Each version has a different "bridge" to convert SAT to the respective connector.
3.5": These contain 3.5" drives with the same chip as 2.5", plus a power supply.
These need external power supplies since the disks are heavier, and need more power than the port can supply. These also come in USB, Firewire and Thunderbolt versions. They are also referred to as external hard drives.
There are a few differences in the 2.5" and 3.5" external drives. 2.5" is more convenient, but 3.5" is more heavy duty and reliable. Also, 3.5" drives tend to be faster on average, thanks to the dedicated power supply.
Coming to your specific questions:
For laptops, for internal storage, you can use pretty much any 2.5" SATA drive. SSDs are obviously faster, but for a lot more money, you get a lot less storage. PCIe drives are not supported unless your laptop came with one.
For external storage, with laptops, it depends on your needs. If you need something to carry around, go for a portable USB 3.0 drive. It is fast enough, reliable and convenient to haul around. You do not need a power socket to get it to work, which is another advantage. If, however, you need something nly for backup, I would recommend an external USB 3.0 drive. They are faster, even more reliable, and you get more storage for the same money.
I personally use a 1TB Seagate Backup Plus portable drive when I'm travelling to have enough space to drain my camera's memory card, but at home, for my movies and other big data. I use a 1TB Western Digital MyBook.