They are 2 glands located directly inside the anus, at about the 4 and 8 o'clock positions. They fill with "stuff" and most dogs naturally are able to empty them on their own with each bowel movement. Some dogs however can't empty them, and they end up getting full, and can eventually rupture and abscess. When a dog starts scooting his butt on the floor, it's a good idea to check and see if he needs his anal glands expressed. It's something anyone can learn to do at home, but let me tell you.... I'm a vet tech (a very new one, granted) and I still can't do it lol.
Not all dogs have issues. None of my dogs ever have. It's more common for small breeds to need help. We have some dogs that come to the clinic every 2 or 3 weeks to have theirs expressed!