Probably a lot of people with more expertise than me, but I'll give it a stab.
Much of your decision depends on what you want to use your photos for, and how much money you want to spend. What kind of computer you have, and what the OS (operating system) is can also be a factor.
Some of the things to consider:
Megapixels -- This basically indicates how good your photo resolution will be. In other words, the more megapixels, the greater the detail of your photos, and the larger you can blow up the photo before it starts looking grainy and loses detail. If you don't care about blowing photos up much larger than 8x10, 3-4 megapixels would be fine.
Batteries -- Digital cameras eat battery power like crazy. You might want to consider one that allows you to use both a lithium pack type battery as well as rechargeable AA's. If you're on vacation and your battery goes out, you can have a charger with you and use the AA's.
Memory Card -- These are what the pictures are stored on in your camera. They come in Mega-Byte increments. As a rough rule of thumb, figure 1 MB per picture. Thus, a 16 MB card will allow you to take 16 photos. If you don't buy a camera that comes with at least a 64 MB card, go get a larger card. There are different types of cards (media) as well. I have an Olympus (see below) that uses Smart Media cards. The largest card is 128 MB. I bought a second one so I can take up to 256 pictures at one time. There are other cards for different cameras that allow more pictures on them.
Downloading your photos -- Personally, I download from my camera to my home computer. Then back up my photos by burning them to CD. (If you take a lot of pictures, and store them on your computer, you eat up storage space -- which may, or may not, be a consideration depending on the size of your hard drive and how many pictures you take.). My camera comes with a USB connector cable. Provided your computer's motherboard is new enough to support USB 2.0 (1.0 is a lot slower), you can download a lot of photos pretty quickly. There are also cameras using FireWire (don't know much about it), or cameras with docking stations that hook up to your computer. For me, the USB cable works just fine.
Printing photos -- I have literally hundreds of photos, but I have not yet printed out a single one. I just built a new computer and don't have a printer for it yet. So, hopefully, someone can talk about printing photos. I know that some photo developers allow you to go in with a CD with your photos on them and print your own. Haven't tried that yet.
Recovery time -- This is not the correct term for what I'm talking about, but I can't remember what it's called. Basically, it means how quickly you can take photos back to back. If you need to snap off a bunch of quick photos, you need to look into this. If it's just your normal family photo shoot, it probably is not important. There is also wait time for flash to recover.
I own an Olympus C-4000 (4 megapixels) that I bought in April. Got it for about 279 + tax at Costco. Not touting this vendor, this is just a link I found in case you're interested in this camera (DO NOT pay what they say is the MSRP price -- way too high):
There's a ton of cameras out there, so look around on the net before you settle on one. I bought the Olympus because the price was good for the camera.
Hope this at least gets you started -- good luck.