This is a difficult topic because for those who do not yet have children, any complaints by those who have at least one can be construed as insensitive. But let's delve into this anyway.
It is not unusual for couples trying to conceive baby #2 or #3 or even #4 to encounter obstacles. Miscarriage after a successful pregnancy is common, although many couples do not discuss it openly because 1) they have a child already to care for and don't want to wallow in grief around that child, and 2) other people often don't understand and offer painful advice.
I have a friend who had no trouble conceiving baby #1, and when it came time to try for the next baby, she was shocked that it was not happening easily. Finally, after two years, she went for help, and a hysterogram (HSG) revealed blocked tubes. How did that happen? Perhaps some tissue went the wrong way when she delivered and blocked her tubes or perhaps it happened later, but tubes can get blocked at any time. Luckily, just having the HSG unblocked her tubes, and she got pregnant.
Pregnancy is hard on your body from a nutrient standpoint. The baby takes what he/she needs, and that means that the mother's iron stores are depleted, as are most other nutrients. That's why eating well when you are pregnant is so important--it's not just for the baby. Your body when pregnant needs a lot more protein and calories and essential nutrients, and they say that it takes two years for the mother's iron stores to replenish.
A previous successful pregnancy is a very good sign that you can get pregnant again. However, it is not a guarantee. Taking some time to enjoy your baby is the most important first step. At the same time, work to get your own body back into prime health. I don't necessarily mean what most people mean when they say that--I am not talking about losing the "baby weight." I mean you need to eat well, sleep well, take your supplements, and get some moderate exercise. You did all those things the first time around, right?
It is okay to dream about the next baby. It is not selfish. We have desires deep down inside us, and nobody can tell you that you do not deserve to have those desires. Find a friendly ear when you need one, and take very good care of yourself and your family. However large that family ends up being, enjoy it!