Every hobby has its expenses and stamp collecting is no exception to this rule. With a hobby, you have some control over how much money you want to spend on it. Granted you may not have any judgment and just jump in and spend your money recklessly. But stamp collecting can be done on the cheap. The good news is that most stamps are pretty cheap to purchase. But you need to proceed with caution until you have determined what you want to collect. There are so many different stamps in existence, you could never hope to collect all of them due to cost, storage issues, and various other reasons.
In other words you will need to specialize. Specializing your collection can be as easy or as hard as you want to make it. If you collected coins, there is an old adage that states that you should buy the book before you buy the coin. And I suppose this could be applied to collecting stamps too. Applied knowledge will be your best investment when collecting stamps. You can never know enough about your specialty.
Why you want to collect stamps can be interesting in its own right. Here are some of the reasons people collect stamps:
Their collection is for pure enjoyment.
Their collection is meant to be an investment.
Some collectors involve the whole family in the hobby.
Some collectors are very competitive.
You may want to build a collection that sends a message.
Some collectors look to collecting stamps as a form of investing. They want to buy low and sell high. Most wont be very successful at this strategy. Most lack sufficient market knowledge to ever make this approach work for them. Money can be made with rare stamps, but it tends to be more speculation than investing. Serious study of what you want to collect and what the market is willing to pay for that material is one way of achieving this dream.
You have to make sure that you are not overpaying for flawed stamps that will never return your investment. You will need to learn what makes a stamp rare and collectable. As previously mentioned most stamps are very common and cheap to buy. These stamps are highly unlikely to ever become rare and profitable to sell for an huge profit.
If you are serous about trying to speculate on buying and selling rare stamps, serious knowledge is the only way to go. You will need to buy suitable reference books to know what you are buying is indeed rare and of interest to serious collectors.