Know the various issues. During the Confederate era, notes were issued by the federal Confederate government, the individual states of the confederacy, county, city and town governments, and "bank notes" issued by individual banks. Each of these genres can become a hobby unto itself, so don't get too overwhelmed!
Pick a theme. Many collectors will focus on bills that have a particular pictoral theme (railroads, landscapes, etc.) Some collectors will strive to collect all of the notes from a particular geographic issue (only notes from the State of South Carolina, for instance). Others may simply enjoy collecting whatever notes they come across. The "scattered" approach can be a lot of fun, but most people will decide to focus their collecting efforts on one particular area.
Get some books. There are several excellent books on Confederate currency collecting that will list the varieties of bills produced. My favorite books are by Arlie Slabaugh (Confederate States Paper Money) and Criswell (Criswell's Currency Series). Both give price listings for bills in various conditions, and are an excellent resource to make sure you are paying fair market value (or less than fair market value) for a particular bill.
Buy the best bill you can afford. As a general rule of thumb, it's better to buy the best quality bills that you can afford. Most collectors become "traders", and better quality bills will always be welcomed by other collectors for bartering. "Junk" bills are good for filling the gaps in your collection, but don't expect to be able to trade them easily.
Invest in a good storage book. There are many ways to store paper currency. The best approach is to get a dedicated currency binder that has clear sleeves for holding each bill. These sleeves aren't cheap, but they will protect your investment and allow you to enjoy the bills for many years to come.
NEVER MODIFY A BILL! I can't emphasize this point strongly enough. Even something as simple as a piece of tape will irreversably alter a bill and lower its value. NEVER cut a bill (this includes "trimming" the edges), and NEVER try to remove marks on the bill. Leave it just as it is; the wear and tear it has is part of its history and charm.
Beware of counterfeits. There have been *many* counterfeit bills produced since the Civil War. Some people collect older counterfeit bills, but this takes considerable skill and experience. The best way to identify counterfeit bills is to compare it to known "good" specimens. Pay particular attention to the quality of the paper and the printing, and the overall condition. It's rare to find a civil-war era bill in "perfect" condition. If in doubt, have a professional currency grader examine the bill before purchasing it.
Enjoy your bills! Don't forget to flip through your collection regularly, to enjoy the beautiful pieces of artwork you have collected! Some of the old currencies were amazingly detailed works of art, and all of them have an important story to tell!
)