How do I get the guild to erase my bounty?

At first you can only clear your bounty in Riften, but after completing enough small jobs for Vex and Delvin in each city (Markarth, Whiterun, Windhelm and Solitude), you will be able to do a city influence quest where you will be asked to complete a certain task from an influential individual from that city. Once each quest has been completed and the guild will have influence in that city, a merchant will move into the Ragged Flagon, some characters will be a fence for you (this one will be obsolete if you have the fence perk), and you will be able to clear your bounty by stating you're with the guild (this will cost some gold depending on your bounty).