Get some pieces of wood and some sanding pieces, you can find them at good hardware stores. Take the strips and gently rub them across any rough spots, or ridges to smooth them out. You can also use electric wood sanders, if needed.
For outdoor use, I recommend using Douglas fir or vertical grain Douglas fir. These woods hold up very well in outdoor conditions over time. For indoor signs, good choices are...
Use a furniture refinisher to dissolve the stain and scrub it away with a bit of steel wool. Although it still involves some scrubbing, this approach is less damaging to...
I recommend sanding down the wood for a smooth finish. After, apply primer, sanding in between coats of primer in order to get a better adhesion to the paint.