What if he doesn't talk to you anymore?

If you tell your guy friend no and he stops talking to you, it is likely he has decided to end your friendship, or at least needs time to get over you. If he won't talk to you anymore, that is his prerogative. If this bothers you, you may want to confront him about it, but it is likely that his feelings for you are too strong to just be friends anymore. If you care for him as a friend and truly do not want to be with him, then let him go. Deal with this loss of friendship as best you can on your own. Otherwise you would be leading him and making things more painful for him.