Refactoring is a huge part of the programming workflow. Over the past year there has been a resurgence of the refactoring practice. With so much emphasis on the process and techniques of refactoring, the bigger picture is often missed.
When you refactor, you should stop and ask: "Why do I need to do this? What's different now? Did a perception change, was something missed, etc.? How can I prevent this the next time?"
Remember, not all smells are "bad" and it's a subjective scale.