Among the many factors that lead to a decrease in sex drive, more than half can be eliminated on your own, while the rest require the help of a doctor. In summary, there are ten major factors that can cause a reduction in sex drive.