Why is it often so difficult to let go of sinful desires?

You have described the challenge of every human who is trying to develop her spiritual capacities and qualities. Thinking of the tendency to adopt the standards of a decadent and morally bankrupt society in opposition to our desire to develop our spiritual capacity like opposing muscles puts it in perspective. Like building strength in our muscles, without the opposition, our spiritual growth would be impossible.