More Affection Equals More Sex—Why is it so difficult?

This article has really given me pause for thought.
I readily admit that I am very guilty of neglecting affection in relationships. As time goes by, I just don’t do the things I used to do. Random hugs and stolen kisses. Shoulder rubs, knee rubs. Sweet, kind, considerate comments.
Even when I think about these things, I will usually forget and fall into the same patterns.
Why is maintaining affection so difficult?
I don’t think I’m a total asshole, but I do definitely want to me more affectionate.
I do have a theory: we just aren’t allowed to be as affectionate with people, in general. I mean friends, family, acquaintances. Casual touch and kind language aren’t the norm in society. Instead, personal space and innocuous patter rule the day. It is rare that I will just hug a friend, or pat a workmate’s shoulder after a job well done. And language, that is so very guarded.
Now, I am fully aware that there are very good reasons for touch barriers and word sensitivity. I just wish there were a middle ground to be found.