Family - Very easily. I say "I love you" to my family all the time, and visa versa.
Friends - It can seem a little awkward, but not so much with very close friends.
SOs, beginning of relationship - It depends on how I feel about the person, but I wouldn't be the first to say it.
SOs, after the relationship has matured - Easily.
People I don't truly love - No. I don't like being dishonest or insincere.