I find that I only get pissy with my husband when he does this is when I had a sort of mental plan going on as to how things were going to go down later. For example, if he gets off of work late and I was expecting cuddles when he was done but then he never comes in and goes to pass out on the couch instead after reading for a while. These are things I like to know about ahead of time so that I don't have to deal with expecting one thing and then getting an unpleasant surprise. If I am very clear with him before hand about my wants/needs, this doesn't seem to be an issue, but sometimes I still forget to tell him and end up hurt anyway.
Proper communication usually solves this problem though.
It is very helpful for an INFJ to be explicit with an INTP about feelings and expectations. For the former, it's because we just aren't wired to understand feelings, at least not in our natural state, untutored state. For the latter, don't forget, we're "P's" and we don't have a great need to finish things or stick to a schedule.