As you decide whether or not something your partner says or does is okay with you, remember a few things: It doesn't mean the situation is all his fault; it doesn't mean you should threaten to break up with him (unless you'll really do it); it doesn't mean he "owes" you; and it doesn't mean that you should punish him in some way.
When he does something that's not okay, it means that it's time for you to take a second look at how things are in your relationship. It's likely that whatever is going on is not beneficial for you, and it's probably not healthy for your relationship either. Decide what your next move will be. Will you create agreements with him to put changes in motion? Will you consider whether or not it's wise for you to stay in the relationship at all? Will you make some internal changes that could significantly affect his behavior?
The choice is yours to make. Remember, you deserve to be treated with respect, kindness and love.