11 Signs You're Only The 'Problem' Because You Don't Let People Do Whatever They Want To You

Written on Mar 11, 2026

assertive woman who doesn't let people do whatever they want to her Dean Drobot | Shutterstock
Advertisement

There are few things as uncomfortable and humiliating as being labeled "the problem" when you know that’s not you. Unfortunately, toxic people who like to take advantage of others will often do this when you set a boundary that prevents them from controlling you. When you set a healthy boundary, it means you know how you expect others to treat you and what you simply will not put up with.

This obviously creates a big issue for manipulative people because they can no longer treat you poorly and expect you to bend over backwards for them in return. That’s when they’ll start treating you like you’re the problem. In these situations, it can be easy to wonder if that may be true, but there will actually be some subtle signs you’re only the "problem" because you don’t let people do whatever they want to you. You’re not the problem at all; rather, they’re just irritated that they can’t continue to mistreat you.

Here are 11 signs you’re only the ‘problem’ because you don’t let people do whatever they want to you

1. Saying 'no' means you're selfish

woman who thinks her partner is selfish for saying no Photoroyalty | Shutterstock

A lot of people struggle with saying "no," especially if they’re a people-pleaser, but we all have to learn when to do so to protect ourselves and our health. Someone who has no respect for you and wants to be able to treat you however they see fit will see your "no" as selfish instead of healthy.

Beyond just the general idea of people-pleasing, saying "no" could make you feel guilty for not helping the other person out, or it could make you worry that you’ll upset that person and your relationship dynamic will shift. It’s even possible that you just want to be the person who can do it all.

Toxic people that you don’t need in your life will prey on these concerns and weaponize them against you. But it’s never selfish to say "no" if it’s the right answer for you. Good, fair people will understand that.

RELATED: If A Man In Your Life Uses These 11 Phrases, He's Likely A Very Selfish Person

Advertisement

2. They give you vague criticism

toxic woman giving her friend vague criticism MDV Edwards | Shutterstock

Healthy relationships require people to call each other out at times, even though they’re close. This is done from a place of love because they want to help you become a better person. In these cases, that person will tell you the specific thing you should have done differently instead of just making a blanket statement.

But if someone gives you vague criticism, it's just one of the glaring signs you’re only the problem because you don’t let people do whatever they want to you. Clinical psychologist Ellen Hendriksen explained that this is the opposite of constructive criticism. Instead, it makes you feel like you’ve been rejected and like you did something wrong.

It’s one thing for someone to say that they think you shouldn’t bring up a certain topic in conversation, but it’s an entirely different thing for them to say you’re just a bad conversationalist. This small difference is very important.

RELATED: People Who Can’t Handle Criticism Always Use These 11 Phrases

3. They never adjust, but they expect you to

woman on the phone with someone who never adjusts but expects her to MAYA LAB | Shutterstock

A clear sign that someone falsely thinks of you as the problem is that they expect you to shift and change to fit their agenda, but they never do the same. They want you to drop a subject because it upsets them, or they want you to be more open-minded and flexible. And while they expect you to accommodate them in that way, the behavior is never reciprocated.

If someone you’re close to does want you to change in some way, there’s a right way for them to ask you to do so. They need to both demonstrate how this change will benefit your relationship, and support you while you make the adjustment. If someone presents you with some vague idea like “be nicer,” it doesn’t really fit. This can show how much that person really cares, and how self-centered their request is.

RELATED: 12 Subtle Ways You're Being Manipulated By A Toxic Person

Advertisement

4. You feel better when you’re alone

woman who feels better when she's alone and away from toxic people DukiPh | Shutterstock

Humans aren’t meant to be solitary creatures and are hard-wired for connection, but sometimes it truly is better to be alone than to subject yourself to being around a specific person. If you feel more content when you’re not around them, or you breathe a huge sigh of relief whenever you can get away from them, it means that they’re probably baselessly acting like you’re the problem.

Toxic people are known for leaving you feeling drained, ignoring your boundaries, and hurting your health. In some cases, they may even be cruel or narcissistic. That’s not the kind of person that’s going to fill your cup. Instead, you’ll leave those interactions feeling worse than you did to begin with. Never think that you have to put up with someone who sees you as a problem.

RELATED: Women Who Feel More At Peace When They’re Alone Usually Have These 10 Distinct Personality Traits

5. They say you’ve changed

toxic woman telling her friend that she's changed ArtMari | Shutterstock

If someone suddenly notices that they can’t manipulate you the way they could before, they may accuse you of changing. In this case, they’ll withdraw any support they offered and treat you like an outcast. It’s best to end these relationships if possible, or at least try to spend less time with that person.

It’s entirely possible that you did change. Maybe you used to go along with letting them treat you however they wanted to, but you came to the realization that this was wrong and hurtful. Even if it’s true, it means that you experienced positive growth. If someone really cares about you, they’ll celebrate that growth instead of trying to tear you down for it.

RELATED: 12 Telltale Traits Of Someone Who Is A Conniving, Manipulative Person

Advertisement

6. You feel guilty for having boundaries

woman who feels guilty for having boundaries YURII MASLAK | Shutterstock

There is absolutely nothing wrong with putting healthy, reasonable boundaries in place. Establishing boundaries is actually the best way to avoid facing problems like burnout and anger, and is a way to show ourselves some love. Good people understand that and will show your boundaries the respect they deserve.

But if you're feeling guilty for having and enforcing those boundaries, it's another of the unfortunate signs you’re only the problem because you don’t let people do whatever they want to you. People who think you’re a problem will treat your boundaries like a burden, and they’ll make it clear that they think you should let them go.

RELATED: People Who Lack Personal Boundaries Say These 11 Phrases On A Near Daily Basis

7. You can’t talk about things that bother you

woman turned away from man not listening to him talk about things that bother him DexonDee | Shutterstock

We’ve all been in situations where it would be better to just leave things alone and not bring up issues from the past. But if you want to have a strong relationship with someone, you both have to be willing to discuss things that rubbed you the wrong way.

If someone invalidates your pain and claims you’re causing a problem by trying to work through something that was hurtful for you, they don’t care about you, and they probably see you as a problem. Research has actually proven that talking through a problem you’re having helps you feel better, and you should always be able to address that problem with the person who caused it.

If you feel like someone is doing their best to keep you quiet and avoid the hard conversations, they probably are, and that’s not a good sign.

RELATED: A Person Who Does These 11 Things While Talking To You Is Almost Always More Narcissistic Than You Realize

Advertisement

8. They offer approval when you agree

woman offering approval when other woman agrees with her PaeGAG | Shutterstock

We all struggle with our self-confidence at times, so it can be really helpful when someone you care about gives you some validation. Someone who thinks of you as the problem isn’t going to give you the validation you want unless you’re agreeing with them, though. They’ll show that you’ve earned their approval when they find your actions acceptable.

This is actually the exact opposite of what validation should be. Therapist Karyn Hall noted, “Validation is a way of communicating that the relationship is important and solid even when you disagree on issues. Validation is the recognition and acceptance of another person's thoughts, feelings, sensations, and behaviors as understandable.” 

This means the other person shouldn’t feel the need to agree with you to approve of you. If they can’t follow this formula, you’ve become a problem in their eyes.

RELATED: 7 Mindset Shifts That Will Set You Free From Other People's Approval

9. They take issue with your self-respect

dad who takes issue with his daughter's self-respect sebra | Shutterstock

Self-respect is a good thing, but some people don’t see it that way. If you lack self-respect, you’ll be easier to manipulate and mistreat. When someone takes issue with your high level of self-respect, it's yet another of the signs you’re only the problem because you don’t let people do whatever they want to you.

Having self-respect is really the bare minimum for wanting to be treated well. If you respect yourself, you’ll expect others to do the same, but having self-respect doesn’t mean you’re special or egotistical.

Being able to enforce boundaries and expect the treatment you deserve starts with self-respect. Psychologist Dr. Judith Zackson said, “It’s hard, if not impossible, to live a fulfilled, meaningful, and joyous life if you don’t respect yourself first.” The right people will admire your sense of self-respect, but it will bother people who think you’re being a problem.

RELATED: People Who Lack Self-Respect Almost Always Do These 11 Things On Repeat

Advertisement

10. They liked you better when you gave your all

woman who liked her friend better when she gave her all Dragana Gordic | Shutterstock

In many cases, someone will decide you’re the problem because you used to act differently. If you didn’t have firm boundaries before, you might have let them walk all over you, and they got used to that. They loved being able to treat you however they wanted to and depended on you to always bring your all to every situation, even if they didn’t do the same.

Now that you’ve changed, you don’t give them everything you have and accept nothing in return, which is an issue. There’s no way they’re going to like you as much in this situation. It can feel extremely anxiety-inducing to do something you know won’t make people like you, but it’s important to be true to yourself and your needs.

RELATED: 7 Everyday Habits That Genuinely Confident People Adopted — Not Because They Wanted To Be Liked But Because They Realized It Earned Them Respect

11. You could ‘fix’ the dynamic by compromising

woman who thinks man could fix their dynamic by compromising Zamrznuti tonovi | Shutterstock

Every friend group and relationship has its own dynamic. Oftentimes, we hold that dynamic together by all playing different roles. Maybe you’re the “mom” of the group, or you’re the one who’s always supportive, for example. If you come to respect yourself more and enforce boundaries, there’s a good chance your place in the group will change, as will the dynamic.

A toxic person might look at this kind of situation and think that it’s your fault the pieces don’t fit the way they used to, and you just need to compromise so that everyone can get along the way they did before. But this would be compliance, and that’s never worth it. Psychology educator Kendra Cherry defined compliance as “changing your behavior because someone asked you to do so.”

In some cases, compliance is necessary, but doing so at the expense of any independence and confidence you’ve gained is not right. They should love who you are even if it’s different, not think you’re the problem.

RELATED: 10 Ways Emotionally Manipulative People Make Everyone Around Them Feel Small

Mary-Faith Martinez is a writer with a bachelor’s degree in English and Journalism who covers news, psychology, lifestyle, and human interest topics.

Advertisement
Loading...