can anyone tell me if this is legal?

So on defence, I always try to box people out almost immediately for the rebound, or the put back if a miss happens on offense, the moment the ball flies through the air. However, I don't know if the referee is not seeing this, but I've started seeing more of my opponents use their shoulder to push me out of the way. He's not pushing me straight up, but these players are using their shoulder and bumping, or sometimes almost jumping into me like they want the defensive rebound so badly and they want to knock me down. It all happens too fast, so I can't really see what the opponent is doing that the referee doesn't call when they bump into me. I swear pushing people this hard is illegal, or is it okay to push with your shoulder as long as it doesn't extend out or instead lean into them with your shoulder? I mean, it hurts like mad, but my core is strong so I don't get pushed out of the way easily, although after me or the opponent gets the rebound, the pain starts to set in like crazy and I usually pass the ball out because I need some time for it to go away. Is it also okay if I do what these guys are doing when I'm trying to get the defensive or offensive rebound? Use my shoulder and body and push or lean into them really hard? Is that a legal rebounding tactic?