my mouse keeps leaving the screen while i’m in an xbox party (game bar)

so i noticed my mouse will leave the screen when i look down (stacked monitors) when i’m attacking and i accidentally click out of the game and i have to go back up and click back in, but when i kill game bar and xbox via task manager and restart my game it doesn’t leave my screen, does anyone know how I can prevent this while being in a party?