FSD almost unusable on highways

I have always been a heavy user of Autopilot. I drove from California to Connecticut over 6 days and used it 80% of the time on highways. But for the occasional phantom braking, it was extremely useful and was a big stress reliever on long, uninterrupted drives. I got FSD when it became generally available and have since used it on highways and local roads. I have heard on these forums from many who seem to be fans of it and use it regularly. In my own experience, while it does some weird moves occasionally, it's impressive how much it has improved while driving on side streets. But it's another story while driving on highways, so much so that I've almost stopped using it. It has to do with how it handles speed limits.

Connecticut's highways look like something out of Mad Max most times. When speed limit is 65, it feels like 85 is the norm. I usually fix it at 75 and drive in the middle or the right lane, and can still feel everyone zooming past me. That's all fine, until the speed limit changes. At many places, it drops to 55 or even to 45 fairly suddenly. That's when the FSD detects it and drops the speed immediately and quite suddenly. A drop down from 75 to 55 can be quite jarring and 75 to 45 can be positively scary, almost like the phantom braking. Of course, none of this would be a problem if everybody did the same but that's never the case. People don't slow down just because the speed limit drops down (most don't even notice it), they do so only when compelled to do so, like when they come to a steep curve in the road. Now, I can set a speed offset, and I do (about 15% above the limit), but that's no solution because the offset applies at both the higher and the lower speed, so the drop is still quite steep. I have to constantly monitor for this happening and be ready to step on the accelerator suddenly, just so the car coming behind won't ram into me when mine starts slowing rather aggressively. I've had many a time, been honked at aggressively and cursed by annoyed drivers behind. This becomes so stressful in that it takes away the from the supposed 'automatic' driving mode.

What is interesting about this is - while all the intelligence is applied to ensure that the car avoids colliding with the car ahead - with multiple controls and settings, there's almost none to make sure that it doesn't end up being hit from behind. Of course, I know that there's no radar on the back (with the disabling of the one in the front, that point is moot anyway), and there aren't as many cameras as in the front. But, since it's almost all camera dependent now, there should be some precaution to watch out for following traffic when braking. The only solution I see, is an option 'follow surrounding traffic behavior for downward speed limit changes'. I don't know if that's possible but I can't think of any other alternative.

What I'm trying to find out is, how is everybody handling this? I don't recall many people mentioning this, so I find it quite puzzling. Does no one else find this annoying? Am I missing something?