AITA for refusing my parents' request to be trustee for my brother's inheritance?

Throwaway account.  Context: I (45M) have one brother (40M).   My parents (71M,F) both had successful careers and have accumulated enough wealth that they need to do estate planning.  I got very lucky and have had a lucrative finance career that has left me quite a bit wealthier than they are (which they know, roughly).  My brother and his wife both work for non-profits, but have climbed their respective org charts and have a good middle-class income.  My parents have helped them from time to time, for example with a downpayment for a house when they got married and private school tuition for my young nephew.  He’s a little more into shopping and costly hobbies  than I am – but I live an unusually spartan life. Brother and I have a good relationship, though we live on opposite sides of the country, so we only get together a few times a year. 

When my parents last visited me, they pulled me aside to tell me about their estate plans.  They intend to leave everything to my brother and his family, because I’m already financially set and because they don’t want a giant disparity in wealth between their two sons.  I’m entirely fine with that.  It’s their money, obviously; I don’t need any; and I love my brother and his family and want them to be comfortable.  I told my parents as much – so far so good.

Here's where it got problematic.  Parents said, “it’s a lot of money and we don’t want it to just drop in [Bro’s] lap.  So we are setting up a trust to receive the money, to benefit [Bro] and then [nephew].  We want you to be the Trustee. ”  I told them that wasn’t going to be possible.  I don’t want to superintend my brother’s finances.  It seems like a recipe for strife and division. If I just let him do what he pleases and the money is dissipated, he could resent that I didn’t invest it better or exert more control; if I am conservative about disbursements, then he could resent my “heavy hand.”   I also told them that while my bro is not quite the penny-pincher I am, he is far from irresponsible with money (or generally).

My parents reacted poorly.  They claimed that it is not a big ask.  I said that, actually, it is kind of a big ask for me to serve as Trustee for who-knows-how-long , and that the one sliver lining of getting no inheritance should be that I don’t need to be involved in the estate.  I offered to refer them to a professional trust company. They accused me of refusing out of jealousy.  That was too much for me, so I pulled out my banking app and showed them that what was just sitting in my checking account was close to their estate (to be honest, had just received a large deferred comp wire from my firm, so it was unusually large –but, look, I was annoyed at their accusation and wanted to rebut it as quickly and convincingly as possible).  When we parted, it was a little tense and my mom said, “well, I guess we were wrong to think we could count on you.”

Now I am thinking maybe I should just suck it up and do it.  AITA?