I've never cared what passing strangers thought, and friends come and go. The only friendships worth having come about from connections far deeper than money, position and convenience anyway and I make more than enough of those to keep me happy. The only reactions that have caused me problems have been those of some extended family members and neighbours.
One (affluent) neighbour in particular suddenly started talking to us again when he thought we'd got a new car - and stopped when it turned out to be a courtesy car! My mother was so worried that we'd inevitably become a drain on her (much more ample) resources that our relationship was often extremely strained. There were some other reasons for that, but I think money was a main one. Even my old dad, who lives almost as happily in relative poverty as we do, keeps wishing that we'd "win the lottery" - a difficult feat, since we don't buy tickets!
In general, it's a bit like the elephant in the room. It seems to make people feel uncomfortable, though it's taken me many years to empathise with the other person's point of view and realise to some extent why my decision might have caused so much general discomfort. I think people might have felt primarily bewildered by our refusal to follow the herd, which is similar to the reaction we get to our our off-grid plans and home education. There's a kind of glazing over of the eyes, and you can see them trying not to think about it until they've had enough time to process it. (Another neighbour, who'd blanked me for five years after I deregistered the children, excitedly knocked on my door one day to inform me that she'd just heard about home ed on Radio 4, "So it must be ok!")
I think, unhappily, that we invoke feelings of guilt in some relatives though, and possibly shame in others. I'm sorry about that, but I can't help it. We don't go about pushing ourselves in their faces though, so if they want to forget about us then they can - with impunity, as far as I'm concerned. It's been many years since I stopped hoping or even wishing for anything else. I still want to be part of a big, mutually supporting tribe but I think my best hope for that will come from the younger generation now, and/or from those deeper connections that always withstand the test of time.