Right, Simon, it IS nice that you can simply “pull in” (to a JOSM edit session) only the part(s) of a gigantic multipolygon that you actually need to be in the buffer while editing. One of my favorite features of JOSM actually (and there are lots, I’m a big fan).
The reason it has a performance hit is simple: the data are huge. I’ve noticed a slowdown in JOSM performance with very large datasets for at least 10 (13?) years. My solution has been to buy hardware (RAM, especially) that accommodates such editing, and tweaking editor settings from my command-line interface accordingly as I have illustrated. And BTW, the near-constant improvement in JOSM, including performance, extensibility, bug fixes and general improvements (it is solid, mature, even awesome software!) has been nothing short of tremendous for the benefit of both JOSM (human) editors and OSM’s downstream data consumers.