I suspect there is not unique answer for "too big". As you have said it depends on…, and the "on…" can vary with each spreadsheet and amount of installed RAM. If someone is looking for a definitive number, like "when you hit x MB you will crash", that number just won't happen.

In your case, as previously suggested by many, the simplest solution may be to add more RAM and extend the life of your spreadsheets, regardless of size.

On a Mac since 1984.
Currently: 24" M1 iMac, M2 Pro Mac mini with 27" BenQ monitor, M2 Macbook Air, MacOS 14.x; iPhones, iPods (yes, still) and iPads.