Will it really be much worse than it is now?
Losing your entire volume is not all that much worse than losing your home folder, is it?
Depends on how you use your computer, I reckon. I actually keep very little in my home folder; I store most of my data on the top level of the hard drive.
One big difference I can see for folks who use their home folders as they ought is applications. If you lose access to your home folder, you lose your data; if you lose access to the entire disk, you lose your data and your apps. (Well, assuming you don't have backups. But everyone keeps good backups, right? Right?)
I generally don't recommend FileVault for most folks. It's really only useful for people who have data with significant economic, legal, industrial, or military value, or folks with legally regulated data like health records. (I also don't keep my banking password in my Keychain, so take that for what it's worth.) The only time I've ever used FileVault is when I went on vacation overseas, because I didn't want customs pawing through my iPhoto library when I came back, which they sometimes do.
