Try Shift+Command+L (⇧⌘L) to go directly to your user Library, BUT you must click on the desktop first, to switch the focus to Finder.

Actually I had missed that addition to the GO menu blush

If we knew what it was we were doing, it wouldn't be called research, would it?

— Albert Einstein