I emailed Kurt Lang and he was kind enough to respond quickly and in detail:

____________

The main issue is likely scale factor. The default (and native) resolution for that screen is 1920x1080.

Here's the problem. The lower resolutions are all proportional (same scale factor in height and width), which means a circle will still look like a circle and not an ellipse. But, the scale percentages are all oddball. As in, not whole number divisions of 2, 3, 4, etc.

1600 x 900 = 83.33%
1344 x 756 = 70%
1280 x 720 = 66.66%

Computers with LED screens don't like these kinds of screen values, which is something joemikeb already explained.

It didn't matter on old CRT screens since an electron gun was far from anything you'd call accurate. And a pulse would almost always at least partially excite a neighboring pixel, causing it to blur out the pixel being sent to the screen. Meaning, nothing was ever super sharp on a CRT and jaggies practically didn't exist. It had to be something obvious before you'd notice.

On an LED, when you're trying to increase the image size by lowering the resolution, you run into this percentage problem. Take 1600 x 900. Remember, the screen itself is always 1920 x 1080, so what's drawn has to be dumbed down to fit.

Okay, the OS needs to draw one pixel. Which pixel on the screen is it supposed to use? The call to display one pixel is now larger than the scale factor. Should it go to pixel 1122 in the row, or 1123? Does it mathematically land right in the middle and the OS displays black on both neighboring pixels? Or does it skip drawing it at all and those two spots remain the background color? As you round the curve of say, the letter S, is the pixel on row 82 going to draw on pixel position 448, and on row 83 the next pixel of the curve draws on pixel position 450 rather than 449 because that's how the math came out for the nearest pixel position?

There isn't much you can do about this when using non-native resolutions on screens with exactly defined pixel locations.

Though one thing that could help is using HiDPI. I don't know if this works anymore, but you can give it a try. Open Terminal and copy/paste this line into it:

sudo defaults write /Library/Preferences/com.apple.windowserver.plist DisplayResolutionEnabled -bool true

Quit Terminal.

Since the file you're trying to add this to is in the root Library folder, you may get a message that you don't have permissions. If that's the case, you'll have to temporarily disable System Integrity Protection so you can run the command, then turn SIP back on.

Open the System Preferences and click Displays. Click the Scaled radio button. Now, hold down the Option key and click directly on the word Scaled.

If this still works, you'll find lower resolutions with (HiDPI) listed after the values. The largest one you can pick is half the native resolution. What this does is cause the Mac to draw everything with much better smoothing results.