I thought Mike Bombich of Carbon Copy Cloner fame has said that indeed, the only way (for now) to create a bootable external drive, was to use Apple's Installer first to install the OS on the external drive then use CCC to clone the Data volume. Clunky but bootable. You as much said this in your fifth bullet point above.
Unfortunately Mike's solution does work on M1 Macs. I just spent the last four hours trying to create a bootable external drive with zero success. What I have learned is:
- The App Store installer does not work any better than the Recovery Disk installer, just a bit differently.
- Either installer requires a password for an admin user on the system to install on any drive/volume on the system
- Copying user settings is an option when initiating the installation.
- Whether the user data is copied or not, if the authorizing account is on an encrypted volume the install fails with the error unable to transfer ownership
- Once the installer is downloaded the full installation took 18½ minutes to the first boot attempt. (The installer claims 54 minutes)
- Even if the installer completes normally the external volume boot fails with an error 104 (dsNeedToWriteBootBlocks)
- I went through the steps to create a bootable thumb drive using the command line and Apple's own tool for creating a bootable drive. The process completed successfully, but the drive will not boot.
If I sound discouraged, I am. I have no idea what to try next.