There’s always some trust involved when using Ledger hardware wallets since their firmware is closed source, their hardware is closed source, the secure elements they use are closed source, etc.
Ledger’s new recovery service, Ledger Recover, involves exporting the seed from a hardware wallet and storing it in an encrypted, sharded form at several custodial companies. Ledger if facing criticism for this since they’ve previously stated that a firmware update couldn’t allow the seed to be exported from a hardware wallet.
I personally don’t recommend anyone use Ledger’s recovery service since it appears to use the same key to decrypt everyone’s seeds based on this comment.
If they were encrypted, then the funds would not be accessible if the shares were revealed, e.g. in subpoena. The Ledger cofounder cited above says that shares accessed via subpoena would allow access to the funds. This is only possible if the shares are not encrypted.
That’s not what the Ledger founder said, but even so would that be any better? If your keys are stored on 3rd party servers and accessible by 3rd parties, the rest doesn’t really matter. At that point you’ve lost the whole reason of using a hardware wallet.
I’ve always said people were foolish for trusting Ledger (or other similar companies), but anyone that continues to use ledger now is just willfully ignorant. They’ve lost the plot.