OpenSSH 10.4p1 uplift

[andrewyounkers]

This PR merges in upstream commits from the OpenSSH-portable V_10_4_P1 tag by running a git merge V_10_4_P1. The main changes that I noticed during the uplift causing conflicts had to do with the upstream addition of enabling the ssh-mldsa44-ed25519 hybrid signature algorithm.

All merge conflict resolutions are shown in commit: 1dc4dc1. I used the following methodology to resolve all merge conflicts

• Accept current to preserve OQS specific logic
• Accept any combinations needed to bring in updated upstream code but keep OQS specific code
• Accept incoming for all other conflicts to bring in new upstream logic

In the past OQS-SSH has rejected upstream OpenSSH hybrid PQ algorithms due to them already being implemented in the project, but this is a case where the upstream is adding a new algorithm the project doesn't already support. Since there was no conflict in bringing in this new algorithm, I kept the upstream code enabled and as such ssh-mldsa44-ed25519@openssh.com is now available in this project. I added a note to the README to address this in the following commit: 1ef3c3a.

[baentsch]

In the past OQS-SSH has rejected upstream OpenSSH hybrid PQ algorithms due to them already being implemented in the project,

Thanks for pointing this out @andrewyounkers ! This in my eyes is a statement and stance that surprises me: Can you point to such instances? What's your personal thought around this: Are you of the opinion that OQS has the resources and development principles allowing it to replace (presumably) well-maintained upstream code?

fwiw, in oqs-provider (for openssl) I took the exact opposite stance, namely to remove PQ algs (and of course also their hybrids) that the upstream began to carry on the grounds that the upstream is the professionally maintained crypto lib and OQS doesn't have resources nor development principles warranting it over-ruling such well-maintained upstream code.

On the alg itself,

the upstream addition of enabling the ssh-mldsa44-ed25519 hybrid signature algorithm

allow me to ask the question what this update then removes from the upstream: The complete crypto, i.e., also an ML-DSA from the upstream, or just the hybrid-ization, i.e., ML-DSA still comes from the upstream? If the former, are there no other hybrids (that then would differ in terms of where the PQ logic comes from)?

FWIW, for openssl I began work on a provider that does hybrid-ization independent of the original crypto provider to still allow PQ experimentation -- but not by-default "overrule" the upstream: Might the same approach be worth-while considering here?

All told, I see the principle

Accept current to preserve OQS specific logic

highly skeptical and wouldn't follow it nor approve a PR doing so. Good I have not been asked to do so :-). So just take this statement as a correction of possibly committed prior crimes (should I have set the precedent you mentioned above of rejecting upstream OpenSSH hybrid PQ algorithms: I am willing to stand corrected when I see I made mistakes).

[andrewyounkers]

In the past OQS-SSH has rejected upstream OpenSSH hybrid PQ algorithms due to them already being implemented in the project,

Thanks for pointing this out @andrewyounkers ! This in my eyes is a statement and stance that surprises me: Can you point to such instances? What's your personal thought around this: Are you of the opinion that OQS has the resources and development principles allowing it to replace (presumably) well-maintained upstream code?

The two examples that I was referencing were sntrup761x25519-sha512@openssh.com and mlkem768x25519-sha256. The openssh implementation of these algorithms are disabled in defines.h and OQS-SSH/liboqs logic is used instead to provide these algorithms. This logic was in place before my involvement with OQS-SSH so I had preserved it throughout uplifts.

I am personally of the opinion to prefer the upstream code for the reason you mentioned of it being well-maintained code. I am willing to swap these algorithms to the OpenSSH implementation in this PR for all the reasons you provided above and carry on with the precedent set by the oqs-provider.

fwiw, in oqs-provider (for openssl) I took the exact opposite stance, namely to remove PQ algs (and of course also their hybrids) that the upstream began to carry on the grounds that the upstream is the professionally maintained crypto lib and OQS doesn't have resources nor development principles warranting it over-ruling such well-maintained upstream code.

On the alg itself,

the upstream addition of enabling the ssh-mldsa44-ed25519 hybrid signature algorithm

allow me to ask the question what this update then removes from the upstream: The complete crypto, i.e., also an ML-DSA from the upstream, or just the hybrid-ization, i.e., ML-DSA still comes from the upstream? If the former, are there no other hybrids (that then would differ in terms of where the PQ logic comes from)?

This update doesn't replace the complete crypto implementation of ssh-mldsa44-ed25519 from the upstream. The upstream calls OpenSSH's bundled libcrux ML-DSA implementation where all other signature algorithms call liboqs. If the above algorithms were swapped to enable the OpenSSH code, only the three algorithms mentioned in this comment would differ in terms of where the PQ logic comes from.

FWIW, for openssl I began work on a provider that does hybrid-ization independent of the original crypto provider to still allow PQ experimentation -- but not by-default "overrule" the upstream: Might the same approach be worth-while considering here?

I do think the same approach of the hybrid provider would be worthwhile here especially as more PQ algorithms become supported by OpenSSH. I can make an issue for tracking this feature in the long-term. Moving forward until that is implemented, I think it is best to prefer the upstream algorithm when OpenSSH makes one available.

All told, I see the principle

Accept current to preserve OQS specific logic

highly skeptical and wouldn't follow it nor approve a PR doing so. Good I have not been asked to do so :-). So just take this statement as a correction of possibly committed prior crimes (should I have set the precedent you mentioned above of rejecting upstream OpenSSH hybrid PQ algorithms: I am willing to stand corrected when I see I made mistakes).

Thank you @baentsch for the thoughtful comments, I hope this addressed your questions but will be able to further discuss if there is a follow-up. I will include a reply to the last section in a new comment as my response is a bit long - sorry :-)

[andrewyounkers]

@baentsch In regards to the comments around " Accept current to preserve OQS specific logic", I would like to clarify what I meant by that bullet point and seek guidance for the future.

When resolving merge conflicts my priority is to always bring in the new upstream code first. If there is OQS specific logic that after investigating needs to be kept, I bring in a combination of the changes to get the incoming logic without breaking the OQS specific logic in place. I will note there are not very many of these cases.

Majority of the merge conflicts that I have experienced during the uplifts come from OQS code surrounded by ///// OQS_TEMPLATE_FRAGMENT comments or the OQS specific case statement CASE_KEY_OQS:. Most of what I have seen is that the incoming changes just remove these sections, so accepting "current" for these conflicts just preserves what is needed for OQS algorithms without losing any new upstream logic. If there are upstream changes in the conflicts surrounding the OQS code, the new code gets brought in as a combination. This is what I was referencing in the above bullet point, which I can now see wasn't a very good or clear description on my end.

With this explanation does it alleviate any of your concerns? If this still is troubling is there a better process that I can follow in future uplifts? I apologize for not being more clear in my description, and will include more thorough explanations of the merge conflict resolution process in future uplift PRs.