Add support for Wrap Tokens v1

[MikhailMS]

This PR:

• fixes Improve the support of gssapi/sasl, fully realize wraptoken. #460
• enables the fix for GSSAPI Kerberos Authentication: wrong Token ID IBM/sarama#2022

This was tested against life Cloudera Kafka clusters (multiple versions) and everything is working nicely

I'm not the greatest Go expert, so if there are any issues with the code, let me know & I'd get it fix asap
Really kin to get this one merged, so Sarama can also get patched, which in turn would allow other applications in our stack to receive the fix

[MikhailMS]

@jcmturner would appreciate if this PR could be reviewed and accepted :) if anything else needs to be done on this one - let me know & I'd try to get it all sorted

[tzsebe]

@MikhailMS thanks for this contribution! It looks like the final change set adds wrapTokenV1, but I don't see any additional code upstream that's detecting the format and using the V1 token where needed. Does that still need to be added, or does this just work out of the box through some mechanism I'm not seeing? Thanks again!

[MikhailMS]

@tzsebe hello there, thank you for the time given to review this PR :)

That is correct note - from this library perspective, the change proposed in this PR is all that needed to be done

Then, to make use of this new code, upstream application would need to do some work to get the support for V1 token (see this PR I made to Sarama package some time ago)

Probably the code change I've proposed in Sarama project could be added to this library instead - I cannot remember why exactly I did it this way, probably to separate concerns (as in actual application needs to know when to use which, not the library to auto-do so for the app) 🤔

[tzsebe]

@MikhailMS thanks for getting back to me!

I'm looking at colinmarc's HDFS library, which would need a similar change in a few places.

I'm not enough of a SME to know if there's any deterministic way to pick V1 or V2, or if V2 tokens are actually supported by everyone. The Java Kerberos client seems to select between V1 and V2 tokens based on "old" vs "new" encryption schemes:

• Their wrap method: https://github.com/openjdk/jdk/blob/jdk-11+0/src/java.security.jgss/share/classes/sun/security/jgss/krb5/Krb5Context.java#L990
• If you trace it down, that's ultimately driven by: https://github.com/openjdk/jdk/blob/master/src/java.security.jgss/share/classes/sun/security/krb5/internal/crypto/EType.java#L320

I don't know if this is a limitation of tokens (where V1 MUST be used for those weak encryption schemes due to format), or some kind of historical backwards-compatibility decision that no longer matters in practice, but if that logic is important, it'd probably be good to incorporate it into GoKRB5 as well.

Perhaps it'd be as simple as adding some top level function calls that handle the switching, e.g. Parse(), which inspects the first two bytes and decides whether to construct and unmarshal into a V1 or a V2 token, and New(...), which would take in encryption scheme as one of the parameters, and instantiate according to logic similar to how the Java client does it. This assumes there's a reasonable interface that can capture both types (would Marshal(), Unmarshal(), and Payload() be sufficient?)

We can also be sneaky and compose the v1 into the existing one, delegating to it as needed, which would result in a backwards-compatible change that probably works for every client, but would be kind of a hacky structure. The other option feels cleaner to me.

[MikhailMS]

Perhaps it'd be as simple as adding some top level function calls that handle the switching, e.g. Parse(), which inspects the first two bytes and decides whether to construct and unmarshal into a V1 or a V2 token, and New(...), which would take in encryption scheme as one of the parameters, and instantiate according to logic similar to how the Java client does it. This assumes there's a reasonable interface that can capture both types (would Marshal(), Unmarshal(), and Payload() be sufficient?)

This is it I believe; but that would require a bit more code written :)

Althought I am not yet seeing a way to generalise this nicely, as some functions are not currently sharing the same signature and there is a diff logic between the versions, of which the caller need to be aware anyway, soo 🤔

(another semi-problem is that I no longer have an access to the setup against which I've tested this;
I however know that there is still a need for this fix to be completed 👍)

[tzsebe]

Ah, too bad about the setup. Is it hard to recreate? Bringing up a Kerberos-fronted HDFS cluster is easy with dataproc, but I wasn't quite able to configure it to support RC4 in a way that reproduces the issue. The old encryption schemes aren't really supported, unsurprisingly.

When I get some time, I'll try to get a deeper understanding of how and why the public interfaces differ between the two.

[MikhailMS]

Is it hard to recreate?

Tbh, have no idea; my last attempt at setting up Kerberos + Kafka was back in 2018, and it drove me crazy, so not looking forward trying again :)

Functions-wise, the only functions which currently different (as in different signatures) are Marshal & NewInitiatorWrapToken*

// For Token V1, we MUST encrypt SndSeqNum, so
Marshal(key types.EncryptionKey) ([]byte, error)

// For Token V2, it seems to be not the case, so
Marshal() ([]byte, error)

// For Token V1, we MUST encrypt SndSeqNum, so
NewInitiatorWrapTokenV1(initial_token *WrapTokenV1, key types.EncryptionKey) (*WrapTokenV1, error)

// For Token V2, it seems to be not the case, so
NewInitiatorWrapToken(payload []byte, key types.EncryptionKey) (*WrapToken, error)

we cannot really move the encrypt call out of Marshal, so the only real way out here is to change V2 signature to match V1, which would be odd 🤔

for New*, we can do some changes to allow []byte for V1 instead of WrapToken but that may be sub-optimal

And then, while signature is similar, but actual parameters that needs to be passed are different

// For V1, we need Verfiy to be called as
wrapTokenReq.Verify(krbAuth.encKey, keyusage.GSSAPI_ACCEPTOR_SIGN)

// For V2, we can go with
wrapTokenReq.Verify(krbAuth.encKey, keyusage.GSSAPI_ACCEPTOR_SEAL)

so, the caller still need to know what Token Version they are working with (albeit we can probably hide this under higher level Verify function with signature of Verify(token, krbAuth.encKey), but not sure how to make this pretty)

🤔

[oiweiwei]

@MikhailMS thanks for getting back to me!

I'm looking at colinmarc's HDFS library, which would need a similar change in a few places.

I'm not enough of a SME to know if there's any deterministic way to pick V1 or V2, or if V2 tokens are actually supported by everyone. The Java Kerberos client seems to select between V1 and V2 tokens based on "old" vs "new" encryption schemes:

• Their wrap method: https://github.com/openjdk/jdk/blob/jdk-11+0/src/java.security.jgss/share/classes/sun/security/jgss/krb5/Krb5Context.java#L990
• If you trace it down, that's ultimately driven by: https://github.com/openjdk/jdk/blob/master/src/java.security.jgss/share/classes/sun/security/krb5/internal/crypto/EType.java#L320

I don't know if this is a limitation of tokens (where V1 MUST be used for those weak encryption schemes due to format), or some kind of historical backwards-compatibility decision that no longer matters in practice, but if that logic is important, it'd probably be good to incorporate it into GoKRB5 as well.

Perhaps it'd be as simple as adding some top level function calls that handle the switching, e.g. Parse(), which inspects the first two bytes and decides whether to construct and unmarshal into a V1 or a V2 token, and New(...), which would take in encryption scheme as one of the parameters, and instantiate according to logic similar to how the Java client does it. This assumes there's a reasonable interface that can capture both types (would Marshal(), Unmarshal(), and Payload() be sufficient?)

We can also be sneaky and compose the v1 into the existing one, delegating to it as needed, which would result in a backwards-compatible change that probably works for every client, but would be kind of a hacky structure. The other option feels cleaner to me.

Just stumbled across this discussion and maybe what I've discovered can be useful for your further considerations:

As per https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-kile/e94b3acd-8415-4d0d-9786-749d0c39d550

If the session key encryption type is AES128-CTS-HMAC-SHA1-96 or AES256-CTS-HMAC-SHA1-96 (as specified in [[RFC3961]](https://go.microsoft.com/fwlink/?LinkId=90450)):

The base line is [[RFC4121]](https://go.microsoft.com/fwlink/?LinkId=90459).

If the session key encryption type is RC4-HMAC or RC4-HMAC-EXP per [RFC3961]:

The base line is [[RFC4757]](https://go.microsoft.com/fwlink/?LinkId=90488).

The ordered list contains the header ([RFC4757] section 7.3).

The data (excluding the conf_req_flag set to FALSE) is encrypted in place.

The "to-be-signed data" is a concatenation of all the input buffers for which sign==TRUE. The InitialContextToken pseudo ASN.1 header is included at the beginning of the token header.

So this may give some explanation why "old" ciphers or "new" ciphers are what they are from format perspective.

Here you can find some code where WrapToken (with encryption only) and MICToken are implemented for RC4/AES ciphers: https://github.com/oiweiwei/go-msrpc/blob/main/ssp/krb5/crypto/aes_cts_hmac_sha1.go