Giving access to lower-level foundational cryptographic APIs

[mrts]

Hi JJWT team!

As 1.0 is getting closer now, I would like to propose officially giving access to lower-level foundational cryptographic APIs like in the following example:

import io.jsonwebtoken.Jwts;
import io.jsonwebtoken.impl.security.DefaultVerifySecureDigestRequest;
import io.jsonwebtoken.security.SignatureAlgorithm;

...

    boolean isSignatureValid(String jwAlgorithmName, byte[] signedData, byte[] signature, PublicKey publicKey) {
        var signatureAlgorithm = (SignatureAlgorithm) Jwts.SIG.get().forKey(jwAlgorithmName);
        var verificationRequest = new DefaultVerifySecureDigestRequest<PublicKey>(
            new ByteArrayInputStream(signedData),
            null, null,
            publicKey, signature);
        return signatureAlgorithm.verify(verificationRequest);
    }

This would enable others to build upon the good work that has gone into the implementation of the foundational APIs.

However, there are two concerns with this:

1. Dependence on io.jsonwebtoken.impl namespace and jjwt-impl JAR to access DefaultVerifySecureDigestRequest. The JJWT documentation clearly advises against adding the jjwt-impl.jar to projects with compile scope.
2. The two null parameters in DefaultVerifySecureDigestRequest<PublicKey> constructor and having to use ByteArrayInputStream feel somewhat cumbersome for this use-case, perhaps there is a way to streamline it - or maybe not, this is a minor issue.

What are your thoughts regarding this?

[lhazlewood]

Hi @mrts !

We don't have any plans to expose these lower-level implementation details at this time - they are considered internal implementation details and are not curated/planned/designed for extensibility, or more importantly, supportability.

In other words, the impl module is what is required to implement the public APIs as efficiently and securely as possible without significant thought for elegance or usability or longer term support. Your reference of the two null parameters is a great reflection of this - that implementation is designed only for utility and convenience for the JJWT implementation team, since it's not intended to be used by application developers. (If we did intend for app developer usage, we'd probably enable request creation with a nice builder, but that's beside the point).

As such, the impl module is explicitly documented as private/internal to allow the JJWT implementation team to fix or adjust or change anything at all that might be necessary for bug fixes or new features, or even breaking previous impl compatibility if desired. That is, we explicitly intended this module to be a 'we can change what we want, whenever we want' in the name of speed and quick support for the JJWT user community without fear of breaking users' application code. If we were to expose this module as public, it would greatly tie our hands in implementation speed and flexibility, as we'd have to be much more careful not to break anything for anyone.

It would be best to think of the JJWT impl module in the same light as the JDK 9+ effort to encapsulate internal APIs, which was fully realized in JDK 17, making even reflection access to internal APIs impossible. When we enable JDK 9+ module-info.java files in JJWT soon, this restriction will be enforced just as the JDK does.

So, that's our 'official' stance on why things are they way they are.

I think there could be future work dedicated to enhancing 'supporting' API concepts for application developers that wish to create their own Algorithm implementations - for example, if you create your own SigantureAlgorithm implementation, you might want to quickly create VerifySecureDigestRequest instances during testing, without having to implement that interface yourself. That's a level of 'extra' niceness that we didn't really have time for when trying to release 0.12.x since the very large majority of JJWT users never implement their own SignatureAlgorithm, EncryptionAlgorithm, etc instances. But if there's enough demand for 'provider support' APIs, we could add that to the JJWT public API or perhaps as a secondary spi ("service provider interface(s)") module that only algorithm implementors would want to depend on.

But if the desire is not "I'm using JJWT, and I want to create a custom algorithm implementation", but instead "I prefer a simpler way than the JDK's cryptography APIs, and I'd like to use JJWT's cryptographic primitives out of convenience", then that would be the reason I've created the https://github.com/lhazlewood/scrapi project. It is even possible one day that the next version of JJWT's impl implementation delegates to scrapi and removes the bulk of the crypto convenience classes we created within JJWT.

Does this help? Thoughts?

[mrts]

Hi @lhazlewood!

Thank you for the clarification regarding the impl module. I understand and respect the rationale behind keeping these details internal for the sake of efficient and secure implementation of public APIs. The comparison with the JDK's encapsulation efforts makes the intentions and practices quite clear and I can fully understand the need for flexibility in maintaining and updating these internal APIs without the constraints of external dependencies.

You are right that the desire is not "I'm using JJWT, and I want to create a custom algorithm implementation", but it's not quite "I prefer a simpler way than the JDK's cryptography APIs, and I'd like to use JJWT's cryptographic primitives out of convenience" either. We use JJWT in the web-eid-authtoken-validation-java library for validating Web eID authentication tokens during authentication with electronic ID smart cards in web applications.

Web eID authentication tokens (WeIDAT) are intentionally not in JWT format [1]. Since there does not exist a standardized format of an authentication proof that fulfills the needs of the Web eID authentication protocol, we use a special purpose format for the WeIDAT. However, we still use the proven basic building blocks of JWT in WeIDAT: the JSON format, base64-encoding and JSON Web Signature Algorithms as defined in RFC 7518. The WeIDAT signature verification procedure is therefore pretty much identical to JWT, here is the current implementation using JJWT 0.12.

So, our desire is to leverage JJWT's robust cryptographic components for specialized validation of Web eID authentication tokens, which require the security and reliability of JJWT, close to, but not quite within the standard JWT framework.

Now, the following APIs are already part of the public JJWT api module:

SignatureAlgorithm signatureAlgorithm = (SignatureAlgorithm) Jwts.SIG.get().forKey(algorithm)
VerifySecureDigestRequest<PublicKey> verificationRequest
signatureAlgorithm.verify(verificationRequest)

So, the only point of contention is the availability of the DefaultVerifySecureDigestRequest<PublicKey> class, which would perfectly meet our needs, but is currently only accessible through the impl module. This issue is similar to concerns raised by another user, @icecreamhead, about the DefaultJws class being practical yet classified as internal, as discussed here.

Considering the utility and fit of the DefaultVerifySecureDigestRequest<PublicKey> and DefaultJws classes for specialized usage requirements like ours and @icecreamhead's, I propose considering making similar functionality accessible through the public API without compromising the integrity of the internal API design. I believe this could be a valuable addition for a broader range of use cases.

Thanks in advance for considering this and hats off for your excellent work with JJWT!

---

[1] The OpenID Connect ID Token JWT format was initially considered, but it was found that any similarities of the Web eID authentication token to the JWT format are actually undesirable, as they would imply that the claims presented in the Web eID authentication token can be trusted and processed, while actually they cannot be trusted as the token is created in the user's untrusted environment. The solutions are fundamentally different – the purpose of OpenID Connect (and JWT in general) is to exchange identity claims that are signed by a trusted party (usually an authentication server), while the purpose of the Web eID authentication token is to prove that the user is able to create signatures with the private key that corresponds to the presented certificate. The claims (person name, ID code, possibly e-mail address) can be extracted from the certificate only after it has been validated.

[lhazlewood]

So, our desire is to leverage JJWT's robust cryptographic components for specialized validation of Web eID authentication tokens, which require the security and reliability of JJWT, but not within the standard JWT framework.

Now, the following APIs are already part of the public JJWT api module:

SignatureAlgorithm signatureAlgorithm = (SignatureAlgorithm) Jwts.SIG.get().forKey(algorithm)
VerifySecureDigestRequest<PublicKey> verificationRequest
signatureAlgorithm.verify(verificationRequest)

So, the only point of contention is the availability of the DefaultVerifySecureDigestRequest<PublicKey> class, which would perfectly meet our needs, but is currently only accessible through the impl module. This issue is similar to concerns raised by another user, @icecreamhead, about the DefaultJws class being practical yet classified as internal, as discussed here.

I see now, thank you for clarifying!

So the SignatureAlgorithm interface (and other *Algorithm public interfaces) was/were always intended to allow custom implementations to be plugged in to JwtBuilder and JwtParserBuilder usages. The methods within them have to be public so that the builders can invoke them - they were not intended to be used outside of this context (which is exactly why there is no convenient way in the JJWT api module to create Request objects since only the JJWT Builder implementations 'need' to do that).

That said, this newer 'SPI' concept I was brainstorming above could fulfill your needs:

But if there's enough demand for 'provider support' APIs, we could add that to the JJWT public API or perhaps as a secondary spi ("service provider interface(s)") module that only algorithm implementors would want to depend on.

This could basically provide some abstract classes and request builders for those implementing and testing *Algorithm implementations, but you would be able to leverage that to invoke the algorithms directly (as would be done during testing of a custom algorithm). For example:

var request = Jwts.SIG.request(publicKey).data(concatSignedFields).digest(decodedSignature).build();
if (!signatureAlgorithm.verify(request)) {
    throw new AuthTokenSignatureValidationException();
}

I agree this would be nice to have for your use case, as well as testing custom implementations, and also something the Builder implementations could leverage themselves (instead of instantiating DefaultVerifySecureDigestRequest directly as they do today), so we can evaluate adding it to a future release.

Other than Request builders, is there anything else that you felt you needed?

[mrts]

Thank you very much for considering my proposal!

The API sketch you shared aligns closely with what we need:

var request = Jwts.SIG.request(publicKey).data(concatSignedFields).digest(decodedSignature).build();
if (!signatureAlgorithm.verify(request)) {
    throw new AuthTokenSignatureValidationException();
}

We might want to validate that the algorithm string from the Web eID authentication token matches the public key algorithm, but this is a minor detail.
It would be wonderful to have this capability, cheers!

Other than Request builders, is there anything else that you felt you needed?

Regarding additional needs, not at this time. However, considering your suggestion about the possibility for developers to add their own SignatureAlgorithm implementations, we might need support for non-standard algorithms in the future, such as those used by the German electronic health card which employs the Brainpool ECC algorithm, not currently covered by JWA RFC 7518.

Additionally, we are keeping an eye on developments like this post-quantum web authentication proof-of-concept solution that uses Web eID. Although this particular project does not use Java, it points towards new horizons of digital authentication where JJWT could play a significant role.

[lhazlewood]

Thanks for the feedback @mrts, I'll use this discussion to spawn a new issue to track work associated with this :)

[lhazlewood]

For those that may not have seen it yet, JDK 21 enabled a new Key Encapsulation Mechanism via the JDK 21+ KEM class to support post-quantum algorithms (like Kyber). JEP 452 also included an implementation of RFC 9180.

Without having tried it, I think the KEM API could be used easily enough within a new JJWT KeyAlgorithm implementation, and possibly without any changes or additions to the JJWT API🤞.

[lhazlewood]

I just created #940 to track associated work based on this discussion, so I'm marking it as closed (please use the new ticket for further associated discussion and/or requests). If you'd like to re-open it however, please let me know!

[mrts]

Excellent, thank you very much!