Handle unregistered users in BearerTokenAuthMechanism and implement user registration mechanism

doc/release-notes/10959-bearer-token-user-registration.md

@@ -0,0 +1,5 @@
+The OIDC Bearer token API authentication feature (available through a feature flag) has been extended to allow the registration of new users in Dataverse when there is no user account associated with the bearer token. 
+
+Specifically, a new endpoint (users/register) has been implemented, to which the bearer token and new user account information are sent, allowing the identity provider user to be linked to a Dataverse account. 
+
+In this way, the user will be recognized in future requests using the bearer token in the BearerTokenAuthMechanism.

doc/sphinx-guides/source/api/auth.rst

@@ -81,6 +81,29 @@ To test if bearer tokens are working, you can try something like the following (
 
   curl -H "Authorization: Bearer $TOKEN" http://localhost:8080/api/users/:me
 
+It may happen that when you try to authenticate a user for the first time with a bearer token, it does not have an associated user account in Dataverse. In this case, it is necessary to register the user using the following endpoint:
+
+.. code-block:: bash
+
+  curl  -H "Authorization: Bearer $TOKEN" -X POST http://localhost:8080/api/users/register --data '{"termsAccepted":true}'
+
+It is essential to send a JSON that includes the property ``termsAccepted`` set to true, which indicates that you accept the terms of service of Dataverse. Otherwise, you will not be able to create an account.
+
+In this JSON, we can also include the fields ``position`` or ``affiliation``, in the same way as when we register a user through the Dataverse UI. These fields are optional, and if not provided, they will be persisted as empty in Dataverse.
+
+Beyond the ``api-bearer-auth`` feature flag, there is another flag called ``api-bearer-auth-json-claims`` that can be enabled to allow sending missing user claims in the registration JSON. This is useful when the identity provider does not supply the necessary claims. However, this flag will only be considered if the ``api-bearer-auth`` feature flag is enabled. If the latter is not enabled, the ``api-bearer-auth-json-claims`` flag will be ignored.
+
+With the ``api-bearer-auth`` feature flag enabled, you can include the following properties in the request JSON:
+
+- ``username``
+- ``firstName``
+- ``lastName``
+- ``emailAddress``
+
+Note that even if they are included in the JSON, if it is possible to retrieve the corresponding claims from the identity provider, these values will be ignored and the ones from the identity provider will be used instead.
+
+This functionality is included under a feature flag because using it may introduce user impersonation issues, for example if the identity provider does not provide an email field and the user submits an email address they do not own.
+
 Signed URLs
 -----------
 

doc/sphinx-guides/source/installation/config.rst

@@ -3343,6 +3343,12 @@ please find all known feature flags below. Any of these flags can be activated u
     * - api-session-auth
       - Enables API authentication via session cookie (JSESSIONID). **Caution: Enabling this feature flag exposes the installation to CSRF risks!** We expect this feature flag to be temporary (only used by frontend developers, see `#9063 <https://github.com/IQSS/dataverse/issues/9063>`_) and for the feature to be removed in the future.
       - ``Off``
+    * - api-bearer-auth
+      - Enables API authentication via Bearer Token.
+      - ``Off``
+    * - api-bearer-auth-provide-missing-claims
+      - Enables sending missing user claims in the request JSON provided during OIDC user registration, when these claims are not returned by the identity provider and are required for registration. This feature only works when the feature flag ``api-bearer-auth`` is also enabled. **Caution: Enabling this feature flag exposes the installation to potential user impersonation issues.**
+      - ``Off``
     * - avoid-expensive-solr-join
       - Changes the way Solr queries are constructed for public content (published Collections, Datasets and Files). It removes a very expensive Solr join on all such documents, improving overall performance, especially for large instances under heavy load. Before this feature flag is enabled, the corresponding indexing feature (see next feature flag) must be turned on and a full reindex performed (otherwise public objects are not going to be shown in search results). See :doc:`/admin/solr-search-index`. 
       - ``Off``

docker-compose-dev.yml

@@ -17,6 +17,7 @@ services:
       SKIP_DEPLOY: "${SKIP_DEPLOY}"
       DATAVERSE_JSF_REFRESH_PERIOD: "1"
       DATAVERSE_FEATURE_API_BEARER_AUTH: "1"
+      DATAVERSE_FEATURE_API_BEARER_AUTH_PROVIDE_MISSING_CLAIMS: "1"
       DATAVERSE_MAIL_SYSTEM_EMAIL: "dataverse@localhost"
       DATAVERSE_MAIL_MTA_HOST: "smtp"
       DATAVERSE_AUTH_OIDC_ENABLED: "1"

src/main/java/edu/harvard/iq/dataverse/api/AbstractApiBean.java

@@ -14,14 +14,11 @@
 import edu.harvard.iq.dataverse.dataset.DatasetTypeServiceBean;
 import edu.harvard.iq.dataverse.engine.command.Command;
 import edu.harvard.iq.dataverse.engine.command.DataverseRequest;
-import edu.harvard.iq.dataverse.engine.command.exception.CommandException;
-import edu.harvard.iq.dataverse.engine.command.exception.IllegalCommandException;
-import edu.harvard.iq.dataverse.engine.command.exception.PermissionException;
+import edu.harvard.iq.dataverse.engine.command.exception.*;
 import edu.harvard.iq.dataverse.engine.command.impl.GetDraftDatasetVersionCommand;
 import edu.harvard.iq.dataverse.engine.command.impl.GetLatestAccessibleDatasetVersionCommand;
 import edu.harvard.iq.dataverse.engine.command.impl.GetLatestPublishedDatasetVersionCommand;
 import edu.harvard.iq.dataverse.engine.command.impl.GetSpecificPublishedDatasetVersionCommand;
-import edu.harvard.iq.dataverse.engine.command.exception.RateLimitCommandException;
 import edu.harvard.iq.dataverse.externaltools.ExternalToolServiceBean;
 import edu.harvard.iq.dataverse.license.LicenseServiceBean;
 import edu.harvard.iq.dataverse.pidproviders.PidUtil;
@@ -56,6 +53,7 @@
 import java.net.URI;
 import java.util.Arrays;
 import java.util.Collections;
+import java.util.Map;
 import java.util.UUID;
 import java.util.concurrent.Callable;
 import java.util.logging.Level;
@@ -631,10 +629,22 @@ protected <T> T execCommand( Command<T> cmd ) throws WrappedResponse {
              * sometimes?) doesn't have much information in it:
              *
              * "User @jsmith is not permitted to perform requested action."
+             *
+             * Update (11/11/2024):
+             *
+             * An {@code isDetailedMessageRequired} flag has been added to {@code PermissionException} to selectively return more
+             * specific error messages when the generic message (e.g. "User :guest is not permitted to perform requested action")
+             * lacks sufficient context. This approach aims to provide valuable permission-related details in cases where it
+             * could help users better understand their permission issues without exposing unnecessary internal information.
              */
-            throw new WrappedResponse(error(Response.Status.UNAUTHORIZED,
-                                                    "User " + cmd.getRequest().getUser().getIdentifier() + " is not permitted to perform requested action.") );
-
+            if (ex.isDetailedMessageRequired()) {
+                throw new WrappedResponse(error(Response.Status.UNAUTHORIZED, ex.getMessage()));
+            } else {
+                throw new WrappedResponse(error(Response.Status.UNAUTHORIZED,
+                        "User " + cmd.getRequest().getUser().getIdentifier() + " is not permitted to perform requested action."));
+            }
+        } catch (InvalidFieldsCommandException ex) {
+            throw new WrappedResponse(ex, badRequest(ex.getMessage(), ex.getFieldErrors()));
         } catch (CommandException ex) {
             Logger.getLogger(AbstractApiBean.class.getName()).log(Level.SEVERE, "Error while executing command " + cmd, ex);
             throw new WrappedResponse(ex, error(Status.INTERNAL_SERVER_ERROR, ex.getMessage()));
@@ -809,6 +819,18 @@ protected Response badRequest( String msg ) {
         return error( Status.BAD_REQUEST, msg );
     }
 
+    protected Response badRequest(String msg, Map<String, String> fieldErrors) {
+        return Response.status(Status.BAD_REQUEST)
+                .entity(NullSafeJsonBuilder.jsonObjectBuilder()
+                        .add("status", ApiConstants.STATUS_ERROR)
+                        .add("message", msg)
+                        .add("fieldErrors", Json.createObjectBuilder(fieldErrors).build())
+                        .build()
+                )
+                .type(MediaType.APPLICATION_JSON_TYPE)
+                .build();
+    }
+
     protected Response forbidden( String msg ) {
         return error( Status.FORBIDDEN, msg );
     }

src/main/java/edu/harvard/iq/dataverse/api/Users.java

@@ -8,29 +8,33 @@
 import edu.harvard.iq.dataverse.api.auth.AuthRequired;
 import edu.harvard.iq.dataverse.authorization.users.ApiToken;
 import edu.harvard.iq.dataverse.authorization.users.AuthenticatedUser;
+import edu.harvard.iq.dataverse.authorization.users.GuestUser;
 import edu.harvard.iq.dataverse.authorization.users.User;
-import edu.harvard.iq.dataverse.engine.command.impl.ChangeUserIdentifierCommand;
-import edu.harvard.iq.dataverse.engine.command.impl.GetUserTracesCommand;
-import edu.harvard.iq.dataverse.engine.command.impl.MergeInAccountCommand;
-import edu.harvard.iq.dataverse.engine.command.impl.RevokeAllRolesCommand;
+import edu.harvard.iq.dataverse.engine.command.impl.*;
+import edu.harvard.iq.dataverse.settings.FeatureFlags;
+import edu.harvard.iq.dataverse.util.BundleUtil;
 import edu.harvard.iq.dataverse.util.FileUtil;
 
+import static edu.harvard.iq.dataverse.api.auth.AuthUtil.extractBearerTokenFromHeaderParam;
 import static edu.harvard.iq.dataverse.util.json.JsonPrinter.json;
 
+import java.text.MessageFormat;
 import java.util.Arrays;
 import java.util.List;
+import java.util.Optional;
 import java.util.logging.Level;
 import java.util.logging.Logger;
+
+import edu.harvard.iq.dataverse.util.json.JsonParseException;
+import edu.harvard.iq.dataverse.util.json.JsonUtil;
 import jakarta.ejb.Stateless;
 import jakarta.json.JsonArray;
+import jakarta.json.JsonObject;
 import jakarta.json.JsonObjectBuilder;
+import jakarta.json.stream.JsonParsingException;
 import jakarta.ws.rs.*;
 import jakarta.ws.rs.container.ContainerRequestContext;
-import jakarta.ws.rs.core.Context;
-import jakarta.ws.rs.core.MediaType;
-import jakarta.ws.rs.core.Request;
-import jakarta.ws.rs.core.Response;
-import jakarta.ws.rs.core.Variant;
+import jakarta.ws.rs.core.*;
 
 /**
  *
@@ -266,4 +270,24 @@ public Response getTracesElement(@Context ContainerRequestContext crc, @Context
         }
     }
 
+    @POST
+    @Path("register")
+    public Response registerOIDCUser(String body) {
+        if (!FeatureFlags.API_BEARER_AUTH.enabled()) {
+            return error(Response.Status.INTERNAL_SERVER_ERROR, BundleUtil.getStringFromBundle("users.api.errors.bearerAuthFeatureFlagDisabled"));
+        }
+        Optional<String> bearerToken = extractBearerTokenFromHeaderParam(httpRequest.getHeader(HttpHeaders.AUTHORIZATION));
+        if (bearerToken.isEmpty()) {
+            return error(Response.Status.BAD_REQUEST, BundleUtil.getStringFromBundle("users.api.errors.bearerTokenRequired"));
+        }
+        try {
+            JsonObject userJson = JsonUtil.getJsonObject(body);
+            execCommand(new RegisterOIDCUserCommand(createDataverseRequest(GuestUser.get()), bearerToken.get(), jsonParser().parseUserDTO(userJson)));
+        } catch (JsonParseException | JsonParsingException e) {
+            return error(Response.Status.BAD_REQUEST, MessageFormat.format(BundleUtil.getStringFromBundle("users.api.errors.jsonParseToUserDTO"), e.getMessage()));
+        } catch (WrappedResponse e) {
+            return e.getResponse();
+        }
+        return ok(BundleUtil.getStringFromBundle("users.api.userRegistered"));
+    }
 }

src/main/java/edu/harvard/iq/dataverse/api/auth/ApiKeyAuthMechanism.java

@@ -9,6 +9,7 @@
 
 import jakarta.inject.Inject;
 import jakarta.ws.rs.container.ContainerRequestContext;
+
 import java.util.logging.Logger;
 
 /**
@@ -49,7 +50,7 @@ public User findUserFromRequest(ContainerRequestContext containerRequestContext)
             authUser = userSvc.updateLastApiUseTime(authUser);
             return authUser;
         }
-        throw new WrappedAuthErrorResponse(RESPONSE_MESSAGE_BAD_API_KEY);
+        throw new WrappedUnauthorizedAuthErrorResponse(RESPONSE_MESSAGE_BAD_API_KEY);
     }
 
     private String getRequestApiKey(ContainerRequestContext containerRequestContext) {
@@ -59,15 +60,15 @@ private String getRequestApiKey(ContainerRequestContext containerRequestContext)
         return headerParamApiKey != null ? headerParamApiKey : queryParamApiKey;
     }
 
-    private void checkAnonymizedAccessToRequestPath(String requestPath, PrivateUrlUser privateUrlUser) throws WrappedAuthErrorResponse {
+    private void checkAnonymizedAccessToRequestPath(String requestPath, PrivateUrlUser privateUrlUser) throws WrappedUnauthorizedAuthErrorResponse {
         if (!privateUrlUser.hasAnonymizedAccess()) {
             return;
         }
         // For privateUrlUsers restricted to anonymized access, all api calls are off-limits except for those used in the UI
         // to download the file or image thumbs
         if (!(requestPath.startsWith(ACCESS_DATAFILE_PATH_PREFIX) && !requestPath.substring(ACCESS_DATAFILE_PATH_PREFIX.length()).contains("/"))) {
             logger.info("Anonymized access request for " + requestPath);
-            throw new WrappedAuthErrorResponse(RESPONSE_MESSAGE_BAD_API_KEY);
+            throw new WrappedUnauthorizedAuthErrorResponse(RESPONSE_MESSAGE_BAD_API_KEY);
         }
     }
 }

src/main/java/edu/harvard/iq/dataverse/api/auth/AuthUtil.java

@@ -0,0 +1,24 @@
+package edu.harvard.iq.dataverse.api.auth;
+
+import java.util.Optional;
+
+public class AuthUtil {
+
+    private static final String BEARER_AUTH_SCHEME = "Bearer";
+
+    /**
+     * Extracts the Bearer token from the provided HTTP Authorization header value.
+     * <p>
+     * Validates that the header value starts with the "Bearer" scheme as defined in RFC 6750.
+     * If the header is null, empty, or does not start with "Bearer ", an empty {@link Optional} is returned.
+     *
+     * @param headerParamBearerToken the raw HTTP Authorization header value containing the Bearer token
+     * @return An {@link Optional} containing the raw Bearer token if present and valid; otherwise, an empty {@link Optional}
+     */
+    public static Optional<String> extractBearerTokenFromHeaderParam(String headerParamBearerToken) {
+        if (headerParamBearerToken != null && headerParamBearerToken.toLowerCase().startsWith(BEARER_AUTH_SCHEME.toLowerCase() + " ")) {
+            return Optional.of(headerParamBearerToken);
+        }
+        return Optional.empty();
+    }
+}