Michel
Michel

Reputation: 9440

Export all users from KeyCloak

I have a specific use case in which we want to ask Keycloak for all the users and the groups and roles for each user, on a daily basis. For reconciliation purposes with other internal systems.

Currently we are using the provided Keycloak endpoints in the UsersResource for this. But we see that performance slows down after each call to a point we can't use this solution anymore. There are more then 30K users in the realm.

We've also seen that Keycloak can export the database, but only on system boot (I guess for migration purposes). Given that we want to extract all the users on a daily basis we cannot use this.

Are there some known functionalities or workarounds?

Upvotes: 24

Views: 59944

Answers (4)

Kornext
Kornext

Reputation: 315

you can add to your realm.json export file manually

//import-file.json
{
    "realm": "<realm-name>",
    "users": [
      {
        "username": "admin",
        "enabled": true,
        "emailVerified": true,
        "firstName": "Rootus",
        "lastName": "Adminus",
        "email": "local-admin@mail.com",
        "credentials": [
          {
            "type": "password",
            "value": "admin"
          }
        ],
        "realmRoles": [
          "ADMIN"
        ]
      },
      {
        "username": "operator",
        "enabled": true,
        "emailVerified": true,
        "firstName": "Operatus",
        "lastName": "Operando",
        "email": "local-operator@mail.com",
        "credentials": [
          {
            "type": "password",
            "value": "operator"
          }
        ],
        "realmRoles": [
          "ROLE_OPERATOR"
        ]
      }
    ]
}

Upvotes: 6

Rosa Persiani
Rosa Persiani

Reputation: 21

We used this query to export user related data from the db:

SELECT U.EMAIL, U.FIRST_NAME, U.LAST_NAME, U.USERNAME, R.NAME, R.DESCRIPTION
FROM (USER_ENTITY U
join USER_ROLE_MAPPING M 
ON U.ID = M.USER_ID) join KEYCLOAK_ROLE R
ON M.ROLE_ID= R.ID;

the output looks like this:

| EMAIL | FIRST_NAME | LAST_NAME | USERNAME | NAME | DESCRIPTION | | rosa@example.com | Rosa | Persiani | rsper | view-profile | ${role_view-profile} |

the output can be easily converted to cvs or excel

Upvotes: 2

you need in your docker-compose-yml to bind your folder, not just the realm json file, like this:

keycloak:
    image: jboss/keycloak:8.0.1
    container_name: "keycloak"
     volumes:
      - ./realms/:/tmp/
    environment:
      - KEYCLOAK_USER=admin
      - KEYCLOAK_PASSWORD=admin
      - KEYCLOAK_IMPORT=/tmp/realm-export.json -Dkeycloak.profile.feature.upload_scripts=enabled 

where realms is your folder beside the yaml file. At this point you can run docker-compose up -d with your basic realm-export.json as always, go in your admin panel, adding users with credentials and roles, and then with this command you will able to export the entire configuration:

docker exec -it keycloak /opt/jboss/keycloak/bin/standalone.sh -Djboss.socket.binding.port-offset=100 -Dkeycloak.migration.action=export -Dkeycloak.migration.provider=singleFile -Dkeycloak.migration.realmName=ed-realm -Dkeycloak.migration.usersExportStrategy=REALM_FILE -Dkeycloak.migration.file=/tmp/export.json

You will see in your realms folder that a new file will be created, and it will contain the entire configuration, so you can run docker-compose down, replace your old file with this new and run again docker-compose up as many time you want, and redoing the process when you will change your realm again.

Upvotes: 17

Gernot Grames
Gernot Grames

Reputation: 173

I have done it with an parallel starting container via docker, which connects to the existing keycloak db.

Please use the same Version of the container keycloak as the real keacloak has. Because of db schema differences between versions.

EXPORT

docker run --rm\
    --name keycloak_exporter\
    -v /tmp:/tmp/keycloak-export:Z\
    -e POSTGRES_DATABASE=keycloak\
    -e POSTGRES_PASSWORD=PASSOWRD_PLEASE\
    -e POSTGRES_USER=keycloak\
    -e DB_VENDOR=POSTGRES\
    -e POSTGRES_PORT_5432_TCP_ADDR=postgresql.local\
    jboss/keycloak:3.4.3.Final\
    -Dkeycloak.migration.action=export\
    -Dkeycloak.migration.provider=dir\
    -Dkeycloak.migration.dir=/tmp/keycloak-export\
    -Dkeycloak.migration.usersExportStrategy=SAME_FILE\
    -Dkeycloak.migration.realmName=therealm

IMPORT

docker run --rm\
    --name keycloak_importer\
    -v /tmp:/tmp/keycloak-import:Z\
    -e POSTGRES_DATABASE=keycloak_dest\
    -e POSTGRES_PASSWORD=PASSOWRD_DEST_PLEASE\
    -e POSTGRES_USER=keycloak\
    -e DB_VENDOR=POSTGRES\
    -e POSTGRES_PORT_5432_TCP_ADDR=postgresql2.local\
    jboss/keycloak:3.4.3.Final\
    -Dkeycloak.migration.action=import\
    -Dkeycloak.migration.provider=dir\
    -Dkeycloak.migration.dir=/tmp/keycloak-import\
    -Dkeycloak.migration.strategy=IGNORE_EXISTING\
    -Dkeycloak.migration.usersExportStrategy=SAME_FILE\
    -Dkeycloak.migration.realmName=therealm

Possible config options: https://github.com/keycloak/keycloak-documentation/blob/master/server_admin/topics/export-import.adoc

Upvotes: 15

Related Questions