Skip to content

P4: Custos Deployment Custos Deployment Step 4

Jump to bottom
anbadrin edited this page May 6, 2022 · 6 revisions

Custos Deployment Step 4

Back to Home

Deploy Custos

  1. On the master node, assign labels to all the 3 VM's using the following command
kubectl label nodes node_name custosServiceWorker="enabled"

img

  1. Delete all the pods in the ingress-nginx namespace
kubectl delete all -n ingress-nginx

img

  1. Redeploy ingress-nginx services using the following command
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v0.44.0/deploy/static/provider/baremetal/deploy.yaml

img

  1. Verify if the services are deployed using the following command
kubectl get svc -n ingress-nginx

img

From the above output, get the https port number of the ingress-nginx controller. In our case it is 31701

  1. Check if the vault url works using this port number.

https://js-157-10.jetstream-cloud.org:31701/ui/vault/auth

img

  1. Also verify keycloak using the following url

https://js-157-10.jetstream-cloud.org:31701/auth

img

At this step, we created a new VM for the maven installation. This can be done from your local machine as well.

  1. On the new VM, generate a ssh key pair and add that public key to the master node's authorized_keys file in the .ssh folder

img

  1. We cloned the apache airavata custos repository in our local machine
git clone https://github.com/apache/airavata-custos.git
cd airavata-custos
git checkout develop
  1. Open the folder in your IDE and make the following changes

  • In the file custos-core-services/utility-services/custos-configuration-service/pom.xml, modify the following on line 83 to false

    <skip>true</skip>

  • Search for iam.server.url in all files in the following folder,

    custos-core-services/utility-services/custos-configuration-service/src/main/resources/

  • Replace all the iam.server.url parameters to iam.server.url=https://{host_name}:{port_number}/auth for the dev.properties and staging.properties files

  • There is a possibility that the values in below 2 files will not be changed. Change them manually.

    a. scimService-staging.properties

    b. scimService-dev.properties

  • Open custos-integration-services/tenant-management-service-parent/tenant-management-service/src/main/java/tasks/TenantActivationTask.java file and comment lines 225-249

RegisterClientResponse registerClientResponse = federatedAuthenticationClient
                    .addClient(clientMetadataBuilder.build());


            CredentialMetadata credentialMetadataCILogon = CredentialMetadata
                    .newBuilder()
                    .setId(registerClientResponse.getClientId())
                    .setSecret(registerClientResponse.getClientSecret())
                    .setOwnerId(tenant.getTenantId())
                    .setType(Type.CILOGON)
                    .build();

            credentialStoreServiceClient.putCredential(credentialMetadataCILogon);


            ConfigureFederateIDPRequest request = ConfigureFederateIDPRequest
                    .newBuilder()
                    .setTenantId(tenant.getTenantId())
                    .setClientID(registerClientResponse.getClientId())
                    .setClientSec(registerClientResponse.getClientSecret())
                    .setScope(tenant.getScope())
                    .setRequesterEmail(tenant.getRequesterEmail())
                    .setType(FederatedIDPs.CILOGON)
                    .build();
            iamAdminServiceClient.configureFederatedIDP(request);

  • We used scp to copy these files from our local machine to the maven vm. Before this step is performed, it is mandatory that either the ssh or password setup is done on the vm.

    #scp -r /path/to/copy/from/local username@ipaddress:~
scp -r /Users/anita/Desktop/airavata-custos/airavata-custos [email protected]:~

  • Give 777 permission to the folder on the vm using chmod

  1. Docker login on the vm using the following command

    sudo docker login

You will be prompted to enter the docker hub credentials. Enter the credentials and docker will be connected

  1. Build code using maven

    mvn clean install -P container

  2. Push code images to repo

    mvn dockerfile:push -P container

  3. Deploy the artifacts

    mvn antrun:run -P scp-to-remote

img

  1. On the master node, check if the pods in custos namespace are running successfully
kubectl get all -n custos

The custos-messaging-core-service might be in CrashLoopBackOff state. This is not causing any issue. We can proceed

img

  1. Run the following command
helm install cluster-management-core-service /home/ssh_user/custos/artifacts/cluster-management-core-service-1.1-SNAPSHOT.tgz -n keycloak

img

  1. Get the names of the services installed in the previous command using the following command
helm list -n custos --short

img

  1. Uninstall the iam-admin-core-service using the following command
helm uninstall iam-admin-core-service -n custos

img

  1. Uninstall the identity-core-service using the following command
helm uninstall identity-core-service -n custos

img

  1. Login to vault and click on enable new engine on the top right corner. The following UI will be displayed.

Select the generic type KV

img

  1. Give the path as secret and version as 1

img

  1. Click on enable engine to create the secret engine

img

  1. Similarly create the resourcesecret engine

  2. Registering tenant

On Postman, modify and import the following curl command

curl --location --request POST 'https://{host_name}:{port_name}/tenant-management/v1.0.0/oauth2/tenant' \
--header 'Content-Type: application/json' \
--data-raw '{
    "client_name":"{Give any name}",
    "requester_email":"{A valid email}",
    "admin_username":"{Set the admin username}",
    "admin_first_name":"{First Name}",
    "admin_last_name":"{Last Name}",
    "admin_email":"{A valid email}",
    "contacts":["[email protected]","[email protected]"],
    "redirect_uris":["http://localhost:8080/callback*",
    "https://{host_name}/callback*"],
    "scope":"openid profile email org.cilogon.userinfo",
    "domain":"https://{host_name}",
    "admin_password":"{Set a password}",
    "client_uri":"https://{host_name}",
    "logo_uri":"https://{host_name}",
    "application_type":"web",
    "comment":"Custos super tenant for production"
}'
  1. The output to this would be of the following form:
{
    "client_id": "{client_id}",
    "client_secret": "{client_secret}",
    "is_activated": false,
    "client_id_issued_at": 1651783581000,
    "client_secret_expires_at": 0,
    "registration_client_uri": "https://custos.scigap.org/apiserver/tenant-management/v1.0.0/oauth2/tenant?client_id={client_id}",
    "token_endpoint_auth_method": "client_secret_basic",
    "msg": "Use Base64 encoded clientId:clientSecret as auth token for authorization, Credentials are activated after admin approval"
}

  1. Open secret in vault, open id, open custos and change supertenant to true.

  2. Set supertenant to active by importing, modifying and running the following curl command on postman

curl --location --request POST 'https://{host_name}:{port_number}/tenant-management/v1.0.0/status' \
--header 'Content-Type: application/json' \
--data-raw '{
"client_id":"{client_id}",
"status":"ACTIVE",
"super_tenant":true,
"updatedBy":"{username}"
}'

The output would be of the form

{
    "tenant_id": "{tenant_id}",
    "status": "ACTIVE"
}

References

We referred the deployment steps of Team Scapsulators. Following are the 2 links for the same:-

We also got some guidance from Thank you Team Neo

Clone this wiki locally