cdpy authentication

[tomateoooh]

Hi,

I am trying to use cdpy in Lambda and as such, I want to present the credentials as environment variables from AWS Secrets Manager. I've set the following env vars; CDP_ACCESS_KEY_ID and CDP_PRIVATE_KEY as strings containing the credentials, as they are presented in an API key from the CDP UI.

But it seems like cdpy is expecting CDP_PRIVATE_KEY to be a path to a file, not the actual credential string because I get the following error:

cdpy.common.CdpError: CdpError(NoCredentialsError('Unable to locate CDP credentials: Private key file <credentialString> does not exist.'))

Is there a way to pass the credential directly to cdpy from the environment variable? I don't want to persist the credential to a file as this increases the likelihood that it can be compromised.

Thanks,
Tom

[wmudge]

Yes, in fact, we do so all the time for the same reasons. Can you confirm that the underlying cdpcli application can connect to the CDP control plane from your Lambda function?

[tomateoooh]

Hi,

Thanks for getting back to me.

In this first case, I'm testing the Python script running locally on my machine before creating it as a Lambda. I've configured cdpcli to connect to our environment using the ~/.cdp/credentials file, which contains the same cdp_access_key_id and cdp_private_key as I'm going to use in both my Python script and my Lambda. Below is my terminal output for running sync users from the cdpcli:

Next is my terminal output when trying to use the same credentials as I will in Lambda and I did above, presented as environment variables:

Then for completeness, here is my python script:

Thanks,
Tom

[wmudge]

Hi Tom,

cdpy wraps/delegates to the underlying EnvProvider in the cdpcli package. This code tests the value of the environment variable CDP_PRIVATE_KEY by first checking if it is a file and then if the value is hashed. If not, it throws the error you are seeing, which is a bit misleading.

if not os.path.isfile(private_key):
    if Ed25519v1Auth.detect_private_key(private_key):
        private_key_value = private_key
    else:
        raise NoCredentialsError(
            err_msg='Private key file {} does not exist'.format(private_key))
else:
    private_key_value = open(private_key).read()

https://github.com/cloudera/cdpcli/blob/master/cdpcli/credentials.py#L187-L194

It appears that indeed, your script is picking up the environment variables, but it fails to validate the CDP_PRIVATE_KEY value for some reason.

Thanks,
Webster

[tomateoooh]

Hi Webster,

Apologies, I'm not going to have much time to look into this over the next couple of weeks, I'm pursuing this project for work and I'm going to be on annual leave from Wednesday onwards. I'll investigate when I return and let you know what I find.

Thanks for your help so far,
Tom

[tomateoooh]

Hi Webster,

That has clarified things a lot. I should be hashing the private key.

Thanks very much for your help,
Tom

[wmudge]

The API key should already be hashed when you get it from your user profile. Can you check to make sure the value is intact within the Lambda function?

[tomateoooh]

Hi Webster,

After a lot of experimentation, I managed to get it working as you described above. I also discovered what was going wrong. I had downloaded the credentials file from the CDP web GUI, then used the terminal cat command to print out the file contents, before copying them into Secrets Manager. I didn't realise that cat had appended a % character to the end of the private key string, so this was preventing it from hitting true on this if statement and meaning I got the reported error:

if Ed25519v1Auth.detect_private_key(private_key):
        private_key_value = private_key
    else:
        raise NoCredentialsError(
            err_msg='Private key file {} does not exist'.format(private_key))

Thanks for your support on this and sorry to take up your time with a trivial error,
Tom

[wmudge]

Hey @tomateoooh! Glad to hear that you were able to get it working. I am very happy to help!