Storing and Retrieving Secret Files in AWS Secrets Manager

AWS Secrets Manager is a powerful tool for securely storing sensitive data, such as usernames and passwords. It supports key/value pairs and allows retrieval of secrets using the AWS Management Console, AWS CLI, or AWS API.

Secrets Manager can handle various sensitive information like database credentials, API keys, and OAuth tokens. It also offers features for rotating, monitoring, and managing access to these secrets. By using Secrets Manager API calls instead of hardcoded secrets, you can avoid exposing plaintext secrets in your applications.

In this guide, we will walk through the process of storing the content of a secret file (.env) in AWS Secrets Manager.

Creating a Secret File in AWS Secrets Manager Using the AWS Console

• Log in to your AWS account and navigate to Secrets Manager and click "Store a new secret".

• Choose "Other type of secret" since we are storing a custom secret.

• Enter Key/Value Pairs:
  • Key: Name it env (or any other relevant name).
  • Value: Convert your .env file content to base64 to avoid line issues.
• Content of .env file:

username=testusername
password=mypassword

• Convert the .env file to base64:

cat .env | base64

• Copy the value and paste it into the AWS Secret Manager

• Select the Encryption Key: If not default, select your encryption key and click "Next".

• Name Your Secret: Use a path-like format, e.g., test/webapp/env, to organize secrets logically.

• Review and Store. Ensure automatic rotation is off, review all settings, and click "Store".

Creating a Secret File in AWS Secrets Manager Using the AWS CLI

If you prefer the AWS CLI, follow these steps:

Step 1 - Open AWS CloudShell: Click on the Terminal icon in the AWS Console to access AWS CLI.

Step 2 - Prepare JSON Input: AWS Secrets Manager CLI expects a JSON string for key/value pairs.

Create env.json from your .env file:

echo "{\"env\": \"$(cat .env | base64)\"}" > env.json

Content of env.json:

{"env": "dXNlcm5hbWU9dGVzdHVzZXJuYW1lCnBhc3N3b3JkPW15cGFzc3dvcmQK"}

Step 3 - Create the Secret:

aws secretsmanager create-secret --name secret-create-from-file --secret-string file://env.json

The secret is created successfully.

Retrieving Secrets Using AWS Console

Go to Secrets Manager > Secret Value > Retrieve secret value

The base64 secret value will be displayed.

Decode the Value

echo "dXNlcm5hbWU9dGVzdHVzZXJuYW1lCnBhc3N3b3JkPW15cGFzc3dvcmQK" | base64 -d

Retrieving Secrets Using AWS CLI

Get Secret Value:

aws secretsmanager get-secret-value --secret-id secret-create-from-file --region=us-east-1

This returns the secret in JSON format:

{
    "ARN": "arn:aws:secretsmanager:us-east-1:xxxxxx:secret:secret-create-from-file-aTHAfZ",
    "Name": "secret-create-from-file",
    "VersionId": "bc77af0-1231-4e59-be1f-43f32d3d2f03",
    "SecretString": "{\"env\": \"dXNlcm5hbWU9dGVzdHVzZXJuYW1lCnBhc3N3b3JkPW15cGFzc3dvcmQK\"}\n",
    "VersionStages": [
        "AWSCURRENT"
    ],
    "CreatedDate": "2024-07-11T12:17:21.643000+00:00"
}

Extract and Decode the Secret:

Using jq and base64, we can filter and decode the secret value directly:

aws secretsmanager get-secret-value --secret-id secret-create-from-file --region=us-east-1 | jq -r .SecretString | jq -r .env | base64 -d

Output

Conclusion

Storing and managing secrets securely is critical for any application. AWS Secrets Manager offers a robust solution for storing the base64 encoded content of your files. This guide covered how to create and retrieve secrets using both the AWS Management Console and AWS CLI. By following these steps, you can ensure your sensitive information is stored securely and accessed safely.