If you've been developing web apps and/or working with cloud, you'd already have realized that managing secrets is a nightmare, and having a multi stage workflow with different secrets even more so. There are different types of secrets you'd prefer to handle and manage differently. AWS provides several powerful tools and services for secrets management, however the fundamentals are lost within their bulky documentations. This blog aims to provide a concise overview of secret management in AWS.

‍

In this article, we'll explore:

• Common types of secrets in modern applications
• Why do they need to be managed
• How AWS helps in managing them
• Other challenges in secret management

Let's dive in.

‍

Table of Contents

1. Top Challenges in AWS Secrets Management
2. Why Managing Secrets in AWS is Crucial for Security & Compliance
3. Different Types of Secrets in AWS
4. Enhancing AWS Secrets Management with Security, Rotation, and Compliance Best Practices
5. AWS Solutions for Secret Management
   
   1. AWS Parameter Store
   2. AWS Secrets Manager
   3. AWS Key Management Service (KMS)
6. Real-World Use Cases for AWS Secrets Management

‍

Top Challenges in AWS Secrets Management

1. Security Risks: Hardcoded secrets in code repositories might remain for a long period of time because of the VCS, even after removed later which can lead to severe security breaches if repositories are compromised
2. Scalability Issues: As applications grow, the number of secrets increases exponentially, Managing hundreds of secrets across environments becomes a growing pain and handling versions just adds more to the complexity.
3. Access Control: Ensuring proper permissions and audit trails
4. Rotation Complexity: Implementing secure key rotation policies ensures better security and compliance
5. Multi-Region Deployment: Synchronizing secrets across different AWS regions

‍

Why Managing Secrets in AWS is Crucial for Security & Compliance

Before diving into the solutions, let's understand why proper secret management is a must.

1. Reduces the risk of accidental exposure of secrets: Data breaches are expensive, compliance fines can also be massive and recovery costs and other fees add up quickly.
2. Professional Reputation: Nobody wants to be "that guy" who leaked creds, orgs loses trust when breaches happen and professional & reputation matters in the industry.
3. Efficiency: No need to reach out to multiple people to gather env vars to run one service out of many, rinse and repeat for other services. Also faster onboarding for new team members. And less time saved communicating back all the updated env to everyone who needs access.
4. Compliance Requirements: Many standards ( SOC 2, HIPAA, PIC) require proper secret management. It also helps make the audit process smoother and ensures regulatory compliance.
5. Risk Management: Managing secrets properly helps in quick responses for potential breach, and makes things easy to continue maintaining security best practises
6. Future Proofing: This builds the foundation for zero-trust security architecture and eases out integration with new services and applications

‍

Different Types of Secrets in AWS

There are different types of secrets you might come across which requires a different approach for managing them:

1. Database Credentials
   
   1. Connection strings
   2. Usernames and passwords

2. API Keys and Other secrets Vars
   
   1. Application secrets like SMTP user/pass, hash keys etc, env vars etc
   2. Third-party service APIs, Endpoints, tenant IDs etc
   3. Different Social auth tokens/client IDs

3. Encryption Keys
   
   1. Data encryption keys
   2. SSL/TLS certificates
   3. Cryptographic secrets

‍

Enhancing AWS Secrets Management with Security, Rotation, and Compliance Best Practices

Now, let's look into what are some best practices for managing secrets effectively.

• Never Hardcode Secrets, always use env vars for any type of secrets, ensure the secrets are always ignored in vsc/git.
• Whenever possible, Instead of creds, try to use different auth roles like OIDC to interact with 3rd party services.
• Use rotating secrets for most sensitive secrets like Database credentials and other infra related services credentials.
• Use Monitoring via Cloudtrail or similar tools, to monitor the types of secrets that are getting accessed.

‍

How AWS Secrets Manager, Parameter Store, and KMS Enhance Secrets Management

AWS provides a robust ecosystem for managing secrets securely, Here are some of the services that will help us manage secrets effectively.

‍

1. AWS Secrets Manager
   
   1. Automated secret rotation
   2. Fine-grained access control
   3. Integration with RDS, Redshift, and DocumentDB

2. AWS Systems Manager Parameter Store
   
   1. Hierarchical organization of parameters
   2. Secure string storage
   3. Version control and history tracking

3. AWS Key Management Service (KMS)
   
   1. Centralized key management
   2. Encryption key rotation
   3. Support for custom keys

Let's dive in how can we leverage these services to manage secrets and tackle the challenges effectively

‍

AWS Parameter Store

For all the environment variables, it's common practice to use a parameter store. It's dirt cheap, crosses all the boxes and is compliance ready. It provides different modes of securing credentials like String, SecureString, StringList.

• Use SecureString for all the 3rd party API creds, and/or sensitive secrets
• Use String for less sensitive secrets

The difference is that, when getting parameter, from SSM, it returns encrypted values if it's stored as secureString and can only be decrypted with proper permissions

A common pattern that is used to manage secrets for different stages and different app is as follow: /{stage}/{service}/{platform}/type For example, if we want to have stripe webhook secret stored, we can create a parameter like this: /prod/payment-service/stripe/whsecret

Now, let's see how we can get the added secret from parameter store:

import boto3

ssm = boto3.client('ssm')
response = ssm.get_parameter(Name='/prod/payment-service/stripe/whsecret:', WithDecryption=True)
print(response['Parameter']['Value'])
  

Here, the version is optional, however, when specified, it will retrieve that specific version of the secret. So this resolved all of our problems, for multi-stage, multi-versioned, multi-service platforms, this works great!

If we want to inject a secret in a running CI, it provides a CLI based implementation and we can use it like this:

aws ssm get-parameters --names /prod/payment-service/stripe/whsecret --with-decryption --query Parameters[].Value
  

We can set up the CI with an OIDC role to provide access to get-parameter operation and inject the necessary secrets into running CI.

As great as Parameter store really is, there are some limitations of this service:

• Max of 4kb secret can be stored
• Has lower rate limits than secrets manager
• Does not support rotation

‍

AWS Secrets Manager

Parameter store is great and all, however, when it comes to secrets rotation or larger secrets, it fails there. but, worry not, AWS provides another service Secrets Manager which is a bit pricey but helps tackle these challenges effectively i.e. secrets which has size > 4kb and/or need rotation ability

‍

We can retrieve a secret from secrets manager as follows:

import boto3

# Retrieve a secret from Secrets Manager
session = boto3.Session()
secrets_manager = session.client('secretsmanager')
secret = secrets_manager.get_secret_value(SecretId='my-database-secret')
  

This is very similar to a parameter store, but with increased rate limits and size. We can even enable rotation for secrets through a lambda function. The following code snippet shows how we can enable rotation for a secret:

def enable_rotation():
    client = boto3.client('secretsmanager')
    
    try:
        response = client.rotate_secret(
            SecretId='payment-service/prod/db-credentials',
            RotationLambdaARN='arn:aws:lambda:region:account:function:rotation-function',
            RotationRules={
                'AutomaticallyAfterDays': 30
            }
        )
        return response
    except Exception as e:
        print(f"Error enabling rotation: {str(e)}")
          

AWS Key Management Service

AWS KMS is a fully managed service which makes it easy to create and manage encryption keys for various use cases for our application. Unlike the above 2 services, KMS focuses specifically on cryptographic operations and key management.

‍

KMS integrates seamlessly with various AWS services:

• Amazon S3 server-side encryption
• Amazon RDS database encryption
• AWS Lambda encryption
• Amazon EBS volume encryption

Key Concepts in AWS KMS

1. Customer Master Keys (CMKs)
   
   1. AWS-managed keys
   2. Customer-managed keys
   3. AWS-owned keys

2. Key Rotation
   
   1. Automatic key rotation
   2. Manual key rotation
   3. Cryptographic key versioning

3. Encryption Contexts
   
   1. Additional authenticated data
   2. Provides an extra layer of security

Real-World Use Cases for AWS Secrets Management

1. Encrypting and Decrypting Data

import boto3

# Initialize KMS client
kms_client = boto3.client('kms')

def encrypt_data(key_id, plaintext_data):
    """Encrypt sensitive data using a KMS key"""
    try:
        response = kms_client.encrypt(
            KeyId=key_id,
            Plaintext=plaintext_data.encode(),
            EncryptionContext={
                'purpose': 'data-protection'
            }
        )
        return response['CiphertextBlob']
    except Exception as e:
        print(f"Encryption error: {e}")
        return None

def decrypt_data(ciphertext, key_id):
    """Decrypt data using the same KMS key and encryption context"""
    try:
        response = kms_client.decrypt(
            KeyId=key_id,
            CiphertextBlob=ciphertext,
            EncryptionContext={
                'purpose': 'data-protection'
            }
        )
        return response['Plaintext'].decode()
    except Exception as e:
        print(f"Decryption error: {e}")
        return None

# Example usage
key_id = 'arn:aws:kms:us-west-2:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab'
sensitive_data = "This is a top-secret message!"
encrypted_data = encrypt_data(key_id, sensitive_data)
decrypted_data = decrypt_data(encrypted_data, key_id)
  

2. Generating Data Keys for Client-Side Encryption

def generate_data_key(key_id, key_spec='AES_256'):
    """Generate a data key for client-side encryption"""
    try:
        response = kms_client.generate_data_key(
            KeyId=key_id,
            KeySpec=key_spec
        )
        return {
            'plaintext_key': response['Plaintext'],
            'encrypted_key': response['CiphertextBlob']
        }
    except Exception as e:
        print(f"Data key generation error: {e}")
        return None
          

Best Practices for AWS KMS

1. Use Customer-Managed Keys (CMKs) for more control over key policies and rotation
2. Enable automatic key rotation for enhanced security
3. Use encryption contexts to add an extra layer of authentication
4. Implement least privilege access using IAM policies
5. Monitor key usage through AWS CloudTrail

‍

Frequently Asked Questions

What is secret management, and why is it important in AWS?

Secrets management involves securely storing, accessing, and rotating sensitive credentials like API keys, database passwords, and encryption keys. In AWS, proper secrets management prevents unauthorized access, data breaches, and compliance violations while improving operational security.

What are the most common challenges in managing secrets?

Managing secrets comes with several challenges, including:

• Security risks: Hardcoded secrets in code repositories can be leaked.
• Scalability: Handling secrets across multiple applications and environments becomes complex.
• Access control: Ensuring only authorized services and users can access specific secrets.
• Rotation complexity: Securely rotating credentials without downtime.
• Multi-region synchronization: Keeping secrets consistent across AWS regions.

What types of secrets should be managed in AWS?

AWS applications require managing different types of secrets, such as:

• Database credentials: Connection strings, usernames, passwords.
• API keys: Access tokens for third-party services, social authentication tokens.
• Encryption keys: SSL/TLS certificates, AWS KMS keys for encrypting data.

What are AWS’s best practices for secrets management?

To manage secrets securely, follow these AWS best practices:

• Never hardcode secrets in source code—use AWS Secrets Manager or Parameter Store.
• Enable automatic secret rotation for database credentials and API keys.
• Use IAM roles instead of static credentials whenever possible.
• Encrypt secrets at rest and in transit using AWS KMS.
• Monitor secret access with AWS CloudTrail logs.



Want to learn more about securing AWS applications? Check out AppSecEngineer’s AWS Security Courses!