AWS Secrets Manager Tutorial: Stop Hardcoding the $4.2M Security Mistake | AWSight
AWSight
AWS Security Insights

AWS Secrets Manager Tutorial: Stop Hardcoding the $4.2M Security Mistake

Learn how to implement secure credential management before your startup loses funding like the 39 million companies that leaked secrets in 2024

📅 Published: January 2025
⏱️ Read time: 12 minutes
🔧 Implementation: 20 minutes
🎯 Difficulty: Intermediate

🚨 The GitHub Leak That Killed a Series B Round

In 2024, a promising fintech startup was days away from closing their $25M Series B when their lead investor discovered something devastating during due diligence: database credentials, API keys, and OAuth tokens hardcoded across 47 GitHub repositories.

$4.2M

Lost in valuation adjustments, legal fees, security audits, and delayed funding. The round eventually closed 8 months later at a 60% lower valuation.

The killer detail? Among the 39 million secrets leaked on GitHub in 2024, their production database credentials had been publicly accessible for 18 months.

39M
secrets leaked on GitHub in 2024
91.6%
of leaked credentials remain valid after 5 days
85%
of leaks occur in developers' personal repos
50%
of cyberattacks use compromised credentials

🎯 Want Our Complete AWS Security Checklist?

Don't just secure your credentials—get our comprehensive 20-point security checklist that covers all critical AWS configurations. Used by 500+ companies to prevent security incidents and pass investor due diligence.

📥 Download Free Security Checklist 🔍 Get Free 72-Hour Security Assessment

🎯 Why Hardcoded Credentials Are a $4.2M Mistake

Hardcoded credentials—API keys, database passwords, and access tokens embedded directly in source code—represent one of the most dangerous yet common security vulnerabilities in modern applications. When investors conduct due diligence, discovering hardcoded credentials is often a deal-breaker.

The Three Deadly Sins of Credential Management

1
Source Code Exposure

Every time you commit hardcoded credentials to version control, you create a permanent security vulnerability. Even if you later remove the credentials, they remain in Git history forever. Version control systems like GitHub, GitLab, and Bitbucket are actively scanned by automated tools seeking exposed credentials.

# NEVER DO THIS - Hardcoded credentials const config = { database_url: "postgresql://admin:SuperSecret123@prod-db.amazonaws.com/main", api_key: "sk-1234567890abcdef", jwt_secret: "my-super-secret-key" };
2
Rotation Impossibility

Hardcoded credentials make rotation nearly impossible. When you need to change a password or regenerate an API key, you must update every instance across your entire codebase, redeploy applications, and coordinate timing across multiple services. This complexity leads most teams to avoid rotation entirely.

3
Privilege Escalation Risk

Hardcoded credentials often grant broader access than necessary because developers use convenient, high-privilege accounts. A single compromised credential can provide attackers with administrative access to databases, cloud services, and critical infrastructure.

⚠️ Critical Insight: According to GitHub's 2024 security report, 91.6% of leaked credentials remain valid after 5 days, and 50% of all cyberattacks now use compromised credentials as their primary attack vector.

🔍 AWS Secrets Manager vs. Parameter Store vs. Hardcoding

AWS provides multiple options for managing secrets and configuration data. Understanding when to use each service is crucial for both security and cost optimization.

Feature Hardcoded Parameter Store Secrets Manager
Security ❌ Exposed in code ✅ Encrypted at rest ✅ Encrypted + KMS
Automatic Rotation ❌ Manual only ❌ Not supported ✅ Built-in rotation
Cost ✅ Free ✅ $0.05/10K requests ❌ $0.40/secret/month
Version Control ❌ Git history risk ✅ Versioned ✅ Versioned + rollback
Cross-Region ❌ Manual sync ❌ Regional only ✅ Automatic replication
Audit Logging ❌ No visibility ✅ CloudTrail ✅ CloudTrail + detailed
Best For Never Config data, non-sensitive DB credentials, API keys
✅ Recommendation: Use Secrets Manager for database credentials, API keys, and OAuth tokens. Use Parameter Store for application configuration and non-sensitive data. Never hardcode credentials.
1
Create Your First Secret in AWS Secrets Manager (5 minutes)

Prerequisites:

  • AWS CLI configured with appropriate IAM permissions
  • Access to AWS Management Console
  • Existing database or service credentials to migrate

Console Method:

1.1 Navigate to Secrets Manager

  • Open the AWS Management Console
  • Search for "Secrets Manager" in the services search bar
  • Click on "AWS Secrets Manager"

1.2 Create New Secret

  • Click "Store a new secret"
  • Select secret type based on your needs:
  • RDS database: For database credentials with automatic rotation
  • Other type: For API keys, OAuth tokens, or custom credentials
# For database credentials, choose your engine Database engine: PostgreSQL Username: app_user Password: [Generate secure password] Database: production_db Host: your-db-instance.region.rds.amazonaws.com Port: 5432

1.3 Configure Secret Details

  • Secret name: prod/database/postgresql
  • Description: "Production database credentials for main application"
  • KMS encryption key: Choose aws/secretsmanager (free) or custom key
  • Tags: Add environment, team, and cost allocation tags

CLI Method (Recommended for Automation):

# Create a database secret with JSON credentials aws secretsmanager create-secret \ --name "prod/database/postgresql" \ --description "Production PostgreSQL credentials" \ --secret-string '{ "username": "app_user", "password": "YourSecurePassword123!", "host": "prod-db.cluster-xyz.region.rds.amazonaws.com", "port": 5432, "dbname": "production" }' \ --tags '[ {"Key": "Environment", "Value": "production"}, {"Key": "Team", "Value": "backend"}, {"Key": "Application", "Value": "main-app"} ]' # Create an API key secret aws secretsmanager create-secret \ --name "prod/external-api/stripe" \ --description "Stripe API keys for payment processing" \ --secret-string '{ "publishable_key": "pk_live_abc123...", "secret_key": "sk_live_xyz789...", "webhook_secret": "whsec_def456..." }'
✅ Success! Your secret is now stored securely in AWS Secrets Manager with encryption at rest using AWS KMS.
2
Migrate from Hardcoded Credentials (8 minutes)

Now let's replace hardcoded credentials in your application with secure calls to Secrets Manager. This step involves updating your application code and deployment configuration.

Step 2.1: Install AWS SDK

# Node.js npm install @aws-sdk/client-secrets-manager # Python pip install boto3 # Go go get github.com/aws/aws-sdk-go-v2/service/secretsmanager # Java implementation 'software.amazon.awssdk:secretsmanager'

Step 2.2: Update Application Code

Before (Hardcoded - NEVER DO THIS):

// DANGEROUS: Hardcoded credentials const dbConfig = { host: 'prod-db.cluster-xyz.region.rds.amazonaws.com', user: 'app_user', password: 'MyHardcodedPassword123!', database: 'production' };

After (Secure with Secrets Manager):

// SECURE: Dynamic credential retrieval const { SecretsManagerClient, GetSecretValueCommand } = require('@aws-sdk/client-secrets-manager'); async function getDbCredentials() { const client = new SecretsManagerClient({ region: 'us-west-2' }); try { const response = await client.send( new GetSecretValueCommand({ SecretId: 'prod/database/postgresql' }) ); return JSON.parse(response.SecretString); } catch (error) { console.error('Failed to retrieve credentials:', error); throw error; } } // Usage in your application async function initializeDatabase() { const credentials = await getDbCredentials(); const pool = new Pool({ host: credentials.host, user: credentials.username, password: credentials.password, database: credentials.dbname, port: credentials.port }); return pool; }

Step 2.3: Python Example

# SECURE: Python implementation import boto3 import json from botocore.exceptions import ClientError def get_secret(secret_name, region_name): # Create a Secrets Manager client session = boto3.session.Session() client = session.client( service_name='secretsmanager', region_name=region_name ) try: get_secret_value_response = client.get_secret_value( SecretId=secret_name ) except ClientError as e: raise e secret = get_secret_value_response['SecretString'] return json.loads(secret) # Usage credentials = get_secret('prod/database/postgresql', 'us-west-2') connection_string = f"postgresql://{credentials['username']}:{credentials['password']}@{credentials['host']}:{credentials['port']}/{credentials['dbname']}"

Step 2.4: Environment Variables (Alternative)

For applications that can't directly integrate with AWS SDK, use the AWS CLI to populate environment variables:

#!/bin/bash # Deployment script that retrieves secrets # Retrieve secret and extract values SECRET_JSON=$(aws secretsmanager get-secret-value \ --secret-id "prod/database/postgresql" \ --query 'SecretString' \ --output text) # Parse JSON and export as environment variables export DB_HOST=$(echo $SECRET_JSON | jq -r '.host') export DB_USER=$(echo $SECRET_JSON | jq -r '.username') export DB_PASS=$(echo $SECRET_JSON | jq -r '.password') export DB_NAME=$(echo $SECRET_JSON | jq -r '.dbname') # Start your application node app.js
⚠️ Security Note: Always use IAM roles instead of access keys when possible. For EC2, Lambda, or ECS, assign appropriate IAM roles rather than embedding AWS credentials.
AWS Secrets Manager Tutorial: Stop Hardcoding the $4.2M Security Mistake | AWSight
3
Set Up Automatic Rotation (4 minutes)

Automatic rotation is one of Secrets Manager's most powerful features. It regularly updates your credentials without requiring application downtime or manual intervention.

Step 3.1: Enable Rotation for RDS Database

# Enable automatic rotation for RDS database secret aws secretsmanager rotate-secret \ --secret-id "prod/database/postgresql" \ --rotation-rules '{"AutomaticallyAfterDays": 30}' # Set up rotation with custom Lambda function aws secretsmanager update-secret \ --secret-id "prod/database/postgresql" \ --description "Auto-rotated PostgreSQL credentials" # Configure rotation interval aws secretsmanager put-rotation-configuration \ --secret-id "prod/database/postgresql" \ --rotation-lambda-arn "arn:aws:lambda:region:account:function:SecretsManagerRotation" \ --rotation-rules '{"AutomaticallyAfterDays": 30}'

Step 3.2: Console Configuration

  • Open your secret in the Secrets Manager console
  • Click "Edit rotation"
  • Enable "Automatic rotation"
  • Set rotation interval (recommended: 30-90 days)
  • Choose rotation function:
    • Single user: One set of credentials, brief downtime during rotation
    • Alternating users: Two sets of credentials, zero downtime

Step 3.3: Custom Rotation for API Keys

For non-database secrets like API keys, create a custom Lambda function:

# Lambda function for custom API key rotation import json import boto3 import requests def lambda_handler(event, context): # Extract secret ARN and step from event secret_arn = event['SecretId'] step = event['Step'] secrets_client = boto3.client('secretsmanager') if step == 'createSecret': # Generate new API key from service new_api_key = generate_new_api_key() # Store pending version of secret secrets_client.update_secret_version_stage( SecretId=secret_arn, SecretString=json.dumps({'api_key': new_api_key}), VersionStage='AWSPENDING' ) elif step == 'setSecret': # Activate new API key with service activate_api_key(secret_arn) elif step == 'testSecret': # Test new API key functionality test_api_key(secret_arn) elif step == 'finishSecret': # Move AWSPENDING to AWSCURRENT finish_rotation(secret_arn) def generate_new_api_key(): # Call your service's API to generate new key response = requests.post('https://api.yourservice.com/keys', headers={'Authorization': 'Bearer admin_token'}) return response.json()['api_key']
✅ Rotation Active: Your credentials will now automatically rotate according to your schedule, significantly reducing the window of exposure for compromised credentials.
4
Implement Production-Ready Application Integration (3 minutes)

For production applications, implement caching, error handling, and retry logic to ensure reliable access to secrets while minimizing AWS API calls.

Step 4.1: Implement Secret Caching

// Production-ready secrets manager client with caching class SecretsCache { constructor(region, ttlMinutes = 5) { this.client = new SecretsManagerClient({ region }); this.cache = new Map(); this.ttl = ttlMinutes * 60 * 1000; // Convert to milliseconds } async getSecret(secretId, forceRefresh = false) { const cacheKey = secretId; const cached = this.cache.get(cacheKey); // Return cached value if valid and not forcing refresh if (!forceRefresh && cached && (Date.now() - cached.timestamp) < this.ttl) { return cached.value; } // Fetch fresh value from AWS try { const response = await this.client.send( new GetSecretValueCommand({ SecretId: secretId }) ); const secret = JSON.parse(response.SecretString); // Cache the result this.cache.set(cacheKey, { value: secret, timestamp: Date.now() }); return secret; } catch (error) { // Return stale cache if AWS call fails if (cached) { console.warn('Using stale cache due to AWS error:', error.message); return cached.value; } throw error; } } // Force refresh all secrets (call during rotation events) async refreshAll() { const promises = Array.from(this.cache.keys()).map(secretId => this.getSecret(secretId, true) ); await Promise.all(promises); } } // Global instance with singleton pattern const secretsCache = new SecretsCache('us-west-2', 5); module.exports = secretsCache;

Step 4.2: IAM Permissions Setup

{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "secretsmanager:GetSecretValue" ], "Resource": [ "arn:aws:secretsmanager:us-west-2:*:secret:prod/database/*", "arn:aws:secretsmanager:us-west-2:*:secret:prod/external-api/*" ] }, { "Effect": "Allow", "Action": [ "kms:Decrypt" ], "Resource": "arn:aws:kms:us-west-2:*:key/*", "Condition": { "StringEquals": { "kms:ViaService": "secretsmanager.us-west-2.amazonaws.com" } } } ] }

Step 4.3: Docker Integration

# Dockerfile with secrets manager integration FROM node:18-alpine # Install AWS CLI for secret retrieval RUN apk add --no-cache aws-cli jq # Copy application code COPY . /app WORKDIR /app RUN npm install # Startup script that retrieves secrets COPY start.sh /start.sh RUN chmod +x /start.sh CMD ["/start.sh"]
#!/bin/bash # start.sh - Startup script for containerized applications set -e # Wait for AWS metadata service (for ECS/EC2) until aws sts get-caller-identity > /dev/null 2>&1; do echo "Waiting for AWS credentials..." sleep 2 done # Retrieve and export database credentials echo "Retrieving database credentials..." DB_SECRET=$(aws secretsmanager get-secret-value \ --secret-id "prod/database/postgresql" \ --query 'SecretString' \ --output text) export DB_HOST=$(echo "$DB_SECRET" | jq -r '.host') export DB_USER=$(echo "$DB_SECRET" | jq -r '.username') export DB_PASS=$(echo "$DB_SECRET" | jq -r '.password') export DB_NAME=$(echo "$DB_SECRET" | jq -r '.dbname') echo "Starting application..." exec node app.js
✅ Production Ready: Your application now securely retrieves credentials with caching, error handling, and proper IAM permissions.

🔍 Validation: Verify Your Secrets Manager Implementation

Complete these checks to ensure your secrets management is working correctly and securely:

  • Secret Retrieval Test: Verify your application can successfully connect to databases and external APIs using retrieved credentials.
  • Rotation Test: Manually trigger rotation and confirm your application handles the credential change without downtime.
  • IAM Permission Verification: Ensure your application uses minimal necessary permissions and cannot access unrelated secrets.
  • Caching Validation: Monitor AWS API calls to confirm caching is working and reducing request costs.
  • Error Handling Test: Simulate AWS service unavailability and verify graceful degradation with cached credentials.
  • Audit Log Review: Check CloudTrail logs to ensure all secret access is properly logged and monitored.

Comprehensive Validation Script

Run this script to validate your implementation:

#!/bin/bash # Secrets Manager Implementation Validation Script echo "Validating AWS Secrets Manager implementation..." # Test 1: Verify secret exists and is accessible echo "Testing secret accessibility..." SECRET_ARN=$(aws secretsmanager describe-secret \ --secret-id "prod/database/postgresql" \ --query 'ARN' \ --output text) if [ -n "$SECRET_ARN" ]; then echo "Secret accessible: $SECRET_ARN" else echo "Cannot access secret!" exit 1 fi # Test 2: Verify rotation configuration echo "Checking rotation configuration..." ROTATION_ENABLED=$(aws secretsmanager describe-secret \ --secret-id "prod/database/postgresql" \ --query 'RotationEnabled' \ --output text) if [ "$ROTATION_ENABLED" == "True" ]; then echo "Automatic rotation is enabled" else echo "Consider enabling automatic rotation" fi # Test 3: Check encryption configuration echo "Verifying encryption..." KMS_KEY=$(aws secretsmanager describe-secret \ --secret-id "prod/database/postgresql" \ --query 'KmsKeyId' \ --output text) if [ -n "$KMS_KEY" ]; then echo "Encrypted with KMS key: $KMS_KEY" else echo "No KMS encryption detected!" fi # Test 4: Validate secret format echo "Validating secret format..." SECRET_VALUE=$(aws secretsmanager get-secret-value \ --secret-id "prod/database/postgresql" \ --query 'SecretString' \ --output text) if echo "$SECRET_VALUE" | jq -e '.username and .password and .host' > /dev/null; then echo "Secret contains required database fields" else echo "Secret format is invalid or incomplete" fi # Test 5: Check for hardcoded credentials in codebase echo "Scanning for hardcoded credentials..." HARDCODED_FOUND=$(grep -r -i "password\|secret\|token" . \ --include="*.js" --include="*.py" --include="*.java" \ --exclude-dir=node_modules --exclude-dir=.git | wc -l) if [ "$HARDCODED_FOUND" -eq 0 ]; then echo "No obvious hardcoded credentials found" else echo "Found $HARDCODED_FOUND potential hardcoded credentials - review manually" fi echo "Validation complete!"

🛡️ Production-Ready Security Practices

Secret Naming Conventions

Implement consistent naming patterns for easy management and automated policies:

# Recommended naming pattern: environment/service/resource prod/database/postgresql prod/external-api/stripe staging/cache/redis dev/messaging/slack-webhook # IAM policies can target by pattern arn:aws:secretsmanager:*:*:secret:prod/* arn:aws:secretsmanager:*:*:secret:staging/*

Cross-Region Replication

Replicate critical secrets to multiple regions for disaster recovery:

# Enable cross-region replication aws secretsmanager replicate-secret-to-regions \ --secret-id "prod/database/postgresql" \ --add-replica-regions '[ { "Region": "us-east-1", "KmsKeyId": "arn:aws:kms:us-east-1:account:key/key-id" }, { "Region": "eu-west-1", "KmsKeyId": "arn:aws:kms:eu-west-1:account:key/key-id" } ]'