Skip to main content

Auto-Remediate GuardDuty Findings with Lambda

Build an AWS auto-remediation pipeline: GuardDuty detects threats, EventBridge routes them, and Lambda quarantines the instance and alerts Slack.

Blueprint Walkthrough

Architecture Overview & Problem Statement

This project builds an automated incident-response pipeline on AWS. GuardDuty detects a threat, EventBridge routes high-severity findings to a Lambda function, and Lambda quarantines the affected EC2 instance, deactivates a leaked IAM access key, and posts the evidence to Slack, with no human needing to be online. You build it with the AWS CLI and Python, and you add the guard rails (dry-run mode, an opt-out tag and a rollback runbook) that make unattended remediation safe.

What You Will Learn

  • How GuardDuty findings are structured (type, severity, resource) and how they reach EventBridge
  • How to write an EventBridge event pattern that matches a numeric severity
  • How to isolate an instance without destroying forensic evidence
  • How to write a remediation function that is safe to run unattended: dry-run mode, idempotency, an opt-out tag and least-privilege IAM
  • How to keep a secret (the Slack webhook) out of environment variables

What You Can Show

A GitHub repo with the Lambda code, a README of the CLI steps, a screenshot of a real Slack alert (webhook removed), your measured response time, and the rollback runbook.

Prerequisites, Time and Cost

  • An AWS account you can safely experiment in (not a shared production account)
  • AWS CLI v2 configured, basic Python, and a Slack workspace where you can create an incoming webhook
  • About 2-3 hours
  • GuardDuty bills on the data it analyses, and new accounts typically get a free-trial period. Lambda, EventBridge and an SSM standard parameter cost effectively nothing at this volume. Check the current GuardDuty pricing page and run the cleanup at the end.

Architecture Overview

A security alert with no automation is just noise. When GuardDuty flags an EC2 instance talking to a known command-and-control server at 3am, nobody is awake to respond. By the time someone at a company like Hotstar reads the alert in the morning, the instance may have been compromised for hours during peak streaming traffic.

This project automates the first response step. GuardDuty detects the threat. EventBridge routes the finding to a Lambda function. Lambda quarantines the instance by swapping its security groups for a deny-all group, tags it with what happened, and sends a Slack alert, with no human needing to be online.

flowchart LR A[EC2 or IAM
activity] --> B[GuardDuty
finding] B --> C[EventBridge
rule] C --> D[Lambda] classDef blue fill:#326CE5,stroke:#1E4FB8,color:#fff classDef green fill:#10B981,stroke:#0B7A56,color:#fff classDef orange fill:#F97316,stroke:#B85410,color:#fff classDef red fill:#EF4444,stroke:#B02D2D,color:#fff classDef purple fill:#8B5CF6,stroke:#6238C2,color:#fff classDef slate fill:#475569,stroke:#2B3544,color:#fff class A slate class B orange class C purple class D blue

When the rule matches, one Lambda function does the response. It first checks DRY_RUN and the opt-out tag, then takes up to three actions:

flowchart TD L[Lambda] --> Q[Quarantine SG
and tags] L --> K[Deactivate
IAM key] L --> S[Slack alert] classDef blue fill:#326CE5,stroke:#1E4FB8,color:#fff classDef green fill:#10B981,stroke:#0B7A56,color:#fff classDef orange fill:#F97316,stroke:#B85410,color:#fff classDef red fill:#EF4444,stroke:#B02D2D,color:#fff classDef purple fill:#8B5CF6,stroke:#6238C2,color:#fff classDef slate fill:#475569,stroke:#2B3544,color:#fff class L blue class Q red class K red class S green
Tip

Do not promise a response time you have not measured. In the Validation section you measure the part you control (EventBridge to Lambda to isolated instance) and write the number in your README. GuardDuty's own detection delay is separate and is not something your code can speed up.

Problem Solved

Manual incident response has a timing problem: the gap between "a threat occurs" and "a human notices and reacts" can be hours, especially overnight or at weekends. In that gap a compromised instance can exfiltrate data, mine cryptocurrency on your bill, or pivot to other resources, a real risk for any company running workloads around the clock, like Zerodha during market hours.

This project shrinks that gap by automating the first step. GuardDuty analyses CloudTrail activity, VPC Flow Logs and DNS logs using threat intelligence and anomaly detection, so you do not write any detection logic yourself. Your job is the response.

Isolation is not termination. The standard pattern is to cut a compromised instance off from the network while keeping its disk and memory for forensics, not to destroy the evidence by terminating it.

Security

Automated remediation limits blast radius, but it can also cause an outage when a finding is a false positive. That is why this project adds three guard rails that most tutorials skip: a DRY_RUN switch, an opt-out tag for instances that need a human decision, and a rollback runbook. Treat the automation as the first responder, not the investigator.


Milestone 1: Enable GuardDuty

Concept

Enabling GuardDuty is one API call. There is no agent to install. GuardDuty is regional: a detector in ap-south-1 only watches ap-south-1. This project works in a single region. In a real company you would enable it in every region you use, usually through AWS Organizations with a delegated administrator account.

Steps

Check whether a detector already exists, because an account can only have one per region:

Bash
export AWS_REGION=ap-south-1
aws guardduty list-detectors --region $AWS_REGION
JSON
{
"DetectorIds": []
}

If the list is empty, create one:

Bash
DETECTOR_ID=$(aws guardduty create-detector \
--enable \
--finding-publishing-frequency FIFTEEN_MINUTES \
--region $AWS_REGION \
--query 'DetectorId' \
--output text)
echo $DETECTOR_ID

If a detector already exists, reuse its ID instead:

Bash
DETECTOR_ID=$(aws guardduty list-detectors --region $AWS_REGION \
--query 'DetectorIds[0]' --output text)
Tip

FIFTEEN_MINUTES controls how often updates to an existing finding are re-published to EventBridge. New findings are published to EventBridge as they are generated, and GuardDuty detects continuously.

Common Mistake

Running create-detector when a detector already exists. The call fails, so always list detectors first and reuse the existing ID.


Milestone 2: Create the Deny-All Quarantine Security Group

Concept

Isolation relies on one security group with zero inbound and zero outbound rules. Swapping it onto an instance blocks new network traffic in both directions without touching the disk or memory.

A security group belongs to one VPC, and an instance can only use groups from its own VPC. The Lambda in this project therefore looks up the quarantine group by name inside the instance's VPC, instead of hard-coding one group ID. You create a group with the same name in each VPC you want to protect.

Steps

Bash
VPC_ID=$(aws ec2 describe-vpcs \
--filters Name=isDefault,Values=true \
--query 'Vpcs[0].VpcId' \
--output text \
--region $AWS_REGION)
QUARANTINE_SG=$(aws ec2 create-security-group \
--group-name "QUARANTINE-deny-all" \
--description "Quarantine - no inbound or outbound traffic" \
--vpc-id $VPC_ID \
--region $AWS_REGION \
--query 'GroupId' \
--output text)
echo $QUARANTINE_SG

A new group has no inbound rules, but it does have a default allow-all outbound rule. Remove it:

Bash
aws ec2 revoke-security-group-egress \
--group-id $QUARANTINE_SG \
--ip-permissions '[{"IpProtocol": "-1", "IpRanges": [{"CidrIp": "0.0.0.0/0"}]}]' \
--region $AWS_REGION

Confirm both rule lists are empty:

Bash
aws ec2 describe-security-groups \
--group-ids $QUARANTINE_SG \
--region $AWS_REGION \
--query 'SecurityGroups[0].{In:IpPermissions,Out:IpPermissionsEgress}'
JSON
{
"In": [],
"Out": []
}
Common Mistake

Forgetting to revoke the default outbound rule. The group would block inbound traffic but still allow outbound, so a compromised instance could keep exfiltrating data.

Security

Security groups track connections. Per AWS's documentation on security group connection tracking, connections that were already established before the change can keep working for a while, even though the new group has no rules. If you must cut every existing connection immediately, a network ACL change on the subnet or stopping the instance is stronger, but stopping loses memory contents. Decide this trade-off before you rely on this pattern in production. The Extensions section covers it.


Milestone 3: Store the Slack Webhook Securely

Concept

A Slack incoming-webhook URL is a secret: anyone who has it can post to your channel. Putting it in a Lambda environment variable leaves it visible in the console and in get-function-configuration. Instead you store it in SSM Parameter Store as a SecureString and let the function read it at runtime.

Steps

Create an incoming webhook in Slack (Slack app, then Incoming Webhooks, then add to a test channel). Then read it into a shell variable without it landing in your shell history:

Bash
read -rs SLACK_WEBHOOK
Bash
aws ssm put-parameter \
--name /security/slack-webhook-url \
--type SecureString \
--value "$SLACK_WEBHOOK" \
--region $AWS_REGION
unset SLACK_WEBHOOK
Common Mistake

Pasting the webhook URL into your README, a screenshot, or a Git commit. If it leaks, delete the webhook in Slack and create a new one.


Milestone 4: Write the Lambda Remediation Function

Concept

The function does three things per finding: contain the resource, tag what happened and why, and notify. It is written to be safe to run unattended:

  • DRY_RUN logs what it would do without changing anything. You start with it on.
  • Idempotent: a second finding for an already-quarantined instance is skipped, not re-processed.
  • Opt-out tag: an instance tagged SecurityAutoResponse=exempt is never auto-isolated. A human decides.
  • Narrow blast radius: for access-key findings it deactivates only the key named in the finding, not every key the user has.
  • Deactivate, never delete: deactivation is reversible and keeps the audit trail.

Findings about assumed-role or instance-profile credentials are not handled automatically here. Those need a different response (for example a deny policy on the role session), so the function reports "manual review" to Slack instead of guessing.

Steps

Bash
mkdir -p lambda
cat > lambda/remediation.py << 'EOF'
import json
import logging
import os
import urllib.request
from datetime import datetime, timezone
import boto3
from botocore.exceptions import ClientError
logger = logging.getLogger()
logger.setLevel(logging.INFO)
QUARANTINE_SG_NAME = os.environ.get('QUARANTINE_SG_NAME', 'QUARANTINE-deny-all')
SLACK_WEBHOOK_PARAM = os.environ['SLACK_WEBHOOK_PARAM']
DRY_RUN = os.environ.get('DRY_RUN', 'true').lower() == 'true'
SEVERITY_THRESHOLD = float(os.environ.get('SEVERITY_THRESHOLD', '7.0'))
EXEMPT_TAG_KEY = 'SecurityAutoResponse'
ssm = boto3.client('ssm')
_slack_url = None
def now_iso():
return datetime.now(timezone.utc).isoformat()
def get_slack_url():
"""Read the webhook once per container, not once per invocation."""
global _slack_url
if _slack_url is None:
resp = ssm.get_parameter(Name=SLACK_WEBHOOK_PARAM, WithDecryption=True)
_slack_url = resp['Parameter']['Value']
return _slack_url
def lambda_handler(event, context):
"""GuardDuty findings arrive from EventBridge as event['detail']."""
finding = event.get('detail', {})
finding_id = finding.get('id', 'unknown')
finding_type = finding.get('type', 'Unknown')
severity = float(finding.get('severity', 0))
account_id = finding.get('accountId', 'unknown')
region = finding.get('region', os.environ.get('AWS_REGION', 'unknown'))
# Log identifiers only - the full finding can contain sensitive details
logger.info("Finding %s type=%s severity=%s dry_run=%s",
finding_id, finding_type, severity, DRY_RUN)
if severity < SEVERITY_THRESHOLD:
logger.info("Severity below threshold - skipping")
return {'status': 'skipped', 'reason': 'below_severity_threshold'}
resource = finding.get('resource', {})
resource_type = resource.get('resourceType', 'Unknown')
actions = []
if resource_type == 'Instance':
instance_id = resource.get('instanceDetails', {}).get('instanceId')
if instance_id:
actions.extend(isolate_ec2_instance(instance_id, finding_id, finding_type, region))
else:
actions.append("No instance ID in finding - manual review needed")
elif resource_type == 'AccessKey':
details = resource.get('accessKeyDetails', {})
if details.get('userType') == 'IAMUser' and details.get('userName') and details.get('accessKeyId'):
actions.extend(deactivate_access_key(
details['userName'], details['accessKeyId'], finding_id, finding_type))
else:
actions.append(
f"No automatic action for principal type {details.get('userType')} - manual review needed")
else:
actions.append(f"No automatic action for resource type {resource_type} - manual review needed")
send_slack_alert(finding_type, severity, finding_id, account_id, region, resource_type, actions)
failed = any(a.startswith('ERROR') for a in actions)
return {'status': 'error' if failed else 'handled',
'finding_id': finding_id, 'dry_run': DRY_RUN, 'actions': actions}
def isolate_ec2_instance(instance_id, finding_id, finding_type, region):
ec2 = boto3.client('ec2', region_name=region)
try:
reservations = ec2.describe_instances(InstanceIds=[instance_id])['Reservations']
instance = reservations[0]['Instances'][0]
tags = {t['Key']: t['Value'] for t in instance.get('Tags', [])}
if tags.get(EXEMPT_TAG_KEY) == 'exempt':
return [f"SKIPPED {instance_id}: tagged {EXEMPT_TAG_KEY}=exempt - human decision required"]
if tags.get('SECURITY_STATUS') == 'QUARANTINED':
return [f"SKIPPED {instance_id}: already quarantined"]
# The quarantine group must live in the same VPC as the instance
vpc_id = instance['VpcId']
groups = ec2.describe_security_groups(Filters=[
{'Name': 'group-name', 'Values': [QUARANTINE_SG_NAME]},
{'Name': 'vpc-id', 'Values': [vpc_id]},
])['SecurityGroups']
if not groups:
return [f"ERROR {instance_id}: no security group named {QUARANTINE_SG_NAME} in {vpc_id}"]
quarantine_sg = groups[0]['GroupId']
original_sgs = [sg['GroupId'] for sg in instance['SecurityGroups']]
if DRY_RUN:
return [f"DRY RUN: would replace security groups {original_sgs} on {instance_id} with {quarantine_sg}"]
# Record the original groups BEFORE changing anything, so rollback is always possible
ec2.create_tags(Resources=[instance_id], Tags=[
{'Key': 'ORIGINAL_SGS', 'Value': ','.join(original_sgs)},
{'Key': 'FINDING_ID', 'Value': finding_id},
{'Key': 'FINDING_TYPE', 'Value': finding_type},
])
ec2.modify_instance_attribute(InstanceId=instance_id, Groups=[quarantine_sg])
ec2.create_tags(Resources=[instance_id], Tags=[
{'Key': 'SECURITY_STATUS', 'Value': 'QUARANTINED'},
{'Key': 'QUARANTINE_TIME', 'Value': now_iso()},
])
return [f"Quarantined {instance_id}: security groups {original_sgs} replaced with {quarantine_sg}"]
except (ClientError, IndexError, KeyError) as e:
logger.error("Failed to isolate %s: %s", instance_id, e)
return [f"ERROR isolating {instance_id}: {e}"]
def deactivate_access_key(user_name, access_key_id, finding_id, finding_type):
iam = boto3.client('iam')
try:
if DRY_RUN:
return [f"DRY RUN: would deactivate key {access_key_id} for user {user_name}"]
iam.update_access_key(UserName=user_name, AccessKeyId=access_key_id, Status='Inactive')
iam.tag_user(UserName=user_name, Tags=[
{'Key': 'SECURITY_STATUS', 'Value': 'KEY_DEACTIVATED'},
{'Key': 'DEACTIVATED_KEY_ID', 'Value': access_key_id},
{'Key': 'FINDING_ID', 'Value': finding_id},
{'Key': 'FINDING_TYPE', 'Value': finding_type},
])
return [f"Deactivated access key {access_key_id} for user {user_name}"]
except ClientError as e:
logger.error("Failed to deactivate key for %s: %s", user_name, e)
return [f"ERROR deactivating key for {user_name}: {e}"]
def send_slack_alert(finding_type, severity, finding_id, account_id, region,
resource_type, actions):
if severity >= 9.0:
emoji, color, label = ":rotating_light:", "#FF0000", "CRITICAL"
elif severity >= 7.0:
emoji, color, label = ":warning:", "#FF8C00", "HIGH"
else:
emoji, color, label = ":information_source:", "#FFA500", "MEDIUM"
mode = " (DRY RUN)" if DRY_RUN else ""
actions_text = "\n".join(f"- {a}" for a in actions)
message = {
"text": f"{emoji} GuardDuty {label} finding{mode}",
"attachments": [{
"color": color,
"blocks": [
{"type": "header",
"text": {"type": "plain_text", "text": f"Security Alert - {label}{mode}"}},
{"type": "section", "fields": [
{"type": "mrkdwn", "text": f"*Finding Type:*\n{finding_type}"},
{"type": "mrkdwn", "text": f"*Severity:*\n{severity}/10"},
{"type": "mrkdwn", "text": f"*Account:*\n{account_id}"},
{"type": "mrkdwn", "text": f"*Region:*\n{region}"},
{"type": "mrkdwn", "text": f"*Resource Type:*\n{resource_type}"},
{"type": "mrkdwn", "text": f"*Finding ID:*\n{finding_id}"},
]},
{"type": "section",
"text": {"type": "mrkdwn", "text": f"*Actions:*\n{actions_text}"}},
],
}],
}
req = urllib.request.Request(
get_slack_url(),
data=json.dumps(message).encode('utf-8'),
headers={'Content-Type': 'application/json'},
)
try:
with urllib.request.urlopen(req, timeout=10) as resp:
logger.info("Slack notification sent: %s", resp.status)
except Exception as e: # a Slack outage must never undo a successful remediation
logger.error("Failed to send Slack notification: %s", e)
EOF
cd lambda
zip function.zip remediation.py
cd ..
Common Mistake

Deleting a compromised access key instead of deactivating it. Deletion is irreversible and destroys evidence. If the finding is a false positive, or you need to trace what the credential accessed, a deactivated key is far more useful.


Milestone 5: Create the Lambda IAM Role

Concept

The responder is itself a high-value target: it can change security groups and disable credentials. So its permissions are scoped as tightly as the APIs allow. Read-only EC2 describe calls need *. The write actions are limited to instances and security groups in your account and region, to IAM users in your account, and to a single SSM parameter.

Steps

Bash
ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)
cat > trust-policy.json << 'EOF'
{
"Version": "2012-10-17",
"Statement": [{
"Effect": "Allow",
"Principal": {"Service": "lambda.amazonaws.com"},
"Action": "sts:AssumeRole"
}]
}
EOF
ROLE_ARN=$(aws iam create-role \
--role-name guardduty-remediation-role \
--assume-role-policy-document file://trust-policy.json \
--query 'Role.Arn' \
--output text)
aws iam attach-role-policy \
--role-name guardduty-remediation-role \
--policy-arn arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole

The policy file below is not quoted (EOF without quotes), so $ACCOUNT_ID and $AWS_REGION are filled in by the shell:

Bash
cat > remediation-policy.json << EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "ReadEc2",
"Effect": "Allow",
"Action": ["ec2:DescribeInstances", "ec2:DescribeSecurityGroups"],
"Resource": "*"
},
{
"Sid": "QuarantineInstances",
"Effect": "Allow",
"Action": ["ec2:ModifyInstanceAttribute", "ec2:CreateTags"],
"Resource": [
"arn:aws:ec2:$AWS_REGION:$ACCOUNT_ID:instance/*",
"arn:aws:ec2:$AWS_REGION:$ACCOUNT_ID:security-group/*"
]
},
{
"Sid": "DeactivateKeys",
"Effect": "Allow",
"Action": ["iam:UpdateAccessKey", "iam:TagUser"],
"Resource": "arn:aws:iam::$ACCOUNT_ID:user/*"
},
{
"Sid": "ReadSlackWebhook",
"Effect": "Allow",
"Action": "ssm:GetParameter",
"Resource": "arn:aws:ssm:$AWS_REGION:$ACCOUNT_ID:parameter/security/slack-webhook-url"
}
]
}
EOF
aws iam put-role-policy \
--role-name guardduty-remediation-role \
--policy-name remediation-permissions \
--policy-document file://remediation-policy.json
Tip

If a call later fails with AccessDenied, the error message names the action and resource that was denied. Read it before widening the policy. Widen only the one statement that needs it.

Security

iam:UpdateAccessKey on user/* is still broad. For a production version, add a condition or limit it to a path prefix (for example user/service-accounts/*) so the responder cannot disable your break-glass admin user.

Common Mistake

Putting Resource: "*" on every statement just to make AccessDenied errors disappear. Read which action and resource were denied, then widen only that statement.


Milestone 6: Deploy the Lambda Function

Concept

IAM is eventually consistent: the role you just created may not be visible to the Lambda service for a few seconds. The function starts with DRY_RUN=true, so nothing is changed until you have seen the logs and Slack message and are happy with them.

Steps

Bash
## Wait for IAM role propagation
sleep 10
aws lambda create-function \
--function-name guardduty-remediation \
--runtime python3.12 \
--role $ROLE_ARN \
--handler remediation.lambda_handler \
--zip-file fileb://lambda/function.zip \
--timeout 30 \
--environment "Variables={QUARANTINE_SG_NAME=QUARANTINE-deny-all,SLACK_WEBHOOK_PARAM=/security/slack-webhook-url,DRY_RUN=true,SEVERITY_THRESHOLD=7.0}" \
--region $AWS_REGION
aws lambda wait function-active-v2 \
--function-name guardduty-remediation \
--region $AWS_REGION

When you change the code later, redeploy with:

Bash
aws lambda update-function-code \
--function-name guardduty-remediation \
--zip-file fileb://lambda/function.zip \
--region $AWS_REGION
Common Mistake

Skipping the sleep. Creating the function immediately after the role often fails with an error that the role cannot be assumed, because the role has not propagated yet. Wait and retry.

Tip

The runtime name python3.12 was current when this project was written. Lambda retires old runtimes over time, so check the Lambda runtimes page and use a currently supported Python version if this one is rejected.


Milestone 7: Create the EventBridge Rule

Concept

The event pattern decides which findings ever reach your Lambda. GuardDuty severity is a number (0.1 to 10.0), not a label, so the pattern needs a numeric comparison. Common bands are Low (below 4.0), Medium (4.0-6.9) and High (7.0-8.9). Newer GuardDuty documentation also describes a Critical band at 9.0 and above. Confirm the current bands in the GuardDuty docs. The threshold of 7.0 used here catches High and anything above it.

The severity threshold is checked twice, once in the rule and once in the Lambda. That is intentional: if someone loosens the rule later, the function still refuses to act on low-severity findings.

Steps

Bash
aws events put-rule \
--name guardduty-high-severity-findings \
--event-pattern '{
"source": ["aws.guardduty"],
"detail-type": ["GuardDuty Finding"],
"detail": {
"severity": [{"numeric": [">=", 7.0]}]
}
}' \
--state ENABLED \
--description "Route High-severity GuardDuty findings to the remediation Lambda" \
--region $AWS_REGION
LAMBDA_ARN=$(aws lambda get-function \
--function-name guardduty-remediation \
--region $AWS_REGION \
--query 'Configuration.FunctionArn' --output text)
aws events put-targets \
--rule guardduty-high-severity-findings \
--targets "Id=RemediationLambda,Arn=$LAMBDA_ARN" \
--region $AWS_REGION
aws lambda add-permission \
--function-name guardduty-remediation \
--statement-id eventbridge-invoke \
--action lambda:InvokeFunction \
--principal events.amazonaws.com \
--source-arn "arn:aws:events:$AWS_REGION:$ACCOUNT_ID:rule/guardduty-high-severity-findings" \
--region $AWS_REGION
Common Mistake

Matching severity as a string. {"severity": ["HIGH"]} never matches, because GuardDuty sends a number. Use {"numeric": [">=", 7.0]}.


Validation & Testing

Verification Steps

Test in three stages, from safest to most realistic. Do not skip stage 1.

Stage 1: Dry Run Against a Disposable Instance

Launch a throwaway instance (it runs in your default VPC, where you created the quarantine group):

Bash
INSTANCE_ID=$(aws ec2 run-instances \
--image-id resolve:ssm:/aws/service/ami-amazon-linux-latest/al2023-ami-kernel-default-x86_64 \
--instance-type t3.micro \
--tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=guardduty-test-victim}]' \
--query 'Instances[0].InstanceId' --output text \
--region $AWS_REGION)
aws ec2 wait instance-running --instance-ids $INSTANCE_ID --region $AWS_REGION
echo $INSTANCE_ID

Build a fake finding that points at it:

Bash
cat > test-event.json << EOF
{
"detail": {
"id": "test-finding-0001",
"type": "Backdoor:EC2/C&CActivity.B!DNS",
"severity": 8.0,
"accountId": "$ACCOUNT_ID",
"region": "$AWS_REGION",
"resource": {
"resourceType": "Instance",
"instanceDetails": {"instanceId": "$INSTANCE_ID"}
}
}
}
EOF
aws lambda invoke \
--function-name guardduty-remediation \
--cli-binary-format raw-in-base64-out \
--payload file://test-event.json \
--region $AWS_REGION \
response.json && cat response.json
JSON
{"status": "handled", "finding_id": "test-finding-0001", "dry_run": true, "actions": ["DRY RUN: would replace security groups ['sg-...'] on i-... with sg-..."]}

Check Slack for the alert marked (DRY RUN), and confirm the instance's security groups did not change.

Stage 2: Real Isolation

Turn dry-run off. Note that update-function-configuration replaces the whole variable set, so resend every variable:

Bash
aws lambda update-function-configuration \
--function-name guardduty-remediation \
--environment "Variables={QUARANTINE_SG_NAME=QUARANTINE-deny-all,SLACK_WEBHOOK_PARAM=/security/slack-webhook-url,DRY_RUN=false,SEVERITY_THRESHOLD=7.0}" \
--region $AWS_REGION
aws lambda wait function-updated-v2 --function-name guardduty-remediation --region $AWS_REGION
aws lambda invoke \
--function-name guardduty-remediation \
--cli-binary-format raw-in-base64-out \
--payload file://test-event.json \
--region $AWS_REGION \
response.json && cat response.json

Verify the instance is quarantined and tagged:

Bash
aws ec2 describe-instances --instance-ids $INSTANCE_ID --region $AWS_REGION \
--query 'Reservations[0].Instances[0].{SGs:SecurityGroups[*].GroupId,Tags:Tags}'

You should see only the quarantine group, plus the SECURITY_STATUS, QUARANTINE_TIME, FINDING_ID, FINDING_TYPE and ORIGINAL_SGS tags. Invoke the function a second time and confirm the response says SKIPPED ... already quarantined (idempotency).

Stage 3: End to End Through GuardDuty and EventBridge

Generate a sample finding:

Bash
aws guardduty create-sample-findings \
--detector-id $DETECTOR_ID \
--finding-types 'Backdoor:EC2/C&CActivity.B!DNS' \
--region $AWS_REGION
Tip

Use single quotes around finding types. The ! character triggers history expansion inside double quotes in interactive bash.

Then read the Lambda logs:

Bash
aws logs tail /aws/lambda/guardduty-remediation --since 15m --region $AWS_REGION

Sample findings use synthetic instance IDs that do not exist in your account, so the EC2 step will log an ERROR ... InvalidInstanceID.NotFound action. That is expected. What you are proving is that the whole chain fires: GuardDuty to EventBridge to Lambda to Slack. The sample finding's severity depends on its type, so confirm in the log line that it was at or above 7.0. If nothing arrives, check the rule state and the Lambda permission first.

Measure Your Response Time

Compare the QUARANTINE_TIME tag from Stage 2 against the time you invoked the function, and look at the Duration line in the CloudWatch REPORT entry. Write the numbers you measured in your README. Do not copy a number from anyone else.

Rollback Runbook

A false positive that isolates a production instance is an outage. Practise the rollback now, not during an incident:

Bash
## 1. Read the original security groups from the tag
ORIG=$(aws ec2 describe-tags \
--filters Name=resource-id,Values=$INSTANCE_ID Name=key,Values=ORIGINAL_SGS \
--query 'Tags[0].Value' --output text --region $AWS_REGION)
## 2. Restore them (the tag stores a comma-separated list)
aws ec2 modify-instance-attribute \
--instance-id $INSTANCE_ID \
--groups ${ORIG//,/ } \
--region $AWS_REGION
## 3. Clear the quarantine marker so the function can act again if a new finding arrives
aws ec2 delete-tags --resources $INSTANCE_ID \
--tags Key=SECURITY_STATUS Key=QUARANTINE_TIME \
--region $AWS_REGION

For a deactivated key, re-enable it with aws iam update-access-key --user-name <user> --access-key-id <key> --status Active, but only after a human has confirmed the finding was a false positive.

Cleanup

Remove everything so you are not billed for it:

Bash
aws events remove-targets --rule guardduty-high-severity-findings --ids RemediationLambda --region $AWS_REGION
aws events delete-rule --name guardduty-high-severity-findings --region $AWS_REGION
aws lambda delete-function --function-name guardduty-remediation --region $AWS_REGION
aws iam delete-role-policy --role-name guardduty-remediation-role --policy-name remediation-permissions
aws iam detach-role-policy --role-name guardduty-remediation-role \
--policy-arn arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
aws iam delete-role --role-name guardduty-remediation-role
aws ssm delete-parameter --name /security/slack-webhook-url --region $AWS_REGION
aws ec2 terminate-instances --instance-ids $INSTANCE_ID --region $AWS_REGION
aws ec2 wait instance-terminated --instance-ids $INSTANCE_ID --region $AWS_REGION
aws ec2 delete-security-group --group-id $QUARANTINE_SG --region $AWS_REGION
aws guardduty delete-detector --detector-id $DETECTOR_ID --region $AWS_REGION
Common Mistake

Deleting the security group before terminating the instance that uses it. The delete fails with a dependency error. Terminate the instance first.

Common Mistakes Recap

Mistake Why It Breaks Fix
Default outbound rule left on the quarantine group The instance can still send data out Revoke the allow-all egress rule
Quarantine group created in only one VPC Instances in other VPCs cannot use it Create a group with the same name in each VPC
String match on severity Rule never fires Use a numeric comparison
No sleep after role creation Role cannot be assumed yet Wait a few seconds and retry
Missing lambda add-permission Rule matches but Lambda never runs Grant EventBridge invoke permission
update-function-configuration with only one variable All other variables are wiped Always resend the full variable set
Slack webhook in an env var or Git Secret is exposed Keep it in SSM SecureString
Deleting keys or terminating instances Destroys forensic evidence Deactivate and isolate first
Never testing rollback A false positive becomes a long outage Practise the runbook before you need it

Portfolio Deliverable

Publish a repo containing: lambda/remediation.py, the CLI steps as a README, an architecture diagram, a screenshot of a real Slack alert (webhook URL removed), the response time you measured, and the rollback runbook. In the README add a short "limitations" section that covers connection tracking, single-region scope, and the credential types the function does not handle. Hiring managers value honest limits.

Extensions

  • Snapshot the instance's EBS volumes before isolation, so evidence is preserved even if someone later stops the instance
  • Add a network ACL step for faster cut-off of established connections
  • Add an SQS dead-letter queue so failed invocations are not lost
  • Rebuild the whole stack in Terraform
  • Enable GuardDuty across an AWS Organization with a delegated administrator
Found a mistake, an outdated step or a broken link?