AWS – The CLI: Profiles, Queries and the Flags That Bite

The AWS CLI is how you check what is actually deployed, script anything repeatable, and answer questions the console makes you click through. Version 2 is the current one and the only one worth installing.

Profiles, so you never hit the wrong account

Credentials live in ~/.aws/credentials and settings in ~/.aws/config. Give every account a named profile and never configure a default:

aws configure --profile folau
aws sts get-caller-identity --profile folau

Leaving [default] empty is a deliberate safety measure. With no default, a command missing --profile fails asking for credentials instead of quietly running against whatever account happened to be configured. That is the difference between an error and an incident.

get-caller-identity is the first command to run whenever something is denied. It answers "who am I actually authenticated as", which is often not what you assumed.

For an organisation, use SSO rather than long-lived keys — the credentials are short-lived and there is nothing on disk worth stealing:

aws configure sso
aws sso login --profile production

--query turns JSON into an answer

Most commands return far more than you want. --query takes a JMESPath expression and runs it client-side:

# just the ids and states, as a table
aws ec2 describe-instances \
  --query "Reservations[].Instances[].[InstanceId,InstanceType,State.Name]" \
  --output table

# filter inside the expression
aws ec2 describe-instances \
  --query "Reservations[].Instances[?State.Name=='running'].InstanceId" \
  --output text

[] flattens nested arrays, [?…] filters, and --output text gives you something you can pipe into xargs. Note this is different from a service-side --filters, which the API applies before sending — prefer --filters when it exists, because --query still transfers everything first.

Three flags that have cost this site real time

These are not theoretical. Each one shipped a bug on lovemesomecoding.com.

s3 sync skips unchanged files, so metadata never updates

aws s3 sync compares size and timestamp and skips files it thinks match. That includes files whose content is identical but whose metadata you have changed — so editing Cache-Control in a deploy script does nothing at all to objects already in the bucket. The deploy succeeds, the header stays wrong, and nothing indicates a problem.

The fix is to upload those files unconditionally rather than sync them:

aws s3 cp ./out s3://lovemesomecoding.com --recursive \
  --exclude "_next/static/*" \
  --cache-control "public, max-age=0, must-revalidate"

--exact-timestamps is load-bearing when downloading

Downloading, sync skips a same-sized object unless the S3 copy is newer than the local file. Two things break that. Small derived files change without changing size — "count":12 to "count":13 is byte-identical in length. And S3's timestamp is UTC while the local mtime was stamped at download time, so a fresh write can look older than the file it should replace.

Together those shipped a page reading "12 tutorials" above a list of 13.

aws s3 sync s3://my-bucket/prod/ ./content --exact-timestamps

That skips only on an exact timestamp match, so a same-sized change is always re-fetched.

--metadata-directive REPLACE without --content-type

Changing metadata on an existing object requires --metadata-directive REPLACE, which replaces all metadata — including the content type. Omit --content-type and every object becomes binary/octet-stream, at which point browsers download your HTML instead of rendering it.

Always pass both, and never run it across mixed file types in one command.

Pagination is not automatic in the way you expect

The CLI pages through results for you and prints the whole set, which is usually what you want and occasionally very slow. --max-items limits the total returned; --page-size changes the requests made underneath without changing the result.

If a listing command is timing out against a large account, a smaller --page-size is the fix — the default page is too large for the API to build in time. Increasing --max-items will not help.

Small things that pay for themselves

# stop the pager opening on every command
export AWS_PAGER=""

# build a valid request skeleton instead of guessing at nested JSON
aws ecs register-task-definition --generate-cli-skeleton > task.json
aws ecs register-task-definition --cli-input-json file://task.json

# dry run: check permissions without doing anything
aws ec2 stop-instances --instance-ids i-0abc123def456 --dry-run

--generate-cli-skeleton is the one people discover late. For any command taking a deeply nested structure, it prints the exact shape expected, which beats assembling it from the documentation.

--dry-run exists on most EC2 mutating calls and reports whether you would be permitted, without acting. DryRunOperation means yes; UnauthorizedOperation means no.

Output formats

Four are available and each has a job. json is the default and the one to pipe into jq. table is for reading with your eyes. text is tab-separated and the one to pipe into shell tools — though note that a field containing a tab or an empty value will misalign your columns, so prefer json plus jq for anything you rely on. yaml exists and is rarely the answer.

aws ec2 describe-instances \
  --query "Reservations[].Instances[].InstanceId" --output text \
  | xargs -n1 echo "found:"

Combining --query with --output text is the workhorse: it turns a listing into a list of ids you can loop over, without a JSON parser in the pipeline.

When output looks wrong

Two habits worth having. Add --debug to see the actual request, signature and response — verbose, but it answers "which region did that go to" definitively. And remember that --region beats the profile's region, which beats AWS_DEFAULT_REGION: a resource that "does not exist" is very often in a region you did not mean to query.