Testing Professor
Testing ProfessorMulti-Cloud Guidebook · AWS
All scenarios

End-to-End QA Scenario

GitHub Actions → AWS via OIDC (Keyless Deploy + QA Gates)

Federated OIDC trust replaces long-lived AWS keys. PR builds run unit + Playwright on an ephemeral CloudFormation stack; merge promotes to staging behind a CodeDeploy canary with CloudWatch auto-rollback.

Architecture

GitHub PR ─► Actions workflow
   ├─ aws-actions/configure-aws-credentials@v4 (OIDC, no static keys)
   ├─ docker build + push → ECR (scan-on-push)
   ├─ deploy ephemeral CFN stack (PR-{number}) → ECS Fargate
   ├─ matrix: Playwright shards 1..N against the PR URL
   │      └─ JUnit + trace → S3 (artifacts/pr-{n}/)
   ├─ post status checks back to PR (pass-rate, p95)
   └─ on PR close → CFN delete stack
GitHub main push
   └─ promote image tag → CodeDeploy canary (10%) → CloudWatch alarms → 100% or rollback

Workflow steps

  1. 1

    One-time trust

    Create an IAM OIDC provider for `token.actions.githubusercontent.com` and a role with a trust policy scoped to `repo:org/repo:ref:refs/heads/main` and `pull_request`. No access keys live in GitHub.

  2. 2

    Workflow auth

    Job sets `permissions: id-token: write` and calls `aws-actions/configure-aws-credentials@v4` with `role-to-assume`. The short-lived STS token is used by all subsequent AWS CLI/SDK calls.

  3. 3

    Build + scan

    `docker/build-push-action` pushes to ECR; ECR scan-on-push blocks the workflow on HIGH/CRITICAL CVEs via a follow-up `aws ecr describe-image-scan-findings` step.

  4. 4

    Ephemeral env per PR

    `aws cloudformation deploy` creates stack `qa-pr-{{ github.event.number }}` with an ECS Fargate service + ALB. Stack outputs the public URL consumed by later steps.

  5. 5

    Parallel e2e

    Matrix strategy fans out Playwright across N shards; each shard uploads `junit.xml`, trace and video to `s3://qa-artifacts/pr-{n}/shard-{i}/`. Failures attach screenshots directly to the PR via `actions/upload-artifact`.

  6. 6

    PR feedback

    Final job aggregates shard results and posts a sticky PR comment with pass-rate, slowest 5 specs and a link to the S3-hosted HTML report.

  7. 7

    Teardown

    A `pull_request: closed` workflow runs `aws cloudformation delete-stack` so PR envs do not accumulate cost.

  8. 8

    Promote

    On merge to `main`, a deploy workflow registers a new ECS task definition and triggers CodeDeploy in `CodeDeployDefault.ECSCanary10Percent5Minutes`. CloudWatch alarm on 5xx auto-rolls back.

Key takeaways

  • OIDC eliminates the #1 GitHub Actions security incident: leaked AWS keys.
  • Ephemeral per-PR stacks make every review a real staging environment.
  • Sticky PR comments turn QA results into reviewer-visible signal, not buried logs.