# Infrastructure stacks reference

> Reference: Pulumi stack layout, reusable resources, service stacks, buckets, queues, Datadog sidecars, FusionAuth, and deployment inputs.

- Repository: macro-inc/macro
- GitHub: https://github.com/macro-inc/macro
- Human docs: https://grok-wiki.com/public/docs/macro-inc-macro-bb988e1a448e
- Complete Markdown: https://grok-wiki.com/public/docs/macro-inc-macro-bb988e1a448e/llms-full.txt

## Source Files

- `infra/README.md`
- `infra/packages/resources/src/index.ts`
- `infra/packages/lambda/src/index.ts`
- `infra/packages/service/src/index.ts`
- `infra/stacks/document-storage/index.ts`
- `infra/stacks/mcp-server/index.ts`
- `infra/stacks/fusionauth-instance/README.md`

---

---
title: "Infrastructure stacks reference"
description: "Reference: Pulumi stack layout, reusable resources, service stacks, buckets, queues, Datadog sidecars, FusionAuth, and deployment inputs."
---

Macro infrastructure is a Bun/TypeScript Pulumi monorepo under `infra/`, with deployable projects in `infra/stacks/*` and reusable components in `infra/packages/*`. AWS resources target `us-east-1`, and each stack is deployed from its own stack directory with Pulumi.

## Stack layout

```text
infra/
  package.json                 # Bun workspaces: stacks/* and packages/*
  packages/
    resources/                 # Shared AWS components: buckets, queues, ALB, RDS, Redis, Datadog
    lambda/                    # Rust Lambda packaging and worker trigger helpers
    service/                   # ECR image builder component
    shared/                    # stack/config constants, service URLs, shared StackReference helpers
    vpc/                       # legacy Coparse API VPC/subnet IDs
  stacks/
    document-storage/          # document S3 bucket, replication bucket, shared ECS cluster
    cloud-storage-service/     # application service and docx upload bucket
    mcp-server/                # public ECS/Fargate MCP server
    fusion-auth/               # FusionAuth runtime service and RDS database
    fusionauth-instance/       # FusionAuth tenant/application configuration
    ...
```

Run Pulumi from a stack directory:

```bash
cd infra/stacks/mcp-server
pulumi up --stack dev
pulumi up --stack prod
```

<Info>
The root `infra/package.json` enforces Bun as the package manager and exposes `bun run check`, `bun run lint`, and `bun run format` for TypeScript validation.
</Info>

## Shared packages

| Package | Main exports | Purpose |
| --- | --- | --- |
| `packages/resources` | `createBucket`, `createBucketV2`, `Queue`, `QueueAlarms`, `serviceLoadBalancer`, `DynamoDBTable`, `Database`, `Redis`, Datadog helpers | Reusable AWS building blocks for stacks |
| `packages/lambda` | `Lambda`, `WorkerTrigger`, `generateContentHash`, `SourceCodeHash` | Rust Lambda deployment and ECS worker trigger helpers |
| `packages/service` | `EcrImage` | ECR repository plus Docker image build/push |
| `packages/shared` | `stack`, `config`, constants, service URL helpers, stack-reference helpers | Shared Pulumi config, hard-coded account constants, cross-stack outputs |
| `packages/vpc` | `get_coparse_api_vpc`, `COPARSE_API_VPC` | Existing VPC and subnet IDs used by services |

## Cross-stack dependency model

```mermaid
flowchart LR
  subgraph Foundation
    DS["document-storage stack\nS3 bucket + ECS cluster"]
    VPC["packages/vpc\nget_coparse_api_vpc()"]
  end

  subgraph SharedOutputs
    CSS["cloud-storage-service\nDOCX bucket + queues"]
    Email["email-service\nscheduled queue"]
    Link["link-sharing\nCloudFront signer outputs"]
  end

  subgraph ServiceStacks
    MCP["mcp-server\nMcpServer component"]
    Cog["document-cognition-service\nDocumentCognitionService component"]
    Contacts["contacts-service\nContactsService component"]
  end

  DS --> MCP
  DS --> Cog
  DS --> Contacts
  VPC --> MCP
  VPC --> Cog
  VPC --> Contacts
  CSS --> MCP
  Email --> MCP
  Link --> MCP
  CSS --> Cog
  Email --> Cog
  Link --> Cog
```

`document-storage` is both a storage stack and a foundational service stack: it exports the document bucket and the shared ECS cluster outputs consumed by ECS service stacks.

## Buckets

### `createBucket`

`createBucket` creates an `aws.s3.Bucket` with common Macro defaults:

| Input | Behavior |
| --- | --- |
| `id` | Pulumi resource ID |
| `bucketName` | Physical S3 bucket name |
| `transferAcceleration` | Enables S3 acceleration when true |
| `enableVersioning` | Enables bucket versioning with MFA delete disabled |
| `lifecycleRules` | Passed through to S3 bucket lifecycle rules |
| `exposeHeaders` | Added to default CORS exposed headers |
| `tags` | Applied to the bucket |

Default behavior:

- `forceDestroy` is enabled outside `prod`.
- Production buckets log to `macro-logging-bucket` with prefix `${bucketName}/`.
- CORS allows `GET`, `PUT`, `POST`, `DELETE`, and `HEAD`.
- CORS allows all headers and exposes `ETag` plus any caller-provided headers.
- Allowed origins come from `packages/resources/src/resources/cors.ts`.

### `createBucketV2`

`createBucketV2` uses `aws.s3.BucketV2` and creates separate resources for:

- CORS configuration
- lifecycle configuration
- versioning
- transfer acceleration
- production logging

Use this helper when a stack needs the newer S3 V2 resource model or finer-grained Pulumi dependencies.

### `document-storage` bucket

The `document-storage` stack creates the primary document bucket:

| Stack | Bucket name |
| --- | --- |
| `prod` | `macro-document-storage-prod` |
| non-`prod` | `doc-storage-${stack}` |

Configured behavior:

- Versioning enabled.
- Transfer acceleration enabled only in `prod`.
- `temp_files/` objects expire after 1 day.
- expired delete markers are removed.
- noncurrent versions expire after 30 days.
- EventBridge bucket notifications are enabled.
- CORS exposes `Content-Length` and `Content-Range`.

The stack exports:

| Output | Value |
| --- | --- |
| `documentStorageBucketId` | bucket ID/name |
| `documentStorageBucketArn` | bucket ARN |
| `documentStorageBucketName` | same value as bucket ID |
| `documentStorageBucketReplicationRoleArn` | replication IAM role ARN |
| `cloudStorageClusterName` | shared ECS cluster name |
| `cloudStorageClusterArn` | shared ECS cluster ARN |

### Replication bucket

`document-storage/replication-bucket.ts` creates a cross-region replication target in `us-west-1`.

| Resource | Naming |
| --- | --- |
| bucket | `macro-doc-storage-replication` in `prod`, otherwise `macro-doc-storage-replication-${stack}` |
| IAM role | `replication-role-${stack}` |
| admin group | `document-store-admin-${stack}` |

The replication bucket is versioned. In production it logs to `macro-access-log-bucket-uswest2`. Its bucket policy denies non-admin access and allows the replication role to read replication-related object versions.

## Queues

### `Queue`

`Queue` is a Pulumi component that creates:

- an SQS queue
- a DLQ
- a DLQ visible-message alarm
- a default queue age alarm through `QueueAlarms`

Naming:

| Resource | Name pattern |
| --- | --- |
| main queue | `${name}-queue-${stack}` |
| DLQ | `${name}-dlq-${stack}` |
| FIFO queue | appends `.fifo` to both names |

Defaults:

| Option | Default |
| --- | --- |
| `maxReceiveCount` | `5` |
| `visibilityTimeoutSeconds` | `30` |
| DLQ retention | `1209600` seconds |
| `fifoQueue` | `false` |

The DLQ alarm fires when `ApproximateNumberOfMessagesVisible > 0` and sends actions to `CloudTrailSNS`.

### `QueueAlarms`

`QueueAlarms` creates an `ApproximateAgeOfOldestMessage` CloudWatch alarm for a queue.

| Option | Default |
| --- | --- |
| `approximateAgeOfOldestMessageEvaluationPeriods` | used as the alarm period, default `60` seconds |
| `approximateAgeOfOldestMessageThreshold` | `120` seconds |
| metric namespace | `AWS/SQS` |
| alarm action | `CloudTrailSNS` |

### Custom queue stacks

Some stacks define queues directly instead of using `Queue`. For example, `search-event-queue` creates:

- `search-event-queue-${stack}`
- `search-event-queue-dlq-${stack}`
- an SNS topic named `search-event-queue-alerts-${stack}`
- production-only alarms for DLQ messages, old messages, and visible message count

## Service stacks

Service stacks usually follow this pattern:

1. Read Pulumi config with `config.require(...)`.
2. Resolve AWS Secrets Manager values with `aws.secretsmanager.getSecretVersionOutput(...)`.
3. Import shared VPC values with `get_coparse_api_vpc()`.
4. Read other stack outputs with `pulumi.StackReference`.
5. Create an ECS/Fargate component for the service.
6. Export service URLs, role ARNs, queue names, bucket names, or Lambda names needed by downstream stacks.

### Shared VPC

`get_coparse_api_vpc()` returns a fixed VPC ID and fixed public/private subnet IDs. Service tasks run in private subnets. ALBs use public subnets unless `isPrivate` is true.

### Load balancers

`serviceLoadBalancer` creates:

| Resource | Behavior |
| --- | --- |
| target group | HTTP target group with `targetType: "ip"` and caller-provided health check path |
| ALB | public or internal application load balancer |
| HTTPS listener | port `443`, TLS policy `ELBSecurityPolicy-TLS13-1-2-2021-06`, shared Macro ACM certificate |
| HTTP listener | port `80`, redirects to HTTPS with `HTTP_301` |
| access logs | enabled in `prod` to `macro-alb-logging` |

`isPrivate` controls subnet selection:

| `isPrivate` | ALB subnets |
| --- | --- |
| `true` | VPC private subnets |
| false/undefined | VPC public subnets |

### ECR images

`EcrImage` creates:

- mutable ECR repository
- force-delete repository behavior
- skipped default AWSX lifecycle policy
- explicit lifecycle rule to expire untagged images older than 1 day
- Docker image tagged `latest`

If `USE_PREBUILT_SERVICE_BINARIES=true`, supported Dockerfiles are remapped:

| Dockerfile | Prebuilt Dockerfile |
| --- | --- |
| `Dockerfile` | `Dockerfile.prebuilt` |
| `Dockerfile.convert_service` | `Dockerfile.convert_service.prebuilt` |
| `Dockerfile.search_processing_service` | `Dockerfile.search_processing_service.prebuilt` |

Unsupported Dockerfile names throw an error when prebuilt binaries are enabled.

### ECS/Fargate service conventions

The `mcp-server` and `document-cognition-service` components show the current ECS service shape:

- tasks run in private subnets
- security group ingress is only from the ALB security group on the service port
- ALB accepts public HTTP/HTTPS when the service is public
- deployment circuit breaker is enabled with rollback
- task role is passed through `taskDefinitionArgs.taskRole`
- Route53 creates an A alias for the service domain
- CPU, memory, ALB 5xx, and target-tracking scaling policies are declared in the service component

`mcp-server` uses:

| Setting | Value |
| --- | --- |
| domain | `mcp-server.macro.com` in `prod`, `mcp-server-${stack}.macro.com` otherwise |
| container port | `8080` |
| health check | `/health` |
| desired count | `1` |
| autoscaling min/max | min `1`, max `10` in `prod`, max `3` otherwise |
| scaling targets | ALB request count `1000`, CPU `70%`, memory `70%` |
| service alarms | CPU `80%`, memory `80%`, ALB 5xx count `25` |

## Datadog integration

Datadog support is centralized in `packages/resources`.

### Secrets and sidecars

`DATADOG_API_KEY` is read from Secrets Manager secret `datadog-api-key`.

`fargateLogRouterSidecarContainer` uses:

- image `amazon/aws-for-fluent-bit:latest`
- FireLens type `fluentbit`
- config file `/fluent-bit/configs/parse-json.conf`
- `ECS_FARGATE=true`
- `DD_ENV=${stack}`
- memory reservation `50`

`datadogAgentContainer` uses:

- image `public.ecr.aws/datadog/agent:latest`
- `DD_SITE=us5.datadoghq.com`
- APM enabled
- OTLP gRPC endpoint `0.0.0.0:4317`
- sample rate `0.1` in `prod`, `1.0` otherwise
- max traces per second `100`
- memory reservation `256`

Service containers that use FireLens send logs to `http-intake.logs.us5.datadoghq.com` with `dd_service`, `dd_source=fargate`, `dd_tags`, and `provider=ecs`.

### Lambda logs

The shared `Lambda` component creates a CloudWatch log group at:

```text
/aws/lambda/${baseName}-${stack}
```

It also creates a log subscription filter to the shared Datadog Kinesis Firehose stream and filters out Lambda runtime boilerplate lines:

```text
START RequestId
REPORT RequestId
END RequestId
INIT_START
```

### Datadog Software Catalog

`DatadogServiceEntity` creates a Datadog Software Catalog service entity with:

- service name
- display name
- owner
- GitHub repository link
- GitHub code path
- language metadata
- dependency on `service:document-storage`

## Lambda components

### `Lambda`

The generic `Lambda<T>` component deploys Rust Lambdas using the `provided.al2023` runtime.

Key behavior:

| Field | Value |
| --- | --- |
| handler | `bootstrap` |
| runtime | `provided.al2023` |
| architecture | `x86_64` |
| timeout default | `30` seconds |
| log retention | `7` days |
| deployment package | `pulumi.asset.FileArchive(zipLocation)` |

If a VPC is supplied, the component creates a Lambda security group, attaches all-in/all-out rules, and places the Lambda in private subnets.

The component adds `SOURCE_CODE_FILE_HASH` to the Lambda environment and depends on a `SourceCodeHash` resource so Pulumi detects source-code changes after a build.

### `WorkerTrigger`

`WorkerTrigger` creates a Lambda that runs an ECS task.

Environment variables:

| Name | Value |
| --- | --- |
| `TASK_DEFINITION` | task definition ARN |
| `CLUSTER` | ECS cluster ARN |
| `SUBNETS` | comma-separated private subnet IDs |
| `ENVIRONMENT` | Pulumi stack |
| `RUST_LOG` | `worker_trigger=info` |

The trigger role can call `ecs:RunTask` for the supplied task definition and cluster, and can `iam:PassRole` to `ecs-tasks.amazonaws.com`.

## AI tools deployment inputs

`getAiToolsInfra()` returns environment variables and IAM resource ARNs needed by services that host the Rust `ai_tools` crate.

It reads stack outputs from:

| StackReference name | Target stack |
| --- | --- |
| `ai-tools-cloud-storage-stack` | `macro-inc/document-storage/${stack}` |
| `ai-tools-cloud-storage-service-stack` | `macro-inc/cloud-storage-service/${stack}` |
| `ai-tools-email-service-stack` | `macro-inc/email-service/${stack}` |
| `ai-tools-linksharing-stack` | `macro-inc/link-sharing/${stack}` |

Returned environment variables include:

| Name | Source |
| --- | --- |
| `INTERNAL_API_SECRET_KEY` | `document-storage-service-auth-key-${stack}` secret value |
| `DOCUMENT_STORAGE_SERVICE_URL` | shared service URL map |
| `EMAIL_SERVICE_URL` | shared service URL map |
| `SYNC_SERVICE_URL` | shared service URL map |
| `LEXICAL_SERVICE_URL` | shared service URL map |
| `STATIC_FILE_SERVICE_URL` | shared service URL map |
| `SYNC_SERVICE_AUTH_KEY` | `sync-service-key-${stack}` secret name |
| `MCP_CREDENTIALS_KEY_SECRET_NAME` | `mcp-credentials-key-${stack}` |
| `DOCUMENT_STORAGE_BUCKET` | document-storage stack output |
| `DOCX_DOCUMENT_UPLOAD_BUCKET` | cloud-storage-service stack output |
| `EMAIL_SCHEDULED_QUEUE` | email-service stack output |
| `DOCUMENT_STORAGE_SERVICE_CLOUDFRONT_DISTRIBUTION_URL` | link-sharing stack output |
| `DOCUMENT_STORAGE_SERVICE_CLOUDFRONT_SIGNER_PUBLIC_KEY_ID` | link-sharing stack output |
| `DOCUMENT_STORAGE_SERVICE_CLOUDFRONT_SIGNER_PRIVATE_KEY_SECRET_NAME` | `linksharing-private-key-${stack}` |

Returned IAM inputs include:

- secret ARNs for sync service auth, CloudFront private key, and MCP credentials
- queue ARN for the email scheduled queue
- bucket ARNs for document storage and DOCX uploads

`getAiToolsServiceRoleArns()` returns role ARNs for services that host `ai_tools`, currently from `mcp-server`, `document-cognition`, and `agent-schedule-service` stack outputs.

## MCP server stack

`infra/stacks/mcp-server` deploys an ECS/Fargate service using the shared document-storage ECS cluster.

Required Pulumi config consumed by the stack:

| Key | Meaning |
| --- | --- |
| `macro_db_secret_key` | Secrets Manager key for database URL |
| `jwt_secret_key` | Secrets Manager key name for JWT secret |
| `fusionauth_client_id` | Secrets Manager key for FusionAuth client ID |
| `fusionauth_base_url` | FusionAuth base URL |
| `fusionauth_issuer` | FusionAuth issuer |
| `fusionauth_client_secret` | Secrets Manager key for FusionAuth client secret |
| `fusionauth_tenant_id` | FusionAuth tenant ID |
| `fusionauth_api_key` | Secrets Manager key for FusionAuth API key |
| `google_client_id` | Secrets Manager key for Google client ID |
| `google_client_secret` | Secrets Manager key for Google client secret |
| `macro_cache_secret_key` | Secrets Manager key for Redis endpoint |
| `anthropic_api_key` | Secrets Manager key for Anthropic API key |

The service also receives all `getAiToolsInfra()` environment variables.

Exports:

| Output | Meaning |
| --- | --- |
| `mcpServerUrl` | service URL |
| `mcpServerRoleArn` | ECS task role ARN |

## FusionAuth stacks

Macro uses two separate FusionAuth-related stacks.

| Stack | Responsibility |
| --- | --- |
| `fusion-auth` | Runs FusionAuth itself on ECS/Fargate with an RDS PostgreSQL database |
| `fusionauth-instance` | Configures FusionAuth tenants, application, templates, lambdas, webhooks, signing key, and OAuth settings through `pulumi-fusionauth` |

### `fusion-auth`

The runtime stack creates:

- RDS PostgreSQL database named `fusionauth`
- ECS cluster named `fusionauth-${stack}`
- ECS/Fargate service using image `fusionauth/fusionauth-app:1.62.1`
- ALB and Route53 record
- CloudWatch alarms for CPU, memory, and ALB 5xx
- production-only service autoscaling

Domain names:

| Stack | Domain |
| --- | --- |
| `prod` | `auth.macro.com` |
| non-`prod` | `fusionauth-${stack}.macro.com` |

Database behavior:

| Stack | Behavior |
| --- | --- |
| `prod` | endpoint output is hard-coded to `fusionauthdb-prod.macro.com` |
| non-`prod` | endpoint comes from the created RDS instance |
| `dev` | database is publicly accessible |
| `prod` | database is not publicly accessible |

Required config:

| Key | Meaning |
| --- | --- |
| `fusion-auth:db-password-secret-key` | Secrets Manager key containing the database password |

### `fusionauth-instance`

The configuration stack uses the `pulumi-fusionauth` provider. Provider inputs are read from the `fusionauth` Pulumi config namespace:

| Key | Meaning |
| --- | --- |
| `fusionauth:host` | FusionAuth API host |
| `fusionauth:apiKey` | FusionAuth API key |

Stack config inputs include:

| Key | Meaning |
| --- | --- |
| `fusionauth-instance:fusionauth-issuer` | tenant issuer |
| `fusionauth-instance:fusionauth-signing-key-id` | signing key ID |
| `fusionauth-instance:fusionauth-license-key-secret-key` | Secrets Manager key for FusionAuth license |
| `fusionauth-instance:smtp-user-secret-key` | Secrets Manager key containing `{ username, password }` |
| `fusionauth-instance:default-from-email` | tenant default sender email |
| `fusionauth-instance:authentication-service-domain` | authentication service base URL |
| `fusionauth-instance:authentication-service-internal-secret-key` | optional Secrets Manager key for webhook auth |
| `fusionauth-instance:fusionauth-default-tenant-id` | required outside `local` |
| `fusionauth-instance:fusionauth-client-id` | required outside `local` |
| `fusionauth-instance:fusionauth-client-secret-key` | required outside `local` |

Configured FusionAuth resources:

- Reactor license
- system CORS for Apple ID POST callbacks
- passwordless login email template
- email verification template
- create-user and delete-user webhooks to the authentication service
- default tenant with SMTP, email verification, event configuration, theme, issuer, logout URL, and unverified-user deletion after 30 days
- `populate_macro_jwt` JWTPopulate lambda
- `reconcile_secondary_idp_link` OpenIDReconcile lambda
- HS256 signing key
- Macro application with passwordless login, JWT, refresh tokens, and OAuth redirects

OAuth redirect URLs include:

- `${AUTHENTICATION_SERVICE_DOMAIN}/oauth/redirect`
- `https://mcp-server.macro.com/oauth/callback` in `prod`
- `https://mcp-server-${stack}.macro.com/oauth/callback` outside `prod`
- localhost redirect patterns in `local` and `dev`

<Warning>
The Google/Gmail secondary IdP reconcile lambda is declared in Pulumi, but the IdP wiring is documented in code as an admin-UI step until the IdP itself is managed by Pulumi.
</Warning>

### Local FusionAuth setup

`infra/stacks/fusionauth-instance/justfile` provides local setup commands.

```bash
cd infra/stacks/fusionauth-instance
just setup
```

`just setup` installs workspace dependencies, downloads the FusionAuth Docker Compose `.env`, starts FusionAuth, waits for `/api/status`, initializes the local Pulumi stack, applies the FusionAuth configuration, patches the root `.env`, and stops the container.

<Warning>
The local `fusionauth-instance` stack is intended for a personal Pulumi account. Do not create it with the `macro-inc/` organization prefix.
</Warning>

## Deployment checklist

<Steps>
  <Step title="Install dependencies">
    Run `bun i` from `infra/` or use the stack-specific setup command when one exists.
  </Step>
  <Step title="Select the stack directory">
    Run Pulumi from `infra/stacks/<stack-name>`, not from the repository root.
  </Step>
  <Step title="Verify config">
    Confirm `Pulumi.<stack>.yaml` contains `aws:region: us-east-1` and every `config.require(...)` key consumed by that stack.
  </Step>
  <Step title="Check stack references">
    For service stacks, verify upstream stacks have exported the referenced outputs. Common dependencies are `document-storage`, `cloud-storage-service`, `email-service`, and `link-sharing`.
  </Step>
  <Step title="Deploy">
    Run `pulumi up --stack dev` or `pulumi up --stack prod`.
  </Step>
</Steps>

## Related pages

<CardGroup>
  <Card title="Service stack implementation" href="/infrastructure/service-stacks">
    ECS/Fargate service component shape, load balancer behavior, and service-specific deployment patterns.
  </Card>
  <Card title="FusionAuth operations" href="/infrastructure/fusionauth">
    Runtime FusionAuth service, tenant configuration, local setup, and OAuth integration details.
  </Card>
</CardGroup>