AWS Hub
AWS hosts the EC2 Mac Ultra (openclaw-mac-ultra, arm64) that is the mandatory execution environment for all InvestorLift scraping. The VPS IP is CloudFront-blocked by InvestorLift (HTTP 403); the Mac has a different IP + valid cookies that bypass the WAF. AWS also hosts the sendlift S3 bucket (public-read) used for IL property images. Read this hub before any InvestorLift work or before any AWS credential/IAM change.
⚠️ CRITICAL STATUS ALERTS
EC2 Mac Ultra — IMPAIRED since 2026-05-02 22:15 UTC
The openclaw-mac-ultra instance is currently in impaired state per AWS health checks (paused 2026-05-02 22:15 UTC). EBS state is preserved. No data loss has occurred.
Recovery action (next session):
- Open AWS Console → EC2 → Instances →
openclaw-mac-ultra - Actions → Reboot instance
- Wait for status checks to pass (typically 3-5 min for Mac instances)
- Verify Tailscale connectivity:
tailscale status | grep openclaw-mac-ultra - Verify SSH:
ssh -i ~/.ssh/openclaw-mac.pem ec2-user@100.123.248.46 "uptime"
Impact while impaired: All InvestorLift scraping (_scrapeILDeal() in lovable-api-server.js) is blocked. The IL skill, il-marketplace-pull, and all IL enrichment workflows will fail until the instance is restored.
ROOT KEY ROTATION — P0 PENDING
AWS root/IAM access keys are currently stored in plaintext in master.env. These keys were used during the 2026-05-02/03 session. This is a G-NO-PLAINTEXT-CREDS violation.
Required action (P0, before next AWS operation):
- Rotate AWS access key pair in IAM console
- Store new keys in 1Password:
op://Aurora/aws/access-key-idandop://Aurora/aws/secret-key - Remove plaintext keys from
master.env— replace withop://Aurora/aws/...references - Sync new keys to all scripts that use
AWS_ACCESS_KEY_ID/AWS_SECRET_ACCESS_KEY - Verify old keys are revoked in IAM (cannot be used even if leaked)
Cross-ref: 1password — op:// access pattern via op CLI.
Quick reference
| Field | Value |
|---|---|
| Vendor | Amazon Web Services |
| URL | https://console.aws.amazon.com |
| Dashboard | https://console.aws.amazon.com |
| KB doc | 2026-05-02 — audit report only; no API.md yet |
| Auth method | IAM access key + secret (target: IAM role) |
| Auth credential | op://Aurora/aws/access-key-id + op://Aurora/aws/secret-key (target state; currently plaintext in master.env — P0 rotation pending) |
| Cred-proxy port | n/a |
| Webhook port | n/a |
| Webhook handler | n/a (AWS is outbound SSH target, not inbound webhook source) |
| Webhook dedup table | n/a |
| Tunnel path | n/a |
| Outbound API base | https://aws.amazon.com (EC2 API varies by region) |
| Rate limits | EC2 API: varies by operation; S3: no hard rate limit on public-read GETs |
| Rate-limit action | Varies by service; exponential backoff; Discord ops alert on sustained 429 |
| Cost | EC2 Mac Ultra: dedicated host pricing (arm64 Mac instance, 24h minimum); S3 sendlift: public-read, minimal egress cost |
| Backup/recovery | EBS volume preserved on impaired state; no automated snapshot confirmed |
| Discord alert channel | ops |
| Drift cadence | Manual — no automated AWS health check integration yet |
| Status | production (Mac Ultra impaired — see above) |
EC2 Mac Ultra — InvestorLift scraping (MANDATORY)
Rule: ALWAYS scrape InvestorLift via the Mac, NEVER on the VPS
MANDATORY: InvestorLift deal page scraping (_scrapeILDeal() in lovable-api-server.js)
MUST use SSH to the AWS Mac (ec2-user@100.123.248.46), NOT local Playwright on the VPS.
Why: The VPS IP is CloudFront-blocked by IL (403). The Mac has a different IP + valid
cookies that bypass the WAF. Attempting Playwright on the VPS returns HTTP 403 and 0 photos.
This rule is encoded in CLAUDE.md §InvestorLift Scraping — ALWAYS Via AWS Mac and in reference_mac_hosts_ssh. It is non-negotiable until the VPS IP is unblocked (no timeline).
Instance details
| Property | Value |
|---|---|
| Instance name | openclaw-mac-ultra (also referred to as aws-mac) |
| Tailscale hostname | 100.123.248.46 (Tailscale IP; also accessible as openclaw-mac-ultra via MagicDNS) |
| Architecture | arm64 (Apple Silicon Mac — M1 Ultra) |
| SSH user | ec2-user |
| SSH key | ~/.ssh/openclaw-mac.pem (AWS-only; not used for other Mac hosts) |
| Cookie file | /Users/ec2-user/openclaw/workspace/data/investorlift-cookies-raw.txt |
| Current status | IMPAIRED since 2026-05-02 22:15 UTC |
IL scraping pattern (proven)
From workspace/scripts/il-folder-photos.js:
1. SSH to Mac: ssh -i ~/.ssh/openclaw-mac.pem ec2-user@100.123.248.46
2. curl IL deal page using Mac's cookie file (investorlift-cookies-raw.txt)
3. Extract sendlift/property-images/{ID}.jpg image IDs from raw HTML
4. Build public S3 URLs: https://s3.us-east-2.amazonaws.com/sendlift/property-images/{ID}.jpg
5. Download photos from S3 on VPS (S3 bucket is public-read, no auth needed)
Never deviate from this pattern. See investorlift for the full IL pipeline.
Other Mac hosts (disambiguation)
| Host | Type | Purpose |
|---|---|---|
aws-mac / openclaw-mac-ultra | EC2 Mac arm64 (100.123.248.46) | IL scraping — THIS HUB |
openclaw-mac-cloud | MacinCloud Intel | Legacy; not actively used |
auroras-mac-mini | Home Mac mini (100.93.7.6) | BlueBubbles iMessage relay |
Source: reference_mac_hosts_ssh
LaunchDaemon: com.openclaw.tailscaled
A system-level Tailscale daemon is installed on openclaw-mac-ultra as a LaunchDaemon. This ensures Tailscale persists across reboots without requiring a logged-in user session.
| Property | Value |
|---|---|
| LaunchDaemon | com.openclaw.tailscaled |
| Host | EC2 Mac Ultra (openclaw-mac-ultra, 100.123.248.46) |
| Plist path | /Library/LaunchDaemons/com.openclaw.tailscaled.plist |
| Binary | /opt/homebrew/bin/tailscaled --state=/var/lib/tailscale/tailscaled.state --socket=/var/run/tailscaled.socket |
| Logs | /var/log/tailscaled.log, /var/log/tailscaled.err.log |
| Purpose | System-level Tailscale daemon; persistent across reboots without logged-in user |
Why system-level: The previous per-user Tailscale agent was fragile — it required an active user session and would disconnect on reboots. The LaunchDaemon runs as root at boot time and maintains the Tailscale connection reliably.
After instance reboot (see impaired recovery above): verify Tailscale is connected before attempting any IL scraping or SSH.
Plan reference: openclaw-obsidian-vault-2026-05-02 §Phase 6.1.2
S3 — sendlift bucket
| Property | Value |
|---|---|
| Bucket name | sendlift |
| Region | us-east-2 |
| Access | Public-read — no auth required for GET requests |
| URL pattern | https://s3.us-east-2.amazonaws.com/sendlift/property-images/{ID}.jpg |
| Purpose | InvestorLift property images |
| Auth for downloads | None — download directly on VPS without AWS credentials |
The sendlift bucket is InvestorLift’s own S3 bucket (not OpenClaw-owned). It is public-read, so S3 downloads can happen directly on the VPS — only the initial IL deal page scrape (to get image IDs from HTML) must go through the Mac.
Components
workspace/scripts/il-folder-photos.js— IL scraping via Mac SSH (proven pattern)workspace/scripts/sync-il-api-to-supabase.js— IL API sync to Supabaseworkspace/scripts/enrich-il-deals-detail-api.js— IL deal enrichment~/.ssh/openclaw-mac.pem— SSH key for EC2 Mac Ultra/Library/LaunchDaemons/com.openclaw.tailscaled.plist(on Mac) — Tailscale daemonworkspace/knowledge-base/aws/_audit/2026-05-02.md— vendor audit report (no API.md yet)
How it’s used
- Trigger: IL skill invoked (
/il-marketplace-pull) OR dailyinvestorlift-daily-sync.shcron - Flow: VPS cron/skill → SSH to Mac → curl IL deal page with cookies → extract image IDs → build S3 URLs → download images on VPS → store in Supabase
- Agents involved: _summary (IL ops), _summary (deal ingestion)
- Failure mode: Mac impaired/offline → IL scraping fails with SSH timeout. Recovery: reboot via AWS Console → verify Tailscale → retry. No fallback available (VPS IP blocked).
- Success criteria:
ssh -i ~/.ssh/openclaw-mac.pem ec2-user@100.123.248.46 "uptime"returns; IL scraping returns >0 images for a known deal ID.
Cross-links
Agents that touch this
- _summary — InvestorLift operations, infrastructure monitoring
- _summary — consumes IL deal data for acquisition pipeline
Skills that invoke this
- il-marketplace-pull — wraps IL scraping + enrichment; MUST go through Mac (MANDATORY)
Plans that govern this
- openclaw-obsidian-vault-2026-05-02 — LaunchDaemon added Phase 6.1.2
- osil-il-ai-replication-2026-05-03 — B13 displacement track (OSIL IL AI replication)
- project_vps_reboot_5_internal_cascade_F1F4_2026-05-01 — VPS incident context
Feedback rules
- reference_mac_hosts_ssh — authoritative Mac host disambiguation (aws-mac vs mac-cloud vs mac-mini)
- reference_il_marketplace_pipeline — IL working curl/cookie pattern; supersedes Playwright
- feedback_il_enrichment_runs_on_mac_ultra — IL enrichment MUST run on Mac Ultra, not VPS
- feedback_no_plaintext_creds — G-NO-PLAINTEXT-CREDS; AWS keys MUST move to op://Aurora/aws/
- feedback_action_gate_violation_repeated — service/instance actions require explicit auth
KB / source docs
- 2026-05-02 — vendor audit report; documents deprecated services (Pinpoint, Connect Voice ID) + re:Invent 2025 launches
- SOURCE MISSING:
workspace/knowledge-base/aws/API.md— no REST API reference created yet. Full AWS API docs at https://docs.aws.amazon.com/
System maps
- infrastructure — full infra topology showing Mac Ultra
- integrations — IL → Mac → S3 data flow
Related: Infra/compute cluster
This hub is part of the Infra/compute cluster:
- hetzner — primary VPS substrate (the execution environment for all agents)
- cloudflare — public edge, tunnel, WAF; protects the VPS
- github — vault backup;
traewayrer/openclaw-vaultprivate repo
Mandatory cross-refs (PF-A):
- aws → investorlift: Mac Ultra is a hard dependency for all IL scraping. No IL data without the Mac.
- aws → 1password: root key rotation P0 pending; target state is
op://Aurora/aws/{access-key-id,secret-key}.
Open issues / TODOs
- P0 — ROOT KEY ROTATION: AWS keys in
master.envare plaintext. Rotate now, store in 1Password, remove from master.env. See above. - P0 — Mac impaired: Reboot
openclaw-mac-ultravia AWS Console. All IL workflows blocked until resolved. - AWS API.md missing:
workspace/knowledge-base/aws/has only_audit/2026-05-02.md. No API reference doc. Create and add to G-KB-SYNC-WITH-CLAUDEMD platforms list. - EBS snapshot: no automated snapshot confirmed for
openclaw-mac-ultra. Mac instances have 24h minimum billing — confirm EBS backup policy before any termination. - IMDSv2: AWS audit 2026-05-02 recommends enforcing IMDSv2 on EC2 instances to mitigate metadata attack vectors.
- Cookie refresh automation:
investorlift-refresh-cookies.jsexists but no monitoring on cookie expiry. Stale cookies = silent scraping failure.
Recent activity
- 2026-05-03: hub created (W1-S7 sub-agent)
- 2026-05-02 22:15 UTC: Mac Ultra entered
impairedstate; paused pending AWS Console reboot - 2026-05-02: Root key rotation identified as P0 (keys used in session, plaintext in master.env)
- 2026-05-02:
com.openclaw.tailscaledLaunchDaemon installed (Plan §Phase 6.1.2) - 2026-05-01: Reboot #5 VPS cascade — Mac Ultra was unaffected; confirmed IL scraping depends solely on Mac availability