Quartz 4

SKILL

3 min read

Skill: dispo-learning-rollup

Pulls all dispo SMS feedback signals and runs LLM analysis to produce ranked improvement recommendations, blind spots, and quick wins. Posts to Slack. Runs automatically every 6h via systemd timer, or on demand here.

Trigger

User says any of:

  • /dispo-learning-rollup
  • “run the learning rollup”
  • “what does the system need to improve”
  • “pull feedback analysis”
  • “what are we getting wrong in dispo SMS”
  • “show me the rollup”
  • “dispo improvement report”

Usage

# Default: last 6 hours, posts to Slack, writes to DB
node /home/opsadmin/.openclaw/workspace/scripts/dispo-learning-rollup.js
 
# Custom window
node /home/opsadmin/.openclaw/workspace/scripts/dispo-learning-rollup.js --window-hours 24
 
# Dry run (no Slack, no DB write) — use for ad-hoc review
node /home/opsadmin/.openclaw/workspace/scripts/dispo-learning-rollup.js --dry-run
 
# JSON output for skill consumption
node /home/opsadmin/.openclaw/workspace/scripts/dispo-learning-rollup.js --json --dry-run

What it analyzes

Feedback signals (draft_feedback_log):

  • Original draft vs revised draft comparison
  • Score before and after feedback (delta = improvement signal)
  • Feedback category (missing_context, wrong_intent_signal, persona_drift, irrelevant_content, tone_mismatch, wrong_mode, incomplete_answer)
  • Feedback round number (1st correction vs 3rd correction = different severity)

Approval metrics (message_approvals):

  • Score distribution and trend
  • Finish=length occurrences (Kimi token truncation)
  • Low-score drafts approved by human anyway
  • Context gaps (drafts generated with no thread, no deal context)
  • Expired approvals (drafted but never sent)
  • Multi-round feedback conversations (3+ rounds = systemic failure)

Error patterns (webhook_events):

  • aurora_regen_failed: what feedback triggered failures
  • Pool saturation events
  • Rate by error type

Buyer behavior (salesmsg_inbox):

  • Reply rate within 4h of approved drafts
  • Engagement after revision vs first-attempt drafts

Output

  1. Top failure categories — ranked by frequency, each with root cause + exact fix (file + what to change)
  2. Systemic issues — cross-conversation patterns
  3. Blind spots — what should be measured/handled but isn’t
  4. Quick wins — <30min fixes with high impact
  5. Score trend — improving/declining/stable
  6. Health verdict — one sentence

Posted to: Slack dispo-a-socal (DISPO_TEAM) Stored in: system_improvement_log table (queryable for trend analysis)

Query past rollups

SELECT created_at, window_hours, health_verdict, score_trend,
       avg_score_before, avg_score_after,
       feedback_count, approval_count
FROM system_improvement_log
ORDER BY created_at DESC
LIMIT 10;

Timer status

systemctl --user status dispo-learning-rollup.timer
systemctl --user list-timers | grep rollup
# Next fire time + last run

Files

  • Script: /home/opsadmin/.openclaw/workspace/scripts/dispo-learning-rollup.js
  • Log: /home/opsadmin/.openclaw/logs/dispo-learning-rollup.log
  • Tables: draft_feedback_log, system_improvement_log
  • Slack fn: scripts/lib/slack-notify.js → notifyLearningRollup()
  • Timer: ~/.config/systemd/user/dispo-learning-rollup.timer

When to use on-demand

  • After a session with heavy feedback corrections → run 1h window to see patterns
  • Before modifying the system prompt → run 24h window to see what Henry has been correcting
  • After a deploy → run 6h window to confirm scores aren’t degrading
  • Weekly strategic review → run 168h window for full week patterns

Graph View

Backlinks