Add scripts/validate-agents.sh for agent validation

2026-02-03 19:23:26 +01:00
parent fddc22e55e
commit 1f320f1c95
2 changed files with 369 additions and 0 deletions
--- a/.sisyphus/notepads/chiron-agent-framework/learnings.md
+++ b/.sisyphus/notepads/chiron-agent-framework/learnings.md
@@ -46,3 +46,205 @@

 ### Files Created
 - `prompts/chiron-forge.txt` - Chiron-Forge build mode system prompt
+
+## Task: Create prompts/calliope.txt
+
+### Calliope Prompt Pattern
+- Purpose: Writing specialist for documentation, reports, meeting notes, and professional prose
+- Identity: Greek muse of epic poetry and eloquence
+- Core distinction: Focuses on eloquence and structure, not technical implementation
+- Second-person addressing: "You are..." format
+
+### Key Components
+- **Identity**: "Greek muse of epic poetry and eloquence, specializing in writing assistance"
+- **Capabilities**: Draft documentation, create reports, transform notes to summaries, professional writing
+- **Workflow**: Understand → Clarify (Question tool) → Gather → Draft → Refine → Review
+- **Quality Standards**: Clarity, logical structure, consistent terminology, accurate representation
+- **Output Format**: Clear hierarchy, bullet/numbered lists, tables, executive summaries, action items
+
+### Scope Boundaries
+- DO NOT execute code or run commands (delegate to technical agents)
+- DO NOT handle short communication (Hermes's domain)
+- DO NOT manage wiki knowledge bases (Athena's domain)
+- DO NOT make factual assertions without verification
+- DO NOT write specialized domain content without input
+
+### Verification
+- File size: 3390 chars (well above 500 minimum)
+- Keywords present: writing, document, report, summaries
+- Lines: 48
+- Follows standard prompt structure from agent-development skill
+
+### Files Created
+- `prompts/calliope.txt` - Calliope writing specialist system prompt (48 lines)
+
+## Task: Create skills/msteams/SKILL.md
+
+### MS Teams Skill Pattern
+- Description format: Clear trigger conditions with numbered use cases
+- Structure: Quick start → Core API capabilities → Common workflows → API endpoint reference → Best practices → Integration examples
+- Scope: Channels, messages, meetings, chat operations
+- Excluded: Authentication flows (handled externally), Outlook email functionality
+
+### Verification
+- YAML frontmatter validation: `python3 -c "import yaml; yaml.safe_load()"`
+- Required fields: name, description, compatibility
+- Body content: >7400 chars, comprehensive coverage
+
+### Files Created
+- `skills/msteams/SKILL.md` - Complete Teams Graph API integration documentation
+
+## Task: Create skills/obsidian/SKILL.md
+
+### Obsidian Skill Pattern
+- Purpose: Document Obsidian Local REST API integration
+- API base URL: http://localhost:27124
+- Core capabilities: vault operations, note CRUD, search, daily notes
+
+### SKILL.md Requirements
+- YAML frontmatter with quoted description when using special characters
+- Required fields: name, description, compatibility: opencode
+- Description must include "Use when:" triggers and "Triggers:" examples
+- Body follows markdown structure with ATX headers (#, ##, ###)
+
+### Key Sections
+1. Quick Start - API overview and base URL
+2. Core Workflows - Step-by-step API operations
+3. Note Structure Patterns - Frontmatter and WikiLink formats
+4. Search Patterns - Tag-based, path-based, combined searches
+5. Integration with Other Skills - Handoff triggers to other skills
+6. Error Handling - HTTP status codes
+7. Best Practices - Usage guidelines
+8. Example Workflows - Practical curl examples
+
+### YAML Frontmatter Gotchas
+- Must quote description if it contains colons, parentheses, or other special YAML characters
+- Example: description: "Text with (parentheses) and colons: like this"
+- Without quotes: causes YAML parsing errors
+
+### Obsidian Local REST API Endpoints
+- GET /vault/ - List all files
+- GET /active/ - Get active note
+- POST /notes/ - Create note
+- GET /notes/{path} - Read note
+- PUT /notes/{path} - Update note
+- DELETE /notes/{path} - Delete note
+- GET /search/simple - Search content
+- POST /search/ - Advanced search
+- POST /daily/note - Create daily note
+- GET /daily/note - Get daily note
+
+### Frontmatter Pattern
+```yaml
+---
+date: YYYY-MM-DD
+created: ISO8601_timestamp
+type: note_type
+status: draft|final|archived
+tags: #tag1 #tag2
+---
+```
+
+### WikiLink Format
+- [[Link]] - Basic link
+- [[Link|Alias]] - Link with alias
+- [[#Section]] - Section link
+- [[Note#Section]] - Section in other note
+
+### Search Query Patterns
+- tag:#tagname - Tag search
+- path:directory/ - Path restriction
+- Combined with contextLength parameter
+
+### Files Created
+- skills/obsidian/SKILL.md (250 lines, 6443 bytes)
+
+## Task: Create skills/outlook/SKILL.md
+
+### Outlook Skill Pattern
+- Purpose: Document Outlook Graph API integration via MCP
+- Core capabilities: Mail CRUD, folder management, calendar events, contacts, search
+- Excluded: Graph API authentication (handled externally), Teams functionality
+
+### SKILL.md Structure for API Skills
+- YAML frontmatter with clear "Use when:" triggers and "Triggers:" examples
+- Description follows pattern: "API integration via MCP. Use when: (1) ..., (2) ... Triggers: '...', '...', '...'"
+- Body sections: Core Operations → Common Workflows → IDs and Discovery → Important Notes
+
+### API Documentation Pattern
+- Group operations by domain (Mail Access, Mail CRUD, Folder Management, Calendar Events, Contacts, Search)
+- Each operation format: **Function name**: `function_name(params)` - Brief description
+- Include parameter defaults: `param=value` in function signatures
+- Document common workflows with step-by-step examples
+
+### Verification
+- YAML frontmatter validation: `python3 skills/skill-creator/scripts/quick_validate.py skills/outlook/`
+- Required fields: name, description, compatibility: opencode
+- File size: 7572 chars (comprehensive but not excessive)
+
+### Files Created
+- skills/outlook/SKILL.md (160 lines, 7572 bytes)
+## Task: Create scripts/validate-agents.sh
+
+### Validation Script Pattern
+- Bash script follows existing test-skill.sh conventions
+- Shebang: `#!/usr/bin/env bash`
+- Strict mode: `set -euo pipefail`
+- Color output: RED, GREEN, YELLOW, NC (no color)
+- Helper functions for specific checks
+- Main case statement for option handling
+
+### Script Structure
+```bash
+# Setup
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+REPO_ROOT="$(dirname "$SCRIPT_DIR")"
+
+# Colors
+RED='\033[0;31m'
+GREEN='\033[0;32m'
+YELLOW='\033[1;33m'
+NC='\033[0m'
+
+# Functions
+check_json_valid() - Validates JSON syntax
+check_required_fields() - Validates agent structure
+check_prompt_file() - Verifies prompt file exists and non-empty
+
+# Main case statement handles --help, default validation
+```
+
+### Agent Validation Checks
+1. JSON validity: `python3 -m json.tool`
+2. Required fields: description, mode, model, prompt, permission
+3. Prompt file extraction: Regex from `{file:./prompts/filename}` format
+4. File existence: Check prompt file exists in prompts/ directory
+5. Non-empty check: Use `-s` test operator
+
+### Regex Pattern for Prompt References
+```bash
+[[ $prompt_ref =~ \{file:./prompts/([^}]+)\} ]]
+# BASH_REMATCH[1] contains the filename
+```
+
+### Python Integration in Bash Scripts
+- Validate JSON: `python3 -m json.tool file`
+- Parse JSON: `python3 -c "import json; json.load(open('file'))"`
+- Extract data: `python3 -c "import json; data = json.load(open('file')); print('\n'.join(data.keys()))"`
+- Check fields: `python3 -c "import sys, json; data = json.load(sys.stdin); exit(0 if 'field' in data else 1)"`
+
+### Comment Discipline
+- Keep file header documentation (usage, purpose, checks)
+- Remove inline comments for self-explanatory code (function names, clear operations)
+- Keep comments for complex/regex patterns
+- Follow existing repository conventions (e.g., "# Main" section marker)
+
+### Files Created
+- `scripts/validate-agents.sh` - Bash validation script (130 lines)
+
+### Verification
+- Script executable: `chmod +x`
+- Runs successfully: Validates all 6 agents
+- Error handling: Detects missing files, invalid JSON, missing fields
+- Help documentation: `--help` flag
+
--- a/scripts/validate-agents.sh
+++ b/scripts/validate-agents.sh
@@ -0,0 +1,167 @@
+#!/usr/bin/env bash
+#
+# Validate agents.json structure and prompt files
+#
+# Usage:
+#   ./scripts/validate-agents.sh              # Validate agents.json
+#   ./scripts/validate-agents.sh --help       # Show help
+#
+# Checks:
+#   - agents.json is valid JSON
+#   - Each agent has required fields (description, mode, model, prompt, permission)
+#   - All referenced prompt files exist
+#   - All prompt files are non-empty
+
+set -euo pipefail
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+REPO_ROOT="$(dirname "$SCRIPT_DIR")"
+
+RED='\033[0;31m'
+GREEN='\033[0;32m'
+YELLOW='\033[1;33m'
+NC='\033[0m'
+
+usage() {
+    echo "Usage: $0 [OPTIONS]"
+    echo ""
+    echo "Validate agents.json structure and prompt files."
+    echo ""
+    echo "Options:"
+    echo "  --help      Show this help message"
+    echo ""
+    echo "Validates:"
+    echo "  - agents.json is valid JSON"
+    echo "  - Each agent has required fields"
+    echo "  - All referenced prompt files exist"
+    echo "  - All prompt files are non-empty"
+}
+
+check_json_valid() {
+    local agents_file="$1"
+
+    if ! python3 -m json.tool "$agents_file" > /dev/null 2>&1; then
+        echo -e "${RED}❌ agents.json is not valid JSON${NC}"
+        return 1
+    fi
+
+    echo -e "${GREEN}✅ agents.json is valid JSON${NC}"
+    return 0
+}
+
+check_required_fields() {
+    local agents_file="$1"
+    local agent_name="$2"
+    local agent_data="$3"
+
+    local required_fields=("description" "mode" "model" "prompt" "permission")
+    local missing_fields=()
+
+    for field in "${required_fields[@]}"; do
+        if ! echo "$agent_data" | python3 -c "import sys, json; data = json.load(sys.stdin); exit(0 if '$field' in data else 1)" 2>/dev/null; then
+            missing_fields+=("$field")
+        fi
+    done
+
+    if [[ ${#missing_fields[@]} -gt 0 ]]; then
+        echo -e "  ${RED}❌ Missing required fields for '$agent_name': ${missing_fields[*]}${NC}"
+        return 1
+    fi
+
+    echo -e "  ${GREEN}✅ '$agent_name' has all required fields${NC}"
+    return 0
+}
+
+check_prompt_file() {
+    local agent_name="$1"
+    local prompt_ref="$2"
+
+    # Extract filename from {file:./prompts/filename}
+    if [[ ! $prompt_ref =~ \{file:./prompts/([^}]+)\} ]]; then
+        echo -e "  ${RED}❌ '$agent_name': Invalid prompt reference format: $prompt_ref${NC}"
+        return 1
+    fi
+
+    local prompt_file="prompts/${BASH_REMATCH[1]}"
+
+    if [[ ! -f "$REPO_ROOT/$prompt_file" ]]; then
+        echo -e "  ${RED}❌ '$agent_name': Prompt file not found: $prompt_file${NC}"
+        return 1
+    fi
+
+    if [[ ! -s "$REPO_ROOT/$prompt_file" ]]; then
+        echo -e "  ${RED}❌ '$agent_name': Prompt file is empty: $prompt_file${NC}"
+        return 1
+    fi
+
+    echo -e "  ${GREEN}✅ '$agent_name': Prompt file exists and is non-empty ($prompt_file)${NC}"
+    return 0
+}
+
+validate_agents() {
+    local agents_file="$REPO_ROOT/agents/agents.json"
+
+    echo -e "${YELLOW}Validating agents.json...${NC}"
+    echo ""
+
+    if [[ ! -f "$agents_file" ]]; then
+        echo -e "${RED}❌ agents.json not found at $agents_file${NC}"
+        exit 1
+    fi
+
+    check_json_valid "$agents_file" || exit 1
+
+    local agent_names
+    agent_names=$(python3 -c "import json; data = json.load(open('$agents_file')); print('\n'.join(data.keys()))")
+
+    local failed=0
+
+    while IFS= read -r agent_name; do
+        [[ -z "$agent_name" ]] && continue
+
+        echo -n "  Checking '$agent_name': "
+
+        local agent_data
+        agent_data=$(python3 -c "import json; data = json.load(open('$agents_file')); print(json.dumps(data['$agent_name']))")
+
+        if ! check_required_fields "$agents_file" "$agent_name" "$agent_data"; then
+            ((failed++)) || true
+            continue
+        fi
+
+        local prompt_ref
+        prompt_ref=$(python3 -c "import json, sys; data = json.load(open('$agents_file')); print(data['$agent_name'].get('prompt', ''))")
+
+        if ! check_prompt_file "$agent_name" "$prompt_ref"; then
+            ((failed++)) || true
+        fi
+
+    done <<< "$agent_names"
+
+    echo ""
+
+    if [[ $failed -eq 0 ]]; then
+        echo -e "${GREEN}✅ All agents validated successfully!${NC}"
+        exit 0
+    else
+        echo -e "${RED}❌ $failed agent(s) failed validation${NC}"
+        exit 1
+    fi
+}
+
+# Main
+case "${1:-}" in
+    --help|-h)
+        usage
+        exit 0
+        ;;
+    "")
+        validate_agents
+        ;;
+    *)
+        echo -e "${RED}Unknown option: $1${NC}"
+        echo ""
+        usage
+        exit 1
+        ;;
+esac