arc-humans-interface-db/todo.md

ARC Database & Interface TODO

✅ Completed

ConceptARC Corpus

• Add ConceptARC corpus to repository
• Upload ConceptARC to database (160 puzzles, 16 categories)
• Categorize puzzles in DB (stored in set column)
• Fix solution extraction for multiple test cases (array of solutions)
• Remove duplicate entries from arc_puzzles table (260 duplicates cleaned)
• Verify solution formats across all corpora

🔄 In Progress

Database

Fixes

• Corpora Mixup (V1 vs V2)
• Make id the index

Views & Queries

• Create database view with skills/concepts mapping
  • Map ConceptARC categories to skill types

Data Quality

• Verify all V1/V2 puzzles have solutions
• Validate JSON integrity across all puzzles
• Document database schema

Schema Updates

• Add created_at timestamps to relevant tables
• Create N-to-N tags table structure
  • Create tags table
  • Create arc_puzzle_tags junction table

Retool Interface

Core Features

• Update puzzle renderer to handle ConceptARC multiple solutions
  • Show all 3 test outputs for ConceptARC
• Remove header frame, consolidate to single-frame interface
• Add puzzle filtering by:
  • Corpora (V1, V2, ConceptARC)
  • Category/Set
  • Difficulty
  • Skills/Concepts

User Features

• Implement user input system
  • Design input capture mechanism
  • Store user solutions in database
• Create puzzle assignment system
  • Add assignment toggle in interface
  • Assign puzzles to specific users
  • Track completion status
  • Record time spent on puzzles

Tag System

• Design tag taxonomy
  • Define tag types (concept, difficulty, skill, etc.)
• Implement tag database schema
  • Create tags table
  • Create puzzle_tags junction table
  • Add tag management endpoints
• Build tag UI in Retool
  • Tag creation/editing interface
  • Bulk tagging functionality
  • Tag-based search/filtering
  • Test skill selection with many items (search function?)
• Populate initial tags
  • Auto-tag ConceptARC by category
  • Tag V1/V2 by known concepts

Administration

• Publish Retool app to production

📋 Backlog

Documentation

• Create user guide for Retool interface
• Document API/database schema

Testing

• Test puzzle rendering across all corpora
• Validate solution checking logic
• Test multi-user concurrent access
• Performance testing with full dataset

📝 Notes

Database Stats (Current)

• Total puzzles: 2,400
  • V1: 1,120
  • V2: 1,120
  • ConceptARC: 160 (16 categories × 10 puzzles)
• All puzzles have solutions stored
• ConceptARC uses array format (3 solutions per puzzle)

Scripts Available

• upload_conceptarc_to_db.py - Upload ConceptARC puzzles
• check_schema.py - Inspect database structure
• find_duplicates.py - Find duplicate puzzle entries
• remove_duplicates.py - Clean duplicate entries
• fix_conceptarc_solutions.py - Fix ConceptARC solution format
• verify_conceptarc_upload.py - Verify upload integrity
• verify_solution_format.py - Check solution formats