# ARC Database & Interface TODO

## ✅ Completed

### ConceptARC Corpus
- [x] Add ConceptARC corpus to repository
- [x] Upload ConceptARC to database (160 puzzles, 16 categories)
- [x] Categorize puzzles in DB (stored in `set` column)
- [x] Fix solution extraction for multiple test cases (array of solutions)
- [x] Remove duplicate entries from arc_puzzles table (260 duplicates cleaned)
- [x] Verify solution formats across all corpora

## 🔄 In Progress

### Database

#### Fixes
- [x] Corpora Mixup (V1 vs V2)
- [x] Make id the index

#### Views & Queries
- [x] Create database view with skills/concepts mapping
  - [x] Map ConceptARC categories to skill types


#### Data Quality
- [x] Verify all V1/V2 puzzles have solutions
- [x] Validate JSON integrity across all puzzles
- [ ] Document database schema

#### Schema Updates
- [x] Add created_at timestamps to relevant tables
- [x] Create N-to-N tags table structure
  - [x] Create tags table
  - [x] Create arc_puzzle_tags junction table

### Retool Interface

#### Core Features
- [x] Update puzzle renderer to handle ConceptARC multiple solutions
  - [x] Show all 3 test outputs for ConceptARC
- [x] Remove header frame, consolidate to single-frame interface
- [ ] Add puzzle filtering by:
  - [ ] Corpora (V1, V2, ConceptARC)
  - [ ] Category/Set
  - [ ] Difficulty
  - [ ] Skills/Concepts

#### User Features
- [x] Implement user input system
  - [x] Design input capture mechanism
  - [x] Store user solutions in database
- [ ] Create puzzle assignment system
  - [ ] Add assignment toggle in interface
  - [ ] Assign puzzles to specific users
  - [ ] Track completion status
  - [ ] Record time spent on puzzles

### Tag System
- [x] Design tag taxonomy
  - [x] Define tag types (concept, difficulty, skill, etc.)
- [x] Implement tag database schema
  - [x] Create tags table
  - [x] Create puzzle_tags junction table
  - [x] Add tag management endpoints
- [x] Build tag UI in Retool
  - [x] Tag creation/editing interface
  - [ ] Bulk tagging functionality
  - [ ] Tag-based search/filtering
  - [x] Test skill selection with many items (search function?)
- [ ] Populate initial tags
  - [ ] Auto-tag ConceptARC by category
  - [ ] Tag V1/V2 by known concepts

### Administration
- [ ] Publish Retool app to production

## 📋 Backlog

### Documentation
- [ ] Create user guide for Retool interface
- [ ] Document API/database schema

### Testing
- [ ] Test puzzle rendering across all corpora
- [ ] Validate solution checking logic
- [ ] Test multi-user concurrent access
- [ ] Performance testing with full dataset



## 📝 Notes

### Database Stats (Current)
- Total puzzles: 2,400
  - V1: 1,120
  - V2: 1,120
  - ConceptARC: 160 (16 categories × 10 puzzles)
- All puzzles have solutions stored
- ConceptARC uses array format (3 solutions per puzzle)

### Scripts Available
- `upload_conceptarc_to_db.py` - Upload ConceptARC puzzles
- `check_schema.py` - Inspect database structure
- `find_duplicates.py` - Find duplicate puzzle entries
- `remove_duplicates.py` - Clean duplicate entries
- `fix_conceptarc_solutions.py` - Fix ConceptARC solution format
- `verify_conceptarc_upload.py` - Verify upload integrity
- `verify_solution_format.py` - Check solution formats