1. Overview: The O×A Framework
AlwaysAI uses a dual-scoring architecture that evaluates both what you produce (Outcome) and how you collaborate with AI (AI Collaboration). This reflects the reality that in modern work, both the quality of your thinking and your ability to leverage AI tools matter.
Three Score Components
| Component | What It Measures | How It's Assessed |
|---|
| Outcome | Quality of your final work product | AI evaluates your submission against task-specific rubric criteria |
| AI Collaboration | How effectively you used AI as a thinking partner | Analyzed from your chat interactions — prompt quality, iteration patterns, critical thinking |
| Interaction | Coherence between your AI use and outcome quality | Computed from the alignment between Outcome and AI scores |
Assessment Philosophies
Different courses weight these components differently depending on learning goals:
| Philosophy | Outcome | AI Collaboration | Interaction | Best For |
|---|
| Professional | 50% | 30% | 20% | Workplace readiness |
| Formative | 30% | 50% | 20% | Learning & skill building |
| Certification | 60% | 20% | 20% | High-stakes exams |
| AI Proficiency | 20% | 60% | 20% | AI fluency courses |
Example: In a Formative assessment, a student who writes a mediocre submission (Outcome: 40%) but demonstrates excellent AI collaboration skills (AI: 85%) and good coherence (Interaction: 75%) would earn:
Total = 0.30 × 40 + 0.50 × 85 + 0.20 × 75 = 12 + 42.5 + 15 = 69.5%
2. AI Interaction Score 6 Dimensions
Your AI Interaction score measures how skillfully you collaborate with AI during any activity. It is computed automatically by analyzing every chat message you send.
Prompt Precision 20%
Are your prompts clear, specific, and well-contextualized?
What we look for: Detailed questions, specific terminology, edge cases mentioned, context provided.
Low: "Help me with this"
High: "Can you explain how the trade-off between recall and precision affects our model when the dataset is heavily imbalanced?"
Iteration Depth 20%
Do you refine and build on AI responses, or accept the first answer?
What we look for: Follow-up questions, requests for elaboration, building on previous answers, exploring alternatives.
Low: Single question, no follow-up
High: "That's helpful. Now what about the edge case where...?" → "Can you compare this to the approach we discussed earlier?"
Critical Evaluation 20%
Do you verify, question, and challenge AI outputs?
What we look for: Checking correctness, spotting errors, requesting proof, expressing healthy skepticism.
Low: Accepting all AI responses without question
High: "Are you sure about that formula? I think the denominator should be n-1 for sample variance, not n."
Integration/Synthesis 20%
Do you combine AI insights with your own knowledge?
What we look for: References to personal experience, domain knowledge, connecting AI output to course concepts.
Low: Using AI output verbatim
High: "Based on what you explained and my experience with our project's codebase, I think we should adapt this approach by..."
Metacognitive Awareness 10%
Do you reflect on when and why AI helps or falls short?
What we look for: Understanding AI limitations, recognizing where human judgment is needed.
Low: No awareness of AI's role
High: "This is useful for technical analysis, but I'll need my own judgment for the ethical implications since AI might miss cultural nuances."
Efficiency 10%
Do you use AI interactions wisely, neither too few nor too many?
What we look for: Purposeful interactions (optimal: 2–6 per activity), each adding value.
Low: 0–1 interactions (underuse) or 10+ repetitive exchanges
High: 3–5 focused exchanges that systematically build understanding
AI Interaction Score = (Prompt Precision × 0.20) + (Iteration Depth × 0.20) + (Critical Evaluation × 0.20) + (Integration × 0.20) + (Metacognitive × 0.10) + (Efficiency × 0.10)
How it's computed: The system analyzes every prompt you send to the AI. It counts keywords and patterns that indicate each dimension. For example, words like "verify", "check", "are you sure" contribute to Critical Evaluation; words like "also", "building on", "what about" contribute to Iteration Depth. Each dimension is scored 0–100, then weighted.
3. Learning Activity Score 5 Dimensions
Learning activities (PBL, Project-based, Case-based) are multi-session journeys where you explore topics, work through phases, and build understanding over time. Your score reflects the quality of your learning process, not just the final output.
How Learning Activities Work
- Start a journey: Begin with defined phases (e.g., Research → Analyze → Design → Reflect).
- Work through sessions: You can pause and resume across multiple sessions. Each tracks your AI interactions, notes, and phase progress.
- AI assistance: Use the AI chat to explore concepts, ask questions, and get guidance. Interaction quality is tracked.
- Complete phases: Progress through each phase by completing deliverables, notes, and quiz questions.
- Final scoring: When you complete the journey, the system evaluates your entire learning process across all sessions.
Learning Score Components
| Component | PBL | Project | Case | What It Measures |
|---|
| Learning Process | 35% | 25% | 40% | Depth of exploration, refinement, conceptual growth, synthesis |
| AI Collaboration | 35% | 30% | 35% | Quality of AI interaction (same 6 dimensions) |
| Engagement | 15% | 20% | 10% | Session consistency, persistence, time investment |
| Outcome | 15% | 25% | 15% | Phase completion quality |
Learning Total = (Process × weight) + (AI Collab × weight) + (Engagement × weight) + (Outcome × weight)
The 5 Learning Process Dimensions
Phase Progression 20%
How well did you move through the learning phases? Did you follow the intended sequence while also revisiting earlier phases when deeper understanding was needed?
Indicators: Phase completion rate, revisitation patterns, transition quality.
Session Management 15%
Did you space your learning sessions effectively? Consistent, productive sessions show better learning habits than cramming.
Indicators: Session spacing, productivity per session, consistency.
Engagement Depth 25%
How deeply did you engage with the material? Quality of notes, deliverables, and time investment matter more than speed.
Indicators: Note quality, deliverable content, time per phase, exploration depth.
Reflective Practice 20%
Did you reflect on your own learning? Self-assessment, goal-setting, and metacognition demonstrate mature learning.
Indicators: Reflection quality, self-assessment, learning goal articulation.
Knowledge Construction 20%
Did you build on prior sessions, connecting new ideas to what you already learned? This shows genuine understanding.
Indicators: Cross-session references, concept connections, synthesis across phases.
How it's computed: When your journey is complete, an AI evaluator reviews all your sessions, interactions, notes, and phase deliverables holistically. Each dimension is scored 0–100 based on evidence found in your work — concrete behaviors and patterns, not just keyword matching.
4. Assessment Score 5 Dimensions
Assessments test your ability to produce quality work with AI assistance. You receive a task with specific requirements, use AI as a collaborator, and submit your work for evaluation.
How Assessments Work
- Receive a task: Each assessment has a defined problem, requirements, and evaluation rubric.
- Use AI strategically: Chat with AI to research, brainstorm, draft, and refine. How you use AI matters.
- Complete structured sections: Fill in response sections addressing different aspects of the task.
- Submit for evaluation: Your submission is evaluated by AI against the task rubric and your AI collaboration is analyzed.
- Receive scores: You get Outcome, AI Collaboration, and Interaction scores with detailed feedback.
Outcome Scoring
For each rubric criterion, the AI evaluator assesses:
- Score: 0 to maximum points for that criterion
- Feedback: What you did well and what could improve
- Evidence: Specific quotes from your submission
Total outcome: (Sum of criterion scores / Maximum possible) × 100
The 5 Assessment Process Dimensions
Task Comprehension 20%
Did you understand what the task required before diving in? Evidence includes readiness quiz performance, initial planning, and alignment of work with requirements.
Workflow Discipline 20%
Did you follow a systematic approach? Working through sections in order, completing each step, following the suggested workflow.
Revision Quality 20%
Did you revise and improve your work? The system tracks how you used AI editing tools (compose, improve, expand, shorten) and whether revisions improved quality.
Independent Thinking 20%
Did you bring your own reasoning, or just copy AI output? Step notes, personal reflections, and critical engagement demonstrate independent thinking.
Time Management 20%
Did you balance your time across sections effectively? Spending too long on one section at the expense of others indicates poor management.
How it's computed: An AI evaluator reviews your complete assessment process: readiness quiz, workflow progression, step notes, revision patterns, and time distribution. Each dimension is scored 0–100 based on evidence from your work.
5. Your Report Card Interactive
The Report Card is your comprehensive dashboard for understanding every aspect of your performance. It brings together Learning Process, Assessment Process, and AI Interaction scores for each week or lesson, with detailed explanations and actionable improvement tips — all in one place.
Three Category Cards per Week
When you expand a week in your Report Card, you see three scored categories:
Learning Process
Your overall learning score, broken into 4 tappable sub-scores (Process, AI Collab, Engagement, Outcome) and 5 detailed process dimensions. Each sub-score shows how it was calculated and how to improve.
Assessment Process
Your assessment workflow and answer quality, broken into 5 process dimensions (Task Comprehension, Workflow Discipline, Revision Quality, Independent Thinking, Time Management) plus the rubric-based outcome score.
AI Interaction Quality
Your AI collaboration effectiveness, showing 6 tappable dimension scores (Prompt Precision, Iteration Depth, Critical Evaluation, Integration, Metacognitive Awareness, Efficiency) drawn from both learning and assessment activities.
Tappable Score Explanations
Every score in the Report Card is interactive — designed for both desktop and touch devices (iPad, phone). Tap any score to see a detailed popup:
What You See When You Tap a Score
| Section | Description |
|---|
| What This Measures | Plain-English explanation of what the score evaluates |
| How It's Scored | The method used — AI evaluation, keyword analysis, or session timing patterns |
| How to Improve | 4 numbered, actionable tips with concrete examples you can apply immediately |
| Your Score Means | Contextual interpretation — Excellent (≥80), Good (60–79), Developing (40–59), or Needs Work (<40) |
Example: Tapping "AI Collab: 58" in the Learning Process card opens a popup explaining that AI Collaboration measures how effectively you partnered with AI, that it's scored across 6 dimensions based on your chat interactions, and gives you 4 specific tips like "Ask specific, well-structured questions rather than vague ones" and "Follow up on AI responses — build on them, challenge them, ask for alternatives."
Dimension Drill-Down
Each of the 5 Learning Process dimensions and 5 Assessment Process dimensions is also tappable. Tap a dimension row (like "Phase Progression: 45") to open a detailed drill-down showing:
- Score & Rating: Numerical score (0–100) with rating level (Minimal, Below Expectations, Meets Basic, Good, Excellent)
- Summary: 2–3 sentence assessment of your work in this dimension
- Evidence: Specific quotes from your notes, deliverables, and AI conversations
- Strengths: What you did well
- How to Improve: Actionable steps to raise this score
Question Any Score with AI
At the bottom of every dimension drill-down, there's a text input where you can ask AI about the score. This is your chance to understand why you received that rating:
You ask: "Why did I only get 25 for Session Management when I had 8 sessions?"
AI responds: "While you opened 8 sessions, most were very short (under 2 minutes) with little progress. Session Management measures productive, spaced learning — not just session count. Your sessions lacked consistency and showed minimal phase advancement between them."
Discuss feature: The AI discussion is multi-turn — you can ask follow-up questions. All discussions are saved so you can review them later. Both teachers and students have access to the same discussions, maintaining full transparency.
View Details Popup
Each category card has a "View Details" link that opens a full-screen popup with three tabs:
| Tab | What It Includes |
|---|
| Learning | Learning Process score, 4 tappable sub-scores, 5 process dimension bars with summaries/strengths/improvements, full verbal analysis (expandable) |
| Assessment | Assessment Process score, Outcome score, 5 process dimension bars with summaries/strengths/improvements, full verbal analysis (expandable) |
| AI Interaction | AI Collaboration score, 6 tappable dimension breakdown, AI verbal analysis from both learning and assessment, total chat count |
Semester Summary
At the bottom of the Report Card, a Semester Summary aggregates your performance across all weeks:
- Overall Score & Grade: A single number (0–100) and letter grade (A–F) for the semester
- Trajectory: Whether you're improving, stable, declining, or inconsistent over time
- Narrative: An AI-written summary of your semester performance covering all three categories
- Strengths: Top 3 strengths identified across all weeks
- Areas for Growth: Key concerns that need attention
- Recommendations: Actionable next steps tailored to your performance patterns
Generating Summaries
Important: The Report Card displays pre-computed summaries — it does not regenerate scores each time you open it. To generate a summary, open the learning activity or assessment and click the Progress Summary or Assessment Summary button. Once generated, summaries are stored and appear instantly in the Report Card.
6. How Final Scores Are Calculated
Assessment Final Score
Final Score = (Outcome Weight × Outcome) + (AI Weight × AI Collaboration) + (Interaction Weight × Interaction Score)
The Interaction Score rewards coherence between outcome and AI collaboration:
- Gap < 15 points: +20 bonus (strong alignment)
- Gap 15–30 points: +10 bonus (moderate alignment)
- Gap > 30 points: no bonus (misalignment)
Interaction Score = min(100, (Outcome + AI) / 2 + Coherence Bonus)
Example: (Formative): Outcome: 70%, AI: 65%, Gap = 5 (<15 → +20 bonus)
Interaction = min(100, 67.5 + 20) = 87.5%
Final = 0.30×70 + 0.50×65 + 0.20×87.5 = 21 + 32.5 + 17.5 = 71%
Learning Activity Final Score
Final Score = (Process × weight) + (AI Collab × weight) + (Engagement × weight) + (Outcome × weight)
Weights vary by activity type (PBL, Project, or Case-based) as shown in section 3.
7. Class Standing & Percentiles
The "Where I Stand in Class" feature shows how your scores compare to classmates using statistical percentiles across all 16 dimensions.
How It Works
- The system collects scores from all students in your class for the selected week.
- For each dimension, it computes the class average and spread (standard deviation).
- Your position is a z-score: how many standard deviations above or below average.
- This converts to a percentile: "Top X%" means X% of students scored lower.
Performance Tiers
| Tier | Percentile | Meaning |
|---|
| Elite | Top 10% | Outstanding — among the highest performers |
| Advanced | Top 30% | Above average — consistently strong |
| Proficient | Top 60% | Meets expectations — solid work |
| Developing | Top 90% | Progressing — room for improvement |
| Foundational | Bottom 10% | Early stage — focused support recommended |
Four Aggregate Scores
| Aggregate | What It Includes | Source |
|---|
| AI Score | Weighted average of 6 AI dimensions | Best learning or assessment |
| Learning Score | Total learning score (all 4 components) | Best completed learning activity |
| Assessment Score | Assessment outcome score | Best submitted assessment |
| Overall Score | Combined weighted total | Best submitted assessment |
Minimum requirement: Class standings require at least 5 students with scores. Rankings use the best attempt if you have multiple submissions.
8. Diagnostic Patterns
Assessment Diagnostics
| Pattern | Outcome | AI | What It Means |
|---|
| Effective Augmentation | ≥60% | ≥60% | Strong work through skilled AI collaboration |
| Underutilized AI | ≥60% | <60% | Good outcome, AI potential not fully leveraged |
| Process Strong, Domain Gap | <60% | ≥60% | Good AI skills, domain knowledge needs work |
| Foundational | <60% | <60% | Both areas need development |
Learning Diagnostics
| Pattern | Conditions | Recommendation |
|---|
| Master Learner | Process ≥60, AI ≥60, Engagement ≥60 | Ready for advanced challenges |
| Intensive Learner | Process ≥60, AI ≥60, Engagement <60 | Develop regular learning habits |
| Independent Learner | Process ≥60, AI <60, Engagement ≥60 | Experiment with more AI collaboration |
| Process Developing | Process <60, AI ≥60, Engagement ≥60 | Focus on deeper exploration |
| Growth Trajectory | Improving trend across sessions | Continue your momentum |
| Building Foundations | Other | Structured guidance recommended |
9. Multiple Attempts & Best Score
How It Works
- After submitting, you'll see a "Try Again" button to start a new attempt.
- Each attempt is independent — fresh workspace, different approach possible.
- All attempts are visible in My Learning, numbered (#1, #2, etc.).
- The attempt with the highest total score is marked with a "Best" badge.
- Class rankings and analytics use only your best attempt.
Example: You submit Assessment L1 twice:
Attempt #1: Outcome 45%, AI 60%, Total 52%
Attempt #2: Outcome 72%, AI 65%, Total 68% Best
Your grade and ranking will use 68%.
Score Scale Reference
| Range | Level | Description |
|---|
| 80–100 | Excellent | Exceptional work demonstrating mastery |
| 60–79 | Good | Solid work meeting expectations with strengths |
| 40–59 | Developing | Adequate work with clear areas for improvement |
| 0–39 | Foundational | Significant gaps — focused support needed |
For teachers: Average student work should typically score 40–60. Scores above 70 indicate genuinely strong work. The system is calibrated to be realistic, not inflated.
10. Frequently Asked Questions
Can I see exactly how my score was calculated?▼
Yes. Open your Report Card from the module page to see all your scores organized by week. Every score is tappable — tap any sub-score (Process, AI Collab, Engagement, Outcome) or AI dimension to see a popup explaining what it measures, how it's scored, and how to improve. Tap any process dimension row for a drill-down with evidence quotes, strengths, and improvement tips. You can even ask AI to explain why you got a specific score. Both teachers and students see the same Report Card — nothing is hidden.
Does the AI read my entire submission?▼
Yes. When you submit, an AI evaluator reads your full submission, all your structured section inputs, your step notes, your complete chat history with the AI assistant, and any revision actions you took. It evaluates each rubric criterion individually and provides specific feedback with quotes from your work as evidence.
Is it better to use AI a lot or a little?▼
Quality over quantity. The optimal range is 2–6 purposeful AI interactions per activity. Too few (0–1) suggests you didn't leverage AI; too many (10+) may indicate unfocused use. What matters most is how you interact: asking specific questions, iterating on responses, thinking critically, and integrating AI insights with your own knowledge.
Will I get a higher score if I just copy what the AI says?▼
No. The scoring system specifically measures independent thinking and critical evaluation. If you copy AI output without engaging critically, your Integration/Synthesis and Independent Thinking scores will be low. The best scores come from using AI as a thinking partner — challenging its responses, combining its output with your own ideas, and reflecting on its limitations.
Can I redo an assessment to improve my score?▼
Yes. After submitting, you can click "Try Again" to start a fresh attempt. Each attempt is scored independently. The system tracks all your attempts and uses your highest total score for your class ranking and analytics. All attempts remain visible in your My Learning view so you can compare your progress.
What happens if I run out of time?▼
The timer shows a suggested duration, not a hard limit. When the time expires, the timer switches to an overtime indicator and you can continue working. The actual time you spend is recorded for analytics, but you won't be locked out or auto-submitted. Take the time you need to do your best work.
What is the "Interaction Score" and why does it matter?▼
The Interaction Score measures coherence — whether your AI collaboration actually helped produce a better outcome. If your Outcome and AI Collaboration scores are close (within 15 points), you get a bonus because it means you used AI effectively to enhance your work. A big gap suggests either you didn't apply AI insights to your work, or your AI interaction didn't address the task requirements.
How is "Where I Stand in Class" calculated?▼
Your position is calculated using z-scores (a standard statistical method). For each of the 16 dimensions, the system computes the class average and spread. Your score is then expressed as a percentile — "Top 30%" means 70% of students scored lower than you on that dimension. Requires at least 5 students with scores to be meaningful. Your identity is never revealed to other students.
Are learning activities and assessments scored differently?▼
Yes. Learning activities emphasize process — how you explored, engaged, and grew over multiple sessions. Assessments emphasize outcome — the quality of your final submission evaluated against a rubric. Both include AI collaboration scoring. Learning uses 4 components (Process, AI, Engagement, Outcome), while assessments use 3 (Outcome, AI Collaboration, Interaction).
Can I discuss my scores with my teacher?▼
Absolutely. Transparency is a core design principle. Both you and your teacher see the exact same Report Card with identical scores, breakdowns, and AI feedback. In the Report Card, tap any dimension to open the drill-down, then use the "Question this Rating" chat at the bottom to ask the AI evaluator why you received that score. The AI will respond with specific evidence from your work. These discussions are saved and visible to both you and your teacher, making them a great starting point for in-person conversations.
What does it mean if my AI Score is high but my Outcome is low?▼
This is the "Process Strong, Domain Gap" pattern. It means you're skilled at using AI as a thinking partner (good prompts, critical evaluation, iteration), but the subject matter quality of your final work needs improvement. Focus on deepening your domain knowledge — use the AI to help you understand concepts better, not just to produce output.
Is my data private?▼
Your submissions, chat history, and detailed scores are visible only to you and your teachers. Class rankings show your percentile position without revealing individual student identities to other students. All data is stored securely and used solely for educational assessment purposes.
How do I open my Report Card?▼
Navigate to your module page and click the Report Card button. The Report Card streams in your scores week by week. You can toggle between "By Week" and "By Lesson" views. If you're viewing a specific week's learning activity or assessment, you can also open a focused Report Card for just that week.
Why does my Report Card show "No summaries generated yet"?▼
The Report Card displays pre-computed summaries, not live calculations. You need to generate summaries first by opening each activity and clicking the Progress Summary button (for learning) or Assessment Summary button (for assessments). Once generated, summaries are stored permanently and appear instantly in the Report Card every time you open it.
What do the scores in the Report Card mean in terms of grades?▼
Each week receives a letter grade based on its combined score: A (90+), B (80–89), C (70–79), D (60–69), F (below 60). The Semester Summary at the bottom of the Report Card shows your overall semester grade and trajectory. Scores are color-coded throughout: green (≥80), yellow (60–79), orange (40–59), and red (<40).
Can I print my Report Card?▼
Yes. Click the Print button in the Report Card header. This generates a clean, formatted printout including all your week/lesson scores, letter grades, and the Semester Summary. You can print it or save it as a PDF for your records.