What Is AlwaysAI?
AlwaysAI is an AI-augmented assessment and learning platform that treats AI collaboration as a core educational competency. Instead of banning, ignoring, or detecting AI use, AlwaysAI makes AI collaboration mandatory, visible, and graded in every assessment and learning activity.
The platform implements the O×A Framework (Outcome × AI Collaboration)—a dual-scoring architecture that evaluates both what students produce and how they collaborate with AI to produce it. Every submission receives two scores: a traditional Outcome score and an AI Collaboration score, combined through configurable weights into a final grade.
Two Complementary Modes
Structured Assessment
Evaluate task completion with AI collaboration in a single session. Focuses on outcome quality and AI collaboration quality at a point in time.
Learning Activity
Support extended learning journeys across multiple sessions (days or weeks). Focuses on process and AI collaboration growth over time.
Learning activities span three pedagogical frameworks—Problem-Based Learning (PBL), Project-Based Learning, and Case-Based Learning—each with distinct scoring weights that prioritize learning process alongside AI collaboration.
| Aspect | Structured Assessment | Learning Activity |
|---|---|---|
| Purpose | Evaluate task completion | Support learning journeys |
| Duration | Single session | Multi-session (days/weeks) |
| Primary Focus | Outcome + AI collaboration | Process + AI collaboration growth |
| AI Evaluation | Point-in-time snapshot | Trend analysis across sessions |
| Feedback | Post-submission | Real-time + post-completion |
| Pedagogical Fit | Exams, assignments, projects | PBL, Project-Based, Case-Based |
Why AlwaysAI?
The Problem: A Three-Way Failure
Educational institutions face a paradox. AI tools are ubiquitous, yet assessment systems were designed for a world without them. The three dominant responses have all failed:
🚫 Ban AI Entirely
Creates artificial conditions disconnected from real-world professional practice. Students who comply are disadvantaged; enforcement is unreliable.
🙈 Ignore AI
Evaluates only outcomes while remaining blind to how they were achieved. Students are incentivized to accept AI output uncritically.
🔍 Detect AI as Cheating
Treats AI use as academic dishonesty. Detection tools have unacceptable false positive rates, particularly for non-native speakers.
None of these approaches develop the AI collaboration competencies students need for modern professional work.
The Insight: Assessment Drives Behavior
Building on Biggs' (1996) principle that “students learn what they think they will be assessed on,” AlwaysAI applies a simple insight: if you want students to use AI correctly, assess their AI usage directly.
This transforms AI literacy from a curriculum topic into an emergent property of assessment design. Students don’t need a separate course on prompt engineering if their regular coursework requires and evaluates effective prompting, critical evaluation of AI outputs, and thoughtful human-AI task allocation.
The Philosophical Shift
| Aspect | Traditional Approach | AlwaysAI Approach |
|---|---|---|
| AI use | Cheating | A skill to develop |
| Response | Detect and penalize | Measure and improve |
| Assessment | Binary (used / didn't use) | Multi-dimensional quality |
| Focus | Output authenticity | Process quality |
| Assumption | AI diminishes learning | AI can enhance learning |
Student-Centered Education in the AI Era
True student-centeredness in the AI era means:
- Universal AI access during all learning activities
- Scaffolded AI collaboration with embedded guidance
- Transparent assessment criteria for AI usage quality
- Diagnostic feedback that helps students improve their human-AI collaboration
- Adaptive support that meets students where they are
Withholding AI from students—or permitting it without guidance—is not student-centered. It either denies access to transformative learning tools or abandons students to discover usage patterns through trial and error.
The O×A Scoring Formula
Four Assessment Philosophies
Different educational contexts call for different emphases:
Professional Readiness
Preparing students for AI-augmented workplaces
Formative / Learning
Emphasizing AI skill development during learning
High-Stakes Certification
When domain outcomes matter most
AI Proficiency
Courses specifically targeting AI collaboration skills
AI collaboration is always assessed—the question is how much weight it receives relative to task outcomes. Even in high-stakes certification (60% outcome), 40% of the grade still comes from how students collaborated with AI.
Six Dimensions of AI Collaboration
The AI Collaboration score is not a single number but a composite of six measurable dimensions:
1. Prompt Precision
The clarity, specificity, and contextual richness of prompts. Students who provide clear context, specify constraints, and articulate actual needs receive more useful AI assistance.
2. Iteration Depth
Systematic refinement through multi-turn dialogue. Expert AI users rarely accept first responses—they probe, refine, and build understanding iteratively.
3. Critical Evaluation
Healthy skepticism and verification of AI outputs. AI systems can be confidently wrong; students must develop habits of verification rather than blind acceptance.
4. Integration / Synthesis
Combining AI assistance with personal knowledge and domain expertise. Effective AI collaboration is not outsourcing thinking—it is augmenting it.
5. Metacognitive Awareness
Understanding when AI assistance is appropriate, what AI does well vs. poorly, and the respective contributions of human and AI in the collaboration.
6. Efficiency
Achieving good outcomes with appropriate (neither excessive nor insufficient) AI interaction. The optimal range is typically 2–6 meaningful interactions per task.
Three-Layer Evaluation
Rather than relying on a single method, the platform evaluates through three complementary layers:
Pattern-Based Heuristics
Fast, consistent keyword and pattern matching against compiled regular expressions, detecting quality indicators across all six dimensions. Provides immediate, real-time scoring.
Linguistic Analysis
Deeper textual analysis including lexical density, speech act classification, prompt complexity scoring, and temporal pattern analysis (how prompts evolve over a session).
LLM-Powered Evaluation
Contextual, nuanced assessment that scores each rubric criterion with specific evidence quotes from student work, evaluates all six AI collaboration dimensions with examples, and provides actionable feedback.
Learning Activity Frameworks
For multi-session learning activities, the O×A framework adapts to three pedagogical models, each with distinct scoring weights:
| Framework | Learning Process | AI Collaboration | Engagement | Outcome |
|---|---|---|---|---|
| Problem-Based Learning | 35% | 35% | 15% | 15% |
| Project-Based Learning | 25% | 30% | 20% | 25% |
| Case-Based Learning | 40% | 35% | 10% | 15% |
Learning Process Sub-Dimensions
Learning process is itself evaluated across four equally-weighted sub-dimensions:
Exploration Depth
Breadth and depth of inquiry into the problem space. 25% weight.
Iterative Refinement
Quality of revision and improvement over time. 25% weight.
Conceptual Growth
Development of understanding across sessions. 25% weight.
Synthesis Quality
Integration of insights into coherent conclusions. 25% weight.
Diagnostic Profiles
Instead of just a score, students receive a diagnostic profile based on their performance pattern, with targeted recommendations:
Master Learner
Intensive Learner
Independent Learner
Process Developing
Growth Trajectory
Building Foundations
AI Interaction Modes & Roles
Three Interaction Modes
Students interact with AI through three modalities, all tracked and scored equally:
Chat
Text-based conversation. Always available as the default interaction mode.
Voice
Real-time voice interaction via OpenAI Realtime API for hands-free exploration.
Note: Voice mode requires an advanced API (e.g., OpenAI Realtime API) and is not enabled by default. Please contact the application creator to enable this feature.
Multimodal Upload
Images, PDFs, code, and data alongside text for complex analysis tasks.
Five AI Roles
The system classifies each AI interaction by the role the AI plays, helping instructors understand what students use AI for:
| Role | Description | Example Use |
|---|---|---|
| Executor | Computation, formatting, syntax | Math, code, formatting tasks |
| Retriever | Finding and summarizing information | Research, literature review |
| Interlocutor | Socratic dialogue and clarification | Concept exploration |
| Critic | Reviewing and providing feedback | Content review, editing |
| Generator | Producing drafts, code, designs | Content creation |
Module Structure
Course content follows a competency-based progression organized across 14 weeks:
Three Phases
Destabilize
Reconstruct
Stabilize
Three Levels per Week
L1 — Acquire
Remember and understand. Foundational knowledge building.
L2 — Deepen
Apply and analyze. Intermediate skill development.
L3 — Create
Evaluate and create. Advanced mastery demonstration.
Data Persistence & Analytics
All student work is tracked through dual storage to ensure both session resilience and comprehensive analytics:
Google Cloud Storage
Full session snapshots for restore capability. Auto-saved every 10 seconds and on every AI interaction.
xAPI Learning Record Store
Timestamped event logs for analytics and grading evidence. Complete audit trails of every learning journey.
Every student-AI interaction is captured with full prompt and response text, enabling complete audit trails of learning journeys.
Multi-Provider LLM Support
The platform supports 8 LLM providers to avoid vendor lock-in:
All providers are evaluated equally—the system does not bias scoring toward any specific model.
The Core Principle
Students who graduate without understanding how to collaborate effectively with AI are underprepared for modern professional work. By making AI collaboration mandatory and graded, AlwaysAI aligns assessment incentives with real-world demands.