User Research & Feedback Systems

FEEDBACK CHANNEL ARCHITECTURE

CHANNEL 1: IN-APP FEEDBACK
├── Floating feedback button (always visible)
├── Post-transaction rating (1-5 stars + optional comment)
├── Feature-specific feedback prompts
├── Bug report functionality
├── Processing: Automated categorization
└── Response: Acknowledgment within 24 hours

CHANNEL 2: SUPPORT TICKETS
├── In-app chat support
├── Email support
├── Phone support (if offered)
├── Ticket categorization
├── Processing: Agent + AI classification
└── Response: SLA-based (4hr/24hr/48hr)

CHANNEL 3: SURVEYS
├── In-app survey prompts
├── Email survey invitations
├── Incentivized surveys (for detailed feedback)
├── NPS tracking
├── Processing: Automated analysis
└── Frequency: Per user lifecycle stage

CHANNEL 4: SOCIAL MEDIA
├── Official account mentions
├── Hashtag monitoring
├── Sentiment tracking
├── Influencer mentions
├── Processing: Social listening tools
└── Response: As appropriate

CHANNEL 5: APP STORE
├── Review monitoring
├── Rating tracking
├── Response management
├── Competitive monitoring
└── Processing: Daily review

CHANNEL 6: COMMUNITY
├── Official forums
├── User community platforms
├── Merchant community
├── Feature request voting
└── Processing: Community management
```

FEEDBACK PROCESSING PIPELINE

STAGE 1: COLLECTION
├── All channels → Central database
├── Timestamp
├── User ID (if available)
├── Channel source
├── Raw content
└── Automatic: Real-time

STAGE 2: CLASSIFICATION
├── Category assignment
│ ├── Bug/Issue
│ ├── Feature request
│ ├── Compliment
│ ├── Complaint
│ └── Question
├── Severity assessment
├── Product area tagging
├── Sentiment scoring
└── Automatic + Manual review

STAGE 3: PRIORITIZATION
├── Volume analysis (how many users affected)
├── Severity weighting
├── Strategic alignment
├── Effort estimation
└── Priority scoring

STAGE 4: ROUTING
├── Bugs → Engineering
├── UX issues → Design
├── Policy questions → Policy team
├── Urgent issues → Incident response
└── Features → Product backlog

STAGE 5: ACTION
├── Issue resolution
├── Feature development
├── Communication/response
├── Documentation update
└── Knowledge base update

STAGE 6: CLOSURE
├── User notification (where applicable)
├── Satisfaction verification
├── Metrics update
├── Learning documentation
└── Trend analysis input
```

---

CONFIRMATION BIAS IN CBDC RESEARCH

WHAT IT LOOKS LIKE:

Selection Bias:
├── "We talked to 10 users who love the app"
├── Problem: Didn't include critics
├── Fix: Random sampling, include churned users
└── Red flag: Only positive quotes in reports

Question Bias:
├── "Don't you love the new feature?"
├── Problem: Leading questions
├── Fix: Neutral phrasing
└── Red flag: Questions assume positive answer

Interpretation Bias:
├── "75% rated 3+ stars, so users are satisfied"
├── Problem: 3 stars isn't satisfied
├── Fix: Use proper benchmarks
└── Red flag: Spinning neutral as positive

Reporting Bias:
├── "We highlight what's working well"
├── Problem: Hiding problems
├── Fix: Report all findings
└── Red flag: Steering committee only sees good news

Survivorship Bias:
├── "Active users say it's great"
├── Problem: Ignoring churned users
├── Fix: Include exit interviews
└── Red flag: Only measuring current users
```

OBJECTIVE RESEARCH PRACTICES

SAMPLING:
□ Random selection within segments
□ Include hard-to-reach populations
□ Proportional representation
□ Exit surveys for churned users
□ Document selection methodology

QUESTION DESIGN:
□ Neutral language
□ Avoid leading questions
□ Include negative options
□ Open-ended questions for depth
□ Third-party review of instruments

DATA COLLECTION:
□ Standardized protocols
□ Multiple interviewers
□ Recorded sessions (with consent)
□ Interviewer calibration
□ Quality control checks

ANALYSIS:
□ Pre-define success criteria
□ Include negative findings
□ Statistical significance testing
□ Multiple analysts review
□ Devil's advocate role

REPORTING:
□ Report all findings (positive and negative)
□ Include confidence intervals
□ Acknowledge limitations
□ Present dissenting views
□ External review option

GOVERNANCE:
□ Research team independence
□ Separation from marketing
□ Direct board reporting option
□ External validation for major decisions
□ Transparency commitments
```

RED TEAM RESEARCH APPROACH

PURPOSE: Deliberately look for problems

RED TEAM MANDATE:
├── Find reasons CBDC will fail
├── Interview dissatisfied users
├── Identify edge cases and failures
├── Challenge positive findings
└── Present to decision-makers

RED TEAM METHODS:
├── Churned user interviews
├── Merchant rejection analysis
├── Competitor user interviews
├── Privacy advocate consultation
├── Accessibility audit
├── Fraud attempt simulation
└── Support escalation review

RED TEAM OUTPUT:
├── "Why this will fail" report
├── Top 10 risks to adoption
├── User segment vulnerability analysis
├── Competitive threat assessment
└── Recommendations for improvement

RED TEAM RULES:
├── Report directly to senior leadership
├── Independence from project team
├── Findings cannot be suppressed
├── Constructive recommendations required
└── Regular cadence (quarterly)
```

---

INSIGHT TO ACTION FRAMEWORK

STEP 1: CONSOLIDATE INSIGHTS
├── Synthesize all feedback sources
├── Identify patterns and themes
├── Quantify where possible
├── Prioritize by impact
└── Document with evidence

STEP 2: CATEGORIZE ACTIONS
├── Quick wins (low effort, high impact)
├── Strategic initiatives (high effort, high impact)
├── Consider later (low effort, low impact)
├── Deprioritize (high effort, low impact)
└── Map to product roadmap

STEP 3: ASSIGN OWNERSHIP
├── Each action has owner
├── Clear deliverables
├── Timeline commitment
├── Resource allocation
└── Escalation path

STEP 4: TRACK PROGRESS
├── Action item dashboard
├── Weekly status updates
├── Blockers surfaced
├── Dependencies managed
└── Metrics tied to actions

STEP 5: CLOSE THE LOOP
├── Communicate changes to users
├── Measure impact of changes
├── Update feedback providers
├── Document learnings
└── Feed into next cycle
```

USER FEEDBACK DASHBOARD COMPONENTS

VOLUME METRICS:
├── Feedback received (by channel)
├── Feedback categorization breakdown
├── Response rate to surveys
├── Support ticket volume
└── Trend: Week-over-week

SENTIMENT METRICS:
├── Overall sentiment score
├── NPS (Net Promoter Score)
├── App store rating
├── Sentiment by category
└── Trend: Week-over-week

QUALITY METRICS:
├── Response time (by channel)
├── Resolution time
├── First-contact resolution rate
├── Customer satisfaction post-resolution
└── Trend: Week-over-week

ACTION METRICS:
├── Issues identified
├── Issues resolved
├── Features shipped from feedback
├── Open action items
└── Time from insight to action

LEARNING METRICS:
├── Key insights this period
├── Hypothesis validations/invalidations
├── Segment-specific learnings
├── Competitive insights
└── Research gaps identified
```

---

✅ Multi-channel feedback captures more: Single-channel feedback misses significant user perspectives.

✅ Exit surveys reveal problems: Users who leave have valuable insights about why.

✅ Confirmation bias is pervasive: Without deliberate countermeasures, research will support pre-existing beliefs.

⚠️ Whether research predicts adoption: Even excellent research may not predict actual behavior at scale.

⚠️ What users say vs. what they do: Stated preferences often don't match actual behavior.

🔴 Only talking to satisfied users: Creates false confidence.

🔴 Leading questions in surveys: Invalidates findings.

🔴 Suppressing negative findings: Prevents necessary course corrections.

---

Assignment: Create a comprehensive user research plan for a CBDC pilot.

Time investment: 2-3 hours

---

Q1: What is the primary purpose of pre-pilot research?
A) Prove CBDC will work B) Understand user needs before building C) Generate marketing content D) Satisfy regulators
Answer: B

Q2: Which user group is often missing from CBDC research?
A) Early adopters B) Power users C) Churned users D) Employees
Answer: C

Q3: What is confirmation bias in research?
A) Confirming user identity B) Seeking/interpreting data to support existing beliefs C) Getting user confirmation D) Testing twice
Answer: B

Q4: What is the purpose of red team research?
A) Test security B) Deliberately look for reasons CBDC will fail C) Market research D) Compliance testing
Answer: B

Q5: What should happen after identifying a user insight?
A) File it B) Discuss in meeting C) Assign owner, track progress, close loop D) Add to backlog
Answer: C

---

End of Lesson 10