top of page

How Creators Publicly Request Feedback in an Open Online Community ?

Accepted to ACM CSCW 2020 full paper

UX Research
feedback banner.jpg

Principle Investigator  

Team Member             

Skills & Tools              

Prof. Steven Dow (UC San Diego, UCSD The Design Lab)
Regina Cheng (University of Washington), MaySnow Liu
4 Months

Qualitative Coding
Semantic Analysis
Literature Review
Statistical Modeling


Recently, creative workers frequently use online critique communities for feedback on their work, especially those with limited access to formal feedback resources. For example, design students like to ask for feedback for their portfolio and project work, product designers want to hear ideas about the logo they created, etc. Being aware of this trend, we want to understand:

" what strategies seekers use to request feedback and whether those strategies are effective ? "


We present two studies to explore the dynamics between feedback request and feedback in subreddit r/design_critiques, which is part of r/design_critiques is a large and active community dedicated to feedback exchange across a range of design domains, including website, graphic design, animation, etc. It allows everyone to publicly share their work through text-based forum post (including embedded URLs) and receive feedback in the form of public comments.


Community member's opinions about how to request feedback

  • ​Conducted 12 Semi-structured Interviews

  • Analyzed Interview Results

  • Built Affinity Diagrams


How request strategies affect community feedback

  • ​Data collection (post request and comment from 2013.1 - 2018.10)

  • Developed coding scheme

  • Qualitatively coded 900 sampled posts out of total 24587 posts

  • Built regression models

  • Conducted big data analysis

Our Data

All posts w/URLs; associated comments from 2013 to 2018 from r/design_critique subreddit community

24,587 feedback requests (Including title, body, author, timestamp)

98,475 pieces of feedback

Our Participants

12 active users in r/design_critique community aged over 18
On average
12.64 posts history (min = 8, max = 23)

4.04 years of experience in the community (min = 0.5, max = 7)

Building the Affinity Diagram

Sample Interview transceript.png

Sample of Interview Transcript

Affinity Diagram 2.HEIC

Affinity Diagram Process


Affinity Diagram Process

Qualitative Coding

Sample Code Scheme.png

We selected 900 post requests from the corpus of 24,867 posts by randomly sampling 150 posts from each year of the six years.

We iteratively developed a coding scheme describing the feedback request strategies in the community. The first coding scheme has 21 strategies, and we refined it into 7 at the end. Then, we binary coded for the presence of 7 strategies.

To investigate how the features of feedback requests influence the resulting feedback, we built regression models with binary-coded strategies as independent variables (IVs) and added length of request as a covariate in the model.


We analyzed several features of the resulting feedback as dependent variables (length of feedback, the quantity of feedback, waiting time for the first feedback) and created a different regression model for each. In the end, we report the effect sizes of our regression models.

Sample Coding Scheme


Although I cannot provide detailed information about method, discussion or result of our research until our paper is published, here's a high-level overview of our findings. 

Feedback Seeker

  1. Seekers often provide design (but not personal) details. 

  2. Seeker's disclosure of personal information may distract providers and frame them to generate biased feedback.

  3. Seekers prompt specific feedback, yet still want comprehensive critique.

  4. Seekers want expert's feedback, but avoid explicitly requesting it.

Feedback Provider

  1. Providers want more contextual information.

  2. Providers try to empathize with seekers.

  3. Providers prefer specific, not vague feedback prompts.

  4. Providers avoid lengthy and complicated requests.

Signaling novice led to faster and better-justified feedback

Critiquing one's design in request resulted in faster and more actionable feedback

Providing specific prompts, especially showing variants, yielded faster and more justified feedback.

Self-critiques establish context and surface meta-cognitive thinking

Last, based on our research, we presented some design implications for future feedback systems.

For more detailed information, feel free to read our full paper


Our team.JPG
bottom of page