DATE
MAR 2025 - SEPT 2025
COMPANY
ARROWSTER
TEAM
1 PRODUCT DESIGNER - 5 ENGINEERS - 1 PM
STATUS
IN PRODUCTION
TABLE OF CONTENTS
My role: Sole product designer. I ran user research, wireframes, hi-fi prototypes, user testing, and the final product.
Arrowster is building a platform that helps students navigate the university application process, from choosing a major and schools to developing personal essays. I helped design and build the 1.0 version of the platform, and we’re now turning it into 2.0.
This case study covers one part of that work: redesigning the AI research agents to create a more connected and effective research experience (highlighted in purple below)
The AI Research Agents, highlighted in purple, are one part of Arrowster’s broader end-to-end college counseling journey that I redesigned
I.
THE PROBLEM
V1.0 brought the agents together, but students left after one answer.
Version 1.0 gave each stage of studying abroad its own AI expert. It cut 20+ scattered agent windows down to 7: choosing a major, finding colleges, finding scholarships, planning extracurriculars, preparing for tests, writing personal statements, and applying for a visa.
That was a real improvement, but the numbers showed students weren't getting value from it:
78.7%
DROP-OFF RATE
of students who started an agent, left after getting a result
3 minutes
PER SESSION
The average time a user interacts with the product, which is too short for a research session
The V1.0's 7-agent structure. Each agent was a capable standalone tool, with in-house resources for students to learn about.
The V1.0's 7-agent research tool design. Most of the responses are text-based.
II.
RESEARCH
So the question I set out to answer was: Why do students leave after they get a result?
To understand how students actually research colleges, and where Arrowster's product fit into that process, I triangulated three research methods, since a change this large needed more than one source of truth to validate it:
Secondary research (Netnography): Based on early conversations with students, I found several active Facebook groups where students discuss the college research and application process. I collected relevant threads and grouped them into themes. These themes shaped my baseline understanding of the counseling business and informed the questions I used in the focus group and usability testing.
Focus groups: I ran two groups of students to talk through their experience researching and applying to college, and their experience using Arrowster's current product.
Qualitative research (Chat history): I coded and grouped 6000+ anonymized student questions from V1.0 into recurring behavioral themes.
One of the focus groups of students
Key insights
INSIGHT #1
Students don't know what to ask AI agents
Most students were new to applying abroad. They didn't know which question to ask first, which agent fit their situation, or what Sophie AI could do.
The chat history backs this up: every agent opened with 3 suggested prompts, and 83% of students picked one as their first move instead of typing their own question.
INSIGHT #2
Our chat responses are not insightful enough
Unlike generic responses students can easily get from other AI chatbots, they need something more reliable and in-depth, something insightful that only consultants with years of experience would have.
Digging further, we found students really needed situational advice based on their specific case like financing, visa likelihood, scholarship odds, immigration pathways.
We don't want another information provider, we want a real consultant
— Student participant during focus group
INSIGHT #3
Students usually come to Arrowster for profile prediction and financial/scholarship advice
The 6000+ questions fell into five groups:
Admission Prediction ("Chancing") — 35%: "Am I eligible?", "What are my chances with a 3.2 GPA?"
Financial & Scholarships — 27%: "Scholarships I'm eligible for," "What's my budget?"
School & Program Discovery — 15%: "Show me best-fit schools," "Universities in Germany for Biotechnology"
Major & Career Guidance — 13%: "How do I choose the right major?", "Majors I'm suited for"
Application Support — 10%: "Rate my college essay," "How to apply to UCLA"
Coding anonymized students' questions from V1.0
III.
COMPETITIVE ANALYSIS
I looked at how conversational AI patterns are being used across products. This includes Claude, ChatGPT, and several other AI platforms acting as counseling or advisory tools in fields outside education (finance, legal, human resources). I wanted to answer two questions:
What are the usual patterns these products have?
Where does each pattern actually work? Where does it break down?
Findings
FINDING #1
AI chat tools are using more interactive design to display information
There are many design archetypes that these AI chat tools are using. There are patterns for information retrieval, messaging/drafting, and eliciting more information from users. Below are some patterns from Claude.
Claude adapts its response format: elicitation buttons, interactive maps, step cards,... to match what the task actually needs.
FINDING #2
For non-expert users, checklists and predefined options help them decide faster than open text does.
This comes down to recognition vs. recall: choosing from a visible list of relevant options is cognitively easier than generating a question from scratch. Our students were closer to this profile than to the "power user" profile that V1.0's chat-first design assumed.
Takeaway
We'd defaulted to the expert-user pattern (open chat) for a user base that needed the average-user pattern (guided, recognition-based choices). This directly shaped the direction of the redesign below.
IV.
SOLUTION
In V1.0, every agent sat at the same level under Home. Version 2.0 groups them by where the student is in their journey:
Research: explore majors, colleges, scholarships, budget, and visa eligibility
Evaluate: get an in-depth review of their own profile and strengths
Apply (paid plan): prepare documents, tests, visa documents, essays, and extracurriculars
Admission chances, the top need at 35%, now has its own Evaluate step. This case study focuses on the Research step, with three goals that come straight from the insights:
GOAL # 1
Show what agents can do
We added guidelines to the research tools. The new version must be able to clearly communicate to students what our AI agents can do and how students can navigate through the site
GOAL # 2
Ask for details before answering
We made research more interactive: Agents collect students’ details through multiple-choice questions, open questions, and saved lists, then ask follow-up questions to dig deeper, so every answer is personal and complete.
GOAL # 3
Give answers students can't get elsewhere
We added in-house data from our partner agencies and past successful students, including school data and scholarship chances.
Version 2: Features now follow the student's journey, instead of simply providing separate agents
V.
ITERATIONS
Preliminary research AI Agent
Here are some iterations of the research agents that I designed.
Instead of dropping students into an empty text field with no prior guidance or context, I explored ways to make the interface more instructive without overwhelming students with text they might not read.
Preliminary research includes many separate agents, which confused and overwhelmed students
Option 1
In this option, I combined all agents into a single text field. Below it, a checklist shows which details the agent still needs (for example, Finding College has a checklist for GPA, major, budget, country, degree, intake year, and preferences). As students type, the matching items get checked off automatically. Each agent also has its own animated character, which makes it easier for students to tell that each one does a different job.
PROS
Provides students with better guidance: The checklist tells students what information leads to better results, so they don’t have to guess what to write.
Freedom to add personal details: Because it’s open text, students can include things a form wouldn’t capture
CONS
Looks too much like a general chatbot. Since the layout is close to ChatGPT or Claude, students may think it’s just an open chat and miss that there are specialized agents behind it.
Agents are hidden. The agent list only appears after clicking the dropdown, so many students may never find it or switch between agents.
The checklist isn’t explained. Students might not understand that the items check off automatically, or they may think every item is required before they can continue.
Option 2
For Option 2, I designed a unified command interface that brought multiple agents together. Each command included essential context to help the agents answer students’ questions more effectively. Students could also add more criteria if they wanted to personalize their requests further.
PROS
A clear place to start: Students see a sentence to fill in instead of an empty text box, so they know what to write.
Room to personalize. “Add criteria” lets students include extra details only when they care about them, so the default sentence stays short.
One place for every agent. Students switch tasks by changing the first blank instead of moving between separate tools, so agents are easy to find.
CONS
Once the text goes past three lines, the input box gets crowded and hard to read. The more details students add, the harder it is for them to scan, review, and edit what they wrote.
Option 3
Option 3 builds on Option 2. The command line now asks only for the basics the agent needs to get started. Once students fill these in, the agent asks follow-up questions that fit their request. These can be a text field for detailed, personal answers that regular filter interface cannot capture, or a short quiz with ready-made options to tap.
PROS
Easy to start and less to take in at once. Students only see a few blanks at the start, so the first screen feels quick and easy.
Tapping instead of typing. Students can pick from options instead of writing everything out or digging through filters and multiple screens.
Questions that fit the student. Follow-ups change based on what they entered, so a student who is “exploring options” gets a quiz, while one who knows their major skips it.
CONS
More steps. Students have to go through several screens before they see results, which may feel slow for those who just want a quick answer.
More work to build and scale. Every agent needs its own sentence and its own set of follow-ups, so adding new agents takes more design and writing effort than a single text field.
VI.
PROTOTYPING & TESTING
I designed all three options in Figma, then used Claude Code to turn them into working prototypes. This let students try a real flow instead of imagining one from static screens.
We tested with 10 new high school students. 5 students tested Option 1, and 5 others tried both Option 2 and Option 3 to compare them. Each student told us about their background information and what they were currently researching, then used Arrowster to find more information.
Our findings
FINDING # 1
Students prefer clear layouts and step-by-step guidance when using an interface they haven't seen before
3/5 users who tried Option 1, were confused about the checklist below the text field. They either thought it was required to fill out all items on the checklist before moving on, or didn’t understand what it meant.
2/5 users who tried Option 1 missed the agent selection button: "It looks similar to the ChatGPT interface, so when I wanted to find other agents I immediately looked at the side navigation first."
FINDING # 2
They lose interest quickly when a screen has too much text
4/5 users prefer Option 3 over 2 for its simple layout:
"For Option 2 I have to click a lot at once. It looks like a wall of text I have to get through"
"Option 3 is more engaging to me. It's like talking to an actual person who understands my situation"
Based on these findings, we chose Option 3. It starts with only a few basic blanks, then guides students through short follow-up questions one step at a time, which keeps each screen light and easy to follow. This ensures that the agent has more qualified input while still giving users the freedom to ask more personal questions, which is crucial for consulting businesses.
Final prototype of the product. Click to explore!
VII.
FINAL DESIGN
The final flow takes students from a short sentence to an answer based on their own profile, in three steps:
Pick a task and fill in the basics. The first blank chooses the agent, and a few more blanks collect what it needs to start. Answers are saved, so students don’t have to fill them in again when they move to another agent.
Answer follow-up questions. The agent asks only what fits the request, using options to tap or a text field.
Get results based on their profile. For Find Universities, students choose how they want their list built: a balanced mix of reach, match, and safety schools, or schools that are easier to get into.
Final design for Find universities feature
Final design for Cost breakdown feature
What’s new on the result page
Answers based on Arrowster’s own data. Results draw on past accepted students, similar student profiles, and average living costs obtained from real cases. This gives students the specific advice they asked for in research, not general answers they could get from any chatbot.
Suggested next questions. Each answer ends with follow-up questions that fit the result, so students always know what to ask next and don’t hit a dead end.
Cards instead of long text. Key information is shown in clickable cards with sources, so students can scan it quickly and dig deeper on what interests them.
VIII.
RESULTS
1 agent -> 3 agents
PER SESSION
Because it’s easier to switch between agents, students now try 3 agents per session on average, up from 1.
78.7% -> 23%
DROP-OFF RATE AFTER 1 AGENT
Students who used to stop after one agent now move on to try more. This is the behavior the original chat-first design had been trying, and failing, to produce. It assumed students already knew what to ask, creating a barrier before they could even begin.
3 minutes -> 14 minutes
PER SESSION
Sessions got longer because students now share more details before getting an answer, receive a detailed answer based on their own profile, and get follow-up questions that keep the research going.
IX.
MY LESSONS
Prototype earlier, and with real functionality
This project involved several rounds of testing, and a lot of time went into updating the prototype between each round. I learned that low-fidelity prototypes made it difficult for students to evaluate an unfamiliar product concept because too much was left to their imagination.
Using Claude Code to turn Figma designs into working prototypes made it possible to test something students could actually use, rather than something they had to imagine.
Don't default to the open chat box
The dominant AI products put an open prompt at the center of the experience, but that assumes users already know what to ask. Our research showed that this wasn't true for our audience.
AI made it possible to handle highly specific and personalized questions, but the interface still needs to elicit the context required to make those answers useful. Instead of waiting for the perfect prompt, we learned to guide students toward the information they need to make a decision.