Grocery HelpAR

UX Design / Mobile Application / Augmented Reality

*Awarded 2nd place in Georgia Tech Convergence Innovation Competition


Problem space

Grocey HelpAR is a mobile application that helps international students who are not accustomed to grocery shopping in the US. Based on our user research, we found several problems which our users encountered: first, the difficulty translating different kinds of food from their own language to English, which often led to them buying the wrong items; second, users also had a hard time finding their way around complicated store layouts; and, finally, they had difficulty comparing products to choose the best option.


Grocery HelpAR helps our target users to overcome the obstacles they face during grocery shopping. When you make a list, Grocery HelpAR automatically translates your shopping list into English. When you enter the store with your shopping list, Grocery HelpAR will calculate the optimal route for the shopping list and let you know where you need to go. When you are close to the item, Grocery HelpAR will inform you with an alarm, vibration and a popup message. Grocery HelpAR provides the best solution when you are not sure which item is best for you. It provides a list of items which are most frequently recommended by your ethnic group. This is helpful because our target users tend to rely on recommendations from their community, because they want to find the items most suited to their own tastes. The users also find the detailed information of items easily and quickly by using the AR scan function of Grocery HelpAR.

My contribution

I participated in this project as a user researcher and UX designer along with great teammates, Geunbae, Miro, and Shuhan. As a team member, I participated in every step of the research and the design. I conducted two contextual inquiries and four semi-constructed interviews. Based on the findings from observation and interviews, I conducted a structured hierarchical task analysis and created a storyboard, which allowed us to have a better standing of our users group. I was in charge of one of three alternative design concepts (recipes & shopping list) and made paper prototypes for the concept. I also shoot and edited the video clip above.


Target user

For our project, the target user group is international students who prepare their own meals. During our research, we discovered that many of the international students at Georgia Tech are new to the U.S. To elaborate more on some of the main issues they face, we have things like: translation of different By conducting interviews and contextual inquiries, we also found out that many of them had a range of different shopping experiences from back home dealing with different types of food, the layout of the store and other things which made it challenging for them to shop for groceries in the U.S. kinds of food from their own language to English which often led them to buy wrong items, choosing the substitutions for their traditional ingredients in dishes, in store finding their ways around stores; organizing shopping lists, product comparison to choose the most satisfying product (which often led to wasting time). We were aware that shopping for some users may still bring them enjoyable and satisfying experiences, but as they are Georgia Tech international students who have to manage their time wisely in order to succeed in their academic agendas, and based on various research methods that we have introduced and navigated, the process is time consuming and often, frustrating.


Task analysis

User research

Semi-structed interview

Once we defined the problem areas, we wanted to quantify and confirm that the issues persist with users who are accustomed to the store’s layout and the types of food. The semi structured interview was a great way to gather this data as well as some of the detailed feedback and responses from the users. Moreover, we decided to conduct semi-structured interviews with a topic focused on grocery shopping. By asking the international students about their overall experience with grocery shopping in the U.S (and around the Georgia Tech campus), we aimed to discover the current problems and hardships that they face in addition to positive (or surprising) experiences. We envisioned that we would definitely get the most useful information about user needs and the problem space by conducting these interviews.

Contextual inquiry

Following the semi-structured interviews, we thought it was necessary to continue on with our research by doing some contextual inquiries. Since our topic is about grocery shopping, we wanted to witness the tasks and the process that the international students often go through firsthand by shadowing them. Also, by asking them to do think-aloud protocol (ex. of what they are trying to accomplish and why they are taking such a route, etc), we wanted to discover more about the users that we are working with. We thought it was necessary to understand the users’ behaviors and their problems. Since the problem we want to solve involves wayfinding, the fact that many users had difficulties communicating and correctly identifying the type of food, (which were biggest issues we found), it was essential for us to observe the current users while they performed the task. It helped us to understand the problem space as well as the challenges that they face with our own eyes.

Affinity mapping

Findings from user research

We started the diagram during one of our class sessions, but we ran out of time, so we revised the diagram by meeting together at a separate time as a group afterwards. Based on the affinity diagram, we first found the behavioral patterns of international students at Georgia Tech while shopping for groceries. We found out that they often have to manage their time wisely (because they are students) and spend the least time as possible at the store. However, during some of our open ended and semi-structured interviews, one person said that they often enjoy hanging out at the store to look around. Most of them would make a list before going to the store, physically (on apps or notes) or mentally. They seldom ask the shop assistants for help because some of the items they are looking for are hard to find. Moreover, many of the them use dictionary app or search engines to translate the item names they are looking for. Lastly, we found out that because many are not familiar with U.S brands which are different from their own countries, they heavily rely on friends’ recommendations to select the right items for their taste and their traditional dishes.

With the help from our affinity diagram, we were able to discover that the most time-consuming part of the user’s journey was locating and selecting items. For locating items, the hardship was related to accurately translating the name of the items quickly and easily.  And for selecting items, the utmost problem was that the users are unfamiliar with U.S brands which motivates them to ask for help / recommendations from their friends.

Diverging design ideas

Low fidelity prototype 1: AR shopping aid

The idea behind “AR shopping aid” came from reusing existing solutions and enhancing them. Almost everyone already have mobile devices that are capable of using AR applications so there is no need to introduce new technology. Other apps also exist but they don’t provide complete solution. Our idea is to make application that provides users with necessary tools for shopping (such as shopping list, translator, store map with list items location, suggestions when choosing items etc) and to add AR option to provide faster solution when choosing items from the shelf by overlaying information (suggestions from online community) over product which is significantly faster than browsing internet.

Low fidelity prototype 2: Smart shopping cart

Since a shopping cart is the most necessary assistant in a shopping behavior and typically needs two hands of users to use, we feel it would be more natural if the interaction only takes place between the user and shopping cart. The basic idea is to add a smart device with a relatively big screen like an iPad to the shopping cart so the hands of users can be liberated from holding the device. The smart device is able to provide diverse services. There would be also a mobile app to complement. The whole system is designed to solve three main problems of international students, which are translation of item names, wayfinding and selecting brands.

To use the system users would need to sign up an account completing the nationality and language information. After that they can make a list in their mother language on the app which will translate the list into English automatically. Then the user signs in the smart device of the shopping cart with the list loaded. The device will calculate the shortest route and navigate users through store. When users arrive at one item, the device provides recommendation for the brand based on the popularity in the people from user’s country.

Low fidelity prototype 3: Making list from recipe

This solution is a mobile application that integrates a cookbook for traditional dishes and a shopping list. We starts from the fact that our target users prefer to cook their own traditional dishes. They also search online recipes for their traditional dishes before grocery shopping and cooking. We think that if we can provide a service that makes a shopping list automatically when they decide on a recipe from a cookbook, users can reduce their time and effort during grocery shopping.

Converging design ideas

From our classmates’ feedback, the AR shopping aid generally got the most approval while smart shopping carts received divergent opinions. And attitudes towards recipes & shopping lists were relatively neutral. Most of the classmates liked the idea of an AR shopping aid, although some doubted that it was feasible to get all the information for the items. One brought up that international students may be too shy to scan every item label on the shelf. For the smart shopping cart, the biggest concern was the safety problem of the using smart devices on shopping carts as users may be too focussed on the screen rather than on where they are doing (like texting while driving) which can be difficult to resolve. Besides that, our classmates liked this idea. Recipes & shopping lists was quite acceptable to all the classmates. A lot suggested that it can be incorporated into the former two ideas. Some of them also mentioned that some users might decide what they cook based on the ingredients that they already have. If the list include the ingredients the users have in their refrigerator, the solution will be able to provide more options to start working on.

High fidelity prototype for evaluation

After the in-class design sprint, we made a prototype which will be used in the usability test. Based on the feedback from the sprint, we decided to develop the AR shopping aid idea by merging some design concepts from the other two design ideas. First of all, the function that enables users to make a shopping list from recipes is incorporated into the AR shopping aid concept. We also developed translation and wayfinding functions by comparing the similarities and differences between the AR shopping aid idea and the smart shopping cart.


We combined A/B testing with usability test to evaluate our prototype. A/B testing helped us to compare and see how our app changes user’s behavior in grocery shopping and test the effectiveness of our solution. The usability test helped us identify the specific problems of our design and prototype.

In test A, we assigned tasks to our participant without our application. In the test B, the participants will perform similar tasks by using our application. The performance metrics including success, time on task and errors were measured in both tests. To assure the validity of time measurements, we chose to use retrospective think-aloud protocol. Neither did the participant think aloud nor did we ask questions during the task. We only asked interview questions right after each task when the timing was stopped. Most of the questions were self-reported using rating scales in regard of the usability of specific part of the app. We also asked open-ended questions to get clues of what should be modified.

What worked:

Based on our observation we can conclude that, overall the design was a success.
Indoor navigation received great feedback from all users regardless of how well they were familiar with the store layout.
Recommendation (needs some improvement with “filter by”) and AR function also performed very well.

What didn't worked:

To our biggest surprise, Translation was not highly rated function, even though it came out of initial research as one of the biggest problems. We are not sure at this point if users got accustomed to the groceries they are looking for. It is something that needs further investigation.
List creation performed only as 5.5 which is most likely due to low fidelity prototype. List was one of the “easy to implement” features for “all in one” solution.Reason behind is most likely due to a low fidelity.
Due to some design issues with UI users have missed the “Location on shelf” which was one of the important features for time saving. However, once users were introduced to it they had very positive feedback.
UI elements, such as icons, seems to be a bit unclear as well. Workflow between adding ingredients to the list and opening in the store map was a bit confusing.

Design suggestions from evaluation

Number of reviews

The app doesn’t show how many people rated the product so the ratings are not very reliable. We should add numbers of reviews right next to the ratings.

Feedback after adding items

We need to add feedback that let the users know the item has been successfully added to the list.

Make cards look more clickable

Many participants didn’t know the cards of product are clickable on the recommendation page. We need to add signifiers for the cards, for example a ‘view more’ button on the cards.