Category: HCI Project Diary

In the “virtually getting together” project, we decided to focus on the online learning and created a VR campus on Mozilla Hub.

Emphasizing & Brainstorming

Since we all have plenty of experience on online learning, we are all aware of the problem produced when zoom meeting substitutes face-to-face teaching — the lack of interaction. I talked with some of my teachers, they all confirmed that the interaction is obviously decreased. Through the VR hub, students can express their basic feelings with head movement and hub interaction features, and seeing each other’s character would give them a sense of actually being in the campus.

Design & Implementation

The interview & need-finding work were conducted by my partners, then Andrew and me began to work on the implementation of the 3D model used by VR. We’ve proposed a Zen Garden scene to replace the traditional classroom, and put the lecture hall, breakout room and some other decorations into the planning part. Andrew then created a blender model implementing the basic design; I added the garden part to the model, migrated it to the Mozilla Hub and added hub components to the scene. For example, the water and particle emitter are used in the pond to settle the relaxing and focusing atmosphere; the audio zones in the breakout room are used to reduce interference from other groups while keeping the original layout.

Usability Testing

We received a lot of feedbacks from the usability testing period. We’ve implemented other scenes based on that: for example a classic classroom as some users mentioned that it would be better to have a feeling of in the school.

Demo Video

Based on the scene we’ve built, I also made the demo video with some recordings of usability testing.

Reflection

This time we did the whole project in a timely manner, and the quality of each part is prominently increased, as most of our ideas can be fully considered. 3D modeling is also a new thing for me to try, and with the help of my teammate, I’ve learned a lot on the building of model although it is a really challenging work.

For the VR classroom, I also have some further thoughts on utilizing body/facial sensor to provide more interaction. The Apple Memoji (based on Face ID) is a great way to show students’ facial expression or body language on the model during a class, although it is hard to implement such technical thing in this project, I believe this could be a trend in the future of VR getting together.

In this human-robot interaction project, we created a chatbot prototype designated for hotel quarantine life with RASA framework.

Emphasizing

Quarantine is a relatively common thing during the COVID-19 pandemic, and a lot of people have been experienced a short period of quarantine life. I talked to some of my friends to learn about the problems occurred during quarantine, and try to think about what the bot could do to improve their quarantine experience.

New policies about the quarantine period, test requirements and hotel rules are often confusing, really need a better channel to know these messages. Also there are too few ways to entertain oneself, and often feel lonely due to lack of communication.

— Chu (Translated)

Brainstorming

During the brainstorm session, we’ve considered various demands of quarantine life, including food, information, entertainment and mental & physical health. Based my own experience, the entertainment demand is critical in days without talking to people, so I proposed that the bot could offer personalized recommendations on entertainment activities, such as movies, musics, etc.

Implementation

Finally we’ve chosen 3 functions to implement into our demo: food ordering, quarantine information and emotional guiding. We consider these functions as essential and helpful during one’s quarantine.

Food ordering is one of the basic demand of human being, and the food condition in a quarantine hotel is often limited and not supplied in 24 hours. Therefore we designed a user-friendly catering process, with up-to-date menu and on-time order processing.

Quarantine information gives users a convenient, reliable and personalized way to know about the probably complex and rapid-changing policies. The bot can retrieve and process personal information, combined with the latest policy of local government to answer users’ questions. This could reduce the workload of public health departments and also help users to save time on looking up these information.

The bot also offers emotional guidance as part of its function. It contains professional suggestions and also practical way to get appropriate help. The bot try to detect user emotions from context and find the best match on its database.

We’ve also conducted usability testing and collected a lot of informative advices, which were used to further iterate our design.

Demo video

I also edited the showcase video recorded by Simon.

Reflection

The usability test reveals to me that it’s really important to look at the software we made from different perspectives: a perfect function in developer’s eyes could be full of problems in actual use.

Also we may need to bring forward the whole process of the project, as there appears to be a lot of time pressure by the deadline. All in all, a great teamwork of all of our group members, and the project itself is very helpful and educational, both in the designing works and programming works.

Is the campus intelligent enough? Which part of the campus can be more smarter?

Teamwork!

As an exchange student here in UST, it’s a really nice experience to team up with people in different backgrounds, and all of us put our endeavor into different parts of the project.

Brainstorming

Starting from the brainstorming session, we’ve considered a lot of aspects in our campus, and came up with different ideas regarding the inconvenience of our daily lives. For example, we’ve discussed integrating student card & room card, also better path advisor based on our current location. Finally we turned to the library, where students in UST are “living in”.

My teammates raised the problem that the library is often full, and they’d be very disappointed after coming all the way to the library and finding no seats. We also interviewed the staff at the library counter, and got to know that they are using a very old way of counting people.

After talking with more people, we confirmed to further develop into this problem. So our next goal is to design a smart way to count people inside the library, and deliver this information effectively to anyone who wants to know it.

Library people counter

How to count the people? We’ve come across different plans, including an image identification based on the existing security cameras or body temperature sensors.

But then we came up with a simpler, low-cost solution: break beam sensor. As all the entrances and exits of the library is one-way, we can deploy such sensors on each exit, and breaking the beam indicates a person in/out. Moreover, we can utilize the existing entrance gate to count incoming students by uploading the total card scanned.

We also discussed about the possibility of counting people in different floors. Theoretically if we put enough break beam sensors at the lifts and the end of the stairs, we could get a statistic of how many people are in the certain floor. However, it’s relatively hard and inaccurate to use break beam sensor to tell the direction of moving, and our goal is to help students decide whether to go to the library, so we did not expand this discussion.

Design

My teammate designed the user interface with the Google Material Design, showing the occupancy and the trend of student number in library, other space is used to display news and available rooms.

Video Prototype & Presentation

To demonstrate our ideas & designs, we decided to make two videos in our presentation: one for emphasizing the problem, the other for illustrating how our system works. I takes some videos and edited them, fast-forwarding the way to library to emphasize the problem we had.

Then, my teammate made the technical demo with excellent skills in 3D modeling and animation, vividly demonstrated how our system supposed to work.

Reflection

In the whole process of designing, we keep think about some complicated designs to improve some aspects of our prototype; but soon we realized that we just need to keep simple & easy. Complicated designs are often expensive and impractical, while simple, delicate design are often more valuable.

Having good communication & cooperation is definitely a key to the overall progress. With a good structure and division of the whole task, we can significantly accelerate the whole process.