MIST

Multimodal Interactive Speech Toy

Type
iOS
Time
Role
UI design
Development
Icon design
Illustration
Sound design
Tools
Xcode
Figma
Sketch
Logic Pro
Procreate

Introduction

MIST stands for Multimodal Interactive Speech Toy. It responds to user interactions like pressing buttons for each emotion, touching the screen, and using the joystick. On this page, I’ll guide you through MIST’s development process.

The Idea

MIST is a toy-themed app that doesn’t aim to boost productivity or play songs like Droplay!. Its development began as an experiment following the release of iOS 18 and its SwiftUI APIs. This year, Apple introduced TextRenderer, a protocol that allows for more flexible, character-by-character string rendering. This immediately caught my attention. While exploring ideas utilizing this renderer, I recalled a childhood game known as モンタージュ遊び (Montage Play).

In this game, you randomly pick a number for each facial part—like eyes and mouth—and end up with a surprising combination. It was my favorite back then, though I am still not sure why. This simple childhood game weirdly stuck in my head, and I thought it would make a fun little app if I could combine creating different faces with speech rendering using TextRenderer.

The First Concept

Since this app main appeal is in offering brief moments of visual expression, it needed to be both aesthetically engaging and visually intuitive. The first iteration explored a handheld gaming console-inspired theme, typically featuring a joystick, speaker, settings button, and the familiar four-colored buttons. However, since the app didn’t require sequential actions with buttons, I eventually removed them. During these iterations, I came across the classic Speak & Spell, a popular children’s toy from the 70s, which provided further inspiration.

This toy actually inspired me when developing MIST (which was originally called “Misty,” by the way). I thought, “What if I added buttons so MIST could speak its own sentences in its own voice? That would be neat!” This idea aligned perfectly with the initial vision of giving MIST a face and speech visualization. That was the moment when the basic concept for MIST really came together.

Development

I have always loved apps that serve a single purpose without many transitions between interfaces, which MIST follows closely (just like Droplay!). Because of this simplicity, there was not a need for extensive design work in Figma, and I was able to dive directly into Xcode, designing and implementing at the same time—a process I always find enjoyable.

I started by defining what MIST should express: what sentences, how long, and in what tones? To make it relatable and evoke user empathy, I decided it should convey the major emotions felt by living creatures (mainly Homo sapiens): Peace, Anger, Joy, Confusion (Messy), Sadness, and Emptiness (Void), so to speak.

Face

To give MIST a unique personality, achieving vividness and a touch of goofiness in each expression was essential. To fine-tune the animations and values with precision, I built an internal preview tool that displays every expression of MIST. I also had to ensure smooth transitions between expressions; otherwise, the facial components would move awkwardly during changes.

This was especially challenging when transitioning from one complex expression to another. For instance, moving from a frowning face to a confused face requires more intricate path adjustments than transitioning from a neutral face to a closed smile. Achieving fluidity and minimizing unnatural movements is key to bringing software to life.

Text

Text movements add significant momentum in showcasing the personality of the software, which is where the TextRenderer comes in—I spent a lot of time perfecting it. I also built an internal preview tool to refine this process. Tweaking the text movement was more challenging than other elements because I needed to balance legibility, expressiveness, and how well it blended with facial movements.

Interestingly, finding the right animations for positive emotions was relatively straightforward (though the mathematical calculations were anything but). Scaling and rotating provided just the right effects to convey them. However, for negative emotions like messiness or sadness, these methods did not quite work, and I had to explore other approaches.

In the end, the movements really helped in expressing emotions non-verbally. I am pleased with how they came together in MIST.

User Interface

It was clear that the user interface of MIST is crucial for its aesthetic integrity. While aiming to make it as intuitive and simple as possible, it also needed to incorporate the design and interactions of a handheld gaming console. For instance, all operations, including navigating the menu, are controlled with the joystick, with no tap actions included.

1 / 3

Sound

Sound design for MIST was one of the most challenging parts since I had never tried it before. Although I know the basics of music, I researched and learned a lot from the Internet and through playing instruments. The most helpful resources came from the gaming industry, which has shared so much valuable knowledge and helped me in many ways. I am quite happy with the results for my first project involving sound, and it has definitely expanded my skills as a creator.

App Icon

To be honest, designing app icons is always the most fun part, and it literally took forever to get to where I am now. Here are some of the first drafts, though they are not too far from the final versions. I have always loved logos or icons with a rainbow or rainbow-like gradient, probably inspired by the classic Apple aesthetic. Both Flashback and Droplay! have it, almost for no particular reason. However, MIST needed a rainbow since it represents six different emotions, one of its standout features. After countless iterations and switching ideas back and forth, I decided the app icon should truly reflect what the app is all about, so I chose to depict its face and controls.

Working on MIST gave me a great opportunity to experiment with various app icon themes and the new appearance styles introduced in iOS 18: Light, Dark, and Tinted. I will admit, it was a bit challenging to apply these styles while keeping the original layout and elements intact.

Light
Dark
Tinted
Light
Dark
Tinted
1 / 3

Illustration

To bring the world of MIST to life, creating App Store cover illustrations was essential. I started with a theme inspired by 60s magazine ads, and I have to say, I am really happy with how it turned out—the grainy texture of aged paper and the classic magazine layout feel just right.

Reactions

MIST was released on November 7, 2024, and I am incredibly thankful for the attention it has received—far more than I expected. It seems people genuinely appreciate MIST, and here are some of my favorite reactions.

Building MIST was truly challenging—I spent countless nights fine-tuning tiny details that most people might not even notice, all in pursuit of achieving a balance I could be fully satisfied with. It is incredibly rewarding to see that there are people out there who resonate with my philosophy and appreciate the effort behind it.

Download

I hope you enjoy the page and find MIST intriguing! If you’d like to check it out, here is the product page. I would love to hear any feedback you have. In the meantime, have fun with MIST!