How do you feel? What are you doing? Where are you?
Emojis, tiny images people use instead of words when texting on smart phones and social media, are the answer. But what do they really say? How do we know? Why are they used when it sometimes takes longer to find the emoji than to type its meaning?
The answers to these questions (and many more) are the topics of Dr. Jing Ge’s work at UC Berkeley and her passion.
Emojis are more than an image on social media telling the world how someone feels. Emojis, when used together, can become a complete thought like an actual language. Dr. Ge studies the ways emojis are used around the world and how emojis are used to communicate.
Emoji’s tend to be visually universal, but each can have a unique cultural meaning. A positive emoji in China could be a negative one in the United States.
Some think using emojis is going backwards in communications, going back to using symbols instead of words in language.
Dr. Ge explores the expanding uses for emojis, for instance, in the medical field when a patient has a difficulty expressing themselves. Emojis can bridge the communication gap. Similarly, people with trauma can benefit from emojis since emojis can reveal much more than feelings, possibly even revealing crisis or medical needs.
check this out
We recently sat down with Jing Ge, a Postdoctoral Research Fellow and Lecturer in the Department of Anthropology at UC Berkeley, to talk emojis. We met up with her at the Web Conference in San Francisco, where she presented her research with emojis.
Editor’s note: Not only is Dr. Ge interesting 
because she studies emojis, but also when she’s not teaching. You may find her, running marathons, in a race car in Laguna Seca or in her art studio—working on her newest piece. Including our cover art.
Chet Cooper: Can you talk about the presentation that you gave at the Web Conference?
Ge: The topic was emoji sequence use to enact personal identity. We talk a lot about emojis, but one thing I want to argue is the use of a single emoji to express sentiments and emotions. Yet, there’s one thing they’re missing which is the phenomenon of people who use two or more than two emojis to express different semantic meanings. This idea is huge.
Celebrities and politicians all use emoji sequences to convey an idea or tell a story. If you think this doesn’t matter, or you don’t care, that’s fine. But, if you really look at it’s history and growth—there is even an emoji writing class, an elementary teacher helping her children communicate, and how children can write emoji as part of their education.
Cooper: Do you think it’s going to change in the way we communicate, that we’re using symbols—are we kind of going backward?
Ge: This is the discussion, is emoji a new language or an old language? I won’t say it’s really fully changed. Formal language is always there. But I would say it is a digitally mediated graphical language. There are no grammar properties, so you can’t say it’s a formal language. But it is evolving into a digitally mediated graphical language. It can substitute for words and phrases and convey a complete idea. So based on this argument, I have looked at the pragmatic and communicative functions of emojis. Now I look at the grammar-like patterns.
What are the patterns? When they string the different emoji together, it looks pretty crazy, like nonsense, but there are certain patterns that they string together. I won’t call it grammar, but there’s a grammatical pattern.
Cooper: What they’re doing is creating a string as a sequence to communicate?
Ge: Yeah. In order to make it more understandable. Otherwise there are no patterns and you won’t understand. It is not randomly put together.
Cooper: But an author understands the language of their emojis, where maybe an outsider would look at it and say it’s nonsense?
Ge: Yes.
Cooper: It’s both sides of reading these symbols. But maybe the general population hasn’t caught up to that.
Ge: A few of these emoji sequences are really something (used by) millennials and teenagers; they put them together for having fun. A student shared a story that really, really touched my heart.
check this out
She volunteered working in a care home. The people have dementia, and there were two people who had dementia who couldn’t speak. They couldn’t type because they didn’t know how to spell things. They couldn’t communicate. They felt frustrated, that is, until my student signed up for my emoji course. She taught (residents) how to use an emoji to communicate to text back and forth.
People who have dementia use emojis to talk. And, they talk more. And, they are happier. They have really bonded with my student because they can communicate. In the beginning, they use one emoji, like hearts. My student asked, “Do you like ice cream?” And, the response would be “heart ice”
Cooper: They taught them emoji language?
Ge: Yes. They’re more and more confident, and these two ladies just put up two or three emoji to convey the idea. Like “love ice cream” or “arm cry” like, “My arm feels pain.” When we look at this real-life example, can you say that these are nonsense? They’re not. They’re not nonsense.
Cooper: It’s also not nonsense for people who have dyslexia and have a hard time writing something, where the emoji can easily represent the thought process without having to write something that an emoji can convey with one symbol.
Ge: In my talk this morning, I specifically looked at how (people) use emojis. Every time we talk about things, when we talk in public, we try to express ourselves and try to establish our own personal identity—who we are. So this morning, I talked about how users use emoji sequences to express who they are—to establish their own personal identity. That’s the basic topic that I delivered this morning. I looked at the linguistic elements.
When we talk about things from the linguistic perspective, we use different linguistic elements to enact our personal identity. We can express our stance as a first-person pronoun, or use an attitude marker, like “I.” You use the first-person pronoun to express your personal stance. I look at what type of linguistic elements are embedded in emoji sequences that are used to express themselves and establish their personal identity.
Cooper: Can you give an example?
Ge: An example, the first emoji is green tea. The second emoji is sad face or disappointed face. The third emoji is a fox, an animal emoji that means fox. In this sentence, the user establishes or enacts their personal identity through the second emoji, disappointed face. That’s her attitudinal marker. And also through self-mention, because that fox is the emoji to represent herself as a first-person pronoun.
Cooper: So she can’t have tea with her friend?
Ge: This is nonlinear. This is also the beautiful part. They don’t have a subjective verb and object. It’s completely opposite. They say, “For green tea, this like I.” Basically, “I dislike green tea.” The user uses the fox emoji to represent herself, that is the first-person pronoun I.
check this out
Cooper: But that’s going left to right?
Ge: Yes. It’s nonlinear. That’s the beautiful part of emoji sequences.
Cooper: Nonlinear? I thought it had to be linear.
Ge: There are a lot of nonlinear looking at the patterns. They don’t follow subject-verb-object. The order is object-verb-subject. It’s amazing. It’s more pragmatic. They put the word they want to emphasize in the front.
Cooper: So she was saying she doesn’t like the tea. So she shows the tea first, she’s not happy about the tea, and then her avatar is the fox.
Ge: Exactly. There are two linguistic elements. It’s attitudinal marker—
Cooper: But how do the people know she’s the fox?
Ge: This is because I collect data from celebrities. That fox is her name in the TV show, so that’s more well-recognized. And again, it needs a lot of contextual clues to analyze. It’s more like discourse analogies, when you look at the whole context.
Cooper: So linguistically. it makes sense for you to study this?
Ge: I want to apply this in a different context, to make a greater impact. In the next phase I want to apply emoji research on students mental health and disabilities.
Cooper: Can you expand on what that means to you?
Ge: For example, with students right now, we know, specifically at Berkeley, depression rates are increasing. Students get more depressed. What are their problems? But, if we interview them and say, “Hey, what’s your problem?” they don’t want to tell you.
I want to use emojis with different emotions and sentiments—this is one direction I can pursue, more straightforward—that help them acknowledge their feelings and become more aware of their emotions. Whether they’re depressed or angry or frustrated, it can be expressed.
This is straightforward. And even better, they can generate an emoji and use the emoji to write something. And I can use the emoji to know their thoughts, their insights about their courses, their lives, their campus life, their opinion of their professors and so on. This is a way to know students’ insights, their needs, their wants, their problems, and their emotions.
We know they’re not happy, but “happy” is too abstract. Why aren’t they happy?
Cooper: And, to what degree of unhappiness, and what degree concerning mental health issues that are severe.
Ge: Yes, and after this course I’m working on a kind of self-reflection article, student engagement through happiness. It’s more like a blog post that will be published by UC Berkeley. I want to see how to engage students through happiness.
Cooper: Is that still emoji-based? Is that separate?
Ge: Separate. Happiness, because I’m really interested in students’ mental health?
Cooper: You don’t care about anybody else’s happiness?
Ge: (laughs) Of course I care about everybody’s health! That’s why I study emojis.
Cooper: When you say that, you’re supposed to put a happy face.
check this out
Ge: (laughs) I should! 
Do you use emojis?
Cooper: Sometimes, I did this morning. We had a conference call before I came here. Our side was having a difficult time understanding another company’s proposal. I emailed our team, “Let’s talk on the phone before the meeting,” my emojis were a head blowing up and then a happy face.
Ge: (laughs) Nice! I like it! Perfect!
Cooper: I find it frustrating when I attach an emoji on my cell, but when it goes into text, it changes.
Ge: Yes, that’s one thing they have to fix. Sometimes I send one emoji, like “I’d enjoy sauna,” my emoji clipboard shows that. And when my friend receives it, it’s “A guy enjoys sauna!” Google, Microsoft, Apple have their own emoji keyboards, so that’s why it renders differently on different platforms and different devices. This is a problem.
Cooper: I sent a thumbs-up, and it came across as a middle finger.
Ge: Oh, that’s impossible! (laughs)
Cooper: You mentioned doing some work on disabilities.
Ge: I’m really interested in digital anthropology. Have you heard of the nonprofit organization Design Justice? Today we talked about AI (artificial intelligence) technology. One thing I realized is how these fancy technologies can facilitate people from different groups like disabilities, people who have dementia, people who have hearing difficulties, and any group of vulnerable people. To what extent can they benefit from these technologies—AI?
Cooper: What country are you from?
Ge: Originally, I’m from China, but I’m half-Japanese and half-Chinese.
Cooper: Is there a difference in cultural use of emojis?
Ge: Yes, definitely, especially the sentiment. Smiling face in a Western cultural context means you are happy, but in an Asian context, especially in China, the smiling face is to be polite. If you send me something, then I send you a smiling face, it doesn’t mean I’m happy. I just try to avoid the silence, to be polite, just like “Got it.” It doesn’t mean that I’m happy. It’s more the cultural context, and also the emoji that’s smiling, you cover your mouth.
Cooper: I see that in Asia all the time. They constantly put their hands over their mouths.
Ge: There’s an emoji for that. I’m thinking of Western context of my students and some of my colleagues. I don’t know what scenario you use emoji with mouth covered. How do you interpret it if I send you an emoji with my hand covering my mouth?
Cooper: When I’ve made a joke and they laugh I thought they were thinking, “Maybe I shouldn’t be laughing about this.”
Ge: OK, but in Chinese and also Japanese context, as women, we’re supposed to laugh without showing our teeth.
Cooper: It’s a cultural thing about teeth showing?
Ge: Yes, in the Japanese or Chinese culture, it’s not nice smiling with our teeth exposed. We wish to be more elegant.
Cooper: I don’t see men doing that.
Ge: That’s just for females, to be more elegant, more well-educated and you’re not supposed to show your teeth. If you watch the old movies that go back to the Asian times, you’ll see, it’s just like that. That’s also a deeply culture-related emoji. So in this class we cover social uses of emoji, social functions of emoji, and then emojis used in different cultural contexts, emoji and gender, and emoji and race.
Cooper: Can you give me some samples of those last ones?
Ge: This is really interesting. I didn’t study this, but there are some studies on why the people don’t use the white skin tone color emoji online. I asked my students, and they said they won’t, either. In emojis, we have a different skin color. You have black, yellow, white. You’re supposed to use the white emoji skin color because you’re white, right?
This study shows that white people don’t use the white skin tone color to indicate their real race. And in some scenarios, white people use black skin tone color when they use emojis. You know why? First of all, they don’t want to show their white privilege. That’s why—for example, say, black people or people of color post something to gather support from somebody, white people don’t want to say, “Yeah, I support you, and by the way, I’m white.” They don’t want to overemphasize—I think color and race is really overemphasized in the West, especially North America, in this context.
So people try to de-emphasize, downplay their race online, so they don’t use it. And for the people who use black skin tone color, they try to create this affiliation with the black community because the black community is huge here in the U.S. So the white people use black skin tone color. They try to have this affiliation to build this intimacy with this community.
Cooper: And you’re picking up this data from Twitter?
check this out
Ge: Yes, Twitter. And also, white people use black emoji because their skin is dark, not necessarily to express their race. So there are some ambiguities there, but there are some motivations why they don’t use white skin.
Cooper: Picking up these analytics, the metrics of whoever has done that study, do you know if they talked to some of these white privileged people to see if that’s the purpose of them doing that? 
Ge: In this class I have a lot of white students, and I just asked them. And it’s the same answer from them. It’s quite interesting. They said, “We don’t want to overemphasize our skin tone color online and exclude ourselves from this online community.” They just don’t want to do that. “Hey, I’m white, I’m so awesome!”
Cooper: And when you’re using emojis, do you identify your background?
Ge: Because I’m Asian, we just use the default, the yellow skin tone color.
Cooper: I’m using the default. But these people are specifically grabbing skin tone emoji that are beyond what they are, purposely? They’re not defaults?
Ge: Not default. They can choose black, yellow, or white to indicate their race. This is part of emojis, gender and race. In this whole course, it’s really about emoji from anthropological perspective, how people from different communities, different cultures, different societies use emojis to interact with each other and to form the communities. It’s quite interesting, a quite interesting course.
Cooper: Tell me about your class at Berkeley.
Ge: The name of my course is “Anthropology of Social Media: The Study of the Emoji.” This course looks at how emojis have been used in the different social and cultural contexts and how emojis are integrated by people from different groups and communities. We have also looked at emoji and gender, how female and male emoji users use emojis differently.
Cooper: Can you give me an example?
Ge: For example, females use much more emojis than males. Females use more face-related emoji to express their emotions, versus guys use the heart emoji more.
Cooper: Guys use heart more than women?
Ge: Yeah, but women use more face-related emojis.
Cooper: So they’re showing more specific emotional connection with the emoji because the faces have different expressions, where the male is just more general?
Ge: That makes sense when we think about language and gender. Women tend to express more emotions than guys. This is not surprising, but I think the most surprising part we have talked about in class is when females talk to females, they use more polite and intimate emojis like hearts or the face-related emoji, like the smiling face. But when we put a female in the mixed gender group, when females talk in this online group, there’s males and females. When females use emojis in this scenario, they use the winking face more. They express humor and flirt with somebody else and try to express that they have a sense of humor.
Cooper: So a person uses humor, and they consider that flirting? Or is that just a general concept?
Ge: That’s a general concept. So definitely, females use more emojis than guys. And guys use a different type of emoji, and the pattern’s different. Females use more emojis dispersed in different locations in one sentence, versus the guys who use a string of emojis. The guys use repetitive emojis as a sequence or a sequence of repetitive emojis, but the females use many emojis in one sentence, but they disperse emojis in different locations in one sentence.
Cooper: How are you gathering data? Are you using some form of AI?
Ge: There’s some Twitter data that’s public, and also I look at Chinese Weibo that’s also public data. I just collect the data that’s open to the public domain.
Cooper: Is there such a thing as an emoji community?
check this out
Ge: This morning, this conference, this emoji workshop, is a community. I was there last year at Stanford, the first International Emoji Workshop. And this year it’s the same. We meet and stay in touch and we share our work. We do a lot of collaboration with each other. So yes, it is a community.
Cooper: Have you heard of anyone starting a community purposely only to use emojis, no regular text?
Ge: Yes. There is an app. This guy, he presented his work, I just now talked to him. He built this app called “Emoji First.” You’re not allowed to put text. You share where you are and you share your thoughts, your opinions about the service at the restaurant, whatever. You only use emojis. There’s a platform. Now they have 3,000 users. And I would say that’s a community.
Cooper: What was the most surprising thing you found out about emojis?
Ge: I would say it’s really an eye-opening experience. As a researcher, I focused on my own topic, quite a narrow one. I focused on one or two perspectives, that’s it. But during this process, I expanded the horizon of my understanding about emojis, because I have read a lot of articles about this emoji research community, what’s going on there, how emojis have been studied from different perspectives and also different angles. This is the first thing.
The second, I think, personally is—the intellectual capacity. Emojis are emojis. In order to teach students, I have to answer one question: to what extent, or how can an emoji course contribute the anthropology. Otherwise my course cannot be passed.
I gave myself the space—this surprised me. I confronted the huge theoretical foundation. And also, by studying emojis, I can advance some of the communication theories, anthropology-related theories and methodologies. This is the contribution. I never thought an emoji course could make such a contribution.
These theories are old, and they’re used to study texts or photographs or visual communication. But by applying these theories to study emoji, it surprised me how this type of research can advance different types of communication and cultural theories. That is an encouragement because I see the impact of my research, not only the practical impact, but also the theoretical impact. This also surprised me.
Cooper: How would you say to a layman that you’ve gone from A to B?
Ge: Whatever we do, whatever we say, we can categorize our speech, our words, into different categories. This is direct if I ask you a question.
We have one assertive category, where you make an announcement or give factual information. That’s assertive. And then when you make a compliment on my boots, that’s expressive. You express your emotion, opinion, attitude.
When you ask me—do you have time later, that’s directive, or if I ask you something or ask you to do something.
There are five categories of speech act. When I study emojis’ differences, I want to apply these different categories. What category can be categorized by using these different categories? Some emoji sequences just cannot fit into these categories. What to do? I have to expand; I have to create new categories.
Traditionally, speech acts only have five categories. But these five categories cannot be really fully used to categories emoji sequences. I have leftover data. I have some emojis with nowhere to go. They’re like orphans. What to do? I create a new category.
Cooper: So you adopted them?
Ge: Yeah, I adopted them. (laughs) I gave them a new name—
Cooper: —and purpose.
Ge: Yeah, and now they’re family.
Cooper: How are you describing the sixth or seventh categories?
Ge: I look at their features. When you want to describe something, this cell phone, there are some characteristics, some features, and I use these features to come up with a name. Does that make sense?
Cooper: Dealing with objects, descriptive? Did you come up with a name?
Ge: Yes, I did.
Cooper: What were the orphans’ names?
Ge: It’s still in the process. Just like, you have your Monday, Tuesday, Wednesday, Thursday dress code. And then if all of a sudden you need to go to another occasion or another social event, and then you realize that your traditional dress code categories cannot be used with that occasion, what to do? You create another category of dress code. You need to bring in new things. That’s the theoretical contribution.
Cooper: Let’s go back about the disenfranchised groups you mentioned before. You were talking about people who are blind, deaf, etc. How are you envisioning the emojis support them, if that’s where you’re going?
Ge: I think first is the categories from the design perspective, there’s an LGBTQ group. Again, we want to use emojis to represent ourselves. I talked to people from transgender and LGBTQ community and asked them, “Can you find an emoji to describe or express yourself?” The answer is, “No.”
check this out
People with differences, and there are a lot of—even people who have, let’s say, mental health issues or feel depressed. Can you see an emoji to indicate, “I feel depressed?” If I send this to you, you know you probably need to give me more support. This knowledge will most likely translate to your action. If you know, you probably will change the way you talk to this person, you’ll be more careful or more attentive.
Cooper: Are you saying there should be something more serious in the emojis that are developed that can express depression?
Ge: Mm-hmm. And, also for people with eating disorders. I did some data mining, but I didn’t finish. But the preliminary finding is that a lot of millennials and teenagers use the pig head emoji to express when they have an eating disorder on Twitter.
Cooper: Are they saying they have an eating disordered, or someone else?
Ge: They have. If I have an eating disorder—
Cooper: They’ve chosen the face of a pig?
Ge: Yeah. They use this pig emoji a lot either in a text or beside my user name.
Cooper: So it’s the only thing they could find that they felt like they could use?
Ge: Yeah, they are not satisfied with their body shape and they have an eating disorder. And this is one thing I realized.
Cooper: That’s interesting. It’s just the opposite.
Ge: Exactly. This is the research I presented in California at the Academy of Science. This year is the Year of the Pig.
Cooper: For China.
Ge: Right. So my topic is how the pig from China meets the pig from North America because in China the pig is really cute, humble. It’s a nice symbol. But in North America it’s negative. People use this to convey information regarding corruption, bankruptcy, eating disorder, mental health.
Cooper: That’s a real difference. So you’ve always thought of the pig as being adorable and cute and not anything in a negative context?
Ge: Yeah. I looked at the data from Twitter and it was pretty amazing. The emoji can really help people to—I don’t know, express their problems. This is huge. We really want to predict something, that this person has the intention to commit suicide. If we can extract some information to help us to predict something, we can save people’s lives!
Cooper: Do you know of anyone who’s talking to psychiatrists about this, looking into any way to gather some data about this, usage, clinical trials?
Ge: No. This is my next step because I’m really interested in mental health, especially students’ mental health.
Cooper: Do you know any psychiatrists to work with?
Ge: I’m still working on—I’m approaching professors from public health at UC Berkeley. I’m really interested in students’ mental health. If I can put emojis in this context, I could make a little contribution. I think that’s a great idea. It’s another unknown path to pursue to make a bigger impact.
Globally, you don’t have to have learned grammar rules and spelling. I looked at some other papers, about rural China where a lot of farmers don’t have a higher education, people with lower literacy use emojis a lot. They don’t have to think about grammar and how to make sentences. They just express their ideas. If that works, why not?
Cooper: If you have dyslexia, an emoji is a quick answer without taking time to check spelling and sentence structure. Emoji with a purpose. That’s your new presentation.
Ge: (laughs) Yes, emojis can be much more than cute symbols.
