# communikation ulster
[e-mail]

Dear Mr Moore,

we hope you are doing well in these trying times!

We are five students from the University of Applied Sciences Augsburg. Three of us participated at the soHappy project with Leo last summer and know you already.
We would like to get in touch with you, since we are looking forward to our project at the Ulster University this summer.

We have been thinking about the upcoming project and would like to ask you: Do you have some project ideas in mind? Are you interested in topic proposals from our side?

We wish you all the best and are looking forward on working together.

Your Team Ulster 2.0
Anna, Carina, Kilian, Lukas and Marcel

Thank you for making contact and helpful to know that some of your are returning, which I am sure will help the new members of the team.

As for topics ideas, it would be possible to extend the soHappy app, if that was of interest, or to work on a related affective computing topic. For example, I have always been intrigued by the challenge of giving devices, anything from robots to simply IoT devices, the ability to emulate emotion in their interaction with humans.

Alternatively, and a little related to soHappy, I have worked in the area of cognitive fatigue assessment using a smartphone in the past. We explored a range of measures to determine cognitive fatigue and started to identify facial cues which might indicate cognitive fatigue. We had thought about how this could be useful for assessing fatigue for those working in critical environments, e.g. clinical staff in hospitals. There is some data available from this work that could also be explored, although the study size was quite small.

Another interesting challenge, that is related to both of the areas above, is the potential to explore the use of voice to determine emotional or cognitive fatigue.

Those are the areas I am particularly interested in but I am also happy for you to suggest areas that you have an interest in.

Hope that helps,
George

## topics

- soHappy
- cognitiv fatique (speech emotion (fatique) recognition)
- robot emotions (affectiv computing)

Hi George,

we heard about violent demonstrations that happened in Belfast more freuqent in the recent past. How do you experience the current situation?
We hope you and your loved ones are doing well.
Also, thank your for the quick reply and for sending us your thoughts and ideas about the upcoming project.

Since we have new members we agreed to choose a new topic for the upcoming project.
Considering your ideas (affective computing, cognitive fatigue), we would prefer to have a closer look at cognitive fatigue.

1. Could you please provide us some more general information about the topic?
2. What kind of research can you provide, that we could possibly look into?
3. What do you prefer as the main project goal (mobile or desktop app)? Or do you have another result in mind?
4. We were thinking about Dart & Flutter as a programming language for the project, escpecially to support multiple platforms. Do you have experiences regarding Dart & Flutter or can you recommend something more appropriate?
5. Which indicator(s) (voice \ speech recognition, face...) should we use to detect cognitive fatigue?

Thank you in advance and we are still :) looking forward to working with you.

Your Team Ulster 2.0
Anna, Carina, Kilian, Lukas and Marcel

Anna, Carina, Kilian, Lukas and Marcel,

Thank you for asking how we are. As you probably know, Northern Ireland has some trouble at times but it is usually quite local in nature and tends to settle down again quite quickly in recent years. I expect this will be the same, although it will probably come and go, mostly during the summer months.

To address each of your questions, regarding focusing on cognitive fatigue, I have written quite a long email. If you need anything explained further then please ask. At some point a call might be the best way to answer your questions.

1. Could you please provide us some more general information about the topic?
and
2. What kind of research can you provide, that we could possibly look into?

A good place to start might be with the papers we have written on the topic. As well as providing some background and reference to related material, it will also be focused on our area of interest. Perhaps consider the following articles, both Open Access:

Validation of a Smartphone-Based Approach to In Situ Cognitive Fatigue Assessment
Towards Mobile Cognitive Fatigue Assessment as Indicated by Physical, Social, Environmental, and Emotional Factors

3. What do you prefer as the main project goal (mobile or desktop app)? Or do you have another result in mind?
and
5. Which indicator(s) (voice \ speech recognition, face…) should we use to detect cognitive fatigue?

The above research finished with an understanding that the games we have explored could be used to assess cognitive fatigue. It also found that there might be some potential to assess cognitive fatigue using facial analysis. However, the iOS app that was made is very out-of-date and no longer functions. Moreover, it is limited to iOS.

As such, possible goals could be:

Rework all or some aspects of the previous mobile app to be up-to-date and cross-platform. This could be a straight port of the existing features to a new app. This would most likely be a more practical goal than researched based, although it would allow for both areas of activity.

Explore facial analysis further with a mobile app, again perhaps cross-platform. This would probably require some machine learning to support the facial analysis, perhaps related to last year’s work. This would perhaps be a more difficult challenge

I also mentioned the potential to explore voice or emotion as cues for cognitive fatigue. This would take the work into a new area and be more research centred as a result. It would also facilitate development work, again perhaps in the form of a cross-platform mobile app.

All of the above would be of interest and value to me, so it perhaps depends on the nature of the challenge you feel interested in, and what your skills and capacity are as a team. Indeed, the goals above are not mutually exclusive and all or aspects of them could be combined in some way, if that was of interest to you.

The answer below, particularly in regard to what is possible, due to various OS and browser constraints, might also influence this choice.

4. We were thinking about Dart & Flutter as a programming language for the project, especially to support multiple platforms. Do you have experiences regarding Dart & Flutter or can you recommend something more appropriate?

I do not have any experience regarding Dart/Flutter, other than having experimented with it when it was first released. I note that it has recently been updated quite substantially and seems to be very interesting.

While the ability to release cross-platform mobile apps is of obvious interest here, the ability to release them as a Web (for mobile) app might be even more interesting, given that it would allow the app to be made available without the needed to use app stores. As such, Flutter/Dart could be worth considering.

However, from experience of cross-platform development, some care might be needed to ensure that Flutter/Dart will support everything that you how to achieve before selecting it, especially if taking a Web app approach. Typically, access to the camera and microphone, for recording video and audio, is often a troublesome area when taking a cross-platform approach. Largely due to the restrictions Android, and in particular, iOS, or various web browsers, place on this.

Hope that helps,
George

23.8 - 24.9

Hi George,

thank you very much for the quick answer and the detailed information.

Overall, we are very interested in implementing a stand-alone web application.
Currently we are working on some projects where we are able to gain experience with different web frameworks such as Angular, React and Django / Flask. We could use those experiences and expand them further but we are also open to use other frameworks.
We would like to keep the research part small and focus on implementing the application.

In order to talk about that in detail we would like to take your suggestion and we would appreciate a video call in the near future.
We are all available every Tuesday from 5 p.m. and Friday from 11 a.m.
What time slot fits you best?

Regarding time frames for the project, we were considering 23.08.2021 - 24.09.2021. Are you available during this period?

Your Team Ulster 2.0
Anna, Carina, Kilian, Lukas and Marcel

---

2021.05.07

Hi George,

thank you again for the last meeting, it was a pleasure to hear your thoughts and the background story of the previous studies about cognitive fatigue.
We would like to schedule a second meeting, because we think it would be interesting to dig deeper into your interests (affective computing) or fantasize about the project.

Mainly, this would be less about organisational stuff and more about getting to know each other better.
How about Friday next week, same time (12:00 CEST) also in BBB: https://public.senfcall.de/ulsterunteraugsburg

You could also give us a chance to work on our small talk skills ;)

Your Team Ulster 2.0
Anna, Carina, Kilian, Lukas and Marcel