Category: F2023 – Shiloh

For my midterm project, I’m attempting to avoid games in favor of creating something that involves experience and knowledge for the user to explore, as well as material that is helpful, positive, and reassuring. As a result, I decided to design something resembling an old museum with a theme resembling an underground archive, with some art pieces featuring positive words or advice from historical personalities, and another area of books containing Quran verses in Arabic and English. I’m going to try to provide calming and helpful verses. 

In my main page, I will try to add more aesthetics and try to have a mix of European, Greek and a bit of Arabic aspects since the underground museum would involve historical figures from different backgrounds as well as verses from the Quran. For example, the temples in the main page represent Doric Greek temples, and the lanterns can be considered part of many cultures but I initially associated it with the Arab / Islamic culture. The user would be able to click the “Enter” button and enter the museum, and they would be able to use arrows to move between the two sections in the museum. I’m also aiming to give the user the ability to click on the books and pictures to learn more about the historical background, take advice, or enjoy flipping through  positive quotes or reassuring and guiding verses from the Quran. I also plan to add fire sound, audio sounds for certain clicks and papers.

Most frightening part 

I’m mostly worried about making the buttons take the user to another page or scene, I’m currently trying to do that for my main page which will take me to the first section of the museum. I’m not sure if I will be able to have two sections but it’s definitely something I’m aiming for. I’m also trying to figure out if I can make a book and the user will be able to flip through a few pages. 

 

For the midterm, I initially had many ideas some were games, and others were experiences, However, the overarching concept that resonated with me the most was the idea of exploring the underwater world, as it has always been a significant source of inspiration for me. I envisioned creating an immersive underwater experience, inspired by my childhood favorite movie “Finding Nemo.”

The whole idea is to bring human experience to the fish world. As a result, I wanted to create a street feel life but under the sea. There fish can go shopping and view artworks in a museum.

I initially was trying to find a picture of already made imagined under the sea life, but I could not, so I had to create my own. Using vector-based images I created different layers of sea life and buildings to seem they are under the ocean using Adobe Illustrator.

More details on the midterm

I first created all the main layers I wanted. Then I tried to figure out the best way to implement it. I was initially worried about how I would figure the logic out but after I while I organized it so I had different functions and classes doing different things.

IMG_8096

Loading the images and sound was challenging because the P5js is case-sensitive. I honestly was struggling through the whole process of coding but I wanted it to work. The code has some bugs but the feel is there. I still have to figure out why the sound in P5js sounds different from when I play it on my computer, and how will the user view the artwork and the bags in the mall. There is still a lot of layers to be added and improved to make it even more interactive.

Computer Vision for Artists and Designers

This article illustrates the historical development of computer vision techniques using notable examples, and then provides practical instruction to artists and designers on computer vision principles and optimizing their physical environment for the camera. I found it interesting how the development of technology always seems to welcome a process of democratization but also capitalization and surveillance with negative consequences. For example, before the advent of the printing press and similar technology for mass printing, only a select few people could print books. The ability to disseminate texts is a tool, but what people do with it can cause both positive and negative consequences — such as distributing hate speech or propaganda. Similarly, the ethics of computer vision, now that it is so democratized, is intriguing to contemplate.

Two example usages of computer vision particularly stood out to me, Krueger’s Videoplace, and Suicide Box by the Bureau of Inverse Technology. The concept behind Videoplace, that it involves the whole human body, is something that I think could be used to positively offset the fact that modern humans sit at screens all day. Would it be possible to make word processors or computer operating systems controlled by larger upper body movements, or even foot movements, like floor pianos, so that we can still move while doing our schoolwork/jobs? This line of thinking was inspired by the podcast Body Electric, exploring how the human body has changed in response to technology.

The Suicide Box also raised many questions for me. How did it know to only detect human vertical movement, and not if someone threw something from the bridge? How come the numbers of suicides it recorded were so much more than the officially recorded amount by the port authority? For a topic as emotionally charged and taboo as suicide, is it empowering to record data to bring awareness to the topic, or does it cause more harm than good (as 13 Reasons Why’s suicide depiction did)?

CONCEPT

Title translation: Welcome, a Thousand Welcome.

When I saw the coffee shop experience example, I was inspired to create an educational/cultural experience that has a fun aspect to it. The concept of this midterm project will be a dialect learning experience from which the user will be able to learn a few words in Darija the Moroccan dialect, in a fun way.

Here is a preliminary user guide for what I am referring to:

UI DESIGN:

Frame 1: WELCOME SCREEN

Press PLAY to start

Frame 2: INSTRUCTIONS SCREEN

Press arrow for next screen

FRAME 3: THE HOUSE

Hover the mouse and choose one of the six spaces

FRAME n: 1/6 space

Click on an object to hear the word for the object in Darija

Click on arrow to go back

LOOP

COMPLEX PARTS:

-SOUND RECORDINGS: I’ll be recording all the audio files for the objects. My goal is to at least have 10 recordings per space (6 spaces makes them 60 recordings in total). I’m anxious about managing all the files as well as how will I be able to incorporate all files into the project without making it too heavy and laggy. I’ll test it and in case it does not work, I’ll bring down the number to 4 recordings/space.

-SCENES MANAGEMENT: I have about 9 scenes in this project. The scene switching routine will be a lot of work. I started looking into libraries to help with this matter and found P5 Scene Manager  a library which may make my life much easier. Still looking into how it works.

-HOW TO INCLUDE TEXT: I want to include the English word of each object along with each object. I’m still brainstorming ways to make this possible.

Looking forward to see the result!

After thinking for what felt like forever and hating a ton of games i tried to design an idea came to my mind, why not go old school and turn a physical game into a digital game and what’s better than using my favorite stress relief game “Wack-a-mole”, the game that generations have played, so why not give it a modern twist.

Description: A game where moles appear from various holes on a hill and the user must “whack” them by clicking on them. The aim is to get as many moles as possible within a certain time frame.

This is the inspo i want to recreate:

And this is the rough sketch i came up with so far:

I choose to leave the visuals till later and start off with what was frightening me the most at the time which was having the mole appear in exact random holes and for the mouse click to be able to detect it no matter where it’s touched which I was able to fix using a dist() function, ensuring precise and reliable click detection. However, other challenges remain such as making the game visually appealing to fit the inspo picture as much as possible and in general just look good and fun, adding the score, and as time gets closer to finish the game becomes harder yet remaining seamless.

Risks and ways to minimize them:

1. Game becomes boring: The speed gets faster as time goes by

User Interaction Design:

The end result will have:

1. Start Screen: Press “Start” to begin the game with instructions above.
2. Gameplay: Moles pop up from holes at random intervals. The player clicks on them to score points. (I’m thinking of making the mouse be a mallet and have the mallet hit the moles but i’m not really sure how to do that)
3. Timer at the bottom: A long 40 sec bar that runs down as time passes that changes color based on time left and in the last 10 second it will print the remaining time under
4. Sound feedback: Every time a mole is hit a funny sound will play
5. Difficulty increase:As the timer gets close to running out the moles will become faster at disappearing
6. Scoring: Player earns points for each mole whacked.
7. End Screen: Displays score when time runs out and an option to replay.

 

 

 

Concept

For my midterm, my ideas were all over the place. Eventually, I settled on the concept of creating the disco night project because I think it’s one of those projects with a lot of flexibility so you can be super creative with it.

I want to create a disco night where users have the option to select different characters to dance to different songs and possibly other features too that I might come up with during the coding process.

At first, I considered using sprites or GIFs of random characters dancing. However, I soon discovered that GIFs of real humans didn’t quite blend well with a 2D background, and it lacked the charm I was aiming for. So, I made a decision to bring to life the ultimate disco night with some of my favorite cartoon characters.

One aspect that initially concerned me was finding the right sprites for my chosen characters. As I scoured the internet for suitable sprites, I quickly realized that there were none available for the specific characters I wanted to feature in my project. This left me with two options: create my own sprites or opt for GIFs instead. After some experimentation, I found that using GIFs turned out to be a visually appealing choice that added the dynamic quality I was aiming for.

Computer Vision

I found the reading on “Computer Vision for Artists and Designers” to be quite interesting and informative. It introduced a variety of projects that piqued my curiosity. While I was familiar with the term “computer vision” before, this reading provided me with a better understanding of its scope and the various concepts it encompasses. One key takeaway from the reading was the realization that computer vision techniques can be used to gather real-time information about various aspects of human interactions, including identities, locations, gestures, facial expressions, gait characteristics, and gaze directions. What intrigued me the most was the accessibility of some of these techniques, which can be implemented by novice programmers in a relatively short time.

I didn’t notice biases in the reading and didn’t affect my beliefs on computer vision much. However, as I read the material, I couldn’t help but wonder about the connection between computer vision and virtual reality (VR). It just so happens that I am currently enrolled in a virtual reality research course, and the reading prompted me to explore this relationship further. Hence I searched online and most sources highlighted how computer vision plays a crucial role in AR and VR, particularly in areas such as object detection, gaze tracking, spatial mapping, gesture recognition, and real-time image processing. These applications not only enhance the immersion and interactivity of AR and VR experiences but also bridge the gap between the virtual and physical worlds. I was also able to make a connection between VR experience and one passage in the reading that particularly resonated with me was the description of the “Messa di Voce” project, which used whole-body vision-based interactions, speech analysis, and projection-based augmented reality to create an audiovisual performance. This project demonstrated the power of computer vision algorithms in tracking performers’ movements and analyzing audio signals to create real-time visualizations, which seem indispensable when it comes to creating a virtual reality experience. Overall, this reading expanded my understanding of computer vision and its wide-ranging applications in various fields, including the arts and virtual reality. It sparked my interest in exploring the intersection of computer vision and VR further and how these technologies can be harnessed to create more immersive and interactive virtual experiences that are safe and ethical for different age groups.

Concept

I had a hard time deciding between my two ideas: a game and a musical interactive experience. But after taking my time, I decided to work on the second idea. I had a big project in mind on implementing different instruments, but after thinking thoroughly about implementing those ideas, I realized how many difficulties can appear; therefore, I decided that the best strategy would be to focus on one particular instrument but give it more flexibility and interactivity. That instrument would be the piano.

The most challenging part

While thinking about creating a piano keyboard, I came up with two strategies on how the idea could be implemented. The first one is creating a class for every key and functions inside which will either create the graphical design or program the click functions. Or I could just find the photos of the piano keyboard and assign the click functions depending on the position of the mouse.

I decided to create a class because it gives me more flexibility with animations and making changes.

But the most challenging part of creating the class is creating a sound. If I download a lot of sounds for every note, the program will run slowly, but if I choose to use an oscillator, it will not sound like the piano; therefore, I’m still deciding what should be done. I will most probably stick to importing actual music files and work on program optimization.

Design

I made a beta version of my project, which is not fully interactive and doesn’t contain all the graphical objects that would be added. I will work on the design of the keyboard, and I also came up with the idea of adding music sheets near the piano to learn new melodies.

Furthermore, I am considering the idea of implementing the click function, not only mouseClick but also keyPressed, which would allow to execute the sounds faster, creating a more natural sound. I will try to make the experience more entertaining by creating more options for a person apart from playing the instrument.

Concept:

In my concept, I attempt to take the viewer through an interactive journey of my development and learning through this course. We play a cat, who we can move with the arrow keys. and as we proceed the background reflects the projects from my initial weeks in the class. At the end of the journey, after week 4, lies a final interactive demonstration of what lies ahead for me in the second half of this course.

I decided on doing this project, over a more mainstream idea such as a game, due to its nature of tying together my previous development. I feel this will be a rather poetic tribute, since I am proud of my previous works and what i have learnt!

 

 

Design:

Designing the code has been a more complex challenge than I imagined. A lot of the issues stem from implementational difficulties such as bad organization of my prior code which results in cluttered and unusable code when I attempt to combine elements. Nonetheless, I seem to have made headway into the same.

Other important design elements for me include a dependence on vibrant colors on a black background. My previous projects followed this aesthetic and I believe it is the most aesthetically pleasing to continue doing the same. I have also chosen relevant copyright-free music that shows a sense of ‘adventure’ as the user moves through the adventure.

 

Frightening Aspects:

The most frightening, volatile, and uncertain aspect of this project for me is the final ‘interactive stage’. This seems rather obvious, since I should have the code for the building blocks of the initial stages. However, the issue is deeper than being that of just a technical implementation.

Conceptually, it has been tough to find a way to showcase the ‘future possibilities’ of this course. Technically, a couple ideas I had included features such as minigames where a robot attempts to attack you – which seem far too intricate to design. Finally, I have decided to settle on an interactive ‘task board’, tasks on which would be clickable and would show pictures of robots/AI/ other things related to our future topics of discussion, play music, etc.

Reducing the Risk:

An attempt I have made to reduce the risk is rigorously try out different possibilities that I have brainstormed – in a manner which leaves no doubt as to whether a method is viable.

 

Moreover, I am attempting to see if libraries can simplify my work and code organization beyond what it is now.

 

 

Having conducted research in computer vision, and considering it as one of my areas of specialization, I believe this reading wasn’t specifically targeted towards me. Nonetheless, it was a fun reading and exploratory review of how computer vision can be used for the purpose of interactive medium. But in another way, it has always seemed to me as if those who are closest to a work often fail to recognize it beyond the framework they use it for. In this way, I feel like it may be that the exploration of computer vision and AI in interactive media which is explored in this reading (and that we will explore further in class) may perhaps be the most enlightening for me.

For this reason however, in this reading response, I would like to go a bit beyond the reading. One of the first things that surprised me here was the date of the publication of this article. From what I know, most modern computer vision techniques evolved in the 2010s – thus it was surprising for me to see such creative use of relatively primitive technology going back to the previous century. This also shows that the far more advanced technology available now can probably help us to even more boundary-crossing media pieces.

At this point, I also want to go on an aside about the usage of AI in art. The topic of the ethics of generative art has been in contention over the past year (and rightly so). But if we can somehow have such technology ethically created -then this adds several new dimensions to what art can do and show. Let’s take for example – a technology like ChatGPT. Even though this class rightly discourages its use for generating code, there is still something to be said of its power in being a tool to learn, and how it makes computer art more accessible even to those who have less of a background in coding.