Month: October 2023

Golan Levin’s reading sheds light on computer vision and its role in interactive art and creative design. It’s intriguing how the seemingly contradictory worlds of computers and art come together to create interactive art that deeply engages people. This irony lies in the fact that digital technology often absorbs individuals, disconnecting them from the physical world. However, when art and computers combine, they captivate and transport individuals into alternate realms that encourage technological interaction. During this interaction, artists embed powerful meanings into their pieces, which become all the more memorable because people physically engage with them.

On a different note, one aspect that stands out is how computer vision has evolved beyond art and found applications in diverse fields, including healthcare, transportation, security, and entertainment. This evolution reflects the pivotal role of computer vision in our modern lives. However, its important to note that the reading also raises ethical and societal questions, particularly in the context of surveillance-themed artworks like “Sorting Daemon” and “Suicide Box.” These pieces challenge us to consider the implications of using computer vision for both artistic and surveillance purposes, which ultimately blurs the line between observation and intrusion.

In conclusion, computer vision has undergone a remarkable transformation, becoming a powerful tool for artists, designers, and creators across various fields. Its integration with cleverly designed physical environments showcases the interdisciplinary nature of interactive art and design, bridging the gap between the virtual and physical worlds. While offering incredible creative possibilities that often proffer a deep and meaningful message, it also prompts us to reflect on the ethical and societal implications of this technology.

The study by Golan Levin delves into the integration of computer vision algorithms within the realm of interactive artworks and creative design. While importance of cleverly designed algorithms that would allow computers to make smart interpretations about digital images and video is unquestionable, I kept thinking about the role of other factors influencing computer vision. One such factor is the optimization of the physical environment where the video data is collected. Especially in the case of artworks and the creative field where the physical environment can be controlled, it is important to consider how the visual data collection is staged, as it can either increase the reliability of computer vision or make it more difficult to analyse. Cleverly designed physical environments can potentially decreased the need for more complex software solutions. This concept highlights the creativity necessary in the world of interactive art and design. It implies that artists and designers can use their spatial and environmental design skills to develop setups that allow for accurate data collection. They can extend the capabilities of computer vision without resorting to advanced and intricate software development by deliberately altering the physical context. This approach reflects the field’s interdisciplinary nature, in which creativity goes beyond the digital domain and demands a complete understanding of both the virtual and physical components of interactive artworks.

A compelling example of the principles discussed in the paper can be found in the artwork “LimboTime.” “LimboTime” is a simple game that was conceived and implemented in a remarkably short span of a single afternoon. Its creators utilized a wall-sized sheet of white Foamcore as an ideal environment for brightness-based thresholding. The usage of this material enabled them to effectively find the game’s three players against the background, demonstrating how physical environment optimization may contribute to computer vision success. Rather than depending on complex software solutions, the detection of players’ hands and heads was accomplished by creative heuristics, such as recognizing the limboer’s head as the uppermost point of the middle ball of black pixels. This elegant solution showcases that a well-designed physical environment can improve the performance of computer vision techniques, simplifying the implementation process, and fostering innovation in the realm of interactive art and design.

For my mid-term project, my initial inspiration was to create a minimalist cooking game, a concept deeply rooted in my childhood experiences. During those early years, I would eagerly scour websites in search of every cooking and Barbie game available, often spending entire days immersed in these virtual culinary adventures. Drawing from my love for baking, I decided to develop a game focused on making cupcakes.

The core interaction in this game revolves around the user’s ability to add ingredients and craft a delightful cupcake. The user can click the ingredients to add them, and a cupcake would appear. To enhance the overall experience, I incorporated lively and playful background music that would accompany the gameplay.

 However, when I began coding the game, I encountered my first major hurdle: I couldn’t figure out how to call the function responsible for the cupcake’s creation. While I successfully implemented the ingredient addition aspect, the expected output of a fully-formed cupcake remained difficult to track down. These are basic coding that I have done till now. I would like to change the background to a self created portrait and add additional elements. 

Additionally, I encountered difficulties when attempting to integrate music seamlessly into the gameplay as I had initially envisioned. This obstacle prompted me to consider a more straightforward approach, where I would include a continuous background music track instead.

In summary, my mid-term project revolves around a nostalgic cooking game centered on baking cupcakes. Despite encountering challenges with function calls for cupcake creation and integrating music as part of the interactive experience, I am determined to overcome these obstacles and provide an enjoyable and immersive gaming experience for players.

Computer Vision for Artists and Designers

We always see the stage but the backstage remains secret for most of us until we get interested and look at it by ourselves. Today’s material opened the curtains and showed the backstage of the computer vision. We all have been imposed on face and movement recognition to some extent, starting from the face masks in the Instagram filters and the face ID of the iPhone to the movement recognition in various artworks. Although I always wondered how it works, I didn’t have a chance to look at it by myself until today. 

Although I could predict beforehand that the motion is detected by the comparison of the pixels in each subsequent frame, the information about the detection through brightness threshold was new for me, yet so simple. It was interesting to read all these operations made by the computer, which is hidden from our eyes. After reading the article, I have a question whether the advanced technologies still operate the same way, or whether their way of detection might be changed over time. 

I am surprised by the amount of constraints and limitations of the detection of presence and motion, object tracking, etc. Although the technologies have developed rapidly since the day of the first interactive media artworks such as Videoplace, there are still many things, that need to be developed. For instance, simply the face ID in my iPhone recognizes me and my 11-year-old sister as one person. 

Last but not least, I would like to mention examples of artwork using computer vision. While some create an entertaining and aesthetically pleasing experience with voice recognition as Messa di Voce or movement recognition as Videoplace, others use computer vision to raise social issues such as Standards and Double Standards and Suicide Box. I liked the idea of Suicide Box in a way that this project could be developed further as a safety tracker. Every time the vertical motion is recognized, the net can be released under the bridge to catch the falling person and the signal can be sent to the police about possible suicide. Considering the statistics of 17 suicides in 100 days of deployment, the issue of suicide remains significant and needs a solution.

Design and Concept

Over the first few weeks of this class and our class projects, I’ve found myself leaning towards pieces that would work well in an art installation setting ( or at least I think so ), ones that don’t require much user interaction but is just pleasant to look at.

For my midterm, I wanted to do something similar, but at the same time incorporating some more interaction in it while retaining the essence of being an ‘installation piece’ . I was inspired by Chris Milk’s “The Treachery of Sanctuary”, Romy Achituv & Camille Utterback’s “Text Rain”, and more than the others, Scott Snibbe’s “Deep Walls”. I loved the concept of leaving behind little snippets of yourself, seeing the snippets left behind by other people, and the idea that for every memory that is added, a memory is lost.

Picture of Scott Snibbe’s “Deep Walls”

For my midterm project, I’d like to do something similar with a camera and a keyboard. On the screen, there will be a few black and white photo threshold’d to be fully black or white so that the person is not identifiable, along with some text that was left behind by that person. The next viewer will be given one of a few prompts, such as “What is your favorite memory?” , “Who do you miss the most?”, “What are you looking forward to?”, and the viewer can type their answers with the keyboard while the camera takes a photo of them without a prompt. Their memory and photo will be left on the screen, replacing one of the existing ‘memories’, causing it to be lost forever.

I’ve been thinking about the concept of memory for a while, and I thought that this will be a good project to incorporate my thoughts into a physical piece. The biggest challenge for this project I think will be getting the camera and photo-taking to work with Processing, and doing so in a meaningful way. I would also have to think of some good prompts for the piece, as some prompts would be too simple, while having them be too complicated might discourage viewers from typing.

It’s fascinating how many technologies were developed first for military use, before finding it’s way into the commercial space! This was the first time I’ve heard of computer vision originating from military origins, but I knew that the Internet was first invented for war-time communications, and super-glue was an accidental invention from other military-based research. Those inventions that were first made for something as violent as harm has eventually settled down and turned into technologies that we use in our everyday lives, and even in art.

I still think it’s funny how some tasks that are so simple for humans are extremely difficult problems in computation, and some tasks that are nigh-impossible for humans is easy for computers. Computers can calculate 3 to the power of 100 almost instantly, a task that no sane human can do. In the case of computer vision, any human can identify a pigeon in a picture, yet it took computers decades of research before it started being able to identify objects well.

I also found it interesting how many of the early applications of computer vision in projects has a theme of surveillance around it, and I think it reflects how the scene viewed cameras at the time — tools of surveillance. Standards and Double Standards and Suicide Box both use computer vision to highlight societal observations through the lens of an array of pixels.

Concept

For people who know me, I was a gamer my entire life, and I loved the concept of video games as art. I always believed that games themselves can convey meaning like any other artwork, and for some their aesthetic values might even exceed what people traditionally identify as art. Also due to games’ nature of being an interactive medium, it can achieve much more that traditional art forms cannot. For example, the immersion that games bring to people playing them goes far beyond just viewing them, but actually experiencing them yourselves. Therefore, I decided to look for these artistic values in the earliest games, played on the arcades. When I was reading the webpage on The Art of Video Games exhibition in 2012 on the Smithsonian Art Museum website, I came across this game that I had heard of but never played before (because it was too old, obviously), Pac-Man. I knew it was a famous game and almost as old as Tetris, and then soon found out it was one of the first video games that “took this leap beyond the arcade, beyond the screen, into the broader popular culture. That changed the way the world looked at video games and understood that there was something bigger happening here than just a place to waste a quarter.” I took a great interest in this and decided to modify it by combining some other game elements into it and upgrading its interface for my midterm assignment.

I plan to leave the core gameplay of Pac-Man alone and focus on other aspects of the game. In the original version of the game, The map is always the same. I want to combine some roguelike game elements into my project by randomly generating the map every time the player resets the game. therefore the player would always get a new experience when in the game. I also want to make it an endless mode, therefore the player can play to update their personal records. I plan to achieve that by using the tunnels on the left and right sides of the map. Once the player finishes eating all the dots in one map, the tunnels turn into portals that teleport them to other maps with different backgrounds and different theme songs.

Functions and Classes

In this game, I think the main classes are Ghosts, Map, Player, Dots, and Game.

For ghosts, the four types of ghosts (Blinky (red), Pinky (pink), Inky (cyan), and Clyde (orange)) inherit the Ghost class. The Ghost class has certain attributes: intposition for identifying its absolute position, also responsible for calculating destination position; realposition for identifying the true position of the ghost and for determining collisions; targetposition for identifying the target the ghost tries to reach; speed for the speed of the ghosts, and state for determining the current state the ghost is in for it to have different actions. The functions the Ghost class includes are checkCollision, updateTarget, stateChange, directionUpdate, etc. In these functions, updateTarget is different for different types of ghosts.

For the map, it will only have a few attributes. One is a 2D array used to represent the map, and one is its state. it will only have one function, that is mapGen, called every time it is initialized.

For Player, it will have attributes: intposition, realposition, direction, and speed. The only functions it will have are movement and checkDot.

For Dots, the only attribute is its position and type, and the only function is to destroy itself after being touched.

Lastly, for the game class, will have attributes such as points, lives, gamestate(pause, ongoing, in between, etc). It will take care of loading each state of the game and contain data necessary to keep the game going.

This is just a brief outlook of what the system would be like. In practice, some parts may be modified.

Uncertain Part of Project

I find the most uncertain part to be the random map generator. I need to come up with a set of rules so that the map is playable and won’t include too many long corridors the player can be cornered, long dead ends, or areas the player cannot access. However, I have made a preliminary generator that follows one set of rules. This set of rules makes sure that the map is different every time and will not have corridors with width 2 in it. However, this set of rules could result in inaccessible areas. Right now I have an idea about another set of rules to be applied after this so that the inaccessible areas and dead ends can be connected to the other corridors so that the map could be more reasonable. I plan to make any tile that should be accessible with only one or fewer connections with other accessible tiles have at least two connections with other accessible tiles. There will be tiles that cannot find two connection points. Then this tile will be discarded into a wall tile. This process will repeat until no action can be done, and a usable map is then created. I haven’t implemented the second rule set yet, but I have a prototype of the first rule set and it yields satisfying results. It generates grids shown below:

Therefore I am confident that this map generator will be achievable, and I only need to tweak some details.

DiDecor- decorate the room and interact with the character

My concept: When I was a child, I used to play games involving the decoration of rooms of different kinds. When Professor Mang showed the example of a midterm coffee shop, I recalled this experience as the coffee shop looked a lot like one of those games. While looking at that project, I really wanted to change the furniture and play around with the different images. At that moment, I realized that I would like to create my own game with room decoration, so the furniture such as sofas, cupboards, tables, etc could be changed as shown in Fig 1. Fig 1. Room decor game example. Source: 1,334 × 750

The interactivity of the game: I will create an array of images of each of the furniture types, which can be changed by clicking the mouse on top of that particular piece of furniture, and the last shown picture will remain in the game until the user clicks one more. Additionally, I would like to create a character, who lives in this room. The user can move the character and make her sit, and talk with another character (this one isn’t movable) when clicking on that character. I liked the idea of the radio, which changes the music by clicking or stopping it. I would like to implement that with the TV or radio as well because the music sets the vibe of the room too. Additionally, if I have time, I would like to add the story at the beginning of the game how the young lady bought the house and wants the interior designer (the user) to create a design of the room and explain the rules of the game such as which buttons to click. 

Complex part of the game and minimization of risk: First, one of the challenging parts would be finding the images of pieces of furniture. Secondly, talking about the code, it seems hard for now to set the conversation between the characters. For now, I am thinking about making conditionals, so the user can choose what to answer from the given options and the reply of another character changes depending on that answer. If this is too complicated, I will make a random conversation. Each time the user clicks on the second character, a random conversation is generated between the characters, so each time, they are talking about different things but without the interaction of the user.