Author: Areebah Farooq

Concept

Who Pays? is about the fight that happens at the end of every meal with my friends. Nobody wants to split the bill. Everyone wants to be the one who pays, and it turns into an actual race to get your card to the card machine first. So in this game, paying is winning.

The project starts as a short comic. Four pixel art frames I made tell the story of one lunch at Campus East. The food is gone, you casually reach for your uni card, everyone notices, and suddenly three real cards are out and pointed at the machine. The game happens inside that fourth frame. You hold the black card and race Eshaal and Eman by mashing Space, and whoever’s card touches the card machine first pays.

How it works

The sketch has three scenes, and a variable called scene decides whether draw() shows the title screen, the story or the race. Each story frame is a Panel object that knows its picture, its caption and the frame it came on screen. That’s all it needs to type the caption out one letter every two frames. The title screen is frame 1 blurred with grain on top, and it waits for Enter or a click before anything starts.

Each hand in the race is a Racer object. It holds the hand picture cut out of frame 4, where it starts, how far it has to travel to reach the card machine, and a progress number from 0 to 1. Pressing Space adds 0.045 to your progress, and Eshaal and Eman add a small random amount every frame. Where a card sits on screen is just its start position plus the distance times its progress.

Problems and what I’d improve

The hardest design problem was three cards racing toward one small card machine. If they all moved toward the same point, they’d pile on top of each other halfway through and you couldn’t tell whose card was ahead. So each hand got its own straight path, measured from frame 4: mine comes in from the right, Eshaal’s from the left and Eman’s from below, and each path ends at a slightly different spot on the machine. The cards only really meet at the very end. When they do overlap, the draw order decides who’s on top, and your card is drawn last so it’s always the one you can see.

Then I had to decide what “winning” actually means in a mashing game. My first thought was to check when a card entered the card reader’s area on screen. But the cards are tilted and come from three directions, so that area would be a different shape for each of them. It was much simpler to make each path end exactly where the card touches the machine. That way reaching the machine just means progress hits 1, and after every frame checkforWInner() looks for the first card that got there.

Most of the other problems came from the p5 web editor. An older version of the grain effect looped over every pixel on the screen, and the editor stopped it for running longer than 100 milliseconds and called it a “potential infinite loop”. Drawing a few thousand random dots each frame fixed that.

There’s still a lot I’d change. Eshaal and Eman add a random amount every frame, but over a whole race that averages out, so they move at an almost steady speed. Random bursts of speed would make them feel more like people. A tie goes to whoever is first in the racers list, which isn’t really fair. The speeds are also per frame, so on.

Code I am proud of

The part I’m proudest of is the arm stretching in the Racer class. It was the trickiest thing to get right, and you can see it every time the cards move.

// in the constructor, the forearm slice gets cut out of the hand picture once
this.armSlice = this.img.get(this.armX - this.x, this.armY - this.y, info.arm[2], info.arm[3]);

show() {
  // The forearm slice is drawn again and again, from where the hand started to where
  // it is now. Stacked up like that it looks like the arm is stretching toward the machine.
  for (let step = 0; step < this.progress; step += 0.01) {
    image(this.armSlice, this.armX + this.reachX * step, this.armY + this.reachY * step);
  }

  // hand and card go on top, moved along by however far they've got
  image(this.img, this.x + this.reachX * this.progress, this.y + this.reachY * this.progress);
}

When the hands first moved, they floated away from their arms and left a gap in the drawing. The fix was a thin slice of each forearm, taken from exactly where the hand was cut out, drawn over and over along the path behind the hand. Because the slice comes from the real drawing, the arm keeps its shading and outline, and it looks like everyone is actually leaning across the table.

Reflection

I made this game because of how every meal with my friends ends. In Pakistan, paying is hospitality. Covering the bill is how you show someone you care about them, and letting them pay can feel like you’ve failed as a host, even when nobody was hosting anyone. So nobody splits. Everyone reaches for their card at the same time, someone says put that away, someone tries to sneak their card to the cashier before the bill even comes, and it turns into a full fight that everyone is secretly enjoying.

I wanted the game to work the same way, so winning means you’re the one who pays. In most games you win something. Here you win by being the first to give your money away, and that’s honestly what it feels like at the table. Even the second frame, where you casually go for your uni card, is a move every one of us has tried at some point.

References

These Coding Train videos cover the parts of p5.js this project leans on.

Classes in ES6, for the Panel and Racer classes. https://thecodingtrain.com/tracks/code-programming-with-p5-js/code/6-objects/2-classes

Constructor Arguments with Classes, for giving each Racer its name, hand image and path. https://thecodingtrain.com/tracks/code-programming-with-p5-js/code/6-objects/3-constructor

Arrays of Objects, for the racers and panels arrays and looping over them. https://thecodingtrain.com/tracks/code-programming-with-p5-js/code/7-arrays/3-arrays-objects

Objects & Images, for each Racer drawing its own hand picture. https://thecodingtrain.com/tracks/code-programming-with-p5-js/code/7-arrays/8-objects-images

p5.js Web Editor: Uploading Media Files, for getting the frames, hands and sounds into the editor. https://thecodingtrain.com/tracks/code-programming-with-p5-js/code/7-arrays/upload-editor

random(), for Eshaal and Eman’s speed and the grain on the title screen. https://thecodingtrain.com/tracks/code-programming-with-p5-js/code/2-variables/4-random

while & for Loops, for the grain loop and the arm loop. https://thecodingtrain.com/tracks/code-programming-with-p5-js/code/4-loops/1-while-for

Loading and Playing, for loadSound(), play(), loop() and setVolume(). https://thecodingtrain.com/tracks/sound/sound/1-loading-and-playing

Adding Sound Effects, for the beep, approved and declined sounds. https://thecodingtrain.com/tracks/sound/sound/5-adding-sound-effects

Press Start 2P by CodeMan38, the pixel font. https://fonts.google.com/specimen/Press+Start+2P

Improve loop protection warning for slow loops, a forum thread that helped me understand the “infinite loop” warning. https://discourse.processing.org/t/improve-loop-protection-warning-for-slow-loops-that-are-not-infinite/49147

AI Usage

It was most useful when something broke or confused me. It found that the preload error came from the editor defaulting to p5.js 2.0, that my uploads weren’t loading because file names have to be typed out in full in quotes, and that the “infinite loop” was the editor’s 100 millisecond limit. When I switched to portrait frames, it worked out that my frame 4 was still landscape inside the file and rotated the cut outs and every hand position to match. When I didn’t understand the grain, it pulled the title screen apart into layers so I could see it’s just thousands of faint random dots drawn on top of the blurred frame. It also explained how frameCount controls the flicker, and gave me simple pseudocode for the moving hands (start position plus distance times progress), which is the line in show().

I also used gemini to help me generate the graphics, so like that is why the hands or fingers may not be complete haha.

Reading Reflection

The Minsky story at the start is funny, but I think it explains why computer vision is so hard to appreciate. Seeing feels effortless to us, so it seems like it should be simple, when really it’s one of the things we understand least about our own minds. Levin points out that text already comes as symbols, while video is just rectangles of pixels with nothing telling the computer what any of them are. What I kept thinking about is that human vision isn’t assumption free either. We fill in gaps, fall for optical illusions, see faces in random objects. The difference is that our assumptions are invisible to us, while a computer’s have to be written down by someone. In LimboTime, the program treats the limboer’s head as the topmost point of the middle blob of black pixels. The computer isn’t seeing a head there. A few workshop participants decided what a head is for one afternoon, and the computer can only ever find what they told it to look for.

Looked at that way, each technique in the reading is a different guess about what matters in a scene. Frame differencing guesses that what matters is whatever moves, which is why it works on the Tour de France and fails in an office waiting room. Background subtraction guesses it’s whatever wasn’t there before. Thresholding guesses it’s whatever is darker or lighter than everything else, and brightest pixel tracking is basically a max search over an array. None of this is hard to code. The harder part, and what Levin spends most of his time on, is making the world fit the guess with backlighting, infrared, polarizing filters, or retroreflective material (the same stuff 3M makes for safety uniforms). His conclusion says a scene’s “quality” is defined by the algorithm analyzing it, which means there’s no such thing as a good image in general, only an image that’s good for one particular piece of code. The idea I found most useful is at the end of section three, where he says to choose features that are easy to detect and also carry meaning. The mouth is an easy dark shape to find, and its roundness happens to track vowel sounds. Pupils reflect infrared and also show where someone is looking. Power of real design skill is embodied by the difference of what’s computable and what’s meaningful.

That same idea is also what makes surveillance work, and Suicide Box shows it most clearly. The camera never knows anyone is jumping. It picks up vertical motion through simple frame differencing, the same technique a beginner would use for a game, and the location of the camera turns that motion into a death. Levin mentions in the abstract that computer vision was mostly funded for military and law enforcement, and then his conclusion lists everything it can now track, including identities, facial expressions, gait and gaze direction. In 2006 that probably read as a list of creative possibilities. Reading it now, it sounds a lot like a description of the systems that watch us in airports and on streets.

Since the technology is identical, I think the line between interactive art and surveillance comes down to whether the person being tracked can see what the system sees and gets something back. Videoplace shows you your own silhouette, so you’re playing with the system instead of being processed by it. Sorting Daemon turns people on the street into color swatches, and most of them probably never find out. Cheese sits somewhere in between. The actresses knew they were being watched, and an alarm pushed them to make their smiles look more sincere to a machine. Every interactive piece with a camera does a softer version of this, getting you to move in ways the camera can read. Usually that’s the fun part, but Cheese shows how quickly responding to someone turns into making them perform. If I were to determine for what counts as a head or a smile, I would let people see what’s being tracked instead of hiding the camera and letting it feel like magic.

Midterm Report

I was thinking on notes of what is culturally important to me and values wise. Being from Pakistan, hospitality is a big part of the culture, and I believe I have embodied it in a nuance way. Whenever, my friends and I go out I want to pay for us. There is an element of looking after people that brings me a lot of joy, and I wanted to embody that in the game.

So, I will be making a game where the faster you approach the card reader you win. It will mainly be pressing a key to get to the card reader faster, while the other 2 players reach the machine.

I have prepared the 4 frames that will be needed for the game, and I have made a pixelated version of the frames. I wanted to have the first 3 frames just like appear and kind of build a storyline, and the fourth frame has the game in it.

Here is a wire-frame of the game:

Then, here are the 4 frames of the storyline. 

Parts I am worried about

When the user starts playing the game, I am concerned about how the hand would elongate. I will have to parse the image and maybe separate the hands from the arms like save it as a separate asset too. Then, as the user plays the hand extends towards the card machine, so i am concerned about the game mechanics too.

I will be doing something like following to move the separate assets:

progress = 0                        // 0 = start, 1 = touching the card machine

on key press:
    progress = progress + 0.05      // each press moves the card a bit closer

every frame:
    progress = progress - 0.002     // slides back if you stop pressing
    handX = startX + (machineX - startX) * progress
    handY = startY + (machineY - startY) * progress
    draw hand at (handX, handY)

    if progress >= 1: you pay

Progress so far

Reading Reflection

This is the first time I have came across literature on design, and it is refreshing to read about design principles which ground artistic choices in a project. I believe technology itself is both a mode and barrier to communication; a barrier because human connection is supposed to be real, felt, and not interrupted, and a mode because of its ability to reach so many across the world just through sophisticated internet systems.  Norman discusses this as well with how machines are rule based and rigid, but humans are flexible and develop relationships over understanding. The principles of discoverability and understanding are great design principles to begin with. Especially with code, I believe a well-commented code can actually get across the developer’s thinking process, and also makes the codebase a place for others to collaborate. I believe a well-commented code adds up to the discoverability of the code, such as what feature attributes to what code line specifically. I think this is a major part where it co-relates to interactive media.

I also find it interesting how Human Centered Design is called a philosophy and not an area of focus. I think this argument isn’t developed enough in the reading, as the reading refers to this as a philosophy but the rest are referred to as areas of focus.

Something that drives me crazy is delivery riders trying to find buildings on campus. Almost every time I order food the rider calls me because they are lost somewhere on the Highline. All the residences look the same from outside and the names are not visible from the road. There are basically no signifiers for them, so they go in circles until someone comes down to get the food. A map at each entrance (a stair leading from the ground floor or every road connection to the high line) that is laid out the same way the buildings actually are so the map and the real space match. It is such a small thing but it wastes time for both the rider and the student, and by Norman’s logic it is not the rider’s fault at all, it is the design.

For interactive media, feedback is the principle I keep coming back to. If a user clicks on something in a sketch and nothing happens right away, they just keep clicking, same as people pressing the elevator button over and over. A color change or a small sound is enough to tell them the program got their input. The user should also know what to click or press without me explaining it; a button has to look like a button. Mapping is the easy one in a sketch since moving the mouse up can move something up on the screen and nobody has to think twice. The system image part stayed with me the most though. The user can never ask me what I meant while they are using my project. Whatever is on the screen is all they get.

Coding Assignment

I decided to make a room simulation by using the data of the CO2 and temperature from an office room. I used the Occupancy Detection dataset from Kaggle, where they taken in different motion readings from the day including the CO2, temperature, and timing of the day. I map a graph directly below the simulated “room” to give us an idea of the CO2 levels of the room as the time of the day changes. I have also made a window which changes color as the time of the day changes, and then the color of the room changes as CO2 levels get higher or lower (Red for high CO2 and blue for lower).

Code I am proud of

The part I’m most proud of is getting the room to change color with the CO2 level. Each frame I take the current minute’s reading and use map() to turn it into a number between 0 and 1, where 400 ppm (roughly outdoor air) is 0 and 1400 ppm is 1. I wrap it in constrain() so a reading above 1400 can’t push the value past 1 and break the color. That number then goes into lerpColor(), which blends between a calm blue and a warm red, so the room slowly shifts from blue to red as the air gets stuffier over the day.

What I like about this is that one line of math carries the whole idea of the project. You don’t need to read the numbers to know the air is getting worse, you can just see the room turning red. It also made the lag really visible. People arrive around 7:36 but the room doesn’t turn noticeably red until after 9, because CO2 takes time to build up.

let r = rows[idx];

  // ROOM: floor color follows CO2 (blue = fresh, red = stuffy)
  let t = constrain(map(r.co2, 400, 1400, 0, 1), 0, 1);
  fill(lerpColor(color(120, 180, 230), color(230, 90, 80), t));
  stroke(60);
  strokeWeight(4);
  rect(100, 40, 600, 260);

How this was made

I started with importing the datatest.txt and cleaning it. There were a few errors with empty lines which I got rid off. To be particular, there was one less column header compared to the number of columns in the file, and when I was parsing each line to get each column value into the array, I ran into an error. There was an empty row, so I had to look for such things in the dataset too. Then I looked at the names of the headers, and assigned variables to each one.

Since the headers were shifted by one, I couldn’t rely on the names, so I read each value by its column number instead. I only kept the rows from February 3 because it’s a full day, midnight to midnight, which gave me 1440 readings, one for every minute. When I printed the first line of data, I noticed the file starts at 14:19 on February 2, so that day was incomplete. The file also stops partway through February 4, which left February 3 as the only full day, so I kept just that one. I saved each minute as an object with its time, temperature, light, CO2 and occupancy so the rest of the code would be easier to read.

After that I built the visual in layers. First the room, with the floor color changing from blue to red as CO2 goes up. Then the window, which gets brighter with the light reading, and two people who show up at the desks when the room is occupied. Under the room I drew a line graph of CO2 for the whole day, with a dashed line at 800 ppm since that’s around where ventilation usually kicks in. Last, I added the timeline so the day plays on its own, and you can hover over the canvas to scrub to any minute.

Embedded Sketch

Reflection and Improvement

I would want to make this simulation for dataset2 too, which was in the kaggle dataset. Those readings were taken with a closed room and therefore CO2 builds faster compared to this where it is slower. I would also love a 3d model of the room with actual sunlight changing as the day changes and the sunlight coming in from the window. I would probably make a comparison between the both.

 

My concept

I wanted to build a portrait that resonated with me. Features like the eye lash and my face shape were something I wanted to particularly be mindful of. I wasn’t going for a moving or dynamic sketch but rather wanted to focus on the basic sketch and understanding javascript. Other then that, I just wanted to build off what we learnt in class. I also included that I was very tired this week from moving in and settling, so I drew a change of state to sleeping haha.

The main functionality would be the changing day and night.

Code

// I have kept a isNight variable which tracks if it is day or night
// since it is starting as false it means it is day time
let isNight = false; 

function setup() {
  createCanvas(400, 400);
}

function draw() {
  // background changes based on state
  // so if isNight is true which is if it is night time the sky is dark blue
  if (isNight) {
    // dark navy sky
    background(10, 5, 80); 
  } else {
    // else it was day time
    background(100, 180, 230);
  }

  // sun or moon same depends on state
  noStroke();
  if (isNight) {
    // pale gray-white moon
    fill(240); 
    circle(50, 50, 50);
  } else {
    fill("yellow");
    circle(50, 50, 50);
  }

  //hair like the back of it
  fill("black")
  circle(200,220,300);
  // left ear
  fill(230, 190, 150);
  ellipse(95, 220, 18, 22);
  //right ear
  fill(230, 190, 150);
  ellipse(305, 220, 18, 22);
  
  // face
  fill(250, 200, 160); 
  noStroke();
  ellipse(200, 200, 210, 220); 
  // hair at the front of head
  fill("black");
  noStroke();
  ellipse(200, 120, 200, 70);

  // eyes
  if (isNight) {
    // closed eyes at night
    stroke(0);
    strokeWeight(3);
    noFill();
    arc(165, 190, 45, 35, PI, TWO_PI);
    arc(235, 190, 45, 35, PI, TWO_PI);
    noStroke();

    
  } else {
    // open eyes during the day
    fill(255);
    noStroke();
    ellipse(170, 180, 45, 35);
    ellipse(230, 180, 45, 35);
    fill(60);
    ellipse(170, 180, 20, 20);
    ellipse(230, 180, 20, 20);
  }

  // nose
  fill(230, 190, 150);
  ellipse(200, 220, 18, 22);

  // eyelashes
  
    stroke(0);
    strokeWeight(2);
    line(145, 170, 135, 165);
    line(152, 166, 148, 154);
    line(162, 161, 162, 150);
    line(255, 170, 262, 160);
    line(248, 166, 252, 154);
    line(240, 153, 235, 163);
    noStroke();
  

  //emoji for the lips
  textSize(65);
  text("👄", 168, 290);
}

function mousePressed() {
  // flips true to false or false to true, any click any where flips this
  isNight = !isNight; 
}

I struggled a bit with the eyelash placement. I wanted them to be at an angle, and I achieved exactly what I wanted. Especially placing them over the eyes was challenging and finding that placement. This was a trial and error process for sure and took me considerable time, but I am happy it looks like the way I want.

Embedded Sketch

 

How this was made

I started with the face and nose since they would be the same in both day and night. I couldn’t make the mouth like I wanted, so I looked at the “Getting Started with p5” where I saw the text function. I used the lip emoji in that function and then added the eyelashes and hair. I wanted these 2 to be static too with the day/night change. I added the sun in the back. I just used one main function ‘draw’ and had everything inside. I used the “isNight” variable for day and night. I changed the shape of the eye depending on the state. One interesting thing I learned was that each line of code draws on top of the last, so shapes drawn later cover the ones drawn earlier. It’s not about compiling, it’s just the order things get painted onto the canvas.

Reflection and improvements

I would make a dynamic sketch in the future and make it more personal as for now I just wanted this to be a reflection of myself (like quite literally) out of the shapes and techniques we learnt. I think I would add my neck and body too. Probably changes in state that reflect more things about me too.