Author: Nelson Mbigili

NYUAD Puzzle Adventure

My final project was inspired by the beauty of the NYU Abu Dhabi campus. I designed a simple game that involved beautiful images of the campus some taken by myself and some from online sources. Initially the game begins with images in 4 square pieces and a player can move and arrange the pieces to make the image whole. The control can be done entirely through a simple controller that I designed. The game keeps track of time in each chapter as the player solves a specific image and the game intentionally designed to loop, Increasing in level of difficulty every time. The level of difficulty increases by increasing need for precision and increasing number of image pieces.

Video Demonstration 

Demonstration 01: Mr. Godbless Osei

Demonstration 02: Mr Aimable Tuyisenge

Demonstration 03: Screen Recording

Implementation

Interaction

Interaction being key in any game design, NYUAD puzzle adventure integrates interaction through a custom controller that is used to move and arrange the image pieces. The controller can be seen below:

Given my game design, I wanted to make a simple controller, So I used four Switches and a Joystick .  I used the switches to change between game states in my game and the joystick to control movement of pieces. The electrical connections the controller can be seen  on a schematic below:

In addition to interaction through the hardware design, I also used sounds that significantly provided cues. One such was a sound that notifies player that two pieces are already attached together. And a celebration rewarding player when an image is completed.

Arduino code

My Arduino code, which was was the heart of the controller involved the following :

1. Reading data from the buttons and the joystick

2. Sending data to P5.js via serial connection.

3. Receiving data from P5.js for the speakers in the controller.

My Arduino code can be see here:

// ------------Buttons pins-------------------------------
int changePin = 3;       
int playPin = 5;   
int instructionsPin = 2;   
int nextPin = 11;   
// ------------Joystick X and Y pins-----------------------
const int xPin = A2; 
const int yPin = A1; 
// ------------Speaker Pin---------------------------------
const int speaker1Pin = 1;
const int speaker2Pin = 4;
// ------------Control Speaker------------------------------
int playOrNot = 0;
const int play1 = 440;
const int play2 = 39;

int xValue = 0;
int yValue = 0;

void setup() 
{
  pinMode(changePin, INPUT);        
  pinMode(instructionsPin, INPUT); 
  pinMode(playPin, INPUT);      
  pinMode(nextPin, INPUT); 
  pinMode(speaker1Pin, OUTPUT);
  pinMode(speaker2Pin, OUTPUT);  
  Serial.begin(9600);           
}

void loop() 
{
  xValue = analogRead(xPin); 
  yValue = analogRead(yPin); 
  int changeState = digitalRead(changePin);
  int playState = digitalRead(playPin);
  int instructionsState = digitalRead(instructionsPin);
  int nextState = digitalRead(nextPin);

  // Serial output in order -> Instructions, Next, Change, Play, X, Y
  Serial.print(instructionsState == HIGH ? "HIGH," : "LOW,");
  Serial.print(nextState == HIGH ? "HIGH," : "LOW,");
  Serial.print(changeState == HIGH ? "HIGH," : "LOW,");
  Serial.print(playState == HIGH ? "HIGH," : "LOW,");
  Serial.print(xValue);
  Serial.print(",");
  Serial.println(yValue);

  if (Serial.available()) 
  {
    playOrNot = Serial.parseInt();
  }
  if (playOrNot == 1) 
  {
    tone(speaker1Pin, play1); 
  } 
  else if (playOrNot == 2)
  {
    tone(speaker2Pin, play2); 
  }
  else
  {
    noTone(speaker1Pin);
    noTone(speaker2Pin);
  }
}

P5.js sketch

My P5.js sketch manages  multiple states such as “WELCOME,” “INSTRUCTIONS,” “PLAY” and uses a serial connection for controller inputs.  Some key features includes

1. Welcome screen: Displays an introductory screen and a short game summary.

2. Controller Integration: Reads serial data for buttons and joystick inputs to navigate states and interact with puzzle pieces.

3. Dynamic Screens and Feedback: Provides visual and audio feedback, including fading hints, a timer, and celebration effects when puzzles are solved.

4. State Management: Transition between screens based on button presses and game progress.

In my code, I also integrated keyboard controls to facilitate debugging.

The P5j.s Sketch can be seen here: -> Fullscreen

Communication 

Data is sent between Arduino and p5.js through a serial connection. Arduino sends data as a string, in a particular order e.g  playButton, value1, value2,…. p5.js listens for this data, splits it into individual pieces, and uses those pieces to control the game states and actions, such as pressing buttons or moving the joystick.

Things I am Proud of

I am proud with how the whole game has turned out. However, some things that really stood out to me are as follows:

1. The controller design:

Designing this controller has helped me understand how complex controllers like those for the Xbox or PS5 work. I could envision my design integrated with many other elements to and become just as complex  as those in market today.

2. The graphics

The graphics used were quite simple, but very appealing in my opinion.

Reflections for Future

Several areas could be improved in this project. Some of them includes:

1. Using different shapes, like jigsaw patterns, instead of simple rectangles. While I initially wanted to use such complex shapes and pattens for my pieces, I failed due to complexity in wrapping images, managing positioning and many other challenges.

2. The controller design could also be improved  by adding sensors like sensors such as touch sensors and adding  to make the controller more interactive. Adding vibration motors would give players a better feel during gameplay.

My game involves solving puzzles of NYU Abu Dhabi images.
The game involves some nice images of the NYU Abu Dhabi campus which are initially is square pieces. I decided to go with square pieces instead of jigsaw pattern because it was easy to  combine move and manipulate square pieces than any other shape.
I invited a friend of mine to test the game and the video can be seen below:

My schematic is as shown here below:

I have four buttons to control game states and actions and the joystick to move image pieces. The speakers are attached to the controller to provide feedback when the pieces move.

Reflection

1. Simplifying Instructions: I realized that users will skip long instructional text. To address this, I need to make my design as intuitive and self-explanatory as possible.

2. Sound Feedback: The sound feedback when pieces attach is effective in making players aware of their actions and creating a sense of progress. This helps avoid confusion or mistaken attachments.

3. Adjusting Final Sound: The final pop-up sound when an image is correctly solved was too loud and needs to be adjusted for a better user experience.

4. Speed of Image Pieces: The speed of the image pieces was too slow, which could make the gameplay feel boring. Increasing the speed would make the experience more engaging.

5. Magnetic Precision: To improve ease of use, I need to reduce the precision required for pieces to attach. Allowing pieces to magnetically snap together would make it easier for players to combine and move them.

6. Background Music: Given that my user testing was quite, and my volunteer was shocked by the celebration sound, I would like to add a background music in order to make the game more interesting.

Finalised Idea

My finalised idea, from last week’s two options is the NYUAD Puzzle Adventure. 

On P5js the game will display a puzzle with shuffled pieces of  different places of NYU Abu Dhabi campus.  The pieces will be telling a story as a hole in the game. The entire game control will be from  the Arduino hardware.

Progress 

Up until this week, I have been working on the P5js designing the possible structure for the puzzle as seen on the image below:

The hardware design will consist of two potentiometer that will be used to move the puzzle pieces one potentiometer for horizontal movements and one for vertical movements. I will also have two buttons that will be used whenever player wants to lift or drop a piece in the game. I will also have hidden motors (vibration motors potentially) to provide players with feedback as the game progresses. The hardware box will also have an opening mechanism to reward players with NYUAD inspired present given that they correctly solve the puzzle and the grand prize for whoever beats the fastest time.

My hardware schematic will be as shown below:

 

 

First Idea

My first Idea was to develop an interactive game called “Connect Four.” The game  involves two (or one player vs the computer)  taking turns dropping coins into a 7×6 grid displayed in p5.js. Each player is assigned a color—red or blue—and the goal is to be the first to connect four coins in a row, column, or diagonal or the grid is full resulting into a tie. My design will combine physical interaction with a  hardware that allows players to select columns and insert coins. Then whole grid display will be shown in the P5.js in real time.

I hope to develop two modes :

1.Two-Player Mode: In this mode, two players alternate turns, dropping coins into the grid to win against their opponent.

2. Single-Player Mode (vs. Computer): In this mode, the player competes against the computer that follows either of two approaches one being for easy and one for difficult play to accommodate different skill levels.

I hope that the physical  interaction adds a unique experience and the game nature makes it competitive and fun.

Second Idea

I am hoping to develop an interactive game called “NYUAD Puzzle Adventure.” The game involves solving digital jigsaw puzzles displayed in p5.js sketch, controlled entirely through user input on the hardware. Players will use two adjustable controllers (potentiometers) to move puzzle pieces, one controlling horizontal movement and the other controlling vertical movement. A slide button will be used to select and lift a piece or release it into the desired position on the board.

The game will feature images of the NYU Abu Dhabi campus, and a timer will track how long players take to complete each puzzle. To make the game even more interactive, I will use hidden motors to provide physical feedback like vibrations whenever players move pieces and whenever the Game is solved correctly. 

Whenever the game is completed and the player manages to solve the puzzle setting a new record in the shortest time possible the wooden box controlled will open to reveal an NYU Abu Dhabi inspired gift to the player.

I hope that with this design:

Players’ interaction with the puzzle through adjustable controls and physical feedback makes the game more engaging.

The storytelling nature of the puzzle, by using NYUAD images, and the rewarding mechanism will make it fun and relevant to players.

Design Meets Disability

The reading highlights some important points, especially on how we should rethink design for disability, emphasizing a balance between function and beauty. The examples given, such as the Eames’ leg splint, stood out to me because they show that disability products can be both useful and attractive. I was also inspired by the story of how glasses changed from a medical tool to a fashion item. This change made glasses a symbol of personal style rather than something to hide.

For my own designs going forward, I hope to focus on some key ideas from this text. First, I want to make designs that feel good to use—not only functional but also enjoyable and comfortable—so that even if a design is intended for people with disabilities, they feel comfortable using it. I plan to use creative solutions that combine usefulness with visual appeal. Second, I’ll seek to work with people from different backgrounds—such as artists, fashion designers, and people with disabilities—to create designs that are more thoughtful and inclusive. Lastly, I’ll avoid “one-size-fits-all” designs, instead creating products that allow people to show their unique personalities.

In the end, I believe designing for disability is a chance to make products that improve people’s lives in meaningful ways, so designers should prioritize combining beauty and function.

 

Task 1: Sensor Controlled Ellipse

I used a potentiometer to control the horizontal movement of an ellipse in p5.js. Additionally, I displayed text showing the connection status and the current position of the ellipse in (x, y) format.

My Sketch (with the P5js Code) can be found here:

My Arduino Code is here:

// Pin for reading analog values from sensor
int Pin=A0;
void setup() 
{
 Serial.begin(9600);
}
void loop() 
{ 
//Reading values from sensor
   int sensor = analogRead(Pin);
   delay(5);
//Writting the sensor value through serial
   Serial.println(sensor);
}

Task 2: LED Brightness control from P5

I created a simple custom interface with On and Off Button to turn on the LED. While the LED is On there is a slider that can be adjusted to increase or decrease the brightness of the LED.

My Sketch (with the P5js Code) can be found here:

My Arduino Code is here:

 // Setting Global Variables
const int ledPin = 9; 
int brightness = 0;

void setup()
{
  pinMode(ledPin, OUTPUT);  
 // Start serial communication
  Serial.begin(9600);      
}

void loop()
{
// Check if data is available to read
  if (Serial.available())  
  {
// Read Data as Integer 
    int value = Serial.parseInt(); 
// Clear the serial data 
    Serial.flush();
    if (value > 0) 
    {
      brightness = value;
      Serial.println(brightness);
  // Analog write the brightness   
      analogWrite(ledPin, brightness);  
    }
  }else
  {
    analogWrite(ledPin, 0);  
  }
}

Task 3: Bouncing Ball and Winds

In this activity, I started with the given sketch. I added LED lighting that activates whenever the balls touch the ground. The LEDs are controlled digitally by the balls as they make contact with the ground. Additionally, I included two potentiometers that act as wind controls for the ball’s horizontal movement. When the right wind is stronger than the left, the balls move toward the right, and vice versa. I also added an arrow to indicate the direction of the wind.

My Schematics can be seen here:

Here is the Video Demonstration:

My Sketch (with the P5js Code) can be found here:

My Arduino Code is here:

 // Setting Global Variables
const int ledPin1 = 9; 
const int ledPin2 = 6; 
int WindLeftPin=A0;
int WindRightPin=A1;
int brightness = 0;

void setup()
{

  pinMode(ledPin1, OUTPUT);  
  pinMode(ledPin2, OUTPUT);  
 // Start serial communication
  Serial.begin(9600);     
}

void loop()

{if (Serial.available())  
{
  int WindLeft = analogRead(WindLeftPin);
  int WindRight = analogRead(WindRightPin);
  Serial.print(WindLeft);
  Serial.print(",");
  Serial.print(WindRight);
  Serial.println();
  int value = Serial.parseInt();
  if (value == 0) 
  {
    digitalWrite(ledPin1, LOW);
    digitalWrite(ledPin2, LOW);
  }
  else if (value == 1)
  {
    digitalWrite(ledPin1, HIGH);
    digitalWrite(ledPin2, HIGH);
  } 
}
}

 

 

A Brief Rant on the Future of Interaction Design.

I agree with many points the author makes about the current vision of future technology. The author believes that these “pictures under glass” — screens we interact with using simple gestures like sliding or tapping — limit the true potential of our hands and bodies. This reading made me think more about how I use my hands in ways I usually don’t notice.

The author highlights an important problem, which she also addresses in his response to critics. It’s essential to consider how people actually use technology when creating designs, so that the designs fit naturally into human behavior, not the other way around. Since this is already an issue today, it’s crucial to avoid imagining a future with similar limitations. The author is doing his part by writing about this problem to raise awareness and inspire more research and funding in this area.

This reading has encouraged me to think beyond the devices we have now. Instead of just improving screens, we could push for innovations that respect and enhance our human abilities. It makes me hopeful that the future of technology can be something much richer and more connected to our senses and actions. Looking forward, I hope to create interactive designs that put human capabilities first, rather than adding interaction elements just for the sake of it.

Concept 

In this assignment, I collaborated with @Ruslan and we both love Christmas. The famous song Jingle Bells brings memories of the the times. So we explored various possibilities and decided to come up with speed variation of the Jingle Bells melody with respect to distance.

Here is the demonstration Video:

Schematic 

Here is the Schematic  for our Arduino connections:

Code:

In the implementation of our our idea, we searched for possible combinations of the notes and durations to match the Jingle Bells melody and stored them in an array. We then implemented the code mapping distance with durations. The variations in durations for each note make it seem playing faster or slower. Here is the code:

#include "pitches.h"
#define ARRAY_LENGTH(array) (sizeof(array) / sizeof(array[0]))

// Notes and Durations to match the Jingle Bells 
int JingleBells[] = 
{
  NOTE_E4, NOTE_E4, NOTE_E4, NOTE_E4, NOTE_E4, NOTE_E4, NOTE_E4, NOTE_G4,
  NOTE_C4, NOTE_D4, NOTE_E4, NOTE_F4, NOTE_F4, NOTE_F4, NOTE_F4, NOTE_F4,
  NOTE_E4, NOTE_E4, NOTE_E4, NOTE_E4, NOTE_E4, NOTE_D4, NOTE_D4, NOTE_E4,
  NOTE_D4, NOTE_G4, NOTE_E4, NOTE_E4, NOTE_E4, NOTE_E4, NOTE_E4, NOTE_E4, NOTE_E4, NOTE_G4,
  NOTE_C4, NOTE_D4, NOTE_E4, NOTE_F4, NOTE_F4, NOTE_F4, NOTE_F4, NOTE_F4,
  NOTE_E4, NOTE_E4, NOTE_E4, NOTE_E4, NOTE_E4, NOTE_D4, NOTE_D4, NOTE_E4,
  NOTE_D4, NOTE_G4,
};

int JingleBellsDurations[] = {
  4, 4, 4, 4, 4, 4, 4, 4,
  4, 4, 4, 4, 4, 4, 4, 4,
  4, 4, 4, 4, 4, 4, 4, 4,
  4, 4, 4, 4, 4, 4, 4, 4, 4, 4,
  4, 4, 4, 4, 4, 4, 4, 4,
  4, 4, 4, 4, 4, 4, 4, 4,
  4, 4
};

const int echoPin = 7;
const int trigPin = 8;;
const int Speaker1 = 2;
const int Speaker2 = 3;
int volume;

void setup() 
{
// Initialize serial communication:
  Serial.begin(9600);
  pinMode(echoPin, INPUT);
  pinMode(trigPin, OUTPUT);
  pinMode(Speaker1,OUTPUT);
}

void loop() 
{
  long duration,Distance;
  
// Distance Sensor reading
  digitalWrite(trigPin, LOW);
  delayMicroseconds(2);
  digitalWrite(trigPin, HIGH);
  delayMicroseconds(5);
  digitalWrite(trigPin, LOW);
  duration = pulseIn(echoPin, HIGH);
  Distance = microsecondsToCentimeters(duration);

// Map Distance to volume range (0 to 255)
  volume = map(Distance, 0, 100, 0, 255);  
  volume = constrain(volume, 0, 255); 

// Play melody with adjusted volume


 playMelody(Speaker1 , JingleBells, JingleBellsDurations, ARRAY_LENGTH(JingleBells), volume);
  
// Debug output to Serial Monitor
  Serial.print("Distance: ");
  Serial.print(Distance);
  Serial.print("    Volume: ");
  Serial.print(volume);
  Serial.println();
}
// Get Centimeters from microseconds of Sensor
long microsecondsToCentimeters(long microseconds) 
{
  return microseconds / 29 / 2;
}
// PlayMelody function to accept volume and adjust note duration
void playMelody(int pin, int notes[], int durations[], int length, int volume) 
{
  for (int i = 0; i < length; i++) 
  {
// Adjust the note Duration based on the volume
    int noteDuration = (1000 / durations[i]) * (volume / 255.0);  

// Play the note with adjusted Durations
    tone(pin, notes[i], noteDuration);
// Delay to separate the notes
    delay(noteDuration * 1.3);  
    noTone(pin); 
  }
}

Reflections

Reflecting on this project, I learned a lot about working with notes, melodies. I was interested my the fact that  even complex music arrangements are made up of simple notes. The song “Jingle Bells” in particular really made me appreciate the structure of music on a new level. Each note represents a small part of the song, and adjusting the timing or pitch make the whole melody.

Working with @Ruslan made the process even more interesting, as we were both curious and explored various aspects before coming up with the final decision on music’s speed. I hope to continue working on musical notes in future projects.

Concept

This week, I created a simple switch to control the lights of LEDs both digitally and in an analog way. For the digital control, I used basic switches, and for the analog control, I used a potentiometer. The potentiometer detects and measures the voltage based on how much the knob or slider is turned.

Schematic Diagram

 

Code

// Pin definitions for the RGB LED
const int redPin = 6;
const int greenPin = 5;
const int bluePin = 3;

// Analog input pins for potentiometers
const int redControl = A1;
const int greenControl = A2;
const int blueControl = A3;

void setup() {
  // Set RGB LED pins as outputs
  pinMode(redPin, OUTPUT);
  pinMode(greenPin, OUTPUT);
  pinMode(bluePin, OUTPUT);

  // Initialize serial communication for debugging
  Serial.begin(9600);
}

void loop() {
  // Read values from potentiometers (0-1023 range)
  int redValue = analogRead(redControl);
  int greenValue = analogRead(greenControl);
  int blueValue = analogRead(blueControl);

  // Map potentiometer values to PWM range (0-255)
  int redBrightness = map(redValue, 0, 1023, 0, 255);
  int greenBrightness = map(greenValue, 0, 1023, 0, 255);
  int blueBrightness = map(blueValue, 0, 1023, 0, 255);

  // Set the brightness of each color channel
  analogWrite(redPin, redBrightness);
  analogWrite(greenPin, greenBrightness);
  analogWrite(bluePin, blueBrightness);

  // Print the brightness values to the Serial Monitor
  Serial.print("Red:   ");
  Serial.print(redBrightness);
  Serial.print("\tGreen:   ");
  Serial.print(greenBrightness);
  Serial.print("\tBlue:   ");
  Serial.println(blueBrightness);
  
  // Short delay to make output more readable
  delay(100);
}

Video Demonstration of my design can be seen below:

Reflection for future works

I am proud of the progress I made in designing and implementing the analog and digital switch controls for the LEDs. This simple project has greatly enhanced my understanding of how these systems work. I look forward to taking on more challenging tasks, particularly those that involve further manipulation of analog signals.