I was inspired by old school computer graphics that you would see in movies like The Matrix. Because of this, I knew that I wanted to make some ASCII art in the signature green color that most retro graphics used. After some experimenting, I decided to make an ASCII representation of a Menger Sponge, which is a fractal geometry that I thought would be very interesting to look at.
Process
I began by creating a sample video that I could use to turn into ASCII art. To do this, I created a 3D cube in Processing, which is a predecessor of P5.js. I attempted to do this in P5.js, but found the saveFrame() function too limiting. I created a simple box using the 3D renderer in Processing, and added some lighting to give the sketch some dynamic range. This is important as I needed to use the difference in brightness later on when converting the video to ASCII, and the greater the dynamic range is the easier it is to perceive the ASCII video.
I incremented the rotation angle by a fraction of pi because I wanted to be able to count when the cube resets to its original position. This made it easier to create a video that could be looped seamlessly.
Once I had the output frames, I combined them together using Microsoft Movie Maker. The final result was this video:
Next, I wanted to work on converting this footage to ASCII art. I followed Daniel Schiffman’s coding challenge on creating ASCII text images. After experimenting with the video size and character density arrays, the following sketch was the result I got:
However, I wanted to create something a bit more complex. This is when I remembered an old project that I worked on by following another one of The Coding Train‘s challenges, which was the Menger Sponge coding challenge. After generating the frames and compiling them into a video, this was the result:
All I had to do then is to insert this video into the original code and play around with different parameters until I got the desired result.
Code Highlights
I’m really proud of the code that makes the animation look like its being built up slowly using ASCII characters. The way I achieved this is basically by filtering out the highlights on the Menger sponge. When I compiled the video, I saw that the lower right corner of the sponge had a bright highlight on it that was blocky.
//finding the right character based on brightness of pixel
let len = charArray.length;
let charIndex;
//playing with map values for the building up effect
charIndex = floor(map(apparentBrightness, 0, 100, len, 0));
When I filtered the brightest points of the sponge out, I essentially removed the lower left corner until it got a bit darker later in the animation, which created the building-up effect.
Reflection
Compared to the first assignment, I had a more solid idea of what I wanted to achieve. Because of this, I had planned out my workflow beforehand and that streamlined the entire creative process. I knew I had to create source animation and then convert it to ASCII characters. This made my code more readable, and I had better control of the sketch overall.
However, the building-up animation that I am most proud of is dependent on the source video. It looks the way it is because in the source animation the highlights are blocky as well. If I wanted to recreate this project, I want to work on some logic that allows the effect to be more generalizable. Maybe I could filter out sections of the object based on a distance function instead of the brightness levels. That way I can substitute different source videos and still get the cool effect.
const int blueButton = 6;
const int servoPin = 9;
const int songLength = 8;
const int tempo = 115;
const int noteC3 = 130;
const int noteD3 = 146;
const int noteE3 = 164;
const int noteF3 = 174;
const int noteG3 = 196;
const int noteA3 = 220;
const int noteB3 = 246;
const int noteC4 = 261;
const int noteD4 = 293;
const int noteE4 = 329;
const int noteF4 = 349;
const int noteG4 = 392;
const int noteA4 = 440;
const int noteB4 = 493;
const int noteC5 = 523;
const int noteD5 = 587;
const int noteE5 = 659;
const int noteF5 = 698;
const int noteG5 = 784;
const int noteA5 = 880;
const int noteB5 = 987;
//int musicNotes[] = {noteC4, noteD4, noteE4, noteF4, noteG4, noteA4, noteB4, noteC5};
int musicNotes[] = {noteC5, 0, noteE5, 0, noteG5, 0, noteB5, 0};
int musicNotes2[] = {noteC4, 0, noteE4, 0, noteG4, 0, noteB4, 0};
int musicNotes3[] = {noteC3, 0, noteE3, 0, noteG3, 0, noteB3, 0};
int noteDuration[] = {2, 4, 2, 4, 4, 2, 4, 2};
void setup() {
pinMode(servoPin, OUTPUT);
pinMode(blueButton, INPUT_PULLUP);
Serial.begin(9600);
}
void loop() {
lightSensor();
button();
}
//new tab: button
void button() {
bool bluebuttonState = digitalRead(blueButton);
if (bluebuttonState == HIGH) {
for (int i = 0; i < songLength; i++) {
int duration = noteDuration[i] * tempo;
tone(servoPin, musicNotes2[i], duration);
delay(duration); //make the length of the time = length of the musical note(frequency)
delay(15);
}
} else {
for (int i = 0; i < songLength; i++) {
int duration = noteDuration[i] * tempo;
tone(servoPin, musicNotes[i], duration);
delay(duration); //make the length of the time = length of the musical note(frequency)
delay(15);
};
};
};
//new tab light sensor
void lightSensor() {
int analogValue = analogRead(A0);
Serial.print(analogValue);
if (analogValue < 10) {
Serial.println(" - Dark");
// //add music note here
} else if (analogValue < 200) {
Serial.println(" - Dim");
for (int i = 0; i < songLength; i++) {
int duration = noteDuration[i] * tempo;
tone(servoPin, musicNotes3[i], duration);
delay(duration); //make the length of the time = length of the musical note(frequency)
delay(15);
}
} };
Documentation:
Idea: Create sound directly from Arduino, like a musical drone sound by changing the musical notes from C4 to C5 when you click on the button or change from either to a 3rd octave when you dim the light sensor (all by changing the frequency from within the motor).
Positives:
I like how I could manipulate the motor sound based on its frequency to create a tone and tempo.In addition to that I was able to play around with that frequency within the motor to create an array of musical notes with different octaves. Once I could adjust the tempo and time spacing between notes through looping the array, I was able to integrate that into different parts of the code.
I like that I was able to introduce different noises from the motor by adding in different components that trigger different sounds like the button and sensor.
This was also surprisingly fun compared to other assignments for me because I learned that Arduino could be used for more than just LED circuits etc but you can incorporate and manipulate other media like sound.
Negatives:
I don’t think its particularly musical, however I think it follows the rules of music in terms of octaves and musical notes.
I would like each candy spiral to go through several colored versions of itself:
for (int a= 0; a<1; a++) {
save(random(20)+"spiral");
}
This code allows different colored versions from random to be saved as a new version of itself , I plan on using the saved photos as part of a sprite sheet to add to the animation above.
Then, I didn’t really know what to do with it in terms of animation….
I discovered the built in P3D and started different experiments, but I didn’t know how to integrate more effects while using the While loop, so I experimented further and created the animation at the top of the post.
The inspiration of this post for me was to experiment as much as possible, since it’s been helping me understand the mechanics of the code. And to save different blocks of code on GitHub if I find something interesting I could integrate in a different art piece later.
I used a photoresistor as input to control the position of the sphere.
p5js:
let xcord = 0;
let left=0;
let right=0;
function setup() {
createCanvas(640, 480);
}
function draw() {
background("white");
if (!serialActive) {
text("Press Space Bar to select Serial Port", 20, 30);
} else {
text("Connected", 20, 30);
ellipse(xcord,height/2,100,100);
console.log(xcord);
}
}
function keyPressed() {
if (key == " ") {
// important to have in order to start the serial connection!!
setUpSerial();
}
}
function readSerial(data) {
if (data != null) {
let fromArduino = trim(data);
xcord = int(fromArduino);
}
}
Arduino:
int inPin = A0;
void setup() {
// Start serial communication so we can send data
// over the USB connection to our p5js sketch
Serial.begin(9600);
pinMode(LED_BUILTIN, OUTPUT);
pinMode(inPin, INPUT);
}
void loop() {
digitalWrite(LED_BUILTIN, HIGH); // led on while receiving data
int sensor = analogRead(A0);
delay(5);
Serial.println(sensor - 300);
digitalWrite(LED_BUILTIN, LOW);
}
2.
p5js:
let ycord;
function setup() {
createCanvas(640, 480);
textSize(18);
}
function draw() {
if (!serialActive) {
text("Press Space Bar to select Serial Port", 20, 30);
} else {
text("Connected", 20, 30);
}
if (mouseIsPressed) {
ycord=mouseY;
console.log(ycord);
}
}
function keyPressed() {
if (key == " ") {
// important to have in order to start the serial connection!!
setUpSerial();
}
}
function readSerial(data) {
if (data != null) {
let fromArduino = trim(data);
let sendToArduino = ycord + "\n";
writeSerial(sendToArduino);
console.log("sent");
}
}
Arduino:
int inPin = A0;
int outpin = 8;
void setup() {
// Start serial communication so we can send data
// over the USB connection to our p5js sketch
Serial.begin(9600);
pinMode(LED_BUILTIN, OUTPUT);
pinMode(inPin, INPUT);
pinMode(outpin, INPUT);
}
void loop() {
digitalWrite(LED_BUILTIN, HIGH); // led on while receiving data
int brightness = Serial.parseInt();
if (Serial.read() == '\n') {
digitalWrite(outpin, HIGH);
delay(brightness/100);
digitalWrite(outpin, LOW);
delay(4.80-brightness/100);
}
Serial.println(brightness);
}
3. It works but the lag is insane.
Arduino:
int inPin = A0;
int outpin = 8;
void setup() {
// Start serial communication so we can send data
// over the USB connection to our p5js sketch
Serial.begin(9600);
pinMode(LED_BUILTIN, OUTPUT);
pinMode(inPin, INPUT);
pinMode(outpin, INPUT);
}
void loop() {
digitalWrite(LED_BUILTIN, LOW);
while (Serial.available()) {
// led on while receiving data
digitalWrite(LED_BUILTIN, HIGH);
int left = Serial.parseInt();
if (Serial.read() == '\n') {
int sensor = analogRead(A0);
Serial.println(sensor);
}
if (left >= 350 && left <= 360) {
digitalWrite(outpin, HIGH);
} else {
digitalWrite(outpin, LOW);
}
}
int sensor = analogRead(A0);
Serial.println(sensor);
}
P5:
let velocity;
let gravity;
let position;
let acceleration;
let wind;
let drag = 0.99;
let mass = 50;
function setup() {
createCanvas(640, 360);
noFill();
position = createVector(width/2, 0);
velocity = createVector(0,0);
acceleration = createVector(0,0);
gravity = createVector(0, 0.1*mass);
wind = createVector(0,0);
}
function draw() {
background(255);
applyForce(wind);
applyForce(gravity);
velocity.add(acceleration);
velocity.mult(drag);
position.add(velocity);
acceleration.mult(0);
ellipse(position.x,position.y,mass,mass);
if (position.y > height-mass/2) {
velocity.y *= -0.9; // A little dampening when hitting the bottom
position.y = height-mass/2;
}
}
function readSerial(data) {
////////////////////////////////////
//READ FROM ARDUINO HERE
////////////////////////////////////
if (data != null) {
// make sure there is actually a message
let fromArduino = trim(data);
//console.log(fromArduino);
let sensorVal = int(fromArduino);
if (sensorVal < 600){
wind.x=1;
}
else if(sensorVal >= 600 && sensorVal < 800){
wind.x = 0;
}
else {
wind.x = -1;
}
// //////////////////////////////////
// //SEND TO ARDUINO HERE (handshake)
// //////////////////////////////////
}
else{
console.log("empty input");
}
//height of ball sent to arduino to check if ball on floor or not
let sendToArduino = position.y + 0.5 * mass + "\n";
writeSerial(sendToArduino);
}
function applyForce(force){
// Newton's 2nd law: F = M * A
// or A = F / M
let f = p5.Vector.div(force, mass);
acceleration.add(f);
}
function keyPressed(){
if (keyCode==LEFT_ARROW){
wind.x=-1;
}
if (keyCode==RIGHT_ARROW){
wind.x=1;
}
// if (key==' '){
// mass=random(15,80);
// position.y=-mass;
// velocity.mult(0);
// }
if (key == " ") {
if (!serialActive) {
setUpSerial();
console.log("serial")
}
mass=random(15,80);
position.y=-mass;
velocity.mult(0);
}
}
For my final project, I want to make a surveillance camera based project where a servo motor (on top of which will be attached a cctv clone) moves according to face/motion-tracking. I think I will probably have to use ml5.js to figure this out. I am not too sure if it will work out by the final deadline so I might change the proposal but yea.
While going through the text, a specific passage that took my attention is the discussion about hearing aids. Unlike glasses, whose design has undergone comparatively less change, the writer highlights the evolving nature of hearing aids, emphasizing not only the modifications in their appearance but also the constant transformation of their discreet placement.
And in the part where, ‘Simple meets universal’, the author explores the complexities of designing for special needs, especially when dealing with a minority of the population. On one hand, there is a compelling business argument against further fragmenting the market by tailoring designs for small percentages of users. The concern here is that such a specialized approach might limit the product’s reach and economic viability.
Conversely, the concept of inclusive design, also known as universal design, introduces a principle-based contention. Inclusive design, by definition, aims to cater to the entire population. This definition intertwines two critical aspects: the acknowledgment that individuals possess varying abilities, and a recognition that people may have diverse needs and preferences regardless of their abilities. The former is commonly addressed through multimodal interfaces, incorporating visual, audible, and tactile cues to accommodate those with impaired touch, hearing, and/or sight. Meanwhile, the latter is often managed through multifunctional platforms, incorporating numerous features to appeal to a broad range of users.
A central question raised by the author is whether the pursuit of universal design, aiming to accommodate a diverse user base, might inadvertently lead to overly complex designs. There is a tension between the goal of inclusivity and the risk of creating designs that are intricate and potentially challenging to use. This prompts a consideration of how truly inclusive such designs are, and whether there are insights to be gained from navigating these design complexities.
For the first idea, I’m thinking of using either real-time data or just existing data and creating a creative visualization that can be controlled by the user through an ultrasonic sensor. My second option for data visualization is to apply the concept of sonification in the field of astronomy where data is converted into sound as an output demonstrating some waves that are being radiated in space or a black hole studied by NASA. If I go with this idea, I’ll probably try to connect the sound of those waves with the user’s interaction with the ultrasonic sensor and connect it to what’s displayed on p5js.
Idea 2
My second idea does not involve any data its more like a game. The game would either involve a journey in an old house and the user will be moving through different rooms or it would be a whole night journey in multiple settings like a home, forest, streets, etc. The main concept is that they will be using their phones as a flashlight to search for certain objects asked by the game in order to win or complete the narrative. The light would be detected by the light sensor and it would be connected to p5js so the space they are in is bright enough for them to look for the item. This idea might be tough because I’m not sure to what extent it is plausible to implement.
Something that uses only one sensor on Arduino and makes the ellipse in p5 move on the horizontal axis, in the middle of the screen, and nothing on arduino is controlled by p5.
P5JS code
let redValue = 0;
let transparency = 255;
function setup() {
createCanvas(640, 480);
textSize(18);
}
function draw() {
if (key == " ") {
initiateSerialConnection();
}
background(map(redValue, 0, 1023, 0, 255), 255, 255);
fill(255, 0, 255, map(transparency, 0, 1023, 0, 255));
if (!isSerialActive) {
text("Press Space Bar to select Serial Port", 20, 30);
} else {
ellipse(redValue / 2, 240, 100, 100);
}
}
function keyPressed() {
if (key == " ") {
initiateSerialConnection();
}
}
function readSerial(data) {
if (data != null) {
let fromArduino = split(trim(data), ",");
if (fromArduino.length == 2) {
redValue = int(fromArduino[0]);
transparency = int(fromArduino[1]);
}
}
}
Arduino code
void setup() {
Serial.begin(9600);
pinMode(LED_BUILTIN, OUTPUT);
// start the handshake
while (Serial.available() <= 0)
{
Serial.println("0,0"); // send a starting message
delay(300); // wait 1/3 second
}
}
void loop()
{
// wait for data from p5 before doing something
while (Serial.available())
{
digitalWrite(LED_BUILTIN, HIGH); // led on while receiving data
// Read sensor value
int sensorValue = analogRead(A0);
Serial.print(sensorValue);
// Map sensor value to screen width
int screenValue = map(sensorValue, 0, 1023, 0, 800);
// Send mapped value to p5.js
Serial.println(screenValue);
delay(50); // for stability
}
digitalWrite(LED_BUILTIN, LOW);
}
Exercise 2
Something that controls the LED brightness from p5. LED’s brightness is changed by mouseX and the other’s by mouse Y.
P5JS code
let rVal = 0;
let alpha = 255;
let left = 0; // True (1) if mouse is being clicked on left side of screen
let right = 0; // True (1) if mouse is being clicked on right side of screen
function setup() {
createCanvas(255, 255);
textSize(18);
}
function draw() {
// one value from Arduino controls the background's red color
background(map(rVal, 0, 1023, 0, 255), 255, 255);
// the other value controls the text's transparency value
fill(255, 0, 255, map(alpha, 0, 1023, 0, 255));
if (!serialActive) {
text("Press Space Bar to select Serial Port", 20, 30);
} else {
text("Connected", 20, 30);
// Print the current values
text('rVal = ' + str(rVal), 20, 50);
text('alpha = ' + str(alpha), 20, 70);
}
// click on one side of the screen, one LED will light up
// click on the other side, the other LED will light up
if (mouseIsPressed) {
if (mouseX <= width / 2) {
left = 1;
} else {
right = 1;
}
} else {
left = right = 0;
}
}
function keyPressed() {
if (key == " ") {
// important to have in order to start the serial connection!!
setUpSerial();
}
}
// This function will be called by the web-serial library
// with each new *line* of data. The serial library reads
// the data until the newline and then gives it to us through
// this callback function
function readSerial(data) {
////////////////////////////////////
//READ FROM ARDUINO HERE
////////////////////////////////////
if (data != null) {
// make sure there is actually a message
// split the message
let fromArduino = split(trim(data), ",");
// if the right length, then proceed
if (fromArduino.length == 2) {
// only store values here
// do everything with those values in the main draw loop
// We take the string we get from Arduino and explicitly
// convert it to a number by using int()
// e.g. "103" becomes 103
rVal = int(fromArduino[0]);
alpha = int(fromArduino[1]);
}
//////////////////////////////////
//SEND TO ARDUINO HERE (handshake)
//////////////////////////////////
let sendToArduino = mouseX + "," + mouseY + "\n";
writeSerial(sendToArduino);
}
}
Arduino code
int leftLedPin = 2;
int rightLedPin = 5;
void setup() {
// Start serial communication so we can send data
// over the USB connection to our p5js sketch
Serial.begin(9600);
// We'll use the builtin LED as a status output.
// We can't use the serial monitor since the serial connection is
// used to communicate to p5js and only one application on the computer
// can use a serial port at once.
pinMode(LED_BUILTIN, OUTPUT);
// Outputs on these pins
pinMode(leftLedPin, OUTPUT);
pinMode(rightLedPin, OUTPUT);
// Blink them so we can check the wiring
digitalWrite(leftLedPin, HIGH);
digitalWrite(rightLedPin, HIGH);
delay(200);
digitalWrite(leftLedPin, LOW);
digitalWrite(rightLedPin, LOW);
// start the handshake
while (Serial.available() <= 0) {
digitalWrite(LED_BUILTIN, HIGH); // on/blink while waiting for serial data
Serial.println("0,0"); // send a starting message
delay(300); // wait 1/3 second
digitalWrite(LED_BUILTIN, LOW);
delay(50);
}
}
void loop() {
// wait for data from p5 before doing something
while (Serial.available()) {
digitalWrite(LED_BUILTIN, HIGH); // led on while receiving data
int posX = Serial.parseInt();
int posY = Serial.parseInt();
if (Serial.read() == '\n') {
analogWrite(leftLedPin, posX);
analogWrite(rightLedPin, posY);
int sensor = analogRead(A0);
delay(5);
int sensor2 = analogRead(A1);
delay(5);
Serial.print(sensor);
Serial.print(',');
Serial.println(sensor2);
}
}
digitalWrite(LED_BUILTIN, LOW);
}
Exercise 3
Bouncing ball: potentiometer is used as the analog sensor to control the breeze.
P5JS code
let velocity;
let gravity;
let position;
let acceleration;
let breeze;
let drag = 0.99;
let mass = 50;
let heightOfBall = 0;
function setup() {
createCanvas(640, 360); // Create a canvas of 800x400 pixels
noFill();
position = createVector(width/2, 0);
velocity = createVector(0,0);
acceleration = createVector(0,0);
gravity = createVector(0, 0.5*mass);
breeze = createVector(0,0);
}
function draw() {
background(215);
fill(0);
if (!serialActive) {
text("Press the space bar to select the serial Port", 20, 30);
}
else
{
text("Arduino is connected! Press b to jump.", 20, 30);
applyForce(breeze);
applyForce(gravity);
velocity.add(acceleration);
velocity.mult(drag);
position.add(velocity);
acceleration.mult(0);
ellipse(position.x,position.y,mass,mass);
if (position.y > height-mass/2) {
velocity.y *= -0.9; // A little dampening when hitting the bottom
position.y = height-mass/2;
heightOfBall = 0;
}
else {
heightOfBall = 1;
}
}
}
function applyForce(force){
// Newton's 2nd law: F = M * A
// or A = F / M
let f = p5.Vector.div(force, mass);
acceleration.add(f);
}
function keyPressed() {
if (key == " ") {
// important to have in order to start the serial connection!!
setUpSerial();
}
else if (key=='b'){
mass=random(15,80);
position.y=-mass;
velocity.mult(0);
}
}
// this callback function
function readSerial(data) {
////////////////////////////////////
//READ FROM ARDUINO HERE
////////////////////////////////////
if (data != null) {
// make sure there is actually a message
let fromArduino = split(trim(data), ",");
// if the right length, then proceed
if (fromArduino.length == 1) {
//sensor value is the input from potentiometer
let sensorVal = int(fromArduino[0]);
//potentiometer value ranges from 0 - 1023
//for values less than 400,wind blows to right
if (sensorVal < 400){
breeze.x=1
}
//if value between 400 and 500, wind stops so ball stops
else if(sensorVal >= 400 && sensorVal < 500){
breeze.x = 0
}
//if value greater than 500, wind blows to left
else {
breeze.x = -1
}
//////////////////////////////////
//SEND TO ARDUINO HERE (handshake)
//////////////////////////////////
}
//height of ball sent to arduino to check if ball on floor or not
let sendToArduino = heightOfBall + "\n";
writeSerial(sendToArduino);
}
}
Arduino code
const int poten_pin = A0;
const int ledPin = 2;
void setup() {
Serial.begin(9600); // Start serial communication at 9600 bps
pinMode(LED_BUILTIN, OUTPUT);
pinMode(ledPin, OUTPUT);
pinMode(poten_pin, INPUT);
// start the handshake
while (Serial.available() <= 0) {
digitalWrite(LED_BUILTIN, HIGH); // on/blink while waiting for serial data
Serial.println("0,0"); // send a starting message
delay(300); // wait 1/3 second
digitalWrite(LED_BUILTIN, LOW);
delay(50);
}
}
void loop()
{
// wait for data from p5 before doing something
while (Serial.available())
{
digitalWrite(LED_BUILTIN, HIGH);
digitalWrite(ledPin, LOW);
//read the position of ball from p5
int position = Serial.parseInt();
if (Serial.read() == '\n') {
// Read potentiometer value
int sensorValue = analogRead(poten_pin);
//send value to p5
Serial.println(sensorValue);
}
//if ball is touching the ground i.e. height is zero, turn LED on
if (position == 0)
{
digitalWrite(ledPin, HIGH);
}
else{
digitalWrite(ledPin, LOW);
}
}
digitalWrite(LED_BUILTIN, LOW);
}
make something that uses only one sensor on Arduino and makes the ellipse in p5 move on the horizontal axis, in the middle of the screen, and nothing on arduino is controlled by p5.
3. take the gravity wind example (https://editor.p5js.org/aaronsherwood/sketches/I7iQrNCul) and make it so every time the ball bounces one led lights up and then turns off, and you can control the wind from one analog sensor.
For the first exercise, Nafiha and I decided to use a light sensor as the analog input value for the movement of the ellipse on the p5js screen.
P5js Code
//exercise 1 p5js
let redValue = 0;
let transparency = 255;
function setup() {
createCanvas(640, 480);
textSize(18);
}
function draw() {
if (key == " ") {
initiateSerialConnection();
}
background(map(redValue, 0, 1023, 0, 255), 255, 255);
fill(255, 0, 255, map(transparency, 0, 1023, 0, 255));
if (!isSerialActive) {
text("Press Space Bar to select Serial Port", 20, 30);
} else {
ellipse(redValue / 2, 240, 100, 100);
}
}
function keyPressed() {
if (key == " ") {
initiateSerialConnection();
}
}
function readSerial(data) {
if (data != null) {
let fromArduino = split(trim(data), ",");
if (fromArduino.length == 2) {
redValue = int(fromArduino[0]);
transparency = int(fromArduino[1]);
}
}
}
Arduino Code
//exercise 1 arduino
void setup() {
Serial.begin(9600);
pinMode(LED_BUILTIN, OUTPUT);
// start the handshake
while (Serial.available() <= 0)
{
Serial.println("0,0"); // send a starting message
delay(300); // wait 1/3 second
}
}
void loop()
{
// wait for data from p5 before doing something
while (Serial.available())
{
digitalWrite(LED_BUILTIN, HIGH); // led on while receiving data
// Read sensor value
int sensorValue = analogRead(A0);
Serial.print(sensorValue);
// Map sensor value to screen width
int screenValue = map(sensorValue, 0, 1023, 0, 800);
// Send mapped value to p5.js
Serial.println(screenValue);
delay(50); // for stability
}
digitalWrite(LED_BUILTIN, LOW);
}
For the second exercise the LED’s brightness is changed by mouseX and the other’s by mouse Y.
P5js Code
//exercise 2 p5js
let rVal = 0;
let alpha = 255;
let left = 0; // True (1) if mouse is being clicked on left side of screen
let right = 0; // True (1) if mouse is being clicked on right side of screen
function setup() {
createCanvas(255, 255);
textSize(18);
}
function draw() {
// one value from Arduino controls the background's red color
background(map(rVal, 0, 1023, 0, 255), 255, 255);
// the other value controls the text's transparency value
fill(255, 0, 255, map(alpha, 0, 1023, 0, 255));
if (!serialActive) {
text("Press Space Bar to select Serial Port", 20, 30);
} else {
text("Connected", 20, 30);
// Print the current values
text('rVal = ' + str(rVal), 20, 50);
text('alpha = ' + str(alpha), 20, 70);
}
// click on one side of the screen, one LED will light up
// click on the other side, the other LED will light up
if (mouseIsPressed) {
if (mouseX <= width / 2) {
left = 1;
} else {
right = 1;
}
} else {
left = right = 0;
}
}
function keyPressed() {
if (key == " ") {
// important to have in order to start the serial connection!!
setUpSerial();
}
}
// This function will be called by the web-serial library
// with each new *line* of data. The serial library reads
// the data until the newline and then gives it to us through
// this callback function
function readSerial(data) {
////////////////////////////////////
//READ FROM ARDUINO HERE
////////////////////////////////////
if (data != null) {
// make sure there is actually a message
// split the message
let fromArduino = split(trim(data), ",");
// if the right length, then proceed
if (fromArduino.length == 2) {
// only store values here
// do everything with those values in the main draw loop
// We take the string we get from Arduino and explicitly
// convert it to a number by using int()
// e.g. "103" becomes 103
rVal = int(fromArduino[0]);
alpha = int(fromArduino[1]);
}
//////////////////////////////////
//SEND TO ARDUINO HERE (handshake)
//////////////////////////////////
let sendToArduino = mouseX + "," + mouseY + "\n";
writeSerial(sendToArduino);
}
}
Arduino Code
//exercise 2 arduino
int leftLedPin = 3;
int rightLedPin = 6;
void setup() {
// Start serial communication so we can send data
// over the USB connection to our p5js sketch
Serial.begin(9600);
// We'll use the builtin LED as a status output.
// We can't use the serial monitor since the serial connection is
// used to communicate to p5js and only one application on the computer
// can use a serial port at once.
pinMode(LED_BUILTIN, OUTPUT);
// Outputs on these pins
pinMode(leftLedPin, OUTPUT);
pinMode(rightLedPin, OUTPUT);
// Blink them so we can check the wiring
digitalWrite(leftLedPin, HIGH);
digitalWrite(rightLedPin, HIGH);
delay(200);
digitalWrite(leftLedPin, LOW);
digitalWrite(rightLedPin, LOW);
// start the handshake
while (Serial.available() <= 0) {
digitalWrite(LED_BUILTIN, HIGH); // on/blink while waiting for serial data
Serial.println("0,0"); // send a starting message
delay(300); // wait 1/3 second
digitalWrite(LED_BUILTIN, LOW);
delay(50);
}
}
void loop() {
// wait for data from p5 before doing something
while (Serial.available()) {
digitalWrite(LED_BUILTIN, HIGH); // led on while receiving data
int xPos = Serial.parseInt();
int yPos = Serial.parseInt();
if (Serial.read() == '\n') {
analogWrite(leftLedPin, xPos);
analogWrite(rightLedPin, yPos);
int sensor = analogRead(A0);
delay(5);
int sensor2 = analogRead(A1);
delay(5);
Serial.print(sensor);
Serial.print(',');
Serial.println(sensor2);
}
}
digitalWrite(LED_BUILTIN, LOW);
}
For the third exercise, we used a potentiometer as our analog sensor to control the breeze.
//exercise 3 p5js
let velocity;
let gravity;
let position;
let acceleration;
let breeze;
let drag = 0.99;
let mass = 50;
let heightOfBall = 0;
function setup() {
createCanvas(640, 360);
noFill();
position = createVector(width/2, 0);
velocity = createVector(0,0);
acceleration = createVector(0,0);
gravity = createVector(0, 0.5*mass);
breeze = createVector(0,0);
}
function draw() {
background(215);
fill(0);
if (!serialActive) {
text("Press the space bar to select the serial Port", 20, 30);
}
else
{
text("Arduino is connected! Press b to jump.", 20, 30);
applyForce(breeze);
applyForce(gravity);
velocity.add(acceleration);
velocity.mult(drag);
position.add(velocity);
acceleration.mult(0);
ellipse(position.x,position.y,mass,mass);
if (position.y > height-mass/2) {
velocity.y *= -0.9; // A little dampening when hitting the bottom
position.y = height-mass/2;
heightOfBall = 0;
}
else {
heightOfBall = 1;
}
}
}
function applyForce(force){
// Newton's 2nd law: F = M * A
// or A = F / M
let f = p5.Vector.div(force, mass);
acceleration.add(f);
}
function keyPressed() {
if (key == " ") {
// important to have in order to start the serial connection!!
setUpSerial();
}
else if (key=='b'){
mass=random(15,80);
position.y=-mass;
velocity.mult(0);
}
}
// this callback function
function readSerial(data) {
////////////////////////////////////
//READ FROM ARDUINO HERE
////////////////////////////////////
if (data != null) {
// make sure there is actually a message
let fromArduino = split(trim(data), ",");
// if the right length, then proceed
if (fromArduino.length == 1) {
//sensor value is the input from potentiometer
let sensorVal = int(fromArduino[0]);
//potentiometer value ranges from 0 - 1023
//for values less than 400,wind blows to right
if (sensorVal < 400){
breeze.x=1
}
//if value between 400 and 500, wind stops so ball stops
else if(sensorVal >= 400 && sensorVal < 500){
breeze.x = 0
}
//if value greater than 500, wind blows to left
else {
breeze.x = -1
}
//////////////////////////////////
//SEND TO ARDUINO HERE (handshake)
//////////////////////////////////
}
//height of ball sent to arduino to check if ball on floor or not
let sendToArduino = heightOfBall + "\n";
writeSerial(sendToArduino);
}
}
Arduino Code
const int poten_pin = A0;
const int ledPin = 2;
void setup() {
Serial.begin(9600); // Start serial communication at 9600 bps
pinMode(LED_BUILTIN, OUTPUT);
pinMode(ledPin, OUTPUT);
pinMode(poten_pin, INPUT);
// start the handshake
while (Serial.available() <= 0) {
digitalWrite(LED_BUILTIN, HIGH); // on/blink while waiting for serial data
Serial.println("0,0"); // send a starting message
delay(300); // wait 1/3 second
digitalWrite(LED_BUILTIN, LOW);
delay(50);
}
}
void loop()
{
// wait for data from p5 before doing something
while (Serial.available())
{
digitalWrite(LED_BUILTIN, HIGH);
digitalWrite(ledPin, LOW);
//read the position of ball from p5
int position = Serial.parseInt();
if (Serial.read() == '\n') {
// Read potentiometer value
int sensorValue = analogRead(poten_pin);
//send value to p5
Serial.println(sensorValue);
}
//if ball is touching the ground i.e. height is zero, turn LED on
if (position == 0)
{
digitalWrite(ledPin, HIGH);
}
else{
digitalWrite(ledPin, LOW);
}
}
digitalWrite(LED_BUILTIN, LOW);
}
I want to create a color-mixing game. Using potentiometers that represent primary colors red, green and blue, players will have to match a presented color as closely as possible by tuning the RGB values. I will use some sort of a system or an equation to calculate a score as to how close to the target color the match is. The feedback would be instant, meaning that the color changes would appear at the same time the potentiometers are used. I think it would be nice to have multiple rounds, let’s say 5, which would progress in difficulty, and perhaps a time limit for each round. There would be a final score that could be used for the players to compete with each other. I believe that the game would be a valuable interactive experience in terms of providing a better understanding of color theory and enhancing visual perception.
Task: make something that uses only one sensor on Arduino and makes the ellipse in p5 move on the horizontal axis, in the middle of the screen, and nothing on arduino is controlled by p5
Code for P5 and Arduino commented at the end of it:
Demonstration:
EXERCISE 2
Task: make something that controls the LED brightness from p5
Code for P5 and Arduino commented at the end of it:
Demonstration:
EXERCISE 3
Task: take the gravity wind example and make it so every time the ball bounces one led lights up and then turns off, and you can control the wind from one analog sensor.
This was the hardest exercise to do. I made it work, however, instead of fully turning off when the ball is fully on the ground, the LED lights up. This would be a problem to solve in the future.
Code for P5 and Arduino commented at the end of it:
For the final project, I have decided to create an immersive experience where people could make their own NYUAD ID cards.
Usually, people do not like the photographs on their ID cards. This issue inspired me to program a way in which we can take our own photos for the ID. For instance, a user can press a button and take their photo live by looking at the camera. Maybe I will also mirror the final photograph so that it looks just as when you look at yourself in the mirror. According to my research, people actually think that a mirror reflection is much more attractive to you than a photograph.
Moreover, I have some friends who chose or go by a different name than the one in the passport. This will also be incorporated in this program, where a person could write their preferred name on a touch screen, which will be reflected on their ID.
Also, I believe that many students can relate to the issue that our ID cards are not unique and do not show any individuality. In my program, there will be an option to add accessories to their photos. For instance, I plan to incorporate different images of scarves, jewelry, hats, mustaches, etc.
NYU New York has already made some changes to their ID cards, where students can change their ID cards for free.
NYU community members who would like to:
• have their NYU ID card replaced to reflect their chosen/preferred name, or
• have the photo on their card replaced to better reflect their gender identity/expression
Furthermore, as suggested by my classmates and Professor Shiloh during the class discussion, I can add an option to control lighting using a ring light (brightness through a potentiometer and color of the light by linking the remote to Arduino). In this way, people can control the setting in which they take the photo, which is also very important to the quality of your final photograph.
The ID card will resemble the NYU ID, however, incorporating all the features mentioned above should produce a fun and engaging project.
