For my final project, I first decided to utilize NeoMatrix to create an interactive pixel art board. In the beginning everything was going okay, I spent a few nights trying to connect the wires to the tiny holes in the NeoMatrix and in the end connect two out of four boards, mainly because two other boards failed to light up. The soldering process was probably the hardest part of the entire project. Since I was not used to the machine and had barely any experience beforehand, I was left in quite a tough situation, with a bunch of burn marks and a hope which was ultimately extinguished after failing to connect a 16×16 board.
Although the situation was hard, I finished my code for the p5js end. A snipped of the most important bits can be seen below.
const WINDOW_WIDTH = 800
const WINDOW_HEIGHT = 400
const TILE_WIDTH = WINDOW_WIDTH/2
const TILE_HEIGHT = WINDOW_HEIGHT
const NUMBER_ROWS = 8
const NUMBER_COLUMNS = 8
const WIDTH = TILE_WIDTH/NUMBER_COLUMNS
const HEIGHT = TILE_HEIGHT/NUMBER_ROWS
let serial;
let pressed_id = 0;
let state = 0;
let r = 255;
let g = 255;
let b = 255;
boxes = []
class Box{
constructor(x, y, w, h, id){
this.x_pos = x
this.y_pos = y
this.box_width = w
this.box_height = h
this.id = id
this.clicked_state = 0
}
display(){
if(this.clicked_state == 1){
fill(255, 0, 0)
}
else{
fill(255, 255, 255)
}
rect(this.x_pos, this.y_pos, this.box_width, this.box_height)
}
}
function setup() {
createCanvas(WINDOW_WIDTH, WINDOW_HEIGHT);
background(220);
id = 0
for(let n = 0; n < 3; n++){
for(let i = 0; i < NUMBER_ROWS; i++){
box_rows = []
for(let j = 0; j < NUMBER_COLUMNS; j++){
append(box_rows, new Box(n*TILE_WIDTH+j*WIDTH, i*HEIGHT, WIDTH, HEIGHT, id))
id++
}
append(boxes, box_rows)
}
}
}
function draw() {
for(let i = 0; i < boxes.length; i++){
for(let j = 0; j < NUMBER_COLUMNS; j++){
boxes[i][j].display()
}
}
}
function mouseClicked(){
for(let i = 0; i < boxes.length; i++){
for(let j = 0; j < NUMBER_COLUMNS; j++){
if(mouseX > boxes[i][j].x_pos && mouseX < boxes[i][j].x_pos+boxes[i][j].box_width &&
mouseY > boxes[i][j].y_pos && mouseY < boxes[i][j].y_pos+boxes[i][j].box_width){
// console.log("(" + boxes[i][j].x_pos + ", " + boxes[i][j].y_pos + ", " +boxes[i][j].id +")")
if(boxes[i][j].clicked_state == 0){
boxes[i][j].clicked_state = 1
}
else{
boxes[i][j].clicked_state = 0
}
state = boxes[i][j].clicked_state
pressed_id = boxes[i][j].id
}
}
}
}
function keyPressed(){
if (key == " ") {
// important to have in order to start the serial connection!!
setUpSerial();
}
}
// This function will be called by the web-serial library
// with each new *line* of data. The serial library reads
// the data until the newline and then gives it to us through
// this callback function
function readSerial(data) {
////////////////////////////////////
//READ FROM ARDUINO HERE
////////////////////////////////////
if (data != null) {
// make sure there is actually a message
// split the message
let fromArduino = split(trim(data), ",");
// if the right length, then proceed
if (fromArduino.length == 1) {
// only store values here
// do everything with those values in the main draw loop
// We take the string we get from Arduino and explicitly
// convert it to a number by using int()
// e.g. "103" becomes 103
rVal = int(fromArduino[0]);
// alpha = int(fromArduino[1]);
}
//////////////////////////////////
//SEND TO ARDUINO HERE (handshake)
//////////////////////////////////
let sendToArduino = pressed_id + "," + state + "," + g + "," + r + "," + b + "\n";
console.log(pressed_id + "," + state + "," + g + "," + r + "," + b + "\n")
writeSerial(sendToArduino);
}
}
The code ran perfectly when it came to execution. The tiles were matched with the pixel placement of the NeoMatrix, which went in a zigzag pattern. After reading many pages of documentation for the Adafruit electronics, I was able to properly display the selected pixel from the p5js screen. Although all seem dandy, disaster struck at 12am in the morning.
The serial connection was working properly and the Adafruit library was maintaining the pixel array. However, unbeknownst to me, who was testing the board with only one color, green, the board did not display colors properly. After realizing this fact, I set out to work and spent a good couple of hours trying to fix the issue. After reviewing the documentation for what felt like the hundredth time, the board was still not working. Within the NeoMatrix setup, there was a line called “NEO_GRB” which stands for Green Red Blue. Without the serial connection, this line would produce normal colors that were set on a pixel. However, when paired, the matrix would produce only green, blue and turquoise. When changed to “NEO_RGB”, the pixel would be colored either red, blue or purple. No matter which way I tried, only two out of three colors would show up. By checking on similar issues online, I found that many people were met with the same issue, but little knew of how to help. I tried many possible solutions but none of them worked. With only 2 days left until the showcase and the deadline had already passed, I was forced to abruptly change my final project. I decided to recreate an Etch-A-Sketch.
I admit that it was my issue that I did not start the serial connection process sooner, but having finished both the code and physical part, I believed there to be no issue with the process. Unfortunately, some things are just bound to fail and to save myself tears I decided to not waste time on something that I would not be able to fix. At least, if I tried something else, I would have somewhat of a satisfactory product by the end. Although my original idea is something I am very much more proud of, I believe that navigating through this crisis taught me to think of every detail that could go wrong in the future.
The Etch-A-Sketch was a project I believed to be very easily doable, even within the few days I had left. I already had code prepared for this project as it was also one of my original ideas at the start. All that was left was to create the physical components. By using potentiometers and a flex sensor, I would attempt to recreate a cult classic. The flex sensor would be used to detect rapid movement of an external device and delete the screen, imitating the shaking needed to erase an Etch-A-Sketch drawing.
For the Arduino, the code was very simple. I only needed to collect the analog reads and calculate the angle of flex that would be used to clear the screen. The Arduino code can be seen below. The basic premise of the code is to gather the inputs from the flex sensor and the two potentiometers and send them to p5js for use. The LED code was used as control to check whether there was serial connection happening between Arduino and p5js.
// Measure the voltage at 5V and the actual resistance of your
// 47k resistor, and enter them below:
const float VCC = 4.98; // Measured voltage of Arduino 5V line
const float R_DIV = 47500.0;
// Upload the code, then try to adjust these values to more
// accurately calculate bend degree.
const float STRAIGHT_RESISTANCE = 37300.0; // resistance when straight
const float BEND_RESISTANCE = 90000.0; // resistance at 90 deg
int leftLedPin = 2;
int rightLedPin = 5;
void setup() {
// Start serial communication so we can send data
// over the USB connection to our p5js sketch
Serial.begin(9600);
// We'll use the builtin LED as a status output.
// We can't use the serial monitor since the serial connection is
// used to communicate to p5js and only one application on the computer
// can use a serial port at once.
pinMode(LED_BUILTIN, OUTPUT);
// Outputs on these pins
pinMode(leftLedPin, OUTPUT);
pinMode(rightLedPin, OUTPUT);
// Blink them so we can check the wiring
digitalWrite(leftLedPin, HIGH);
digitalWrite(rightLedPin, HIGH);
delay(200);
digitalWrite(leftLedPin, LOW);
digitalWrite(rightLedPin, LOW);
// start the handshake
while (Serial.available() <= 0) {
digitalWrite(LED_BUILTIN, HIGH); // on/blink while waiting for serial data
Serial.println("0,0,0"); // send a starting message
delay(300); // wait 1/3 second
digitalWrite(LED_BUILTIN, LOW);
delay(50);
}
}
void loop() {
// wait for data from p5 before doing something
while (Serial.available()) {
digitalWrite(LED_BUILTIN, HIGH); // led on while receiving data
int left = Serial.parseInt();
int right = Serial.parseInt();
if (Serial.read() == '\n') {
digitalWrite(leftLedPin, left);
digitalWrite(rightLedPin, right);
int shaker = analogRead(A0);
float flexV = shaker * VCC / 1023.0;
float flexR = R_DIV * (VCC / flexV - 1.0);
float angle = map(flexR, STRAIGHT_RESISTANCE, BEND_RESISTANCE,
0, 90.0);
int sensor = analogRead(A2);
delay(5);
int sensor2 = analogRead(A5);
Serial.print(angle);
Serial.print(',');
Serial.print(sensor);
Serial.print(',');
Serial.println(sensor2);
}
}
digitalWrite(LED_BUILTIN, LOW);
}
To emulate the screen of a Etch-A-Sketch, p5js was utilized with a class that would serve as the tip of the drawing needle. The code, which can be seen below, received sensor data from Arduino in order to move the elements on screen as well as delete them.
class Ball{
constructor(x_pos, y_pos, radius, color){
this.x = x_pos
this.y = y_pos
this.r = radius
this.c = color
this.weight = 2
this.x_prev = x_pos
this.y_prev = y_pos
}
moveX(value){
this.x = value;
}
moveY(value){
this.y = value;
}
display(){
stroke(this.c)
if(this.x_prev == 0 && this.y_prev == 0 ||
this.x_prev == this.r && this.y_prev == this.r){
strokeWeight(0)
}
else{
strokeWeight(this.weight)
}
line(this.x_prev, this.y_prev, this.x, this.y)
this.x_prev = this.x
this.y_prev = this.y
}
changeCol(col){
this.c = col;
}
changeWeight(weight_){
this.weight = weight_
}
}
let shaker = 0;
let rVal = 0;
let alpha = 0;
let left = 0;
let right = 0;
let canvas;
let current_theme = 255;
let button_state = 0;
let colorPicker;
let button;
let slider;
function setup() {
canvas = createCanvas(600, 600);
textSize(18);
background(255)
colorPicker = createColorPicker('#ed225d');
colorPicker.position(width+5, 0);
colorPicker.size(50,200)
button = createButton("Switch Theme");
button.position(width+5, 205)
button.size(50,50)
button.mousePressed(switchTheme)
button.style('font-size', '10px')
slider = createSlider(0, 10, 0, 1);
slider.position(width-35, 320);
slider.style("transform","rotate(90deg)");
button2 = createButton("Save Drawing");
button2.position(width+5, 400)
button2.size(50,50)
button2.mousePressed(saveDrawing)
button2.style('font-size', '10px')
mover = new Ball(rVal, alpha, 2, "#000")
clearScreen();
}
function draw() {
let val1 = map(rVal, 0, 1023, mover.r, width-mover.r)
let val2 = map(alpha, 0, 1023, mover.r, height-mover.r)
mover.display();
mover.moveX(val1);
mover.moveY(val2);
mover.changeCol(colorPicker.color())
clearScreen();
mover.changeWeight(slider.value()+2)
// one value from Arduino controls the background's red color
//background(map(rVal, 0, 1023, 0, 255), 255, 255);
// the other value controls the text's transparency value
//fill(255, 0, 255, map(alpha, 0, 1023, 0, 255));
// if (!serialActive) {
// text("Press Space Bar to select Serial Port", 20, 30);
// } else {
// text("Connected", 20, 30);
// // Print the current values
// text('rVal = ' + str(rVal), 20, 50);
// text('alpha = ' + str(alpha), 20, 70);
// }
// let writer = createWriter('newFile.txt');
// writer.write([rVal, alpha]);
// writer.close()
}
function keyPressed() {
if (key == " ") {
// important to have in order to start the serial connection!!
setUpSerial();
}
}
function clearScreen(){
if(shaker > 200){
background(current_theme)
}
}
function switchTheme(){
button_state++;
if(button_state >= 2){
button_state = 0
}
if(button_state == 0){
background(255)
current_theme = 255
}
else{
background(0)
current_theme = 0
}
}
function saveDrawing(){
save(canvas, "myCanvas", 'png');
}
function readSerial(data) {
////////////////////////////////////
//READ FROM ARDUINO HERE
////////////////////////////////////
if (data != null) {
// make sure there is actually a message
// split the message
let fromArduino = split(trim(data), ",");
// if the right length, then proceed
if (fromArduino.length == 3) {
// only store values here
// do everything with those values in the main draw loop
// We take the string we get from Arduino and explicitly
// convert it to a number by using int()
// e.g. "103" becomes 103
shaker = int(fromArduino[0]);
rVal = int(fromArduino[1]);
alpha = int(fromArduino[2]);
}
//////////////////////////////////
//SEND TO ARDUINO HERE (handshake)
//////////////////////////////////
let sendToArduino = left + "," + right + "\n";
writeSerial(sendToArduino);
}
}
Although the code is quite simple, due to the time constraint posed on me after the initial project failed, there was little to be done to increase the complexity of it. To give the users some degree of freedom, I added a color picker, a stroke changer and buttons to change the theme as well as save the canvas drawing, if they wish to do so. The screen is deleted upon shaking the shaker that comes with the Arduino.
Reflection
This project was quite fun to work on. Although my initial idea failed, I am happy to have produced at least something to show my passion for Interactive Media. I would love to rework my initial project and attempt to make it work after I have finished with this class. Knowing what went wrong and how I could have fixed it with a saner state of mind might just be what pushed me to pursue Interactive Media and Physical Computing in the long run. For now, I am just grateful to have a physical product with me to show at the Showcase and even though I wish I could have done more, I realized it is sometimes important to start anew in order to learn more than I would have if I had continued to be stuck on the broken project. I loved taking Introduction to Interactive Media this semester and I hope that by continuing my Interactive Media Minor journey, I will be able to learn much more about both coding and the physical components that bring our code to life.
