Author: Joonha

Concept

My project is inspired by Etch-a-sketch, a simple drawing tool with just two knobs that control the x and y coordinates of a drawing cursor. My initial thought was to create a digital replica of this tool, however, Prof. Shiloh made me I decided to challenge myself: add another drawing axis so it becomes a 3D drawing tool. It is a 3D-Etch-a-Sketch.

Interaction Design

3D-Etch-a-Sketch means that there are at least 3 knobs (potentiometers) for a drawing cursor, each for x, y, and z coordinate. Also, since the drawing can occur in front or behind a previous drawing, there need to be a way to shift/rotate the viewing angle to see where the user is drawing. So I added 2 more knobs for rotation of viewing angle in x-axis and y-axis.

In addition to this, I thought to myself, why not color, so I added 3 additional knobs for r, g, b values of drawing cursor’s color.

Two more switches (buttons) are added for eraser/reset function and change of display mode. Those functionalities will be described later.

As a result, my final product looks like the below image. A black control panel with 8 knobs and 2 buttons. Laptop screen is used to display the drawings.

Arduino Code

The Arduino code reads values from sensors, stores converted or raw values into different variables, convert, and send values to p5.js. It also receives whether p5.js finished a reset process. The detail is as the following:

• • • Sensors: 8 potentiometers and 2 switches (buttons)
      • 3 potentiometers: x, y, z coordinate of the drawing block (0-30)
      • 3 potentiometers: R, G, B value of the drawing block (0-255)
      • 2 potentiometers: x, y angle of rotation of entire drawing (0-360)
      • 1 switch: choose among displayAll/disableIndicators/disableAll
        • displayAll: display 3D space indicators and block outline
        • disableIndicators: disable 3D space indicators
        • disableAll: disable 3D space indicators and block outline
      • 1 switch: toggle eraser mode & long press to reset all
        • Build: normal drawing block
        • Remove: invisible drawing block (erase blocks)
        • Reset Warning: press 1.2 seconds to enter, warn that if pressing is continued, entire drawing will be removed
        • Reset: continued press (total 2.4 seconds) to enter, clear all blocks

Arduino code is embedded in the p5.js sketch.

P5.js Code

The role of p5.js code is to display the drawing and other miscellaneous information.

• • • On setup, a 31 x 31 x 31 block space will be created
    • Convert x, y, z values into 3D space coordinates
    • Convert R, G, B values into a color of the drawing block
    • Convert px, py values into a 3D perspective (view angle)
    • displayAll/disableIndicators/disableAll
      • displayAll: display 3D space indicators and block outline
      • disableIndicators: disable 3D space indicators
      • disableAll: disable 3D space indicators and block outline
    • toggle eraser mode & reset warning/reset all
      • Build: normal drawing block
      • Remove: invisible drawing block (erase blocks)
      • Reset Warning: press 1.2 seconds to enter, warn that if pressing is continued, entire drawing will be removed
      • Reset: continued press (total 2.4 seconds) to enter, clear all blocks

Here is the link to the p5.js code.

Communication between Arduino and p5.js

Arduino to p5.js:

• • • x, y, z coordinates of the drawing block (cursor)
    • r, g, b color of the drawing block
    • px, py (rotation axis of the viewing angle)
    • eraserMode
    • displayMode

p5.js to Arduino

• • • whether the sketch finished the reset process

Details are describes in above sections.

Highlights

p5.js:

show() {
  let fd = this.voxelSize * this.d; // used to go to next row
  let cd = fd / 2 + this.voxelSize / 2; // used to center voxel display
  translate(-cd, -cd, -cd);
  ...
  for (let i = 0; i < this.d; i++) {
    translate(this.voxelSize, 0, 0);
    for (let j = 0; j < this.d; j++) {
      translate(0, this.voxelSize, 0);
      for (let k = 0; k < this.d; k++) {
        translate(0, 0, this.voxelSize);
        this.voxels[i][j][k].show();
      }
      translate(0, 0, -fd);
    }
    translate(0, -fd, 0);
  }
  ...
}

This code displays each blocks (31 x 31 x 31) in the correct position. This code is nice because it is quite intuitive yet powerful.

Arduino:

void setEraserMode() {
  bool switchState = !digitalRead(btn_eraser);
  if (switchState) { // on
    unsigned long currentMillis = millis();
    if (!prevEraserSwitchState) { // when off > on
      eraserMillis = currentMillis;
    } else if (currentMillis - eraserMillis >= eraserTriggerDelay) { // 1.2 sec press: promptReset
      if (currentMillis - eraserMillis >= eraserTriggerDelay*2) {  // 2.4 sec press: resetAll
        eraserMode = 3;
      } else {
        eraserMode = 2;
      }
    }
  } else { // off
    if (prevEraserSwitchState) { // when on > off
      eraserMode = !eraserMode; // 0 becomes 1, nonzero becomes 0
    }
  }
  prevEraserSwitchState = switchState;  
}

This code lets a switch(button) to serve multiple functions. You can press to toggle between two values (0 and 1) and, when pressed long, this code goes through different stages (3 and 4). If released in any time during higher stages, the value resets. This code is used in setting the eraserMode/reset.

Physical component:

The panel and casing seems to be made nicely. It is clean and sturdy.

Future Improvements

During the showcase, I realized that there could be a few improvements in my project:

• • • Visible indication of color changes. People tend to turn knobs slowly (partly due to it being a bit stiff), and it is often that people do not realize the color changes on the screen. They move on to other visibly interactive sensors.
    • Removal of reset warning. Because reset warning shows a drastic change in view (a clear screen with warning message), people do not see it as a warning sign but instead an indication of actual reset. Also, people tend to not press the button for long period (1~ sec), so it may be better to have 2~2.5 sec delay with no reset warning.
    • Improved stabilization of analog read. Although there were multiple measures of making analog read stable (delay, double check, average, heat shrink tubing, etc.), there were noises in reading and it was difficult to make the drawing precise.

This post is a complementary post to my previous post, Final Project: Progress.

Previously, as a part of the progress, I posted the following video.

The video is recorded using a debug mode, which is controlling interactives with keyboard. My physical component was incomplete to create a user testing video back then.

However, I completed my physical component before the showcase.

And here is the user testing.

Finalized Concept

The finalized concept of my project is 3D Etch-a-Sketch, with dimension being 31 x 31 x 31.

Breakdown: Arduino & Circuit

• • • Direct Input (+processed): 8 potentiometers and 2 switches (buttons)
      • 3 potentiometers: x, y, z coordinate of the drawing block (0-30)
      • 3 potentiometers: R, G, B value of the drawing block (0-255)
      • 2 potentiometers: x, y angle of rotation of entire drawing (0-360)
      • 1 switch: choose among displayAll/disableIndicators/disableAll
        • displayAll: display 3D space indicators and block outline
        • disableIndicators: disable 3D space indicators
        • disableAll: disable 3D space indicators and block outline
      • 1 switch: toggle eraser mode & long press to reset all
        • Build: normal drawing block
        • Remove: invisible drawing block (erase blocks)
        • Reset Warning: press 3 seconds to enter, warn that if pressing is continued, entire drawing will be removed
        • Reset: continued press (total 6 seconds) to enter, clear all blocks
    • Direct Output: None. p5.js will take this part.
    • Serial Input: reset
      • reset: receive value 1 when the blocks are cleared. Resets some variables, including display mode and eraser mode.
    • Serial Output: All values generated by direct input
      • x, y, z coordinate of the drawing block (0-30)
      • R, G, B value of the drawing block (0-255)
      • x, y angle of rotation of entire drawing (0-360)
      • displayAll/disableIndicators/disableAll (0-2)
      • build/remove/reset warning/reset (0-3)

Breakdown: P5.js

• • • Direct Input: serial communication setup & test inputs
      • Click canvas to trigger setUpSerial()
      • Basic keyboard controls for testing.
        • Will likely be disabled in final product
    • Direct Output: displaying of 3D Etch-a-Sketch
      • (many are already described in Breakdown: Arduino & Circuit)
      • On setup, a 31 x 31 x 31 block space will be created
      • Convert x, y, z values into 3D space coordinates
      • Convert R, G, B values into a color of the drawing block
      • Convert px, py values into a 3D perspective (view angle)
      • displayAll/disableIndicators/disableAll
      • toggle eraser mode & reset warning/reset all
    • Serial Input: All values coming from Arduino
      • (already described in Breakdown: Arduino & Circuit)
      • x, y, z
      • R, G, B
      • px, py
      • displayAccessoryMode
      • eraserMode
    • Serial Output: reset
      • (already described in Breakdown: Arduino & Circuit)

Summary

Arduino will take care of all inputs. P5.js will take care of all outputs. All interactions have one-to-one direct outcomes.

Progress (Dec 3, 2022): p5.js

Since the sketch will not work without Arduino side being set up, the sketch will be linked: https://editor.p5js.org/ob2sd/sketches/2C81Ie2p_

As of now, the test mode can be enabled with the below modification:

function draw() {
  if (!serialActive) {   // serialActive  -->  !serialActive
    ...
  }
}

Use wheel click or right click + Esc to interact with the canvas. Left click will trigger setUpSerial(). Use arrow keys to change perspective (rotation angle of entire drawing). Use numpad (123  789) to control x, y, z coordinate of the drawing block.

Progress (Dec 6, 2022): p5.js

Test mode can be enabled with the following modification:

let debug = true;

Use Ctrl + F and find variable debug to see which interactions are implemented in debug mode.

The physical controller is in the process of being built.

Group

I will be likely working as solo, although I am open to anyone who can help me with the physical side of the project.

Groupmates: Hessa, Aisha

Schematics

The schematic is made to be compatible throughout the assignment.

This schematic is used for Exercise 2.

Exercise 1:

1. • make something that uses only one sensor  on arduino and makes the ellipse in p5 move on the horizontal axis, in the middle of the screen, and nothing on arduino is controlled by p5

Arduino Code (Credit: Hessa):

void setup() {
  Serial.begin(9600); // initialize serial communications
}
 
void loop() {
  // read the input pin:
  int potentiometer = analogRead(A0);                  
  // remap the pot value to fit in 1 byte:
  int mappedPot = map(potentiometer, 0, 1023, 0, 255);
  // print it out the serial port:
  Serial.write(mappedPot);
  //Serial.println(mappedPot);
  // slight delay to stabilize the ADC:
  delay(1);                                            
}

Exercise 2:

• • make something that controls the LED brightness from p5

Arduino Code (Credit: Aisha):

int LED = 5;

void setup() {
  Serial.begin(9600);
  pinMode(LED, OUTPUT);
  // start the handshake
  while (Serial.available() <= 0) {
    Serial.println("Wait");  // send a starting message
    delay(300);               // wait 1/3 second
  }
}

void loop() {
  // wait for data from p5 before doing something
  while (Serial.available()) {
    int brightness = Serial.parseInt(); 
    if (Serial.read() == '\n') {
      analogWrite(LED, brightness); // turn on LED and adjusts brightness
      Serial.println("LIT"); 
    }
  }
}

Exercise 3:

1. • take the gravity wind example (https://editor.p5js.org/aaronsherwood/sketches/I7iQrNCul) and make it so every time the ball bounces one led lights up and then turns off, and you can control the wind from one analog sensor

Code is included in the above sketch.

Working Video of Exercise 3

Inspiration: Etch a Sketch

Etch a Sketch is a simple drawing tool. Users can rotate two knobs to alter x and y positions of a drawing dot, creating a continuous drawing. To erase the drawn image, users have to shake the board.

Brainstorming

Creating an Arduino version of Etch and Sketch does not seem to be too difficult, so I am thinking of a few things that make the Arduino version somewhat unique. However, there are a few guidelines I would like to maintain:

1. 1. 1. Since it is a drawing tool, the control of the dot has to be precise.
      2. Related to (1), the final product should be intuitive to use.

So, the Arduino version of Etch and Sketch will likely have potentiometers to act as knobs and a button to erase the drawing. The below is the list of additional features that may or may not be implemented in my project:

1. 1. 1. 3D. Introducing another drawing axis, people can create a 3D drawing. However, this may be a challenging task, and other features may not be implemented if I focus on creating this.
      2. Partial erasing. Similar to how texts are erased on smartphones, people will be able to undo their drawing, and erasing will become faster if keep erasing.
      3. Color/stroke thickness.
      4. Pause/Resume drawing. Similar to lifting a pen to draw things on another side, the drawing of the dot may pause and resume at any time. Maybe this can be implemented with the color feature using RGBA.

Hopefully, unlike my midterm project, the final product of this project will be not too complex.

This post is a mirror to the following post: Link

Concept: 

To be frank, we both didn’t really have a mind-blowing, creative inspiration when we got started on this project, which is why we decided to create something that’s simple but also making sure that it will leave enough room for user to be creative with it on our behalf. The idea that we had was to create an instrument that is able to be controlled by the user, such as its beats, pitch, etc., and it consisted of one digital sensor (switch), one analog sensor (potentiometer), and a Piezo speaker, along with the according wires that were needed. Here’s the picture of our initial state that we began with, along with the schematic:

Process/Coding:

While building the circuit itself was fairly easy and efficient since it was just a matter of connecting the wires into the respective places, the coding process was a little bit more complicated as we ran into a few obstacles:

• Initially, the turn-off function of the switch wasn’t working — this was because of the delay time, so we realized that we had to press on the switch for a while instead of just pressing lightly.
• We also had to adjust the tone range of our “instrument” — initially it was around 5000, but we later made it to 2000 so that it doesn’t hurt our ears as much even when we turn it up to the maximum tone.
• Sometimes, the instrument won’t turn off at all even when we pressed down on the switch with enough strength and time; this was due to a small error in the code.

But after a few trials and errors, we were able to fulfill our initial vision for our instrument by controlling both the beat and the pitch either simultaneously or separately.

There was one aspect of the code that was still slightly flawed when we went back to review it, but the final product seemed to work fine despite it so we kept it, which is shown here:

To see the entire code, click here.

Final Product: 

Here’s what the final product looked like:

Here’s a brief instruction on how to use the instrument:

1. When you press on the switch button, the Piezo buzzer will start making sounds.
2. You can use the potentiometer to control the pitch/tone of the sound and you can use the switch button to control the beat length of each sound. Mind the fact that each time you press on the switch button, the beats will get faster and faster. There will be 5 different beat speed offered.
3. By pressing the switch button for the 6th time, you will be able to turn off the instrument completely.

…and here’s the video of it running:

Reflection:

Despite some minor hiccups that we’ve faced during the making process, this project was actually pretty manageable and simpler than we thought it’d be! It was exciting to hear how the sound changed as we adjusted the potentiometer/pressed the switch button to create a melody. Potentially creating a song or playing an already existing one by using this instrument that we made will be really fun, and we’re proud of the final product that we ended up with!

Inspiration

I am pretty sure when people saw blue and red LED’s together, many of them thought of police light in some way. So did I. Because there were tight requirements in this assignment, I felt those requirements somewhat limited my options.

Schematics

As shown in the picture, there are 1 button, 1 potentiometer, and two LED’s. The parts were chosen to meet the following requirements:

1. 1. One analog sensor
   2. One digital sensor
   3. LED output in analog fashion
   4. LED output in digital fashion

Setup Overview

The setup is identical to the schematics.

Code Highlight

In the code, I implemented three different output(LED light) modes. Those modes can be set via a switch, and the speed of progress for each modes can be controlled via potentiometer. The below three functions are responsible for different output modes. Brief descriptions of each mode can be found below.

void blink();  // both turn on&off at the same time
void alternate();  // alternate lighting
void glow(); // only blue LED; gradually increase and decrease brightness

These different modes are chosen through a switch, which will be processed in the following code:

void manageOutputPattern() {  // button press -> change output pattern
  bool switchState = digitalRead(powerSwitch);
  if (switchState && !prevSwitchState) {  // ensures unique ON signal
    outputPattern = (outputPattern + 1) % 3;
    counter = 0;
    allOff();
  }
  prevSwitchState = switchState;
}

If the previous button state was false and the current state is true, the output mode will change to the next one. The previous state has to be tested because a button will keep sending true while it is being pressed. We only want the moment where the signal changes from false to true.

Other parts of the code can be found here.

Operation

The video shows three different output modes and modified progress speeds (slow & fast).

Reflection

Although it took quite a bit of trials and errors, I became familiarized with using circuit components, Arduino IDE, and fritzing. In the future, I would like to learn how to efficiently design my circuit.

Brainstorming

At first, I tried to create a switch that works via static electricity. Attach a conductive tape to a piece of plastic bag and make it connect each end of wires when static electricity is applied to the switch. However, I soon realized that there are two major problems:

1. 1. 1. I need a lot of static electricity to effectively lift the plastic bag.
      2. A bit of wind can mess up the result.

To resolve these issues, I could have tried a balloon with lots of physical/spatial constraints (such as a box) that limit the position and rotation of the balloon, but since large, delicate systems often lead to lots of errors, I decided build something smaller and simpler: a water sensor.

Overview

Overview

The idea here is very simple. I arranged so that each end of wire is placed very close to each other but not touching. With the fact that tap water is conductive, I can activate the switch with a drop of water.

Operation

Of course, if the water is removed manually or through evaporation, the switch disconnects and the light turns off.

Possible Application

Because the circuit completes at the presence of water, it may be able to tell whether you should water the plant.

In-game display

This game is inspired by Tanks, a game which user(s) can control tanks to destroy enemy tanks. To play this game, each player needs to apply basic geometry and trigonometry to target and hit enemy tanks.

My version of Tanks has similar mechanics with some differences. The game will rotate turns between players, limiting each turns to take no more than 20 seconds. During their turns, players can adjust launch angle, launch power, position, and launch shells to attack opponent(s) using a keyboard. Launched shells will follow a simple physics law and eventually land on either the landscape or an opponent tank, permanently changing the landscape or damaging the opponent tank. While the game is running, there will be a dashboard which indicate which player is playing, how much HP (life) is remaining, what angle the tank is aiming, how strong the launching power will be, and how much time left before the turn ends. The match/game ends when there is one or zero (there can be a tie!) players on the map with some health.

On browser that blocks canvas-based fingerprinting, the game may not work as intended.
 

When I started working on this project, I tried to implement a lot of features. And later, I realized that I again underestimated the work I need to do to write this game. The below is the list of canceled/partially implemented features of this game due to limitations of reality.

1. 1. Adjustable number of players
   2. Customizable tank colors
   3. Map generator (users can draw their own map)
   4. More sound effects

However, there are also many features that are implemented successfully.

1. 1. Interactable instructions page
   2. In-game menu for pause, restart, and exit the game.
   3. Multiplayer (hardcoded 2)
   4. Image-based maps (drawn on mspaint.exe)
   5. Interactable modifications on the map
   6. Tanks and their controllable features
   7. Turns, player indicator (the yellow arrow) and time limits
   8. Functioning dashboard
   9. Physics of launched shells
   10. Some sound effects
   11. Etc.

Among those, I am especially proud of 4 and 5. Below, I will spend some time explaining how they are done.

function preload() {
  for (let i=1;i<=5;i++) {
    append(gameMaps, loadImage("maps/map_"+i+".png"));
  }
  ...
}

First, images of maps needs to be loaded before the game starts. To be used as maps, they need to at least satisfy two things. First, the pixel color for air must be similar to [190, 251, 254, 255]. Second, there must be enough non-air pixels for tanks to land on when the game starts. If these two are satisfied, pretty much any image can be used as a playable map. Once the maps are loaded, Tank objects and Shell objects can interact with air and non-air pixels.

function compareColors(a, b) {
  // used to compare colors
  if (!a || !b) {
    return false;
  } // if one of them is null, false
  if (a.length != b.length) {
    return false;
  } // if length differs, false
  for (let i = 0; i < a.length; i++) {
    if (abs(a[i] - b[i]) > 7) {
      return false;
    } // difference may be negligible
  }
  return true;
}

// Partial code from class Shell
...
if (this.y>0&&!compareColors(get(this.x, this.y), c.ac)) {
  this.explode();
}
if (this.y > height) {
  this.explode();
}
...

The above code uses a get() function to check whether a launched shell is in the air or overlapped with non-air pixels (ground, map boundaries, and tanks) and decide if it will explode or not. If it chooses to explode, it will trigger the below code.

append(c.explosions, [this.x, this.y, this.explosionRadius]);

It records where the explosion happened and the size of the explosion. The recorded explosions are later used in the below code so the pixels affected by the explosions are “erased” or filled with air color.

modifyMap() {
  push();
  noStroke();
  fill(this.ac);
  for (let i = 0; i < this.explosions.length; i++) {
    let tmp = this.explosions[i];
    circle(tmp[0], tmp[1], tmp[2]);
  }
  pop();
}

Of course, there were(and still are) a lot of difficulties when writing this game, and below are the major ones.

• • Object overlapping
    • Because the map is modifiable, positions of all objects are relative to their surroundings, and in some cases, tanks can overlap each other and show unintended behavior. There seems no easy fix for this.
  • Continuous sound effect
    • When tried to use sound effects for continuous events (such as tanks moving, aim adjusting, etc.), it results in undesirable sound. This may be solvable by finding a good sound file that has matching start and end tone, but I could not find those. So I removed those sound effects.
  • Uphill/Downhill movement
    • Similar to object overlapping, tanks moving uphill and downhill causes them to stutter.
  • Interaction failure with air and non-air pixels
    • In the early stage of coding this game, tanks could stop in the air or indefinitely fall down into the ground. This is mainly due to pixels being meshed together when pixel density is not 1, but it is fixed by having threshold on color differences.
  • Simultaneous keyboard control
    • There were times when multiple keys were pressed, only one (the latest) one will be recognized, and this key will remain recognized unless all keys are released or a new key is pressed. Fixed this issue by using keyIsDown() in draw() instead of keyPressed().
  • There may be other major issues that I have encountered, but I cannot remember them at this point.

From this project, I learned that even a simple-looking feature can be long and complex to implement. For the later projects,  I will try not to focus on implementing multiple features; I will instead focus on making small features more reliable and smooth.

This time, I had to think what kind of data I wanted to visualize. For some reason, I did not want to use the real-world data. Maybe I wanted to represent something more general. After thinking awhile, I decided to use a list of prime numbers. I got the numbers from here.

This time, I will show the results first.

Click to view the full image – you can zoom in to see the details

Which is generated from the below:

It does not mean much as data, but it shows an interesting pattern. Roughly speaking, what I am doing here is that I am printing out 9 different results based on different approximations of PI, each starting from the center and incrementing the distance from the center for each point. The position of each points is determined by the distance from the center and an angle, which is determined by the value of prime numbers multiplied by an approximate PI value. Those are shown in the code below.

function plot(num,row,col) {
  push();
  translate(col*width/4,row*height/4);
  for (let i=(frameCount-1)*500;i<frameCount*500&&i<numbers.length/2;i++) {
    push();
    rotate(Number(numbers[i])*num);
    point((-i/350),0)
    pop();
  }
  noStroke();
  text(num,0,numbers.length/700+40);
  stroke(80,200,80);
  pop();
}

The for condition here is slightly modified so it is 500 times faster than associating the drawing 1:1 with frameCount. It still plots all  the points I want.

Also, the visualization does not have to involve prime numbers. It could have been the incrementing number i, although it may have a bit different outcome.

Click to view the full image – you can zoom in to see the details

The label here is wrong, but the results are still interesting. It appears as if a tangled ball of string is unfolding as we approximate more precisely.

For the next project, if feasible, I will try to reuse the modified for condition because it has a big advantage of processing every step while speeding up. There is no need for frame rate modification.