Making Interactive Art: Set the Stage, Then Shut Up and Listen
“Don’t interpret your own work”
As creators and artists, the idea of providing the user an interactive experience without description is always the hardest part of the creation. Since we are used to giving description of the “what” and “how” of a product, we are often faced with questions how to let the users flow through the interaction without these pre-described elements. Personally, I have always tried to create a unique experience for the user through my creation, but I tend to end up trying to guide the users in a certain direction – limiting them to think in different perspective on the product. After reading this blog article, I realized I have to make the operation of the product discoverable, setting only the necessary elements to “begin the conversation”. This brought me back to the importance of signifiers- to allow the users to easily navigate through the experience without the creator mentioning any instructions. Also, I realized that using personas – the different cases of the experience based on potential users – will help the creators better visualize and understand what elements are necessary to be added or omitted to make the product suitable for users to interact without the creator stepping in.
My mid-term project definitely showed this fault of having myself to explain how it works and what to expect from the product. For the final project, I will encompass these traits so that I do not have to limit the possible interactive possibilities and let the users take their own approach.
Physical Computing’s Greatest Hits (and misses)
The articles provides with a number of variations of a physical computing products as well as the necessary components involved to create the product. The article definitely opened up the various pathway that I can take for my future projects since it also provided concrete example that I can take as an influence. Especially, I was intrigued by the idea of using flex sensors in gloves – as a drum gloves. From this idea, I wanted to implement a MIDI system of multiple EDM sounds that you can manipulate using different fingers to make your own electronic song – a launch pad transferred to gloves.
Moreover, the idea of “originality” used to be one of the main aspects that I consider during my creation process. I have always believed that the products I create has to be “unique”. So, whenever I encounter an interesting idea for the product, I would search on Google to see whether there is an existing creation. If I do find on on the internet, I would try to think of something else. However, after reading this article, it led me to think again about my process. First, I have never copy & pasted an existing code to my creation. Second, if I have been influenced by an existing idea or theme, I always gave a tweak. With these two parts of the process, I realized that the product is already original. And, this led me to look back at all the different ideas that I discarded.
#1: Making Interactive Art: Set the Stage, Then Shut Up and Listen
This blog post’s main message is to not “interpret your own work”. The author argues that interactive art does not require an explanation from the creator as this takes away from the interactive element of the art in the first place. It should be the audience’s decision and interpretation of your art that allows the ‘conversation’ to occur between them, you and the art.
Personally, I agree with this sentiment to some extent. I think a very important part of interactive art is its ability to be understood and performed by everyone without the need of too much interaction or explanation on the creator’s part, as its goal should be to allow its own interpretation to take place without interference. However, I also think that some art works may need some sort of introduction to them – not to interfere with the perspectives of the audience – but maybe as a form of introduction and aims. Although, if this could be done in a discrete way in which the creator is not involved, it would be ideal.
Looking at all the examples of physical computing in the author’s list made me super excited for the endless possibilities that computing opens into the world of interactive art. Most forms of work that were listed did seem the typical projects that most of us have seen before (either on the internet, art galleries etc.); however, I agree very much with the author when he says that although these projects have been done multiple times, people shouldn’t give up as they still “allow a lot of room for originality”. I felt this constructing many of my own projects for the class as they felt “unoriginal” sometimes but adding my own layer of creativity or perspective to them is what made the projects so personal and fun to make still.
But this reading brings up an extremely important point: just because it’s already been done, does not mean you should not do it.
My theater professor always says, “It may have already been done, but is hasn’t been done by YOU”. So maybe we’ve seen a Tilty controller before, but your own personal interests, strengths, and creativity can make it into something really interesting that should be pursued. The fact that something has already been explored before does not mean that it should not be explored further. Maybe there’s a hidden gem that the first explorer did not find before. Okay I feel like I’m gonna start getting metaphorical so I’ll change the subject.
Overall, this reading was really generative for ideas and reflective on how the class has been going so far. I saw my peer’s projects in some of the ideas presented, but I could also recall projects that the reading did not bring up. Not that those are more valuable or more creative, just that they were different.
Man, I am so bad at that! I always want to start my presentations with an in depth explanation of all the bells and whistles of my projects. But it’s an important thing I need to learn; let the audience interact with the piece.
One thing that really stuck with me in this article was that yes, interactive art is still self expression, but it is more of a conversation than a statement. The art piece itself is the seeds of a conversation between artist and audience. It’s important to consider that the expression will come from the experience of interacting with the piece and as they interact, more of the expression will be revealed.
It’s also important to note that the expression will be added to, as part of the conversation, by the way the audience responds to and understands the experience they had with the installation. The artist should take the time to listen to how they are interacting and responding to such.
How will the audience create their own story with the piece? Are they meant to create their own story or uncover the artist’s?
How does the audience complete the work? Is it incomplete before? At what stage is the work complete?
For my midterm project I wanted to make something more challenging and thought-out. After a lot of brainstorming, I noticed something that I have several of… secret storage boxes disguised as books.
I decided to make a safe that is disguised as a book, and make it so that it can only be locked/unlocked if a combination of gestures were done. I was at first going to use wood for the entire safe, but decided to use a combination of wood, cardboard, and transparent acrylic to make it more interesting. The paper part of the book is made from acrylic, so that if someone picked up the book from a shelf, they would see that it is not a normal book. So the transparent acrylic I decided to use is a twist to the point of safes, which is to keep objects hidden safe, in secret. The transparent barrier would let them see the money and valuables inside it, but no matter what they tried, they would not be able to open it, because only its owner would know the correct combination of gestures. This project was inspired by a combination of my love for Harry Potter and Sherlock Holmes. I didn’t notice this when I actually made it, but the safe box is a combination of the two storage boxes in the first image above. Although I had originally not thought of using transparency, the final result of my work was somehow subconsciously a mix of the two boxes, using the book-like shape of one and the transparent sides and solid top and bottom of the other.
I thought of adding a buzzer to make a sound when the person got the code wrong or an alarm after 3 wrong tries, but I thought that that wouldn’t make as much sense for my idea, because I want people to think that the box does not open, and a buzzer would give it away, and encourage them to try again. For this reason I also decided not to use any LEDs. I initially had a different combination of gestures, also using an up gesture, but decided to not use it as results were inconsistent with the up gesture.
materials/equipment:
-wood
-handsaw
-cardboard
-transparent acrylic
-laser cutter (for the 3 acrylic pieces)
-servo
-ZX gesture sensor
-Breadboard
-Redboard
-Jumper wires
-Glue (I mainly used acrylic glue, super glue, and a glue gun)
-paper (for book cover) – I printed and used the cover of the ‘getting started with Arduino’, which was co-written by Michael Shiloh, an NYUAD IM professor. I initially thought of using this cover just as a pleasant and funny surprise for the professor, since I knew he would be there when showing our projects, but this was perfect because not only did the color of the sensor match the color scheme of the cover but there was also a UNO board on it, that I put my sensor on to disguise.
I was a bit worried that because the gesture wasn’t too complex, it would open easily if people did any gesture, but it was great, and very entertaining I might add, to watch everyone try to guess how to open it and not be able to, until they eventually gave up and let me show them how it’s done! The product turned out how I wanted it to, and worked consistently.
#include <Servo.h>
#include <Wire.h>
#include <ZX_Sensor.h>
//const int ZX_ADDR = 0x10;
int gestureSequence[] = {
0, 1, 2 // 0 is right swipe and 1 is left, 2 is up
};
int index = 0;
const int ZX_ADDR = 0x10;
ZX_Sensor zx_sensor = ZX_Sensor(ZX_ADDR);
bool leftGestureStarted = false;
bool rightGestureStarted = false;
bool upGestureStarted = false;
int prevZ = 240;
uint8_t x_pos = 0;
uint8_t z_pos = 0;
Servo myservo;
void setup() {
// put your setup code here, to run once:
uint8_t ver;
myservo.attach(9);
Serial.begin(9600);
Serial.println();
Serial.println("-----------------------------------");
Serial.println("SparkFun/GestureSense - safebox");
Serial.println("-----------------------------------");
if ( zx_sensor.init() ) {
Serial.println("ZX Sensor initialization complete");
} else {
Serial.println("Something went wrong during ZX Sensor init!");
}
ver = zx_sensor.getModelVersion();
if ( ver == ZX_ERROR ) {
Serial.println("Error reading model version number");
} else {
Serial.print("Model version: ");
Serial.println(ver);
}
if ( ver != ZX_MODEL_VER ) {
Serial.print("Model version needs to be ");
Serial.print(ZX_MODEL_VER);
Serial.print(" to work with this library. Stopping.");
while (1);
}
//
// Read the register map version and ensure the library will work
ver = zx_sensor.getRegMapVersion();
if ( ver == ZX_ERROR ) {
Serial.println("Error reading register map version number");
} else {
Serial.print("Register Map Version: ");
Serial.println(ver);
}
//
if ( ver != ZX_REG_MAP_VER ) {
Serial.print("Register map version needs to be ");
Serial.print(ZX_REG_MAP_VER);
Serial.print(" to work with this library. Stopping.");
while (1);
}
myservo.attach(9);
}
void loop() {
// put your main code here, to run repeatedly:
//gesture stuff
int gesture = -1;
if (zx_sensor.positionAvailable()) {
x_pos = zx_sensor.readX();
z_pos = zx_sensor.readZ();
if (x_pos != ZX_ERROR) {
Serial.print("X: ");
Serial.print(x_pos);
}
if (z_pos != ZX_ERROR) {
Serial.print(" Z: ");
Serial.print(z_pos);
}
Serial.print(" Index: ");
Serial.print(index);
Serial.print(" Left Started: ");
Serial.print(leftGestureStarted);
Serial.print(" Right Started: ");
Serial.print(rightGestureStarted);
Serial.print(" Up Started: ");
Serial.println(upGestureStarted);
// ===================== //
if (z_pos < 220 && index < 3) {
// Serial.println("z less than 100");
if (leftGestureStarted == false && rightGestureStarted == false && upGestureStarted == false)
{
if (x_pos > 130 && x_pos < 200)
{
leftGestureStarted = true;
}
else if (x_pos < 110 && x_pos > 80) {
rightGestureStarted = true;
}
// else if (x_pos > 110 && x_pos < 170 && z_pos < 40) {
// upGestureStarted = true;
// }
}
}
} else {
leftGestureStarted = false;
rightGestureStarted =false;
upGestureStarted = false;
}
//then in another if statement
if (leftGestureStarted == true) {
if (x_pos < 100)
{
gesture = 1;
leftGestureStarted = false;
}
}
if (rightGestureStarted == true) {
if (x_pos > 130 )
{
gesture = 0;
rightGestureStarted = false;
}
}
// if (upGestureStarted == true) {
// if (z_pos > 60)
// {
// //Copy and Paste
// gesture = 2;
// upGestureStarted = false;
// }
// }
// then check if the gesture is the current correct gesture:
if (gesture >= 0) {
if (gestureSequence[index] == gesture) {
index++;
} else {
index = 0;
Serial.println("wrong gesture");
}
}
if (index == 2) {
myservo.write(0);
// 3 in this case because we have 4 gestures so starting from gesture 0=> counting 0,1,2,3
// unlock safe
delay(3000);
index=0;
} else myservo.write(90);
prevZ = z_pos;
}
I have become increasingly interested in organic movement and mimicking it via mechanical means. Considering the skills I have come to develop with servos, my first thoughts moved towards a peacock or lizard. As I began to think of materiality and found that I might enjoy the cardboard texture on a lizard as compared to a peacock, an animal meant to be stunning and pristine.
The goal was to create a simple creature that would respond to an approaching “threat”. First, at a safe distance, the creature would light up, establishing discomfort. As one walks closer, it would growl, and when too close it would flare its fan to assert its dominance over the situation.
These reactions were centred around an IR Distance Sensor, which would calculate proximity. A series of LEDs would construct the eyes, a small piezo buzzer for the growl, and two servos connected to an difficult-to-glue set of fans.
The following is the code that organizes the system.
#include <Servo.h>
//For distance sensor: Establish which pin sends the signal, and which pin will recive it.
const int trigPin = 7;
const int echoPin = 6;
const int ledPinR1 = 13;
const int ledPinR2 = 12;
const int ledPinR3 = 9;
const int ledPinL1 = 8;
const int ledPinL2 = 5;
const int ledPinL3 = 4;
const int buzzer = 3;
#define ROAR 100
//Declare the servos
Servo servo1;
Servo servo2;
int servoPin1 = 10;
int servoPin2 = 11;
void setup() {
// put your setup code here, to run once:
//The trigger pin sends the signal. THe echo pin recieves it.
pinMode(trigPin, OUTPUT);
pinMode(echoPin, INPUT);
pinMode (ledPinR1, OUTPUT);
pinMode (ledPinR2, OUTPUT);
pinMode (ledPinR3, OUTPUT);
pinMode (ledPinL1, OUTPUT);
pinMode (ledPinL2, OUTPUT);
pinMode (ledPinL3, OUTPUT);
pinMode (buzzer, OUTPUT);
servo1.attach(servoPin1);
servo2.attach(servoPin2);
Serial.begin (9600);
}
void loop() {
// put your main code here, to run repeatedly:
// Send a signal, the stop sending a signal.
long duration, distance;
digitalWrite(trigPin, LOW);
delayMicroseconds(2);
digitalWrite(trigPin, HIGH);
delayMicroseconds(20);
digitalWrite(trigPin, LOW);
// Detect the signal sent.
duration = pulseIn(echoPin, HIGH);
distance = duration / 29 / 2 ;
Serial.println (distance);
//Establish when each thing (LED, buzzer, Servos) does what, and when.
if (distance <= 20) { // If at a CLOSE distance move the servos.
servo1.write(100);
servo2.write(100);
} else {
servo1.write(0);
servo2.write(0);
}
if (distance <= 50) { // If at a MED distance move the servos.
tone(3, ROAR);
} else {
noTone(3);
}
if (distance <= 100) { // If at a FAR distance move the servos.
digitalWrite(ledPinR1, HIGH);
digitalWrite(ledPinR2, HIGH);
digitalWrite(ledPinR3, HIGH);
digitalWrite(ledPinL1, HIGH);
digitalWrite(ledPinL2, HIGH);
digitalWrite(ledPinL3, HIGH);
} else {
digitalWrite(ledPinR1, LOW);
digitalWrite(ledPinR2, LOW);
digitalWrite(ledPinR3, LOW);
digitalWrite(ledPinL1, LOW);
digitalWrite(ledPinL2, LOW);
digitalWrite(ledPinL3, LOW);
}
delay(100);
}
We also had to do some readings on physical computing and interactive art. Texts like these are always able to provide me with comfort in the notion of creativity as iteration. No ideas are original, but all ideas are unique.
Out readings on affordance and signalling speak to this idea, in that technology can be additive. Naturally occurring objects have particular forms of affordance, and as humans we add signals. Primitive hatchets and blades were mere sharp rocks many centuries ago. Now, however, we find ourselves in an intense era of weaponry. Of course, weapons aren’t the most advisable artistic creation, but creativity within the small scope of the 20th and 21st century can function the same way.
Instead of attempting to steal an idea, I hope to build off of others’ work. I also hope that we come to create a climate under the assumption of goodwill. If a work looks similar to another, we should praise dialogue between them. However, such ideas will take time to manifest in a capitalist structure. But I expect that we as artists, designers, and creatives reach this space.
At first I wanted to make a project that would make sounds through moving small objects – and when thinking of what would be the interactive input, I discovered the stress ball. However, talking to Aaron, who brought some insightful inspiration and ideas, pushed me incredibly in the direction very close to the final piece.
The project is a simple game that consists of two stress balls and two servo-driven boxes installed above the heads of each contestant. There is one pressure sensor on each stress ball which detects a certain range – so that the person has to press it down and release it, which counts as one press. Whoever reaches the count of 50 first, will trigger opponent’s servo and make the box tilt over their head and spill what’s inside (aluminium foil tiny balls, paper clips, pieces of paper anything). There is also a restart button included so that the game can be very easily started over.
However, there were several issues that I struggled a lot with – one of which is how fragile the pressure sensors are. I soldered wires to it to make it possible for the ball to be held in a bigger distance, taped it to a ball and sewed it in a pouch of cloth to secure the pressure sensor (and to make the ball look a bit prettier, nicer to touch as well as visually signify where to press). Yet, when I was testing it with couple of people – in the heat of the game, the soldered part was broken very easily. When it happened for the first time, I changed one sensor in terms of soldering, the second one in terms of taping. And as I found out after the class started and found one the sensors broken again – it was the way it was soldered, which put a way too much pressure on the pressure sensor (a bit of irony in that, isn’t it). Of course there were issues with getting the code as well – for example when I tried to incorporate LEDs and the servos got very lazy, but luckily there was Aaron to help me get moving.
Here is a set up of the breadboard, as well as the box and mechanism attached to a chair.
Video of game in action:
The code:
#include <Servo.h>
//restart button SET UP
int restartButton = 2;
int prevButtonState = LOW;
//PLAYER A SET UP pressure
int pressurePinA = A0;
int pressureValueA = 0;
int prevPressureValueA = 0;
bool pressureA_enabled = true;
//PLAYER A SET UP servo
Servo servoA;
bool servoStateA = false;
//PLAYER B SET UP pressure pin
int pressurePinB = A1;
int pressureValueB = 0;
int prevPressureValueB = 0;
bool pressureB_enabled = true;
//PLAYER B SET UP servo
Servo servoB;
bool servoStateB = false;
//counting the amount of times the two players meet the pressure range ("presses the ball") - starting at 0
int triggerCounterA = 0;
int triggerCounterB = 0;
void setup() {
// pinMode(ledPinA, OUTPUT);
servoA.attach(5);
servoB.attach(6);
pinMode(restartButton, INPUT);
Serial.begin(9600);
}
void loop() {
//read what the pressure values of the pins are
int pressureValueA = analogRead(pressurePinA);
int pressureValueB = analogRead(pressurePinB);
//trigger counter PRESSURE condition for PLAYER A
if (pressureValueA >= 800 && pressureA_enabled == true) {
triggerCounterA = triggerCounterA + 1;
pressureA_enabled = false;
}
if (pressureValueA < 600 && prevPressureValueA > 600) {
pressureA_enabled = true;
}
//print out the values for the trigger count and pressure of PLAYER A
// Serial.print(pressureValueA);
Serial.print(" ");
Serial.println(triggerCounterA);
// Serial.print(" | ");
// Serial.print(pressureValueB);
Serial.print(" ");
Serial.print(triggerCounterB);
//trigger counter PRESSURE condition for PLAYER B
if (pressureValueB >= 800 && pressureB_enabled == true) {
triggerCounterB = triggerCounterB + 1;
pressureB_enabled = false;
}
if (pressureValueB < 600 && prevPressureValueB >600) {
pressureB_enabled = true;
}
//print out the values for the trigger count and pressure of PLAYER B
// Serial.print(pressureValueB);
// Serial.print(" ");
// Serial.println(triggerCounterB);
//if the trigger count of PLAYER A is bigger or equal to 100, the servo B will trigger
if (triggerCounterA >= 50) {
servoB.write(180);
delay(200);
// triggerCounterB = 0;
} else
servoB.write(0);
//if the trigger count of PLAYER B is bigger or equal to 100, the servo A will trigger
if (triggerCounterB >= 50) {
servoA.write(180);
delay(200);
} else
servoA.write(0);
//reverse the values in the end of the loop
prevPressureValueA = pressureValueA;
prevPressureValueB = pressureValueB;
if (triggerCounterA >= 51) {
triggerCounterA = 0;
triggerCounterB = 0;
}
if (triggerCounterB >= 51) {
triggerCounterA = 0;
triggerCounterB = 0;
}
// //LED brightness loop
// brightnessA = map(triggerCounterA, 0, 5, 0, 1023);
// analogWrite(ledPinA, brightnessA);
//making the button restart the whole thing
int currentButtonState = digitalRead(restartButton);
if (currentButtonState == HIGH && prevButtonState == LOW) {
triggerCounterA = 0;
triggerCounterB = 0;
} prevButtonState = currentButtonState;
}
So for my midterm I decided to make a cover for one of my favorite songs, Pumped Up Kicks.
DRUM & BASS
First things first, I needed a drum or a bass sound that would imitate the original low pitch sounds of the song itself. What I built was a drum kit of three Servos, with different sizes of foam balls and spoon heads attached to each. In order to amplify the sound, I built structures around the tapping so that the sound waves would flow through one direction that was not blocked and the spoon heads also increased the noise of tapping as inside was hollow. Below is a picture of the drum kit. The one on the middle has the top opened because I constantly need to fix the Servo as it was harder to reach than other Servos and it had some initial malfunctions. Lastly, the Servos, or bass drums, were hitting through a function called Bass() which matched the actual beat of the song.
ACOUSTIC (VERSE & CHORUS)
For the sound inside, I used a Piezo Buzzer but mask-taped the buzzer so that I could amplify the sound. Most of my acoustic part was coded. Basically I had a BassHelper() function which would match with Bass() [Servo Drums] and also provide an intro to the Verse() function, which was the tone for the first verse. Then came the Chorus() function, and the song ended there. In order to run all of this together, I had to use millis() instead of delay as well as create the song from scratch by myself. Here is the whole code: It may be hard to read, but it is an easy concept: just using millis() to code every tone for each function.
For the input, I used a potentiometer to crank up, and if the meter reached at least a value of 1000, then the bass and basshelper() would start playing. Then, I had a counter to say that when basshelper() ended, add a count and if that counter matched the number with the next function then the next function would play. Like that, I was able to play the Tone functions one at a time while having the Bass() on for the whole time. Also, I could check if my potentiometer was on by connecting the value to an ledPin and seeing if the light lit up as I cranked the value past 1000. Here is the picture of the board below.
FINAL PRODUCT
Here is a video of a demonstration of the final product. There are also audio explanation of the features as well.
After our class discussion about breaking circuits with water, I stupidly thought I should make a boat just to see whether I could short circuit it. Unsurprisingly, I spent most of my time being worried about putting my boat in the water in the case that it would just sink and burn.
What:
Simply, it is my attempt at create a remote controlled boat without the remote, but instead replaced by a joystick. It detects when the boat is too close to surrounding walls and stops the motor from moving. The joystick allows the boat to go backward, forward, left and right. If there is no interaction with the joystick, then the boat does not move.
Problems I Faced:
The first problem I had was trying to make the boat float. My original plan was to use half a styrofoam ball, which would hold the DC motors. However, the motor would easily touch water and when I tried it in a tub of water, the styrofoam ball would always tip over. To fix it, I decided to put that on top of a flat styrofoam surface, but it was very awkward and nothing fit well.
I finally decided to stack multiple foam planes on top of each other to give it enough height to stay away from the water and to carve out a tip in the front to make it resemble a boat. Although this was not the best solution, it still stopped water from touching any of the circuits. Water, however, still leaked through if it stays in water for too long. I ended up duct taping the bottom, but it was definitely just a temporary solution.
I also had trouble with figuring out how to use the motor driver especially in terms of how to hook it up to the breadboard and where the wires for the DC motors should go. I also had to learn about PWMs and how they work and how to make the DC motor turn left/right using digitalWrite(). It took a bit of time to configure everything correctly, but it worked in the end. Unfortunately, I still couldn’t figure out why the motor driver isn’t giving me the full 255 value for PWM. It did not give me the max speed I needed to turn the propeller under water.
If I had more time:
The boat is, ascetically speaking, not a boat. It’s just styrofoam with a tip that makes it look like a boat. To improve on that, I would probably make the boat out of some lightweight wood that is curved correctly on the tip and sides so that it floats. There would be a hollow area for the breadboard and SparkFun circuit to sit on. It would also be covered in some way so that the water from the propeller would not accidentally carry water over the circuit.
For the joystick, I would detach it from the circuit and create a wireless board that has a joystick attached to it. By doing so, I would not have to follow the boat with the joystick and I wouldn’t have to worry about pulling out a wire by accident.
Additionally, if I have more Ultrasonic sensors, I would attach one on each side of the boat so it can detect all sides. In that way, I could either control the boat to stop completing if it’s close to anything or I could disable one of the directions to prevent the boat from hitting the side.
Last Thoughts:
I really enjoyed creating this project, although I spent a majority of the time figuring out the motor power, which led to a not-so-well designed boat. I enjoyed making the propellers and figuring out a way to extend the legs of the propeller so that it wouldn’t touch the water. It was also frustrating because I spent a long time trying to make a propeller out of a different material because I thought the plastic was too weak. However, it was because the motor wasn’t giving enough current to push the water.
My idea for my midterm changed a few times, beginning with a noise pollution detector, and then changing to a noise-canceling project to a wheel that spins depending on the pitch of the user’s voice. I just knew I wanted to do something with sounds. I wanted to discover more about sound sensors and how to link them to servos.
I couldn’t get a pitch reading using my RedBoard, so instead, I used a Teensy 3.2 board that can take in values and record the pitch and frequency of certain noises. It can also record a certain volume of sound. So I could set it to only read beyond a certain level of decibels and not below that.
For the wheel, I have 7 pieces of art, the cloud one being the most irrelevant one of the batch, and the rest are there to highlight women’s day this weekend. I also realized that the paintings have nothing to actually relate to the whole “using pitch to reveal things” concept so I decided to create a side three-part story inspired by “Goldilocks and the Three Bears”. The father, mother, and baby bear are bound to have different pitches because of very apparent reasons. The user has to use the pitch of their voice and adjust it to fit the set tone to make the servo turn to the specific bear horrified at some strange girl touching their food.
// Audio Tone Input
// Copyright 2013 Tony DiCola (tony@tonydicola.com)
// This code is part of the guide at http://learn.adafruit.com/fft-fun-with-fourier-transforms/
// Additions made by Aaron Sherwood 2019
#define ARM_MATH_CM4
#include <arm_math.h>
////////////////////////////////////////////////////////////////////////////////
// CONIFIGURATION
// These values can be changed to alter the behavior of the spectrum display.
////////////////////////////////////////////////////////////////////////////////
#include <Servo.h> // add servo library
Servo myservo; // create servo object to control a servo
int SAMPLE_RATE_HZ = 9000; // Sample rate of the audio in hertz.
const int TONE_LOWS[] = { // Lower bound (in hz) of each tone in the input sequence.
200, 400, 600
};
const int TONE_HIGHS[] = { // Upper bound (in hz) of each tone in the input sequence.
200, 400, 600
};
int TONE_ERROR_MARGIN_HZ = 0; // Allowed fudge factor above and below the bounds for each tone input.
int TONE_WINDOW_MS = 4000; // Maximum amount of milliseconds allowed to enter the full sequence.
///////////
////////// SET THRSHOLED HERE DHABIA
float TONE_THRESHOLD_DB = 15.0; // Threshold (in decibels) each tone must be above other frequencies to count.
///////////
///////////
const int FFT_SIZE = 256; // Size of the FFT. Realistically can only be at most 256
// without running out of memory for buffers and other state.
const int AUDIO_INPUT_PIN = 14; // Input ADC pin for audio data.
const int ANALOG_READ_RESOLUTION = 10; // Bits of resolution for the ADC.
const int ANALOG_READ_AVERAGING = 16; // Number of samples to average with each ADC reading.
const int POWER_LED_PIN = 13; // Output pin for power LED (pin 13 to use Teensy 3.0's onboard LED).
// any other changes to the program.
const int MAX_CHARS = 65; // Max size of the input command buffer
////////////////////////////////////////////////////////////////////////////////
// INTERNAL STATE
// These shouldn't be modified unless you know what you're doing.
////////////////////////////////////////////////////////////////////////////////
IntervalTimer samplingTimer;
float samples[FFT_SIZE * 2];
float magnitudes[FFT_SIZE];
int sampleCounter = 0;
char commandBuffer[MAX_CHARS];
int tonePosition = 0;
unsigned long toneStart = 0;
////////////////////////////////////////////////////////////////////////////////
// MAIN SKETCH FUNCTIONS
////////////////////////////////////////////////////////////////////////////////
void setup() {
// Set up serial port.
Serial.begin(38400);
myservo.attach(3); // attaches the servo on pin 9 to the servo object
// Set up ADC and audio input.
pinMode(AUDIO_INPUT_PIN, INPUT);
analogReadResolution(ANALOG_READ_RESOLUTION);
analogReadAveraging(ANALOG_READ_AVERAGING);
// Turn on the power indicator LED.
pinMode(POWER_LED_PIN, OUTPUT);
// Clear the input command buffer
memset(commandBuffer, 0, sizeof(commandBuffer));
// Begin sampling audio
samplingBegin();
}
void loop() {
// Calculate FFT if a full sample is available.
if (samplingIsDone()) {
// Run FFT on sample data.
arm_cfft_radix4_instance_f32 fft_inst;
arm_cfft_radix4_init_f32(&fft_inst, FFT_SIZE, 0, 1);
arm_cfft_radix4_f32(&fft_inst, samples);
// Calculate magnitude of complex numbers output by the FFT.
arm_cmplx_mag_f32(samples, magnitudes, FFT_SIZE);
// Detect tone sequence.
toneLoop();
// Restart audio sampling.
samplingBegin();
}
// Parse any pending commands.
parserLoop();
}
////////////////////////////////////////////////////////////////////////////////
// UTILITY FUNCTIONS
////////////////////////////////////////////////////////////////////////////////
// Compute the average magnitude of a target frequency window vs. all other frequencies.
void windowMean(float* magnitudes, int lowBin, int highBin, float* windowMean, float* otherMean) {
*windowMean = 0;
*otherMean = 0;
// Notice the first magnitude bin is skipped because it represents the
// average power of the signal.
for (int i = 1; i < FFT_SIZE / 2; ++i) {
if (i >= lowBin && i <= highBin) {
*windowMean += magnitudes[i];
}
else {
*otherMean += magnitudes[i];
}
}
*windowMean /= (highBin - lowBin) + 1;
*otherMean /= (FFT_SIZE / 2 - (highBin - lowBin));
}
// Convert a frequency to the appropriate FFT bin it will fall within.
int frequencyToBin(float frequency) {
float binFrequency = float(SAMPLE_RATE_HZ) / float(FFT_SIZE);
return int(frequency / binFrequency);
}
// Convert intensity to decibels
float intensityDb(float intensity) {
return 20.0 * log10(intensity);
}
////////////////////////////////////////////////////////////////////////////////
// SPECTRUM DISPLAY FUNCTIONS
///////////////////////////////////////////////////////////////////////////////
void toneLoop() {
for (int i = 0; i < sizeof(TONE_LOWS)/sizeof(int); i++) {
// Calculate the low and high frequency bins for the currently expected tone.
int lowBin = frequencyToBin(TONE_LOWS[i]);
int highBin = frequencyToBin(TONE_HIGHS[i]);
// Get the average intensity of frequencies inside and outside the tone window.
float window, other;
windowMean(magnitudes, lowBin, highBin, &window, &other);
window = intensityDb(window);
other = intensityDb(other);
// Check if tone intensity is above the threshold to detect a step in the sequence.
if ((window - other) >= TONE_THRESHOLD_DB) {
if (i == 0){
Serial.println("low triggered");
myservo.write(25);
toneDetected();
} else if (i == 1) {
Serial.println("mid triggered");
myservo.write(60);
toneDetected();
}
else if (i == 2) {
Serial.println("high triggered");
myservo.write(110);
toneDetected();
}
}
// // Check if the entire sequence was passed through.
// if (tonePosition >= sizeof(TONE_LOWS) / sizeof(int)) {
// toneDetected();
// tonePosition = 0;
// }
}
}
void toneDetected() {
// Flash the LEDs four times.
int pause = 150;
for (int i = 0; i < 4; i++) {
digitalWrite(POWER_LED_PIN, HIGH);
delay(pause);
digitalWrite(POWER_LED_PIN, LOW);
delay(pause);
}
}
////////////////////////////////////////////////////////////////////////////////
// SAMPLING FUNCTIONS
////////////////////////////////////////////////////////////////////////////////
void samplingCallback() {
// Read from the ADC and store the sample data
samples[sampleCounter] = (float32_t)analogRead(AUDIO_INPUT_PIN);
// Complex FFT functions require a coefficient for the imaginary part of the input.
// Since we only have real data, set this coefficient to zero.
samples[sampleCounter + 1] = 0.0;
// Update sample buffer position and stop after the buffer is filled
sampleCounter += 2;
if (sampleCounter >= FFT_SIZE * 2) {
samplingTimer.end();
}
}
void samplingBegin() {
// Reset sample buffer position and start callback at necessary rate.
sampleCounter = 0;
samplingTimer.begin(samplingCallback, 1000000 / SAMPLE_RATE_HZ);
}
boolean samplingIsDone() {
return sampleCounter >= FFT_SIZE * 2;
}
////////////////////////////////////////////////////////////////////////////////
// COMMAND PARSING FUNCTIONS
// These functions allow parsing simple commands input on the serial port.
// Commands allow reading and writing variables that control the device.
//
// All commands must end with a semicolon character.
//
// Example commands are:
// GET SAMPLE_RATE_HZ;
// - Get the sample rate of the device.
// SET SAMPLE_RATE_HZ 400;
// - Set the sample rate of the device to 400 hertz.
//
////////////////////////////////////////////////////////////////////////////////
void parserLoop() {
// Process any incoming characters from the serial port
while (Serial.available() > 0) {
char c = Serial.read();
// Add any characters that aren't the end of a command (semicolon) to the input buffer.
if (c != ';') {
c = toupper(c);
strncat(commandBuffer, &c, 1);
}
else
{
// Parse the command because an end of command token was encountered.
parseCommand(commandBuffer);
// Clear the input buffer
memset(commandBuffer, 0, sizeof(commandBuffer));
}
}
}
// Macro used in parseCommand function to simplify parsing get and set commands for a variable
#define GET_AND_SET(variableName) \
else if (strcmp(command, "GET " #variableName) == 0) { \
Serial.println(variableName); \
} \
else if (strstr(command, "SET " #variableName " ") != NULL) { \
variableName = (typeof(variableName)) atof(command+(sizeof("SET " #variableName " ")-1)); \
}
void parseCommand(char* command) {
if (strcmp(command, "GET MAGNITUDES") == 0) {
for (int i = 0; i < FFT_SIZE; ++i) {
Serial.println(magnitudes[i]);
}
}
else if (strcmp(command, "GET SAMPLES") == 0) {
for (int i = 0; i < FFT_SIZE * 2; i += 2) {
Serial.println(samples[i]);
}
}
else if (strcmp(command, "GET FFT_SIZE") == 0) {
Serial.println(FFT_SIZE);
}
GET_AND_SET(SAMPLE_RATE_HZ)
GET_AND_SET(TONE_ERROR_MARGIN_HZ)
GET_AND_SET(TONE_WINDOW_MS)
GET_AND_SET(TONE_THRESHOLD_DB)
}
