(initial) User testing

Testing

So far, my code is able to recognize the user’s different poses, for the rock, paper, scissors game. By uploading the training data (images of different poses) to Teachable Machine, the model is now able to reliably distinguish between the 3 actions in the game. I tested this out myself, shown in the video below. I was glad to see that the image recognition model also worked well when users other than myself tested it out, even though the training data only included me.

https://youtube.com/shorts/dKZt07FMDyg?feature=share

user testing

First of all, I changed my topic again. Now, it is about creating an “immersive” experience of the nature of my country, Kazakhstan. I just miss home and got inspired after listening to some Kazakh songs. For this project, I am using a potentiometer to change the backgrounds and a ZX detection sensor to move a horse on the screen and change the colour of led lights from green to yellow, so that there could be an interactive experience with the environment.

Generally, I could finish the main functionality part of the program, and establish the communication between p5js and Arduino. However, I keep having struggles with the working of the led strip. I tried to solder it many times, but it keeps falling off (and the strip is becoming shorter and shorter). I asked for help, but although it seemed that the connection was good, it still remains unresponsive to the code. During the weekends, I worked with a soldered led strip, which is much shorter, sorry but one of the wires fell off just yesterday……So, now it does not work either, although I created a code based on it. A ZX detection sensor is another story. Theoretically, it is a really fun and useful sensor with endless possibilities… but it keeps giving me random values, which makes it complicated to control.

I asked my friend to test the program so that I could have a clear view of some new additions and changes to the program. Firstly, I need to add instructions. The second thing is to adjust my sensor. Also, she asked me if I could add more interaction with the sensor, I will think about it when it starts to normally work without creating a messy picture 😀 Yet she loved the design and the music in the background.

User Testing – Cybertruck by Zaeem&Dev

Progress

So far, we have added extra an obstacle detection system using the ultrasonic sensor and a boost in maneuverability to the car. Earlier, the car would only curve slightly when moving left and right. Now, it stays in place and adjusts directions before moving forward or backward.

Car Movement

After user testing, we have concluded that the car works as it was intended to. The ultrasonic sensor adds the automatic breaking functionality that we wanted to add initially. There are a few problems with the sensor though.

 

Furthermore, we have also figured out how to use the XBee shield to establish wireless connection between two computers. Although, we ran into a problem here too. We will need to book office hours with the Professor to figure this out.

Challenges

As observed in the video shared above, the car moves forward in intervals. This only happens when the code for the motion sensor is uploaded. We have deduced that this must be because of all the system delays associated with the working of the ultrasonic sensor. Furthermore, we cannot figure out how to connect the Xbee shield to p5js. We figured out how to connect the two components of the shield together, but we cannot properly connect it to p5js and then to Arduino. Solving this issue will mark the end of our project.

Final Project Progress – Arcane Dash

I finally decided on a theme and decided to go with an Arcane themed game. I realised that most of my work will revolve around the game design on p5.js with timing the music to the obstacles so I will have to do a lot of trial and error to make sure that is working. I managed to get the jumping function working for the character but right now I’m mainly focusing on the level design and art as that is what will take me the most time. I also selected all the songs that I will include for the levels which are from the Arcane soundtrack.

Here is some pictures of how the levels are looking so far. I took inspiration from pictures of the bridges in the tv show:

Arcane: League of Legends

The square will be replaced later with designs of the 3 main characters that the player will be able to choose from as well as select the level they want to play.

Everything is still a work in progress and theres a lot to do but I am really passionate about Arcane so I think that will motivate me a lot in order to finish the game and make it look like how I imagine it.

Final Project Progress

My final project is about creating a Visual Generative Art. I plan to use pressure sensors or 8 input touch sensors to generate art. I am going to experiment with both these sensors to decide which one to use.

For now, I have worked on the design or the art that would be generated. I am torn between two ideas: 1. Have multiple art pieces generated based on sensors OR 2. Manipulate one art piece using the sensor.

Idea 1
I have this p5.js code, which generates circles on mouse press. Circles are just a placeholder for the art, where I would put the art code, and instead of mouse press, it would be connected by Arduino.

This is an example of the art that I generated using p5js. I kinda want it to be like fireworks. The color scheme is not finalized yet.
Screen Recording 2022-04-27 at 10.16.12 PM

Idea 2
This is the code I wrote for the generative art.

And here’s a video of how it would be manipulated:
Screen Recording 2022-04-27 at 10.19.15 PM

 

 

 

 

 

Please note: Some things might get different as mentioned here as I will be working on these files.

Final project (initial) progress

I want to build off of this code that was on ml5js. I need to train the program to recognize the different hand gestures (rock, paper, scissors).

I started with the Arduino part of the ‘rock, paper, scissors’ game, by trying to build a reward system. When the project is complete, the green LED should light up whenever the user wins a round, and the red LED should light up whenever the user loses a round. And if the user wins the game (eg. wins 5 rounds before the computer) he/she receives  candy that is hidden under the box (video link below).

#include <Servo.h>

Servo servo;
int servoPos = 100;
int gameOver = 1 ;

int gledPin = 2;
int rledPin = 4;
int winloss = 1;

void setup() {
  servo.attach(9);  
  pinMode(gledPin, OUTPUT);
  pinMode(rledPin, OUTPUT);
  Serial.begin(9600);
}

void loop() {
  if (gameOver == 1){
    servoPos = 20;
    servo.write(servoPos);
    }
   servoPos = 100;
   servo.write(servoPos);

  if (winloss == 0){
    digitalWrite(rledPin, HIGH);
    }
   else{
    digitalWrite(gledPin, HIGH);}

  delay(2000);
  
}

https://youtube.com/shorts/Zce_tFlzj7E?feature=share

final project progress: not 90%

So, according to the ambitions that I had in the previous post, I wanted to create a 3d interface controller to play a snake game in 3d format. I built a box, it worked, when I placed a hand inside of it. Also, apparently, the project, which I wanted to implement in my project, was created 10 years ago and it is not relevant anymore (oops). At least, I learned soldering and could do a 220k resistor, so if you need one, I have it…

However, the given values were inaccurate and I could not check if the p5js/or any 3d processing code would work with it. Had some trouble planning the project accordingly.

After talking to Professor Aaron, he suggested using a ZX distance and gesture sensor instead. I might use two sensors to provide a 3d effect, but it does not work anyways. So I hope that even one sensor could work for now.

In my p5js code, I just re-created a simple snake game, which is controlled by the keys for now.

Sorry, my project has not progressed much, since my idea was changing the whole week.

Nicholas and Shama Final Project Progress

The project has been moving along well, with progress being done on all fronts. The simple Node backend has been complete with the Alpaca trading API, and looks as follows:

import express from 'express'
import cors from 'cors'
const Alpaca = require('@alpacahq/alpaca-trade-api')
const fs = require('fs')
require('dotenv').config()
const app = express()
const port = 8000

const stocklist = [];
let alpaca;
let startval;
if(process.env.ENV==='DEVELOPMENT'){
    alpaca = new Alpaca({
        keyId: process.env.TEST_ALPACA_KEY_ID,
        secretKey: process.env.TEST_ALPACA_SECRET_KEY,
        paper: true,
    })
    startval = 100000;
}else if(process.env.ENV==='PRODUCTION'){
    alpaca = new Alpaca({
        keyId: process.env.ALPACA_KEY_ID,
        secretKey: process.env.ALPACA_SECRET_KEY,
        paper: false,
    })
}else{
    throw new Error("Error: Environment Not Properly Selected")
}

//read stocklist 
fs.readFile('stocklist.txt', function(err, data) {
    if(err) throw err;
    var array = data.toString().split("\n");
    for(const stock of array) {
        const [symbol, name] = stock.split(' ')
        if(symbol&&name){
            let currstock ={
                symbol,
                name:name.replace("\"","")
            }
            stocklist.push(currstock)
        }
    }
});

app.use(express.json())
app.use(cors())


app.post('/buy_order', async(req, res)=>{
    try{
        const {ticker} = req.body;
        const order = await alpaca.createOrder({
            symbol:ticker,
            qty:1,
            side: 'buy',
            type:'market',
            time_in_force:'day'
        })
        res.json({order})
    }catch(e){
        console.log(e)
        res.status(400).send('err')
    }
})


app.post('/sell_all', async(req, res)=>{
    try{
        await alpaca.closeAllPositions();
        res.json({'success':'success'})
    }catch(e){
        console.log(e)
    }
})

app.get('/timeline', async(req, res)=>{
    try{
        const history = await alpaca.getPortfolioHistory({
            period: '1D',
            timeframe:'1Min'
          })
        const last_elem = await history.equity.findIndex(e=>e===null);
        const cut_history = {
            equity:history.equity.slice(0,last_elem),
            profit_loss: history.profit_loss[last_elem-1],
            profit_loss_pct: history.profit_loss_pct[last_elem-1]
        };
        res.json({...cut_history})
    }catch(e){
        console.log(e)
        res.send('err')
    }
})

app.get('/positions', async(req, res)=>{
    try{
        const positions = await alpaca.getPositions();
        res.json(positions)
    }catch(e){
        res.send('err')
    }
})

app.get('/randtickers', async(req,res)=>{
    try{
        const rand1 = Math.floor(stocklist.length * Math.random());
        let rand2 = Math.floor(stocklist.length * Math.random());
        while(rand2===rand1){
            rand2 = Math.floor(stocklist.length * Math.random());
        }
        res.json({tickers: [stocklist[rand1],stocklist[rand2]]})
    }catch(e){
        res.status(400).send('err')
    }
})


app.listen(port, () => {
  console.log(`Example app listening on port ${port}`)
})

The data for the stocks is manipulated and returned from the server between 5:30pm-12:00am GST.

On the p5js end, we have created the functions to get, post, and display data. It looks like this when the stock market is open:

let equity;
let profit_loss;
let profit_loss_pct;
let tickers = [{symbol:"",name:""},{symbol:"",name:""}];
const FPS = 60;
function setup() {
  updateEquity()
     .then(()=>getTickers())
     .then(data=>tickers=data.tickers)
  createCanvas(640,320)
}

function draw() {
  background(220);
  if(frameCount%FPS*60===0){
    updateEquity();
  }
  if(frameCount%FPS===0){
    getTickers().then(data=>{tickers=data.tickers;});
  }
  
  push();
  fill(255,0,0)
  rect(0,20+height/4,1*width/3,3*height/4-20)
  fill(100,100,100)
  rect(width/3,20+height/4,1*width/3,3*height/4-20)
  fill(0,0,255)
  rect(2*width/3,20+height/4,1*width/3,3*height/4-20)
  pop();
  
  push();
  fill(255)
  noStroke();
  textSize(24)
  textAlign(CENTER)
  text(tickers[0].symbol, width/6, 2*height/4)
  text(tickers[0].name, width/6, 3*height/4)
  text("SKIP", 2*width/4, 11*height/16)
  fill(255)
  text(tickers[1].symbol, 5*width/6, 2*height/4)
  text(tickers[1].name, 5*width/6, 3*height/4)
  pop();
  plotTimeline(equity);
  
  push();
  textSize(18);
  fill(profit_loss<0?255:0,profit_loss>0?255:0,0)
  text(`${profit_loss?'$'+profit_loss.toString():""}`, 5*width/6+10, height/7)
  text(`${profit_loss_pct?(profit_loss_pct*100).toString().slice(0,7)+"%":""}`, 5*width/6+10, height/7+20)
  pop();
  
}

async function makeOrder(ticker){
  try{
    const data = await fetch('http://localhost:8000/buy_order',{
      method:"POST",
      headers : { 
        'Content-Type': 'application/json',
        'Accept': 'application/json'
       },
      body:JSON.stringify({ticker})
    })
    jsonData = await data.json();
    return jsonData;
  }catch(e){
    console.log(e)
  }
}

async function sellAll(){
  try{
    await fetch('http://localhost:8000/sell_all',{
        method:"POST",
        headers : { 
          'Content-Type': 'application/json',
          'Accept': 'application/json'
         },
    })
    return 'success'
  }catch(e){
    console.log(e)
  }
}

async function getTimeline(){
  try{
    const data = await fetch('http://localhost:8000/timeline')
    const jsonData = await data.json()
    return jsonData;
  }catch(e){
    console.log(e)
  }
}

async function getTickers(){
  try{
    const data = await fetch('http://localhost:8000/randtickers')
    return data.json()
  }catch(e){
    console.log(e)
  }
}

function plotTimeline(equity){
  if(!equity){
    return;
  }
  const Emin = Math.min(...equity);
  const Emax = Math.max(...equity);
  const maxGraphHeight = height/4;
  const plotEquity = [];
  push();
  noFill();
  stroke(0)
  beginShape();
  for(let i =0; i<equity.length; i++){
    vertex(map(i,0,equity.length, 10, 5*width/6),map(equity[i], Emin, Emax, 10, maxGraphHeight))
  }
  endShape();
  pop();
}

function updateEquity(){
  getTimeline().then(resp=>{
      equity = resp.equity;
      profit_loss = resp.profit_loss
      profit_loss_pct = resp.profit_loss_pct
    })
}

 

The graph at the top shows the progress made, and the numbers on the right show current profits. The 3 bars in the middle display the current options.

We have been also working on the physical computing end, where we’re testing implementations for our beloved airhorn.

To wrap up our project, we have to hook up the hardware and software end, which is the “hardest” 10%, as the professor mentioned in class

Final Project Progress

Before starting the Arduino side of things, I wanted to finish the p5js side of my project. So far, I’ve finished all the main parts of the game like the transitions and different buttons. I’ve managed to make the game go full screen when the user clicks the starting screen.

All that is left for me to add in the p5js code is the part that is related to Arduino and small parts like the sounds and other small visual effects which I haven’t decided on yet.

Sounds

I will add the following sounds:

  • Background music for the entire duration of the game including menu and score screen
  • Crowd sounds to play during the main part of the game
  • Excited crowd sounds when the user scores

P5js sketch

Arduino

As for the Arduino side of my project, I have decided that I will use an infrared emitter and receiver under the hoop on either side so that when the ball falls through the hoop, it will disrupt the signal.

I will also build a boxy frame in which I will have my hoop on a slanted surface so that the ball rolls down to the user after they shoot it.

Dance Breaker: The Beginning 💃🏼 🤸

Planning Stage

 

 

 

 

Before we began to work, we needed to sort out a grand variety of details. Which materials were we going use?  How would we use them? When would we meet and build our project? Given that Lily is a senior and has her capstone paper due in less than 2 days, and her presentation a few days before the final submission, our team needed to make sure we were properly managing our time and planning realistic deadlines according to our schedule. We wrote a detailed outline with parts of our project and internal deadlines that we plan to keep in order to make our project the best final project. During this initial planning meeting, we also gave our project the name Dance Breaker. It is an ode to breakdancing AND hints to how the game will work: the music will stop (break) and you need to press the sensors to un-break it. Get it?

SEE OUR PLAN HERE

We also created our internal deadline system into google calendar and set working hours. Once we had our work cut out for us, we began the execution of our plan.

Building Progress

Game Mechanics:

      •  We worked mostly on the P5.js code for the songs and serial connection. To do this he constructed a board with smaller sensors and pieces from a previous project that would allow for testing on a smaller scale. He also began to work with computer vision. The current p5.js sketch loads pixels and has a light threshold akin to previous class exercises. It took a long time to figure out how to mirror the image in order to have a comprehensive experience. Furthermore, he built conditional statements so that all sensors need to be pressed in order for a song to play. The arduino code should not require any changes after today except for the addition of the reward system.
      • Next steps for the game mechanics are:
          • Implement a functioning random timer (which was already created but not aligned with the music) to stop the song randomly and resume the game.
          • Randomize the sensors that need to be pressed in order for the music to continue.
          • Add a design to the computer vision beyond a threshold.

 

void setup() {
  Serial.begin(9600);
 
}
void loop() {
  int oneValue = analogRead(A0);
   
  int twoValue = analogRead(A1);

  int threeValue = analogRead(A2);
 
  int fourValue = analogRead(A3);
   
  int fiveValue = analogRead(A4);


        
    int sensorValue = oneValue;
    Serial.print(sensorValue);
    Serial.print(",");
    sensorValue = twoValue;
    Serial.print(sensorValue);
    Serial.print(",");
    sensorValue = threeValue;
    Serial.print(sensorValue);
    Serial.print(",");
    sensorValue = fourValue;
    Serial.print(sensorValue);
    Serial.print(",");
    sensorValue = fiveValue;
    Serial.print(sensorValue);
    Serial.println();
    delay(1);     

  }

Physical Component: Dance Mat and Projector

      • We booked the necessary materials (a vertical projector, a camera) through the portal.
      • We also picked up the necessary sensors and acrylic to build our dance mats. We ran some simple tests with the materials in order to make sure that they work as imagined, and have begun to build the illustrator sketches for the laser cutting.
      • Next steps for the physical components are:
        • Finish the illustrator design and laser cut the materials
        • Print Stickers and Logo
        • Cut and assemble the base.