Passing by Interactive Media Lab, I always noticed those TVs that used webcams to track our movement and demonstrate interesting visualization using predefined symbols. I always wondered how it actually worked. So for this data visulization assignment I wanted to recreate it and perhaps add some more adjustments. I followed Coding Train’s coding challenge tutorial regarding ASCII Text images where most of my code comes from.
The basic mechanism behind this program is to map the pixel brightness values which are calculated by dividing their average RGB values and mapping it to characters from density character strings. In this case, I am using ‘ÑYUAD876543210?!abc;:+=-,._’; since initial symbols take up more density and show contrast better but it can realistically be anything. Super bright parts of the image have no space value at all. Additionally, there is no Canvas, and this is direct html implementation with little bit of CSS. I also added a slider for density value which adjusts number of spaces that are added to the string, which acts similiar to how contrast slider would act in photo editing app. If uyou think about it, thats actually whats happening. There are more spaces for wider value of brighter pixels. There is also a Toggle Color button on top left, which assigns the symbols colors based on initial pixel value. This is done per frame basis.
To ensure that you see full working project, make sure to follow P5 link, since this website does not correctly display it and also you need camera permissions anyway since it takes the video of your webcam.
Here is perhaps the most important code out of entire project:
//loop to iterate over pixels
for (let j = 0; j < video.height; j++) { //iterates over rows (height)
for (let i = 0; i < video.width; i++) { //iterates over columns (width)
const pixelIndex = (i + j * video.width) * 4;
//calculates index of pixel in videos pixel array based on its x and y cordinates (i and j). basically vertical and horizontal displacement. Video width indicates how many pixels are in each row. r, g b, a so we multiply by 4 since pixel takes up 4 spaces in array
const r = video.pixels[pixelIndex];
const g = video.pixels[pixelIndex + 1];
const b = video.pixels[pixelIndex + 2];
const avg = (r + g + b) / 3;
const len = adjustedDensity.length;//to later map brightness value
const charIndex = floor(map(avg, 0, 255, 0, len));
//maps avg value from 0 to 255 to 0 to len. floor is used for rounding
const c = adjustedDensity.charAt(charIndex);
//brighter pixel = higher char index lower density
const charColor = `rgb(${r},${g},${b})`;
//its a template literal, javascript uses it for embedding expressions within the string
if (c == " ") {
asciiImage += " ";
} else if (check == true) {
asciiImage += `<span style="color:${charColor};">${c}</span>`;
} else {
asciiImage += c; //adds c to our image
}
//span element is inline container in hztml used ot apply styles without line break
//if our c is empty by mapping, correspond that to true space (html does not conventionally display empty spaces)
}
asciiImage += '<br/>'; //line break to start fresh rows
}
asciiDiv.html(asciiImage); //sets html content of ascidiv to our finalized asciimage, by continously drawing we update div content and thus motion is live
}
The code is fully commented but the general idea is to find out r g b values for individual pixels. After that, we find average value which corresponds to our brightness. We map our string index to brightness values. Therefore we have a constant C which is a character for every-frame and we add it to our frame. If color is on, then its color is modified.
In the future, I could work on optimizing since. Refreshing html elements so many times, especially when color is applied is very taxing on CPU and the process becomes laggy. Lowering frame-rate does not help much. Additionally, I could add more adjustmenets and variables, such as video size, font options and more. The latter is especially interesting since the font I am using has same horizontal length. Variable font would be harder to implement. Overall I am very glad with how this project turned out and would like to work on it more in the future.