Concept

The game is called The Magic Studio. The original idea was to create a highly interactive room where users could generate anything they wanted—sounds, objects, and more—giving them full creative freedom. However, due to time constraints, I scaled the project down to a painting game. In the current version, users are given a reference painting, and they must recreate it by generating objects that match the original as closely as possible. The goal is to maintain an element of creativity while also challenging players to be precise in their recreations.

How it works

The game starts with an instructions screen. It then moves the scene where the user can generate objects to match the desired painting. New objects can be generated by prompting again and the previous one gets removed automatically. Once the user is satisfied with their design, they can click on the next button to move to the next painting. Once all the painting tasks are completed, the user can restart the game.

Challenges

One of the biggest challenges and something that I am the most proud of was making the language model (LLM) generate objects as accurately as possible. Initially, I used a model with limited capabilities, which struggled to create detailed objects correctly. This led to some frustrating results, as the generated objects often didn’t match the intended design. Eventually, I switched to the Gemini model, which significantly improved performance. The new model generated objects more accurately, making the gameplay experience much smoother and more enjoyable.

Another challenge was ensuring that the interaction between the user and the game felt intuitive. Since p5.js is primarily a visual tool, integrating AI-based object generation in a way that seamlessly fit into the game mechanics took a lot of trial and error.

  // Create input field and Gemini button, but hide them until the game starts
  promptInput = createInput("Provide Object Description");
  promptInput.position(350, 100);
  promptInput.hide();
  
  button = createButton("Create your painting!");
  button.position(promptInput.x+20,promptInput.y + promptInput.height + 10);
  button.mousePressed(() => {
    let userText = promptInput.value();
    let geminiPrompt = `
You are an AI code generator working within an online p5.js editor environment. In this environment, the following conditions apply:
- The p5.js library is loaded along with its DOM addon (p5.dom), so functions like createCanvas, createButton, and createInput are available.
- A canvas of 800x600 pixels is created as part of a "dream room" simulation.
- The dream room maintains an array of dream objects. Each dream object must be defined as a JavaScript object with the following properties:
    - x: a numeric value representing the horizontal coordinate (default value: 100)
    - y: a numeric value representing the vertical coordinate (default value: 100)
    - size: a numeric value representing the object’s size (default value: 50)
    - draw: a function that uses p5.js drawing commands (for example, ellipse) to render the object at (x, y) using its size. Ensure you combine multiple shapes for a rich rendering.
    - move: a function that accepts two parameters, dx and dy, and updates the x and y coordinates respectively

Your task is to generate a valid p5.js code snippet that creates a new dream object (named \`dreamObj\`) with these properties. The object must be defined using "let".

Output Requirements:
- Your output must be a valid JSON object with exactly two keys: "code" and "description".
- The "code" key’s value must be a string containing the p5.js code that defines the dream object as described.
- The "description" key’s value must be a concise explanation of what the code does.
- Do not include any additional keys or text; output only the JSON.

Now, generate the object: 
`;
    let prompt = geminiPrompt + userText;
    generateDreamObject(prompt);
  });
  button.hide();

Future Improvements

There are the ways I can enhance the project in the future:

1. AI-Based Scoring System – One idea is to allow players to take a screenshot of their generated painting, and then use AI to analyze it and give a score based on accuracy and similarity to the reference image.
2. AI-Generated Reference Objects – Instead of only providing a static reference painting, we could allow AI to generate a new image based on the original reference. The AI could create a new rendition of the image in a slightly altered style, and players could then attempt to recreate that version using p5.js.
3. Comparing AI vs. Player Renderings – We could take screenshots of both the AI-generated image and the player-generated image, then compare them using an AI model to determine which one is a better match to the original reference. This would add another layer of challenge and gamification to the experience.
4. More Creative Freedom – To bring the project closer to the original concept, I could add more interactive elements, such as sound generation or more diverse object creation tools, allowing users to express their creativity beyond just painting.