Talking to an AI Version of Me in 3D

I'm excited to announce a new feature on my website: you can now chat with an AI version of me in a 3D environment. It's like stepping into The Matrix, but instead of fighting agents, you're having a conversation with a digital clone of yours truly. Go to albertotaiuti.com and give it a try!

What?

When you visit the landing page of my website, you'll notice a new option to enter "The Construct" (another Matrix reference ;)). Click on it, and you'll be transported into a 3D scene where you can interact with an AI version of me powered by a Large Language Model (LLM).

I thought that it would be fun to combine two of my favorite things: 3D and LLMs. So over the last weekend I put this together, and it was both easy and hard at the same time!

Learnings

Creating 3D on the web has become very easy thanks to frameworks like react-three-fiber, and using LLMs as helpers. However, finding the right 3D assets, and animating them, remains challenging. I'm baffled by how clunky Sketchfab is to this day, and let's not even talk about Turbosquid. Then if you want to even slightly modify them, you're pretty much out of luck: learning Blender takes too much time.

Someone will come around and make this part as easy as drag and drop, but it hasn't happened yet.

The Tech Behind It

This new feature is built using a combination of technologies:

  • Three.js and React Three Fiber for the 3D rendering
  • OpenAI's GPT model for the conversational AI
  • Blender for the 3D assets
  • Mixamo for the animations

One of the coolest parts is the transition animation between the 2D website and the 3D scene. I think it really adds to the immersive feel of the experience. I also added a MatrixRain component in the code - it's a fun little easter egg that adds to the sci-fi vibe, which keeps you entertained while the assets are loading.

Watch It Being Built

For those interested in the development process, I recorded myself coding this feature. You can watch the video on YouTube here. It's a great way to see how all the pieces come together, from setting up the 3D environment to integrating the AI model.

Try It Out!

I had a blast building this over a weekend, and I hope it'll inspire others to add more 3D to their websites. Head over to my website, step into The Construct, and let me know what you think. Who knows, you might even learn something new about me that the real Alberto doesn't know!

In this world, there is no spoon. And apparently, there's an AI version of me. Enjoy!