Breaking My Brain

I do not know how I ended up with a working prototype at all, if I'm honest. This is so far out of my skillset, plus add the fact that I work a fulltime job and for the majority of this contest my 5-year-old has been home. Not an excuse for how primitive the "final" project is, but I definitely could use more time to figure out how to fix all the issues.

First, I love VR. I have wanted to expand on my 2D Midjourney artwork just hanging on the wall of an oncyber gallery for a while. My initial idea was to have a virtual garden full of my art pieces and in my aesthetic to walk through.

A few images I started working with

Once I had a few images, I started looking for some sort of AI help to get 3D models. I tried quite a few (https://3d.csm.ai/ CSM and https://app.meshy.ai/discover Meshy to name a few) but ended up using https://3daistudio.com/ The devs were so nice and helpful and the 3D models I started making were so much better.

The first step I had to do was use Photoshop to remove the background of my images. The way I create images I couldn't make a clean, ready for 3D image so I had to spend time to clean them up.

Then I threw them into 3D Ai Studio.

https://video.wixstatic.com/video/8965f3_40b89e1072b546b8bffe6dcd73b104e4/360p/mp4/file.mp4

Next, I had to figure out an environment to place them in. Here is where my brain started cracking. I do not have the chops (yet) to create a full 3D environment, or room, in my aesthetic. I figured I could create a panoramic image and some how place the objects inside. But I wanted a pseudo-3D effect at the very least.

Enter https://www.blockadelabs.com/ I had tried to use Automatic1111 to create something but I couldn't get the models and all the bits to work. Blockade Labs uses SD to create an already pano-ready 360 image you can export. So I did. That's where I went back into A1111 and used something I found on GitHub to create a depth map (https://github.com/thygate/stable-diffusion-webui-depthmap-script) of the image I created in Blockade Labs.

But how to create the environment? I started the free trial for the Adobe Substance suite. They weren't doing what my brain wanted. I tried another online app https://fondant.design/ but Fondant also wouldn't work for me. So I had to suck it up and try Blender.

Enter too many YouTube videos to share. I had to figure out how to get my image and depth map into Blender and how to wrap it around the center of my viewing point. I started out trying to create a custom home screen for my Meta Quest but in the end, it needed to be compatible with any platform.

100% a mess but it was functional. But new issues. How do I share and experience it?

Google, google, google ... and I find https://home.swiftxr.io/ SwiftXR seemed to be functional so I tried it. Installed their Blender add-on, exported my scene, tested it in their app, and published to its own website.

https://weirdmomma-garden.swiftxr.app/

Once you hit that button in the corner you can actually view it. This is so last minute I haven't had the chance to test it on my own headset's browser but it works on my PC and my phone as AR. You can also scan the QR Code:

https://video.wixstatic.com/video/8965f3_2388e2e8960e4ce985153e617adcd8ec/1080p/mp4/file.mp4

Anyway, this was a crazy few weeks of trying to put all these pieces together with very little time to do it. I'm typing this while I have company in my house because someone else is watching my kid in the pool.

It's not pretty. It's basically a prototype of an idea. But I'm proud of stretching my brain and actually having a semi-finished product at the end. I spent a significant amount of time in 11 different products to figure this out.