Disclaimer: I was part of the freelance people working for GROSSE8 on this one.
Showcase
Check out Elastiques documentation of the project for more videos and details. (I was not part of the on location team, therefore I don’t have many final images/videos I own the rights to)
Description
First things first. I have no hat in the ring when it comes to NFTs. I haven’t really had any points of contact with that world either. I mainly joined the project because I’ve had many great projects together with GROSSE8. As a bonus, I was really excited about the technical challenge that was asked. We had to build a system, that is able to connect to the users meta mask wallet, get their Apes and display them on our touch panel. Then the user can select one to sticker it on the car. After an intro animation the Ape lands on a specific spot, where it stays over the duration of the whole event. The more users sticker their Ape on the car, the more crowded it gets.
Questions
Quickly many questions arose. Where does each Ape get placed? Can the grid change dynamically over time? Is there any algorithmic way or do we hardcode it? How do we select the correct Ape from the user? How do we even do this at 30-60fps while projecting over 8000x12000 pixels across the surface of the car?
Answers
The Grid
As recalculating the grid in real time would lead to a lot of randomness or computing power (looking at you dart throwing technique) or tweaking the parameters very intensely, we resorted to hardcoding the grid. Building a Houdini setup where you can prototype and place every ape in real time, one by one. While this is very manual and click intensive, it is also very art directable and precise. If the client wants to move on sticker over because the face is masked by an other sticker, that’s no problem.
The correct Ape
Simply put, the meta mask wallet sends a unique user ID and we matched that to our data base of Apes. For that we just needed thousands of apes in our repo. Talk about Git LFS. A more complicated part was to create a save game of the state of the installation without running into infinite loops. That ensured the Apes always holding the correct order and not glitching on top or under each other.
Performance
Kind of disappointing, but it was a mix of a bigger GPU and many hours of performance optimization. Nowadays hardware is very powerful and many people don’t realize how wasteful they manage their resources. Once you hit a bottleneck, that ship has sailed for the moment and you often have to work backwards to find what’s causing it. Often it’s a complex combination of many small things. A lot of people don’t know it but the Probe is an essential palette tool. It shows CPU and GPU load and draw times and helps immensely to find the root cause.
Learnings
While working procedural and dynamic is a blessing in many situations, it can feel like diminishing returns depending on the project. While learning Houdini I wanted to proceduralize everything because I was coming from a very manual workflow. But afters years of having this routine, one can get blind to the upsides of “just doing it by hand”. My main takeaway was to be dynamic and choose the approach depending on the needs and circumstances of the situation. Other than that I learned a lot of maths to calculate and move points in real time and developed a very deep understanding of the 3d Texture node in TouchDesigner. Huge shout out to Markus Heckmann for taking the time and explaining the inner workings and order of operations of that node on project start up.
Nice Detail
The projectors had over 6000 ANSi lumen each. I got told that one is able to see the projection on the car in bright daylight, without any issues. It’s insane how much power these projectors must have. We had to hardcode how many Apes can be placed on the car. Therefore we calculated the average time the experience runs per person. We factored in how long the exhibition is open and added ±40 slots as a safety buffer. In the end we hardcoded 250 slots (I placed every point by hand). After closing doors of the exhibition the project lead texted me with relief that we used up 238 of the slots and had a queue that went over half of the venue.