is a software engineering manager at Spotify in NYC.
FloraJS + Audio
At JSConf EU 2013, I gave a talk about mapping natural input like audio to forces in a natural simulation. Combining the HTML5 getUserMedia API, with FloraJS, I demoed two audio apps that attracted particles based on a tone frequency I generated from an iPhone app.
The HTML5 getUserMedia API standard is not complete. But the latest Google Chrome should run the demos just fine.
Flora tone pair - At the prompt, allow the app to use your microphone. Next, use a tone generator to attract the agents. Or if you don’t have a tone generator, just whistle.
Flora tone circle - At the prompt, allow the app to use your microphone. Next, use a tone generator to cycle through different frequencies and guide the agents.
If you want to check out the code, you can find it in the GitHub repo.
Rules are forces like gravity, friction, and drag. The world is inhabited by autonomous agents that carry properties like mass, width and height. Agents also have behaviors like steer, flock, and attract.
When you combine enough rules, the agents in your system behave is if the system is ‘natural’.
Agents can carry sensors tuned to specific stimuli. They can also attach behaviors to these sensors. For example, if a sensor detects heat, the agent can choose to turn away.
Sensors can be combined for an especially realistic simulation.
When we combine sensors, at some point we stop seeing the individual rules and accept that we’re witnessing something intelligent that could inhabit the real world. This is the premise behind Braitenberg Vehicles, imaginary autonomous vehicles described in Valentino Braitenberg’s book, Vehicles: Experiments in Synthetic Psychology.
Please checkout the repo's gh-pages for more examples or visit the GitHub repo for a look at the code.
The article is super detailed. Maybe a bit too detailed. If you want to learn all the steps, start at the beginning. If you want to just get the code, skip to the end. Or you could go straight to the GitHub repo.
If you decide to start from the beginning, you’ll cover the thinking behind FloraJS, a larger framework for natural animation.
ittybitty8bit was my first attempt at a large project. The initial idea was to provide a platform for people to send each other 8-bit art via SMS.
The SMS contained a message and a link to an 8-bit scene that featured one of eight cats repeating the SMS message.
After I got it up and running, it was a fun toy to play with. I learned about SMS Gateways and bought a bunch of phones for testing. But eventually, I needed to pay rent and had to take a full-time job which meant less time for ittybitty8bit.
I decided to take down the SMS functionality and just leave up some of the artwork.
Even now and then I think I’ll turn it into something else. Meow.
Do I look familiar?
Once calculated, the script renders the pixelated image with each block positioned relative to the mouse position. Moving the mouse renders a parallax effect.
Clockstagram was my first iOS app. It displayed the current time using popular Instagram photos.
It’s not in the App Store anymore. But early on, it was fun watching the downloads tick up. Every now and then a blog would link to the app in a different country. Out of nowhere I would see spikes in Turkey or the Philippines.
In the end the app had over 29K downloads with the US, Saudi Arabia and Thailand as the top three countries.
Clockstagram was also a web app. You could go to clockstagram.com with a desktop or mobile browser and get the same functionality.
Not long after Instagram first launched, they issued a challenge similar to DARPA’s Shredder Challenge... create an algorithm that reconstructs a "shredded" photograph into its original form.
At the time, I was experimenting with Processing. Since challenge was essentially a visual problem, Processing seemed like a good tool for the problem.
I was also a huge Instagram fan and entertained the idea that I could possibly get an interview if I came up with an interesting solution.
Eventually, I finished and posted a solution along with a hundred or so eager hackers. A few days later, I got a call from Kevin Systrom but didn’t get the job.
A week later Facebook bought Instagram.
I included additional images in my solution to prove my algorithm was flexible.
Just showing off with this one.
The original shredded image from Instagram.
Once I had thisisnotflash.com up and running, I did the same thing for the iPad.
I remember the rumors about Apple’s upcoming tablet in 2009. Everyone was wondering what Apple would call it.
When “iPad” was announced, I had a few domain names queued up in my registrar’s shopping cart thinking I would resell them. iPadPixels.com was one of them.
But instead of dumping it, I’m glad I put it to good use. Long live divs.
This is not Flash
Remember the pissy fight between Adobe and Apple over the restrictions on Flash playback on mobile iOS? Back when I was building Big Block Framework, I realized I could easily create animation and interactivity the mobile Safari web browser.
“Ah hah.. who needs Flash. I have divs!" I thought. I built thisisnotflash.com as an demo.
Looking back, it wasn’t so much about doing something Flash-like as doing something creative within a set of strict constraints.
Every now and then I run the demo on my iPhone and test if it still works. So far so good.
Big Block Framework
For a long while I was obsessed with creating 8-bit style graphics and animation in desktop and mobile web browsers. It seemed reasonable a modern web browser on an average personal computer could render a grid of solid color blocks as 8-bit art at a comfortable 60fps.
Imagine a grid of 8px by 8px divs. Animating those divs requires changing their properties. I realized it was much faster to change a div’s class name than individual properties.
Big Block Framework pre-computed all the possible locations in the grid as well as all the possible colors. It then created CSS style rules for each. At 640 x 480, that’s close to 5,000 rules. There lies the enormity of the hack.
But it worked.
I even created a typeface, Werds Fer Nerds, that was just a bunch of style rules for each letter.
This was my first real attempt build a creative tool for other people. I'm not sure anyone actually used it. But it was just the beginning of several JS-based solutions to deliver 8-bit art in a browser.