Digital artist Danil Krivoruchko taps into the deep neural algorithm with his reenvisioning of Tatiana Stolpovskaya’s NYC Flow.
The footage was captured using an iPhone 6 at 240fps and then transformed using a post-processing algorithm into a trippy world of vivid watercolours and paint.
The computer algorithm was created by researchers at Cornell University to explore how artificial neural networks simulate how neurons behave to stimulus (images, sound etc). In this instance, the algorithm reproduces the painterly style of Vincent Van Gogh and Pablo Picasso by taking a video feed and creating the stylised sequence from it. Previously, the task needed to be added manually frame by frame i.e. A Scanner Darkly.
The code was shared on GitHub to give others the freedom to experiment with the artistic style transfer for videos. Krivoruchko notes that, “you will need Ubuntu, really fast GPU and some spare time to make it work. And after that some After Effects compositing work is still needed – like basic colour correction, motion blur, removing glitches, adding details to faces etc.”
Krivoruchko also shared the source video for comparison (see below). If you have some time on your hands feel free to dive into the code; otherwise, you can download Prisma (now available on Android) and create artwork with the click of a button while patiently waiting for video to make its way to the app.