Much has been said about the potential of neural networks in the modification and creation of images; a field with much potential, as demonstrated by projects capable of creating photos of non-existent people, only learning from huge databases of images. But although this may seem impressive, the truth is that the potential is even greater, as Nvidia has shown.
Nvidia is one of the companies that is betting more for the diverse utilities of the automatic learning and Artificial Intelligence; it offers GPUs very capable in the necessary calculations to make these projects reality. To prove it, they have presented GauGAN, a program capable of creating images from simple sketches drawn with Paint by the user.
Using this system, GauGAN is able to turn any drawing into a realistic photograph; all parts of the drawing are represented with textures and objects as they would be in real life. Green blocks become big grasses, gray lines become clouds, and a blue line becomes a waterfall in the middle of the mountain. Everything translates into an image very similar to what we would have in our head when we made the drawing.
The truth is that GauGAN does something of a trick; although it looks like a program similar to Paint, when we choose the colors we are indicating the type of material that we want to use. The lightest blue is the sky, the grey is the clouds, and the different shades of green are trees, plants, bushes, etc. The program “only” has to change those parts for the appropriate textures; the real challenge is to integrate them in a natural way. In the above example of the waterfall, the system only knows that it is a line of “water” in the middle of a portion of “rock”; it is its job to interpret that and come to the conclusion that it is a waterfall that falls into the downstream river.