Introduction to the Causal Image Generation Model
How to create a causal image generation model?
Problem Statement
To first build a causal image generative model, we need to have a valid problem statement, In our case, we are creating an image where two game characters with certain attributes interact with each other. Their actions and reactions are affected by each other's attributes. By using a causal model, it gives us the ability to condition and intervene in the image generation process and get a corresponding image. Sample condition queries can be like, "How would the image look had the character's action was Attack"? Conditioning on the observable classes is possible via conditioned GAN's too. So, how is it different from a causal model?
Causal Model Vs Conditioned GAN
In a conditioned GAN, we can condition on the values of the observed classes but this doesn't work in a case where the other observed classes and unobserved variables are affected by the class we conditioned on. We need a way to represent the causality in the system and how different entities in the image are getting affected by each other. The relationship between the different entities in the image is described by the causal DAG.
DAG
To build a causal model, we must construct a DAG that states the causal assumptions in the image generation process. In the below image, we see that there are 2 variables from which an image is generated. The DAG is an assumption of how the different latent and/or observed variables might interact together to produce the image.
Data
To start developing a causal image generative model, we need images. We use procedural generation to create a dataset of game characters. Procedural generation is an automated way of generating a dataset. We used a small set of game characters and merged them to create multiple images of two-game characters interacting. A Sample image can be found below.
The above image is merged by the following individual sprites.
The reactor image is just flipped to make it seem obvious that the reaction of the reactor game character is caused due to the action of the actor game character.
Pyro
Pyro is a universal probabilistic programming language (PPL) written in Python and supported by PyTorch on the backend. Pyro enables flexible and expressive deep probabilistic modeling, unifying the best of modern deep learning and Bayesian modeling. To get started,
More info about pyro can be found at
Pytorch
Pytorch is needed for building a deep neural network to learn the distribution of images. In this tutorial, we will use a Variational Auto Encoder architecture to build a generative model. A general understanding of how to build a neural network using PyTorch is needed to fully understand this tutorial. To get started,
Last updated
