Building the Image Generation Directed Graph

What is the DAG for our use case ? What does it mean ?

In the previous section, we took a look at how to probabilistic-ally reason with causal DAGs and inference algorithms. In this section, we take a look at our image generation use case.

Game character Image Generation DAG

Each image is a situation with 2 characters where there is an instigator also known as the Actor and a character reacting to the instigation named the Reactor. All the nodes in the DAG are Discrete random variables except the image.

Node Abbr	Node	Values	Explanation
AC	Actor Character	Satyr, Golem	The Actor can be any character
RC	Reactor Character	Satyr, Golem	The Reactor can be any character
AT	Actor Type	Type1, Type2, Type3	There are 3 types for each character
RT	Reactor Type	Type1, Type2, Type3	There are 3 types for each character
AD	Actor Defense	Low, High	Defense of the actor. Unobserved
AS	Actor Strength	Low, High	Strength of the actor. Unobserved
AA	Actor Attack	Low, High	Attacking capabilities of the actor. Unobserved.
RD	Reactor Defense	Low, High	Defense of the reactor. Unobserved
RS	Reactor Strength	Low, High	Strength of the reactor. Unobserved.
RA	Reactor Attack	Low, High	Attacking capabilities of the reactor. Unobserved
AACT	Actor's Action	Attack, Walk, Taunt	The Action of the actor.
RRCT	Reactor's Reaction	Die, Hurt, Idle, Attack	The Reaction of the reactor
IMG	Image	NA	Generated Image

Sample Image

The joint probability can be written by factorizing the DAG.

P(AC,RC,AT,RT,AD,AS,AA,RD,RS,RA,AACT, RRCT, IMG) = P(AC) * P(RC) * P(AT|AC) * P(RT|RC) * P(AD | AC, AT) * P(AS|AC,AT) * P(AA|AC,AT) * P(RD|RC,RT) * P(RS|RC,RT) * P(RA|RC,RT) * P(AACT| AD,AS,AA) * P(RRCT|RA,RS,RD,AACT) * P(IMG|AACT,RRCT,AT,RT,AC,RC)

P(IMG|AACT,RRCT,AT,RT,AC,RC) is learnt by a neural network. The decoder of the learned network generates an image

Like in the previous section, we need to specify certain prior conditional probabilities to make the sampling easy for us. Instead of computing intervention/condition distributions using inference algorithms at run time, we use probability propagation methods to compute conditional probabilities for the queries that we are interested in. This is done to simplify the problem and avoid unnecessary troubles during inference.

Need to figure out a way to do inference at run time, instead of pre-computing them.

The next section deals with how to compute these probabilities using R packages gRain and bnlearn.