On another note, after posting, I went to the main website and they display several user examples. They promise 'text' in images but everything I saw failed miserably, so I guess we are not there yet.

For the metric to evaluate this kind of network, to me it's impossible to define it, or at least correctly, it's too subjective, if you put a metric on it, it will influence the results and therefore limit the images you can create. (not even speaking about the kind of images we want it to create)

So maybe the best we can have is a very generic metric for training, and finetuning it to some style afterward depending on your usage.

So I don't really trust 'objective ranking', but we could take all those networks, fine tune them on some well defined style and evaluate them with ground truth images when we ask to make the original piece, it would at least be some kind of proxy on the finetuning-ability of the network.

I'm skipping a lot of details such as: how to generate the prompt corresponding to the desired output, but without thinking too much about it, it could be like finding some fix-point function, and there, we can create a metric. (but this fixed function would depend on the model, and probably we can't use it to evaluate other models, and having a fixed function for all models would not compare fairly)

Note: thinking out loud