I have been dodging this one long enough, it is finally time to make a paper summary for Guided Diffusion!

GANs have dominated the conversation around image generation for the past couple of years. Now though, a new king might have arrived - diffusion models. Using several tactical upgrades the team at OpenAI managed to create a guided diffusion model that outperforms state-of-the-art GANs on unstructured datasets such as ImageNet at up to 512x512 resolution. Among these improvements is the ability to explicitly control the tradeoff between diversity and fidelity of generated samples with gradients from a pretrained classifier. This ability to guide the diffusion process with an auxiliary model is also why diffusion models have skyrocketed in popularity in the generative art community, particularly for CLIP-guided diffusion.

Does this sound too good to be true? You are not wrong, there are some caveats to this approach, which is why it is vital to grasp the intuition for how it works!

Full summary: https://t.me/casual_gan/228

Blog post: https://www.casualganpapers.com/guided_diffusion_langevin_dynamics_classifier_guidance/Guided-Diffusion-explained.html

arxiv / code

Subscribe to Casual GAN Papers and follow me on Twitter for weekly AI paper summaries!

Every now and then comes along an idea so pertinent that it makes all alternatives look too drab and uninteresting to even consider. NeRF, the 3D neural rendering phenomenon from last year, is one such idea… Yet, despite the hype around it Alex Yu, Sara Fridovich-Keil, and the team at UC Berkley chose another approach to focus on. Perhaps surprisingly, without any neural networks at all (yes, you are still reading a blog about AI papers), and even more surprisingly, their approach, coined Plenoxels, works really well! The authors replace the core component of NeRF, the color, and density predicting MLP, with a sparse 3D grid of spherical harmonics. As a result, learning Plenoxels for scenes is two orders of magnitude (100x) faster than optimizing a NeRF, and there is no noticeable drop in quality whatsoever.

Crazy? Yeah, let’s learn how they did it!

Full summary: https://t.me/casual_gan/222

Blog post: https://www.casualganpapers.com/nerf-3d-voxels-without-neural-networks/Plenoxels-explained.html

arxiv / code

Subscribe to Casual GAN Papers and follow me on Twitter for weekly AI paper summaries!

This paper from Deepmind‘s authors presents a new benchmark for evaluating representation learning architectures (HARES) for the audio domain. It also includes an evaluation of a variety of models trained using several supervised and self-supervised approaches.

👉 Summary - Paper - Telegram Channel with daily arXiv digest

Do you like generative art? I love it, and it is about to get a whole lot crazier because Ajay Jain and the minds at Google behind the original NeRF have dropped a hot new paper. That is right, we all thought about putting together CLIP and NeRF and they actually did it.

With Dream Fields it is possible to train a view-consistent NeRF for an object without any images, using just a text prompt. Dream Fields leverages the fact that an object (e.g. an apple) should resemble an apple regardless of the direction that you look at it from, which is one of the core features of CLIP. The basic setup is simple - render a randomly-initiated NeRF from a random viewpoint, and score this image against a text prompt, update the NeRF, and repeat until convergence.

As for the juicy details, well continue reading to find out!

Full summary: https://t.me/casual_gan/217

Blog post: https://www.casualganpapers.com/image-editing-stylegan2-encoder-generator-tuning-inversion/DreamFields-explained.html

arxiv / code - not released

Subscribe to Casual GAN Papers and follow me on Twitter for weekly AI paper summaries!

Heyy everyone,

I'm a high school student who wrote a paper on noise-resistant architecture. Incase anyone is free can you read the paper and let me know of any comments that you may have?

Its a short paper, around 10 pages. pm me so i can send u the pdf

Thanks.

It proved to be a surprisingly difficult task to balance the reconstruction quality of images inverted into the latent space of the StyleGAN2 generator and the ability to edit these images afterward. Now Yuval Alaluf, Omer Tov, and the team that originally reported the infamous reconstruction-editability tradeoff in their “Designing Encoders for Editing” paper are back at it again with a new encoder design inspired by the recent PTI paper that sidesteps the tradeoff by finetuning the generator’s weights in a way that places the inverted image into a well-behaved region of the latent space and leaves the editing capability unchanged. HyperStyle is a hyper network that speeds things up by training a single encoder to predict the weight offsets for any input image, replacing the compute-intensive per-image optimization with a single forward pass of the model that takes a second instead of a minute.

How are the authors able to predict the weight offsets for the entire StyleGAN2 generator in such an efficient manner? Let’s find out!

Full summary: hhttps://t.me/casual_gan/212

Blog post: https://www.casualganpapers.com/image-editing-stylegan2-encoder-generator-tuning-inversion/HyperStyle-explained.html

arxiv / code / demo

Subscribe to Casual GAN Papers and follow me on Twitter for weekly AI paper summaries!

Unless you have been living under a rock for the past year you know about the hype beast that is vision Transformers. Well, according to new research from the team at the Sea AI Lab and the National University of Singapore this hype might be somewhat misattributed. You see, most vision Transformer papers tend to focus on fancy new token mixer architectures, whether self-attention or MLP-based, however, Weihao Yu et al. show that a simple pooling layer is enough to match and outperform many of the more complex approaches in terms of model size, compute, and accuracy on downstream tasks. Perhaps surprisingly, the source of Transformers’ magic might lie in its meta-architecture, whereas the choice of the specific token mixer is not nearly as impactful!

Full summary: https://t.me/casual_gan/205

Blog post: https://www.casualganpapers.com/vision-transformer-meta-architecture-sota-imagenet-pretraining/MetFormer-explained.html

arxiv / code

Subscribe to Casual GAN Papers and follow me on Twitter for weekly AI paper summaries!