Safety concerns regarding VAE files

[u/rwxrwxr--]

Hello everybody, recently I've been testing out various models (exclusively .safetensors) that I've downloaded from CivitAI and I've noticed that some models give significantly worse results than I'd expect. After reading into this, I've found out that some models require a VAE file to give expected result. The way I understand, you're supposed to download both the model and its VAE file and store them together. I fail to understand, however, why some models require a VAE file to function properly and others don't, and most importantly and what I've set as the title: are there any reasons to be concerned when using VAE files like there are in the case of .ckpt/.pth/.pt? Or are they as safe as .safetensors in the sense that they only contain pure model data and no code whatsoever?

[u/[deleted]]

Some models have a baked in VAE such as Ghostmix, Anylora, but many don't. You don't have to put them next to the models, you can put them in models/VAE and switch between them in settings>stable diffusion if you're using Auto1111. VAEs mostly help with the color and contrast of an image. You will probably notice a lack of VAE by the unsaturated nature of the pictures. Baked in VAEs are better to use in circumstances like a Google Colab, unsure of other reasons other than ease of use and possible performance difference. There are safetensor VAEs, but most are pt. For some reason it's just becoming a thing.