Sign In
homemodelsimagesvideos3D Modelshubsarticlescomicschallengesupdatesshop

Anima Base 1.0 GSQ-RCO quantized (safetensor+gguf) proof of concept - v1.0 Showcase

ZE

zeuss194

Loading Images

TL:DR

The custom int8 x bf16 is very close to base model for only 2.4GB

The custom GGUF is the best I've manage to get before quality take a hit and lose to closeness to base model.

Based on ISTA-DAS labs Non-uniform GGUF quantizations via GSQ + RCO: per-tensor mixed precision in standard GGUF form.

There is also a int8 convrot created using the same pattern, where a handfull of tensor have been "buffed" to BF16.

Both models went throught some overnight gpu compute to be done and manual curating by me.

I post here the 2 models I think are the closest to base model at a fraction of the weight.

It's mostly experimental, can be usefull for 2 class of people

  • User with "old" 4gb or 6gb gpu, This give some headroom for lora stacks and higher res and batchs

  • User with massive workflow who want everything to in VRAM.

  • People like me who just want to try ^

Comments