Zephyr - v4.0 - Illustrous Showcase

Same training dataset, same rank; batch size doubled from 32 to 64 so gradients should in theory be more accurate. Also tripled training time to 300 steps from 100. Abandoned cosine scheduler with restarts for normal cosine as it didn't prove itself in the previous version.

Mixed results, in that it's not much better. Still having trouble getting meaningful updates beyond the first 100 or so steps. I'm considering either increasing the rank back up to 64 or making a finetune, as the style is not really coming through, or maybe I'm doing something wrong.