MOSS architecture converter to int8_convrot

A script that allows you to convert a bf16 checkpoint of the MOSS architecture into an int8 ConvRot format with a single click. It even supports non-standard attention block configurations. In short, it can convert just about anything into int8 ConvRot without any unnecessary hassle.

It works great on CPU, actually I recommend installing CPU libraries to avoid OOM.

1. Install Python 3.11, making sure to check the "Add python.exe to PATH" box.

2. Extract the converter archive to a simple folder, e.g., C:\MOSS_ConvRot_Converter.

3. Run INSTALL_DEPS.bat.

4. Run CHECK_SETUP.bat (to ensure all required libraries are installed correctly).

5. Drop your .safetensors checkpoint into the converter folder.

6. Run CONVERT.bat.

7. The final output file will appear right next to it: *_int8_convrot.safetensors.