After scrounging around for data, was able to get this character created. Mind you, took several attempts at training her so it can finally have prompts where she had the option without wearing her hat. Keep in mind, that it might have her hat on sometimes even with negative prompts; be patient. :P
Thanks to waifuliberator for the assist in getting the dataset.