This is meant to be an image to image workflow and not a video workflow.
Using Xinsir's Union Controlnet in Tile mode with low weight results in a very nice pose transfer. This can be used in conjunction with LoRAs to transfer pose from real to Anime.