thedeoxen

/

refcontrol-flux-kontext-reference-pose-lora

like 42

Hello, I tried the LoRA model you shared, and it works great. I compared it with the pose reference kontext-lora that I trained myself, and the effect is worse than yours. I would like to ask how you trained it, including data production, data volume, training parameters, etc.The following is a comparison of the effects of our models


If it's convenient for you, can I contact you in some way to consult you alone?

Hey guys, sorry for the delay. I am going to prepare all information about training in the github repo, but write here some initial info:
I used images pairs like this around 100 pairs.




For training I used aitoolkit with default parameters exept training steps I set 8000 and resolutions used 768 and 512 to fit to 4090 gpu memory.
I selected weights from 6500 step, selected by comparing results from validation generation.
Let me know if i missed something important.

Great work! Did you generate the different views of the character using Flux pro?

Hey guys, sorry for the delay. I am going to prepare all information about training in the github repo, but write here some initial info:
I used images pairs like this around 100 pairs.




For training I used aitoolkit with default parameters exept training steps I set 8000 and resolutions used 768 and 512 to fit to 4090 gpu memory.
I selected weights from 6500 step, selected by comparing results from validation generation.
Let me know if i missed something important.

Thank you very much for your sharing. I used 2000 pairs of data and trained for 15000 steps, but the effect is not as good as your lora. My data is as follows: on the left is a skeleton diagram, on the right is a model diagram, and the target diagram is a single model diagram with the target posture. The posture reference effect after training is quite good, but the hands and feet are likely to have abnormal distortions. I am looking forward to the sharing of your github repo.

Hey guys, sorry for the delay. I am going to prepare all information about training in the github repo, but write here some initial info:
I used images pairs like this around 100 pairs.




For training I used aitoolkit with default parameters exept training steps I set 8000 and resolutions used 768 and 512 to fit to 4090 gpu memory.
I selected weights from 6500 step, selected by comparing results from validation generation.
Let me know if i missed something important.

Thank you very much for your sharing. I used 2000 pairs of data and trained for 15000 steps, but the effect is not as good as your lora. My data is as follows: on the left is a skeleton diagram, on the right is a model diagram, and the target diagram is a single model diagram with the target posture. The posture reference effect after training is quite good, but the hands and feet are likely to have abnormal distortions. I am looking forward to the sharing of your github repo.

Can you also share git? cause 2K pairs sounds impressive. Also 15k steps for 2k is 7.5 steps per image, which is very low, even 10 is low, so your lora didnt actually analyze images a lot. Also your problem may be in "single model diagram with the target posture". Does it mean you input squre image with skeleton and character in each half of image and output is just full square pic of character? if so it may not exactly match skeloton due to resize or relocation and its harder for flux the more of image should be changed, ehich in your case always is more than half, and for <1o steps per image it just can't get it

@thedeoxen Great work. Are you planning to opensource your dataset? That may help others to understand more. Thanks

hey @hardikdava123 actually it is already opensourced :)
https://huggingface.co/datasets/thedeoxen/refcontrol-flux-kontext-dataset

how should i use workflow for inference, just with this input ?

why should i use this node for inference?