Skip to content

Instantly share code, notes, and snippets.

@tori29umai0123
Created November 7, 2025 07:30
Show Gist options
  • Save tori29umai0123/e51935e6d68b7aa71263d42e7dcaca03 to your computer and use it in GitHub Desktop.
Save tori29umai0123/e51935e6d68b7aa71263d42e7dcaca03 to your computer and use it in GitHub Desktop.
Total tensors in 1: 1440
Total tensors in 2: 1680
⚠️ Keys only in second file:
transformer_blocks.0.img_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.0.img_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.0.txt_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.0.txt_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.1.img_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.1.img_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.1.txt_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.1.txt_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.10.img_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.10.img_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.10.txt_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.10.txt_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.11.img_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.11.img_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.11.txt_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.11.txt_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.12.img_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.12.img_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.12.txt_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.12.txt_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.13.img_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.13.img_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.13.txt_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.13.txt_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.14.img_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.14.img_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.14.txt_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.14.txt_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.15.img_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.15.img_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.15.txt_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.15.txt_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.16.img_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.16.img_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.16.txt_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.16.txt_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.17.img_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.17.img_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.17.txt_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.17.txt_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.18.img_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.18.img_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.18.txt_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.18.txt_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.19.img_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.19.img_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.19.txt_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.19.txt_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.2.img_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.2.img_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.2.txt_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.2.txt_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.20.img_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.20.img_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.20.txt_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.20.txt_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.21.img_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.21.img_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.21.txt_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.21.txt_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.22.img_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.22.img_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.22.txt_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.22.txt_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.23.img_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.23.img_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.23.txt_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.23.txt_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.24.img_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.24.img_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.24.txt_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.24.txt_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.25.img_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.25.img_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.25.txt_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.25.txt_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.26.img_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.26.img_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.26.txt_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.26.txt_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.27.img_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.27.img_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.27.txt_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.27.txt_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.28.img_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.28.img_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.28.txt_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.28.txt_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.29.img_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.29.img_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.29.txt_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.29.txt_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.3.img_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.3.img_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.3.txt_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.3.txt_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.30.img_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.30.img_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.30.txt_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.30.txt_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.31.img_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.31.img_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.31.txt_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.31.txt_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.32.img_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.32.img_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.32.txt_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.32.txt_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.33.img_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.33.img_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.33.txt_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.33.txt_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.34.img_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.34.img_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.34.txt_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.34.txt_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.35.img_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.35.img_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.35.txt_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.35.txt_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.36.img_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.36.img_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.36.txt_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.36.txt_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.37.img_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.37.img_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.37.txt_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.37.txt_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.38.img_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.38.img_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.38.txt_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.38.txt_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.39.img_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.39.img_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.39.txt_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.39.txt_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.4.img_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.4.img_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.4.txt_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.4.txt_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.40.img_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.40.img_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.40.txt_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.40.txt_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.41.img_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.41.img_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.41.txt_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.41.txt_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.42.img_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.42.img_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.42.txt_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.42.txt_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.43.img_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.43.img_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.43.txt_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.43.txt_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.44.img_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.44.img_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.44.txt_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.44.txt_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.45.img_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.45.img_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.45.txt_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.45.txt_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.46.img_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.46.img_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.46.txt_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.46.txt_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.47.img_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.47.img_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.47.txt_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.47.txt_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.48.img_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.48.img_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.48.txt_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.48.txt_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.49.img_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.49.img_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.49.txt_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.49.txt_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.5.img_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.5.img_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.5.txt_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.5.txt_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.50.img_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.50.img_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.50.txt_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.50.txt_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.51.img_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.51.img_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.51.txt_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.51.txt_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.52.img_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.52.img_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.52.txt_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.52.txt_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.53.img_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.53.img_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.53.txt_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.53.txt_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.54.img_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.54.img_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.54.txt_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.54.txt_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.55.img_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.55.img_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.55.txt_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.55.txt_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.56.img_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.56.img_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.56.txt_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.56.txt_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.57.img_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.57.img_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.57.txt_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.57.txt_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.58.img_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.58.img_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.58.txt_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.58.txt_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.59.img_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.59.img_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.59.txt_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.59.txt_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.6.img_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.6.img_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.6.txt_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.6.txt_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.7.img_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.7.img_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.7.txt_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.7.txt_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.8.img_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.8.img_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.8.txt_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.8.txt_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.9.img_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.9.img_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
transformer_blocks.9.txt_mlp.net.0.proj.lora_A.default.weight [(4, 3072)] torch.bfloat16
transformer_blocks.9.txt_mlp.net.0.proj.lora_B.default.weight [(12288, 4)] torch.bfloat16
🔍 Mismatched tensors:
transformer_blocks.0.attn.add_k_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.0.attn.add_k_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.0.attn.add_q_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.0.attn.add_q_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.0.attn.add_v_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.0.attn.add_v_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.0.attn.to_add_out.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.0.attn.to_add_out.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.0.attn.to_k.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.0.attn.to_k.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.0.attn.to_out.0.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.0.attn.to_out.0.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.0.attn.to_q.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.0.attn.to_q.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.0.attn.to_v.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.0.attn.to_v.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.0.img_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.0.img_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.0.img_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.0.img_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.0.txt_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.0.txt_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.0.txt_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.0.txt_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.1.attn.add_k_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.1.attn.add_k_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.1.attn.add_q_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.1.attn.add_q_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.1.attn.add_v_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.1.attn.add_v_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.1.attn.to_add_out.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.1.attn.to_add_out.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.1.attn.to_k.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.1.attn.to_k.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.1.attn.to_out.0.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.1.attn.to_out.0.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.1.attn.to_q.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.1.attn.to_q.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.1.attn.to_v.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.1.attn.to_v.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.1.img_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.1.img_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.1.img_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.1.img_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.1.txt_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.1.txt_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.1.txt_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.1.txt_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.10.attn.add_k_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.10.attn.add_k_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.10.attn.add_q_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.10.attn.add_q_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.10.attn.add_v_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.10.attn.add_v_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.10.attn.to_add_out.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.10.attn.to_add_out.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.10.attn.to_k.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.10.attn.to_k.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.10.attn.to_out.0.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.10.attn.to_out.0.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.10.attn.to_q.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.10.attn.to_q.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.10.attn.to_v.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.10.attn.to_v.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.10.img_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.10.img_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.10.img_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.10.img_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.10.txt_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.10.txt_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.10.txt_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.10.txt_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.11.attn.add_k_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.11.attn.add_k_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.11.attn.add_q_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.11.attn.add_q_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.11.attn.add_v_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.11.attn.add_v_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.11.attn.to_add_out.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.11.attn.to_add_out.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.11.attn.to_k.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.11.attn.to_k.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.11.attn.to_out.0.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.11.attn.to_out.0.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.11.attn.to_q.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.11.attn.to_q.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.11.attn.to_v.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.11.attn.to_v.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.11.img_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.11.img_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.11.img_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.11.img_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.11.txt_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.11.txt_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.11.txt_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.11.txt_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.12.attn.add_k_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.12.attn.add_k_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.12.attn.add_q_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.12.attn.add_q_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.12.attn.add_v_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.12.attn.add_v_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.12.attn.to_add_out.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.12.attn.to_add_out.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.12.attn.to_k.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.12.attn.to_k.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.12.attn.to_out.0.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.12.attn.to_out.0.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.12.attn.to_q.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.12.attn.to_q.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.12.attn.to_v.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.12.attn.to_v.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.12.img_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.12.img_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.12.img_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.12.img_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.12.txt_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.12.txt_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.12.txt_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.12.txt_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.13.attn.add_k_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.13.attn.add_k_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.13.attn.add_q_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.13.attn.add_q_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.13.attn.add_v_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.13.attn.add_v_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.13.attn.to_add_out.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.13.attn.to_add_out.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.13.attn.to_k.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.13.attn.to_k.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.13.attn.to_out.0.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.13.attn.to_out.0.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.13.attn.to_q.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.13.attn.to_q.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.13.attn.to_v.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.13.attn.to_v.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.13.img_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.13.img_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.13.img_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.13.img_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.13.txt_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.13.txt_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.13.txt_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.13.txt_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.14.attn.add_k_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.14.attn.add_k_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.14.attn.add_q_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.14.attn.add_q_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.14.attn.add_v_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.14.attn.add_v_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.14.attn.to_add_out.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.14.attn.to_add_out.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.14.attn.to_k.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.14.attn.to_k.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.14.attn.to_out.0.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.14.attn.to_out.0.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.14.attn.to_q.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.14.attn.to_q.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.14.attn.to_v.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.14.attn.to_v.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.14.img_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.14.img_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.14.img_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.14.img_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.14.txt_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.14.txt_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.14.txt_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.14.txt_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.15.attn.add_k_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.15.attn.add_k_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.15.attn.add_q_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.15.attn.add_q_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.15.attn.add_v_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.15.attn.add_v_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.15.attn.to_add_out.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.15.attn.to_add_out.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.15.attn.to_k.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.15.attn.to_k.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.15.attn.to_out.0.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.15.attn.to_out.0.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.15.attn.to_q.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.15.attn.to_q.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.15.attn.to_v.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.15.attn.to_v.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.15.img_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.15.img_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.15.img_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.15.img_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.15.txt_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.15.txt_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.15.txt_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.15.txt_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.16.attn.add_k_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.16.attn.add_k_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.16.attn.add_q_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.16.attn.add_q_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.16.attn.add_v_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.16.attn.add_v_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.16.attn.to_add_out.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.16.attn.to_add_out.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.16.attn.to_k.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.16.attn.to_k.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.16.attn.to_out.0.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.16.attn.to_out.0.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.16.attn.to_q.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.16.attn.to_q.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.16.attn.to_v.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.16.attn.to_v.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.16.img_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.16.img_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.16.img_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.16.img_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.16.txt_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.16.txt_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.16.txt_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.16.txt_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.17.attn.add_k_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.17.attn.add_k_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.17.attn.add_q_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.17.attn.add_q_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.17.attn.add_v_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.17.attn.add_v_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.17.attn.to_add_out.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.17.attn.to_add_out.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.17.attn.to_k.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.17.attn.to_k.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.17.attn.to_out.0.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.17.attn.to_out.0.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.17.attn.to_q.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.17.attn.to_q.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.17.attn.to_v.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.17.attn.to_v.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.17.img_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.17.img_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.17.img_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.17.img_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.17.txt_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.17.txt_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.17.txt_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.17.txt_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.18.attn.add_k_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.18.attn.add_k_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.18.attn.add_q_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.18.attn.add_q_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.18.attn.add_v_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.18.attn.add_v_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.18.attn.to_add_out.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.18.attn.to_add_out.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.18.attn.to_k.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.18.attn.to_k.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.18.attn.to_out.0.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.18.attn.to_out.0.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.18.attn.to_q.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.18.attn.to_q.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.18.attn.to_v.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.18.attn.to_v.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.18.img_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.18.img_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.18.img_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.18.img_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.18.txt_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.18.txt_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.18.txt_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.18.txt_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.19.attn.add_k_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.19.attn.add_k_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.19.attn.add_q_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.19.attn.add_q_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.19.attn.add_v_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.19.attn.add_v_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.19.attn.to_add_out.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.19.attn.to_add_out.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.19.attn.to_k.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.19.attn.to_k.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.19.attn.to_out.0.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.19.attn.to_out.0.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.19.attn.to_q.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.19.attn.to_q.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.19.attn.to_v.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.19.attn.to_v.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.19.img_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.19.img_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.19.img_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.19.img_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.19.txt_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.19.txt_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.19.txt_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.19.txt_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.2.attn.add_k_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.2.attn.add_k_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.2.attn.add_q_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.2.attn.add_q_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.2.attn.add_v_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.2.attn.add_v_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.2.attn.to_add_out.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.2.attn.to_add_out.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.2.attn.to_k.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.2.attn.to_k.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.2.attn.to_out.0.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.2.attn.to_out.0.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.2.attn.to_q.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.2.attn.to_q.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.2.attn.to_v.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.2.attn.to_v.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.2.img_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.2.img_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.2.img_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.2.img_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.2.txt_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.2.txt_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.2.txt_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.2.txt_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.20.attn.add_k_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.20.attn.add_k_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.20.attn.add_q_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.20.attn.add_q_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.20.attn.add_v_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.20.attn.add_v_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.20.attn.to_add_out.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.20.attn.to_add_out.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.20.attn.to_k.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.20.attn.to_k.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.20.attn.to_out.0.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.20.attn.to_out.0.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.20.attn.to_q.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.20.attn.to_q.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.20.attn.to_v.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.20.attn.to_v.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.20.img_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.20.img_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.20.img_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.20.img_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.20.txt_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.20.txt_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.20.txt_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.20.txt_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.21.attn.add_k_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.21.attn.add_k_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.21.attn.add_q_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.21.attn.add_q_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.21.attn.add_v_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.21.attn.add_v_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.21.attn.to_add_out.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.21.attn.to_add_out.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.21.attn.to_k.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.21.attn.to_k.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.21.attn.to_out.0.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.21.attn.to_out.0.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.21.attn.to_q.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.21.attn.to_q.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.21.attn.to_v.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.21.attn.to_v.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.21.img_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.21.img_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.21.img_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.21.img_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.21.txt_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.21.txt_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.21.txt_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.21.txt_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.22.attn.add_k_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.22.attn.add_k_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.22.attn.add_q_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.22.attn.add_q_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.22.attn.add_v_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.22.attn.add_v_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.22.attn.to_add_out.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.22.attn.to_add_out.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.22.attn.to_k.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.22.attn.to_k.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.22.attn.to_out.0.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.22.attn.to_out.0.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.22.attn.to_q.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.22.attn.to_q.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.22.attn.to_v.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.22.attn.to_v.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.22.img_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.22.img_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.22.img_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.22.img_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.22.txt_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.22.txt_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.22.txt_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.22.txt_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.23.attn.add_k_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.23.attn.add_k_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.23.attn.add_q_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.23.attn.add_q_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.23.attn.add_v_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.23.attn.add_v_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.23.attn.to_add_out.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.23.attn.to_add_out.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.23.attn.to_k.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.23.attn.to_k.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.23.attn.to_out.0.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.23.attn.to_out.0.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.23.attn.to_q.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.23.attn.to_q.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.23.attn.to_v.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.23.attn.to_v.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.23.img_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.23.img_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.23.img_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.23.img_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.23.txt_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.23.txt_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.23.txt_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.23.txt_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.24.attn.add_k_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.24.attn.add_k_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.24.attn.add_q_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.24.attn.add_q_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.24.attn.add_v_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.24.attn.add_v_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.24.attn.to_add_out.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.24.attn.to_add_out.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.24.attn.to_k.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.24.attn.to_k.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.24.attn.to_out.0.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.24.attn.to_out.0.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.24.attn.to_q.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.24.attn.to_q.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.24.attn.to_v.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.24.attn.to_v.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.24.img_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.24.img_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.24.img_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.24.img_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.24.txt_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.24.txt_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.24.txt_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.24.txt_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.25.attn.add_k_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.25.attn.add_k_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.25.attn.add_q_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.25.attn.add_q_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.25.attn.add_v_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.25.attn.add_v_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.25.attn.to_add_out.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.25.attn.to_add_out.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.25.attn.to_k.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.25.attn.to_k.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.25.attn.to_out.0.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.25.attn.to_out.0.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.25.attn.to_q.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.25.attn.to_q.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.25.attn.to_v.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.25.attn.to_v.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.25.img_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.25.img_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.25.img_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.25.img_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.25.txt_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.25.txt_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.25.txt_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.25.txt_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.26.attn.add_k_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.26.attn.add_k_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.26.attn.add_q_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.26.attn.add_q_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.26.attn.add_v_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.26.attn.add_v_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.26.attn.to_add_out.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.26.attn.to_add_out.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.26.attn.to_k.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.26.attn.to_k.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.26.attn.to_out.0.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.26.attn.to_out.0.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.26.attn.to_q.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.26.attn.to_q.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.26.attn.to_v.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.26.attn.to_v.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.26.img_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.26.img_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.26.img_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.26.img_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.26.txt_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.26.txt_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.26.txt_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.26.txt_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.27.attn.add_k_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.27.attn.add_k_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.27.attn.add_q_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.27.attn.add_q_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.27.attn.add_v_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.27.attn.add_v_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.27.attn.to_add_out.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.27.attn.to_add_out.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.27.attn.to_k.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.27.attn.to_k.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.27.attn.to_out.0.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.27.attn.to_out.0.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.27.attn.to_q.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.27.attn.to_q.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.27.attn.to_v.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.27.attn.to_v.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.27.img_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.27.img_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.27.img_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.27.img_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.27.txt_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.27.txt_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.27.txt_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.27.txt_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.28.attn.add_k_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.28.attn.add_k_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.28.attn.add_q_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.28.attn.add_q_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.28.attn.add_v_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.28.attn.add_v_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.28.attn.to_add_out.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.28.attn.to_add_out.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.28.attn.to_k.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.28.attn.to_k.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.28.attn.to_out.0.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.28.attn.to_out.0.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.28.attn.to_q.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.28.attn.to_q.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.28.attn.to_v.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.28.attn.to_v.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.28.img_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.28.img_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.28.img_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.28.img_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.28.txt_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.28.txt_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.28.txt_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.28.txt_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.29.attn.add_k_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.29.attn.add_k_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.29.attn.add_q_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.29.attn.add_q_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.29.attn.add_v_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.29.attn.add_v_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.29.attn.to_add_out.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.29.attn.to_add_out.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.29.attn.to_k.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.29.attn.to_k.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.29.attn.to_out.0.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.29.attn.to_out.0.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.29.attn.to_q.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.29.attn.to_q.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.29.attn.to_v.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.29.attn.to_v.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.29.img_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.29.img_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.29.img_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.29.img_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.29.txt_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.29.txt_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.29.txt_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.29.txt_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.3.attn.add_k_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.3.attn.add_k_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.3.attn.add_q_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.3.attn.add_q_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.3.attn.add_v_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.3.attn.add_v_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.3.attn.to_add_out.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.3.attn.to_add_out.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.3.attn.to_k.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.3.attn.to_k.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.3.attn.to_out.0.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.3.attn.to_out.0.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.3.attn.to_q.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.3.attn.to_q.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.3.attn.to_v.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.3.attn.to_v.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.3.img_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.3.img_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.3.img_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.3.img_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.3.txt_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.3.txt_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.3.txt_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.3.txt_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.30.attn.add_k_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.30.attn.add_k_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.30.attn.add_q_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.30.attn.add_q_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.30.attn.add_v_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.30.attn.add_v_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.30.attn.to_add_out.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.30.attn.to_add_out.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.30.attn.to_k.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.30.attn.to_k.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.30.attn.to_out.0.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.30.attn.to_out.0.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.30.attn.to_q.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.30.attn.to_q.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.30.attn.to_v.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.30.attn.to_v.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.30.img_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.30.img_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.30.img_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.30.img_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.30.txt_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.30.txt_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.30.txt_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.30.txt_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.31.attn.add_k_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.31.attn.add_k_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.31.attn.add_q_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.31.attn.add_q_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.31.attn.add_v_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.31.attn.add_v_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.31.attn.to_add_out.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.31.attn.to_add_out.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.31.attn.to_k.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.31.attn.to_k.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.31.attn.to_out.0.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.31.attn.to_out.0.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.31.attn.to_q.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.31.attn.to_q.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.31.attn.to_v.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.31.attn.to_v.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.31.img_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.31.img_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.31.img_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.31.img_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.31.txt_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.31.txt_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.31.txt_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.31.txt_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.32.attn.add_k_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.32.attn.add_k_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.32.attn.add_q_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.32.attn.add_q_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.32.attn.add_v_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.32.attn.add_v_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.32.attn.to_add_out.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.32.attn.to_add_out.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.32.attn.to_k.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.32.attn.to_k.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.32.attn.to_out.0.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.32.attn.to_out.0.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.32.attn.to_q.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.32.attn.to_q.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.32.attn.to_v.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.32.attn.to_v.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.32.img_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.32.img_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.32.img_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.32.img_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.32.txt_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.32.txt_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.32.txt_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.32.txt_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.33.attn.add_k_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.33.attn.add_k_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.33.attn.add_q_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.33.attn.add_q_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.33.attn.add_v_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.33.attn.add_v_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.33.attn.to_add_out.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.33.attn.to_add_out.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.33.attn.to_k.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.33.attn.to_k.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.33.attn.to_out.0.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.33.attn.to_out.0.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.33.attn.to_q.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.33.attn.to_q.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.33.attn.to_v.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.33.attn.to_v.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.33.img_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.33.img_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.33.img_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.33.img_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.33.txt_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.33.txt_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.33.txt_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.33.txt_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.34.attn.add_k_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.34.attn.add_k_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.34.attn.add_q_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.34.attn.add_q_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.34.attn.add_v_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.34.attn.add_v_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.34.attn.to_add_out.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.34.attn.to_add_out.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.34.attn.to_k.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.34.attn.to_k.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.34.attn.to_out.0.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.34.attn.to_out.0.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.34.attn.to_q.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.34.attn.to_q.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.34.attn.to_v.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.34.attn.to_v.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.34.img_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.34.img_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.34.img_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.34.img_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.34.txt_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.34.txt_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.34.txt_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.34.txt_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.35.attn.add_k_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.35.attn.add_k_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.35.attn.add_q_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.35.attn.add_q_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.35.attn.add_v_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.35.attn.add_v_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.35.attn.to_add_out.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.35.attn.to_add_out.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.35.attn.to_k.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.35.attn.to_k.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.35.attn.to_out.0.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.35.attn.to_out.0.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.35.attn.to_q.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.35.attn.to_q.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.35.attn.to_v.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.35.attn.to_v.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.35.img_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.35.img_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.35.img_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.35.img_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.35.txt_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.35.txt_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.35.txt_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.35.txt_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.36.attn.add_k_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.36.attn.add_k_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.36.attn.add_q_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.36.attn.add_q_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.36.attn.add_v_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.36.attn.add_v_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.36.attn.to_add_out.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.36.attn.to_add_out.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.36.attn.to_k.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.36.attn.to_k.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.36.attn.to_out.0.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.36.attn.to_out.0.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.36.attn.to_q.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.36.attn.to_q.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.36.attn.to_v.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.36.attn.to_v.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.36.img_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.36.img_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.36.img_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.36.img_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.36.txt_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.36.txt_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.36.txt_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.36.txt_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.37.attn.add_k_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.37.attn.add_k_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.37.attn.add_q_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.37.attn.add_q_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.37.attn.add_v_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.37.attn.add_v_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.37.attn.to_add_out.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.37.attn.to_add_out.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.37.attn.to_k.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.37.attn.to_k.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.37.attn.to_out.0.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.37.attn.to_out.0.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.37.attn.to_q.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.37.attn.to_q.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.37.attn.to_v.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.37.attn.to_v.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.37.img_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.37.img_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.37.img_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.37.img_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.37.txt_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.37.txt_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.37.txt_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.37.txt_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.38.attn.add_k_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.38.attn.add_k_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.38.attn.add_q_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.38.attn.add_q_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.38.attn.add_v_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.38.attn.add_v_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.38.attn.to_add_out.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.38.attn.to_add_out.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.38.attn.to_k.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.38.attn.to_k.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.38.attn.to_out.0.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.38.attn.to_out.0.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.38.attn.to_q.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.38.attn.to_q.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.38.attn.to_v.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.38.attn.to_v.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.38.img_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.38.img_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.38.img_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.38.img_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.38.txt_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.38.txt_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.38.txt_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.38.txt_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.39.attn.add_k_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.39.attn.add_k_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.39.attn.add_q_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.39.attn.add_q_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.39.attn.add_v_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.39.attn.add_v_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.39.attn.to_add_out.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.39.attn.to_add_out.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.39.attn.to_k.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.39.attn.to_k.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.39.attn.to_out.0.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.39.attn.to_out.0.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.39.attn.to_q.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.39.attn.to_q.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.39.attn.to_v.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.39.attn.to_v.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.39.img_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.39.img_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.39.img_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.39.img_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.39.txt_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.39.txt_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.39.txt_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.39.txt_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.4.attn.add_k_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.4.attn.add_k_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.4.attn.add_q_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.4.attn.add_q_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.4.attn.add_v_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.4.attn.add_v_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.4.attn.to_add_out.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.4.attn.to_add_out.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.4.attn.to_k.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.4.attn.to_k.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.4.attn.to_out.0.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.4.attn.to_out.0.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.4.attn.to_q.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.4.attn.to_q.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.4.attn.to_v.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.4.attn.to_v.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.4.img_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.4.img_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.4.img_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.4.img_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.4.txt_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.4.txt_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.4.txt_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.4.txt_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.40.attn.add_k_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.40.attn.add_k_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.40.attn.add_q_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.40.attn.add_q_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.40.attn.add_v_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.40.attn.add_v_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.40.attn.to_add_out.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.40.attn.to_add_out.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.40.attn.to_k.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.40.attn.to_k.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.40.attn.to_out.0.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.40.attn.to_out.0.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.40.attn.to_q.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.40.attn.to_q.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.40.attn.to_v.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.40.attn.to_v.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.40.img_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.40.img_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.40.img_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.40.img_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.40.txt_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.40.txt_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.40.txt_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.40.txt_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.41.attn.add_k_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.41.attn.add_k_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.41.attn.add_q_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.41.attn.add_q_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.41.attn.add_v_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.41.attn.add_v_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.41.attn.to_add_out.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.41.attn.to_add_out.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.41.attn.to_k.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.41.attn.to_k.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.41.attn.to_out.0.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.41.attn.to_out.0.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.41.attn.to_q.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.41.attn.to_q.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.41.attn.to_v.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.41.attn.to_v.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.41.img_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.41.img_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.41.img_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.41.img_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.41.txt_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.41.txt_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.41.txt_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.41.txt_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.42.attn.add_k_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.42.attn.add_k_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.42.attn.add_q_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.42.attn.add_q_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.42.attn.add_v_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.42.attn.add_v_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.42.attn.to_add_out.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.42.attn.to_add_out.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.42.attn.to_k.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.42.attn.to_k.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.42.attn.to_out.0.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.42.attn.to_out.0.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.42.attn.to_q.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.42.attn.to_q.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.42.attn.to_v.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.42.attn.to_v.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.42.img_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.42.img_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.42.img_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.42.img_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.42.txt_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.42.txt_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.42.txt_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.42.txt_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.43.attn.add_k_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.43.attn.add_k_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.43.attn.add_q_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.43.attn.add_q_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.43.attn.add_v_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.43.attn.add_v_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.43.attn.to_add_out.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.43.attn.to_add_out.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.43.attn.to_k.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.43.attn.to_k.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.43.attn.to_out.0.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.43.attn.to_out.0.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.43.attn.to_q.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.43.attn.to_q.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.43.attn.to_v.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.43.attn.to_v.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.43.img_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.43.img_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.43.img_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.43.img_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.43.txt_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.43.txt_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.43.txt_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.43.txt_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.44.attn.add_k_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.44.attn.add_k_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.44.attn.add_q_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.44.attn.add_q_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.44.attn.add_v_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.44.attn.add_v_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.44.attn.to_add_out.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.44.attn.to_add_out.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.44.attn.to_k.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.44.attn.to_k.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.44.attn.to_out.0.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.44.attn.to_out.0.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.44.attn.to_q.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.44.attn.to_q.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.44.attn.to_v.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.44.attn.to_v.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.44.img_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.44.img_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.44.img_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.44.img_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.44.txt_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.44.txt_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.44.txt_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.44.txt_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.45.attn.add_k_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.45.attn.add_k_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.45.attn.add_q_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.45.attn.add_q_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.45.attn.add_v_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.45.attn.add_v_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.45.attn.to_add_out.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.45.attn.to_add_out.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.45.attn.to_k.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.45.attn.to_k.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.45.attn.to_out.0.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.45.attn.to_out.0.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.45.attn.to_q.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.45.attn.to_q.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.45.attn.to_v.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.45.attn.to_v.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.45.img_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.45.img_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.45.img_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.45.img_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.45.txt_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.45.txt_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.45.txt_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.45.txt_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.46.attn.add_k_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.46.attn.add_k_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.46.attn.add_q_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.46.attn.add_q_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.46.attn.add_v_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.46.attn.add_v_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.46.attn.to_add_out.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.46.attn.to_add_out.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.46.attn.to_k.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.46.attn.to_k.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.46.attn.to_out.0.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.46.attn.to_out.0.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.46.attn.to_q.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.46.attn.to_q.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.46.attn.to_v.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.46.attn.to_v.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.46.img_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.46.img_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.46.img_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.46.img_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.46.txt_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.46.txt_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.46.txt_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.46.txt_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.47.attn.add_k_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.47.attn.add_k_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.47.attn.add_q_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.47.attn.add_q_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.47.attn.add_v_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.47.attn.add_v_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.47.attn.to_add_out.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.47.attn.to_add_out.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.47.attn.to_k.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.47.attn.to_k.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.47.attn.to_out.0.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.47.attn.to_out.0.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.47.attn.to_q.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.47.attn.to_q.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.47.attn.to_v.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.47.attn.to_v.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.47.img_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.47.img_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.47.img_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.47.img_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.47.txt_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.47.txt_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.47.txt_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.47.txt_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.48.attn.add_k_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.48.attn.add_k_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.48.attn.add_q_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.48.attn.add_q_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.48.attn.add_v_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.48.attn.add_v_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.48.attn.to_add_out.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.48.attn.to_add_out.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.48.attn.to_k.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.48.attn.to_k.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.48.attn.to_out.0.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.48.attn.to_out.0.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.48.attn.to_q.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.48.attn.to_q.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.48.attn.to_v.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.48.attn.to_v.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.48.img_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.48.img_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.48.img_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.48.img_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.48.txt_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.48.txt_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.48.txt_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.48.txt_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.49.attn.add_k_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.49.attn.add_k_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.49.attn.add_q_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.49.attn.add_q_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.49.attn.add_v_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.49.attn.add_v_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.49.attn.to_add_out.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.49.attn.to_add_out.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.49.attn.to_k.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.49.attn.to_k.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.49.attn.to_out.0.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.49.attn.to_out.0.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.49.attn.to_q.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.49.attn.to_q.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.49.attn.to_v.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.49.attn.to_v.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.49.img_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.49.img_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.49.img_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.49.img_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.49.txt_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.49.txt_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.49.txt_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.49.txt_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.5.attn.add_k_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.5.attn.add_k_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.5.attn.add_q_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.5.attn.add_q_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.5.attn.add_v_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.5.attn.add_v_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.5.attn.to_add_out.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.5.attn.to_add_out.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.5.attn.to_k.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.5.attn.to_k.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.5.attn.to_out.0.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.5.attn.to_out.0.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.5.attn.to_q.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.5.attn.to_q.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.5.attn.to_v.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.5.attn.to_v.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.5.img_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.5.img_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.5.img_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.5.img_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.5.txt_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.5.txt_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.5.txt_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.5.txt_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.50.attn.add_k_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.50.attn.add_k_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.50.attn.add_q_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.50.attn.add_q_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.50.attn.add_v_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.50.attn.add_v_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.50.attn.to_add_out.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.50.attn.to_add_out.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.50.attn.to_k.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.50.attn.to_k.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.50.attn.to_out.0.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.50.attn.to_out.0.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.50.attn.to_q.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.50.attn.to_q.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.50.attn.to_v.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.50.attn.to_v.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.50.img_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.50.img_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.50.img_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.50.img_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.50.txt_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.50.txt_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.50.txt_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.50.txt_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.51.attn.add_k_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.51.attn.add_k_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.51.attn.add_q_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.51.attn.add_q_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.51.attn.add_v_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.51.attn.add_v_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.51.attn.to_add_out.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.51.attn.to_add_out.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.51.attn.to_k.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.51.attn.to_k.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.51.attn.to_out.0.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.51.attn.to_out.0.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.51.attn.to_q.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.51.attn.to_q.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.51.attn.to_v.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.51.attn.to_v.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.51.img_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.51.img_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.51.img_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.51.img_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.51.txt_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.51.txt_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.51.txt_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.51.txt_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.52.attn.add_k_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.52.attn.add_k_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.52.attn.add_q_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.52.attn.add_q_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.52.attn.add_v_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.52.attn.add_v_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.52.attn.to_add_out.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.52.attn.to_add_out.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.52.attn.to_k.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.52.attn.to_k.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.52.attn.to_out.0.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.52.attn.to_out.0.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.52.attn.to_q.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.52.attn.to_q.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.52.attn.to_v.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.52.attn.to_v.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.52.img_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.52.img_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.52.img_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.52.img_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.52.txt_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.52.txt_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.52.txt_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.52.txt_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.53.attn.add_k_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.53.attn.add_k_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.53.attn.add_q_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.53.attn.add_q_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.53.attn.add_v_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.53.attn.add_v_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.53.attn.to_add_out.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.53.attn.to_add_out.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.53.attn.to_k.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.53.attn.to_k.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.53.attn.to_out.0.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.53.attn.to_out.0.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.53.attn.to_q.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.53.attn.to_q.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.53.attn.to_v.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.53.attn.to_v.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.53.img_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.53.img_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.53.img_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.53.img_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.53.txt_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.53.txt_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.53.txt_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.53.txt_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.54.attn.add_k_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.54.attn.add_k_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.54.attn.add_q_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.54.attn.add_q_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.54.attn.add_v_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.54.attn.add_v_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.54.attn.to_add_out.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.54.attn.to_add_out.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.54.attn.to_k.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.54.attn.to_k.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.54.attn.to_out.0.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.54.attn.to_out.0.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.54.attn.to_q.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.54.attn.to_q.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.54.attn.to_v.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.54.attn.to_v.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.54.img_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.54.img_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.54.img_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.54.img_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.54.txt_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.54.txt_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.54.txt_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.54.txt_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.55.attn.add_k_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.55.attn.add_k_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.55.attn.add_q_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.55.attn.add_q_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.55.attn.add_v_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.55.attn.add_v_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.55.attn.to_add_out.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.55.attn.to_add_out.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.55.attn.to_k.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.55.attn.to_k.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.55.attn.to_out.0.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.55.attn.to_out.0.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.55.attn.to_q.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.55.attn.to_q.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.55.attn.to_v.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.55.attn.to_v.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.55.img_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.55.img_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.55.img_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.55.img_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.55.txt_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.55.txt_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.55.txt_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.55.txt_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.56.attn.add_k_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.56.attn.add_k_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.56.attn.add_q_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.56.attn.add_q_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.56.attn.add_v_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.56.attn.add_v_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.56.attn.to_add_out.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.56.attn.to_add_out.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.56.attn.to_k.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.56.attn.to_k.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.56.attn.to_out.0.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.56.attn.to_out.0.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.56.attn.to_q.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.56.attn.to_q.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.56.attn.to_v.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.56.attn.to_v.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.56.img_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.56.img_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.56.img_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.56.img_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.56.txt_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.56.txt_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.56.txt_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.56.txt_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.57.attn.add_k_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.57.attn.add_k_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.57.attn.add_q_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.57.attn.add_q_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.57.attn.add_v_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.57.attn.add_v_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.57.attn.to_add_out.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.57.attn.to_add_out.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.57.attn.to_k.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.57.attn.to_k.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.57.attn.to_out.0.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.57.attn.to_out.0.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.57.attn.to_q.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.57.attn.to_q.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.57.attn.to_v.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.57.attn.to_v.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.57.img_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.57.img_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.57.img_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.57.img_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.57.txt_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.57.txt_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.57.txt_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.57.txt_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.58.attn.add_k_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.58.attn.add_k_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.58.attn.add_q_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.58.attn.add_q_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.58.attn.add_v_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.58.attn.add_v_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.58.attn.to_add_out.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.58.attn.to_add_out.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.58.attn.to_k.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.58.attn.to_k.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.58.attn.to_out.0.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.58.attn.to_out.0.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.58.attn.to_q.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.58.attn.to_q.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.58.attn.to_v.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.58.attn.to_v.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.58.img_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.58.img_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.58.img_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.58.img_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.58.txt_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.58.txt_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.58.txt_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.58.txt_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.59.attn.add_k_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.59.attn.add_k_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.59.attn.add_q_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.59.attn.add_q_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.59.attn.add_v_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.59.attn.add_v_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.59.attn.to_add_out.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.59.attn.to_add_out.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.59.attn.to_k.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.59.attn.to_k.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.59.attn.to_out.0.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.59.attn.to_out.0.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.59.attn.to_q.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.59.attn.to_q.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.59.attn.to_v.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.59.attn.to_v.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.59.img_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.59.img_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.59.img_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.59.img_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.59.txt_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.59.txt_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.59.txt_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.59.txt_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.6.attn.add_k_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.6.attn.add_k_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.6.attn.add_q_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.6.attn.add_q_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.6.attn.add_v_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.6.attn.add_v_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.6.attn.to_add_out.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.6.attn.to_add_out.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.6.attn.to_k.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.6.attn.to_k.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.6.attn.to_out.0.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.6.attn.to_out.0.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.6.attn.to_q.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.6.attn.to_q.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.6.attn.to_v.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.6.attn.to_v.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.6.img_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.6.img_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.6.img_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.6.img_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.6.txt_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.6.txt_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.6.txt_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.6.txt_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.7.attn.add_k_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.7.attn.add_k_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.7.attn.add_q_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.7.attn.add_q_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.7.attn.add_v_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.7.attn.add_v_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.7.attn.to_add_out.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.7.attn.to_add_out.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.7.attn.to_k.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.7.attn.to_k.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.7.attn.to_out.0.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.7.attn.to_out.0.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.7.attn.to_q.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.7.attn.to_q.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.7.attn.to_v.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.7.attn.to_v.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.7.img_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.7.img_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.7.img_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.7.img_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.7.txt_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.7.txt_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.7.txt_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.7.txt_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.8.attn.add_k_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.8.attn.add_k_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.8.attn.add_q_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.8.attn.add_q_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.8.attn.add_v_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.8.attn.add_v_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.8.attn.to_add_out.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.8.attn.to_add_out.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.8.attn.to_k.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.8.attn.to_k.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.8.attn.to_out.0.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.8.attn.to_out.0.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.8.attn.to_q.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.8.attn.to_q.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.8.attn.to_v.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.8.attn.to_v.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.8.img_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.8.img_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.8.img_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.8.img_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.8.txt_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.8.txt_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.8.txt_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.8.txt_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.9.attn.add_k_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.9.attn.add_k_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.9.attn.add_q_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.9.attn.add_q_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.9.attn.add_v_proj.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.9.attn.add_v_proj.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.9.attn.to_add_out.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.9.attn.to_add_out.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.9.attn.to_k.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.9.attn.to_k.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.9.attn.to_out.0.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.9.attn.to_out.0.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.9.attn.to_q.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.9.attn.to_q.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.9.attn.to_v.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.9.attn.to_v.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.9.img_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.9.img_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.9.img_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.9.img_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
transformer_blocks.9.txt_mlp.net.2.lora_A.default.weight
file1: (16, 12288) torch.bfloat16
file2: (4, 12288) torch.bfloat16
transformer_blocks.9.txt_mlp.net.2.lora_B.default.weight
file1: (3072, 16) torch.bfloat16
file2: (3072, 4) torch.bfloat16
transformer_blocks.9.txt_mod.1.lora_A.default.weight
file1: (16, 3072) torch.bfloat16
file2: (4, 3072) torch.bfloat16
transformer_blocks.9.txt_mod.1.lora_B.default.weight
file1: (18432, 16) torch.bfloat16
file2: (18432, 4) torch.bfloat16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment