Replicate LoRAs

comfy-replicate-lora.md

Thank you for providing the updated documentation links. Based on the information in these guides:

• Running ComfyUI on Replicate
• ComfyUI Guide on Replicate

Updated Insight:

Yes, fofr/any-comfyui-workflow on Replicate does allow the use of custom-trained LoRAs (Low-Rank Adaptations). The Replicate platform has been updated to support more flexible workflows with ComfyUI, including the ability to load external resources like custom models and LoRAs.

How to Use Custom LoRAs:

1. Host Your LoRA Files:

   • Public URLs: Upload your custom LoRA files to a location accessible via a direct URL (e.g., GitHub, Google Drive with direct link sharing, or a public file hosting service).
   • Ensure Accessibility: The URL must be publicly accessible without authentication.

2. Modify Your ComfyUI Workflow:

   • Add a Load LoRA Node: In your ComfyUI workflow JSON, include a node that loads your custom LoRA.
   • Specify the LoRA URL: In the node parameters, set the model field to the direct URL of your hosted LoRA file.

   {
     "type": "Load LoRA",
     "id": "load_lora_node",
     "inputs": {
       "model": "https://your-hosted-url.com/your-custom-lora.safetensors",
       "weight": 1.0
     }
   }

3. Integrate the LoRA into Your Workflow:

   • Connect Nodes: Ensure the Load LoRA node is correctly connected to the rest of your workflow, typically feeding into the KSampler or equivalent node that handles model conditioning.

4. Run the Workflow on Replicate:

   • Submit the Workflow JSON: When running the fofr/any-comfyui-workflow model on Replicate, provide your modified workflow JSON.
   • Monitor Resource Usage: Be mindful of the execution time and resource limits on Replicate to avoid any runtime errors.

Considerations:

• File Size Limits: Replicate may have limits on the size of files that can be downloaded during model execution. Ensure your LoRA files are reasonably sized.
• Security and Privacy: Hosting your LoRA publicly means anyone with the URL can access it. Consider using a secure, temporary hosting solution if privacy is a concern.
• Compatibility: Verify that your custom LoRA is compatible with the base model specified in your workflow to prevent any compatibility issues.

Example Workflow Snippet:

Here's an example snippet of how you might include a custom LoRA in your workflow:

{
  "workflow": [
    {
      "type": "Load VAE",
      "id": "load_vae",
      "inputs": {
        "model": "vae-ft-mse-840000-ema-pruned.ckpt"
      }
    },
    {
      "type": "Load Checkpoint",
      "id": "load_checkpoint",
      "inputs": {
        "model": "sd-v1-5.ckpt"
      }
    },
    {
      "type": "Load LoRA",
      "id": "load_lora",
      "inputs": {
        "model": "https://your-hosted-url.com/your-custom-lora.safetensors",
        "weight": 0.8
      }
    },
    {
      "type": "KSampler",
      "id": "ksampler",
      "inputs": {
        "seed": 12345,
        "steps": 20,
        "cfg_scale": 7.0,
        "model": "load_checkpoint",
        "vae": "load_vae",
        "lora": "load_lora"
      }
    }
    // ... rest of your workflow nodes ...
  ]
}

Steps to Follow:

1. Prepare Your LoRA: Ensure your LoRA file is in a format compatible with ComfyUI (e.g., .safetensors or .pt).

2. Host the File: Upload it to a reliable hosting service that allows direct downloads.

3. Update the Workflow: Edit your ComfyUI workflow JSON to include the Load LoRA node with the correct URL.

4. Test Locally: If possible, test your workflow locally with ComfyUI to ensure it functions as expected before running it on Replicate.

5. Run on Replicate: Use the updated workflow JSON as input to the fofr/any-comfyui-workflow model on Replicate.

Additional Tips:

• Caching: Replicate may cache downloaded models for efficiency. Subsequent runs might be faster if using the same LoRA.
• Error Handling: If you encounter errors, check the logs provided by Replicate for troubleshooting.
• Community Support: Consider reaching out to the Replicate community or forums if you need assistance with complex workflows.

Conclusion:

With the updated capabilities outlined in the Replicate documentation, you can indeed use custom-trained LoRAs with fofr/any-comfyui-workflow. By hosting your LoRA files and correctly referencing them in your ComfyUI workflow, you can integrate custom models into your image generation tasks on Replicate.

---

References:

• Replicate Guide - Running ComfyUI: Run ComfyUI on Replicate
• ComfyUI Documentation: ComfyUI Guide on Replicate

Let me know if you need further assistance setting up your workflow or if you have any other questions!