Generate Images Using Image to Image Guidance

This feature allows for uploaded or generated Images as reference, apply ControlNet and add finer adjustments to your image appearance.

πŸ‘

Update

Image guidance via API now supports multiple ControlNets

Follow these recipes with code snippets to generate images using image guidance

Style Reference

This guide will recreate the following Style Reference functionality in the Web UI via API.

Using multiple Style Reference on the Web App

Using multiple Style Reference on the Web App

Generation using multiple Style Reference

Generation using multiple Style Reference

Parameter breakdown:

  • Specify your image type in initImageType as either GENERATED for Leonardo.Ai generated images or UPLOADED if you upload your own image.
  • The parameter weight is a numeric value between 0-2, and 'strengthType' is a string comprised of the values between Low-Max. This will be bucketed under these settings:
strengthTypeweight
Low0-0.4
Mid0.4-0.8
High0.8-1.2
Ultra1.2-1.6
Max1.6-2

Either parameters can be used for the guidance strength, but not both at the same time.

  • influence will only appear if multiple Style References are used, and only used for Style Reference. This is a ratio of the influence of the two images. i.e. The right image from the example will have a higher influence on the output.

E.g. If Image A has an influence of 1 and Image B also has an influence of 1, their ratio will be 1:1. Similarly, if Image A has an influence of 0.5 and Image B has an influence of 0.5, the ratio remains 1:1. It's important to note that the total of both image influences does not necessarily need to add up to 1.

  • If only using one style reference image, you don’t need to include influence in the body parameter.

Sample Request

curl --request POST \
     --url https://cloud.leonardo.ai/api/rest/v1/generations \
     --header 'accept: application/json' \
     --header 'authorization: Bearer <YOUR_API_KEY>' \
     --header 'content-type: application/json' \
     --data '
{
  "height": 512,
  "modelId": "aa77f04e-3eec-4034-9c07-d0f619684628",//Leonardo Kino XL
  "prompt": "A wistful young woman stands in the beaming doorway of a sunlit room",
  "presetStyle":"CINEMATIC",
  "width": 1024,
  "photoReal": true,
  "photoRealVersion":"v2",
  "alchemy":true,
  "controlnets": [
        {
            "initImageId": <YOUR_GENERATED_IMAGE_ID>,
            "initImageType": "GENERATED",
            "preprocessorId": 67, //Style Reference ID
            "strengthType": "High",
            "influence": 0.39
        },
        {
            "initImageId": <YOUR_INIT_IMAGE_ID>,
            "initImageType": "UPLOADED",
            "preprocessorId": 67,
            "strengthType": "High",
            "influence": 0.64
        }
    ]
}

Character Reference

This guide will recreate the following functionality in the Web UI via API.

Using Character Reference with Style Reference

Using Character Reference with Style Reference

Generation using Character and Style Reference

Generation using Character and Style Reference

Parameter breakdown:

  • Specify your image type in initImageType as either GENERATED for Leonardo.Ai generated images or UPLOADED if you upload your own image.
  • The parameter weight is a numeric value between 0-2, and 'strengthType' is a string comprised of the values between Low-Max. This will be bucketed under these settings:
strengthTypeweightDescription
Low0-0.66Provide greatest flexibility at the cost of resemblance
Mid0.66-1.32Display reasonable flexibility and more resemblance.
High0.32-2Less flexible but have the best resemblance.

πŸ“˜

Note

Character Reference is not intended as a face swap feature and does not guarantee a perfect replica of a person in the output.

Sample Request

curl --request POST \
     --url https://cloud.leonardo.ai/api/rest/v1/generations \
     --header 'accept: application/json' \
     --header 'authorization: Bearer <YOUR_API_KEY>' \
     --header 'content-type: application/json' \
     --data '
{
  "height": 576,
  "modelId": "aa77f04e-3eec-4034-9c07-d0f619684628",//Leonardo Kino XL
  "prompt": "A mesmerizing lady with cascading strands of blonde hair gazes through a misty train window",
  "presetStyle":"CINEMATIC",
  "width": 1024,
  "photoReal": true,
  "photoRealVersion":"v2",
  "alchemy":true,
  "controlnets": [
        {
            "initImageId": <YOUR_INIT_IMAGE_ID>,
            "initImageType": "UPLOADED",
            "preprocessorId": 133,//Character Reference Id
            "strengthType": "Mid",
        },
        {
            "initImageId": <YOUR_GENERATED_IMAGE_ID>,
            "initImageType": "GENERATED",
            "preprocessorId": 67,//Style Reference Id
            "strengthType": "High",
        }
    ]
}

Image to Image Guidance

Using Image to Image with Style Reference

Using Image to Image with Style Reference

Sample Request

Image to Image does not require ControlNet preprocessor IDs, below is an example request.

πŸ“˜

Note

  • Image to Image with multiple ControlNets is incompatible with most SDXL models, except for Style Reference (67) and Character Reference (133) Preprocessor IDs.
curl --request POST \
     --url https://cloud.leonardo.ai/api/rest/v1/generations \
     --header 'accept: application/json' \
     --header 'authorization: Bearer <YOUR_API_KEY>' \
     --header 'content-type: application/json' \
     --data '
{
  "height": 644,
  "modelId": "b24e16ff-06e3-43eb-8d33-4416c2d75876" //Leonardo Lightning XL,
  "prompt": "Vibrant makeup bottle in a showroom",
  "presetStyle":"DYNAMIC",
  "width": 1040,
  "alchemy":true,
  "controlnets": [
        {
            "initImageId":<YOUR_GENERATED_IMAGE_ID>,
            "initImageType": "GENERATED",
            "preprocessorId": 67, //Style Reference ID
            "strengthType": "High",
        }
    
    ],
  "init_image_id": <YOUR_INIT_IMAGE_ID> ,
  "init_strength": 0.5,
}

Other ControlNets

Image Guidance uses different preprocessor IDs depending on the base model for different ControlNets. Refer to the table below to combine multiple ControlNets.

ControlNetSD1.5SD2.1SDXL
Style Referencexx67
Character Referencexx133
Content Referencexx100
Edge to Image11219
Depth to Image31320
Pose to Image71621
Text Image Input111822
Sketch to Image1017x
Normal Map615x
Line Art5xx
Pattern to Image8xx
QR Code to Image9xx

Sample Request

curl --request POST \
     --url https://cloud.leonardo.ai/api/rest/v1/generations \
     --header 'accept: application/json' \
     --header 'authorization: Bearer <YOUR_API_KEY>' \
     --header 'content-type: application/json' \
     --data '
{
  "height": 512,
  "modelId": "aa77f04e-3eec-4034-9c07-d0f619684628",//Leonardo Kino XL
  "prompt": "Intricately swirling nebulas dance across a vividly colored galactic map",
  "width": 1024,
  "alchemy":true,
  "controlnets": [
        {
            "initImageId": <YOUR_GENERATED_IMAGE_ID>,
            "initImageType": "GENERATED",
            "preprocessorId": 19, //Edge to Image ID
            "weight": "0.5"
        }   
    ]
}

Image Guidance using Uploaded or Generated Images (Legacy format)

The guide will recreate the following functionality in the Web UI via API.

Applying previously generated image as Image to Image Guidance

Applying previously generated image as Image to Image Guidance

🚧

Limitations

  • Legacy Image Guidance features on API does not have full parity with the Web UI.

controlNetType,controlNet and weighting are now legacy ControlNet parameters.

ControlNetcontrolNetType
Edge to ImageCANNY
Pose to ImagePOSE
Depth to ImageDEPTH

Sample Request For Uploaded Images

This request has no ControlNets added, it is only using Image to Image Guidance.

curl --request POST \
     --url https://cloud.leonardo.ai/api/rest/v1/generations \
     --header 'accept: application/json' \
     --header 'authorization: Bearer <YOUR_API_KEY>' \
     --header 'content-type: application/json' \
     --data '
{
  "height": 512,
  "modelId": "1e60896f-3c26-4296-8ecc-53e2afecc132",
  "prompt": "An oil painting of an orange cat",
  "width": 512,
  "init_image_id": <YOUR_INIT_IMAGE_ID> ,
  "init_strength": 0.5,
}

Sample Request for Generated Images

🚧

Legacy Feature

The following request is a legacy ControlNet parameter and will be soon be deprecating. We recommend to use the new ControlNet parameters which also allows for multiple ControlNets.

This request adds Edge to Image ControlNet using your previously generated image.

curl --request POST \
     --url https://cloud.leonardo.ai/api/rest/v1/generations \
     --header 'accept: application/json' \
     --header 'authorization: Bearer <YOUR_API_KEY>' \
     --header 'content-type: application/json' \
     --data '
{
  "height": 512,
  "modelId": "1e60896f-3c26-4296-8ecc-53e2afecc132",
  "prompt": "An oil painting of an orange cat",
  "width": 512
  "init_generation_image_id": "<YOUR_GENERATED_IMAGE_ID>",
  "init_strength": 0.5,
  "controlNet": True,
  "controlNetType": "CANNY"  
}


What’s Next

View our other API References below