Seamless texture generation support for qwen image, z-image, and flux

[Phylliida]

Adds support for qwen image to generate seamless textures, using a --circular flag. Edit: Also includes a --circularx and --circulary if you want only tiling on those axes

Requires this PR ggml-org/ggml#1374 (Edit: see ggml-org/llama.cpp#16985) for ggml that adds a "circular" mode that can be used.

I also had to tweak rope so the period of the sinusods would evenly tile.

./bin/sd  --diffusion-model /home/bepis/Documents/world/models/qwen-image-lighting-8steps-V1.0-Q4_K_S.gguf  --vae /home/bepis/Documents/world/models/qwen_image_vae.safetensors  --qwen2vl /home/bepis/Documents/world/models/Qwen2.5-VL-7B-Instruct-IQ4_XS.gguf --cfg-scale 2.5 --sampling-method euler -p "onion" --circular --seed 420

[Phylliida]

Some conflicts, lemme resolve those...

[leejet]

Nice Work! Once ggml merges the circular-pad changes, I will merge this PR.

[oscarbg]

llama.cpp PR merged already!

[Phylliida]

Needs to be modified slightly bc you can do it only circular pad no circular conv

[Phylliida]

In particular like this:

__STATIC_INLINE__ struct ggml_tensor* ggml_nn_conv_2d(struct ggml_context* ctx,
                                                      struct ggml_tensor* x,
                                                      struct ggml_tensor* w,
                                                      struct ggml_tensor* b,
                                                      int s0      = 1,
                                                      int s1      = 1,
                                                      int p0      = 0,
                                                      int p1      = 0,
                                                      int d0      = 1,
                                                      int d1      = 1,
                                                      bool direct = false,
                                                      float scale = 1.f) {
    if (scale != 1.f) {
        x = ggml_scale(ctx, x, scale);
    }
    const bool use_circular = sd_is_circular_padding_enabled();
    LOG_DEBUG("use circular conv %d", use_circular ? 1 : 0);
    const bool is_depthwise = (w->ne[2] == 1 && x->ne[2] == w->ne[3]);

    if (use_circular && (p0 != 0 || p1 != 0)) {
        x  = ggml_pad_ext_circular(ctx, x, p0, p0, p1, p1, 0, 0, 0, 0);
        p0 = 0;
        p1 = 0;
    }

    if (direct) {
        if (is_depthwise) {
            x = ggml_conv_2d_dw_direct(ctx, w, x, s0, s1, p0, p1, d0, d1);
        } else {
            x = ggml_conv_2d_direct(ctx, w, x, s0, s1, p0, p1, d0, d1);
        }
    } else {
        x = ggml_conv_2d(ctx, w, x, s0, s1, p0, p1, d0, d1);
    }
    if (scale != 1.f) {
        x = ggml_scale(ctx, x, 1.f / scale);
    }
    if (b != NULL) {
        b = ggml_reshape_4d(ctx, b, 1, 1, b->ne[0], 1);
        x = ggml_add_inplace(ctx, x, b);
    }
    return x;
}

I can clean up this PR to that on wednesday

[leejet]

I think we can add a field in GGMLRunnerContext to control whether circular padding is used.

[Phylliida]

looked into this, though it was merged into llama.cpp, we'll need to wait for ggml to be synced with llama.cpp's ggml (it seems to only be synced every few weeks), or alternatively adjust the reference to point to llama.cpp's ggml folder instead of ggml

[pwilkin]

@ggerganov any chance for a GGML sync?

[ggerganov]

It's synced now - thanks

[Phylliida]

in progress, wait a few min...

[Phylliida]

Okay it is working for qwen image, should also work for other models but I haven't tested them yet, anyone else feel free to

[Phylliida]

Works for z-image too now

./bin/sd  --diffusion-model /home/bepis/Downloads/z_image_turbo-Q4_0.gguf --vae /home/bepis/Documents/cyborgism/ComfyUI/models/vae/ae.safetensors  --llm /home/bepis/Downloads/Qwen3-4B-Instruct-2507-Q4_K_M.gguf  --cfg-scale 1 --sampling-method euler -p "anime style girl with massive fluffy fennec ears and a big fluffy tail blonde messy long hair blue eyes wearing a maid outfit with a long black gold leaf pattern dress and a white apron, it is a postcard held by a hand in front of a beautiful realistic city at sunset and there is cursive writing that says \"Seamless ZImage\"" --circular --seed 42 --width 1024 --height 1024 --steps 8

(You can use this to test out seamless textures)

[SkutteOleg]

Is there any way to make it circular only horizontally? To generate equirectangular 360 panoramas, for example?

[Phylliida]

Is there any way to make it circular only horizontally? To generate equirectangular 360 panoramas, for example?

In principle it would be fairly easy, but does require an additional pr to llama.cpp

[Phylliida]

Actually, it is easier than that, we can just adjust the rope. I can add an option for that....

[Phylliida]

Ok added a -circularx and -circulary parameter to just be on those axes

./bin/sd  --diffusion-model /home/bepis/Downloads/z_image_turbo-Q4_0.gguf --vae /home/bepis/Documents/cyborgism/ComfyUI/models/vae/ae.safetensors  --llm /home/bepis/Downloads/Qwen3-4B-Instruct-2507-Q4_K_M.gguf  --cfg-scale 1 --sampling-method euler -p "anime style girl with massive fluffy fennec ears and a big fluffy tail blonde messy long hair blue eyes wearing a maid outfit with a long black gold leaf pattern dress and a white apron, it is a postcard held by a hand in front of a beautiful realistic city at sunset and there is cursive writing that says \"Seamless ZImage on x axis\"" --circularx --seed 42 --width 1024 --height 1024 --steps 8

./bin/sd  --diffusion-model /home/bepis/Downloads/z_image_turbo-Q4_0.gguf --vae /home/bepis/Documents/cyborgism/ComfyUI/models/vae/ae.safetensors  --llm /home/bepis/Downloads/Qwen3-4B-Instruct-2507-Q4_K_M.gguf  --cfg-scale 1 --sampling-method euler -p "anime style girl with massive fluffy fennec ears and a big fluffy tail blonde messy long hair blue eyes wearing a maid outfit with a long black gold leaf pattern dress and a white apron, it is a postcard held by a hand in front of a beautiful realistic city at sunset and there is cursive writing that says \"Seamless ZImage on y axis\"" --circulary --seed 42 --width 1024 --height 1024 --steps 8

[SkutteOleg]

Thanks! Confirmed working with Z-Image-Turbo:

Doesn't work with Chroma-based models:

[Phylliida]

Doesn't work with Chroma-based models

You can try it now, but I think my implementation for chroma isn't quite right/we're still missing something because it's mostly seamless but still a little blurry around borders

[Phylliida]

Ok there's some source of non-circular I'm still missing for chroma, it mostly tiles is a little fuzzy at the edges (maybe something with the vae?), but otherwise things are refactored into -circularx or -circulary or -circular for qwen, z-image, and chroma/flux

[SkutteOleg]

Thank you very much! It's pretty much perfect with Chroma now, I almost can't find the seam:

[pwilkin]

This is just in time for me wanting to get a seamless background tile for my blog ;)