[HELP NEEDED]Getting bad quality output when executing stable-diffusion kernel

[StudenteChamp2]

I am adding ai based image enhancement to my 3D rendering software. I found your wonderful framework and decided to integrate it. Here my test image:

Here the output of Stable-diffusion:

For comparison here the output of Google Gemini Api:

The difference is crazy. Here my implementation code:

`

void OfflineRenderer::performRenderImageEnhancing()
{
	_ASSERT(m_inputImage->isFloatingPointImage());
	_ASSERT(m_inputImage->getNbChannels() == 3);

	// First build the RGB8 image from RGB32F to feed the enhancer.
	const auto imageSrc = Graphics::Texture::Helper::create_RGB8FromRGB32F(*m_inputImage);

	// Build the stable diffusion Api native input image.
	sd_image_t nativeInputImage;
	{
		nativeInputImage.width = getRenderingWidth();
		nativeInputImage.height = getRenderingHeight();
		nativeInputImage.channel = 3u;
		nativeInputImage.data = imageSrc->getRawData();
	}

	// Init the context parameters.
	sd_ctx_t* ctx = nullptr;
	{
		sd_ctx_params_t sd_ctx_params = {};
		sd_ctx_params.model_path = "D:\\Stable-diffusion\\Models\\v1-5-pruned-emaonly.safetensors";
		//sd_ctx_params.vae_path = "/path/to/your/vae.safetensors";
		sd_ctx_params.n_threads =  Hardware::HardwareInformationSingleton::instance()->nbOfPhysicalCores();
		sd_ctx_params.wtype = sd_type_t::SD_TYPE_F16;
		sd_ctx_params.backend = "cpu";
		//sd_ctx_params.backend = "vulkan0";

		ctx = new_sd_ctx(&sd_ctx_params);
		_ASSERT(ctx != nullptr);
	}

	// Configure generation parameters.
	sd_img_gen_params_t sd_img_gen_params;
	{
		sd_img_gen_params_init(&sd_img_gen_params);
		sd_img_gen_params.prompt = "Generate a beautiful daytime bathroom at 12pm.";
		sd_img_gen_params.negative_prompt = "";
		sd_img_gen_params.clip_skip = 1;
		sd_img_gen_params.init_image = nativeInputImage;
		sd_img_gen_params.sample_params.sample_method = EULER_A_SAMPLE_METHOD;

		sd_img_gen_params.width = getRenderingWidth();
		sd_img_gen_params.height = getRenderingHeight();
		sd_img_gen_params.strength = 0.75f;
		sd_img_gen_params.seed = 9999;
		sd_img_gen_params.control_strength = 0.9f;
	}

	// Set the generation callback.
	sd_set_progress_callback(ProgressCallback, &m_stopRender);

	sd_image_t* result = nullptr;
	try
	{
		result = generate_image(ctx, &sd_img_gen_params);
		_ASSERT(result && result[0].data);
	}
	catch (const std::runtime_error&)
	{
		goto FreeMemory;
	}

	_ASSERT(result->width == imageSrc->getWidth());
	_ASSERT(result->height == imageSrc->getHeight());
	_ASSERT(result->channel == imageSrc->getNbChannels());

	// Read back result.
	const uint32_t nbTasks = getRenderingWidth() * getRenderingHeight();
	tbb::parallel_for(size_t(0), size_t(nbTasks), [&](size_t tbbIdx) {

		if (m_stopRender)
		{
			tbb::task::current_context()->cancel_group_execution();
			return;
		}

		const uint32_t pixelLinearIdx = (uint32_t)tbbIdx;
		const uint32_t pixelStartInArray = (uint32_t)tbbIdx * 3u;
		const uint32_t pixelPosX = (uint32_t)(pixelLinearIdx % getRenderingWidth());
		const uint32_t pixelPosY = (uint32_t)(pixelLinearIdx / getRenderingWidth());
		const Math::Uvec2 pixelPos = Math::Uvec2(pixelPosX, pixelPosY);

		const uint8_t R = result[0].data[pixelStartInArray];
		const uint8_t G = result[0].data[pixelStartInArray + 1u];
		const uint8_t B = result[0].data[pixelStartInArray + 2u];

		// OpenCV layout is BGR
		const RGBFColor outputColor = RGBFColor((float)B/ 255.0f, (float)G/255.0f, (float)R/ 255.0f);
		m_outputImage->setPixelFromPosition(outputColor, pixelPos);
	});


	// Cleanup resources.
FreeMemory:;
	free_sd_ctx(ctx);
	if (result)
	{
		free(result[0].data);
		free(result);
	}
}`

I think i am doing something wrong but what?
Thanks for helping!

[stduhpf]

• Stable diffusion 1.5 is an ancient model (from 2022), comparing it to Gemini is really unfair
• It's also only trained to generate images of 512x512 resolution or 0.25 Mpx
• Img2Img mode isn't good at preserving details, but you could try lowering the strength to preserve more of the original image

To edit images like that you should probably use a more recent model like Flux2[Klein], Qwen Image Edit or LongCat Image Edit. If you want something lite and fast like sd1.5, you can give instruct-pix2pix a shot, it is a sd1.5 fine-tune that is able to use reference images to control generation.

[StudenteChamp2]

Thank you very much <3

[wbruna]

@StudenteChamp2 , a quick test with a quantized Flux.2 Klein 4b in edit mode:

sd-cli --backend Vulkan --diffusion-model flux-2-klein-4b-Q8_0.gguf --llm Qwen3-4B-UD-Q4_K_XL.gguf --vae flux2.full_encoder_small_decoder.safetensors --vae-conv-direct -p 'turn the image into a high quality photograph' -r bathroom_3d.png -o bathroom.png --cfg-scale 1 --steps 4 --fa

[StudenteChamp2]

This is sensational and the type of output i am expecting.

[StudenteChamp2]

I am able to get the same results as you with the CLI.

But I am getting a "cartoon" output with my C++ implementation and i dont know why.
cartoon.bmp

I attached Visual Studio to the CLI process to check the diffusion parameters with breakpoints. I then set the same values to my C++ code. Code now look like this:

`

void OfflineRenderer::performRenderImageEnhancing()
{
	_ASSERT(m_inputImage->isFloatingPointImage());
	_ASSERT(m_inputImage->getNbChannels() == 3);

	// First build the RGB8 image from RGB32F to feed the enhancer.
	const auto imageSrc = Graphics::Texture::Helper::create_RGB8FromRGB32F(*m_inputImage);

	// Build the stable diffusion Api native input image.
	sd_image_t nativeInputImage;
	{
		nativeInputImage.width = getRenderingWidth();
		nativeInputImage.height = getRenderingHeight();
		nativeInputImage.channel = 3u;
		nativeInputImage.data = imageSrc->getRawData();
	}

	// Init the context parameters.
	sd_ctx_t* ctx = nullptr;
	{
			sd_ctx_params_t sd_ctx_params = {};
			sd_ctx_params.diffusion_model_path = "D:\\Stable-diffusion\\Models\\flux-2-klein-4b-Q8_0.gguf";
			sd_ctx_params.llm_path = "D:\\Stable-diffusion\\Models\\Qwen3-4B-UD-Q4_K_XL.gguf";
			sd_ctx_params.vae_path = "D:\\Stable-diffusion\\Models\\flux2.full_encoder_small_decoder.safetensors";
			sd_ctx_params.vae_conv_direct = true;
			//sd_ctx_params.flash_attn = true;

			sd_ctx_params.n_threads = 12;
			sd_ctx_params.backend = "cpu";
			sd_ctx_params.wtype = sd_type_t::SD_TYPE_Q8_0;
			//sd_ctx_params.wtype = sd_type_t::SD_TYPE_COUNT; // CRACHES

			sd_ctx_params.rng_type = CUDA_RNG;
			sd_ctx_params.sampler_rng_type = RNG_TYPE_COUNT;
			sd_ctx_params.prediction = PREDICTION_COUNT;
			sd_ctx_params.lora_apply_mode = LORA_APPLY_AUTO;
			sd_ctx_params.vae_conv_direct = true;
			sd_ctx_params.chroma_use_dit_mask = true;
			sd_ctx_params.chroma_t5_mask_pad = 1;
			sd_ctx_params.vae_format = SD_VAE_FORMAT_AUTO;
			sd_ctx_params.max_vram = nullptr;
			sd_ctx_params.stream_layers = false;
			sd_ctx_params.params_backend = "";
			sd_ctx_params.rpc_servers = "";

			ctx = new_sd_ctx(&sd_ctx_params);
			_ASSERT(ctx != nullptr);
	}

	// Configure generation parameters.
	sd_img_gen_params_t sd_img_gen_params;
	{
			sd_img_gen_params_init(&sd_img_gen_params);
			sd_img_gen_params.prompt = "turn the image into a high quality photograph";
			sd_img_gen_params.negative_prompt = "";
			sd_img_gen_params.clip_skip = 1;
			sd_img_gen_params.init_image = nativeInputImage;
			sd_img_gen_params.sample_params.sample_method = EULER_A_SAMPLE_METHOD;

			sd_img_gen_params.width = getRenderingWidth();
			sd_img_gen_params.height = getRenderingHeight();
			sd_img_gen_params.strength = 0.75f;
			sd_img_gen_params.seed = 42;
			sd_img_gen_params.control_strength = 0.9f;

			sd_img_gen_params.ref_images = &nativeInputImage;
			sd_img_gen_params.ref_images_count = 1;
			sd_img_gen_params.auto_resize_ref_image = false;
			sd_img_gen_params.increase_ref_index = false;

			sd_image_t generated_mask = { 0, 0, 1, nullptr };
			generated_mask.data = (uint8_t*)malloc(getRenderingWidth() * getRenderingHeight());
			generated_mask.width = getRenderingWidth();
			generated_mask.height = getRenderingHeight();
			memset(generated_mask.data, 255, getRenderingWidth() * getRenderingHeight());
			sd_img_gen_params.mask_image = generated_mask;

			sd_img_gen_params.sample_params.scheduler = DISCRETE_SCHEDULER;

			std::vector<int> skip_layers = { 7, 8, 9 };
			sd_img_gen_params.sample_params.guidance.slg.layers = skip_layers.empty() ? nullptr : skip_layers.data();
			sd_img_gen_params.sample_params.guidance.slg.layer_count = skip_layers.size();
			//sd_img_gen_params.sample_params.guidance.img_cfg = 1.0f;

			sd_img_gen_params.sample_params.custom_sigmas = nullptr;
			sd_img_gen_params.sample_params.custom_sigmas_count = 0;
			sd_img_gen_params.sample_params.extra_sample_args = nullptr;
			//sd_img_gen_params.sample_params.sample_steps = 4;

			sd_img_gen_params.batch_count = 1;
			sd_img_gen_params.control_image = {};


			sd_pm_params_t pm_params = {
				nullptr,
				0,
				nullptr,
				20 };

			sd_img_gen_params.pm_params = pm_params;

			sd_img_gen_params.vae_tiling_params.enabled = false;
			sd_img_gen_params.vae_tiling_params.temporal_tiling = false;
			sd_img_gen_params.vae_tiling_params.tile_size_x = 0;
			sd_img_gen_params.vae_tiling_params.tile_size_y = 0;
			sd_img_gen_params.vae_tiling_params.target_overlap = 0.5f;
			sd_img_gen_params.vae_tiling_params.rel_size_x = 0;
			sd_img_gen_params.vae_tiling_params.rel_size_y = 0;
			sd_img_gen_params.vae_tiling_params.extra_tiling_args = nullptr;

			sd_img_gen_params.cache.mode = SD_CACHE_DISABLED;

			sd_img_gen_params.hires.enabled = false;
			sd_img_gen_params.hires.upscaler = {};
			sd_img_gen_params.hires.model_path = nullptr;
			sd_img_gen_params.hires.scale = 2.0f;
			sd_img_gen_params.hires.target_width = 0;
			sd_img_gen_params.hires.target_height = 0;
			sd_img_gen_params.hires.steps = 0;
			sd_img_gen_params.hires.denoising_strength = 0.7f;
			sd_img_gen_params.hires.upscale_tile_size = 128;
			sd_img_gen_params.hires.custom_sigmas = nullptr;
			sd_img_gen_params.hires.custom_sigmas_count = 0;
	}

	// Set the generation callback.
	sd_set_progress_callback(ProgressCallback, &m_stopRender);

	sd_image_t* result = nullptr;
	try
	{
		result = generate_image(ctx, &sd_img_gen_params);
		_ASSERT(result && result[0].data);
	}
	catch (const std::runtime_error&)
	{
		goto FreeMemory;
	}

	_ASSERT(result->width == imageSrc->getWidth());
	_ASSERT(result->height == imageSrc->getHeight());
	_ASSERT(result->channel == imageSrc->getNbChannels());

	// Read back result.
	const uint32_t nbTasks = getRenderingWidth() * getRenderingHeight();
	tbb::parallel_for(size_t(0), size_t(nbTasks), [&](size_t tbbIdx) {

		if (m_stopRender)
		{
			tbb::task::current_context()->cancel_group_execution();
			return;
		}

		const uint32_t pixelLinearIdx = (uint32_t)tbbIdx;
		const uint32_t pixelStartInArray = (uint32_t)tbbIdx * 3u;
		const uint32_t pixelPosX = (uint32_t)(pixelLinearIdx % getRenderingWidth());
		const uint32_t pixelPosY = (uint32_t)(pixelLinearIdx / getRenderingWidth());
		const Math::Uvec2 pixelPos = Math::Uvec2(pixelPosX, pixelPosY);

		const uint8_t R = result[0].data[pixelStartInArray];
		const uint8_t G = result[0].data[pixelStartInArray + 1u];
		const uint8_t B = result[0].data[pixelStartInArray + 2u];

		const RGBFColor outputColor = RGBFColor((float)R/ 255.0f, (float)G/255.0f, (float)B/ 255.0f);
		m_outputImage->setPixelFromPosition(outputColor, pixelPos);
	});


	// Cleanup resources.
	FreeMemory:;
	free_sd_ctx(ctx);
	if (result)
	{
		free(result[0].data);
		free(result);
	}
}

`

What am I missing? Thanks for helping!

[stduhpf]

Looks like you didn't set the sample_params.txt_cfg. I think by default it is set to a value of 7, which is too much for Flux2, that could very well explain the oversaturation we see in your image. You can set it to 1, and that should help.

[StudenteChamp2]

Oh i forgot to change the sampling method;
This
sd_img_gen_params.sample_params.sample_method = EULER_A_SAMPLE_METHOD;
Now becomes this:

sd_img_gen_params.sample_params.sample_method = EULER_SAMPLE_METHOD;
And it is way better:
way_better.bmp

I am getting close to the reference but i am not yet there.

[wbruna]

You do not need to initialize all those fields: just call the init functions (sd_ctx_params_init and sd_img_gen_params_init), which would also get you the proper non-zero default values. After that, the values that you'll need to change for Klein 4b should roughly match the values from the command line (basically the models, reference image, sample_params.txt_cfg as stduhpf pointed out, and step count).

--vae-conv-direct and --fa from my example are not strictly needed: they mainly reduce memory requirements, and may help performance depending on the device.

[StudenteChamp2]

I was missing the sd_ctx_params_init(&sd_ctx_params); call
And as suggested in previous answer i should set sd_img_gen_params.sample_params.guidance.txt_cfg to 1
Also as suggested i will rely on the initialization functions rather than setting values manually.

I now get this at 1024x1024:

It is beautiful but i have some artifacts on the white walls:

Changed the quantization to SD_TYPE_F32 and the artefacts are gone

`

void OfflineRenderer::performRenderImageEnhancing()
{
	_ASSERT(m_inputImage->isFloatingPointImage());
	_ASSERT(m_inputImage->getNbChannels() == 3);

	// First build the RGB8 image from RGB32F to feed the enhancer.
	const auto imageSrc = Graphics::Texture::Helper::create_RGB8FromRGB32F(*m_inputImage);

	// Build the stable diffusion Api native input image.
	sd_image_t nativeInputImage;
	{
		nativeInputImage.width = getRenderingWidth();
		nativeInputImage.height = getRenderingHeight();
		nativeInputImage.channel = 3u;
		nativeInputImage.data = imageSrc->getRawData();
	}

// Init the context parameters.
	sd_ctx_t* ctx = nullptr;
	{
		sd_ctx_params_t sd_ctx_params = {};
		sd_ctx_params_init(&sd_ctx_params);

		sd_ctx_params.diffusion_model_path = "D:\\Stable-diffusion\\Models\\flux-2-klein-4b-Q8_0.gguf";
		sd_ctx_params.llm_path = "D:\\Stable-diffusion\\Models\\Qwen3-4B-UD-Q4_K_XL.gguf";
		sd_ctx_params.vae_path = "D:\\Stable-diffusion\\Models\\flux2.full_encoder_small_decoder.safetensors";
		//sd_ctx_params.vae_conv_direct = true;
		//sd_ctx_params.flash_attn = true;

		sd_ctx_params.n_threads = 12;
		sd_ctx_params.backend = "cpu";
		sd_ctx_params.wtype = sd_type_t::SD_TYPE_Q8_0;

		ctx = new_sd_ctx(&sd_ctx_params);
		_ASSERT(ctx != nullptr);
	}

	// Configure generation parameters.
	sd_img_gen_params_t sd_img_gen_params;
	{
		sd_img_gen_params_init(&sd_img_gen_params);

		sd_img_gen_params.prompt = "turn the image into a high quality photograph";
		sd_img_gen_params.negative_prompt = "";
		sd_img_gen_params.sample_params.sample_method = EULER_SAMPLE_METHOD;

		sd_img_gen_params.width = getRenderingWidth();
		sd_img_gen_params.height = getRenderingHeight();
		sd_img_gen_params.strength = 0.75f;
		sd_img_gen_params.seed = 42;
		sd_img_gen_params.control_strength = 0.9f;

		sd_img_gen_params.ref_images = &nativeInputImage;
		sd_img_gen_params.ref_images_count = 1;
		sd_img_gen_params.auto_resize_ref_image = false;
		sd_img_gen_params.increase_ref_index = false;

		sd_img_gen_params.sample_params.guidance.txt_cfg = 1.0f;
		sd_img_gen_params.sample_params.sample_steps = 4;
	}

	// Set the generation callback.
	sd_set_progress_callback(ProgressCallback, &m_stopRender);

	sd_image_t* result = nullptr;
	try
	{
		result = generate_image(ctx, &sd_img_gen_params);
		_ASSERT(result && result[0].data);
	}
	catch (const std::runtime_error&)
	{
		goto FreeMemory;
	}

	_ASSERT(result->width == imageSrc->getWidth());
	_ASSERT(result->height == imageSrc->getHeight());
	_ASSERT(result->channel == imageSrc->getNbChannels());

	// Read back result.
	const uint32_t nbTasks = getRenderingWidth() * getRenderingHeight();
	tbb::parallel_for(size_t(0), size_t(nbTasks), [&](size_t tbbIdx) {

		if (m_stopRender)
		{
			tbb::task::current_context()->cancel_group_execution();
			return;
		}

		const uint32_t pixelLinearIdx = (uint32_t)tbbIdx;
		const uint32_t pixelStartInArray = (uint32_t)tbbIdx * 3u;
		const uint32_t pixelPosX = (uint32_t)(pixelLinearIdx % getRenderingWidth());
		const uint32_t pixelPosY = (uint32_t)(pixelLinearIdx / getRenderingWidth());
		const Math::Uvec2 pixelPos = Math::Uvec2(pixelPosX, pixelPosY);

		const uint8_t R = result[0].data[pixelStartInArray];
		const uint8_t G = result[0].data[pixelStartInArray + 1u];
		const uint8_t B = result[0].data[pixelStartInArray + 2u];

		const RGBFColor outputColor = RGBFColor((float)R/ 255.0f, (float)G/255.0f, (float)B/ 255.0f);
		m_outputImage->setPixelFromPosition(outputColor, pixelPos);
	});


// Cleanup resources.
FreeMemory:;
free_sd_ctx(ctx);
if (result)
{
	free(result[0].data);
	free(result);
}

}

`