inpaint error

[xuechengjinxiu]

I try to run this project locally with Docker, but every time I use the inpaint function, the model input parameters from frontend to the backend are wrong in the dev branch. I want to know how your backend inpaint model is used. At the same time, in main branch I met mask_image and image in tensor.cat error, and image input channels Inconsistent error

[gadicc]

Hey, @xuechengjinxiu... Welcome and thanks for the issue! I have some big changes coming and definitely intend to improve the docs with better examples. In the meantime, try with one of these inputs (they differ slightly based on the model being used):

docker-diffusers-api/test.py

Lines 246 to 276 in 5c7c2f6

	test(
	"inpaint-v1-4",
	{
	"modelInputs": {
	"prompt": "a cat sitting on a bench",
	"init_image": b64encode_file("overture-creations-5sI6fQgYIuo.png"),
	"mask_image": b64encode_file("overture-creations-5sI6fQgYIuo_mask.png"),
	},
	"callInputs": {
	"MODEL_ID": "CompVis/stable-diffusion-v1-4",
	"PIPELINE": "StableDiffusionInpaintPipelineLegacy",
	"SCHEDULER": "DDIMScheduler", # Note, as of diffusers 0.3.0, no LMS yet
	},
	},
	)
	test(
	"inpaint-sd",
	{
	"modelInputs": {
	"prompt": "a cat sitting on a bench",
	"image": b64encode_file("overture-creations-5sI6fQgYIuo.png"),
	"mask_image": b64encode_file("overture-creations-5sI6fQgYIuo_mask.png"),
	},
	"callInputs": {
	"MODEL_ID": "runwayml/stable-diffusion-inpainting",
	"PIPELINE": "StableDiffusionInpaintPipeline",
	"SCHEDULER": "DDIMScheduler", # Note, as of diffusers 0.3.0, no LMS yet
	},
	},
	)

(you can also run these tests automatically like e.g. python test.py inpaint-sd which will use those default values above)

I know also that diffusers library made some changes about how image vs init_image is handled... (it's meant to still work and give a deprecation warning, but I think it was broken for a while and I'm sure which version we're using in dev branch), so I hope this all works. In our next release, we'll have automated testing for a lot of this stuff, it's what I'm busy working on now 😅