Skip to content

Update image generation demo #4266

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
atobiszei wants to merge 1 commit into
base: main
Choose a base branch
from
+13 −46
Open

Update image generation demo #4266

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 13 additions & 46 deletions demos/image_generation/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ This command pulls the `OpenVINO/stable-diffusion-v1-5-int8-ov` quantized model

> **NOTE:** Optionally, to only download the model and omit the serving part, use `--pull` parameter.

### CPU
### GPU

::::{tab-set}
:::{tab-item} Docker (Linux)
Expand All @@ -45,13 +45,15 @@ Start docker container:
```bash
mkdir -p ${HOME}/models

docker run -d --rm --user $(id -u):$(id -g) -p 8000:8000 -v ${HOME}/models:/models:rw \
docker run -d --rm -p 8000:8000 -v ${HOME}/models:/models:rw \
--user $(id -u):$(id -g) --device /dev/dri --group-add=$(stat -c "%g" /dev/dri/render* | head -n 1) \
-e http_proxy=$http_proxy -e https_proxy=$https_proxy -e no_proxy=$no_proxy \
openvino/model_server:latest \
openvino/model_server:latest-gpu \
--rest_port 8000 \
--model_repository_path /models \
--task image_generation \
--source_model OpenVINO/stable-diffusion-v1-5-int8-ov
--source_model OpenVINO/stable-diffusion-v1-5-int8-ov \
--target_device GPU
```
:::

Expand All @@ -65,45 +67,7 @@ Assuming you have unpacked model server package, make sure to:

as mentioned in [deployment guide](../../docs/deploying_server_baremetal.md), in every new shell that will start OpenVINO Model Server.


```bat
if not exist c:\models mkdir c:\models

ovms --rest_port 8000 ^
--model_repository_path c:\models ^
--task image_generation ^
--source_model OpenVINO/stable-diffusion-v1-5-int8-ov
```
:::

::::

### GPU

::::{tab-set}
:::{tab-item} Docker (Linux)
:sync: docker
In case you want to use Intel GPU device to run the generation, add extra docker parameters `--device /dev/dri --group-add=$(stat -c "%g" /dev/dri/render* | head -n 1)` to `docker run` command, use the docker image with GPU support. Export the models with precision matching the GPU capacity and adjust pipeline configuration.
It can be applied using the commands below:
```bash
mkdir -p ${HOME}/models

docker run -d --rm -p 8000:8000 -v ${HOME}/models:/models:rw \
--user $(id -u):$(id -g) --device /dev/dri --group-add=$(stat -c "%g" /dev/dri/render* | head -n 1) \
-e http_proxy=$http_proxy -e https_proxy=$https_proxy -e no_proxy=$no_proxy \
openvino/model_server:latest-gpu \
--rest_port 8000 \
--model_repository_path /models \
--task image_generation \
--source_model OpenVINO/stable-diffusion-v1-5-int8-ov \
--target_device GPU
```
:::

:::{tab-item} Bare metal (Windows)
:sync: bare-metal

If you run on GPU make sure to have appropriate drivers installed, so the device is accessible for the model server.
Make sure to have appropriate GPU drivers installed, so the device is accessible for the model server.

```bat
if not exist c:\models mkdir c:\models
Expand Down Expand Up @@ -227,10 +191,12 @@ If you already have a model on disk (downloaded via Option 1 with `--pull`, or v

```bash
docker run -d --rm -p 8000:8000 -v ${HOME}/models:/models:rw \
openvino/model_server:latest \
--device /dev/dri --group-add=$(stat -c "%g" /dev/dri/render* | head -n 1) \
openvino/model_server:latest-gpu \
--rest_port 8000 \
--model_name OpenVINO/stable-diffusion-v1-5-int8-ov \
--model_path /models/OpenVINO/stable-diffusion-v1-5-int8-ov
--model_path /models/OpenVINO/stable-diffusion-v1-5-int8-ov \
--target_device GPU
```
:::

Expand All @@ -240,7 +206,8 @@ docker run -d --rm -p 8000:8000 -v ${HOME}/models:/models:rw \
```bat
ovms --rest_port 8000 ^
--model_name OpenVINO/stable-diffusion-v1-5-int8-ov ^
--model_path c:\models\OpenVINO\stable-diffusion-v1-5-int8-ov
--model_path c:\models\OpenVINO\stable-diffusion-v1-5-int8-ov ^
--target_device GPU
```
:::

Expand Down