Update openvino example #234
Conversation
There was a problem hiding this comment.
Reviewable status: 0 of 4 files reviewed, 4 unresolved discussions, not enough approvals from maintainers (3 more required), not enough approvals from different teams (2 more required, approved so far: )
Examples/openvino/ubuntu24.04-openvino.dockerfile line 1 at r1 (raw file):
FROM openvino/ubuntu24_dev:2024.6.0
What are the docker images available for openvino 2024? Is there any image available which is not dev?
Examples/openvino/ubuntu24.04-openvino.dockerfile line 3 at r1 (raw file):
FROM openvino/ubuntu24_dev:2024.6.0 USER root
Why is this required now?
Examples/openvino/ubuntu24.04-openvino.dockerfile line 5 at r1 (raw file):
USER root ENV MODELS="resnet-50-tf bert-large-uncased-whole-word-masking-squad-0001 ssd_mobilenet_v1_coco"
what about the other three models brain-tumor-segmentation-0001, brain-tumor-segmentation-0002 and bert-large-uncased-whole-word-masking-squad-int8-0001?
Examples/openvino/README.md line 38 at r1 (raw file):
gsc-ubuntu24.04-openvino -i <image files> \ -m /model/<public | intel>/<model_dir>/<INT8 | FP16 | FP32>/<model_xml_file> \ -d CPU -b 1 -t 20 -nstreams 72 -nthreads 72 -nireq 72 -hint none
What is this option '-hint none' about?
There was a problem hiding this comment.
Reviewable status: 0 of 4 files reviewed, 4 unresolved discussions, not enough approvals from maintainers (2 more required), not enough approvals from different teams (2 more required, approved so far: ), "fixup! " found in commit messages' one-liners (waiting on @sahason)
Examples/openvino/README.md line 38 at r1 (raw file):
Previously, sahason wrote…
What is this option '-hint none' about?
By setting -hint none, you are indicating that the inference engine should not apply any particular optimizations or performance hints, and should run the model as in the given params.
If not set it results in this, so this is needed
Code snippet:
[Step 1/11] Parsing and validating input arguments
[ INFO ] Parsing input parameters
[ ERROR ] -nstreams, -nthreads and -pin options are fine
tune options. To use them you should explicitely set
-hint option to none. This is not OpenVINO limitation
(those options can be used in OpenVINO together),
but a benchmark_app UI rule.Examples/openvino/ubuntu24.04-openvino.dockerfile line 1 at r1 (raw file):
Previously, sahason wrote…
What are the docker images available for openvino 2024? Is there any image available which is not dev?
Here are all docker images - docker_ci/dockerfiles/ubuntu24 at master · openvinotoolkit/docker_ci
For 2024.6 we have dev and runtime pacakges, both providing openvino runtime. Could use anyone of them. What to prefer?
Examples/openvino/ubuntu24.04-openvino.dockerfile line 3 at r1 (raw file):
Previously, sahason wrote…
Why is this required now?
If we don't give this then it results in this.
For downloading models to /model this is needed
Code snippet:
13.20 PermissionError: [Errno 13] Permission denied: '/model'
------
ubuntu24.04-openvino.dockerfile:9
--------------------
8 |
9 | >>> RUN for model_name in $MODELS; do \
10 | >>> omz_downloader --name $model_name -o /model && \
11 | >>> omz_converter --name $model_name -d /model -o /model; \
12 | >>> done
13 |
--------------------Examples/openvino/ubuntu24.04-openvino.dockerfile line 5 at r1 (raw file):
Previously, sahason wrote…
what about the other three models brain-tumor-segmentation-0001, brain-tumor-segmentation-0002 and bert-large-uncased-whole-word-masking-squad-int8-0001?
The models are picked up from - open_model_zoo/models/public at master · openvinotoolkit/open_model_zoo
brain-tumor-segmentation-0001 is not found
brain-tumor-segmentation-0002 is there
bert-large-uncased-whole-word-masking-squad-int8-0001 is there
So keeping
Code snippet:
ENV MODELS="resnet-50-tf \
bert-large-uncased-whole-word-masking-squad-0001 \
ssd_mobilenet_v1_coco \
brain-tumor-segmentation-0002 \
bert-large-uncased-whole-word-masking-squad-int8-0001"
adarshan-intel
force-pushed
the
adarsh/openvino_ex
branch
from
fae15b4 to
254c286
Compare
There was a problem hiding this comment.
Reviewed 3 of 4 files at r1, 1 of 1 files at r2, all commit messages.
Reviewable status: all files reviewed, 9 unresolved discussions, not enough approvals from maintainers (2 more required), not enough approvals from different teams (2 more required, approved so far: ), "fixup! " found in commit messages' one-liners (waiting on @sahason)
Examples/openvino/ubuntu24.04-openvino.dockerfile line 1 at r1 (raw file):
Previously, adarshan-intel (Adarsh Anand) wrote…
Here are all docker images - docker_ci/dockerfiles/ubuntu24 at master · openvinotoolkit/docker_ci
For 2024.6 we have dev and runtime pacakges, both providing openvino runtime. Could use anyone of them. What to prefer?
I'd prefer using ubuntu24_runtime if it works (I guess it's slimmer?).
Examples/openvino/ubuntu24.04-openvino.dockerfile line 3 at r1 (raw file):
Previously, adarshan-intel (Adarsh Anand) wrote…
If we don't give this then it results in this.
For downloading models to /model this is needed
Why not download the models to a dir that doesn't require root access? Alternatively, could we set the permissions for that dir explicitly instead of switching to a ROOT user?
Examples/openvino/README.md line 68 at r2 (raw file):
## Running the benchmark natively To run the benchmark in a native Docker container (outside Gramine), run the above commands with the following modifications:
Why this change? Shouldn't we wrap to the line limit?
Code quote:
To run the benchmark in a native Docker container (outside Gramine), run the above commands with the following modifications:Examples/openvino/README.md line 75 at r2 (raw file):
The above `docker run` commands are for a 36-core system. Please check https://github.com/gramineproject/examples/blob/master/openvino/README.md for an
Well, the content of that doc doesn't match w/ this then.
Code quote:
https://github.com/gramineproject/examples/blob/master/openvino/README.mdExamples/openvino/ubuntu24.04-openvino.dockerfile line 3 at r2 (raw file):
FROM openvino/ubuntu24_dev:2024.6.0 USER root
trailing space
Code quote:
·Examples/openvino/ubuntu24.04-openvino.dockerfile line 5 at r2 (raw file):
USER root ENV MODELS="resnet-50-tf \
Why do we need this ENV? Can't we just use for...do... in RUN like we did before?
Code quote:
ENV MODELSExamples/openvino/ubuntu24.04-openvino.dockerfile line 16 at r2 (raw file):
done ENTRYPOINT ["/opt/intel/openvino/samples/cpp/samples_bin/samples_bin/benchmark_app"]
Just wanted to double-check, is this a typo or is it supposed to look like this?
Code quote:
samples_bin/samples_bin
There was a problem hiding this comment.
Reviewable status: all files reviewed, 9 unresolved discussions, not enough approvals from maintainers (2 more required), not enough approvals from different teams (2 more required, approved so far: ), "fixup! " found in commit messages' one-liners (waiting on @kailun-qin and @sahason)
Examples/openvino/README.md line 68 at r2 (raw file):
Previously, kailun-qin (Kailun Qin) wrote…
Why this change? Shouldn't we wrap to the line limit?
Done.
Examples/openvino/README.md line 75 at r2 (raw file):
Previously, kailun-qin (Kailun Qin) wrote…
Well, the content of that doc doesn't match w/ this then.
What should be done?
Should I remove this note?
Examples/openvino/ubuntu24.04-openvino.dockerfile line 1 at r1 (raw file):
Previously, kailun-qin (Kailun Qin) wrote…
I'd prefer using
ubuntu24_runtimeif it works (I guess it's slimmer?).
For ubuntu24_runtime image it does not find the downloader and converter tool?
So keeping dev, is this ok?
Or do you want to me to install openmodelzoo separately?
docker_ci/dockerfiles/ubuntu24/openvino_cg_dev_2024.6.0.dockerfile at master · openvinotoolkit/docker_ci
Ig that will be longer
Code snippet:
0.310 + omz_downloader --name bert-large-uncased-whole-word-masking-squad-int8-0001 -o /model
0.311 /bin/bash: line 1: omz_downloader: command not found
------
ubuntu24.04-openvino.dockerfile:5
--------------------
4 |
5 | >>> RUN for model_name in resnet-50-tf \
6 | >>> bert-large-uncased-whole-word-masking-squad-0001 \
7 | >>> ssd_mobilenet_v1_coco \
8 | >>> brain-tumor-segmentation-0002 \
9 | >>> bert-large-uncased-whole-word-masking-squad-int8-0001; do \
10 | >>> omz_downloader --name $model_name -o /model && \
11 | >>> omz_converter --name $model_name -d /model -o /model; \
12 | >>> done
13 | Examples/openvino/ubuntu24.04-openvino.dockerfile line 3 at r2 (raw file):
Previously, kailun-qin (Kailun Qin) wrote…
trailing space
Done.
Examples/openvino/ubuntu24.04-openvino.dockerfile line 5 at r2 (raw file):
Previously, kailun-qin (Kailun Qin) wrote…
Why do we need this ENV? Can't we just use
for...do...in RUN like we did before?
Done.
Examples/openvino/ubuntu24.04-openvino.dockerfile line 16 at r2 (raw file):
Previously, kailun-qin (Kailun Qin) wrote…
Just wanted to double-check, is this a typo or is it supposed to look like this?
I went inside the docker image and checked.
This is how it is supposed to be
There was a problem hiding this comment.
Reviewable status: 2 of 4 files reviewed, 9 unresolved discussions, not enough approvals from maintainers (2 more required), not enough approvals from different teams (2 more required, approved so far: ), "fixup! " found in commit messages' one-liners (waiting on @kailun-qin and @sahason)
Examples/openvino/ubuntu24.04-openvino.dockerfile line 3 at r1 (raw file):
Previously, kailun-qin (Kailun Qin) wrote…
Why not download the models to a dir that doesn't require root access? Alternatively, could we set the permissions for that dir explicitly instead of switching to a ROOT user?
I was thinking we can create another user and give permission for it of /models folder?
Is that fine ?
|
@kailun-qin Any updates? |
There was a problem hiding this comment.
Reviewed 2 of 2 files at r3, all commit messages.
Reviewable status: all files reviewed, 5 unresolved discussions, not enough approvals from maintainers (2 more required), not enough approvals from different teams (2 more required, approved so far: ), "fixup! " found in commit messages' one-liners (waiting on @adarshan-intel and @sahason)
Examples/openvino/README.md line 75 at r2 (raw file):
Previously, adarshan-intel (Adarsh Anand) wrote…
What should be done?
Should I remove this note?
I think it's still useful to keep the link, as it might be helpful for users. Just add a note mentioning that it can be outdated, but the performance suggestions provided there are generally applicable here.
Examples/openvino/ubuntu24.04-openvino.dockerfile line 1 at r1 (raw file):
Previously, adarshan-intel (Adarsh Anand) wrote…
For ubuntu24_runtime image it does not find the downloader and converter tool?
So keeping dev, is this ok?
Or do you want to me to install openmodelzoo separately?
docker_ci/dockerfiles/ubuntu24/openvino_cg_dev_2024.6.0.dockerfile at master · openvinotoolkit/docker_ciIg that will be longer
Ok, in such case, I'm fine w/ keeping dev.
Examples/openvino/ubuntu24.04-openvino.dockerfile line 3 at r1 (raw file):
Previously, adarshan-intel (Adarsh Anand) wrote…
I was thinking we can create another user and give permission for it of /models folder?
Is that fine ?
Yes, I'm fine w/ it.
There was a problem hiding this comment.
Reviewable status: 3 of 4 files reviewed, 5 unresolved discussions, not enough approvals from maintainers (2 more required), not enough approvals from different teams (2 more required, approved so far: ), "fixup! " found in commit messages' one-liners (waiting on @kailun-qin and @sahason)
Examples/openvino/README.md line 75 at r2 (raw file):
Previously, kailun-qin (Kailun Qin) wrote…
I think it's still useful to keep the link, as it might be helpful for users. Just add a note mentioning that it can be outdated, but the performance suggestions provided there are generally applicable here.
Added Note.
Examples/openvino/ubuntu24.04-openvino.dockerfile line 1 at r1 (raw file):
Previously, kailun-qin (Kailun Qin) wrote…
Ok, in such case, I'm fine w/ keeping dev.
Done.
Examples/openvino/ubuntu24.04-openvino.dockerfile line 3 at r1 (raw file):
Previously, kailun-qin (Kailun Qin) wrote…
Yes, I'm fine w/ it.
I tried creating a regular user and giving permissions of /models dir to it for installing models. But while running the benchmark tests the user is not able to access /dev/sgx_enclave.
RUN useradd -m -s /bin/bash modeluser
RUN mkdir /model && chown modeluser:modeluser /model
USER modeluser
Should I keep it as USER root? If not, what could be done here?
Code snippet:
intel@intel-M50CYP2SBSTD:~/adarsh/gsc$ docker run --rm --device /dev/sgx_enclave gsc-ubuntu24.04-openvino \
-m /model/public/resnet-50-tf/FP16/resnet-50-tf.xml \
-d CPU -b 1 -t 20 -nstreams 2 -nthreads 2 -nireq 2 -hint none
Gramine is starting. Parsing TOML manifest file, this may take some time...
error: Cannot open /dev/sgx_enclave (permission denied). This may happen because the current user has insufficient permissions to this device.
error: load_enclave() failed with error: Permission denied (EACCES)
There was a problem hiding this comment.
Reviewed 1 of 1 files at r4, all commit messages.
Reviewable status: all files reviewed, 4 unresolved discussions, not enough approvals from maintainers (1 more required), not enough approvals from different teams (1 more required, approved so far: Intel), "fixup! " found in commit messages' one-liners (waiting on @sahason)
Examples/openvino/ubuntu24.04-openvino.dockerfile line 3 at r1 (raw file):
Previously, adarshan-intel (Adarsh Anand) wrote…
I tried creating a regular user and giving permissions of /models dir to it for installing models. But while running the benchmark tests the user is not able to access /dev/sgx_enclave.
RUN useradd -m -s /bin/bash modeluser RUN mkdir /model && chown modeluser:modeluser /model USER modeluserShould I keep it as USER root? If not, what could be done here?
Ah, ok. Since we need to access /dev/sgx_* anyway and the previous Dockerfile also uses a root user, I'm fine w/ keeping it as is.
Signed-off-by: Adarsh <[email protected]>
Signed-off-by: Adarsh Anand <[email protected]>
Signed-off-by: Adarsh Anand <[email protected]>
Signed-off-by: Adarsh Anand <[email protected]>
Signed-off-by: Adarsh Anand <[email protected]>
Signed-off-by: Adarsh <[email protected]>
adarshan-intel
force-pushed
the
adarsh/openvino_ex
branch
from
63d68e5 to
90a5a21
Compare
Description of the changes
According to this - #226 (review)
Docker optimization: Replaced manual Ubuntu setup with OpenVINO CI Dockerfile
(openvino_cg_dev_2024.6.0.dockerfile) :
Model management improvements:
How to test this PR?
CI
This change is