Description
I am trying to convert an image using basic soci convert with default settings. But when I try to pull the image using soci snapshotter I get this error -
sudo nerdctl run --platform linux/amd64 --snapshotter soci --net host --rm -it --platform linux/amd64 xxx-container-registry.xxx.com/r1/r2:0.1.0-soci /bin/bash
/bin/bash: error while loading shared libraries: /app/var/data/conda-env/lib/libtinfo.so.6: cannot read file data: Input/output error
Soci error log -
Jan 08 18:01:20 lima-default soci-snapshotter-grpc[23927]: {"digest":"sha256:f3f9d548535bfa8739e2b6c6464f1ff5c94bbb3ec28cf6xxxx",
"error":"file.Read: error reading tar header at 51757568, size 512: EOF",
"fetchedPercent":0,"fetchedSize":0,
"level":"error","msg":"statFile error","size":89539606}
In the ztoc info I see that its at a offset of 199165440 with span 0 whereas it should have been at span 50 assuming 4mb spans? -
{
"filename": "app/var/data/conda-env/lib/libtinfo.so.6",
"offset": 199165440,
"size": 0,
"type": "symlink",
"start_span": 0,
"end_span": 0
},
{
"filename": "app/var/data/conda-env/lib/libicuuc.so.75",
"offset": 199165952,
"size": 0,
"type": "symlink",
"start_span": 0,
"end_span": 0
},
Here is the end of the ztoc, all are at span 0 while the whole uncompressed layer is 250Mb+
{
"filename": "app/var/data/conda-env/ssl/misc/CA.pl",
"offset": 255863296,
"size": 13398,
"type": "reg",
"start_span": 0,
"end_span": 0
},
{
"filename": "app/var/data/conda-env/ssl/misc/tsget.pl",
"offset": 255877632,
"size": 6746,
"type": "reg",
"start_span": 0,
"end_span": 0
},
{
"filename": "app/var/data/conda-env/ssl/misc/tsget",
"offset": 255885312,
"size": 0,
"type": "symlink",
"start_span": 0,
"end_span": 0
},
{
"filename": "app/var/data/conda-env/ssl/ct_log_list.cnf.dist",
"offset": 255885824,
"size": 412,
"type": "reg",
"start_span": 0,
"end_span": 0
}
I have tried creating spans of 8mb as well and I still see start_span as 0 for all the index in ztoc.
More info for both the ztoc created (the issue is with the first one) -
sudo soci ztoc info sha256:59ecf25ca78e1edc5d17bd6d48f77de1d45073c11b2cb86b684abxxxx | jq '{num_spans, num_files, num_multi_span_files, span_size, size, version, build_tool}'
{
"num_spans": 1,
"num_files": 9033,
"num_multi_span_files": 0,
"span_size": 4194304,
"size": 1916800,
"version": "0.9",
"build_tool": "AWS SOCI CLI v0.2"
}
sudo soci ztoc info sha256:a0beb7eab9f45aa3fb5220b77953db4c19dcf845bad33219f18yyyy | jq '{num_spans, num_files, num_multi_span_files, span_size, size}'
{
"num_spans": 58,
"num_files": 10,
"num_multi_span_files": 2,
"span_size": 4194304,
"size": 1908120
}
Steps to reproduce the bug
No response
Describe the results you expected
- Ztoc index to have proper span start and end and respect the span size of 4mb.
file.Read: error reading tar header at xxx to not occur
Host information
- OS: Tried both arm64 and amd64
- Snapshotter Version: v0.12.1
- Containerd Version: v2.2.0
Any additional context or information about the bug
When pulling the image without SOCI everything works fine.
Wrt the error above, I am able to manually read the tar file after downloading it locally -
gunzip -c /tmp/layer.tar.gz | dd bs=512 skip=$((51757568/512)) count=1 2>/dev/null | xxd | head
00000000: 6170 702f 7661 722f 6461 7461 2f63 6f6e app/var/data/con
00000010: 6461 2d65 6e76 2f6c 6962 2f6c 6962 7469 da-env/lib/libti
00000020: 6e66 6f2e 736f 2e36 2e35 0000 0000 0000 nfo.so.6.5......
00000030: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000040: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000050: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000060: 0000 0000 3030 3030 3737 3500 3030 3131 ....0000775.0011
00000070: 3631 3100 3030 3030 3030 3000 3030 3030 611.0000000.0000
00000080: 3130 3733 3531 3000 3030 3030 3030 3030 1073510.00000000
00000090: 3030 3000 3030 3135 3331 3400 3000 0000 000.0015314.0...
No response
Description
I am trying to convert an image using basic soci convert with default settings. But when I try to pull the image using soci snapshotter I get this error -
Soci error log -
In the ztoc info I see that its at a offset of
199165440with span 0 whereas it should have been at span 50 assuming 4mb spans? -Here is the end of the ztoc, all are at span 0 while the whole uncompressed layer is 250Mb+
I have tried creating spans of 8mb as well and I still see start_span as 0 for all the index in ztoc.
More info for both the ztoc created (the issue is with the first one) -
Steps to reproduce the bug
No response
Describe the results you expected
file.Read: error reading tar header at xxxto not occurHost information
Any additional context or information about the bug
When pulling the image without SOCI everything works fine.
Wrt the error above, I am able to manually read the tar file after downloading it locally -
No response