Skip to content

Commit 77e282b

Browse files
authored
Merge pull request #66 from awslabs/feature/concurrent-camera-stream-viewing
Concurrent camera stream viewing (multi-viewer live preview)
2 parents 376d0ac + 309d892 commit 77e282b

50 files changed

Lines changed: 12493 additions & 3 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
{"specId": "b4b904e3-cf37-4065-a121-966005904c7c", "workflowType": "requirements-first", "specType": "feature"}

.kiro/specs/concurrent-camera-stream-viewing/design.md

Lines changed: 593 additions & 0 deletions
Large diffs are not rendered by default.

.kiro/specs/concurrent-camera-stream-viewing/requirements.md

Lines changed: 140 additions & 0 deletions
Large diffs are not rendered by default.

.kiro/specs/concurrent-camera-stream-viewing/tasks.md

Lines changed: 313 additions & 0 deletions
Large diffs are not rendered by default.

README.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -282,6 +282,14 @@ version, and compile models for the matching compilation target (Jetson JetPack
282282

283283
**Debugging:** `VERBOSE=1 ./gdk-component-build-and-publish.sh`
284284

285+
**Publishing without rebuilding (`publish-ecr-only.sh`).** `gdk-component-build-and-publish.sh` always does a clean rebuild before publishing. If you have already built the images (e.g. the build succeeded but the publish failed because an auth/login session expired), use `publish-ecr-only.sh` to publish the **already-built** images without spending ~15-20 minutes on another build:
286+
287+
```bash
288+
./publish-ecr-only.sh # aarch64 JetPack 5 (aws.edgeml.dda.LocalServer.arm64JP5)
289+
```
290+
291+
It reuses the locally-built `flask-app:latest` / `react-webapp:latest` images and the existing `custom-build/` staging directory, and runs only the ECR + S3 publish path used for >2 GB artifacts: bump to the next patch version, push the images to ECR, upload the small scripts/compose zip to S3, rewrite the recipe to reference the `docker:` + S3 artifacts, register the new component version, and tag it for portal discovery. It fails fast if AWS credentials are invalid or the built images/staging dir are missing (run a build first). It currently targets the arm64 JetPack 5 component; edit `COMPONENT_NAME`/`ARCH` near the top for other targets.
292+
285293
> **Notice**: Stop the EC2 build server after building to avoid unnecessary costs.
286294
287295
<details>
@@ -937,6 +945,7 @@ defect-detection-application/
937945
├── datasets/ # Sample datasets
938946
├── build-custom.sh # Custom build logic
939947
├── gdk-component-build-and-publish.sh
948+
├── publish-ecr-only.sh # Publish already-built images (no rebuild)
940949
└── setup-build-server.sh
941950
```
942951

publish-ecr-only.sh

Lines changed: 136 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,136 @@
1+
#!/bin/bash
2+
set -e
3+
set -o pipefail
4+
# Publish-only (ECR + S3) for an already-built LocalServer component, WITHOUT
5+
# rebuilding. Mirrors the >2GB ECR path in gdk-component-build-and-publish.sh,
6+
# reusing the locally-built flask-app:latest / react-webapp:latest images and the
7+
# existing custom-build staging dir. For aarch64 JetPack 5.
8+
9+
ARCH="aarch64"
10+
COMPONENT_NAME="aws.edgeml.dda.LocalServer.arm64JP5"
11+
12+
echo "Checking AWS credentials..."
13+
aws sts get-caller-identity >/dev/null || { echo "ERROR: AWS credentials invalid/expired"; exit 1; }
14+
15+
PUB_REGION=$(aws configure get region 2>/dev/null || true)
16+
[ -z "$PUB_REGION" ] && PUB_REGION="${AWS_REGION:-${AWS_DEFAULT_REGION:-}}"
17+
PUB_ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)
18+
if [ -z "$PUB_REGION" ] || [ -z "$PUB_ACCOUNT_ID" ] || [ "$PUB_ACCOUNT_ID" = "None" ]; then
19+
echo "ERROR: cannot resolve region/account (region='$PUB_REGION' account='$PUB_ACCOUNT_ID')"; exit 1
20+
fi
21+
22+
ECR_REGISTRY="${PUB_ACCOUNT_ID}.dkr.ecr.${PUB_REGION}.amazonaws.com"
23+
ECR_REPO_BACKEND="${ECR_REGISTRY}/dda/flask-app"
24+
ECR_REPO_FRONTEND="${ECR_REGISTRY}/dda/react-webapp"
25+
S3_BUCKET="dda-component-${PUB_REGION}-${PUB_ACCOUNT_ID}"
26+
27+
# Pre-flight: the images and staging dir must already exist (built, not rebuilt here).
28+
docker image inspect flask-app:latest >/dev/null 2>&1 || { echo "ERROR: flask-app:latest not found"; exit 1; }
29+
docker image inspect react-webapp:latest >/dev/null 2>&1 || { echo "ERROR: react-webapp:latest not found"; exit 1; }
30+
STAGE_DIR="custom-build/${COMPONENT_NAME}"
31+
[ -d "$STAGE_DIR" ] || { echo "ERROR: staging dir $STAGE_DIR not found"; exit 1; }
32+
33+
# Next version = bump patch of the latest registered version (start at 1.0.0).
34+
LATEST_VERSION=$(aws greengrassv2 list-component-versions \
35+
--arn "arn:aws:greengrass:${PUB_REGION}:${PUB_ACCOUNT_ID}:components:${COMPONENT_NAME}" \
36+
--query 'componentVersions[0].componentVersion' --output text 2>/dev/null || echo "None")
37+
if [ "$LATEST_VERSION" = "None" ] || [ -z "$LATEST_VERSION" ]; then
38+
COMPONENT_VERSION="1.0.0"
39+
else
40+
V_MAJOR=$(echo "$LATEST_VERSION" | cut -d. -f1)
41+
V_MINOR=$(echo "$LATEST_VERSION" | cut -d. -f2)
42+
V_PATCH=$(echo "$LATEST_VERSION" | cut -d. -f3)
43+
COMPONENT_VERSION="${V_MAJOR}.${V_MINOR}.$((V_PATCH + 1))"
44+
fi
45+
echo "Latest registered: ${LATEST_VERSION}; publishing version: $COMPONENT_VERSION"
46+
47+
# Authenticate to ECR and ensure repos exist.
48+
aws ecr get-login-password --region "$PUB_REGION" | docker login --username AWS --password-stdin "$ECR_REGISTRY"
49+
aws ecr describe-repositories --repository-names dda/flask-app --region "$PUB_REGION" >/dev/null 2>&1 || \
50+
aws ecr create-repository --repository-name dda/flask-app --region "$PUB_REGION" >/dev/null
51+
aws ecr describe-repositories --repository-names dda/react-webapp --region "$PUB_REGION" >/dev/null 2>&1 || \
52+
aws ecr create-repository --repository-name dda/react-webapp --region "$PUB_REGION" >/dev/null
53+
54+
echo "Pushing flask-app to ECR (${ECR_REPO_BACKEND}:${COMPONENT_VERSION})..."
55+
docker tag flask-app:latest "${ECR_REPO_BACKEND}:${COMPONENT_VERSION}"
56+
docker push "${ECR_REPO_BACKEND}:${COMPONENT_VERSION}"
57+
echo "Pushing react-webapp to ECR (${ECR_REPO_FRONTEND}:${COMPONENT_VERSION})..."
58+
docker tag react-webapp:latest "${ECR_REPO_FRONTEND}:${COMPONENT_VERSION}"
59+
docker push "${ECR_REPO_FRONTEND}:${COMPONENT_VERSION}"
60+
61+
# Repackage a scripts-only zip (same name/layout as the full zip, minus the image tars).
62+
rm -f "${STAGE_DIR}/flask-app.tar" "${STAGE_DIR}/react-webapp.tar" "${STAGE_DIR}"/.tmp-* 2>/dev/null || true
63+
APP_ZIP="custom-build/${COMPONENT_NAME}-${ARCH}.zip"
64+
rm -f "$APP_ZIP"
65+
zip -r -X "$APP_ZIP" "custom-build/${COMPONENT_NAME}" -x '*/.tmp-*'
66+
67+
S3_KEY="${COMPONENT_NAME}/${COMPONENT_VERSION}/${COMPONENT_NAME}-${ARCH}.zip"
68+
S3_URI="s3://${S3_BUCKET}/${S3_KEY}"
69+
echo "Uploading scripts artifact to ${S3_URI} ($(numfmt --to=iec "$(stat --format=%s "$APP_ZIP")" 2>/dev/null || echo "?"))..."
70+
aws s3 cp "$APP_ZIP" "$S3_URI" --region "$PUB_REGION"
71+
72+
# Rewrite the recipe: docker image artifacts + S3 scripts artifact, add Docker/TES
73+
# deps, and swap the in-Install `docker load -i ...tar` for `docker tag <ecr> ...:latest`.
74+
python3 -c "import yaml" 2>/dev/null || pip3 install --user pyyaml >/dev/null 2>&1 || true
75+
ECR_RECIPE="greengrass-build/recipes/recipe-ecr.yaml"
76+
mkdir -p greengrass-build/recipes
77+
python3 - "recipe.yaml" "$ECR_RECIPE" "$ECR_REPO_BACKEND" "$ECR_REPO_FRONTEND" "$COMPONENT_VERSION" "$S3_URI" <<'PYEOF'
78+
import re, sys, yaml
79+
80+
src, out, ecr_backend, ecr_frontend, version, s3_uri = sys.argv[1:7]
81+
82+
with open(src) as f:
83+
recipe = yaml.safe_load(f)
84+
85+
recipe['ComponentVersion'] = version
86+
87+
deps = recipe.setdefault('ComponentDependencies', {})
88+
deps['aws.greengrass.DockerApplicationManager'] = {'VersionRequirement': '~2.0.0'}
89+
deps['aws.greengrass.TokenExchangeService'] = {'VersionRequirement': '~2.0.0'}
90+
91+
def rewrite_install(script: str) -> str:
92+
script = re.sub(r'docker load -i \S*flask-app\.tar',
93+
f'docker tag {ecr_backend}:{version} flask-app:latest', script)
94+
script = re.sub(r'docker load -i \S*react-webapp\.tar(?:\.gz)?',
95+
f'docker tag {ecr_frontend}:{version} react-webapp:latest', script)
96+
return script
97+
98+
changed = False
99+
for manifest in recipe.get('Manifests', []):
100+
lifecycle = manifest.get('Lifecycle', {})
101+
install = lifecycle.get('Install')
102+
if isinstance(install, dict) and 'Script' in install:
103+
new_script = rewrite_install(install['Script'])
104+
if new_script != install['Script']:
105+
changed = True
106+
install['Script'] = new_script
107+
manifest['Artifacts'] = [
108+
{'URI': f'docker:{ecr_backend}:{version}'},
109+
{'URI': f'docker:{ecr_frontend}:{version}'},
110+
{'URI': s3_uri, 'Unarchive': 'ZIP'},
111+
]
112+
113+
if not changed:
114+
sys.stderr.write('ERROR: did not find docker load commands to rewrite in Install; recipe format may have changed.\n')
115+
sys.exit(2)
116+
117+
with open(out, 'w') as f:
118+
yaml.dump(recipe, f, default_flow_style=False, sort_keys=False)
119+
print(f'Wrote ECR recipe: {out}')
120+
PYEOF
121+
122+
echo "Creating component version via API..."
123+
aws greengrassv2 create-component-version --inline-recipe fileb://"$ECR_RECIPE" --region "$PUB_REGION"
124+
125+
echo "Tagging component for portal discovery..."
126+
COMPONENT_ARN=$(aws greengrassv2 list-components --scope PRIVATE --region "$PUB_REGION" \
127+
--query "components[?componentName=='${COMPONENT_NAME}'].arn | [0]" --output text 2>/dev/null || true)
128+
if [ -n "$COMPONENT_ARN" ] && [ "$COMPONENT_ARN" != "None" ]; then
129+
aws greengrassv2 tag-resource --resource-arn "$COMPONENT_ARN" \
130+
--tags "dda-portal:managed=true" --region "$PUB_REGION" 2>/dev/null \
131+
&& echo "Tagged $COMPONENT_ARN" || echo "WARN: tagging failed (non-critical)"
132+
else
133+
echo "WARN: could not resolve component ARN for tagging (non-critical)"
134+
fi
135+
136+
echo "DONE: published ${COMPONENT_NAME} v${COMPONENT_VERSION} (ECR + S3)"

src/backend/app.py

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -108,7 +108,8 @@
108108
image_source,
109109
auth_info,
110110
download_file,
111-
inference_result
111+
inference_result,
112+
streams
112113
)
113114

114115
import dao.sqlite_db.models as models
@@ -147,6 +148,7 @@
147148
app.include_router(auth_info.router)
148149
app.include_router(download_file.unauthenticated_router)
149150
app.include_router(inference_result.router)
151+
app.include_router(streams.router)
150152

151153

152154
def cleanup_workflow_digital_inputs():

src/backend/endpoints/image_source.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -118,6 +118,7 @@ def get_frame(image_source_dict, image_source_config_override=None):
118118
camera_id = image_source_dict.get('cameraId')
119119
return get_camera_frame(camera_id, camera_config)
120120

121+
121122
@router.post("/image-sources/{imageSourceId}/preview")
122123
def preview_image(imageSourceId, request: GetPreviewImageRequest = GetPreviewImageRequest(), db: Session = Depends(get_db)) -> GetPreviewImageResponse:
123124
try:

0 commit comments

Comments
 (0)