[Queue Time Histogram] Add Job Queue Time Lambda (#6435)

# Description
Add aws lambda to generate in-queue job histgram, steps:
   1. add logic to generate snapshot of in-queue jobs **[This PR]**
   2. add logic to generate histogram
   3. suppport multi-threading for backfilling

the snapshot data we generate includes:
{ queue_s, repo, workflow_name , job_name, htm_url, machine_type, time,
runner_labels}

the runner_labels includes the machine_type, and other categories such
as linux, dynamic etc

Design
Doc:[doc](https://docs.google.com/document/d/1OiPv-ku_NvMgvnaMIbtnIEHTixJY28CBaKeMKJtglEk/edit?tab=t.0#heading=h.87bg49h503n7)

# working result in s3 (Ran locally)
[s3
link](https://us-east-1.console.aws.amazon.com/s3/object/ossci-raw-job-status?region=us-east-1&bucketType=general&prefix=job_queue_times_historical/pytorch/pytorch/1742367310.txt)

Author: yangw-dev

.github/workflows/tests.yml

@@ -39,6 +39,24 @@ jobs:
         # Test tools
         python3 -m unittest discover -vs tools/tests -p 'test_*.py'
 
+  test-aws-lambda:
+    name: Test aws lambda
+    if: ${{ github.repository == 'pytorch/test-infra' }}
+    uses: ./.github/workflows/linux_job_v2.yml
+    with:
+      docker-image: python:3.10-slim-bullseye
+      runner: linux.large
+      script: |
+        # Environment setup
+        echo ::group::setup Python environment
+        python -m venv .venv/
+        source .venv/bin/activate
+        pip install -r aws/lambda/tests/test_requirements.txt
+        echo ::endgroup::
+
+        # Test aws lambda
+        pytest -v aws/lambda/tests
+
   test-github-scripts:
     name: Test github scripts
     if: ${{ github.repository == 'pytorch/test-infra' }}

.lintrunner.toml

@@ -43,6 +43,7 @@ include_patterns = [
     'torchci/**/*.py',
     'torchci/**/*.pyi',
     '.github/scripts/*.py',
+    'aws/lambda/oss-ci-job-queue-time/*.py',
     'aws/lambda/whl_metadata_upload_pep658/**/*.py',
 ]
 command = [

aws/lambda/oss_ci_job_queue_time/.gitignore

@@ -0,0 +1,3 @@
+*.zip
+deployment/
+venv/

aws/lambda/oss_ci_job_queue_time/Makefile

@@ -0,0 +1,19 @@
+all: run-local
+
+clean:
+	rm -rf deployment
+	rm -rf venv
+	rm -rf deployment.zip
+
+venv/bin/python:
+	virtualenv venv
+	venv/bin/pip install -r requirements.txt
+
+deployment.zip:
+	mkdir -p deployment
+	cp lambda_function.py ./deployment/.
+	pip3.10 install -r requirements.txt -t ./deployment/. --platform manylinux2014_x86_64 --only-binary=:all: --implementation cp --python-version 3.10 --upgrade
+	cd ./deployment && zip -q -r ../deployment.zip .
+
+.PHONY: create-deployment-package
+create-deployment-package: deployment.zip