Fix/dagster round 2 (RonaldHensbergen#117)

* Fixing Dagster user-code being unhealthy

* Test-case T1 succeeds!

* Updating the test checklist

* Updated test with correct Dockerfile

* Linting issues (why a file I didn't change?)

Author: RonaldHensbergen

.github/ISSUE_TEMPLATE/bug_report.yml

@@ -23,10 +23,10 @@ body:
     label: Severity
     description: How severely does this bug affect you?
     options:
-      - Critical (system unusable / data loss)
-      - High (major feature broken)
-      - Medium (feature partially broken)
-      - Low (minor inconvenience)
+    - Critical (system unusable / data loss)
+    - High (major feature broken)
+    - Medium (feature partially broken)
+    - Low (minor inconvenience)
   validations:
     required: true
 

docs/mvp-test/checklist-signoff.md

@@ -20,13 +20,13 @@ Run tests in this order:
 
 ## T1 Bootstrap proof
 
-- [ ] **T1.1** Fresh bootstrap  
+- [x] **T1.1** Fresh bootstrap  
   Clone repo, copy env/config, render profile, and start stack from docs only.
 
-- [ ] **T1.2** Preflight/doctor  
+- [x] **T1.2** Preflight/doctor  
   Verify Docker, Compose, ports, env vars, disk, and memory before startup.
 
-- [ ] **T1.3** Container health  
+- [x] **T1.3** Container health  
   Confirm Dagster, Postgres, and Superset all become healthy within timeout.
 
 ---
@@ -169,7 +169,7 @@ Run tests in this order:
 
 Minimum required before calling the profile proven:
 
-- [ ] **T1.1–T1.3**
+- [x] **T1.1–T1.3**
 - [ ] **T2.1–T2.3**
 - [ ] **T3.1–T3.3**
 - [ ] **T4.1–T4.5**

images/dagster/Dockerfile-user-code images/dagster/Dockerfile.user-codeimages/dagster/Dockerfile-user-code renamed to images/dagster/Dockerfile.user-code

@@ -21,7 +21,7 @@ COPY workdirs/dagster /opt/dagster/app
 # Run dagster code server on port 4000
 EXPOSE 4000
 
-HEALTHCHECK --timeout=1s --start-period=3s --interval=3s --retries=20 CMD ["dagster", "api", "grpc-health-check", "-p", "4000"]
+HEALTHCHECK --timeout=10s --start-period=5s --interval=10s --retries=20 CMD ["dagster", "api", "grpc-health-check", "-p", "4000"]
 
 # CMD allows this to be overridden from run launchers or executors to execute runs and steps
 CMD ["dagster", "code-server", "start", "-h", "0.0.0.0", "-p", "4000", "-f", "/opt/dagster/app/definitions.py"]

images/dagster/workspace.yaml

@@ -1,5 +1,5 @@
 load_from:
 - grpc_server:
-  host: dagster-user-code
-  port: 4000
-  location_name: example_user_code
+    host: dagster-user-code
+    port: 4000
+    location_name: example_user_code

modules/orchestration/dagster/module.yaml

@@ -130,7 +130,7 @@ spec:
         user-code:
           build:
             context: ../../../
-            dockerfile: images/dagster/Dockerfile-user-code
+            dockerfile: images/dagster/Dockerfile.user-code
           image: local/dagster-user-code:custom
           restart: always
           hostname: dagster-user-code
@@ -160,7 +160,7 @@ spec:
             - "${config.webPort}:3000"
           volumes:
             - /var/run/docker.sock:/var/run/docker.sock
-            - /tmp/io_manager_storage:/tmp/io_manager_storage
+            - dagster-io-manager-storage:/tmp/io_manager_storage
           environment:
             DAGSTER_HOME: "${config.homeDir}"
             DAGSTER_POSTGRES_HOST: "${bindings.run-storage.host}"
@@ -195,11 +195,15 @@ spec:
             - dagster-daemon run
           volumes:
             - /var/run/docker.sock:/var/run/docker.sock
-            - /tmp/io_manager_storage:/tmp/io_manager_storage
+            - dagster-io-manager-storage:/tmp/io_manager_storage
           environment:
             DAGSTER_HOME: "${config.homeDir}"
             DAGSTER_POSTGRES_HOST: "${bindings.run-storage.host}"
             DAGSTER_POSTGRES_PORT: "${bindings.run-storage.port}"
             DAGSTER_POSTGRES_DB: "${bindings.run-storage.database}"
             DAGSTER_POSTGRES_USER: "${bindings.run-storage.username}"
             DAGSTER_POSTGRES_PASSWORD: "${bindings.run-storage.password}"
+
+      volumes:
+        dagster-io-manager-storage:
+          enabledFrom: spec.config.storage.enabled

tests/test_render_example_profile.py

@@ -42,7 +42,7 @@ def test_example_profile_validates_plans_and_renders_compose(self):
             self.assertIn("dagster-user-code", compose["services"])
             self.assertEqual(
                 compose["services"]["dagster-user-code"]["build"]["dockerfile"],
-                "images/dagster/Dockerfile-user-code",
+                "images/dagster/Dockerfile.user-code",
             )
             self.assertEqual(
                 compose["services"]["dagster-dagster-webserver"]["depends_on"]["dagster-user-code"]["condition"],