Skip to content

Commit f780406

Browse files
kennodhiambokennodhiambo
authored
Merge branch 'apache:master' into master
2 parents 1ecdd9c + 6006a21 commit f780406

File tree

353 files changed

+17149
-8194
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

353 files changed

+17149
-8194
lines changed

‎.cursor/rules/dev-standard.mdc‎

Lines changed: 125 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,125 @@
1+
---
2+
description: Apache Superset development standards and guidelines for Cursor IDE
3+
globs: ["**/*.py", "**/*.ts", "**/*.tsx", "**/*.js", "**/*.jsx", "**/*.sql", "**/*.md"]
4+
alwaysApply: true
5+
---
6+
7+
# Apache Superset Development Standards for Cursor IDE
8+
9+
Apache Superset is a data visualization platform with Flask/Python backend and React/TypeScript frontend.
10+
11+
## ⚠️ CRITICAL: Ongoing Refactors (What NOT to Do)
12+
13+
**These migrations are actively happening - avoid deprecated patterns:**
14+
15+
### Frontend Modernization
16+
- **NO `any` types** - Use proper TypeScript types
17+
- **NO JavaScript files** - Convert to TypeScript (.ts/.tsx)
18+
- **NO Enzyme** - Use React Testing Library/Jest (Enzyme fully removed)
19+
- **Use @superset-ui/core** - Don't import Ant Design directly
20+
21+
### Testing Strategy Migration
22+
- **Prefer unit tests** over integration tests
23+
- **Prefer integration tests** over Cypress end-to-end tests
24+
- **Cypress is last resort** - Actively moving away from Cypress
25+
- **Use Jest + React Testing Library** for component testing
26+
27+
### Backend Type Safety
28+
- **Add type hints** - All new Python code needs proper typing
29+
- **MyPy compliance** - Run `pre-commit run mypy` to validate
30+
- **SQLAlchemy typing** - Use proper model annotations
31+
32+
## Code Standards
33+
34+
### TypeScript Frontend
35+
- **NO `any` types** - Use proper TypeScript
36+
- **Functional components** with hooks
37+
- **@superset-ui/core** for UI components (not direct antd)
38+
- **Jest** for testing (NO Enzyme)
39+
- **Redux** for global state, hooks for local
40+
41+
### Python Backend
42+
- **Type hints required** for all new code
43+
- **MyPy compliant** - run `pre-commit run mypy`
44+
- **SQLAlchemy models** with proper typing
45+
- **pytest** for testing
46+
47+
### Apache License Headers
48+
- **New files require ASF license headers** - When creating new code files, include the standard Apache Software Foundation license header
49+
- **LLM instruction files are excluded** - Files like LLMS.md, CLAUDE.md, etc. are in `.rat-excludes` to avoid header token overhead
50+
51+
## Key Directory Structure
52+
53+
```
54+
superset/
55+
├── superset/ # Python backend (Flask, SQLAlchemy)
56+
│ ├── views/api/ # REST API endpoints
57+
│ ├── models/ # Database models
58+
│ └── connectors/ # Database connections
59+
├── superset-frontend/src/ # React TypeScript frontend
60+
│ ├── components/ # Reusable components
61+
│ ├── explore/ # Chart builder
62+
│ ├── dashboard/ # Dashboard interface
63+
│ └── SqlLab/ # SQL editor
64+
├── superset-frontend/packages/
65+
│ └── superset-ui-core/ # UI component library (USE THIS)
66+
├── tests/ # Python/integration tests
67+
├── docs/ # Documentation (UPDATE FOR CHANGES)
68+
└── UPDATING.md # Breaking changes log
69+
```
70+
71+
## Architecture Patterns
72+
73+
### Dataset-Centric Approach
74+
Charts built from enriched datasets containing:
75+
- Dimension columns with labels/descriptions
76+
- Predefined metrics as SQL expressions
77+
- Self-service analytics within defined contexts
78+
79+
### Security & Features
80+
- **RBAC**: Role-based access via Flask-AppBuilder
81+
- **Feature flags**: Control feature rollouts
82+
- **Row-level security**: SQL-based data access control
83+
84+
## Test Utilities
85+
86+
### Python Test Helpers
87+
- **`SupersetTestCase`** - Base class in `tests/integration_tests/base_tests.py`
88+
- **`@with_config`** - Config mocking decorator
89+
- **`@with_feature_flags`** - Feature flag testing
90+
- **`login_as()`, `login_as_admin()`** - Authentication helpers
91+
- **`create_dashboard()`, `create_slice()`** - Data setup utilities
92+
93+
### TypeScript Test Helpers
94+
- **`superset-frontend/spec/helpers/testing-library.tsx`** - Custom render() with providers
95+
- **`createWrapper()`** - Redux/Router/Theme wrapper
96+
- **`selectOption()`** - Select component helper
97+
- **React Testing Library** - NO Enzyme (removed)
98+
99+
## Pre-commit Validation
100+
101+
**Use pre-commit hooks for quality validation:**
102+
103+
```bash
104+
# Install hooks
105+
pre-commit install
106+
107+
# Quick validation (faster than --all-files)
108+
pre-commit run # Staged files only
109+
pre-commit run mypy # Python type checking
110+
pre-commit run prettier # Code formatting
111+
pre-commit run eslint # Frontend linting
112+
```
113+
114+
## Development Guidelines
115+
116+
- **Documentation**: Update docs/ for any user-facing changes
117+
- **Breaking Changes**: Add to UPDATING.md
118+
- **Docstrings**: Required for new functions/classes
119+
- **Follow existing patterns**: Mimic code style, use existing libraries and utilities
120+
- **Type Safety**: This codebase is actively modernizing toward full TypeScript and type safety
121+
- **Always run `pre-commit run`** to validate changes before committing
122+
123+
---
124+
125+
**Note**: This codebase is actively modernizing toward full TypeScript and type safety. Always run `pre-commit run` to validate changes. Follow the ongoing refactors section to avoid deprecated patterns.

‎.devcontainer/README.md‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
# Superset Development with GitHub Codespaces
2+
3+
For complete documentation on using GitHub Codespaces with Apache Superset, please see:
4+
5+
**[Setting up a Development Environment - GitHub Codespaces](https://superset.apache.org/docs/contributing/development#github-codespaces-cloud-development)**

‎.devcontainer/devcontainer.json‎

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
{
2+
"name": "Apache Superset Development",
3+
// Keep this in sync with the base image in Dockerfile (ARG PY_VER)
4+
// Using the same base as Dockerfile, but non-slim for dev tools
5+
"image": "python:3.11.13-bookworm",
6+
7+
"features": {
8+
"ghcr.io/devcontainers/features/docker-in-docker:2": {
9+
"moby": true,
10+
"dockerDashComposeVersion": "v2"
11+
},
12+
"ghcr.io/devcontainers/features/node:1": {
13+
"version": "20"
14+
},
15+
"ghcr.io/devcontainers/features/git:1": {},
16+
"ghcr.io/devcontainers/features/common-utils:2": {
17+
"configureZshAsDefaultShell": true
18+
},
19+
"ghcr.io/devcontainers/features/sshd:1": {
20+
"version": "latest"
21+
}
22+
},
23+
24+
// Forward ports for development
25+
"forwardPorts": [9001],
26+
"portsAttributes": {
27+
"9001": {
28+
"label": "Superset (via Webpack Dev Server)",
29+
"onAutoForward": "notify",
30+
"visibility": "public"
31+
}
32+
},
33+
34+
// Run commands after container is created
35+
"postCreateCommand": "chmod +x .devcontainer/setup-dev.sh && .devcontainer/setup-dev.sh",
36+
37+
// Auto-start Superset on Codespace resume
38+
"postStartCommand": ".devcontainer/start-superset.sh",
39+
40+
// VS Code customizations
41+
"customizations": {
42+
"vscode": {
43+
"extensions": [
44+
"ms-python.python",
45+
"ms-python.vscode-pylance",
46+
"charliermarsh.ruff",
47+
"dbaeumer.vscode-eslint",
48+
"esbenp.prettier-vscode"
49+
]
50+
}
51+
}
52+
}

‎.devcontainer/setup-dev.sh‎

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
#!/bin/bash
2+
# Setup script for Superset Codespaces development environment
3+
4+
echo "🔧 Setting up Superset development environment..."
5+
6+
# The universal image has most tools, just need Superset-specific libs
7+
echo "📦 Installing Superset-specific dependencies..."
8+
sudo apt-get update
9+
sudo apt-get install -y \
10+
libsasl2-dev \
11+
libldap2-dev \
12+
libpq-dev \
13+
tmux \
14+
gh
15+
16+
# Install uv for fast Python package management
17+
echo "📦 Installing uv..."
18+
curl -LsSf https://astral.sh/uv/install.sh | sh
19+
20+
# Add cargo/bin to PATH for uv
21+
echo 'export PATH="$HOME/.cargo/bin:$PATH"' >> ~/.bashrc
22+
echo 'export PATH="$HOME/.cargo/bin:$PATH"' >> ~/.zshrc
23+
24+
# Install Claude Code CLI via npm
25+
echo "🤖 Installing Claude Code..."
26+
npm install -g @anthropic-ai/claude-code
27+
28+
# Make the start script executable
29+
chmod +x .devcontainer/start-superset.sh
30+
31+
echo "✅ Development environment setup complete!"
32+
echo "🚀 Run '.devcontainer/start-superset.sh' to start Superset"

‎.devcontainer/start-superset.sh‎

Lines changed: 59 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,59 @@
1+
#!/bin/bash
2+
# Startup script for Superset in Codespaces
3+
4+
echo "🚀 Starting Superset in Codespaces..."
5+
echo "🌐 Frontend will be available at port 9001"
6+
7+
# Find the workspace directory (Codespaces clones as 'superset', not 'superset-2')
8+
WORKSPACE_DIR=$(find /workspaces -maxdepth 1 -name "superset*" -type d | head -1)
9+
if [ -n "$WORKSPACE_DIR" ]; then
10+
cd "$WORKSPACE_DIR"
11+
echo "📁 Working in: $WORKSPACE_DIR"
12+
else
13+
echo "📁 Using current directory: $(pwd)"
14+
fi
15+
16+
# Check if docker is running
17+
if ! docker info > /dev/null 2>&1; then
18+
echo "⏳ Waiting for Docker to start..."
19+
sleep 5
20+
fi
21+
22+
# Clean up any existing containers
23+
echo "🧹 Cleaning up existing containers..."
24+
docker-compose -f docker-compose-light.yml down
25+
26+
# Start services
27+
echo "🏗️ Building and starting services..."
28+
echo ""
29+
echo "📝 Once started, login with:"
30+
echo " Username: admin"
31+
echo " Password: admin"
32+
echo ""
33+
echo "📋 Running in foreground with live logs (Ctrl+C to stop)..."
34+
35+
# Run docker-compose and capture exit code
36+
docker-compose -f docker-compose-light.yml up
37+
EXIT_CODE=$?
38+
39+
# If it failed, provide helpful instructions
40+
if [ $EXIT_CODE -ne 0 ] && [ $EXIT_CODE -ne 130 ]; then # 130 is Ctrl+C
41+
echo ""
42+
echo "❌ Superset startup failed (exit code: $EXIT_CODE)"
43+
echo ""
44+
echo "🔄 To restart Superset, run:"
45+
echo " .devcontainer/start-superset.sh"
46+
echo ""
47+
echo "🔧 For troubleshooting:"
48+
echo " # View logs:"
49+
echo " docker-compose -f docker-compose-light.yml logs"
50+
echo ""
51+
echo " # Clean restart (removes volumes):"
52+
echo " docker-compose -f docker-compose-light.yml down -v"
53+
echo " .devcontainer/start-superset.sh"
54+
echo ""
55+
echo " # Common issues:"
56+
echo " - Network timeouts: Just retry, often transient"
57+
echo " - Port conflicts: Check 'docker ps'"
58+
echo " - Database issues: Try clean restart with -v"
59+
fi

‎.github/copilot-instructions.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
../LLMS.md

‎.github/workflows/claude.yml‎

Lines changed: 82 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,82 @@
1+
name: Claude PR Assistant
2+
3+
on:
4+
issue_comment:
5+
types: [created]
6+
pull_request_review_comment:
7+
types: [created]
8+
9+
jobs:
10+
check-permissions:
11+
if: |
12+
(github.event_name == 'issue_comment' && contains(github.event.comment.body, '@claude')) ||
13+
(github.event_name == 'pull_request_review_comment' && contains(github.event.comment.body, '@claude'))
14+
runs-on: ubuntu-latest
15+
outputs:
16+
allowed: ${{ steps.check.outputs.allowed }}
17+
steps:
18+
- name: Check if user is allowed
19+
id: check
20+
run: |
21+
# List of allowed users
22+
ALLOWED_USERS="mistercrunch,rusackas"
23+
24+
# Get the commenter's username
25+
COMMENTER="${{ github.event.comment.user.login }}"
26+
27+
echo "Checking permissions for user: $COMMENTER"
28+
29+
# Check if user is in allowed list
30+
if [[ ",$ALLOWED_USERS," == *",$COMMENTER,"* ]]; then
31+
echo "allowed=true" >> $GITHUB_OUTPUT
32+
echo "✅ User $COMMENTER is allowed to use Claude"
33+
else
34+
echo "allowed=false" >> $GITHUB_OUTPUT
35+
echo "❌ User $COMMENTER is not allowed to use Claude"
36+
fi
37+
38+
deny-access:
39+
needs: check-permissions
40+
if: needs.check-permissions.outputs.allowed == 'false'
41+
runs-on: ubuntu-latest
42+
permissions:
43+
issues: write
44+
pull-requests: write
45+
steps:
46+
- name: Comment access denied
47+
uses: actions/github-script@v7
48+
with:
49+
script: |
50+
const message = `👋 Hi @${{ github.event.comment.user.login || github.event.review.user.login || github.event.issue.user.login }}!
51+
52+
Thanks for trying to use Claude Code, but currently only certain team members have access to this feature.
53+
54+
If you believe you should have access, please contact a project maintainer.`;
55+
56+
await github.rest.issues.createComment({
57+
owner: context.repo.owner,
58+
repo: context.repo.repo,
59+
issue_number: context.issue.number,
60+
body: message
61+
});
62+
63+
claude-code-action:
64+
needs: check-permissions
65+
if: needs.check-permissions.outputs.allowed == 'true'
66+
runs-on: ubuntu-latest
67+
permissions:
68+
contents: write
69+
pull-requests: write
70+
issues: write
71+
id-token: write
72+
steps:
73+
- name: Checkout repository
74+
uses: actions/checkout@v4
75+
with:
76+
fetch-depth: 1
77+
78+
- name: Run Claude PR Action
79+
uses: anthropics/claude-code-action@beta
80+
with:
81+
anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
82+
timeout_minutes: "60"

‎.github/workflows/superset-python-unittest.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -51,7 +51,7 @@ jobs:
5151
SUPERSET_TESTENV: true
5252
SUPERSET_SECRET_KEY: not-a-secret
5353
run: |
54-
pytest --durations-min=0.5 --cov-report= --cov=superset/sql/ ./tests/unit_tests/sql/ --cache-clear --cov-fail-under=100
54+
pytest --durations-min=0.5 --cov=superset/sql/ ./tests/unit_tests/sql/ --cache-clear --cov-fail-under=100
5555
- name: Upload code coverage
5656
uses: codecov/codecov-action@v5
5757
with:

‎.gitignore‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -127,4 +127,7 @@ docker/*local*
127127
# Jest test report
128128
test-report.html
129129
superset/static/stats/statistics.html
130+
131+
# LLM-related
132+
CLAUDE.local.md
130133
.aider*

0 commit comments

Comments
 (0)