Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 9 additions & 3 deletions .github/workflows/CI.yml
Original file line number Diff line number Diff line change
Expand Up @@ -188,11 +188,17 @@ jobs:
docker stop $(docker ps -q) && docker system prune --volumes -f
docker image ls --format '{{.Repository}}:{{.Tag}}' | grep '^migrations/' | xargs -I {} docker image rm {}

cdk-tests:
node-tests:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
npm-project:
- ./deployment/cdk/opensearch-service-migration
- ./deployment/migration-assistant-solution
defaults:
run:
working-directory: ./deployment/cdk/opensearch-service-migration
working-directory: ${{ matrix.npm-project }}
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
Expand Down Expand Up @@ -227,7 +233,7 @@ jobs:
all-ci-checks-pass:
needs:
- style-check
- cdk-tests
- node-tests
- gradle-tests
- link-checker
- python-e2e-tests
Expand Down
4 changes: 4 additions & 0 deletions deployment/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,10 @@ This directory is aimed at housing deployment/distribution methods for various m
A containerized end-to-end solution (including a source and target cluster as well as the migration services) can be deployed locally using the
[Docker Solution](../TrafficCapture/dockerSolution/README.md).

### Deploying the Migration Assistant Solution to AWS

The Migration Assistant is avaliable as an AWS Solution, [learn more](./migration-assistant-solution/README.md).

### Deploying Migration solution to AWS

**Note**: These features are still under development and subject to change
Expand Down
12 changes: 12 additions & 0 deletions deployment/migration-assistant-solution/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
*.js
!jest.config.js
*.d.ts
node_modules
cdk.context.json
coverage
dist
# CDK asset staging directory
.cdk.staging
cdk.out
certs
cdk-synth-output
6 changes: 6 additions & 0 deletions deployment/migration-assistant-solution/.npmignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
*.ts
!*.d.ts

# CDK asset staging directory
.cdk.staging
cdk.out
45 changes: 45 additions & 0 deletions deployment/migration-assistant-solution/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
# AWS Solutions Deployment

OpenSearch Migration assistant is distributed in AWS Solutions Library, see the most recent release in https://aws.amazon.com/solutions/implementations/migration-assistant-for-amazon-opensearch-service/

## Solutions Architecture

For the solutions project to allow customization of the feature used by Migration assistant first a bootstrap environment is deployed then a secondary step is used to deploy the configured version of Migration Assistant. See more details about this configuration options from [options.md](../cdk/opensearch-service-migration/options.md).

```mermaid
sequenceDiagram
participant User
participant Cfn as Cloud Formation
participant AWS as AWS Solution Environment
participant MA as Migration Assistant Environment

User ->> Cfn: (Optional) Deploy Bootstrap
User ->>+ Cfn: Deploy AWS Solution via Template
Cfn ->>- AWS: Deploy Resources
User ->>+ AWS: Build Migration Assisant (on EC2 instance)
Note over AWS: Migration Assistant is built<br/>with `./initBootstrap.sh` script<br/>and deployed via `cdk` tool
AWS ->> User: Build Complete
User ->> AWS: Configure & Deploy Migration Assisant
AWS ->>+ Cfn: Deploy Migration Assisant
Cfn ->>- MA: Deploy Resources
deactivate AWS
Note over MA: Migration Assistant is ready
User ->> MA: Log into Migration Assistant Console
User ->> MA: Migration Actions
```

### Migration Assistant

The full range of functionality offered by the migration assistant deployed through the opensearch-service-migration project, see its [README.MD](../cdk/opensearch-service-migration/README.md) for additon details.

## Development

This project is writen in TypeScript and uses the cloud developer tookit (CDK) to produce its build artifacts, cloud formation templates that can be used to deploy onto Amazon Web Services.

### Quick Start Guide

* Install Node 18+ & Npm 10+ https://docs.npmjs.com/downloading-and-installing-node-js-and-npm
* Build the project `npm run build`
* Test the project `npm run build`
* Create the deployment artifacts `npm run synth`
* Deploy with the default AWS credentials to the default region `npm run deploy`
38 changes: 38 additions & 0 deletions deployment/migration-assistant-solution/bin/app.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
import 'source-map-support/register';
import { App, DefaultStackSynthesizer } from 'aws-cdk-lib';
import { SolutionsInfrastructureStack } from '../lib/solutions-stack';

const getProps = () => {
const { CODE_BUCKET, SOLUTION_NAME, CODE_VERSION } = process.env;
if (typeof CODE_BUCKET !== 'string' || CODE_BUCKET.trim() === '') {
console.warn(`Missing environment variable CODE_BUCKET, using a default value`);
}

if (typeof SOLUTION_NAME !== 'string' || SOLUTION_NAME.trim() === '') {
console.warn(`Missing environment variable SOLUTION_NAME, using a default value`);
}

if (typeof CODE_VERSION !== 'string' || CODE_VERSION.trim() === '') {
console.warn(`Missing environment variable CODE_VERSION, using a default value`);
}

const codeBucket = CODE_BUCKET ?? "Unknown";
const solutionVersion = CODE_VERSION ?? "Unknown";
const solutionName = SOLUTION_NAME ?? "MigrationAssistant";
const solutionId = 'SO0290';
const description = `(${solutionId}) - The AWS CloudFormation template for deployment of the ${solutionName}. Version ${solutionVersion}`;
return {
codeBucket,
solutionVersion,
solutionId,
solutionName,
description
};
};

const app = new App();
const infraProps = getProps()
new SolutionsInfrastructureStack(app, 'OSMigrations-Bootstrap', {
synthesizer: new DefaultStackSynthesizer(),
...infraProps
});
42 changes: 42 additions & 0 deletions deployment/migration-assistant-solution/cdk.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
{
"app": "npx ts-node --prefer-ts-exts bin/app.ts",
"watch": {
"include": [
"**"
],
"exclude": [
"README.md",
"cdk*.json",
"**/*.d.ts",
"**/*.js",
"tsconfig.json",
"package*.json",
"yarn.lock",
"node_modules",
"test"
]
},
"context": {
"@aws-cdk/aws-apigateway:usagePlanKeyOrderInsensitiveId": true,
"@aws-cdk/core:stackRelativeExports": true,
"@aws-cdk/aws-rds:lowercaseDbIdentifier": true,
"@aws-cdk/aws-lambda:recognizeVersionProps": true,
"@aws-cdk/aws-lambda:recognizeLayerVersion": true,
"@aws-cdk/aws-cloudfront:defaultSecurityPolicyTLSv1.2_2021": true,
"@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true,
"@aws-cdk/aws-ec2:uniqueImdsv2TemplateName": true,
"@aws-cdk/core:checkSecretUsage": true,
"@aws-cdk/aws-iam:minimizePolicies": true,
"@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true,
"@aws-cdk/core:validateSnapshotRemovalPolicy": true,
"@aws-cdk/aws-codepipeline:crossAccountKeyAliasStackSafeResourceName": true,
"@aws-cdk/aws-s3:createDefaultLoggingPolicy": true,
"@aws-cdk/aws-sns-subscriptions:restrictSqsDescryption": true,
"@aws-cdk/aws-apigateway:disableCloudWatchRole": true,
"@aws-cdk/core:enablePartitionLiterals": true,
"@aws-cdk/core:target-partitions": [
"aws",
"aws-cn"
]
}
}
18 changes: 18 additions & 0 deletions deployment/migration-assistant-solution/eslint.config.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
import eslint from '@eslint/js';
import tseslint from 'typescript-eslint';

export default tseslint.config(
eslint.configs.recommended,
...tseslint.configs.strict,
...tseslint.configs.stylistic,
{
ignores: ['**/*.js'],
},
{
languageOptions: {
globals: {
jest: true
}
}
}
);
1 change: 1 addition & 0 deletions deployment/migration-assistant-solution/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
export * from './lib/solutions-stack'
55 changes: 55 additions & 0 deletions deployment/migration-assistant-solution/initBootstrap.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
#!/bin/bash

usage() {
echo "Usage: $0 [--tag <tag_name>] [--branch <branch_name>]"
exit 1
}

tag=""
branch=""

# Parse options
while [[ "$#" -gt 0 ]]; do
case $1 in
--tag)
if [ -n "$branch" ]; then
echo "Error: You cannot specify both --tag and --branch."
usage
fi
tag="$2"
shift
;;
--branch)
if [ -n "$tag" ]; then
echo "Error: You cannot specify both --tag and --branch."
usage
fi
branch="$2"
shift
;;
*)
echo "Unknown parameter passed: $1"
usage
;;
esac
shift
done

yum update && yum install -y git java-11-amazon-corretto-devel docker nodejs https://s3.amazonaws.com/session-manager-downloads/plugin/latest/linux_64bit/session-manager-plugin.rpm
systemctl start docker
git init
git remote | grep "origin" || git remote add -f origin https://github.com/opensearch-project/opensearch-migrations.git

if [ -n "$branch" ]; then
git checkout $branch
elif [ -n "$tag" ]; then
git checkout tags/$tag
else
latest_release_tag=$(curl -s https://api.github.com/repos/opensearch-project/opensearch-migrations/releases/latest | jq -r ".tag_name")
git checkout tags/$latest_release_tag
fi

cd deployment/cdk/opensearch-service-migration || exit
npm install -g aws-cdk 2>&1
npm install 2>&1
./buildDockerImages.sh
7 changes: 7 additions & 0 deletions deployment/migration-assistant-solution/jest.config.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
/** @type {import('ts-jest').JestConfigWithTsJest} **/
module.exports = {
testEnvironment: "node",
transform: {
"^.+.tsx?$": ["ts-jest",{}],
},
};
Loading