AWS (Amazon Web Services.md

AWS (Amazon Web Services)

• Типы инстансов Amazon EC2

---

• AWS Machine Learning Blog
• Изучите возможности для Python на AWS
• AWS Documentation - Find user guides, developer guides, API references, tutorials, and more
• • Создание, обучение и развертывание моделей машинного обучения с помощью Amazon SageMaker
• Hands-on Tutorials - Explore more tutorials from integrating intelligence to your apps to optimizing your models

  Ознакомьтесь с практическими учебными пособиями и обучающими видеоматериалами по работе с сервисом AWS Начните работу и запустите первую рабочую нагрузку с помощью простых пошаговых учебных пособий

• Coursera
  • Getting Started with AWS Machine Learning
• Edx
  • AWS, Free online courses from Amazon Web Services

Title	Description
Amazon SageMaker	Amazon SageMaker Documentation Amazon SageMaker Pipelines - First purpose-built CI/CD service for machine learning How Genworth built a serverless ML pipeline on AWS using Amazon SageMaker and AWS Glue Creating a machine learning-powered REST API with Amazon API Gateway mapping templates and Amazon SageMaker
AWS Lambda	Lambda quotas Deploy Python Lambda functions with .zip file archives AWS Lambda — теория AWS Lambdas and layers for Machine Learning
Amazon Simple Storage Service Documentation	Creating a bucket Bucket naming rules create-bucket Using high-level (s3) commands with the AWS CLI Uploading objects
AWS Elastic Beanstalk	Complete Machine Learning solution(Part 3/3): Deploy Flask application on AWS Elastic Beanstalk Deploy Machine learning Flask app using ElasticBeanStalk service on AWS Deploy a machine learning model with AWS Elastic Beanstalk
Amazon Elastic File System	Shared File System for Your Lambda Functions
Amazon Elastic Container Service
AWS DeepRacer	Developers of all skill levels can get hands on with machine learning through a cloud based 3D racing simulator, fully autonomous 1/18th scale race car driven by reinforcement learning, and global racing league.
Amazon Mechanical Turk	Amazon Mechanical Turk (MTurk) is a crowdsourcing marketplace that makes it easier for individuals and businesses to outsource their processes and jobs to a distributed workforce who can perform these tasks virtually. This could include anything from conducting simple data validation and research to more subjective tasks like survey participation, content moderation, and more. MTurk enables companies to harness the collective intelligence, skills, and insights from a global workforce to streamline business processes, augment data collection and analysis, and accelerate machine learning development.

Examples

GitHub

Title	Description
Amazon SageMaker Script Mode Examples	Amazon SageMaker examples for prebuilt framework mode containers, a.k.a. Script Mode, and more (BYO containers and models etc.)
amazon-sagemaker-immersion-day	Amazon SageMaker Immersion Day help customers and partners to provide end to end understanding of building ML use cases from feature engineering to understanding various in-built algorithm and Train , Tune and Deploy the ML model in production like scenario. It guides you to bring your own model and perform on-premise ML workload Lift-and-Shift to Amazon SageMaker platform. It further demonstrate advance concept like Model Debugging , Model Monitoring and AutoML and guide to evaluate your machine leaning workload through AWS ML Well-architect lens.

SageMaker Studio

Write .csv file into S3 bucket

Using AWS Data Wrangler

Pandas on AWS

• AWS Data Wrangler Documentation
• AWS Data Wrangler on GitHub

!pip install awswrangler

awswrangler.s3.to_csv

❗Note : to make it work, the awswrangler has to be imported first

import awswrangler as wr

wr.s3.to_csv(
    df=df_new,
    path='s3://'
)

Articles

• Deploying A Pre-Trained Sklearn Model on Amazon SageMaker

AWS Lambda

• Building Scikit-Learn For AWS Lambda
  • Error: No module named 'sklearn.__check_build._check_build'
  • Search:
    • Using Scikit-Learn In AWS Lambda
    • Error with scikit-learn module in AWS Lambda
  • Building Scikit-Learn For AWS Lambda
  • sklearn and pandas in AWS Lambda
  • Установка sklearn на AWS deeplearning AMI
  • sklearn not working in aws lambda function. Deployment package issue

Articles

• Возможности AWS Lambda
• AWS Machine Learning Blog
  • Deploying machine learning models as serverless APIs
  • How to Deploy Deep Learning Models with AWS Lambda and Tensorflow
  • Code-free machine learning: AutoML with AutoGluon, Amazon SageMaker, and AWS Lambda
• AWS Documentation
  • Create a Lambda Function and Deploy a Custom Trained Model to AWS DeepLens
  • Using container images to run TensorFlow models in AWS Lambda
  • Tutorial: Configuring a Lambda function to access Amazon RDS in an Amazon VPC
• Towards Data Science / Medium
  • How to deploy a Machine Learning model on AWS Lambda
  • Deploy Machine Learning Models On AWS Lambda
  • Serverless Machine Learning Engineering Project On AWS Lambda
  • Deploying Machine Learning Model to AWS Lambda using Serverless
  • Containerized ML deployment with AWS Lambda
  • How to Deploy ML models with AWS Lambda
  • How to Build an AWS Lambda for Data Science

Doker and Lambda

To create an image from an AWS base image for Lambda

1. On your local machine, create a project directory for your new function.

2. Create a directory named app in in the project directory, and then add your function handler code to the app directory.

3. Use a text editor to create a new Dockerfile.

4. Build your Docker image with the docker build command. Enter a name for the image. The following example names the image hello-world.

    docker build -t hello-world .  
    docker build -t hello-world:latest -f Dockerfile -m 12g .
   

5. Start the Docker image with the docker run command. For this example, enter hello-world as the image name.

    docker run -p 9000:8080 hello-world
   

6. (Optional) Test your application locally using the runtime interface emulator. From a new terminal window, post an event to the following endpoint using a curl command:

    curl -XPOST "http://localhost:9000/2015-03-31/functions/function/invocations" -d '{}'
   

7. Authenticate the Docker CLI to your Amazon ECR registry.

    aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin 123456789012.dkr.ecr.us-east-1.amazonaws.com
   

8. Create a repository in Amazon ECR using the create-repository command.

    aws ecr create-repository --repository-name hello-world --image-scanning-configuration scanOnPush=true --image-tag-mutability MUTABLE
   

9. Tag your image to match your repository name.

    docker tag  hello-world:latest 123456789012.dkr.ecr.us-east-1.amazonaws.com/hello-world:latest
   

10. Deploy the image to Amazon ECR using the docker push command.

    docker push 123456789012.dkr.ecr.us-east-1.amazonaws.com/hello-world:latest
    

🔗 Links

• Creating Lambda container images
• Updating the function code (console) -> Creating Lambda functions defined as container images

AWS

• SERVERLESS DATA PROCESSING ON AWS
• Deploy your Machine Learning model as a REST API on AWS

Libraries

Title	Description
Boto3	You use the AWS SDK for Python (Boto3) to create, configure, and manage AWS services, such as Amazon Elastic Compute Cloud (Amazon EC2) and Amazon Simple Storage Service (Amazon S3). The SDK provides an object-oriented API as well as low-level access to AWS services.