在跑Lambda时,通常需要安装python安装包,这里记录一个自动化方式来完成
构建Docker时,先构建下列文件夹
.
├── container/
│ ├── app/
│ │ ├──app.py
│ ├── Dockerfile
|
├── build_and_push.sh
├── config
├── create_lambda.sh
其中:
config里面放的是配置
s3_path
:Lambda运行的代码在s3上的路径docker_name
:ECR名字,简称lambda_name
:Lambda名字handler
:python文件名.程序名,例如app.py中的handler(),则是app.handlers3_bucket
:Lambda运行的代码所在的s3 buckets3_prefix
:Lambda运行的代码所在的s3 prefixlambda_role
:Lambda运行的Roletimeout
:超时时间(秒)memorysize
:内存大小(MB)ecr
:ECR在s3上的全名role
:Lambda运行的Role
container
: 中放的是 Lambda 所用的 docker 镜像app
:里面放的是要在Lambda里面跑的代码
build_and_push.sh
:是打docker镜像并推导s3上的脚本create_lambda.sh
:构建Lambda,具体步骤1.把container/app里面的代码打包并上传到s3_path
对应的s3目录,2.删除原有的Lambd,3.构建新的Lambda
其中,app.py内容:
import sys
def handler(event, context):
return '[TEST123] Hello from AWS Lambda using Python' + sys.version + '!'
Dockerfile内容:
# Define function directory
ARG FUNCTION_DIR="/function"
FROM python:buster as build-image
# Install aws-lambda-cpp build dependencies
RUN apt-get update && \
apt-get install -y \
g++ \
make \
cmake \
unzip \
libcurl4-openssl-dev
# Include global arg in this stage of the build
ARG FUNCTION_DIR
# Create function directory
RUN mkdir -p ${FUNCTION_DIR}
# Copy function code
COPY app/* ${FUNCTION_DIR}
# Install the runtime interface client
RUN pip install \
--target ${FUNCTION_DIR} \
awslambdaric
# Multi-stage build: grab a fresh copy of the base image
FROM python:buster
# Include global arg in this stage of the build
ARG FUNCTION_DIR
# Set working directory to function root directory
WORKDIR ${FUNCTION_DIR}
# Copy in the build image dependencies
COPY --from=build-image ${FUNCTION_DIR} ${FUNCTION_DIR}
ENTRYPOINT [ "/usr/local/bin/python", "-m", "awslambdaric" ]
CMD [ "app.handler" ]
make_lambda.sh内容:
zip -r9 function.zip lambda_function.py
aws lambda delete-function --function-name $1
aws lambda create-function --function-name $1 --runtime python3.8 --zip-file fileb://function.zip --handler lambda_function.lambda_handler --role arn:aws-cn:iam::468855463735:role/Lambda_for_SageMaker_Endpoint_Predict --timeout 600 --memory-size 512
create_lambda.sh内容
#!/bin/bash
string=$(cat config | grep s3_path)
array=(${string/// })
for var in ${array[@]}
do
if [ $var != 's3_path' ];then
s3_path=$var
fi
done
string=$(cat config | grep lambda_name)
array=(${string/// })
for var in ${array[@]}
do
if [ $var != 'lambda_name' ];then
lambda_name=$var
fi
done
string=$(cat config | grep handler)
array=(${string/// })
for var in ${array[@]}
do
if [ $var != 'handler' ];then
handler=$var
fi
done
string=$(cat config | grep s3_bucket)
array=(${string/// })
for var in ${array[@]}
do
if [ $var != 's3_bucket' ];then
s3_bucket=$var
fi
done
string=$(cat config | grep s3_prefix)
array=(${string/// })
for var in ${array[@]}
do
if [ $var != 's3_prefix' ];then
s3_prefix=$var
fi
done
string=$(cat config | grep ecr)
array=(${string/// })
for var in ${array[@]}
do
if [ $var != 'ecr' ];then
ecr=$var
fi
done
string=$(cat config | grep role)
array=(${string/// })
for var in ${array[@]}
do
if [ $var != 'role' ];then
role=$var
fi
done
# zip lambda code and upload
cd container/app/
zip -gr ../../functions.zip .
cd ../../
aws s3 cp functions.zip s3://zbrl-sagemaker-code/temp/
# cd app
# zip -g functions.zip .
# aws s3 cp functions.zip s3_path
echo s3_path $s3_path
echo lambda_name $lambda_name
echo handler $handler
echo s3_bucket $s3_bucket
echo s3_prefix $s3_prefix
echo ecr $ecr
aws lambda delete-function --function-name $lambda_name
aws lambda create-function --function-name $lambda_name --handler $handler --role $role --timeout 600 --memory-size 512 --code ImageUri=$ecr,S3Bucket=$s3_bucket,S3Key=$s3_prefix'functions.zip' --runtime python3.8
config内容:
s3_path s3://xxx/yyy/
docker_name lambda/xxxyyy
lambda_name test_lambda
handler app.handler
s3_bucket xxx
s3_prefix yyy/
lambda_role arn:aws-cn:iam::xxx:role/xxx
timeout 600
memorysize 512
ecr xxx.dkr.ecr.cn-northwest-1.amazonaws.com.cn/lambda/xxxyyy
role arn:aws-cn:iam::xxx:role/xxx
build_and_push.sh内容
#!/bin/bash
string=$(cat config | grep docker_name)
array=(${string/// })
for var in ${array[@]}
do
if [ $var != 'docker_name' ];then
docker_name=$var
fi
done
echo Docker name is : $docker_name
cd container
account=$(aws sts get-caller-identity --query Account --output text)
# Get the region defined in the current configuration (default to us-west-2 if none defined)
region=$(aws configure get region)
region=${region:-us-west-2}
# 如果不是China Region,删掉.cn
fullname="${account}.dkr.ecr.${region}.amazonaws.com.cn/${docker_name}:latest"
# If the repository doesn't exist in ECR, create it.
aws ecr describe-repositories --repository-names "${docker_name}" > /dev/null 2>&1
if [ $? -ne 0 ]
then
aws ecr create-repository --repository-name "${docker_name}" > /dev/null
fi
# Get the login command from ECR and execute it directly
aws ecr get-login-password --region ${region}|docker login --username AWS --password-stdin ${fullname}
# Build the docker image locally with the image name and then push it to ECR
# with the full name.
docker build -t ${docker_name} .
docker tag ${docker_name} ${fullname}
docker push ${fullname}