Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. SPDX-License-Identifier: CC-BY-SA-4.0
First, we’ll use the AWS CDK to deploy some prerequisites. Our guiding principle is that we’ll use the CDK to deploy static infrastructure and prerequisites that are out of scope for this lab, and use a CI/CD pipeline to deploy the rest. For example, if we’re building a stream processor, we might assume that the Kafka cluster is already in operation, but we need to deploy our actual stream processing application.
Pick an AWS region to work in, such as
us-west-2. We’ll refer to this as
REGION going forward.
Also note your AWS account number. You find this in the console or by running
aws sts get-caller-identity on the CLI. We’ll refer to this as
ACCOUNT going forward.
In the AWS console, go to the Cloud9 service and select
Create environment. Call your new IDE
FargateIDE and click
Next Step. On the next screen, change the instance type to
m4.large and click
Next step again. On the final page, click
Create environment. Make sure that you leave the VPC settings at the default values.
Once the environment builds, you’ll automatically redirect to the IDE. Take a minute to explore the interface, and note that you can change the color scheme if you like (AWS Cloud9 menu -> Preferences -> Themes).
Next, let’s update the Cloud9 environment to let you run the labs from the environment.
AWS servicefor the entity and leave the service set to
AWS managed temporary credentials.
Note that this role grants a very broad set of permissions to your Cloud9 instance, allowing it to use the CDK to create several types of AWS resources.
In your Cloud 9 environment, install the CDK and update some dependencies:
npm install -g email@example.com
Next clone the Git repo:
git clone https://github.com/aws-samples/amazon-ecs-fargate-workshop-dev-ops-data.git fargate-workshop cd fargate-workshop
Next we need to create a Python virtual environment.
virtualenv .env source .env/bin/activate
Now we install some CDK modules.
pip install awscli pip install --upgrade aws-cdk.core pip install -r labs/requirements.txt
Create the file
~/.aws/config with these lines:
[default] region=REGION account=ACCOUNT
Run these commands to produce a zip file with our function code:
cd labs/fargate-workshop-cdk/fargate_workshop_cdk pip install --target ./package kafka-python cd package zip -r9 ../function.zip . cd .. zip -g function.zip kafka-producer.py
We’re now ready to deploy the prerequisites. Run the following, making sure to substitute the proper values for your
cd ~/environment/fargate-workshop/labs/fargate-workshop-cdk touch ~/.aws/credentials cdk bootstrap aws://ACCOUNT/REGION cdk synth cdk deploy fargate-workshop-network # Also deploys stack fargate-workshop-dataeng-cluster cdk deploy fargate-workshop-discovery # Also deploys stack fargate-workshop-dataeng cdk deploy fargate-workshop-dataeng-lambda
This is what we’ll have after the deployment is complete: