Giter VIP home page Giter VIP logo

awslabs / web-client-for-aws-transfer-family Goto Github PK

View Code? Open in Web Editor NEW
71.0 9.0 40.0 411 KB

This solution creates a web portal for your customers to access your corporate Secure Shell File Transfer Protocol (SFTP) environment. It combines the benefits of using AWS Transfer for SFTP with an intuitive web browser interface for your non-technical users.

Home Page: https://aws.amazon.com/solutions/implementations/web-client-for-aws-transfer-family/?did=sl_card&trk=sl_card

License: Apache License 2.0

Shell 12.52% JavaScript 1.49% TypeScript 26.31% HTML 10.29% Dockerfile 0.99% Python 38.99% CSS 3.53% SCSS 5.88%
aws-transfer sftp-client aws

web-client-for-aws-transfer-family's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

web-client-for-aws-transfer-family's Issues

Error 413 Request Entity Too Large When uploading file

We got "413 Request Entity Too Large:The data value transmitted exceeds the capacity limit." when trying to upload 500MB files via the web client.
The error when inspecting the page during the upload is
{
"message": "Invalid request: The data value transmitted exceeds the capacity limit.",
"status": 500
}
When investigating on the ecs logs we got "413 Request Entity Too Large:The data value transmitted exceeds the capacity limit."
Based on logs in sftp (transfert family) we know that the request didnt arrive to the sftp server, it's inside the backend code running on ecs we are assuming.

To repoduce:
Connect to the web client and try to upload file up to 150 MB

SFTP Endpoint Stack Create failures

03- sftp-endpoint.template

SFTPCustomAuthLambda | CREATE_FAILED | Properties validation failed for resource SFTPCustomAuthLambda with message: #/Code/S3Bucket: failed validation constraint for keyword [pattern]


AWSTemplateFormatVersion: "2010-09-09"
Description: (SO0103-sftpendpoint) %%SOLUTION_NAME%% - This template creates Lambda-backed API Gateway for integrating with custom identity provider in AWS Transfer Family with SFTP.

Parameters:

AWSTransferForSFTPS3Bucket:
Type: String
Description : The name of the S3 bucket used for the SFTP server

#TODO: what are requirements? Just create it instead of relying on default sg.
AWSTransferVPCSecGroup:
Type: AWS::EC2::SecurityGroup::Id
Description: Default Security Group for the VPC identified before

VPCResourceStack:
Type: String
MinLength: 1
MaxLength: 255
AllowedPattern: "^[a-zA-Z][-a-zA-Z0-9]*$"
Default: sftp-vpc-stack

CognitoResourceStack:
Type: String
MinLength: 1
MaxLength: 255
AllowedPattern: "^[a-zA-Z][-a-zA-Z0-9]*$"
Default: sftp-cognito-stack

Mappings:
SourceCode:
General:
S3Bucket: "%%BUCKET_NAME%%"
KeyPrefix: "%%SOLUTION_NAME%%/%%VERSION%%"

Resources:

TransferSFTPCloudWatchLogsWriteLogs:
Description: Allows AWS Transfer Family to write CloudWatchLogs
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Principal:
Service: transfer.amazonaws.com
Action: sts:AssumeRole
Path: "/"
Policies:
- PolicyName: TransferSFTPWriteLogsPolicy
PolicyDocument:
Version: "2012-10-17"
Statement:
Sid: VisualEditor0
Effect: Allow
Action:
- logs:CreateLogStream
- logs:DescribeLogStreams
- logs:CreateLogGroup
- logs:PutLogEvents
Resource: "arn:aws:logs:::log-group:/aws/transfer/"
TransferSFTPS3Access: # Allows AWS Transfer Family to access S3 bucket
Description: Allows AWS Transfer Family to access S3 bucket
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Principal:
Service: transfer.amazonaws.com
Action: sts:AssumeRole
Path: "/"
Policies:
- PolicyName: TransferSFTPS3BucketAccessPolicy
PolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Action:
- s3:DeleteObject
- s3:PutObject
- s3:GetObject
- s3:DeleteObjectVersion
- s3:GetObjectVersion
- s3:GetObjectACL
- s3:PutObjectACL
Resource:
Fn::Sub: arn:aws:s3:::${AWSTransferForSFTPS3Bucket}/

- Effect: Allow
Action:
- s3:ListBucket
- s3:GetBucketLocation
Resource:
Fn::Sub: arn:aws:s3:::${AWSTransferForSFTPS3Bucket}
IdentityProviderApi:
Type: AWS::ApiGateway::RestApi
Properties:
Name:
Fn::Join:
- "-"
- - "Transfer Identity Provider basic template API"
- Fn::ImportValue: !Join [':', [!Ref 'VPCResourceStack', 'StackID']]
Description: API used for GetUserConfig requests
FailOnWarnings: true
EndpointConfiguration:
Types:
- REGIONAL

DynamoDB backup vault using AWS Backup

BackupVaultWithDailyBackups:
Type: "AWS::Backup::BackupVault"
Properties:
BackupVaultName:
Fn::Join:
- "-"
- - "DynamoDBBackupVaultWithDailyBackups"
- Fn::ImportValue: !Join [':', [!Ref 'VPCResourceStack', 'StackID']]

DynamoDB backup plan using AWS Backup

BackupPlanWithDailyBackups:
Type: "AWS::Backup::BackupPlan"
Properties:
BackupPlan:
BackupPlanName:
Fn::Join:
- "-"
- - "DynamoDBBackupPlanWithDailyBackups"
- Fn::ImportValue: !Join [':', [!Ref 'VPCResourceStack', 'StackID']]
BackupPlanRule:
- RuleName: "RuleForDailyBackups"
TargetBackupVault: !Ref BackupVaultWithDailyBackups
ScheduleExpression: "cron(0 5 ? * * *)"
DependsOn: BackupVaultWithDailyBackups

DynamoDB Table

SFTPUserDirectoryMapping:
Type: AWS::DynamoDB::Table
Metadata:
cfn_nag:
rules_to_suppress:
- id: W78
reason: "DynamoDB backup defined using AWS Backups"
- id: W28
reason: "Unique resource name defined for consistency across all resource names."
Properties:
TableName:
Fn::Join:
- "-"
- - "SFTPUserDirectoryMapping"
- Fn::ImportValue: !Join [':', [!Ref 'VPCResourceStack', 'StackID']]
AttributeDefinitions:
- AttributeName: "username"
AttributeType: "S"
KeySchema:
- AttributeName: "username"
KeyType: "HASH"
BillingMode: "PAY_PER_REQUEST"
SSESpecification:
SSEEnabled: True
Tags:
- Key: "ddbbackup"
Value: "daily"

IAM Role for AWS Backup

BackupRole:
Type: "AWS::IAM::Role"
Properties:
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: "Allow"
Principal:
Service:
- "backup.amazonaws.com"
Action:
- "sts:AssumeRole"
ManagedPolicyArns:
- "arn:aws:iam::aws:policy/service-role/AWSBackupServiceRolePolicyForBackup"

Tag based DynamoDB Backup

TagBasedBackupSelection:
Type: "AWS::Backup::BackupSelection"
Properties:
BackupSelection:
SelectionName: "TagBasedBackupSelection"
IamRoleArn: !GetAtt BackupRole.Arn
ListOfTags:
- ConditionType: "STRINGEQUALS"
ConditionKey: "ddbbackup"
ConditionValue: "daily"
BackupPlanId: !Ref BackupPlanWithDailyBackups
DependsOn: BackupPlanWithDailyBackups

LambdaExecutionRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Principal:
Service:
- lambda.amazonaws.com
Action:
- sts:AssumeRole
Policies:
- PolicyName: CWLogsPermissionPolicy
PolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Action:
- logs:CreateLogGroup
Resource: !Sub 'arn:aws:logs:${AWS::Region}:${AWS::AccountId}:'
- Effect: Allow
Action:
- logs:CreateLogStream
- logs:PutLogEvents
Resource: !Sub 'arn:aws:logs:${AWS::Region}:${AWS::AccountId}:log-group:/aws/lambda/
'
- PolicyName: DynomoDBTableAccess
PolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Action:
- dynamodb:GetItem
Resource:
Fn::GetAtt: SFTPUserDirectoryMapping.Arn
- PolicyName: ParameterStoreAccess
PolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Action:
- ssm:GetParameter
- ssm:PutParameter
Resource: !Sub 'arn:aws:ssm:${AWS::Region}:${AWS::AccountId}:parameter/sftpui-*'
- PolicyName: CognitoUserPoolAccess
PolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Action:
- cognito-idp:DescribeUserPoolClient
- cognito-idp:AdminInitiateAuth
Resource:
Fn::ImportValue: !Sub '${CognitoResourceStack}:UserPoolArn'

ApiCloudWatchLogsRole:
Description: IAM role used by API Gateway to log API requests to CloudWatch
Type: AWS::IAM::Role
Metadata:
cfn_nag:
rules_to_suppress:
- id: W11
reason: "resource created after IAM role creation."
Properties:
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Principal:
Service:
- apigateway.amazonaws.com
Action:
- sts:AssumeRole
Policies:
- PolicyName: ApiGatewayLogsPolicy
PolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Action:
- logs:CreateLogGroup
- logs:CreateLogStream
- logs:DescribeLogGroups
- logs:DescribeLogStreams
- logs:PutLogEvents
- logs:GetLogEvents
- logs:FilterLogEvents
Resource: ""
ApiLoggingAccount:
Type: AWS::ApiGateway::Account
DependsOn:
- IdentityProviderApi
Properties:
CloudWatchRoleArn:
Fn::GetAtt: ApiCloudWatchLogsRole.Arn
ApiStage:
Type: AWS::ApiGateway::Stage
Metadata:
cfn_nag:
rules_to_suppress:
- id: W69
reason: "AccessLogSetting not required for this use case."
- id: W64
reason: "UsagePlan not required for this use case."
Properties:
DeploymentId:
Ref: ApiDeployment
MethodSettings:
- DataTraceEnabled: false
HttpMethod: "
"
LoggingLevel: INFO
ResourcePath: "/"
RestApiId:
Ref: IdentityProviderApi
StageName: prod
ApiDeployment:
DependsOn:
- GetUserConfigRequest
Type: AWS::ApiGateway::Deployment
Metadata:
cfn_nag:
rules_to_suppress:
- id: W68
reason: "UsagePlan not required for this use case."
Properties:
RestApiId:
Ref: IdentityProviderApi
StageName: stagefordeployment
TransferIdentityProviderRole:
Type: AWS::IAM::Role
Metadata:
cfn_nag:
rules_to_suppress:
- id: W11
reason: "resource created after IAM role creation."
Properties:
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Principal:
Service: transfer.amazonaws.com
Action:
- sts:AssumeRole
Policies:
- PolicyName: TransferCanInvokeThisApi
PolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Action:
- execute-api:Invoke
Resource:
Fn::Sub: arn:${AWS::Partition}:execute-api:${AWS::Region}:${AWS::AccountId}:${IdentityProviderApi}/prod/GET/

- PolicyName: TransferCanReadThisApi
PolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Action:
- apigateway:GET
Resource: "*"
- PolicyName: TransferCanVerifyReturnedUserRole
PolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Action: iam:PassRole
Resource:
Fn::GetAtt: TransferSFTPS3Access.Arn

SFTPCustomAuthLambda:
Type: AWS::Lambda::Function
Metadata:
cfn_nag:
rules_to_suppress:
- id: W58
reason: "CW Logs permissions defined"
- id: W89
reason: "VPC configuration for Lambda not supported for this use case."
- id: W92
reason: "reserved concurrency not required for this use case."
- id: W24
reason: "approprate permissions are assigned to the role."
Description: A function to provide IAM roles and policies for given user and serverId.
Properties:
Handler: SFTPCustomAuthLambdaFunction.lambda_handler
Role:
Fn::GetAtt: LambdaExecutionRole.Arn
Runtime: python3.7
Timeout: 5
Environment:
Variables:
USER_POOL_ID:
Fn::ImportValue: !Sub '${CognitoResourceStack}:UserPoolId'
CLIENT_ID:
Fn::ImportValue: !Sub '${CognitoResourceStack}:UserPoolClientId'
SFTPUSRDIRMAP: !Ref SFTPUserDirectoryMapping
STACK_ID:
Fn::ImportValue: !Join [':', [!Ref 'VPCResourceStack', 'StackID']]
botoConfig: '{"user_agent_extra": "AwsSolution/SO0103/%%VERSION%%"}'
ROLE_ARN:
Fn::GetAtt: TransferSFTPS3Access.Arn
Code:
S3Bucket:
!Join [
"-",
[
!FindInMap ["SourceCode", "General", "S3Bucket"],
Ref: "AWS::Region",
],
]
S3Key:
!Join [
"/lambda/",
[
!FindInMap ["SourceCode", "General", "KeyPrefix"],
"SFTPCustomAuthLambdaFunction.zip",
],
]

SFTPCustomAuthLambdaPermission:
Type: AWS::Lambda::Permission
Metadata:
cfn_nag:
rules_to_suppress:
- id: W24
reason: "approprate permissions are assigned to the role."
Properties:
Action: lambda:invokeFunction
FunctionName:
Fn::GetAtt: SFTPCustomAuthLambda.Arn
Principal: apigateway.amazonaws.com
SourceArn:
Fn::Join:
- ""
- - "arn:aws:execute-api:"
- Ref: AWS::Region
- ":"
- Ref: AWS::AccountId
- ":"
- Ref: IdentityProviderApi
- "/*"
ServersResource:
Type: AWS::ApiGateway::Resource
Properties:
RestApiId:
Ref: IdentityProviderApi
ParentId:
Fn::GetAtt:
- IdentityProviderApi
- RootResourceId
PathPart: servers
ServerIdResource:
Type: AWS::ApiGateway::Resource
Properties:
RestApiId:
Ref: IdentityProviderApi
ParentId:
Ref: ServersResource
PathPart: "{serverId}"
UsersResource:
Type: AWS::ApiGateway::Resource
Properties:
RestApiId:
Ref: IdentityProviderApi
ParentId:
Ref: ServerIdResource
PathPart: users
UserNameResource:
Type: AWS::ApiGateway::Resource
Properties:
RestApiId:
Ref: IdentityProviderApi
ParentId:
Ref: UsersResource
PathPart: "{username}"
GetUserConfigResource:
Type: AWS::ApiGateway::Resource
Properties:
RestApiId:
Ref: IdentityProviderApi
ParentId:
Ref: UserNameResource
PathPart: config
GetUserConfigRequest:
Type: AWS::ApiGateway::Method
DependsOn: GetUserConfigResponseModel
Properties:
AuthorizationType: AWS_IAM
HttpMethod: GET
Integration:
Type: AWS
IntegrationHttpMethod: POST
Uri:
Fn::Join:
- ""
- - "arn:aws:apigateway:"
- Ref: AWS::Region
- ":lambda:path/2015-03-31/functions/"
- Fn::GetAtt:
- SFTPCustomAuthLambda
- Arn
- "/invocations"
IntegrationResponses:
- StatusCode: 200
RequestTemplates:
application/json: |
{
"username": "$input.params('username')",
"password": "$util.escapeJavaScript($input.params('Password')).replaceAll("\'","'")",
"serverId": "$input.params('serverId')"
}
RequestParameters:
method.request.header.Password: false
ResourceId:
Ref: GetUserConfigResource
RestApiId:
Ref: IdentityProviderApi
MethodResponses:
- StatusCode: 200
ResponseModels:
application/json: UserConfigResponseModel
GetUserConfigResponseModel:
Type: AWS::ApiGateway::Model
Properties:
RestApiId:
Ref: IdentityProviderApi
ContentType: application/json
Description: API response for GetUserConfig
Name: UserConfigResponseModel
Schema:
"$schema": http://json-schema.org/draft-04/schema#
title: UserUserConfig
type: object
properties:
HomeDirectory:
type: string
Role:
type: string
Policy:
type: string
PublicKeys:
type: array
items:
type: string
AWSSFTPEIPIP01:
Type: AWS::EC2::EIP
Properties:
Domain: vpc
AWSSFTPEIPIP02:
Type: AWS::EC2::EIP
Properties:
Domain: vpc
AWSSFTPEIPIP03:
Type: AWS::EC2::EIP
Properties:
Domain: vpc
AWSSFTPEndPoint:
Type: AWS::Transfer::Server
Properties :
EndpointType: VPC
EndpointDetails:
AddressAllocationIds:
- !GetAtt AWSSFTPEIPIP01.AllocationId
- !GetAtt AWSSFTPEIPIP02.AllocationId

    SubnetIds:
      - Fn::ImportValue: !Sub '${VPCResourceStack}:PublicSubnet1'
      - Fn::ImportValue: !Sub '${VPCResourceStack}:PublicSubnet2'
      
    VpcId:
      Fn::ImportValue: !Sub "${VPCResourceStack}:VPCID"
  IdentityProviderType: API_GATEWAY
  IdentityProviderDetails:
    InvocationRole: !GetAtt TransferIdentityProviderRole.Arn
    Url:
      Fn::Join:
      - ""
      - - https://
        - Ref: IdentityProviderApi
        - .execute-api.
        - Ref: AWS::Region
        - .amazonaws.com/
        - Ref: ApiStage
  LoggingRole: !GetAtt TransferSFTPCloudWatchLogsWriteLogs.Arn

AWSSFTPVPCSecGroupIngressRule:
Type: AWS::EC2::SecurityGroupIngress
Properties:
Description: "Allow external SFTP traffic"
IpProtocol: tcp
CidrIp: 0.0.0.0/0
FromPort: 22
ToPort: 22
GroupId: !Ref AWSTransferVPCSecGroup

#Custom resource section to get DNSentries for the AWS Transfer VPC EndpointType
CustomResourceLambdaExecutionRole:
Type: AWS::IAM::Role
Metadata:
cfn_nag:
rules_to_suppress:
- id: W11
reason: "resource created after IAM role creation."
Properties:
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Principal:
Service:
- lambda.amazonaws.com
Action:
- sts:AssumeRole
Path: "/"
Policies:
- PolicyName: root
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Action:
- logs:CreateLogGroup
- logs:CreateLogStream
- logs:PutLogEvents
Resource: arn:aws:logs:::*
- Effect: Allow
Action:
- ec2:DescribeVpcEndpointServices
- ec2:DescribeVpcEndpointServicePermissions
- ec2:DescribeVpcEndpointServices
- ec2:DescribeVpcEndpointServiceConfigurations
- ec2:DescribeVpcEndpoints
- ec2:DescribeVpcEndpointConnections
Resource: ""
- Effect: Allow
Action:
- transfer:DescribeServer
Resource: !GetAtt 'AWSSFTPEndPoint.Arn'
- PolicyName: ParameterStoreAccess
PolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Action:
- ssm:GetParameter
- ssm:PutParameter
- ssm:DeleteParameter
Resource: !Sub 'arn:aws:ssm:${AWS::Region}:${AWS::AccountId}:parameter/sftpui-
'
TransferMiscConfigDetails:
Type: Custom::TransferMiscConfigDetails
Properties:
ServiceToken: !GetAtt 'TransferMiscConfigLambdaFunction.Arn'
TransferServerId: !GetAtt 'AWSSFTPEndPoint.ServerId'
TransferServerArn: !GetAtt 'AWSSFTPEndPoint.Arn'
TransferMiscConfigLambdaFunction:
Type: AWS::Lambda::Function
Metadata:
cfn_nag:
rules_to_suppress:
- id: W58
reason: "CW Logs permissions defined"
- id: W92
reason: "reserved concurrency not required for this use case."
- id: W89
reason: "VPC configuration not required for this use case."
Properties:
Environment:
Variables:
STACK_ID:
Fn::ImportValue: !Join [':', [!Ref 'VPCResourceStack', 'StackID']]
botoConfig: '{"user_agent_extra": "AwsSolution/SO0103/%%VERSION%%"}'
Code:
S3Bucket:
!Join [
"-",
[
!FindInMap ["SourceCode", "General", "S3Bucket"],
Ref: "AWS::Region",
],
]
S3Key:
!Join [
"/lambda/",
[
!FindInMap ["SourceCode", "General", "KeyPrefix"],
"TransferMiscConfigLambdaFunction.zip",
],
]
Handler: TransferMiscConfigLambdaFunction.lambda_handler
Runtime: python3.7
Timeout: 10
Role: !GetAtt CustomResourceLambdaExecutionRole.Arn

Outputs:
StackArn:
Value:
Ref: AWS::StackId
TransferIdentityProviderUrl:
Description: URL to pass to AWS Transfer CreateServer call as part of optional IdentityProviderDetails
Value:
Fn::Join:
- ""
- - https://
- Ref: IdentityProviderApi
- .execute-api.
- Ref: AWS::Region
- .amazonaws.com/
- Ref: ApiStage
TransferIdentityProviderInvocationRole:
Description: IAM Role to pass to AWS Transfer CreateServer call as part of optional IdentityProviderDetails
Value:
Fn::GetAtt: TransferIdentityProviderRole.Arn
TransferVPCEndpointDNSEntry:
Description: Private DNS entry from the Transfer VPC endpoint. This will be used by the application running in Private subnet to interact with AWS Transfer endpoint.
Value:
Fn::GetAtt: TransferMiscConfigDetails.Data
Export:
Name: !Join [ ':', [ !Ref 'AWS::StackName', 'TransferVPCEndpointDNSEntry' ] ]
JWTSecretKeyParameterARN:
Description: ARN of the parameter that stores JWT secret key. It will be used in later templates
Value:
Fn::GetAtt: TransferMiscConfigDetails.JWT_Secret_Key_Parameter_ARN
Export:
Name: !Join [ ':', [ !Ref 'AWS::StackName', 'JWTSecretKeyParameterARN' ] ]
AWSTransferVPCSecGroup:
Description: AWS Transfer VPC security group.
Value: !Ref 'AWSTransferVPCSecGroup'
Export:
Name: !Join [ ':', [ !Ref 'AWS::StackName', 'AWSTransferVPCSecGroup' ] ]

Template 03 Fails to Deploy

On deployment of the third tempalte - 03-sftp-endpoint.template the deployment fails every attempt. The deployment fails on the following error:

"Properties validation failed for resource SFTPCustomAuthLambda with message:
#/Code/S3Bucket: failed validation constraint for keyword [pattern]"

I went into the template and changed the template under Mappings for the S3Bucket and hard coded the "bucketname" into this field. This seems to have helped but causes my deployment to fail on a new error every time:

Resource handler returned message: "Error occurred while GetObject. S3 Error Code: NoSuchBucket. S3 Error Message: The specified bucket does not exist (Service: Lambda, Status Code: 400, Request ID: redacted)" (RequestToken: redacted, HandlerErrorCode: InvalidRequest)

I am unable to move past this step and seems like the template fails no matter where I deploy it on any bucket or any AWS account.

Step 3 fails due to NoSuchBucket

| 2021-11-13T22:24:34.486Z | SFTPCustomAuthLambd | CREATE_FAILED | Resource handler returned message: "Error occurred while GetObject. S3 Error Code: NoSuchBucket.

does this require some prefix or something?

Lambda@Edge function creation fails

Running script 06a-add-security-header.sh the ClouFormation stack doesn't get created with error:
"The runtime parameter of nodejs10.x is no longer supported for creating or updating AWS Lambda functions."

How to improve web interface performance?

I noticed user login and interaction with SFTP server is slower in web interface. Is there any way to improve the performance? Adding more CPU and RAM in Faragate will help?
One other question. How to configure MFA and create reset password for SFTP server user?
@shardhak

what creates SFTPCustomAuthLambdaFunction.zip? 03-sftp-endpoint stack fails

I'm on step 3 in creating the 03-sftp-endpoint stack.
The error is when creating the SFTPCustomAuthLambda resource.

The code for that lambda says it lives here:

      Code:
        S3Bucket:
          !Join [
            "-",
            [
            !FindInMap ["SourceCode", "General", "S3Bucket"],
            Ref: "AWS::Region",
            ],
          ]
        S3Key:
          !Join [
            "/lambda/",
            [
            !FindInMap ["SourceCode", "General", "KeyPrefix"],
            "SFTPCustomAuthLambdaFunction.zip",
            ],
          ]

But I can' find any instructions or shell scripts on what creates and uploads that zip file.

Any idea?

Unable to Authenticate to AWS SFTP Transfer Family using Crowd

Ive built a Lambda in Python to auth with Atlassian Crowd, I get the correct HomeDirectory Response and so on, but I am unable to authenticate that to the AWS SFTP Transfer Family Service. When I try I get:

msg: "Missing cookie "access_token_cookie""

Using Cognito is not an option for this request. For comparison this Lambda gets the exact response Cognito provides, with different request headers.

From the documentation for AWS SFTP Transfer Family , verbatim, it says this:

"The JWT tokens are set in the access cookie (default name is set to access_token_cookie)
and refresh cookie (default name is set to refresh_token_cookie) respectively. The tokens are used in
subsequent HTTPS requests from the client to the web application where the python backend decrypts
JWT tokens using KMS Decrypt API to get the username and password for that request and authenticate"

Trying to replicate that ^

Cannot login, blank red box after typing username and password

Hello,

We have deployed the solution and created user + dynamodb mapping for the user. But while logging into the URL, it gives red box with no information. While checking the browser logs, observe below ones.

SyntaxError: Unexpected token '<', ")

I saw that two more issues were raised with similar issue but not sure if that was resolved as i don't see the solution.

Bucket cannot have ACLs set with ObjectOwnership's BucketOwnerEnforced setting

When i was using "05-sftp-fargate.template" template I faced the error
_Bucket cannot have ACLs set with ObjectOwnership's BucketOwnerEnforced setting (Service: Amazon S3; Status Code: 400; Error Code: InvalidBucketAclWithObjectOwnership_
I tried changing object ownership , changing ACL's also tried the solution given in stackoverflow, but didn't got any solution on this, I request anyone to give a appropriate solution (edited)

Unreadable architecture diagram

The architecture diagram that appears in the README.md as well as on the AWS storage blog article that introduces the web client uses tiny, blurry fonts. It's nearly unreadable. It should be reexported in a much larger resolution/size (and perhaps with a transparent background) so as to be usable, especially by people with vision difficulties (think Section 508).

Another possibility might be to use Mermaid to create the diagram.

Fargate Deployment issue and resolution

Hi Team,

This is to inform you about updating cloudformation template for Fargate. It was failing as it was unable to create S3 buckets due to the recent policy changes in S3 bucket. Adding below two lines to override the default behaviour helped me. We need to change it twice in the template as two buckets are being created.

Same issue for Webclient Template as well.

image

ECS Tasks keep terminating while Fargate Service is creating new ones.

My SFTP Backend ECS Task keep terminating with the logs here. Looks like its failing to import a bunch of modules, not sure why is it doing this.

2022-03-16 18:44:26[2022-03-16 13:14:26 +0000] [10] [ERROR] Exception in worker process
2022-03-16 18:44:26Traceback (most recent call last):
2022-03-16 18:44:26 File "/root/.local/lib/python3.8/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker
2022-03-16 18:44:26 worker.init_process()
2022-03-16 18:44:26 File "/root/.local/lib/python3.8/site-packages/gunicorn/workers/base.py", line 119, in init_process
2022-03-16 18:44:26 self.load_wsgi()
2022-03-16 18:44:26 File "/root/.local/lib/python3.8/site-packages/gunicorn/workers/base.py", line 144, in load_wsgi
2022-03-16 18:44:26 self.wsgi = self.app.wsgi()
2022-03-16 18:44:26 File "/root/.local/lib/python3.8/site-packages/gunicorn/app/base.py", line 67, in wsgi
2022-03-16 18:44:26 self.callable = self.load()
2022-03-16 18:44:26 File "/root/.local/lib/python3.8/site-packages/gunicorn/app/wsgiapp.py", line 49, in load
2022-03-16 18:44:26 return self.load_wsgiapp()
2022-03-16 18:44:26 File "/root/.local/lib/python3.8/site-packages/gunicorn/app/wsgiapp.py", line 39, in load_wsgiapp
2022-03-16 18:44:26 return util.import_app(self.app_uri)
2022-03-16 18:44:26 File "/root/.local/lib/python3.8/site-packages/gunicorn/util.py", line 358, in import_app
2022-03-16 18:44:26 mod = importlib.import_module(module)
2022-03-16 18:44:26 File "/usr/local/lib/python3.8/importlib/init.py", line 127, in import_module
2022-03-16 18:44:26 return _bootstrap._gcd_import(name[level:], package, level)
2022-03-16 18:44:26 File "", line 1014, in _gcd_import
2022-03-16 18:44:26 File "", line 991, in _find_and_load
2022-03-16 18:44:26 File "", line 975, in _find_and_load_unlocked
2022-03-16 18:44:26 File "", line 671, in _load_unlocked
2022-03-16 18:44:26 File "", line 843, in exec_module
2022-03-16 18:44:26 File "", line 219, in _call_with_frames_removed
2022-03-16 18:44:26 File "/app/transfer_sftp_backend.py", line 7, in
2022-03-16 18:44:26 from flask import Flask
2022-03-16 18:44:26 File "/root/.local/lib/python3.8/site-packages/flask/init.py", line 19, in
2022-03-16 18:44:26 from . import json
2022-03-16 18:44:26 File "/root/.local/lib/python3.8/site-packages/flask/json/init.py", line 15, in
2022-03-16 18:44:26 from itsdangerous import json as _json
2022-03-16 18:44:26ImportError: cannot import name 'json' from 'itsdangerous' (/root/.local/lib/python3.8/site-packages/itsdangerous/init.py)
2022-03-16 18:44:26[2022-03-16 13:14:26 +0000] [11] [ERROR] Exception in worker process
2022-03-16 18:44:26Traceback (most recent call last):
2022-03-16 18:44:26 File "/root/.local/lib/python3.8/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker
2022-03-16 18:44:26 worker.init_process()
2022-03-16 18:44:26 File "/root/.local/lib/python3.8/site-packages/gunicorn/workers/base.py", line 119, in init_process
2022-03-16 18:44:26 self.load_wsgi()
2022-03-16 18:44:26 File "/root/.local/lib/python3.8/site-packages/gunicorn/workers/base.py", line 144, in load_wsgi
2022-03-16 18:44:26 self.wsgi = self.app.wsgi()
2022-03-16 18:44:26 File "/root/.local/lib/python3.8/site-packages/gunicorn/app/base.py", line 67, in wsgi
2022-03-16 18:44:26 self.callable = self.load()
2022-03-16 18:44:26 File "/root/.local/lib/python3.8/site-packages/gunicorn/app/wsgiapp.py", line 49, in load
2022-03-16 18:44:26 return self.load_wsgiapp()
2022-03-16 18:44:26 File "/root/.local/lib/python3.8/site-packages/gunicorn/app/wsgiapp.py", line 39, in load_wsgiapp
2022-03-16 18:44:26 return util.import_app(self.app_uri)
2022-03-16 18:44:26 File "/root/.local/lib/python3.8/site-packages/gunicorn/util.py", line 358, in import_app
2022-03-16 18:44:26 mod = importlib.import_module(module)
2022-03-16 18:44:26 File "/usr/local/lib/python3.8/importlib/init.py", line 127, in import_module
2022-03-16 18:44:26 return _bootstrap._gcd_import(name[level:], package, level)
2022-03-16 18:44:26 File "", line 1014, in _gcd_import
2022-03-16 18:44:26 File "", line 991, in _find_and_load
2022-03-16 18:44:26 File "", line 975, in _find_and_load_unlocked
2022-03-16 18:44:26 File "", line 671, in _load_unlocked
2022-03-16 18:44:26 File "", line 843, in exec_module
2022-03-16 18:44:26 File "", line 219, in _call_with_frames_removed
2022-03-16 18:44:26 File "/app/transfer_sftp_backend.py", line 7, in
2022-03-16 18:44:26 from flask import Flask
2022-03-16 18:44:26 File "/root/.local/lib/python3.8/site-packages/flask/init.py", line 19, in
2022-03-16 18:44:26 from . import json
2022-03-16 18:44:26 File "/root/.local/lib/python3.8/site-packages/flask/json/init.py", line 15, in
2022-03-16 18:44:26 from itsdangerous import json as _json
2022-03-16 18:44:26ImportError: cannot import name 'json' from 'itsdangerous' (/root/.local/lib/python3.8/site-packages/itsdangerous/init.py)
2022-03-16 18:44:26[2022-03-16 13:14:26 +0000] [12] [ERROR] Exception in worker process
2022-03-16 18:44:26Traceback (most recent call last):
2022-03-16 18:44:26 File "/root/.local/lib/python3.8/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker
2022-03-16 18:44:26 worker.init_process()
2022-03-16 18:44:26 File "/root/.local/lib/python3.8/site-packages/gunicorn/workers/base.py", line 119, in init_process
2022-03-16 18:44:26 self.load_wsgi()
2022-03-16 18:44:26 File "/root/.local/lib/python3.8/site-packages/gunicorn/workers/base.py", line 144, in load_wsgi
2022-03-16 18:44:26 self.wsgi = self.app.wsgi()
2022-03-16 18:44:26 File "/root/.local/lib/python3.8/site-packages/gunicorn/app/base.py", line 67, in wsgi
2022-03-16 18:44:26 self.callable = self.load()
2022-03-16 18:44:26 File "/root/.local/lib/python3.8/site-packages/gunicorn/app/wsgiapp.py", line 49, in load
2022-03-16 18:44:26 return self.load_wsgiapp()
2022-03-16 18:44:26 File "/root/.local/lib/python3.8/site-packages/gunicorn/app/wsgiapp.py", line 39, in load_wsgiapp
2022-03-16 18:44:26 return util.import_app(self.app_uri)
2022-03-16 18:44:26 File "/root/.local/lib/python3.8/site-packages/gunicorn/util.py", line 358, in import_app
2022-03-16 18:44:26 mod = importlib.import_module(module)
2022-03-16 18:44:26 File "/usr/local/lib/python3.8/importlib/init.py", line 127, in import_module
2022-03-16 18:44:26 return _bootstrap._gcd_import(name[level:], package, level)
2022-03-16 18:44:26 File "", line 1014, in _gcd_import
2022-03-16 18:44:26 File "", line 991, in _find_and_load
2022-03-16 18:44:26 File "", line 975, in _find_and_load_unlocked
2022-03-16 18:44:26 File "", line 671, in _load_unlocked
2022-03-16 18:44:26 File "", line 843, in exec_module
2022-03-16 18:44:26 File "", line 219, in _call_with_frames_removed
2022-03-16 18:44:26 File "/app/transfer_sftp_backend.py", line 7, in
2022-03-16 18:44:26 from flask import Flask
2022-03-16 18:44:26 File "/root/.local/lib/python3.8/site-packages/flask/init.py", line 19, in
2022-03-16 18:44:26 from . import json
2022-03-16 18:44:26 File "/root/.local/lib/python3.8/site-packages/flask/json/init.py", line 15, in
2022-03-16 18:44:26 from itsdangerous import json as _json
2022-03-16 18:44:26ImportError: cannot import name 'json' from 'itsdangerous' (/root/.local/lib/python3.8/site-packages/itsdangerous/init.py)
2022-03-16 18:44:26[2022-03-16 13:14:26 +0000] [12] [INFO] Worker exiting (pid: 12)
2022-03-16 18:44:23[2022-03-16 13:14:23 +0000] [12] [INFO] Booting worker with pid: 12
2022-03-16 18:44:23[2022-03-16 13:14:23 +0000] [11] [INFO] Booting worker with pid: 11
2022-03-16 18:44:23[2022-03-16 13:14:23 +0000] [10] [INFO] Booting worker with pid: 10
2022-03-16 18:44:23[2022-03-16 13:14:23 +0000] [8] [INFO] Listening at: http://0.0.0.0:80 (8)
2022-03-16 18:44:23[2022-03-16 13:14:23 +0000] [8] [INFO] Using worker: sync
2022-03-16 18:44:23[2022-03-16 13:14:23 +0000] [8] [INFO] Starting gunicorn 20.0.4

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.