AWS Transfer Family with Golang for File Transfers

In the world of modern technology, secure and efficient file transfer is a critical necessity for many organizations. AWS Transfer Family, a managed secure file transfer solution offered by Amazon Web Services, allows file transfers using standard protocols such as SFTP, FTPS, FTP and AS2. When combined with Golang, a programming language known for its efficiency and ease of use, a powerful combination is created to manage and automate file transfers effectively.

In this article, we will explore how to integrate AWS Transfer Family with Golang to optimize your file transfer processes. From configuring AWS Transfer Family to implementing an SFTP client with Golang, I will guide you step by step so you can make the most of these technologies.

What is AWS Transfer Family?

AWS Transfer Family is a secure, fully managed service that allows for the transfer of files into and out of AWS storage services such as Amazon S3 and Amazon EFS using protocols like SFTP, AS2, FTPS, and FTP.

Key Features

• Protocols Supported: SFTP, FTPS, FTP, AS2


• AWS Storage Integrations: Amazon S3, Amazon EFS


• Use Cases: Data lakes, internal transfers, data distribution, compliance workflows, supply chain logistics, B2B transactions.

Benefits

• Real-time scalability.


• No need for infrastructure management.


• Leverage native AWS services for data processing and analytics.


• Fully managed, serverless workflow service.


• Pay-as-you-go pricing.

Prerequisites

• AWS Account: An active Amazon Web Services account. If you don’t have one yet, you can sign up on the AWS website.


• Golang: Knowledge of the Go (Golang) programming language.


• Administrative Privileges in AWS.


• AWS Transfer Family: Administrative read and write permissions to configure and manage
  
  
  

  The API proposed in this article serves as a basic example to demonstrate its fundamental functionality. In a real-world scenario, the API can be more complex and include additional features. Its primary function is to upload files to the AWS Transfer Family SFTP server and store them in an S3 bucket, whether for backup purposes or integration with other systems. Use cases vary and depend on the needs of each organization. Here are some additional use case examples:

  
  
  

  
  
  
  Create Transfer Family Service
  

  
  
  

  Log in to the AWS Management Console and select the AWS Transfer Family service and create server.

  
  
  
  
  

  Choose an identity provider and select "Service managed."

  
  
  
  
  

  We select "Amazon S3" as the storage option and click Next.

  
  
  

  
  
  

  
  
  
  

  
  
  
  Create a role
  

  
  
  

  Create a new role for the AWS transfer family to s3 service. To do this, go to the IAM service and create new role of the AWS service type and in the use case select transfer and next.

  
  
  
  
  

  We proceed to assign the name of the role "role-example-transfer-to-s3", we review and click "Create role".

  
  
  
  
  

  We set the name of the bucket, leaving the default configuration and create bucket.

  
  
  

  Bucket name: "bucket-example-transfer".

  
  
  
  
  

  At the end of the creación we will have something like this:

  
  
  
  
  

  To follow this article, you will need to provide the public key that was generated at the start aws_example_ssh_key.pub.

  
  
  

  To view the content of the public key, use the following command:

  
  
  

  $ cat aws_example_ssh_key.pub

  
  
  

  The public key looks like this:
  
  

  
  
  
  
  

  CODE Kopieren

  ssh-rsa AAAAB3NzaC1yc2EBBAADAQABAAACAQC/qNMJaCz8+I3G71bTFf8Cpe+AAAA0BBBBBBBA6C+/a8fyl9Dw4wjj0huc4ItlX2auKG3skxTYP1ZgO5/7M7xmp2Hf+AAAA0BBBBBBBA6CynWwfwkS5XEIfNtoNWCSeB6ra4anljFdLJH4ZReOM18xxBZCyz2gGA7ePprM0nfpgoQKjp8eDoz6HFcvH4Qynea06W72D+tUWjCRH6bgPBsw/cCfj+VrcqsQXif/X8nViS679+V+AAAA0BBBBBBBA6C+t/5dD3uFPyc8yJn2fNEHB4kfIB9JR5fPIo4dVqPebloPg9vKT2M+AAAA0BBBBBBBA6C/AAAA0BBBBBBBA6C+XLTJSAljgq9JQ4l3vnT4RSsKuK+DX2um5DsN/Zwk9Qwf8JRJey/AAAA0BBBBBBBA6C+/AAAA0BBBBBBBA6CNkVaeDOYSIeGkMBUmmMaeVNeaR967SBLoHNzBfodKKaIpsmW2yHvIitg/YKu2YXKe/DnYU3G8om13uL8qIOB85CugVQ== rodrigoquispe@AAAA0BBBBBBBA6C
  
  

  
  
  
  
  
  
  
  

  This is the key that we need to include in the "SSH Public Keys" field. Ensure that you copy and paste the entire key accurately into this field to enable secure access to the server.

  
  
  
  
  

  
  
  
  Project structure
  

  
  
  

  Below, we provide a detailed overview of the Golang project structure and include a link to the

  
  
  

  Now, we upload a file to the SFTP server using Postman. Then, we navigate to S3 to verify that the file has been uploaded correctly.

  
  
  
  
  

  The files were successfully uploaded without any issues.

  
  
  

  Thank you for reading.

  Vollständiger Original-Bericht

  Ausführliche Details, Code-Beispiele & Hersteller-Stellungnahme auf dev.to.

  ↗ Original-Artikel auf dev.to lesen