Recently I had to view a S3 object using CLI in a read-only environment. I was stuck for a while before I found the solution. Here are my learnings from it.
First of all, let’s look at s3 and s3api commands. This article explained very well about the differences and use cases.
In a word, aws s3api
is low-level while aws s3
is high-level. What does it mean? s3api commands directly invokes S3 API and has near one-to-one mapping to API. When we define policies and permissions for S3, we need to specify actions. One commonly used command is GetObject
. You will find that s3api has corresponding commands for that, aws s3api get-object
. aws s3
commands simplify managing objects and buckets.
Now, let’s take a deep dive into aws s3 cp
and aws s3api get-object
respectively.
aws s3 cp
The basic syntax is as follows,
aws s3 cp
<LocalPath> <S3Uri> or <S3Uri> <LocalPath> or <S3Uri> <S3Uri>
It can copy files between local storage and S3, or between S3 locations.
For example,
aws s3 cp s3://mybucket/example.txt ./example.txt
Back to the initial task I was given, I couldn’t create a new file nor save it into an existing file. Gladly, example 13 from the official documentation shows a way to stream the file as standard output by using a magical -
.
aws s3 cp s3://mybucket/example.txt -
aws s3api get-object
The simplest command needs to include the bucket, key and outfile. Note that the outfile parameter is specified without an option name such as “ — outfile”. The name of the output file must be the last parameter in the command.
aws s3api get-object
--bucket <value>
--key <value>
<outfile>
For example,
aws s3api get-object --bucket mybucket --key example.txt ./example.txt
The output file doesn’t need to be created before running this command. It would create a new one if it doesn’t exist.
If it’s running successful, the output would be in JSON format.
{
"AcceptRanges": "bytes",
"LastModified": "",
"ContentLength": ,
"ETag": "\"\"",
"ContentType": "",
"ServerSideEncryption": "",
"Metadata": {}
}
I’ve searched intensively in the document or the Internet but couldn’t find any clues of presenting the file without saving it locally.
Conclusion
aws s3 cp
and aws s3api get-object
are two representatives from s3
and s3api
commands which are similar but different in many ways. aws s3 cp
is indeed simple to use, suitable for copying files between local storage and S3, or between S3 locations. It can quickly show you the content of the file from the terminal. aws s3api get-object
has more options hence provides granular control. The output of the command also contains rich information which might be useful for some.