s3curl

S3 Curl is a tool that allows users to interact with Walrus by adding security parameters as curl headers.

You may create, delete, list buckets, put, get, delete objects, set access control policies, etc. Please refer to the Amazon S3 documentation for the S3 interface specification.

You will need the perl-Digest-HMAC package for S3 Curl, if it is not already installed on your system.

You will need to modify the file s3curl.pl in your favorite editor to change the hostname endpoint.

For example, change

my @endpoints = ( 's3.amazonaws.com' );

my @endpoints = ( 'your-host' );

where, your-host is the IP or the hostname on which Walrus runs.

For authentication to succeed, it is crucial that your-host is the same as the host portion of the $S3_URL environment variable set by eucarc.

When modifying the file s3curl.pl, please specify just the hostname and not the port number or the entire path (e.g., you would specify xyz.com instead of xyz.com:8773/services/Walrus. However, you need to use the entire path in the command-line when using the tool).

Be sure to source your 'eucarc' file before running the commands below.

s3curl.pl --id $EC2_ACCESS_KEY --key $EC2_SECRET_KEY <curl options>

It is preferable to add your credentials to the ".s3curl" config file. Please read the README file bundled with s3curl for details.

For example, to create a bucket,

s3curl.pl --id $EC2_ACCESS_KEY --key $EC2_SECRET_KEY --put /dev/null -- -s -v $S3_URL/bucketName

where bucketName is the name of the bucket that you want to create.

To put an object,

s3curl.pl --id $EC2_ACCESS_KEY --key $EC2_SECRET_KEY --put <filename> -- -s -v $S3_URL/bucketName/objectName

where objectName is the name of the object that you want to create.

To get the MD5 checksum, size and last modification time for an object.

s3curl.pl --id $EC2_ACCESS_KEY --key $EC2_SECRET_KEY --head -- -s -v $S3_URL/bucketName/objectName > object.head

cat object.head To get an object

s3curl.pl --id $EC2_ACCESS_KEY --key $EC2_SECRET_KEY --get -- -s -v $S3_URL/bucketName/objectName > object

To delete an object,

s3curl.pl --id $EC2_ACCESS_KEY --key $EC2_SECRET_KEY --del -- -s -v $S3_URL/bucketName/objectName

To delete a bucket,

s3curl.pl --id $EC2_ACCESS_KEY --key $EC2_SECRET_KEY --del -- -s -v $S3_URL/bucketName

Note that, according to the S3 specification, a bucket needs to be empty before it can be deleted.

You may pipe the output through "xmlindent" (you might need to install it, if it is not already installed on your system).

For example, to get the access control policy for a bucket,

s3curl.pl --id $EC2_ACCESS_KEY --key $EC2_SECRET_KEY --get -- -s -v $S3_URL/bucketName?acl | xmlindent > bucket.acl

To get the contents of a bucket.

s3curl.pl --id $EC2_ACCESS_KEY --key $EC2_SECRET_KEY --get -- -s -v $S3_URL/bucketName  | xmlindent > bucket.list

To list all buckets for a user.

s3curl.pl --id $EC2_ACCESS_KEY --key $EC2_SECRET_KEY --get -- -s -v $S3_URL  | xmlindent > bucketlisting