Sep 24, 2019

Some Command Line Tools for AWS

I’ve been working a lot in AWS lately and while I like the tech stack, going through the console to do stuff is really annoying to a command-line junkie such as myself. I could cook up my own scripts via the aws-cli or using boto3 and Python. But luckily others have already done that for me. Here are a couple of tools I use everyday.

awslogs

If I want to check the logs for my lambda function, I used to log in to the console, go to CloudWatch, find the my lambda (which is made difficult because the search bar doesn’t do substrings), click on my lambda group, and then sort by time, because it doesn’t necessarily sort by time, and then click on the link for the logs. That is exhausting.

Now I use awslogs, which does a lot of the work for me. awslogs groups will list all the existing CloudWatch groups and then awslogs get <group name> will display the latest entries. Of course you pipe it to grep or ag and search for messages. Or you can use --search in the awslogs command. You can also set a start time and/or an end time for the log messages.

This command doesn’t seem to honor the standard AWS_PROFILE environment variable, but you can use --profile <name> at the end of the awslogs command (it doesn’t seem to work anywhere else in the command, unfortunately).

s3cmd

I think of s3cmd as a little Swiss Army knife for all things S3. It does syncing, copying, tree-traversals. It even knows when to use multi-part uploads. Most of my work is in S3 lately, and I use this all the time.

My biggest problem with s3cmd is it doesn’t use AWS’s standard configs but it’s own config file. That makes switching between profiles (something I do a lot) tricky. I ended up making a wrapper script that reads ~/.aws/credentials and takes the given profile name and executes the S3 call. You can see the script here. Usage is:

s3ctl.py <profile_name> <s3cmd command>

so if you are putting fancy-file into s3://my-fancy-bucket/some-folder using your development AWS profile, it’s:

s3ctl.py development put fancy-file  s3://my-fancy-bucket/some-folder 

About the Author

Mike Hostetler profile.

Mike Hostetler

Principal Technologist

Mike has almost 20 years of experience in technology. He started in networking and Unix administration, and grew into technical support and QA testing. But he has always done some development on the side and decided a few years ago to pursue it full-time. His history of working with users gives Mike a unique perspective on writing software.

Leave a Reply

Your email address will not be published. Required fields are marked *

Related Blog Posts
An Exploration in Rust: Musings From a Java/C++ Developer
Why Rust? It’s fast (runtime performance) It’s small (binary size) It’s safe (no memory leaks) It’s modern (build system, language features, etc) When Is It Worth It? Embedded systems (where it is implied that interpreted […]
Getting Started with CSS Container Queries
For as long as I’ve been working full-time on the front-end, I’ve heard about the promise of container queries and their potential to solve the majority of our responsive web design needs. And, for as […]
Simple improvements to making decisions in teams
Software development teams need to make a lot of decisions. Functional requirements, non-functional requirements, user experience, API contracts, tech stack, architecture, database schemas, cloud providers, deployment strategy, test strategy, security, and the list goes on. […]
JavaScript Bundle Optimization – Polyfills
If you are lucky enough to only support a small subset of browsers (for example, you are targeting a controlled set of users), feel free to move along. However, if your website is open to […]