Refreshing AWS credentials with Python

What to do about botocore.exceptions.RefreshWithMFAUnsupportedError?

One of my stories on secure code, IAM, and Application Security

Free Content on Jobs in Cybersecurity | Sign up for the Email List

I noted on Twitter recently that I ran into the following exception:

Bummer! I was running a long-running program I wrote to assess all the potential lateral movements within a customer account for a cloud security assessment. I didn’t think it would run that long but after enhancing the program and resolving a number of other errors, turns out that even in a small-ish account there a multitude of connections to evaluate. In the case of AWS, you need to look at multiple layers of networking as well as ingress and egress traffic to determine if a host is truly accessible or not.

OK what to do so I can provide a complete assessment? I need to allow my program to continue to run until the end. I thought it would be a simple Google search to find an answer to refresh a boto3 session, but as it turns out, there were a lot of questionable blog posts out there. I don’t think some of them even work, after testing them out. I came up with this solution.

Python Session Refresh Code

The code below checks if a refresh is needed each time a client gets requested. The boto3 client is instantiated for a particular service (such as S3, EC2, etc) in a particular region. You’ll need to call get_region to set the region before you call get_client. Once you have a client you can use it to make AWS requests as you normally would with the AWS python SDK (boto3).

Still getting timeouts?

If you continue to get timeouts before a session refresh, reduce the timeout value, or request a new client sooner in your application to trigger the check before a session timeout. Each time you call get_client it checks if the session needs to be refreshed. If you don’t call that function again before a timeout then your session will end.

MFA and Session Refresh:

I use MFA for penetration tests and assessments when checking cloud configurations, so it triggers MFA a bit before the session ends, asking for a new code to create a new session before the timeout, so my program can continue.

Requesting a new session triggers the MFA check, so this presumes you have an application that you are actively monitoring, if using MFA. I presume if you are not using MFA the program will simply keep running but I haven’t tested that yet.

Although I could probably work around that too if I really tried, I like to use MFA each time a session refreshes for additional security. I’ve been thinking about ways to make it easier to input the MFA token, but why would I use MFA at all if I just eliminate it through fancy coding later? Yes, you should use MFA as much as possible. It will prevent many attacks and stolen credential abuse. In my case, I want to avoid having my credentials abused during a penetration test or assessment in a client account, so I use MFA whenever possible.

Profiles

You can configure different AWS profiles to use when calling AWS APIs. Often I’m testing client environments that have multiple AWS accounts. I automatically generate a profile for each account and loop through them when testing and scanning AWS environments. This code allows you to pass in a profile. It attempts to use the default profile if you don’t pass one in, but I haven’t tested that yet.

The Code:

#file: refreshsession.py
import boto3
from session import Session
from datetime import datetime

class RefreshSession:

  def __init__(self, service:str, profile:str=None, \
     client_timeout:int=None):
    
    #Max session is 3600. Depending on how often 
    #you request a client in your application,
    #your refresh period may need to be shorter
    if client_timeout != None and client_timeout > 3500:
        raise Exception('Timeout must be less than 3500 Seconds')

    #how long sessions should last
    self.timeout = int(client_timeout) if client_timeout else 3500
        
    #when the last session started
    self.start_time = datetime.utcnow()

    #profile configured with ec2 role on host
    if profile == None: profile = 'default'
    self.profile = profile 
    
    #triggers MFA code request
    self.session = Session(self.profile)

    #ec2, s3, etc.
    self.service = service

    #will set these later
    self.client = None
    self.region = None

  #get regions does not require a client
  def get_regions(self):
    return self.session.get_regions()

  #if we need to change regions,
  #update the client to the new region
  def set_region(self, region):
    self.region = region
    self.client = self.session.get_client(self.service, self.region)

  #return a client corresponding to the current session
  def get_client(self):

    if (self.region == None): 
      raise Exception("call set_region before calling set_client.")
    
        #####################################################
        #If we are nearing the end of the session, refresh it
        # 1. Start new session which triggers request for MFA code
        # 2. Reinstantiate the client with new session
        # 3. Set the session start time for future checks
        #####################################################
        time_now = datetime.utcnow()
        time_diff = (time_now - self.start_time).seconds
        if self.timeout < int(time_diff):
            self.session = Session(self.profile)
            self.client = \
              self.session.get_client(self.service, self.region)
            self.start_time = time_now
           
        return self.client

To run the code I do something like this:

#file: main.py imports refreshsession.py in same folder
import refreshsession as rs

profile="profile_name_in_aws_config_file"
service="ec2"
session = rs.RefreshSession(service, profile, 3000)
regions = session.get_regions()

for region in regions:
    session.set_region(region)
    client = session.get_client()

    #now run commands as you would with a boto3 client

No warranty, written quickly, but it seems to be working for me.

Someone later told me this code was failing. I looked into it here:

Follow for updates.

Teri Radichel | © 2nd Sight Lab 2022

The best way to support this blog is to sign up for the email list and clap for stories you like. That also helps me determine what stories people like and what to write about more often. Other ways to follow and support are listed below. Thank you!

About Teri Radichel:
~~~~~~~~~~~~~~~~~~~~
Author: Cybersecurity for Executives in the Age of Cloud
Presentations: Presentations by Teri Radichel
Recognition: SANS Difference Makers Award, AWS Security Hero, IANS Faculty
Certifications: SANS
Education: BA Business, Master of Software Engineering, Master of Infosec
Company: Cloud Penetration Tests, Assessments, Training ~ 2nd Sight Lab

Like this story? Use the options below to support this blog.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
❤️ Clap
❤️ Referrals
❤️ Medium: Teri Radichel
❤️ Email List: Teri Radichel
❤️ Twitter: @teriradichel
❤️ Mastodon: @[email protected]
❤️ Facebook: 2nd Sight Lab
❤️ YouTube: @2ndsightlab
❤️ Buy a Book: Teri Radichel on Amazon
❤️ Request a penetration test, assessment, or training
 via LinkedIn: Teri Radichel 
❤️ Schedule a consulting call with me through IANS Research