Potential Bugs and Security Problems Due to Change in Module Precedence from Python 2 to Python 3

Is Python 3 on an AWS Ubuntu instance not behaving the way the documentation says it should?

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

⚙️ Check out my series on Automating Cybersecurity Metrics | Code.

🔒 Related Stories: Bugs | AWS Security | Secure Code

💻 Free Content on Jobs in Cybersecurity | ✉️ Sign up for the Email List

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

A summary of the problem is here. I included everything I did below trying to sort out this problem. It’s not fully resolved yet.

I was running some code that use to work, but now failing saying it cannot run the code it ran before because the credentials don’t have permission. I know they work in the AWS CLI but not with Boto3. What is the problem??

So I walk through the code and I figure out that I created a custom session object hardcoded to EC2 and us-west-2 to pull a list of regions. Well, if the client for whom I’m pentesting or performing an assessment for does not allow actions in us-west-2 that is going to fail. I need to use the default region for the profile I’m using for the test which is configured to an accessible region. #TODO

But in this case, the client does allow me to use us-west-2 so that is not the issue.

I notice that I have named a class “Session” which overlaps with Boto3’s session. Is that the problem? No. I can see in my code that I reference boto3.Session where appropriate. I each case, the correct code from the correct location gets executed.

Then it hits me. This client doesn’t allow me to make calls in every region. So I take a look at my code that is looping through regions to make queries. Aha. I’m failing on the error when I don’t have access to a specific region. Duh.

I add a try-catch and that error is, actually expected. I get the error on regions where I don’t have permission and that is correct. I add some code to log the error instead of crashing the program. Ok, that solves the credential problem.

But where’s my output? I only see error messages and no output. Well I’m looking for network IPs in my customer account that can reach other IPs, but my code ends up with 0 results, and therefore no output. So technically if the client has zero trust networking and no access between a single interface, that is correct. But that is never the reality. Systems need to connect, so there should be some output.

So the first thing I do is re-reroute the errors to separate log file explained in the first answer here:

It would be better to create a separate logging mechanism and a way to configure the log file location rather than hard code it but for the moment:

Then I log the error:

logger.error("Error message here")

#or

except Exception as e:
  logger.error(e)

#or some combination thereof

Now I have to sort through my own code. It’s pretty simple to read. I decide to just test with one region instead of all the failing regions so I hard code that region and comment out the loop.

My code nicely looks at each different networking component and whether or not two interfaces have access to each other for each component. If any component resolves to false, then those interfaces cannot communicate and the result is not logged. Hmm. I must have a logic error somewhere. But why haven’t I seen this before? I typically get hundreds of results. Zero trust networks are rare.

Finally I see it. Where I check the routes I log if the two interfaces are in the same VPC, in which case the access should be true and the interfaces can communicate as far as routes are concerned. Technically it’s not a route but you know what I mean. They are in the same VPC so routes are not preventing access.

Apparently I did not notice this issue before because most clients have many VPCs and I always get large results for this test. But in this case, the client set up a test environment and apparently, it is all in one VPC.

So that’s fine. One VPC. The code evaluates that the interfaces are in the same VPC. But my code is returning false. What?

Aha.

Two little stinky equals signs. That’s it.

I had this:

access == True

instead of

access = False

So that is fixed but now I hit another problem. I get an error that says ipaddress.check_access is not a valid method. There’s that IP address module again.

I check the Python 3 module ipaddress.py and I don’t see the missing method. I search around on the Internet and for other versions. Nada.

I look at the inputs to this method, and they seem to be specific to AWS content. I doubt the code that evaluates those inputs is in boto3 or python. That was something specific I was evaluating I’m guessing and therefore this function should be in my own code.

Oh my goodness. It dawns on me what I did. I copied over a class I wrote when I fixed the original issue with ipaddress.py in a a very hokey way in this prior post. I hadn’t looked at this code in a bit and didn’t realize I already had an ipaddress.py file.

Then I removed it when I realized everything was working again for no apparent reason.

Now the root of the problem comes to light. I copy back in my own custom ipaddress.py class that has my missing function into my application for GitHub.

But how did this ever work in the first place?! I have two conflicting classes and either one or the other should cause an error.

Somehow, my code ran on a prior occasion and it needed both versions of this ipaddress.py class and I was not specific as to which version in either pace I called it.

Regardless of the cause, I need to solve the problem. I could rename my class and rewrite my code. Or I could do something to try to force Python to use the correct version in the one place where I need it. How do you get Python to use the root python class?

This absolute_import method is the cleanest mechanism I’ve found:

You can read about the feature here:

What’s weird is that the default modules that come with Python should be picked up first in Python 3 so you shouldn’t need that line of code for what I’m trying to do. It should be working that way. Python 3 is behaving like Python 2 on my Ubuntu VM.

So if you’ve been following along …That. is. weird.

Or did I read it backwards. I really don’t have time for this right now. I’ll have to revisit it more later.

I verified in a prior post that I was using Python 3.10. However, the fix to my problem of Python picking up a module other than the default was to overwrite the default ipaddress.py in my local directory.

That should not have solved the problem. That file should not take precedence in the built-in python version of that file in Python 3 if I understand correctly. My application should have been using the ipaddress.py from the default Python installation and failing on my custom module, not the other way around. Hmm.

But as I mentioned, things magically started working the next day without me doing anything, that I know of. Let’s re-run the program to see what happens now. I copied my custom ipaddress.py from my GitHub repository back into my code and deleted the python cached objects.

Well odd. The original error is back and it comes from importing boto3. All this code should be using the Python 3.10 ipaddress.py file. The disturbing thing here is that this has something to do with cryptography. Not troubling at all (sarcasm).

But why am I only getting this error when I add ipaddress.py back into my own project — a completely different ipaddress.py than would be used by OpenSSL, if it had one?

So many questions an a penetration test due. I do not have time to look into that further at this very second. Thankfully I am not using crypto in what I am currently working on. If you are, you should look into this and figure it out.

I need to ensure the default python class is used in one of my modules and my local class in another. I tried adding the absolute_import option but it did nothing.

So for sanity’s sake, I removed my local ipaddress.py module again. I was back to this error which indicates that my local module was taking precedence, even though Python3 should be doing things the other way around.

OK so now I’m wondering ==> Has AWS done something funky to make Python 3 work like Python 2 in terms of precedence? Or Canonical? Or something on this machine?

I don’t know I just need this to work. So what I can do is simply take the Python3 IP address file and put it in my program and add my custom function right?

But wait, there’s more.

My custom ipaddress.py has this in it:

from netaddr import IPNetwork, IPAddress

Is that going to cause a conflict? Well, let’s see whats in ipaddress.py from Python and try it. Run locate, which I showed you how to install in an earlier Ubuntu post.

These are all the versions of ipaddress.py on my VM:

I want to look at the imports in that last one.

less /usr/lib/python3.10/ipaddress.py

Only one import named functools. Conflicting names?

I run this:

cat /usr/lib/python3.10/ipaddress.py | grep netaddr
cat /usr/lib/python3.10/ipaddress.py | grep IPAddress
cat /usr/lib/python3.10/ipaddress.py | grep IPNetwork

Doesn’t seem like there are any conflicts

Ok I’ll grab that file and add in my code. I add it to my directory in my application and add my necessary imports and custom functions. On execution the other error goes away. I’ve still got some troubleshooting to do but it seems like the correct code is getting used and boto3 is working for what I need it to do.

I can see, however, how this default behavioral change between Python 2 and Python 3 has the potential to cause confusing bugs — and possibly security problems. Make sure you know the source of the code you are running.

Follow for updates.

Teri Radichel | © 2nd Sight Lab 2023

About Teri Radichel:
~~~~~~~~~~~~~~~~~~~~
⭐️ Author: Cybersecurity Books
⭐️ Presentations: Presentations by Teri Radichel
⭐️ Recognition: SANS Award, AWS Security Hero, IANS Faculty
⭐️ Certifications: SANS ~ GSE 240
⭐️ Education: BA Business, Master of Software Engineering, Master of Infosec
⭐️ Company: Penetration Tests, Assessments, Phone Consulting ~ 2nd Sight Lab

Need Help With Cybersecurity, Cloud, or Application Security?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
🔒 Request a penetration test or security assessment
🔒 Schedule a consulting call
🔒 Cybersecurity Speaker for Presentation

Follow for more stories like this:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 
❤️ Sign Up my Medium Email List
❤️ Twitter: @teriradichel
❤️ LinkedIn: https://www.linkedin.com/in/teriradichel
❤️ Mastodon: @teriradichel@infosec.exchange
❤️ Facebook: 2nd Sight Lab
❤️ YouTube: @2ndsightlab