Issues With Bash Arrays

Just figuring a few things out and writing it down for future self and anyone facing similar problems

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

⚙️ Check out my series on Automating Cybersecurity Metrics | Code.

🔒 Related Stories: Bugs | AWS Security | Secure Code

💻 Free Content on Jobs in Cybersecurity | ✉️ Sign up for the Email List

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

I don’t necessarily consider myself a bash expert but I feel like I am being forced to become one.

Initially I just wanted to write a POC in bash and move it to another language. But I’ve been writing this POC for like a year now…

So I’m currently working with bash arrays and it a few glitches. There are different older and newer ways to declare or use bash arrays. Many, many posts show the older ways because bash has been around so long. I’m going to stick to the newer ways because I *hope* you are not using outdated software if you are using my code for secure deployments with proper governance.

First of all, I recently discovered “local” because none of the older posts reference it. This is a very important construct because it helps you keep variables specific to a function and not overwritten by some other function that has a variable with the same name.

I’ve written about the importance of using proper scope in programming languages in other posts and thought bash was simply not capable of doing this but it is! I have had so many weird bugs because I had a function calling a function and happen to be using the same name in both. Even though I understand all this stuff that threw me for a loop for quite some time in a couple of cases.

Use local whenever possible. Now that I know about it trying to update all my code.

OK but how do you do that with an array? Because you can use declare to declare an empty array like this:

declare -a myarray

Well I discovered you can use local or declare to define an array.

Apparently either one will define a local variable. Cool.

So this should work like the above statement, right?

local -a myarray

But the difference, I think, is that declare defines the variable specifically as an array. Where as local imports the array into a variable but the variable is not specifically for holding an array. You could change it to a string or anything else. If I understand correctly.

So I think it would be better to use declare to specifically define an array.

Along the way I was researching how to pass an array between functions. I think I have done that before but I always for get and somewhere I ran across this construct:

declare -n myarray = $1

So what does -n do? It appears to pass a variable by reference instead of name (something I also just learned you could do in bash):

The variable becomes a name reference for another variable. Unset the attribute with +n .

But then I changed declare to local to this before I realized that local and declare both create a locally scoped variable.

local -n myarray = $1

But when I passed the array from one function to another using that construct on Amazon Linux it failed. You’d have to see if you get the same behavior on other versions of Linux. Mind you this was Amazon Linux 2.

shared/parse_config.sh: line 110: local: `[first line of my array]’: invalid variable name for name reference

So I’m using local to create a local variable, but I’m referencing the array from another function by reference, and for bash is using the value of the array as the name instead of the name I defined (myarray).

Interesting because I got that idea from this post:

I thought it was working earlier. The only difference I see is the spaces so I changed my code to this with no spaces before or after the equal sign:

local -n myarray=$1

Nope.

In any case I changed it back to declare -a and it worked. Found some other interesting ways to use array here might coming in handy in the future like referencing an element by name.

Also this post goes into arrays on one or multiple lines and has a very long and verbose explanation about what a bunch of reasons why other answers are wrong followed by an answer that contains this:

readarray -t lines <<<"$string"

Not sure how long readarray has been arround but did not see it in a lot of posts so searched around more and found this interesting post on new lines and why the above command above has -t. You’ll want to use -t to get rid of newline characters in a multi-line array:

Ok so here’s the next problem. I’m looping through lines in my array and I want to put a subset of the lines into a second array. But what I’m getting is my full array on one line like this when I loop through the lines instead of separate lines.

So here’s what I’ve got in the line function that is accepting the argument as an array (truncated):

It appears that the whole value of my configuration in a multi-line source is coming through as one string.

I add declare -p in the function passing in the array to see what’s in it:

I can see that everything is getting stuffed into one element in the array

That’s odd because this was working before I used readarray.

I changed it back to read -a:

and it works in the calling function:

Now each line is a separate element in the array, except I see another problem. It’s splitting out anything with a dash into a separate element.

Well that’s not going to work.

In addition my receiving function still sees it as all one line. Let’s change the input parameter back to local -n and see what happens now.

At this point I start to break it down into a more simple script.

I figure out that if you put a $ in front of the a in the second line you get an error:

Must be this:

Finally after fiddling and searching I come up with this:

I think that gives me what I want:

I see some extra spaces but I’m going to trim those off and the size of my data is not really large, nor is this my final programming language of choice so it works well enough for now.

The next step is to see if I can pass that to another function.

Here’s what I’ve got:

I’m going to create a function above that and see if I can get and print the array successfully.

So my attempts to get an array above were all wrong and I forgot that I had this working according this post prior:

Duh.

Pass the array like this:

handle_array "${array[@]}"

Receive the array in the calling function like this:

 local arr=("$@")

Here’s my test:

That works — if you only have one argument. The $@ represents the positional arguments passed to a function. If you have multiple arguments to pass to a function, then you have to put the array at the end of the argument list and there can only be one array.

This example makes use of shift to shift the array over one when there is one other argument so the array becomes the first argument, and then accesses as above:

But wait! There’s more…

When I tried running this in CloudShell which runs on Amazon 2023, it didn’t work. Finally figured out this syntax works:

readarray -t -d ' ' a <<<$c

Apparently the IFS option does not.

Basically, this is kind of odd syntax altogether and looking forward to getting back to a real programming language but works for now.

Follow for updates.

Teri Radichel | © 2nd Sight Lab 2024

About Teri Radichel:
~~~~~~~~~~~~~~~~~~~~
⭐️ Author: Cybersecurity Books
⭐️ Presentations: Presentations by Teri Radichel
⭐️ Recognition: SANS Award, AWS Security Hero, IANS Faculty
⭐️ Certifications: SANS ~ GSE 240
⭐️ Education: BA Business, Master of Software Engineering, Master of Infosec
⭐️ Company: Penetration Tests, Assessments, Phone Consulting ~ 2nd Sight Lab

Need Help With Cybersecurity, Cloud, or Application Security?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
🔒 Request a penetration test or security assessment
🔒 Schedule a consulting call
🔒 Cybersecurity Speaker for Presentation

Follow for more stories like this:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 
❤️ Sign Up my Medium Email List
❤️ Twitter: @teriradichel
❤️ LinkedIn: https://www.linkedin.com/in/teriradichel
❤️ Mastodon: @teriradichel@infosec.exchange
❤️ Facebook: 2nd Sight Lab
❤️ YouTube: @2ndsightlab