avatarTeri Radichel

Summary

The author, a teacher of an Azure class, shares their frustrating experience with Azure support, including long wait times, unhelpful responses, and difficulty resolving issues.

Abstract

The author, who teaches an Azure class, has been struggling with Azure support for almost a month. They initially contacted support to help with teaching the class, but have encountered numerous issues, including unhelpful responses, long wait times, and difficulty resolving problems. The author has also heard similar complaints from their clients. They eventually cancelled their support due to the lack of assistance and the time wasted. Despite this, Azure continued to contact them even after the support was cancelled. The author hopes that their post will bring attention to these issues and encourage Microsoft to address them.

Opinions

  • The author is frustrated with Azure support and has had a negative experience.
  • The author believes that Azure support is not providing adequate assistance and is wasting their time.
  • The author has heard similar complaints about Azure support from their clients.
  • The author hopes that their post will bring attention to these issues and encourage Microsoft to address them.

Azure Support Diary (or Diatribe)

A few submissions for the Azure Wishlist

One of my posts on Azure Security.

Free Content on Jobs in Cybersecurity | Sign up for the Email List

TLDR: It took almost a month for Azure to admin the first support ticket I submitted was a bug. Almost every support ticket was a bug or I resolved it myself exception ONE case where I got a quick answer. There were I think 3 or 4 I never followed up on in the end due to way too much time invested and the end of my project for which I opened support.
I opened Azure support because I was teaching an Azure class to 35 students and I wanted to see what had changed. There are a lot of interesting features on Azure, but when you can’t get the support you need or even create a VM in some cases, it makes it a bit hard to justify using the platform. I also have heard the same issues about Azure support from my clients.
In my case, I got done with a 6 week class (7 including a week skipped for a holiday) and simply cancelled support at the end of my class. I was originally intending to get an answer on the first ticket I opened so I could demonstrate the functionality in class. After Azure finally admitted it was a bug, I was on to the next fiasco - I couldn't create a single VM. 
At one point I was told one of the things I reported was a security incident but they could not provide the details about their super-secret systems or what exactly happened. I got blocked using a service because it said my IP address was an internal Microsoft IP address rather than my own.
Since the class had ended it was all a moot point so I left 4 cases hanging - one was to test my first support ticket after Azure fixed the bug. I really didn't want to deal with the person on that case any longer as it was an extremely challenging experience to communicate with that person for whatever reason. 
At times I was told so-and-so's manager was contacting me but even in those cases things often did not get resolved. I feel like I paid $200 to report bugs and go around in circles with Azure support. I think I would have done about as well on my own and saved the money.
After I cancelled Azure support - they still did not close the tickets and continued contacting me. As you can see from the repeated theme below, Azure has a real issue closing support cases. They could probably help more customers and stop wasting time on tickets that should be closed if they would just let the customers close tickets themselves.
Hopefully some of this will get resolved. If you are struggling with Azure support give this post a like so Microsoft realizes it's a problem. Your cases may not exactly align with my own, but in general, the issue is that cases are not getting resolved in a timely manner, if at all.
Additionally there are too many bugs - and I started a separate blog about that:
https://medium.com/bugs-that-bite

I’ve been working on some new ideas on Microsoft Azure that intrigue me such as using a remote IdP to handle user authentication with Azure handling authorization. I also signed up for Standard support ($100/month) to see if it can help me resolve the issues more quickly. Some of the features I’m testing out are in Preview. Here are a few items on my Azure wishlist today.

Apologies for typos I have no time today.

Caveat: I’m sure running a global cloud platform is not a simple feat. So many moving parts it boggles the mind how a CEO or security executive could even keep track of all of this and manage it. Hoping this helps provide some insight to someone who can actually address the issues as some things need to be addressed over and above support tickets.
BTW: Although I'm writing about Azure here, all other cloud platforms, technical product companies, and telecommunications companies should take note. The tips in this document apply to most support teams I deal with across the board.
  • Very confusing to report a case when you search for Support Request in the Azure portal. FIX: Add a button on the page you get to when you search for “Support Request” to have a button to create a support request instead of having to click a link at the top to get to that page.
  • I have to login to GitHub to provide feedback on documentation. FIX: Don’t make people log into GitHub to report a problem with documentation or I won’t bother. I’m busy and don’t want to be logged into GitHub unless I need to be.
e.g. “The identities of the virtual network and the subnet are also transmitted with each request.” Networks have identities? What does that mean, exactly?
  • Support staff is responding to questions without reviewing or maybe understanding screenshots provided with the case. FIX: Please look at the provided screenshots before answering the question.
  • Support staff are asking me to do a remote session. That is a security risk if people in an account are allowing them to do that and security teams and administrators don’t know about it. FIX 1: Don’t ask for remote sessions unless it is actually required. The person could have reproduced my steps with the information I provided. FIX 2: Remote sessions should require higher privileges to allow and should be a button you have to toggle on in the portal, with a time limit, that turns back off after the session. (I was looking for the information on how the person intended to initiate the remote session. I don’t want to allow it. I just wanted to understand what the person wanted to do exactly, so I could analyze the associated risk.)
If I give someone access via a remote session can they make changes in my account? 
How do you grant a remote session? I never would so I haven't looked into it. Shouldn’t checking the box to provide diagnostic access be enough? 
The person asked what error message I was getting, but I had already provided the error in a screenshot.
I provided in a detailed blog that had step by step the actions I took in two products to set up an IdP. I presume someone could follow those steps to try to reproduce the problem. The IdP I'm using has a free trial option.
I don't want Azure support viewing my private IdP configuration information. That is why I provided a screenshot of the page without the details.
Now I have requests out to the support teams on both sides and this is where it gets tricky. Which vendor is actually going to help me solve the problem instead of pointing to the other vendor? We'll see who gets this done first.
  • As global admin you won’t necessarily see all the subscriptions and resources in your account until you toggle a special button. Even after you toggle the magic button, some of the logs and information is not accessible. Even after assigning the global admin the “Security Reader” role I can’t see them all without making myself an owner of the subscription. You have to go in and manually assign yourself as owner of every subscription. Then you’re supposed to turn off the magic toggle again. The problem is, someone comes along and creates a new subscription. Repeat. Also, what’s the point of the magic toggle if anyone who gains access to that account can simply go in and flip the switch? I am trying to understand the purpose of that feature but haven’t figured out a good reason for it yet. FIX 1: Just make the global admin a true global admin and tell people not to use it and store the credentials in a safe place. FIX 2: Definitely make every and all logs accessible when you flip the magic elevation toggle.
  • The Global Administrator does not have access to certain aspects of a new tenant when created. Fix: It seems like the Global admin should have access to everything in the account by default. What if developers have created something the GA doesn’t know about and it’s hidden from their view and they don’t realize it?
  • Azure DevOps looks interesting, but it requires a gazillion URLs which makes firewall rules very difficult. FIX: Create some sort of proxy for these requests so people can add a small number of firewall rules. Perhaps distinctly different features have a different and clearly defined domain so organizations can block access to a certain feature using network rules. However, this is a bit unwieldy.
Also note that although you can use a proxy for private access, it may be that the traffic still goes over the public Internet from a remote location not in Azure once it leaves the Azure private access proxy. If you want your traffic to be truly private make sure it’s not just allowing access from a private IP address and then sending it over the Internet anyway unless that’s what you want. I haven’t tested this out yet.
  • While viewing audit logs support is telling me that a filter is causing logs not to show up, but there is no filtering capability on the page. I see an error message. The support person I don’t think looked at the screen shot and saw that the problem was an error message, not a filter. In the end, this was a bug I think. FIX 1: Look at support case attachments. FIX 2: There’s a bug here after further analysis, I believe.
  • There is no way for me to mark a support case as closed. FIX: As a customer I should be able to close a case when I figure out the answer myself or the system magically starts working again (as it did last night).
  • After the support teams says they will close a case upon my request, the case still shows that it is open. FIX: Mark the case as closed and have it show up as closed so it drops off the active list.
  • After the support team says it will close the ticket it remains open and people keep replying to it. FIX: Save us all time and close the ticket so people don’t keep replying to it after it’s resolved.
Example: Azure Activity Logs were not working. This is initially why I signed up for support.
I submitted a support case and magically the logs started showing up as soon as I submitted my request?!
I added a comment to the ticket shortly after creating it that the issue was resolved. Since then I have received three responses, all telling me “Thank you. I will close the ticket.” I think I see what the problem is with the system that is causing this.
1. I enter my request. I get an automated response. 
2. I respond to my request saying it's ok now, please close the request (since I cannot close it myself.)
3. Someone responds to my first request and apparently can't see my response that it's ok now please close?
4. I respond again saying the problem is resolved.
5. I respond again asking to close the request.
[One of my faults, I tend to send a couple of messages in spurts like one would send a text message and that's not good in a support portal. Trying not to do that but it happens.]
7. Someone else responds to #2 that they will close the case.
8. Someone else responds to #4 that they will close the case.
So what you have is people responding multiple times to the same case that I want closed saying they will archive and close the case but it is still in the open status and people are still responding. 
I'm just waiting now for one more person to respond to #5 telling me they will close the case. Yep...just got it as I am writing this. That's 4 responses telling me they will close the case and it's still open and people are still responding.
Do people have to meet some kind of quota related to responses because they respond with messages like "I'm glad I could help you with this." And yet the case is still open.
Meanwhile, my other cases that I need help with are missing*** from the portal, and are unresolved. 
*** Note they were not missing (I don't think) there are just quirky things with the filters and some bugs shown in the screen shots below that I figured out later. ***
Update: Took an hour long consulting call and when I came back this case is finally marked close. However, I have another case in the same state. People keep replying to it and it magically started working as soon as I signed up for support. It's related to Azure Monitor.
  • Support case information is disappearing. I made comments on a case and later they didn’t show up after someone responded. Is this a bug? FIX: Make sure comments show up after added and cannot be removed.
  • Emails outside of the support portal did not show up in the support portal. Someone emailed an answer to a question and when I logged into respond that person’s answer was not there. Is this a bug or a person going around the system? FIX 1: Fix the bug that that caused that person’s answer not to show up or FIX 2: Make sure people aren’t taking actions outside the support portal so there’s an accurate log of the activity on the ticket.
  • This may be just me but the switch tenant functionality is not obvious enough. You have to click on a tenant to get the switch function to become active so you can use it. FIX: Potentially show the status of the tenant in the list showing which one is active with a button on the others to make it the active tenant.
  • I submitted tickets in the evening. I went to bed and people responded. Now apparently those people are off shift and no one is looking at the tickets. Fix: When someone is off shift have someone else take a look at the tickets and see if they can resolve them instead of waiting until that person comes back on again. People just respond to a ticket with minimal information so it meets the required timeframe and then it just sits there. (I had the same problem with other cloud providers.)
  • I submitted a new ticket asking if someone could pick up the open tickets which mostly revolved around permissions. The person that picked up my request did not read the ticket or respond to what it was about. FIX: Read and respond to what is written in the case.
  • For support tickets, the list doesn’t tell me who’s turn it is to respond. FIX: Add a column that shows who’s got the next action.
  • I submitted a case to resolve the following: Cases not showing up in portal — please add them, cases I’ve requested to be closed are still open, and reassignment of cases that have not been resolved in a timely manner. Now I wait…… This is the point where there’s nothing for me to do because all the cases are deadlocked in non-responsiveness and unhelpful answers and usually I give up and go do something else. I never get around to logging back in because I’ve just given up and finally someone closes them because I don’t respond. I’ll try to come back and check later. Fix: Improve the whole process before it gets to this point.

Update:

  • Got this in one of the support responses. It is extraneous and makes the response much more wordy than it needs to be. Fix: Only respond with the required information.
We will consider this issue resolved when one of the following conditions are met:
Once we are able to determine why you are getting this error.
A workaround has been provided that will allow you to solve your business need.
It has been clearly identified and/or shown to you that what you are attempting to do cannot be supported by us.
The issue that you are experiencing is by design.
Microsoft works on ‘one issue per incident’ basis. As per Microsoft an incident is defined as an issue that cannot be broken down any further. In case you have any other issue after this, you would have to create a separate incident for it.
During the course of troubleshooting, it is possible that the issue may be existent as a result of a problem from a Third-Party Software/Hardware side. In such situations, you would have to contact the respective vendor(s).
We will now begin working together to resolve your issue. If you do not agree with the scope defined above, or would like to amend it, please let me know as soon as possible.
  • I actually got an answer on the first try for one case and requested to close it. I also figured out that one of my requests is an Azure bug. There are now 4 cases in the portal which I’ve requested to be closed. Fix: Allow customers to close their own cases but for now, close cases as soon as a customer asks — without responding because that is a waste of time and inbox.

Update:

  • Moments later my support requests have disappeared again. Fix: Clearly this portal has an issue. Note that I took the time to figure out what the bug is and provided screen shots below.
  • I just figured out what is confusing about tenants. Fix: When you click on Azure Active Directory “Manage Tenants” should be in the left menu with everything else.

Update:

  • Took a break. Now the only support requests I can see are ones that I asked support to close. They are still all open except one.

Update:

  • Went to eat dinner and came back to this. Was still confused by filters and support bugs and couldn’t find tickets. Figured out later this was the bug below but it was very confusing. Fix: Fix the bug.

Update:

  • Cannot add two files when creating a support case, unless I’m missing it. I can go back after I create the case and add the files — if I can see it. I just created a case and it’s not immediately showing up and I need to add two more screenshots. Fix: Allow upload of multiple files on initial creation.

Update:

  • Went to bed. Got up to check my cases.
Sometimes I wonder if people are intentionally trying to be unhelpful or they really don’t understand or take the time to read what I am writing. We had this saying at Capital One: Assume good intentions. So that’s what I’ll do.
  • In the case where I provided a step by step walk through with detailed screenshots and instructions of the actions I had taken and the resulting error, the person responded with a couple of sentences pointing to the option in the menu in the Azure portal to perform the action which I provided detailed information showing I had already done. He or she did not address the error message I provided that occurred at the end of those steps. Fix: Read the message provided in the case and look at the screenshots. If the customer provides steps they took that led to an error, take those steps yourself to see if you can reproduce the problem.
  • In a case where I asked a person if they could see VMs in my account, the person responded that there are multiple subscriptions in my account. Um, yes, I know that. Fix: Answer the question that was asked.
  • I figured out that the global filter is tricky. I’ve been working with multiple tenants which I didn’t do so much in the past in order to test out permission boundaries, etc. If you set the “global filter” it appears that it’s not truly global. It only applies to one tenant I think. So even though I had set the global filter to all subscriptions and turned on the magic elevation toggle, I wasn’t seeing all my subscriptions. Fix: Somehow there has got to be a better way than using this hidden global filter and magic button. Remove those and use filtering on the subscription page.
  • I asked a question about not being able to see all the subscriptions when logged in as a user that was assigned the security reader role in two tenants. The person that responded to that last case replied that I have one tenant in my account (I provided screenshots and explained there are two.) The person explained to me how to add the security reader role. That was not my question. The user is already assigned the security reader role as I had explained. I figured it out myself and asked the person to close the case. Fix: Read the message in the case and look at the screenshots.
  • It seems like when I turn off the global filter it doesn’t stay off. Fix: Make sure if someone turns off the global filter that it stays off until they turn it on again.
  • When I started creating support tickets I set a default email address. My last two tickets went to a different address. I don’t know if that is because I submitted them from a different place in the portal that was in a different tenant or subscription or something. Fix: The default email address for a user for support tickets should be the same across all of Azure and then have a more granular way to set different emails for different resources.
Here’s the whole problem causing most of the confusion, besides what appears to be a bunch of bugs when navigating certain paths of the portal:
You need a filter at the top of every page that has the following: 
- A drop down for region
- A drop down for subscription
- A drop down for tenant
- A check box to apply the global filter or not and a link to edit it
- For global admins an indication as to whether or not they have elevated privileges (or get rid of that because what is the point? People should not be using the global admin account - I'm only doing it for research purposes.)
- A way to click on something that shows which roles apply to the user on the current screen and with the current selections.

Other discoveries:

  • When you have multiple tenants the “global filter” is not actually “global” it applies only to the current tenant. Fix: Make that clear on the screen where the person is editing the “global filter” to ensure the person understands it only is global for one tenant.
  • When you toggle the magic elevation button, that also only applies to one tenant. Fix: Make it clear in the documentation — highlight it somehow — that you have to elevate in each tenant where you are having an issue to see all subscriptions across all tenants.
  • When you are on the subscriptions page and you change the filter to uncheck the global filter, the next time you come back the filter reapplies the global filter. Fix: Save the settings the user selected.
  • When you select “all subscriptions” it still applies the global filter. All subscriptions does not mean “all subscriptions” in this case. Fix: Uncheck the global filter when someone selects all subscriptions.
  • In order to view subscriptions in another tenant you have to navigate to confusing pages as noted above and switch to a different tenant and then navigate back to the subscriptions page to the subscriptions for that tenant. Fix: Allow switching to a different tenant from the subscriptions page. These two things seem closely related.

Update:

Been handling cloud security and container consulting calls all day and checking support requests in between. Here’s the latest:

  • I posted a support case number on Twitter and asked that it be closed because nothing I request to be closed in the portal is getting closed and because that case had a title that apparently confused the person that read it. He or she only read the title and not the case contents. I wanted that case to be closed to avoid confusion. The person keeps responding after I asked to close it — multiple times. Fix 1: When a customer requests to close a case, close it. Fix 2: Better yet, let customers close cases. Fix 3: Either that or allow the customer to edit categories and subjects if they make a mistake.
  • I created a new case with a different appropriate title and provided the information required to restore the missing cases in the second ticket (which I found later, see bug below). I tried to be more clear in my request to avoid confusion. I reported the bug in this ticket later. So far I have gotten no help with that case I requested to have escalated on Twitter. I am trying Twitter because I don’t know how else to get my case escalated. It’s not working. Fix 1: Provide a better way to handle escalations. Fix 2: I have a number of issues about at the same level, however one is more urgent that the others. I wish there was a way to indicate that. However, I do not want phone calls, so I am not going to pick priority A. I want the information support provides documented in the portal for later reference. I also find that it’s easier to communicate in writing with people who speak another language.

Side note. I love this checkbox in the Okta support portal:

  • Another thing I noticed was that I was able to solve some cases myself, so I moved the priority down to C. (A, B, C). Once I realized that the cases were not getting worked in the order I wanted, I tried to downgrade some to C, but once an engineer picks it up that can’t be changed. Fix: A customer should be able to change the priority as new information or other urgent matters arise related to their cloud account.
  • Directory is in small print top right nav. Same issue with regions in AWS where people don’t realize what region they are in and think all their resources are gone. Fix: The currently selected tenant could be more prominent in view when dealing with subscriptions. The pop out there works exactly like I recommended above. The currently selected directory is very clear and nicely designed.
  • Filter on the subscription page has a lot of space between the subscription list and the global filter checkbox. If a user has made the height of their screen very small so they can look at two screens at once they will not see the global filter unless they scroll down. This is know as “below the fold” in UI design. Fix: Move the global filter checkbox to the top of that screen so that regardless of the height of the filter popup, the user always sees it. Fix: Move the button at the bottom to the end of the list (remove the extraneous space).

Moving along…

  • Adding roles to a user in a tenant won’t work until the user is invited to the tenant and accepts the invitation. Although my user had the necessary roles the user could not switch to the tenant. Suggestions: 1. Add a note to that page where you assign roles that they won’t work until the user accepts the invitation. 2. On the user’s page where they switch to a new directory, add a note that says an email invitation is waiting. 3. If the email bounces, indicate that on the users list where it says “Invitation.” 4. On the users list where it says “invitation” indicate whether or not the invitation has been accepted.
  • I got an email that looks like this, FYI. Suggestion: Possibly a bit of design work to make it look more trustworthy?

This domain also popped up. Hopefully that is expected.

  • After going through this process it tried to force me to login with the Microsoft Authenticator app. I had to back up and try again to use a different authenticator. Fix: Make it easier to use a different authenticator app during this process.
  • Finally…I was able to get to the tenant and had assigned the security reader role to all the subscriptions and could see them.
  • Although that works, creating yourself as a guest user of your own tenant seems a bit odd. I kept asking support if there was a way to assign a user the security reader role across all subscriptions in my account. They kept repeating that I had to assign the user the reader role separately in each subscription. In a few cases they said, oh by the way you have multiple tenants but didn’t provide a solution for granting the user access to the second tenant and the subscriptions in it globally. I figured there must be a better way so I kept looking.

You can grant tenant-wide access to Microsoft Defender for Cloud:

This page mentions cross-tenant management.

With Azure Lighthouse:

This article is also useful:

None of the support people mentioned this in any of my cases when I was trying to achieve this objective. Some of the above still seems to be written from the perspective of a third-party managing your Azure resources versus an in-house security team, but it seems like it would still be good to mention.

  • So now every single case I have open can be closed except the most important one I was hoping to complete in one day….got help from someone at Microsoft who is trying to help me with that. Thank you!!

Update:

  • Spoke too soon. The issue I mentioned about with the screenshot about non-compliant VMS: Once again I got a response in email instead of in the portal. I didn’t see it. I copied and pasted into the portal. Initially, I thought the person had confirmed it was a bug. Then I realized I misread what she wrote. I have no VMs in my account and Microsoft Defender for Cloud appears to be telling me in that screenshot above that I have non-compliant VMs. I asked the person assigned to this ticket if she could confirm, by looking in my account which I set up for testing, that there are no VMs in my account. Instead of doing that she keeps telling me that I have other subscriptions. Yes. I know. I have multiple subscriptions and multiple tenants. She also is trying to get me on the phone. Why does Microsoft support always do this? I don’t think this is a complicated request, is it? I have a very small account I have set up specifically for testing something. It seems like it is a very simple ask. Look at the tenants and subscriptions and tell me if you see any VMs because I do not. Fix: Check what the customer asks to confirm you see the same thing the customer does, unless you cannot for some reason. If you see something different, explain how you got to that view. If you cannot do what the customer is asking, then respond and tell the customer that due to constraints on your end, such as not being able to view all the tenants or subscriptions, that you cannot do what they are asking. Note that this person did ultimately help me and confirmed this is a bug.

Update:

  • Got up in the morning. Haven’t submitted any new tickets. Figured out all the issues myself except one and two open tickets. Hoping to see all tickets closed. Nope. Fix: You know.
  • The views in the support portal are confusing and don’t show me all my cases. I figured out what is going on and seems to be some kind of bug. Here are screen shots explaining the confusion and bug:
When I first login and go to support cases this is what I see:
Note that the last case says “audit logs” and is one of the very first things I submitted. You used to be able to type “audit logs” in the search bar and get to the audit logs when logged in as a global administrator. Now it tells me I don’t have permissions. I concluded that this is a bug. I don’t think the support person ever took the steps I provided or understood me. I requested to close that ticket over a day ago I believe and someone responded 20 hours ago saying they would close it. As you can see it’s still open.
Next I click on “See all support requests.” Wouldn’t you expect to see the “audit logs” case when you click on “See all support requests”? I did but maybe I don’t understand the purpose of that link. When I click it this is what I see:
This list does not have the “audit logs” case. 
On that same screen you can filter by Open, Closed, and All. If you select "All" on this screen, you actually get "All."
Of these cases only the first one should be "Open."
It also shows you that I still have a number of cases open after requesting to close all of them but one now. The first one named “Support Cases” is the one where I requested help to close all the cases. An operations manager responded in email and said he can’t respond in the portal due to some technical limitation. He said he is working on closing all the cases. I’m not sure how long it is supposed to take or if he can’t affect what is going on in other departments, but they are still open.
Support Cases + Tenants
The other issue is that I found three cases under my other tenant. I never signed up for support before with multiple tenants. I generally don’t even bother with support due to issues such as the above. I research and figure out things myself. So this is my first attempt at support with multiple tenants.
When I submitted the cases I presumed they would be global. I also thought I had selected the same tenant each time I submitted a case. I want to go back and test that but I don't want to proliferate more support tickets in my account.
When I signed up for support, I think I had to choose a specific tenant. If I was not allowed to submit cases in another tenant, the portal should prevent me from doing that so I presume it is OK. If it wasn't, people wouldn’t be responding to those tickets and they are.
So having tickets in multiple tenants was part, but not all of the problem. I expected the support system to be global. But it's similar to AWS where you're logging into different accounts and your support requests are specific to each account. 
Fix: Drop down on support page to switch tenants and ability to globally filter on all support requests.
  • Tip for me: When I used to work with development teams in India, I used to have to provide very explicit, step-by-step instructions to get things done correctly. 1. Do this. 2. Do that. 3. Next do that. 4. Don’t do that. I noticed that one of the support tickets I wrote in that manner it got resolved. It was the ONLY support ticket that actually got resolved without me figuring it out first. I got an error in the portal. I provided the 3 or 4 steps to get the same error. the person told me that in order to view what I was trying to view I had to go over to some other screen to register Microsoft Insights or something. Why, when you sign up for a security service you have to go to some other magic screen and register some service from a list is Fix 1: Automatically register what is required for a service to work properly when someone signs up for it. Had this same problem with Azure Data Shares. Fix 2: Note to self: Provide explicit, step by step instructions including every click you made to get to the point where you go the error. It appears that providing a screenshot is not enough because the support people don’t look at them or maybe they don’t know the system well enough to get to those screens. They are also speaking another language (something I cannot do so that is not an insult — I’m not that good.) and it may be harder for them to understand. Fix 3: If you provide a screenshot tell the person to look at the screenshot. Fix 4: If you provide a link to a step-by-step walk through with all the details in screenshots and steps to reproduce the problem tell the person you provided that and tell them to go look at it explicitly. However, even when you do #3 and #4, people still aren’t looking at things in every case. That’s when it gets frustrating.

Update:

  • Came back a few hours later. After explaining step by step what I was after in the non-compliant VM issue have now confirmed that my assessment is correct. There’s a bug in Azure. It reports non-compliant resources even though I have no resources in this test account. Here’s another screenshot:
Considering what could cause this. Here are some guesses:
A software bug with an incorrect calculation.
Would support staff working on tickets in my account with non-compliant VMs cause this?
Is there a resource in an Azure service causing this?
Don’t know but reported so hopefully it gets fixed soon.
  • As I mentioned I later realized that I had submitted tickets inadvertently in a different tenant. What I realized along with that is that is why the recommended categories when I created the ticket didn’t make sense. I recreated a case related to an IdP integration I was working on from the correct tenant, in case that improved visibility into the error message. I still was not sure which category to pick, but I found one with SAML in the description. Fix: Align the categories with the menus customers pick from in the portal so it’s easier to get requests routed to the correct team.
  • In the new case I 1.) Provided an attachment 2.) Said I provided an attachment with step by step instructions and screenshots 3.) Repeated that I provided attachment. 4.) Asked the person to look at the attachment and follow the steps. Here is the response I got:
Are you able to share screenshots? I’ll need more information on this so I can help you with this issue.

…………………………………………………………………………………

…………………………………………………………………………………

…………………………………………………………………………………

Fix: Train people to look at attachments submitted with tickets!!!
  • In this new request I explained that I hadn’t gotten a response for almost 24 hours. I asked if they could escalate the ticket. I waited a bit then went off to do other things. When I got back nothing was done on the ticket. The person asked for a the other ticket number. When I provided it, the person said they would close the ticket since another one was open. Now you want to close a ticket? Well the person wasn’t really helping me anyway so I said to go ahead and close it. Fix: When someone asks to escalate a ticket, don’t ask to close it. Help the customer get the problem resolved because that’s why they are submitting a second ticket and asking for help with escalation in the first place.
  • As mentioned, a manager contacted me yesterday via email stating that he would help me get all my tickets closed. I’m sure that manager has to deal with a lot of different departments and doesn’t really have control over closing all the tickets so maybe this is not that person’s fault. A bunch of tickets are still open. In the ticket where I had asked to get those tickets closed here’s what another person responded:
"We can ask the Support Engineers questions like potentially asking them to close the case but for them to be following process they need to hear directly from you that case closure is permitted."
So when I write into the ticket "you can close this" or "please close this" does that not qualify is permission to close the case?
Fix: Let customers close their own cases.

Update:

Back for another day.

  • Last night I had two tickets open. In my ticket related to issues with support, the person wanted to downgrade the ticket to a lower priority. However, one of the three items in that ticket was to escalate my only remaining open ticket because the person wasn’t reading it the steps I sent over to reproduce the problem or looking at the screen shot with the error message. I didn’t want to downgrade a request for help resolving what should be my only open ticket (it’s not, the others are still not closed). I sent some clarifying information to hopefully make my ask more clear.
  • My one outstanding ticket has been assigned to another team. I hope it is the right team. I added a new message on the request reiterating my request. I provided a link to the blog post with detailed screen shots and attached the document where I printed that blog post as well in case they didn’t want to click a link. Waiting to hear back. If that team can resolve the issue will be pretty cool and I’ll share the results.
I was thinking about *why* a company would not have the obvious function of letting customers close their own tickets. It’s such a simple thing. I imagine the support team complains because customers close a ticket and open a new similar ticket. But the ultimate problem in that case is not that the customer is closing and open new tickets. 
Consider the Five Whys. Why is the customer submitting multiple cases? Why is the customer closing and re-opening the same case? Because the support person responding to them is not helping them. Their problem is not getting resolved. 
Why is the problem not getting resolved? It could be one of many factors including: The support person doesn’t understand what the customer is asking either because the customer is not clear or the support person doesn’t understand English well enough. It could be the support person is afraid to let the ticket go to another team who could solve it faster for fear of being deemed “not good enough” or losing their job. It could be that the support person doesn’t have sufficient training. Perhaps the support person is afraid to ask for help due to draconian management or being derided by others on the same or a different team. The support operations management may be under pressure to meet quotas and that is driving pressure down to support teams to handle support cases too quickly and not be thorough when reading them. Support teams may not have the tools to do their jobs. Support systems which are not customer-facing may consist of broken, buggy software. So many factors could be causing this. 
I can only guess. Those running support teams need to do internal analysis themselves to resolve the root causes. Once they resolve those root causes, customers will be happier. Support teams will be happier. Cases will be handled more efficiently. The company will probably save money too.
  • And…I just got another response on a ticket I asked the person to close.

Update:

  • I’ve been going back and forth with the support person on an urgent ticket. For some reason, the support person keeps asking me for information that is in the document I attached. I don’t understand why, but it seems like the person doesn’t want to open the document and follow the steps I provided to reproduce the error I am getting. The document includes step by step instructions with screenshots of each step I took and a screenshot of the error at the end. I’ve explained this problem repeatedly above and I don’t understand why the support people at Azure don’t want to look at the information I provided and ask me questions I’ve already answered. Fix: Maybe it’s a communication breakdown of some kind but I really can’t understand why the support team refuses to follow the steps I took to reproduce the error so they can tell me how to fix it???
  • The error message I’m getting from Azure in that last bullet point is a very unhelpful error message that doesn’t really say anything. I already wrote about writing better error messages and why that is important here:

Update:

  • On the way to pick up my mail it hit me. Perhaps the person responding to my ticket doesn’t understand the meaning of the word “federation.” That’s the only thing I can think of. If the person read my document I am obviously federating authentication, but if they don’t understand what federation is they might not understand that the steps I’m taking are exactly that. Maybe my ticket still hasn’t gotten to the correct team. I’m not sure.
  • I wondered if the person wasn’t really referring to federation but specifically the issue that Microsoft only supports certain domain configurations. If so, the person did not read the document I sent over which specifically refers to the DNS documentation in this section of the document.

Update:

  • Yep. I received a response to my ticket which was a copy and paste out of the above link that I have in my steps that I took that led to the error I’m getting.
  • In addition to linking to the document from which the person copied and pasted, right below that link, I provide a screenshot of the error message you will get when you do not take those steps, and explain how to fix it.
  • So I think, if I understand correctly, the person is sending me information and asking if I took the steps in the information I sent to them.
  • I tried to understand why the person was sending this. Did I update the wrong domain? Did I miss something? I reviewed it all and I can’t see that I did anything wrong. I went back and reviewed my instructions, my document, and repeated the steps in them.
  • I still cannot get the person to answer whether they actually went through and tried the steps I provided and got this to work.
  • If they want to validate I added the DNS information correctly, they could ask for the domain name to see if I did something wrong or find the DNS records in my logs (I presume the domain I’m trying to set up is logged? I should check that….) ****

**** Note: after going around in circles on this I decided I did not want to provide my domain name to this particular person because the person will not answer simple questions about the documentation. I started pondering what someone could do with the information if I provide it and need to think through that a bit more. Then thing is: I provided all the information to do exactly what I did on their own systems and with their own domains so they should not need my domains to begin with. Also, the person will not answer the following questions which strikes me as a bit odd so I have requested to escalate this ticket: 1.) Have you followed my steps and successfully implemented what I am trying to get working? 2.) What domain name out of the metadata do I need to put in the first text box when creating a remote IdP? 3.) Which domain from the metadata requires the DNS TXT records? 4.) Where should the logs be when this process fails and are there any logs with more detailed information? See more below on my thought process regarding external IdPs and SAML and potential risks I’m still thinking about at this point. ****

  • I asked for clarification and waiting.

Update

  • I asked earlier today to close four tickets on Twitter. The person on Twitter told me to ask the engineers who own the tickets. I responded that I already did. Please help. The reason I asked this was because people keep responding to tickets I requested to close. How does this make sense?
  • I got two responses to tickets I requested to close today. The first one was the “I was glad I could help you” type response. OK fine. I haven’t responded to that response all day. It’s hours later. I just got a detailed summary of everything that was said and done in the ticket. I didn’t even read it. However, I have a question. Seeing as how the ticket is already resolved and I asked to close it, is that really necessary? Couldn’t the person be helping other customers instead who need help more than I need that summary? (I don’t need the summary it’s just distracting and like I said, I didn’t read it, so what’s the point of rehashing the entire ticket?) I did look in the portal and it is closed. Thank you.
  • Regarding my other ticket can’t find any logs related to the action I’m taking to register an external IdP. I requested more information in my support ticket in case I’m just missing the information in the logs. It appears that it should be in the audit logs and would be concerning if activities like this are not logged:

Update:

Got up this a.m. 4 days after submitting initial tickets. No response on my IdP question as to whether my understanding of the DNS information is incorrect or if there are any additional logs that would provide more insight into the problem. The other ticket I submitted a day ago: Someone responded saying they would be working on my ticket about 24 hours ago. I responded asking if certain things might be causing the problem. Today the portal says someone updated the ticket 2 hours ago but there’s no new response in the portal.

Update:

A couple of days later I’m checking back in. First of all, needed to get some work done. Secondly. Taxes. Bleh.

Now support is telling me that “they” (whomever they is) cannot see the domain I’m setting up for federation. Alright. We’re getting closer to what I need. Someone is actually looking at my specific configuration. However, the reason they can’t see the domain is because I can’t add it. That’s what the ticket is all about. I get an error message provided in the steps I asked support to go through to tell me what I am doing wrong. I still don’t know if someone took the steps I provided in the ticket. What I wonder is, why not look specifically at my configuration and account in the first place and tell me what is wrong with it instead of asking generic questions? That seems to be part of the reason this is dragging out. I provided the specific steps I took and if someone duplicated those steps they should get the same error. I’m not getting a response as to whether those steps *should* work or if there is a problem with the steps? But at least finally someone is not just sending me links to documentation I already have or steps I have already taken but telling me what they need to solve the problem. They are looking at what I have done and the steps I’ve taken. Hoping for a resolution soon and will keep plugging away on it.

I’m going to stop this post here. I need to get work done but hopefully somehow support teams everywhere can think about how you could make things less painful for your customers, because personally I find things like this very painful. Azure is a good cloud platform, and they are not the only ones with this support issue.

This week end I had someone from AT&T basically yelling and talking over me, repeating the same thing in a condescending voice to the point where I just hung up. I probably sounded a little frustrated when I called in about a $400 bill where I was charged incorrect amounts and wanted to know the status of the phones I had shipped in. (I do not recommend buying anything online from AT&T go to a store.) I learned how to get around basic support thanks to someone at the AT&T store. That next person I got in a resolution-oriented department (because the company doesn’t want to lose business) calmly re-assured me that basically everything the last person yelling at me was saying was wrong. The issues are not totally resolved but I have my fingers crossed that the phone that got lost to me on the way to me will get processed correctly when it got sent back. I got two phones on pretty much the same day after I called and the support person submitted a case because I had not received the original phone for two weeks. One of them looked like it had been opened. Now I’m hoping I don’t get charged for the phone I sent back because I’m honest. I could have just pretended it never showed up and would have no risk of being charged for it.

Update:

I wasn’t going to write anything more here but this response was too much:

I understand there may be some confusion regarding the process that occurs when support tickets are filed.
You may find the linked documentation helpful.
Manage an Azure support request — Azure supportability | Microsoft Docs

Another condescending response. Here’s my response:

Not helpful. What would be helpful would be if I could get the things working that I’m submitting support requests about.

I feel like I’m close after 5 days of back and forth with Azure and another vendor on fixing the main thing I want to get working, but I don’t think either company has actually tested the steps I provided to tell me what I’m doing wrong. Just keep plugging away.

One other note is that the Azure support portal shows that tickets have bene updated or changed when there is actually no response or action showing in the portal. For example, a ticket I created 5 days ago and was resolved shortly after I opened it shows that it was updated in the last hour. I have received no messages (and don’t need any since it is resolved and should be closed) and the case is still open. If anyone is looking at that last updated time in the portal, it seems to be inaccurate.

Update:

Well, I’m still updating here because I wanted to note that I was testing out using Azure Data Share in my class instead of an alternate cloud service from another provide since I am teaching an Azure class. However, when I tried to use that service I set it up incorrectly somehow and I got an error. I submitted a ticket asking how to fix the error. I never got a response in the Azure portal on that ticket. I think someone may have responded in email only but as noted above I would like a record of the responses in the portal, and I’m getting so many emails for non-issues that I can’t keep track of what email goes with what ticket. See the example above where 4 people responded to a ticket that was an azure bug that seems to have resolved itself. The responses came after I requested to close the ticket. Just now I got a lengthy three paragraph response on a ticket submitted 5 days ago that also was a bug and I reported it to Azure and asked to close the ticket. After the 3 paragraph response telling me the engineer was going to report the bug and close the ticket, I got another email from a support manager on some ticket asking me how the engineer did. I don’t know which ticket that was for and I don’t have time to look it up. So back to Azure Data Store. I couldn’t get it working in the required time frame and don’t have more time to spend on it right now. Also, after testing it out it requires way too many steps for my simple use case. Other cloud services for something like this are much simpler depending on what you are trying to do. If you must store your data in a particular Azure data storage service and need to share it, this service may be your only option. In my case, that’s not a requirement. I responded to the ticket after a few days saying never mind, please close the case.

So now I have one case that should be open.

That one case is the main case I have been trying to resolve all along. I think I figured out by reverse-engineering a question from an Azure support person that I put the DNS TXT records on the incorrect domain. I asked specifically which domain in which text box in Azure should get the DNS TXT records. I also asked for clarification on what should go into one of the boxes for domain federation. In a prior request I asked for where I can find detailed logs that provide more information than “failed” which is basically what the error message I get in the Azure console says. I never got a response on the error logs and waiting to hear back on the rest as to what I am doing wrong that is causing Azure to report the error that doesn’t say anything.

In the meantime the other vendor is sending me links unrelated to the problem I’m trying to solve. I have only sent them one or two questions so not calling them out here. I’ve had repeated problems with Azure support for years which is why I finally wrote this blog post. I asked the other vendor which domain I need to put in which Azure text box to make this work. Maybe they have some logs or information that can help me figure this out and get it working.

Now I wait…

Update:

The person who is currently assigned to my IdP ticket told me yesterday the ticket would be escalated…

But at the same time the person again asked for private domain name and IdP information again with a link to a folder to submit the information. I have repeatedly indicated that I do not want to provide that information but that I want generic answers to my questions about the documentation.

My response: Please let me know when the ticket has been escalated.

This person seems intent on *not* answering my questions. The person keeps asking me for private information about by my IdP and domain names. I provided the detailed steps I took in order to avoid sending the specific details of my configuration. Anyone could follow those steps to reproduce the problem and see if they could get the same results. Anyone could tell me on step X you say to do Y but that’s wrong, you need to do Z. For some reason I don’t understand, this person doesn’t want to or can’t do that but they won’t come out and say it. They just keep tying to get my private IdP information.

That got me to thinking about the risks associated with sharing the private information you use to set up an IdP between different cloud systems. What are the chances someone in Azure support could do something nefarious if they have the details of your IdP configuration? I have yet to fully evaluate this threat model.

I’ve been starting to look more closely at SAML configurations and security related to this. If someone in Azure support has the details about your IdP configuration (the metadata you provide to configure the IdP) could they then impersonate your IdP? Could they trick users into logging into the wrong IdP and grab authentication tokens along the way? Could the individual set up an Azure IdP in their own account using your IdP information (similar but different than subdomain hijacking)?

Azure requires that the domain name you use in this type of federation is NOT in Azure as I documented in my steps that I sent over. That means that if you use this type of federation, there’s nothing to say that someone couldn’t take your metadata and set up this configuration in some other account. You must set up the appropriate DNS records on the associated domain. But is that information specific to only your account? I need to revisit that.

Waiting for someone new to respond indicating the request has been escalated.

Update:

I started this ticket on April 12th and it’s now April 29th. To be honest I haven’t been looking at it closely since it all feels like a waste of time at this point. The person I’m talking to is not answering my questions and seems determined to ask for information I do not want to provide. I just want generic answers because at this point I don’t trust the person. I just want to get the ticket escalated.

Today I got two responses neither of which were from a new person and neither of which escalated the case. I replied that I appreciate the effort but I don’t want to provide anymore information until the case is escalated.

I submitted three separate cases with individual questions about the Azure documentation — not specific to my configuration or the error I got on the Azure platform — to see if I can get answers another way. I haven’t logged back in yet to see if those got answered. I’m very busy and this is all very time consuming.

Right now…I need to get some work done…

Oh wait. The person just responded and said they did not say they would escalate my ticket. I’m very confused.

I asked if the person could please escalate my case because we seem to be having communication issues. I do not feel comfortable providing additional information at this point. Also, the question I asked in the beginning was for someone to follow the steps I did and reproduce the error message and tell me what I’m doing wrong in my steps. As far as I can tell, no one has done that yet.

~~~~~~~

Alright this was all so distracting that I stopped the urgent matter I’m working on and logged into the Azure portal to see if my other questions got answered. Just curious. No high expectations, but hoping they got routed to a person that truly wants to help.

The first one I looked at did and the answer was reasonable and seems like it should be correct:

Where should the logs show up in Azure when you create a new external IdP. The person tells me they are supposed to show up in the audit logs. That’s what I understood from the documentation so that confirms one thing the other person absolutely refused to answer for some reason. Thank you.

But now the problem is, I can’t find any entries related to my action to add an external IdP using SAML federation. I could just be looking for it incorrectly or not know what to search on or what it looks like. Or there could be a bug related to the specific error message I get. I asked the person if they could send me a sample or tell me what to search on to see the entries related to the 4 steps I took to try to add a new IdP. (I provided those four steps when I submitted the ticket). Hopefully the person can send me a sample or answer this. Then I will repeat the steps and look again to see if it’s a bug or I’m missing something.

For the other two questions, which were distinctly different if the support people and bothered to open and read them, I got two responses which both said something to the effect of, “It looks like you submitted two duplicate tickets. Please confirm.” Clearly neither person (or the same person?) bothered to read the contents of the ticket. Even if they presumed it was a duplicate, why did you not answer at least one of what you incorrectly presumed to be a duplicate?? At least one of my questions would have been answered. What a waste of everyone’s time.

All this is why I never pay for Azure support. But I responded hopefully in a civil manner and avoiding the words going through my mind. I clarified that they were two separate tickets with two different questions in each of them. Could you please answer my questions?

I was thinking about alternate ways to resolve this problem and I wanted to research some other security issues anyway related to this. I’m about to set up my ADFS server and start reverse engineering this whole process myself so I can figure out what is going on. Asking support is definitely not saving me any time. I’ve set up ADFS servers in the past but was hoping to avoid the hassle.

~~~~~

Based on my experiences to date I would not use Azure, personally, if I was a small business getting started. AWS is much more start up friendly, though have challenges with their support as well sometimes. The thing is, when I wrote about it, they reached out to me to help resolve the problem. If you have a large company with a dedicated account manager hopefully you are getting better support than this.

As a side note, I just had an interaction with Google Support that was heavenly compared to all of this. To be fair, it may have been a simpler question, but still, it was far superior to this whole experience. My only problem with Google support is that I can’t find my ticket list in Google Workspace, but that could just be me. I did have challenges when I accidentally registered a workspace through Google Domains. I can’t recommend that. Create your Workspace directly at Google. But I digress.

Update May 6

I had another issue in Azure that was actually resolved. The support person said it was a bug. There’s a mystical magical property on an Azure service which you can get to from the CLI but you cannot see in the portal. It’s also very interesting because you might think you have restricted private networking with an endpoint on this particular service but the that can be overridden.

Anyway finally someone resolved the issue. I had already asked for the case to be close. Now the manager is sending me emails about the case. REALLY? Is this problem keeping cases open because of Azure support managers? Just close the case.

By the way I gave up on the initial issue I submitted when I started this blog post for now. As you can see it’s taking days of my life and I’m really busy. The point is past where I would use it in a class. I might pick it up again or try to reach out directly to people at Microsoft and Okta that I know to resolve the problem instead. I’m not sure yet but right now I’m too busy to think about it.

Update:

I had to submit another request today. I tried to create a VM in a subscription and the error is that the size is not available. The other issue is that I can’t create VMs at all according to the error message. When I click on the “See all sizes” list I randomly do or do not get alternate sizes but I can’t choose any of them. It could be an access issue but if it is, these error messages are completely inaccurate and confusing, so I submitted a request to see what is going on with screen shots so Azure can see what the experience looks like.

Since I was in there I replied to the open ticket the manager replied to and explained that I had figured out how to fix my issue myself (add my IP to a service firewall) but then that wasn’t working so I came back to re-open the ticket. That’s when the support person had responded with a bug and resolved the problem. All good. I didn’t want to reply and generate more emails. I had already requested to close the ticket so I presumed that would be the end of it. But, I explained the three bugs related to what I was trying to do to the manager including the fact that the documentation and Azure portal doesn’t have the public access option and asked to close it.

Just for kicks I looked at my original ticket on the IdP. FINALLY, FINALLY, FINALLY!!! I got a technical manager on the ticket who said they reproduced the error and confirmed it is a bug. The ETA for the fix is 5/13/22.

THANK YOU.

Now I just need someone to answer a couple of questions related to putting in the proper IdP domain but I’m busy so I’ll come back to that later.

  • I discovered something recently with the help of students in a class which I overlooked for too long — but I never would have expected a cloud platform to work this way. Azure private IPs are not by default disallowed Internet traffic like they are on AWS and Google Cloud Platform. Azure creates a magical outbound IP address on private IPs that gives them outbound access to C2 hosts, oops I mean the Internet. It appears there is no way to turn it off but you can do things like add a NAT to your networking. OK so to block that you have to add a NAT and pay for it and then disallow all outbound Internet traffic to the NAT? Hopefully NSG rules apply so if you block all outbound traffic to a subnet or host it can’t reach out to the Internet. I haven’t tested that. There should be a way to turn that off.

May 12, 2022

I submitted a support request on May 1st because I cannot create any VMs in a particular subscription. The person that answered the case told me to put in a limit increase request. However, when I looked at my Azure quota I should have had 10 of numerous types of VMs. I put in the limit increase again anyway according to the person’s instructions just in case it would help. I had to put in 11 because my existing quota was 10. I had only created one VM ever in this account up to that point because it’s a new account I created specifically for a class. That shows that I do have a quota of 10 already — unused — since I cannot put in a limit increase for less than 10 in the region I’m trying to create VMs in.

The person who responded to the quota increase could not grant access to some of the types of VM images I requested as they are not usable with current hardware. Why are they in the list in that case???

The quota increase did not work. I cannot create VMs in almost any region.

Initially I thought it was a permission issue because I figured out I could create a VM with the global administrator in a particular account and region. However, a later test produced different results.

I asked people on Twitter if they were experiencing this problem and a number of other people are so this is not just specific to my account.

I got an error message about a “low priority” request failure. I am not requesting low priority VMS or spot VMs:

This is not a free account and I am paying $100/month for support which so far has cost me a lot of time and in most cases I resolved the issue myself. In the other cases they were mostly bugs except I think one.

I let the person who increased the quota and the person who responded on the case where I said I could not create VMs know that a limit increase is not solving the problem. I submitted the error message I am getting.

The person working on the quota did not respond. That person responded initially that they are overloaded with requests. I wonder if that is because something is broken in relation to quotas and limit increase requests.

The person to whom I submitted the original case is requesting a phone call — Again Azure support??? If I wanted a phone call I would have put that on the ticket. Additionally, why is it that this person needs a phone call in the first place? This is a simple request:

I want to create a VM in region X with user Y.

I provided access to diagnostic logs.

I provided a copy of the error message when I create a VM. Why is this person asking for a phone call instead of looking in the logs to resolve the problem? What more information could possibly be required to solve this problem on the Azure side? Maybe there is something but I’m too busy right now and honestly not in the mood for a phone call with Azure support.

I will now methodically go through my account and see if I can create a VM anywhere. I noticed that when I tried to create a VM in AUSTRALIA I had access to do so, while a bunch of other regions in the US say my subscription is not allowed to create VMs in those regions. I never asked for access in Australia.

I am also going to methodically test which users and VMs are allowed to create a VM and what permissions may be lacking. This seems like something that would be easy enough for Azure support to do — especially given that this is happening to others, not just me.

Time for a deep dive into the logs and configuration….

By the way, I also checked on the two tickets where I submitted questions about an IdP configuration on April 25th yesterday and no one had responded to those questions yet. Don’t have time to look at those right now.

………

Update with screenshots. Troubleshooting…

It used to be that you could open an Azure account and create a VM with the global administrator account. I don’t really recommend that but it used to work. Maybe something changed. But here’s what happens when I do that:

As you can see below there are no available sizes. This is also what I see when I chose other types of VMs.

This is what I seen when I click “all sizes.”

Now expand the list:

I’m in US East 1. I tried a different availability zone and didn’t work either. Next I switched to East US 2 on this VM creation screen. This seems like a bug. There’s no pricing and the error message references East US instead of East US 2:

Let’s go back and start over since switching regions on this screen seems to have a bug.

Let’s choose a Windows VM — same thing:

With this Windows VM option East US produces the same result but interestingly East US 2 produces a different result when you click to see all the size options. The E-Series is in a separate category. It seems to be selectable (and is expensive). Choosing this option seems to pass validation. Checking to see if it actually works.

I’ve already provided feedback multiple times and through support cases. Not sure this is working.

Nope. Even though the VM passed validation here’s the error message (which is different than the one I saw previously about low priority):

Can’t create a VM in South Central US, apparently.

Oh wait…I got an error above that I could not create that VM with the E-series size. I clicked the delete button on that page after I got the error just to clean up anything that may have been partially deleted. However, when I got to create a new VM I see that it appears in my list of VMS. No error message appears here.

However, when you click on the VM you see an error message:

Apparently the VM failed, but the following resources were still created an hanging around in my account:

That’s problematic. First, I’d be billed for them if I didn’t realize they were there. A network interface is hanging around for someone to attach to something else and also a public IP address. I’m going to force-delete those.

I get an interesting error message when I *delete* resources?

So Azure doesn’t have capacity to delete resources??

It worked after two more attempts and refresh after each attempt.

It appears that my subscription also cannot create resources in West US 3.

Did I put some restriction on this account? I don’t recall doing that. I am going to teach about Azure governance but I don’t believe I applied any such restrictions up to this point that I remember.

OK interesting. Australia Southeast is OK. If I had put restrictions on this I don’t remember at the moment, this region wouldn’t be working as I have no reason to allow this region. So that’s odd.

It also appears that I can select various resources in this region.

Let’s give it a try. Why not. I’ve been to Australia. I love it there!

Azure tends to default to expensive sizes. I’m going to choose the least expensive one.

Oh, also interesting. I couldn’t launch this VM until I put in a phone number. I didn’t put in a real one as I do NOT want someone vishing or trying to get at my phone number used for authenticating to a lot of different things.

Well, that seems to work. I can create VMs in Australia. That’s super. I guess it’s better than nothing.

Went to submit this information in a new support request since I’m in a different tenant and subscription for this test. Here’s what it says. This is absolutely not helpful because this hasn’t worked for almost two weeks now. Additionally, customers do not want to have to guess by trial and error like I’m doing above to figure out where they can create a VM. Additionally, VMs are not available in almost every US region??? Either Azure has a serious capacity issue or some kind of bug related to quotas and VM availability.

I submitted a new case with the a document and all these screenshots attached to it. I requested that this problem be escalated as it seems like some kind of bug. Does Azure really not have capacity in the entire United States to the point where I have to create a VM in Australia?? Seems like something is wrong here.

I also once again requested EMAIL support, with responses in the portal only. Hopefully, the Azure support engineer that gets this request will honor that preference and the escalation request. Hopefully they will look at the attachment. As with all my other requests I provided access to diagnostic logs as this is a test account. It doesn’t seem like anyone is actually looking at those diagnostic logs as far as I can tell. The suggested response in the initial case to increase quota was not aligned with the error messages, the quota allocated to my account, and the number of virtual machines I can created to date.

There’s another minor issue with the above screen. When you change from priority C to priority B you have to re-enter your preferred contact method and business hours. It’s not a big deal but mildly irritating that it can’t just save those settings.

For the sake of completeness, I have been getting different error messages for the same issue. In the past I got this quota error even though the limits in my account showed that I had available capacity. This has to do with “LowPriorityCores” quota but I was not choosing a low priority option.

The error message I got today is completely different when choosing the same options so it seems like there’s some kind of bug in all of this (or 6).

Also, I searched on my quota in East US and East US 2 for the BS instance size as an example:

Unless I’m looking at it wrong I should be able to create ten of those in East US 2. I haven’t been able to create one, ever. It’s not just a “capacity at the moment” thing. It seems like the support engineer that told me to put in a request to increase my quota did not even check to see if that was actually the problem. That may be why the team that is handling quota increases is severely overloaded at the moment per their message back to me. Other support staff are not verifying that a limited quota is the issue by looking at the settings in the customer account or error messages they are assuming that is the problem and telling people to submit limit increase requests. These requests create more work for the quota increase team without actually solving the problem.

Update May 13, 2022

Last night, I also continued to investigate the issue in my account that is preventing creating a VM in any US region with AZs. At some point, I realized that I could not create a VM with AZs in the Australian region above where I took a screen shot. The VM that worked did not include AZs. I started to think maybe I can only create a VM with no AZs, However, I went on to create a VM with AZs in another Australian region with the latests Microsoft VM image. I presume the latest version of Microsoft datacenter works with AZs? I’m digging around for any documentation that tells me limitations that might be preventing me from doing this.

After interacting with Azure support on Twitter, they said they found the ticket and were going to do something about it. They did not. When I logged into my account this a.m. here’s what I found:

  1. The case where I requested escalation — I provided the screen shots above that shows exactly what is failing in an attachment. Once gain the person simply asks me what I am trying to do. *sigh*
  2. The case where I initially reported the error messages on the VM creation screen ….no response.
  3. The case where I asked for an increased quota request and reiterated that after increasing the quota I still can’t create a VM: The person asked me what quota and regions I want to use. *sigh* The person did say this which is interesting:

“There are certain restrictions on multiple Azure subscriptions and because of that you may not be able to create VM in the requested region.”

OK so I create a new Azure account, I’m located near the East US regions so Azure defaults to that. I can’t create a single VM of any type or size in that region in the default tenant and subscription. I asked for clarification as to exactly what the problem is in this case. On Twitter the Azure support team says this is not documented. Hoping to get more information from my Azure support case where I requested an explanation.

So for now I replied to two tickets asking for the following:

The ability to create any Linux or Windows VMs with AZs in East US, East US2, West US2 in any B series or D series size.

I mean, I just want to create one VM.

We’ll see what happens.

Meanwhile…I searched around for others that might experiencing the same issues. Seems to be a bigger problem in Europe:

I found this page on allocation issues on Azure.

Interesting that they refer to StackOverflow prior to Azure support:

One thing that I’m trying to figure out when looking in the Azure portal is which are the latest/best VM sizes to use. I’ve tried many different types of VMs and a lot of them don’t work. Also, it seems like some of the less expensive options are being deprecated. The article above explains which VMs not to use but really this should be obvious when choosing a size in the portal.

Additionally, as shown in the screenshots above — I can’t choose *any* size in US regions even though I have quota for all of them.

Other examples of people having similar issues on various forums:

https://techcommunity.microsoft.com/t5/azure/virtual-machine-sizing-plan-not-available-in-creating/td-p/1258985

Update:

Creating a VM in an availability set works. It doesn’t work with the availability zone option. That’s one way to work around the VM problem. Well, almost. It let me pick from various machine sizes. But it still fails. And…the diagnose blade fails as well.

Self-diagnostics…no issues.

Oh. Just wait a few minutes and then seems to be OK. Incorrect error messages.

~~~~~

That’s it for now. Back to trying to test new features in preview and prepare for an Azure class! I go through the struggles of figuring new things out so my students and customers don’t have to. =)

Follow for updates.

Teri Radichel | © 2nd Sight Lab 2022

About Teri Radichel:
~~~~~~~~~~~~~~~~~~~~
⭐️ Author: Cybersecurity Books
⭐️ Presentations: Presentations by Teri Radichel
⭐️ Recognition: SANS Award, AWS Security Hero, IANS Faculty
⭐️ Certifications: SANS ~ GSE 240
⭐️ Education: BA Business, Master of Software Engineering, Master of Infosec
⭐️ Company: Penetration Tests, Assessments, Phone Consulting ~ 2nd Sight Lab
Need Help With Cybersecurity, Cloud, or Application Security?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
🔒 Request a penetration test or security assessment
🔒 Schedule a consulting call
🔒 Cybersecurity Speaker for Presentation
Follow for more stories like this:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 
❤️ Sign Up my Medium Email List
❤️ Twitter: @teriradichel
❤️ LinkedIn: https://www.linkedin.com/in/teriradichel
❤️ Mastodon: @teriradichel@infosec.exchange
❤️ Facebook: 2nd Sight Lab
❤️ YouTube: @2ndsightlab
Azure
Cloudsecurity
Cybersecurity
Cloud
Security
Recommended from ReadMedium
avatarArmandkeza
Architecturing your entreprise azure landing zone

After some initial PoC Kewing Inc is ready to start migrating application to azure cloud.

3 min read
avatarSukhpinder Singh | .Net Developer
Day 1 — Azure Open AI Challenge

Learnings include creating Azure Open AI services, deploying GPT models, interacting with GPT in Playground, and optimizing model…

4 min read
avatarKokaTic
Deploy fullstack application with TerraForm & Azure in 5 minutes !

PostgreSQL Database / Azure Function as the backend, Azure Web Apps as Frontend

5 min read
avatarRahul S
Security of LLM Apps: Prompt Injection & Jailbreaking

THE GEN AI SERIES | KEEP IN TOUCH

6 min read
avatarSousouni Bajis
A Comprehensive Microsoft 365 Security & Incident Response Guide

In our prior discussion, we’ve journeyed through the worlds of Azure AI and Microsoft 365 Copilot, exploring their impact within SharePoint…

18 min read
avatarJonathan Mondaut
How ChatGPT Turned Me into a Hacker

Discover how ChatGPT helped me become a hacker, from gathering resources to tackling CTF challenges, all with the power of AI.

4 min read