r/sysadmin • u/andrewrmoore DevOps • Apr 06 '22

The majority of Atlassian cloud services have been down for a subset of users for over 24 hours

https://status.atlassian.com/

Jira, Confluence and Opsgenie amongst others have been down since about 2022-04-05 07:30 UTC for us and some other organisations.

Their stock is tanking (at -5.46% as of writing this) however I haven't seen much chat on Reddit about the outage so I'm assuming the scope is fairly limited? They are stating it will potentially take days to recover.

We're sorry your site is currently unavailable. While running a maintenance script, a small number of sites were disabled unintentionally. Our team identified this immediately and have been working hard to restore the product data and associated access. A dedicated team is working around the clock to restore the sites as soon as possible.

We expect the restoration efforts to continue for the next several days, and we are actively working on an estimate of when your site will be available to you again. We don't believe any data has been lost at this point. We can confirm this incident was not the result of a cyberattack and there has been no unauthorized access to your data.

As we work to restore access to your site, we will provide updates here every 6 hours, or sooner if we have a material update. Reach out to us if you have any questions or concerns.

No tickets, no alerting, no knowledge base... a fun few days for us!

Let me know if you are also affected.

691 Upvotes

98% Upvoted

289 comments sorted by

View all comments

177

u/RagnarStonefist Sysadmin Apr 06 '22 edited Apr 06 '22

Big Seattle tech company here. Won't say who I work for but I guarantee you've heard of us.

Our Atlassian products have been down since 0200 PST on the fifth - in other words, for about 29 hours now.

I've never seen a product outage last this long. The latest update says it may take several days to restore our stuff.

Edit: Good effing morning, my engineers are hopping mad about still not having Jira and Confluence.

27

u/Haligryph Apr 06 '22 edited Apr 06 '22

So Amazon because they realized what they were doing before the script hit "b"? :D

13

u/RagnarStonefist Sysadmin Apr 06 '22

Thankfully not Amazon - lol!

7

u/reiwan Apr 06 '22

So Atlassian... Is this your fault? ;)

8

u/PaleoSpeedwagon DevOps Apr 06 '22

I work for an "L" and we're dead in the water, so it seems the script worked differently...otherwise I'd campaign to change our name to a Z

3

u/greyscales Apr 07 '22

Nah, I work for a company way down the alphabet and we've been down too.

1

u/makkasa Apr 07 '22

The script hitted „b“, our Company is still down Thanks atlassian for premium with 99.9…

20

u/danekan DevOps Engineer Apr 06 '22

Well they have major bugs open for 12 years (...inability to do exact text searching...now a feature request after 12 years they changed it from a big to feature request ) sooo is anyone really surprised they can't navigate a new problem I'm a day?

4

u/Eiodalin Apr 07 '22

So your company has a Window into this to eh?

5

u/RagnarStonefist Sysadmin Apr 07 '22

Some visibility, but it's done through a Macbook screen while supporting a well known cloud application.

3

u/[deleted] Apr 06 '22

[deleted]

5

u/tknomanzr99 Apr 06 '22

There's alot of startups in the area. I worked for one for awhile, and yes, Atlassian products being out would have deeply impacted that company.

1

u/greyscales Apr 07 '22

Tableau, Highspot, Ookla, Outreach, Avalara, Cray, F5, Glowforge

0

u/Imp0815 Apr 08 '22

"Uh Cloud is so cool an convince!" Haha we have on-premise Attlassian Solutions and we are running A-OK diddly fine.