SoundCloud Status

We’re performing maintenance on the database that handles overview stats starting at 10am CET on Thursday, January 19th. This maintenance will only affect the overview stats section of the site, i.e. “Your Stats” (http://soundcloud.com/you/stats) — the rest of the site will be up and running. While we’re doing this, the “Your Stats” section will be inaccessible, but all data during this time will be recorded.

We’ll be as quick as possible, and will keep you updated as we usually do by posting our progress on Twitter.

Thanks for your patience with this.

Update 11:07AM CET: Work is finished!

We’re performing necessary maintenance on the database that handles overview stats starting at 8:30am CET on Tuesday, January 17th. This maintenance will only affect the overview stats section of the site, i.e. “Your Stats” (http://soundcloud.com/you/stats) — the rest of the site will be up and running. While we’re doing this, the “Your Stats” section will be inaccessible, but all data during this time will be recorded.

We’ll be as quick as possible, and will keep you updated as we usually do by posting our progress on Twitter.

Thanks a lot for everyone’s patience and understanding!

Update 9:46am CET: Maintenance is finished! “Your Stats” are available again.

Hey everyone!

We’ll be performing maintenance on the site starting at 8:00am CET on Wednesday, December 21st. During this time, we’re expecting to be up, but actions across the site will be delayed until we are finished.

Examples of actions that will be affected/delayed:

• posting comments to tracks
• play counts and other stats won’t appear until maintenance is done
• track and image transcoding will be delayed — recommendation: it will be best to wait until the maintenance is over to upload tracks and/or images
• any deletes, edits or updates to tracks and/or your account will be stalled

Please note: updates to our API will also be delayed, which will affect apps

All data will be captured during this time, just updated after maintenance is finished. We will be posting our progress on Twitter.

Apologies if this causes an inconvenience. We’ll be as quick as possible!

Thanks very much for everyone’s patience!

Update Berlin 11:33am: Maintenance is finished!

11:56 CET: Hey all! Play counts are currently not updating as there is a backlog causing a delay. We are currently working hard to get them back on track. Don’t worry, all plays are being logged correctly and will be recovered once the queue is cleared.

To continue improving the Dashboard, we’ll be performing maintenance on the Dashboard tabs: ‘Activity on your tracks’, ‘Incoming tracks’ and ‘Exclusive tracks’, starting at 10am CET — this has been moved to 10:15am CET. The main Dashboard and the rest of the site will be functioning as normal during this time. 

The items in your tabs are safe and will be back soon, thanks a lot for your patience while we make these improvements!

Update 10:50am CET: Dashboard tab maintenance is done! Tabs are available again.

9:39 CET : Very sorry for the current downtime. We are performing maintenance. The team is working hard to get the site back up as soon as possible. Sorry again for the inconvenience this causes.

11:07 CET: The site is back online but we are working through a transcoding backlog. This should be resolved and cleared shortly. Thank you again for your patience!

11:20 CET: Transcoding and backlog are clear. Thank you for the patience and again sorry for the inconvenience this has caused!

We can confirm that the ongoing service outage that started at 8:40pm last night CET has been a distributed denial-of-service attack.

We’re continuing to working hard to mitigate it.

7:47pm CET: our attempts at mitigating was successful and we’re now stabilizing. We might experience some intermittent hiccups, transcoding delays and unavailable players but it looks like we’re on track on getting back to full capacity.

Update: transcoding backlog is eliminated now!

Thank you all for your patience and supportive tweets, comments and emails. It feels great to feel the support of such a great community.

01:08pm CET: After seeing the service stabilize for a while this morning CET, we’re still experiencing an unidentified network related outage resulting from an unusual high amount of traffic and are working all hands on deck to identify and fix the root cause for the outage.

Our CTO, VP Engineering, Head of Operations and all available Operations engineering resources have been working around the clock to bring the service back up as soon as possible. We want you all to know that we take this issue very seriously and are doing everything we can to resolve the situation. We’ll post updates to our Status blog as soon as we receive clarification about progress of the issue. At this time, we don’t have an estimated time about when we can expect the service to be fully operational. Your data is safe, we deeply apologize and will not let you down.

Previous updates

4:08pm CET: service is back up and stabilizing. We’re still seeing intermittent connection errors and will continue to monitor performance.

5:27pm CET: down again

11:57AM PDT: Very sorry for the current downtime. We’re aware of this and the team is investigating and working hard to get the site back up as soon as possible. Sorry again for the inconvenience this causes.

12:25PM PDT: Thanks very much for waiting. Our team is still investigating and working very hard to restore service. Again, sorry about this and thanks for your patience!

1:00PM PDT: We’re currently in the process of troubleshooting and still working hard at restoring service. 

1:52PM PDT: Process still ongoing, we’re working as fast and hard as possible. Apologies again!

3:43PM PDT: Still working very hard to resolve this and the process is ongoing. Extremely sorry for the delay, thanks for your continued patience.

5:20PM PDT: We’re very, very sorry for the downtime, everyone. We have as many people working on this issue as possible. We’re still addressing this and troubleshooting. Again (and again), we really appreciate your patience and we apologize for the inconvenience this is causing.

6:50PM PDT: Once more, extreme apologies for this. We can assure you that our team is working hard and the best they can to bring SoundCloud back up for everyone. We’ll continue to post here once we have more details. Thanks so much for waiting, everyone!

9:19PM PDT: Huge thanks to everyone for sticking through that. SoundCloud.com is up again! We’re currently experiencing a transcoding backlog though, so if you’re stuck in this, please wait it out. 

11:11PM PDT: The SoundCloud API is currently still down (hoping to bring it back soon), but the regular site is up and the transcoding backlog is eliminated now! 

01:08pm CET: After seeing the service stabilize for a while this morning CET, we’re still experiencing an unidentified network related outage resulting from an unusual high amount of traffic and are working all hands on deck to identify and fix the root cause for the outage.

Our CTO, VP Engineering, Head of Operations and all available Operations engineering resources have been working around the clock to bring the service back up as soon as possible. We want you all to know that we take this issue very seriously and are doing everything we can to resolve the situation. We’ll post updates to our Status blog as soon as we receive clarification about progress of the issue. At this time, we don’t have an estimated time about when we can expect the service to be fully operational. We deeply apologize for this and will not let you down.

Hey all, we’re seeing an elevated amount of errors on the site and are currently investigating the cause for this.

And we’re back. Something went wrong in the system and our Ops team just went all Zach de la Rocha on it.