Steadfast dead again! Go figure the response given.

[adamnp]

Been 45 minutes now...Claiming major storms in chicago..Go figure, weather.com says Partly Cloudy in chicago...I, only host voice servers here through NationVoice on SF network....Their response to me was '1 server is down, be up in 15 minutes' ....I respond with, <[GGS]Adam> Seems more than 1 machine, pings to the whole network are dead...Gameservers down, voice servers down...Sure just 1 machine? ... "Yes 1 machine thats it" <[GGS]Adam> Ah, curious then why SF is telling customers DC Power emergency>"

 

Go figure I get the response, "Actually the whole rack went down....no wait, the whole DC is down atm! ETR ~15-30 minutes"

...Then no response again...

 

Another reason to host w/cc instead of these guys. I don't understand the lies from both providers, why not offer the truth to customers?

 

Is it me? Just frustrating to me, I really feel for customers when I see companies lying blatently like that. I don't understand it.

 

I understand issues happen and am happy to work with the provider, I just like to be given the truth, so I can pass this on to my customers easily, quickly and in good faith. Thats the aggrivating part :]

 

Adam ...Frustrated tonight! dont mind me :]

[Daersun]

huh?

[adamnp]

Bunch of racks must be back up, total time 1 hour 52 minutes. Still some down, as 50% chicago vent customers are still down.

Dearson, check NationVoice, this is where I got there info from, and the info from Steadfast was provided via livechat until their website dropped aswell. Website is back up now, livechat offline, but still machines are down. Woops, SF didn't say anything out of ordinary my apologies, it was NV. Now that I re-read what I posted :]. SF Gave me truth, NV told me pony tales.

 

Apologies on that SF.

 

:/

 

Adam

[adamnp]

All are up now...Just didn't understand the odd/misleading reponses from NV over the outage.

[Daersun]

Ok, I guess I need to clear this up.

 

Servers were never DOWN.

 

On the multi honed network, ATT was clogged. The DDOS protection protected the switches to make sure they didn't get maxed out. All incoming though ATT was hit with bad loss etc.

 

Servers never went down and Steadfast explained everything on WHT.

http://www.webhostingtalk.com/showthread.php?p=4596752#post4596752

 

Colocrossing is a good provider, but no reason to trash steadfast who has been nothing but professional to us.

[adamnp]

Thats fine, I do understand that and I did apologize for my acusations to SF, but regardless what they told you they specifically told me repeated times there was a "Datacenter power failure" and "Major DC Power emergency" and the same issue was again repeated by NationVoice Staff to their entire public channel, stating the same thing "Power failure at datacenter". And I did see the post at WHT ~40 minutes After I reported the outage to NV/SF, and was still recieving the same messages from them both regarding 'power issues' ..This was reasoning for my post, why both were claiming power outages when that had nothing to do with the reason it was down, and had infact posted the real reason elsewhere. That was my only bitch to begin with :] (Keep in mind I was acting as a normal customer inquiring about information)

 

So, like I said, what was aggrivating was the false information given to me. The course of action was understandable.

[Defcon|Rich]

ATT/T is causing problems throughout Chicago tonight it's not specific to Steadfast. If you have a server hosted in Chicago and are routing through att chances are your not having much fun tonight.

 

Ok, I guess I need to clear this up.

 

Servers were never DOWN.

 

On the multi honed network, ATT was clogged. The DDOS protection protected the switches to make sure they didn't get maxed out. All incoming though ATT was hit with bad loss etc.

 

Servers never went down and Steadfast explained everything on WHT.

http://www.webhostingtalk.com/showthread.php?p=4596752#post4596752

 

Colocrossing is a good provider, but no reason to trash steadfast who has been nothing but professional to us.

[jbiloh]

We didn't see any abnormal AT&T traffic tonight.

 

From what I've read seems Steadfast's xconnects to AT&T we're pegged due to DDOS traffic. That's not an internal AT&T issue.

[Defcon|Rich]

We didn't see any abnormal AT&T traffic tonight.

 

From what I've read seems Steadfast's xconnects to AT&T we're pegged due to DDOS traffic. That's not an internal AT&T issue.

 

 

You sure about that Jon?

 

 

Tracing route to unknown.ord.colocrossing.com [205.234.152.151]

over a maximum of 30 hops:

 

1 <1 ms <1 ms <1 ms 192.168.0.1

2 6 ms 8 ms 7 ms 64.185.160.1

3 9 ms 7 ms 8 ms 208-117-2-25.block5.gvtc.com [208.117.2.25]

4 6 ms 7 ms 7 ms 208-117-2-42.block5.gvtc.com [208.117.2.42]

5 17 ms 16 ms 16 ms 12.86.9.45

6 46 ms 48 ms 48 ms tbr1.hs1tx.ip.att.net [12.122.103.10]

7 43 ms 44 ms 44 ms tbr2.dlstx.ip.att.net [12.122.10.9]

8 42 ms 44 ms 44 ms tbr1.sl9mo.ip.att.net [12.122.10.89]

9 47 ms 48 ms 48 ms tbr2.sl9mo.ip.att.net [12.122.9.142]

10 46 ms 44 ms 44 ms tbr2.cgcil.ip.att.net [12.122.10.45]

11 42 ms 44 ms 44 ms gar1.chail.ip.att.net [12.123.4.65]

12 179 ms 173 ms 181 ms att-gw.chicagoequinix.velocity-servers.net [12.1

19.137.50]

13 175 ms 173 ms 185 ms unknown.ord.colocrossing.com [205.234.152.151]

 

Trace complete.

[jbiloh]

Rich,

 

Thats a return path issue between us and your ISP.

 

We operated normally all night.

 

Tracing route to 208-117-2-25.block5.gvtc.com [208.117.2.25]
over a maximum of 30 hops:

 1     1 ms    <1 ms     1 ms  ge-1-45.cr.ChicagoEquinix.colocrossing.com [66.225.194.1]
 2     1 ms    <1 ms    <1 ms  att-gw.chicagoequinix.velocity-servers.net [12.119.137.49]
 3    33 ms    34 ms    33 ms  tbr2.cgcil.ip.att.net [12.123.4.66]
 4    31 ms    33 ms    32 ms  tbr2.sl9mo.ip.att.net [12.122.10.46]
 5    31 ms    31 ms    31 ms  tbr1.sl9mo.ip.att.net [12.122.9.141]
 6    31 ms    31 ms    31 ms  tbr2.dlstx.ip.att.net [12.122.10.90]
 7    34 ms    34 ms    34 ms  tbr1.hs1tx.ip.att.net [12.122.10.10]
 8    33 ms    33 ms    33 ms  gar6.hs1tx.ip.att.net [12.122.103.9]
 9    39 ms    39 ms    39 ms  12.86.9.46
10    39 ms    39 ms    39 ms  208-117-2-25.block5.gvtc.com [208.117.2.25]

Trace complete.

[Defcon|Rich]

That's a trace from one of the guys we sent over to you. They were having issues through att as you can see by the trace.

[adamnp]

I experianced no issues with the Jon's network last night, all our public servers remained pegged at 50 users all night long, you can check the graphs here..

 

http://www.game-monitor.com/GameServer/66.225.195.179:27015/

 

Also had no complaints, about it aswell. I would imagine it was just that users issue...Oddly enough actually at the time it pooped, we were actually in the SF vent, and the CC server. The vent dropped and we started talking ingame, maintaining normal server pings.

 

I'm not an IP Engineer though, this is why I was hunting for info..

[Defcon|Rich]

The issue seemed to show up in the same time frame as SF's problems which is why I mentioned it. Could be a number of things causing the bad hops.

[Jdubz31]

None of our monitors detected abnormalities on our AT&T Circuit. If the issues are still occurring, please open a ticket.

[Daersun]

I disagree that a 1.5gb ddos shouldnt take down connection.

 

I personaly think up to the distribution layer it should be ok. Unless your using a 6500 with 6x48 100meg ports for your layer3 your pretty much screwed.

 

Most people dont run 10gbit to layer3

[Monk]

a 1.5gb ddos can take down anything. Most attacks can generate 1000mbit but 500,000pps (1500 * pps = bandwidth) which will kill anything that is limited to a specific PPS, for example, a 7201 can only handle 2M/pps before it runs out of CPU.

 

When i worked at abovenet back in '97 IAD4 got hit by a 100mbit synflood and it was a 12016 GSR, it died.