Crapfast Network Issue

[JasonF]

Steadfast is having network issues today. Probably another Dos attack their network can not handle.

[Daersun]

First off, I am having no problems.

(from ATLANTA)

Ping statistics for 208.100.37.1:

Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),

Approximate round trip times in milli-seconds:

Minimum = 30ms, Maximum = 31ms, Average = 30ms

 

Ok,

Network lesson time...

If a DDOSS comes in though 1 provider - and it is bigger then the pipe that provider is on, anyone coming into that pipe (whos ISP routes into that pipe) is going to be affected. Normally this doesn't happen, and it splits it between multiple pipes. Steadfast uses 6506's with sup 720 3bxls, quite honestly the ddos isnt going to affect its core and max out the CPUs.

Their Aggregation layer is not going to get hit either. When it drops down to the layer3 or layer2 switch your connected to, thats only a 100mbit or MAX gbit uplink to the L3/distribution switch your on. THAT switch is going to get owned. absolutely demolished.

 

Funny thing is... With a 2gbit + ddos, there is not a single provider out there, (bar steadfasts new seatle DC) that runs 10gbit to the rack, thus there is not a single provider out there who can handle a ddos of that size.

 

DDOS protection works well - it has about a 30 second delay before it kicks in, and in a large DDOS 4-10gbit, your going to start to see some lines maxed out.

 

Funny thing is the only ddos I have EVER been hit with @ steadfast is when ATT had their pipe maxed and because they do not allowed community blackhole, steadfast couldnt nullroute the IP and had to depeer. That dropped about 5% of my traffic, other then that, everything has been solid for years and WELL above 5 nines of uptime.

 

How to make things better?

Unless steadfast's network performance is worse on the dedicated side (which my understanding is your on) I think your problems are a case of seeing what you want to see because ddoses are a normal part of the internet that no one can avoid - NO ONE.

[JasonF]

Ping times to one of my Steadfast boxes was >300ms with packet loss. I called and they confirmed a major network issue.

 

A few months ago they had a Dos attack and caused all of my boxes to be inaccessible by myself and my clients.

[Daersun]

Maybe its time to upgrade to colo then Because obviously this side is MUCH more stable

[JasonF]

I don't use Steadfast anymore. I have been using Equinix via Colocrossing for all of my Chicago servers after learning what a bust Steadfast was in both pricing and network. I have 2 boxes at Steadfast still and can't dump them due to IP change issues.

[Daersun]

Gotta say bud, I have alot more boxes there for alot longer with almost no issues - sounds like user error ;P

[JasonF]

Gotta say bud, I have alot more boxes there for alot longer with almost no issues - sounds like user error ;P

 

I guess steadfast should hire you since you are capable of resolving issues beyond your control.

[KarlZimm]

Yes, there was a major DDoS this morning, multiple Gigabit and caused some carrier based congestion. Our DDoS filtering was blocking it, but that doesn't do much good when it is congesting the line into our network. As was said above, that is a rare occurrence and will happen to everyone at some point. If Colocrossing had multiple Gbit/sec sent over their AT&T line you would see the same issues, at best. The benefit of our DDoS protection is that the attack will not completely congest our access switch, which would basically make all systems on that switch inaccessable.

 

If you do have the solution for preventing DDoS attacks though I would be more than happy to hear it.

 

As a note, the affects were not felt by all customers and the total time service would have been affected is less than half an hour. That was also at around 10AM on a weekend, one of our slowest times.

[Jissh]

LOL was wondering if Karl was going to see that post. Personally I have no problems with Steadfast, only a couple small outages in just over 2 years. Really we're quite happy with them and the service we get.

[Defcon|Rich]

Every provider WILL have issues from time to time. Nobody is immune, that's a fact. Trouble is when you reach a certain level and host xx amount of servers your margin of error greatly increases on the odds something will come up.. Being popular is a bitch sometimes

 

Honestly speaking when we hosted servers with Karl I don't recall having any problems with the network or hardware over the course of 3-4 years we were there so I would say it's a little premature to start throwing rocks

[JasonF]

so I would say it's a little premature to start throwing rocks

 

I have servers at Equinix, Softlayer, The Planet, Voxel and Steadfast. The only issues I have had with dos attacks affecting service to my servers has been at Steadfast. This has happened 2 times at Steadfast and I have only been there 7 months. Add the fact that Steadfast is the most expensive out of those listed and it it makes perfect sense and is everything but premature.

 

Also, since this is Karl's first post. I am sure he was summoned here by a forum member.

[adamnp]

Id have to agree with Jordan in the fact that no host is able to be completely freefrom DDoS...

Karl/Gary might remember, ba.best.com , and some other /famous sites that used to tote DDoS Free, but even the best hosts can be dropped further up the line, theres always a way to attack it and you'd be surprised at how determined people can get.

 

Definately have to understand that its going to happen once in awhile.

 

I've had issues in the past with *all* the providers I currently host with....That includes pretty much all of the ones listed in this thread..It's just going to happen once in awhile

 

-Adam

[JasonF]

I will attribute the network issues to bad luck I guess. In all fairness, Steadfast's support has been excellent but have been unlucky with network issues.

[Defcon|Rich]

 

Definately have to understand that its going to happen once in awhile.

 

I've had issues in the past with *all* the providers I currently host with....That includes pretty much all of the ones listed in this thread..It's just going to happen once in awhile

 

-Adam

 

 

Like you said, It's inevitable. The most important thing is how your provider handles the issues when they come up. That's what sets everyone apart.

[Daersun]

A provider willing to come into forums like this to clear up some questions gets kudos in my book

[Defcon|Rich]

A provider willing to come into forums like this to clear up some questions gets kudos in my book

 

 

It's pretty obvious he was tipped off about this thread and came over to do some damage control so let's not put the horse before the cart here.. It's not a bad thing but I don't think a slap on the back or an "atta boy" is quite in order

[JasonF]

Thanks Rich for stating that. I have been reluctant to add anymore to this thread to avoid adding anymore mud to the forums. If anyone would like to speak to me anymore about this topic please message me off the board and I will state my reasons. I would also be happy to include Karl in the email to give him a chance to respond.