Beyond.ca - how much data?

[bituerbo]

Hey Rage2 (or anyone else 'in the know').

How much transfer do you see monthly from this site?

Sincerely,

Concerned.

[em2ab]

I'm interested as well. I'd guess close to 100 gigs.

[bituerbo]

Originally posted by em2ab
I'm interested as well. I'd guess close to 100 gigs.
I'd HIGHLY suspect it's in the TB range, but waiting for official input!

[benyl]

It is mostly text. I can't see it being in the TB range at all.

[Eleanor]

Originally posted by benyl
It is mostly text. I can't see it being in the TB range at all. JB & post a hot pic thread? :D

[kenny]

Originally posted by benyl
It is mostly text. I can't see it being in the TB range at all.

There are a lot of attachments on the forums.

Back in 2003 we were moving about 70 - 75GB / month... I'm curious what we're moving today. We have made numerous changes to the forums to reduce our bandwidth usage.

[Mibz]

Originally posted by Eleanor
JB & post a hot pic thread? :D Those don't count towards Beyond's transfer since they're hosted elsewhere. Beyond does host avatars though, which, I imagine, would have a significant impact if they're not resized before being stored.

[rage2]

We average 18GB/day. Yesterday was a little above average at 18.452GB.

We run full HTTP compression, we've streamlined as much text/code as we can on all pages, but ya, avatars eat up a lot of bandwidth.

There's size restrictions for avatars, and they're automatically disabled under heavy load (it's the biggest CPU killer on the DB side).

Our biggest day in the last year was 132.040GB, I believe that was the day of the claw.

[EK 2.0]

Originally posted by rage2

We average 18GB/day. Yesterday was a little above average at 18.452GB.



I know nothing about this sort of stuff...but that seems awfully low....haha...

I was thinking a number like 100 or something...

[Kloubek]

Originally posted by rage2

Our biggest day in the last year was 132.040GB, I believe that was the day of the claw.

"The Day of The Claw". Awesome.

Perhaps we should just shorten it to "Claw Day". I don't know about anyone else, but I'm willing to push for it to be a stat.

[Eleanor]

Originally posted by Mibz
Those don't count towards Beyond's transfer since they're hosted elsewhere. Beyond does host avatars though, which, I imagine, would have a significant impact if they're not resized before being stored. Shows you what I know about the internetz :rofl:

[benyl]

Originally posted by Eleanor
Shows you what I know about the internetz :rofl:

That is why so many sites don't allow hot linking or save their photos without a .jpg, .png extension. They are trying to stop people from stealing their bandwidth.

[em2ab]

Originally posted by EK 2.0




I know nothing about this sort of stuff...but that seems awfully low....haha...

I was thinking a number like 100 or something...

It's closer to 20,000 megabytes if that makes you feel better! :)

[adam c]

18G per day?

That seems pretty high, aren't most images cached in everyone's machines and with most content being text...

I dunno?

[EK 2.0]

Originally posted by em2ab

It's closer to 20,000 megabytes if that makes you feel better! :)



Actually yeah...it sort of does haha...

But I equate the usage to the storage in my 160GB iPod haha....how I measure anything computer storage related haha...

And 18BG doesn't seem like a lot....I mean for all that beyond does in a day...haha...

[em2ab]

Originally posted by EK 2.0




Actually yeah...it sort of does haha...

But I equate the usage to the storage in my 160GB iPod haha....how I measure anything computer storage related haha...

And 18BG doesn't seem like a lot....I mean for all that beyond does in a day...haha...

vBulletin which is the software Beyond runs on is very good at caching and data management. It builds all the pages dynamically so all it transfers out from the server is each post from a database, the logo and a few background images. The blue line across the top of each box is even built dynamically, it's actually the following blue image stretched across the page. So it only sends this small image, then changes the size on display:

http://forums.beyond.ca/i/bg.gif

Each page is dynamically built and transfers very little data.

[Mibz]

Originally posted by em2ab


vBulletin which is the software Beyond runs on is very good at caching and data management. It builds all the pages dynamically so all it transfers out from the server is each post from a database, the logo and a few background images. The blue line across the top of each box is even built dynamically, it's actually the following blue image stretched across the page. So it only sends this small image, then changes the size on display:

http://forums.beyond.ca/i/bg.gif

Each page is dynamically built and transfers very little data. I didn't know that. That's fucking cool.

[GQBalla]

ahah i remember when the avatars shut down when the claw thread was going about

[em2ab]

If you really want to hammer the server, set your browser to not cache images and always pull new content from the server.....:nut:

[Elaenor]

Originally posted by rage2
We average 18GB/day. Yesterday was a little above average at 18.452GB.

We run full HTTP compression, we've streamlined as much text/code as we can on all pages, but ya, avatars eat up a lot of bandwidth.

There's size restrictions for avatars, and they're automatically disabled under heavy load (it's the biggest CPU killer on the DB side).

Our biggest day in the last year was 132.040GB, I believe that was the day of the claw.

Cool! Thanks for the info!

[Mibz]

Originally posted by em2ab
If you really want to hammer the server, set your browser to not cache images and always pull new content from the server.....:nut: And run the original Fasterfox with Prefetch enabled. Hahaha, it'll be a new DDOS:
"Hey man, install this extension, it makes your shit go way faster. Oh yeah, turn it up to prefetch 50 pages at a time. Sweet, yeah, now disable caching. Sweet, okay, now at 4:45 go to this website for me, k?"
Rinse and repeat.

[chkolny541]

^^lol, just to be dicks

[rage2]

Originally posted by Mibz
And run the original Fasterfox with Prefetch enabled. Hahaha, it'll be a new DDOS:
"Hey man, install this extension, it makes your shit go way faster. Oh yeah, turn it up to prefetch 50 pages at a time. Sweet, yeah, now disable caching. Sweet, okay, now at 4:45 go to this website for me, k?"
Rinse and repeat.
We have standby servers that can easily handle the load. Like I said, once the load hits a pre-determined point, avatars gets disabled, and that opens up enough database CPU to serve more than 15 times the normal volume (the DB is basically our non scalable bottleneck at this point). If I upgrade the DB server, we can probably double or triple that amount, but there's been no need.

Bandwidth wise, I think we're alright...

http://www.speedtest.net/result/412493707.png

I think that might've been on a bad night :).

[GQBalla]

dannggg

bawlin - ^%^^

[UndrgroundRider]

Originally posted by rage2

We have standby servers that can easily handle the load. Like I said, once the load hits a pre-determined point, avatars gets disabled, and that opens up enough database CPU to serve more than 15 times the normal volume (the DB is basically our non scalable bottleneck at this point). If I upgrade the DB server, we can probably double or triple that amount, but there's been no need.

Bandwidth wise, I think we're alright...

http://www.speedtest.net/result/412493707.png

I think that might've been on a bad night :).

I'm not a vB expert, but if the avatars require that much CPU utilization, why not store them as files instead? That's what most php galleries do, and then just store the index information in the db.

[rage2]

Originally posted by UndrgroundRider
I'm not a vB expert, but if the avatars require that much CPU utilization, why not store them as files instead? That's what most php galleries do, and then just store the index information in the db.
That's actually what vb3 does. I started to convert vb2 to use a file based system, but I got lazy, and just implemented the avatar disable on high load instead which works just as well. We've only seen it get disabled 2 or 3 times last year (again, around the claw days), so it works well enough for now.

[911fever]

When are you seeing peak hours for beyond data use? Night time between 6-9pm?