Peak traffic

Written By: - Date published: 9:31 pm, November 10th, 2008 - 16 comments
Categories: admin - Tags: , , ,

Sysop

Sysop

Some comments on Sunday about this site performing well induced me to have a look at the traffic over the last week. I was extremely surprised at how much load the system can now take without stalling. From the WordPress blog stats, It turns out that the traffic for the week was 34.6% higher in page views compared to the week before. It was a staggering 61.9% higher than it was two weeks before. It makes me more confident about running the site at a technical level towards the 2011 (or earlier) election.

I’d love to take credit for this (and I will in part). However the reason people read the site is largely because of the content, both from our posters and our knowledgeable and informed commentators*. This is a completely voluntary site run on a very loose cooperative arrangement – it just shows what kind of talent is there to be tapped from the grass-roots. A special mention for Julie Fairey from The Hand Mirror for her election evening coverage who kept everyone coming back for more. Despite being offline for comments for 19 hours, it is still in our top ten days for traffic.

However that is just the visible portion of the site. We’re reliant on a number of technical supports.

Webfarm also deserves a strong mention. Their systems have been utterly reliable since I moved the site there in Feburary. Currently the site runs on virtual private server with 512MB of RAM running Fedora Linux with unlimited local traffic costing $159+GST per month.

However, my biggest kudo’s has to go to the open source people at WordPress, Fedora project, K2 theme and the wordpress plugin authors without whom this system would have failed dramatically under the load. The site now runs without any code kludges apart from some css styles for the display. It is quite a different site from what was running last year and failing under load.

* Well almost all commentators – I specifically exclude the annoying, largely incoherent, and definitely anti-social trolls. To a man, they seem to mainly be act supporters** hyped on their badly self-assessed intelligence, in love with their own egos, and who take the credo of individualism to the point of being incapable of meaningful social interaction with others. It is the one area that I pity John Key in – he is going to have to deal with adult versions of these clowns. He also doesn’t have the “send to spam” button. However he has done well with an early ban.

** But not all act supporters are trolls

16 comments on “Peak traffic ”

  • Comments are now closed

  1. Pascal's bookie 1
    10 November 2008 at 9:59 pm

    “He also doesn’t have the “send to spam’ button. However he has done well with an early ban.”

    lolnui.

  2. mickysavage 2
    10 November 2008 at 10:03 pm

    Go Lprent

    This is the first site that I view in the morning. It will become more and more important in the future as a number of campaigns heat up.

    The goal will be to elect a progressive administration next term. There is amongst parts of the community a euphoria about the Key administration but it appears that they do not understand that Obama is a leftie progressive and Key is a right wing troll posing as a “Labour lite” Helen.

    Over the next few months the reality will start to become obvious and support will flood back.

    I am sad to see Helen go. If she hung around I am sure that many people would work to have her returned as PM.

  3. Lew 3
    10 November 2008 at 10:11 pm

    mickysavage: Obama is only a `leftie progressive’ in the context of two-party post-Reagan USian politics. A close examination of policies will reveal that he’s about as far left as National – maybe not quite so far. I grant, however, that ideas of `left’ and `right’ are blunt objects with which to beat a candidate into shape more than they are precision tools with which to measure them. And the whole affair is skewed by a statist Bush Jr administration which pissed off (US types of) liberals, conservatives and libertarians alike.

    L

  4. Mike Collins 4
    10 November 2008 at 11:01 pm

    I too thought Julie’s coverage was good. I was reading it the following day. Must have been tough to do bearing in mind the results were probably not to her liking.

  5. burt 5
    10 November 2008 at 11:41 pm

    lprent

    Great post. Most excellent analysis. You certainly have nailed the issue.

  6. lprent 6
    11 November 2008 at 2:51 am

    MC: I’d definitely say that Julie wouldn’t have liked the results. As you say, it made the effort all the more difficult. She would have also had to contend with posting on a site that was definitely under strain. My logs show close to a 0% idle system for much of the night.

    burt: Have you been reading my spam queue? There is a particular type of spambot that shows up with almost those exact words. However they usually have a link attached pointing to something like “MILF anal” (I don’t even want to know what that is) or “online blackjack” or “insurancesitesearch”. There are alternatives in the wording – usually something like “Excellent post. I might not agree fully with what you say”

    Oh well. Got up to send the better half off to a sinking island in PNG.

  7. Peter J 7
    11 November 2008 at 9:37 am

    Very interesting, thanks for the insight. I know what it is like having to move hosting providers, have had to do it a couple of times due to reliability issues. The current provider I am using is awesome, only costs me $29.95 a month too, and has taken a few hammerings over the years.

    I too work for an entirely voluntary organisation, and fund all the website related costs out of my own pocket, just as other key members fund things like advertising, printing etc out of their own pockets, and also put in a lot of unpaid time on top of a paid job.

    I dont have the headache of having to manage forums and comments as you do however. You have my sympathies on that one!

    Regards
    Peter Jenkins
    webmaster for Sensible Sentencing Trust

  8. Tane 8
    11 November 2008 at 9:41 am

    Lynn, you forgot all the granny porn links. Honestly, what is it with those?

  9. John Stevens 9
    11 November 2008 at 9:52 am

    Where is your post on JK being the most expensive limo user in the past year? Getting a bit slack boys.
    http://www.stuff.co.nz/vote08/4756615a28435.html

    ‘John Key, who pinned his campaign on slashing bureaucratic excess, spent more on chauffeur-driven limousines than any politician – including the prime minister – last year.’

  10. Killinginthenameof 10
    11 November 2008 at 10:00 am

    He’s only used them somewhere between 5% and 10% more than Helen Clarke, that’s not exactly a big deal in my books. Still stoked on those BMW’s though, i’m very glad they chose them, nice cars indeed.

  11. Killinginthenameof 11
    11 November 2008 at 10:01 am

    Bugger!! sorry previous comment in moderation!! Miss spelled Helen Clark;s name too, brain freeze moment! (im up way to early!)

  12. P.I. 12
    11 November 2008 at 10:44 am

    [Tane: And that’s you banned…. hmm, appears you’ve already been permanently banned under a different name.]

    [lprent: some people never learn]

  13. lprent 13
    11 November 2008 at 12:48 pm

    granny porn

    Yeah, there is a some very strange spam in that queue…..

  14. lprent 14
    11 November 2008 at 12:57 pm

    Peter J:

    he current provider I am using is awesome, only costs me $29.95 a month too, and has taken a few hammerings over the years.

    I looked at a few like that (mainly offshore). All our traffic is local so an offshore solution wasn’t all that enticing. It just added 200 msec latency.

    But the real problem is that it is very hard to run a database server the size of this one with appropriate off site backups and all of the site tuning without having console access. We’re getting literally thousands of comments per week, and a considerable number of posts per day. The lurker population is immense – over 50x the commentator population.

    What is astonishing is that the traffic doesn’t appear to ever fall off. It goes up, plateaus, and a few weeks later goes up again. I thought it’d fall post-election and so far it hasn’t by much.

  15. Julie Fairey 15
    13 November 2008 at 10:04 am

    Thanks lprent, I enjoyed it. No I didn’t like the results much, but then I’m pretty used to that, from my years in student politics and then in the Alliance party!

    Thanks for the opportunity 🙂

    [lprent: I’ll leave the login in place. If you get something that doesn’t fit on The Hand Mirror, but will here then feel free to post. (Alliance! wanders away shaking his head)…]

Links to post

The server will be getting hardware changes this evening starting at 10pm NZDT.
The site will be off line for some hours.