web analytics

Site health – comments and posts

Written By: - Date published: 4:31 pm, February 12th, 2012 - 36 comments
Categories: admin, The Standard - Tags:

vto made a comment that he thought that the number of comments was diminishing at kiwiblog. Now I have no idea because I usually avoid the sewer that is the comments section there. However it did remind me that I should have a look at the last year or so here. It feels like it has been getting more popular, but intuition is a poor substitute for actually looking at the numbers.

So I ran some SQL against the comments table pulling out the numbers of approved comments and the number of posts with approved comments*, tossed it into LibreOffice Calc and pulled out some numbers.

So looking at the yearly picture for full years, our comments are rising in the last few years and our posts remain pretty constant, so our comments per post has been rising.

The Standard – yearly comments and posts 
Year Comments Posts Comments per Post
2008 64218 2199 29.20
2009 64594 2687 24.04
2010 94558 2888 32.74
2011 131249 2595 50.58

We have been keeping the numbers of posts limited because we’re not really that interested in burning out authors. We all have lives outside of the blogs, don’t want to spend our lives writing posts, and don’t really want to become some kind of news clipping service. You can see the election year effect in 2008 and 2011.

The question for this site in 2012 is if we keep similar numbers of the readers and commentators, and their page views and comments post election year.

You can see the post election slump in December 2008 and Jan 2009 had a virtual reprise of the December 2010 and 2012, albeit from a much higher base and for much the same reasons. The effects of the very short election season in 2011 shows in our comments which actually decreased while the world cup was on in September and the first part of October and then went through the roof after the final game.

But there has been healthy growth throughout the year and the numbers for the latter part of Jan and early Feb point to the comments on the site remaining pretty healthy, but unlikely to repeat the November numbers for a while. Probably in the post-winter commenting boosts. But of course commentators don’t just come to argue with each other. They come for the posts as well.

And then we look at the numbers of posts. There you can see the usual post election slump adding to the usual reduction in authors writing. But overall we maintaining the numbers of posts with spiking going on in times of interest. However what we are seeing is a change upwards in the number of comments per post.

 The spike in later 2007 and early 2008 shows exactly what happens when there is no moderation. You get a considerable increase in trash commenting as the trolls pile in. The average size of the comments goes down (if I get time I’ll put a post up on that) and you notice the lousy commentators drowning out anyone that looks interesting – the sewer provides a good example. When the moderation kicks in and becomes effective the number of comments slumps but the size of the comments, their quality increases, and you see interesting commentators return.

But since early 2010 we have had a steady increase both in the number of comments and crucially in the number of commentators.  We’re also seeing that interesting effect that even when the authors are taking their hard earned rest and the numbers of posts decrease, the commnets carry on as people make their own fun. The threaded nature of the comments sections makes it feasible to do.

Not everyone is particularly happy with the style of moderation on this site or even the style of the site. But it works and I’m more than happy with what we’re doing together.

* Code:

YEAR(comment_date) as year,
MONTH(comment_date) as month,
COUNT(*) as comments,
COUNT(DISTINCT comment_post_id) as posts
FROM wp_comments
WHERE comment_approved=1
GROUP BY YEAR(comment_date), MONTH(comment_date)
ORDER BY YEAR(comment_date), MONTH(comment_date)

This particular query eliminates the small number of posts that have no comments (mostly from 2007) and includes the few image pages that have comments that were never moved.  It is close enough for my purposes.


36 comments on “Site health – comments and posts”

  1. G 1

    Whle you are talking about site health… can you please restore the RSS feed to the entire post? For some of us, work tracks what we look at, but they are too dumb (to be fair, their programme is too dumb) to realise that reader.google.com is Google Reader and not Google Search (which could be work-related). So I read blogs through RSS feeds only.

    • Lanthanide 1.1

      There was a problem where posts that were edited/updated did not have those edits reflected in the RSS feed. Sometimes posts are put up with chunks missing or error that are later corrected, but people reading the RSS feed didn’t see the updates.

      I think turning the RSS off was the short-term workaround for this problem.

    • lprent 1.2

      Plus the overseas traffic running on the RSS feed. The spam bots have been reading the RSS feed to look for new posts, which they then try to spam. Since there are literally thousands of the little buggers and I have no easy way to determine that they are spam there is no easy way to discriminate. I wish I could get at google analytics spambot discriminators. I’d use them on the reader and RSS reader side.

      The RSS feed got dropped from full to excerpt on about the 10th of last month. Since then we have dropped a lot of overseas traffic Gb compared to December. That costs $3 + GST per Gb.

      Of course the actual solution around NZ’s crazy costs for overseas traffic (when are we getting some competition for the southern cross?) is to bounce an up to date replication to a US server and let the rest of the world read their 45GB/month from there while NZ read their local ~300GB/month per month of it. However it costs about 50GB of overseas traffic to maintain a replication on a site that is this active.

      Of course I could fix everything by moving everything back to a US server. Of course it I did that then the actual traffic over the southern would increase to the full 300GB and the response times would slow.

      I have been playing with a astrill VPN setup at present. Hopefully that will push the replication costs elsewhere out of my budget. But I am finding it hard to get it to restart the connections reliably when I put in failures.

      Next one I am going to try is to change the RSS feeder to discriminate between known ‘good’ aggregation sites (like Google reader and feedburner) and everyone else. To be precise only allow a couple of feeders to use the RSS feeds.

      • infused 1.2.1

        Use a better provider. DTS don;t charge for outbound, which is what most of your traffic is.

        • lprent

          If it is the provider I just looked at then they don’t do dedicated servers. They do have collocation rackspace, but in Wellington and Christchurch. But I really don’t have time to chase around with hardware anyway. Their GB rates for the racks look ruinous for our volumes. http://dts.net.nz/hosting/co-location/ $6 + GST per 5Gb per month. We move at least 300GB per month.

          Unless there is something I am not seeing?

  2. Gruntie 3

    I mostly read the standard on my iphone 4s – (no I’m not a rich prick) but find The Standard blog the most buggy if all blogs – very slow and crashes a lot – have given up writing comments lately ( also coincided with my depression following 26 Nov 2011)
    Does anyone else have same issue with iOS on iPhone?

    • McFlock 3.1

      I got a similarissue on my android – but then it was the only blog I looked at that 100+ comments in a thread, so figured it was just that.
      Bloody stupid trying to go through all those with a 4″ screen 🙂

    • lprent 3.2

      I will have a look at it. We’re using the wpPhone ‘theme’ for phones. I have to confess that I haven’t tested it for a while.

      I should probably look at finding or even writing a barebones themes for the smart phones. Most of the ones I’ve looked at appear to be too sluggish when there are large numbers of comments with our 3G data. They need to paginate them.

    • Trowlie 3.3

      I’m the same on my iPhone. The page takes an age to load and will then crash as soon as I start to scroll down.

    • SHG 3.4

      Agreed – The Standard is a dog with the mobile theme enabled.

      • lprent 3.4.1

        Looking around for another theme. Tried it out on the my old iphone 3G yesterday and it was sluggish on wifi! It was interesting because the theme wasn’t nearly as bad last time I used it.

  3. Congrats Lprent.  Good to see the Standard is going from strength to strength.

    One thing I am interested in.  Did the 2011 post election dip occur before or after the Labour leadership campaign?  My impression is that the campaign caused considerable interest but it may be that it was amongst the Labour members rather than your readership as a whole.

    • lprent 4.1

      After. We had a hell of a spike for the leadership campaign. The December figures look like they dropped a lot, but the reality is that they are higher than almost every month.

      When it hit Dec 25 the page views dropped to about 5k per day and stayed there until the Ports of Auckland dispute.resurfaced in Jan. Comments had the same effect.

  4. Tangled up in blue 5

    vto made a comment that he thought that the number of comments was diminishing at kiwiblog.

    Imo “lefties” should stay away from Kiwiblog completely. I mean don’t even go there for a laugh to see what those clowns are up to. Giving it traffic is unnecessary and indirectly supports the site. It used to be that a left voice on there was helpful to show up the usual biased tripe but lately (since about a month before last election) it’s gotten so hard-right and hateful that people are realising it’s just a place for angry 1% Act supporters to rage. Stay away and let Kiwiblog fade into obscurity.

    • Populuxe1 5.1

      Really? Keep your friends close and your enemies closer. I like to know what the bastards are saying, otherwise you’re just reinforcing the stereotype that the Left lives in an ivory tower.

      • McFlock 5.1.1

        Much as I tend to agree with the sentiment, if they’re saying anything much more tory than kk, james3 or so on I probably wouldn’t want to burn my eyeballs.

  5. Anne 6

    Imo “lefties” should stay away from Kiwiblog completely.

    Absolutely. Don’t give Farrablog the traffic. He holds it up as evidence of his supposed political insight and importance. The MSM are too thick – or lazy – to investigate the reality.

  6. Blue 7

    @Anne – “Imo “lefties” should stay away from Kiwiblog completely.’,Absolutely”

    Ah yes, ‘we can’t debate, so lets run away’, Why am I not surprised that you’re not prepared to listen to an opposing view. Ever wonder why the National Party handed you your asses again? Its this sort of head in the sand thinking that costs you your support. Mindless stuff.

    • Colonial Viper 7.1

      Why should anyone be prepared to listen to shite? You know like how selling off our country is good for us, how kicking the poor a bit harder will be “motivational” or how SOEs don’t need to respect Treaty obligations after being sold off.

      Ever wonder why the National Party handed you your asses again?

      You must be talking about the National led coalition which came in 4 seats down on 2008.

    • McFlock 7.2

      “Opposing view” is one thing.
      Calling people “parasites” is another thing entirely.
      One will never persuade people who freely use that (and similar language) to recognise that society extends beyond their personal chequebook, and their pretensions to logic are futile at best. “Debating” with them on their territory lends them a legitimacy they don’t deserve.
      Let KB become a mad corner of the internet, the NZ equivalent of militias and supremacists. After a while even the corporate media will have to admit that “newsworthy” story needs to be based on something more than the fanciful masturbations of a national party tool.

    • Gosman 7.3

      Interestingly I’d hazzard a guess that the posts with the mosts comments are the ones that have the most right wing comments on them. So to make a big deal about this stat you also have to acknowedge that opposing comment viewpoints make the blog more popular. You just need to lookat what has happened to Tumeke since Martyn Bradbury elbowed aside Tim Selwyn and implemented his Stalinist moderation policy. I believe the blog averages in the low single digits now per post.

      • Draco T Bastard 7.3.1

        And here’s me thinking that it was Tumeke’s layout and general user unfriendliness that did it.

        • Gosman

          How has that changed over the past couple of years?

        • just saying

          I’ve tried to comment at Tumeke and been unable to navigate the system.

          While it’s true posts that generate large numbers of comments are often those that generate debate and or argument (often over several days) it is just as likely (if not more) to be between differing views within the left. I think the overwhelming majority of regular commenters here come here to read the posts, follow some links, and exchange ideas with other left-wingers, and even those willing to engage the regular trolls do so mostly as a side-sport.

  7. Rich 9

    I actually think you can have too many comments (though maybe not from a revenue point of view).

    I tend to have this site down my list because of the number of Tories that comment on it (along with the Grauniad). If I wanted to know what RWS think, I’d read the mass media or listen to talkback.

    If there was a Tory-free politics site with reasonable traffic, I’d read it a lot.

    • lprent 9.1

      Yeah, well there is always that. But in their defence, the RWS that survive here tend to be somewhat “smarter” than those who frequent the mass media or talkback or the sewer. I feel it is healthy for the left to engage with the right, partially for the cross-fertilisation, but also because left activists tend to start arguing over minutiae of dogma that few voters care about if they are only talking to the like minded.

      The filter mechanism is simple. We are interested in people who can and will argue. We tend to turf the commentators that don’t appear to have a brain and simply repeat statements over and over again like some kind of dumbarse parrot – in other words the talkback crowd. Cameron Slater is pretty good example of the genre.

      In the usual link between stupidity, short-term thinking, arrogance, and conservative, the bulk of the right get winnowed here because they simply can’t help themselves. They drop into repetitive or troll behaviours, attack the authors, or pick up one of the other site’s Darwin awards that we will moderate for and ban on.

      There are some on the left with the similar dogmatic ways, but I generally find them either indignantly dropping off the site or changing their behaviour when they get a warning.

      Obviously I like the balance. It covers a pretty wide range of the political spectrum, it is pretty well limited to people who can and do argue mostly without too much rancour and aren’t too thin-skinned. And with the exception of the prohibitions about our losing authors allows a lot of freedom in what can be commented about.

      Tory-free politics site with reasonable traffic

      I have seen a few over the years – there were several interesting ones in the usenet days and you can find some quite doctrinaire ones in various places. I haven’t been that impressed. The further they seem to get from having to actually win votes in general elections, the less rational they seem to be.

      In my opinion their comments sections usually look distinctly like the bullying behaviour in the sewer. Fortunately they seldom seem to last that long as being busy with comments either.

    • lprent 9.2

      Oh and comments don’t seem to have that much of a correlation with page views, unique visitors or what data I have on clickthru. Those are what attracts advertisers and what the site gets paid for on the advertising.

      The best overall correlation for those things is with what is happening in the political world and what is not getting reported. We get enormous numbers of hits when the media start missing a story or trying to spin it. The Ports of Auckland dispute is the most recent example (26k page views in a day), with the Crafar farms one less so. I’ve been intrigued with this for a while because it tends to indicate that people bullshit meters go off the scale then and they come and see what we are saying.

  8. Same issue. Goes crazy once the comments get to about 50.

Recent Comments

Recent Posts


  • A charge on plastic bags – debunking some myths
    The launch of my Members’ Bill last week, which would introduce a 15 cent charge on single-use plastic bags at the check-out, has generated a lot of comment on mainstream and social media. From The Paul Henry Show at the ...
    GreensBy Denise Roche
    10 hours ago
  • National’s $1trillion property sandcastle
    The National government's failure to fix the housing crisis has seen the ballooning and unsustainable property market touch the $1 trillion mark, Labour’s Finance spokesperson Grant Robertson says. "Labour wants an economy that creates high wage work that is based ...
    11 hours ago
  • Government failure on housing crisis drives Reserve Bank to add tools
    If the Government was delivering a comprehensive plan to fix the housing crisis, it is unlikely that the Reserve Bank would be continuing to pursue debt to income limits for lending for housing, says Labour’s Finance spokesperson Grant Robertson. ...
    1 day ago
  • International embarrassment for NZ likely over National’s failure to protect Maui dolphin
    New Zealanders who care about Maui dolphin should prepare to feel embarrassed: the Government is about to be put to shame on the international stage for its lack of action to protect Maui’s dolphin. The International Whaling Commissions’ 66th Biennial ...
    GreensBy Eugenie Sage
    4 days ago
  • Why don’t we spend $1b to keep people out of jail, rather than spending it on keeping them in?
    Earlier this week, Corrections Minister Judith Collins announced the government’s ‘solution’ to our burgeoning prison population. It seems that most, if not all, of Bill English’s hard-won surplus is going to disappear into another round of prison-building.  That must be ...
    GreensBy David Clendon
    5 days ago
  • PKE Ship Sent Packing – Not Too Soon
    It is appropriate that the palm kernel expeller (PKE) ship off Tauranga has been sent packing. For weeks I have been saying this ship needed to be sent away, but it seems as if MPI has been trying to find ...
    GreensBy Steffan Browning
    5 days ago
  • Do you #LoveSnow?
    I was a lucky kid. When I was about five or six my mum and auntie took me up to Whakapapa on Mt Ruapehu and taught me to ski. As a young kid I thought there was no bigger ...
    GreensBy James Shaw
    5 days ago
  • Awa Kairangi/Hutt River – Swimmable?
    On Thursday night I hosted a great swimmable rivers meeting organised by the local Greens in Heretaunga (Hutt Valley). It was great to see about 70 people attend and engage in the topic. We were welcomed by Te Atiawa representative ...
    GreensBy Catherine Delahunty
    5 days ago
  • September benefit figures disappointing
    The Government is out of touch with the reality that fewer people are going off the benefit and into employment or study, says Labour’s Social Development spokesperson Carmel Sepuloni.  “The quarterly benefit numbers for September are concerning. They show that ...
    5 days ago
  • MFAT officials refuse to back Prime Minister on Saudi sheep claims
    An Ombudsman’s interim decision released about the existence or otherwise of legal advice on the multimillion dollar Saudi sheep deal shows MFAT has failed to back up the Prime Minister’s claims on the matter, says Labour MP David Parker. “The ...
    6 days ago
  • Barry Coates on his first weeks in Parliament
    Week one in Parliament has been quite an occasion. I would like to share the experience. I had given up on the prospect of getting into Parliament before the election and had been enjoying the diverse work I was doing ...
    GreensBy Barry Coates
    6 days ago
  • Nats still planning to take Housing NZ dividend
    Housing New Zealand’s Statement of Performance Expectations shows that the National Government intends to pocket $237m from Housing New Zealand this year including a $54m “surplus distribution”, despite promises that dividends would stop, says Labour’s Housing spokesperson Phil Twyford. “After ...
    6 days ago
  • Parliament must restore democracy for Ecan
    Parliament has a chance to return full democracy to Canterbury with the drawing of a member’s bill that would replace the Government’s appointed commissioners with democratically elected councillors, says Labour’s Canterbury Spokesperson Megan Woods. “In 2010, the Government stripped Cantabrians ...
    7 days ago
  • Police struggle to hold the line in Northland
    Labour’s promise of a thousand extra police will go a long way to calming the fears of people in the North, says the MP for Te Tai Tokerau Kelvin Davis.  “Police are talking about the Northland towns of Kaitaia and ...
    7 days ago
  • Vote Sooty Shearwater/Tītī for Bird of the Year
    Sooty shearwater (Puffinus griseus) are amazing and deserve your vote in Forest and Bird’s Bird of the Year competition.  They make one of the longest known bird migrations, flying an annual round trip of 64,000 kms across the entire Pacific ...
    GreensBy Eugenie Sage
    7 days ago
  • Urgent action on agriculture emissions needed
    Immediate action is required to curb agricultural emissions is the loud and clear message from Climate change & agriculture: Understanding the biological greenhouse gases report released today by the Parliamentary Commissioner for the Environment, says Labour’s Climate Change spokesperson Megan ...
    1 week ago
  • Super Fund climate change approach a good start
    Labour Finance Spokesperson Grant Robertson and Climate Change Spokesperson Dr Megan Woods have welcomed the adoption of a climate change investment strategy by the New Zealand Super Fund. “This is a good start. It is a welcome development that the Super ...
    1 week ago
  • Energy use going in the wrong direction
    New data out this week from Statistics NZ paints a concerning picture of energy use across the economy under this National Government. You won’t be surprised to hear that there is some seriously worrying information here about how dirty our ...
    GreensBy Gareth Hughes
    1 week ago
  • Raising the age the right thing to do
    The announcement today that the Government will leave the door open for young people leaving state care still means there is a lot of work to do, says Labour's Spokesperson for Children, Jacinda Ardern "The Government indicated some time ago ...
    1 week ago
  • Junior Doctors go on Strike
    Thousands of junior doctors took strike action for 24 hours this week for better working conditions and safer working hours.  The Green Party supports their cause, and particularly their claims to reduce the number of days worked from up to ...
    GreensBy Denise Roche
    1 week ago
  • Coleman plays down the plight of junior doctors
    Junior doctors are crucial to our health services and the industrial action that continues tomorrow shows how desperately the Government has underfunded health, says Labour’s Health spokesperson Annette King.  “Jonathan Coleman’s claim that he has not seen objective evidence of ...
    1 week ago
  • Inflation piles pressure on National and Reserve Bank
    While many households will welcome the low inflation figures announced today, they highlight serious questions for both the National government and the Reserve Bank, Labour’s  Finance Spokesperson Grant Robertson said.  "While low inflation will be welcomed by many, the ...
    1 week ago
  • Officials warned Nat’s $1b infrastructure fund ineffective and rushed
    Treasury papers show the Government rushed out an infrastructure announcement officials told them risked making no significant difference to housing supply, says Labour’s housing spokesperson Phil Twyford.  “Like so much of National’s housing policy, this was another poll-driven PR initiative ...
    1 week ago
  • More cops needed to tackle P
    New Police statistics obtained in Written Questions show John Key is losing his War on P, highlighting the need for more Police, says Opposition Leader Andrew Little.  “New Zealanders expect serious action on P but today’s hodgepodge of half-measures won’t ...
    1 week ago
  • Strengthening our relationship with the Rātana movement
    It was a privilege to visit Rātana Pā last week with fellow Greens’ Co-leader James Shaw, our Māori Caucus and senior staff to meet with the leaders of te iwi mōrehu, to strengthen the ties between the Green Party and ...
    GreensBy Metiria Turei
    1 week ago
  • MBIE docs show country needs KiwiBuild, not Key’s pretend “building boom”
    John Key’s spin that New Zealand is in a building boom does not change the massive shortfall in building construction as new MBIE papers reveal, says Labour Party housing spokesperson Phil Twyford.  “We can fix the housing crisis, by the ...
    1 week ago
  • 1 in 7 Akl houses now going to big property speculators
    Speculators are running riot in the Auckland housing market making life tougher for first home buyers, says Labour’s housing spokesperson Phil Twyford.  Newly released data from Core Logic shows a 40 per cent increase in the share of house sales ...
    2 weeks ago
  • Disconnected thinking dirties the water
    Iain Rabbitts’ belief that drinking water quality, charging for water use and the land use that leads to water quality degradation should be treated separately is part of the problem we have right now in this country. The connection is ...
    GreensBy Catherine Delahunty
    2 weeks ago
  • Report back from Hands Off Our Tamariki hui
    This week I attended a hui in Otaki organised by Hands Off Our Tamariki about the proposed reforms to the Child Young Persons and their Families Act. Moana Jackson and Paora Moyle spoke.  They expressed deep, profound concern about the proposed ...
    GreensBy Jan Logie
    2 weeks ago
  • Labour mourns passing of Helen Kelly
    Helen Kelly was a passionate advocate for working New Zealanders and for a safe and decent working life, Leader of the Opposition Andrew Little says.  “Helen Kelly spent her adult life fighting for the right of every working person to ...
    2 weeks ago
  • National’s visionless immigration policy
    National’s recent immigration announcement is a continuation of the visionless approach to government that it has displayed in the last three terms. Rather than using the levers of government to implement a sustainable immigration policy that benefits new and current ...
    GreensBy Denise Roche
    2 weeks ago
  • Andrew Little: Speech to the Police Association Conference 2016
    Police Association delegates, Association life members and staff, representatives from overseas jurisdictions. Thank you for inviting me here today. The Police Association has become a strong and respected voice for Police officers and for policing in New Zealand. There is ...
    2 weeks ago
  • 1,000 more police for safer communities
    Labour will fund an extra 1,000 Police in its first term to tackle the rising rate of crime, says Leader of the Opposition Andrew Little. “Labour will put more cops on the beat to keep our communities safe. ...
    2 weeks ago
  • Call for all-party round table on homelessness
    Labour is calling on the Government to take part in a roundtable meeting to hammer out a cross-party agreement on ending homelessness.  Labour’s housing spokesperson Phil Twyford said the country wanted positive solutions to homelessness, and wanted the political parties ...
    2 weeks ago
  • Seclusion rooms in schools
    Schools are undoubtedly stretched and underfunded to cope with students with high learning support needs. But this cannot justify the use of rooms (or cupboards) as spaces to forcibly isolate children. It has emerged via media that this practice continues ...
    GreensBy Catherine Delahunty
    2 weeks ago
  • Public should get a say on new Waikato power station
    I had an opinion piece published in the Waikato Times about a controversial proposal to build a new gas-fired power station. It’s not on their website yet, so here it is: If you think the public would get a say ...
    GreensBy Gareth Hughes
    2 weeks ago
  • MSD and their investment approach
    The Government talks about investment but there is no investment. It is not investment if it isn’t over the whole of life and if there is no new money  — Shamubeel Eaqub   Investment sounds like adequate resourcing but this ...
    GreensBy Jan Logie
    2 weeks ago
  • Certainty needed for community services
    A couple of months ago I was at a seminar where three community organisations were presenting. Two of the three presenters were waiting to find out if their organisation would get a contract renewed with MSD. Not knowing if their ...
    GreensBy Jan Logie
    2 weeks ago
  • Domestic Violence – some advice for the media
    For the purpose of this piece, I’m going to use Domestic Violence (DV) as a proxy for intimate partner violence. DV is not isolated to physical abuse in a relationship between people with the same power. DV is a pattern of ...
    GreensBy Jan Logie
    2 weeks ago
  • Leroy’s New Paw Prints
    Leroy, an Auckland great dane recently received a new 3D printed bionic leg after cancer was discovered. I think this is a fantastic story and highlights the real potential of additive manufacturing, or 3D printing Leroy’s prosthetic was printed in titanium and was ...
    GreensBy Gareth Hughes
    2 weeks ago