700,000 comments

Written By: - Date published: 5:13 pm, March 6th, 2014 - 31 comments
Categories: The Standard - Tags:

TheStandard at 700k commentsWe passed 700 thousand comments sometime today over our last 6 and a half years. I was going to have a whole pile of statistics ready for the occasion. However a dose of flu caught me on friday after threatening for the whole of the previous wee. Having three days in bed asleep only waking up for panadol put paid to that idea. But we’re pretty much tracking towards a million comments in August 2015 when the site turns eight years old. Eddie predicted that back in December 2012, and we were bang on prediction when we hit 600 thousand comments in August last year

 

a million comments

It looks like our active moderation policies have contradicted the naysayers who said that having it would make the site moribund. Of course most of them said it as they were being booted off the site so the rest of us could live without their mosquito like mindless whining.

Last august, I was preparing to shift the site completely into cloud space. That is now completed. So most of the niggles I was talking about back in August have disappeared.

Of course (in another long tradition) our very success also revealed more technical issues. The volume of comments coupled with a new database caused a problem with the database not coping with simultaneous comment posting, and last months total includes 98 duplicates that I have to figure out how to clean out (without disturbing comment threading)

These days we’re running on a database server, a file server, and a memcache server that are the  back end for a cluster of web servers going on and off line depending on the need of the site. Overnight it slumbers with a single web server. During the day I’ve seen it with seven running. Most of the time (like today) it peaks at 3 or 4. It should scale well into the coming election as the site winds up with an extra 50% or more of load.

BTW: I’ll have a hunt for some “free” time to pull stats out this weekend. The main one of interest to me is that the trend for the density of comment detail is continuing to rise.

31 comments on “700,000 comments ”

  1. Skinny 1

    Congrats Team well done! It is the best Left political blog going and long may it continue to evolve!

  2. tinfoilhat 2

    “The main one of interest to me is that the trend for the density of comments is continuing to rise.”

    Nice double entendre !

  3. Clemgeopin 3

    Nice to know. Congrats. Of those 700,000 I think at least 50 or so are mine!

    • lprent 3.1

      That is easy enough to check. Click your name in the Comments tab on the right and it will give you a listing of every comments and/or post under that name.

      Umm got to implement the “walk the pages” on those search “posts”

      • Lanthanide 3.1.1

        Commenting so I show up on the right…

        Right, that doesn’t actually tell me how many comments I’ve made in total. Only goes back 33 pages of history, too.

        • lprent 3.1.1.1

          Ok. A final task for tonight (I hope).

        • lprent 3.1.1.2

          Ok, I remember that there were several problems with this – that is why the code was turned off..

          The first of which was that it simply didn’t like comments. It was built based on posts or comments, but not the mixture that I’d used for the search..

          The second was that it wasn’t too good on the search criteria and I needed to nick some code off another package.

          The third was that it didn’t say how comments/posts that there were.

          And finally I needed to resize the comments so that they were closer to the size of the posts that the search page was based on OR reduce the post snippets because otherwise the page finished far too early (or too late)

          I’ll leave it there for the moment whilst I have a read of the code. But I suspect that it is a weekend job

  4. bad12 4

    Kia ora Lprent for keeping the truck rolling, hows the financial situation while we are talking about keeping things rolling along…

    • lprent 4.1

      Mildly bad at present because scoop (who do our advertising) are still sorting themselves out and the tech work on servers has been more draining than I’d like over the last 6 months. In theory those changes and the work should start paying for themselves now.

      On the other hand I’ve pretty well cleared the financial residue of the leaky building fix, so I’m better able to cover it while they sort themselves out (or we shift to different model).

  5. karol 5

    Thanks for all your work, Lynn, and for keeping an eye on the stats. Sorry you had the flu.

    • lprent 5.1

      I keep an eye on the stats because it is a good early warning indicator of approaching capacity issues.

      I did leave it rather hanging over the weekend with the flu. I have two posts that I started to write before dropping back into a stricken stupor. My advice. If you see anyone with the flu this year, then avoidance is good policy.

      • karol 5.1.1

        I did wonder if you were ill when I thread I sent to moderation stayed there for a couple of days – which maybe was a good thing for that thread/commenter.

        As I get older I wonder if I should take a flu jab – but usually I get a fairly mild dose f the current flu/cold, if that.

        • lprent 5.1.1.1

          I think that I’m going to have to start getting them. The couple of ones have been nasty.

          • Flip 5.1.1.1.1

            I’d recommend flu jabs. They leave you feeling a bit unwell for a couple of days but to avoid the flu I’ve found it is worth it.

            • lprent 5.1.1.1.1.1

              The unwell part was always a problem for me. But I gather that they’re less violent now than they were back in the 90s when I tried them last.

  6. phil 6

    Well done. Pity about those able to collate and spy on us all;-)

    • lprent 6.1

      Reminds me now that the servers are performing correctly for TS. Time to reinstitute SSL on the site.

      It does feel rather like wearing an edible condom and expecting it to be useful in pregnancy protection. But hopefully it will be better than nothing.

      Also time to check the IPv6 to help out with rerouting

  7. swordfish 7

    By my reckoning, trythisone’s comment of 17 August 2007 (12:32 pm to be exact) was the first ever comment here. http://thestandard.org.nz/key-plays-a-big-bertha-off-a-par-3/#comments

    And the historic comment, itself ?……”Where will it lead? 😉 ” in response to a post by Z K MUGGLETONSPOFIN on Key’s golf exploits.

    I wonder where trythisone is today ? Current commenter with different pseudonym ? By comparison, I’m fresh off the boat – October 2010, if I remember rightly. (or, indeed, leftly)

  8. The Real Matthew 8

    Glad to be a part of it

  9. greywarbler 9

    Do whaleoil and kiwiblog have the same issues I wonder? Do bigboy and Father have to worry about money? I bet they don’t, they have advertising don’t they, and integrate their blogs into other ‘business’ interests. I bet they don’t follow in Rutherford’s footsteps with his pragmatic thinking as in, “We didn’t have the money, so we had to think”?

    I feel it’s just The Standard team and Lynn Prentice’ engineering of pumps and levers that are the example of doing a lot with little in the time-honoured NZ innovative way? Congrats with making it work so well, and go so high. And presently, needing another injection of the good oil to prevent any squeaks and groans?

    Incidentally also from Wikiquotes:
    Scientists [Blogs] are not dependent on the ideas of a single man, but on the combined wisdom of thousands of men, all thinking of the same problem, and each doing his little bit to add to the great structure of knowledge which is gradually being erected.
    Footnote – He embraces she!

    • Murray Olsen 9.1

      We all stand on the shoulders of giants, and some of us stand on their toes.

      It’s actually sad for me to look back and see what pioneers like Rutherford said about science. Very few of the revolutionary physicists of the early 20th century tried to take sole credit for their work. These days, with the role of the funding agencies, it is rare to acknowledge anyone else’s contribution. I suspect it was different before Rogernomics, but I only started university in 1988, so I have nothing to guide me here except my prejudices.

      On the comments – happy to have the opportunity to contribute. I hope I manage to say something sensible now and then. Looking at the graph, they seem to be growing exponentially. This brings to mind a physicist joke about the ever increasing number of articles published, and how if they were stacked on top of each other, the pile would grow faster than the speed of light. This would not be in violation of special relativity because there would be no transfer of information. (We do not get paid for our comedic talents, thank god.)

      • lprent 9.1.1

        Looking at the graph, they seem to be growing exponentially.

        That was what Eddie observed and based his prediction on. Presumably there is a plateau point somewhere.

      • greywarbler 9.1.2

        Murray O
        This:

        We stand on the shoulders of giants, but some of us stand on their toes.

        FIFY. Is that you,r line, original? It’s a good one.
        I think you add good stuff to the blog. Those trying, toes or shoulders, we all do our bit as we can.

        • Murray Olsen 9.1.2.1

          As far as I know it’s original. It comes from an interaction with a PhD student I was supervising. She had discovered that someone famous was not without their flaws and was a bit reluctant to report her result. I said that sometimes it can be just as important to stand on the toes of giants as on their shoulders.

  10. fambo 10

    Congrats to The Standard, my favourite blog site with The Daily Blog, and No Right Turn

The server will be getting hardware changes this evening starting at 10pm NZDT.
The site will be off line for some hours.