Why you’re getting auto-moderated

Written By: - Date published: 12:49 pm, June 13th, 2011 - 20 comments
Categories: admin, The Standard - Tags:

There has been a repeated question over the last few days coming up in comments about auto-moderation. For instance prism asks

Some were wondering what initiates moderation. I have just used Latin quotation – quid pro quo and the system may not know that one. ‘Lickspiitle’ politicians might be new. Mentioning Matthew Hooton? (Okayed in a few minutes.)

Right now the most common reason for auto-moderation is coming from akismet‘s servers are getting hammered in the battle against spam. It is affecting the rate at which comments are getting automatically moderated on this site.

Normally we’d get akismet to check  300-600 comments in a day for spamness. This includes both the actual user comments and the bot comments.  But if you look at the graph above, it shows the numbers of spam comments that akismet has processed from this site over the past 60 days. You’ll notice that just under a week ago we went from having less than 50 spam per day to having more 600-900. With the legitimate comments on top, it means that we have massively increased the number of spam being passed to be checked. Part of this appears to be because it counts resubmitted comments (more on that shortly), but the base load of comments arriving appears to have increased markedly.

I’m seeing the same pattern in several other sites that I have admin access to. I suspect that over the last 7 days the number of comments being processed at akismet’s server has gone up about 10 fold or more. Most sites get the same amount of spam comments, but will only have a fraction of our daily comments. Their server system is struggling under the load.

How does all this relate to auto-moderation? Well, when our akismet plugin is unable to get a timely answer from the akismet servers, it will pop the comment into auto-moderation. You’ll see this showing as it being moderated. The akismet plugin will automatically resubmit the comments shortly after and will keep doing so until it gets an answer (also increasing the load on their servers). Most comments are clearing within a couple of minutes automatically.

So for the moment, you’ll have to just put up with it. If it persists past this week, then I’ll start manually tagging IP ranges in .htaccess to not accept comments from (as I have already done for trackbacks and pings).

20 comments on “Why you’re getting auto-moderated ”

  • Comments are now closed

  1. Portion Control 1
    13 June 2011 at 3:03 pm

    Thank you Lyn I thought it was just because I’m a new commenter but I did notice that my comments are coming through very quickly so your moderators are working really hard. Bugger these spam artists.

    • lprent 1.1
      13 June 2011 at 4:35 pm

      If you’re a first time commentator by e-mail address, then your messages will get held until one is approved manually by a moderator. After that happens your comment don’t require manual releases. This prevents the bots that almost look human from getting free access.

      (Always amazes me how often idiot humans will leave a troll 101 style comment as their first. We send them straight to the spam filter. Eventually they figure out that they need to read the comments to find out what is acceptable.) 

      The auto-moderation I was describing above happens after you’ve had a comment approved.

  2. Jum 2
    13 June 2011 at 4:29 pm

    Just wait until the gummint gets to work on its Labour Supporters’ list and we will get to see what it really means to have a fascist, authoritarian state.

    PS And that post took 3 attempts to get it through. We have no idea in New Zealand what it is like to not have a voice, but Key and Co will soon let us know.

    • lprent 2.1
      13 June 2011 at 4:41 pm

      Probably because you used several of my manual words that I use to filter out trolls – “gummint”, and “fascist”.

      There is a selection of standard words that experience has shown me to be strongly associated with trolls, and which they appear to be helpless to resist using. Don’t talk like a troll and it is easier to leave comments that show up fast.

  3. ianmac 3
    13 June 2011 at 4:59 pm

    Suppose the huge increase of Spam was the reason for Clear.net to put blocks from certain sources. I phoned them but I didn’t really understand the Philipino operator who hadn’t seen the edict.

    • lprent 3.1
      13 June 2011 at 11:42 pm

      Likely. There is something new out there. There have been a few botnets taken out recently and the whole net has gotten quieter. But we seem to have something new tuning out there.

    • Deadly_NZ 3.2
      14 June 2011 at 1:49 am

      Don’t waste your time with the Telstra help line they are hopeless

      • M 3.2.1
        14 June 2011 at 7:26 pm

        Amen to that Deadly – try writing a strongly worded email from their site as I did and got a person in NZ phoning me (I had been overcharged twice and got a penalty as well) as the poor person in Manila or wherever was not qualified to give me a yes.

        Do not take no from someone who is not qualified to give you a yes.

        Luckily I have kept the cell phone number of the guy who signed me up and I got on to him and he also gave Telstra accounts the hurry-up.

        Good luck

  4. prism 4
    13 June 2011 at 6:07 pm

    Thanks lprent – you put in great work required to keep the blog going. Good to know that individuals count even when you’re one of the biggest in NZ.

    • lprent 4.1
      13 June 2011 at 11:57 pm

      Probably the biggest by one measure – page views. But the winter slowdown is finally happening – well apart from the last couple of days.

      But that personal service (especially to spoiled brat trolls like schroedinger at 5 who can’t do whatever they want) seems to work pretty well.

  5. schroedinger 5
    13 June 2011 at 6:42 pm

    Yeah, like awesome. No relationship with trying to clamp down on inconvenient comments eh.

    • ianupnorth 5.1
      13 June 2011 at 7:14 pm

      In my experience the inconvenient commentators usually get scared off by robust arguments and strong fact.

      Exactly where is Monty these days anyway?

      • lprent 5.1.1
        13 June 2011 at 7:29 pm

        I had a burst earlier this year of banning people until after the election if I had banned them too often previously. I think that he was one of them. I was having to expend too much time and effort moderating, as were the other mods. I wasn’t feeling too shit hot health wise (heart attack in Feb) and figured it was time to reduce the workload on the site.

        Behavior improved markedly. As is common when I escalate the moderation, the number of pageviews, comments and commentators started rising more rapidly. Of course some of that could be because it is election year. We will find out in march/April next year after the election and the slowness of summer.

        • Deadly_NZ 5.1.1.1
          14 June 2011 at 1:51 am

          Ahhh but did millsey cop it as`well?

          • lprent 5.1.1.1.1
            14 June 2011 at 8:32 am

            From memory he caught a weeks ban for going over the edge after I had warned a couple of times. Pretty sure I have seen him back after that.

    • McFlock 5.2
      13 June 2011 at 9:06 pm

      I guess you won’t know if there’s anything in that until you sniff the box, eh?
       
      “Schroedinger’s crap” sounds like a phrase to keep in one’s hip pocket.

  6. Deadly_NZ 6
    14 June 2011 at 1:52 am

    Dunno if you would call this Crap lol.
    http://www.schrodinger.com/

  7. Vicky32 7
    14 June 2011 at 3:48 pm

    I am glad to know why it’s happening, so no probs… 🙂

  8. schroedinger 8
    14 June 2011 at 5:53 pm

    I consider myself on notice. seriously. yeah.

  9. lprent 9
    30 June 2011 at 5:09 pm

    Damn – it is still rising…

The server will be getting hardware changes this evening starting at 10pm NZDT.
The site will be off line for some hours.