web analytics

Argghh! The comment bug

Written By: - Date published: 1:44 pm, October 18th, 2015 - 10 comments
Categories: admin, The Standard - Tags: ,

I have spent a few hours been chasing the bug that has been causing people to go into moderation automatically. Found it, killed it, and I thought I’d share it with you.

It was in Wordfence, a great security program for the site.  Essentially it is one of the main plugins to block out irritating behaviours by bots treating the site as their own personal space. Because there are a lot of bots running around the net probing for security holes, reading content, and generally being a pain in the arse when trying to predict the computing resources required to operate the site.

Since we run the site on less than $300 per month funded by voluntary welcome donations (see here), I spend a lot of time keeping robot free loaders out of the site.

Our direct1 human traffic this year is about 80 gigabytes per month to mainly NZ resident humans. Our welcome and expected2 bot traffic averages about 150 gigabytes of traffic per month. And we routinely provide about another 70-80 gigabytes of traffic to various bots who follow our rules.

Crawler rules

Otherwise we’d have to provide systems that are capable of handling many times our legitimate traffic.

But there are other rules in the system designed to block bots and humans from trying to take over our site and use it for purposes nefarious – like spam and free advertising. A lot of those are focused around getting logins as admins and authors, or placing comments as new or existing commenters on the site. These are known as spambots.

So Wordfence and other systems provide preventative measures for those as well. Some of the Wordfence security options are here

Login rules

One of the reasons3 that we have run this site for many years without requiring logins is because it removes a major potential security issue. These days the only logins in active use are those of the authors and a few people who got them before I turned off access to obtaining them in 2011.

We handle robots leaving spam comments with some subtle programs that bank on human vs robot behaviours. The odd robot spam or troll that gets past those has to have an approved comment by a moderator before they can post freely.

However there are the odd persons who leave comments purporting to be by other people. Usually we know because of gravator differences because they have the wrong ’email’.

And then there is this behaviour that Wordfence protects against.

Hold anonymous comments using member emails for moderation
In WordPress it’s possible for someone who is not a member on your website to post a comment and to specify an email address of a real member on your site. This behavior is suspicious and may be incorporated into a more sophisticated attack. So we suggest that you leave this option enabled which will hold those comments for moderation before they’re published.

Thats ok – good security. It isn’t particularly relevant to our site however. Many of our authors comment without logging in because it is faster. Not having a special page generated for you is a whole lot slower than getting a cached page (apart from the replies tab). Our longer term commenters who have actual logins have mostly long since forgotten their passwords 🙂

I never noticed this option. It was added in an upgrade after I did the Wordfence setup years ago. It was off by default as are all new features. And it never got triggered when I did plugin upgrade testing.

However somehow this one button amonst the hundreds that control this site turned on and caused a week of nuisance for moderators and commenters  ! ^&*%^*^%&%^$$!!!!!

Looking at the errors at wordfence, I see this.

Hi All,

We have released a fix that addresses a bug introduced with the release that went out 15 hours ago. The issue is that if a Wordfence options checkbox is unchecked and is different from the default setting, the release resets the Wordfence setting to the default setting of checked.

Arggghhhhh… The joys of upgrades and testing..

 


 

  1. Not counting the graphics. Those are mostly served up from a content delivery network, in our case Cloudfront from Amazon.
  2. Legitimate googlebots and some other selected web search crawlers, Cloudfront, backup and synchronising systems, Internet Archive (Wayback Engine) and the National Library web harvesting (see Kiwiblogblog is not completely dead).
  3. See the privacy section of the policy.

10 comments on “Argghh! The comment bug ”

  1. r0b 1

    Hah! Thanks for tracking it down, and as usual for the work that goes on behind the scenes…

    • lprent 1.1

      More for the moderators than anyone else in this case. They were having to release the comments from moderation. And we usually get a lot of comments each day.

      Right now we’re inching towards our first million published comments accumulated over the last 8 and bit years.

      For me it was mostly a matter of finding enough clean and uninterrupted time to check everything that may have introduced this problem

  2. Thank you – I never realised the value of having realtime comments as much – it is just hard to converse when the lag is there – such an important feature that supports the Standard community – thanks again.
    Edit It works!!! thanks the gods it works 🙂 and thanks to the mods who let my comments through – a pain I know, but appreciated.

  3. Stephen 3

    Man, you do great work, LP. Long may you continue.

  4. wyndham 4

    Lyn. Alas, every post that I open carries the heading “No Comments” irrespective of the fact that there may be umpteen comments from a variety of folk. The only way I can open these comments is to click on “No Comments”. Is this in my PC or is it at your end ?

    • lprent 4.1

      Sounds like the caching, either server or client side. I will have a look at it after I get the tabs for the feeds working.

      But in the meantime, try Shift + F5 to force a reload and see if it fixes it. If it does then it is likely to be client side and probably related to date time.

      You don’t see comments for anything?

  5. Lanthanide 5

    Lynn, have you considered Patreon?

    Potentially to cover existing server support, as well as a pot of money to be split between authors in some sort of manner (may be difficult).

    • lprent 5.1

      Interesting. The reason that I went the other way; reducing cost, was because of the scarce resource we have – our time. There was no time to do structured work hours work. Instead we tend to do everything async, as and when there was a block of time.

      Just to give an idea, while I could make a call to someone during the day, if I didn’t get them first time then I didn’t leave messages. When I’m working, I don’t answer phone calls. Same for email. I keep an eye on it, but I can’t answer non-work things in a timely manner when I’m programming. It takes a mind dump to become human again, and then usually about 30 minutes to get back into the groove.

      I’m probably a bit extreme because of what I work on. But most of the people who author and moderate here seem to have the same kinds of time pressures. So the idea of doing the business kinds of things to run a voluntary blog in our “spare time” isn’t something that we can do. We don’t have the kinds of block time required, and the local market isn’t big.

      But something run for creatives who have the same issues.. That has more possibilities.

  6. infused 6

    Teach you for upgrading too quick…

Recent Comments

Recent Posts

  • Twenty highlights of 2020
    As we welcome in the new year, our focus is on continuing to keep New Zealanders safe and moving forward with our economic recovery. There’s a lot to get on with, but before we say a final goodbye to 2020, here’s a quick look back at some of the milestones ...
    3 weeks ago

  • Bay Cadets learn skills to protect environment
    Bay Conservation Cadets launched with first intake Supported with $3.5 million grant Part of $1.245b Jobs for Nature programme to accelerate recover from Covid Cadets will learn skills to protect and enhance environment Environment Minister David Parker today welcomed the first intake of cadets at the launch of the Bay ...
    BeehiveBy beehive.govt.nz
    18 hours ago
  • Cook Islanders to resume travel to New Zealand
    The Prime Minister of New Zealand Jacinda Ardern and the Prime Minister of the Cook Islands Mark Brown have announced passengers from the Cook Islands can resume quarantine-free travel into New Zealand from 21 January, enabling access to essential services such as health. “Following confirmation of the Cook Islands’ COVID ...
    BeehiveBy beehive.govt.nz
    4 days ago
  • Supporting communities and landowners to grow employment opportunities
    Jobs for Nature funding is being made available to conservation groups and landowners to employ staff and contractors in a move aimed at boosting local biodiversity-focused projects, Conservation Minister Kiritapu Allan has announced. It is estimated some 400-plus jobs will be created with employment opportunities in ecology, restoration, trapping, ...
    BeehiveBy beehive.govt.nz
    5 days ago
  • Border exception for some returning international tertiary students
    The Government has approved an exception class for 1000 international tertiary students, degree level and above, who began their study in New Zealand but were caught offshore when border restrictions began. The exception will allow students to return to New Zealand in stages from April 2021. “Our top priority continues ...
    BeehiveBy beehive.govt.nz
    5 days ago
  • Tiwai deal gives time for managed transition
    Today’s deal between Meridian and Rio Tinto for the Tiwai smelter to remain open another four years provides time for a managed transition for Southland. “The deal provides welcome certainty to the Southland community by protecting jobs and incomes as the region plans for the future. The Government is committed ...
    BeehiveBy beehive.govt.nz
    5 days ago
  • New member for APEC Business Advisory Council
    Prime Minister Jacinda Ardern has appointed Anna Curzon to the APEC Business Advisory Council (ABAC). The leader of each APEC economy appoints three private sector representatives to ABAC. ABAC provides advice to leaders annually on business priorities. “ABAC helps ensure that APEC’s work programme is informed by business community perspectives ...
    BeehiveBy beehive.govt.nz
    6 days ago
  • Govt’s careful economic management recognised
    The Government’s prudent fiscal management and strong policy programme in the face of the COVID-19 global pandemic have been acknowledged by the credit rating agency Fitch. Fitch has today affirmed New Zealand’s local currency rating at AA+ with a stable outlook and foreign currency rating at AA with a positive ...
    BeehiveBy beehive.govt.nz
    6 days ago
  • Additional actions to keep COVID-19 out of NZ
    The Government is putting in place a suite of additional actions to protect New Zealand from COVID-19, including new emerging variants, COVID-19 Response Minister Chris Hipkins said today. “Given the high rates of infection in many countries and evidence of the global spread of more transmissible variants, it’s clear that ...
    BeehiveBy beehive.govt.nz
    7 days ago
  • 19 projects will clean up and protect waterways
    $36 million of Government funding alongside councils and others for 19 projects Investment will clean up and protect waterways and create local jobs Boots on the ground expected in Q2 of 2021 Funding part of the Jobs for Nature policy package A package of 19 projects will help clean up ...
    BeehiveBy beehive.govt.nz
    1 week ago
  • New Zealand Government acknowledges 175th anniversary of Battle of Ruapekapeka
    The commemoration of the 175th anniversary of the Battle of Ruapekapeka represents an opportunity for all New Zealanders to reflect on the role these conflicts have had in creating our modern nation, says Associate Minister for Arts, Culture and Heritage Kiri Allan. “The Battle at Te Ruapekapeka Pā, which took ...
    BeehiveBy beehive.govt.nz
    2 weeks ago
  • Better care for babies with tongue-tie
    Babies born with tongue-tie will be assessed and treated consistently under new guidelines released by the Ministry of Health, Associate Minister of Health Dr Ayesha Verrall announced today. Around 5% to 10% of babies are born with a tongue-tie, or ankyloglossia, in New Zealand each year. At least half can ...
    BeehiveBy beehive.govt.nz
    2 weeks ago
  • Prisoner disorder event at Waikeria Prison over
    The prisoner disorder event at Waikeria Prison is over, with all remaining prisoners now safely and securely detained, Corrections Minister Kelvin Davis says. The majority of those involved in the event are members of the Mongols and Comancheros. Five of the men are deportees from Australia, with three subject to ...
    BeehiveBy beehive.govt.nz
    2 weeks ago
  • Pre-departure COVID-19 test for travellers from the UK and the US from 15 January
    Travellers from the United Kingdom or the United States bound for New Zealand will be required to get a negative test result for COVID-19 before departing, and work is underway to extend the requirement to other long haul flights to New Zealand, COVID-19 Response Minister Chris Hipkins confirmed today. “The new PCR test requirement, foreshadowed last ...
    BeehiveBy beehive.govt.nz
    2 weeks ago
  • PM congratulates New Year Honour recipients
    Prime Minister Jacinda Ardern has added her warm congratulations to the New Zealanders recognised for their contributions to their communities and the country in the New Year 2021 Honours List. “The past year has been one that few of us could have imagined. In spite of all the things that ...
    BeehiveBy beehive.govt.nz
    3 weeks ago
  • David Parker congratulates New Year 2021 Honours recipients
    Attorney-General and Minister for the Environment David Parker has congratulated two retired judges who have had their contributions to the country and their communities recognised in the New Year 2021 Honours list. The Hon Tony Randerson QC has been appointed a Companion of the New Zealand Order of Merit for ...
    BeehiveBy beehive.govt.nz
    3 weeks ago
  • New Year’s Honours highlights outstanding Pacific leadership through challenging year
    Minister for Pacific Peoples Aupito William Sio says the New Year’s Honours List 2021 highlights again the outstanding contribution made by Pacific people across Aotearoa. “We are acknowledging the work of 13 Pacific leaders in the New Year’s Honours, representing a number of sectors including health, education, community, sports, the ...
    BeehiveBy beehive.govt.nz
    3 weeks ago
  • Supporting seniors to embrace technology
    The Government’s investment in digital literacy training for seniors has led to more than 250 people participating so far, helping them stay connected. “COVID-19 has meant older New Zealanders are showing more interest in learning how to use technology like Zoom and Skype so they can to keep in touch ...
    BeehiveBy beehive.govt.nz
    3 weeks ago