web analytics

Argghh! The comment bug

Written By: - Date published: 1:44 pm, October 18th, 2015 - 10 comments
Categories: admin, The Standard - Tags: ,

I have spent a few hours been chasing the bug that has been causing people to go into moderation automatically. Found it, killed it, and I thought I’d share it with you.

It was in Wordfence, a great security program for the site.  Essentially it is one of the main plugins to block out irritating behaviours by bots treating the site as their own personal space. Because there are a lot of bots running around the net probing for security holes, reading content, and generally being a pain in the arse when trying to predict the computing resources required to operate the site.

Since we run the site on less than $300 per month funded by voluntary welcome donations (see here), I spend a lot of time keeping robot free loaders out of the site.

Our direct1 human traffic this year is about 80 gigabytes per month to mainly NZ resident humans. Our welcome and expected2 bot traffic averages about 150 gigabytes of traffic per month. And we routinely provide about another 70-80 gigabytes of traffic to various bots who follow our rules.

Crawler rules

Otherwise we’d have to provide systems that are capable of handling many times our legitimate traffic.

But there are other rules in the system designed to block bots and humans from trying to take over our site and use it for purposes nefarious – like spam and free advertising. A lot of those are focused around getting logins as admins and authors, or placing comments as new or existing commenters on the site. These are known as spambots.

So Wordfence and other systems provide preventative measures for those as well. Some of the Wordfence security options are here

Login rules

One of the reasons3 that we have run this site for many years without requiring logins is because it removes a major potential security issue. These days the only logins in active use are those of the authors and a few people who got them before I turned off access to obtaining them in 2011.

We handle robots leaving spam comments with some subtle programs that bank on human vs robot behaviours. The odd robot spam or troll that gets past those has to have an approved comment by a moderator before they can post freely.

However there are the odd persons who leave comments purporting to be by other people. Usually we know because of gravator differences because they have the wrong ’email’.

And then there is this behaviour that Wordfence protects against.

Hold anonymous comments using member emails for moderation
In WordPress it’s possible for someone who is not a member on your website to post a comment and to specify an email address of a real member on your site. This behavior is suspicious and may be incorporated into a more sophisticated attack. So we suggest that you leave this option enabled which will hold those comments for moderation before they’re published.

Thats ok – good security. It isn’t particularly relevant to our site however. Many of our authors comment without logging in because it is faster. Not having a special page generated for you is a whole lot slower than getting a cached page (apart from the replies tab). Our longer term commenters who have actual logins have mostly long since forgotten their passwords 🙂

I never noticed this option. It was added in an upgrade after I did the Wordfence setup years ago. It was off by default as are all new features. And it never got triggered when I did plugin upgrade testing.

However somehow this one button amonst the hundreds that control this site turned on and caused a week of nuisance for moderators and commenters  ! ^&*%^*^%&%^$$!!!!!

Looking at the errors at wordfence, I see this.

Hi All,

We have released a fix that addresses a bug introduced with the release that went out 15 hours ago. The issue is that if a Wordfence options checkbox is unchecked and is different from the default setting, the release resets the Wordfence setting to the default setting of checked.

Arggghhhhh… The joys of upgrades and testing..

 


 

  1. Not counting the graphics. Those are mostly served up from a content delivery network, in our case Cloudfront from Amazon.
  2. Legitimate googlebots and some other selected web search crawlers, Cloudfront, backup and synchronising systems, Internet Archive (Wayback Engine) and the National Library web harvesting (see Kiwiblogblog is not completely dead).
  3. See the privacy section of the policy.

10 comments on “Argghh! The comment bug ”

  1. r0b 1

    Hah! Thanks for tracking it down, and as usual for the work that goes on behind the scenes…

    • lprent 1.1

      More for the moderators than anyone else in this case. They were having to release the comments from moderation. And we usually get a lot of comments each day.

      Right now we’re inching towards our first million published comments accumulated over the last 8 and bit years.

      For me it was mostly a matter of finding enough clean and uninterrupted time to check everything that may have introduced this problem

  2. Thank you – I never realised the value of having realtime comments as much – it is just hard to converse when the lag is there – such an important feature that supports the Standard community – thanks again.
    Edit It works!!! thanks the gods it works 🙂 and thanks to the mods who let my comments through – a pain I know, but appreciated.

  3. Stephen 3

    Man, you do great work, LP. Long may you continue.

  4. wyndham 4

    Lyn. Alas, every post that I open carries the heading “No Comments” irrespective of the fact that there may be umpteen comments from a variety of folk. The only way I can open these comments is to click on “No Comments”. Is this in my PC or is it at your end ?

    • lprent 4.1

      Sounds like the caching, either server or client side. I will have a look at it after I get the tabs for the feeds working.

      But in the meantime, try Shift + F5 to force a reload and see if it fixes it. If it does then it is likely to be client side and probably related to date time.

      You don’t see comments for anything?

  5. Lanthanide 5

    Lynn, have you considered Patreon?

    Potentially to cover existing server support, as well as a pot of money to be split between authors in some sort of manner (may be difficult).

    • lprent 5.1

      Interesting. The reason that I went the other way; reducing cost, was because of the scarce resource we have – our time. There was no time to do structured work hours work. Instead we tend to do everything async, as and when there was a block of time.

      Just to give an idea, while I could make a call to someone during the day, if I didn’t get them first time then I didn’t leave messages. When I’m working, I don’t answer phone calls. Same for email. I keep an eye on it, but I can’t answer non-work things in a timely manner when I’m programming. It takes a mind dump to become human again, and then usually about 30 minutes to get back into the groove.

      I’m probably a bit extreme because of what I work on. But most of the people who author and moderate here seem to have the same kinds of time pressures. So the idea of doing the business kinds of things to run a voluntary blog in our “spare time” isn’t something that we can do. We don’t have the kinds of block time required, and the local market isn’t big.

      But something run for creatives who have the same issues.. That has more possibilities.

  6. infused 6

    Teach you for upgrading too quick…

Recent Comments

Recent Posts

  • Fast-tracked Northland water project will accelerate economic recovery
    The Government has welcomed the decision to approve a new water storage reservoir in Northland, the first of a number of infrastructure projects earmarked for a speedy consenting process that aims to accelerate New Zealand’s economic recovery from Covid-19.  The Matawii Water Storage Reservoir will provide drinking water for Kaikohe, ...
    BeehiveBy beehive.govt.nz
    2 hours ago
  • Tokelau Language Week reminds us to stay united and strong
    Staying strong in the face of challenges and being true to our heritage and languages are key to preserving our cultural identity and wellbeing, is the focus of the 2020 Tokelau Language Week. Minister for Pacific Peoples, Aupito William Sio, says this year’s theme, ‘Apoapo tau foe, i nā tāfea ...
    BeehiveBy beehive.govt.nz
    5 days ago
  • NZ announces a third P-3 deployment in support of UN sanctions
    The Government has deployed a Royal New Zealand Air Force P-3K2 Orion (P-3) maritime patrol aircraft to support the implementation of United Nations Security Council (UNSC) resolutions imposing sanctions against North Korea, announced Minister of Foreign Affairs Winston Peters and Minister of Defence Ron Mark. “New Zealand has long supported ...
    BeehiveBy beehive.govt.nz
    1 week ago
  • Pacific trade and development agreement a reality
    Pacific regional trade and development agreement PACER Plus will enter into force in 60 days now that the required eight countries have ratified it. Trade and Export Growth Minister David Parker welcomed the announcement that the Cook Islands is the eighth nation to ratify this landmark agreement. “The agreement represents ...
    BeehiveBy beehive.govt.nz
    2 weeks ago
  • Securing a pipeline of teachers
    The Government is changing its approach to teacher recruitment as COVID-19 travel restrictions continue, by boosting a range of initiatives to get more Kiwis into teaching. “When we came into Government, we were faced with a teacher supply crisis,” Education Minister Chris Hipkins said. “Over the past three years, we ...
    BeehiveBy beehive.govt.nz
    2 weeks ago
  • Border exceptions for a small number of international students with visas
    The Government has established a new category that will allow 250 international PhD and postgraduate students to enter New Zealand and continue their studies, in the latest set of border exceptions. “The health, safety and wellbeing of people in New Zealand remains the Government’s top priority. Tight border restrictions remain ...
    BeehiveBy beehive.govt.nz
    2 weeks ago
  • First COVID-19 vaccine purchase agreement signed
    The Government has signed an agreement to purchase 1.5 million COVID-19 vaccines – enough for 750,000 people – from Pfizer and BioNTech, subject to the vaccine successfully completing all clinical trials and passing regulatory approvals in New Zealand, say Research, Science and Innovation Minister Megan Woods and Health Minister Chris Hipkins. ...
    BeehiveBy beehive.govt.nz
    2 weeks ago
  • International statement – End-to-end encryption and public safety
    We, the undersigned, support strong encryption, which plays a crucial role in protecting personal data, privacy, intellectual property, trade secrets and cyber security.  It also serves a vital purpose in repressive states to protect journalists, human rights defenders and other vulnerable people, as stated in the 2017 resolution of the ...
    BeehiveBy beehive.govt.nz
    2 weeks ago
  • Ministry of Defence Biodefence Assessment released
    The Ministry of Defence has today released a Defence Assessment examining Defence’s role across the spectrum of biological hazards and threats facing New Zealand. Biodefence: Preparing for a New Era of Biological Hazards and Threats looks at how the NZDF supports other agencies’ biodefence activities, and considers the context of ...
    BeehiveBy beehive.govt.nz
    2 weeks ago
  • New Approaches to Economic Challenges: Confronting Planetary Emergencies: OECD 9 October 2020
    New Approaches to Economic Challenges: Confronting Planetary Emergencies: OECD 9 October 2020 Hon David Parker’s response following Thomas Piketty and Esther Duflo. Good morning, good afternoon, and good evening, wherever in the world you might be. I first acknowledge the excellent thought provoking speeches of Thomas Piketty and Esther ...
    BeehiveBy beehive.govt.nz
    3 weeks ago
  • Kaipara Moana restoration takes next step
    A Memorandum of Understanding has been signed today at Waihāua Marae between the Crown, local iwi and councils to protect, restore and enhance the mauri of Kaipara Moana in Northland. Environment Minister David Parker signed the document on behalf of the Crown along with representatives from Ngā Maunga Whakahī, Ngāti ...
    BeehiveBy beehive.govt.nz
    3 weeks ago
  • New Zealand and Uruguay unite on reducing livestock production emissions
    Agriculture Minister Damien O’Connor and Uruguayan Minister of Livestock, Agriculture and Fisheries Carlos María Uriarte have welcomed the launch of a three-year project that will underpin sustainable livestock production in Uruguay, Argentina, and Costa Rica.  The project called ‘Innovation for pasture management’ is led by Uruguay’s National Institute of Agricultural ...
    BeehiveBy beehive.govt.nz
    3 weeks ago
  • 3100 jobs created through marae upgrades
    Hundreds of marae throughout the country will be upgraded through investments from the Provincial Growth Fund’s refocused post COVID-19 funding to create jobs and put money into the pockets of local tradespeople and businesses, Regional Economic Development Minister Shane Jones and Māori Development Minister Nanaia Mahuta have announced. “A total ...
    BeehiveBy beehive.govt.nz
    3 weeks ago
  • Health volunteers recognised in annual awards
    Health Minister Chris Hipkins has announced 9 teams and 14 individuals are the recipients of this year’s Minister of Health Volunteer Awards.  “The health volunteer awards celebrate and recognise the thousands of dedicated health sector volunteers who give many hours of their time to help other New Zealanders,” Mr Hipkins ...
    BeehiveBy beehive.govt.nz
    3 weeks ago
  • Community COVID-19 Fund supports Pacific recovery
    The Minister for Pacific Peoples, Aupito William Sio says a total of 264 groups and individuals have successfully applied for the Pacific Aotearoa Community COVID-19 Recovery Fund, that will support Pacific communities drive their own COVID-19 recovery strategies, initiatives, and actions. “I am keen to see this Fund support Pacific ...
    BeehiveBy beehive.govt.nz
    3 weeks ago
  • Community benefits from Māori apprenticeships
    Up to 50 Māori apprentices in Wellington will receive paid training to build houses for their local communities, thanks to a $2.75 million investment from the Māori Trades and Training Fund, announced Employment Minister Willie Jackson today. “This funding will enable Ngāti Toa Rangatira Incorporated to provide its Ngā Kaimahi ...
    BeehiveBy beehive.govt.nz
    3 weeks ago