web analytics

Maintenance during the weekend

Written By: - Date published: 9:32 pm, May 27th, 2010 - 27 comments
Categories: admin - Tags:


During the weekend, amongst the other chores, the system will get an operating system upgrade. I’ll also be doing a general check for filesystem errors and look for stray files accumulating after its first 30 odd days of operation.

In theory, this should only result in a few minutes of outage. In practice it will probably have the usual unexpected issues and take a bit longer. So don’t get alarmed if the site disappears for a while on saturday midday (one of our lowest loading times in reasonable hours).

I was hoping to upgrade to WordPress 3.0 at the same time (the beta works great), but the final release hasn’t come out yet. So that will have to wait.

I have to say that so far I’m really happy with this new server. The only hassle has been some moron in San Antonio, Texas flooding us with ping backs. I’ve now blocked anything that has a similar profile, and barred some of the more persistent ping back offenders worldwide. By doing it when they touch the web server rather than after they access the sites code, there has been a noticeable reduction load.

27 comments on “Maintenance during the weekend ”

  1. infused 1

    You will have to pickup on my other comment. I always like your comments regarding the server. I assume this is a vm of some kind?

    I like the reverse dns 😛 That aside, why the need for a checkdisk? Why is the server crashing on high cpu load?

    Why 21 hops?

    All for keeping jobs in nz, but you source your server from overseas 😀 Good one 😀

    SSH is nice and open btw.

    • Ryan 1.1

      Hmm… so they’re outsourcing their web hosting? Do as I say, not as I do.

      • Marty G 1.1.1

        outsourcing is when a company takes part of its activities and gets another company overseas to do it.

        We were always going to need to buy server from somewhere. This was the best, most reliable deal. Free trade baby.

    • lprent 1.2

      Shifting from the other post…

      This site has been running on effectively a single shared CPU with 256MB of RAM in a virtual server container up until this year (in 2008, I bumped it up from 128MB). I bumped it to 512 at the end of last year. Now it is on 2GB and multi-core Xeons quite different

      Last time I looked at hits (we usually look at page impressions instead) was at the start of last year when it was getting a million or so per week. Traffic has jumped since then.

      Count the number of links per page, including in the comments. It’d be rare when a post + comments don’t have at least 20 links in it. Some of the posts with hundreds of comments will have a lot more, especially the climate change ones.

      As I said you’re looking at a system in IIS that runs well when given heaps of resources which I couldn’t afford

      Try running IIS6 + windows of your choice in 256MB with a active dynamic website on top of it. Don’t even think of putting IIS7 on it you can’t push the required OS into that.

      • Ryan 1.2.1

        Why are you running WordPress on a Windows server?

        • lprent

          I’m not. Infused was asking me why I wasn’t running it on IIS.

          But you’re right – WordPress is a bit crippled on IIS. Found that out when I ran it in a Dual CPU windows 2003 server at home for a while Bloody thing was a pain in the arse,

  2. infused 2

    False. My company does outsourcing. We outsource IT. We don’t outsource it overseas.

    • lprent 2.1

      We looked at local, in fact we ran local until the start of 2009. But basically we needed better knowledgeable support.

      Had a look at NZ sourcing when we did this new server. Could finally get the type of support we needed here at a reasonable price.

      But there was a wee clause in the contracts which on further investigation essentially said that if they got a complaint about the site from a lawyer that we’d have to remove the offending material or have the site taken down. That was regardless of the validity of the complaint. The redress was to go to court to prove the complaint was invalid – taking quite a few months. That wasn’t acceptable.

      There are similar clauses where we are now. However you’d have to apply in the legalese of the state that it is in – ie hiring a lawyer there rather than having local lawyer mate do it. That makes the cost of a nuisance complaint somewhat higher.

      • infused 2.1.1

        I don’t think that’s a good enough excuse. There are pleanty of NZ hosters that would not have held you to ransom like that.

        • lprent

          Yeah – find one? For about $250 per month. How many minutes of legal advice is that?

          I am quite specific about the type of questions of ask the hosters. They are the various ‘legal’ threats that we have had over the years.

          Think it through…

        • Marty G

          to get a site kicked off the new zealand servers we looked at all you have to do is claim it’s saying something defamatory.

          simply doesn’t meet our needs because the right would just make false complaints and keep us offline.

          • infused

            Seriously. Get a good host and that wouldn’t happen.

            I know, been there done that.

            Without much effort you can find the site I run. Probably the busuinesses I run too.

            The site I run most probably host the most questionable material in NZ.

            • lprent

              It isn’t a question of the content. We’re almost always well within the legal bounds. What I’m concerned about are nuisance complaints that have no basis, but will force the hosting company to expend legal resources and time. We’ll get them because there are a awful lot of people around who’d prefer that this site didn’t exist.

              In my opinion, based on the way that the right have previously behaved about this site, if we were hosted locally then there would be one or more attempts to close us down or stifle what we have to say coming up towards the next election.

              But I can’t see a good reason to expose friends and hosts to that, if all it takes to prevent that it is a minor payment every month and offshore hosting.

              If the site gets big enough and makes sufficient money that we can afford to set up the site on our own link and hardware, then we will resite here. In that case we’d force the complainants to put anything that is questionable in front of a court for determination. We would also be looking for costs plus damages if they fail to make a case, in fact I’d be interested in appealing low awards so we get some case law on nuisance complaints.

  3. infused 3

    Well, my site, has 2xdual core xeons @ 3.0ghz (4 cores), 2gb memory.

    Here are some stats:

    Feb 2010 747912 157.52 GB
    Mar 2010 820433 149.09 GB
    Apr 2010 639627 129.30 GB
    May 2010 510310 100.70 GB

    I\’m strugging to find why your server is stressing so hard, since the requirements under linux are probably 50% less.

    This is a Windows 2008 R2 server as well.

    Also, why host overseas? Keen to know. Since you support keeping everything here. Castle as well? sheesh.

    • lprent 3.1

      Replied above. It is more the legal question. But I’ve found the tech support for linux to be better as well.

      Your volumes are somewhat higher. I think that we’re hitting about 80-100GB at present (with a cap of 750GB).

      Your hits are well below (I’d have to look them up). Have a look at the front page sometime and figure out how many links that there are. It is all of those thumbnails. Of course most of the links are serviced with a 304 because I tuned the site that way (my .htaccess is pretty specific).

      However the pages themselves is highly dynamic. Every time a comment is changed then so is the main page. That is why comments are locked after a month. It minimizes bot traffic, and we tell the well-behaved bots what pages to fetch with what frequencies..

      • infused 3.1.1

        Ah lprent, you don’t bite. A couple of questions.

        1) I believe you’re wrong about NZ hosting. Most of the people I know, know more about Linux than Windows (bare in mind, I’m talking about sys admins, not average joe).

        2) Stored procedures? Maybe it’s time to move the DB to a seperate server

        At the end of the day, I believe if you guys are going to be preaching about keeping it in NZ, you should move your server and show your support. DPF has his here. I’m sure someone is willing to host it. I bet I could. If I can, I bet many others can. In the ISP arena, NZ has some huge talent. You should really explore it.

        My server was donated for free, my hosting is donated for free. My site is operated for free, with no ads. 100mbit, no limits on data. You need to ask around.

        • lprent

          DPF has a mate that runs his servers under some kind of special arrangement. You can see the companies ad on the site.

          We could probably talk our way into something. But really none of us has the time in real life to chase that kind of arrangement.

          It is simpler in my already cluttered life to pay a few hours of work value than it is to spend more hours running around looking for a cheaper local deal or sponsorship. Same with everyone else who writes here. We’re far more interested in writing posts than cozying up to anyone.

          It took us more than a year to find the time to put advertising on the site – and that was with the advertiser wanting us to do it. None of us had the time.

          There are no stored procedures. It is running on a bog standard WordPress system with our own theme and a pile of plugins.

          Besides, if you remember – what was the reaction when we did site on a donated system at the start of 2008? We had bozos like you running around saying that the sky had fallen. In fact I can probably find your exact comments at the time.

          I had Whaleoil going off and attacking the company that I had used to work for. Why? Because they were acting as my secondary DNS (and I theirs), when I wasn’t even working there. With a pile of fools on wingnut sites crowing in behind in the usual braying pack of mindless lynchers.

          I’d prefer The Standard to stay quite independent, not beholden to anyone who can be pressured, and out of easy legal targeting range. The wingnuts taught me that during that episode – they would literally do damn near any underhanded tactics to get the site down. It is easier and safer for the site to run off shore than dealing with the fuckwits attacking friends and allies here.

          You notice that with the exception of Whale (who finally had a totally justifiable defamation suit against him, and an even more justifiable court action now) that the same attacks are not mounted against the sites on the right? That is because the left tend to be fundamentally more civilized and committed to open debate.

          Basically, I’d have to say that right and especially the wingnuts are fundamentally uncivilized as a group when it comes to freedom of speech for people other than themselves. Sort of a mutual bully sub-society.

  4. Jim Nald 4

    I’d like to make a donation but not via online.
    I’d be happy for you to send your bank account number to my email.
    I might as well donate what I would have subscribed or intermittently pay re newspapers.

    captcha: pay
    (yes, you are psychic)
    (forgot to type captcha, trying again .. captcha: standard … yes, The Standard)

    • lprent 4.1

      Always happy to take money for the server. I’ll send you the details for The Standards new bank account after I dig them out again

      • Jim MacDonald 4.1.1

        No need to publicise it and you can just email it through if you prefer.

  5. Brett 5

    Basic stuff I know but have you set you geographic target to New Zealand for google? otherwise it defaults to whatever country your site is hosted in.

    • lprent 5.1

      Yep, it is quite aware of the country that the site is for (you can see it if you look at searches in NZ vs elsewhere). The google bots are very very well behaved – they follow the hints that we give them. So do slurp and bing and most of the majors.

      Unlike others where I had to put limitIPconn on to stop one bloody bot from using over a hundred simultaneous connections at a time to suck data off the site. The site now limits to 16 connections per IP, and another system looks at the number of connections from an IP over a 15 minute period.

  6. jcuknz 6

    All the above is way way over me, I just want the site to stay running, so best wishes for the weekend 🙂

  7. lprent 7

    Upgrade went well. Did it this morning because I’m finishing up at work with a half day which would run through midday. Start at the next project on Monday.

  8. Turn off the TV 8

    Have you tried http://wordpress.org/extend/plugins/wp-super-cache/ ??

    Are you using a PHP accelerator of some kind?

    • lprent 8.1

      APC. It was the most stable out of the ones I tested. Made a significant difference to the speed in serving the pages.

Recent Comments

Recent Posts

  • Tokelau Language Week reminds us to stay united and strong
    Staying strong in the face of challenges and being true to our heritage and languages are key to preserving our cultural identity and wellbeing, is the focus of the 2020 Tokelau Language Week. Minister for Pacific Peoples, Aupito William Sio, says this year’s theme, ‘Apoapo tau foe, i nā tāfea ...
    BeehiveBy beehive.govt.nz
    2 days ago
  • NZ announces a third P-3 deployment in support of UN sanctions
    The Government has deployed a Royal New Zealand Air Force P-3K2 Orion (P-3) maritime patrol aircraft to support the implementation of United Nations Security Council (UNSC) resolutions imposing sanctions against North Korea, announced Minister of Foreign Affairs Winston Peters and Minister of Defence Ron Mark. “New Zealand has long supported ...
    BeehiveBy beehive.govt.nz
    6 days ago
  • Pacific trade and development agreement a reality
    Pacific regional trade and development agreement PACER Plus will enter into force in 60 days now that the required eight countries have ratified it. Trade and Export Growth Minister David Parker welcomed the announcement that the Cook Islands is the eighth nation to ratify this landmark agreement. “The agreement represents ...
    BeehiveBy beehive.govt.nz
    1 week ago
  • Securing a pipeline of teachers
    The Government is changing its approach to teacher recruitment as COVID-19 travel restrictions continue, by boosting a range of initiatives to get more Kiwis into teaching. “When we came into Government, we were faced with a teacher supply crisis,” Education Minister Chris Hipkins said. “Over the past three years, we ...
    BeehiveBy beehive.govt.nz
    2 weeks ago
  • Border exceptions for a small number of international students with visas
    The Government has established a new category that will allow 250 international PhD and postgraduate students to enter New Zealand and continue their studies, in the latest set of border exceptions. “The health, safety and wellbeing of people in New Zealand remains the Government’s top priority. Tight border restrictions remain ...
    BeehiveBy beehive.govt.nz
    2 weeks ago
  • First COVID-19 vaccine purchase agreement signed
    The Government has signed an agreement to purchase 1.5 million COVID-19 vaccines – enough for 750,000 people – from Pfizer and BioNTech, subject to the vaccine successfully completing all clinical trials and passing regulatory approvals in New Zealand, say Research, Science and Innovation Minister Megan Woods and Health Minister Chris Hipkins. ...
    BeehiveBy beehive.govt.nz
    2 weeks ago
  • International statement – End-to-end encryption and public safety
    We, the undersigned, support strong encryption, which plays a crucial role in protecting personal data, privacy, intellectual property, trade secrets and cyber security.  It also serves a vital purpose in repressive states to protect journalists, human rights defenders and other vulnerable people, as stated in the 2017 resolution of the ...
    BeehiveBy beehive.govt.nz
    2 weeks ago
  • Ministry of Defence Biodefence Assessment released
    The Ministry of Defence has today released a Defence Assessment examining Defence’s role across the spectrum of biological hazards and threats facing New Zealand. Biodefence: Preparing for a New Era of Biological Hazards and Threats looks at how the NZDF supports other agencies’ biodefence activities, and considers the context of ...
    BeehiveBy beehive.govt.nz
    2 weeks ago
  • New Approaches to Economic Challenges: Confronting Planetary Emergencies: OECD 9 October 2020
    New Approaches to Economic Challenges: Confronting Planetary Emergencies: OECD 9 October 2020 Hon David Parker’s response following Thomas Piketty and Esther Duflo. Good morning, good afternoon, and good evening, wherever in the world you might be. I first acknowledge the excellent thought provoking speeches of Thomas Piketty and Esther ...
    BeehiveBy beehive.govt.nz
    2 weeks ago
  • Kaipara Moana restoration takes next step
    A Memorandum of Understanding has been signed today at Waihāua Marae between the Crown, local iwi and councils to protect, restore and enhance the mauri of Kaipara Moana in Northland. Environment Minister David Parker signed the document on behalf of the Crown along with representatives from Ngā Maunga Whakahī, Ngāti ...
    BeehiveBy beehive.govt.nz
    2 weeks ago
  • New Zealand and Uruguay unite on reducing livestock production emissions
    Agriculture Minister Damien O’Connor and Uruguayan Minister of Livestock, Agriculture and Fisheries Carlos María Uriarte have welcomed the launch of a three-year project that will underpin sustainable livestock production in Uruguay, Argentina, and Costa Rica.  The project called ‘Innovation for pasture management’ is led by Uruguay’s National Institute of Agricultural ...
    BeehiveBy beehive.govt.nz
    2 weeks ago
  • 3100 jobs created through marae upgrades
    Hundreds of marae throughout the country will be upgraded through investments from the Provincial Growth Fund’s refocused post COVID-19 funding to create jobs and put money into the pockets of local tradespeople and businesses, Regional Economic Development Minister Shane Jones and Māori Development Minister Nanaia Mahuta have announced. “A total ...
    BeehiveBy beehive.govt.nz
    2 weeks ago
  • Health volunteers recognised in annual awards
    Health Minister Chris Hipkins has announced 9 teams and 14 individuals are the recipients of this year’s Minister of Health Volunteer Awards.  “The health volunteer awards celebrate and recognise the thousands of dedicated health sector volunteers who give many hours of their time to help other New Zealanders,” Mr Hipkins ...
    BeehiveBy beehive.govt.nz
    2 weeks ago
  • Community COVID-19 Fund supports Pacific recovery
    The Minister for Pacific Peoples, Aupito William Sio says a total of 264 groups and individuals have successfully applied for the Pacific Aotearoa Community COVID-19 Recovery Fund, that will support Pacific communities drive their own COVID-19 recovery strategies, initiatives, and actions. “I am keen to see this Fund support Pacific ...
    BeehiveBy beehive.govt.nz
    2 weeks ago
  • Community benefits from Māori apprenticeships
    Up to 50 Māori apprentices in Wellington will receive paid training to build houses for their local communities, thanks to a $2.75 million investment from the Māori Trades and Training Fund, announced Employment Minister Willie Jackson today. “This funding will enable Ngāti Toa Rangatira Incorporated to provide its Ngā Kaimahi ...
    BeehiveBy beehive.govt.nz
    2 weeks ago
  • Training fund supports Māori jobseekers
    Rapidly growing sectors will benefit from a $990,000 Māori Trades and Training Fund investment which will see Wellington jobseekers supported into work, announced Employment Minister Willie Jackson today. “This funding will enable Sapphire Consultants Ltd. to help up to 45 Māori jobseekers into paid training initiatives over two years through ...
    BeehiveBy beehive.govt.nz
    2 weeks ago
  • Ruakura Inland Port development vital infrastructure for Waikato
    The Government is investing $40 million to develop an inland port at Ruakura which will become a freight super-hub and a future business, research and residential development for the Waikato, Urban Development and Transport Minister Phil Twyford, and Māori Development Minister Nanaia Mahuta announced today. The funding has been has ...
    BeehiveBy beehive.govt.nz
    3 weeks ago
  • Appointments made to Defence Expert Review Group
    Defence Minister Ron Mark announced today the establishment of an Expert Review Group to review a number of aspects of the New Zealand Defence Force’s (NZDF) structure, information management and record-keeping processes.  The Expert Review Group’s work arises out of the first recommendation from the Report of the Government’s Inquiry ...
    BeehiveBy beehive.govt.nz
    3 weeks ago
  • No active community cases of COVID-19
    There are no active community cases of COVID-19 remaining in the country after the last people from the recent outbreak have recovered from the virus, Health Minister Chris Hipkins said today. “This is a big milestone. New Zealanders have once again through their collective actions squashed the virus. The systems ...
    BeehiveBy beehive.govt.nz
    3 weeks ago
  • Clean energy upgrade for more public buildings
    More public buildings will be supported by the Government to upgrade to run on clean energy, the Minister for Climate Change James Shaw announced today. Minister Shaw announced that Lincoln and Auckland universities will receive support through the Clean-Powered Public Service Fund to replace fossil fuel boilers. Southern, Taranaki, and ...
    BeehiveBy beehive.govt.nz
    3 weeks ago
  • Schools back donations scheme for the second year
    More schools have opted in to the donations scheme for 2021, compared to 2020 when the scheme was introduced. “The families of more than 447,000 students will be better off next year, with 94% of eligible schools and kura opting into the scheme,” Education Minister Chris Hipkins said. “This is ...
    BeehiveBy beehive.govt.nz
    3 weeks ago