Open Mike – 04/04/2010

Written By: - Date published: 6:00 am, April 4th, 2010 - 53 comments
Categories: open mike - Tags:

Open mike is your post.

It’s open for discussing topics of interest, making announcements, general discussion, whatever you choose.

Comment on whatever takes your fancy.

The usual good behaviour rules apply (see the link to Policy in the banner).

Step right up to the mike…

53 comments on “Open Mike – 04/04/2010 ”

  • Comments are now closed

  1. Bored 1
    4 April 2010 at 7:08 am

    Terry Heffernan RIP.

    Reading the Dom yesterday there was an obituary for Terry, I was rather shocked and reminded of our own mortality having known him well at Uni. Terry was that wonderful thing, a truly honest man who had the enviable record of standing for every party in NZ and never winning. The true political chameleon whose views did not change as those around him did.

    I first met Terry in a tutorial, he was years senior to a spotty first year but displayed his true nature, friendly and prepared to debate, not dictate. He tolerated our extreme leftism and was never dismissive. More importantly he was always friendly, ready to engage and cheerful. This attracted the epithet “Heffernana” to which he would smile broadly. Odd ball perhaps but a lovely guy.

    I ran into Terry only a few times over the years, kept an eye on him through the papers. It saddened me that his demise did not attract much media coverage, he might not have been a political winner but he was a trier and as they say God loves a trier.

    So, heres to you Terry, RIP.

  2. Cnr Joe 2
    4 April 2010 at 8:26 am

    Nice to see the Standard back up and running.
    What would the current political debate be like without it?

    • (NOT) Galeandra 2.1
      4 April 2010 at 8:39 am

      Less facile ?

      [lprent: Bugger off gitmo. Go back to playing with yourself. ]

  3. The Chairman 3
    4 April 2010 at 8:56 am

    On a sewer project that was originally supposed to cost $250 million, the county now owed a total of $1.28 billion just in interest and fees on the debt.

    The destruction of Jefferson County reveals the basic battle plan of these modern barbarians, the way that banks like JP Morgan and Goldman Sachs have systematically set out to pillage towns and cities from Pittsburgh to Athens.

    Read more here:

    http://www.rollingstone.com/politics/story/32906678/looting_main_street/1

  4. prism 4
    4 April 2010 at 9:10 am

    Less factual?

  5. sk 5
    4 April 2010 at 9:22 am

    nah Galendra, that is the Prime Minister you are referring to.

    Chris Carter’s comments on Red Alert re: NZ whaling policy being made up on the hoof, and pissing off the Australians and the UK highlights the very real limitations of our PM. The ‘Forrest Gump’ of NZ politics; in all the photos with a great smile (smirk), but . . .

  6. jcuknz 6
    4 April 2010 at 9:47 am

    Congrats on being up and running again … the world might have been less facile without you but less interesting too and you don’t have a monopoly on facilism. Good to see you back 🙂

  7. ropata 7
    4 April 2010 at 10:52 am

    I was going nuts without the Standard. Haere mai ki te korero!

    • lprent 7.1
      4 April 2010 at 11:17 am

      Didn’t realize you were a fan 🙂

      Testing the comment tab speed after a comment gets posted

      • bill brown 7.1.1
        4 April 2010 at 11:21 am

        There’s a spam leak in the “Nats fail on crime” thread

        • lprent 7.1.1.1
          4 April 2010 at 11:30 am

          Ok – I’ll have a look at it. We’re being targeted a bit at present – I guess because of the site change..

      • lprent 7.1.2
        4 April 2010 at 11:24 am

        Ok – that is significantly better response

        Changed from


        SELECT wp_comments.* FROM wp_comments
        JOIN wp_posts ON wp_posts.ID = wp_comments.comment_post_ID
        WHERE comment_approved = '1' AND post_status = 'publish'
        ORDER BY comment_date_gmt DESC LIMIT 15

        To

        select A.* FROM
        (
        SELECT wp_comments.* FROM wp_comments
        ORDER BY comment_date_gmt DESC LIMIT 100
        ) AS A, wp_posts
        WHERE wp_posts.ID = A.comment_post_ID
        AND comment_approved = '1'
        AND post_status = 'publish'

        It pulled 288 records in total to extract 100 records rather than 156k..

        Now it does everything using indexes rather than generating a temp table of 156k records. Even for a hash table that is quite a bit…

        Guess I’d better lumber up the svn and post that back to wordpress…

        • Lanthanide 7.1.2.1
          4 April 2010 at 4:55 pm

          156k temp table vs 288? That’s crazy bad for original code used widely.

          • lprent 7.1.2.1.1
            4 April 2010 at 5:24 pm

            Yeah. This is in the default widgets code for recent comments. I guess there aren’t that many sites having 150k+ comment showing recent comments at all.

      • ropata 7.1.3
        4 April 2010 at 2:33 pm

        Yeah. Well I only usually comment when I want to argue about something otherwise I appreciate the worthwhile commentary round here, even if I don’t always agree it makes me think. Rangimarie.

        PS. Hurrah for indexes. Also you might want to get rid of the inline query and make it a stored proc. (if that’s possible in your DB)

        • lprent 7.1.3.1
          4 April 2010 at 3:12 pm

          Yeah it is. MySQL does most of the standard database ops.

          However I try to keep wordpress as conformant as possible to the standard version as it reduces headaches when they do an upgrade – like in a few weeks. WordPress 3.0 has released their near to final beta that I’m anxious to play with.

          Just at present I have that one tweak for that SQL query because it was REALLY slowing down the posting of comments. Apart from that there is the theme written by mummybot with a few tweaks by me, and a series of 18 standard plugins, 2 plugins by me doing a few operations mainly for advertising support, and a plugin by mummybot supporting the theme.

          Most of the custom work is in the (L)AMP stack. PHP and MySQL have performance driven configurations. Apache2 uses mpm_worker and fast cgi to enhance performance, with mods like limitconn to keep the pesky bots under control. It also has some custom .htaccess for the site to minimize the server loadings.

          But this latest bug was in linux – the absolute last thing I suspected. It looked initially like a hardware fault. After that was eliminated I went through all of the rest first and finally wound up concluding it was something weird in the OS. I sure hope that is what it was, because I’d hate to go through all this again…

  8. prism 8
    4 April 2010 at 11:32 am

    Interesting interviews on death penalty, treatment of criminals on Chris Laidlaw by Jeremy Rose 11am Sunday 4/4/10. One small area of texas Harris County has greater death penalties than whole of Texas, and kills 40% of all USA
    criminal death sentences.
    NZ Prof Flynn effect talked about, and China talked about also. So quite wide, and interesting books written about.

  9. gingercrush 9
    4 April 2010 at 11:46 am

    Hmm what is going on with Scoop and thus having ads on here, Kiwiblog, PA and Pundit all mucking up?

  10. mickysavage 10
    4 April 2010 at 11:52 am

    Anyone read the Herald this morning? There is this fascinating article about how there is a new group of working class conservatives and some comments about the last election.

    Did you know,

    1. Labour lost 4 “middle electorates”. I thought it was only three?
    2. Sam Lotu Liga bet Mark Gosche, I must ask Carol Beaumont about this.
    3. Pansy Wong winning Botany was a surprise, as if.
    4. Paula Bennett “stormed home” in Waitakere, if you coult winning by 600 votes “storming home”.
    5. John Banks has a chance of being the Super City Mayor, yeah right!
    6. An opinion poll from 5 months ago is quoted as evidence of continued support.

    They quote someone who was impressed by National because it has a “long term” view. I must check up the meaning of that phrase because it seems to have changed dramatically.

    Someone else voted National because she wanted more people paying tax. Boy she must be disappointed now.

    Another on a low income because she would get a tax cut. I wonder if she could sue for a broken promise.

    An accountant voting National is offered as evidence of the decline in Labour’s fortune.

    According to Stephen Joyce “The war for the Beehive, he says, was been won in places like Waitakere.” Give that man a dictionary!

    In the most telling statement a woman in Onehunga stated “I don’t think so, they just haven’t hit the right balance between what people want and what’s good for the company – sorry – country.”

    She was right first time!

    The link is at http://www.nzherald.co.nz/nz/news/article.cfm?c_id=1&objectid=10636275&pnum=0. I thought April 1 was a couple of days ago?

  11. Leopold 11
    4 April 2010 at 12:18 pm

    Beware – names and emails are turning up in your “leave a reply” section at end of comments

    • lprent 11.1
      4 April 2010 at 12:39 pm

      Ouch… That may explain what gitmo was doing. Thanks. I’ll check

    • lprent 11.2
      4 April 2010 at 12:45 pm

      They are meant to show your own values from cookies (stored on your machine) once you’ve used them.

      Are you saying that you can see other peoples name, e-mail, and URL?

      If so, are you behind a company type firewall where the cookies could have become cached? What browser? Can anyone else see them…

      I can’t reproduce it…

      captcha: dictionary

  12. gingercrush 12
    4 April 2010 at 12:22 pm

    I rather liked the article but something that is missing is any comments about lower turnout which did affect some electorates and notably the splitting of the left vote in electorates such as Auckland Central.

    [lprent: Damn – gitmo just used your details. Now to figure out what is going on. I’d suggest that people login in (that can’t be seen) until I get this fixed. In the meantime I’m going to disable anonymous comments. ]

  13. lprent 13
    4 April 2010 at 2:03 pm

    Ok. That ‘should’ fix it. I’ll test in another browser

  14. AncientGeek 14
    4 April 2010 at 2:09 pm

    Test comment…

    Maybe it is to do with the redit

  15. lprent 15
    4 April 2010 at 2:14 pm

    Looks good. Php.ini was still in the default development mode for cookies. Updated with the correct file. There was a nasty flag on in the wordpress caching. One of those was probably the problem.

    If anyone sees a recurrence, please e-mail lprent [at] primary.geek.nz

    Thanks.

    Now to get mail set up for the registrations etc – after lunch

  16. vto 16
    4 April 2010 at 2:23 pm

    Just harking back to recent debate about the profitability of Coro mines it seems that between 1888 and 1951 Martha Hill (the mine in central Waihi occupying about 300 acres) gave up 35,000,000 ounces (yes 35million) of gold. At today’s prices that is about $NZ55 billion. Not bad. And definitely profitable.

  17. (NOT) Sanctuary 17
    4 April 2010 at 3:13 pm

    Leopold speaks:
    Not the real Sanctuary here -but it appears that names, e-mail addresses and URLs are still appearing on my computer (having logged out since last time). Don’t mind not being anonymous, but some would be thoroughly pissed off about messages sent under their own names
    (Sorry, Sanctuary!)

    [lprent: Thanks. You should find that it is not happening now (let me know if it is), because I turned supercache off. The problem is that there is a switch somewhere that is writing then at the server side rather than from cookies at the client-side. Hunting… ]

  18. lprent_test 18
    4 April 2010 at 3:23 pm

    The stored e-mail values are still showing.

    Turned SuperCache off. It and the updated ajax reedit are the only possible remaining ones. I still haven’t managed to sight it myself. So I’m just going to read the HTML to see what loads those fields. They’re meant to all come from the client side. But they must be being set at the server.

  19. lprent_test 19
    4 April 2010 at 3:28 pm

    Damn it is writing the values in at the server. I can see them in the code. Therefore they will get cached at the server by supercache.

    Since that didn’t used to happen, I have a setting wrong. Hunting….

  20. Lindsey 20
    4 April 2010 at 3:50 pm

    It is increasingly obvious that The Herald no longer employs people who are capable of doing the most basic of fact searches, or sub-editors who are capable of spotting the most basic of factual errors. They do their political masters no service when their repeaters are not even capable of establishing the basic facts upon which to hang their pre-determined lines of “analysis”.

  21. lprent_testing 21
    4 April 2010 at 5:21 pm

    Test message to see if I’ve gotten rid of the redisplay of comment emails

    Yes… Now to get it to redisplay at the client side

  23. lprent 23
    4 April 2010 at 7:23 pm

    Ok that is solved in between other bits and pieces. It now loads the defaults on the client side from the cookies using javascript.

    You’re safe to use the anonymous commenting.

    I think I’ll leave the requirement to validate the first comment by a user on. It’ll make it easier to constrain the people that like leaving multiple comments under different names.

    The e-mail at the server is up for registration and passwords again after I convinced the mail serve at my side that it wasn’t a rogue mail server. Generally it is a lot easier for people to login. For a start the captcha is off if you are logged in, and it is a hell of a lot more secure.

  24. armchair critic 24
    4 April 2010 at 8:04 pm

    It looks like it is working properly.

  25. lprent 25
    4 April 2010 at 10:31 pm

    Ok search is back online.
    The RSS feeds appear to be working (including a slow query that I’ll have to fix later).
    The slow queries have virtually disappeared except under load.

    I suppose I’d better get the banner ad running again, and do the upgrades I’d planned for friday

    Oh and put some batteries in my mouse… Looks like it has been overused ..

    • Jenny 25.1
      4 April 2010 at 11:02 pm

      It’s fascinating reading about your struggles Lynn. Great work, I must say. I only pray you don’t have to do it ever again. (or is this what you do in you day job, day in day out?)

      • lprent 25.1.1
        4 April 2010 at 11:46 pm

        Hell no. Being a permanent sysop would drive me nuts. Usually I figure out how to make the systems largely bullet-proof so I don’t have to deal with these issues. But we haven’t the resources at The Standard to do that.

        Mostly I write library level c++/c# code with a number of other languages. Sometimes as a member of a team of developers, sometimes as the team lead (which I generally prefer to avoid), sometimes as the development manager (which I really try to avoid). I like programming. I tend to specialize in GUI, architectural design, and my favorite – solving structural design problems – solving the unsolvable.

        Knowing how to get systems working is a incidental by-product and so has been the commercial web development (although I like writing webapps). Most of the web stuff has been done in my spare time.

        • Lanthanide 25.1.1.1
          5 April 2010 at 12:39 am

          What actually happened with the site? I haven’t seen any post or comment anywhere about the downtime. If I’ve missed it, please link it. I’ve gleaned that you’ve moved the site to a new server/hosting provider, presumably for bandwidth reasons?

          • lprent 25.1.1.1.1
            5 April 2010 at 1:15 am

            Still too busy to write one..

            Nope. There was something deadlocking up at the bottom of the system. The number of php5 processes went up massively as connections came in and weren’t serviced. It’d happened a few times over the last couple of months and I’d gotten around it by various tweaks and a hefty number of service restarts.

            This time it was so bad that the bots just looking at the maintenance screen (ie wordpress options database access only) were sufficient to cause a failure cascade.

            I initially thought it was hardware, but we eliminated that after the techs looked at the system. They also checked the other VPS’es to see of there was something hogging something. There was one contender, but after they eliminated that, I started looking for something in our configuration.

            Read all of the /var/log files looking for something. They showed the slowdown results but not the cause.

            The database worked fine when I hit it from several systems at the same time. Another wordpress site (Lyn’s doco site) on the same system ran fine with their limited traffic.

            So I started on our wordpress configuration. First I had to write a variant index.php so I could startup and shutdown the site access fast by renaming a file to prevent the cascade from making ssh unusable. Then reversing out the recent changes in wordpress one at a time (and wait 10-20 minutes to see if the bots could still kill at the maintenance screen, which they did). Then I reimplemented a new version of the site from scratch – still failed. Created a new site which worked. Loaded it with data and made it the standard, and it failed.

            Then I looked at the slow queries and eliminated them. Checked all of the mutexes and lock files in use in wordpress for permission issues. Did the same with the pid and sock files.

            Then looked at the LAMP stack and reloaded that in several configurations. Dropped apache from worker/fast cgi down to prefork, and eliminated most of the mods. Played with the MySQL and PHP configurations.

            Eventually was left with the OS. Spend considerable time backing everything off (and managed to lose a couple of months of images because (as I found out later) filezilla ftp doesn’t like symbolic linked directories, and hadn’t been picking then up in backups (back to ncftp push next time..)).

            Reimaged, reloaded the final test system on the installed pacakages and presto… It worked correctly. Bloody slow, but didn’t fall over.

            Then started putting everything back one bit at a time to get to the far more speedy system testing all of the way.

            Short answer is that I have no frigging idea what was actually going wrong. I suspect it was the actual VPS container because this one seems a lot different (more constrained) than the last one. I think that we hit it when wordpress shifted from largely using temp mem hash tables to temp disk tables and we hit something in the filesystem, and it was something about the size of data we were using (as the comments went up). But I can’t prove anything….. But when I grew the wordpress database to a similar size with random generated records, I got similar effects

            The only upside is that I’m happy with the software and services configuration. Now I know it wasn’t the cause, I can turn APC back on for PHP.

            But hardwarewise, I think that we need to shift to something like a dual Xeon level box to give us some headroom. Growth last month in page views was 9% greater than feb, and feb was our largest month ever. The bots and rss feeds are loving us to death… So are the unique visitors and page views per visitor..

            Time to grow the server again. But it is clear that we’re going to have to get a warm backup/load spreader as well for precisely this type of issue. It outgrew my old dual Opteron last year as the warmish backup. Time to start setting up database replication (which is fun) and looking at mod_proxy so we can tandem the system for peak loading…

            We knew this was coming up. That is why the advertising went on. It is getting past my ability to pay for the monthly running costs. Marty reckons that we’re really going to start to get some serious traffic this mid-term year, and election year will be crazy.. Plus I’d like to start a few sister sites.

            • Lanthanide 25.1.1.1.1.1
              5 April 2010 at 8:21 am

              Ouch, no wonder it took so long to fix.

              • mickysavage …
                5 April 2010 at 12:10 pm

                You should put a serious plan for financing together, sort of a “friends of the Standard” group. If you get a hundred people pledging $10 per month then you would have some cash coming in. I have no idea now much time or money it takes to keep the site going but by the look of your description concerning the remedial work required above it must be considerable.

                • lprent …
                  5 April 2010 at 12:33 pm

                  Normally it isn’t that bad. It is more the unpredictability that is a problem. It always seems to happen while I’m at work, which limits my remedial powers.

                  This time I was just lucky that the really bad one happened at easter. Makes you feel like there is some kind of divine intervention.

            • mickysavage 25.1.1.1.1.2
              5 April 2010 at 2:24 pm

              Have you thought about a “The Standard” T shirt? There are a few Labour conferences and meetings coming up where they could be sold and I am sure the Greens would do the same.

              I would buy a few!

  26. ianmac 26
    5 April 2010 at 12:24 pm

    As a superannuant I would be happy to pay $10 per month Micky Savage. When the Standard went down its abscence made me realise how important the site is. A terrible responsibility eh Iprent! 🙂

    • lprent 26.1
      5 April 2010 at 12:37 pm

      Oh yeah… Working on the upgrades for advertising now so we may have revenues when we have a bank account (trust starts on the new financial year).

      But there is always the donate button, but you have to have a paypal account or credit card. I’ll add the account number when we know what it is. Revenue is always useful 🙂

  27. RedLogix 27
    5 April 2010 at 12:26 pm

    Well done Lyn…..that’s your Easter fracked for all time. But the rest of us can do more than just express appreciation, Mickey’s idea is a good one and I’d for one would be prepared to back it.

Links to post

The server will be getting hardware changes this evening starting at 10pm NZDT.
The site will be off line for some hours.