Outage

Written By: - Date published: 9:56 am, February 1st, 2012 - 33 comments
Categories: uncategorized - Tags:

There was some maintenance done on the main NZ server last night.

All looked ok when I went to sleep at about 6am, but appears to have started having problems from about 8:30 onwards with intermittently waiting on some network resource.

Problem looks external to my server and the server’s support staff are chasing it at present. It is intermittent so hopefully this post will display.

If it persists for too long, then I’ll switch to a backup server. But I suspect they will find it shortly. In the meantime I’d better head off to work.

 

33 comments on “Outage”

  1. Lanthanide 1

    Reloading pages generally seems to work after a couple of tries.

    • lprent 1.1

      Yeah. top shows the system going into wait (over 90%) and just sitting there for up to a minute, and the suddenly releasing.

      But the network system just stopped completely just then and looked like it restarted.

      Ummm started ok, but looks like we are back to waits

      • lprent 1.1.1

        Looks like the techs have found the system bit that has a problem though…

        • lprent 1.1.1.1

          Mind you, anything has to be better than the day in 2002 (?) that a contractor in the US manager to hammer a metal pole through the main underground power line to the hosting centre that our main server was housed at. Turned out the backup power system fried itself while trying to turn on.

          Or the earlier incident when a hurricane went through Florida and took out power to a hosting centre there.

  2. felix 2

    Gosman crashed the server again?

    • fender 2.1

      lol…way to go gosman
      bombard the site with repative emptyheaded crap like comparing farmland to trademe and trying to impress with his superior knowledge of kindergarden economics, playdough stuff.

    • lprent 2.2

      I don’t think that he has ever managed to crash the server. But computers aren’t too fussed about quality, it is the quantity that they tend to notice…

      All-time comments.
      1: Colonial Viper (13,071)
      2: Draco T Bastard (9,451)
      3: Felix (9,326)
      4: lprent (8,759)
      5. Pascal’s bookie (6,399)

      Out of 371,487 approved comments, of which you contributed 2.51% and CV has did 3.52%

      31: Gosman (2,220)

      Now at this point I’d like to emphasize for any young kiwi’s that it is NOT a contest. If I see any wannabe record breakers they will wind up like

      164: kiwiteen123 (438)

      😈

      • McFlock 2.2.1

        League tables!
        That will help the quality of comments appearing on The Standard! Maybe the administrators could also enter into a charter partnership with a business to improve its performance?

        🙂

        • lprent 2.2.1.1

          No way. I pull them out occasionally to stir people with. However I learnt my lesson from kiwiteen who treated them as an exercise in the unimportance of quantity.

          BTW:
          22. mcflock (2,818)

        • Colonial Viper 2.2.1.2

          I’d back league tables. Currently I’m well ahead of the league – by quantity anyways 😈

          • fender 2.2.1.2.1

            IMO your comments are of a consistently high quality CV
            When are you running for parliament?

  3. lprent 3

    Ok fixed..

    Looks like there was some kind of a configuration problem in apache2.2-common. Pulled the whole of it out and reinstalled. Then fixed the enabled modules. Ummm still getting highish waits

  4. lprent 4

    Ok. Dropped the fastcgi which is where the problem seemed to be happening.

    Site will be somewhat slower until I have a look at it tonight.

    • lprent 4.1

      At present the best guess is that sometime in the past there was a bad upgrade of apache2 which was waiting for restart that I never gave it. There was some moaning in the apache groups back before the election, and I haven’t taken the server down since at least October.

      The maintenance reboot triggered the problem on fastcgi which showed up when the load started winding up for the morning.

      I’ll probably take the whole site down on the weekend and completely reinstall apache2. But I’ll run a clean test on a different machine to see if I can isolate the problem in fastCGI. There is no way that it should start getting 90+% wait time.

      For the moment, I will take it off default apache process mode this evening and move to one of the threaded systems.

      But now I’d better get back to these interprocess named semaphores. Next release is due shortly, and this is the stuff I get paid for 🙂

  5. lprent 5

    The only thing I can see that is a problem is the very very large comment areas. The one that is at almost 600 comments looks like it is giving up before it all sent.

    I’ll (finally) put my fix in later.

  6. Jackal 6

    I don’t seem to be able to log in. All the links including the homepage have been downloading a small document saying:

  7. Jackal 7

    <?php
    /**
    * Front to the WordPress application. This file doesn't do anything, but loads
    * wp-blog-header.php which does and tells WordPress to load the theme.
    *
    * @package WordPress
    */

    /**
    * Tells WordPress to load the WordPress theme and output it.
    *
    * @var bool
    */
    define('WP_USE_THEMES', true);

    /** Loads the WordPress Environment and Template */
    require('./wp-blog-header.php');
    ?

    • lprent 7.1

      Just logged in on my three test identities.

      Try flushing your browser cache. It probably caught a php fragment in it while I was purging and reinstalling apache2-mod-php5.

  8. lprent 8

    Testing old version of ajax edit

  9. The Voice of Reason 9

    Cheers for all the hard work, LP. I’ve got no idea what it is you do to make this site function, might as well be voodoo for all I know, but the results are terrific!
     
    If you compare The Standard with KB, the differences are obvious. KB still looks like it was designed as a school project some time in the early nineties and the comments’ layout is just a context free spew down the page. Actually, so are most of the comments, so perhaps that bit is deliberate!
     
    Anyhoo, thanks again.

    • lprent 9.1

      I get rather slack about it in summer, mostly because my office at home is a bit cramped and way way to hot with a few computers running (and I haven’t been able to squeeze in the air-cond unit (yet)).. And last year I was concerned with other matters during winter – mostly the balance between cigarettes and weight.

      But the layout is mostly from mummybot. I just tweak it for functionality. But I do have to do the tweaks now. If only the weather would turn to a nice wholesome programmer friendly rain and sleet…..

  10. ropata 10

    I thought this news may have had something to do with it … now is a good time to patch the system

    http://www.theinquirer.net/inquirer/news/2142701/hundreds-wordpress-web-sites-hacked

    • lprent 10.1

      Nope. But it is the reason that I tend to upgrade wordpress, plugins, and security updates pretty fast usually within a few days of release after I try them out on the test system.

  11. taxicab 11

    Have you tried to access either of the following today “norightturn , tumeke , maui street ” all blocked by norton quite a coincidence !!

    • lprent 11.1

      That is in that curious operating system called windoze isn’t it?

      However all three of those sites have the words .blogspot.com after it. I suspect that therein lies the coincidence….

  12. taxicab 12

    sites back up now

  13. Drongo 13

    Odgers paid some nerds in Hong Kong to hack in to grab commenters’ IP addresses. She has them now and is emailing them to Slater at the minute. Things should settle down soon.

    • Colonial Viper 13.1

      Nice thing is we can pay those same nerds to counterhack her systems. Should be hilarious.

  14. felix 14

    Hey Lynn, I’ve been having a weird issue the last day or so. If I click on a link in the sidebar comments box, it loads the post instead of the specific comment. However the comment URL does appear in the address bar of the browser and on reload it brings up the comment.

    Possibly related to the other issues? Using XP & Firefox.

Leave a Comment

Show Tags

Recent Comments

Recent Posts

  • Nats’ housing policy fails to keep pace with population growth
    Auckland got less than half the new houses it needed in the past year to keep up with record population growth, says Labour’s Housing spokesperson Phil Twyford. ...
    19 hours ago
  • Urgent action needed on dirty rivers
    The Our Fresh Water Environment 2017 report re-confirms that we need urgent action to clean up our rivers. Meanwhile, National is standing by as our rivers get even more polluted, says Labour’s Environment spokesperson David Parker. “This report is yet ...
    2 days ago
  • Where there’s smoke and mirrors, there’s Steven Joyce
    Steven Joyce’s much vaunted pre-Budget speech is simply an underwhelming response to the infrastructure deficit National has created, says Labour’s Finance Spokesperson Grant Robertson. “Steven Joyce has belatedly come to the realisation that everyone else has a long time ago, ...
    2 days ago
  • Time to stamp out cold, mouldy rentals
    New figures show a small number of landlords are letting down the sector by renting cold, mouldy rentals. These houses need to be brought up to a decent standard for people to live in by Andrew Little’s Healthy Homes Bill, ...
    3 days ago
  • Time for fresh approach on immigration
    Latest figures showing another record year for immigration underlines the need for an urgent rethink on how this country can continue to absorb so many people, says Labour Leader Andrew Little. “New Zealand needs immigrants and is all the better ...
    3 days ago
  • Bring back the Mental Health Commission
    The People’s Mental Health Review is a much needed wake up call for the Government on mental health, says the Leader of the Opposition Andrew Little.   “I applaud their proposal to restore a Mental Health Commission and their call for ...
    5 days ago
  • And the band played on…
    Making Amy Adams the Housing Minister five months out from the election is just the orchestra playing on as National’s Titanic housing crisis slips below the waves – along with the hopes and dreams of countless Kiwi families, says Labour’s ...
    5 days ago
  • Hotel no place for children in care
    ...
    1 week ago
  • Maybe not, Minister? Nick Smith’s housing measure suppressed
    Sir Humphrey: Minister, remember the Housing Affordability Measure work you asked us to prepare back in 2012? Well, it’s ready now.Minister Smith: Oh goodie, what does it say?Sir Humphrey: Nothing.Minister Smith: Nothing?Sir Humphrey: Well, sir, you asked us to prepare ...
    1 week ago
  • Inflation data shows many New Zealanders are worse off under National
    The latest inflation data from Statistics New Zealand shows that too many New Zealanders are now worse off under the National Government, said Labour’s Finance Spokesperson Grant Robertson “Consumer Price Inflation (CPI) is now running at 2.2 per cent, and ...
    1 week ago
  • Another emergency housing grant blow out
      Emergency housing grants data released today show another blow out in spending on putting homeless people up in motels, says Labour’s Housing spokesperson Phil Twyford.   ...
    1 week ago
  • Families struggle as hardship grants increase
    The considerable increase in hardship grants shows that more and more Kiwi families are struggling to put food on the table and pay for basic schooling, says Labour’s Social Development spokesperson Carmel Sepuloni. ...
    1 week ago
  • More tinkering, no leadership from Nats on immigration
    National’s latest tinkering with the immigration system is another attempt to create the appearance of action without actually doing anything meaningful, says Leader of the Opposition Andrew Little. ...
    1 week ago
  • Suicide figures make for grim reading
    The 506 suspected suicides of Kiwis who have been in the care of mental health services in the last four years show that these services are under severe stress, says Labour’s Health spokesperson David Clark.   “If you do the ...
    2 weeks ago
  • Pay equity deal a victory for determination and unions
    The pay equity settlement revealed today for around 55,000 low-paid workers was hard-won by a determined Kristine Bartlett backed by her union, up against sheer Government resistance to paying Kiwis their fair share, says Labour Leader Andrew Little. “Labour welcomes ...
    2 weeks ago
  • DHB’s forced to make tough choices
    The Minister of Health today admitted that the country’s District Health Boards were having to spend more than their ring fenced expenditure on Mental Health, says Labour’s Health spokesperson David Clark.   “The situation is serious with Capital and Coast ...
    2 weeks ago
  • Nats break emergency housing pledge – deliver just five more places
    Despite National’s promises of 2,200 emergency housing beds, just 737 were provided in the March Quarter, an increase of only five from six months earlier, says Labour’s Housing spokesperson Phil Twyford. ...
    2 weeks ago
  • Research underlines need for KiwiBuild
    New research showing the social and fiscal benefits of homeownership underlines the need for a massive government-backed building programme like KiwiBuild, says Labour’s housing spokesperson Phil Twyford. ...
    2 weeks ago
  • Social data security review too little, too late
    The independent review into the Ministry of Social Development’s individual client level data IT system is too little, too late, says Labour’s Social Development spokesperson Carmel Sepuloni. “The Minister of Social Development has finally seen some sense and called for ...
    2 weeks ago
  • More questions raised on CERA conflicts
    With the admission that three more former CERA staff members are under suspicion of not appropriately managing conflicts of interest related to the Canterbury rebuild, it’s imperative that CERA’s successor organisation Ōtākaro fronts up to Parliamentary questions, says Labour’s Canterbury ...
    2 weeks ago
  • Labour to tackle Hutt housing crisis
    Labour will build a mix of 400 state houses and affordable KiwiBuild homes in the Hutt Valley in its first term in government to tackle the housing crisis there, says Leader of the Opposition Andrew Little. “Housing in the Hutt ...
    2 weeks ago
  • Farewell to John Clarke
    This wonderfully talented man has been claimed by Australia, but how I remember John Clarke is as a young Wellington actor who performed satirical pieces in a show called “Knickers” at Downstage Theatre. The show featured other future luminaries like ...
    GreensBy Catherine Delahunty
    3 weeks ago
  • Valedictory Speech
    Te papa pounamu Aotearoa NZ Karanga karanga karanga; Nga tupuna Haere haere haere; Te kahui ora te korowai o tenei whare; E tu e tu ... tutahi tonu Ki a koutou oku hoa mahi ki Te Kawanatanga; Noho mai noho ...
    3 weeks ago
  • Buck stops with Gerry Brownlee
    The fact that the State Services Commission has referred the CERA conflict of interest issue to the Serious Fraud Office is a positive move, but one that raises serious questions about the Government’s oversight of the rebuild, says Labour Canterbury ...
    3 weeks ago
  • Teachers deserve a democratic Education Council
    Teachers around New Zealand reeling from the news that their registration fees could more than double will be even angrier that the National Government has removed their ability to have any say about who sits on the Council that sets ...
    3 weeks ago
  • Free trade backers are simply out of touch
    Are the backers of free trade out of touch with public opinion? This was the question asked when the Chartered Accountants launched their Future of Trade study. I was astonished by the answer in a room of free trade enthusiasts ...
    GreensBy Barry Coates
    3 weeks ago
  • John Clarke aka Fred Dagg will be missed by all Kiwis
    The man who revolutionised comedy on both sides of the Tasman, John Clarke, will be sadly missed by Kiwis and Aussies alike, says the Leader of the Opposition Andrew Little.   “I grew up with Fred Dagg and I am ...
    3 weeks ago
  • Labour’s modern approach to monetary policy
    A commitment to full employment and a more transparent process to provide market certainty are the hallmarks of Labour’s proposals for a new approach to monetary policy, says Labour’s Finance spokesperson Grant Robertson. ...
    3 weeks ago
  • Greens back Labour’s plan for monetary policy reform
    Labour plans to change the way we do monetary policy in New Zealand and the Green Party supports them fully. We’re now of a single mind on this. Labour will move away from our reliance on a single, unelected person ...
    GreensBy robert.ashe
    3 weeks ago
  • Greens back Labour’s monetary policy reform
    Labour plans to change the way we do monetary policy in New Zealand and the Green Party supports them fully. We’re now of a single mind on this. Labour will move away from our reliance on a single, unelected person ...
    GreensBy James Shaw
    3 weeks ago
  • Govt drops ball on Masters Games housing squeeze
    Families currently living in emergency accommodation face being forced out onto the street as motel accommodation in Auckland is filled up by contestants and visitors of the World Masters Games in coming weeks, says Labours social development spokesperson Carmel Sepuloni. ...
    3 weeks ago
  • State inquiry for Nga Morehu – The Survivors of State Abuse
    The Prime Minister must show humanitarian leadership and launch an independent inquiry into historic claims of abuse of children who were in State care, says Labour’s Deputy Leader Jacinda Ardern. ...
    3 weeks ago