Outage last night

Written By: - Date published: 8:19 am, November 8th, 2014 - 17 comments
Categories: admin, The Standard - Tags:

Dolphin Over World 2010The site had a strange problem with a database solid state drive last night while restarting after doing operating system upgrades. This morning it was completely offline and causing problems with the other drives in the array.

I took the entire database raid array offline this morning because I didn’t trust its integrity, shifted to a backup drive, and restored to the last good backup from 2300 last night.

There will have been some comments lost between 2300 and 0044 when the outage was reported.

This is the first time I have had a SSD fail on me, and it looks like it was just a connector problem. The behaviour of the raid was more of an issue, and one that I will be looking at tomorrow.

There was a planned hardware upgrade with a upgrade of my home server this weekend to a 8 core water cooled CPU and motherboard, but I will leave that for another evening. It is mainly there to allow me to get my workstation back for some game playing.

17 comments on “Outage last night”

  1. There I was thinking I had copped a ban for drunken waffling lol. Sounded like a bit of a headache, probably just as bad as mine this morning.

  2. Manuka AOR 2

    Really good it is back. I was starting to get anxiety/ withdrawal symptoms 😀

  3. Chooky 3

    us Chooks in the hen house are very concerned ….we think John Key Five Eyes and the ferrets are on to us….we will be lurking and moving incognito today

  4. It is more of a flex issue. The fail over system isn’t working. But the system I need to test it from is the same one that TS is running on.

    Get a second system to the same standard, workstation gets freed up, I can do the testing without taking TS out.

    Just waiting for a night when I can send a few hours after midnight.

  5. GregJ 5

    Whew – glad its working again. I live on the other side of the World – when The Standard ain’t up I begin to fear Armageddon is coming!! 😯

  6. greywarshark 6

    lprent
    I was trying to send this to you last night thinking you might have some time to kill.
    Wrong. Thanks for getting all going again.

    All the best with what sounds an interesting project Lyn. I had a look at the donation site for Lyn and saw that she received more than her hoped for amount so that is good.

    This is a link to a trip over India on a C90 motorcycle Lynn. You could get into the ambience of the country while Lyn is there though you are still at home. It is funny and fascinating. But you may have seen it.
    http://www.c90adventures.co.uk/news/76-india

  7. One Anonymous Bloke 7

    I have to ask: what games are you playing?

    • lprent 7.1

      Starting diablo 3 running under wine. Have been using some large new maps on homeworld2 in wine. 0ad occasionally.

      I don’t have time to play many games these days. Those that I do play have to be shortish and reasonably strategic or at least tactical. Then I can play them while I mull on something else and not get bored.

      • Draco T Bastard 7.1.1

        You may like Ancient Space then. Similar game play to Homeworld but not with the sense of vastness that Homeworld has as the graphics are used to enclose space (They really are fantastic graphics though). I’m not overly fond of the resource system as it comes across as artificial but do enjoy the game.

        Don’t know if it would work under WINE.

  8. infused 8

    let me guess. you have some shitty on board raid card? that’s your issue.

  9. Kiwiri - Raided of the Last Shark 9

    As far as I am concerned, The Standard is also, for me when I come online to look for material to read, The Sunshine.

  10. Instauration 10

    Lynn
    SSD promise much – but unless you pay mega for SAS, non-nearline you essentially have a device that is optimised for read, and DB writes will quickly erode the mtbf.
    Avoid RAID5 – it rarely provides an optimal r/w charachteristic for any DB.
    RAID10 – SAS – an easy life 🙂

    • lprent 10.1

      It depends what you are putting on it.

      Think about what it means on a blog database in reality rather than in mere theory. The database write usage is pretty minimal compared to the read simply because the numbers of writes in day are in the thousands, and the reads are massive. It is essentially a write once database because few records are deleted.

      A blog database needs to be optimized for read.

      My back of the envelope calcs indicate that I will have a write lifetime of >15 years. The SSD that failed had a connector problem. Probably because it had a crappy clipin. I tested it with 5 SATA cables and none of them locked in properly. Yet all of the other SSDs were fine clipping in.

Recent Comments

Recent Posts

  • Confirmation bias
    Something slightly deeper. Facebook is an out of control dangerous institution that neatly divides us up into our own tribes and lets us reinforce our beliefs with each other while at the same time throw rocks ...
    Confirmation bias
    1 hour ago
  • Andrew Little leads NZ delegation on global anti-terrorism taskforce
    Justice Minister Andrew Little leaves for the United States today to take part in a global task force that’s tackling terrorism and anti-money laundering. “I’m looking forward to leading the New Zealand delegation to the Financial Action Task Force (FATF) ...
    1 week ago
  • Third reading: Arms (Prohibited Firearms, Magazines, and Parts) Amendment Bill
    Mr Speaker We have travelled a long way in eight days, since the bill was read a first time. It has been a punishing schedule for MPs and submitters and public servants who have played a role in this process. ...
    2 weeks ago
  • Legal framework for gun buyback scheme announced
    Police Minister Stuart Nash has announced a legal framework for the gun buyback will be established as a first step towards determining the level of compensation. It will include compensation for high capacity magazines and parts. Mr Nash has outlined ...
    2 weeks ago
  • Second reading: Arms (Prohibited Firearms, Magazines and Parts) Amendment Bill
    Mr Speaker, it is Day 25 of the largest criminal investigation in New Zealand history. Not a day, or a moment, has been wasted as we respond to the atrocity that is testing us all. That is true also of ...
    2 weeks ago
  • First reading: Arms (Prohibited Firearms, Magazines and Parts) Amendment Bill
    Mr Speaker, as we meet today New Zealand is under a terror threat level of HIGH. As we meet today, Police are routinely carrying firearms, Bushmaster rifles and Glock pistols, in a significant departure from normal practice. As we meet ...
    3 weeks ago
  • NZ-China economic ties strengthened
    Economic ties between New Zealand and China are being strengthened with the successful negotiation of a new taxation treaty. The double tax agreement was signed by New Zealand’s Ambassador to China and by the Commissioner of the State Taxation Administration ...
    3 weeks ago
  • Tighter gun laws to enhance public safety
    Police Minister Stuart Nash has introduced legislation changing firearms laws to improve public safety following the Christchurch terror attacks. “Every semi-automatic weapon used in the terrorist attack will be banned,” Mr Nash says. “Owning a gun is a privilege not ...
    3 weeks ago