web analytics

Outage last night

Written By: - Date published: 8:19 am, November 8th, 2014 - 17 comments
Categories: admin, The Standard - Tags:

Dolphin Over World 2010The site had a strange problem with a database solid state drive last night while restarting after doing operating system upgrades. This morning it was completely offline and causing problems with the other drives in the array.

I took the entire database raid array offline this morning because I didn’t trust its integrity, shifted to a backup drive, and restored to the last good backup from 2300 last night.

There will have been some comments lost between 2300 and 0044 when the outage was reported.

This is the first time I have had a SSD fail on me, and it looks like it was just a connector problem. The behaviour of the raid was more of an issue, and one that I will be looking at tomorrow.

There was a planned hardware upgrade with a upgrade of my home server this weekend to a 8 core water cooled CPU and motherboard, but I will leave that for another evening. It is mainly there to allow me to get my workstation back for some game playing.

17 comments on “Outage last night ”

  1. Skinny 1

    There I was thinking I had copped a ban for drunken waffling lol. Sounded like a bit of a headache, probably just as bad as mine this morning.

  2. Manuka AOR 2

    Really good it is back. I was starting to get anxiety/ withdrawal symptoms 😀

  3. Chooky 3

    us Chooks in the hen house are very concerned ….we think John Key Five Eyes and the ferrets are on to us….we will be lurking and moving incognito today

  4. lprent 4

    It is more of a flex issue. The fail over system isn’t working. But the system I need to test it from is the same one that TS is running on.

    Get a second system to the same standard, workstation gets freed up, I can do the testing without taking TS out.

    Just waiting for a night when I can send a few hours after midnight.

  5. GregJ 5

    Whew – glad its working again. I live on the other side of the World – when The Standard ain’t up I begin to fear Armageddon is coming!! 😯

  6. greywarshark 6

    I was trying to send this to you last night thinking you might have some time to kill.
    Wrong. Thanks for getting all going again.

    All the best with what sounds an interesting project Lyn. I had a look at the donation site for Lyn and saw that she received more than her hoped for amount so that is good.

    This is a link to a trip over India on a C90 motorcycle Lynn. You could get into the ambience of the country while Lyn is there though you are still at home. It is funny and fascinating. But you may have seen it.

  7. One Anonymous Bloke 7

    I have to ask: what games are you playing?

    • lprent 7.1

      Starting diablo 3 running under wine. Have been using some large new maps on homeworld2 in wine. 0ad occasionally.

      I don’t have time to play many games these days. Those that I do play have to be shortish and reasonably strategic or at least tactical. Then I can play them while I mull on something else and not get bored.

      • Draco T Bastard 7.1.1

        You may like Ancient Space then. Similar game play to Homeworld but not with the sense of vastness that Homeworld has as the graphics are used to enclose space (They really are fantastic graphics though). I’m not overly fond of the resource system as it comes across as artificial but do enjoy the game.

        Don’t know if it would work under WINE.

  8. infused 8

    let me guess. you have some shitty on board raid card? that’s your issue.

    • lprent 8.1

      Why would I use one of those. They keep dying.

      Nope this was mdadm. The problem was with the shitty data connector on the SSD.

  9. Kiwiri - Raided of the Last Shark 9

    As far as I am concerned, The Standard is also, for me when I come online to look for material to read, The Sunshine.

  10. Instauration 10

    SSD promise much – but unless you pay mega for SAS, non-nearline you essentially have a device that is optimised for read, and DB writes will quickly erode the mtbf.
    Avoid RAID5 – it rarely provides an optimal r/w charachteristic for any DB.
    RAID10 – SAS – an easy life 🙂

    • lprent 10.1

      It depends what you are putting on it.

      Think about what it means on a blog database in reality rather than in mere theory. The database write usage is pretty minimal compared to the read simply because the numbers of writes in day are in the thousands, and the reads are massive. It is essentially a write once database because few records are deleted.

      A blog database needs to be optimized for read.

      My back of the envelope calcs indicate that I will have a write lifetime of >15 years. The SSD that failed had a connector problem. Probably because it had a crappy clipin. I tested it with 5 SATA cables and none of them locked in properly. Yet all of the other SSDs were fine clipping in.

Recent Comments

Recent Posts

  • NZ announces a third P-3 deployment in support of UN sanctions
    The Government has deployed a Royal New Zealand Air Force P-3K2 Orion (P-3) maritime patrol aircraft to support the implementation of United Nations Security Council (UNSC) resolutions imposing sanctions against North Korea, announced Minister of Foreign Affairs Winston Peters and Minister of Defence Ron Mark. “New Zealand has long supported ...
    BeehiveBy beehive.govt.nz
    4 days ago
  • Pacific trade and development agreement a reality
    Pacific regional trade and development agreement PACER Plus will enter into force in 60 days now that the required eight countries have ratified it. Trade and Export Growth Minister David Parker welcomed the announcement that the Cook Islands is the eighth nation to ratify this landmark agreement. “The agreement represents ...
    BeehiveBy beehive.govt.nz
    1 week ago
  • Securing a pipeline of teachers
    The Government is changing its approach to teacher recruitment as COVID-19 travel restrictions continue, by boosting a range of initiatives to get more Kiwis into teaching. “When we came into Government, we were faced with a teacher supply crisis,” Education Minister Chris Hipkins said. “Over the past three years, we ...
    BeehiveBy beehive.govt.nz
    1 week ago
  • Border exceptions for a small number of international students with visas
    The Government has established a new category that will allow 250 international PhD and postgraduate students to enter New Zealand and continue their studies, in the latest set of border exceptions. “The health, safety and wellbeing of people in New Zealand remains the Government’s top priority. Tight border restrictions remain ...
    BeehiveBy beehive.govt.nz
    2 weeks ago
  • First COVID-19 vaccine purchase agreement signed
    The Government has signed an agreement to purchase 1.5 million COVID-19 vaccines – enough for 750,000 people – from Pfizer and BioNTech, subject to the vaccine successfully completing all clinical trials and passing regulatory approvals in New Zealand, say Research, Science and Innovation Minister Megan Woods and Health Minister Chris Hipkins. ...
    BeehiveBy beehive.govt.nz
    2 weeks ago
  • International statement – End-to-end encryption and public safety
    We, the undersigned, support strong encryption, which plays a crucial role in protecting personal data, privacy, intellectual property, trade secrets and cyber security.  It also serves a vital purpose in repressive states to protect journalists, human rights defenders and other vulnerable people, as stated in the 2017 resolution of the ...
    BeehiveBy beehive.govt.nz
    2 weeks ago
  • Ministry of Defence Biodefence Assessment released
    The Ministry of Defence has today released a Defence Assessment examining Defence’s role across the spectrum of biological hazards and threats facing New Zealand. Biodefence: Preparing for a New Era of Biological Hazards and Threats looks at how the NZDF supports other agencies’ biodefence activities, and considers the context of ...
    BeehiveBy beehive.govt.nz
    2 weeks ago
  • New Approaches to Economic Challenges: Confronting Planetary Emergencies: OECD 9 October 2020
    New Approaches to Economic Challenges: Confronting Planetary Emergencies: OECD 9 October 2020 Hon David Parker’s response following Thomas Piketty and Esther Duflo. Good morning, good afternoon, and good evening, wherever in the world you might be. I first acknowledge the excellent thought provoking speeches of Thomas Piketty and Esther ...
    BeehiveBy beehive.govt.nz
    2 weeks ago
  • Kaipara Moana restoration takes next step
    A Memorandum of Understanding has been signed today at Waihāua Marae between the Crown, local iwi and councils to protect, restore and enhance the mauri of Kaipara Moana in Northland. Environment Minister David Parker signed the document on behalf of the Crown along with representatives from Ngā Maunga Whakahī, Ngāti ...
    BeehiveBy beehive.govt.nz
    2 weeks ago
  • New Zealand and Uruguay unite on reducing livestock production emissions
    Agriculture Minister Damien O’Connor and Uruguayan Minister of Livestock, Agriculture and Fisheries Carlos María Uriarte have welcomed the launch of a three-year project that will underpin sustainable livestock production in Uruguay, Argentina, and Costa Rica.  The project called ‘Innovation for pasture management’ is led by Uruguay’s National Institute of Agricultural ...
    BeehiveBy beehive.govt.nz
    2 weeks ago
  • 3100 jobs created through marae upgrades
    Hundreds of marae throughout the country will be upgraded through investments from the Provincial Growth Fund’s refocused post COVID-19 funding to create jobs and put money into the pockets of local tradespeople and businesses, Regional Economic Development Minister Shane Jones and Māori Development Minister Nanaia Mahuta have announced. “A total ...
    BeehiveBy beehive.govt.nz
    2 weeks ago
  • Health volunteers recognised in annual awards
    Health Minister Chris Hipkins has announced 9 teams and 14 individuals are the recipients of this year’s Minister of Health Volunteer Awards.  “The health volunteer awards celebrate and recognise the thousands of dedicated health sector volunteers who give many hours of their time to help other New Zealanders,” Mr Hipkins ...
    BeehiveBy beehive.govt.nz
    2 weeks ago
  • Community COVID-19 Fund supports Pacific recovery
    The Minister for Pacific Peoples, Aupito William Sio says a total of 264 groups and individuals have successfully applied for the Pacific Aotearoa Community COVID-19 Recovery Fund, that will support Pacific communities drive their own COVID-19 recovery strategies, initiatives, and actions. “I am keen to see this Fund support Pacific ...
    BeehiveBy beehive.govt.nz
    2 weeks ago
  • Community benefits from Māori apprenticeships
    Up to 50 Māori apprentices in Wellington will receive paid training to build houses for their local communities, thanks to a $2.75 million investment from the Māori Trades and Training Fund, announced Employment Minister Willie Jackson today. “This funding will enable Ngāti Toa Rangatira Incorporated to provide its Ngā Kaimahi ...
    BeehiveBy beehive.govt.nz
    2 weeks ago
  • Training fund supports Māori jobseekers
    Rapidly growing sectors will benefit from a $990,000 Māori Trades and Training Fund investment which will see Wellington jobseekers supported into work, announced Employment Minister Willie Jackson today. “This funding will enable Sapphire Consultants Ltd. to help up to 45 Māori jobseekers into paid training initiatives over two years through ...
    BeehiveBy beehive.govt.nz
    2 weeks ago
  • Ruakura Inland Port development vital infrastructure for Waikato
    The Government is investing $40 million to develop an inland port at Ruakura which will become a freight super-hub and a future business, research and residential development for the Waikato, Urban Development and Transport Minister Phil Twyford, and Māori Development Minister Nanaia Mahuta announced today. The funding has been has ...
    BeehiveBy beehive.govt.nz
    2 weeks ago
  • Appointments made to Defence Expert Review Group
    Defence Minister Ron Mark announced today the establishment of an Expert Review Group to review a number of aspects of the New Zealand Defence Force’s (NZDF) structure, information management and record-keeping processes.  The Expert Review Group’s work arises out of the first recommendation from the Report of the Government’s Inquiry ...
    BeehiveBy beehive.govt.nz
    2 weeks ago
  • No active community cases of COVID-19
    There are no active community cases of COVID-19 remaining in the country after the last people from the recent outbreak have recovered from the virus, Health Minister Chris Hipkins said today. “This is a big milestone. New Zealanders have once again through their collective actions squashed the virus. The systems ...
    BeehiveBy beehive.govt.nz
    2 weeks ago
  • Clean energy upgrade for more public buildings
    More public buildings will be supported by the Government to upgrade to run on clean energy, the Minister for Climate Change James Shaw announced today. Minister Shaw announced that Lincoln and Auckland universities will receive support through the Clean-Powered Public Service Fund to replace fossil fuel boilers. Southern, Taranaki, and ...
    BeehiveBy beehive.govt.nz
    2 weeks ago
  • Schools back donations scheme for the second year
    More schools have opted in to the donations scheme for 2021, compared to 2020 when the scheme was introduced. “The families of more than 447,000 students will be better off next year, with 94% of eligible schools and kura opting into the scheme,” Education Minister Chris Hipkins said. “This is ...
    BeehiveBy beehive.govt.nz
    3 weeks ago
  • Ruapehu cycle trails gets PGF boost
    The spectacular Mountains to Sea cycle trail in Ruapehu District will receive $4.6 million in funding from the Provincial Growth Fund for two additional trails, Regional Economic Development Minister Shane Jones announced today. “This is an exciting development for the local community, and one that will provide significant economic opportunities ...
    BeehiveBy beehive.govt.nz
    3 weeks ago