web analytics

Outage last night

Written By: - Date published: 8:19 am, November 8th, 2014 - 17 comments
Categories: admin, The Standard - Tags:

Dolphin Over World 2010The site had a strange problem with a database solid state drive last night while restarting after doing operating system upgrades. This morning it was completely offline and causing problems with the other drives in the array.

I took the entire database raid array offline this morning because I didn’t trust its integrity, shifted to a backup drive, and restored to the last good backup from 2300 last night.

There will have been some comments lost between 2300 and 0044 when the outage was reported.

This is the first time I have had a SSD fail on me, and it looks like it was just a connector problem. The behaviour of the raid was more of an issue, and one that I will be looking at tomorrow.

There was a planned hardware upgrade with a upgrade of my home server this weekend to a 8 core water cooled CPU and motherboard, but I will leave that for another evening. It is mainly there to allow me to get my workstation back for some game playing.

17 comments on “Outage last night”

  1. Skinny 1

    There I was thinking I had copped a ban for drunken waffling lol. Sounded like a bit of a headache, probably just as bad as mine this morning.

  2. Manuka AOR 2

    Really good it is back. I was starting to get anxiety/ withdrawal symptoms 😀

  3. Chooky 3

    us Chooks in the hen house are very concerned ….we think John Key Five Eyes and the ferrets are on to us….we will be lurking and moving incognito today

  4. lprent 4

    It is more of a flex issue. The fail over system isn’t working. But the system I need to test it from is the same one that TS is running on.

    Get a second system to the same standard, workstation gets freed up, I can do the testing without taking TS out.

    Just waiting for a night when I can send a few hours after midnight.

  5. GregJ 5

    Whew – glad its working again. I live on the other side of the World – when The Standard ain’t up I begin to fear Armageddon is coming!! 😯

  6. greywarshark 6

    lprent
    I was trying to send this to you last night thinking you might have some time to kill.
    Wrong. Thanks for getting all going again.

    All the best with what sounds an interesting project Lyn. I had a look at the donation site for Lyn and saw that she received more than her hoped for amount so that is good.

    This is a link to a trip over India on a C90 motorcycle Lynn. You could get into the ambience of the country while Lyn is there though you are still at home. It is funny and fascinating. But you may have seen it.
    http://www.c90adventures.co.uk/news/76-india

  7. One Anonymous Bloke 7

    I have to ask: what games are you playing?

    • lprent 7.1

      Starting diablo 3 running under wine. Have been using some large new maps on homeworld2 in wine. 0ad occasionally.

      I don’t have time to play many games these days. Those that I do play have to be shortish and reasonably strategic or at least tactical. Then I can play them while I mull on something else and not get bored.

      • Draco T Bastard 7.1.1

        You may like Ancient Space then. Similar game play to Homeworld but not with the sense of vastness that Homeworld has as the graphics are used to enclose space (They really are fantastic graphics though). I’m not overly fond of the resource system as it comes across as artificial but do enjoy the game.

        Don’t know if it would work under WINE.

  8. infused 8

    let me guess. you have some shitty on board raid card? that’s your issue.

    • lprent 8.1

      Why would I use one of those. They keep dying.

      Nope this was mdadm. The problem was with the shitty data connector on the SSD.

  9. Kiwiri - Raided of the Last Shark 9

    As far as I am concerned, The Standard is also, for me when I come online to look for material to read, The Sunshine.

  10. Instauration 10

    Lynn
    SSD promise much – but unless you pay mega for SAS, non-nearline you essentially have a device that is optimised for read, and DB writes will quickly erode the mtbf.
    Avoid RAID5 – it rarely provides an optimal r/w charachteristic for any DB.
    RAID10 – SAS – an easy life 🙂

    • lprent 10.1

      It depends what you are putting on it.

      Think about what it means on a blog database in reality rather than in mere theory. The database write usage is pretty minimal compared to the read simply because the numbers of writes in day are in the thousands, and the reads are massive. It is essentially a write once database because few records are deleted.

      A blog database needs to be optimized for read.

      My back of the envelope calcs indicate that I will have a write lifetime of >15 years. The SSD that failed had a connector problem. Probably because it had a crappy clipin. I tested it with 5 SATA cables and none of them locked in properly. Yet all of the other SSDs were fine clipping in.

Recent Comments

Recent Posts

  • Provincial Growth Fund to fund Waipukurau cultural development and tourism
    The Ngā Ara Tipuna -  Waipukurau Pā Site Interpretation project will receive $2.798 million from the Provincial Growth Fund to create an authentic cultural tourism experience, Under-Secretary for Regional Economic Development Fletcher Tabuteau announced today “The project will inform visitors about the history of six pā sites in Waipukurau with a combination ...
    BeehiveBy beehive.govt.nz
    2 hours ago
  • 21 new judges boost diversity, improve access to justice
    Twenty-one new District Court judges have been appointed in a move that will improve access to justice and boost diversity on the bench. The new judges include replacements for retirements and 10 new positions. Attorney-General David Parker today announced the 14 judges who can immediately be named, with the remainder ...
    BeehiveBy beehive.govt.nz
    2 hours ago
  • Puhinui to Auckland Airport in 10 minutes
    Aucklanders are another step closer to getting rapid transit to the airport, with the start of construction to upgrade State Highway 20B to the airport, Transport Minister Phil Twyford announced today. SH20B will be upgraded with additional lanes in each direction, dedicated to bus and high-occupancy vehicles between Pukaki Creek ...
    BeehiveBy beehive.govt.nz
    5 hours ago
  • Advancing New Zealand’s trade agenda focus of Europe meetings
    World Trade Organisation reform, agricultural trade and a free trade agreement with the United Kingdom will be the focus of Minister for Trade and Export Growth David Parker’s visit to Europe this week. David Parker leaves on Tuesday for a series of meetings in the UK and Switzerland that aim ...
    BeehiveBy beehive.govt.nz
    1 day ago
  • Minister of Defence to visit counterparts in US and Canada
    The Minister of Defence, Ron Mark, departed today for the United States and Canada where he will meet with his counterparts.  While in Canada Minister Mark will meet with his counterpart, Minister of National Defence Harjit Sajjan.  “New Zealand and Canada are close friends, and share an instinctive like-mindedness on ...
    BeehiveBy beehive.govt.nz
    1 day ago
  • Government to deliver family carers $2000 pay rise, expand scheme to spouses this year
    The Coalition Government is delivering this year the changes to Funded Family Care the disability sector has long-asked for, says Associate Minister of Health Jenny Salesa. “Today we are announcing the details of our big changes to Funded Family Care, including an annual average pay boost of $2,246.40 for funded ...
    BeehiveBy beehive.govt.nz
    2 days ago
  • Ko te reo kua mū: Piri Sciascia
    Minister for Māori Development Nanaia Mahuta joins te ao Māori in their sorrow as they learn of the loss of one of the great orators and spokespersons of a generation – Piri Sciascia.  “The son of Pōrangahau was a staunch advocate for Māori development and served his people for over ...
    BeehiveBy beehive.govt.nz
    3 days ago
  • Minister opens new ecosanctuary at Cape Farewell
    A new ecosanctuary with a predator proof fence on Golden Bay’s Cape Farewell, which will restore a safe home for sea birds, rare native plants, giant snails, and geckos, was officially opened today by the Minister of Conservation Eugenie Sage. “There has been a fantastic community effort supported by the ...
    BeehiveBy beehive.govt.nz
    3 days ago
  • Pacific partners work together to provide additional support to Australia
    The NZDF continues to support the Australian Defence Force (ADF) as it battles fires in Victoria and New South Wales, including by transporting Republic of Fiji Military engineers from Nadi to Australia, announced Defence Minister Ron Mark. On Saturday morning a NZDF Boeing 757 will depart New Zealand to uplift ...
    BeehiveBy beehive.govt.nz
    4 days ago
  • Kaikōura $10.88 million boost in tourism & business
    The Provincial Growth Fund (PGF) is investing $10.88 million to boost business and tourism opportunities in Kaikōura, Parliamentary Undersecretary for Regional Economic Development, Fletcher Tabuteau announced today. As part of the Kaikōura Marina Development Programme, the following two projects will receive PGF funding: A $9.88 million investment to begin the ...
    BeehiveBy beehive.govt.nz
    4 days ago
  • Govt accounts in surplus, debt remains low
    The Government’s books are in good shape with the accounts in surplus and expenses close to forecast, Finance Minister Grant Robertson says. The Treasury today released the Crown accounts for the five months to November. The operating balance before gains and losses (OBEGAL) was above forecast by $0.7 billion resulting ...
    BeehiveBy beehive.govt.nz
    4 days ago
  • Auckland focus for first Police graduation of 2020
    The number of Police on the Auckland frontline is increasing with the graduation today of a special locally-trained wing of new constables. Police Minister Stuart Nash says the graduation of eighteen officers from Recruit Wing 333-5 means that more than 1900 new Police have been deployed since the Coalition Government ...
    BeehiveBy beehive.govt.nz
    5 days ago
  • Wairarapa gets $7.11m PGF water boost
    The Provincial Growth Fund is putting $7.11 million into creating a sustainable water supply for Wairarapa, Parliamentary Under-Secretary for Regional Economic Development Fletcher Tabuteau announced today. The following two projects will receive Provincial Growth Fund (PGF) funding: A $7 million investment in Wairarapa Water Limited for the pre-construction development of ...
    BeehiveBy beehive.govt.nz
    6 days ago
  • Progress with new Police station in Mahia
    Community safety and crime prevention in the East Coast community of Mahia has moved forward with the opening of a new Police station to serve the growing coastal settlement. Police Minister Stuart Nash has officially opened the new station, which was relocated almost 20 kilometres along the coast from the nearby ...
    BeehiveBy beehive.govt.nz
    1 week ago
  • Plans to protect the future of whitebaiting announced
    With several native whitebait species in decline the Minister of Conservation Eugenie Sage has today released proposals to standardise and improve management of whitebait across New Zealand. “The need for action for a healthy whitebait fishery has never been greater,” Eugenie Sage said.  “Four of the six whitebait species are ...
    BeehiveBy beehive.govt.nz
    1 week ago
  • New resource for schools to increase awareness and understanding of climate change
    A new Ministry of Education resource available for schools in 2020 will increase awareness and understanding of climate change, Education Minister Chris Hipkins said today. “The resource, Climate Change – prepare today, live well tomorrow, will help students understand the effects of climate change at a local, national and global ...
    BeehiveBy beehive.govt.nz
    1 week ago
  • Getting more out of our most productive firms
    Finance Minister Grant Robertson has approved the terms of reference for an Inquiry into the economic contribution of New Zealand's frontier firms. Frontier firms are the most productive firms in the domestic economy within their own industry. “These firms are important as they diffuse new technologies and business practices into ...
    BeehiveBy beehive.govt.nz
    1 week ago
  • NZDF sends more support to Australia
    The New Zealand Defence Force (NZDF) is sending an Environmental Health Team, a Primary Health Care Team and a Chaplain to Australia, boosting New Zealand support for the Australian Defence Force (ADF) as it battles bush fires in Victoria and New South Wales, Defence Minister Ron Mark announced today. The ...
    BeehiveBy beehive.govt.nz
    2 weeks ago
  • New Zealand joins partners in calling for full investigation into air crash in Iran
    Acting Prime Minister and Minister of Foreign Affairs Rt Hon Winston Peters says that developments suggesting a surface-to-air missile is responsible for the downing of the Ukrainian International Airlines flight in Iran is disastrous news. “New Zealand offers its deepest sympathies to the families of the 176 victims. It is ...
    BeehiveBy beehive.govt.nz
    2 weeks ago
  • Staying connected to Australian agriculture
    Agriculture Minister, Damien O’Connor, says the Ministry for Primary Industries is continuing to stay connected to federal authorities in Australia as devastating fires affect the country.  “The Ministry is using an existing trans-Tasman forum for discussions on the agricultural impact of the fires and the future recovery phase,” says Damien ...
    BeehiveBy beehive.govt.nz
    2 weeks ago
  • Investment in schools – a commitment to communities
    Thousands of school-age children, their teachers and wider communities are benefiting from the Government’s multi-million dollar investment upgrading and renewing schools, Education Minister Chris Hipkins said today. “We want New Zealand to be the best place to be a child and that means learning in warm, comfortable and modern classrooms,” ...
    BeehiveBy beehive.govt.nz
    2 weeks ago
  • New Zealand Defence Force sends support to Australia
    Minister of Defence Ron Mark today announced New Zealand is sending three Royal New Zealand Air Force NH90 helicopters and crew, and two NZ Army Combat Engineer Sections as well as a command element to support the Australian Defence Force efforts in tackling the Australian fires.  The New Zealand Defence Force ...
    BeehiveBy beehive.govt.nz
    2 weeks ago
  • Better access to books for blind and low vision citizens on World Braille Day
    "Today is World Braille Day and I am delighted to announce that an international treaty giving blind and low vision New Zealanders access to books and literary works comes into force today,” Minister for Disability Issues Carmel Sepuloni announced today. “Today the Marrakesh Treaty and the associated amendments to the ...
    BeehiveBy beehive.govt.nz
    3 weeks ago
  • New Zealand to send further firefighter support to Australia
    The New Zealand Government is sending a further 22 firefighters to help fight the Australian fires. “The devastation caused by these fires is taking a substantial toll on our Australian neighbours and we will continue to do what we can to assist as they deal with this extremely dynamic, dangerous ...
    BeehiveBy beehive.govt.nz
    3 weeks ago
  • Reducing the cost of education
    Twenty-two more schools have opted into the Government’s policy of providing $150 per child to schools who don’t ask parents for donations– bringing the total number of schools in the policy to 1,585. The Ministry of Education has accepted late opt ins past the November 14 deadline from schools that ...
    BeehiveBy beehive.govt.nz
    3 weeks ago