Maintenance time

Written By: - Date published: 11:51 am, January 2nd, 2022 - 15 comments
Categories: infrastructure, The Standard, uncategorized - Tags:

The Standard server is going down for a few hours later today for maintenance.

Server waiting for maintenance

It needs a replacement for the steadily failing hard drives in the archive array (RAID-Z3) and a replacement of 2Tb disks (some now over 7 years old) with new 4Tb drives. Rather than having 8 in the array, one has fully died and another faulted. So I’ll replace with 4x4Tb drives. About to head to PBTech to get a one more drive (I prefer getting them out of different batches).

The metaSAS card needs to move so that it doesn’t occasionally get overheating errors. I also have an large quiet exterior fan to attach to provide more even ventilation.

The SSDs that run The Standard array are starting to get wear issues, most notably in the log regions. Plus I need to start replacing those Intel 120Gb SSDs with something that is a bit bigger.

A good vacuum of the chassis interior wouldn’t go amiss. Even with the dust covers, the fine dust is slowly building up. Plus I think that the cat hair is starting to get in. She does like to groom herself just in front (after having a good sunbath).

I was thinking about updating the motherboard, CPU and RAM. But that can wait for another year.

As usual, I can’t really give you a precise time-frame because of the vagaries of shopping (urrgh) and balky hardware. However everything is backed up (and that takes a while to archive close to 10Tb to AWS S3). It should be a matter of only a few hours at worst to get up and running again, and then days to get the non-critical backup data down again.

In the mean time – enjoy the sunny day. I know that Luna will…

Luna in the sun
Luna bathing in the sun and preparing to shed cat hair into the server.

 Updated: The debris of old drives..

The residual drives – to be stored or disposed of.

15 comments on “Maintenance time ”

  1. Puckish Rogue 1

    So does this mean there'll be a dictionary function added? (I'd chip in a few bucks if it was money thing)

  2. Good luck Luna, you know this is only temporary and you will soon be able to get back into cat shedding mode in front of that consistently nice warm 'thing'.

    Support PR. If there is anything that a few $$$$ will help I would be happy to help too.

    This site and ALL its users/poster/commenters brings joy to me not the least of which is the intelligence and writing skills of all. And not forgetting the impecable moderating. smiley

  3. Dennis Frank 3

    I wonder if an AI module will eventually be a worthwhile addition? It could self-diagnose times for regenerative replacements to the system, put in the order to PBTech.

    You'd need a slot in your front door for the courier driver to drop the packages through. Then a package-stripping robot (along the lines of them automatic vacuumers) to extricate the replacement modules from the packages.

    Writing the instruction program to teach the robot to insert the new bits would be the tricky part. Robot would need eyes to see the slots, and an internalised photo of the equipment from the inserter's perspective to match up with.

    Anyway, if you need a creative project for your retirement plan you could consider how to make the TS system resilient via self-maintenance… enlightened

    • Shanreagh 3.1

      I vote for Luna being trained to do all this. She is on site, fewer $$$$ in travel costs and may be keen. So often when recruiting for new tasks the existing 'employees' are overlooked. After AI is pretty scary, almost as scary as having a vaccine…'ooops who said that?'

      We should keep this within the animal kingdom. Failing Luna then Gezza's Pukeko or Elvira the eel may be free for training. wink

    • lprent 3.2

      Curiously enough, a server self-diagnosis software module is what I'm writing at present professionally. Probably not the AI yet, I'm just pulling enough data to allow alerts. But also to enable data mining over time for expert system rules.

  4. higherstandard 4

    Has your lord and master thought of doing a Mayor Stubbs …. might be just what the city needs.

    • lprent 4.1

      She is no Governor Grey, ruler of government house.

      I'd could say that she rules my partner – the soft-hearted food giver to a cat whose finicky eating habits a a legend. However Luna doesn't sleep on my side of the bed because I auto-kick when bitten through the covers. Luna comes to me when she wants to be rubbed up the wrong way – as a hair removal.

      However all these must be ridiculous rumours because I just got told so….

  5. lprent 5

    Ok – starting to shutdown. I located where my screw driver set went to..

    • lprent 5.1

      And we're back.

      Had a problem that I had three identical Intel SSD drives, and I'd forgetten to get the serial numbers before shutdown

      Like this

      $ lsblk --nodeps -o name,serial
      NAME SERIAL
      sda S1DHNSAF817333Y
      sdb ZDH8TXGX
      sdc WD-WX42D31KS5SE
      sdd WD-WXA2D21RV3HE
      sde ZDHAR4D8
      sdf ZDHAQ4NC
      sdg CVCV4302022A120BGN
      sdh CVLT614108Z2120GGN
      sdi PHWL5433003G120LGN
      sdj S2R4NX0H410553P
      sdk 130801400034
      sdl S3YBNB0N402586X
      sdm CVMP2355042L120BGN
      nvme0n1 50026B76839DD779
      $ lsscsi
      [0:0:16:0] disk ATA ST4000VN008-2DR1 SC60 /dev/sdb
      [0:0:17:0] disk ATA WDC WD40EFZX-68A 0B81 /dev/sdc
      [0:0:18:0] disk ATA WDC WD40EFZX-68A 0B81 /dev/sdd
      [0:0:19:0] disk ATA ST4000VN008-2DR1 SC60 /dev/sde
      [0:0:20:0] disk ATA ST4000VN008-2DR1 SC60 /dev/sdf
      [1:0:0:0] disk ATA Samsung SSD 840 BB6Q /dev/sda
      [2:0:0:0] disk ATA INTEL SSDSC2CW12 400i /dev/sdg
      [3:0:0:0] disk ATA INTEL SSDSC2KW12 G200 /dev/sdh
      [4:0:0:0] disk ATA INTEL SSDSC2BB12 0370 /dev/sdi
      [5:0:0:0] disk ATA Samsung SSD 850 2B6Q /dev/sdj
      [6:0:0:0] disk ATA SanDisk SDSSDP06 0 /dev/sdk
      [7:0:0:0] disk ATA Samsung SSD 860 4B6Q /dev/sdl
      [8:0:0:0] disk ATA INTEL SSDSC2CT12 300i /dev/sdm
      [N:0:1:1] disk KINGSTON SA2000M81000G__1 /dev/nvme0n1

      That would have chopped nearly half an hour off the time. I kept putting the wrong one back in.

      But also not getting the right battery for the UPS systems didn’t help.

  6. lprent 6

    Turns out the drives were older than I thought.

    • 1 WD from 2012
    • 5 WD from 2013
    • 1 Seagate from 2013 (which was the dead one).
    • 1 WD from 2012 2016

    Lets hope that 5 x 4 terabyte drives (3 Seagate IronWolf and 2 WD Red Plus) last as long. Turns out I forget that I had a 4Tb Ironwolf that I had alreday brought in 2020 for testing.

    Now there are just these 8 2Tb antiques plus a few Intel 120Gb to clean and dispose of. Most of them are still ok with few errors. I will tuck them away in a backup e-sata system.

  7. McFlock 7

    any reason wordfence would be throwing 403 errors?

    • lprent 7.1

      Yeah, zpool creation of archive created it at /archive rather than /mnt/archive

      rsync to copy Hold to /mnt/archive carefully created a /mnt/archive and filled up the root directory. When it ran out of space, it started to fill RAM, and then quit. That meant that there wasn't a lot of spare RAM available, so large requests to use it – like wordfence would cause a 403 error because it it couldn't get enough RAM to do whatever it was wanting to do.

      Fixed finally. Pretty freaky that TS carried on working from reading. Not so hot on saving anything.

      The rsyncs and copies are in full overnight pulls. Hopefully all done before morning.

  8. Descendant Of Smith 8

    Not related to your rebuild but I thought interesting anyway.

    Son is just dealing with a few of these errors on different businesse's systems. Affecting email for some.

    The FIP-FS "Microsoft scan engine failed to load. Can't convert "2201010001" too long.

    YYMMDDhhmm formatted time exceed signed INT range.

    Seems 2022 is a problem not just 2000.

    Found a link.

    https://borncity.com/win/2022/01/02/microsoft-besttigt-exchange-year-2022-problem-fip-fs-scan-engine-failed-to-load-1-jan-2022/

    • lprent 8.1

      Oh dear. Who would code that into a 32 bit signed integer?
      If they'd done it in a 32 bit unsigned integer then it wouldn't be a problem until later.

      Hex 0xFFFFFFFF == Dec 4,294,967,295 (ie with a no signed bit)
      Hex 0x7FFFFFFF == Dec 2,125,483,647 (can be negative)

      Some dumbarse corporate programming fool obviously thought it'd be more 'efficient' to convert a string to a numeric to break it up. Which is stupid in the face of it. It is more efficient to split the string and convert the date elements because 2 string bytes at a datetime. That is a trivial conversion to date (see std c++ datetime), whereas converting 10 bytes at a time to a number is laborious, especially when you have to then split that number to get to a datetime.

      (reads article) Ok – Microsoft Exchange – figures. One bit of software that I swore I’d never recommend two decades ago.

Recent Comments

Recent Posts

  • Comprehensive Partnership the goal for NZ and the Philippines
    Prime Minister Christopher Luxon held a bilateral meeting today with the President of the Philippines, Ferdinand Marcos Jr.  The Prime Minister was accompanied by MP Paulo Garcia, the first Filipino to be elected to a legislature outside the Philippines. During today’s meeting, Prime Minister Luxon and President Marcos Jr discussed opportunities to ...
    BeehiveBy beehive.govt.nz
    1 hour ago
  • Government commits $20m to Westport flood protection
    The Government has announced that $20 million in funding will be made available to Westport to fund much needed flood protection around the town. This measure will significantly improve the resilience of the community, says Local Government Minister Simeon Brown. “The Westport community has already been allocated almost $3 million ...
    BeehiveBy beehive.govt.nz
    7 hours ago
  • Taupō takes pole position
    The Government is proud to support the first ever Repco Supercars Championship event in Taupō as up to 70,000 motorsport fans attend the Taupō International Motorsport Park this weekend, says Economic Development Minister Melissa Lee. “Anticipation for the ITM Taupō Super400 is huge, with tickets and accommodation selling out weeks ...
    BeehiveBy beehive.govt.nz
    8 hours ago
  • Cost of living support for low-income homeowners
    Local Government Minister Simeon Brown has announced an increase to the Rates Rebate Scheme, putting money back into the pockets of low-income homeowners.  “The coalition Government is committed to bringing down the cost of living for New Zealanders. That includes targeted support for those Kiwis who are doing things tough, such ...
    BeehiveBy beehive.govt.nz
    10 hours ago
  • Government backing mussel spat project
    The Coalition Government is investing in a project to boost survival rates of New Zealand mussels and grow the industry, Oceans and Fisheries Minister Shane Jones has announced. “This project seeks to increase the resilience of our mussels and significantly boost the sector’s productivity,” Mr Jones says. “The project - ...
    BeehiveBy beehive.govt.nz
    11 hours ago
  • Government focused on getting people into work
    Benefit figures released today underscore the importance of the Government’s plan to rebuild the economy and have 50,000 fewer people on Jobseeker Support, Social Development and Employment Minister Louise Upston says. “Benefit numbers are still significantly higher than when National was last in government, when there was about 70,000 fewer ...
    BeehiveBy beehive.govt.nz
    13 hours ago
  • Clean energy key driver to reducing emissions
    The Government’s commitment to doubling New Zealand’s renewable energy capacity is backed by new data showing that clean energy has helped the country reach its lowest annual gross emissions since 1999, Climate Change Minister Simon Watts says. New Zealand’s latest Greenhouse Gas Inventory (1990-2022) published today, shows gross emissions fell ...
    BeehiveBy beehive.govt.nz
    14 hours ago
  • Earthquake-prone buildings review brought forward
    The Government is bringing the earthquake-prone building review forward, with work to start immediately, and extending the deadline for remediations by four years, Building and Construction Minister Chris Penk says. “Our Government is focused on rebuilding the economy. A key part of our plan is to cut red tape that ...
    BeehiveBy beehive.govt.nz
    19 hours ago
  • Thailand and NZ to agree to Strategic Partnership
    Prime Minister Christopher Luxon and his Thai counterpart, Prime Minister Srettha Thavisin, have today agreed that New Zealand and the Kingdom of Thailand will upgrade the bilateral relationship to a Strategic Partnership by 2026. “New Zealand and Thailand have a lot to offer each other. We have a strong mutual desire to build ...
    BeehiveBy beehive.govt.nz
    1 day ago
  • Government consults on extending coastal permits for ports
    RMA Reform Minister Chris Bishop and Transport Minister Simeon Brown have today announced the Coalition Government’s intention to extend port coastal permits for a further 20 years, providing port operators with certainty to continue their operations. “The introduction of the Resource Management Act in 1991 required ports to obtain coastal ...
    BeehiveBy beehive.govt.nz
    1 day ago
  • Inflation coming down, but more work to do
    Today’s announcement that inflation is down to 4 per cent is encouraging news for Kiwis, but there is more work to be done - underlining the importance of the Government’s plan to get the economy back on track, acting Finance Minister Chris Bishop says. “Inflation is now at 4 per ...
    BeehiveBy beehive.govt.nz
    1 day ago
  • School attendance restored as a priority in health advice
    Refreshed health guidance released today will help parents and schools make informed decisions about whether their child needs to be in school, addressing one of the key issues affecting school attendance, says Associate Education Minister David Seymour. In recent years, consistently across all school terms, short-term illness or medical reasons ...
    BeehiveBy beehive.govt.nz
    2 days ago
  • Unnecessary bureaucracy cut in oceans sector
    Oceans and Fisheries Minister Shane Jones is streamlining high-level oceans management while maintaining a focus on supporting the sector’s role in the export-led recovery of the economy. “I am working to realise the untapped potential of our fishing and aquaculture sector. To achieve that we need to be smarter with ...
    BeehiveBy beehive.govt.nz
    2 days ago
  • Patterson promoting NZ’s wool sector at International Congress
    Associate Agriculture Minister Mark Patterson is speaking at the International Wool Textile Organisation Congress in Adelaide, promoting New Zealand wool, and outlining the coalition Government’s support for the revitalisation the sector.    "New Zealand’s wool exports reached $400 million in the year to 30 June 2023, and the coalition Government ...
    BeehiveBy beehive.govt.nz
    2 days ago
  • Removing red tape to help early learners thrive
    The Government is making legislative changes to make it easier for new early learning services to be established, and for existing services to operate, Associate Education Minister David Seymour says. The changes involve repealing the network approval provisions that apply when someone wants to establish a new early learning service, ...
    BeehiveBy beehive.govt.nz
    3 days ago
  • RMA changes to cut coal mining consent red tape
    Changes to the Resource Management Act will align consenting for coal mining to other forms of mining to reduce barriers that are holding back economic development, Resources Minister Shane Jones says. “The inconsistent treatment of coal mining compared with other extractive activities is burdensome red tape that fails to acknowledge ...
    BeehiveBy beehive.govt.nz
    3 days ago
  • McClay reaffirms strong NZ-China trade relationship
    Trade, Agriculture and Forestry Minister Todd McClay has concluded productive discussions with ministerial counterparts in Beijing today, in support of the New Zealand-China trade and economic relationship. “My meeting with Commerce Minister Wang Wentao reaffirmed the complementary nature of the bilateral trade relationship, with our Free Trade Agreement at its ...
    BeehiveBy beehive.govt.nz
    3 days ago
  • Prime Minister Luxon acknowledges legacy of Singapore Prime Minister Lee
    Prime Minister Christopher Luxon today paid tribute to Singapore’s outgoing Prime Minister Lee Hsien Loong.   Meeting in Singapore today immediately before Prime Minister Lee announced he was stepping down, Prime Minister Luxon warmly acknowledged his counterpart’s almost twenty years as leader, and the enduring legacy he has left for Singapore and South East ...
    BeehiveBy beehive.govt.nz
    3 days ago
  • PMs Luxon and Lee deepen Singapore-NZ ties
    Prime Minister Christopher Luxon held a bilateral meeting today with Singapore Prime Minister Lee Hsien Loong. While in Singapore as part of his visit to South East Asia this week, Prime Minister Luxon also met with Singapore President Tharman Shanmugaratnam and will meet with Deputy Prime Minister Lawrence Wong.  During today’s meeting, Prime Minister Luxon ...
    BeehiveBy beehive.govt.nz
    3 days ago
  • Antarctica New Zealand Board appointments
    Foreign Minister Winston Peters has made further appointments to the Board of Antarctica New Zealand as part of a continued effort to ensure the Scott Base Redevelopment project is delivered in a cost-effective and efficient manner.  The Minister has appointed Neville Harris as a new member of the Board. Mr ...
    BeehiveBy beehive.govt.nz
    3 days ago
  • Finance Minister travels to Washington DC
    Finance Minister Nicola Willis will travel to the United States on Tuesday to attend a meeting of the Five Finance Ministers group, with counterparts from Australia, the United States, Canada, and the United Kingdom.  “I am looking forward to meeting with our Five Finance partners on how we can work ...
    BeehiveBy beehive.govt.nz
    3 days ago
  • Pet bonds a win/win for renters and landlords
    The coalition Government has today announced purrfect and pawsitive changes to the Residential Tenancies Act to give tenants with pets greater choice when looking for a rental property, says Housing Minister Chris Bishop. “Pets are important members of many Kiwi families. It’s estimated that around 64 per cent of New ...
    BeehiveBy beehive.govt.nz
    3 days ago
  • Long Tunnel for SH1 Wellington being considered
    State Highway 1 (SH1) through Wellington City is heavily congested at peak times and while planning continues on the duplicate Mt Victoria Tunnel and Basin Reserve project, the Government has also asked NZ Transport Agency (NZTA) to consider and provide advice on a Long Tunnel option, Transport Minister Simeon Brown ...
    BeehiveBy beehive.govt.nz
    4 days ago
  • New Zealand condemns Iranian strikes
    Prime Minister Christopher Luxon and Foreign Minister Winston Peters have condemned Iran’s shocking and illegal strikes against Israel.    “These attacks are a major challenge to peace and stability in a region already under enormous pressure," Mr Luxon says.    "We are deeply concerned that miscalculation on any side could ...
    BeehiveBy beehive.govt.nz
    4 days ago
  • Huge interest in Government’s infrastructure plans
    Hundreds of people in little over a week have turned out in Northland to hear Regional Development Minister Shane Jones speak about plans for boosting the regional economy through infrastructure. About 200 people from the infrastructure and associated sectors attended an event headlined by Mr Jones in Whangarei today. Last ...
    BeehiveBy beehive.govt.nz
    6 days ago
  • Health Minister thanks outgoing Health New Zealand Chair
    Health Minister Dr Shane Reti has today thanked outgoing Health New Zealand – Te Whatu Ora Chair Dame Karen Poutasi for her service on the Board.   “Dame Karen tendered her resignation as Chair and as a member of the Board today,” says Dr Reti.  “I have asked her to ...
    BeehiveBy beehive.govt.nz
    6 days ago
  • Roads of National Significance planning underway
    The NZ Transport Agency (NZTA) has signalled their proposed delivery approach for the Government’s 15 Roads of National Significance (RoNS), with the release of the State Highway Investment Proposal (SHIP) today, Transport Minister Simeon Brown says.  “Boosting economic growth and productivity is a key part of the Government’s plan to ...
    BeehiveBy beehive.govt.nz
    6 days ago
  • Navigating an unstable global environment
    New Zealand is renewing its connections with a world facing urgent challenges by pursuing an active, energetic foreign policy, Foreign Minister Winston Peters says.   “Our country faces the most unstable global environment in decades,” Mr Peters says at the conclusion of two weeks of engagements in Egypt, Europe and the United States.    “We cannot afford to sit back in splendid ...
    BeehiveBy beehive.govt.nz
    6 days ago
  • NZ welcomes Australian Governor-General
    Prime Minister Christopher Luxon has announced the Australian Governor-General, His Excellency General The Honourable David Hurley and his wife Her Excellency Mrs Linda Hurley, will make a State visit to New Zealand from Tuesday 16 April to Thursday 18 April. The visit reciprocates the State visit of former Governor-General Dame Patsy Reddy ...
    BeehiveBy beehive.govt.nz
    7 days ago
  • Pseudoephedrine back on shelves for Winter
    Associate Health Minister David Seymour has announced that Medsafe has approved 11 cold and flu medicines containing pseudoephedrine. Pharmaceutical suppliers have indicated they may be able to supply the first products in June. “This is much earlier than the original expectation of medicines being available by 2025. The Government recognised ...
    BeehiveBy beehive.govt.nz
    7 days ago
  • NZ and the US: an ever closer partnership
    New Zealand and the United States have recommitted to their strategic partnership in Washington DC today, pledging to work ever more closely together in support of shared values and interests, Foreign Minister Winston Peters says.    “The strategic environment that New Zealand and the United States face is considerably more ...
    BeehiveBy beehive.govt.nz
    7 days ago
  • Joint US and NZ declaration
    April 11, 2024 Joint Declaration by United States Secretary of State the Honorable Antony J. Blinken and New Zealand Minister of Foreign Affairs the Right Honourable Winston Peters We met today in Washington, D.C. to recommit to the historic partnership between our two countries and the principles that underpin it—rule ...
    BeehiveBy beehive.govt.nz
    7 days ago
  • NZ and US to undertake further practical Pacific cooperation
    Foreign Minister Winston Peters has announced further New Zealand cooperation with the United States in the Pacific Islands region through $16.4 million in funding for initiatives in digital connectivity and oceans and fisheries research.   “New Zealand can achieve more in the Pacific if we work together more urgently and ...
    BeehiveBy beehive.govt.nz
    7 days ago
  • Government redress for Te Korowai o Wainuiārua
    The Government is continuing the bipartisan effort to restore its relationship with iwi as the Te Korowai o Wainuiārua Claims Settlement Bill passed its first reading in Parliament today, says Treaty Negotiations Minister Paul Goldsmith. “Historical grievances of Te Korowai o Wainuiārua relate to 19th century warfare, land purchased or taken ...
    BeehiveBy beehive.govt.nz
    1 week ago
  • Focus on outstanding minerals permit applications
    New Zealand Petroleum and Minerals is working to resolve almost 150 outstanding minerals permit applications by the end of the financial year, enabling valuable mining activity and signalling to the sector that New Zealand is open for business, Resources Minister Shane Jones says.  “While there are no set timeframes for ...
    BeehiveBy beehive.govt.nz
    1 week ago
  • Applications open for NZ-Ireland Research Call
    The New Zealand and Irish governments have today announced that applications for the 2024 New Zealand-Ireland Joint Research Call on Agriculture and Climate Change are now open. This is the third research call in the three-year Joint Research Initiative pilot launched in 2022 by the Ministry for Primary Industries and Ireland’s ...
    BeehiveBy beehive.govt.nz
    1 week ago
  • Tenancy rules changes to improve rental market
    The coalition Government has today announced changes to the Residential Tenancies Act to encourage landlords back to the rental property market, says Housing Minister Chris Bishop. “The previous Government waged a war on landlords. Many landlords told us this caused them to exit the rental market altogether. It caused worse ...
    BeehiveBy beehive.govt.nz
    1 week ago
  • Boosting NZ’s trade and agricultural relationship with China
    Trade and Agriculture Minister Todd McClay will visit China next week, to strengthen relationships, support Kiwi exporters and promote New Zealand businesses on the world stage. “China is one of New Zealand’s most significant trade and economic relationships and remains an important destination for New Zealand’s products, accounting for nearly 22 per cent of our good and ...
    BeehiveBy beehive.govt.nz
    1 week ago
  • Freshwater farm plan systems to be improved
    The coalition Government intends to improve freshwater farm plans so that they are more cost-effective and practical for farmers, Associate Environment Minister Andrew Hoggard and Agriculture Minister Todd McClay have announced. “A fit-for-purpose freshwater farm plan system will enable farmers and growers to find the right solutions for their farm ...
    BeehiveBy beehive.govt.nz
    1 week ago
  • New Fast Track Projects advisory group named
    The coalition Government has today announced the expert advisory group who will provide independent recommendations to Ministers on projects to be included in the Fast Track Approvals Bill, say RMA Reform Minister Chris Bishop and Regional Development Minister Shane Jones. “Our Fast Track Approval process will make it easier and ...
    BeehiveBy beehive.govt.nz
    1 week ago

Page generated in The Standard by Wordpress at 2024-04-18T11:37:59+00:00