web analytics

A quiet day

Written By: - Date published: 8:34 am, June 4th, 2010 - 6 comments
Categories: admin - Tags:

Sysop

The Internet is a modern wonder. However if you have to run servers on it, often the biggest wonder is that it works at all. There are so many things that can go wrong with the transport of data from one place to another.

Yesterday, several of the NZ networks were having problems communicating with our server. There was a routing problem in San Diego where packets coming back from the server were routed into oblivion. So the site appeared to be offline. This happened if you were (like I was) on the Orcon, IHug, and probably other networks. Other local networks were feeding in and getting responses from different routes.

I detected it in the morning and informed the hosting company and my ISP by voice message and e-mail. Unfortunately I’ve just started a new job where I’m doing a lot of design work with other people that occupied my full day. So I wasn’t able to have a good look at it until late in the afternoon.

I found my e-mails to the hosting company still sitting in the outbox because the mail server wasn’t able to talk to the hosting company. By resending those e-mails using a g-mail account I was able to contact hosting support who hadn’t been able to detect the problem (my rushed voice message in the morning was a bit short of required information). They started to look for the problem router where the routing table was sending returning packets to oblivion. They also started to work with to give the site an IP that would bypass the affected route.

Meanwhile I’d managed to talk to the server using putty from my iPhone. By doing a traceroute out from the server, and a traceroute from my home system going to the server we managed to pinpoint the router with a problem. The hosting support then organised a reroute around that problem system.

It got implemented last night. So everything should be back online this morning, and I’m able to write you this post. Looking at the traffic from yesterday, it clipped about 30-40% off the usual load of page views. As well as the people who weren’t able to read or comment, some of our authors were unable to put up their posts.

A quiet day, that is unlikely to be repeated today. It looks like the government is providing lots of material for us and you to write about.

6 comments on “A quiet day ”

  1. Name 1

    I’m with Slingshot and had no problem connecting to your servers at any time yesterday.

    A traceroute to you this morning records 22 external hops:

    Tracing route to 69-55-230-31.in-addr.arpa.johncompanies.com [69.55.230.31]
    over a maximum of 30 hops:

    1 1 ms <1 ms <1 ms 192.168.88.1
    2 713 ms 23 ms 17 ms 172.16.4.1
    3 18 ms 88 ms 56 ms 172.16.2.1
    4 316 ms 95 ms 92 ms p32.argent.cust.commverge.net.nz [202.180.81.32]
    5 30 ms 51 ms 81 ms vlan93-cpcak3-e1.tranzpeer.net [202.180.76.68]
    6 60 ms 137 ms 890 ms vlan7-cpcak3-s1.tranzpeer.net [202.180.81.49]
    7 110 ms 95 ms 53 ms gi10-0-0-405.gw1.akl1.asianetcom.net [203.192.167.29]
    8 29 ms 84 ms 52 ms ge-0-0-0-0.gw1.akl2.asianetcom.net [203.192.185.45]
    9 171 ms 209 ms 174 ms po1-1-0.gw3.lax1.asianetcom.net [203.192.185.62]
    10 249 ms 306 ms 188 ms te7-6.mpd01.lax05.atlas.cogentco.com [154.54.11.9]
    11 196 ms 189 ms 430 ms te2-1.ccr02.lax05.atlas.cogentco.com [154.54.28.66]
    12 239 ms 211 ms 198 ms te4-2.ccr01.lax01.atlas.cogentco.com [154.54.6.229]
    13 251 ms 173 ms 210 ms te4-2.ccr01.lax04.atlas.cogentco.com [154.54.24.70]
    14 214 ms 195 ms 178 ms audio-visual-services-group-inc.demarc.cogentco.com [38.104.82.234]
    15 267 ms 223 ms 195 ms border11.po2-20g-bbnet1.lax.pnap.net [216.52.255.104]
    16 173 ms 218 ms 189 ms core2.ge2-3.ocy002.pnap.net [64.94.109.6]
    17 220 ms 201 ms 207 ms border1.po1-2g-bbnet1.ocy002.pnap.net [216.52.96.34]
    18 190 ms 192 ms 232 ms core2-g2-1.ocy002.pnap.net [70.42.209.10]
    19 240 ms 172 ms 234 ms border3.ge2-0-bbnet1.sdg.pnap.net [63.251.127.7]
    20 461 ms 211 ms 187 ms castleaccess-2.border3.sdg.pnap.net [63.251.125.10]
    21 240 ms 313 ms 202 ms ge1-1-6509-a.castleaccess.com [69.43.169.80]
    22 196 ms 229 ms 211 ms 69.43.129.84
    23 177 ms 172 ms 195 ms 69-55-233-164.in-addr.arpa.johncompanies.com [69.55.233.164]
    24 175 ms 185 ms 241 ms 69-55-233-153.in-addr.arpa.johncompanies.com [69.55.233.153]
    25 206 ms 227 ms 312 ms 69-55-230-31.in-addr.arpa.johncompanies.com [69.55.230.31]

    Trace complete.

    For an .org.nz URL this seems a bit excessive.

    • lprent 1.1

      The reasons for this have little to do with tech and a lot to do with getting the sites server out of the way of nuisance complaints. This was extensively described in the comments on this post.

      It used to be that we sited offshore partially because of cost and partially because of legal issues. The cost issue has gone away – you can get just as cheap servers here.

      The legal one hasn’t. We don’t trust this government or its wingnut minions not to attempt to shut the dite down with nuisance complaints at a time that is convenient for them. It’d be one of those things that a devious control freak (say McCully) would probably find useful…

      Unfortunately hosting services here will limit their liability by forcing us to remove material or shut the site on receipt of a letter from a lawyer alleging a complaint. The complainant doesn’t have to prove anything – just make a complaint. And who hasn’t got a mate who is a lawyer? So nuisance complaints are cheap and easy to make.

      By siting in the US, we significantly raise the cost of nuisance complaints. You now have to find a lawyer to frame such complaints in the legal framework of the state the server or hosting company is in. So people can’t take us out cheaply.

      Of course if anyone does feel that they have been unfairly criticized by this site, then they are welcome to complain to us. We’re usually pretty good for handling the effects of our inadvertent factual errors on people affected. If they receive no satisfaction then they can take a case to our local courts. But they’d have to prove their complaint and damage to the satisfaction of a court.

      If a local hosting company is willing to give us a contract without one of those drop-out clauses in it, at least requiring some proof of the validity of a claim, we’d be happy to resite back into NZ.

  2. prism 2

    Thanks for that interesting glimpse of the perils and the active problem solving needed in maintaining the continuance of the blog.

  3. jcuknz 3

    I don’t understand what you are doing but I value it … thanks for your efforts.

  4. Good job Mr L!

Recent Comments

Recent Posts

  • Fast-tracked Northland water project will accelerate economic recovery
    The Government has welcomed the decision to approve a new water storage reservoir in Northland, the first of a number of infrastructure projects earmarked for a speedy consenting process that aims to accelerate New Zealand’s economic recovery from Covid-19.  The Matawii Water Storage Reservoir will provide drinking water for Kaikohe, ...
    BeehiveBy beehive.govt.nz
    18 hours ago
  • Tokelau Language Week reminds us to stay united and strong
    Staying strong in the face of challenges and being true to our heritage and languages are key to preserving our cultural identity and wellbeing, is the focus of the 2020 Tokelau Language Week. Minister for Pacific Peoples, Aupito William Sio, says this year’s theme, ‘Apoapo tau foe, i nā tāfea ...
    BeehiveBy beehive.govt.nz
    6 days ago
  • NZ announces a third P-3 deployment in support of UN sanctions
    The Government has deployed a Royal New Zealand Air Force P-3K2 Orion (P-3) maritime patrol aircraft to support the implementation of United Nations Security Council (UNSC) resolutions imposing sanctions against North Korea, announced Minister of Foreign Affairs Winston Peters and Minister of Defence Ron Mark. “New Zealand has long supported ...
    BeehiveBy beehive.govt.nz
    1 week ago
  • Pacific trade and development agreement a reality
    Pacific regional trade and development agreement PACER Plus will enter into force in 60 days now that the required eight countries have ratified it. Trade and Export Growth Minister David Parker welcomed the announcement that the Cook Islands is the eighth nation to ratify this landmark agreement. “The agreement represents ...
    BeehiveBy beehive.govt.nz
    2 weeks ago
  • Securing a pipeline of teachers
    The Government is changing its approach to teacher recruitment as COVID-19 travel restrictions continue, by boosting a range of initiatives to get more Kiwis into teaching. “When we came into Government, we were faced with a teacher supply crisis,” Education Minister Chris Hipkins said. “Over the past three years, we ...
    BeehiveBy beehive.govt.nz
    2 weeks ago
  • Border exceptions for a small number of international students with visas
    The Government has established a new category that will allow 250 international PhD and postgraduate students to enter New Zealand and continue their studies, in the latest set of border exceptions. “The health, safety and wellbeing of people in New Zealand remains the Government’s top priority. Tight border restrictions remain ...
    BeehiveBy beehive.govt.nz
    2 weeks ago
  • First COVID-19 vaccine purchase agreement signed
    The Government has signed an agreement to purchase 1.5 million COVID-19 vaccines – enough for 750,000 people – from Pfizer and BioNTech, subject to the vaccine successfully completing all clinical trials and passing regulatory approvals in New Zealand, say Research, Science and Innovation Minister Megan Woods and Health Minister Chris Hipkins. ...
    BeehiveBy beehive.govt.nz
    2 weeks ago
  • International statement – End-to-end encryption and public safety
    We, the undersigned, support strong encryption, which plays a crucial role in protecting personal data, privacy, intellectual property, trade secrets and cyber security.  It also serves a vital purpose in repressive states to protect journalists, human rights defenders and other vulnerable people, as stated in the 2017 resolution of the ...
    BeehiveBy beehive.govt.nz
    2 weeks ago
  • Ministry of Defence Biodefence Assessment released
    The Ministry of Defence has today released a Defence Assessment examining Defence’s role across the spectrum of biological hazards and threats facing New Zealand. Biodefence: Preparing for a New Era of Biological Hazards and Threats looks at how the NZDF supports other agencies’ biodefence activities, and considers the context of ...
    BeehiveBy beehive.govt.nz
    2 weeks ago
  • New Approaches to Economic Challenges: Confronting Planetary Emergencies: OECD 9 October 2020
    New Approaches to Economic Challenges: Confronting Planetary Emergencies: OECD 9 October 2020 Hon David Parker’s response following Thomas Piketty and Esther Duflo. Good morning, good afternoon, and good evening, wherever in the world you might be. I first acknowledge the excellent thought provoking speeches of Thomas Piketty and Esther ...
    BeehiveBy beehive.govt.nz
    3 weeks ago
  • Kaipara Moana restoration takes next step
    A Memorandum of Understanding has been signed today at Waihāua Marae between the Crown, local iwi and councils to protect, restore and enhance the mauri of Kaipara Moana in Northland. Environment Minister David Parker signed the document on behalf of the Crown along with representatives from Ngā Maunga Whakahī, Ngāti ...
    BeehiveBy beehive.govt.nz
    3 weeks ago
  • New Zealand and Uruguay unite on reducing livestock production emissions
    Agriculture Minister Damien O’Connor and Uruguayan Minister of Livestock, Agriculture and Fisheries Carlos María Uriarte have welcomed the launch of a three-year project that will underpin sustainable livestock production in Uruguay, Argentina, and Costa Rica.  The project called ‘Innovation for pasture management’ is led by Uruguay’s National Institute of Agricultural ...
    BeehiveBy beehive.govt.nz
    3 weeks ago
  • 3100 jobs created through marae upgrades
    Hundreds of marae throughout the country will be upgraded through investments from the Provincial Growth Fund’s refocused post COVID-19 funding to create jobs and put money into the pockets of local tradespeople and businesses, Regional Economic Development Minister Shane Jones and Māori Development Minister Nanaia Mahuta have announced. “A total ...
    BeehiveBy beehive.govt.nz
    3 weeks ago
  • Health volunteers recognised in annual awards
    Health Minister Chris Hipkins has announced 9 teams and 14 individuals are the recipients of this year’s Minister of Health Volunteer Awards.  “The health volunteer awards celebrate and recognise the thousands of dedicated health sector volunteers who give many hours of their time to help other New Zealanders,” Mr Hipkins ...
    BeehiveBy beehive.govt.nz
    3 weeks ago
  • Community COVID-19 Fund supports Pacific recovery
    The Minister for Pacific Peoples, Aupito William Sio says a total of 264 groups and individuals have successfully applied for the Pacific Aotearoa Community COVID-19 Recovery Fund, that will support Pacific communities drive their own COVID-19 recovery strategies, initiatives, and actions. “I am keen to see this Fund support Pacific ...
    BeehiveBy beehive.govt.nz
    3 weeks ago