Back to work time

Written By: - Date published: 11:55 pm, January 9th, 2013 - 24 comments
Categories: admin, infrastructure, internet, The Standard - Tags:

Thought I’d share this portrait of my holidays….

xmas_and_new_year

This is why we’re a bit slack about posts of the Xmas-New year period. At various times over the last five end of years we’ve tried having lot of posts, few posts, and at the end of 2011 we mostly had OpenMike (and the least drop in visitor numbers – looks like people like making their own fun). The pattern always looks much the same.

Page views start falling through December with visit numbers dropping a lot less (people read pages fewer times), then abruptly drop just before the 25th. The 25th is invariably the worst day of the year. We get a spike a few days after Christmas day as people realize there is nothing much to do. But then they sink back into a summer turpitude until work is inflicted upon them again (ie about now) when our page views start rising rapidly. Then it becomes worthwhile writing posts again.

Nice for me. It means that there is one part of the year that I can safely move servers without doing the allnighters.

The primary server has been moved out into the offshore cloud far away from the expensive costs of having to pay for the costs of the Southern Cross cable monopoly. The ~5% of our traffic from human readers offshore wasn’t a problem. However the searchbots (most recently the bloody bingbot) and spambots causing us to exceed our “free” datacap on overseas traffic. This was $3/GB when we started on the old server and is now $1/GB.

However it has cost us thousands of dollars in variable per GB charges since I moved the server back to NZ in 2011. Does nasty things to the site’s operating budget. For instance in November it cost $103 extra because the Bing searchbot tried to read the entire site including every comment link. They nearly did a million page views in November and 600k in December. The really irritating thing is that we get very few incoming searches from Bing. I’ve now constrained Bing to a more limited diet.

But now the server is back offshore, I can budget without those dratted gouging variable costs from the Southern Cross cable monopoly. Because of the cloudflare content delivery network I don’t need dedicated servers any more. We were heading up to the main server straining on 30-40% of the available processing power prior to using cloudflare. These days the same server seldom uses more than 10%. So in doing the shift we sould drop the size of server required and the base cost of running a server has dropped dramatically. And that is despite the ever increasing numbers of visitors, comments, and page views over the last two years.

So my current limit is set at 800GB per month for all traffic and I’ll review if I need to lift that later this month.

24 comments on “Back to work time ”

  • Comments are now closed

  1. xtasy 1
    10 January 2013 at 12:01 am

    Hey Lprent – I already feel very happy with the “cloud’ outsourcing, as it is a bit like “cloud nine” to me.

    Maybe the NZ SIS and others will find it a bit more difficult to track and trace us in future, that is a very optimistic kind of “hope” of course.

    Otherwise saving costs makes sense, and cloud server use is now the norm, so we move with them times.

    Thanks for all!

  2. lprent 2
    10 January 2013 at 3:06 am

    Ok – off to bed. Caching appears to be better tuned.

  3. Bob Simmons 3
    10 January 2013 at 8:06 am

    The difference between say, Fisher and Paykell moving overseas to save costs (which you lefties were pissing yourselves over) and you moving your website overseas to save cost is?

    Both make good business sense, yet one is worse then the other?

    I smell hypocrite.

    • Colonial Viper 3.1
      10 January 2013 at 8:59 am

      Shit Bob you must think you are hot stuff.

      In the first case, hundreds of skilled NZers lost jobs, the country lost significant manufacturing capability and ownership of the company and its IP was transferred to the Chinese, to enrich wealthy shareholders.

      In the second case, The Standard stopped paying monopoly rents on the Southern Cross Cable to overseas corporates and overseas shareholders.

      • MrV 3.1.1
        10 January 2013 at 9:13 pm

        More convenient bullshit from the human colonic viper.
        Fact is you are directly leading to lost internet hosting activity etc in NZ. But you fail to see the hypocrisy here

        Fact:So you’d rather have your NZ visitors pay those “monopoly rents on the Southern Cross Cable to overseas corporates and overseas shareholders.” – and don’t forget to ad the bogeyman to your list. Boo.
        Fact:If you hosted in NZ most traffic is NZ based so wouldn’t be routed onto the cable.
        Fact:You can configure your site so external crawlers are limited traffic-wise.

    • lprent 3.2
      10 January 2013 at 9:00 am

      I thought that I made it pretty clear in the post that the main problem was paying for overseas traffic that we have little control or use for. Why should we pay for the overseas parasites? They aren’t our readership who are 95% on the local network. Why should we pay that money to a gouging near monopoly effectively owned by offshore interests? It isn’t like they’re investing in NZ because that would involve their destroying their own monopoly that sucks the life out of local businesses trying to develop locally.

      Whereas the rationale for F&P to move their production offshore was because they were largely selling into a global market where the freight for both purchases for the Bill of Materials (BOM) and the freight for market was a large part of the basic cost. The labour savings just aren’t that much of a cost in the BOM and wasn’t the main reason for the move. They kept the development, R&D, and much of the marketing here and communicated via the net with the production closer to suppliers and markets. In other words they kept the high paying intellectual property and jobs here. That is exactly what NZ wants and needs. We don’t want to be a low wage economy.

      I have no argument that there was a business logic to the F&P production move decision. I have no idea why you think I would – but I suspect it has to do with one of those rather stupid myths that some of the right like to cling to. I suspect what you smell is the rectal stench of your own bigotry.

      The actual argument here was that the National government is completely crap at fostering businesses and jobs here. But I guess taht was somewhat too subtle for your observational abilities. That the government who should be managing our structural infrastructure for businesses allows a profit gouging monopoly here like the Southern Cross cable is just plain daft. Effectively they should be trying to fix that market distortion so that businesses (like the export businesses I work for) don’t automatically move their servers offshore.

      As bandwidth requirements for running offshore sales and production operations increase, it will become more and more of a pain to maintain businesses here. If NZ businesses working largely offshore continue to have the life sucked out of them by the parasites then they will move both the server systems and eventually development offshore, taking those high paying jobs away.

      BTW: Unlimited bandwidth inside NZ was paid by the base charge on the server. However over the last two years when the server was back in NZ we wound up paying nearly as much for the overseas traffic as we did for all of the traffic inside the country. More inmportantly it was unscheduled and largely uncontrollable costs for something that we didn’t want nor need. Paradoxically moving to the other side of the Southern Cross both increases the costs to our actual readers as they will have increased charges on their ISP bills, while decreasing our site bills. It will actually increase costs to NZ consumers of our site.

      But in neither case are the bills in proportion to either the costs of running the Southern Cross nor to pay for new cables that will be required in the medium term (not that they’re doing that anyway). It is simple profit gouging.

      • Steve H 3.2.1
        10 January 2013 at 9:34 am

        L,

        The reason that they couldn’t get investors for a new cable was that it isn’t needed.

        It is no-where near capacity, and there are upgrades happening to it as we speak. And can you name me a company which charges less and less for its services every year, like SCCS does?

        The reason nobody invested was because it doesn’t make sense to an investor – build something which will lose you money long term providing a service which is dirt cheap anyway?

        • lprent 3.2.1.1
          10 January 2013 at 10:20 am

          That is the problem. It isn’t dirt-cheap to people providing content online. If it was then we wouldn’t be having this discussion.

          Dirt cheap is what the local network costs. The Southern Cross was and still is outrageously expensive by comparison.

          This site does between 400Gb and 700Gb per month if I didn’t put constraints on it. The majority of that traffic is inside NZ. But typically between 150GB and 250GB of that traffic comes from offshore. Most of that is from searchbots, most of the remainder is from spambots, and a small fraction is from humans.

          But it means that if I had 250GB of traffic from overseas – drop the 25GB of “free” limit from the hosting provider, and say 25GB from the actual overseas readers. Then at $1/GB we’re coughing up $200 for traffic that isn’t readers. The server cost was only $233/mo. And of course to make it worse we’re paying 15% tax on that to a lazy government.

          The pricing structures of the Southern Cross distort the market.Now some of that we want – searchbots for instance. While they could operate their searchbots on the local network, they don’t? Why? Because they don’t get charged for the southern cross – they force the cost on to us.

          Now look at this the other way. We just moved our servers offshore so we don’t get penalized for running a server in NZ by the Southern Cross. This means that literally hundreds of GB’s more traffic has to go over the Southern Cross from our servers into NZ. Ok so someone has to pay for that. It goes on the bills of the readers. Our move offshore has probably added slightly to the already very high ISP costs of local users.

          Meanwhile our local servers industry dies because there are market distortions

          BTW: It is pretty noticeable that the only time that the costs actually reduce on the Southern Cross is when people start talking about putting in more cables

        • mike e vipe e 3.2.1.2
          10 January 2013 at 1:19 pm

          S#*t Head what utter trash monopolies charge what their research shows the market will pay`!
          One of the main reasons why the morgan push for another cable failed was that the charges the US government wanted to charge to let it come on shore in the US made it enviable they had vested interests in keeping the existing southern cross cable as a monopoly one being they want us to start paying for content on the internet ie Rupert Murdoch and mates!

          • Steve H 3.2.1.2.1
            10 January 2013 at 2:49 pm

            Thats silly, Mike – why would the US government not want to make money off 2 providers?

            Do some math, 2 is bigger than 1.

            @ lprent, why not ban bing then? As you say, nobody uses it to search your site and then you could have save some cash.

            It sounds like you don’t want to accept that on one hand you complain about ISP’s charging too much, but on the other hand you take advantage of user pays, causing cost to ISP’s

            • lprent 3.2.1.2.1.1
              10 January 2013 at 4:39 pm

              I’d love to. I even went so far as to find out all of their current searchbot IP’s ranges.

              However when I compared it to the actual ranges of bingbot IP’s requesting data from our site, I found that the list was quite incomplete (and I verified that they were in fact real bingbots). A quick hunt around google showed that everyone has the same problem.

              This means that I can’t kill at the firewall by refusing a connection. I have to ban using the agent at apache. But by the time it gets that far into the server it becomes a bit of a waste of time. I’m still paying for the request data and the CPU time to throw them off.

              Eventually I just went into the bing webmaster tools and got them to limit themselves.

              Not to mention that they are merely the worst of the current crop of search bots from slurp to baidu to google. And the innumerable spambots where there are no civilising tools. It would be a never-ending task. Strategically it is smarter to bypass the problem. I’d change cable suppliers, but there isn’t any other choice apart from satellite latency. So I’ve done the next best thing.

              This isn’t exactly a paid job and I don’t have time to fight bots on behalf of the Southern Cross cable. It is easier to remove our site from the costs. It is unfortunate that Southern Cross will wind up penalizing our readers instead, but that is less of a problem to me than wasting my time.

    • Mary 3.3
      10 January 2013 at 9:10 am

      Your comment is as silly and non-sensical as something Cameron Slater would say.

      • Steve H 3.3.1
        10 January 2013 at 3:06 pm

        HOw is his comment nonsense?

        You guys are all for “keeping jobs and everything in new zealand to be paid for by everyone but the end user”, then your dear blog owner goes and makes it user-pays, to save some cash

        He’s just as bad as a corporate, but I don’t see you guys getting all angry about him doing so.

        Keep The Standard in New Zealand!

        [lprent: So you’re going to contribute. Good to hear – so how much? And when can I expect to see it? ]

        • Colonial Viper 3.3.1.1
          10 January 2013 at 3:19 pm

          To repeat my earlier comments for the Right Wingnuts

          http://thestandard.org.nz/back-to-work-time/#comment-571732

          In the first case (Fisher & Paykel), hundreds of skilled NZers lost jobs, the country lost significant manufacturing capability and ownership of the company and its IP was transferred to the Chinese, to enrich wealthy shareholders.

          In the second case, The Standard stopped paying monopoly rents on the Southern Cross Cable to overseas corporates and overseas shareholders.

          • Steve H 3.3.1.1.1
            10 January 2013 at 3:35 pm

            So you support competition then? Odd, I’m sure in some of your other posts you write are for supporting government owned monopoly’s on things like power.

            Government owned Air NZ was great, wasn’t it?

            • McFlock 3.3.1.1.1.1
              10 January 2013 at 4:52 pm

              Are you really that incapable of understanding that a person’s opinions on different things can vary according to the characteristics of each different thing, yet still be part of a single coherent and consistent world-view?

            • ropata 3.3.1.1.1.2
              12 January 2013 at 11:10 am

              err, Air NZ went bust under the “guidance” of Brierly and their mates in the NZX. Don’t you recall the Labour govt bailout of $1 billion? Air NZ is now 80% govt owned and doing much better with responsible leadership.

              • Colonial Viper …
                14 January 2013 at 6:47 pm

                So you support competition then? Odd, I’m sure in some of your other posts you write are for supporting government owned monopoly’s on things like power.

                Critical infrastructure and services must be government owned. Monopolies like that need to be kept out of the hands of the profiteering private sector.

    • tc 3.4
      10 January 2013 at 9:11 am

      F&P was sold Bob try and keep up. The standard has reduced it’s costs so that it can continue to deliver on it’s charter and play a role in NZ society….spot the difference, use crayons if it helps.

  4. QoT 4
    10 January 2013 at 7:31 pm

    Bing = the devil.

    I seriously hate Bing, primarily because if I encounter it, it means I’m being forced to use a computer which doesn’t have Firefox/Chrome.

    • lprent 4.1
      10 January 2013 at 11:07 pm

      I have never seen bing itself. Just their bots rampaging and the webmaster tools. Coming to think of it I haven’t used Windows 7, apart from booting up on it long enough to download and burn an ubuntu disk at work. There is half of a terabyte disk with windows there that I never use. And I forgot the damn password.

      There is a vista partition on my laptop, that I will be putting windows 8 in the next few weeks. I want to do some testing of some cross compiling using MinGW/gcc to make sure I know what I have to check.

  5. SHG (not Colonial Viper) 5
    13 January 2013 at 7:07 pm

    searchbots (most recently the bloody bingbot) and spambots causing us to exceed our “free” datacap on overseas traffic

    robots.txt, learn it, love it.

    • lprent 5.1
      14 January 2013 at 12:18 pm

      Of course that is set automatically, and is in fact a virtual file. The problem is that each individual bingbot bot seems to follow it individually (if at all). And of course most of the bots simply ignore it.

Links to post

The server will be getting hardware changes this evening starting at 10pm NZDT.
The site will be off line for some hours.