web analytics

Bluehost have been a pain, but it is all over now.

Written By: - Date published: 4:59 am, June 30th, 2009 - 33 comments
Categories: admin, crosby textor, scoundrels - Tags: ,



Ok. The new server is up and running after a few painful days with the system being suspended, and then after moving it to a backup and having a partition run out of space (yesterdays problem).

The ostensible reason that Bluehost were suspending my account was because the processing usage on their CPU’s was exceeding their (unspecified) limits.  Bluehost thought that suspending the account manually and sending me a e-mail after they’d done it was an appropriate way of handling the issue. The way that this was expressed to me was that there was likely to be a rogue process running on my account.

I wasted considerable time looking for problems in the code and processes. This included staying up most of the night on Friday morning because all they could show me were some evidence of zombie processes from the day before. So I started looking at the server processes at the time that the issue was apparently happening the previous night. Eventually I went to bed. Half an hour later the dipshits suspended the account again. The ‘evidence’ was from when I was watching the site. The only CPU usage was from spiders, crawlers and other ‘bots scanning the posts and comments for the day.

My other thought had been that the site was being targeted in a denial of service attack. However there is no evidence of that either. A careful run through the logs shows that there was a substantial increase in traffic going on. Over the last 5 weeks, there has been close to a third increase in traffic on the site and not much from the ‘bots. The rise in traffic is largely in page-views of the front page and an increase in RSS feeds. Looks like the Mt Albert by-election, Worth, super-city, and other issues have been ramping interest up in political commentary. Hardly surprising as this government seem to be descending into a defensive cycle of  governing incontinence.

About the time I’d figured all of this out, the arseholes at Bluehost had cut me off in their usual version of ‘good’ service. I’d come to the conclusion that Bluehost simply didn’t like sites with growing traffic. Bluehost had been fine for the last 5 months while the site grew back from the post-election and holiday drop in traffic. Now it is up to election month levels, which is obviously more than they care for. Mind you, I can understand why.

Over the weekend I shifted the site to a backup server. It was slow and unfortunately a bit short of disk space. The latter I found out after I went to work on Monday and the database went down while I was reading the mobile version on the bus. Turns out that the log files  grow pretty fast these days and while there was a lot of room across the hard disks, there was very little on the disk partition running the database. But what was interesting was that even on the weekend the site traffic was showing interesting graphs of CPU usage and traffic – like averaging 30% CPU on the relatively quiet weekend traffic. I did some optimisation which should help on the new site, but I’ve got to seriously look at how to channel the site growth so it stops blowing out our servers.

Now we’re back on a server far from Crosby Textor and their habit of using litigation to hide their nasty political tactics. Fools like that, along with the costs, was the primary reason for shifting offshore. It makes it harder to use money as a gagging tool.

This time the server is with a technically orientated company rather than one looking at cost-cutting. Hopefully we’ll now get another 12 months before the site growth forces another crisis.

My apologies for the break in service. But The Standard is a site run on the smell of an oily rag with a lot of volunteers who have jobs and lives outside of politics. And on that note, I’d better hit the sack so I can code in a few hours.



33 comments on “Bluehost have been a pain, but it is all over now.”

  1. infused 1

    Overseas sites are slow zzzz. Should have put the word out for some discounted hosting, im sure many would have helped rather than shifting it overseas.

  2. infused 2

    btw, site is really slow.

  3. infused 3

    some info for you. From home, cable connection. Takes 15-20 seconds for a page load.

    Tracing route to thestandard.org.nz []
    over a maximum of 30 hops:

    1 <1 ms <1 ms <1 ms
    2 * 12 ms 14 ms ****
    3 11 ms 11 ms 11 ms ge0-0.v4wlg0.acsdata.co.nz []
    4 21 ms 19 ms 21 ms ge0-0-35.v4akl1.acsdata.co.nz []
    5 20 ms 20 ms 19 ms Gi1-0.415.gw1.akl1.asianetcom.net [
    6 177 ms 175 ms 174 ms po8-1.gw1.sjc1.asianetcom.net []
    7 146 ms 149 ms 246 ms ip-202-147-50-250.asianetcom.net [
    8 * * * Request timed out.
    9 185 ms 188 ms 188 ms
    10 159 ms 160 ms 160 ms ge1-2-6509-a.castleaccess.com []
    11 160 ms 161 ms 161 ms
    12 160 ms 159 ms 160 ms 69-55-233-164.in-addr.arpa.johncompanies.com [6
    13 160 ms 162 ms 159 ms 69-55-233-153.in-addr.arpa.johncompanies.com [6
    14 159 ms 160 ms 161 ms virt12.johncompanies.com []
    15 * * * Request timed out.
    16 * * * Request timed out.
    17 * * * Request timed out.
    18 * * * Request timed out.

    From work citylink

    1 <1 ms <1 ms <1 ms *****
    2 1 ms 1 ms 1 ms ******
    3 2 ms 2 ms 2 ms dts-slx-gw.dts.net.nz []
    4 4 ms 4 ms 3 ms
    5 * * * Request timed out.
    6 12 ms 13 ms 13 ms
    7 12 ms 15 ms 11 ms Gi1-2.gw1.akl1.asianetcom.net []

    8 138 ms 139 ms 141 ms po8-1.gw1.sjc1.asianetcom.net []
    9 166 ms 166 ms 166 ms ip-202-147-50-250.asianetcom.net [

    10 * * * Request timed out.
    11 299 ms 308 ms 299 ms
    12 153 ms 153 ms 155 ms ge1-1-6509-a.castleaccess.com []
    13 154 ms 152 ms 152 ms
    14 152 ms 153 ms 153 ms 69-55-233-164.in-addr.arpa.johncompanies.com [69
    15 183 ms 156 ms 152 ms 69-55-233-153.in-addr.arpa.johncompanies.com [69
    16 151 ms 152 ms 154 ms virt12.johncompanies.com []
    17 * * * Request timed out.
    18 * * * Request timed out.
    19 ^C

  4. lprent 4

    Yeah I’m puzzled about that. I can’t see any obvious slowdowns.

    But it is definitely slower. I just pulled the YesVote as being the one thing that isn’t cached

  5. infused 5

    Here is the load times… 😉


    very, very, very slow… Cheaper isn’t better.

  6. Maynard J 6

    4:59 am? To classify as ‘a few hours’ sleep you need more than two – I am not sure you would have got that. Thanks, as always, for the work you put into this site. Category seems very appropriate fror Bluehost.

    • lprent 6.1

      Lyn dragged me out of bed after I had about 3 and a bit hours. I usually only sleep for 5 or 6 hours anyway. However this site is seriously cutting into the reading I promised myself after the by-election

  7. Andrew 7

    Site seems to be slower than it has ever been 🙁

  8. Hmmmmm… my hosting is with Bluehost and I haven’t had issues – though I guess I don’t have anywhere near the traffic levels of here.

    The site is very slow for me too so far this morning.

  9. lprent 9

    Fixed the speed issue (I think). Apache was set to some very sub-optimal levels for accepting clients

  10. jarbury 10

    Yeah definitely faster now.

    Back to normal, I might even say. (Touch wood)

  11. lprent 11

    I’ll have a look back in a couple of hours. Have some code to dig through now.

  12. Chris S 12


    I don’t know much about your setup, but if you want to host overseas I suggest investing in VPS’s.

    You can get a few… one with a bunch of RAM to run your database, one or two to run your webservers and get the host to backwire them with private networking so it doesn’t count towards your cap.

    I don’t know what you’re paying for your colo but 3 vps’s will be < $100usd/mo by a lot and you get full root access to your slices.

    Try http://www.linode.com/ some good specs and good prices. If you do, provision them in the Atlanta data center (I get get a solid 1MB/S+ to my vps although latency is higher).

    Feel free to email me if you want a hand… I do this for a job and would be more than happy to help out 🙂 chris@nevermind.co.nz

  13. infused 13

    VPS are crap. For one reason, disks. They can be shared with like 10 users. Try running a database on raid 6 or 10 with 10 or so other ‘servers’ on it. Doesn’t work. Been there, done that.

    • Chris S 13.1

      With the relatively small amount of data used by a blog, the database can keep most of it cached in RAM.

      • lprent 13.1.1

        It does. Most of the 3000 odd posts and 100k comments aren’t accessed except by search-bots and people finding results off them. The bulk goes on 30 posts at the front and the thousand or so comments on those.

        However generating pages is a big exercise on a site that keeps having comments and posts added.

  14. SeaJay 14

    Your efforts are much appreciated Standards.
    Who knew so much goes on behind the pages huh?

  15. The Baron 15

    Nope, still ultra slow on pretty much every device I visit on.

    • lprent 15.1

      I’ll keep tuning it. Only finished shifting it at about 3am. Too many damn graphics

      • The Baron 15.1.1

        Yeah, good luck – substantially improved over the last couple of hours.

        Was hanging on the blackberry and on desktop +firefox, now almost back to normal.

        • lprent

          Bit more tweaking to do. I managed to stop the services when I logged in on the web console. Guess I’d better play with that later. Besides I can do everything ssh

  16. jbc 16

    I’ll second Chris S in that VPS are definitely worth investigation if you ever look to re-home again.

    Like everything else there are good ones and bad ones, but a well-managed VPS host will generally offer far better “bang for the buck” than just about anything else (including a bare-rack colocation service). For one thing: you don’t have to worry about managing the metal 10,000 km away.

    The most important thing by far is how clueful the people are who run your service (as you have discovered).

  17. lprent 17

    This one is a VPS – they just have a different name for it. The site was also on a VPS from Feb 2008 to March 2009 in NZ with a memory upgrade on the way through from 256MB to 512MB RAM.

    It got moved offshore because of cost (web hosting was about 10x cheaper than the NZ VPS and far more capable than any webhost I could find here). A secondary reason is that makes it harder for someone to use the threat of litigation to shut down the site. It forces the expenditure of bigger bucks up front by the litigant to force the host site to topple.

    A website really doesn’t do much database activity, it is maybe 10% of CPU, and little gets done on the disk. Most of the ‘cost’ is generating the pages from the data and compressing them before transmission.

    Unlike the webapp I wrote for campaigning. Now that has some serious database usage.

    infused: most of the VPS’es I was looking at had separate HDD’s per client.

  18. felix 18

    Hey Lynn, just noticed that when I click a “recent comment” link to a comment on the currently loaded page, the whole page reloads instead of just jumping to the comment as it used to.

    • lprent 18.1

      I’ll add it to the buglist (Anita keeps updating it). But not tonight…

      • felix 18.1.1

        Yes, her attention to detail can be staggering 🙂

        I wasn’t raising it as a user bug as such, just thought it might be a (small) waste of your data and/or cpu cycles.

  19. George Darroch 19

    Bluehost have a worsening reputation for doing things like this. They’re also likely to pull your site at the first sign of a cease and desist letter, or any such legalese. See this example regarding Zimbabwe blogs

    I’m glad you’re away from them, and wish you trouble-free hosting!

  20. I’m glad you are back – I did wonder what was going on.

    Sounds like you need to keep searching for another host though.

    I hate coding nightmares like that though – I really felt for you reading through it all. Our comments feed died just over a months ago and I have gone around in circles trying to solve the issue staying awake for hours and wasting time trying to get answers out of Blogger. I have now officially given up and we are going to be migrating ourselves to a hosted WordPress platform as a result – thankfully we have been donated free hosting so we shouldn’t have your issues.

    • lprent 20.1

      Have fun. WordPress is great – lots of flex.

      The problem in this site is that we get a hell of a lot of comments on a lot of posts. Therefore a lot of page views by both readers coming in from the searches and by the various search bugs.

      The front page isn’t too much of a hassle despite the updates from comment counts. It gets the majority of views, but caches reasonably easily. The speed that comments are added isn’t a real hassle.

      The pages getting commented on was why I put a time limit of about a month on leaving comments on the posts at the end of last year. That immediately reduced the load because those could be reported to the search engines as essentially being static.

      However the search engines are all over any page that has comments being updated. I think that googlebot is the most assidous readers of this site. There are a proliferating number of bot’s. They kind of provide a background noise for the whole site (but we ignore them for reporting) which is why our page views are down compared to some sites. We encourage it by having a sitemap plugin that actively tells a number of search engines about what areas are ‘hot’.

      However I think I’ll have to start discouraging anyone who isn’t a major SE.

      Then of course there are the RSS and Feedcatchers, etc. They are restricted to the last 15 posts..

      Anyway, the point is that the actual commentators aren’t actually the real issue. It all those observing them…. 😈

      Boots the traffic up pretty damn high as a base load. Looks to me like we’re going to have to look at some kind of dedicated systems shortly. That means it will move from the kind of penny-pinching that we’ve run it on to date.

  21. Noddy (used to be Dr.No) 21

    Classic. Full marks for introducing Crosby Textor references to an issue that is simply about your inability to manage a website. You guys crack me up sometimes…

    IrishBill: And that’s you on your first warning. Troll again and I’ll ban you for as long as I feel is suitable.

    • lprent 21.1

      It was relevant, If there weren’t litigatous fuckwits like Crosby-Textor around the local political sphere, then we’d just dig out some local sponsorship for the servers.

      However the hardware sponsors would be the first target for anyone wanting to use the courts to silence opposition websites. Just as they have tried with Media7 (read the link)

      It isn’t purely a technical issue.

      Anyway, I think you have the wrong name – should be Noddy. Ummm now it is…….. everywhere through the site. Generally it does pay to annoy a tired sysop 😈

Links to post

Recent Comments

Recent Posts


  • A charge on plastic bags – debunking some myths
    The launch of my Members’ Bill last week, which would introduce a 15 cent charge on single-use plastic bags at the check-out, has generated a lot of comment on mainstream and social media. From The Paul Henry Show at the ...
    GreensBy Denise Roche
    1 day ago
  • National’s $1trillion property sandcastle
    The National government's failure to fix the housing crisis has seen the ballooning and unsustainable property market touch the $1 trillion mark, Labour’s Finance spokesperson Grant Robertson says. "Labour wants an economy that creates high wage work that is based ...
    1 day ago
  • Government failure on housing crisis drives Reserve Bank to add tools
    If the Government was delivering a comprehensive plan to fix the housing crisis, it is unlikely that the Reserve Bank would be continuing to pursue debt to income limits for lending for housing, says Labour’s Finance spokesperson Grant Robertson. ...
    2 days ago
  • International embarrassment for NZ likely over National’s failure to protect Maui dolphin
    New Zealanders who care about Maui dolphin should prepare to feel embarrassed: the Government is about to be put to shame on the international stage for its lack of action to protect Maui’s dolphin. The International Whaling Commissions’ 66th Biennial ...
    GreensBy Eugenie Sage
    4 days ago
  • Why don’t we spend $1b to keep people out of jail, rather than spending it on keeping them in?
    Earlier this week, Corrections Minister Judith Collins announced the government’s ‘solution’ to our burgeoning prison population. It seems that most, if not all, of Bill English’s hard-won surplus is going to disappear into another round of prison-building.  That must be ...
    GreensBy David Clendon
    6 days ago
  • PKE Ship Sent Packing – Not Too Soon
    It is appropriate that the palm kernel expeller (PKE) ship off Tauranga has been sent packing. For weeks I have been saying this ship needed to be sent away, but it seems as if MPI has been trying to find ...
    GreensBy Steffan Browning
    6 days ago
  • Do you #LoveSnow?
    I was a lucky kid. When I was about five or six my mum and auntie took me up to Whakapapa on Mt Ruapehu and taught me to ski. As a young kid I thought there was no bigger ...
    GreensBy James Shaw
    6 days ago
  • Awa Kairangi/Hutt River – Swimmable?
    On Thursday night I hosted a great swimmable rivers meeting organised by the local Greens in Heretaunga (Hutt Valley). It was great to see about 70 people attend and engage in the topic. We were welcomed by Te Atiawa representative ...
    GreensBy Catherine Delahunty
    6 days ago
  • September benefit figures disappointing
    The Government is out of touch with the reality that fewer people are going off the benefit and into employment or study, says Labour’s Social Development spokesperson Carmel Sepuloni.  “The quarterly benefit numbers for September are concerning. They show that ...
    6 days ago
  • MFAT officials refuse to back Prime Minister on Saudi sheep claims
    An Ombudsman’s interim decision released about the existence or otherwise of legal advice on the multimillion dollar Saudi sheep deal shows MFAT has failed to back up the Prime Minister’s claims on the matter, says Labour MP David Parker. “The ...
    6 days ago
  • Barry Coates on his first weeks in Parliament
    Week one in Parliament has been quite an occasion. I would like to share the experience. I had given up on the prospect of getting into Parliament before the election and had been enjoying the diverse work I was doing ...
    GreensBy Barry Coates
    6 days ago
  • Nats still planning to take Housing NZ dividend
    Housing New Zealand’s Statement of Performance Expectations shows that the National Government intends to pocket $237m from Housing New Zealand this year including a $54m “surplus distribution”, despite promises that dividends would stop, says Labour’s Housing spokesperson Phil Twyford. “After ...
    1 week ago
  • Parliament must restore democracy for Ecan
    Parliament has a chance to return full democracy to Canterbury with the drawing of a member’s bill that would replace the Government’s appointed commissioners with democratically elected councillors, says Labour’s Canterbury Spokesperson Megan Woods. “In 2010, the Government stripped Cantabrians ...
    1 week ago
  • Police struggle to hold the line in Northland
    Labour’s promise of a thousand extra police will go a long way to calming the fears of people in the North, says the MP for Te Tai Tokerau Kelvin Davis.  “Police are talking about the Northland towns of Kaitaia and ...
    1 week ago
  • Vote Sooty Shearwater/Tītī for Bird of the Year
    Sooty shearwater (Puffinus griseus) are amazing and deserve your vote in Forest and Bird’s Bird of the Year competition.  They make one of the longest known bird migrations, flying an annual round trip of 64,000 kms across the entire Pacific ...
    GreensBy Eugenie Sage
    1 week ago
  • Urgent action on agriculture emissions needed
    Immediate action is required to curb agricultural emissions is the loud and clear message from Climate change & agriculture: Understanding the biological greenhouse gases report released today by the Parliamentary Commissioner for the Environment, says Labour’s Climate Change spokesperson Megan ...
    1 week ago
  • Super Fund climate change approach a good start
    Labour Finance Spokesperson Grant Robertson and Climate Change Spokesperson Dr Megan Woods have welcomed the adoption of a climate change investment strategy by the New Zealand Super Fund. “This is a good start. It is a welcome development that the Super ...
    1 week ago
  • Energy use going in the wrong direction
    New data out this week from Statistics NZ paints a concerning picture of energy use across the economy under this National Government. You won’t be surprised to hear that there is some seriously worrying information here about how dirty our ...
    GreensBy Gareth Hughes
    1 week ago
  • Raising the age the right thing to do
    The announcement today that the Government will leave the door open for young people leaving state care still means there is a lot of work to do, says Labour's Spokesperson for Children, Jacinda Ardern "The Government indicated some time ago ...
    1 week ago
  • Junior Doctors go on Strike
    Thousands of junior doctors took strike action for 24 hours this week for better working conditions and safer working hours.  The Green Party supports their cause, and particularly their claims to reduce the number of days worked from up to ...
    GreensBy Denise Roche
    1 week ago
  • Coleman plays down the plight of junior doctors
    Junior doctors are crucial to our health services and the industrial action that continues tomorrow shows how desperately the Government has underfunded health, says Labour’s Health spokesperson Annette King.  “Jonathan Coleman’s claim that he has not seen objective evidence of ...
    1 week ago
  • Inflation piles pressure on National and Reserve Bank
    While many households will welcome the low inflation figures announced today, they highlight serious questions for both the National government and the Reserve Bank, Labour’s  Finance Spokesperson Grant Robertson said.  "While low inflation will be welcomed by many, the ...
    1 week ago
  • Officials warned Nat’s $1b infrastructure fund ineffective and rushed
    Treasury papers show the Government rushed out an infrastructure announcement officials told them risked making no significant difference to housing supply, says Labour’s housing spokesperson Phil Twyford.  “Like so much of National’s housing policy, this was another poll-driven PR initiative ...
    1 week ago
  • More cops needed to tackle P
    New Police statistics obtained in Written Questions show John Key is losing his War on P, highlighting the need for more Police, says Opposition Leader Andrew Little.  “New Zealanders expect serious action on P but today’s hodgepodge of half-measures won’t ...
    1 week ago
  • Strengthening our relationship with the Rātana movement
    It was a privilege to visit Rātana Pā last week with fellow Greens’ Co-leader James Shaw, our Māori Caucus and senior staff to meet with the leaders of te iwi mōrehu, to strengthen the ties between the Green Party and ...
    GreensBy Metiria Turei
    1 week ago
  • MBIE docs show country needs KiwiBuild, not Key’s pretend “building boom”
    John Key’s spin that New Zealand is in a building boom does not change the massive shortfall in building construction as new MBIE papers reveal, says Labour Party housing spokesperson Phil Twyford.  “We can fix the housing crisis, by the ...
    1 week ago
  • 1 in 7 Akl houses now going to big property speculators
    Speculators are running riot in the Auckland housing market making life tougher for first home buyers, says Labour’s housing spokesperson Phil Twyford.  Newly released data from Core Logic shows a 40 per cent increase in the share of house sales ...
    2 weeks ago
  • Disconnected thinking dirties the water
    Iain Rabbitts’ belief that drinking water quality, charging for water use and the land use that leads to water quality degradation should be treated separately is part of the problem we have right now in this country. The connection is ...
    GreensBy Catherine Delahunty
    2 weeks ago
  • Report back from Hands Off Our Tamariki hui
    This week I attended a hui in Otaki organised by Hands Off Our Tamariki about the proposed reforms to the Child Young Persons and their Families Act. Moana Jackson and Paora Moyle spoke.  They expressed deep, profound concern about the proposed ...
    GreensBy Jan Logie
    2 weeks ago
  • Labour mourns passing of Helen Kelly
    Helen Kelly was a passionate advocate for working New Zealanders and for a safe and decent working life, Leader of the Opposition Andrew Little says.  “Helen Kelly spent her adult life fighting for the right of every working person to ...
    2 weeks ago
  • National’s visionless immigration policy
    National’s recent immigration announcement is a continuation of the visionless approach to government that it has displayed in the last three terms. Rather than using the levers of government to implement a sustainable immigration policy that benefits new and current ...
    GreensBy Denise Roche
    2 weeks ago
  • Andrew Little: Speech to the Police Association Conference 2016
    Police Association delegates, Association life members and staff, representatives from overseas jurisdictions. Thank you for inviting me here today. The Police Association has become a strong and respected voice for Police officers and for policing in New Zealand. There is ...
    2 weeks ago
  • 1,000 more police for safer communities
    Labour will fund an extra 1,000 Police in its first term to tackle the rising rate of crime, says Leader of the Opposition Andrew Little. “Labour will put more cops on the beat to keep our communities safe. ...
    2 weeks ago
  • Call for all-party round table on homelessness
    Labour is calling on the Government to take part in a roundtable meeting to hammer out a cross-party agreement on ending homelessness.  Labour’s housing spokesperson Phil Twyford said the country wanted positive solutions to homelessness, and wanted the political parties ...
    2 weeks ago
  • Seclusion rooms in schools
    Schools are undoubtedly stretched and underfunded to cope with students with high learning support needs. But this cannot justify the use of rooms (or cupboards) as spaces to forcibly isolate children. It has emerged via media that this practice continues ...
    GreensBy Catherine Delahunty
    2 weeks ago
  • Public should get a say on new Waikato power station
    I had an opinion piece published in the Waikato Times about a controversial proposal to build a new gas-fired power station. It’s not on their website yet, so here it is: If you think the public would get a say ...
    GreensBy Gareth Hughes
    2 weeks ago
  • MSD and their investment approach
    The Government talks about investment but there is no investment. It is not investment if it isn’t over the whole of life and if there is no new money  — Shamubeel Eaqub   Investment sounds like adequate resourcing but this ...
    GreensBy Jan Logie
    2 weeks ago
  • Certainty needed for community services
    A couple of months ago I was at a seminar where three community organisations were presenting. Two of the three presenters were waiting to find out if their organisation would get a contract renewed with MSD. Not knowing if their ...
    GreensBy Jan Logie
    2 weeks ago
  • Domestic Violence – some advice for the media
    For the purpose of this piece, I’m going to use Domestic Violence (DV) as a proxy for intimate partner violence. DV is not isolated to physical abuse in a relationship between people with the same power. DV is a pattern of ...
    GreensBy Jan Logie
    3 weeks ago
  • Leroy’s New Paw Prints
    Leroy, an Auckland great dane recently received a new 3D printed bionic leg after cancer was discovered. I think this is a fantastic story and highlights the real potential of additive manufacturing, or 3D printing Leroy’s prosthetic was printed in titanium and was ...
    GreensBy Gareth Hughes
    3 weeks ago