Bluehost have been a pain, but it is all over now.

Written By: - Date published: 4:59 am, June 30th, 2009 - 33 comments
Categories: admin, crosby textor, scoundrels - Tags: ,

Sysop

Sysop

Ok. The new server is up and running after a few painful days with the system being suspended, and then after moving it to a backup and having a partition run out of space (yesterdays problem).

The ostensible reason that Bluehost were suspending my account was because the processing usage on their CPU’s was exceeding their (unspecified) limits.  Bluehost thought that suspending the account manually and sending me a e-mail after they’d done it was an appropriate way of handling the issue. The way that this was expressed to me was that there was likely to be a rogue process running on my account.

I wasted considerable time looking for problems in the code and processes. This included staying up most of the night on Friday morning because all they could show me were some evidence of zombie processes from the day before. So I started looking at the server processes at the time that the issue was apparently happening the previous night. Eventually I went to bed. Half an hour later the dipshits suspended the account again. The ‘evidence’ was from when I was watching the site. The only CPU usage was from spiders, crawlers and other ‘bots scanning the posts and comments for the day.

My other thought had been that the site was being targeted in a denial of service attack. However there is no evidence of that either. A careful run through the logs shows that there was a substantial increase in traffic going on. Over the last 5 weeks, there has been close to a third increase in traffic on the site and not much from the ‘bots. The rise in traffic is largely in page-views of the front page and an increase in RSS feeds. Looks like the Mt Albert by-election, Worth, super-city, and other issues have been ramping interest up in political commentary. Hardly surprising as this government seem to be descending into a defensive cycle of  governing incontinence.

About the time I’d figured all of this out, the arseholes at Bluehost had cut me off in their usual version of ‘good’ service. I’d come to the conclusion that Bluehost simply didn’t like sites with growing traffic. Bluehost had been fine for the last 5 months while the site grew back from the post-election and holiday drop in traffic. Now it is up to election month levels, which is obviously more than they care for. Mind you, I can understand why.

Over the weekend I shifted the site to a backup server. It was slow and unfortunately a bit short of disk space. The latter I found out after I went to work on Monday and the database went down while I was reading the mobile version on the bus. Turns out that the log files  grow pretty fast these days and while there was a lot of room across the hard disks, there was very little on the disk partition running the database. But what was interesting was that even on the weekend the site traffic was showing interesting graphs of CPU usage and traffic – like averaging 30% CPU on the relatively quiet weekend traffic. I did some optimisation which should help on the new site, but I’ve got to seriously look at how to channel the site growth so it stops blowing out our servers.

Now we’re back on a server far from Crosby Textor and their habit of using litigation to hide their nasty political tactics. Fools like that, along with the costs, was the primary reason for shifting offshore. It makes it harder to use money as a gagging tool.

This time the server is with a technically orientated company rather than one looking at cost-cutting. Hopefully we’ll now get another 12 months before the site growth forces another crisis.

My apologies for the break in service. But The Standard is a site run on the smell of an oily rag with a lot of volunteers who have jobs and lives outside of politics. And on that note, I’d better hit the sack so I can code in a few hours.

G’night

33 comments on “Bluehost have been a pain, but it is all over now.”

  1. infused 1

    Overseas sites are slow zzzz. Should have put the word out for some discounted hosting, im sure many would have helped rather than shifting it overseas.

  2. infused 2

    btw, site is really slow.

  3. infused 3

    some info for you. From home, cable connection. Takes 15-20 seconds for a page load.

    Tracing route to thestandard.org.nz [69.55.236.241]
    over a maximum of 30 hops:

    1 <1 ms <1 ms <1 ms 192.168.3.1
    2 * 12 ms 14 ms ****
    3 11 ms 11 ms 11 ms ge0-0.v4wlg0.acsdata.co.nz [202.21.136.193]
    4 21 ms 19 ms 21 ms ge0-0-35.v4akl1.acsdata.co.nz [202.61.3.2]
    5 20 ms 20 ms 19 ms Gi1-0.415.gw1.akl1.asianetcom.net [202.147.41.2
    5]
    6 177 ms 175 ms 174 ms po8-1.gw1.sjc1.asianetcom.net [203.192.185.50]
    7 146 ms 149 ms 246 ms ip-202-147-50-250.asianetcom.net [202.147.50.25
    ]
    8 * * * Request timed out.
    9 185 ms 188 ms 188 ms 72.165.184.6
    10 159 ms 160 ms 160 ms ge1-2-6509-a.castleaccess.com [69.43.169.112]
    11 160 ms 161 ms 161 ms 69.43.129.84
    12 160 ms 159 ms 160 ms 69-55-233-164.in-addr.arpa.johncompanies.com [6
    .55.233.164]
    13 160 ms 162 ms 159 ms 69-55-233-153.in-addr.arpa.johncompanies.com [6
    .55.233.153]
    14 159 ms 160 ms 161 ms virt12.johncompanies.com [69.55.227.70]
    15 * * * Request timed out.
    16 * * * Request timed out.
    17 * * * Request timed out.
    18 * * * Request timed out.

    From work citylink

    1 <1 ms <1 ms <1 ms *****
    2 1 ms 1 ms 1 ms ******
    3 2 ms 2 ms 2 ms dts-slx-gw.dts.net.nz [202.68.80.1]
    4 4 ms 4 ms 3 ms 172.24.5.97
    5 * * * Request timed out.
    6 12 ms 13 ms 13 ms 172.24.5.105
    7 12 ms 15 ms 11 ms Gi1-2.gw1.akl1.asianetcom.net [203.192.167.173]

    8 138 ms 139 ms 141 ms po8-1.gw1.sjc1.asianetcom.net [203.192.185.50]
    9 166 ms 166 ms 166 ms ip-202-147-50-250.asianetcom.net [202.147.50.250

    10 * * * Request timed out.
    11 299 ms 308 ms 299 ms 72.165.184.6
    12 153 ms 153 ms 155 ms ge1-1-6509-a.castleaccess.com [69.43.169.80]
    13 154 ms 152 ms 152 ms 69.43.129.84
    14 152 ms 153 ms 153 ms 69-55-233-164.in-addr.arpa.johncompanies.com [69
    55.233.164]
    15 183 ms 156 ms 152 ms 69-55-233-153.in-addr.arpa.johncompanies.com [69
    55.233.153]
    16 151 ms 152 ms 154 ms virt12.johncompanies.com [69.55.227.70]
    17 * * * Request timed out.
    18 * * * Request timed out.
    19 ^C

  4. lprent 4

    Yeah I’m puzzled about that. I can’t see any obvious slowdowns.

    But it is definitely slower. I just pulled the YesVote as being the one thing that isn’t cached

  5. infused 5

    Here is the load times… 😉

    http://iforce.co.nz/i/r3b3j2jg.jpg

    very, very, very slow… Cheaper isn’t better.

  6. Maynard J 6

    4:59 am? To classify as ‘a few hours’ sleep you need more than two – I am not sure you would have got that. Thanks, as always, for the work you put into this site. Category seems very appropriate fror Bluehost.

    • lprent 6.1

      Lyn dragged me out of bed after I had about 3 and a bit hours. I usually only sleep for 5 or 6 hours anyway. However this site is seriously cutting into the reading I promised myself after the by-election

  7. Andrew 7

    Site seems to be slower than it has ever been 🙁

  8. Hmmmmm… my hosting is with Bluehost and I haven’t had issues – though I guess I don’t have anywhere near the traffic levels of here.

    The site is very slow for me too so far this morning.

  9. lprent 9

    Fixed the speed issue (I think). Apache was set to some very sub-optimal levels for accepting clients

  10. jarbury 10

    Yeah definitely faster now.

    Back to normal, I might even say. (Touch wood)

  11. lprent 11

    I’ll have a look back in a couple of hours. Have some code to dig through now.

  12. Chris S 12

    lprent,

    I don’t know much about your setup, but if you want to host overseas I suggest investing in VPS’s.

    You can get a few… one with a bunch of RAM to run your database, one or two to run your webservers and get the host to backwire them with private networking so it doesn’t count towards your cap.

    I don’t know what you’re paying for your colo but 3 vps’s will be < $100usd/mo by a lot and you get full root access to your slices.

    Try http://www.linode.com/ some good specs and good prices. If you do, provision them in the Atlanta data center (I get get a solid 1MB/S+ to my vps although latency is higher).

    Feel free to email me if you want a hand… I do this for a job and would be more than happy to help out 🙂 chris@nevermind.co.nz

  13. infused 13

    VPS are crap. For one reason, disks. They can be shared with like 10 users. Try running a database on raid 6 or 10 with 10 or so other ‘servers’ on it. Doesn’t work. Been there, done that.

    • Chris S 13.1

      With the relatively small amount of data used by a blog, the database can keep most of it cached in RAM.

      • lprent 13.1.1

        It does. Most of the 3000 odd posts and 100k comments aren’t accessed except by search-bots and people finding results off them. The bulk goes on 30 posts at the front and the thousand or so comments on those.

        However generating pages is a big exercise on a site that keeps having comments and posts added.

  14. SeaJay 14

    Your efforts are much appreciated Standards.
    Who knew so much goes on behind the pages huh?

  15. The Baron 15

    Nope, still ultra slow on pretty much every device I visit on.

    • lprent 15.1

      I’ll keep tuning it. Only finished shifting it at about 3am. Too many damn graphics

      • The Baron 15.1.1

        Yeah, good luck – substantially improved over the last couple of hours.

        Was hanging on the blackberry and on desktop +firefox, now almost back to normal.

        • lprent 15.1.1.1

          Bit more tweaking to do. I managed to stop the services when I logged in on the web console. Guess I’d better play with that later. Besides I can do everything ssh

  16. jbc 16

    I’ll second Chris S in that VPS are definitely worth investigation if you ever look to re-home again.

    Like everything else there are good ones and bad ones, but a well-managed VPS host will generally offer far better “bang for the buck” than just about anything else (including a bare-rack colocation service). For one thing: you don’t have to worry about managing the metal 10,000 km away.

    The most important thing by far is how clueful the people are who run your service (as you have discovered).

  17. lprent 17

    This one is a VPS – they just have a different name for it. The site was also on a VPS from Feb 2008 to March 2009 in NZ with a memory upgrade on the way through from 256MB to 512MB RAM.

    It got moved offshore because of cost (web hosting was about 10x cheaper than the NZ VPS and far more capable than any webhost I could find here). A secondary reason is that makes it harder for someone to use the threat of litigation to shut down the site. It forces the expenditure of bigger bucks up front by the litigant to force the host site to topple.

    A website really doesn’t do much database activity, it is maybe 10% of CPU, and little gets done on the disk. Most of the ‘cost’ is generating the pages from the data and compressing them before transmission.

    Unlike the webapp I wrote for campaigning. Now that has some serious database usage.

    infused: most of the VPS’es I was looking at had separate HDD’s per client.

  18. felix 18

    Hey Lynn, just noticed that when I click a “recent comment” link to a comment on the currently loaded page, the whole page reloads instead of just jumping to the comment as it used to.

    • lprent 18.1

      I’ll add it to the buglist (Anita keeps updating it). But not tonight…

      • felix 18.1.1

        Yes, her attention to detail can be staggering 🙂

        I wasn’t raising it as a user bug as such, just thought it might be a (small) waste of your data and/or cpu cycles.

  19. George Darroch 19

    Bluehost have a worsening reputation for doing things like this. They’re also likely to pull your site at the first sign of a cease and desist letter, or any such legalese. See this example regarding Zimbabwe blogs

    I’m glad you’re away from them, and wish you trouble-free hosting!

  20. I’m glad you are back – I did wonder what was going on.

    Sounds like you need to keep searching for another host though.

    I hate coding nightmares like that though – I really felt for you reading through it all. Our comments feed died just over a months ago and I have gone around in circles trying to solve the issue staying awake for hours and wasting time trying to get answers out of Blogger. I have now officially given up and we are going to be migrating ourselves to a hosted WordPress platform as a result – thankfully we have been donated free hosting so we shouldn’t have your issues.

    • lprent 20.1

      Have fun. WordPress is great – lots of flex.

      The problem in this site is that we get a hell of a lot of comments on a lot of posts. Therefore a lot of page views by both readers coming in from the searches and by the various search bugs.

      The front page isn’t too much of a hassle despite the updates from comment counts. It gets the majority of views, but caches reasonably easily. The speed that comments are added isn’t a real hassle.

      The pages getting commented on was why I put a time limit of about a month on leaving comments on the posts at the end of last year. That immediately reduced the load because those could be reported to the search engines as essentially being static.

      However the search engines are all over any page that has comments being updated. I think that googlebot is the most assidous readers of this site. There are a proliferating number of bot’s. They kind of provide a background noise for the whole site (but we ignore them for reporting) which is why our page views are down compared to some sites. We encourage it by having a sitemap plugin that actively tells a number of search engines about what areas are ‘hot’.

      However I think I’ll have to start discouraging anyone who isn’t a major SE.

      Then of course there are the RSS and Feedcatchers, etc. They are restricted to the last 15 posts..

      Anyway, the point is that the actual commentators aren’t actually the real issue. It all those observing them…. 😈

      Boots the traffic up pretty damn high as a base load. Looks to me like we’re going to have to look at some kind of dedicated systems shortly. That means it will move from the kind of penny-pinching that we’ve run it on to date.

  21. Noddy (used to be Dr.No) 21

    Classic. Full marks for introducing Crosby Textor references to an issue that is simply about your inability to manage a website. You guys crack me up sometimes…

    IrishBill: And that’s you on your first warning. Troll again and I’ll ban you for as long as I feel is suitable.

    • lprent 21.1

      It was relevant, If there weren’t litigatous fuckwits like Crosby-Textor around the local political sphere, then we’d just dig out some local sponsorship for the servers.

      However the hardware sponsors would be the first target for anyone wanting to use the courts to silence opposition websites. Just as they have tried with Media7 (read the link)

      It isn’t purely a technical issue.

      Anyway, I think you have the wrong name – should be Noddy. Ummm now it is…….. everywhere through the site. Generally it does pay to annoy a tired sysop 😈

Links to post

Recent Comments

Recent Posts

  • Government pays twice the price for emergency housing motels – with two more on the way
    Under Labour’s plan to build at least 1000 state houses each year, New Zealand wouldn’t be paying more than double the valuations for motels to house Kiwis needing emergency housing, says Labour’s Social Development spokesperson Carmel Sepuloni. “Under questioning today, ...
    12 hours ago
  • HAM shows country needs Labour on housing
    The latest Housing Affordability Measure report shows affordability dramatically worsening for Auckland first home buyers, says Labour’s Housing spokesperson Phil Twyford. ...
    15 hours ago
  • Canterbury kids get more support for mental health
    Children in Canterbury and Kaikoura will get dedicated mental health support to help them overcome the trauma of the earthquakes, says the Leader of the Opposition Jacinda Ardern. “We’ll fund an extra eighty mental health professionals for the next three ...
    19 hours ago
  • Statement on Julie Bishop’s comments
    It is highly regrettable that the Australian Foreign Minister Julie Bishop has chosen to make false claims about the New Zealand Labour Party. I have been utterly transparent about this situation. I stand by my statements this morning that I ...
    2 days ago
  • Labour stands with Pike families
    A Labour Government will stand with the families of Pike River and reaffirm its commitment to safe workplaces by ensuring there will be a Minister responsible for Pike River, says Leader of the Opposition Jacinda Ardern. “The Pike River disaster ...
    2 days ago
  • Yes to Sallies – Labour will build more state houses
    The Salvation Army’s latest report ‘Taking Stock’ shows why New Zealand needs a Labour-led Government committed to a massive house building programme, says Labour’s Housing spokesperson Phil Twyford. “When the Sallies say the country needs 2000 extra state houses a ...
    2 days ago
  • New Zealanders deserve better than scaremongering over water
    New Zealanders need to hear from National about how they will fund the clean-up of our rivers and lakes for future generations. Instead, National has broadened its scare-mongering, says Labour’s Water spokesperson David Parker. ...
    3 days ago
  • School Leavers’ Toolkit to equip young people for adult life
    Labour will give school leavers the practical skills and knowledge they need for adult life with a new School Leavers’ Toolkit, says Leader of the Opposition Jacinda Ardern. “Our teachers and schools do a great job of teaching our children ...
    3 days ago
  • Pay equity to be a priority for Labour
      Labour will make sure that the country’s mental health workers are a priority when it comes to pay equity negotiations, says the Leader of the Opposition Jacinda Ardern.  “It is very important for me to right the wrong created ...
    5 days ago
  • Labour’s positive education plan
    Today’s announcement on learning support is more tinkering and proof that only a Labour Government will deliver the resources that schools and parents are crying out for, says Labour’s Education spokesperson Chris Hipkins.  “We have a positive vision for a ...
    6 days ago
  • Pike footage raises questions over government’s actions
    The Government’s seeming determination to turn a blind eye to new questions about what happened at Pike River Mine is troubling, says Labour’s West Coast-Tasman MP Damien O’Connor. ...
    6 days ago
  • Solution to rent rises lies in building houses and stopping speculators
    The spread of rental increases from the big cities to the surrounding regions shows why we need to get on top of the housing shortage build homes our families can afford, and lock out the speculators, says Labour’s Housing spokesperson ...
    6 days ago
  • Clean rivers don’t cost $18 a cabbage
    National is falling into a bad pattern of promising the world and not saying how they will fund it, says Labour’s Water spokesperson David Parker. ...
    7 days ago
  • Time for honest answer on transport funding
    National needs to explain how they will fund the $6 billion funding gap in their 10-year Auckland transport plan, says Labour’s Auckland Issues spokesperson Phil Twyford. ...
    7 days ago
  • Time for true numbers on overseas speculators
    It’s time for the Government to give accurate figures on the number of houses being bought by overseas speculators, says Labour’s Land Information spokesperson Raymond Huo. ...
    7 days ago
  • Fair and sustainable trade: A Green Party vision for New Zealand’s trading relationships
    Trade is a cornerstone of the New Zealand economy. It provides us with the things we want and need, and enables us to pay for those with exports that generate business opportunities and jobs. However, it should be recognised that ...
    GreensBy Barry Coates
    7 days ago
  • Clean rivers for future generations
    Labour will lead a nationwide effort to restore our rivers and lakes to a clean, swimmable state, says Leader of the Opposition Jacinda Ardern. ...
    1 week ago
  • New Zealand really can do better on health
    Labour’s commitment to affordable access to high quality healthcare will provide a better service for New Zealanders than the current Health Minister, who will not apologise for statements that he made that wrongly criticised hard-working staff in the Southern DHB’s ...
    1 week ago
  • Labour’s plan the answer to motorway chaos
    Labour’s plan to build a light rail network and improve heavy rail and bus services across Auckland is the only answer to the kind of motorway congestion Aucklanders endured this morning, says Labour Transport spokesperson Phil Twyford. ...
    1 week ago
  • Labour to build rail to Auckland Airport
    A world class city needs a rail connection from the CBD to its international airport – that’s why Labour will build light rail to Auckland Airport as a priority, says Leader of the Opposition Jacinda Ardern.  “Let’s get Auckland moving ...
    2 weeks ago
  • Is pay equity just too hard for this Govt?
    You are hard pressed these days to find someone that openly admits their misogyny, that men should still be paid more than women. Politicians proclaim that they want to see women paid more, but do their actions back it up? ...
    GreensBy Jan Logie
    2 weeks ago
  • Labour’s commitment to our Rainbow nation
    The Labour Party has reaffirmed its commitment to New Zealand’s rainbow community with its 2017 Rainbow policy, featuring the goal to end HIV in New Zealand by 2025. Grant Robertson says Labour continues a long and proud tradition of advocating ...
    2 weeks ago
  • Labour’s vision for Auckland more than reheated roads
    Labour is more ambitious for Auckland than the reheated set of transport projects proposed by National, says Labour’s Auckland Issues spokesperson Phil Twyford. ...
    2 weeks ago
  • Waiting urology patients are the tip of the iceberg
    The 10 patients waiting for urology surgery at Dunedin Hospital are just the tip of the iceberg, says Labour’s Health spokesperson David Clark.  "Hundreds of patients are waiting for follow-up appointments, but they are not deemed serious enough to be ...
    2 weeks ago
  • Maori Landowners Misled by Maori Party
    Māori landowners are being misled by Government hui being held throughout the country promoting the troubled Māori Land Service (MLS), which underpins the Crown’s unpopular Ture Whenua reforms, says Labour’s Ikaroa-Rāwhiti MP Meka Whaitiri. ...
    2 weeks ago
  • Government’s Johnny-come-lately approach to multinational tax won’t wash
    It’s a case of baby steps for a Government that still allows multinational companies to avoid paying their fair share of tax, says Labour’s Revenue spokesperson Michael Wood. “After nine years in government, five years after the issue of multinational ...
    2 weeks ago
  • Auckland congestion up there with the world’s worst
    Traffic congestion is costing Auckland up to $2 billion in lost productivity according to the latest report from the New Zealand Institute of Economic Research, says Labour’s Transport spokesperson Michael Wood.  “This is a disaster and underlines the need for ...
    2 weeks ago
  • Experience in Youth Parliament 2016
    Being a Member of Youth Parliament was an unexpected, but fabulous opportunity for me. It provided me a way to connect with other young people who have some things in common, and to learn what it is like to be ...
    GreensBy NZ Green Party
    2 weeks ago
  • Labour backs renters’ call for warm, healthy homes
    80 per cent of renters wish their home was warmer and drier, and that’s what Labour will deliver, says Labour’s Housing spokesperson Phil Twyford. ...
    2 weeks ago
  • We can – and must – do better for Kiwi jobs
    Labour has the plan to get more young New Zealanders into jobs and tackle concerns raised in the latest statistics which show an extra 3000 young Kiwis are neither earning or learning compared to the same time last year, says ...
    2 weeks ago
  • Statement from Jacinda Ardern, Leader of the New Zealand Labour Party
    I want to start by giving my thanks to Andrew. His announcement today and the situation we have found ourselves in is not what anyone expected or wanted In my time working with Andrew I know one thing to be ...
    2 weeks ago
  • Better homes for Maori under Labour
    Labour’s vision is that Māori enjoy an equal playing field and have the same home ownership opportunities as non-Māori, says Labour’s Māori Development spokesperson Kelvin Davis. “Today Labour is proud to announce a detailed Māori housing policy from South Auckland’s ...
    3 weeks ago
  • Labour to build replacement for Gorge Road
    Labour will build a new road to replace the Manawatu Gorge Road as quickly as possible, says Leader of the Opposition Andrew Little. ...
    3 weeks ago
  • Government suppresses better transport options for Auckland
    A newly-revealed un-redacted report into options for Auckland freight movement shows the Government has been actively trying to suppress more efficient and cost-effective options to construct a third rail line, says Labour’s Transport spokesperson Michael Wood. “While Auckland’s congestion gets ...
    3 weeks ago
  • How to vote and keep your info private
    Going to vote or enrolling to vote can be scary. Having your details out there can open a can of worms. I wish we lived in a country where everyone was safe and secure enough to be openly enrolled, but ...
    GreensBy Jan Logie
    3 weeks ago
  • Coleman’s lost control of his Ministry
    Basic blunders and chronic underfunding revealed in Treasury documents obtained by Labour clearly show Health Minister Jonathan Coleman has lost control of his ministry, says Labour’s Health spokesperson David Clark. “Every New Zealander deserves affordable access to quality healthcare but ...
    3 weeks ago

%d bloggers like this: