Please! Kill bing

Written By: - Date published: 12:59 pm, December 28th, 2013 - 34 comments
Categories: internet, interweb - Tags: , ,

Matthew Yglesias at Slate is speculating that Microsoft may kill the bing search engine. His logic is compelling. As the person responsible for running the operations of this site, all I can say is to kill it. Please Microsoft – let the pain of your ratshit spiders die away… It isn’t like anyone uses bing anyway.

Steve Ballmer’s run as Microsoft CEO hasn’t been great for Microsoft’s shareholders, but it’s been a boon to the world since his determination to pour money into an Online Services division that competes with Google on several fronts has given Mountain View a dose of competition. But Ballmer’s been more-or-less fired, and Microsoft’s board is supposed to appoint a new CEO next year.

Will Ballmer’s successor share his determination to go head to head with the king of search? If you were given the job, would you? I wouldn’t. Online Services has been a huge financial disaster for Microsoft. But if it goes away, then suddenly Google becomes a real monopolist.

 Google Search 247,205
 Yahoo Search 7,768
 Bing 5,055
 Google Image Search 5,044
 Google Mobile 1,651
 isearch.avg.com 1,168
 Ask.com 438

Just to point out exactly how little bing is used by the users of this site, this year to date wordpress referral stats for search engines on the right shows the interesting pattern. These are where a query was received at a search engine resulting in a jump to one of our pages or images.

Now admittedly we are not a site that bothers to chase overseas traffic via search engine queries. We’re a specialist political blog site concentrating largely in New Zealand. We’re interested in our advertising paying for our server costs and not trying to make a living off the site. With a country of just over four million people, the majority of whom only have a passing interest in politics, trying to make a living off a free public site would require us to put up a awful lot of non-political pap for people from offshore to find on their search browsers.

That is not us. We’re currently getting about 300 thousand unique users per year according to google analytics. The origin of their visits has 87% are from New Zealand, 3.5% from aussie, 1.9% from the US, 1.9% from the UK, and then it tails off into rats and mice.

Browser Visits New visits Bounce Pages/visit Avg Visit
Duration
1. Firefox 25.65% 26.53% 38.41% 3.34 00:07:58
2. Chrome 24.81% 18.63% 36.17% 3.85 00:09:05
3. Safari 22.63% 14.16% 38.98% 2.88 00:05:50
4. Internet Explorer 19.55% 21.07% 31.59% 3.31 00:06:56
5. Android Browser 3.02% 18.46% 41.26% 2.78 00:05:07
6. Safari (in-app) 2.00% 48.92% 69.50% 1.74 00:01:52
7. Opera 0.95% 22.09% 36.06% 3.46 00:08:56

But back to bing. Let us not forget that people actively change away from bing. Bing is the default search engine on Internet Explorer. Over this year IE was still the fourth largest browser accessing this site (I cut off when we hit rats and mice again). Of course it might be different in a more corporate environment than in a left political blog site. But I really don’t think so. We get a lot of traffic during work hours and it will be done from company machines.

Normally, I’d just ignore Bing in the same way that I do with yandex from yahoo, the chinese baidu spider, and most of the other multitudinous search engine spiders that crawl the site every day. They tend to obey the rules that we post for them to follow, and that Wordfence and some lower level anti-denial of service routines enforce.  Mostly those restrictions are there to catch and kill access to spambots.

There are exactly two services that because of good behaviour that I allow unfettered read rights to the site. They are google search engines and FeedBurner (now owned by google) for the RSS feeds. That is because the site tells them what is changed and new as it is updated and added, and when the best time to scan the site is and they follow it. Neither cause any peak loading on the site and mostly quietly run in the background of the site’s traffic to the target humans.

We provide exactly the same information to bing. But it acts like spambot, trying to suck down the entire 13,453 posts and 668,619 comments as fast as it can whenever it feels like – usually several times per month. It also seems to have a habit of trying to do it from several locations at the same time – some of which come from redmond and aren’t in their server lists. Over this past year when it hasn’t been constrained,  it hasn’t been abnormal for Bing to have days where it is more than a third of our daily load. These days I throttle it to a few handfuls of pages per minute.

It is quite costly for our site to support Bing because it is the only search spider that treats this site as a free resource – which it isn’t. It’s server side costs are so high for us to provide compared to the number of people using it that it is simply net vermin like the spambots. The sooner it dies, the happier I’ll be.

Does anyone here actually use the damn thing?

34 comments on “Please! Kill bing ”

  • Comments are now closed

  1. infused 1
    28 December 2013 at 1:28 pm

    Think your wrong on this.

    1) bing is tied to Facebook.
    2) Facebook hate google
    3) bing is tied to ms devices (xbox, phones etc)
    3) people don’t activly move away from bing. Chrome and other apps change the default search engine
    4) would you really want google being the only search,engine?

    also, while not as accurate, its far more advanced than google

    • QoT 1.1
      28 December 2013 at 1:48 pm

      people don’t activly move away from bing

      From personal experience? If I’m being forced to use IE, and mis-type a URL, and it automatically directs me to a bing search page, I will in fact wilfully click back into the address bar and carefully re-type the URL instead of clicking on a bing result. I hate it that much.

    • lprent 1.2
      28 December 2013 at 2:09 pm

      1) & 2) Who cares what search engine facebook prefers. We get referrals from facebook directly. It is a social media engine – not a search engine.

      Besides, here are the stats for the other referrals over the last year. Bing is less than all of them.


      Facebook 52,590
      kiwiblog.co.nz 18,160
      WordPress 15,835
      Twitter 15,684
      bowalleyroad 12,119
      nominister 11,482
      nzherald.co.nz 7,269
      tumeke 6,762
      norightturn 5,373

      Ummm reminds me. I must stop helping out The Daily blog with referrals. Getting less than 2k referrals back from them this year is not good. We’d usually do that to them in a week.

      3) bing is tied to ms devices (xbox, phones etc)

      The ms devices don’t register much. I suspect that they never will.
      Looking at mobile operating systems. Tablets were 10.92% of visits and phones were 9.83% and of the combination of both of those the operating systems were:-

      iOS 71.08%
      Android 27.04%
      Windows Ph 1.05%
      rats and mice...

      I think that the windows phones percentage has actually gone down over the past two years.
      xbox ? who uses that as a browser FFS? According to me it happened a good 252 times over the last year.

      3) people don’t activly move away from bing. Chrome and other apps change the default search engine

      They all ask. At some point the user makes a choice to change the default search engine.. That is a pathetic excuse.

      4) would you really want google being the only search,engine?

      No. But Yahoo is bigger in our referrals than Bing, and yandex is a hell of lot better behaved than bing’s atrocious net-vermin bots. In fact *every* other search engine bots are orders of magnitude better behaved than bings bots. It is better to have a lot of well behaved bots than a name brand corporate’s “highly advanced” and very badly behaved bots.

      • Naturesong 1.2.1
        28 December 2013 at 2:18 pm

        think that the windows phones percentage has actually gone down over the past two years.

        Hmm, I’m seeing an uptick in fellow techies using Windows phone, specifically HTC.
        They’re cheap, and those who have picked them up have said good things about them (intuitive ease of use, functionality etc)

        Not sure how broad the uptick is though, may be limited to curious techies with spare cash trying out new gadgets.

        • lprent 1.2.1.1
          28 December 2013 at 2:31 pm

          I like HTC phones myself with Android on. The biggest pan is finding phones that fit in the watch/coin pocket of my jeans which my current old HTC one V does.

          Since the start of september…

          iOS 68.51%
          Android 28.82%
          Windows Phone 1.34%

          Not a lot so far

        • RedLogix 1.2.1.2
          28 December 2013 at 2:43 pm

          My link to the net at the moment involves a satellite, a secure VPN connection to a desktop, and a WiFi connection to a Nokia 925 Internet sharing on my own personal account.

          So far I’ve been pretty happy with it. I use a Logitech Bluetooth keyboard with it (one of the ones that pairs to multiple devices easily) and I can use it as a portable office.

          The other PC I’m using is an Intel NUC kit. Because there is always a decent screen available to me wherever I go, between the 925 and the NUC I’ve eliminated the clunky , security risk laptop. And because it’s 100% Windows I don’t need to bother crawling up yet another learning curve.

          Having said that, my browser is always Chrome and for that reason I rarely use Bing.

          • lprent 1.2.1.2.1
            28 December 2013 at 3:59 pm

            Sounds like me when I’m out and about and needing to access servers (well without the sat? satphone?).

            Nexus7 tablet or laptop (I7 Win7 or DualCore ubuntu or Macbook) to HTC one V to cellnet. Then copper and ssh/vpn/sftp/nfs to access home server. Then copper and ssh/vpn/sftp/nfs from there to the server I actually want to access.

            It is a pain in the arse because while I’m using passphrase and keys for everything, I also restrict all servers to a very limited number of accessing IPs. So I use a locked down base server with several layers of authentication for any mobile access.

            Ummm now I think on it… Ok just upgraded my phone to a HTC one mini with 4G and a bigger data plan. Should arrive when I get back from down south. Should have done it before xmas so it was here before I left. Oh well….

            Plans are weird. What do I want with unlimited minutes of voice? I barely use 50 minutes a month now. Unlimited txts?

            Drop down to the lowest voice plan, add the extra prepaid GB of data that I run out of regularly. 3 GB data transfer on the phone should hold me for a while.

            Hopefully chorus will finally decide that fibre can be inserted into my apartment block in the new year which should reduce the latency a bit.. *sigh*

            • Ron 1.2.1.2.1.1
              28 December 2013 at 7:45 pm

              I hope you don’t have the problems with Fibre I am facing. Living on a R.O.W. even though I am on street frontage I have to get signed approval from owners of all properties on the R.O.W. before I can have fibre. It doesn’t help that many of the other owners live somewhere in deepest China and have never been to NZ. Come New Year I will have to try and get some sense out of Chorus. If your apartments are Unit Title you may face same problem.

              • lprent …
                29 December 2013 at 11:24 am

                Fortunately the rules for apartment blocks are somewhat simplier. The body corporate is able to make the decision for the owners. Since the costs are minimal to the apartment blocks, most of the aggravation is with Chorus and their process.

      • infused 1.2.2
        28 December 2013 at 3:01 pm

        I’m not talking about The Standard, I’m talking in a general sense. Bing is so tied in to stuff it’s not funny. Specifically the new xbox and windows phones with voice commands.

        The facebook / bing deal is big money.

        Don’t get me wrong, I don’t use big either, because it’s so inaccurate, but you’re not going to see it disappear.

        • lprent 1.2.2.1
          28 December 2013 at 4:01 pm

          Maybe not. However one of the jobs over this holiday is to find or write some code to start treating all bingbots and msnbots as being spambots. That means they will be prevented from reading the standard. I’ll send it back to their microsoft’s sites..

          If I do have to write it, then I’ll publish that plugin.

          • infused 1.2.2.1.1
            28 December 2013 at 4:19 pm

            Why not just correctly direct the bots? What’s so hard? I’ve never had issues with them. It’s probably an issue with how you are handling the bots now.

            • Draco T Bastard 1.2.2.1.1.1
              28 December 2013 at 5:25 pm

              He is correctly directing the bots – the MS bots are ignoring the directions. He said that in the bloody post so why don’t you learn to read.

              • McFlock …
                29 December 2013 at 2:26 am

                maybe he’s saying that bingbots have the typical microsoft attitude of having rules that are different and frequently more convoluted than everyone else who does the same job, better.

      • David H 1.2.3
        28 December 2013 at 3:46 pm

        I treat Bing with the same hatred that I reserve for other nasties like Babylon and Conduit’s Malware. I turn it off and remove it from all machines I repair, and yet the first update the machines do, and there it is back again Just like a bad smell.

      • lanthanide 1.2.4
        29 December 2013 at 8:28 am

        Actually yahoo is powered by bing. I’m surprised you didn’t know that. So effectively you can add those referrals together. It’s also another reason why Microsoft won’t want to kill it – it is bringing in some revenue for them.

        Secondly, like many internet services, popularity varies greatly via country. Bing is much more popular in the US than other countries.

        Finally, the use of Xbox to browse the internet is only going to grow, not shrink, with the release of the xbone. Trying to use a tiny, by net standards, NZ political blog as a barometer for worldwide user trends is laughable at best.

        • lprent 1.2.4.1
          29 December 2013 at 11:27 am

          I’m mostly interested in the effect of the bots. They’re still using the yandex bot (and for that matter the msnbots). So if they have cross connections in their own data they can keep bing updated…

    • Naturesong 1.3
      28 December 2013 at 2:12 pm

      I change away from Bing.

      At first it was habit. I recognised this behaviour and decided to spend some time using bing to inform myself.

      I also work with a number of Microsoft products as part of my job (Enterprise infrastructure design – we’re mostly a MS shop, less than 10% of the stuff we do is Unix/Linux), I often (at least once a week) search technet and MSDN resources for white papers, documentation etc.
      I get better results using google to search microsoft sites – you’d expect that at the very least they’d be better at searching their own online presence wouldn’t you?

      I now change away from bing because I know it’s shit.

      • lprent 1.3.1
        28 December 2013 at 2:52 pm

        I write libraries and frameworks for other programmers in a mostly architectural/pathfinder role on greenfield projects. It means that last week (an exceptional release week) I wrote, compiled code, and made installations for debian linux, windows XP, windows 7/8, OS/x 10.6-9, iOS, and some development code for Android 4. Plus wrote bash code on a build servers. It simply wouldn’t have been possible to run to schedule without google… I’ve tried virtually every search engine from stuff like altavista to baidu….

        The internet is part of my expanded brain these days and search tools are my hands. I spend several hours per day looking up everything from parameters to system level API’s to how other people solved structural issues.

        Google isn’t a particularly precise or sophisticated tool compared to something even like altavista from days of yore. What it is a really good tool at finding what is relevant. That is why people switch to it. It is the same reason why I use my own personal code editor (slickedit) across all of my development systems – it saves a lot of time (and I don’t wind up cursing the dumbarse editor in visual studio, the flakiness of eclipse, and the startling flaws of XCode).

        As you say. Google does a better job of indexing the MSDN than Bing does – which is good when I’m hunting a a half-remembered API call that I last saw reading Petzold in 1992. And I love how I can say something like “boost asio 1.43” and the first few items will contain the exact pages I need.

        But my biggest beef with Bing is how its bots interfere with the operation of this site

        • Naturesong 1.3.1.1
          28 December 2013 at 3:09 pm

          Up until google came along altavista was my favourite. Especially for searching universities.

          I tried to go back a few years ago, but it wasn’t the same search engine after Yahoo bought it mangled it up.

    • Draco T Bastard 1.4
      28 December 2013 at 5:18 pm

      3) people don’t activly move away from bing. Chrome and other apps change the default search engine

      /facepalm

      Because installing chrome and other apps isn’t active.

      • karol 1.4.1
        28 December 2013 at 5:40 pm

        Surely there must be a way of removing bing from chrome. I’ve searched for instructions online, but the instructions I’ve found don’t work for me. Bing drives me crazy as google search just switches to bing automatically when using chrome.

        Edit: ah, I think I just removed it!

  2. Morrissey 2
    28 December 2013 at 7:31 pm

    Bring back Netscape.

  3. RedBaronCV 3
    29 December 2013 at 12:05 pm

    search with DuckDuckgo. Use phone with no pictures. Feel seriously undertechnicalised reading above

  4. Chris 4
    29 December 2013 at 2:51 pm

    I managed to rid my computer of Bing easily enough but and there is always a but…

    This appears to be quite a techy story so I hope you don’t mind if I ask a question 🙂

    I need to purchase a new cell phone in the very near future, what I want to know is can you get rid of the apps that come with the phone.

    I only use the phone for texting and calls, I don’t want the bollocks stuff that the phone comes with it.

    Thank you in advance

    Edit: what has happened to the comments menu?

    [lprent: Getting an upgrade. The test server is broken so it is being done on the real site. Keeps getting interrupted by daft things like the washing machine splattering water and pretending to have a leak. ]

    • Lanthanide 4.1
      29 December 2013 at 6:46 pm

      Any phone you buy is coming to come with a lot of pre-loaded ‘apps’ for things like maps, music etc. If it’s android, you can download ‘quarantine’ apps that would let you freeze / hide a lot of those things, but you can’t get rid of them completely. Not sure why you’d want to anyway – just don’t use them. Why get a smart phone if you aren’t going to use anything but txting and calling people anyway? Just get a dumb phone.

      On my phone it did come with a couple (only) of trial things, but again the quarantine app sorted them out from showing up at all.

      • felix 4.1.1
        29 December 2013 at 7:35 pm

        tried “quarantine” app but it needs a rooted ph to be any use.

        what’s the name of the app you used?

    • Draco T Bastard 4.2
      29 December 2013 at 7:05 pm

      what I want to know is can you get rid of the apps that come with the phone.

      If you get an Android device – yes but it’s capable of destroying the phone. You have to achieve root (Android is a offspring of Linux and root is the master administrator mode) which normally requires a rewrite of the code on the phone and if you use the wrong code or it’s bugged then your new phone become an expensive, but tastefully designed, brick.

      Besides, you can ignore the truly bollocks and use the other stuff that can be quite useful such as Maps.

    • Chris 4.3
      29 December 2013 at 9:18 pm

      the changes look great

      • lprent 4.3.1
        29 December 2013 at 9:56 pm

        Few rough edges still and I still have bugs in the (unseen) dynamic load replies tab. I’ll hopefully clean them up while I’m whiling away time in Invercargill.

        The new server looks pretty damn quick as well. Dumped two older and slower servers and put in one with a wider pipe and more grunt. Be interesting to see how the price goes on it (it had a boxing day kind of effects).

  5. Chris 5
    29 December 2013 at 9:16 pm

    Lanthanide, I have very inquisitive grand babies who like pretty pictures etc.

    And besides I’m forever inadvertently clicking the google icon.

    I like simple things in life, all I want to do is make phone calls and type texts, the app technology is not required.

    Thank you all for your replies 🙂

The server will be getting hardware changes this evening starting at 10pm NZDT.
The site will be off line for some hours.