Written By:
lprent - Date published:
2:01 pm, April 4th, 2015 - 61 comments
Categories: admin, The Standard -
Tags:
Last week, very reluctantly 1, I have coughed up the money for a development version of the mobile system used on The Standard. About a quarter of our readers primarily use mobiles to read the site these days. When you add tablets it gets closer to 40%.
Users of the mobile system will have seen some significiant speed improvements since I installed the upgraded version on Friday evening a week ago.
Now our desktop interface can be used on tablets. But it is barely adequate on 7″ tablets (like the ones I use) and older lower resolution tablets in terms of writing comments. But for smaller screens the mobile interface is required, and the comment writing interface is essentially unusable because it doesn’t have any awareness of threaded replies.
I have no figures on it, but I’d envisage that the number of our commenters using mobiles or tables is a whole lot less than 40%. Simply because you can’t use the threaded replies in mobile mode.
I’ve looked at several alternatives, but they all tend to suffer from speed and simplicity issues. The theme we currently use is pretty fast and efficient. But it lacks a few features. Like:-
So right now I’m setting up the test environment to hopefully finally2 fix this issue using the new tools. This probably involves creating a new theme using the scaffold mode based on our existing one.
So if you are interested in giving some ideas about simple features that I should build in. Now is your chance. Subject to vagaries of work, I hope to have this complete before Lyn gets back from Vietnam on the 13th.
The server will be getting hardware changes this evening starting at 10pm NZDT.
The site will be off line for some hours.
Interesting. Have you ruled out replacing this theme with a single responsive one that also handles mobiles and tablets?
There are several problems with themes that are meant to go and do everything from 360/480px wide to something like the 1920px wide.
The first one is that you are usually excessively reliant on javascript and all of the vagaries of the various DOM models. Because I don’t have time to do a lot of maintenance on the site over years of use, I follow the wordpress standard and prefer not to rely on anything except things that are throughly standardised like HTML, CSS2.1, and minimal pretty boring javascript. As a strategy this has worked well, even so it accounts for about 70% of the maintenance work on the site as new browser engines come into play.
If you don’t do it with lots of client side script, then you wind up with chaos in the backend with the caching. Just to give you an idea, this site fully uses about 2GB of memcache.
The reason is that it is caching the enormous numbers of sweeps for information on the 16 thousand posts and 900 thousand individually addressed comments accessed by search engines and their spiders. If we don’t have that, the the background noise of the spiders uses all of the CPU when we get peaks of user activity. That we cache by type of browser is an additional overhead, but not that much of a problem. Starting to add in server side.
The second one is that it is easier to run seperate themes as that means you can stop trying to make one size fits all, with all of the immense numbers of use-cases that involves, the testing, and the support work. Just to give you a really simple idea. What happens in a javascript based resizing theme with multiple columns when you reduce your browser size from 1920 wide to 360 wide? Or vice versa.
Have a look at Boots Theory to see what I mean. If you looked at the code required to deal with the 15 or so major use cases in that resizing and the absolute dependence on how people have their browsers and machines setup.
We typically get about 45k unique people reading this site in a month – that is more than Stephanies page views. I usually get just a few bug reports. If I had a variant of what Stephanie is using, we’d be getting them every day – especially on browser release days. I wouldn’t be able to moderate or do paid work.
The reality is that for us, we will leave it up to users to decide with a default at each level. Have 3 themes – currenet mobile (enhanced for commenting), current desktop, and a wide screen variant >= 1680px that I still have to find time to code. Each will use the deskspace to their max advantage, and mostly for extra search for wider.
Thank you.
Thanks for your efforts. I access TS on my Samsung S2 and the reply to comment function does work..
Odd. Are you running the mobile theme?
I think there is a reply to the post, but not a reply to particular comment last time I looked.
I am replying to your comment, and I access TS via email link. Hope that helps. Sometimes the submit button is reluctant to work.
I have my S2 set to use the standard desktop theme so that I have the reply to comments function. If I use the mobile theme I don’t get the reply to comments function.
That is what I am trying to fix…
Get rid of the big blocks of red.
On the background? THe ones that only show up while loading and in the title..?
What colour would you prefer?
Just to prove the point. I will change to Cosmic Latte – a kind of BEIGE….
Come on….. I need constructive suggestions
Ok, I’m no expert, just a punter.
I remember the current mobile and iPad versions having a lot of red around and the images appeared in B&W. Not saying it’s easy to design something like that but I ended up going to the full version of the site on both mobile and iPad.
To be honest these “mobile friendly” versions of websites are never as comfortable design-wise as the main site for some reason. I guess it’s to do with the info that can be effectively used on the small screens, etc.
Just do the same font, same wordpress template, same white background, same colour thumbnails, etc, and everyone will feel at home.
Yeah, that I can do in general.
Don’t know what kind of smartphone you have, but my HTC One Mini (picked because it fits in my jeans watch pocket) is impossible with the standard ~1000px wide theme.
can you put the red back at some point so we can compare?
I’ve been using the desktop version for ages now because of the reply buttons. Can’t remember what the red looked like. Is it just at the top? ie the The Standard banner?
The white on grey for the posts looks pretty good to me, which is a surprise.
I looked at it before. The header was red, and it showed red whenever anything was repainting further down.
You can see it at present as it goes from beige to grey.
A recent comments thingy would be good.
Goodonya
Good point. Have to provide off the menu
Threaded comments from a mobile device are possible. Just use Opera Mini and go to the desktop site. I can do it from my old nokia with a 4″ screen
Yes but no reply button so replies from a mobile don’t get threaded, which makes some conversations confusing.
wow.. from the desktop!
Opera mini lets you switch the user agent
The entire point of this is to come up with a mobile theme that can also do threaded comments. You haven’t solved the problem, simply avoided it.
Also Chrome on Android has always allowed this.
First thoughts:
Fuk ’em! You’ll only encourage them to ‘The Standard whilst driving’ when they can’t even walk in a straight line whilst ‘The Standarding’
:p
Be it on your head.
Second thoughts:
neo-yuppiedom; post-modern neo-telephonic instantanium
Third thoughts:
the inevitability you’ll come to regret (in terms of your being expected to maintain it and keep it all looking simply spiffing)
The scaffolding is pretty good. It is very good so far for not requiring much code.
Sorry to say but the mobile version on iPad 4 is often unbearably slow to load – seems to hang/pause for 30s then becomes responsive again. On an iPhone 6 it’s marginally better but still quite slow.
Keep up the great work btw!
Ummm not much I can use to look at that right now. Lyn dragged her iPad to Vietnam with her.
I will try with the iPod touch I have been working with for work.
i mostly use my phone to read the standard, but commenting is a bit of a problem on it cos of the thread issue. so even if there’s something i madly want to reply to i don’t, and then by the time i get back home i feel like the moment’s gone.
but my major problem is that when i click on an article or link (or whatever the touchscreen version of clicking is), i always load what i want to read in a new tab, but the original tab loads that too. and i only have this problem on the standard. happens about 95% of the time and it’s an irritant cos my phone takes ages to load a page (i’m living in chinastan).
anyway, thanks heaps to everyone who keeps this blog going. without it my opinion of nz would be quite a few shades darker.
Exactly why I want to get this in.
All web development appears to be going mobile. I have built a few mobile friendly web sites, its really not that hard.
That said, I personally detest mobile phones and I’m no fan of tablets either. Noisy chattering monsters that eat up your day when you could be having a chat face to face over a coffee.
All I know is the future is smart devices not bloody clunky pc’s or laptops. Keep up the good work and make it happen!
This comes to mind 😉
“My baby saw the future
She doesn’t want to live there any more
It’s lousy science fiction
Gets on your skin and seeps into your bones.”
(Byrne: ‘Dance on Vaseline’)
Oh – and here it is should anyone actually read this comment and then think of having a listen
User experience is always going to be better on a bigger screen with a proper keyboard. Esp for those of us who write long comments 😉 Perhaps we will see a change in debate culture via tech, with those with the bigger machines influencing culture differently than the twitter brigade.
Sorry, but your costs are a pittance. I run so many sites that I make no money off whatsoever, not sure why you are complaining about $100 a year. It’s cheap.
Also, you should be looking to one theme that supports both. There are some very high end themes that do this at very little cost. http://themeforest.net/search?utf8=%E2%9C%93&term=&view=list&sort=sales&date=&category=wordpress&price_min=&price_max=&sales=&rating_min=&platform=
X Theme or Avada.
Most purchased themes also tend to look beautiful. But they have a big problems for our users.
Avada for instance in grid mode sucks client site CPU like crazy. I just used their demo on a single core constrained CPU XP VM with limited RAM and a slow network link to test it and it ran like a over sized brick. We have a lot of people reading this on pretty slow machines (hopefully not as limited as that).
Others just waste space, especially in the front page space. We are primarily a news site The majority of our users don’t want to scroll down to read the days posts. X ethos being a good example of how to build a beautiful blog site that acts like requires several pages to see a days worth of post excerpts on a moderately narrow screen. It was a half brick on the slow system test as well.
Our current desktop theme reads in pretty fast for reading on the slow test for front page and pages. It is slow because of the right column screen saver tabs. But doesn’t stop people from immediately scanning. The biggest pain is the damn synchronous statcounter, which slows things down a lot.
Every multifunction theme I have looked at so far depends excessively on client side javascript. Almost of them are slow at the client side. They are suitable for reading sites, not places where people have the site running for hours in a tab wanting instant refreshes like the commenters do. Sure our current WPTouch mobile theme uses quite a lot of javascript, but that has some real tradeoffs between screen size and speed. There isn’t any reason to carry that through to desktops.
I also found a few that largely did the same chores on the server side not dynamically on client side. But when I tested them on the variety of use-cases for different browsers and systems they cost a lot for caching and server side CPU.
BTW: My job is in writing system level code, not focused on writing scripts or web sites. I don’t run any other websites apart from this one. I literally have no need for this tool except for this purpose on this site designing and maintaining a mobile theme (which means that the other 29 instances I’m paying for are rather redundant). The package costs a bit less than half of the monthly cost for the site for a year.
I take care to keep the costs for this site down. Way way down. That is a whole lot simpler and less time wasting for me than depending on advertising. Not having it speeds the site up a lot for the readers. These days the site is now almost entirely funded by unsought voluntary donations.
I just don’t want to lose money on maintaining the site. So I pennypinch when it comes to adding anything to it. Outside of the server costs, I buy a few tools. For instance the extensions to the RSS feeder, W3 Total Cache, the re-edit, etc and then I adapt them into the site. But they are all performing functions that would take too much time to do myself when there is a perfectly good tool to write them with. This WP Touch upgrade is the same.
Full article text in RSS feed please!
Why? The comments are usually as important as the posts themselves in any political context. It became a choice of feeding everything (which was an issue as the comments came later) or just a teaser. I decided on the teaser.
My RSS reader has an offline mode, I spend part of my day in areas with no Internet access, reading the full article offline helps me keep up to date
There are several problems with RSS on a site like that. One is the way that it is collected and sent at the server side. The system that used to run here would be requested to grab the last 10-30 or so posts by RSS feeds. and peoples RSS readers were set to collect that in 5-15 minute intervals. After the site got popular, this meant that there would be 20+ of those per minute and climbing. Even worse were the people who collected all of the comments. Nett effect was that about 10% of our CPU was going into those, and about 25% of our bandwidth.
So I shifted it to feedburner, which mean that that one source came in every few minutes to pick up on stuff because I was informing them of new comments and posts. Still expensive, but manageable if a number of people found it useful.
But I put a tracker on it last year to see what was picking it up, and how many people were coming back to the site. It turned out that only about 4 or 5 people ever replied to comments over a month and that spambots adored the comments feed. I got tens of thousands of attempted spam comments in a month using the tracked links. It was their primary feed for posting crap to the site.
Less than 20 people came in from posts. And that over 94% of the post RSS ‘readers’ were in fact automatic consolidators or spambots grabbing content because they were areas completely outside our 97% audience catchment area. In the last 30 days our incoming sessions (aka visits) for the site look like this
NZ 88.5%
Aussie 3.1%
US 1.9%
UK 1.0%
Germany 0.7%
India 0.4%
Canada 0.4%
Italy 0.3%
South Korea 0.2%
Malaysia 0.2%
In other words, NZ, expats, ESOL and kiwis travelling. It wasn’t much different when I looked at it last year.
The pattern for posts in feedburner was about 3.5% for NZ, and about 6% for all of those areas combined.
In other words, RSS just opened the site to attacks, chewed CPU and bandwidth, and did nothing for more than a tiny fraction of our readers.
Doing only a post excerpt means that feedburner now picks up when post has been made or the excerpt modified. It is enough for the feed columns on other sites, and details what is available to read. Magically it dropped the server load.
Even more magically, as a few weeks that rate of increase on the dumb auto-blocked spam dropped from about four or five thousand per week to a small collection per day. It is still falling. All because I drastically limited links to RSS comments.
But the bot load is still falling as the bots hit the lack of RSS. My bandwidth use is now about 30% compared to what I was seeing at the same time last year. I was using roughly the same other tools to discourage bots then… The NZ percentage of traffic is now about a quarter rather than the 10% it’d gotten down to. That means that the server costs are potentially lower.
What you need is something different. It is a PDF dump of a post and its comments, with no links. Especially for reading. But as far as I can see I don’t have more than 20 people likely to need it. But you’d find that what would do it equally well is to just to use a site dumper (I am sure people will tell you the best ones) to store HTML on your disk.
Don’t set it to run more than a limited pages per minute (I’d have to look it up) as a bot or human or you will hit a lockout designed to rate limit bots.
BTW: For other readers, I allow google unlimited googlebot access because they do it neatly. Their bots are polite and obey the rules and have defined servers. On average the system blocks several hundred false googlebots a day. Everything else is rate limited.
97% of searches on the site come from google according to sitemeter, wordpress stats, and google themselves.
Obviously your intended usage pattern for the site doesn’t match my actual usage. I don’t tend to read the comments. I have higher priorities for my limited time. I try to read the articles to keep up to date with the politics of the country. I have this site and kiwiblog for a left/right balance. Currently it is easier for me to keep up to date with kiwiblog since the full article is in the rss feed.
My ‘RSS Reader’ probably doesn’t even appear to be in NZ as it is a web based service that loads the feed and provides that feed to my devices, or web browser. It does have the ability to try and scrape the story, but it requires an active internet connection for this. I believe it still does this through their servers though. So again, I don’t tend to appear as if I am in New Zealand. Neither would the other people using services like this.
I may be one of the only few that use the site like this, but every vote counts right? and informed voters are better too…
I’ve been in that situation too. Not with ts, but other sites where I’ve had limited internet access and wanted my rss reader to grab the whole post for me to read later.
I’ve yet to find a decent site dumper (I’m on a mac). They’re all too geeky or don’t give me enough control over what I am wanting to do. Not sure how it would work on ts. Setting it to grab the front page plus all internal links one deep?
That is why I looked to see how many people came from RSS to the posts. 20 per month out of 45k unique readers really is a micro-minority. But running a site isn’t free, it costs
David Farrar can afford to cater to the few. He is the pollster for National and seems to use his site as a message testing site for them. He also gets some kind of sweet server deal from friends, probably friends of National, that means his server costs are nothing to minimal. Effectively he is subsidised directly or indirectly by National. We don’t have any similar affiliations.
But we have to pay our way as a true volunteer site. Since we’re now running the sites operational costs at about $250 per month and a lot of volunteer time. That means we cut anything that costs a lot compared to the people we reach. The problem is that those few RSS readers cost the site a hell of a lot. Nothing else does
We provide the excerpts with all of the tags and categories. If something interests you and you don’t comment, then read after the event
In exact dollar terms how much more would it cost per month to include the full article text in the RSS feed?
I don’t know exactly because I don’t know what effects just that change has on the server and data loads compared to dropping RSS comments (I changed them at the same time and it takes months to see effects). To change it is a flick of switch.
The real cost is if I have to step up to more computers to handle peak election loads, which is what I was doing from August (?) 2013 to July 2014. The step then was from $350 per month to $700 per month on AWS as I shifted from having two servers doing the main load to having four.
Now I am on one server with a malfunctioning offshore backup server (that i still have to fix). It is an 8 core system with 32GB RAM running SSDs. It peaks at 25% and averages 8% during the day during the intense interest on the buy-election, whereas during the election (similar levels) with comments RSS on it peaked at 80% and averaged 25%.
Since I have to listen to it running with fans on high when it gets about 30% for any length of time, I’m sensitive that way as well 🙂
But if you really want a feed, find out the IP’s of your scraper and I’ll let them pick a full post feed (I’m a programmer and can make such exceptions easily)
Hi I was scrolling O/M and it gave me the option to alter or delete one of Paul’s comments thought you might want to know.
Ouch. That has to be a caching issue. The question is where.
Mobile or desktop and were you using your own link or at an organization (they often do their own bad webpage caching).
Anyone else seen something similar.
Could you make that feature apply to Puckish Rogue’s comments?
I was on a tablet at home, set to desk top.
I had to resist the urge to change his comment into a full acceptance of the official 9/11 line.
Thank you so much. That is an amazing response. I’ve been given the following ranges as valid for their cloud provider, Digital Ocean, where their VMs run.
45.55.0.0 – 45.55.255.255
46.101.0.0 – 46.101.255.255
80.240.128.0 – 80.240.143.255
104.131.0.0 – 104.131.255.255
104.236.0.0 – 104.236.255.255
107.170.0.0 – 107.170.255.255
128.199.0.0 – 128.199.255.255
162.243.0.0 – 162.243.255.255
178.62.0.0 – 178.62.255.255
185.14.184.0 – 185.14.187.255
188.166.0.0 – 188.166.255.255
192.241.128.0 – 192.241.255.255
192.34.56.0 – 192.34.63.255
192.81.208.0 – 192.81.223.255
198.199.64.0 – 198.199.127.255
198.211.96.0 – 198.211.127.255
208.68.36.0 – 208.68.39.255
Ok. It will probably not be until next weekend, unless I don’t come home from work coded out. I wanted to have a closer look at the RSS code anyway, but it is basically a matter of doing a .htaccess rule.
But, could you remind me on Friday (I’m not a diary kind of person). Email me on lprent at primary.geek.nz
For a while now I’ve not been able to paste into the standard’s comment box on my iphone 5c (8.1.3). I thought it was the iphone after an update, but I’ve just tried another couple of forums and it worked fine. When I try and paste into ts, the ‘paste’ floating button appears briefly but then disappears again.
Paste works fine in Mercury browser but not Safari.
In drop down menu, the archives page doesn’t have the archives on it.
Ah good point. I put it in, but it doesn’t have a page that the mobile theme picks up.
If you have a look at the desktop archives you can see that they are heavily modified.
Updated: Just removed Archive as it doesn’t work. Looks like the changes to WordPress a few years ago killed it..
A jump to top or bottom of page button would be useful for this long threads.
If I get the number threading being able to add and search the number?
Popped to mind earlier for the desktop.
On my laptop I can use command +arrow to get up or down. I think there is a way to do that on the phone but I can’t remember and it’s not obvious. It’s only really a problem on the long threads.
We do get a few of those, and routinely these days, and the sub comments may go on forever. Usually no keyboards on smartphones or tablets unless someone is still using a blackberry.
Maybe a navigation jumper on comments to junk to the next at the same level… Musing..
I think the main time I use it is when I’ve am far down the comments thread and I want to jump back up to the recent comments list. That’s on the desktop version obviously. If you’re going to put the recent comments in the drop down menu, and that’s accessible without having to be at the top of the page, that’d work.
edit, I’ve just seen the back to top of page link at the bottom of the page, doh! Was that always there? I guess it’s those long threads where I’m in the middle or a long way down. But scrolling to the bottom and clicking that button is probably easier than what I do now which is usually just open a new page.
Following on from my discussion about RSS further up. I had a peek at locations in NZ in analytics
This probably has some inaccuracies from GeoIP, especially for Auckland and Wellington head offices for ISPs. It is from 88.5% of the sessions because a number of sessions are dropped due to known inaccuracies.
Auckland 46.1%
Wellington 17.24%
Christchurch 10.3%
Dunedin 4.8%
Hamilton 3.8%
Palmerston North 3.7%
Tauranga 2.7%
Whangarei 1.5%
Lower Hutt 1.4%
Napier 1.2%
New users are roughly inline with those figures as well – but 1.9% in Whangarei and 49.3% in Auckland, probably due to the buy election.
Thanks for your ongoing improvements – much appreciated as I only read the TS on iPhone – much improved over the last few years
Infused – beggars can’t be choosers