Summary: Some numbers to show what goes on in sites that do not share information about their visitors (unlike Windows-centric sites which target non-technical audiences)
THE common perception of GNU/Linux is that it is scarcely used, based on statistics gathered from privacy-hostile Web sites that share (or sell) access log data, embed spyware in all of their pages, and so on. Our sites are inherently different because of a reasonable -- if not sometimes fanatic -- appreciation of privacy at both ends (server and client). People who read technical sites know how to block ads, impede spurious scripts etc. These sites also actively avoid anything which is privacy-infringing, such as interactive 'social' media buttons (these let third parties spy on all visitors in all pages).
Techrights and Tux Machines attract the lion's share our traffic (and server capacity). They both have dedicated servers. These are truly popular and some of the leaders in their respective areas. Techrights deals with threats to software freedom, whereas Tux Machines is about real-time news discovery and organisation (pertaining to Free software and GNU/Linux).
The Varnish layer, which protects both of these large sites (nearly 100,000 pages in each, necessitating a very large cache pool), handles somewhere between a gigabyte to 2.5 gigabytes of data per hour (depending on the time of day, usually somewhere in the middle of this range, on average).
The Apache layer, which now boasts 32 GB of RAM and sports many CPU cores, handled 1,324,232 hits for Techrights (ranked 6636th for traffic in Netcraft) in this past week and 1,065,606 for Tux Machines (ranked 6214th for traffic in Netcraft).
Based on VISITORS Web Log Analyzer, this is what we've had in Techrights:
Unknown: (e.g. bots/spiders): (23.0%)
As a graph (charted with LibreOffice):
Tux Machines reveals a somewhat different pattern. Based on
grepping/filtering the of past month's log at the Apache back end (not Varnish, which would have been a more sensible but harder thing to do), presenting the top 3 only:
One month is as far as retention goes, so it's not possible to show long-term trends (as before, based on Susan's summary of data). Logs older than that are automatically deleted, as promised, for both sites -- forever! We just need a small tail of data (temporarily) for DDOS prevention. █
IN the coming days we will prioritise very recent news and of course important news, but at the same time we shall be catching up with some older but important news that we missed. This means that some older items (one or two weeks old) may occasionally appear. In lieu with requests from readers we will also stop abbreviating long summaries of news, such as today's leftovers and howto roundups. █
THIS COMING WEEK, starting Tuesday in particular, will be a lot less busy than usual because Rianne and I are flying away and will be absent for a couple of weeks. Depending on availability of Wi-Fi, we ought to be able to still post some links, just not the usual volume of links.
We kindly ask anyone who is interested and willing to submit links highlighting relevant news, as every registered user can do that. It will greatly help us run the site while we are very far away in east Asia. █
TOMORROW is my birthday, so we are going away to Liverpool for a while. Over the holidays we won't be too active in this site, at the very least because there is no major news, no announcements of substance, and we also wish to spend some time with our extended family.
As always, anyone in Tux Machines can create an account and submit stories to the front page (as of late only spammers have been doing that almost every morning). We encourage readers to submit any links which they find relevant and of interest to the community. █
My husband has been spending many of his hours fighting blow by blow in the back end, saving Tux Machines from a cyber attacker who really spent his freaking time hammering the website in an attempt to cripple Tux Machines. At first I was bit astonished by how the website behaved while I was posting some articles, I thought of checking the load to make sure the server worked well and to see that every visitor's page request had been served well, only to know that slowness of the website was been masterminded by an attacker. Perhaps this person is so desperate to put the Tux Machines website down, perhaps an enemy of FOSS and Linux advocacy.
We want to reaffirm our visitors and readers and apologise for the slight inconvenience and weired behavior of the website for the previous hours. All we have done is to protect our readers and visitors from this an acceptable gesture even until now he/she has been trying to penetrate the website. My message to this attacker is, leave Tux Machines in peace and go find some games to play with. █
There are rogue bots hammering on this site all day long. It has gone on for quite a few days and it is getting worse. The bots are getting harder to block. Strategies are changing. They are all acting like zombies/botnet and they all have a "Microsoft Windows" in their HTTP header.
The corporate media seems preoccupied with a bug in GNU Bash. It predicts gloom and doom, just as it did when there was a bug in OpenSSL that Microsoft partners dubbed "heartbleed" (although not so much actually happened in terms of damages).
Perhaps it is time to remind that media that Microsoft, with its back doors, is causing turbulence on the Web. Among the outcomes there are GNU/Linux Web sites that are brought down, with administrators who work around the clock trying to block Windows-running PCs from trying to take down their sites. █
Aggregators in Tux Machines have been universally disabled (temporarily we hope) after a week or so of heavy load that took the site down (well, over capacity and hence not accessible). The culprit seems to be mostly -- although not exclusively -- a bunch of bots that hammer on the aggregators with spammy requests. It's sad that so many hours need to be spent just keeping script kiddies out of the site, resulting in fewer bits of output, slower pageloads (performance degradation), and restlessness (monitoring alerts all day long), not to mention crafting of rules that merely keep the site running. Running Tux Machines is not quite as peaceful and trivial/simple as it may seem from the outside. It's like a full-time job, or at least it feels like it, especially whenever the site gets flooded by rogue bots, necessitating special attention 24/7. █