Language Selection

English French German Italian Portuguese Spanish

New Content/Layout OK?

Varnish Proxy

Silly me, a poll would not work on the new server. I forgot that with the Varnish cache proxy at the front almost all visitors arrive from the same IP address (the proxy), which means that Drupal would allocate just one vote to all (except registered and presently logged in users). With Drupal upgrade we can perhaps find polling software that overcomes this.

rpaf

You must use mod_rpaf to fix this problem that Varnish introduces.
See eg https://www.varnish-cache.org/lists/pipermail/varnish-misc/2008-September/016470.html
mod_rpaf for EL6 64bit here: http://centos.alt.ru/repository/centos/6/x86_64/mod_rpaf-0.6-2.el6.x86_64.rpm

Proxy

Thank, we will look into it. Currently, a lot of stuff other than the poll (e.g. views being counted) are not compatible with Varnish and it makes it look as though not many people visit and can participate in the site.

For sheer stats you could use

For sheer stats you could use an external (i.e. not cached by varnish) service, such as Google Analytics or run your own Piwik.

Piwik

Google Analytics is spyware, but Piwik would be a possibility (Stallman recently told me that it's good). Can it be installed on a cache proxy? I'd have to gain access to it first. Either way, this would not facilitate per-post page request count. Susan had it set up with a module, but it's no longer working correctly. In turn, rating/sorting posts by popularity is no longer possible, and that's the real downside (the front page can no longer list popular items for today).

The problem is not just that IP addresses are not unique. Some requests are never seen by the CMS and Apache.

For the non-unique addresses

For the non-unique addresses look at mod_rpaf, it was made for this situations.
Is this drupal6 or 7? With 6 varnish integration sucks from what I've seen.

See also
https://drupal.org/project/varnish
https://fourkitchens.atlassian.net/wiki/display/TECH/Configure+Varnish+3+for+Drupal+7

Agreed on Google Analytics. You can just install Piwik on the same host and tell Varnish either not to cache it or you can just set its virtualhost on a port other than 80 so it bypasses Varnish completely.

Varnish

Thanks for the pointers.

Yes, it's Drupal 6 and there are other issues that I am beginning to see, such as lack of updates from the RSS feeds around the page (I am currently investigating this, maybe it's related to a cron job or module config although I very much doubt the latter as I haven't changed configs).

Non-unique addresses could be bypassed as an issue even by writing random IP addresses, but that would enable easy poll rigging. I guess it's not essential for operation of the site, but it's a nice-to-have...

From Drupal.org: "This module provides integration between your Drupal site and the Varnish HTTP Accelerator, an advanced and very fast reverse-proxy system. Basically, Varnish handles serving static files and anonymous page-views for your site much faster and at higher volumes than Apache, in the neighborhood of 3000 requests per second."

I have had such issues with Varnish on top of WordPress and MediaWiki (pages served improperly from cache) and it all makes me wonder if removing Varnish altogether is the best way to proceed.

As for Piwik, I have never tried it before, so I will look into it.

I would keep Varnish on for

I would keep Varnish on for static files (css, js, jpeg etc) and to clean up HTTP traffic (Varnish will not forward incomplete or malformed HTTP requests to the backend, it should also be the front line against synfloods etc).

Here's a sample of what I use (test it first, I'm just beginning with Varnish myself)

director default dns {
.list = {
.port = "8080";
.connect_timeout = 5s;
.first_byte_timeout = 600s;
.between_bytes_timeout = 600s;
.max_connections = 10000;
"172.16.1.53"/32;
}
}
sub vcl_recv {
if (req.url ~ "\.(png|gif|jpg|swf|css|js)$") {
return(lookup);
}
}
sub vcl_fetch {
if (req.url ~ "\.(png|gif|jpg|swf|css|js)$") {
unset beresp.http.set-cookie;
}
if (req.restarts == 0) {
if (req.http.x-forwarded-for) {
set req.http.X-Forwarded-For =
req.http.X-Forwarded-For + ", " + client.ip;
} else {
set req.http.X-Forwarded-For = client.ip;
}
}
}

Then install mod_rpaf and make sure your Apache is listening on port 8080 and add this to /etc/httpd/conf.d/rpaf.conf:
LoadModule rpaf_module modules/mod_rpaf-2.0.so

RPAFenable On
RPAFproxy_ips 127.0.0.1 IPs_OF_THE_SERVER
RPAFsethostname On
RPAFheader X-Forwarded-For

PS: looks like drupal is messing with my comments, here's a text version http://fpaste.org/74672/raw/

Thanks

Thanks, I will look at it and into it in the weekend.

RSS feeds

The Piwik demo looks impressive, I have just given them a word of endorsement.

I am still trying to resolve some other issues we've identified.

I think I found the source of the issue above (RSS feeds). It seems like any external site access is denied by default, which helps explain why RSS feeds cannot be retrieved by the Drupal part of the site:


[root@tuxmachines ~]# wget lxer.com
--2014-02-05 04:34:37--  http://lxer.com/
Resolving lxer.com... 108.166.170.174
Connecting to lxer.com|108.166.170.174|:80... failed: Connection refused.
[root@tuxmachines ~]# wget linuxtoday.com
--2014-02-05 04:34:54--  http://linuxtoday.com/
Resolving linuxtoday.com... 70.42.23.121
Connecting to linuxtoday.com|70.42.23.121|:80... failed: Connection refused.

Looks like a firewall issue

Looks like a firewall issue at the first glance.

Firewall

Nux wrote:

Looks like a firewall issue at the first glance.

Yes, it was a simply issue to tackle. It works now.

Pageview count and polls

I'll have a look and see if configuration can solve not just the polling issue but also pageview count. The site of this module is down and it seems like it may require configuration on the cache server too.

More in Tux Machines

Games: Surviving Mars and OpenMW

Kernel and Security: BPF, Mesa, Embedded World, Kernel Address Sanitizer and More

  • Concurrency management in BPF
    In the beginning, programs run on the in-kernel BPF virtual machine had no persistent internal state and no data that was shared with any other part of the system. The arrival of eBPF and, in particular, its maps functionality, has changed that situation, though, since a map can be shared between two or more BPF programs as well as with processes running in user space. That sharing naturally leads to concurrency problems, so the BPF developers have found themselves needing to add primitives to manage concurrency (the "exchange and add" or XADD instruction, for example). The next step is the addition of a spinlock mechanism to protect data structures, which has also led to some wider discussions on what the BPF memory model should look like. A BPF map can be thought of as a sort of array or hash-table data structure. The actual data stored in a map can be of an arbitrary type, including structures. If a complex structure is read from a map while it is being modified, the result may be internally inconsistent, with surprising (and probably unwelcome) results. In an attempt to prevent such problems, Alexei Starovoitov introduced BPF spinlocks in mid-January; after a number of quick review cycles, version 7 of the patch set was applied on February 1. If all goes well, this feature will be included in the 5.1 kernel.
  • Intel Ready To Add Their Experimental "Iris" Gallium3D Driver To Mesa
    For just over the past year Intel open-source driver developers have been developing a new Gallium3D-based OpenGL driver for Linux systems as the eventual replacement to their long-standing "i965 classic" Mesa driver. The Intel developers are now confident enough in the state of this new driver dubbed Iris that they are looking to merge the driver into mainline Mesa proper.  The Iris Gallium3D driver has now matured enough that Kenneth Graunke, the Intel OTC developer who originally started Iris in late 2017, is looking to merge the driver into the mainline code-base of Mesa. The driver isn't yet complete but it's already in good enough shape that he's looking for it to be merged albeit marked experimental.
  • Hallo Nürnberg!
    Collabora is headed to Nuremberg, Germany next week to take part in the 2019 edition of Embedded World, "the leading international fair for embedded systems". Following a successful first attendance in 2018, we are very much looking forward to our second visit! If you are planning on attending, please come say hello in Hall 4, booth 4-280! This year, we will be showcasing a state-of-the-art infrastructure for end-to-end, embedded software production. From the birth of a software platform, to reproducible continuous builds, to automated testing on hardware, get a firsthand look at our platform building expertise and see how we use continuous integration to increase productivity and quality control in embedded Linux.
  • KASAN Spots Another Kernel Vulnerability From Early Linux 2.6 Through 4.20
    The Kernel Address Sanitizer (KASAN) that detects dynamic memory errors within the Linux kernel code has just picked up another win with uncovering a use-after-free vulnerability that's been around since the early Linux 2.6 kernels. KASAN (along with the other sanitizers) have already proven quite valuable in spotting various coding mistakes hopefully before they are exploited in the real-world. The Kernel Address Sanitizer picked up another feather in its hat with being responsible for the CVE-2019-8912 discovery.
  • io_uring, SCM_RIGHTS, and reference-count cycles
    The io_uring mechanism that was described here in January has been through a number of revisions since then; those changes have generally been fixing implementation issues rather than changing the user-space API. In particular, this patch set seems to have received more than the usual amount of security-related review, which can only be a good thing. Security concerns became a bit of an obstacle for io_uring, though, when virtual filesystem (VFS) maintainer Al Viro threatened to veto the merging of the whole thing. It turns out that there were some reference-counting issues that required his unique experience to straighten out. The VFS layer is a complicated beast; it must manage the complexities of the filesystem namespace in a way that provides the highest possible performance while maintaining security and correctness. Achieving that requires making use of almost all of the locking and concurrency-management mechanisms that the kernel offers, plus a couple more implemented internally. It is fair to say that the number of kernel developers who thoroughly understand how it works is extremely small; indeed, sometimes it seems like Viro is the only one with the full picture. In keeping with time-honored kernel tradition, little of this complexity is documented, so when Viro gets a moment to write down how some of it works, it's worth paying attention. In a long "brain dump", Viro described how file reference counts are managed, how reference-count cycles can come about, and what the kernel does to break them. For those with the time to beat their brains against it for a while, Viro's explanation (along with a few corrections) is well worth reading. For the rest of us, a lighter version follows.

Blacklisting insecure filesystems in openSUSE

The Linux kernel supports a wide variety of filesystem types, many of which have not seen significant use — or maintenance — in many years. Developers in the openSUSE project have concluded that many of these filesystem types are, at this point, more useful to attackers than to openSUSE users and are proposing to blacklist many of them by default. Such changes can be controversial, but it's probably still fair to say that few people expected the massive discussion that resulted, covering everything from the number of OS/2 users to how openSUSE fits into the distribution marketplace. On January 30, Martin Wilck started the discussion with a proposal to add a blacklist preventing the automatic loading of a set of kernel modules implementing (mostly) old filesystems. These include filesystems like JFS, Minix, cramfs, AFFS, and F2FS. For most of these, the logic is that the filesystems are essentially unused and the modules implementing them have seen little maintenance in recent decades. But those modules can still be automatically loaded if a user inserts a removable drive containing one of those filesystem types. There are a number of fuzz-testing efforts underway in the kernel community, but it seems relatively unlikely that any of them are targeting, say, FreeVxFS filesystem images. So it is not unreasonable to suspect that there just might be exploitable bugs in those modules. Preventing modules for ancient, unmaintained filesystems from automatically loading may thus protect some users against flash-drive attacks. If there were to be a fight over a proposal like this, one would ordinarily expect it to be concerned with the specific list of unwelcome modules. But there was relatively little of that. One possible exception is F2FS, the presence of which raised some eyebrows since it is under active development, having received 44 changes in the 5.0 development cycle, for example. Interestingly, it turns out that openSUSE stopped shipping F2FS in September. While the filesystem is being actively developed, it seems that, with rare exceptions, nobody is actively backporting fixes, and the filesystem also lacks a mechanism to prevent an old F2FS implementation from being confused by a filesystem created by a newer version. Rather than deal with these issues, openSUSE decided to just drop the filesystem altogether. As it happens, the blacklist proposal looks likely to allow F2FS to return to the distribution since it can be blacklisted by default. Read more

gitgeist: a git-based social network proof of concept

Are you tired of not owning the data or the platform you use for social postings? I know I am. It's hard to say when I "first" used a social network. I've been on email for about 30 years and one of the early ad-hoc forms of social networks were chain emails. Over the years I was asked to join all sorts of "social" things such as IRC, ICQ, Skype, MSN Messenger, etc. and eventually things like Orkut, MySpace, Facebook, etc. I'll readily admit that I'm not the type of person that happily jumps onto every new social bandwagon that appears on the Internet. I often prefer preserving the quietness of my own thoughts. That, though, hasn't stopped me from finding some meaningfulness participating in Twitter, Facebook, LinkedIn and more recently Google+. Twitter was in fact the first social network that I truly embraced. And it would've remained my primary social network had they not killed their own community by culling the swell of independently-developed Twitter clients that existed. That and their increased control of their API effectively made me look for something else. Right around that time Google+ was being introduced and many in the open source community started participating in that, in some ways to find a fresh place where techies can aggregate away from the noise and sometimes over-the-top nature of Facebook. Eventually I took to that too and started using G+ as my primary social network. That is, until Google recently decided to pull the plug on G+. While Google+ might not have represented a success for Google, it had become a good place for sharing information among the technically-inclined. As such, I found it quite useful for learning and hearing about new things in my field. Soon-to-be-former users of G+ have gone in all sorts of directions. Some have adopted a "c'mon guys, get over it, Facebook is the spot" attitude, others have adopted things like Mastodon, others have fallen back to their existing IDs on Twitter, and yet others, like me, are still looking. Read more