Language Selection

English French German Italian Portuguese Spanish

New Content/Layout OK?

Varnish Proxy

Silly me, a poll would not work on the new server. I forgot that with the Varnish cache proxy at the front almost all visitors arrive from the same IP address (the proxy), which means that Drupal would allocate just one vote to all (except registered and presently logged in users). With Drupal upgrade we can perhaps find polling software that overcomes this.

rpaf

You must use mod_rpaf to fix this problem that Varnish introduces.
See eg https://www.varnish-cache.org/lists/pipermail/varnish-misc/2008-September/016470.html
mod_rpaf for EL6 64bit here: http://centos.alt.ru/repository/centos/6/x86_64/mod_rpaf-0.6-2.el6.x86_64.rpm

Proxy

Thank, we will look into it. Currently, a lot of stuff other than the poll (e.g. views being counted) are not compatible with Varnish and it makes it look as though not many people visit and can participate in the site.

For sheer stats you could use

For sheer stats you could use an external (i.e. not cached by varnish) service, such as Google Analytics or run your own Piwik.

Piwik

Google Analytics is spyware, but Piwik would be a possibility (Stallman recently told me that it's good). Can it be installed on a cache proxy? I'd have to gain access to it first. Either way, this would not facilitate per-post page request count. Susan had it set up with a module, but it's no longer working correctly. In turn, rating/sorting posts by popularity is no longer possible, and that's the real downside (the front page can no longer list popular items for today).

The problem is not just that IP addresses are not unique. Some requests are never seen by the CMS and Apache.

For the non-unique addresses

For the non-unique addresses look at mod_rpaf, it was made for this situations.
Is this drupal6 or 7? With 6 varnish integration sucks from what I've seen.

See also
https://drupal.org/project/varnish
https://fourkitchens.atlassian.net/wiki/display/TECH/Configure+Varnish+3+for+Drupal+7

Agreed on Google Analytics. You can just install Piwik on the same host and tell Varnish either not to cache it or you can just set its virtualhost on a port other than 80 so it bypasses Varnish completely.

Varnish

Thanks for the pointers.

Yes, it's Drupal 6 and there are other issues that I am beginning to see, such as lack of updates from the RSS feeds around the page (I am currently investigating this, maybe it's related to a cron job or module config although I very much doubt the latter as I haven't changed configs).

Non-unique addresses could be bypassed as an issue even by writing random IP addresses, but that would enable easy poll rigging. I guess it's not essential for operation of the site, but it's a nice-to-have...

From Drupal.org: "This module provides integration between your Drupal site and the Varnish HTTP Accelerator, an advanced and very fast reverse-proxy system. Basically, Varnish handles serving static files and anonymous page-views for your site much faster and at higher volumes than Apache, in the neighborhood of 3000 requests per second."

I have had such issues with Varnish on top of WordPress and MediaWiki (pages served improperly from cache) and it all makes me wonder if removing Varnish altogether is the best way to proceed.

As for Piwik, I have never tried it before, so I will look into it.

I would keep Varnish on for

I would keep Varnish on for static files (css, js, jpeg etc) and to clean up HTTP traffic (Varnish will not forward incomplete or malformed HTTP requests to the backend, it should also be the front line against synfloods etc).

Here's a sample of what I use (test it first, I'm just beginning with Varnish myself)

director default dns {
.list = {
.port = "8080";
.connect_timeout = 5s;
.first_byte_timeout = 600s;
.between_bytes_timeout = 600s;
.max_connections = 10000;
"172.16.1.53"/32;
}
}
sub vcl_recv {
if (req.url ~ "\.(png|gif|jpg|swf|css|js)$") {
return(lookup);
}
}
sub vcl_fetch {
if (req.url ~ "\.(png|gif|jpg|swf|css|js)$") {
unset beresp.http.set-cookie;
}
if (req.restarts == 0) {
if (req.http.x-forwarded-for) {
set req.http.X-Forwarded-For =
req.http.X-Forwarded-For + ", " + client.ip;
} else {
set req.http.X-Forwarded-For = client.ip;
}
}
}

Then install mod_rpaf and make sure your Apache is listening on port 8080 and add this to /etc/httpd/conf.d/rpaf.conf:
LoadModule rpaf_module modules/mod_rpaf-2.0.so

RPAFenable On
RPAFproxy_ips 127.0.0.1 IPs_OF_THE_SERVER
RPAFsethostname On
RPAFheader X-Forwarded-For

PS: looks like drupal is messing with my comments, here's a text version http://fpaste.org/74672/raw/

Thanks

Thanks, I will look at it and into it in the weekend.

RSS feeds

The Piwik demo looks impressive, I have just given them a word of endorsement.

I am still trying to resolve some other issues we've identified.

I think I found the source of the issue above (RSS feeds). It seems like any external site access is denied by default, which helps explain why RSS feeds cannot be retrieved by the Drupal part of the site:


[root@tuxmachines ~]# wget lxer.com
--2014-02-05 04:34:37--  http://lxer.com/
Resolving lxer.com... 108.166.170.174
Connecting to lxer.com|108.166.170.174|:80... failed: Connection refused.
[root@tuxmachines ~]# wget linuxtoday.com
--2014-02-05 04:34:54--  http://linuxtoday.com/
Resolving linuxtoday.com... 70.42.23.121
Connecting to linuxtoday.com|70.42.23.121|:80... failed: Connection refused.

Looks like a firewall issue

Looks like a firewall issue at the first glance.

Firewall

Nux wrote:

Looks like a firewall issue at the first glance.

Yes, it was a simply issue to tackle. It works now.

Pageview count and polls

I'll have a look and see if configuration can solve not just the polling issue but also pageview count. The site of this module is down and it seems like it may require configuration on the cache server too.

More in Tux Machines

today's howtos and leftovers

  • Linux commands for user management
  • CONSOOM All Your PODCASTS From Your Terminal With Castero
  • Install Blender 3D on Debian 10 (Buster)
  • Things To Do After Installing openSUSE Leap 15.2
  • GSoC Reports: Fuzzing Rumpkernel Syscalls, Part 2

    I have been working on Fuzzing Rumpkernel Syscalls. This blogpost details the work I have done during my second coding period.

  • Holger Levsen: DebConf7

    DebConf7 was also special because it had a very special night venue, which was in an ex-church in a rather normal building, operated as sort of community center or some such, while the old church interior was still very much visible as in everything new was build around the old stuff. And while the night venue was cool, it also ment we (video team) had no access to our machines over night (or for much of the evening), because we had to leave the university over night and the networking situation didn't allow remote access with the bandwidth needed to do anything video. The night venue had some very simple house rules, like don't rearrange stuff, don't break stuff, don't fix stuff and just a few little more and of course we broke them in the best possible way: Toresbe with the help of people I don't remember fixed the organ, which was broken for decades. And so the house sounded in some very nice new old tune and I think everybody was happy we broke that rule.

Programming Leftovers

  • Podcast: COBOL development on the mainframe

    Nic reached out when COBOL hit the news this spring to get some background on what COBOL is good for historically, and where it lives in the modern infrastructure stack. I was able to talk about the basics of COBOL and the COBOL standard, strengths today in concert with the latest mainframes, and how COBOL back-end code is now being integrated into front ends via intermediary databases and data-interchange formats like JSON, which COBOL natively supports.

  • What I learned while teaching C programming on YouTube

    The act of breaking something down in order to teach it to others can be a great way to reacquaint yourself with some old concepts and, in many cases, gain new insights. I have a YouTube channel where I demonstrate FreeDOS programs and show off classic DOS applications and games. The channel has a small following, so I tend to explore the topics directly suggested by my audience. When several subscribers asked if I could do more videos about programming, I decided to launch a new video series to teach C programming. I learned a lot from teaching C, and in the process, I came across some meaningful takeaways I think others will appreciate. Make a plan For my day job, I lead training and workshops to help new and emerging IT leaders develop new skills. Outside of regular work, I also enjoy teaching as an adjunct professor. So I'm very comfortable constructing a course outline and designing a curriculum. That's where I started. If you want to teach a subject effectively, you can't just wing it. Start by writing an outline of what topics you want to cover and figure out how each new topic will build on the previous ones. The "building block" method of adding new knowledge is key to an effective training program.

  • Google's Flutter 1.20 framework is out: VS Code extension and mobile autofill support
  • Google Engineers Propose "Machine Function Splitter" For Faster Performance

    Google engineers have been working on the Machine Function Splitter as their means of making binaries up to a few percent faster thanks to this compiler-based approach. They are now seeking to upstream the Machine Function Splitter into LLVM. The Machine Function Splitter is a code generation optimization pass for splitting code functions into hot and cold parts. They are doing this stemming from research that in roughly half of code functions that more than 50% of the code bytes are never executed but generally loaded into the CPU's data cache.

  • Modernize network function development with this Rust-based framework

    The world of networking has undergone monumental shifts over the past decade, particularly in the ongoing move from specialized hardware into software defined network functions (NFV) for data plane1 and packet processing. While the transition to software has fashioned the rise of SDN (Software-defined networking) and programmable networks, new challenges have arisen in making these functions flexible, efficient, easier to use, and fast (i.e. little to no performance overhead). Our team at Comcast wanted to both leverage what the network does best, especially with regards to its transport capacity and routing mechanisms, while also being able to develop network programs through a modern software lens—stressing testing, swift iteration, and deployment. So, with these goals in mind, we developed Capsule, a new framework for network function development, written in Rust, inspired by Berkeley's NetBricks research, and built-on Intel's Data Plane Development Kit (DPDK).

  • This Week in Rust 350
  • Firefox extended tracking protection

    This Mozilla Security Blog entry describes the new redirect-tracking protections soon to be provided by the Firefox browser.

  • Karl Dubost: Browser developer tools timeline

    I was reading In a Land Before Dev Tools by Amber, and I thought, Oh here missing in the history the beautifully chiseled Opera Dragonfly and F12 for Internet Explorer. So let's see what are all the things I myself didn't know.

  • Daniel Stenberg: Upcoming Webinar: curl: How to Make Your First Code Contribution

    Abstract: curl is a wildly popular and well-used open source tool and library, and is the result of more than 2,200 named contributors helping out. Over 800 individuals wrote at least one commit so far. In this presentation, curl’s lead developer Daniel Stenberg talks about how any developer can proceed in order to get their first code contribution submitted and ultimately landed in the curl git repository. Approach to code and commits, style, editing, pull-requests, using github etc. After you’ve seen this, you’ll know how to easily submit your improvement to curl and potentially end up running in ten billion installations world-wide.

Security: Zoom Holes, New Patches and etcd Project Security Committee

  • Zoombomber crashes court hearing on Twitter hack with Pornhub video
  • Security updates for Wednesday

    Security updates have been issued by Debian (net-snmp), Fedora (mingw-curl), openSUSE (firefox, ghostscript, and opera), Oracle (libvncserver and postgresql-jdbc), Scientific Linux (postgresql-jdbc), SUSE (firefox, kernel, libX11, xen, and xorg-x11-libX11), and Ubuntu (apport, grub2, grub2-signed, libssh, libvirt, mysql-8.0, ppp, tomcat8, and whoopsie).

  • The CNCF etcd project reaches a significant milestone with completion of security audit

    This week, a third-party security audit was published on etcd, the open source distributed key-value store that plays a crucial role in scaling Kubernetes in the cloud. For etcd, this audit was important in multiple ways. The audit validates the project’s maturity and sheds light on some areas where the project can improve. This sort of audit is required criteria for any project in the Cloud Native Computing Foundation (CNCF) to qualify for graduation from the CNCF. Read the CNCF blog post that I co-authored to learn more about the audit and what it uncovered. As one of the project maintainers and one of two members of the etcd Project Security Committee, I’d love to share a few reasons I’m hopeful for etcd’s future and why now is a great time to contribute to etcd’s open source community.

Linux Plumbers Conference and Kernel Developments in METRICFS, FS-Cache, HWMON

  • Application Ecosystem Microconference Accepted into 2020 Linux Plumbers Conference

    We are pleased to announce that the Application Ecosystem Microconference has been accepted into the 2020 Linux Plumbers Conference! The Linux kernel is the foundation of the Linux systems, but it is not much use without applications that run on top of it. The application experience relies on the kernel for performance, stability and responsiveness. Plumbers is the perfect venue to have the kernel and app ecosystems under one roof to discuss and learn together and make a better application experience on the Linux platform.

  • Google Opens Patches For "METRICFS" That They Have Used Since 2012 For Telemetry Data

    The METRICFS file-system has been in use internally at Google since 2012 for exporting system statistics to their telemetry systems with around 200 statistics being exported per machine. They are now posting the METRICFS patches as open-source for review and possible upstreaming. A "request for comments" on METRICFS was sent out today on the Linux kernel mailing list. Their motives for now finally publishing these patches is as a result of the recent Statsfs proposal by a Red Hat engineer for a RAM-based file-system for exposing kernel statistics to user-space. METRICFS has a similar aim to Statsfs.

  • FS-Cache Rewritten But Even Its Developers Are Hesitant About Landing It For Linux 5.9

    FS-Cache provides the Linux kernel with a general purpose cache for network file-systems like NFS and AFS but also other special use-cases like ISO9660 file-systems. FS-Cache has been rewritten for better performance and reliability, among other benefits, and while it has been sent in as a pull request for Linux 5.9 even its own developers provide some caution over landing it this cycle. FS-Cache has seen work to "massively overhaul" it with a variety of improvements. The new and improved FS-Cache will now use async direct I/O in place of snooping for updated pages that in turn means less virtual memory overhead. The new FS-Cache implementation has simpler object management, changes to object invalidation, and a variety of other work.

  • Corsair Commander Pro Driver Sent In To Linux 5.9

    The hardware monitoring (HWMON) subsystem has a new driver that is likely to excite some enthusiasts wanting greater control over thermal monitoring and fan control for their systems. The previously covered Corsair Commander Pro Linux driver is now coming with Linux 5.9. The Commander Pro offers six 4-pin fan ports with PWM controls, two RGB LED channels, and four thermal sensors. An interested user/developer created this Linux driver without the support from Corsair. The thermal and fan control support is in place with this new HWMON driver while the RGB lighting controls are available from OpenRGB.