How to: Varnish listen port 80 with systemd

Varnish Cache

This simple tutorial guides you in the setup of Varnish Cache in order to make it work properly with systemd, on Debian Jessie for example.

The Varnish documentation is pretty clear and easy to understand, even if you did not study the entire Debian Administrator Handbook. So you should easily come to the chapter of the official tutorial about putting Varnish on port 80. And this is really easy too. Neverthless, you’re on Debian Jessie, which uses systemd as init system. Cool, but hey, after following the official tutorial, Varnish still doesn’t listen on port 80!

Let’s dive in…

By default (if you installed it with APT) Varnish listens on port 6081, which is good for development purposes, but totally unuseful in production. It acts as a caching reverse proxy for any web server, so it should act as the front door for, let’s say, Apache or nginx. In a production environment you’ll most likely want Varnish to listen on port 80 and put the web server behind the scenes, listening on port 8080.

As said before, the official tutorial explains how to achieve this, let me make a brief summary. First of all, make Apache (let’s take it as an example) listen on port 8080. In order to do this, edit /etc/apache2/ports.conf and replace the line Listen 80 with Listen 8080. The result will be something like this:

Listen 8080

<IfModule ssl_module>
    Listen 443
</IfModule>

<IfModule mod_gnutils.c>
    Listen 443
</IfModule>

Now, you have to update your VirtualHost accordingly. Edit /etc/apache2/sites-available/000-default.conf (or any other VirtualHost config file you use), replace the line <VirtualHost :80> with <VirtualHost :8080> (this may be slightly different in your configuration).

Now it’s time to configure Varnish. Edit the file /etc/default/varnish and search for the configuration with VCL, which looks something like this:

DAEMON_OPTS="-a :6081 
             -T localhost:6082 
             -f /etc/varnish/default.vcl 
             -S /etc/varnish/secret 
             -s malloc,256m"

The value associated with the -a flag is what we actually have to change, in fact it is the port Varnish is listening on. The first line of the snippet above should become:

DAEMON_OPTS="-a :80 

According to the official tutorial, we’re done. Since we changed the default file, we should reload systemd, then restart Varnish:

# systemctl daemon-reload
# systemctl restart varnish.service

Now, check that the new configuration has been correctly applied. Just point your browser to the right URL, for example http://www.example.com. At this stage you should receive a connection error.

Make Varnish listen port 80 with systemd

Want to know the problem? Just have a look at the active processes related to varnish:

$ ps aux | grep varnish

You should see two processes run by the user root with the command /usr/sbin/varnishd -a :6081 -T localhost:6082 -f /etc/varnish/default.vcl -S /etc/varnish/secred -s malloc,256m. Note something familiar? That’s the DAEMON_OPTS we changed before in /etc/default/varnish. So, what happened? Obviously, Varnish didn’t take our new configuration. But why? We followed the official tutorial…

The real problem

The official tutorial is a little bit outdated. Or, better, doesn’t take into account the testing version of Debian, which uses systemd instead of init.d as init system. And this makes a huge difference, explained in a bug report. Basically, /etc/default/varnish is only read by the /etc/init.d/varnish script, not by the systemd init script (/lib/systemd/system/varnish.service).

Now that we know this little detail not reported in the documentation, it’s easy for us to solve the problem.

The easy solution

All we have to do is override the systemd init script of varnish and change something.

# cp /lib/systemd/system/varnish.service /etc/systemd/system/
# nano /etc/systemd/system/varnish.service

We come up with something like this:

[Unit]
Description=Varnish HTTP accelerator

[Service]
Type=forking
LimitNOFILE=131072
LimitMEMLOCK=82000
ExecStartPre=/usr/sbin/varnishd -C -f /etc/varnish/default.vcl
ExecStart=/usr/sbin/varnishd -a :6081 -T localhost:6082 -f /etc/varnish/default.vcl -S /etc/varnish/secret -s malloc,256m
ExecReload=/usr/share/varnish/reload-vcl

[Install]
WantedBy=multi-user.target

Which is similar to what we saw before. We already now that we have to change the port passed as a value to the -a flag:

ExecStart=/usr/sbin/varnishd -a :80 -T localhost:6082 -f /etc/varnish/default.vcl -S /etc/varnish/secret -s malloc,256m

Why don’t just let this script read the /etc/default/varnish file? Because we don’t know, for now, what systemd or other init systems are going to do with default files. They could ignore them in the future, for example. Therefore, the safest solution for us is writing the full command in the script itself.

Problems: reloading Varnish

After discovering what explained above, we could consider the varnish default file completely unuseful. That’s not right. In any moment we could want to reload the Varnish configuration, this way:

# systemctl reload varnish.service

And here come other problems. That command runs the /usr/share/varnish/reload-vcl script, which reads the /etc/default/varnish file. This implies that we have to update both /etc/systemd/system/varnish.service and /etc/default/varnish in order to make Varnish work properly.

Conclusion

I hope this would be helpful for other people that otherwise could waste a lot of time trying to figure out why Varnish doesn’t listen on port 80.

I’ll go on with a series of tutorial related to Varnish, so please consider buying me a coffee to support those free contributions. In particular, we’ll see how to properly configure Varnish 4 to make it work with WordPress. Stay tuned!

CC BY-SA 4.0 How to: Varnish listen port 80 with systemd by Mattia Migliorini is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

deshack

Web Designer freelance, Ubuntu Member, Linux evangelist. Loves working on clear and minimal designs and wants to create beautiful things for different devices.

deshack wrote 82 posts

Post navigation


Comments

  • Kari Kääriäinen

    Nice tutorial, thanks. Helped me understand the init.d/systemd issue. Didn’t work for Ubuntu 16.04 though. Here’s how to do it: https://www.varnish-cache.org/docs/trunk/tutorial/putting_varnish_on_port_80.html#debian-v8-ubuntu-v15-04

  • skonsoft

    Thanks a lot for this tutorial.

    After 2 days, trying to upgrade from ubuntu 14.04 to ubuntu 16.04, i have edcided to keep ubuntu 14.04 until we got clear documentation for this issue.

    Ubuntu 16.04 is not really ready for production…

  • Anon

    Please accept Bitcoin – I’d have tipped you some!

  • Jim Howell

    To follow up on the comment from Greygjhart in more detail, the best solution in my opinion follows.

    A key thing to note below is that you’re not modifying the basic `varnish.service` definition, so the system is prepared to receive upgrades to the package and maintain backwards compatibility without diffing or patching configs. This is the main issue I would have with the article’s suggestion today. This approach also resolves the issue with reloading the service mentioned at the end of the article.

    Create `/etc/systemd/system/varnish.service.d/override.conf` with the following:

    [Service]

    # Actually read the variables from the usual location. This is the crux of the solution.
    EnvironmentFile=/etc/default/varnish

    # Clear existing ExecStart= (required)
    ExecStart=

    # Set a new ExecStart= (using the variables set by the EnvironmentFile)
    # The additional -j and -F options I copied from the original varnish.service definition
    # The -j probably makes sense in any context and thus probably better belongs inside $DAEMON_OPTS
    # The -F only makes sense in this context (systemd) so it should stay here.
    #
    ExecStart=/usr/sbin/varnishd -j unix,user=vcache -F $DAEMON_OPTS

    # Unfortunately there are some additional variables set in the original EnvironmentFile, which are used by the sysv init script.
    # They are NFILES and MEMLOCK, which the sysv init script uses to run `ulimit -n` and `ulimit -l` respectively
    # The base varnish.service file sets these like so:
    # LimitNOFILE=131072
    # LimitMEMLOCK=82000

    # So at least they are set to the same defaults as usual, but if you change them in the EnvironmentFile
    # there is no effect. I'm no systemd expert, but I can't figure out a way to make these values dynamic.
    # I tried the obvious thing:
    # LimitNOFILE=$NFILES
    # But this is a syntax error. I'm not sure if any solution exists.

    So the most common varnish configurations can now be done in the usual place, with pretty good compatibility across sysv, upstart, and systemd. Changes to NFILES and MEMLOCK, if desired, must be done in the `override.conf` drop-in file.

    To go the extra mile we could probably also respect the $START variable somehow, but I’d rather see that variable removed entirely. I don’t think it’s the right place to enable/disable a service. Let your chosen service manager do that.

    • deshack

      Hi Jim, thanks for your contribution. I will update the post ASAP!

    • Ryan H.

      Hey Jim. Brilliant solution.

      When I use $DAEMON_OPTS, I get “Error: socket ${VARNISH_LISTEN_ADDRESS}:${VARNISH_LISTEN_PORT} didn’t resolve”. However, if I use the individual variables, it works, for example:

      ExecStart=/usr/sbin/varnishd -j unix,user=vcache -F -a $VARNISH_LISTEN_ADDRESS:$VARNISH_LISTEN_PORT ...

      This tells me that the override.conf file has access to the vars; looks like the problem might be with the curly braces.

      My /etc/default/varnish file looks something like this:


      VARNISH_LISTEN_ADDRESS=
      VARNISH_LISTEN_PORT=80

      DAEMON_OPTS=”-a ${VARNISH_LISTEN_ADDRESS}:${VARNISH_LISTEN_PORT} \
      -f ${VARNISH_VCL_CONF} \
      -T ${VARNISH_ADMIN_LISTEN_ADDRESS}:${VARNISH_ADMIN_LISTEN_PORT} \
      -t ${VARNISH_TTL} \
      -S ${VARNISH_SECRET_FILE} \
      -s ${VARNISH_STORAGE}”

      Can you show me what your DAEMON_OPTS assignment looks like?

  • Lysergia

    The best option seems to be replacing the ExecStart line with:
    ExecStart=/usr/sbin/varnishd $DAEMON_OPTS

    and adding this line:
    EnvironmentFile=-/etc/default/varnish

    This way, you can just modify /etc/default/varnish instead of copying changes to the systemd file.

    • Lysergia

      No - in EnvironmentFile:

      EnvironmentFile=/etc/default/varnish

  • drollie

    Thanks, very handy. Got me first when upgrading to Jessie.

  • Rian

    thank you boss.. i spent 3 days before found this post. great!

  • Charles

    Hi,

    I’m still on Ubuntu 14.04. What is the Upstart equivalent of the command “systemctl daemon-reload”?

    Thanks and appreciate.

    • deshack

      Hi Charles,
      you have upstart, so please don’t follow this blog posts, as it does not apply to your environment.

  • Hiep Ho

    Awesome! I stuck at changing Varnish parameters and didn’t know why it hadn’t been applied until read your post.
    Keep up good works. Just sent you a cup of coffee.

    • deshack

      Thanks for your support 🙂 I’m glad this post helped you too!

  • Phill

    thanks a million, save me from pulling my hair out 🙂

  • Murray Crowe

    Golly, just upgraded to jessie and I was going round in circles using init.d and editing the default. I think varnish needs to update their docs. My only suggestion is to possibly symlink the systemd varnish.service file rather than copy it. That seems to be what the other packages are doing. Doing an apt upgrade would then detect config changes/conflicts into the future.

    • deshack

      Symlinking it could break in the future, so it’s better to avoid that option.

  • Alexandros

    Can you please let me know why if i update my server, varnish isn’t running automatic?

    • deshack

      Maybe because there’s no init script.

      • Alexandros

        Can you please guide me with how i can create one?

        • deshack

          Well, you have an example in this article 😉

      • Alexandros

        I can see a /etc/init.d/varnish script…

  • Jim Westergren

    Thank you very much and I sent you €15.
    However this “systemctl reload varnish.service” causes:
    Job for varnish.service failed. See ‘systemctl status varnish.service’ and ‘journalctl -xn’ for details.

    And ‘systemctl status varnish.service’ gives me:
    Management port disabled. $DAEMON_OPTS must contain ‘-T hostna…ort’

    However a normal “service varnish restart” works but not “service varnish reload” which gives the same error.

    But it is ok for me for now. I use Debian 8.2 if that matters.

  • Alexandros

    Thanks a lot deshack… You saved my day!

  • Dennis Højgaard

    I can only make it work if i edit /lib/systemd/system/varnish.service directly. Any thougts on that?

    I’m using Debian 8.1

    • deshack

      This is exactly the problem: you should edit that file directly.

      • Ejal de Klerk

        I think you missed something when you replied:
        I had the same issue as Dennis, it only worked when I changed /lib/systemd/system/varnish.service

        This is not mentioned in your article which talks about /etc/

      • Greygjhart

        No you shouldn’t. Files in /lib/systemd/system/ are not meant to be modified directly. You’re supposed to copy them in /etc/systemd/systemd and edit this copy. That way it won’t be erased when you update or anything.

        Dennis, you only missed the ‘systemctl daemon-reload’ that is necessary for systemd to notice that there’s this copy in /etc/ that’s supposed to override the /lib’s one. Then you can start varnish again (‘systemctl start varnish’) and you’ll notice the ‘Loaded’ line in ‘systemctl status varnish’ will have changed : it now references your copy in /etc

        Now, if the Debian varnish packagers had done their job correctly this wouldn’t even have been necessary because the ExecStart would contain variables instead of hard-coded ports and the [Service] section an EnvironmentFile directive pointing to a file setting those variables. And so you wouldn’t have had to copy /lib in the first place, just writing your variable in /etc/default/varnish.

        I don’t know how good a web designer you may be, deshack, but you should really try and understand systemd a bit more before you start giving out pieces of advice like this ‘article’.

        • deshack

          Hi Greygjhart, thank you for your comment. I’m definitely not an expert about systemd and never pretended to be. I found a solution that worked for me and just shared it, since I found none when I needed it. Fortunately in an open source world there are people, like you in this case, that help not only with code. So, thank you for your contribution, I’ll update the post as soon as possible.

  • Lachezar Raychev

    YOU GUYS ARE AWESOME! I TELL YOU… This lost me like a … I don’t know .. 5-6 hours … until I came up on this post. GOOD JOB! GOD BLESS YOU ALL.

  • Stephen Reese

    I found this very helpful and now that Jessie has dropped, I sure others will as well. btw, your “Project” link is broken…

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes:

<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>