RabbitMQ exits with error during install

Get Help
1

hello all, moving this over from github as it is hopefully a support request rather than a bug report. thanks!

today we hit a new snag upon trying to install libretime in a fresh instance of ubuntu 16 running as a virtual machine. From the ./install output:

               * Configuring RabbitMQ *               
-----------------------------------------------------
Error: unable to connect to node rabbit@libretime: nodedown

DIAGNOSTICS
===========

attempted to contact: [rabbit@libretime]

rabbit@libretime:
  * connected to epmd (port 4369) on libretime
  * epmd reports: node 'rabbit' not running at all
                  no other nodes on libretime
  * suggestion: start the node

current node details:
- node name: 'rabbitmq-cli-8940@libretime'
- home dir: /var/lib/rabbitmq
- cookie hash: 7Y30JTYnxpNxLQwmUmWuPQ==

systemctl status rabbitmq-server.service reports that

● rabbitmq-server.service - RabbitMQ Messaging Server
   Loaded: loaded (/lib/systemd/system/rabbitmq-server.service; enabled; vendor preset: enabled)
   Active: failed (Result: timeout) since mar 2018-09-18 17:35:22 CDT; 2s ago
  Process: 28398 ExecStartPost=/usr/lib/rabbitmq/bin/rabbitmq-server-wait (code=killed, signal=TERM)
  Process: 28397 ExecStart=/usr/sbin/rabbitmq-server (code=killed, signal=TERM)
 Main PID: 28397 (code=killed, signal=TERM)

sep 18 17:33:51 libretime systemd[1]: Starting RabbitMQ Messaging Server...
sep 18 17:34:41 libretime rabbitmq[28398]: Waiting for rabbit@localhost ...
sep 18 17:34:41 libretime rabbitmq[28398]: pid is 28417 ...
sep 18 17:35:21 libretime systemd[1]: rabbitmq-server.service: Start-post operation timed out. Stopping.
sep 18 17:35:22 libretime systemd[1]: Failed to start RabbitMQ Messaging Server.
sep 18 17:35:22 libretime systemd[1]: rabbitmq-server.service: Unit entered failed state.
sep 18 17:35:22 libretime systemd[1]: rabbitmq-server.service: Failed with result 'timeout'.

So then I tried to follow the instructions in Preparing the Server to fix the RabbitMQ hostname

Specifically my 2 attempts to configure RabbitMQ were to edit /etc/rabbitmq/rabbitmq-env.conf

Either with

NODE_IP_ADDRESS=127.0.0.1
HOSTNAME=localhost

or by adding a snippet from the stack overflow question linked from libretime documentation.

export RABBITMQ_NODENAME=rabbit@localhost
export RABBITMQ_NODE_IP_ADDRESS=127.0.0.1
export ERL_EPMD_ADDRESS=127.0.0.1

Either of these do appear to change the hostname in the the error message I see when I run sudo service rabbitmq-server start. Now it says says rabbit@localhost instead of rabbit@libretime, but it’s the same error and it still won’t start, and so will most probably still abort the libretime install script.

The only modifications we’ve made to the fresh ubuntu 16 server are the following. Details in the documentation we’ve been writing simultaneously (in spanish) Not sure if any of them could be related.

  • edited hosts file to resolve our custom base_url back to 127.0.0.1
  • virt-manager virtual machine network set as a br0 from the server
  • VM given fixed ip set in /etc/network/interfaces
  • sshfs loads a shared folder from the server upon VM startup

…and so I could really use some direction. :slight_smile:

thanks everyone!

2

So I found this stackoverflow - ubuntu - rabbitmq-server fails to start after hostname has changed for first time - Stack Overflow

It may be with your virtual IPs that you need to point rabbitmq via the /etc/hosts file at the virtual IP of the server you are trying to get it to communicate with.

3

thanks! i’ll try that now

4

So I did try solutions from the answers you linked, and the error message did change, so they may have addressed the “fully qualified domain name” issue – ie. that rabbitmq couldn’t find the instance of itself running on the same server. However, rabbitMQ still times out and can’t start. At this point we’ve concluded that this may be a lack of resources on our old-school donated server from 2006. :pensive: classic moments from the grassroots…

We are now in the process of getting a faster machine, but have learned a few more things:

  1. It would be great to include both minimum and recommended system requirements in documentation. If it already exists I can’t find it. I remember seeing an issue on github as well where somebody tried with 512mb of ram and it failed, so I think this would be useful to others. It’s tough to specify processor speed, but perhaps a recommendation of at least a mid-range processor that’s less than 10 years old? and then I’d say at least 2gb ram in order to not crawl along using swap.
  2. Not 100% sure, but I think we saw a performance reduction in the virtual machine vs. the “barebone” base system. Yet another potential reason to bite the bullet and look into docker. Haven’t so far just because we have no knowledge/experience with it. https://github.com/ned-kelly/docker-multicontainer-libretime right? @ned-kelly g’day!

Many thanks all!
-ryan

5

Yeah, we have seen RabbitMQ fail due to a lack of sufficient system memory. That may or may not be the case here. The docker thing is certainly interesting. I think it makes sense to distribute LibreTime as a docker image because it can work around the need to install packages etc as they can be provided automatically. The notion of separating the services into separate docker instances could also make sense in terms of providing an easy way to upgrade etc but I don’t know enough of how docker works and haven’t tried it. I guess I’ll install LibreTime on docker on a test computer and see how it all works.

6

Hi mate, sing out if you run into any issues - The current build that’s on Github is currently being used by a local station here in AU in Production (hence why it’s been “split out” into multiple containers with proper mapping of volumes etc).

If you’re going to run this in Production make sure you take a look at the “Things to note/Fixes” section of the Readme and also - change the default passwords if you’re going to be exposing it to the internet :slight_smile:

Best of luck!

7

Robbt, Pull down my repo to get started if you’re new to docker - and then have a poke around the actual Dockerfiles, and Docker Compose files - that’s where the magic lives.

8

Yeah, I’m reading about it, trying to solidify how networking etc work with docker. It’s just a matter of finding the time to install, but the concepts are easy enough. I also want to work further on integrating your PRs but we need to have those in separate branches vs. merging your master.

9

No worries - I’m away at the moment but have a few new fixes I’ll push up in a different branch soon.

10

Cool, yeah just looking at the docker install script fixes that should be mergeable no problem as it just adds new options that shouldn’t impact anyone doing things the default way while allowing the multi-container-docker to run off of libretime-master vs. your repo.