Yum update on low ram

After the ‘ShellShock’ vulnerability that came out a couple of days ago, I figured I should probably patch my version of bash.

But there were problems.
I’m running Fedora 20 on a low ram server (512MB). The server has quite a few services (full web-server included stack) running on it. Needless to say RAM was a precious commodity.

When I ran

 yum update bash 

I got

 Error downloading packages: bash-4.2.48-2.fc20.x86_64: [Errno 5] [Errno 12] Cannot allocate memory 

I googled around and found recommendations to cleanup yum/rpm artifacts:

package-cleanup --cleandupes
rpm --rebuilddb
yum clean all

I didn’t have an artifacts and nothing worked.

I checked my ram usage and I still had about 90Mb of free RAM.

Apparently this is not enough.

#fedora told me to add to my ram because Fedora’s minimum RAM requirements are 1Gb.
They also run a full yum update after looking at my system-info.
Given that I hadn’t updated in about 3 months that was probably good advice.

Now it was fairly obvious it was the RAM that was causing problems, so I did what any crazy user would do.

Stop all system services that you can afford downtime on for a couple of minutes/an hour.
This brings down RAM usage. I freed-up about 250Mb RAM like this.

And hopefully you now have enough RAM to power through the update.

run

 yum update 

If all goes well, this probably worked for you.

If it doesn’t the alternative solution would be to do the following.

caveat: it’s nasty, it’s hacky and if you can afford to you should avoid it and you still need some free RAM for this to work, just not as much.

for i in $(yum list updates | tail -n+8 |awk '{print $1}'); do yum update $i -y ; done

This’ll help you install each package one by one, which hopefully takes less RAM than the entire update process, but this is slower as well.
Use at your own risk.

Fedora 20/21 systemctl has problems with mysqld.service

Lots of fun stuff these past couple of hours, so here’s a bunch of short but super quick posts.

Fedora 20 uses MariaDB instead of Oracle’s MySQL (this is a recent change), MariaDB is for most intents and purposes a drop in replacement for MySQL, but ever so often you run into issues.

The way to “start” a service at boot has changed from using init scripts or chkconfig to using systemd’s systemctl enable command.

So to enable mysqld at startup one would run service enable mysqld.service
except that you get an error that looks like this:

Failed to issue method call: No such file or directory

This is because of Bug 1014311 which is basically a problem in systemctl where it doesn’t symlink the symlink.

mysqld.service is a symlink to mariadb.service.

So the workaround is to do

systemctl enable mariadb.service

Setting up: Fedora + Apache + PHP-FPM + MPM_Event

Yello folks, I typically don’t do setup guides, but this one is an exception because I didn’t find a “working” guide for this particular combination anywhere on teh interwebz.
The things I did find were mostly combinations of:

  • Fedora(Old Version)/(Other distribution) + Nginx + PHP-FPM
    or
  • (Some other distribution) + Apache/Nginx + PHP-FPM

None of these guides worked for me and I ended up with PHP’s White Screen of Death aka. WSOD.

First things first, some of you might ask why someone would move to this (weirdish?) setup from
something like the mpm_prefork + mod_php combination that has worked so well for ages.

  1. mpm_prefork is a resource hog compared to other threaded Multi-Processing-Modules (MPM) such as  mpm_worker and mpm_event
  2. mod_php although a very robust plugin, is no longer encouraged for use, the new standard is
    php-fpm or some other form of fcgi scripts, this separates the interpreter pool from the Apache
    workers and restricts things that the scripts can access (since they’re not running under the
    apache user)

Without further ado, the steps:

Install Apache

yum install httpd # Gotta start from the bottom

Install PHP-FPM

yum remove php #You don't need this
yum install php-fpm #Install php-fpm

Setup PHP-FPM

PHP-FPM runs as a separate service on localhost:9000
To start the service, you can do systemctl start php-fpm
Apache forwards all associated requests to PHP-FPM via mod_fcgi_proxy.

Assuming a stock installation, you’ll need to create a configuration file for php-fpm, to tell
apache what to put where.

Create a file called /etc/httpd/conf.d/php-fpm.conf and add the following lines to it,
explanations found in-line.

# Ensure all php files are handled by php-fpm
<FilesMatch .php$> RewriteEngine on
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^ - [R=404,L]
SetHandler "proxy:fcgi://127.0.0.1:9000"
</FilesMatch>

# Public url to test php-fpm is working
# You need to uncomment the coresponding line to enable this option in
# /etc/php-fpm.conf
<Location /ping>
SetHandler "proxy:fcgi://127.0.0.1:9000"
</Location>

AddType text/html .php

# Make index.php default in addition to index.html
DirectoryIndex index.php

To test that everything works well, do the following:

echo "<?php phpinfo() ?>" >/var/www/html/test.php

Then run try accessing the page via the browser at http://your.server.com/test.php.
If everything works as expected, you should see a table with the details of your php installation.
If you see a blank white screen aka the WSOD, check your apache and php-fpm error logs to see what
went wrong. They can be found in /var/log/httpd/error_log and /var/log/php-fpm/error.log

If all goes well you should have a working Apache + PHP-FPM installation.

Replacing MPM_Prefork with MPM_Event

MPM_Event has stayed the default MPM on Linux for a while despite the fact that there are better
performing and more efficient Multithreaded Process Managers out there.

The #httpd community on freenode strongly suggests using mpm_worker or mpm_event wherever one can.

If you don’t know what they are see here.
However if you do know what they are and want to switch from pre-fork because any of the following
reasons:

  • It’s slower to spawn
  • It consumes more resources
  • It gives lower performance vis-a-vis the other mpms

Then continue reading.

All this happens because it needs to create a new process per request and load all the modules into
the child for every request without sharing resources.

To use mpm_event make the following changes:

Open /etc/httpd/conf.modules.d/00-mpm.conf
Uncomment LoadModule mpm_event_module modules/mod_mpm_event.so
Comment LoadModule mpm_prefork_module modules/mod_mpm_prefork.so

Next, Create a file /etc/httpd/conf.d/mpm_event.conf

<IfModule event.c>

#Number of children(servers) Apache will start with
StartServers       2

#This is the combined limit across all servers
MinSpareThreads    128

#This is the combined limit across all servers
MaxSpareThreads    256

#Default value, this limit is per child(server)
# This is the max you can increase ThreadsPerChild to
# without a requiring an httpd stop-start (not restart)
ThreadLimit        768

#Fixed number of threads each child(server) starts up with
ThreadsPerChild    64

#Max number of children(servers) Apache can spawn
ServerLimit        16

#Setting to ServerLimit*ThreadsPerChild + 1024
# Same as MaxClients in older versions of httpd
# Caps the Maximum value of ServerLimiit*ThreadsPerChild that can be set
# to without a requiring an httpd stop-start (not restart)
MaxRequestWorkers  3072
</IfModule>

Feel free to tune the values according to your hardware and server load.

After this simply restart the services:

# Note we don't do systemctl restart httpd
# because certain configs cannot be changed by a restart.
systemctl stop httpd
systemclt start httpd

systemclt restart php-fpm

Moar audio troubles!

Pulseaudio’s been really acting up lately.
For someone who listens to a lot of music (Youtube or otherwise) this is not a nice feeling.
It’s already crashed 17 times today. And the 17th crash proved near fatal. Meaning I wasn’t able to recover from it like the other 16.
Each time pulseaudio crashed earlier I managed to get it back up and running by running these simple commands:

$ # Basically force restart pulseaudio
$ pulseaudio --check ;echo $? # Check if PA is running.
$ pulseaudio -k # Kill any and all existing instances of pa
$ pulseaudio -D # Start PA as a daemon

The 17th time the crash was nastier.
I couldn’t restart it like this, the aforementioned commands refused to start pulseaudio.
So as recommended in the pulseaudio debugging guide I decide to fire up pulse audio like so:
(Also take a look at https://wiki.ubuntu.com/PulseAudio/Log)

$ pulseaudio -vvvv # Extra extra extra verbose. PRINT ALL THE DEBUGZ!

This showed me there was indeed a problem. This was the error I was getting :

E: [pulseaudio] module-ladspa-sink.c: Master sink not found
E: [pulseaudio] module.c: Failed to load module "module-ladspa-sink" (argument: "sink_name=ladspa_output.mbeq_1197.mbeq master=alsa_output.pci-0000_00_1b.0.analog-stereo plugin=mbeq_1197 label=mbeq control=0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0,0"): initialization failed. E: [pulseaudio] main.c: Module load failed.

E: [pulseaudio] main.c: Failed to initialize daemon.

This looked awfully close to what this askubuntu post mentioned http://askubuntu.com/questions/229972/sound-indicator-broken-after-12-10-update-shows-3-dashes-sound-settings-shows.
If ubuntu was having pulseaudio problems then there was bound to be some sort of documentation and some sort of solution.
So I googled around and lo and behold:https://wiki.ubuntu.com/PulseAudio
I’ll paste the relevant bits here:

Troubleshooting

Resetting User Configuration

For some reason, pulseaudio’s user configuration files can become corrupt(unsynced?) in some way, and deleting them (and forcing fresh ones to be generated) fixes a no sound condition. After using the command below, log out/in.

Ubuntu 12.10/Quantal (and earlier)

rm -r ~/.pulse*; pulseaudio -k

Ubuntu 13.04/Raring (and later)

rm -r ~/.config/pulse; pulseaudio -k

Annnd doing this fixed my issue. I still don’t know what causes it.
I’ve filed a bug for the crash https://bugzilla.redhat.com/show_bug.cgi?id=741084 ; but I’m not sure what’s going to become of it.
I talked to the helpful folk over at #fedora on irc.freenode.net
Most of the people there asked me to upgrade to the newer Fedora 20 (Hiesnbug) to get pulseaudio 4.0 which hopefully has a fix for the issue. On a fully up-to-date Fedora 18, the latest PulseAudio version is 2.1
I plan to upgrade to Fedora 20 over the weekend, It’ll give me a chance to give Fedora’s new fedup (Fedora’s new upgrade tool) a whirl. Maybe I’ll blog about it. Maybe I won’t. Anyway stay tuned.

————————
Relevant information:

$ # Technical specs
 $ uname -r
 3.11.4-101.fc18.x86_64
 $ lsb_release -cd
 Description: Fedora release 18 (Spherical Cow)
 Codename: SphericalCow
 $ pulseaudio --version
 pulseaudio 2.1

Chrome volume trouble.

Volume trouble with Google chrome. Flash videos sound muted/softer (about 50% of normal volume).
Firefox still plays everything correctly.
Diagnosis: Have different versions of flash lib installed.

  • /opt/google/chrome/PepperFlash/libpepflashplayer.so
  • /usr/lib64/flash-plugin/libflashplayer.so

According to https://support.google.com/chrome/answer/108086?hl=en pepperflash is for ChromeOS at the time of writing. Disabling it fixes things.

Relevant googlegroups link: http://productforums.google.com/forum/#!topic/chrome/q-MUBwVpnrg


$ lsb_release -cd
Description: Fedora release 18 (Spherical Cow)
$ uname -r
3.10.10-100.fc18.x86_64
a