open list of urls in a bunch of tabs on chrome on os x

I put the urls in a cr-delimited list in a textfile, "url-list.txt".

Initialize a fakey user data dir for Chrome to use:
$ /Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome --user-data-dir=/tmp/
(Chrome will pop up... pick a search engine and all that nonsense, then hit ctrl+c when done)

Open all the urls in a bunch of tabs:
$ cat url-list.txt | xargs /Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome --user-data-dir=/tmp/

If you don't specify a user data dir, and you have Chrome open (of course you do) you'll get an "Unable to obtain profile lock" error message. Also, you may want to keep this list of urls separate from your user profile.

If you don't want to keep the urls separate from your user profile, exit Chrome first, and just use:
$ cat url-list.txt | xargs /Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome

lazy/easy NFS share on CentOS

There are many guides available online for making NFS shares. But where is the guide for the lazy and stupid person like me? It is here! Hooray!

On the Server...

1. Install packages, if missing: yum install portmap nfs-utils nfs-utils-lib
2. Add line to /etc/hosts.allow as needed, e.g.: portmap: nnn.nnn.0.0/255.255.255.0, to allow other servers in your local network to run portmap against you. You can also allow this on an ip-by-ip basis, e.g. portmap: nnn.nnn.nnn.nnn, nnn.nnn.nnn.nnnn, or you can use wildcards. (more info) Note: if wildcards/etc don't work for you at first, try single IP addresses
3. Add lines to /etc/exports specifying the directories you want to share and the hosts to which you want to share them. E.g.:
   

   /directory/to-share	machine.ip.ad.ress(options)
   /somedir/specific-machine	nnn.nnn.nnn.nnn(rw,no_root_squash,sync)
   /somedir/couple-machines	nnn.nnn.nnn.nnn(ro)	nnn.nnn.nnn.nnn(rw,no_root_squash,sync)
   /somedir/entire-network		nnn.nnn.0.0/255.255.255.0(rw)
   /somedir/wildcards		nnn.nnn.nnn.2*(rw,sync)

4. Are you also using iptables? If so, you'll want to open up a bunch of ports, and also edit some of the nfs settings to restrict the ports NFS is using.
   
   In /etc/sysconfig/nfs you'll want to set:
   STATD_PORT=10002
   STATD_OUTGOING_PORT=10003
   MOUNTD_PORT=10004
   RQUOTAD_PORT=10005
   
   In /etc/sysconfig/iptables you'll want to set something like:
   

   -A RH-Firewall-1-INPUT -p udp -m udp -m multiport --dports 111,1110,2049 -j ACCEPT
   -A RH-Firewall-1-INPUT -p tcp -m tcp -m multiport --dports 111,1110,2049 -j ACCEPT
   -A RH-Firewall-1-INPUT -p udp -m udp --dport 32769 -j ACCEPT
   -A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 32803 -j ACCEPT
   -A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 10002 -j ACCEPT
   -A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 10003 -j ACCEPT
   -A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 10004 -j ACCEPT
   -A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 10005 -j ACCEPT
   -A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 10006 -j ACCEPT
   -A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 10007 -j ACCEPT
   -A RH-Firewall-1-INPUT -m state --state NEW -m udp -p udp --dport 10002 -j ACCEPT
   -A RH-Firewall-1-INPUT -m state --state NEW -m udp -p udp --dport 10003 -j ACCEPT
   -A RH-Firewall-1-INPUT -m state --state NEW -m udp -p udp --dport 10004 -j ACCEPT
   -A RH-Firewall-1-INPUT -m state --state NEW -m udp -p udp --dport 10005 -j ACCEPT
   -A RH-Firewall-1-INPUT -m state --state NEW -m udp -p udp --dport 10006 -j ACCEPT
   -A RH-Firewall-1-INPUT -m state --state NEW -m udp -p udp --dport 10007 -j ACCEPT

   And then of course service iptables restart.
5. Set services to start automatically, because we're lazy: for i in nfs portmap; do chkconfig $i on; done
6. Restart services: service portmap restart, service nfs restart
7. Check status: rpcinfo -p localhost.

On the Client...
On the client, you shouldn't need to open up any ports. You can just add a line like:

remote.server.addr:/remote/share	 /local/mount	nfs	noatime

to /etc/fstab, and then use mount /local/mount to mount it. (I use /etc/fstab for laziness, of course.) SO LAZY.

disabling oradiag_user logging for oracle instant client

After getting the Oracle Instant Client and DBD::Oracle successfully installed, users noticed that they were getting enormous log files in oradiag_[username]/diag/clients/user_[username]/host_nnnnnn_nn , and that logging was apparently set to 16, which is full ridiculous logging. E.g. alert/log.xml had:

<msg time='2011-08-10T13:36:50.258-04:00' org_id='oracle' comp_id='clients'
type='UNKNOWN' level='16' host_id='hostname.domain.etc' host_addr='nnn.nnn.nnn.nnn'>
<txt>Directory does not exist for read/write [/usr/lib/oracle/11.2/client64/log] []
</txt>
</msg>

LAME. So lame. Following various advice, I added the following to $ORACLE_HOME/network/admin/sqlnet.ora :

TRACE_LEVEL_CLIENT = OFF
TRACE_DIRECTORY_CLIENT=/dev/null
LOG_DIRECTORY_CLIENT = /dev/null
LOG_FILE_CLIENT = /dev/null
LOG_LEVEL_CLIENT = OFF

... all apparently for nothing. However, I eventually found Oracle document 454927.1 (note: you need to be logged in to support.oracle.com for that link to work), which indicated I needed to also disable ADR (new as of client 11.2, fancy-pants xml-based logging system) using DIAG_ADR_ENABLED = OFF. THEN you are in the old logging mode, at which point your old settings to completely avoid OCI logging will work. So, sqlnet.ora should look like this:

DIAG_ADR_ENABLED = OFF
TRACE_LEVEL_CLIENT = OFF
TRACE_DIRECTORY_CLIENT=/dev/null
LOG_DIRECTORY_CLIENT = /dev/null
LOG_FILE_CLIENT = /dev/null
LOG_LEVEL_CLIENT = OFF

HOWEVER, you must make sure that users have TNS_ADMIN=/usr/lib/oracle/11.2/client64/network/admin/ (or wherever your sqlnet.ora file is). You can add the line "export TNS_ADMIN=/path/to/your/file/" to /etc/profile to set it for all your users by default. Users can override TNS_ADMIN in their ~/.bashrc file to whatever they want.

Note - why would you want to disable the Oracle Instant Client logging? Isn't logging inherently good and disabling it inherently evil?
- If your application (e.g. perl) is already catching Oracle errors.
- If your application is in production and making many requests, the I/O from the logging might slow it down to a measurable extent.

Other sources that helped me find this solution: stackoverflow.com, CERN Savannah Bugs #58917, Oracle Forums thread #959329 (only found via the CERN page - gah!).
... thanks guys!

install perl DBD::Oracle (Lesson learned: CPAN and yum don't mix)

From the many tales of woe on the web about installing perl DBD::Oracle, from "invalid lvalue in assignment" to mysterious make errors, and the pages of intricate instructions doubtfully translated from the French, I assumed that it was a long and difficult process and it was natural that I was having problems installing on 64-bit CentOS. WRONG! It can actually be easy for lazy and dumb people like me.

First, just use yum and hand-compiling. Don't add CPAN to the mix.

1. Add the rpmforge repo and the EPEL repo (see links for instructions) so that you can install perl-DBD and perl-DBI via yum.
2. Install perl-DBD and perl-DBI via yum.
3. Download and install the OCI client "basic" and "sdk/devel" packages from Oracle. Note that you might need an older version if you're connecting to an older version of Oracle. Note also that Oracle makes you log in to download this. Note also that you need both the SDK and the Basic package. I recommend getting the rpms - install with a simple rpm -Uvh .
4. Oracle puts the libraries in a wacky place, e.g. /usr/lib/oracle/11.2/client64/lib if you're using the 64-bit version of 11.2. Therefore, create a new file, e.g. oci.conf, in /etc/ld.so.conf.d/, with the library location in it, and then run (as root) ldconfig -v to add it.
5. Download the DBD::Oracle source from cpan and extract it someplace. 
6. Set some environment variables:
   

   export ORACLE_HOME=/usr/lib/oracle/11.2/client64
   export PATH=$PATH:$ORACLE_HOME/bin
   export LD_LIBRARY_PATH=$ORACLE_HOME/lib

7. Run "perl Makefile.PL -V 11.2.0" in the directory where the OCI client was extracted. Change the version number to whatever the correct version is. This avoids the "I could not determine Oracle client version so I'll just default to version 8.0.0.0" issue.
8. Run "make install".
9. You should be done!

The latest versions of the Oracle Instant Client default to making incredibly verbose logs in the user's home directories - how to disable default oradiag_user instant client logging.

perl cpan "recursive dependency detected"

Using the CPAN shell for the first time on a new server, after install Bundle::CPAN" and then "reload cpan", on the next install I got:

Recursive dependency detected:
    Bundle::CPAN
 => ExtUtils::MakeMaker
 => M/MS/MSCHWERN/ExtUtils-MakeMaker-6.56.tar.gz
 => File::Spec
 => S/SM/SMUELLER/PathTools-3.33.tar.gz
 => File::Path
 => D/DL/DLAND/File-Path-2.08.tar.gz
 => File::Spec.
Cannot continue.

in the perl cpan installer. Alas! I thought. Will I have to install the old-fashioned way? Indeed no! This may just be a sign that you need to "install Bundle::CPAN" and then "reload cpan".

But I just did that, damn it! Apparently, "reload cpan" isn't always completely effective after "install Bundle::CPAN". If you encounter this error, try quitting cpan and reloading.

Proof yet again that almost all tech problems can be solved by restarting the (whatever).

Btw, if you're not using the CPAN shell, use perl -MCPAN -e 'install( q{Bundle::CPAN} )' to install/update the cpan bundle.

xen guest I/O issue: task blocked for more than 120 seconds

(Update to this post here, with more info for G7 blades.)

A lot of people on CentOS/RHEL who run Xen (and apparently also people on Debian/Ubuntu) who have HP or Fujitsu hardware RAIDs are experiencing this issue. Under sustained, heavy I/O on the guest/domU, the guest load average will rise (beyond 20 even on a single-CPU VM) and messages like the following will appear in syslog:

INFO: task syslogd:1500 blocked for more than 120 seconds. 
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. 
syslogd D 0000000000000110 0 1500 1 1503 1491 (NOTLB) 
 ffff8800b0739d88 0000000000000286 ffff8800b8922970 ffff8800b8922970 
 0000000000000009 ffff8800bb2dd0c0 ffff8800baa55080 0000000000002b40 
 ffff8800bb2dd2a8 0000000000000000 
Call Trace: 
 [] :jbd:log_wait_commit+0xa3/0xf5 
 [] autoremove_wake_function+0x0/0x2e 
 [] :jbd:journal_stop+0x1cf/0x1ff 
 [] __writeback_single_inode+0x1d9/0x318 
 [] do_readv_writev+0x26e/0x291 
 [] sync_inode+0x24/0x33 
 [] :ext3:ext3_sync_file+0xcc/0xf8 
 [] do_fsync+0x52/0xa4 
 [] __do_fsync+0x23/0x36 
 [] tracesys+0xab/0xb6 

Two excellent bugs to start out with are:
http://bugs.centos.org/view.php?id=4515
https://bugzilla.redhat.com/show_bug.cgi?id=605444

This problem has apparently cropped up with all -194 kernels and persists into the -238 kernels. I find it continues even with CentOS 5.6 and 2.6.18-238.9.1.el5xen. I've found that it affects all versions of Xen from 3.0.3 to 3.4.3 to 4.1.

There is a firmware update for HP that you should definitely apply, but it doesn't appear to fix the problem completely. (Supposedly, the Fujitsu firmware update does fix the issue.) Particularly, some users got an aggravated version of the problem when a firmware issue caused the onboard battery not to charge. Other users were able to help the issue by disabling irqbalanced, though of course you may not want to do this if your vm uses multiple processors.

One extra source of ambiguity with these forum posts is that it's not clear what other people are doing to cause this issue. Post firmware upgrade, this issue can only reliably be triggered with sustained, heavy I/O. I used dd if=/dev/zero of=./test1024M bs=1024k count=1024 conv=fsync to ensure problems during testing.
- On a dom0, this test would complete in seconds with an average speed of 72-ish mb/second.
- On an affected domU, this test would cause the above-referenced load and dmesg messages, cause the entire system to be unresponsive, and have an average speed around 300 bytes/second.

The two things that made a material difference in our case, with HP Proliant BL460 G5s running CentOS 5.5 and 5.6 with CentOS guest vms, were, after the firmware update:
 - Switching the I/O scheduler on both the dom0 and the domU to noop
 - Capping the max raid speed on both the dom0 and the domU.

You can temporarily switch the I/O scheduler with something like:
echo noop > /sys/block/[your-block-device-name]/queue/schedule
... to change the default on reboot, add "elevator=noop" to the kernel line in /etc/grub.conf .

To set the max raid speed, you  can use
echo "50000" > /proc/sys/dev/raid/speed_limit_max
The default for raid/speed_limit_max is 200K; this sets it to 50K. I had expected that this would only matter on the dom0, since the domU doesn't technically have a raid, but putting it in both places eliminated some intermittent issues.

After changing the scheduler, I found that the domU 1GB copy from /dev/zero would average over 40MB/sec, but would still sometimes freak out, crawl at 2MB/sec, and fill up dmesg with complaints. The load would sometimes climb near 10. After setting the raid max speed, I found that the domU 1GB copy would always be between 47 and 49 MB/sec, the load would never climb above 1.5, and the VM stayed responsive. So it's a band-aid solution, but very effective, and should tide us over until there is a kernel or firmware solution.

*** *** ***

Update 20 June 2011:

Looking over the latest comments (see 28+) on https://bugzilla.redhat.com/show_bug.cgi?id=615543 , it seems that people running certain HP Smart Arrays may completely solve this issue by upgrading their driver AND firmware, though people on other models are not quite so lucky and still need to use the noop scheduler.

I also want to note that if you're in the middle of a slowdown due to this issue and can get any response at all, you'll probably notice that the amount of "Dirty" memory in /proc/meminfo is very high, and running "sync" can sometimes force your system out of the freeze a little quicker than it would find its way out on its own.
awk '/Dirty/ {printf( "%d\n", $2 )}' /proc/meminfo
will give you just the integer by itself, if you want a script to monitor this value.

executing java from php in rhel/centos - not enough space for code cache

The apache user could run java programs just fine from the command line, but even the simplest java command in php would fail:

<?php
system('java -version');
?>

with the error 

Error occurred during initialization of VM
Could not reserve enough space for code cache. 

We already had the setsebool -P httpd_ssi_exec=1 option on, so that Apache had permission to execute whatever. Strangely, however, this "memory" issue is also due, somehow to selinux. Disabling selinux temporarily (e.g. with echo 0 >/selinux/enforce) or setting setsebool -P httpd_disable_trans=1 fixes the problem. I'd like less of a sledgehammer solution, but messing with other selinux settings does not seem to have any effect.


Edit: As a commenter below just indicated, you can use setsebool -P httpd_execmem=1 to disable this setting. If that doesn't work, well, back to the sledgehammer.