Password and group caching

If you have a NIS client on HPUX and you’ve spent a few hours already trying to understand why on the earth “id” command keeps telling something like this:

bash-3.2# id sergeyt
Can't find user sergeyt

And you’ve already checked all the pieces where NIS could brake, i.e. nsswitch.conf, ypwhich -m, ypmatch user_name passwd, nsquery, then try to ‘/sbin/init.d/pwgr stop’ and see if that would make a difference. Linux has a similar service called nscd (name server cache daemon) that has caused me a lot of trouble similar to the one mentioned above because both nscd and pwgrd cache not only positive but a negative responses as well.
Edit /etc/rc.config.d/pwgr file on HPUX server if you prefer this service to be disabled even if the server is rebooted.

An error during APA creation

If you see an error message like this “unplumb operation for lan was unsuccessful” when creating APA interface double check that all members of APA configuration are down and none of them has an IP address assigned.

Expanding SB40c with hpacucli

One of the tasks that we had to deal with during the last trip was to expand SB40c blade storage by replacing 4x146GB SAS disks with 6x30GB SAS disks. Preferably, this operation should be done online causing our client a zero downtime. Since these 4x146GB had been configured into RAID10 the initial idea was quite simple and straightforward:

  1. Pull out any one of the disks from SB40c.
  2. Replace it with a new 300GB disk and wait till the LogicalDrive is reconstructed.
  3. Remove another disk from the array but this time it should be from the other mirror strip.
  4. Insert a new 300GB disk and wait till the reconstruction is over.
  5. Do exectly the same with the two old 146GB disks.
  6. Expand the array by growing the logical drive (we had only one in the array’s configuration).

Replacing the first disk worked as planned and as soon as a 300GB replacement was swapped in a green lid went on indicating that the reconstruction had begun. So far so good. But when we replaced the second disk our joy had diminished – this new disk was giving no signs of life (all lids were black) , the system stalled to the point when it was impossible to gracefully shut it down. So we pressed a reset and thankfully once the system rebooted the reconstruction process continued so the data were safe. The other two 146GB disks were replaced online without a single hiccup, so the rest of the expansion plan was very easy and all we had to do was to insert another 2x300GB disks into the box to make the total number of disks equal to six. After that we just grew the logical drive as show bellow:

=> ctrl all show config

Smart Array P400 in Slot 3                (sn: PAFGL0N9SWK2OA)

   array A (SAS, Unused Space: 584359 MB)

      logicaldrive 1 (273.4 GB, RAID 1+0, OK)

      physicaldrive 1I:1:5 (port 1I:box 1:bay 5, SAS, 300 GB, OK)
      physicaldrive 2I:1:1 (port 2I:box 1:bay 1, SAS, 300 GB, OK)
      physicaldrive 2I:1:2 (port 2I:box 1:bay 2, SAS, 300 GB, OK)
      physicaldrive 2I:1:3 (port 2I:box 1:bay 3, SAS, 300 GB, OK)

   unassigned

      physicaldrive 1I:1:6 (port 1I:box 1:bay 6, SAS, 300 GB, OK)
      physicaldrive 2I:1:4 (port 2I:box 1:bay 4, SAS, 300 GB, OK)

=> ctrl slot=3 ld 1 add drives=allunassigned 
=> ctrl all show config

Smart Array P400 in Slot 3                (sn: PAFGL0N9SWK2OA)

   array A (SAS, Unused Space: 1156500 MB)

      logicaldrive 1 (273.4 GB, RAID 1+0, Transforming, 0% complete)

      physicaldrive 1I:1:5 (port 1I:box 1:bay 5, SAS, 300 GB, OK)
      physicaldrive 1I:1:6 (port 1I:box 1:bay 6, SAS, 300 GB, OK)
      physicaldrive 2I:1:1 (port 2I:box 1:bay 1, SAS, 300 GB, OK)
      physicaldrive 2I:1:2 (port 2I:box 1:bay 2, SAS, 300 GB, OK)
      physicaldrive 2I:1:3 (port 2I:box 1:bay 3, SAS, 300 GB, OK)
      physicaldrive 2I:1:4 (port 2I:box 1:bay 4, SAS, 300 GB, OK)

The initial explanation of what could’ve been a root cause was that one shouldn’t leave hpacucli tool running whilst replacing a disk. Sounds plausible since it resembles a problem when someone deletes a file whilst the other process is writing into it. Or more correctly, when a process read/write over NFS and the server becomes unavailable.
Since we had another SB40c and the task was identical we had the second chance. This time we had double checked that no one was running hpacucli and began replacing the disks. Two disks were replaced flawlessly but the third one hit us exactly with the same problem and we had to do a hard reset once again.
It’s still misty what was the real culprit in the first place. Who knows, maybe it was a firmware issue but we didn’t have the third SB40c to check that theory. Anyway, I think that such badly behavior is unacceptable even if these arrays had the oldest firmware possible.
So if anyone knows how to avoid that in the future or point to the possible error from our side – shoot out. Your comments are truly welcome.

Mirroring root disk on HP-UX 11iv3

This post have been resting in a dust a way too long but now I’m going to fix that by guiding you through a very important and sometimes a vital process as a root disk mirroring on HP-UX. Truth to be told that there is no single grain of rocket science so treat it as a pure reference. Lets imagine that disk3 is a current boot device and disk5 will be our new mirror.

  1. Run the following commands to determine the size (in megabytes) of EFI, HP-UX and HSPS of the boot device:
  2. # diskinfo -b /dev/rdisk/disk3_p1 | awk '{print $1/1024}'
    # diskinfo -b /dev/rdisk/disk3_p2 | awk '{print $1/1024}' 
    # diskinfo -b /dev/rdisk/disk3_p3 | awk '{print $1/1024}'
    
  3. Next, prepare a file, i.e. /tmp/partitionfile, with the following content:
  4. 3
    EFI 500MB
    HPUX 100%
    HPSP 400MB
    
  5. It’s quite convenient to have a list of disks before making the changes so once they’re applied they will be vividly visible at once:
  6. # ioscan -fnC disk
    # ioscan -m dsf
    
  7. Create the partitions on a new disk and with insf command make them available to OS by creating the device files:
    # idisk -f /tmp/partitionfile -w /dev/rdisk/disk5
    # insf -e 
    
  8. Make it bootable:
  9. # pvcreate -f -B /dev/rdisk/disk5_p2 
    
  10. Extend the root volume group:
  11. # vgextend vg00 /dev/disk/disk5_p2 
    Volume group "vg00" has been successfully extended.
    Volume Group configuration for /dev/vg00 has been saved in /etc/lvmconf/vg00.conf
    
  12. To complete the setup of the disk as a boot disk run mkboot command:
  13. # mkboot -e -l /dev/disk/disk5
    

    The -e option tells mkboot to use EFI layout and the -l option tells mkboot that this volume will be used by a volume manager (even if it is not currently used by one)

  14. Create a temporary AUTO file and use the efi_cp command to copy it to the mirror, using the block device of the first (EFI) partition. The –lq option ensures that the system will boot without quorum. In the event of the primary boot disk failing this will allow the mirror disk to boot.
  15. # print 'boot vmunix -lq' > /tmp/AUTO
    # efi_cp -d /dev/disk/disk5_p1 /tmp/AUTO EFI/HPUX/AUTO
    
  16. Now proceed with mirroring your logical volumes. I used pvdisplay to get an ordered list of the volumes to be able to create the mirrored volumes in the same order.
  17. # pvdisplay -v /dev/disk/disk3_p2 | grep "current.*00000"
    # /usr/sbin/lvextend -m 1 /dev/vg00/lvol1 /dev/disk/disk5_p2
    # /usr/sbin/lvextend -m 1 /dev/vg00/lvol2 /dev/disk/disk5_p2
    # /usr/sbin/lvextend -m 1 /dev/vg00/lvol3 /dev/disk/disk5_p2
    # /usr/sbin/lvextend -m 1 /dev/vg00/lvol4 /dev/disk/disk5_p2
    # /usr/sbin/lvextend -m 1 /dev/vg00/lvol5 /dev/disk/disk5_p2
    ...
    
  18. Prepare LVM logical volume to be root, boot, primary swap, or dump volume:
  19. # /usr/sbin/lvlnboot -b /dev/vg00/lvol1
    # /usr/sbin/lvlnboot -r /dev/vg00/lvol3
    # /usr/sbin/lvlnboot -s /dev/vg00/lvol2
    # /usr/sbin/lvlnboot -d /dev/vg00/lvol2
    # /usr/sbin/lvlnboot –R
    
  20. Finally, add the new disk to boot device configuration table:
  21. # /usr/bin/echo “1 /dev/disk/disk5_p2” >> /stand/bootconf
    
  22. Set the alternate boot path to the mirror_disk:
  23. # /usr/sbin/setboot -a /dev/disk/disk5_p2
    
  24. If the mirror disk includes an HPSP partition, use the efi_fsinit command on the character device file for the HPSP (third) partition to initialize it with an EFI file system:
  25. # /usr/sbin/efi_fsinit -d /dev/rdisk/disk5_p3
    

Dop show

How to allow an ordinary user to executed an application or a script with the extended privileges or even as a root user? Having a strong Linux/BSD background the very first option that would come to my mind is – sudo. Solaris is more flexible in that regard due to the presence of RBAC. But what about TRU64? Since I’m not very fluent with it I turned my eyes to Google and typed something like “tru64 sudo”. Thankfully, I came to a forum where a poster was in a similar situation striving for a sudo binary for tru64 platform. But the final comment in this thread mentioned dop as an officially supported way. Hm, that rang a bell.

dop – Allows a user to execute a privileged program without knowing the root password. The dop command also modifies the action database.

That’s looked promising and it turned out to be very easy to configure. My goal was to give an oes user a right to start Oracle Express server, which, for some strange reason, had been configured in a wacky way that only root user was able to start it up.

Back to dop configuration steps. The easiest way is to do everything from “sysman dopconfig” menu driven interface but since the list of built-in privileges wasn’t sufficient some manual intervention was inevitable.
Actually, there is already a built-in privilege called “SystemManagement” but from it’s description:

SystemManagement privilege allows configuration of kernel facilities such as kernel build, shutdown, cfgmgr, system services start/stop, and software asset management

it obvious that the list of rights is too broad. Not a big deal, because one could modify /etc/doprc file, presumably with vi editor, and add a privilege he/she likes. In my case, I added a new entry to /etc/doprc and named it OracleExpressManagement:

OracleExpressManagement {
        {description    { OracleExpressManagement privilege allows to start/stop Oracle Express Server}}
}

Next, with the help of /usr/sbin/dop utility I added a new action to the dop database:

dop -a OracleExpressManagement oracle_express "/sbin/init.d/express *"

If you look at /etc/doprc you would notice a new record added to the end of _Action section:

oracle_express {
            {privs {OracleExpressManagement}}
            {path {/sbin/init.d/express *}}
        }

The last step, is to add our user to a list of who is allowed to run a dop action (oracle_express). This could be achieved in two ways: through sysman dopconfig and, again, by editing /etc/doprc. Whatever approach you prefer the net outcome would be the same – /etc/doprc will be updated slightly:

OracleExpressManagement {
        {users  { oes }}
        {description    { OracleExpressManagement privilege allows start/stop }}
}

That’s it and from now on oes user could run a simple command to start/stop Oracle Express:

/usr/sbin/dop oracle_express start
/usr/sbin/dop oracle_express stop

P.S. I used HP-UX tag purely because now TRU64 is in HP’s hands.

Runlevels in HP-UX and Solaris

In the following post I’d like to dwell upon differences between the run levels in Solaris and HP-UX.
So first, lets take a quick look at the description (man init(1M)) of the run levels which are supported in these two operating systems:

HP-UX

Run-levelDescription
0Shut down HP-UX.
S|sUse for system administration (also known as "single-user state"). When booting into run level S at powerup, the only access to the system is through a shell spawned at the system console as the root user. The only processes running on the system will be kernel daemons started directly by the HP-UX kernel, daemon processes started from entries of type sysinit in /etc/inittab, the shell on the system console, and any processes started by the system administrator. Administration operations that require the system to be in a quiescent state (such as the fsck(1M) operation to repair a file system) should be run in this state. Transitioning into run level S from a higher run level does not terminate other system activity and does not result in a "single-user state"; this operation should not be done.
1Start a subset of essential system processes. This state can also be used to perform system administration tasks.
2Start most system daemons and login processes. This state is often called the "multi-user state". Login processes either at local terminals or over the network are possible.
3Export filesystems and start other system processes. In this state NFS filesystems are often exported, as may be required for an NFS server.
4Activate graphical presentation managers and start other system processes.
5-6These states are available for user-defined operations.

Solaris

Run-levelDescription
0Go into firmware
1Put the system in system administrator mode. All local file systems are mounted. Only a small set of essential kernel processes are left running. This mode is for administrative tasks such as installing optional utility packages. All files are accessible and no users are logged in on the system.
2Put the system in multi-user mode. All multi-user environment terminal processes and daemons are spawned. This state is commonly referred to as the multi-user state.
3Extend multi-user mode by making local resources available over the network.
4Is available to be defined as an alternative multi-user environment configuration. It is not necessary for system operation and is usually not used.
5Shut the machine down so that it is safe to remove the power. Have the machine remove power, if possible.
6Stop the operating system and reboot to the state defined by the initdefault entry in /etc/inittab.
S, sEnter single-user mode. This is the only run level that doesn't require the existence of a properly formatted /etc/inittab file. If this file does not exist, then by default, the only legal run level that init can enter is the single-user mode. When in single-user mode, the filesystems required for basic system operation will be mounted.
Q, qRe-examine /etc/inittab.

Apart from obvious distinctions between run levels with the same numbers, just take a closer look at run levels 5 and 6, there is a fundamental difference in a way that services are started/stopped when transition from on run-level to another takes place.

In HP-UX there is a single script /sbin/rc which acts as a general purpose sequencer invoked upon entering new run level. But what is more important is the following behavior of rc scipt:

If a transition from a lower to a higher run level (i.e., init state) occurs, the start scripts for the new run level and all intermediate levels between the old and new level are executed. If a transition from a higher to a lower run level occurs, the kill scripts for the new run level and all intermediate levels between the old and new level are executed.

Lets compare it to the way that Solaris does the same job. I’m going to concentrate on Solaris 9 leaving the latest Solaris 10 version for the last bit. Just like in HP-UX, Solaris has it’s own /etc/inittab file which controls process dispatching by init. The inittab file is composed of entries that are position dependent and have the following format:

id:rstate:action:process

ap::sysinit:/sbin/autopush -f /etc/iu.ap
ap::sysinit:/sbin/soconfig -f /etc/sock2path
fs::sysinit:/sbin/rcS sysinit >/dev/msglog 2<>/dev/msglog /dev/msglog 2<>/dev/msglog
sS:s:wait:/sbin/rcS >/dev/msglog 2<>/dev/msglog /dev/msglog 2<>/dev/msglog /dev/msglog 2<>/dev/msglog /dev/msglog 2<>/dev/msglog /dev/msglog 2<>/dev/msglog /dev/msglog 2<>/dev/msglog /dev/msglog 2<>/dev/msglog /dev/msglog 2<>/dev/msglog /dev/msglog 2<>/dev/msglog /dev/msglog 2<>/dev/msglog

But, in contrast to HP-UX, there is a single /etc/rc[S012356] script for every run level. More over, in Solaris only the scripts that pertain to the run-level we’re switching into are executed. These scripts are placed in a corresponding directories, i.e. /etc/rc0.d/ for the run level zero, /etc/rc1.d/ for the first and so on. In other words, if we’re transiting from the level S (single user mode) to the second run level then in that case one start/stop scripts from the zero and the first run levels are run by jumping right into the /etc/rc2.d/ directory where kill or K* scripts are run first following by start or S* scripts.

f [ $_INIT_PREV_LEVEL = S -o $_INIT_PREV_LEVEL = 1 ]; then
        echo 'The system is coming up.  Please wait.'

elif [ $_INIT_RUN_LEVEL = 2 ]; then
        echo 'Changing to state 2.'

        if [ -d /etc/rc2.d ]; then
                for f in /etc/rc2.d/K*; do
                        if [ -s $f ]; then
                                case $f in
                                        *.sh)   .        $f ;;
                                        *)      /sbin/sh $f stop ;;
                                esac
                        fi
                done
        fi
fi

if [ $_INIT_PREV_LEVEL != 2 -a $_INIT_PREV_LEVEL != 3 \
    -a $_INIT_PREV_LEVEL != 4 -a -d /etc/rc2.d ]; then

        for f in /etc/rc2.d/S*; do
                if [ -s $f ]; then
                        case $f in
                                *.sh)   .        $f ;;
                                *)      /sbin/sh $f start ;;
                        esac
                fi
        done
fi

The third run level is an exception though. If you take a closer look at the /etc/inittab file from Solaris you’d noticed that /sbin/rc2 is also executed when switching into the third run level.

s2:23:wait:/sbin/rc2 >/dev/msglog 2<>/dev/msglog /dev/msglog 2<>/dev/msglog

By the way, it’s worth mentioning that most Kill scripts could be found in /etc/rc0.d and /etc/rc1.d/, so please pay attention to this.

In Solaris 10 with introduction of SMF (System Management Facility) things have changed noticeably and brought substantial enhancements to the traditional UNIX start-up scripts. I’m not going to linger on that subject in great details but just want to mention that run run-levels have been replaced with the milestones and for legacy purposes startup programs in the /etc/rc?.d directories are executed as part of the corresponding run-level milestone:

/etc/rcS.d milestone/single-user:default
/etc/rc2.d milestone/multi-user:default
/etc/rc3.d milestone/multi-user-server:default

In addition, a new restarter daemon for SMF and for all services has been added to the system – svc.startd.
Quick excerpt from its man page:

svc.startd maintains service state, as well as being responsible for managing faults in accordance with the dependencies of each service. svc.startd is invoked automatically during system startup. It is restarted if any failures occur. svc.startd should never be invoked directly.

So now a good question or even a series of them could start vibrating in your mind:

  • How many milestones do I actually have?
  • How do I check the default run-level or a milestone?
  • How could I list what services are fired up in each milestone?
  • And finally, how to switch from one milestone to another?

To tell the truth, it’s all dead easy.

  1. Simply running the following command you could get an answer to the first question:
    # svcs -a | grep milestone
    online         Dec_11   svc:/milestone/name-services:default
    online         Dec_11   svc:/milestone/network:default
    online         Dec_11   svc:/milestone/devices:default
    online         Dec_11   svc:/milestone/single-user:default
    online         Dec_11   svc:/milestone/sysconfig:default
    online         Dec_11   svc:/milestone/multi-user:default
    online         Dec_11   svc:/milestone/multi-user-server:default

    With “svcs -l” you could get a longer description of each service. More over, this command will also reveal its dependencies and whether they are optional or required:

    # svcs -l  svc:/milestone/multi-user-server:default
    fmri         svc:/milestone/multi-user-server:default
    name         multi-user plus exports milestone
    enabled      true
    state        online
    next_state   none
    state_time   December 11, 2009 12:20:44 PM MSK
    logfile      /var/svc/log/milestone-multi-user-server:default.log
    restarter    svc:/system/svc/restarter:default
    dependency   optional_all/none svc:/system/cluster/cl-svc-enable:default (online)
    dependency   require_all/none svc:/milestone/multi-user (online)
    dependency   optional_all/none svc:/application/management/dmi (online)
    dependency   optional_all/none svc:/application/management/snmpdx (online)
    dependency   optional_all/none svc:/network/rpc/bootparams (disabled)
    dependency   optional_all/none svc:/network/samba (disabled)
    dependency   optional_all/none svc:/network/winbind (disabled)
    dependency   optional_all/none svc:/network/wins (disabled)
    dependency   optional_all/none svc:/network/nfs/server (disabled)
    dependency   optional_all/none svc:/network/rarp (disabled)
    dependency   optional_all/none svc:/network/dhcp-server (disabled)
    dependency   optional_all/none svc:/network/ssh (online)
  2. Using svcprop command one could list all properties which belongs to a given service.
    # svcprop svc:/system/svc/restarter:default
    general/enabled boolean true
    general/entity_stability astring Unstable
    general/single_instance boolean true
    restarter/auxiliary_state astring none
    restarter/next_state astring none
    restarter/state astring online
    restarter/state_timestamp time 1260522889.099316000
    restarter/start_pid count 8
    restarter/contract count 4
    restarter/alt_logfile astring /etc/svc/volatile/svc.startd.log
    restarter/logfile astring /var/svc/log/svc.startd.log
    system/reconfigure boolean false
    options/milestone astring all
    tm_common_name/C ustring master\ restarter
    tm_man_svc_startd/manpath astring /usr/share/man
    tm_man_svc_startd/section astring 1M
    tm_man_svc_startd/title astring svc.startd
    

    As you could see, there is a special property options/milestone that defines the milestone used as the default boot level. Allow me to quote man page once again:

    Acceptable options include only the major milestones:

    svc:/milestone/single-user:default
    svc:/milestone/multi-user:default
    svc:/milestone/multi-user-server:default

    or the special values all or none. all represents an idealized milestone that depends on every service. none is a special milestone where no services are running apart from the master svc:/system/svc/restarter:default.
    By default, svc.startd uses all, a synthetic milestone that depends on every service. If this property is specified, it overrides any initdefault setting in inittab(4).

    To change the default milestone just run the following command:

    # svccfg -s svc:/system/svc/restarter:default setprop options/milestone=svc:/milestone/multi-user-server:default
    # svcprop -p options/milestone svc:/system/svc/restarter:default
    svc:/milestone/multi-user-server:default
    
  3. Use “svcs -D” against the milestone your’re interested in to get the list of all dependents.
  4. # svcs -D  svc:/milestone/multi-user:default
    STATE          STIME    FMRI
    disabled       Dec_11   svc:/network/dhcp-server:default
    disabled       Dec_11   svc:/system/iscsitgt:default
    disabled       10:17:00 svc:/application/cde-printinfo:default
    disabled       10:17:00 svc:/application/graphical-login/cde-login:default
    disabled       10:17:01 svc:/system/vxvm/vxvm-recover:default
    disabled       10:17:32 svc:/application/management/common-agent-container-1:default
    online         Dec_11   svc:/system/cluster/cl-svc-enable:default
    online         Dec_11   svc:/milestone/multi-user-server:default
  5. And finally, to switch from one milestone to another use “svcadm milestone” command:
    # svcs -a | grep milestone
    online         Dec_11   svc:/milestone/name-services:default
    online         Dec_11   svc:/milestone/network:default
    online         Dec_11   svc:/milestone/devices:default
    online         Dec_11   svc:/milestone/single-user:default
    online         Dec_11   svc:/milestone/sysconfig:default
    online         Dec_11   svc:/milestone/multi-user:default
    online         10:55:39 svc:/milestone/multi-user-server:default
    offline        10:55:36 svc:/system/cluster/cl-svc-cluster-milestone:default
    
    # svcadm milestone svc:/milestone/single-user:default
    
    # svcs -a | grep milestone
    disabled       10:57:28 svc:/milestone/multi-user-server:default
    disabled       10:57:28 svc:/system/cluster/cl-svc-cluster-milestone:default
    disabled       10:57:38 svc:/milestone/multi-user:default
    disabled       10:58:06 svc:/milestone/sysconfig:default
    disabled       10:58:06 svc:/milestone/name-services:default
    online         Dec_11   svc:/milestone/network:default
    online         Dec_11   svc:/milestone/devices:default
    online         Dec_11   svc:/milestone/single-user:default

    If you add “-d” option to “svcadm milestone” command then this milestone becomes the default one as well.

That’s it. Feel free to comment if I’ve missed anything or made a mistake.
Cheers.

Plugging HP-UX into SAN

Our task for today is to connect HP-UX (11.31 release) to MSA2312fc through SAN with two Brocade switches in between. First, we need to find out all FC cards installed in our server and once we have that piece of information we could dig for WWN numbers to map them latter to the disk array.

Lets start from the beginning, ioscan is our best friend in revealing inner life of our server:

bash-4.0# ioscan -funC fc
Class     I  H/W Path     Driver S/W State   H/W Type     Description
===================================================================
fc        0  0/3/0/0/0/0  fcd   CLAIMED     INTERFACE    HP 4Gb Dual Port PCIe Fibre Channel Mezzanine (FC Port 1)
                         /dev/fcd0
fc        1  0/3/0/0/0/1  fcd   CLAIMED     INTERFACE    HP 4Gb Dual Port PCIe Fibre Channel Mezzanine (FC Port 2)
                         /dev/fcd1

The output above says everything we need to know. So now, I’m going to use fcmsutil to display WWN information from our cards:

bash-4.0# fcmsutil /dev/fcd0

                           Vendor ID is = 0x1077
                           Device ID is = 0x2432
            PCI Sub-system Vendor ID is = 0x103C
                   PCI Sub-system ID is = 0x1705
                               PCI Mode = PCI Express x4
                       ISP Code version = 4.2.2
                       ISP Chip version = 3
                               Topology = PTTOPT_FABRIC
                             Link Speed = 4Gb
                     Local N_Port_id is = 0x020b01
                  Previous N_Port_id is = None
            N_Port Node World Wide Name = 0x5001438004c2e159
            N_Port Port World Wide Name = 0x5001438004c2e158
            Switch Port World Wide Name = 0x200b00051e868762
            Switch Node World Wide Name = 0x100000051e868762
            N_Port Symbolic Port Name = oamdwh1_fcd0
            N_Port Symbolic Node Name = oamdwh1_HP-UX_B.11.31
                           Driver state = ONLINE
                       Hardware Path is = 0/3/0/0/0/0
                     Maximum Frame Size = 2048
         Driver-Firmware Dump Available = NO
         Driver-Firmware Dump Timestamp = N/A
                         Driver Version = @(#) fcd B.11.31.0803 Jan 20 2008

bash-4.0#
bash-4.0# fcmsutil /dev/fcd1

                           Vendor ID is = 0x1077
                           Device ID is = 0x2432
            PCI Sub-system Vendor ID is = 0x103C
                   PCI Sub-system ID is = 0x1705
                               PCI Mode = PCI Express x4
                       ISP Code version = 4.2.2
                       ISP Chip version = 3
                               Topology = PTTOPT_FABRIC
                             Link Speed = 4Gb
                     Local N_Port_id is = 0x010b01
                  Previous N_Port_id is = None
            N_Port Node World Wide Name = 0x5001438004c2e15b
          N_Port Port World Wide Name = 0x5001438004c2e15a
            Switch Port World Wide Name = 0x200b00051e855e8b
            Switch Node World Wide Name = 0x100000051e855e8b
            N_Port Symbolic Port Name = oamdwh1_fcd1
            N_Port Symbolic Node Name = oamdwh1_HP-UX_B.11.31
                           Driver state = ONLINE
                       Hardware Path is = 0/3/0/0/0/1
                     Maximum Frame Size = 2048
         Driver-Firmware Dump Available = NO
         Driver-Firmware Dump Timestamp = N/A
                         Driver Version = @(#) fcd B.11.31.0803 Jan 20 2008

Using this information we could proceed with zone configuration on FC switches.

alicreate "hpux_fcd1", "50:01:43:80:04:c2:e1:5a"
alicreate "MSA2312_A1", "20:70:00:c0:ff:d8:bb:a4"
zonecreate "hpux_msa2312", "hpux_fcd1;MSA2312_A1"
cfgadd "HPUX_cfg" "hpux_msa2312"

For brevity I omitted the steps required to configure the second switch since they are almost the same. The only difference is the aliases’, zones’ names and WWN numbers.

Don’t forget to save and enable newly created configuration on the switch:

cfgsave HPUX_cfg
cfgenable HPUX_cfg

All that we have to do next, apart from creating and mapping LUNs on our storage, is to tell the system to scan for new disks and to create special files for them:

# insf -C disk

To double check that new disks have been successfully added do the following:

# ioscan -m dsf
Persistent DSF           Legacy DSF(s)
========================================
/dev/rdisk/disk1         /dev/rdsk/c0t0d0
/dev/rdisk/disk1_p1      /dev/rdsk/c0t0d0s1
/dev/rdisk/disk1_p2      /dev/rdsk/c0t0d0s2
/dev/rdisk/disk1_p3      /dev/rdsk/c0t0d0s3
/dev/pt/pt4              /dev/rscsi/c3t0d0
/dev/pt/pt5              /dev/rscsi/c2t0d0
/dev/rdisk/disk6         /dev/rdsk/c4t0d1
                         /dev/rdsk/c5t0d1

/dev/rdisk/disk6 is our new lovely friend. To confirm that, use scsimgr command:

# scsimgr inquiry -D /dev/rdisk/disk6

           INQUIRY INFORMATION FOR LUN: /dev/rdisk/disk6

                Peripheral Device Type: 0 (Direct Access)
                  Peripheral Qualifier: 0 (Peripheral Device Connected)
                       Removable Media: No
                          ANSI Version: 5 (Complies to SPC-3)
                    Normal ACA Support: No
                  Hierarchical Support: 0x1 (Hierarchical addressing model used)
                  Response Data Format: 2 (SPC-3)
                     Additional Length: 155
                           SCC Support: 0 (No Embedded Storage Array Controller)
           Access Controls Coordinator: No
             Target Port Group Support: 0x1 (Implicit Asymmetric Access Support)
                      Third-Party Copy: No
                               Protect: 0 (Protection Information NOT Supported)
                         Basic Queuing: 0
                       Command Queuing: 0x1 (Full Task Management Model (SAM-3))
                    Enclosure Services: No
             Multi-port Device Support: Yes
                Medium Changer Support: No
   Supports 16-bit wide SCSI addresses: No
          Support for 16 Bit Transfers: No
            Synchronous Data Transfers: No
                Linked Command Support: No
                 Vendor Identification: "HP      "
                Product Identification: "MSA2312fc       "
                Product Revision Level: "M110"
                  Vendor Specific Data: 20 20 20 20 20 20 20 20 "        "
                                        43 41 50 49 20 20 41 41 "CAPI  AA"
                                        66 20 20 20             "f   "
                              Clocking: 0 (Supports only ST)
 Quick Arbitration & Selection support: No
    Information Unit transfers Support: No

Finally, when you decide to create LVM configuration on top of newly added disk don’t use legacy DSF, all those c#t#d#, but use persistent DSF instead, i.e. /dev/rdisk/disk6 and HP-UX will deal with multipathing on its own:

# scsimgr lun_map -D /dev/rdisk/disk6

        LUN PATH INFORMATION FOR LUN : /dev/rdisk/disk6

Total number of LUN paths     = 2
World Wide Identifier(WWID)    = 0x600c0ff000d8d17c1b1bf84a01000000

LUN path : lunpath4
Class                         = lunpath
Instance                      = 4
Hardware path                 = 0/3/0/0/0/0.0x247000c0ffd8bba4.0x4001000000000000
SCSI transport protocol       = fibre_channel
State                         = STANDBY
Last Open or Close state      = STANDBY

LUN path : lunpath5
Class                         = lunpath
Instance                      = 5
Hardware path                 = 0/3/0/0/0/1.0x207000c0ffd8bba4.0x4001000000000000
SCSI transport protocol       = fibre_channel
State                         = ACTIVE
Last Open or Close state      = ACTIVE

Enjoy.

Stranger in HP-UX

For the last couple of days I’ve been heavily playing with HP-UX, one of the unknown and never seen operating system. That’s true – I’ve never touch it before. But these days are over and now I’m overfilled with new experience and… mixed fillings. You see, on the one hand it’s UNIX and most of the tools and the concepts have many parts in common with other UNIX based OS. But… On the flip side, there are a lot of stuff that work differently. I’m not talking about things that are HP-UX specific i.e. pseudo-swap, LVM or persistent DSF, etc. but widely used and adopted by the others. For example, ifconfig. You can’t use it to view your networking devices. Or another one – -h option for ls or df commands. No luck. Not mentioning the fact that features that come for free in, lets say Solaris, Linux or *BSD, would require a special license, e.g. mirror/UX if you’re looking for mirroring the disks. Even kmeminfo, ::memstat analog from Solaris mdb, is not publicly available. On no account I’m trying to pin a label but to show that HP-UX is a bit different from other UNIXs, at least myself, have gotten used to.

Anyway, with this post I’m going to create a new category devoted to HP-UX cherishing hopes that one day someone could learn through my pain. I must admit that at times its quite amusing to plunge into something new, because usually it also comes together with mix of pure anger and course wording, especially when something well-known, as I mentioned before, doesn’t work as expected and a childish joy when you’ve finally understood the logic and the idea behind it. And that brings enormous fun into a life. So lets it keep prevailing!