Introduction
This guide is intended for those who are new to installing the VIO server using NIM.
For a more advanced, “official,” or higher level NIM specific guide you can check out the recently released redbooks :
NIM From A-Z in AIX 5L
Just go to http://www.redbooks.ibm.com/ and type in SG24-7296-00 in the search field on the upper right hand side of the page.
You can type SG24-7940-03 in the search field for the VIOS redbook
I will try to keep this as reader friendly as possible. Feel free to make suggestions on how to improve this document, and if there are any procedures you would like to see added in the future, let me know and I’ll do my best to add these procedures.
Finally, there always will be someone to tell you how you should appropriately type out “commands to be executed.” Some people prefer that commands are put in single quotes, others prefer that they’re in italics.....whenever there is an actual command to execute, the example of that command will look just like your command line interface on your AIX system.
For example if I want you to do a listing of files in a directory I would have the example of :
# ls -al
I prefer to do it this way because this is exactly what you’ll see when you type it on your system....(and ok, it’s also easier to cut and paste the commands from my test system into this document once I’ve verified they work correctly).
Does :
--SMIT and Command line processes for most of the operations I cover
Does not:
- Any WebSM interface procedures on how to use NIM. WebSM is not used enough to spend any time going into separate sections on how to use it.
- NIM communication information (including nimsh and firewall issues)
If you are unfamiliar with NIM I highly recommend reading through this section and also use it as a reference while reading through this guide. If you’re figuring, “eh he’ll explain all this later so I’ll just skip this part,” you will be flipping back to this.
So this is sort of like a “reverse-glossary.” I’ll give the best and easiest to understand description of these terms so that you’ll hopefully have a much easier time understanding any new concepts you’re unfamiliar with. In all cases - actually using the files/keywords are handled in greater detail in their corresponding “How To...” sections.
Important Files and Directories:
- /etc/bootptab :
This file will exist on the NIM master. In a quiet NIM environment with no operations that require a client to boot, this file will be empty (except for the pre-existing commented section). This file gets updated automatically by the NIM master when a NIM operation is executed that requires the client machine (from here on out referred to as the "VIO server") to boot from a NIM SPOT. If this file contains incorrect information about either the master or the VIO server, the boot operation will fail. While this file “can” be edited manually to fix a bootp issue - it should not be, as you are only applying a “band-aid” fix to an existing issue in your NIM environment....but, sometimes it’s 5pm on a Friday and you’re ready to go home, right?
(Also note related entry ‘bootp’)
- /etc/exports :
This is not a “NIM specific” file, it is a NIM critical file. Any sort of installation, boot, mksysb, savevg....etc operation requires the use of NFS. This file will be updated with which locations are NFS exported from the master to the VIO server and the permissions associated with those exports. If these entries are incorrect or incomplete you will run into boot failures, permission problems, and other errors commonly associated with NFS. This is a text file and also “can” be edited manually to sometimes “band-aid” a problem, but should only be done so with care in knowing exactly what you’re doing. The good thing is, if we mess up this file we can remove it and recycle NFS. The file can be recreated.
- /etc/hosts :
While not a “NIM specific” file, it is also a NIM critical file. This file is sort of like a phone book. It gives a relationship between a system’s hostname and an IP address. Much like a telephone, if you dial the wrong number you get the wrong person. In NIM, if your IP address does not match up to the correct hostname, your install fails. This is a text file and can be edited manually. There should also only be one entry per IP/hostname. I personally prefer to make sure my NIM master has all entries in the /etc/hosts file and are of the following format :
<ipadress> <shortname> <longname>
- /etc/niminfo:
This file should always exist on the NIM master. This file is built when you first initialize the NIM environment. This is simply a text file so feel free to ‘cat’ or ‘more’ the file and look at the entries included in there. You do not want to manually edit this file if there is a mistake in the definition of the master. In this case you will want to redefine the master, or use the feature in NIM to change the master’s attributes (hostname, gateway....etc).
- /tftpboot :
This directory holds the boot images, along with informational files about the VIO server that is being installed by the NIM master. The file names that are generated generally look like this:
VIO_SPOT.chrp.mp.ent à This is the boot image that will be used to boot the VIO server.
The format for the file is:
<spotname>.<system_architecture>.<processor_type>.<adapter_type>
A <VIOserver_hostname> file is created and linked to the boot image on the NIM master.
Example:
In this example, the VIOS_SPOT.chrp.mp.ent file is the network boot image, which has been created from the SPOT. The fork.austin.ibm.com link points to the boot image and is referenced during the boot operation. The austin.ibm.com.info file contains information that is needed by the client during installation, like which resources should be NFS mounted at boot time.
Important Keywords :
Allocate/Allocation :
This process is what allows your VIO server to access resources in NIM. The master uses NFS to perform the allocation process. You can see which resource types are allocated to your VIO server by using the following command:
# lsnim -a spot
VIOS-A:
spot = VIOS-A_spot
VIOS-B:
spot = VIOS-B_spot
VIOS-C:
spot = VIOS-C_spot
In this example environment we have 3 VIO servers defined as NIM clients. Since the SPOT resources are created from the mksysb image of a VIO server, giving the SPOT resource a descriptive name that relates back to the hostname of your VIO server is helpful.
So we can see that ‘VIOS-A’ has a SPOT called “VIOS-A_spot” allocated, ‘VIOS-B’ has a SPOT called “VIOS-B_spot" allocated, and ‘VIOS-C’ has a SPOT called “VIOS-C_spot” allocated.
Base Operating System Installation :
Also commonly called (and referred to from here on out) as a bos_inst operation. This simply refers to the fact that you are initiating a boot and installation to a VIO server.
Bootp :
This is the initial communication made between the NIM master and the VIO server during a boot or bos_inst operation. In order for this to be successful several factors must be met:
1. - bootpd must be running on the NIM master
2. - You must have the correct IP information for the NIM master and target VIO server
3. - The /etc/bootptab must be populated correctly
4. - If the master and VIO server are on separate networks, the router must be set to forward bootp packets.
There are other causes of failure, but checking/verifying those four will solve most bootp issues.
Tftp (Trivial File Transfer Protocol):
When the VIO server has been rebooted for a bos_inst operation you don’t have access to normal TCP communication. Once bootp connection has successfully been achieved, the NIM master uses tftp to transfer over the <clientname>.info file and the boot image to the VIO server.
if1= : Or interface 1:
This is known as the ‘nim network’. Every machine, even the master, is placed on a defined NIM network. A machine who has multiple adapters defined to NIM will have “if2=” and “if3=”....etc attributes. When your NIM master is initialized, you will create a network name for the master and VIO server on the same subnet as the master. If you name this network “master_net” for example, then all VIO server NIM clients on the same subnet as the master will have their “if1=” line set to “master_net”. If you add additional VIO server NIM clients that are on separate subnets, then you will need to create new network names. You can see the “if1=” information from an :
# lsnim -l master |more
-or-
# lsnim -l <client>
You can get further information about the network name by running an ‘lsnim -l <network_name>’.
Having incorrect networking information is probably the leading cause of NIM installation failures. This attribute and the information used when creating networks is extremely important to make sure you have correct.
Client (NIM client) :
A standalone VIO server, or lpar in a NIM environment, other than the NIM master. Clients (VIO servers) use the resources that reside on the NIM master to be installed.
Master (NIM master) :
The only system in a NIM environment that has permission to run NIM commands remotely on other NIM clients (VIO servers). A NIM client (VIO server) can only have one NIM master. The NIM master must be at a level equal to or higher than the AIX level that is being installed (and accompanying VIOS software) to the VIO server.
The NIM master will also hold all of the NIM resources. Due to this fact, you will want to make sure that the NIM master has plenty of space available for storing these resources. It is ideal to have a separate volume group (example: nimvg) so that the rootvg does not get out of control in size.
Resource (NIM resources) SPOT:
The SPOT resource (stands for Shared Product Object Tree in case you were wondering) is responsible for the following:
- Creating a boot image to send to the VIO server over the network.
- Supplying the commands needed to install the VIO server.
You will need to create a SPOT from your NIM VIOS mksysb resource. The VIOS mksysb contains all of the filesets and commands needed to create the SPOT. A SPOT created from an AIX lpp_source is not a valid SPOT to use when installing a VIO server. Essentially the SPOT is a /usr filesystem just like the one on your NIM master. You can think of it as having multiple “mini-systems” on your NIM master, because each SPOT is its own /usr filesystem. Just like your NIM master’s /usr filesystem, going in there manually and messing around with files can easily corrupt it. The good thing about a SPOT however, is that it is easily rebuilt.
Resource (nim resources) mksysb :
This is simply a mksysb image of the VIO server. The mksysb image resides on the NIM master, and the resource can be defined in one of two ways:
-From an existing mksysb image residing on the NIM master already (extracted from a nim_resources.tar package).
-From the mksysb image copied from the VIOS installation media on to hard disk (this is described in more detail later in this document).
You can also back up an already existing VIO Server and a nim_resources.tar package is created. The contents of this nim_resources.tar package can be stored on the NIM master for installations to new VIO servers, and also for the recovery of VIO servers using NIM. The nim_resources.tar file contains all the necessary resources to restore the Virtual I/O server. This includes the mksysb image, the bosinst.data file, the network boot image, and SPOT resource. When the VIO server is backed up, the nim_resources.tar package will be stored on an NFS mounted file system. More detailed steps for creating the nim_resources.tar file with the ‘backupios’ command will be given later in this document.
Resource (nim resources) bosinst_data:
When booting from an installation image to install a system, the system boots to what are known as the “BOS Menus” or Base Operating System Installation Menus. Here is where you select your console, the language to use during the installation, and what disks to install to amongst other things. When using NIM, a “bosinst_data” resource can be created, and it will answer all of the questions that are asked by the “BOS Menus” automatically. This resource is used when installing to a NIM client that may be in another building, down the road or half way across the country. This way, once you kick off the installation from the NIM master, there is no further interaction that is required. The system should (ideally) install and reboot itself afterwards.
Backupios (VIO server backup):
*Note: Any VIOS 1.X client should be created with 'netboot_kernel=mp', and any VIOS 2.X client should be created with 'netboot_kernel=64'.
From command line:
# nim -o define -t standalone -a platform=chrp -a if1="find_net lucidbso 0" -a netboot_kernel=mp -a connect=shell <clientname>
Breaking down the command
# nim -o define -t standalone : This is the NIM operation of defining an object of type=standalone.
... -a platform=chrp : The platform type of the target system.
...-a if1=”find_net lucidbso 0’ : Used when a client is part of an existing NIM network. In this case my NIM client belongs to my master’s NIM network (i.e. They’re on the same subnet) so I don’t need to give it further information. If this client needed to be on a new network then we would not use the “find_net” but rather the “net_definition=<value>.......” options where you’d need to specify “networktype” “subnetmask” “clientgateway” “networkname” attributes.
... -a netboot_kernel=mp : As noted previously, you should set this attribute to "mp" when installing VIOS 1.X and "64" when installing VIOS 2.1 on the client.
... -a connect=shell : This will be the connection type used. You can either choose shell (rsh) or nimsh.
... <clientname> : The client's resolvable hostname.
From SMIT :
# smitty nim_mkmac
-or-
# smitty nim
==> Administration Tasks ==> Manage Machines ==> Define a Machine
From here you are asked what the hostname of the NIM client (VIO server) is. If the master can resolve the client’s hostname and he already has a NIM network associated with the client’s network then it will “pre-generate” a final acceptance screen for you (see example 1). If the master does not already have a NIM network defined for this client, then you will be prompted for more information and be asked to create a new network for this new client (see example 2).
Example 1:
* NIM Machine Name [clientname]
* Machine Type [standalone]
* Hardware Platform Type [chrp] Make sure this is set to ‘chrp’
Kernel to use for Network Boot [mp]
Communication Protocol used by client []
Primary Network Install Interface
* Cable Type bnc
Network Speed Setting []
Network Duplex Setting []
* NIM Network master_net
* Host Name client_hostname
Network Adapter Hardware Address [0]
Network Adapter Logical Device Name []
IPL ROM Emulation Device []
CPU Id []
Machine Group []
Comments []
Notice that you actually don’t have to fill in anything else as a required field. The master can resolve the VIO server's hostname, and he has put it on the same network (master_net) because in this case, the client and master are on the same network. I recommend leaving the optional fields blank. Pressing <enter> from here will define the client.
-or-
Example 2:
* NIM Machine Name [clientname]
* Machine Type [standalone]
* Hardware Platform Type [chrp]
Kernel to use for Network Boot [mp]
Communication Protocol used by client []
Primary Network Install Interface
* Cable Type bnc
Network Speed Setting []
Network Duplex Setting []
* NIM Network [10_14_20_Net]
* Network Type ent
* Ethernet Type Standard
* Subnetmask [255.255.254.0]
* Default Gateway Used by Machine [10.14.20.1]
* Default Gateway Used by Master [9.3.58.1]
* Host Name clientname
Network Adapter Hardware Address [0]
Network Adapter Logical Device Name []
IPL ROM Emulation Device []
CPU Id []
Machine Group []
Comments []
Notice that there is much more to fill out here, because we’re defining a new network within NIM. We’ll say my VIO server's ip = 10.14.20.34. I’m naming my NIM network 10_14_20_Net so that it is descriptive just by looking at the name. Also, any other VIO server on this same network will now be auto-defined on this new network and the SMIT screens will look like example 1 above. We’ll also need to enter my subnetmask and the gateway used by this client. The master’s gateway is filled in for you. Once this successfully completes I‘ll now have 2 NIM networks.......master_net and 10_14_20_Net.
Now that the client machine has been defined, you are ready to set up the resources needed for the VIOS install using NIM.
Mount the VIOS base DVD and copy the VIOS mksysb image from the DVD (in /usr/sys/inst.images) to your NIM master:
# mount -o ro -v cdrfs /dev/cd0 /mnt
# cd /mnt/usr/sys/inst.images
# cp mksysb_image /export/mksysb/mksysb_image
If using VIOS 1.5 or higher media, the mksysb file may be split into two parts. To combine these two parts and copy them to hdisk, run the following:
# cat /mnt/usr/sys/inst.images/mksysb_image /mnt/usr/sys/inst.images/mksysb_image2 > /export/mksysb/mksysb_image
***You can substitute any path you would like to save the combined mksysb image, for ‘/export//mksysb/mksysb_image'.
# nim -o define -t mksysb -a server=master -a location=/export/mksysb/mksysb_image
VIOS_2_1_mksysb
-or-
# smitty nim_mkres => mksysb =>
Define a Resource
* Resource Name [VIOS_2_1_mksysb] Give the resource a name
* Resource Type mksysb
* Server of Resource [master] + Chose your NIM master
* Location of Resource [/export/mksysb/mksysb_image] / The full path of the mksysb file copied to hdisk
àLeave all other entries set to their default values.
# nim -o define -t spot -a server=master -a location=/export/spot –a source=VIOS_2_1_mksysb VIOS_2_1_SPOT
-or-
# smitty nim_mkres => spot =>
Define a Resource
* Resource Name [VIOS_2_1_spot] give the spot a name
* Resource Type spot
* Server of Resource [master] + chose your NIM master
* Source of Install Images [VIOS_2_1_mksysb] + select the mksysb resource
* Location of Resource [/export/spot] / location to store the SPOT
àLeave all other entries set to their default values.
You'll also want to copy the bosinst.data file from the CD media to the NIM master and define that as a resource:
(If the CD has been mounted as described earlier, run the following command. If the CD was mounted in a different way, then simply change directories to the mount point and copy the file)
# cd /home
# mkdir VIOS_bosinst
# cp /mnt/bosinst.data /export/bosinst_data/VIOS_bosinst
# nim -o define -t bosinst_data -a server=master –a location=/export/bosinst_data/VIOS_bosinst VIOS_2_1_bosinst_data
-or-
# smitty nim_mkres => bosinst_data =>
Define a Resource
* Resource Name [VIOS_2_1_bosinst] Give the resource a name
* Resource Type bosinst_data
* Server of Resource [master] + chose the NIM master
* Location of Resource [/export/bosinst_data/VIOS_bosinst] / location of the file copied from CD
àLeave all other entries set to their default values.
To set the NIM master up to perform the installation, we issue a bos_inst operation:
# nim –o bos_inst –a source=mksysb –a spot=VIOS_2_1_spot -a bosinst_data=VIOS_2_1_bosinst -a accept_licenses=yes –a mksysb=VIOS_2_1_mksysb VIOS-A
# smitty nim_bosinst => select the nim client (VIOS lpar) from the list => select ‘mksysb’ as the installation type => select the VIOS mksysb from the list of mksysbs => select the spot created from the VIOS mksysb =>
Install the Base Operating System on Standalone Clients
* Installation Target VIOS-A
* Installation TYPE mksysb
* SPOT VIOS_2_1_spot
LPP_SOURCE [] +
MKSYSB VIOS_2_1_mksysb
BOSINST_DATA to use during installation [VIOS_2_1_bosinst] + Select the bosinst_data resource created from the VIOS mksysb
ACCEPT new license agreements? [yes] + Set to ‘yes’
Initiate reboot and installation now? [no] + Set to ‘no’ for the pull installation
àLeave all other entries set to their default values.
7. The master is set up....now what ?
The NIM master is now ready and waiting for a bootp request from the VIO server. In order to do this we need to boot the VIO server into SMS (System Management Services). You can simply boot your VIOS server using the SMS profile on the HMC.
The options you see in SMS can depend on the firmware level installed on the target system. I don’t have access to all variations of the menu options, so I’ll just use the one my NIM master has. You should be able to navigate closely enough, as the wording should be similar.
SMS - SYSTEM MANAGEMENT SERVICES -
1. Select Language
2. Change Password Options
3. View Error Log
4. Setup Remote IPL (RIPL (Remote Initial Program Load))
5. Change SCSI Settings
6. Select Console
7. Select Boot Options
First of all we need to provide the IP addresses necessary to tell the VIO server who he is, and where to boot from.
4. Setup Remote IPL (RIPL (Remote Initial Program Load))
Next you’ll have a selection of adapters to use. Select your adapter that corresponds to the adapter/host you defined in NIM. You will not see “ent0/ent1...etc” options. You will however see the hardware addresses and slots.
We then are brought to 3 options :
1. IP Parameters
2. Adapter Configuration
3. Ping Test
Chose the option to set the IP Parameters
Now we need to set our IP Parameters.
1. Client IP Address [###.###.###.###]
2. Server IP Address [###.###.###.###]
3. Gateway IP Address [###.###.###.###]
4. Subnet Mask [###.###.###.###]
Enter the correct addresses for each field.
**VERY IMPORTANT**
If your NIM master and client are on the SAME subnet, then you will set the Gateway IP Address to be the NIM master’s IP address. There were a few firmware levels that made you use 0.0.0.0 if the master and client were on the same network, but running into that is very rare. Use the NIM master’s IP.
If you are unsure or need to verify if your master and client are on the same network you can always go back to your NIM master and check to see what NIM thinks. Of course, NIM is only correct if the information provided to it was correct originally.
On the NIM master you’ll want to compare the “if1=” lines on the master and client:
# lsnim -l master |grep if1
if1 = master_net shadoebso
#lsnim -l VIOS-A |grep if1
if1 = master_net dipperbso
So we know the master and VIO server are defined on the same subnet. If the VIO server is defined on a different NIM network than the master, then use the VIO server’s gateway as the Gateway IP Address entry.
After setting the IP Addresses use ‘M’ to return to the main menu. You typically do not want to go into the “Adapter Parameters” (option 2 on the previous screen) to change the adapter parameters or disable spanning tree. Spanning Tree goes out and wakes up parts of the network that might have been disabled since they’re not in use. This can increase performance but also block bootp if it is not expecting to receive a request through that portion of the network.
Also, with the Ping Test....the ping test is not a reliable way to determine if bootp is going to work. In fact, we use it the other way around. If bootp is failing for some reason we may come back and check the ping test to see if that is having a problem as well. Do not presume bootp will fail if the ping test fails. It may, but it is not a reliable indication.
With our IP Parameters set we should now be back at the main menu.
SMS - SYSTEM MANAGEMENT SERVICES -
1. Select Language
2. Change Password Options
3. View Error Log
4. Setup Remote IPL (RIPL (Remote Initial Program Load))
5. Change SCSI Settings
6. Select Console
7. Select Boot Options
Now we’re ready to select our boot device.
7. Select Boot Options
The next menu should come up :
1. Select Install or Boot Device
2. Configure Boot Device Order
3. Multiboot Startup
You can take either option 1 which will make this a 1 time boot device selection, or option 2 which will permanently change the boot device order until either you or the system changes it back to your boot drive after the install is complete.
1. Select Install or Boot Device
Select Device Type :
1. Diskette
2. Tape
3. CD/DVD
4. IDE
5. Hard Drive
6. Network
7. List all Devices
Trust me, make your life easier and select :
7. List all Devices
The system will go out for you and scan itself to determine which devices are available to boot from. All of your available boot devices will be displayed here. The menu can be a little tricky here. If you have a device pre-selected it will have a 1 next to it under the “Current Position” column. Use the “Select Device Number” listing to choose the device you want to boot from.
The next screen will offer you three choices :
1. Information
2. Normal Mode Boot
3. Service Mode Boot
Select :
2. Normal Mode Boot
It shouldn’t really matter if you select normal or service mode, I always select normal mode.
Finally it asks if you’re sure you want to exit from SMS. Select ‘yes’ and let the boot go.
What you SHOULD see :
You’ll likely see a brief splash screen then the bootp request attempt.
Ideally you’ll see something like :
BOOTP : S=1 R=1
Which indicates it sent and received a request successfully.
Next the master will tftp the boot image over so you should see a number rapidly increasing to roughly 24000-ish. Once that is complete the VIO server will go through all of the necessary LED codes to establish proper communication and NFS mounts. After that you will be presented with the BOS menu screens.
Creating a "nim_resources.tar" file using 'backupios':
To backup the VIO server to a remote file system:
1. Create a mount directory where the backup image, nim_resources.tar package, will be written to.
Example: $ mkdir /VIOS-A/backup
2. Mount an exported directory from the NIM master on the mount directory
Example: $ mount server1:/export/VIOS_backup /VIOS-A/backup
3. Run the backupios command with the –file option. Make sure to specify the path to the mounted directory
Example: $ backupios –file /VIOS-A/backup
This will create the nim_resources.tar package, written to the NFS mounted directory.
Note the $ prompt used in these commands. It's worth noting that you should be logged in as the pdamin user on the VIOS when running these commands. Commands that are run from the # (root) prompt after issuing a setup_oem_env command are not supported.
1. Can an lpp_source and SPOT created from AIX media be used to install the VIO server?
No. The AIX media and the VIOS media are not interchangeable. All of the device drivers and the other software needed to install the VIO server are located on the VIOS media. The AIX media does not contain all of the same device drivers or software necessary to install the VIO server, so AIX resources cannot be used to install a VIO server.
2. Do I need to use an lpp_source to install the VIO server using NIM?
No. All of the device drivers and software packages needed to install the VIO server are in the VIO server mksysb.
3. At what level does the VIO server media start containing two mksysb images that must be combined to create the NIM mksysb resource?
If using VIO server media version 1.5.1 or higher, there will be two mksysb images that must be combined to create the NIM mksysb resource.
This guide is intended for those who are new to installing the VIO server using NIM.
For a more advanced, “official,” or higher level NIM specific guide you can check out the recently released redbooks :
NIM From A-Z in AIX 5L
Just go to http://www.redbooks.ibm.com/ and type in SG24-7296-00 in the search field on the upper right hand side of the page.
You can type SG24-7940-03 in the search field for the VIOS redbook
I will try to keep this as reader friendly as possible. Feel free to make suggestions on how to improve this document, and if there are any procedures you would like to see added in the future, let me know and I’ll do my best to add these procedures.
Finally, there always will be someone to tell you how you should appropriately type out “commands to be executed.” Some people prefer that commands are put in single quotes, others prefer that they’re in italics.....whenever there is an actual command to execute, the example of that command will look just like your command line interface on your AIX system.
For example if I want you to do a listing of files in a directory I would have the example of :
# ls -al
I prefer to do it this way because this is exactly what you’ll see when you type it on your system....(and ok, it’s also easier to cut and paste the commands from my test system into this document once I’ve verified they work correctly).
Does :
- Key words and definitions
- Setup of the client definition on the NIM master
- Setup of NIM resources
- Copy the VIOS mksysb image from the CD to your NIM master
- mksysb
- SPOT
- 'bosinst_data' resource
- Installation of VIOS to NIM clients
- Backup of the VIO server using the "nim_resrouces.tar" package
- VIOS/AIX matrix
- Frequently Asked Questions
--SMIT and Command line processes for most of the operations I cover
Does not:
- Any WebSM interface procedures on how to use NIM. WebSM is not used enough to spend any time going into separate sections on how to use it.
- NIM communication information (including nimsh and firewall issues)
Key words and definitions :
If you are unfamiliar with NIM I highly recommend reading through this section and also use it as a reference while reading through this guide. If you’re figuring, “eh he’ll explain all this later so I’ll just skip this part,” you will be flipping back to this.
So this is sort of like a “reverse-glossary.” I’ll give the best and easiest to understand description of these terms so that you’ll hopefully have a much easier time understanding any new concepts you’re unfamiliar with. In all cases - actually using the files/keywords are handled in greater detail in their corresponding “How To...” sections.
Important Files and Directories:
- /etc/bootptab :
This file will exist on the NIM master. In a quiet NIM environment with no operations that require a client to boot, this file will be empty (except for the pre-existing commented section). This file gets updated automatically by the NIM master when a NIM operation is executed that requires the client machine (from here on out referred to as the "VIO server") to boot from a NIM SPOT. If this file contains incorrect information about either the master or the VIO server, the boot operation will fail. While this file “can” be edited manually to fix a bootp issue - it should not be, as you are only applying a “band-aid” fix to an existing issue in your NIM environment....but, sometimes it’s 5pm on a Friday and you’re ready to go home, right?
(Also note related entry ‘bootp’)
- /etc/exports :
This is not a “NIM specific” file, it is a NIM critical file. Any sort of installation, boot, mksysb, savevg....etc operation requires the use of NFS. This file will be updated with which locations are NFS exported from the master to the VIO server and the permissions associated with those exports. If these entries are incorrect or incomplete you will run into boot failures, permission problems, and other errors commonly associated with NFS. This is a text file and also “can” be edited manually to sometimes “band-aid” a problem, but should only be done so with care in knowing exactly what you’re doing. The good thing is, if we mess up this file we can remove it and recycle NFS. The file can be recreated.
- /etc/hosts :
While not a “NIM specific” file, it is also a NIM critical file. This file is sort of like a phone book. It gives a relationship between a system’s hostname and an IP address. Much like a telephone, if you dial the wrong number you get the wrong person. In NIM, if your IP address does not match up to the correct hostname, your install fails. This is a text file and can be edited manually. There should also only be one entry per IP/hostname. I personally prefer to make sure my NIM master has all entries in the /etc/hosts file and are of the following format :
<ipadress> <shortname> <longname>
- /etc/niminfo:
This file should always exist on the NIM master. This file is built when you first initialize the NIM environment. This is simply a text file so feel free to ‘cat’ or ‘more’ the file and look at the entries included in there. You do not want to manually edit this file if there is a mistake in the definition of the master. In this case you will want to redefine the master, or use the feature in NIM to change the master’s attributes (hostname, gateway....etc).
- /tftpboot :
This directory holds the boot images, along with informational files about the VIO server that is being installed by the NIM master. The file names that are generated generally look like this:
VIO_SPOT.chrp.mp.ent à This is the boot image that will be used to boot the VIO server.
The format for the file is:
<spotname>.<system_architecture>.<processor_type>.<adapter_type>
A <VIOserver_hostname> file is created and linked to the boot image on the NIM master.
Example:
# ls -l /tftpboot
-rw-r--r-- 1 root system 1234567 Jan 1 2010 VIO_SPOT.chrp.mp.ent
lrwxrwxrwx 1 root system 27 Jan 5 2010 fork.austin.ibm.com -> /tftpboot/VIO_SPOT.chrp.mp.ent
-rw-r--r-- 1 root system 256 Jan 5 2010 fork.austin.ibm.com.infoIn this example, the VIOS_SPOT.chrp.mp.ent file is the network boot image, which has been created from the SPOT. The fork.austin.ibm.com link points to the boot image and is referenced during the boot operation. The austin.ibm.com.info file contains information that is needed by the client during installation, like which resources should be NFS mounted at boot time.
Important Keywords :
Allocate/Allocation :
This process is what allows your VIO server to access resources in NIM. The master uses NFS to perform the allocation process. You can see which resource types are allocated to your VIO server by using the following command:
# lsnim -a spot
VIOS-A:
spot = VIOS-A_spot
VIOS-B:
spot = VIOS-B_spot
VIOS-C:
spot = VIOS-C_spot
In this example environment we have 3 VIO servers defined as NIM clients. Since the SPOT resources are created from the mksysb image of a VIO server, giving the SPOT resource a descriptive name that relates back to the hostname of your VIO server is helpful.
So we can see that ‘VIOS-A’ has a SPOT called “VIOS-A_spot” allocated, ‘VIOS-B’ has a SPOT called “VIOS-B_spot" allocated, and ‘VIOS-C’ has a SPOT called “VIOS-C_spot” allocated.
Base Operating System Installation :
Also commonly called (and referred to from here on out) as a bos_inst operation. This simply refers to the fact that you are initiating a boot and installation to a VIO server.
Bootp :
This is the initial communication made between the NIM master and the VIO server during a boot or bos_inst operation. In order for this to be successful several factors must be met:
1. - bootpd must be running on the NIM master
2. - You must have the correct IP information for the NIM master and target VIO server
3. - The /etc/bootptab must be populated correctly
4. - If the master and VIO server are on separate networks, the router must be set to forward bootp packets.
There are other causes of failure, but checking/verifying those four will solve most bootp issues.
Tftp (Trivial File Transfer Protocol):
When the VIO server has been rebooted for a bos_inst operation you don’t have access to normal TCP communication. Once bootp connection has successfully been achieved, the NIM master uses tftp to transfer over the <clientname>.info file and the boot image to the VIO server.
if1= : Or interface 1:
This is known as the ‘nim network’. Every machine, even the master, is placed on a defined NIM network. A machine who has multiple adapters defined to NIM will have “if2=” and “if3=”....etc attributes. When your NIM master is initialized, you will create a network name for the master and VIO server on the same subnet as the master. If you name this network “master_net” for example, then all VIO server NIM clients on the same subnet as the master will have their “if1=” line set to “master_net”. If you add additional VIO server NIM clients that are on separate subnets, then you will need to create new network names. You can see the “if1=” information from an :
# lsnim -l master |more
-or-
# lsnim -l <client>
You can get further information about the network name by running an ‘lsnim -l <network_name>’.
Having incorrect networking information is probably the leading cause of NIM installation failures. This attribute and the information used when creating networks is extremely important to make sure you have correct.
Client (NIM client) :
A standalone VIO server, or lpar in a NIM environment, other than the NIM master. Clients (VIO servers) use the resources that reside on the NIM master to be installed.
Master (NIM master) :
The only system in a NIM environment that has permission to run NIM commands remotely on other NIM clients (VIO servers). A NIM client (VIO server) can only have one NIM master. The NIM master must be at a level equal to or higher than the AIX level that is being installed (and accompanying VIOS software) to the VIO server.
The NIM master will also hold all of the NIM resources. Due to this fact, you will want to make sure that the NIM master has plenty of space available for storing these resources. It is ideal to have a separate volume group (example: nimvg) so that the rootvg does not get out of control in size.
Resource (NIM resources) SPOT:
The SPOT resource (stands for Shared Product Object Tree in case you were wondering) is responsible for the following:
- Creating a boot image to send to the VIO server over the network.
- Supplying the commands needed to install the VIO server.
You will need to create a SPOT from your NIM VIOS mksysb resource. The VIOS mksysb contains all of the filesets and commands needed to create the SPOT. A SPOT created from an AIX lpp_source is not a valid SPOT to use when installing a VIO server. Essentially the SPOT is a /usr filesystem just like the one on your NIM master. You can think of it as having multiple “mini-systems” on your NIM master, because each SPOT is its own /usr filesystem. Just like your NIM master’s /usr filesystem, going in there manually and messing around with files can easily corrupt it. The good thing about a SPOT however, is that it is easily rebuilt.
Resource (nim resources) mksysb :
This is simply a mksysb image of the VIO server. The mksysb image resides on the NIM master, and the resource can be defined in one of two ways:
-From an existing mksysb image residing on the NIM master already (extracted from a nim_resources.tar package).
-From the mksysb image copied from the VIOS installation media on to hard disk (this is described in more detail later in this document).
You can also back up an already existing VIO Server and a nim_resources.tar package is created. The contents of this nim_resources.tar package can be stored on the NIM master for installations to new VIO servers, and also for the recovery of VIO servers using NIM. The nim_resources.tar file contains all the necessary resources to restore the Virtual I/O server. This includes the mksysb image, the bosinst.data file, the network boot image, and SPOT resource. When the VIO server is backed up, the nim_resources.tar package will be stored on an NFS mounted file system. More detailed steps for creating the nim_resources.tar file with the ‘backupios’ command will be given later in this document.
Resource (nim resources) bosinst_data:
When booting from an installation image to install a system, the system boots to what are known as the “BOS Menus” or Base Operating System Installation Menus. Here is where you select your console, the language to use during the installation, and what disks to install to amongst other things. When using NIM, a “bosinst_data” resource can be created, and it will answer all of the questions that are asked by the “BOS Menus” automatically. This resource is used when installing to a NIM client that may be in another building, down the road or half way across the country. This way, once you kick off the installation from the NIM master, there is no further interaction that is required. The system should (ideally) install and reboot itself afterwards.
Backupios (VIO server backup):
The ‘backupios’ command will back up the Virtual I/O Server and user-defined virtual devices to tape, DVD, or a remote file system. You can back up the Virtual I/O Server and user-defined virtual devices using the backupios command.
A mksysb (as discussed above) has a “built in” bosinst.data file. If the option in that file
(PROMPT =) is set to yes, this file really does nothing as the choices you make in the BOS menus will override the options in the file. However, if the mksysb was created to have that option set to no, then we can create a new bosinst_data resource which will trump the one that is part of the mksysb.
(PROMPT =) is set to yes, this file really does nothing as the choices you make in the BOS menus will override the options in the file. However, if the mksysb was created to have that option set to no, then we can create a new bosinst_data resource which will trump the one that is part of the mksysb.
Define the VIOS partition as a NIM client:
*Note: Any VIOS 1.X client should be created with 'netboot_kernel=mp', and any VIOS 2.X client should be created with 'netboot_kernel=64'.
From command line:
# nim -o define -t standalone -a platform=chrp -a if1="find_net lucidbso 0" -a netboot_kernel=mp -a connect=shell <clientname>
Breaking down the command
# nim -o define -t standalone : This is the NIM operation of defining an object of type=standalone.
... -a platform=chrp : The platform type of the target system.
...-a if1=”find_net lucidbso 0’ : Used when a client is part of an existing NIM network. In this case my NIM client belongs to my master’s NIM network (i.e. They’re on the same subnet) so I don’t need to give it further information. If this client needed to be on a new network then we would not use the “find_net” but rather the “net_definition=<value>.......” options where you’d need to specify “networktype” “subnetmask” “clientgateway” “networkname” attributes.
... -a netboot_kernel=mp : As noted previously, you should set this attribute to "mp" when installing VIOS 1.X and "64" when installing VIOS 2.1 on the client.
... -a connect=shell : This will be the connection type used. You can either choose shell (rsh) or nimsh.
... <clientname> : The client's resolvable hostname.
From SMIT :
# smitty nim_mkmac
-or-
# smitty nim
==> Administration Tasks ==> Manage Machines ==> Define a Machine
From here you are asked what the hostname of the NIM client (VIO server) is. If the master can resolve the client’s hostname and he already has a NIM network associated with the client’s network then it will “pre-generate” a final acceptance screen for you (see example 1). If the master does not already have a NIM network defined for this client, then you will be prompted for more information and be asked to create a new network for this new client (see example 2).
Example 1:
* NIM Machine Name [clientname]
* Machine Type [standalone]
* Hardware Platform Type [chrp] Make sure this is set to ‘chrp’
Kernel to use for Network Boot [mp]
Communication Protocol used by client []
Primary Network Install Interface
* Cable Type bnc
Network Speed Setting []
Network Duplex Setting []
* NIM Network master_net
* Host Name client_hostname
Network Adapter Hardware Address [0]
Network Adapter Logical Device Name []
IPL ROM Emulation Device []
CPU Id []
Machine Group []
Comments []
Notice that you actually don’t have to fill in anything else as a required field. The master can resolve the VIO server's hostname, and he has put it on the same network (master_net) because in this case, the client and master are on the same network. I recommend leaving the optional fields blank. Pressing <enter> from here will define the client.
-or-
Example 2:
* NIM Machine Name [clientname]
* Machine Type [standalone]
* Hardware Platform Type [chrp]
Kernel to use for Network Boot [mp]
Communication Protocol used by client []
Primary Network Install Interface
* Cable Type bnc
Network Speed Setting []
Network Duplex Setting []
* NIM Network [10_14_20_Net]
* Network Type ent
* Ethernet Type Standard
* Subnetmask [255.255.254.0]
* Default Gateway Used by Machine [10.14.20.1]
* Default Gateway Used by Master [9.3.58.1]
* Host Name clientname
Network Adapter Hardware Address [0]
Network Adapter Logical Device Name []
IPL ROM Emulation Device []
CPU Id []
Machine Group []
Comments []
Notice that there is much more to fill out here, because we’re defining a new network within NIM. We’ll say my VIO server's ip = 10.14.20.34. I’m naming my NIM network 10_14_20_Net so that it is descriptive just by looking at the name. Also, any other VIO server on this same network will now be auto-defined on this new network and the SMIT screens will look like example 1 above. We’ll also need to enter my subnetmask and the gateway used by this client. The master’s gateway is filled in for you. Once this successfully completes I‘ll now have 2 NIM networks.......master_net and 10_14_20_Net.
Setup of NIM resources:
Now that the client machine has been defined, you are ready to set up the resources needed for the VIOS install using NIM.
Copy the VIOS mksysb image from the CD to your NIM master:
Mount the VIOS base DVD and copy the VIOS mksysb image from the DVD (in /usr/sys/inst.images) to your NIM master:
# mount -o ro -v cdrfs /dev/cd0 /mnt
# cd /mnt/usr/sys/inst.images
# cp mksysb_image /export/mksysb/mksysb_image
If using VIOS 1.5 or higher media, the mksysb file may be split into two parts. To combine these two parts and copy them to hdisk, run the following:
# cat /mnt/usr/sys/inst.images/mksysb_image /mnt/usr/sys/inst.images/mksysb_image2 > /export/mksysb/mksysb_image
***You can substitute any path you would like to save the combined mksysb image, for ‘/export//mksysb/mksysb_image'.
Define the mksysb resource on the NIM master:
# nim -o define -t mksysb -a server=master -a location=/export/mksysb/mksysb_image
VIOS_2_1_mksysb
-or-
# smitty nim_mkres => mksysb =>
Define a Resource
* Resource Name [VIOS_2_1_mksysb] Give the resource a name
* Resource Type mksysb
* Server of Resource [master] + Chose your NIM master
* Location of Resource [/export/mksysb/mksysb_image] / The full path of the mksysb file copied to hdisk
àLeave all other entries set to their default values.
Define a SPOT from the mksysb resource:
# nim -o define -t spot -a server=master -a location=/export/spot –a source=VIOS_2_1_mksysb VIOS_2_1_SPOT
-or-
# smitty nim_mkres => spot =>
Define a Resource
* Resource Name [VIOS_2_1_spot] give the spot a name
* Resource Type spot
* Server of Resource [master] + chose your NIM master
* Source of Install Images [VIOS_2_1_mksysb] + select the mksysb resource
* Location of Resource [/export/spot] / location to store the SPOT
àLeave all other entries set to their default values.
Define the 'bosinst_data' resource on the NIM master:
You'll also want to copy the bosinst.data file from the CD media to the NIM master and define that as a resource:
(If the CD has been mounted as described earlier, run the following command. If the CD was mounted in a different way, then simply change directories to the mount point and copy the file)
# cd /home
# mkdir VIOS_bosinst
# cp /mnt/bosinst.data /export/bosinst_data/VIOS_bosinst
# nim -o define -t bosinst_data -a server=master –a location=/export/bosinst_data/VIOS_bosinst VIOS_2_1_bosinst_data
-or-
# smitty nim_mkres => bosinst_data =>
Define a Resource
* Resource Name [VIOS_2_1_bosinst] Give the resource a name
* Resource Type bosinst_data
* Server of Resource [master] + chose the NIM master
* Location of Resource [/export/bosinst_data/VIOS_bosinst] / location of the file copied from CD
àLeave all other entries set to their default values.
Perform the bos_inst operation to set up the NIM resources:
To set the NIM master up to perform the installation, we issue a bos_inst operation:
# nim –o bos_inst –a source=mksysb –a spot=VIOS_2_1_spot -a bosinst_data=VIOS_2_1_bosinst -a accept_licenses=yes –a mksysb=VIOS_2_1_mksysb VIOS-A
# smitty nim_bosinst => select the nim client (VIOS lpar) from the list => select ‘mksysb’ as the installation type => select the VIOS mksysb from the list of mksysbs => select the spot created from the VIOS mksysb =>
Install the Base Operating System on Standalone Clients
* Installation Target VIOS-A
* Installation TYPE mksysb
* SPOT VIOS_2_1_spot
LPP_SOURCE [] +
MKSYSB VIOS_2_1_mksysb
BOSINST_DATA to use during installation [VIOS_2_1_bosinst] + Select the bosinst_data resource created from the VIOS mksysb
ACCEPT new license agreements? [yes] + Set to ‘yes’
Initiate reboot and installation now? [no] + Set to ‘no’ for the pull installation
àLeave all other entries set to their default values.
7. The master is set up....now what ?
The NIM master is now ready and waiting for a bootp request from the VIO server. In order to do this we need to boot the VIO server into SMS (System Management Services). You can simply boot your VIOS server using the SMS profile on the HMC.
The options you see in SMS can depend on the firmware level installed on the target system. I don’t have access to all variations of the menu options, so I’ll just use the one my NIM master has. You should be able to navigate closely enough, as the wording should be similar.
SMS - SYSTEM MANAGEMENT SERVICES -
1. Select Language
2. Change Password Options
3. View Error Log
4. Setup Remote IPL (RIPL (Remote Initial Program Load))
5. Change SCSI Settings
6. Select Console
7. Select Boot Options
First of all we need to provide the IP addresses necessary to tell the VIO server who he is, and where to boot from.
4. Setup Remote IPL (RIPL (Remote Initial Program Load))
Next you’ll have a selection of adapters to use. Select your adapter that corresponds to the adapter/host you defined in NIM. You will not see “ent0/ent1...etc” options. You will however see the hardware addresses and slots.
We then are brought to 3 options :
1. IP Parameters
2. Adapter Configuration
3. Ping Test
Chose the option to set the IP Parameters
Now we need to set our IP Parameters.
1. Client IP Address [###.###.###.###]
2. Server IP Address [###.###.###.###]
3. Gateway IP Address [###.###.###.###]
4. Subnet Mask [###.###.###.###]
Enter the correct addresses for each field.
**VERY IMPORTANT**
If your NIM master and client are on the SAME subnet, then you will set the Gateway IP Address to be the NIM master’s IP address. There were a few firmware levels that made you use 0.0.0.0 if the master and client were on the same network, but running into that is very rare. Use the NIM master’s IP.
If you are unsure or need to verify if your master and client are on the same network you can always go back to your NIM master and check to see what NIM thinks. Of course, NIM is only correct if the information provided to it was correct originally.
On the NIM master you’ll want to compare the “if1=” lines on the master and client:
# lsnim -l master |grep if1
if1 = master_net shadoebso
#lsnim -l VIOS-A |grep if1
if1 = master_net dipperbso
So we know the master and VIO server are defined on the same subnet. If the VIO server is defined on a different NIM network than the master, then use the VIO server’s gateway as the Gateway IP Address entry.
After setting the IP Addresses use ‘M’ to return to the main menu. You typically do not want to go into the “Adapter Parameters” (option 2 on the previous screen) to change the adapter parameters or disable spanning tree. Spanning Tree goes out and wakes up parts of the network that might have been disabled since they’re not in use. This can increase performance but also block bootp if it is not expecting to receive a request through that portion of the network.
Also, with the Ping Test....the ping test is not a reliable way to determine if bootp is going to work. In fact, we use it the other way around. If bootp is failing for some reason we may come back and check the ping test to see if that is having a problem as well. Do not presume bootp will fail if the ping test fails. It may, but it is not a reliable indication.
With our IP Parameters set we should now be back at the main menu.
SMS - SYSTEM MANAGEMENT SERVICES -
1. Select Language
2. Change Password Options
3. View Error Log
4. Setup Remote IPL (RIPL (Remote Initial Program Load))
5. Change SCSI Settings
6. Select Console
7. Select Boot Options
Now we’re ready to select our boot device.
7. Select Boot Options
The next menu should come up :
1. Select Install or Boot Device
2. Configure Boot Device Order
3. Multiboot Startup
You can take either option 1 which will make this a 1 time boot device selection, or option 2 which will permanently change the boot device order until either you or the system changes it back to your boot drive after the install is complete.
1. Select Install or Boot Device
Select Device Type :
1. Diskette
2. Tape
3. CD/DVD
4. IDE
5. Hard Drive
6. Network
7. List all Devices
Trust me, make your life easier and select :
7. List all Devices
The system will go out for you and scan itself to determine which devices are available to boot from. All of your available boot devices will be displayed here. The menu can be a little tricky here. If you have a device pre-selected it will have a 1 next to it under the “Current Position” column. Use the “Select Device Number” listing to choose the device you want to boot from.
The next screen will offer you three choices :
1. Information
2. Normal Mode Boot
3. Service Mode Boot
Select :
2. Normal Mode Boot
It shouldn’t really matter if you select normal or service mode, I always select normal mode.
Finally it asks if you’re sure you want to exit from SMS. Select ‘yes’ and let the boot go.
What you SHOULD see :
You’ll likely see a brief splash screen then the bootp request attempt.
Ideally you’ll see something like :
BOOTP : S=1 R=1
Which indicates it sent and received a request successfully.
Next the master will tftp the boot image over so you should see a number rapidly increasing to roughly 24000-ish. Once that is complete the VIO server will go through all of the necessary LED codes to establish proper communication and NFS mounts. After that you will be presented with the BOS menu screens.
Backing up the VIO server:
Creating a "nim_resources.tar" file using 'backupios':
To backup the VIO server to a remote file system:
1. Create a mount directory where the backup image, nim_resources.tar package, will be written to.
Example: $ mkdir /VIOS-A/backup
2. Mount an exported directory from the NIM master on the mount directory
Example: $ mount server1:/export/VIOS_backup /VIOS-A/backup
3. Run the backupios command with the –file option. Make sure to specify the path to the mounted directory
Example: $ backupios –file /VIOS-A/backup
This will create the nim_resources.tar package, written to the NFS mounted directory.
Note the $ prompt used in these commands. It's worth noting that you should be logged in as the pdamin user on the VIOS when running these commands. Commands that are run from the # (root) prompt after issuing a setup_oem_env command are not supported.
VIOS/AIX matrix:
FAQs:
1. Can an lpp_source and SPOT created from AIX media be used to install the VIO server?
No. The AIX media and the VIOS media are not interchangeable. All of the device drivers and the other software needed to install the VIO server are located on the VIOS media. The AIX media does not contain all of the same device drivers or software necessary to install the VIO server, so AIX resources cannot be used to install a VIO server.
2. Do I need to use an lpp_source to install the VIO server using NIM?
No. All of the device drivers and software packages needed to install the VIO server are in the VIO server mksysb.
3. At what level does the VIO server media start containing two mksysb images that must be combined to create the NIM mksysb resource?
If using VIO server media version 1.5.1 or higher, there will be two mksysb images that must be combined to create the NIM mksysb resource.
No comments:
Post a Comment