5120's & LDOMs

New Stuff from sun with T5120 T5140's

Power on via the ilom

 start /SYS

then switch to the console

start /SP/console

Default ilom password

root , changeme

Changing the ilom default password

set /SP/users/root password

Enter new Password: *******

Reset machine back to factory defaults

 with a running machine
ldm set-spconfig factory-default
or from the service processor
bootmode config="factory-default"

Enable/ Disable SSH

set /SP/services/ssh state=[enable|disable]

Display the active ILOM sessions

 show /SP/sessions

Display information about commands

show /SP/cli/commands

Add a local user 

create /SP/users/bob password=password role=administrator|operator

Delete a local user 

delete /SP/users/fred


change the ip address to static


cd /SP/network
set pendingipdiscovery=static
set pendingipaddress=xxx.xxx.xxx.xxx
set pendingipnetmask=yyy.yyy.yyy.yyy
set pendingipgatwat=zzz.zzz.zzz.zzz

show to verify setings

set commitpending=true


Enable web interface

set /SP/services/http state=enabled

Reset Service Processor

reset /SP

Boot System

start /SYS
(reset /SYS - for a vulgar system reset)


How do i break into the obp on a 5120

set /HOST send_break_action=break
start /SP/console
r)eboot, o)k prompt, h)alt?
o

 
- why , every domain gets it's own obp

the older revision procedure is to send a break , select r to reboot , then send another break to get to the obp for that ldom


one other way 
set /HOST/bootmode script="setenv auto-boot? false"
then init 0 or / reset /sys

Jumpstart from the control domain ldom

so you have build your jet/jumpstart server and your first ldom host , and try and boot net install

but all you get is arp timeouts

 you even snoop your interfaces looking for packets ... but still nothing!

 ok here is the fix

 (my example uses two interfaces)

first you need to use other interfaces than your e1000g0 g1 etc

remove your vsw interfaces from primary control domain (if configured)

then examine the real mac addesses for your interfaces

-bash-3.00# ifconfig -a
lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1
        inet 127.0.0.1 netmask ff000000

e1000g0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2
        inet xx.yy.zz.40 netmask fffff000 broadcast x.x.x.x
        ether xx:xx:xx:xx:be:7b

e1000g1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 3
        inet zz.yy.zz.40 netmask fffff000 broadcast x.x.x.x
        ether xx:xx:xx:xx:be:7a

 now unplumb both interfaces

-bash-3.00# ifconfig e1000g0 unplumb

-bash-3.00# ifconfig e1000g1 unplumb

 now configure the virtual switch interfaces

-bash-3.00# ldm add-vsw mac-addr=<MAC Addr from above> net-dev=e1000g0 vsw0 primary

-bash-3.00# ldm add-vsw mac-addr=<MAC Addr from above> net-dev=e1000g0 vsw0 primary 

 now ifconfig vsw0 and vsw1

-bash-3.00# ifconfig vsw0 plumb

-bash-3.00# ifconfig vsw1 plumb

 and ifconfig your interfaces as you add them before

 -bash-3.00# ifconfig vsw0 up xx.yy.zz.40 netmask + broadcast +

 -bash-3.00# ifconfig vsw1up zz.yy.zz.40 netmask + broadcast +

no ifconfig will show :

-bash-3.00# ifconfig -a
lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1
        inet 127.0.0.1 netmask ff000000

vsw0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2
        inet xx.yy.zz.40 netmask fffff000 broadcast x.x.x.x
        ether xx:xx:xx:xx:be:7b

vsw1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 3
        inet zz.yy.zz.40 netmask fffff000 broadcast x.x.x.x
        ether xx:xx:xx:xx:be:7a

 copy your interface files from the old e1000g0 files so your changes survive a reboot

-bash-3.00# mv /etc/hostname.e1000g0 /etc/hostname.vsw0

-bash-3.00# mv /etc/hostname.e1000g1 /etc/hostname.vsw1

 

 you should now be able to jumpstart from your control domain

these steps are not required if your jumpstart is remote from the ldom host

 

Set up the hardware mirror (t2000 & t5120)

think really carefully before you do this ... if you dont leave a slice for liveupgrade you will not be able to split the mirror for patching or os upgrade

(best done from cdrom/net boot as you get into the well known catch 22 problem)

/usr/sbin/raidctl [-f] -c cXtXdX cYtYdY


# raidctl
Controller: 1
        Disk: 0.0.0
        Disk: 0.1.0
# format < /dev/null
Searching for disks...done


AVAILABLE DISK SELECTIONS:
       0. c1t0d0 <SUN146G cyl 14087 alt 2 hd 24 sec 848>
          /pci@0/pci@0/pci@2/scsi@0/sd@0,0
       1. c1t1d0 <SUN146G cyl 14087 alt 2 hd 24 sec 848>
          /pci@0/pci@0/pci@2/scsi@0/sd@1,0
Specify disk (enter its number):
# raidctl -c c1t0d0 c1t1d0
Creating RAID volume will destroy all data on spare space of member disks, proceed (yes/no)? y
/pci@0/pci@0/pci@2/scsi@0 (mpt0):
        Physical disk 0 created.
/pci@0/pci@0/pci@2/scsi@0 (mpt0):
        Physical disk 1 created.
/pci@0/pci@0/pci@2/scsi@0 (mpt0):
        Volume 0 is |enabled||optimal|
/pci@0/pci@0/pci@2/scsi@0 (mpt0):
        Volume 0 is |enabled||optimal|
/pci@0/pci@0/pci@2/scsi@0 (mpt0):
        Volume 0 created.
/pci@0/pci@0/pci@2/scsi@0 (mpt0):
        Physical disk (target 1) is |out of sync||online|
/pci@0/pci@0/pci@2/scsi@0 (mpt0):
        Volume 0 is |enabled||degraded|
/pci@0/pci@0/pci@2/scsi@0 (mpt0):
        Volume 0 is |enabled||resyncing||degraded|
Volume c1t0d0 is created successfully!
# raidctl
Controller: 1
        Volume:c1t0d0
        Disk: 0.0.0
        Disk: 0.1.0

List Ldoms

# ./ldm list
NAME             STATE    FLAGS   CONS    VCPU  MEMORY   UTIL  UPTIME
primary          active   -n-c-   SP      32    8064M    0.0%  3m

Setup ldom default service

# /opt/SUNWldm/bin/ldm add-vds primary-vds0 primary
# /opt/SUNWldm/bin/ldm add-vcc port-range=5000-5100 primary-vcc0 primary
# /opt/SUNWldm/bin/ldm add-vsw net-dev=e1000g0 primary-vsw0 primary
# /opt/SUNWldm/bin/ldm set-mau 1 primary
# /opt/SUNWldm/bin/ldm set-vcpu 4 primary
 

List ldom services


# /opt/SUNWldm/bin/ldm list-services primary
VCC
    NAME             PORT-RANGE
    primary-vcc0     5000-5100

VSW
    NAME             MAC               NET-DEV   DEVICE     MODE
    primary-vsw0     00:14:4f:fb:37:32 e1000g0   switch@0

VDS
    NAME             VOLUME         OPTIONS          DEVICE
    primary-vds0

Listing ldom sp config


# /opt/SUNWldm/bin/ldm list-spconfig
factory-default [current]

Saving sp config

# /opt/SUNWldm/bin/ldm add-spconfig initial
#  /opt/SUNWldm/bin/ldm list-spconfig
factory-default [current]
initial [next]

(remember to reboot if you have made changes)

List ldom service bindings

bash-3.00# ./ldm list-bindings primary
NAME             STATE    FLAGS   CONS    VCPU  MEMORY   UTIL  UPTIME
primary          active   -n-cv   SP      4     1G       0.0%  3m

MAC
    00:21:28:ZZ:YY:XX

VCPU
    VID    PID    UTIL STRAND
    0      0      0.0%   100%
    1      1      0.0%   100%
    2      2      0.0%   100%
    3      3      0.0%   100%

MAU
    ID     CPUSET
    0      (0, 1, 2, 3, 4, 5, 6, 7)

MEMORY
    RA               PA               SIZE
    0x8000000        0x8000000        1G

VARIABLES
    auto-boot?=false
    boot-device=/pci@0/pci@0/pci@2/scsi@0/disk@0,0:a disk net
    keyboard-layout=US-English

IO
    DEVICE           PSEUDONYM        OPTIONS
    pci@0            pci
    niu@80           niu

VCC
    NAME             PORT-RANGE
    primary-vcc0     5000-5100

VSW
    NAME             MAC               NET-DEV   DEVICE     MODE
    primary-vsw0     00:14:4f:fb:37:32 e1000g0   switch@0

VDS
    NAME             VOLUME         OPTIONS          DEVICE
    primary-vds0

VCONS
    NAME             SERVICE                     PORT
    SP

Example simple Ldom

bash-3.00# ldm add-domain secondary
bash-3.00# ldm add-vcpu 12 secondary
bash-3.00# ldm add-memory 1G secondary
bash-3.00# ldm add-vnet vnet1 primary-vsw0 secondary
bash-3.00# ldm add-vdsdev /dev/zvol/rdsk/domains/secondary vol1@primary-vds0
bash-3.00# ldm add-vdisk vdisk1 vol1@primary-vds0 secondary
bash-3.00# ldm set-variable auto-boot\?=false secondary
bash-3.00# ldm set-variable boot-device=/virtual-devices@100/channel-devices@20
0/disk@0 secondary
bash-3.00# ldm bind-domain secondary
bash-3.00# ldm start-domain secondary

LDom secondary started
bash-3.00# telnet localhost 5000
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.

Connecting to console "secondary" in group "secondary" ....
Press ~? for control options ..

{0} ok banner

SPARC Enterprise T5120, No Keyboard
Copyright 2008 Sun Microsystems, Inc.  All rights reserved.
OpenBoot 4.28.0, 1024 MB memory available, Serial #66796533.
Ethernet address 0:14:4f:fb:3b:f5, Host ID: 83fb3bf5.


##list physical cdrom / cdr drives

-bash-3.00# cdrw -l
Looking for CD devices...
    Node                   Connected Device                Device type
----------------------+--------------------------------+-----------------
 /dev/rdsk/c0t0d0s2   | TSSTcorp CD/DVDW TS-T632A SR03 | CD Reader/Writer

Set up an iso image as a vdisk (to boot from)

-bash-3.00# ldm add-vdsdev /var/tmp/sol-10-u5-ga-sparc-dvd.iso iso@primary-vds0
-bash-3.00# ldm add-vdisk iso iso@primary-vds0 secondary

Yes it will run a sparc linux system:)

this was a quick test using the iso tip abobe and booting from the cdrom

 

~ # cat /proc/cpuinfo
cpu             : UltraSparc T2 (Niagara2)
fpu             : UltraSparc T2 integrated FPU
prom            : OBP 4.28.0 2008/01/22 21:10
type            : sun4v
ncpus probed    : 12
ncpus active    : 1
D$ parity tl1   : 0
I$ parity tl1   : 0
Cpu0ClkTck      : 00000000457646c0
MMU Type        : Hypervisor (sun4v)


ilom/Sp Is LINUX !


U-Boot 1.1.1 (Apr  3 2008 - 19:06:21)

CPU:   MPC885ZPnn at 133 MHz: 8 kB I-Cache 8 kB D-Cache FEC present
Board: SPARC885
       Watchdog enabled
I2C:   ready
DRAM: 
trying 128 MBytes
trying 64 MBytes
(64 MB SDRAM) 64 MB
Memory Tests: DA A1 A2 00 FF 55 AA T2 T3 T4
POST memory PASSED
FLASH: 32 MB
In:    serial
Out:   serial
Err:   serial
Net:   FEC ETHERNET
POST i2c  c  d 18 20 23 2a 2d 2e 30 40 43 46 51 53 54 56 68 69 6a 6b 70 71 PASSED
POST cpu PASSED
POST ethernet PASSED
Booting linux in 5 seconds...
## Booting image at fe080000 ...
   Image Name:   Linux-2.4.22
   Image Type:   PowerPC Linux Kernel Image (gzip compressed)
   Data Size:    814987 Bytes = 795.9 kB
   Load Address: 00000000
   Entry Point:  00000000
   Verifying Checksum ... OK
   Uncompressing Kernel Image ... OK