In Data ONTAP 8.3 or later, each SVM initiates outbound connections to name servers using its own LIFs and routes, not those of the node_mgmt LIF as in previous releases.
IMPORTANT information for ONTAP 8.3 and later customers:
SVMs (vserver) lacking a specific route, and also having multiple default routes containing the same routing metric (possibly from separate routing groups before the upgrade) could fail to reach the name servers if any of the specified gateways do not provide a path to those servers as the node_mgmt_LIF can no longer be used to reach such services.
Protocols that could be affected:
Any protocol that relies on name services (LDAP, NIS, DNS, Active Directory) is susceptible, for example : NFS and CIFS data protocols are particularly susceptible.
Symptoms:
1. CIFS: clients cannot reach share names.
2. AD: SVM fails to join AD.
3. NFS: Clients cannot mount exports that include host-names instead of IP.
Workaround/Solution:
1. If each gateway can reach all name services, then no changes are needed.
2. Ensure that the correct default route with access to all names services has the lowest metric.
3. Create a specific routing group and add a specific route to the destination.
Courtesy: KBID:1000317
Wednesday 19 December 2018
Thursday 13 December 2018
SMB 1.0 is disabled by default in ONTAP 9.3
For environments with existing CIFS servers running SMB 1.0, migrate to a later SMB version as soon as possible to get best of security and compliance enhancements.
Microsoft extended support has ended for SMB 1.0 :
Windows 2000 (July 13, 2010)
Windows XP (April 8, 2014)
Windows Server 2003 R2 (July 14, 2015)
In ONTAP 9, 9.1, SMB = Enabled by default
In ONTAP 9.2, SMB = Can be disabled
In ONTAP 9.3, SMB = Disabled by-default
Advantage in ONTAP 9.x:
SMB 2.1 Large MTU: allows SMB’s maximum transmission unit (MTU) to be increased from 64KB (default) to 1MB. Doing so significantly improves the speed and efficiency of large file transfers while reducing the number of packets that need to be processed. This approach enables customers using 10-gigabit Ethernet to take advantage of their high-bandwidth networks.
command:
cifs options modify -is-large-mtu-enabled true
When disabled (the default setting), NetApp® ONTAP® software advertises max read and write sizes as 64K.
NOTE: Do not confuse it with Network MTU, SMB MTUs do not refer to MTU sizes used at the network layer.
Microsoft extended support has ended for SMB 1.0 :
Windows 2000 (July 13, 2010)
Windows XP (April 8, 2014)
Windows Server 2003 R2 (July 14, 2015)
In ONTAP 9, 9.1, SMB = Enabled by default
In ONTAP 9.2, SMB = Can be disabled
In ONTAP 9.3, SMB = Disabled by-default
Advantage in ONTAP 9.x:
SMB 2.1 Large MTU: allows SMB’s maximum transmission unit (MTU) to be increased from 64KB (default) to 1MB. Doing so significantly improves the speed and efficiency of large file transfers while reducing the number of packets that need to be processed. This approach enables customers using 10-gigabit Ethernet to take advantage of their high-bandwidth networks.
command:
cifs options modify -is-large-mtu-enabled true
When disabled (the default setting), NetApp® ONTAP® software advertises max read and write sizes as 64K.
NOTE: Do not confuse it with Network MTU, SMB MTUs do not refer to MTU sizes used at the network layer.
Tuesday 11 December 2018
Go for ONTAP 9.3 minimum: iSCSI Performance Improvements
If iSCSI is your MAIN SAN protocol, then you must think about upgrading your controllers to 9.3 minimum:
Reason: The entire iSCSI stack was re-written for ONTAP 9.3. This was done to increase parallelization so that iSCSI processing could take advantage of the higher core controllers that NetApp has been shipping recently.
Prior to the re-write some iSCSI processing was single threaded and therefore had a bottleneck where processing was dependent on the speed of a single processor core. By refactoring the NetApp software iSCSI target to multi-threaded iSCSI processing iSCSI (ONTAP 9.3) is now able to take advantage of numerous CPU cores each of which can concurrently process iSCSI threads.
In addition, increasing multi-threading, other improvements, while perhaps not as substantial, were also included. These include reducing or eliminating locks and context switches and other incremental improvements. The improvements in iSCSI performance seen from this refactoring of iSCSI software target are substantial with the largest improvements seen on larger controllers that have more cores that can be used to concurrently process iSCSI threads.
Compared to 9.2: 2.6x Performance increase @1ms
TR:4080
Reason: The entire iSCSI stack was re-written for ONTAP 9.3. This was done to increase parallelization so that iSCSI processing could take advantage of the higher core controllers that NetApp has been shipping recently.
Prior to the re-write some iSCSI processing was single threaded and therefore had a bottleneck where processing was dependent on the speed of a single processor core. By refactoring the NetApp software iSCSI target to multi-threaded iSCSI processing iSCSI (ONTAP 9.3) is now able to take advantage of numerous CPU cores each of which can concurrently process iSCSI threads.
In addition, increasing multi-threading, other improvements, while perhaps not as substantial, were also included. These include reducing or eliminating locks and context switches and other incremental improvements. The improvements in iSCSI performance seen from this refactoring of iSCSI software target are substantial with the largest improvements seen on larger controllers that have more cores that can be used to concurrently process iSCSI threads.
Compared to 9.2: 2.6x Performance increase @1ms
TR:4080
Friday 7 December 2018
Friday 19 October 2018
SnapCenter 4.1 & Enterprise Level ORACLE 12 c and ONTAP 9.4 [ Practical Information ]
SnapCenter Latest version : 4.1 (Supports ONTAP 9.4)
Oracle Host : [192.168.0.26]
centos-release-7-5.1804.4.el7.centos.x86_64
NetApp SnapCenter Server: [192.168.0.10 : Port 8146]
Windows 2012R2
Plug-in/Agent:
snapcenter-plugin-oracle & netapp-snapcenter-plugin-unix
How to install the plug-in:
You can use either SnapCenter Server GUI to push the plug-in or use the manual local installation on the Linux host (I prefer this method). I always had issues when pushing any sort of agent from Windows to Linux, and I found there is no need to break your head and waste your time, just go for local installation, it always works.
Please note : For Oracle environment hosted on NFS (Like in this demo) or an in-guest iSCSI initiator, you don't need to install 'Plug-in for VMware vSphere', all you need is just :netapp-snapcenter-plugin-oracle & netapp-snapcenter-plugin-unix, they get installed together.
For local installation of plug-in, follow these steps:
1. Go to this location on the SnapCenter server: [Windows]
C:\ProgramData\NetApp\SnapCenter\Package Repository
2. Copy this file to the Linux host.
'snapcenter_linux_host_plugin.bin'
3. Once it is available on the Linux host, simply run this command:
./SnapCenter_linux_host_plugin.bin –i swing
(Make sure you make the file 'executable' after copying it over to Linux, you can use chmod 755)
This will bring up the installation wizard as shown below.
Oracle Host : [192.168.0.26]
centos-release-7-5.1804.4.el7.centos.x86_64
Oracle DB: [Standalone, Enterprise Edition Release 12.2.0.1.0]
Mounted on NetApp FlexVol via NFS - mount point on linux: /mnt/ontap-nfs
NetApp SnapCenter Server: [192.168.0.10 : Port 8146]
Windows 2012R2
Plug-in/Agent:
snapcenter-plugin-oracle & netapp-snapcenter-plugin-unix
How to install the plug-in:
You can use either SnapCenter Server GUI to push the plug-in or use the manual local installation on the Linux host (I prefer this method). I always had issues when pushing any sort of agent from Windows to Linux, and I found there is no need to break your head and waste your time, just go for local installation, it always works.
Please note : For Oracle environment hosted on NFS (Like in this demo) or an in-guest iSCSI initiator, you don't need to install 'Plug-in for VMware vSphere', all you need is just :netapp-snapcenter-plugin-oracle & netapp-snapcenter-plugin-unix, they get installed together.
For local installation of plug-in, follow these steps:
1. Go to this location on the SnapCenter server: [Windows]
C:\ProgramData\NetApp\SnapCenter\Package Repository
2. Copy this file to the Linux host.
'snapcenter_linux_host_plugin.bin'
3. Once it is available on the Linux host, simply run this command:
./SnapCenter_linux_host_plugin.bin –i swing
(Make sure you make the file 'executable' after copying it over to Linux, you can use chmod 755)
This will bring up the installation wizard as shown below.
Note: I always find this interesting on 'linux' based system (Redhat/Centos): Whenever NetApp products such as SnapDrive for UNIX or SnapCenter complains that the 'Host OS' is not supported, all you have to do is just fool the Software by changing the redhat-release text in this file [/etc/redhat-release] to whatever is supported by their Software. When I installed SnapDrive it complained that the OS is not supported (as it was the latest Centos 7 build), so I altered the text in the /etc/redhat-release file to redhat 5.5, and it worked. When I tried to install the plug-in on Linux Host for Oracle DB, it again said not supported (due to older version) so I changed it back to the original latest version, and it worked again. Well, you have this liberty to do these tricks on a 'test' setup, but on a Production environment you would rather make sure its compatible, and when in doubt, always reach out to NetApp Support.
How does SnapCenter server interacts with 'Oracle' DB running on linux host?
Interaction happens via two components:
1) SnapCenter Server [Windows] = SMCore service
2) SnapCenter Plug-in loader [Linux ] = SPL (SnapCenter plug-in loader)
SMCore (Port 8145) = coordinates with the Linux Plug-In (managed by the SnapCenter plug-in loader (SPL)) to perform Oracle database workflows.
SPL = Runs on the Oracle database Host that loads and runs the Oracle plug-in. SMCore coordinates with SPL to perform Oracle data protection workflows like quiesce/unquiesce, backup, RMAN catalog, mount, restore, and clone.
You can verify the status of SPL on linux:
[oracle@redhat ~]$ service spl status
Redirecting to /bin/systemctl status spl.service
● spl.service - SnapCenter Plugin Loader
Loaded: loaded (/etc/systemd/system/spl.service; enabled; vendor preset: disabled)
Active: active (running) since Tue 2018-10-23 11:08:14 BST; 2h 18min ago
Process: 754 ExecStart=/opt/NetApp/snapcenter/spl/bin/spld start (code=exited, status=0/SUCCESS)
As part of the Oracle Backup requirement:
Put the oracle DB in archive Mode [Otherwise Backup will fail]
SQL> connect as sysdba
Enter user-name: sysdba
Enter password:
Connected.
SQL>
SQL> shutdown immediate
Database closed.
Database dismounted.
ORACLE instance shut down.
SQL> startup mount exclusive
ORACLE instance started.
Total System Global Area 1996488704 bytes
Fixed Size 8622336 bytes
Variable Size 587206400 bytes
Database Buffers 1392508928 bytes
Redo Buffers 8151040 bytes
Database mounted.
SQL> alter database archivelog;
Database altered.
SQL> alter database open;
Database altered.
SQL> archive log start;
Statement processed.
SQL> archive log list;
Database log mode Archive Mode
Automatic archival Enabled
Archive destination /mnt/ontap-nfs/u01/app/oracle/product/12.2.0/dbhome_1/dbs/arch
Oldest online log sequence 3
Next log sequence to archive 5
Current log sequence 5
SQL> exit
Next, make a connection to the SnapCenter Server and discover the local Oracle Database:
[oracle@redhat bin]$ ./sccli Open-SmConnection
INFO: A connection session will be opened with SnapCenter 'https://TEST.LAB.COM:8146/'.
Enter the SnapCenter user name: lab\administrator
Enter the SnapCenter password:
INFO: A connection session with the SnapCenter was established successfully.
[oracle@redhat bin]$ ./sccli Get-SmResources
INFO: Using localhost 'redhat.lab.com' as default host for discovering resources.
===============================================================
| Name | Version | Id | Type | Overall Status |
==============================================================
| orcl | 12.2.0.1.0 | redhat.lab.com\orcl | Oracle Single Instance | |
===============================================================
INFO: The command 'Get-SmResources' executed successfully.
[oracle@redhat bin]$./sccli Configure-SmOracleDatabase -AppObjectId <appObject Id> -DatabaseRunAsName <user>
Once this is done, go back to the SnapCenter Server GUI [Windows] : You will see the plugins installed successfully.
Just follow 'Backup' workflow instructions to create a Protection Policy and configure other parameters such as snapshot naming, scheduling, etc etc, just do what it says. There is plenty of help inside the tool to help you figure out. Finally, click 'Back now'.
If the job is successful, you can view the 'Oracle' snapshots on the NetApp SVM:
Open SystemManager GUI: Navigate to SVM, Volume and then view snapshots, and you should see two snapshots there :
1. Data
2. Log
There are more options on the panel which I haven't tested yet, will give it a try in coming days.
That's it for now!! For more info read: 4.1 Documentation
Wednesday 17 October 2018
SnapDrive for UNIX Overview 5.3.1P1: [Quick practical overview]
SnapDrive for UNIX Overview:
SnapDrive for UNIX is an enterprise-class storage and data management utility that simplifies storage management and increases the availability and reliability of application data. Its key functionality includes error-free application storage provisioning, consistent-data NetApp Snapshot copies, and rapid application recovery. It also provides the ability to easily manage data that resides on NetApp Network File System (NFS) shares or NetApp LUNs. SnapDrive for UNIX complements the native file system and volume manager and integrates seamlessly with the clustering technology supported by the host operating system (OS).
My lab details:
[root@redhat oracle]# snapdrived status
Snapdrive Daemon Version : 5.3.1P1 (Change 4326080 Built 'Wed May 24 23:40:39 PDT 2017')
Snapdrive Daemon start time : Tue Oct 16 22:40:38 2018
Total Commands Executed : 4
Host:
Linux redhat.lab.com 3.10.0-862.14.4.el7.x86_64
Config file: /opt/NetApp/snapdrive/snapdrive.conf
#/opt/NetApp/snapdrive/bin/sdconfcheck check
Please note: For cDOT, we have to add the 'VSERVER/SVM' IP, do not use Cluster_Mgmt IP. In the following example: I am using dedicated SVM mgmt IP, and 'vsadmin' account. SVM DNS name could point to multiple IP addresses, hence hard-code the entry in the host file on the Unix/Linux host which points to SVM Mgmt IP.
Following command adds the SVM to the SnapDrive config for Data Management/Provision purpose:
1) [root@redhat ~]# snapdrive config set vsadmin 192.168.0.214
Password for vsadmin:
Retype password:
Mismatch between DNS entry of storage system svm_nfs_iscsi and system name SVM_NFS
Do you want to continue? (y/n) n
Abort the config by pressing 'n'.
2) Add the hostname to the /etc/hosts file.
[root@redhat ~]# cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.0.214 SVM_NFS
3) Ping it once to confirm its working.
[root@redhat ~]# ping svm_nfs
PING SVM_NFS (192.168.0.214) 56(84) bytes of data.
64 bytes from SVM_NFS (192.168.0.214): icmp_seq=1 ttl=64 time=0.430 ms
64 bytes from SVM_NFS (192.168.0.214): icmp_seq=2 ttl=64 time=0.299 ms
4) Now, add the the SVM using the HOSTNAME (Do not use Cluster_mgmt IP):
[root@redhat ~]# snapdrive config set vsadmin SVM_NFS
Password for vsadmin:
Retype password:
[root@redhat ~]# snapdrive config list
username appliance name appliance type
----------------------------------------------
vsadmin SVM_NFS StorageSystem
[root@redhat ~]#
SnapDrive has the 'Wizard' to walk you through creating a LUN/LVM since it's inception, which means you don't have to remember any commands, it will do everything for you. This is how it's run:
5) Create a LUN using Wizard:
[root@redhat ~]# snapdrive storage wizard create
What kind of storage do you want to create {LUN, diskgroup, hostvol, filesys, ?}?[LUN]:
Getting storage systems configured in the host ...
Following are the available storage systems:
SVM_NFS
You can select the storage system name from the list above or enter a
new storage system name to configure it.
Enter the storage system name: SVM_NFS
Enter the storage volume name or press <enter> to list them:
Following is a list of volumes in the storage system:
SMSQL_VOL_DATA SMSQL_VOL_LOGS SVM_NFS_root_LS1 testing_LUN_MOVE
vol_09082018_114009_8vol_CIFS vol_CIFS_SnapMirror_13092018_001828vol_NAS vol_NAS_0
vol_NFS vol_SnapDrive vol_redhat
Please re-enter: vol_SnapDrive
You can provide comma separated multiple entity names e.g: lun1,lun2,
lun3 etc.
Enter the LUN name(s): lun_SD
Checking LUN name(s) availability. Please wait ...
Enter the LUN size for LUN(s) in the below mentioned format. (Default
unit is MB)
<size>k:m:g:t
Where, k: Kilo Bytes m: Mega bytes g: Giga Bytes t: Tera Bytes
Enter the LUN size: 2g
Configuration Summary:
Storage System : SVM_NFS
Volume Name : /vol/vol_SnapDrive
LUN Name : lun_SD
LUN Size : 2048.0 MB
Equivalent CLI command is:
snapdrive storage create -lun SVM_NFS:/vol/vol_SnapDrive/lun_SD -lunsize 2048.0m
Do you want to create storage based on this configuration{y, n}[y]?: y
Creating storage with the provided configuration. Please wait...
LUN SVM_NFS:/vol/vol_SnapDrive/lun_SD to device file mapping => /dev/sdd, /dev/sde
Do you want to create more storage {y, n}[y]?: n
[root@redhat ~]#
To confirm if the SnapDrive provisioned LUN is actually created, verify it with 'multipath -ll' command.
[root@redhat ~]# multipath -ll
3600a09807770457a795d4d4179475475 dm-3 NETAPP ,LUN C-Mode
size=2.0G features='4 queue_if_no_path pg_init_retries 50 retain_attached_hw_handle' hwhandler='1 alua' wp=rw
`-+- policy='service-time 0' prio=50 status=active
|- 3:0:0:1 sdd 8:48 active ready running
`- 4:0:0:1 sde 8:64 active ready running
You can run the following command to check the LUN path & other attributes such as 'ALUA':
[root@redhat ~]# snapdrive lun showpaths
Connected LUNs:
lun path device filename asymmetric access state
-------- --------------- -----------------------
SVM_NFS:/vol/vol_SnapDrive/lun_SD /dev/sdd Optimized
SVM_NFS:/vol/vol_SnapDrive/lun_SD /dev/sde Optimized
[root@redhat ~]#
SnapDrive for UNIX is an enterprise-class storage and data management utility that simplifies storage management and increases the availability and reliability of application data. Its key functionality includes error-free application storage provisioning, consistent-data NetApp Snapshot copies, and rapid application recovery. It also provides the ability to easily manage data that resides on NetApp Network File System (NFS) shares or NetApp LUNs. SnapDrive for UNIX complements the native file system and volume manager and integrates seamlessly with the clustering technology supported by the host operating system (OS).
My lab details:
[root@redhat oracle]# snapdrived status
Snapdrive Daemon Version : 5.3.1P1 (Change 4326080 Built 'Wed May 24 23:40:39 PDT 2017')
Snapdrive Daemon start time : Tue Oct 16 22:40:38 2018
Total Commands Executed : 4
Host:
Linux redhat.lab.com 3.10.0-862.14.4.el7.x86_64
- SnapDrive uses default parameters unless the values in the following config file are un-commented:
Config file: /opt/NetApp/snapdrive/snapdrive.conf
- You can also run sdconfcheck to determine if underlying OS/filesystem drivers are compatible.
#/opt/NetApp/snapdrive/bin/sdconfcheck check
- Once SnapDrive tool is installed (simple : rpm -ivh snadprive_package.rpm), the next step is to add the NetApp storage system [In this example, we are adding cDOT]
Please note: For cDOT, we have to add the 'VSERVER/SVM' IP, do not use Cluster_Mgmt IP. In the following example: I am using dedicated SVM mgmt IP, and 'vsadmin' account. SVM DNS name could point to multiple IP addresses, hence hard-code the entry in the host file on the Unix/Linux host which points to SVM Mgmt IP.
Following command adds the SVM to the SnapDrive config for Data Management/Provision purpose:
1) [root@redhat ~]# snapdrive config set vsadmin 192.168.0.214
Password for vsadmin:
Retype password:
Mismatch between DNS entry of storage system svm_nfs_iscsi and system name SVM_NFS
Do you want to continue? (y/n) n
Abort the config by pressing 'n'.
2) Add the hostname to the /etc/hosts file.
[root@redhat ~]# cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.0.214 SVM_NFS
3) Ping it once to confirm its working.
[root@redhat ~]# ping svm_nfs
PING SVM_NFS (192.168.0.214) 56(84) bytes of data.
64 bytes from SVM_NFS (192.168.0.214): icmp_seq=1 ttl=64 time=0.430 ms
64 bytes from SVM_NFS (192.168.0.214): icmp_seq=2 ttl=64 time=0.299 ms
4) Now, add the the SVM using the HOSTNAME (Do not use Cluster_mgmt IP):
[root@redhat ~]# snapdrive config set vsadmin SVM_NFS
Password for vsadmin:
Retype password:
[root@redhat ~]# snapdrive config list
username appliance name appliance type
----------------------------------------------
vsadmin SVM_NFS StorageSystem
[root@redhat ~]#
SnapDrive has the 'Wizard' to walk you through creating a LUN/LVM since it's inception, which means you don't have to remember any commands, it will do everything for you. This is how it's run:
5) Create a LUN using Wizard:
[root@redhat ~]# snapdrive storage wizard create
What kind of storage do you want to create {LUN, diskgroup, hostvol, filesys, ?}?[LUN]:
Getting storage systems configured in the host ...
Following are the available storage systems:
SVM_NFS
You can select the storage system name from the list above or enter a
new storage system name to configure it.
Enter the storage system name: SVM_NFS
Enter the storage volume name or press <enter> to list them:
Following is a list of volumes in the storage system:
SMSQL_VOL_DATA SMSQL_VOL_LOGS SVM_NFS_root_LS1 testing_LUN_MOVE
vol_09082018_114009_8vol_CIFS vol_CIFS_SnapMirror_13092018_001828vol_NAS vol_NAS_0
vol_NFS vol_SnapDrive vol_redhat
Please re-enter: vol_SnapDrive
You can provide comma separated multiple entity names e.g: lun1,lun2,
lun3 etc.
Enter the LUN name(s): lun_SD
Checking LUN name(s) availability. Please wait ...
Enter the LUN size for LUN(s) in the below mentioned format. (Default
unit is MB)
<size>k:m:g:t
Where, k: Kilo Bytes m: Mega bytes g: Giga Bytes t: Tera Bytes
Enter the LUN size: 2g
Configuration Summary:
Storage System : SVM_NFS
Volume Name : /vol/vol_SnapDrive
LUN Name : lun_SD
LUN Size : 2048.0 MB
Equivalent CLI command is:
snapdrive storage create -lun SVM_NFS:/vol/vol_SnapDrive/lun_SD -lunsize 2048.0m
Do you want to create storage based on this configuration{y, n}[y]?: y
Creating storage with the provided configuration. Please wait...
LUN SVM_NFS:/vol/vol_SnapDrive/lun_SD to device file mapping => /dev/sdd, /dev/sde
Do you want to create more storage {y, n}[y]?: n
[root@redhat ~]#
To confirm if the SnapDrive provisioned LUN is actually created, verify it with 'multipath -ll' command.
[root@redhat ~]# multipath -ll
3600a09807770457a795d4d4179475475 dm-3 NETAPP ,LUN C-Mode
size=2.0G features='4 queue_if_no_path pg_init_retries 50 retain_attached_hw_handle' hwhandler='1 alua' wp=rw
`-+- policy='service-time 0' prio=50 status=active
|- 3:0:0:1 sdd 8:48 active ready running
`- 4:0:0:1 sde 8:64 active ready running
You can run the following command to check the LUN path & other attributes such as 'ALUA':
[root@redhat ~]# snapdrive lun showpaths
Connected LUNs:
lun path device filename asymmetric access state
-------- --------------- -----------------------
SVM_NFS:/vol/vol_SnapDrive/lun_SD /dev/sdd Optimized
SVM_NFS:/vol/vol_SnapDrive/lun_SD /dev/sde Optimized
[root@redhat ~]#
Monday 15 October 2018
Difference between physical WWPNs (7-mode) vs virtual WWPNs (cDOT or ONTAP)?
Physical WWPNs = Is a concept of 7-mode
Virtual* WWPNs = Is a concept of cDOT/ONTAP
*= By virtual it means 'Logical InterFace' or simply LIF, a term that is only applicable to cDOT. One of the major difference between 7-mode & cDOT is taking virtualization to the END Ports such as Physical Adapters be it a NIC or HBA. In 7-mode IP was tied to Physical NIC, and WWPN was tied to the Physical FC HBAs. However, in cDOT it's virtualized by introducing the concept of 'LIF', an abstraction on top of the physical (physical ports). In 7-mode we have VIF, which is also called Virtual Interface, but that is where the similarity ends. The objective of VIF was to support 'Link Aggregation' that's it, it was never truly virtualized the way LIF is in cDOT.
LIF : Is logical and can have either IP or WWPN.
When you open system manager GUI, under section Network : If you click on 'Network Interface' you can note the Logical WWPN and if you click on 'FC/FCoE' you will see Physical WWPN.
Zoning : Do not include 'Physical WWPN', only 'Virtual WWPN'
DO NOT use Physical WWPNs beginning with '50:0a:09:8x' they no longer present a SCSI target service and should not be included in any zone configurations on the FC fabric (though they show as logged in to the fabric).
DO use only virtual WWPNs (WWPNs starting with 20:).
IMPORTANT: NPIV is required for Fibre Channel LIFs to operate correctly. ONTAP uses N_Port ID virtualization (NPIV) to permit every logical interface to log in to an FC fabric with its own worldwide port name (WWPN). What NPIV does is allow a single physical N_Port to have multiple WWPNs, and therefore multiple N_Port_IDs, associated with it.
What is the advantage of Virtual/Logical WWPN: This allows a host (Physical or virtual) connected to the same FC fabric to communicate with the same 'SCSI target=LUN' regardless of which physical node owns which LIF.
Ensure NPIV is enabled on the Switch:
CISCO:
# show npiv status
NPIV is enabled
Brocade:
admin> portcfgshow
NPIV capability
TR-4080 has more details.
Virtual* WWPNs = Is a concept of cDOT/ONTAP
*= By virtual it means 'Logical InterFace' or simply LIF, a term that is only applicable to cDOT. One of the major difference between 7-mode & cDOT is taking virtualization to the END Ports such as Physical Adapters be it a NIC or HBA. In 7-mode IP was tied to Physical NIC, and WWPN was tied to the Physical FC HBAs. However, in cDOT it's virtualized by introducing the concept of 'LIF', an abstraction on top of the physical (physical ports). In 7-mode we have VIF, which is also called Virtual Interface, but that is where the similarity ends. The objective of VIF was to support 'Link Aggregation' that's it, it was never truly virtualized the way LIF is in cDOT.
LIF : Is logical and can have either IP or WWPN.
When you open system manager GUI, under section Network : If you click on 'Network Interface' you can note the Logical WWPN and if you click on 'FC/FCoE' you will see Physical WWPN.
Zoning : Do not include 'Physical WWPN', only 'Virtual WWPN'
DO NOT use Physical WWPNs beginning with '50:0a:09:8x' they no longer present a SCSI target service and should not be included in any zone configurations on the FC fabric (though they show as logged in to the fabric).
DO use only virtual WWPNs (WWPNs starting with 20:).
IMPORTANT: NPIV is required for Fibre Channel LIFs to operate correctly. ONTAP uses N_Port ID virtualization (NPIV) to permit every logical interface to log in to an FC fabric with its own worldwide port name (WWPN). What NPIV does is allow a single physical N_Port to have multiple WWPNs, and therefore multiple N_Port_IDs, associated with it.
What is the advantage of Virtual/Logical WWPN: This allows a host (Physical or virtual) connected to the same FC fabric to communicate with the same 'SCSI target=LUN' regardless of which physical node owns which LIF.
Ensure NPIV is enabled on the Switch:
CISCO:
# show npiv status
NPIV is enabled
Brocade:
admin> portcfgshow
NPIV capability
TR-4080 has more details.
Saturday 13 October 2018
ONTAP HA pairs take-over and give-back time estimation in seconds
These are estimation for reference purpose only:
Migrating LUNs using NetApp 7MTT and where does FLI (Foriegn LUN Import) comes handy.
Take-away: 7MTT is a tool for Data Migration of NAS & SAN (LUNs) between NetApp Storage Arrays only. It is designed by NetApp engineering to help professional services and customers in migrating their data from 7-mode hardware to cDOT (Now called ONTAP). This tool is necessary b'cos NetApp is only pulling resources for cDOT, 7-mode core-ontap code production is stopped, 8.2.x, was the the end of it. Data ONTAP 8.3 and later do not include a 7-Mode version.
Also, it makes sense for any existing 7-mode NetApp customers to move to cDOT, as that is where all the R&D efforts lies.
Can 7MTT migrate LUN?
Yes, it can.
When it cannot?
If the LUN is in 32-bit aggregate. Pre-check in 7MTT tool will not let you go ahead if it finds LUNs hosted on 32-bit aggr, following error will be seen:
What is the remedy?
Convert 32-bit to 64-bit aggregates prior to transitioning to clustered Data ONTAP using 7MTT.
What if my LUN is mis-aligned in 7-mode?
7MTT cannot correct the mis-alignment, it will inherit it.
Which tool can I use to fix the mis-alignment, instead of copying data into newly aligned LUN?
FLI (Foreign LUN Import)
What's cool about FLI?
As the name suggest 'Foreign' LUN Import : It not only PULLs LUNs from 3rd-Party Storage Array, but also between the 7-mode & cDOT (starting Data ONTAP 8.3.1) NetApp Arrays, and can also do the LUN alignment plus auto-transition from a 32-bit to 64-bit aggregate, without having to do it first as in the case of 7MTT. Further, with ONTAP 9.1, even AFF has FLI support enabled by-default.
What protocol FLI supports?
Strictly FC only. It can do iSCSI LUNs as well, but for that the LUN protocol must be changed to iSCSI to FC and then you should be able to do it.
DEMO: Unfortunately, I only have iSCSI in my LAB, hence I cannot do the demo, but if you do have access to FC setup in your lab, then please give it a try.
For more info on :Foreign LUN Import 7-Mode to Clustered Data ONTAP Transition Workflow, Please read this TR-4380, it has all the details.
Also, it makes sense for any existing 7-mode NetApp customers to move to cDOT, as that is where all the R&D efforts lies.
Can 7MTT migrate LUN?
Yes, it can.
When it cannot?
If the LUN is in 32-bit aggregate. Pre-check in 7MTT tool will not let you go ahead if it finds LUNs hosted on 32-bit aggr, following error will be seen:
What is the remedy?
Convert 32-bit to 64-bit aggregates prior to transitioning to clustered Data ONTAP using 7MTT.
What if my LUN is mis-aligned in 7-mode?
7MTT cannot correct the mis-alignment, it will inherit it.
Which tool can I use to fix the mis-alignment, instead of copying data into newly aligned LUN?
FLI (Foreign LUN Import)
What's cool about FLI?
As the name suggest 'Foreign' LUN Import : It not only PULLs LUNs from 3rd-Party Storage Array, but also between the 7-mode & cDOT (starting Data ONTAP 8.3.1) NetApp Arrays, and can also do the LUN alignment plus auto-transition from a 32-bit to 64-bit aggregate, without having to do it first as in the case of 7MTT. Further, with ONTAP 9.1, even AFF has FLI support enabled by-default.
What protocol FLI supports?
Strictly FC only. It can do iSCSI LUNs as well, but for that the LUN protocol must be changed to iSCSI to FC and then you should be able to do it.
DEMO: Unfortunately, I only have iSCSI in my LAB, hence I cannot do the demo, but if you do have access to FC setup in your lab, then please give it a try.
For more info on :Foreign LUN Import 7-Mode to Clustered Data ONTAP Transition Workflow, Please read this TR-4380, it has all the details.
Thursday 11 October 2018
Interesting discovery : SMSQL migration from 7-mode to ONTAP 9.4 (cDOT) using 7MTT
As SQL is an application database sitting on top of the volume/LUN and has it's own VSS writer with in the Microsoft VSS framework, hence its important that it's migrated using an application aware migration tool such as - NetApp SMSQL.
I wrote this post last month and showed the steps required for migrating SQL data from 7-mode to cDOT, which works perfectly fine. However I was intrigued to find out what if I use 7MTT tool to achieve the same result ?
Findings : Used 7MTT and it worked perfectly fine, still amused though. So, this is what I am going to elaborate here tonight.
What made it interesting:
1. I created 'test' db using SQL Management studio and created one table and inserted one row. This DB is initially on the local disk.
2. I created two volumes on the 7-mode filer : One for data and one for logs + snap-info.
3. I launched SnapDrive, added the 7-mode filer and created 3 virtual disk : Data, logs & snap-info. Data on a separate volume and logs & snap-info on the same volume but having separate LUNs.
4. I launched SMSQL console and moved the 'test' DB to 7-mode hosted LUNs : Primary data to Data drive, Logs to Logs drive and snap-info to separate drive.
5. I ran a first FULL backup with VERIFICATION, all came up good.
Here comes the 7MTT Tool:
6. Opened services.msc and stopped : SnapDrive & SnapManager services.
7. Launched 7MTT 3.3.1 tool and selected : Data & Logs volume for migration to cDOT SVM.
8. Migration (Final completion) happened successfully and it turned source Volume offline as I had chosen this option during final completion.
9. Opened system-manager GUI and launched cDOT filer and made sure the two Volumes now showed up, along with 3 LUNs. Please note 7MTT tool not only migrates the LUNs, it also maps them automatically (If chosen to be), I had checked by mistake, but it's ok. We just need to make sure we dis-associate them from the igroup.
Now, I must re-connect the same drives (3 LUNS that are migrated from 7-m to cDOT):
10. I went to cDOT system-manager and went to those 3 LUNs one by one and un-checked the mapping from the igroup.
11. I started the SnapDrive service and launched the SnapDrive management tool - I re-connected the 3 drives using the same 'Drive Letters' as it was with 7-m. However, the disk-signature had changed now, but will it matter ?
Finally:
12. I launched the SMSQL tool, and noticed it says 'test' DB is recovering..its in a hung state.
13. I was expecting this, as it was not a true application level migration and SMSQL must be wondering if these are the same disks ?
14. I didn't bother much to troubleshoot 'recovering...' error, as I knew I had taken a 'good back' post migration to 7-mode filer.
15. I used the restore wizard within SMSQL tool and restored the 'test' db from the last known good back. To my surprise, after the successful restore, when I clicked on the 'Backup' icon it didn't complain this time and it showed the drives correctly as it was with 7-m, except disk signatures.
16. Ran another full back and it worked perfectly. So, basically I have the working SQL 'test' DB hosted on my cDOT (Migrated usign 7MTT) plus the snapshots that were shipped as part of the snamirror replication process, which is the default mechanism for 'copy' based backup within 7MTT.
Tested:
17. I opened SQL Mgmt studio, selected 'test' DB and ran a query on table and it showed the row that I had populated in that table. Hmm...This is very interesting now, b'cos I was expecting the 'restore' to fail, as you remember this backup was made on the drives hosted on the 7-m LUNs, looks like 'disk signature' did not really matter to it. SQL database, snapdrive luns and snapshots are all working nicely.
18. Don't know if anyone has attempted this method on 'test' environment ?
I will be doing few more tests on the around migration concept, and will feed you back on the findings.
I wrote this post last month and showed the steps required for migrating SQL data from 7-mode to cDOT, which works perfectly fine. However I was intrigued to find out what if I use 7MTT tool to achieve the same result ?
Findings : Used 7MTT and it worked perfectly fine, still amused though. So, this is what I am going to elaborate here tonight.
What made it interesting:
1. I created 'test' db using SQL Management studio and created one table and inserted one row. This DB is initially on the local disk.
2. I created two volumes on the 7-mode filer : One for data and one for logs + snap-info.
3. I launched SnapDrive, added the 7-mode filer and created 3 virtual disk : Data, logs & snap-info. Data on a separate volume and logs & snap-info on the same volume but having separate LUNs.
4. I launched SMSQL console and moved the 'test' DB to 7-mode hosted LUNs : Primary data to Data drive, Logs to Logs drive and snap-info to separate drive.
5. I ran a first FULL backup with VERIFICATION, all came up good.
Here comes the 7MTT Tool:
6. Opened services.msc and stopped : SnapDrive & SnapManager services.
7. Launched 7MTT 3.3.1 tool and selected : Data & Logs volume for migration to cDOT SVM.
8. Migration (Final completion) happened successfully and it turned source Volume offline as I had chosen this option during final completion.
9. Opened system-manager GUI and launched cDOT filer and made sure the two Volumes now showed up, along with 3 LUNs. Please note 7MTT tool not only migrates the LUNs, it also maps them automatically (If chosen to be), I had checked by mistake, but it's ok. We just need to make sure we dis-associate them from the igroup.
Now, I must re-connect the same drives (3 LUNS that are migrated from 7-m to cDOT):
10. I went to cDOT system-manager and went to those 3 LUNs one by one and un-checked the mapping from the igroup.
11. I started the SnapDrive service and launched the SnapDrive management tool - I re-connected the 3 drives using the same 'Drive Letters' as it was with 7-m. However, the disk-signature had changed now, but will it matter ?
Finally:
12. I launched the SMSQL tool, and noticed it says 'test' DB is recovering..its in a hung state.
13. I was expecting this, as it was not a true application level migration and SMSQL must be wondering if these are the same disks ?
14. I didn't bother much to troubleshoot 'recovering...' error, as I knew I had taken a 'good back' post migration to 7-mode filer.
15. I used the restore wizard within SMSQL tool and restored the 'test' db from the last known good back. To my surprise, after the successful restore, when I clicked on the 'Backup' icon it didn't complain this time and it showed the drives correctly as it was with 7-m, except disk signatures.
16. Ran another full back and it worked perfectly. So, basically I have the working SQL 'test' DB hosted on my cDOT (Migrated usign 7MTT) plus the snapshots that were shipped as part of the snamirror replication process, which is the default mechanism for 'copy' based backup within 7MTT.
Tested:
17. I opened SQL Mgmt studio, selected 'test' DB and ran a query on table and it showed the row that I had populated in that table. Hmm...This is very interesting now, b'cos I was expecting the 'restore' to fail, as you remember this backup was made on the drives hosted on the 7-m LUNs, looks like 'disk signature' did not really matter to it. SQL database, snapdrive luns and snapshots are all working nicely.
18. Don't know if anyone has attempted this method on 'test' environment ?
I will be doing few more tests on the around migration concept, and will feed you back on the findings.
Sunday 7 October 2018
Unable to verify the graphical display setup. This application requires X display.
Issue : While installing Oracle DB 18c, this error continued to frustrate me, asking for x11 display, tried different suggestions from google but finally one suggestion worked.
Error is seen when you try to launch oracle installation..
[oracle@redhat stage]$ ./runInstaller
ERROR: Unable to verify the graphical display setup. This application requires X display. Make sure that xdpyinfo exist under PATH variable. No protocol specified. Can't connect to X11 window server using ':0.0' as the value of the DISPLAY variable.
Solution that worked in this case:
[root@redhat stage]# yum install xorg-x11-xauth xterm
Package 1:xorg-x11-xauth-1.0.9-1.el7.x86_64 already installed and latest version
Resolving Dependencies
--> Running transaction check
---> Package xterm.x86_64 0:295-3.el7 will be installed
--> Processing Dependency: libXaw.so.7()(64bit) for package: xterm-295-3.el7.x86_64
--> Running transaction check
---> Package libXaw.x86_64 0:1.0.13-4.el7 will be installed
--> Finished Dependency Resolution
Dependencies Resolved
============================================================================================================================================
Package Arch Version Repository Size
============================================================================================================================================
Installing:
xterm x86_64 295-3.el7 base 455 k
Installing for dependencies:
libXaw x86_64 1.0.13-4.el7 base 192 k
Transaction Summary
============================================================================================================================================
Install 1 Package (+1 Dependent package)
Total download size: 647 k
Installed size: 1.7 M
Is this ok [y/d/N]: y
Downloading packages:
(1/2): libXaw-1.0.13-4.el7.x86_64.rpm | 192 kB 00:00:00
(2/2): xterm-295-3.el7.x86_64.rpm | 455 kB 00:00:00
--------------------------------------------------------------------------------------------------------------------------------------------
Total 1.1 MB/s | 647 kB 00:00:00
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : libXaw-1.0.13-4.el7.x86_64 1/2
Installing : xterm-295-3.el7.x86_64 2/2
Verifying : libXaw-1.0.13-4.el7.x86_64 1/2
Verifying : xterm-295-3.el7.x86_64 2/2
Installed:
xterm.x86_64 0:295-3.el7
Dependency Installed:
libXaw.x86_64 0:1.0.13-4.el7
[root@redhat stage]# ssh -Y oracle@redhat.lab.com
Back to non-root user: Run the installer again..
[oracle@redhat stage]$ ./runInstaller
This time it worked, Installation poped-up.
Error is seen when you try to launch oracle installation..
[oracle@redhat stage]$ ./runInstaller
ERROR: Unable to verify the graphical display setup. This application requires X display. Make sure that xdpyinfo exist under PATH variable. No protocol specified. Can't connect to X11 window server using ':0.0' as the value of the DISPLAY variable.
Solution that worked in this case:
[root@redhat stage]# yum install xorg-x11-xauth xterm
Package 1:xorg-x11-xauth-1.0.9-1.el7.x86_64 already installed and latest version
Resolving Dependencies
--> Running transaction check
---> Package xterm.x86_64 0:295-3.el7 will be installed
--> Processing Dependency: libXaw.so.7()(64bit) for package: xterm-295-3.el7.x86_64
--> Running transaction check
---> Package libXaw.x86_64 0:1.0.13-4.el7 will be installed
--> Finished Dependency Resolution
Dependencies Resolved
============================================================================================================================================
Package Arch Version Repository Size
============================================================================================================================================
Installing:
xterm x86_64 295-3.el7 base 455 k
Installing for dependencies:
libXaw x86_64 1.0.13-4.el7 base 192 k
Transaction Summary
============================================================================================================================================
Install 1 Package (+1 Dependent package)
Total download size: 647 k
Installed size: 1.7 M
Is this ok [y/d/N]: y
Downloading packages:
(1/2): libXaw-1.0.13-4.el7.x86_64.rpm | 192 kB 00:00:00
(2/2): xterm-295-3.el7.x86_64.rpm | 455 kB 00:00:00
--------------------------------------------------------------------------------------------------------------------------------------------
Total 1.1 MB/s | 647 kB 00:00:00
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : libXaw-1.0.13-4.el7.x86_64 1/2
Installing : xterm-295-3.el7.x86_64 2/2
Verifying : libXaw-1.0.13-4.el7.x86_64 1/2
Verifying : xterm-295-3.el7.x86_64 2/2
Installed:
xterm.x86_64 0:295-3.el7
Dependency Installed:
libXaw.x86_64 0:1.0.13-4.el7
[root@redhat stage]# ssh -Y oracle@redhat.lab.com
Back to non-root user: Run the installer again..
[oracle@redhat stage]$ ./runInstaller
This time it worked, Installation poped-up.
After extending LUN size from NetApp side, fdisk shows new size but dm-multipath shows old size
Observation:
This is probably already known to most Linux & NetApp administrators, but I discovered so, and hence I am making a note of it in this article.
Steps:
1. Increased LUN size on NetApp side.
2. Ran the iscsi --rescan
3. fdisk shows - new size but, 'multipath -ll' still shows old size
4. dm-multipath which sits between kernel and block devices needs a 'bounce' to see the new size.
5. After running 'service multipathd restart' and running 'multipath -ll' showed the new size correctly.
This is probably already known to most Linux & NetApp administrators, but I discovered so, and hence I am making a note of it in this article.
Steps:
1. Increased LUN size on NetApp side.
2. Ran the iscsi --rescan
3. fdisk shows - new size but, 'multipath -ll' still shows old size
4. dm-multipath which sits between kernel and block devices needs a 'bounce' to see the new size.
5. After running 'service multipathd restart' and running 'multipath -ll' showed the new size correctly.
Saturday 6 October 2018
Configuring Multipath for redhat (centos7-3.10.0-862.el7.x86_64) in simple straight forward steps for ISCSI LUN on ONTAP 9
Straight forward steps from installing multi-path module to configuring multipath.conf file as per NetApp ISCSI redhat recommendation:
[root@redhat ~]# yum install device-mapper-multipath [download the multipath module binary]
[root@redhat ~]# modprobe dm-multipath [insert the module]
[root@redhat ~]# lsmod | grep dm_mod [list the module to confirm]
dm_mod 123941 11 dm_multipath,dm_log,dm_mirror
One of the good feature of Linux is the ability to 'load' modules while the kernel is running: Each piece of code that can be added to the kernel at run-time is called a module. Each module is made up of object code that can be dynamically linked to the running kernel by the insmod (modprobe) program and can be unlinked by the rmmod program.
Next...
[root@redhat /]# mpathconf --enable --with_multipathd y
The command above will create the multipath.conf file, if it does not already exists.
Next...
Blacklist - Non-NetApp devices from multi-path probe:
If there are non-NetApp SCSI devices to exclude, enter the worldwide identifier (WWID) for the devices in the blacklist section of the multipath.conf file.
For example, if /dev/sda is the non-NetApp SCSI device that you want to exclude, you would enter the following:
[root@redhat /]# /lib/udev/scsi_id -gud /dev/sda
3600508e000000000753250f933cc4606 [Just an example, in your system it may be different, this you can copy and paste in the multipath.conf file:]
blacklist {
wwid 3600508e000000000753250f933cc4606
devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*"
devnode "^hd[a-z]"
devnode "^cciss.*"
}
Next...
[root@redhat /]# rdloaddriver=scsi_dh_alua
[root@redhat /]# service multipathd restart
[root@redhat /]# service multipathd restart [bounce it]
Redirecting to /bin/systemctl restart multipathd.service
[root@redhat /]# service multipathd status [use status command to ensure it's up correctly]
Redirecting to /bin/systemctl status multipathd.service
● multipathd.service - Device-Mapper Multipath Device Controller
Loaded: loaded (/usr/lib/systemd/system/multipathd.service; enabled; vendor preset: enabled)
Active: active (running) since Mon 2018-10-08 20:19:15 BST; 8s ago
Process: 49811 ExecStart=/sbin/multipathd (code=exited, status=0/SUCCESS)
Process: 49809 ExecStartPre=/sbin/multipath -A (code=exited, status=0/SUCCESS)
Process: 49808 ExecStartPre=/sbin/modprobe dm-multipath (code=exited, status=0/SUCCESS)
Main PID: 49814 (multipathd)
Tasks: 6
CGroup: /system.slice/multipathd.service
└─49814 /sbin/multipathd
[root@redhat ~]# multipath -ll
3600a09807770457a795d4d4179475456 dm-2 NETAPP ,LUN C-Mode
size=2.0G features='4 queue_if_no_path pg_init_retries 50 retain_attached_hw_handle' hwhandler='1 alua' wp=rw
`-+- policy='service-time 0' prio=50 status=active
|- 4:0:0:0 sdb 8:16 active ready running
`- 3:0:0:0 sdc 8:32 active ready running
[root@redhat ~]#
Note: Make sure you have discovered & logged-into ISCSI target already, otherwise you will not see any output on 'multipath -ll' command. 2 Easy commands to do that are :
[root@redhat /]# iscsiadm -m discovery -t st -p 192.168.0.x:3260
[root@redhat /]# iscsiadm -m node -l
All done!!!
[root@redhat ~]# yum install device-mapper-multipath [download the multipath module binary]
[root@redhat ~]# modprobe dm-multipath [insert the module]
[root@redhat ~]# lsmod | grep dm_mod [list the module to confirm]
dm_mod 123941 11 dm_multipath,dm_log,dm_mirror
One of the good feature of Linux is the ability to 'load' modules while the kernel is running: Each piece of code that can be added to the kernel at run-time is called a module. Each module is made up of object code that can be dynamically linked to the running kernel by the insmod (modprobe) program and can be unlinked by the rmmod program.
Next...
[root@redhat /]# mpathconf --enable --with_multipathd y
The command above will create the multipath.conf file, if it does not already exists.
Next...
Blacklist - Non-NetApp devices from multi-path probe:
If there are non-NetApp SCSI devices to exclude, enter the worldwide identifier (WWID) for the devices in the blacklist section of the multipath.conf file.
For example, if /dev/sda is the non-NetApp SCSI device that you want to exclude, you would enter the following:
[root@redhat /]# /lib/udev/scsi_id -gud /dev/sda
3600508e000000000753250f933cc4606 [Just an example, in your system it may be different, this you can copy and paste in the multipath.conf file:]
blacklist {
wwid 3600508e000000000753250f933cc4606
devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*"
devnode "^hd[a-z]"
devnode "^cciss.*"
}
Next...
[root@redhat /]# rdloaddriver=scsi_dh_alua
[root@redhat /]# service multipathd restart
[root@redhat /]# service multipathd restart [bounce it]
Redirecting to /bin/systemctl restart multipathd.service
[root@redhat /]# service multipathd status [use status command to ensure it's up correctly]
Redirecting to /bin/systemctl status multipathd.service
● multipathd.service - Device-Mapper Multipath Device Controller
Loaded: loaded (/usr/lib/systemd/system/multipathd.service; enabled; vendor preset: enabled)
Active: active (running) since Mon 2018-10-08 20:19:15 BST; 8s ago
Process: 49811 ExecStart=/sbin/multipathd (code=exited, status=0/SUCCESS)
Process: 49809 ExecStartPre=/sbin/multipath -A (code=exited, status=0/SUCCESS)
Process: 49808 ExecStartPre=/sbin/modprobe dm-multipath (code=exited, status=0/SUCCESS)
Main PID: 49814 (multipathd)
Tasks: 6
CGroup: /system.slice/multipathd.service
└─49814 /sbin/multipathd
3600a09807770457a795d4d4179475456 dm-2 NETAPP ,LUN C-Mode
size=2.0G features='4 queue_if_no_path pg_init_retries 50 retain_attached_hw_handle' hwhandler='1 alua' wp=rw
`-+- policy='service-time 0' prio=50 status=active
|- 4:0:0:0 sdb 8:16 active ready running
`- 3:0:0:0 sdc 8:32 active ready running
[root@redhat ~]#
Note: Make sure you have discovered & logged-into ISCSI target already, otherwise you will not see any output on 'multipath -ll' command. 2 Easy commands to do that are :
[root@redhat /]# iscsiadm -m discovery -t st -p 192.168.0.x:3260
[root@redhat /]# iscsiadm -m node -l
All done!!!
Thursday 20 September 2018
Migrating SMSQL Database from 7-mode 8.1.4 LUN to ONTAP 9.4 LUN
Ingredients required:
1. Existing SnapDrive
2. Existing SnapManager for SQL
(Both Licensed)
SnapDrive 7.0 for Windows no longer requires that you have cluster credentials. You are required only to set up your virtual storage server credentials. You are required to have SnapRestore and FlexClone licenses before setting up your virtual storage server credentials.
My lab setup:
SnapManager for SQL Server NetApp, Inc. Version: 7.2.3.2841
SnapDrive Management SnapIn NetApp, Inc. Version: 7.1.5
SQL Server 2012 [11.0.5058]
7-mode : 8.1.4 [Simulator]
cDOT : 9.4 [Simulator]
Protocol : iSCSI via Microsoft Software Initiator
In order to migrate the Database LUNs from 7-mode to cDOT, you should have added both the storage systems to the SnapDrive console.
Make sure the LIF [192.168.0.214] is dedicated iSCSI Mgmt LIF for SVM, with data-protocol-access set to none.
Before Migration, create exactly similar (7-mode) LUNs via SnapDrive on cDOT filer to match with 7-mode LUNs.
Run SnapManager for SQL wizard - Run 'configure wizard:
Just follow the prompts and it should do the trick:
Once it is successfully migrated, all the 'data' will be removed from the 7-mode LUNs [Drive], and if you go to the Windows Drive, they will be empty, all the data would have been moved to cDOT mapped LUNs Windows Drive.
Run a new Full backup, to make sure the migration has happened smoothly.
1. Existing SnapDrive
2. Existing SnapManager for SQL
(Both Licensed)
SnapDrive 7.0 for Windows no longer requires that you have cluster credentials. You are required only to set up your virtual storage server credentials. You are required to have SnapRestore and FlexClone licenses before setting up your virtual storage server credentials.
My lab setup:
SnapManager for SQL Server NetApp, Inc. Version: 7.2.3.2841
SnapDrive Management SnapIn NetApp, Inc. Version: 7.1.5
SQL Server 2012 [11.0.5058]
7-mode : 8.1.4 [Simulator]
cDOT : 9.4 [Simulator]
Protocol : iSCSI via Microsoft Software Initiator
In order to migrate the Database LUNs from 7-mode to cDOT, you should have added both the storage systems to the SnapDrive console.
Make sure the LIF [192.168.0.214] is dedicated iSCSI Mgmt LIF for SVM, with data-protocol-access set to none.
Before Migration, create exactly similar (7-mode) LUNs via SnapDrive on cDOT filer to match with 7-mode LUNs.
Run SnapManager for SQL wizard - Run 'configure wizard:
To move the LUNs to cDOT, select the database and click 'Reconfigure' :
Next, select the existing LUN [Drive] and the LUN [Drive] to be migrated to:
Once done, go to next screen and do the same for 'Snapinfo' directory location, change it to the LUN on cDOT:
Just follow the prompts and it should do the trick:
Run a new Full backup, to make sure the migration has happened smoothly.
Cannot modify 'email notifications' for the existing backup jobs for VSC/SMVI 5.x and onwards...
In SMVI/VSC 5.0 onwards, one cannot modify email notifications on the existing backup jobs after the SMTP server address has changed.
Cause:
This is a known bug:836088
Solution:
Thankfully we don't have to re-create jobs, following workaround will fix this problem.
1. Close vSphere Web Client.
2. Stop the following services from services.msc:
3. Edit the following fields in the 'scheduledBackups.xml' file for every job:
C:\Program Files\NetApp\Virtual Storage Console\smvi\server\repository\
-----------------------------
<notification>
<addresses>
<address>New_To_Address@example.com</address>
</addresses>
<smtphost>NEW_IP_Address</smtphost>
<fromAddress>New_From_Address@example.com</fromAddress>
<type>ALL</type>
</notification>
4. Save and close scheduledBackups.xml
5. Start services that were stopped in step 2.
6. Open vSphere Web Client to see the changes made.
Cause:
This is a known bug:836088
Solution:
Thankfully we don't have to re-create jobs, following workaround will fix this problem.
1. Close vSphere Web Client.
2. Stop the following services from services.msc:
- Virtual Storage Console for VMware vSphere Server
- NetApp SnapManager for Virtual Infrastructure
3. Edit the following fields in the 'scheduledBackups.xml' file for every job:
C:\Program Files\NetApp\Virtual Storage Console\smvi\server\repository\
-----------------------------
<notification>
<addresses>
<address>New_To_Address@example.com</address>
</addresses>
<smtphost>NEW_IP_Address</smtphost>
<fromAddress>New_From_Address@example.com</fromAddress>
<type>ALL</type>
</notification>
4. Save and close scheduledBackups.xml
5. Start services that were stopped in step 2.
6. Open vSphere Web Client to see the changes made.
Tuesday 18 September 2018
Device driver : Sounds familiar, isnt it ? But, it's the single largest contributor to OS code.
Device driver : is a software component that provides an interface between OS and a Hardware device. It is a module of code that can be compiled into kernel or independently of other code.
Purpose : To configure and manage the device and convert requests from the kernel into requests to the Hardware.
Textbook oriented definition of 'device driver' : A device driver can be thought of a translator, that has two functions INPUT (commands) and OUTPUT (Instructions):
Drivers rely on three interfaces:
1. Interface between Driver and Kernel
2. Interface between Driver and Device
3. Interface between Driver and BUS
There are 3 categories of drivers:
1. Character drivers = byte-stream oriented [Media/video/sound]
2. Block drivers = random-access oriented [ata/ide/scsi]
3. Network drivers = packet-stream oriented [Ethernet/wireless/infiband]
Sounds informative, but there is plenty that goes into it, and google is your good friend.
Purpose : To configure and manage the device and convert requests from the kernel into requests to the Hardware.
Textbook oriented definition of 'device driver' : A device driver can be thought of a translator, that has two functions INPUT (commands) and OUTPUT (Instructions):
- INPUT : High level commands such as "retrieve block 123"
- OUTPUT : Low-level hardware specific instructions used by the hardware-controller, which interfaces the IO device to the rest of the system.
Drivers rely on three interfaces:
1. Interface between Driver and Kernel
2. Interface between Driver and Device
3. Interface between Driver and BUS
There are 3 categories of drivers:
1. Character drivers = byte-stream oriented [Media/video/sound]
2. Block drivers = random-access oriented [ata/ide/scsi]
3. Network drivers = packet-stream oriented [Ethernet/wireless/infiband]
Sounds informative, but there is plenty that goes into it, and google is your good friend.
Sunday 16 September 2018
NFS and ONTAP9
If your NFS file system is mounted across a high-speed network, such as 10/40Gigabit Ethernet using JUMBO frame, then larger read and write packet sizes might enhance NFS file system performance.
The chunk size that works best for your NFS configuration depends primarily on the network configuration (MTU) and the applications performing I/O. If your applications are doing lots of small I/O, then a large chunk size might negatively affect the performance.
If the applications are using a mixture of I/O payload sizes, then you must do some experimentation with variable block sizes to select the 'sweet spot size'. However, once you have determined that the 'larger block size' is best for your environment, then MTU size can be very important because it determines 'packet fragments' on the network. If your chunk size is 8KB and the MTU is 1500, it will take six Ethernet frames to transmit the 8KB. If you increase the MTU to 9000 (9,000 bytes), the number of Ethernet frames drops to one.
Chunk vs MTU fragments [1500 bytes] | JUMBO [9000 bytes]
8KB = 6 1
32KB = 22 4
64KB = 43 8
In general factors that affect NFS performance are:
1. OS buffer size
2. NFS wsize/rsize
3. RPC slots
4. TCP Window (Static or Dynamic)
5. Ethernet MTU (Standard / Jumbo)
6. NFS threads
To avoid potentially causing denial of service on a cluster node, modify clients running RHEL 6.3 and later to use, at most, 128 RPC slots. This number corresponds with the current maximum number of RPCs available per TCP connection in ONTAP.
To do this, run the following on the NFS client (alternatively, edit the /etc/modprobe.d/sunrpc.conf file manually to use these values):
# echo "options sunrpc udp_slot_table_entries=64 tcp_slot_table_entries=128 tcp_max_slot_table_entries=128" >> /etc/modprobe.d/sunrpc.conf
Centos7 machine in my lab:
[root@centos7 modprobe.d]# cd /etc/modprobe.d/
[root@centos7 modprobe.d]# ls
lockd.conf mlx4.conf truescale.conf tuned.conf
Note: Looks like my client does not contain this file, let's create one.
[root@centos7 modprobe.d]# touch sunrpc.conf
[root@centos7 modprobe.d]# ls
lockd.conf mlx4.conf sunrpc.conf truescale.conf tuned.conf
[root@centos7 modprobe.d]# echo "options sunrpc udp_slot_table_entries=64 tcp_slot_table_entries=128 tcp_max_slot_table_entries=128" >> /etc/modprobe.d/sunrpc.conf
[root@centos7 modprobe.d]# cat sunrpc.conf
options sunrpc udp_slot_table_entries=64 tcp_slot_table_entries=128 tcp_max_slot_table_entries=128
[root@centos7 modprobe.d]# reboot
In NetApp ONTAP® 9 and later, '-v3-tcp-max-read-size' and '-v3-tcp-max-write-size' have been deprecated.
The recommendation is to leverage the option '-tcp-max-xfer-size' instead. This change also allows TCP transfer sizes of 1MB for both reads and writes. ONTAP versions prior to ONTAP 9 only allowed 1MB for reads.
Following example shows - How to change the NFS 'wsize' & 'rsize' to 1MB. This is just an example showing the commands needed to change read/write size, and it's not a 'recommendation', as every environment (Infrastructure + Application IO requirement) is different.
To change the read/write size:
1) Check the current xfer size on NetApp first:
ONTAP94::*> nfs show -fields tcp-max-xfer-size
vserver tcp-max-xfer-size
------- -----------------
SVM_NFS 65536
2) Check the read/write size on the NFS client:
[root@centos7 /]# mount
192.168.0.215:/vol_NFS on /mnt/netapp-mnt type nfs
(rw,relatime,vers=3,rsize=65536,wsize=65536,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=192.168.0.215,mountvers=3,mountport=635,mountproto=udp,local_lock=none,addr=192.168.0.215)
3) Change the xfer size on NetApp first:
ONTAP94::*> nfs modify -tcp-max-xfer-size 1048576 -vserver SVM_NFS
Warning: Setting "-tcp-max-xfer-size" to a value greater than 65536 may cause performance degradation for existing connections using smaller values. Contact technical support for guidance. Do you want to continue? {y|n}: y
Confirm it again:
ONTAP94::*> nfs show -fields tcp-max-xfer-size
vserver tcp-max-xfer-size
------- -----------------
SVM_NFS 1048576
4) unmount the existing connection(s) on NFS clients:
[root@centos7 /]# umount -f /mnt/netapp-mnt
5) mount it back:
[root@centos7 /]# mount -t nfs 192.168.0.215:/vol_NFS /mnt/netapp-mnt/
6) Confirm the new size:
[root@centos7 /]# mount
192.168.0.215:/vol_NFS on /mnt/netapp-mnt type nfs (rw,relatime,vers=3,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=192.168.0.215,mountvers=3,mountport=635,mountproto=udp,local_lock=none,addr=192.168.0.215)
[root@centos7 /]#
According to TR-4067: In some circumstances, the number of ports used to mount or for NFS operations might be exhausted, which then causes subsequent mount and NFS operations to hang until a port is made available. For example, one scenario seen was with ESX using NFS datastores, because some legacy best practices would call for a data LIF/IP address per datastore. In environments with many volumes/datastores, this created a situation where NFS ports were overrun. The remediation for this situation would be to disable the mount-rootonly and/or the nfs-rootonly options on the NFS server. This removes the 1 to 1,024 port range limit and allows up to 65,534 ports to be used in a NFS server.
Courtesy: TR-4067 (Must read article)
The chunk size that works best for your NFS configuration depends primarily on the network configuration (MTU) and the applications performing I/O. If your applications are doing lots of small I/O, then a large chunk size might negatively affect the performance.
If the applications are using a mixture of I/O payload sizes, then you must do some experimentation with variable block sizes to select the 'sweet spot size'. However, once you have determined that the 'larger block size' is best for your environment, then MTU size can be very important because it determines 'packet fragments' on the network. If your chunk size is 8KB and the MTU is 1500, it will take six Ethernet frames to transmit the 8KB. If you increase the MTU to 9000 (9,000 bytes), the number of Ethernet frames drops to one.
Chunk vs MTU fragments [1500 bytes] | JUMBO [9000 bytes]
8KB = 6 1
32KB = 22 4
64KB = 43 8
In general factors that affect NFS performance are:
1. OS buffer size
2. NFS wsize/rsize
3. RPC slots
4. TCP Window (Static or Dynamic)
5. Ethernet MTU (Standard / Jumbo)
6. NFS threads
To avoid potentially causing denial of service on a cluster node, modify clients running RHEL 6.3 and later to use, at most, 128 RPC slots. This number corresponds with the current maximum number of RPCs available per TCP connection in ONTAP.
To do this, run the following on the NFS client (alternatively, edit the /etc/modprobe.d/sunrpc.conf file manually to use these values):
# echo "options sunrpc udp_slot_table_entries=64 tcp_slot_table_entries=128 tcp_max_slot_table_entries=128" >> /etc/modprobe.d/sunrpc.conf
Centos7 machine in my lab:
[root@centos7 modprobe.d]# cd /etc/modprobe.d/
[root@centos7 modprobe.d]# ls
lockd.conf mlx4.conf truescale.conf tuned.conf
Note: Looks like my client does not contain this file, let's create one.
[root@centos7 modprobe.d]# touch sunrpc.conf
[root@centos7 modprobe.d]# ls
lockd.conf mlx4.conf sunrpc.conf truescale.conf tuned.conf
[root@centos7 modprobe.d]# echo "options sunrpc udp_slot_table_entries=64 tcp_slot_table_entries=128 tcp_max_slot_table_entries=128" >> /etc/modprobe.d/sunrpc.conf
[root@centos7 modprobe.d]# cat sunrpc.conf
options sunrpc udp_slot_table_entries=64 tcp_slot_table_entries=128 tcp_max_slot_table_entries=128
[root@centos7 modprobe.d]# reboot
In NetApp ONTAP® 9 and later, '-v3-tcp-max-read-size' and '-v3-tcp-max-write-size' have been deprecated.
The recommendation is to leverage the option '-tcp-max-xfer-size' instead. This change also allows TCP transfer sizes of 1MB for both reads and writes. ONTAP versions prior to ONTAP 9 only allowed 1MB for reads.
Following example shows - How to change the NFS 'wsize' & 'rsize' to 1MB. This is just an example showing the commands needed to change read/write size, and it's not a 'recommendation', as every environment (Infrastructure + Application IO requirement) is different.
To change the read/write size:
1) Check the current xfer size on NetApp first:
ONTAP94::*> nfs show -fields tcp-max-xfer-size
vserver tcp-max-xfer-size
------- -----------------
SVM_NFS 65536
2) Check the read/write size on the NFS client:
[root@centos7 /]# mount
192.168.0.215:/vol_NFS on /mnt/netapp-mnt type nfs
(rw,relatime,vers=3,rsize=65536,wsize=65536,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=192.168.0.215,mountvers=3,mountport=635,mountproto=udp,local_lock=none,addr=192.168.0.215)
3) Change the xfer size on NetApp first:
ONTAP94::*> nfs modify -tcp-max-xfer-size 1048576 -vserver SVM_NFS
Warning: Setting "-tcp-max-xfer-size" to a value greater than 65536 may cause performance degradation for existing connections using smaller values. Contact technical support for guidance. Do you want to continue? {y|n}: y
Confirm it again:
ONTAP94::*> nfs show -fields tcp-max-xfer-size
vserver tcp-max-xfer-size
------- -----------------
SVM_NFS 1048576
4) unmount the existing connection(s) on NFS clients:
[root@centos7 /]# umount -f /mnt/netapp-mnt
5) mount it back:
[root@centos7 /]# mount -t nfs 192.168.0.215:/vol_NFS /mnt/netapp-mnt/
6) Confirm the new size:
[root@centos7 /]# mount
192.168.0.215:/vol_NFS on /mnt/netapp-mnt type nfs (rw,relatime,vers=3,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=192.168.0.215,mountvers=3,mountport=635,mountproto=udp,local_lock=none,addr=192.168.0.215)
[root@centos7 /]#
According to TR-4067: In some circumstances, the number of ports used to mount or for NFS operations might be exhausted, which then causes subsequent mount and NFS operations to hang until a port is made available. For example, one scenario seen was with ESX using NFS datastores, because some legacy best practices would call for a data LIF/IP address per datastore. In environments with many volumes/datastores, this created a situation where NFS ports were overrun. The remediation for this situation would be to disable the mount-rootonly and/or the nfs-rootonly options on the NFS server. This removes the 1 to 1,024 port range limit and allows up to 65,534 ports to be used in a NFS server.
Courtesy: TR-4067 (Must read article)
Monday 10 September 2018
Mapping NetApp's inode (hexadecimal) to Linux inode (decimal)
OS: ONTAP 9.4
LINUX:Linux centos7 3.10.0-693.el7.x86_64
vserver :SVM_NFS
Volume :vol_NFS
Protocol: NFSv3
First mount the NFS vol to Linux: [Assuming you have already exported the NFS vol to the Linux client(s)]
[root@centos7 /]# mount -t nfs 192.168.0.215:/vol_NFS /mnt/nfs-mount/
Create a text file using touch command:
[root@centos7 nfs-mount]# touch test
Use vi editor to write something inside the file, I have written 'NetApp is fun to work with!!!'
Use cat command to verify what you wrote:
[root@centos7 nfs-mount]# cat test
NetApp is fun to work with!!!
Next : Lets identify the inode or a file-handle in ONTAP:
Go to advanced Cluster shell level: set adv
Use the following command: 'file show-filehandle' ; as shown below.
ONTAP94::*> file show-filehandle -vserver SVM_NFS -path /vol/vol_NFS/test
Vserver Path
---------------------- ---------------------------
SVM_NFS /vol/vol_NFS/test
flags snapid fileid generation fsid msid dsid
------- ------ --------- ---------- ---------- ------------ ------------
0x0 0 0x5d09 0x1b15bd 0x402 0x80024296 0x402
ONTAP94::*>
There is an additional command at the 'node' level to view the file-handle (inode) is called 'showfh':
ONTAP94::*> node run -node ONTAP94-01 -command showfh /vol/vol_NFS/test
flags=0x00 snapid=0 fileid=0x005d09 gen=0x1b15bd fsid=0x000402 dsid=0x00000000000402 msid=0x00000080024296
ONTAP94::*>
At Linux OS, use the command 'stat and file name' to view the inode of the file:
[root@centos7 /]# stat /mnt/nfs-mount/test
File: ‘/mnt/nfs-mount/test’
Size: 30 Blocks: 0 IO Block: 1048576 regular file
Device: 2bh/43d Inode: 1117150473 Links: 1
Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root)
Context: system_u:object_r:nfs_t:s0
Access: 2018-09-10 09:28:40.581214000 +0100
Modify: 2018-09-10 09:28:40.581249000 +0100
Change: 2018-09-10 09:28:40.581294000 +0100
Birth: -
[root@centos7 /]#
We found the inode for the file 'test' from Linux side: Inode: 1117150473
Inode: 1117150473 <---Convert this from decimal into hexadecimal (use google) and you obtain = 42965D09
Now, split this [42965D09] into two halfs = 4296 & 5D09 and compare the two with the output from the 'show-filehandle'command:
NetApp -> fileid (inode) = 0x5d09
Linux -> inode = 5D09
NetApp -> msid = 0x80024296
Linux -> part of inode = 4296
This confirms they match as expected.
TIP: In Cluster ONTAP - Every volume has MSID and DSID [Master Data Set ID & Data Set ID], like Parent & Child relation of a object. Please remember, this will be unique for each new volume, exception if it's a Cloned volume or load sharing Mirrored volume, in which case the MSID will remain the same but DSID will be unique.
Now, let's use the hexdump to view the contents of the file 'test', remember what we wrote - 'NetApp is fun to work with!!!'
This command basically prints the output in hexadump format and contains an additional column shows the corresponding ASCII text translation.
Node level : priv set diag;
This output clearly shows - Where exactly the 'data' is stored, the actual disk, and the disk's block number and the Physical block (vbn or pvbn) which belongs to aggregate. Do not confuse vbn mentioned below with flex-vol block number which is represented by vvbn (virtual volume block number) and is used to represent blocks at flex-vol level (logical container) a way to virtualize storage.
ONTAP94-01*> dumpblock ifile -V vol_NFS 0x005d09
disk v2.19, dbn 63351, vbn 2874103
0x0000: 0x4946424d 0x000007ce 0x000006a5 0x00000000 <MBFI............>
0x0010: 0x00000004 0x00000000 0x00000000 0x00000000 <................>
0x0020: 0x00000000 0x00000000 0x00000000 0x00000000 <................>
0x0030: 0x00000000 0x00000000 0x00000000 0x00000000 <................>
0x0040: 0x00000004 0x00000000 0x00000000 0x00000000 <................>
0x0130: 0x00000000 0x00000000 0x00000000 0x00000000 <................>
0x0140: 0x00000000 0x00000000 0x00000000 0x00000000 <................>
0x0150: 0x00000000 0x00000000 0x00000000 0x00000000 <................>
0x0160: 0x00000104 0x00000000 0x00000000 0x00000001 <................>
0x0170: 0x00000002 0x00000000 0x00000000 0x00000000 <................>
0x0180: 0x5b9629be 0x1ae87168 0x5b9629be 0x1ae909c0 <.).[hq...).[....>
0x0190: 0x5b9629be 0x1ae99a48 0x5b9629be 0x1ae87168 <.).[H....).[hq..>
0x01a0: 0x0000001e 0x00000000 0x00000000 0x00000000 <................>
0x01b0: 0x4174654e 0x69207070 0x75662073 0x6f74206e <NetApp is fun to>
0x01c0: 0x726f7720 0x6977206b 0x21216874 0x00000a21 < work with!!!...>
Please note: Even though Machine only understands binary '0' & '1' at the lowest level, it is much efficient to save them in hexadecimal format in order to save space and it also makes it easy to read, interpret & debug for humans.
LINUX:Linux centos7 3.10.0-693.el7.x86_64
vserver :SVM_NFS
Volume :vol_NFS
Protocol: NFSv3
First mount the NFS vol to Linux: [Assuming you have already exported the NFS vol to the Linux client(s)]
[root@centos7 /]# mount -t nfs 192.168.0.215:/vol_NFS /mnt/nfs-mount/
Create a text file using touch command:
[root@centos7 nfs-mount]# touch test
Use vi editor to write something inside the file, I have written 'NetApp is fun to work with!!!'
Use cat command to verify what you wrote:
[root@centos7 nfs-mount]# cat test
NetApp is fun to work with!!!
Next : Lets identify the inode or a file-handle in ONTAP:
Go to advanced Cluster shell level: set adv
Use the following command: 'file show-filehandle' ; as shown below.
ONTAP94::*> file show-filehandle -vserver SVM_NFS -path /vol/vol_NFS/test
Vserver Path
---------------------- ---------------------------
SVM_NFS /vol/vol_NFS/test
flags snapid fileid generation fsid msid dsid
------- ------ --------- ---------- ---------- ------------ ------------
0x0 0 0x5d09 0x1b15bd 0x402 0x80024296 0x402
ONTAP94::*>
There is an additional command at the 'node' level to view the file-handle (inode) is called 'showfh':
ONTAP94::*> node run -node ONTAP94-01 -command showfh /vol/vol_NFS/test
flags=0x00 snapid=0 fileid=0x005d09 gen=0x1b15bd fsid=0x000402 dsid=0x00000000000402 msid=0x00000080024296
ONTAP94::*>
At Linux OS, use the command 'stat and file name' to view the inode of the file:
[root@centos7 /]# stat /mnt/nfs-mount/test
File: ‘/mnt/nfs-mount/test’
Size: 30 Blocks: 0 IO Block: 1048576 regular file
Device: 2bh/43d Inode: 1117150473 Links: 1
Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root)
Context: system_u:object_r:nfs_t:s0
Access: 2018-09-10 09:28:40.581214000 +0100
Modify: 2018-09-10 09:28:40.581249000 +0100
Change: 2018-09-10 09:28:40.581294000 +0100
Birth: -
[root@centos7 /]#
We found the inode for the file 'test' from Linux side: Inode: 1117150473
Inode: 1117150473 <---Convert this from decimal into hexadecimal (use google) and you obtain = 42965D09
Now, split this [42965D09] into two halfs = 4296 & 5D09 and compare the two with the output from the 'show-filehandle'command:
NetApp -> fileid (inode) = 0x5d09
Linux -> inode = 5D09
NetApp -> msid = 0x80024296
Linux -> part of inode = 4296
This confirms they match as expected.
TIP: In Cluster ONTAP - Every volume has MSID and DSID [Master Data Set ID & Data Set ID], like Parent & Child relation of a object. Please remember, this will be unique for each new volume, exception if it's a Cloned volume or load sharing Mirrored volume, in which case the MSID will remain the same but DSID will be unique.
Now, let's use the hexdump to view the contents of the file 'test', remember what we wrote - 'NetApp is fun to work with!!!'
This command basically prints the output in hexadump format and contains an additional column shows the corresponding ASCII text translation.
Node level : priv set diag;
This output clearly shows - Where exactly the 'data' is stored, the actual disk, and the disk's block number and the Physical block (vbn or pvbn) which belongs to aggregate. Do not confuse vbn mentioned below with flex-vol block number which is represented by vvbn (virtual volume block number) and is used to represent blocks at flex-vol level (logical container) a way to virtualize storage.
ONTAP94-01*> dumpblock ifile -V vol_NFS 0x005d09
disk v2.19, dbn 63351, vbn 2874103
0x0000: 0x4946424d 0x000007ce 0x000006a5 0x00000000 <MBFI............>
0x0010: 0x00000004 0x00000000 0x00000000 0x00000000 <................>
0x0020: 0x00000000 0x00000000 0x00000000 0x00000000 <................>
0x0030: 0x00000000 0x00000000 0x00000000 0x00000000 <................>
0x0040: 0x00000004 0x00000000 0x00000000 0x00000000 <................>
0x0130: 0x00000000 0x00000000 0x00000000 0x00000000 <................>
0x0140: 0x00000000 0x00000000 0x00000000 0x00000000 <................>
0x0150: 0x00000000 0x00000000 0x00000000 0x00000000 <................>
0x0160: 0x00000104 0x00000000 0x00000000 0x00000001 <................>
0x0170: 0x00000002 0x00000000 0x00000000 0x00000000 <................>
0x0180: 0x5b9629be 0x1ae87168 0x5b9629be 0x1ae909c0 <.).[hq...).[....>
0x0190: 0x5b9629be 0x1ae99a48 0x5b9629be 0x1ae87168 <.).[H....).[hq..>
0x01a0: 0x0000001e 0x00000000 0x00000000 0x00000000 <................>
0x01b0: 0x4174654e 0x69207070 0x75662073 0x6f74206e <NetApp is fun to>
0x01c0: 0x726f7720 0x6977206b 0x21216874 0x00000a21 < work with!!!...>
Please note: Even though Machine only understands binary '0' & '1' at the lowest level, it is much efficient to save them in hexadecimal format in order to save space and it also makes it easy to read, interpret & debug for humans.
Thursday 9 August 2018
Saturday 28 July 2018
GUID snapshots {curly brackets} found on the NetApp Volume
Please read the following article for more information.
Thursday 26 July 2018
How to identify the IP address of the source and destination SnapVault
How to identify the IP address of the source and destination SnapVault: Please go to this article,
flow control NetApp
Flow control on NetApp 10-GbE interfaces: Please visit the following article for more information. Here the key-word is 'E' '10-GbE', i.e it applies to dedicated Ethernet NIC only, it does not apply to CNA/UTA Cards, where flow-control cannot be disabled.
Subscribe to:
Posts (Atom)