Openfiler 2.99 HA (Haute dispo)

De $1

Sommaire
  1. 1. Procédure d'installation et de paramétrage d'Openfiler afin d'obtenir un cluster haute dispo
  2. 2. Pré-requis
    1. 2.1. Humains
    2. 2.2. Matériels
  3. 3. Openfiler01
  4. 4. Openfiler02
  5. 5. Installation Openfiler
    1. 5.1. Sur les 2 openfiler
  6. 6. Configuration de base
    1. 6.1. Updates
      1. 6.1.1. Openfiler01
      2. 6.1.2. Openfiler02 
    2. 6.2. Configuration des fichiers /etc/hosts
      1. 6.2.1. Openfiler01
      2. 6.2.2. Openfiler02 
    3. 6.3. Création des clés SSH partagées
      1. 6.3.1. Openfiler01
      2. 6.3.2. Openfiler02
    4. 6.4. Partage des clés SSH
      1. 6.4.1. Openfiler01
      2. 6.4.2. Openfiler02
  7. 7. DRBD
    1. 7.1. Création des partitions
      1. 7.1.1. Openfiler01
      2. 7.1.2. Openfiler02
    2. 7.2. Paramétrages de DRBD
      1. 7.2.1. Openfiler01
        1. 7.2.1.1. Fichier /etc/drbd.conf
        2. 7.2.1.2. Copie de /etc/drbd.conf sur openfiler02
          1. 7.2.1.2.1. Openfiler01
        3. 7.2.1.3. Modification du fichier /etc/drbd.d/global_common.conf
        4. 7.2.1.4. Copie de /etc/drdb.d/global_common.conf sur openfiler02 
    3. 7.3. Création et initialisation du volume meta sur /dev/drbd0 et du volume data sur /dev/drbd1 
      1. 7.3.1. Création des volumes sur Openfiler01
      2. 7.3.2. Création des volumes sur Openfiler02
      3. 7.3.3. Démarrage de drbd sur Openfiler01
      4. 7.3.4. Démarrage de drbd sur Openfiler02
      5. 7.3.5. Statut drbd sur Openfiler01
      6. 7.3.6. Statut drbd sur Openfiler02
      7. 7.3.7. Paramétrage des volumes sur Openfiler01
      8. 7.3.8. Paramétrage du service drbd sur Openfiler02
      9. 7.3.9. Openfiler01
      10. 7.3.10. Modification de l'arborescence openfiler01
      11. 7.3.11. Modification de l'arborescence openfiler02 
  8. 8. Corosync 
      1. 8.1.1. Création de la clé corosync sur Openfiler01 
      2. 8.1.2. Openfiler02 
      3. 8.1.3. Openfiler01 
      4. 8.1.4. Modification des services sur Openfiler02 
      5. 8.1.5. Openfiler01 
      6. 8.1.6. Openfiler02 
      7. 8.1.7. Configuration de corosync sur Openfiler01
  9. 9. Remerciements
    1. 9.1. Sources 
    2. 9.2. Openfiler
    3. 9.3. Corosync
    4. 9.4. Drbd 
  10. 10.  Vous en pensez quoi ?

punaise1.gif

Sommaire

  1. 1. Procédure d'installation et de paramétrage d'Openfiler afin d'obtenir un cluster haute dispo
  2. 2. Pré-requis
    1. 2.1. Humains
    2. 2.2. Matériels
  3. 3. Openfiler01
  4. 4. Openfiler02
  5. 5. Installation Openfiler
    1. 5.1. Sur les 2 openfiler
  6. 6. Configuration de base
    1. 6.1. Updates
      1. 6.1.1. Openfiler01
      2. 6.1.2. Openfiler02 
    2. 6.2. Configuration des fichiers /etc/hosts
      1. 6.2.1. Openfiler01
      2. 6.2.2. Openfiler02 
    3. 6.3. Création des clés SSH partagées
      1. 6.3.1. Openfiler01
      2. 6.3.2. Openfiler02
    4. 6.4. Partage des clés SSH
      1. 6.4.1. Openfiler01
      2. 6.4.2. Openfiler02
  7. 7. DRBD
    1. 7.1. Création des partitions
      1. 7.1.1. Openfiler01
      2. 7.1.2. Openfiler02
    2. 7.2. Paramétrages de DRBD
      1. 7.2.1. Openfiler01
        1. 7.2.1.1. Fichier /etc/drbd.conf
        2. 7.2.1.2. Copie de /etc/drbd.conf sur openfiler02
          1. 7.2.1.2.1. Openfiler01
        3. 7.2.1.3. Modification du fichier /etc/drbd.d/global_common.conf
        4. 7.2.1.4. Copie de /etc/drdb.d/global_common.conf sur openfiler02 
    3. 7.3. Création et initialisation du volume meta sur /dev/drbd0 et du volume data sur /dev/drbd1 
      1. 7.3.1. Création des volumes sur Openfiler01
      2. 7.3.2. Création des volumes sur Openfiler02
      3. 7.3.3. Démarrage de drbd sur Openfiler01
      4. 7.3.4. Démarrage de drbd sur Openfiler02
      5. 7.3.5. Statut drbd sur Openfiler01
      6. 7.3.6. Statut drbd sur Openfiler02
      7. 7.3.7. Paramétrage des volumes sur Openfiler01
      8. 7.3.8. Paramétrage du service drbd sur Openfiler02
      9. 7.3.9. Openfiler01
      10. 7.3.10. Modification de l'arborescence openfiler01
      11. 7.3.11. Modification de l'arborescence openfiler02 
  8. 8. Corosync 
      1. 8.1.1. Création de la clé corosync sur Openfiler01 
      2. 8.1.2. Openfiler02 
      3. 8.1.3. Openfiler01 
      4. 8.1.4. Modification des services sur Openfiler02 
      5. 8.1.5. Openfiler01 
      6. 8.1.6. Openfiler02 
      7. 8.1.7. Configuration de corosync sur Openfiler01
  9. 9. Remerciements
    1. 9.1. Sources 
    2. 9.2. Openfiler
    3. 9.3. Corosync
    4. 9.4. Drbd 
  10. 10.  Vous en pensez quoi ?

 

 

howto04_small.png

Procédure d'installation et de paramétrage d'Openfiler afin d'obtenir un cluster haute dispo

 


Pré-requis

Humains

Du temps et de la patience

Rien à faire du week end

Si possible un temps de merde pour pas démoraliser

Pas d'amis

Pas de copine

Pas de vie sociale

Une tendance native à la schyzophrénie

Matériels

2 Serveurs avec chacun 2 cartes réseau

Openfiler01

LAN Production : 192.168.1.18

LAN Réplication : 10.10.10.18

Openfiler02

LAN Production : 192.168.1.19

LAN Replication : 10.10.10.19

 

Adresse IP HA : 192.168.1.10

Installation Openfiler

http://www.openfiler.com/community/download/

 

Sur les 2 openfiler

 

open001a.png

 

open002a.png

 

open003a.png

 

open004a.png

 

open005a.png

 

open006a.png

 

open007a.png

 

open008a.png

 

open009a.png

 

open010a.png

 

open011a.png

 

open012a.png

 

open013a.png

 

open014a.png

 

open216a.png

 

open016a.png

 

open017a.png

 

 

 

 

Configuration de base

Updates

Openfiler01

[root@openfiler01 ~]# conary update conary

[root@openfiler01 ~]# conary updateall

[root@openfiler01 ~]# conary update openfiler 
[root@openfiler01 ~]# reboot

Openfiler02 

[root@openfiler02 ~]# conary update conary

[root@openfiler02 ~]# conary updateall

[root@openfiler02 ~]# conary update openfiler   

[root@openfiler02 ~]# reboot

Configuration des fichiers /etc/hosts

Openfiler01

Se connecter en ssh

ssh -l root 192.168.1.18

Editer /etc/hosts

[root@openfiler01 ~]# vi /etc/hosts

 

# Do not remove the following line, or various programs
# that require network functionality will fail.
127.0.0.1 openfiler01 localhost.localdomain localhost
10.10.10.19 openfiler02.localdomain    openfiler02

 

Petit test

[root@openfiler01 ~]# ping openfiler02
PING openfiler02 (10.10.10.19) 56(84) bytes of data. 64 bytes from openfiler02 (10.10.10.19): icmp_seq=1 ttl=64 time=0.444 ms 64 bytes from openfiler02 (10.10.10.19): icmp_seq=2 ttl=64 time=0.494 ms ^C --- openfiler02 ping statistics --- 2 packets transmitted, 2 received, 0% packet loss, time 1000ms rtt min/avg/max/mdev = 0.444/0.469/0.494/0.025 ms
 

Openfiler02 

 

Se connecter en ssh

ssh -l root 192.168.1.18

Editer /etc/hosts

[root@openfiler01 ~]# vi /etc/hosts

 

# Do not remove the following line, or various programs
# that require network functionality will fail.
127.0.0.1 openfiler02 localhost.localdomain localhost
10.10.10.18 openfiler01.localdomain    openfiler01

 

Petit test

[root@openfiler02 ~]# ping openfiler01
PING openfiler01 (10.10.10.18) 56(84) bytes of data. 64 bytes from openfiler01 (10.10.10.18): icmp_seq=1 ttl=64 time=1.63 ms ^C --- openfiler01 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 1.633/1.633/1.633/0.000 ms
 

Création des clés SSH partagées

Openfiler01

[root@openfiler01 ~]# ssh-keygen -t dsa
Generating public/private dsa key pair.
Enter file in which to save the key (/root/.ssh/id_dsa): 
Created directory '/root/.ssh'.
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /root/.ssh/id_dsa.
Your public key has been saved in /root/.ssh/id_dsa.pub.
The key fingerprint is:
1e:c7:1a:7f:29:45:73:d2:60:38:fe:7d:92:28:78:be root@openfiler01.localdomain
The key's randomart image is:
+--[ DSA 1024]----+
|           .o    |
|          o. o   |
|         . .+ o  |
|         ... +   |
|        S.o..o . |
|       ..*o.o.+ .|
|        ooo.o  o |
|          .o     |
|          E.     |
+-----------------+
 

 PS : Ne pas entrer de mot de passe

 

 

Openfiler02

[root@openfiler02 ~]# ssh-keygen -t dsa
Generating public/private dsa key pair.
Enter file in which to save the key (/root/.ssh/id_dsa): 
Created directory '/root/.ssh'.
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /root/.ssh/id_dsa.
Your public key has been saved in /root/.ssh/id_dsa.pub.
The key fingerprint is:
77:54:fc:55:7a:15:d9:49:55:d3:a7:73:18:da:aa:c3 root@openfiler02.localdomain
The key's randomart image is:
+--[ DSA 1024]----+
|             .o+#|
|             .o+B|
|            .o.=+|
|           .. =.o|
|        S . .. o |
|         . ..    |
|         . .     |
|          E      |
|           .     |
+-----------------+
 

 PS : Ne pas entrer de mot de passe

Partage des clés SSH

Openfiler01

[root@openfiler01 ~]# scp .ssh/id_dsa.pub root@openfiler02:~/.ssh/authorized_keys2 
The authenticity of host 'openfiler02 (10.10.10.19)' can't be established.
RSA key fingerprint is fb:00:c6:01:73:d0:80:1f:36:fe:29:a8:e4:dc:2a:a2.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'openfiler02,10.10.10.19' (RSA) to the list of known hosts.
root@openfiler02's password: XXXXX
id_dsa.pub                                    100%  618     0.6KB/s   00:00    
 

Openfiler02

[root@openfiler02 ~]# scp .ssh/id_dsa.pub root@openfiler01:~/.ssh/authorized_keys2  
The authenticity of host 'openfiler01 (10.10.10.18)' can't be established.
RSA key fingerprint is 11:18:ec:a6:33:d9:0d:0e:37:b5:d4:65:7e:e6:bb:08.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'openfiler01,10.10.10.18' (RSA) to the list of known hosts.
root@openfiler01's password: XXXXX
id_dsa.pub                                    100%  618     0.6KB/s   00:00    
 

DRBD

Création des partitions

Création des partitions meta et data

Openfiler01

[root@openfiler01 ~]# fdisk -l

Disk /dev/sda: 53.7 GB, 53687091200 bytes
255 heads, 63 sectors/track, 6527 cylinders, total 104857600 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0009c87e

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *          63    62910539    31455238+  83  Linux
/dev/sda2        62910540    71296469     4192965   82  Linux swap / Solaris

Disk /dev/sdb: 1099.5 GB, 1099511627776 bytes
255 heads, 63 sectors/track, 133674 cylinders, total 2147483648 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000c0ff7

   Device Boot      Start         End      Blocks   Id  System
 
[root@openfiler01 ~]# fdisk /dev/sda

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4, default 3): 
Using default value 3
First sector (71714160-104857599, default 71714160): 
Using default value 71714160
Last sector, +sectors or +size{K,M,G} (71714160-104857599, default 104857599): 
Using default value 104857599

Command (m for help): t
Partition number (1-4): 3
Hex code (type L to list codes): 83

Command (m for help): p

Disk /dev/sda: 53.7 GB, 53687091200 bytes
255 heads, 63 sectors/track, 6527 cylinders, total 104857600 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000434d5

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *          63    63328229    31664083+  83  Linux
/dev/sda2        63328230    71714159     4192965   82  Linux swap / Solaris
/dev/sda3        71714160   104857599    16571720   83  Linux

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.

WARNING: Re-reading the partition table failed with error 16: Device or resource busy.
The kernel still uses the old table. The new table will be used at
the next reboot or after you run partprobe(8) or kpartx(8)
Syncing disks.
 
 
[root@openfiler01 ~]# fdisk -l

Disk /dev/sda: 53.7 GB, 53687091200 bytes
255 heads, 63 sectors/track, 6527 cylinders, total 104857600 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0009c87e

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *          63    62910539    31455238+  83  Linux
/dev/sda2        62910540    71296469     4192965   82  Linux swap / Solaris
/dev/sda3        71296470   104857599    16780565   83  Linux

Disk /dev/sdb: 1099.5 GB, 1099511627776 bytes
255 heads, 63 sectors/track, 133674 cylinders, total 2147483648 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000c0ff7

   Device Boot      Start         End      Blocks   Id  System
 
[root@openfiler01 ~]# fdisk /dev/sdb

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4, default 1): 
Using default value 1
First sector (2048-2147483647, default 2048): 
Using default value 2048
Last sector, +sectors or +size{K,M,G} (2048-2147483647, default 2147483647): 
Using default value 2147483647

Command (m for help): t
Selected partition 1
Hex code (type L to list codes): 83

Command (m for help): p

Disk /dev/sdb: 1099.5 GB, 1099511627776 bytes
255 heads, 63 sectors/track, 133674 cylinders, total 2147483648 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0008f610

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1            2048  2147483647  1073740800   83  Linux

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.


[root@openfiler01 ~]# fdisk -l

Disk /dev/sda: 53.7 GB, 53687091200 bytes
255 heads, 63 sectors/track, 6527 cylinders, total 104857600 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0009c87e

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *          63    62910539    31455238+  83  Linux
/dev/sda2        62910540    71296469     4192965   82  Linux swap / Solaris
/dev/sda3        71296470   104857599    16780565   83  Linux

Disk /dev/sdb: 1099.5 GB, 1099511627776 bytes
173 heads, 2 sectors/track, 6206600 cylinders, total 2147483648 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000c0ff7

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1            2048  2147483647  1073740800   83  Linux
 
[root@openfiler01 ~]# reboot 

 

 

Openfiler02

 [root@openfiler02 ~]# fdisk -l

Disk /dev/sda: 53.7 GB, 53687091200 bytes
255 heads, 63 sectors/track, 6527 cylinders, total 104857600 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00043c2a

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *          63    62910539    31455238+  83  Linux
/dev/sda2        62910540    71296469     4192965   82  Linux swap / Solaris

Disk /dev/sdb: 1099.5 GB, 1099511627776 bytes
255 heads, 63 sectors/track, 133674 cylinders, total 2147483648 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000208c5

   Device Boot      Start         End      Blocks   Id  System


[root@openfiler02 ~]# fdisk /dev/sda

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4, default 3): 
Using default value 3
First sector (71296470-104857599, default 71296470): 
Using default value 71296470
Last sector, +sectors or +size{K,M,G} (71296470-104857599, default 104857599): 
Using default value 104857599

Command (m for help): t
Partition number (1-4): 3
Hex code (type L to list codes): 83

Command (m for help): p

Disk /dev/sda: 53.7 GB, 53687091200 bytes
255 heads, 63 sectors/track, 6527 cylinders, total 104857600 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000ea4ee

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *          63    62910539    31455238+  83  Linux
/dev/sda2        62910540    71296469     4192965   82  Linux swap / Solaris
/dev/sda3        71296470   104857599    16780565   83  Linux

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.

WARNING: Re-reading the partition table failed with error 16: Device or resource busy.
The kernel still uses the old table. The new table will be used at
the next reboot or after you run partprobe(8) or kpartx(8)
Syncing disks.
 
 
 
[root@openfiler02 ~]# fdisk -l

Disk /dev/sda: 53.7 GB, 53687091200 bytes
255 heads, 63 sectors/track, 6527 cylinders, total 104857600 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00043c2a

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *          63    62910539    31455238+  83  Linux
/dev/sda2        62910540    71296469     4192965   82  Linux swap / Solaris
/dev/sda3        71296470   104857599    16780565   83  Linux

Disk /dev/sdb: 1099.5 GB, 1099511627776 bytes
255 heads, 63 sectors/track, 133674 cylinders, total 2147483648 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000208c5

   Device Boot      Start         End      Blocks   Id  System
 
[root@openfiler02 ~]# fdisk /dev/sdb

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4, default 1): 
Using default value 1
First sector (2048-2147483647, default 2048): 
Using default value 2048
Last sector, +sectors or +size{K,M,G} (2048-2147483647, default 2147483647): 
Using default value 2147483647

Command (m for help): t
Selected partition 1
Hex code (type L to list codes): 83

Command (m for help): p

Disk /dev/sdb: 1099.5 GB, 1099511627776 bytes
255 heads, 63 sectors/track, 133674 cylinders, total 2147483648 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0009267b

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1            2048  2147483647  1073740800   83  Linux

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.
 
 
[root@openfiler02 ~]# fdisk -l

Disk /dev/sda: 53.7 GB, 53687091200 bytes
255 heads, 63 sectors/track, 6527 cylinders, total 104857600 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00043c2a

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *          63    62910539    31455238+  83  Linux
/dev/sda2        62910540    71296469     4192965   82  Linux swap / Solaris
/dev/sda3        71296470   104857599    16780565   83  Linux

Disk /dev/sdb: 1099.5 GB, 1099511627776 bytes
173 heads, 2 sectors/track, 6206600 cylinders, total 2147483648 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000208c5

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1            2048  2147483647  1073740800   83  Linux
 
[root@openfiler02 ~]# reboot 

 

Paramétrages de DRBD

Openfiler01

root@openfiler01 ~# mv /etc/drbd.conf /etc/drbd.conf.org 

root@openfiler01 ~# vi /etc/drbd.conf

Fichier /etc/drbd.conf

A créer en fonction des partitions concernées (paramétrage des ressources synchronisées par brbd)

# You can find an example in  /usr/share/doc/drbd.../drbd.conf.example
include "drbd.d/global_common.conf";
#include "drbd.d/*.res";

resource meta {
 protocol C;
 on openfiler01 {
  device /dev/drbd0;
# Renseigner votre partition
  disk /dev/sda3; 
# Addresse IP openfiler01
  address 10.10.10.18:7788;
  meta-disk internal;
 }
 on openfiler02 {
  device /dev/drbd0;
# Renseigner votre partition
  disk /dev/sda3; 
# Addresse IP openfiler02
  address 10.10.10.19:7788;
  meta-disk internal;
 }
}


resource data {
 protocol C;
 on openfiler01 {
  device /dev/drbd1;
# Renseigner votre partition
  disk /dev/sdb1; 
# Addresse IP openfiler01
  address 10.10.10.18:7789;
  meta-disk internal;
 }
 on openfiler02 {
  device /dev/drbd1;
# Renseigner votre partition
  disk /dev/sdb1; 
# Addresse IP openfiler02
  address 10.10.10.19:7789;
  meta-disk internal;
 }
}
 
Copie de /etc/drbd.conf sur openfiler02
Openfiler01
[root@openfiler01 ~]# scp /etc/drbd.conf root@openfiler02:/etc 
drbd.conf                                         100% 1473     1.4KB/s   00:00    
 
 
Modification du fichier /etc/drbd.d/global_common.conf
[root@openfiler01 ~]# vi /etc/drbd.d/global_common.conf
 
 
Vérifier la ligne syncer (paramétrage de la bande passante utilisée pour la synchronisation)
 
syncer {

# rate after al-extents use-rle cpu-mask verify-alg csums-alg

rate 100M;

}

 

 
Copie de /etc/drdb.d/global_common.conf sur openfiler02 
[root@openfiler01 ~]# scp /etc/drbd.d/global_common.conf root@openfiler02:/etc/drbd.d/global_common.conf 
global_common.conf                                         100% 1473     1.4KB/s   00:00    
 

Création et initialisation du volume meta sur /dev/drbd0 et du volume data sur /dev/drbd1 

Création des volumes sur Openfiler01

[root@openfiler01 ~]# drbdadm create-md meta



  --==  Thank you for participating in the global usage survey  ==--
The server's response is:

you are the 25616th user to install this version
Writing meta data...
initializing activity log
NOT initialized bitmap
New drbd meta data block successfully created.
success
 
 
[root@openfiler01 ~]# drbdadm create-md data

Writing meta data...
initializing activity log
NOT initialized bitmap
New drbd meta data block successfully created.
success
 
 

 

exclamationpoint01.png

        Si ce message apparait :

 

'meta' ignored, since this host (openfiler01.localdomain) is not mentioned with an 'on' keyword.

Modifier le fichier /etc/drbd.conf en mettant openfilerXX.localdomain au lieu de openfilerXX

 

 

 

 

 

 

 

 

Création des volumes sur Openfiler02

[root@openfiler02 ~]# drbdadm create-md meta

  --==  Thank you for participating in the global usage survey  ==--
The server's response is:

you are the 25616th user to install this version
Writing meta data...
initializing activity log
NOT initialized bitmap
New drbd meta data block successfully created.
success

 
 
[root@openfiler02 ~]# drbdadm create-md data

 

Writing meta data...
initializing activity log
NOT initialized bitmap
New drbd meta data block successfully created.
success

 

 

Démarrage de drbd sur Openfiler01

[root@openfiler01 ~]# service drbd start
Starting DRBD resources: [ d(cluster_metadata) d(vg0drbd) s(cluster_metadata) s(vg0drbd) n(cluster_metadata) n(vg0drbd) ]..........
 
 
 

Démarrage de drbd sur Openfiler02

 
[root@openfiler02 ~]# service drbd start
Starting DRBD resources: [ d(cluster_metadata) d(vg0drbd) s(cluster_metadata) s(vg0drbd) n(cluster_metadata) n(vg0drbd) ].
 
  

Statut drbd sur Openfiler01

[root@openfiler01 ~]# service drbd status
drbd driver loaded OK; device status:
version: 8.3.11 (api:88/proto:86-96)
GIT-hash: 0de839cee13a4160eed6037c4bddd066645e23c5 drbd/linux/drbd_config.h scripts/adjust_drbd_config_h.sh build by rmake-chroot@localhost.localdomain, 2011-11-19 14:15:43
m:res   cs         ro                   ds                         p  mounted  fstype
0:meta  Connected  Secondary/Secondary  Inconsistent/Inconsistent  C
1:data  Connected  Secondary/Secondary  Inconsistent/Inconsistent  C
 
 
 

Statut drbd sur Openfiler02

[root@openfiler02 ~]# service drbd status
drbd driver loaded OK; device status:
version: 8.3.11 (api:88/proto:86-96)
GIT-hash: 0de839cee13a4160eed6037c4bddd066645e23c5 drbd/linux/drbd_config.h scripts/adjust_drbd_config_h.sh build by rmake-chroot@localhost.localdomain, 2011-11-19 14:15:43
m:res   cs         ro                   ds                         p  mounted  fstype
0:meta  Connected  Secondary/Secondary  Inconsistent/Inconsistent  C
1:data  Connected  Secondary/Secondary  Inconsistent/Inconsistent  C
 
 
 

Paramétrage des volumes sur Openfiler01

On paramètre les volumes d' openfiler01 comme volume primaire

[root@openfiler01 ~]# drbdsetup /dev/drbd0 primary -o
[root@openfiler01 ~]# drbdsetup /dev/drbd1 primary -o
[root@openfiler01 ~]# service drbd status
drbd driver loaded OK; device status:
version: 8.3.11 (api:88/proto:86-96)
GIT-hash: 0de839cee13a4160eed6037c4bddd066645e23c5 drbd/linux/drbd_config.h scripts/adjust_drbd_config_h.sh build by rmake-chroot@localhost.localdomain, 2011-11-19 14:15:43
m:res   cs          ro                 ds                     p  mounted  fstype
0:meta  SyncSource  Primary/Secondary  UpToDate/Inconsistent  C
...     sync'ed:    0.1%               (102360/102392)M
...     sync'ed:    13.3%              (911548/1048508)K
1:data  SyncSource  Primary/Secondary  UpToDate/Inconsistent  C
 
 
On paramètre le démarrage automatique du service drbd
[root@openfiler01 ~]# chkconfig --level 2345 drbd on

 

Paramétrage du service drbd sur Openfiler02

 

On paramètre le démarrage automatique du service drbd

[root@openfiler02 ~]# chkconfig --level 2345 drbd on
 

Openfiler01

Formatage de la partition drbd0 (meta) en ext3

[root@openfiler01 ~]# mkfs.ext3 /dev/drbd0
mke2fs 1.41.14 (22-Dec-2010)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
1050576 inodes, 4195003 blocks
209750 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=4294967296
129 block groups
32768 blocks per group, 32768 fragments per group
8144 inodes per group
Superblock backups stored on blocks: 
	32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 
	4096000

Writing inode tables: done                            
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 28 mounts or
180 days, whichever comes first.  Use tune2fs -c or -i to override.
 

 

Editer le fichier /etc/lvm/lvm.conf 

[root@openfiler01 ~]# vi /etc/lvm/lvm.conf   

 

Remplacer :

filter = [ "a/.*/" ]

Par :

filter = [ "a|drbd[0-9]|", "r|.*|" ]

 

Copie du fichier lvm.conf sur openfiler02

[root@openfiler01 ~]# scp /etc/lvm/lvm.conf root@openfiler02:/etc/lvm/lvm.conf

 

Création du volume lvm data 

[root@openfiler01 ~]# pvcreate /dev/drbd1
  Writing physical volume data to disk "/dev/drbd1"
  Physical volume "/dev/drbd1" successfully created

[root@openfiler01 ~]# vgcreate data /dev/drbd1  

[root@openfiler01 ~]# lvcreate -L 400M -n filer data

Montage de la partition meta

[root@openfiler01 ~]# mkdir /meta 

[root@openfiler01 ~]# mount /dev/drbd0 /meta

Modification de l'arborescence openfiler01

 

 root@openfiler01 ~# service openfiler stop

 

root@openfiler01 ~# mv /opt/openfiler/ /opt/openfiler.local

root@openfiler01 ~# mkdir /meta/opt

root@openfiler01 ~# cp -a /opt/openfiler.local /meta/opt/openfiler

root@openfiler01 ~# ln -s /meta/opt/openfiler /opt/openfiler

root@openfiler01 ~# rm /meta/opt/openfiler/sbin/openfiler

root@openfiler01 ~# ln -s /usr/sbin/httpd /meta/opt/openfiler/sbin/openfiler

root@openfiler01 ~# rm /meta/opt/openfiler/etc/rsync.xml

root@openfiler01 ~# ln -s /opt/openfiler.local/etc/rsync.xml /meta/opt/openfiler/etc/

root@openfiler01 ~# mkdir -p /meta/etc/httpd/conf.d

 

root@openfiler01 ~# service nfslock stop

root@openfiler01 ~# umount -a -t rpc-pipefs

root@openfiler01 ~# mkdir /meta/etc

root@openfiler01 ~# mv /etc/samba/ /meta/etc/

root@openfiler01 ~# ln -s /meta/etc/samba/ /etc/samba

root@openfiler01 ~# mkdir -p /meta/var/spool

root@openfiler01 ~# mv /var/spool/samba/ /meta/var/spool/

root@openfiler01 ~# ln -s /meta/var/spool/samba/ /var/spool/samba

root@openfiler01 ~# mkdir -p /meta/var/lib

root@openfiler01 ~# mv /var/lib/nfs/ /meta/var/lib/

root@openfiler01 ~# ln -s /meta/var/lib/nfs/ /var/lib/nfs

root@openfiler01 ~# mv /etc/exports /meta/etc/

root@openfiler01 ~# ln -s /meta/etc/exports /etc/exports

root@openfiler01 ~# mv /etc/ietd.conf /meta/etc/

root@openfiler01 ~# ln -s /meta/etc/ietd.conf /etc/ietd.conf

root@openfiler01 ~# mv /etc/initiators.allow /meta/etc/

root@openfiler01 ~# ln -s /meta/etc/initiators.allow /etc/initiators.allow

root@openfiler01 ~# mv /etc/initiators.deny /meta/etc/

root@openfiler01 ~# ln -s /meta/etc/initiators.deny /etc/initiators.deny

root@openfiler01 ~# mv /etc/proftpd /meta/etc/

root@openfiler01 ~# ln -s /meta/etc/proftpd/ /etc/proftpd

 

root@openfiler01 ~# rm /opt/openfiler/etc/httpd/modules

root@openfiler01 ~# ln -s /usr/lib64/httpd/modules /opt/openfiler/etc/httpd/modules

 

root@openfiler01 ~# service openfiler start

Modification de l'arborescence openfiler02 

 

root@openfiler02 ~# service openfiler stop

root@openfiler02 ~# mkdir /meta

root@openfiler02 ~# mv /opt/openfiler/ /opt/openfiler.local

root@openfiler02 ~# ln -s /meta/opt/openfiler /opt/openfiler

 

root@openfiler02 ~# service nfslock stop

root@openfiler02 ~# umount -a -t rpc-pipefs

root@openfiler02 ~# rm -rf /etc/samba/

root@openfiler02 ~# ln -s /meta/etc/samba/ /etc/samba

root@openfiler02 ~# rm -rf /var/spool/samba/

root@openfiler02 ~# ln -s /meta/var/spool/samba/ /var/spool/samba

root@openfiler02 ~# rm -rf /var/lib/nfs/

root@openfiler02 ~# ln -s /meta/var/lib/nfs/ /var/lib/nfs

root@openfiler02 ~# rm -rf /etc/exports

root@openfiler02 ~# ln -s /meta/etc/exports /etc/exports

root@openfiler02 ~# rm /etc/ietd.conf

root@openfiler02 ~# ln -s /meta/etc/ietd.conf /etc/ietd.conf

root@openfiler02 ~# rm /etc/initiators.allow

root@openfiler02 ~# ln -s /meta/etc/initiators.allow /etc/initiators.allow

root@openfiler02 ~# rm /etc/initiators.deny

root@openfiler02 ~# ln -s /meta/etc/initiators.deny /etc/initiators.deny

root@openfiler02 ~# rm -rf /etc/proftpd

root@openfiler02 ~# ln -s /meta/etc/proftpd/ /etc/proftpd

 

 

exclamationpoint01.png

         Il est conseillé d'attendre la fin des synchronisations avant d'effectuer la suite

 

 

 

 

 

 

Corosync 

Création de la clé corosync sur Openfiler01 

Création de la clé corosync et copie sur openfiler02

[root@openfiler01 ~]# corosync-keygen

[root@openfiler01 ~]# scp /etc/corosync/authkey root@openfiler02:/etc/corosync/authkey

 

Openfiler02 

Modification des droits

[root@openfiler02 ~]# chmod 400 /etc/corosync/authkey 

 

Openfiler01 

Modification du fichier /etc/corosync/service.d/pcmk

[root@openfiler01 ~]# vi /etc/corosync/service.d/pcmk

 

service {

        # Load the Pacemaker Cluster Resource Manager

        name: pacemaker

        ver:  0

 }

 

Copie du fichier sur openfiler02 et modification des paramétrages corosync 

[root@openfiler01 ~]# scp /etc/corosync/service.d/pcmk root@openfiler02:/etc/corosync/service.d/pcmk 

[root@openfiler01 ~]# vi /etc/corosync/corosync.conf

 

# Please read the corosync.conf.5 manual page

compatibility: whitetank

totem {

        version: 2

        secauth: off

        threads: 0

        interface {

                ringnumber: 0

                bindnetaddr: 10.10.10.0

                mcastaddr: 226.94.1.1

                mcastport: 5405

                ttl: 1

        }

}

logging {

        fileline: off

        to_stderr: no

        to_logfile: yes

        to_syslog: yes

        logfile: /var/log/cluster/corosync.log

        debug: off

        timestamp: on

        logger_subsys {

                subsys: AMF

                debug: off

        }

}

amf {

        mode: disabled

}

 

Copie des paramétrages corosync sur openfiler02

[root@openfiler01 ~]# scp /etc/corosync/corosync.conf root@openfiler02:/etc/corosync/corosync.conf

 

Modification des services  

[root@openfiler01 ~]# chkconfig --level 2345 openfiler off

[root@openfiler01 ~]# chkconfig --level 2345 nfslock off

[root@openfiler01 ~]# chkconfig --level 2345 corosync on
 
 

Modification des services sur Openfiler02 

Modification des services

[root@openfiler02 ~]# chkconfig --level 2345 openfiler off

[root@openfiler02 ~]# chkconfig --level 2345 nfslock off

[root@openfiler02 ~]# chkconfig --level 2345 corosync on
 
 
 
Redémarrage des 2 nodes l'un après l'autre (openfiler01 puis openfiler02)
 
 

  

exclamationpoint01.png

         Après les redémarrages, vérifier le statut et le montage des volumes drbd et les repasser primary sur openfiler01 si besoin

 

 

 

 

 

Openfiler01 

[root@openfiler01 ~]# ps auxf

root      3480  0.0  0.8 534456  4112 ?        Ssl  19:15   0:00 corosync

root      3486  0.0  0.5  68172  2776 ?        S    19:15   0:00  \_ /usr/lib64/heartbeat/stonith

106       3487  0.0  1.0  67684  4956 ?        S    19:15   0:00  \_ /usr/lib64/heartbeat/cib

root      3488  0.0  0.4  70828  2196 ?        S    19:15   0:00  \_ /usr/lib64/heartbeat/lrmd

106       3489  0.0  0.6  68536  3096 ?        S    19:15   0:00  \_ /usr/lib64/heartbeat/attrd

106       3490  0.0  0.6  69064  3420 ?        S    19:15   0:00  \_ /usr/lib64/heartbeat/pengine

106       3491  0.0  0.7  76764  3488 ?        S    19:15   0:00  \_ /usr/lib64/heartbeat/crmd

 
 
 

Openfiler02 

[root@openfiler02 ~]# crm_mon --one-shot -V

 

crm_mon[3602]: 2011/03/24_19:32:07 ERROR: unpack_resources: Resource start-up disabled since no STONITH resources have been defined

crm_mon[3602]: 2011/03/24_19:32:07 ERROR: unpack_resources: Either configure some or disable STONITH with the stonith-enabled option

crm_mon[3602]: 2011/03/24_19:32:07 ERROR: unpack_resources: NOTE: Clusters with shared data need STONITH to ensure data integrity

============

Last updated: Thu Mar 24 19:32:07 2011

Stack: openais

Current DC: openfiler01 - partition with quorum

Version: 1.1.2-c6b59218ee949eebff30e837ff6f3824ed0ab86b

2 Nodes configured, 2 expected votes

0 Resources configured.

============

 

Online: [ openfiler01 openfiler02 ]
 
 
 
[root@openfiler02 ~]# crm_mon
 
 

Configuration de corosync sur Openfiler01

[root@openfiler01 ~]# crm configure


crm(live)configure# property stonith-enabled="false"
crm(live)configure# property no-quorum-policy="ignore"
crm(live)configure# property default-action-timeout="240"


crm(live)configure# rsc_defaults $id="rsc-options" \
> resource-stickiness="100"


crm(live)configure# primitive ClusterIP ocf:heartbeat:IPaddr2 \
> params ip="192.168.1.10" cidr_netmask="24" \
> op monitor interval="30s"


crm(live)configure# primitive MetaFS ocf:heartbeat:Filesystem \
> params device="/dev/drbd0" directory="/meta" fstype="ext3"


crm(live)configure# primitive lvmdata ocf:heartbeat:LVM \
> params volgrpname="data"


crm(live)configure# primitive drbd_meta ocf:linbit:drbd \
> params drbd_resource="meta" \
> op monitor interval="15s"


crm(live)configure# primitive drbd_data ocf:linbit:drbd \
> params drbd_resource="data" \
> op monitor interval="15s"


crm(live)configure# primitive openfiler lsb:openfiler
crm(live)configure# primitive iscsi lsb:iscsi-target
crm(live)configure# primitive samba lsb:smb
crm(live)configure# primitive nfs lsb:nfs
crm(live)configure# primitive nfslock lsb:nfslock


crm(live)configure# group g_drbd drbd_meta drbd_data
crm(live)configure# group g_services MetaFS lvmdata openfiler ClusterIP iscsi samba nfs nfslock


crm(live)configure# ms ms_g_drbd g_drbd \
> meta master-max="1" master-node-max="1" \
> clone-max="2" clone-node-max="1" \
> notify="true"


crm(live)configure# colocation c_g_services_on_g_drbd inf: g_services ms_g_drbd:Master
crm(live)configure# order o_g_services_after_g_drbd inf: ms_g_drbd:promote g_services:start 


crm(live)configure# commit 
 
 
 

Remerciements

 
Un petit merci à Jérome pour avoir eu l'insoucience de me laisser accéder à son si précieux matériel afin d'y assouvir mes penchants schyzophréniques. 
 
Espérons seulement que les serveurs n'auront pas trop soufferts ...  ;-)
 

  

recherche01_small04.pngSources 

Drbd 

http://www.drbd.org/users-guide/s-configure-sync-rate.html 

  

 howto05_small.pngVous en pensez quoi ?


 

 

 

 

 

Enrichissez Yakakliker en y contribuant vous aussi.

 
Images (17)
Voir 1 - 6 sur 17 images | Voir tout
Aucune description
open216a....  Actions
Aucune description
open017a....  Actions
Aucune description
open016a....  Actions
Aucune description
open014a....  Actions
Aucune description
open013a....  Actions
Aucune description
open012a....  Actions
Commentaires (14)
Affichage de 14 commentaires sur 14: voir tout
Parametrages Corosync pour le HA iSCSi

https://forums.openfiler.com/viewtopic.php?pid=26962
Posté 20:28, 18 Avr 2012
removed:
primitive iscsi lsb:iscsi-target

added:
primitive iscsi ocf:heartbeat:iSCSITarget \
params iqn="iqn.2006-01.com.openfiler:tsn.01" tid="1" \
op monitor interval="10s"
primitive data_lun ocf:heartbeat:iSCSILogicalUnit \
params target_iqn="iqn.2006-01.com.openfiler:tsn.01" lun="1" path="/dev/data/filer" \
op monitor interval="10s"

and added the data_lun primitive to the g_services group.
Posté 20:28, 18 Avr 2012
bonjour

comment tester que la réplication de /meta s’exécute bien car sur le secondaire je n'ai rien dans meta. je ne dois pas pouvoir lorsqu'il est en secondaire mais au pouvoir lire les infos (il n'y a rien) c'est curieux.
Posté 13:23, 16 Mai 2012
En effet, c'est aussi une constatation que j'ai fait.

Le /meta se monte automatiquement lorsque le primary est hors ligne, ce qui est compréhensible vu qu'on déplace tous les fichiers conf d'openfiler dessus. Si il était monté sur le secondary, nous aurions quelques conflits.

Le mieux pour tester est de faire le classique coup de la panne via une sauvage déconnexion de l'alimentation. modifié 03:15, 17 Mai 2012
Posté 03:15, 17 Mai 2012
bonjour
il y a je pense un souci sur les fichiers à copier car dans meta car celui-ci ne peux remonter les partages créé par openfiler dans l'interface car /etc/fstab ne se retrouve pas dans meta donc pas de montage et pas de lien symbolique donc on n'a plus rien sur meta.
Posté 18:35, 18 Mai 2012
Je regarde ça ...
Posté 20:43, 18 Mai 2012
bonjour
j'ai un peu avancé et constaté des problèmes.
lors de la coupure d'un noeud primary il y a :

le network configuré dans openfiler pour les droits des répertoires a disparu
les points de montage disparaissent
si c'est la première fois le répertoire dans /mnt pour les partages ne se créent pas tout seul sur le second !!! bien que les données sont présentes sur la partition data.
voila
Posté 05:41, 20 Mai 2012
Je me remonte une maquette dès que possible, celle qui a servie pour ce tuto est en prod.

Petite note d'espoir, elle a passée les tests de crashs avec succès. Cependant, elle ne sert que pour des volumes iSCSi.

Je reviens vers toi dès que j'ai des réponses.
Posté 06:33, 20 Mai 2012
d'accord merci mais il est vrai que je n'ai pas encore fait l'essai en iscsi car je n'en avait pas sous la main.
peut être demain si j'ai le temps.
Posté 13:47, 20 Mai 2012
bonjour
j'ai bien testé openfiler en drbd HA avec du iscsi cela fonctionne correctement comme l'a précisé franck.

petite question. savez vous si l'on peut brancher un openfiler en fibre channel FC sur DAS EMC car je vais peut être en récupérer un.
merci d'avance.
Posté 17:19, 25 Mai 2012
Un DAS en FC !!!!!!!
Gros veinard !!!!!!

D'après le forum d'Openfiler, apparemment il gère le FC.
Maintenant, de mon côté ce n'est encore qu'un rêve ... modifié 06:07, 26 Mai 2012
Posté 06:07, 26 Mai 2012
bonjour, je test en ce moment openfiler et je me heurte à un problème dans la partie "modification de l'arborescence", j'ai le message suivant sur le openfiler01 et 02:

Starting openfiler: grep: /opt/openfiler/etc/httpd/conf/httpd.conf: No such file or directory
/bin/bash: /opt/openfiler/sbin/openfiler: No such file or directory
[FAILED]

merci de bien vouloir m'éclairer
Posté 12:35, 16 Jun 2014
As tu jeté un oeil ici :

http://www.howtoforge.com/forums/showthread.php?t=41914
Posté 13:36, 16 Jun 2014
http://community.spiceworks.com/topic/174231-openfiler-drdb-how-to-setup?page=3
Posté 13:38, 16 Jun 2014
Affichage de 14 commentaires sur 14: voir tout
Vous devez être connecté pour poster un commentaire.