오픈스택 스위프트 스토리지 설치 최종 정리

A. 기본 서버 구성

a. 공통

- OS 버젼 :  Ubuntu 16.04LTS 설치
- 네트워크 구성
 * Swift-Proxy Server : 192.168.0.192
 * Swift-Storage Server
    : SAIO 타입 : Proxy Server와 Storage Server를 하나의 서버에서 구동 (192.168.0.192)
    : Production Mode 타입 : Proxy Server와 Storage server가 별도의 서버에서 구동 (192.168.0.193로 구성)

1. 계정 설정
※ ubuntu에서 제공하는 swift 패키지를 설치하면 swift uid는 생성되지만 실질적인 계정이 생성되지는 않아서 문제가 발생하므로 가급적이면 swift 계정을 생성하여 그 계정으로 진행하는 것이 좋음.

etri@newproxy:~$ sudo adduser swift
[sudo] password for etri:
Adding user `swift' ...
Adding new group `swift' (1001) ...
Adding new user `swift' (1001) with group `swift' ...
Creating home directory `/home/swift' ...
Copying files from `/etc/skel' ...
Enter new UNIX password:
Retype new UNIX password:
passwd: password updated successfully
Changing the user information for swift
Enter the new value, or press ENTER for the default
        Full Name []: swift
        Room Number []:
        Work Phone []:
        Home Phone []:
        Other []:
Is the information correct? [Y/n] y
etri@newproxy:~$ sudo vi /etc/sudoers 해서 root밑에 swift도 sudoer로 추가하고 저장한 뒤 vi 종료
# User privilege specification
root    ALL=(ALL:ALL) ALL
swift ALL=(ALL:ALL) ALL

login as: swift
swift@192.168.0.82's password:

swift@newproxy:~$

2. network 설정

- proxy server

swift@newproxy:~$ sudo vi /etc/network/interface

auto ens33
iface ens33 inet static
address 192.168.0.192
netmask 255.255.255.0
gateway 192.168.0.1
dns-nameservers 8.8.8.8 4.4.4.4

- storage server

swift@newproxy:~$ sudo vi /etc/network/interface

auto ens33
iface ens33 inet static
address 192.168.0.193
netmask 255.255.255.0
gateway 192.168.0.1
dns-nameservers 8.8.8.8 4.4.4.4

3. 공통 패키지 설치 및 설정 파일을 저장할 디렉토리를 생성

swift@newproxy:~$ sudo apt-get install python-software-properties swift python-swiftclient openssh-server
swift@newproxy:~$ sudo mkdir -p /etc/swift

b. Proxy Server에서의 작업

1. 스위프트 서비스 구분 해시값 설정 (proxy server에서만 작업함)

swift 서비스 구분용 해시값을 만들기 위해 uuid를 사용하였고 이를 위해 uuid 패키지 설치

swift@newproxy:~$ sudo apt-get install uuid

swift@newproxy:~$ uuid
0b72e4ae-a70a-11e6-9dc4-6b7bc91f17f0  ==> 이 값을 swift-hash 값으로 사용하기 위해 클립보드로 저장해둔다.

swift@newproxy:~$ vi /etc/swift/swift.conf

[swift-hash]
# random unique strings that can never change (DO NOT LOSE)
swift_hash_path_prefix = 0b72e4ae-a70a-11e6-9dc4-6b7bc91f17f0  ==> uuid를 실행해서 얻은 guid 코드 문자열(무작위로 생성된 unique 한 스트링)

저장 후 종료

swift@newproxy:~$

2. proxy-server 동작을 위한 서비스 설치

swift@newproxy:~$ sudo apt-get install swift-proxy memcached

3. 프록시 서버캐싱을 위한 memcache 프록시 서버 설정

swift@newproxy:~$ sudo perl -pi -e "s/-l 127.0.0.1/-l 192.168.0.192/" /etc/memcached.conf

swift@newproxy:~$ sudo service memcached restart   ==> memcache 서비스 재시작

4. 스위프트의 프록시 서버 설정

swift@newproxy:~$ vi /etc/swift/proxy-server.conf

[DEFAULT]
bind_port = 8080
workers = 8
user = swift

[pipeline:main]
pipeline = healthcheck proxy-logging cache tempauth proxy-logging proxy-server

[app:proxy-server]
use = egg:swift#proxy
allow_account_management = true
account_autocreate = true

[filter:proxy-logging]
use = egg:swift#proxy_logging

[filter:tempauth]
use = egg:swift#tempauth
user_system_root = testpass .admin http://192.168.0.192:8080/v1/AUTH_system

[filter:healthcheck]
use = egg:swift#healthcheck

[filter:cache]
use = egg:swift#memcache
memcache_servers = 192.168.0.192:11211

5. 프록시 노드 링에 스토리지 노드를 추가하고 확인

swift@newproxy:~$ cd /etc/swift/
swift@newproxy:/etc/swift$ sudo swift-ring-builder account.builder create 18 2 1
swift@newproxy:/etc/swift$ sudo swift-ring-builder container.builder create 18 2 1
swift@newproxy:/etc/swift$ sudo swift-ring-builder object.builder create 18 2 1
swift@newproxy:/etc/swift$ sudo swift-ring-builder account.builder add r1z1-192.168.0.193:6002/sdb1 50
swift@newproxy:/etc/swift$ sudo swift-ring-builder account.builder add r1z1-192.168.0.194:6002/sdb1 50
swift@newproxy:/etc/swift$ sudo swift-ring-builder container.builder add r1z1-192.168.0.193:6001/sdb1 50
swift@newproxy:/etc/swift$ sudo swift-ring-builder container.builder add r1z1-192.168.0.194:6001/sdb1 50
swift@newproxy:/etc/swift$ sudo swift-ring-builder object.builder add r1z1-192.168.0.193:6000/sdb1 50
swift@newproxy:/etc/swift$ sudo swift-ring-builder object.builder add r1z1-192.168.0.194:6000/sdb1 50
swift@newproxy:/etc/swift$ sudo swift-ring-builder account.builder rebalance
swift@newproxy:/etc/swift$ sudo swift-ring-builder container.builder rebalance
swift@newproxy:/etc/swift$ sudo swift-ring-builder object.builder rebalance
swift@newproxy:/etc/swift$ sudo swift-ring-builder account.builder
swift@newproxy:/etc/swift$ sudo swift-ring-builder container.builder
swift@newproxy:/etc/swift$ sudo swift-ring-builder object.builder

6. proxy 서버 준비 완료

c. Storage Server 설정

1. loop device 생성 및 마운트

- chunk 생성
swift@newstorage:~$ truncate -s 30GB swift-disk1
swift@newstorage:~$ truncate -s 30GB swift-disk2

- xfs 생성
swift@newstorage:~$ mkfs.xfs swift-disk1
swift@newstorage:~$ mkfs.xfs swift-disk2

- fstab 수정

swift@newstorage:~$ sudo vi /etc/fstab

파일 맨 아래쪽에 다음 두줄 추가

/home/etri/swift-disk1  /srv/node/sdb1  xfs noatime,nodiratime,nobarrier,logbufs=8 0 0
/home/etri/swift-disk2  /srv/node/sdb2  xfs noatime,nodiratime,nobarrier,logbufs=8 0 0

- mount point 생성 및 마운트

swift@newstorage:~$ sudo mkdir -p /srv/node/sdb1 /srv/node/sdb2
swift@newstorage:~$ sudo mount /srv/node/sdb1; sudo mount /srv/node/sdb2

2. storage 노드 구성을 위한 패키지 설치

swift@newstorage:~$ sudo apt-get install swift-account swift-container swift-object swift-object-expirer xfsprogs

3. rsync service 설정

- /etc/rsyncd.conf 생성

swift@newstorage:~$ sudo vi /etc/rsyncd.conf

uid = swift
gid = swift
log file = /var/log/rsyncd.log
pid file = /var/run/rsyncd.pid
address = 192.168.0.193  # storage server ip

[account]
max connections = 2
path = /srv/node/
read only = false
lock file = /var/lock/account.lock

[container]
max connections = 2
path = /srv/node/
read only = false
lock file = /var/lock/container.lock

[object]
max connections = 2
path = /srv/node/
read only = false
lock file = /var/lock/object.lock

- rsync service를 활성화

swift@newstorage:~$ sudo vi /etc/default/rsync

.....
RSYNC_ENABLE=TRUE
.....

- 저장 후 rsync service 시작

swift@newstorage:~$ sudo service rsync start

4. swift storage configuration

- account-server.conf 생성 및 설정

swift@newstorage:~$ sudo vi /etc/swift/account-server.conf

[DEFAULT]
bind_ip = 192.168.0.193
workers = 2

[pipeline:main]
pipeline = account-server

[app:account-server]
use = egg:swift#account

[account-replicator]

[account-auditor]

[account-reaper]

- container-server.conf 생성 및 설정

swift@newstorage:~$ sudo vi /etc/swift/container-server.conf

[DEFAULT]
bind_ip = 192.168.0.193
workers = 2

[pipeline:main]
pipeline = container-server

[app:container-server]
use = egg:swift#container

[container-replicator]

[container-updater]

[container-auditor]

[container-sync]

- object-server.conf 생성 및 설정

swift@newstorage:~$ sudo vi /etc/swift/object-server.conf

[DEFAULT]
bind_ip = 192.168.0.193
workers = 2

[pipeline:main]
pipeline = object-server

[app:object-server]
use = egg:swift#object

[object-replicator]

[object-updater]

[object-auditor]

- object-expirer.conf 생성 및 설정

swift@newproxy:~$ sudo vi /etc/swift/object-expirer.conf

[DEFAULT]
user=swift

[object-expirer]
interval = 300

[pipeline:main]
pipeline = catch_errors cache proxy-server

[app:proxy-server]
use = egg:swift#proxy

[filter:cache]
use = egg:swift#memcache

[filter:catch_errors]
use = egg:swift#catch_errors

# See object-expirer.conf-sample for options

- container-reconciler.conf 생성 및 설정

swift@newproxy:~$ sudo vi /etc/swift/container-reconciler.conf

[DEFAULT]
user=swift

[container-reconciler]

[pipeline:main]
pipeline = catch_errors proxy-logging cache proxy-server

[app:proxy-server]
use = egg:swift#proxy
# See proxy-server.conf-sample for options

[filter:cache]
use = egg:swift#memcache
# See proxy-server.conf-sample for options

[filter:proxy-logging]
use = egg:swift#proxy_logging

[filter:catch_errors]
use = egg:swift#catch_errors
# See proxy-server.conf-sample for options

B. 스위프트 구성 및 서비스 시작

a. /etc/swift/swift.conf 복사

- Proxy Server에서 만들어둔 swift.conf를 동일하게 storage 서버에도 같은 위치에 복사한다

swift@newstorage:~$ sudo scp swift@192.168.0.192:/etc/swift/swift.conf /etc/swift/
swift@newstorage:~$ sudo scp swift@192.168.0.192:/etc/swift/*.ring.gz /etc/swift/

To Do :

1. 부팅후 service rsync start 구동시키기 ==> /etc/rc.local에 service rsync start 넣어서 일단 해결
2. storage server에서 404 not found 나는 정확한 이유 찾기 ==> 일단은 sudo chown swift:swift /srv/node/sdb1 /srv/node/sdb2 해서 지금은 오류는 나지 않음