iwinv object(오브젝트)스토리지 curl api 활용

iwinv Object(오브젝트) 스토리지 curl api 활용

테스트 OS : Ubuntu 20.04 LTS
curl 버전 : 7.68.0

# 참고 사항
$url : console.iwinv.kr -> 오브젝트 스토리지 -> 스토리지 관리 -> 인증키 관리 -> API {IDC} Endpoint
$user_id : console.iwinv.kr -> 오브젝트 스토리지 -> 스토리지 관리 -> 인증키 관리 -> User ID
$access_key : console.iwinv.kr -> 오브젝트 스토리지 -> 스토리지 관리 -> 인증키 관리 -> Access Key ID
$secret_key : console.iwinv.kr -> 오브젝트 스토리지 -> 스토리지 관리 -> 인증키 관리 -> Secret Key ID
$token : [토큰 키 생성]으로 생성된 X-Auth-Token 값

토큰 키 생성

# curl -i -g -X GET "$url/auth/" -H "X-Auth-User: $user_id:$access_key" -H "X-Auth-Key: $secret_key"
HTTP/1.1 204 No Content
X-Storage-Url: http://kridc.object.iwinv.krt/swift/v1/AUTH_5b688a1aa8e8a5d7d604e2806c9a2abab64f3fcf73adc6d06151a47c1f817224
X-Storage-Token: AUTH_rgwtk55000000356236383861316161386538613564376436303465323830366339613261626162363466336663663733616463366430363135316134376331663831373232343a4b494d5a304b52514439553239375146374248333a5799f31657becf4e75e85e7a8d6103960f10dee2c3365b6cd357a03fc1a87acd3ec766
X-Auth-Token: AUTH_rgwtk55000000356236383861316161386538613564376436303465323830366339613261626162363466336663663733616463366430363135316134376331663831373232343a4b494d5a304b52514439553239375146374248333a5799f31657becf4e75e85e7a8d6103960f10dee2c3365b6cd357a03fc1a87acd3ec766
X-Trans-Id: tx000000000000000007bda-005ee723ce-1461cdd-default
X-Openstack-Request-Id: tx000000000000000007bda-005ee723ce-1461cdd-default
Content-Type: application/json; charset=utf-8
Date: Mon, 15 Jun 2020 07:31:26 GMT

버킷 생성(버킷 이름 : iwinv)

# curl -i -X PUT $url/swift/v1/AUTH_$user_id/iwinv -H "X-Auth-Token: $token"
HTTP/1.1 201 Created
Content-Length: 0
X-Trans-Id: tx000000000000000008528-005ee82787-1461cdd-default
X-Openstack-Request-Id: tx000000000000000008528-005ee82787-1461cdd-default
Accept-Ranges: bytes
Content-Type: text/plain; charset=utf-8
Date: Tue, 16 Jun 2020 01:59:35 GMT

버킷 삭제(버킷 이름 : iwinv)

# curl -i -X DELETE $url/swift/v1/AUTH_$user_id/iwinv -H "X-Auth-Token: $token"
HTTP/1.1 204 No Content
X-Trans-Id: tx000000000000000008529-005ee82857-1461cdd-default
X-Openstack-Request-Id: tx000000000000000008529-005ee82857-1461cdd-default
Accept-Ranges: bytes
Content-Type: text/plain; charset=utf-8
Date: Tue, 16 Jun 2020 02:03:03 GMT

버킷 리스트 확인

# curl -i -X GET $url/swift/v1/AUTH_$user_id/ -H "X-Auth-Token: $token"
HTTP/1.1 200 OK
X-Timestamp: 1592273237.04130
X-Account-Container-Count: 2
X-Account-Object-Count: 0
X-Account-Bytes-Used: 0
X-Account-Bytes-Used-Actual: 0
X-Account-Storage-Policy-Default-Placement-Container-Count: 2
X-Account-Storage-Policy-Default-Placement-Object-Count: 0
X-Account-Storage-Policy-Default-Placement-Bytes-Used: 0
X-Account-Storage-Policy-Default-Placement-Bytes-Used-Actual: 0
X-Account-Storage-Policy-Temporary-Container-Count: 0
X-Account-Storage-Policy-Temporary-Object-Count: 0
X-Account-Storage-Policy-Temporary-Bytes-Used: 0
X-Account-Storage-Policy-Temporary-Bytes-Used-Actual: 0
Accept-Ranges: bytes
X-Trans-Id: tx00000000000000000853b-005ee82955-1461cdd-default
X-Openstack-Request-Id: tx00000000000000000853b-005ee82955-1461cdd-default
Content-Type: text/plain; charset=utf-8
Content-Length: 12
Date: Tue, 16 Jun 2020 02:07:17 GMT

iwinv
iwinv2

# curl -X GET $url/swift/v1/AUTH_$user_id/?format=json -H "X-Auth-Token: $token"
[{"name":"iwinv","count":0,"bytes":0},{"name":"iwinv2","count":0,"bytes":0}

버킷 메타데이타 설정

# curl -i -X POST $url/swift/v1/AUTH_$user_id/iwinv -H "X-Auth-Token: $token" -H "X-Container-Meta-Iwinv: test"
HTTP/1.1 204 No Content
X-Trans-Id: tx00000000000000000853f-005ee82af8-1461cdd-default
X-Openstack-Request-Id: tx00000000000000000853f-005ee82af8-1461cdd-default
Content-Type: text/plain; charset=utf-8
Date: Tue, 16 Jun 2020 02:14:16 GMT

버킷 메타데이타 확인

# curl --head $url/swift/v1/AUTH_$user_id/iwinv -H "X-Auth-Token: $token"
HTTP/1.1 204 No Content
X-Timestamp: 1592273218.15213
X-Container-Object-Count: 0
X-Container-Bytes-Used: 0
X-Container-Bytes-Used-Actual: 0
X-Storage-Policy: default-placement
X-Storage-Class: STANDARD
X-Container-Meta-Iwinv: test
Last-Modified: Tue, 16 Jun 2020 02:14:16 GMT
X-Trans-Id: tx000000000000000008551-005ee82b79-1461cdd-default
X-Openstack-Request-Id: tx000000000000000008551-005ee82b79-1461cdd-default
Accept-Ranges: bytes
Content-Type: text/plain; charset=utf-8
Date: Tue, 16 Jun 2020 02:16:25 GMT

버킷 메타데이타 제거

# curl -i -X POST $url/swift/v1/AUTH_$user_id/iwinv -H "X-Auth-Token: $token" -H "X-Remove-Container-Meta-Iwinv: test"
HTTP/1.1 204 No Content
X-Trans-Id: tx000000000000000008552-005ee82bfc-1461cdd-default
X-Openstack-Request-Id: tx000000000000000008552-005ee82bfc-1461cdd-default
Content-Type: text/plain; charset=utf-8
Date: Tue, 16 Jun 2020 02:18:36 GMT

버킷 정책 설정

# curl -i -X POST $url/swift/v1/AUTH_$user_id/iwinv -H "X-Auth-Token: $token" -H "X-Remove-Container-Read: .r:*,.rlistings"
HTTP/1.1 204 No Content
X-Trans-Id: tx000000000000000008559-005ee82d3e-1461cdd-default
X-Openstack-Request-Id: tx000000000000000008559-005ee82d3e-1461cdd-default
Content-Type: text/plain; charset=utf-8
Date: Tue, 16 Jun 2020 02:23:58 GMT

버킷 정책 확인

# curl --head $url/swift/v1/AUTH_$user_id/iwinv -H "X-Auth-Token: $token"
HTTP/1.1 204 No Content
X-Timestamp: 1592273218.15213
X-Container-Object-Count: 0
X-Container-Bytes-Used: 0
X-Container-Bytes-Used-Actual: 0
X-Container-Read: .r:*,.rlistings
X-Storage-Policy: default-placement
X-Storage-Class: STANDARD
Last-Modified: Tue, 16 Jun 2020 02:23:58 GMT
X-Trans-Id: tx00000000000000000855a-005ee82d45-1461cdd-default
X-Openstack-Request-Id: tx00000000000000000855a-005ee82d45-1461cdd-default
Accept-Ranges: bytes
Content-Type: text/plain; charset=utf-8
Date: Tue, 16 Jun 2020 02:24:05 GMT

버킷 정책 삭제

# curl -i -X POST $url/swift/v1/AUTH_$user_id/iwinv -H "X-Auth-Token: $token" -H "X-Container-Read: ,"
HTTP/1.1 204 No Content
X-Trans-Id: tx00000000000000000855e-005ee82d73-1461cdd-default
X-Openstack-Request-Id: tx00000000000000000855e-005ee82d73-1461cdd-default
Content-Type: text/plain; charset=utf-8
Date: Tue, 16 Jun 2020 02:24:51 GMT

버킷 versioning 설정

# curl -i -X PUT $url/swift/v1/AUTH_$user_id/iwinv -H "X-Auth-Token: $token" -H "X-Versions-Location: iwinv2"
HTTP/1.1 202 Accepted
Content-Length: 0
X-Trans-Id: tx000000000000000008562-005ee82f65-1461cdd-default
X-Openstack-Request-Id: tx000000000000000008562-005ee82f65-1461cdd-default
Accept-Ranges: bytes
Content-Type: text/plain; charset=utf-8
Date: Tue, 16 Jun 2020 02:33:09 GMT

# 간단히 설명하면 중복되는 파일이 있을 경우 다른 버킷에 복사후 보관하며 상단의 예시는 iwinv 버킷에 중복 되는 파일이 있을경우 iwinv2에 생성 날짜로 보관.

버킷 versioning 설정 확인

# curl --head $url/swift/v1/AUTH_$user_id/iwinv -H "X-Auth-Token: $token"
HTTP/1.1 204 No Content
X-Timestamp: 1592273218.15213
X-Container-Object-Count: 0
X-Container-Bytes-Used: 0
X-Container-Bytes-Used-Actual: 0
X-Storage-Policy: default-placement
X-Storage-Class: STANDARD
X-Versions-Location: iwinv2
Last-Modified: Tue, 16 Jun 2020 02:33:09 GMT
X-Trans-Id: tx000000000000000008563-005ee82fde-1461cdd-default
X-Openstack-Request-Id: tx000000000000000008563-005ee82fde-1461cdd-default
Accept-Ranges: bytes
Content-Type: text/plain; charset=utf-8
Date: Tue, 16 Jun 2020 02:35:10 GMT

버킷 versioning 설정 삭제

# curl -i -X PUT $url/swift/v1/AUTH_$user_id/iwinv -H "X-Auth-Token: $token" -H "X-Remove-Versions-Location: iwinv2"
HTTP/1.1 202 Accepted
Content-Length: 0
X-Trans-Id: tx000000000000000008564-005ee83008-1461cdd-default
X-Openstack-Request-Id: tx000000000000000008564-005ee83008-1461cdd-default
Accept-Ranges: bytes
Content-Type: text/plain; charset=utf-8
Date: Tue, 16 Jun 2020 02:35:52 GMT

오브젝트 업로드(iwinv 버킷에 파일 new_object.txt 업로드)

# curl -i -X PUT -T new_object.txt $url/swift/v1/AUTH_$user_id/iwinv/ -H "X-Auth-Token: $token"
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0HTTP/1.1 100 CONTINUE

100 5 0 0 100 5 0 24 --:--:-- --:--:-- --:--:-- 24HTTP/1.1 201 Created
etag: d8e8fca2dc0f896fd7cb4cb0031ba249
Last-Modified: Tue, 16 Jun 2020 06:15:28 GMT
X-Trans-Id: tx000000000000000000033-005ee86380-150fda5-default
X-Openstack-Request-Id: tx000000000000000000033-005ee86380-150fda5-default
Content-Type: text/plain; charset=utf-8
Content-Length: 0
Date: Tue, 16 Jun 2020 06:15:28 GMT

100 5 0 0 100 5 0 20 --:--:-- --:--:-- --:--:-- 20

오브젝트 다운로드(iwinv 버킷에서 파일 new_object.txt 를 new_object_download.txt로 다운로드)

# curl -s -S -X GET $url/swift/v1/AUTH_$user_id/iwinv/new_object.txt -H "X-Auth-Token: $token" -o new_object_download.txt

오브젝트 복사(PUT, COPYT 메소드 사용하여 버킷 지정 복사)

# curl -i -X PUT $url/swift/v1/AUTH_$user_id/iwinv/new_object_copy.txt -H "X-Auth-Token: $token" -H "X-Copy-From: /iwinv/new_object.txt"
HTTP/1.1 201 Created
etag: d8e8fca2dc0f896fd7cb4cb0031ba249
Last-Modified: Tue, 16 Jun 2020 06:31:22 GMT
X-Copied-From: iwinv/new_object.txt
X-Copied-From-Account: 5b688a1aa8e8a5d7d604e2806c9a2abab64f3fcf73adc6d06151a47c1f817224
X-Copied-From-Last-Modified: Tue, 16 Jun 2020 06:15:28 GMT
X-Trans-Id: tx000000000000000000053-005ee8673a-150fda5-default
X-Openstack-Request-Id: tx000000000000000000053-005ee8673a-150fda5-default
Content-Type: text/plain
Content-Length: 0
Date: Tue, 16 Jun 2020 06:31:23 GMT

# curl -i -X COPY $url/swift/v1/AUTH_$user_id/iwinv/new_object.txt -H "X-Auth-Token: $token" -H "Destination: /iwinv2/new_object_copy.txt"
HTTP/1.1 201 Created
etag: d8e8fca2dc0f896fd7cb4cb0031ba249
Last-Modified: Tue, 16 Jun 2020 06:34:38 GMT
X-Copied-From: iwinv/new_object.txt
X-Copied-From-Account: 5b688a1aa8e8a5d7d604e2806c9a2abab64f3fcf73adc6d06151a47c1f817224
X-Copied-From-Last-Modified: Tue, 16 Jun 2020 06:15:28 GMT
X-Trans-Id: tx000000000000000000054-005ee867fe-150fda5-default
X-Openstack-Request-Id: tx000000000000000000054-005ee867fe-150fda5-default
Content-Type: text/plain
Content-Length: 0
Date: Tue, 16 Jun 2020 06:34:38 GMT

오브젝트 5GByte 이상 파일 3개로 분리후 1개의 파일 형식으로 업로드(X-Object-Manaifest 설정)

# split -b {byte} {원본 파일명} temp

# curl -i -X PUT $url/swift/v1/AUTH_$user_id/iwinv/large_file -H "X-Auth-Token: $token" -H "Content-Type: application/directory" -H "Content-Length: 0"
HTTP/1.1 201 Created
etag: d41d8cd98f00b204e9800998ecf8427e
Last-Modified: Tue, 16 Jun 2020 07:59:11 GMT
X-Trans-Id: tx0000000000000000002c3-005ee87bcf-150fda5-default
X-Openstack-Request-Id: tx0000000000000000002c3-005ee87bcf-150fda5-default
Content-Type: text/plain; charset=utf-8
Content-Length: 0
Date: Tue, 16 Jun 2020 07:59:12 GMT

# curl -i -X PUT $url/swift/v1/AUTH_$user_id/iwinv/large_file/1 -H "X-Auth-Token: $token" -T tempaa
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0HTTP/1.1 100 CONTINUE

99 3718M 0 0 99 3705M 0 28.0M 0:02:12 0:02:11 0:00:01 48.7MHTTP/1.1 201 Created
etag: ccae3b814a60eba635b4d30eb80b5339
Last-Modified: Tue, 16 Jun 2020 08:02:56 GMT
X-Trans-Id: tx0000000000000000002c4-005ee87c2b-150fda5-default
X-Openstack-Request-Id: tx0000000000000000002c4-005ee87c2b-150fda5-default
Content-Type: text/plain; charset=utf-8
Content-Length: 0
Date: Tue, 16 Jun 2020 08:02:56 GMT

100 3718M 0 0 100 3718M 0 27.9M 0:02:13 0:02:13 --:--:-- 36.5M

# curl -i -X PUT $url/swift/v1/AUTH_$user_id/iwinv/large_file/2 -H "X-Auth-Token: $token" -T tempab
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0HTTP/1.1 100 CONTINUE

99 3718M 0 0 99 3684M 0 45.9M 0:01:20 0:01:20 --:--:-- 45.8MHTTP/1.1 201 Created
etag: 7f90c42e1a3f16bb37f5c84f65c36e70
Last-Modified: Tue, 16 Jun 2020 08:05:07 GMT
X-Trans-Id: tx0000000000000000002c5-005ee87ce2-150fda5-default
X-Openstack-Request-Id: tx0000000000000000002c5-005ee87ce2-150fda5-default
Content-Type: text/plain; charset=utf-8
Content-Length: 0
Date: Tue, 16 Jun 2020 08:05:07 GMT

100 3718M 0 0 100 3718M 0 45.6M 0:01:21 0:01:21 --:--:-- 39.5M

# curl -i -X PUT $url/swift/v1/AUTH_$user_id/iwinv/large_file/3 -H "X-Auth-Token: $token" -T tempac
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0HTTP/1.1 100 CONTINUE

99 3718M 0 0 99 3697M 0 45.8M 0:01:21 0:01:20 0:00:01 48.0MHTTP/1.1 201 Created
etag: 63d6e1a125cb98ae177743019b20686d
Last-Modified: Tue, 16 Jun 2020 08:07:34 GMT
X-Trans-Id: tx0000000000000000002c6-005ee87d74-150fda5-default
X-Openstack-Request-Id: tx0000000000000000002c6-005ee87d74-150fda5-default
Content-Type: text/plain; charset=utf-8
Content-Length: 0
Date: Tue, 16 Jun 2020 08:07:34 GMT

100 3718M 0 0 100 3718M 0 45.7M 0:01:21 0:01:21 --:--:-- 44.5M

# curl -i -X PUT $url/swift/v1/AUTH_$user_id/iwinv/large_file -H "X-Auth-Token: $token" -H "X-Object-Manifest: iwinv/large_file/" --data-binary ''
HTTP/1.1 201 Created
etag: d41d8cd98f00b204e9800998ecf8427e
Last-Modified: Tue, 16 Jun 2020 08:11:51 GMT
X-Trans-Id: tx0000000000000000002c9-005ee87ec7-150fda5-default
X-Openstack-Request-Id: tx0000000000000000002c9-005ee87ec7-150fda5-default
Content-Type: text/plain; charset=utf-8
Content-Length: 0
Date: Tue, 16 Jun 2020 08:11:51 GMT

오브젝트 5GByte 이상 파일 다운로드후 원본과 비교(X-Object-Manaifest 설정 파일 다운로드)

# curl -X GET $url/swift/v1/AUTH_$user_id/iwinv/large_file -H "X-Auth-Token: $token" -O
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 10.8G 100 10.8G 0 0 91.2M 0 0:02:02 0:02:02 --:--:-- 90.2M

# md5sum large_file
6b0060afa264fae5acb7b6fd81bbf497 large_file

# md5sum {원본 파일명}
6b0060afa264fae5acb7b6fd81bbf497 {원본 파일명}

오브젝트 메타데이타 설정

# curl -i -X POST $url/swift/v1/AUTH_$user_id/iwinv/new_object.txt -H "X-Auth-Token: $token" -H "X-Container-Meta-Iwinv: test"
HTTP/1.1 202 Accepted
Content-Length: 0
Accept-Ranges: bytes
X-Trans-Id: tx0000000000000000002cc-005ee881b7-150fda5-default
X-Openstack-Request-Id: tx0000000000000000002cc-005ee881b7-150fda5-default
Content-Type: text/plain; charset=utf-8
Date: Tue, 16 Jun 2020 08:24:23 GMT

오브젝트 메타데이타 삭제

$ curl -i -X POST $url/swift/v1/AUTH_$user_id/iwinv/new_object.txt -H "X-Auth-Token: $token" -H "X-Remove-Container-Meta-Iwinv: test"
HTTP/1.1 202 Accepted
Content-Length: 0
Accept-Ranges: bytes
X-Trans-Id: tx0000000000000000002cd-005ee882f9-150fda5-default
X-Openstack-Request-Id: tx0000000000000000002cd-005ee882f9-150fda5-default
Content-Type: text/plain; charset=utf-8
Date: Tue, 16 Jun 2020 08:29:46 GMT