오브젝트스토리지

송용근 에반젤리스트

iwinv Object Storage - Java SDK (with Maven)

                                                                                                         

                                                                                                                                         

 

 

해당 메뉴얼은 AWS S3에서 제공하는 Java SDK를 이용하여 iwinv Object Storage를 사용하는 방법을 설명합니다.  (Apache Maven 사용)

 

참고문서 

https://docs.aws.amazon.com/ko_kr/sdk-for-java/v1/developer-guide

https://github.com/awsdocs/aws-doc-sdk-examples/tree/master/java

 

 

Java SDK 사용하기


테스트 OS : Ubuntu 18.04 LTS

 

[인증키 확인]

Endpoint -> 오브젝트 스토리지 -> 스토리지 관리 -> 인증키 관리 -> API {IDC} Endpoint 

Access_Key  : console.iwinv.kr -> 오브젝트 스토리지 -> 스토리지 관리 -> 인증키 관리 -> Access Key ID

Secret_Key   : console.iwinv.kr -> 오브젝트 스토리지 -> 스토리지 관리 -> 인증키 관리 -> Secret Key ID

 

 

 

 maven 설치하기 

sudo apt install maven

 

maven 패키지 설치  

mvn -B archetype:generate -DarchetypeGroupId=org.apache.maven.archetypes -DgroupId={groupId} -DartifactId={artifactld}


# {groupId} =  패키지 네임스페이스 지정   ex) -DgroupId=org.example.basicapp  => src/main/java/org/example/basicapp

# {artifactId} =  패키지 고유ID (해당 ID값으로 디렉토리가 생성됩니다)       

ex) mvn -B archetype:generate   -DarchetypeGroupId=org.apache.maven.archetypes   -DgroupId=org.example.basicapp -DartifactId=myapp


< 예제 프로젝트 구조 >


├──src
│ ├─main
│ │ └──java
│ │    └──org
│ │       └──example
│ │          └──basicapp
│ │             └──App.java
│ ├─test
│    └──java
│       └──org
│          └──example
│             └──basicapp
│                └──AppTest.java
└──pom.xml

 

 

코드작성하기 

위에 설명한 예제 프로젝트 구조를 기준으로  /src/main/java/org/example 아래에  원하는 java 코드를 작성합니다.  (코드작성시 반드시 코드안에 지정된 class명으로 파일이름을 지정해야합니다.)

ex) CreateBucket.java , ListBuckets.java

 

 

pom.xml 파일 수정하기 

java코드 작성 후 pom.xml 파일을 수정합니다. 

 

java 코드 실행시 main class를 실행하도록 코드에 맞는 main class를 지정합니다. ( maven-shade-plugin 사용)

 

# pom.xml 예제 (프로젝트 구조는 위에 예제와 동일하며 코드는 CreateBucket.java 코드 작성)

 

============================================================================

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>org.example.basicapp</groupId>
  <artifactId>myapp</artifactId>
  <packaging>jar</packaging>
  <version>1.0-SNAPSHOT</version>
  <name>myapp</name>
  <url>http://maven.apache.org</url>
  <dependencies>
    <dependency>
     <groupId>junit</groupId>
     <artifactId>junit</artifactId>
     <version>3.8.1</version>
     <scope>test</scope>
   </dependency>
   <dependency>
     <groupId>com.amazonaws</groupId>
     <artifactId>aws-java-sdk-s3</artifactId>
     <version>1.11.327</version>
   </dependency>
 </dependencies>
    <build>
       <plugins>
           <plugin>
               <groupId>org.apache.maven.plugins</groupId>
               <artifactId>maven-shade-plugin</artifactId>
               <version>2.1</version>
               <executions>
                   <execution>
                       <phase>package</phase>
                       <goals>
                           <goal>shade</goal>
                       </goals>
                       <configuration>
                               <transformers>
                               <transformer
                                   implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
                                   <mainClass>aws.example.s3.ListBuckets</mainClass>
                                   </transformer>
                               </transformers>
                       </configuration>
                   </execution>
               </executions>
           </plugin>
       </plugins>
   </build>
</project>

============================================================================

 

빌드하기  

pom.xml 파일이 있는 위치에서 빌드를 진행합니다.

mvn package 명려어 입력시 빌드가 되고 /target 디렉토리가 생성되며 해당 디렉토리에  jar파일이 생성됩니다.

 

 

실행하기  

java -jar 명령어로 jar 파일을 실행합니다. 

ex) java -jar myapp-1.0-SNAPSHOT.java

 

 

 

# 예제코드

 

버킷 생성

package aws.example.s3;

import com.amazonaws.regions.Regions;
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.AmazonS3ClientBuilder;
import com.amazonaws.services.s3.model.AmazonS3Exception;
import com.amazonaws.services.s3.model.Bucket;
import com.amazonaws.auth.BasicAWSCredentials;
import com.amazonaws.auth.AWSStaticCredentialsProvider;
import com.amazonaws.client.builder.AwsClientBuilder;


import java.util.List;

public class CreateBucket {
    public static Bucket getBucket(String bucket_name) {
        final AmazonS3 s3 = AmazonS3ClientBuilder.standard()
        .withEndpointConfiguration(new AwsClientBuilder.EndpointConfiguration("Endpoint", "default"))
        .withCredentials(new AWSStaticCredentialsProvider(new BasicAWSCredentials("Access_Key ", "Secret_Key")))
        .build();
        Bucket named_bucket = null;
        List buckets = s3.listBuckets();
        for (Bucket b : buckets) {
            if (b.getName().equals(bucket_name)) {
                named_bucket = b;
            }
        }
        return named_bucket;
}

public static Bucket createBucket(String bucket_name) {
    final AmazonS3 s3 = AmazonS3ClientBuilder.standard()
    .withEndpointConfiguration(new AwsClientBuilder.EndpointConfiguration("Endpoint", "default"))
    .withCredentials(new AWSStaticCredentialsProvider(new BasicAWSCredentials("Access_Key ", "Secret_Key")))
    .build();

    Bucket b = null;
    if (s3.doesBucketExistV2(bucket_name)) {
        System.out.format("Bucket %s already exists.\n", bucket_name);
        b = getBucket(bucket_name);
    } else {
        try {
            b = s3.createBucket(bucket_name);
        } catch (AmazonS3Exception e) {
            System.err.println(e.getErrorMessage());
        }
    }
    return b;
}

public static void main(String[] args) {
    final String USAGE = "\n" +
            "CreateBucket - create an S3 bucket\n\n" +
            "Usage: CreateBucket \n\n" +
            "Where:\n" +
            " bucketname - the name of the bucket to create.\n\n" +
            "The bucket name must be unique, or an error will result.\n";

    if (args.length < 1) {
        System.out.println(USAGE);
        System.exit(1);
     }

     String bucket_name = args[0];

     System.out.format("\nCreating S3 bucket: %s\n", bucket_name);
     Bucket b = createBucket(bucket_name);
     if (b == null) {
         System.out.println("Error creating bucket!\n");
     } else {
         System.out.println("Done!\n");
     }
  }
}

 

 

# 실행 명령어 

java -jar 파일명.jar [Bucket_Name]

 

 

버킷 삭제

package aws.example.s3;

import com.amazonaws.AmazonServiceException;
import com.amazonaws.regions.Regions;
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.AmazonS3ClientBuilder;
import com.amazonaws.services.s3.model.*;
import com.amazonaws.auth.BasicAWSCredentials;
import com.amazonaws.auth.AWSStaticCredentialsProvider;
import com.amazonaws.client.builder.AwsClientBuilder;

import java.util.Iterator;


public class DeleteBucket {
   public static void main(String[] args) {
       final String USAGE = "\n" +
               "To run this example, supply the name of an S3 bucket\n" +
               "\n" +
               "Ex: DeleteBucket \n";

       if (args.length < 1) {
           System.out.println(USAGE);
           System.exit(1);
       }

       String bucket_name = args[0];

       System.out.println("Deleting S3 bucket: " + bucket_name);
        final AmazonS3 s3 = AmazonS3ClientBuilder.standard()
        .withEndpointConfiguration(new AwsClientBuilder.EndpointConfiguration("Endpoint", "default"))
        .withCredentials(new AWSStaticCredentialsProvider(new BasicAWSCredentials("Access_Key", "Secret_Key")))
        .build();
        try {
            System.out.println(" - removing objects from bucket");
            ObjectListing object_listing = s3.listObjects(bucket_name);
            while (true) {
                for (Iterator<?> iterator =
                     object_listing.getObjectSummaries().iterator();
                     iterator.hasNext(); ) {
                    S3ObjectSummary summary = (S3ObjectSummary) iterator.next();
                    s3.deleteObject(bucket_name, summary.getKey());
                }


                if (object_listing.isTruncated()) {
                    object_listing = s3.listNextBatchOfObjects(object_listing);
                } else {
                    break;
                }
            }

            System.out.println(" - removing versions from bucket");
            VersionListing version_listing = s3.listVersions(
                    new ListVersionsRequest().withBucketName(bucket_name));
            while (true) {
                for (Iterator<?> iterator =
                     version_listing.getVersionSummaries().iterator();
                     iterator.hasNext(); ) {
                     S3VersionSummary vs = (S3VersionSummary) iterator.next();
                     s3.deleteVersion(
                             bucket_name, vs.getKey(), vs.getVersionId());
                }

                if (version_listing.isTruncated()) {
                    version_listing = s3.listNextBatchOfVersions(
                            version_listing);
                } else {
                    break;
                }
             }

             System.out.println(" OK, bucket ready to delete!");
             s3.deleteBucket(bucket_name);
         } catch (AmazonServiceException e) {
             System.err.println(e.getErrorMessage());
             System.exit(1);
         }
         System.out.println("Done!");
     }
}

 

# 실행 명령어 

java -jar 파일명.jar [Bucket_Name]

 

 

버킷 리스트 확인

package aws.example.s3;

import com.amazonaws.regions.Regions;
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.AmazonS3ClientBuilder;
import com.amazonaws.client.builder.AwsClientBuilder;
import com.amazonaws.services.s3.model.Bucket;
import com.amazonaws.auth.BasicAWSCredentials;
import com.amazonaws.auth.AWSStaticCredentialsProvider;
import java.util.List;

public class ListBuckets {
   public static void main(String[] args) {
       final AmazonS3 s3 = AmazonS3ClientBuilder.standard()
       .withEndpointConfiguration(new AwsClientBuilder.EndpointConfiguration("Endpoint", "default"))
       .withCredentials(new AWSStaticCredentialsProvider(new BasicAWSCredentials("Access_Key", "Secret_Key")))
       .build();
       List buckets = s3.listBuckets();
       System.out.println("Your Amazon S3 buckets are:");
       for (Bucket b : buckets) {
            System.out.println("* " + b.getName());
       }
   }
}

 

 

오브젝트 업로드

package aws.example.s3;

import com.amazonaws.AmazonServiceException;
import com.amazonaws.regions.Regions;
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.AmazonS3ClientBuilder;
import com.amazonaws.auth.BasicAWSCredentials;
import com.amazonaws.auth.AWSStaticCredentialsProvider;
import com.amazonaws.client.builder.AwsClientBuilder;

 
import java.io.File;
import java.nio.file.Paths;

public class PutObject {
    public static void main(String[] args) {
        final String USAGE = "\n" +
                "To run this example, supply the name of an S3 bucket and a file to\n" +
                "upload to it.\n" +
                "\n" +
                "Ex: PutObject \n";

        if (args.length < 2) {
            System.out.println(USAGE);
            System.exit(1);
        }

        String bucket_name = args[0];
        String file_path = args[1];
        String key_name = Paths.get(file_path).getFileName().toString();

        System.out.format("Uploading %s to S3 bucket %s...\n", file_path, bucket_name);
        final AmazonS3 s3 = AmazonS3ClientBuilder.standard()
        .withEndpointConfiguration(new AwsClientBuilder.EndpointConfiguration("Endpoint", "default"))
        .withCredentials(new AWSStaticCredentialsProvider(new BasicAWSCredentials("Access_Key", "Secret_Key")))
        .build();
        try {
            s3.putObject(bucket_name, key_name, new File(file_path));
        } catch (AmazonServiceException e) {
            System.err.println(e.getErrorMessage());
            System.exit(1);
        }
        System.out.println("Done!");
    }
}

 

# 실행 명령어 

java -jar 파일명.jar [Bucket_Name] [Path/File_Name]

ex) java -jar myapp-1.0-SNAPSHOT.java iwinv-bucket /home/ubuntu/test.txt 

 

 

오브젝트 다운로드

package aws.example.s3;

import com.amazonaws.AmazonServiceException;
import com.amazonaws.regions.Regions;
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.AmazonS3ClientBuilder;
import com.amazonaws.services.s3.model.S3Object;
import com.amazonaws.services.s3.model.S3ObjectInputStream;
import com.amazonaws.auth.BasicAWSCredentials;
import com.amazonaws.auth.AWSStaticCredentialsProvider;
import com.amazonaws.client.builder.AwsClientBuilder;


import java.io.File;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.IOException;

public class GetObject {
    public static void main(String[] args) {
        final String USAGE = "\n" +
                "To run this example, supply the name of an S3 bucket and object to\n" +
                "download from it.\n" +
                "\n" +
                "Ex: GetObject \n";

        if (args.length < 2) {
            System.out.println(USAGE);
            System.exit(1);
        }

        String bucket_name = args[0];
        String key_name = args[1];

        System.out.format("Downloading %s from S3 bucket %s...\n", key_name, bucket_name);
        final AmazonS3 s3 = AmazonS3ClientBuilder.standard()
        .withEndpointConfiguration(new AwsClientBuilder.EndpointConfiguration("Endpoint", "default"))
        .withCredentials(new AWSStaticCredentialsProvider(new BasicAWSCredentials("Access_Key", "Secret_Key")))
        .build();

        try {
            S3Object o = s3.getObject(bucket_name, key_name);
            S3ObjectInputStream s3is = o.getObjectContent();
            FileOutputStream fos = new FileOutputStream(new File(key_name));
            byte[] read_buf = new byte[1024];
            int read_len = 0;
            while ((read_len = s3is.read(read_buf)) > 0) {
                fos.write(read_buf, 0, read_len);
            }
            s3is.close();
            fos.close();
        } catch (AmazonServiceException e) {
            System.err.println(e.getErrorMessage());
            System.exit(1);
        } catch (FileNotFoundException e) {
            System.err.println(e.getMessage());
            System.exit(1);
        } catch (IOException e) {
            System.err.println(e.getMessage());
            System.exit(1);
        }
        System.out.println("Done!");
    }
}

 

# 실행 명령어 

java -jar 파일명.jar [Bucket_Name] [Object_Name]

 

 

오브젝트 리스트 확인

package aws.example.s3;

import com.amazonaws.regions.Regions;
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.AmazonS3ClientBuilder;
import com.amazonaws.services.s3.model.ListObjectsV2Result;
import com.amazonaws.services.s3.model.S3ObjectSummary;
import com.amazonaws.auth.BasicAWSCredentials;
import com.amazonaws.auth.AWSStaticCredentialsProvider;
import com.amazonaws.client.builder.AwsClientBuilder;
import java.util.List;

public class ListObjects {
    public static void main(String[] args) {
        final String USAGE = "\n" +
                "To run this example, supply the name of a bucket to list!\n" +
                "\n" +
                "Ex: ListObjects \n";

        if (args.length < 1) {
            System.out.println(USAGE);
            System.exit(1);
        }

        String bucket_name = args[0];

        System.out.format("Objects in S3 bucket %s:\n", bucket_name);
        final AmazonS3 s3 = AmazonS3ClientBuilder.standard()
        .withEndpointConfiguration(new AwsClientBuilder.EndpointConfiguration("Endpoint", "default"))
        .withCredentials(new AWSStaticCredentialsProvider(new BasicAWSCredentials("Access_Key", "Secret_Key")))
        .build();
        ListObjectsV2Result result = s3.listObjectsV2(bucket_name);
        List objects = result.getObjectSummaries();
        for (S3ObjectSummary os : objects) {
             System.out.println("* " + os.getKey());
        }
    }
}

 

# 실행 명령어 

java -jar 파일명.jar [Bucket_Name]

 

 

오브젝트 삭제

package aws.example.s3;

import com.amazonaws.AmazonServiceException;
import com.amazonaws.regions.Regions;
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.AmazonS3ClientBuilder;
import com.amazonaws.auth.BasicAWSCredentials;
import com.amazonaws.auth.AWSStaticCredentialsProvider;
import com.amazonaws.client.builder.AwsClientBuilder;


public class DeleteObject {
    public static void main(String[] args) {
        final String USAGE = "\n" +
                "To run this example, supply the name of an S3 bucket and object\n" +
                "name (key) to delete.\n" +
                "\n" +
                "Ex: DeleteObject \n";

        if (args.length < 2) {
            System.out.println(USAGE);
            System.exit(1);
        }

        String bucket_name = args[0];
        String object_key = args[1];

        System.out.format("Deleting object %s from S3 bucket: %s\n", object_key,
        bucket_name);
        final AmazonS3 s3 = AmazonS3ClientBuilder.standard()
        .withEndpointConfiguration(new AwsClientBuilder.EndpointConfiguration("Endpoint", "default"))
        .withCredentials(new AWSStaticCredentialsProvider(new BasicAWSCredentials("Access_Key ", "Secret_Key")))
        .build();

        try {
            s3.deleteObject(bucket_name, object_key);
        } catch (AmazonServiceException e) {
            System.err.println(e.getErrorMessage());
            System.exit(1);
        }
        System.out.println("Done!");
    }
}

 

# 실행 명령어 

java -jar 파일명.jar [Bucket_Name] [Object_Name]