Docker Registry – S3 storage

Creating a private docker registry can be as simple as running a container provided by docker. Most cloud providers offer a hosted solution (e.g. AWS ECR).

But, sometimes you just want/need to roll your own 🙂

If you are using k8s, this is a quick way of doing it.

The registry stores all its state in the file system. One cheap and reliable way to store the files is using s3.

Configure S3

Create a s3 bucket for your registry files.

Create IAM Policy

{
   "Version": "2012-10-17",
   "Statement": [
     {
       "Effect": "Allow",
       "Action": [
         "s3:ListBucket",
         "s3:GetBucketLocation",
         "s3:ListBucketMultipartUploads"
       ],
       "Resource": "arn:aws:s3:::S3_BUCKET_NAME"
     },
     {
       "Effect": "Allow",
       "Action": [
         "s3:PutObject",
         "s3:GetObject",
         "s3:DeleteObject",
         "s3:ListMultipartUploadParts",
         "s3:AbortMultipartUpload"
       ],
       "Resource": "arn:aws:s3:::S3_BUCKET_NAME/*"
     }
   ]
 }

Replace bucket name with the name of your bucket.

Create User

Configure Container

Create configmap to remove default storage driver.

apiVersion: v1
 data:
   config.yml: |-
     version: 0.1
     log:
       fields:
         service: registry
     http:
       addr: :5000
       headers:
         X-Content-Type-Options: [nosniff]
     health:
       storagedriver:
         enabled: true
         interval: 10s
         threshold: 3
 kind: ConfigMap
 metadata:
   name: my-docker-registry
   namespace: my-docker-registry

You can then override the settings for registry by setting environment variables.

REGISTRY_STORAGE_S3_ACCESSKEY = my_aws_key_id
REGISTRY_STORAGE_S3_SECRETKEY = my_aws_key_secret
REGISTRY_STORAGE_S3_REGION = us-east-1
REGISTRY_STORAGE_S3_BUCKET = my-docker-registry-bucket-name

Since we are deploying to k8s we will create a secret with the environment variables to configure s3 storage for the docker registry.

Create Secret

You can create the yaml file by hand or import using kubectl. The values are base64 encoded in the yaml file.

apiVersion: v1
 data:
   REGISTRY_STORAGE_S3_ACCESSKEY: base64_encoded_my_aws_key_id
   REGISTRY_STORAGE_S3_BUCKET: base64_encoded_my_aws_bucket
   REGISTRY_STORAGE_S3_REGION: base64_encoded_region
   REGISTRY_STORAGE_S3_SECRETKEY: base64_encoded_my_aws_key_secret
 kind: Secret
 metadata:
   name: my-docker-registry
   namespace: my-docker-registry
 type: Opaque

Create Deployment

Now we are ready to create the deployment using the configmap and the s3 storage settings.

apiVersion: apps/v1beta2
 kind: Deployment
 metadata:
   labels:
     app: my-docker-registry
   name: my-docker-registry
   namespace: my-docker-registry
 spec:
   progressDeadlineSeconds: 600
   replicas: 1
   revisionHistoryLimit: 10
   selector:
     matchLabels:
       app: my-docker-registry
   strategy:
     rollingUpdate:
       maxSurge: 1
       maxUnavailable: 0
     type: RollingUpdate
   template:
     metadata:
       labels:
         app: my-docker-registry
     spec:
       containers:
       - envFrom:
         - secretRef:
             name: my-docker-registry
         image: registry:2
         imagePullPolicy: Always
         name: my-docker-registry
         ports:
         - containerPort: 5000
           name: 5000tcp01
           protocol: TCP
         resources: {}
         securityContext:
           allowPrivilegeEscalation: false
           capabilities: {}
           privileged: false
           readOnlyRootFilesystem: false
           runAsNonRoot: false
         stdin: true
         terminationMessagePath: /dev/termination-log
         terminationMessagePolicy: File
         tty: true
         volumeMounts:
         - mountPath: /etc/docker/registry
           name: configmap
           readOnly: true
       dnsPolicy: ClusterFirst
       restartPolicy: Always
       schedulerName: default-scheduler
       securityContext: {}
       terminationGracePeriodSeconds: 30
       volumes:
       - configMap:
           defaultMode: 256
           name: my-docker-registry
           optional: false
         name: configmap

What’s next?

• Create an ingress resource to expose the service.
• Configure authentication

Reference

• https://github.com/docker/docker.github.io/blob/master/registry/storage-drivers/s3.md
• https://docs.docker.com/registry/configuration/
• https://docs.docker.com/registry/deploying/