S3 Gateway: Back Up a Tenancy Every Night

#Overview

Tenancy Managers can use the Dekko S3 Gateway to keep a nightly copy of the tenancy's files on a Windows server that your organisation runs. This article sets it up from start to finish: a backup account in DekkoCORE, the gateway as a Windows service, a nightly rclone job, and a restore.

It also covers the tenancy policy that controls the gateway for everyone. For the end-user steps, see S3 Gateway: Enable Access and Create a Token.

#Turn the gateway on or off for the tenancy

The gateway is controlled by one tenancy policy:

1. Go to Tenancy management → Policies.

2. Under Files & content protection, tick Allow local S3 gateway:

   

3. Click Update policies.

People in the tenancy can then create tokens and connect a gateway to their hubs. Turning the policy off closes the hubs to every gateway on its next request. It does not delete people's tokens.

If the S3 gateway page says the gateway is a paid add-on, contact DekkoSecure to make it available for your tenancy.

#How the backup works

The tenancy's Central Backup Account becomes a member of every Core Hub in the tenancy, including hubs created later. Every file uploaded to a Core Hub after the nomination is shared with it at Download only. The share does not expire, and hub admins and file owners cannot remove it.

One Read-only token for that account therefore reaches all of those files. The gateway shows each hub as a bucket, and a nightly rclone job copies every bucket to the server.

#What you need

• A Tenancy Manager's sign-in to DekkoCORE.
• A Windows server or virtual machine, and a standard Windows account for the backup, not an administrator. This article calls it BACKUPVM\dekkobackup. The gateway's service refuses an administrator account.
• The Dekko S3 Gateway for Windows (x64).
• rclone for Windows. This article uses C:\tools\rclone\rclone.exe.
• A data disk encrypted with BitLocker, with room for the files and the archive. This article uses D:\DekkoBackup.

#Step 1: Create the backup account

As a Tenancy Manager:

1. Go to Tenancy management → Users and click Create user.
2. Enter a User ID, for example backup.service, and a Full name, for example Central Backup.
3. Enter a strong password twice. Nothing is emailed: give the ID and password only to the person who runs the backup server.
4. Give it one Core Hub, for example as Team. Step 2 adds it to the others.
5. Click Create.

You can use an existing account instead, as long as only the backup uses it.

#Step 2: Nominate it as the Central Backup Account

1. Go to Tenancy management → Policies and tick Central Backup Account.
2. In Type email address, type the backup account's user ID and choose it from the list.
3. Type the address DekkoCORE shows to confirm, and click Add Backup Admin.
4. Click Update policies. The bar at the bottom says All changes saved.

#Step 3: Allow the S3 gateway

On the same Policies page, tick Allow local S3 gateway and click Update policies (see Turn the gateway on or off for the tenancy). Without it, the backup account cannot create a token.

#Step 4: Prepare the server

A server administrator does this once:

1. Create the folder C:\Program Files\Dekko S3 Gateway and copy or extract everything from the gateway's ZIP file into it.

   ❗

   Copy or extract the files there. Do not move a folder there: a moved folder keeps permissions that let other programs change it, and the gateway then refuses to set up the service.

2. Give the backup account the right to run scheduled tasks. Open Local Security Policy (secpol.msc) → Local Policies → User Rights Assignment, open Log on as a batch job, and add the backup account.

3. Install rclone in C:\tools\rclone.

4. Create the backup folders and let the backup account write to them, in PowerShell as an administrator:

   New-Item -ItemType Directory -Force D:\DekkoBackup, D:\DekkoBackup\current, D:\DekkoBackup\archive, D:\DekkoBackup\logs
   icacls D:\DekkoBackup /grant "BACKUPVM\dekkobackup:(OI)(CI)M"
   

#Step 5: Create the token and add it to the gateway

Sign in to Windows on the server as the backup account and start C:\Program Files\Dekko S3 Gateway\DekkoS3Gateway.exe. Then, in a browser on the same server:

1. Sign in to DekkoCORE as the backup account. Its Files list shows the files shared with it in each hub.
2. Go to Settings → Account → S3 gateway and click New token.
3. Enter a name, for example Backup server.
4. Under Access, choose Read-only. The backup then cannot change or delete anything in DekkoCORE.
5. Under Reaches, choose All my hubs.
6. Optional: open Advanced and set the Renewal period, from 1 to 365 days (default 30). A running gateway renews its access by itself.
7. Click Create token, confirm with the account's password, and click Copy token. It is shown only once.
8. Within 5 minutes, paste it into the gateway window and click Add.

#Step 6: Run the gateway as a Windows service

In the gateway window, still signed in as the backup account:

1. Tick Run as a Windows service and click Run as a service.
2. Windows asks for an administrator's approval, then for the backup account's own Windows password. The service signs in as that account so that it can open its token.

The window then says Served by the Windows service. The gateway now answers after a restart and while nobody is signed in, and the backup account can sign out.

#Step 7: Configure rclone

Create D:\DekkoBackup\rclone.conf, using the gateway window's Copy buttons for the keys, endpoint and region:

[dekko]
type = s3
provider = Other
endpoint = http://127.0.0.1:9493
region = ap-southeast-2
force_path_style = true
access_key_id = <Access key ID from the gateway window>
secret_access_key = <Secret access key from the gateway window>

Make the file readable only by the backup account and administrators:

icacls D:\DekkoBackup\rclone.conf /inheritance:r /grant:r "BACKUPVM\dekkobackup:(R,W)" "Administrators:F"

Check that rclone sees every Core Hub as a bucket:

C:\tools\rclone\rclone.exe --config D:\DekkoBackup\rclone.conf lsd dekko:

#Step 8: Create the nightly job

Save this as D:\DekkoBackup\backup.ps1:

$ErrorActionPreference = 'Stop'
$run = '{0:yyyy-MM-dd_HHmmss}_{1}' -f (Get-Date), [guid]::NewGuid().ToString('N').Substring(0,8)
& C:\tools\rclone\rclone.exe --config D:\DekkoBackup\rclone.conf sync dekko: D:\DekkoBackup\current `
    --backup-dir "D:\DekkoBackup\archive\$run" `
    --transfers 8 --checkers 16 --multi-thread-streams 8 `
    --log-file "D:\DekkoBackup\logs\$run.log" --log-level INFO
exit $LASTEXITCODE

Each run makes current match every hub, with one folder per hub. A file the run would replace or delete is first moved into a folder under archive named for that run. Each run writes its own log, and a failed run returns its error code to Task Scheduler.

#Step 9: Schedule the job

Run this once in PowerShell as an administrator. It runs the job every night at 02:00 as the backup account, whether or not anyone is signed in:

$account = Get-Credential BACKUPVM\dekkobackup
Register-ScheduledTask -TaskName "Dekko daily backup" -User $account.UserName `
    -Password $account.GetNetworkCredential().Password -RunLevel Limited `
    -Trigger (New-ScheduledTaskTrigger -Daily -At 2am) `
    -Action (New-ScheduledTaskAction -Execute powershell.exe `
        -Argument "-NoProfile -ExecutionPolicy Bypass -File D:\DekkoBackup\backup.ps1")

To try it now, run Start-ScheduledTask "Dekko daily backup", then Get-ScheduledTaskInfo "Dekko daily backup". A LastTaskResult of 0 means it worked. 267011 means the task has not run, usually because the account is missing Log on as a batch job (Step 4).

#What each night's run does

rclone compares each file's size and modification time, so a run downloads only new and changed files. It still lists every file in every hub, so runs take longer as the tenancy grows. A file created and deleted between two runs is never copied.

#Restore a file

• On the server: copy the file from D:\DekkoBackup\current, or an earlier version from a folder under D:\DekkoBackup\archive.
• Back into DekkoCORE: a member of the hub uploads it in the web app. The next run backs it up again. The backup token is read-only, so the backup itself cannot write to DekkoCORE.

#Check that the copy is complete

Once a week, compare the copy with DekkoCORE:

C:\tools\rclone\rclone.exe --config D:\DekkoBackup\rclone.conf check dekko: D:\DekkoBackup\current --download --one-way

This reads every file from DekkoCORE again, so it takes as long as a full download. It checks current only, not the archive folders.

#Keep it running

• Backup account's Windows password changed: the service and the task stop signing in. Clear Run as a Windows service and tick it again, and register the task again with the new password.
• New gateway version: clear the box, then an administrator deletes C:\Program Files\Dekko S3 Gateway and copies the new version there as in Step 4. Start it from there and tick the box again.
• New token: remove the old one in the gateway window and add the new one. The keys change, so update rclone.conf. Then revoke the old token in DekkoCORE under Settings → Account → S3 gateway. Removing it from the gateway window does not revoke it.
• Remove the service: clear the box. If the folder was deleted first, an administrator runs sc.exe delete DekkoS3Gateway. The Dekko Uninstaller also removes the gateway (see S3 Gateway: Enable Access and Create a Token).

#Protect the backup

• The gateway decrypts files on the server, so the copy on its disk is readable. Encrypt that disk with BitLocker.
• Limit who can sign in to the server.
• Keep rclone.conf readable only by the backup account and administrators. Anyone who can read it, and reach the gateway on the server, can read every file the backup account can.
• Keep the backup account's DekkoCORE password with the server team only, and use it for nothing else.
• Keep a second copy on other storage, so one disk failing does not take both.
• Decide how long to keep archive folders. The job never deletes them, so they grow until you remove old ones.

#Test before you rely on it

• Restart the server, sign nobody in, and let a scheduled run happen.
• Make a transfer fail, and check how it shows in Task Scheduler and the log.
• Run a full backup over every hub and time it.
• Restore a file from an archive folder.

#Optional: browse the backup with S3 Browser

To see the same hubs and files in a window on the server, add an account in S3 Browser with the gateway's connection details. See S3 Gateway: Connect an S3 App (S3 Browser, rclone). The free edition of S3 Browser is for non-commercial use only, so an organisation needs the paid edition.