Hi everyone. I was considering backup options to Glacier Deep Archive, and wanted to know:

  1. Which software do you use to encrypt client-side, obfuscate, compress and deduplicate the data before you send it to S3?
  2. What is the difference between Restore Requests (bulk) and Outbound data transfer and which one will I be using when I want to pull my data from AWS?

I’ll be storing approximately 8TB or so of data, which is why I was looking at inexpensive ways to back it up other than buying an HDD outright.

Thanks!

  • 7Sea_Sailor@lemmy.dbzer0.comEnglish
    8·
    1 year ago

    Recently I looked into the same thing, since AWS caught my eye with their apparently ridiculously low prices. Then I found this (presumably indepdenant) review, that changed my view on things: https://b3n.org/b2-vs-s3-nas-backup/

    After reading that, I won’t go with AWS. I’m currently considering to abuse the OneDrive Office Family plan, which costs 99 $ a year for 6 TB of storage (split across 6 accounts), which comes down to 1,40 $ per month per TB. A price that I have not seen beaten by other storage / backup providers.

    • Showroom7561@lemmy.caEnglish
      2·
      1 year ago

      I think that people would be using the service as a last resort, like when all other local or physical offsite backups fail.

      In that sense, the cost to recover shouldn’t be the main factor when considering it.

      • MigratingtoLemmy@lemmy.worldOPEnglish
        1·
        1 year ago

        Is there a less expensive alternative for Cloud storage with a decent SLA? I don’t want to go for the smaller companies, and BackBlaze is quite expensive too!

        • Showroom7561@lemmy.caEnglish
          1·
          1 year ago

          With my Synology NAS, I use icloud e2 for cloud storage. Reasonably priced, and it integrates with Synology’s Hyperbackup software.

          But my needs are relatively small, sending < 5TB to my cloud backup. A few more TB and I may start looking at other options.

          • MigratingtoLemmy@lemmy.worldOPEnglish
            2·
            1 year ago

            I plugged in my numbers into AWS, and I’m looking at $9 a month for storage with $21 for a bulk retrieval. That’s quite inexpensive, which is why I’m starting to think that I’m missing something important

            • g_damian@lemmy.worldEnglish
              1·
              1 year ago

              Scaleway also offers glacier storage class. ~€0.002/GB/month. €0.009/GB retrieval. €0.01/GB transfer.

          • thisispiggy@lemmy.worldEnglish
            1·
            1 year ago

            What other options? I was looking at hezner storage box and it seems pretty reasonable for storage, about $13 for 5 tb

            • Showroom7561@lemmy.caEnglish
              1·
              1 year ago

              I’m not at the “other options yet” as my idrive will review for another year in a week or so.

              At some point, it may be cheaper if I set up a small NAS as a family member’s house and stick an 8TB or 12TB drive in there.

              Really, the cloud backup for me is the last resort, and I have other redundancies available well before I’d need to use a cloud backup.

    • MigratingtoLemmy@lemmy.worldOPEnglish
      2·
      1 year ago

      Wait, I’m looking at the data retrieval cost (bulk request) and it says it’s priced at $0.0025 per GB? That comes out to about $21 for a retrieval! Am i missing something important?

  • CosmicTurtle@lemmy.worldEnglish
    3·
    1 year ago

    1. I don’t encrypt before I push to S3. Probably bad practice on my part. I just rely on AWS encryption to secure my data. My backups are low-risk (imo). That said, I lock down the bucket so that only my account can access the objects. Compression I use tar cjf (bzip). Protip: Once the tar file is made, run tar ljf $archiveFile > archiveFile-ls.txt and store the resulting file along with the tar file in standard storage. That way you know what is in the archive.

    2. Both. Restore Requests is to copy the data out from Glacier into Standard storage. Note that I said copy. When you perform a restore, your original object stays in glacier and AWS creates a copy to somewhere in S3 that you specify. Once the restore is complete, you can then download the copied object like any S3 object, triggering the Outbound data transfer fee.

    • MigratingtoLemmy@lemmy.worldOPEnglish
      1·
      1 year ago

      Thanks, I’ll keep that in mind. I’d encrypt everything client-side since I don’t want anyone to know what I’m storing; including the Cloud provider.

        • MigratingtoLemmy@lemmy.worldOPEnglish
          1·
          1 year ago

          Really? Their pricing is even more expensive than AWS’ S3 Glacier Archive! I’d much rather use BackBlaze B2 than pay that much!

          • 𝕽𝖔𝖔𝖙𝖎𝖊𝖘𝖙@lemmy.worldEnglish
            2·
            1 year ago

            Tbh I don’t really bother with Glacier. It is a lot more expensive than it seems especially when you want to restore anything.

            I generally just use intelligent tiering and it kind of balances out.

            You might think “oh well I’m probably never going to restore from here anyway”

            I am here to tell you that’s a very foolish attitude.

            If you aren’t testing your backups you might as well not have them.

            My honest advice if you must insist on using Glacier is to start off in a normal tier, and keep it there long enough to have tested the backups before transferring it as-is into Glacier.

            It’s not perfect as there’s really no guarantee that data remains safe but at least it mitigates the possibility and reduces the cost to initially use standard tiers before retiring it to Glacier.

            • MigratingtoLemmy@lemmy.worldOPEnglish
              1·
              1 year ago

              You’re right about testing backups. I will have 2 different backups, one for my config and the second for the irreplaceable media. Indeed, restoration from Glacier is too expensive for the data that I plan to back-up.

              I was looking at Scaleway’s Glacier offering, B2 and iDrive. How do you propose I test my backups? I could certainly pull in my config and test it on a VM, but how do I check that I have backed up my media? I plan to encrypt, compress, deduplicate and then ship it off.

        • MigratingtoLemmy@lemmy.worldOPEnglish
          1·
          1 year ago

          Thanks. I was wondering about the reliability of data storage/infrastructure of iDrive specifically. For example, I’m fairly sure that I can keep my data in AWS Glacier/B2 for 10 years or so and nothing much would happen (Assuming Backblaze doesn’t just die). Can I assume that for iDrive? Is this an old company with many years in the business? For their offerings seem amazing, it’s just the perceived risk from lack of information that is holding me back.

      • rentar42@kbin.social
        1·
        1 year ago

        I’m using it too and even the current prices are reasonable (especially if you consider there’s no other fees, no transfer, no ingress, no egress, …). If you put it in S3 glacier and you ever have to restore a relevant chunk of your data (or god forbid, want to do periodic testing of the backed up data) then you’ll be paying quite a bit of fees.