What's the easiest way to backup my entire FreeNAS dataset to Backblaze's B2 service? I've decided to use Backblaze B2 for cloud backups since it is the most cost effective for the amount of data I need to store. What would be the best way for me to backup to it. I'd need to backup my entire dataset or a few user's folders at the very least. Can confirm FreeNAS works as expected - this ticket should be closed. Thanks for your help! The issue was that Backblaze / B2 would retain old copies of files after deletion for about 3-4 days. This happens even when 'File Lifecycle' is set to 'Keep only the last version'. For what it may be worth to FreeNAS users, I replaced /usr/local/bin/rclone with the rclone v1.52.0-023-g399e8c0b-fix-3991-b2-copy-beta and successfully had a b2 sync complete which previously had hung up on the Copy source too big for modification time update issue.
FreeNAS
FreeNAS is an operating system that can be installed on virtually any hardware platform to share data over a network. FreeNAS is the simplest way to create a centralized and easily accessible place for your data. Use FreeNAS with ZFS to protect, store, and back up all of your data. FreeNAS is used everywhere, for the home, small business, and the enterprise.
Unraid
Unraid Server OS 6 is a storage operating system developed by Lime Technology that is designed to scale with a system’s hardware. Its RAID design supports a mix of devices, filesystems, and sharing policies.
| Overview | FreeNAS 11.3 | Unraid |
| Ease of Use | Graphical User Interface | Graphical User Interface |
| Documentation | User Guide, How-To Videos, Training Videos | Wiki |
| Certified Hardware Available | SoHo and Business | No |
| Open Source Licence | Primarily 2-clause BSD | Commercial, requires paid licensing key |
| Base Operating System | FreeBSD 11.3 | unRAID Server OS 6.8 |
| File System | OpenZFS | XFS (default), BTRFS |
| Architecture | 64-bit | 64-bit |
| Downloads | Over 10 million | Unknown |
| Development Language | Python, Angular | Closed source |
| Open Source Repository | https://github.com/freenas | Closed source |
| Core Developers | 25+ | Lime Technology |
| Forums | 66,500+ users and over 550,000 posts | over 630,000 posts |
| YouTube Community Videos | 134,000+ results and 22K+ subscribers | 169,000 results |
| Social Media | 28,800+ Facebook likes, 13,700 Twitter followers | 7,425 twitter followers, 124 Google+ members |
| Restorability | Download/Upload configuration file, default settings restore, boot environments | /previous folder (CLI) |
| Built-in Visual Reporting | CPU, disk, memory, network, processes, uptime, iSCSI, ZFS | CPU, disk, memory, network, processes, uptime, Docker |
| Supported Disk Configurations | Stripe, mirror, RAIDZ1, Z2, Z3, hot-swap | Stripe, single parity, dual parity |
| Encrypted Storage Support | Software Disk Encryption, Self-Encrypting Drives (SED), and Dataset Encryption in TrueNAS CORE 12 | Software Disk Encryption, Command Line Self-Encrypting Disks |
| Third Party Application Support | Plugins, Jails, bhyve VMs, Docker VMs | Docker, KVM, Qemu |
| API | REST, websockets | none |
| Alerting Services | GUI, email, and SNMP alerts, integration with AWS-SNS, InfluxDB, Slack, Mattermost, OpsGenie, PagerDuty, VictorOps | GUI and email alerts |
| Built-in File Sharing Protocols | SMB, NFS, AFP, WebDAV | SMB, NFS, AFP |
| Built-in Block Storage Protocols | iSCSI | none |
| Other Built-in Network Protocols | rsync, FTP, TFTP, LLDP, SSH, DDNS, SNMP | FTP, rsync (CLI), SSH (CLI) |
| Built-in Cloud Synchronization | Amazon S3, Backblaze B2, Box, Dropbox, FTP, Google Cloud Storage, HTTP, Hubic, Mega, Microsoft Azure Blob Storage, Microsoft OneDrive, pCloud, SFTP, WebDAV, Yandex | Command line |
| Built-in Directory Services | Active Directory, LDAP, Kerberos, NIS | Active Directory |
| Built-in Hardware Protocols | S.M.A.R.T., SCSI Enclosure Services (SES), UPS | S.M.A.R.T., UPS (APC only) |
| Built-in Virtualization | bhyve, Docker | KVM, Qemu |
Bug #34414
Cloud sync task to Backblaze B2 doesn't delete files on remote
Backblaze B2 Synology
Description
Setting up a Backblaze B2 backup task runs successfully, but fails to delete data in the bucket.
Settings:
Direction: Push
Credential: Backblaze B2
Path <some test path with files in it>
Transfer Mode: Sync
I'm not sure where to capture relevant log files, but I can provide those if you let me know where to look.
History
#1 Updated by Dru Lavigne almost 3 years ago
- Category changed from OS to Middleware
- Assignee changed from Release Council to Vladimir Vinogradenko
- Private changed from No to Yes
Vlad: please let Daniel know which log files would be useful to attach.
#3 Updated by William Grzybowski almost 3 years ago
Freenas Backblaze B2 Encryption
- Target version changed from Backlog to 11.3
- Severity changed from Med High to Medium
Vladimir, can you investigate this for 11.3, please?
It seems the user is using sync mode already, so perhaps its some issue in rclone backblaze backend?
#4 Updated by Vladimir Vinogradenko almost 3 years ago
- Status changed from Unscreened to Blocked
- Reason for Blocked set to Waiting for feedback
I was not able to repeat that neither on rclone that ships with 11.1 nor in 11.2? Files were deleted correctly. The only reason for behavior described I can imagine is reaching transactions limit. Daniel, have you found the solution?
#5 Updated by Daniel Schuler almost 3 years ago
I'm checking if I can reproduce this problem now - I have a backup that's running that should be finished by Tuesday (7/24) and will provide an update.
I do have about 120k files from a Time Machine backup - when I tried sending this manually using Backblaze's B2 tool, the tool also errored out after deleting 70k files or so, so it's possible that a limit is being reached.
It's also possible that the original backup never had a chance to finish, because the directory being sent to Backblaze kept changing every hour with each incremental Time Machine backup. The currently running backup is based on a cloned ZFS dataset, so it won't change, eliminating this possibility.
In any case, it would be convenient if the cloud sync task would use snapshots like the replication tasks to another FreeNAS host. Of course, that's a feature request, and not a bug :)
Again, I should have an update by Tuesday for this ticket. Thanks.
Freenas Backblaze B2
#6 Updated by Daniel Schuler over 2 years ago
- Reason for Closing set to Behaves as Intended
- Reason for Blocked deleted (
Waiting for feedback)
Can confirm FreeNAS works as expected - this ticket should be closed. Thanks for your help!
The issue was that Backblaze / B2 would retain old copies of files after deletion for about 3-4 days. This happens even when 'File Lifecycle' is set to 'Keep only the last version'. Files are marked as 'hidden' in the web interface, so you should still get the latest copy if pulling from a server.
According to Backblaze support, old copies should be auto deleted in 24 hours, but I consistently saw this take process take several days. During this time, data usage for the bucket will be the sum of current and deleted files.
#7 Updated by Dru Lavigne over 2 years ago
- Status changed from Blocked to Closed
- Target version changed from 11.3 to N/A
- Private changed from Yes to No
Also available in: AtomPDF