Disqus - Latest Comments for metadaddy

Re: Backblaze Drive Stats for Q2 2025

Pat Patterson — Tue, 05 Aug 2025 15:14:55 -0000

I tracked down the error - we inadvertently ran the query to generate the average age data against the previous version of the data set. All fixed now. Thanks!

Re: Backblaze Drive Stats for Q2 2025

Pat Patterson — Tue, 05 Aug 2025 12:06:29 -0000

Thanks again, Kimi. I'll recheck that table, the query that generated it, and compare it to last quarter's data. Watch this space.

Re: Backblaze Drive Stats for Q2 2025

Pat Patterson — Tue, 05 Aug 2025 12:05:14 -0000

Well spotted - thanks, Kimi! We'll fix that right now.

Re: Iceberg on Backblaze B2

Pat Patterson — Tue, 27 May 2025 16:18:15 -0000

Hi there! I wrote a blog post showing how to access drivestats-parquet: Querying a Decade of Drive Stats Data . That post includes a link to a GitHub tutorial that includes credentials for accessing that bucket: https://github.com/backblaze-b2-samples/trino-getting-started-b2/tree/main/hive/trino-b2#accessing-the-backblaze-drive-stats-data-set

Re: Quoth the Drive Stats, Nevermore: An Elegy for Our Seagate 4TB Drives

Pat Patterson — Fri, 01 Nov 2024 13:06:16 -0000

Amazingly, you can still buy them! https://serverdiskdrives.com/products/seagate-st4000dm000-4tb-64mb-sata-6-0gb-s-3-5-hard-drive

Re: How to Zip Files with the Python S3fs Library + Backblaze B2 Cloud Storage

Pat Patterson — Wed, 30 Oct 2024 11:05:56 -0000

You’re right! We’ll get on that!

Re: How We Achieved Upload Speeds Faster Than AWS S3

Pat Patterson — Fri, 03 Nov 2023 21:07:36 -0000

Hi Justin - great question! We do have a partnership agreement with Vultr, but, at present, we do not have any direct peering in place.

Re: Free Image Hosting With Cloudflare Transform Rules and Backblaze B2

Pat Patterson — Mon, 10 Jul 2023 18:40:48 -0000

Hi there - are you talking about the Media Cloud plugin for Wordpress? I'm not sure what you mean by "my website is not accessible". Is there a CDN setting in Media Cloud that you can set with the Cloudflare subdomain? Perhaps the Media Cloud people can help?

Re: NAS RAID Levels Explained: Choosing The Right Level To Protect Your NAS Data

Pat Patterson — Wed, 21 Jun 2023 12:51:31 -0000

1 day is definitely a best-case scenario, and assumes you have a spare on hand. I personally have a 4-drive Synology NAS with a 3-drive RAID array and a spare, so it can be swapped in instantly. The calculations here are intended as a starting point to illustrate some of the trade-offs. Things get much more complicated when the rebuild time is variable!

Re: NAS RAID Levels Explained: Choosing The Right Level To Protect Your NAS Data

Pat Patterson — Wed, 21 Jun 2023 12:44:11 -0000

Hi Anthony - I did the math for this article. Thanks for the correction - I'll revisit the calculations and fix them accordingly.

As disk volumes have grown, we have moved to 16+4 erasure for 16 TB+ drives, because of the longer time needed to recover the array. We mentioned this in the blog post looking at our US East data center. We're also looking at other ways of managing durability. Watch this space!

Another complication in the durability calculation is that we can often clone the good sectors of a failed drive much more quickly than rebuilding from parity. When a drive fails now, we swap in a 'temporary' drive to start rebuilding, and also attempt to clone the failed drive, which is much faster than a rebuild. If the clone succeeds, we swap the clone for the temporary drive and there is a much shorter rebuild of data that arrived since the failure. Andy Klein provided an interesting explanation of the process in a recent podcast.

Thanks also for the note on Genesis codes. I don't see any references to this technology aside from your own website and GitHub repo. Are there any out there?

Re: Go Wild with Wildcards in Backblaze B2 Command Line Tool 3.7.1

Pat Patterson — Thu, 23 Feb 2023 18:00:48 -0000

Hi Justin - thanks for the kudos on incremental mode - I'll pass it on to the developers.

As far as the keys go, if you choose to use b2 authorize-account, then the credentials are stored in a 600 mode (read-write by user only) file in your home directory. This is pretty standard - the AWS CLI does exactly the same thing. I tend to skip b2 authorize-account altogether and instead set the B2_APPLICATION_KEY_ID and B2_APPLICATION_KEY environment variables, since I'm often working with multiple accounts.

Interestingly, Rclone obscures human-readable passwords in the config file, with the caveat that:

Obscuring them is done by encrypting them and writing them out in base64. This is not a secure way of encrypting these passwords as rclone can decrypt them - it is to prevent "eyedropping" - namely someone seeing a password in the rclone config file by accident.

In fact, they go on to say:

Many equally important things (like access tokens) are not obscured in the config file. However it is very hard to shoulder surf a 64 character hex token.

Having said all that, please feel free to share ideas around functionality you'd like to see in the CLI by opening issues on the GitHub project at https://github.com/Backblaz... - it's the perfect forum to discuss them. See you there! 👍🏻

Re: Free Image Hosting With Cloudflare Transform Rules and Backblaze B2

Pat Patterson — Tue, 13 Sep 2022 13:54:12 -0000

Unfortunately, I was not. The Cloudflare terms of service are annoyingly vague:

Use of the Services for serving video or a disproportionate percentage of pictures, audio files, or other non-HTML content is prohibited, unless purchased separately as part of a Paid Service or expressly allowed under our Supplemental Terms for a specific Service.

Cloudflare don't define what a "disproportionate percentage" might be, so I guess it just depends whether you're doing enough for them to notice 🤷🏼‍♂️

Re: Free Image Hosting With Cloudflare Transform Rules and Backblaze B2

Pat Patterson — Mon, 11 Jul 2022 12:27:32 -0000

You can use a private bucket with Cloudflare - instructions here. Cloudflare hotlink protection should work just fine.

Re: Free Image Hosting With Cloudflare Transform Rules and Backblaze B2

Pat Patterson — Sun, 24 Apr 2022 12:39:15 -0000

I asked one of the Cloudflare PMs about this. He told me:

It’s generally nothing to worry about in this use case - it’s generally videos that we take a dim view on.

I'm trying to get a more official response.

Re: Will Containers Replace Virtual Machines?

Pat Patterson — Thu, 07 Apr 2022 13:28:31 -0000

Now that we can run ubuntu docker containers on Windows 10, is this the same as hypervisors providing a homogeneous environment?

Looking at Microsoft's docs on Linux Containers on Windows 10, the Ubuntu containers all run in the same Hyper-V VM, so they do share the same OS kernel.

Is Ubuntu running within a container functionally different from Ubuntu running on a virtual machine?

That's a great question! It's important to remember that a container doesn't run its own kernel. Instead, a container image contains an application and its dependencies, which might include Ubuntu components such as libraries and executables. When Docker starts a container instance from the image, it runs that application on the Docker host's kernel (which is often running on a VM). When the application terminates, so does the container instance.

Ubuntu running in a VM is very different. When the VM starts, it runs the Linux kernel. You can launch any number of applications. An application can terminate without shutting down the entire system. The system shuts down when you tell the kernel to shut down.

I hope this explanation is useful for you!

Re: Free Image Hosting With Cloudflare Transform Rules and Backblaze B2

Pat Patterson — Thu, 10 Feb 2022 13:38:21 -0000

Post updated with the clarification. Thanks again, Dave!

Re: Free Image Hosting With Cloudflare Transform Rules and Backblaze B2

Pat Patterson — Thu, 10 Feb 2022 12:48:40 -0000

James got back to me:

I vaguely recall there being some edge-cases where "x-bz-content-sha1" wasn't set, but I can't find a file that doesn't return that right now. x-bz-content-sha1 should be more than good enough for the ETag.

I'll put a clarification in the text - thanks for bringing this up!

Re: Free Image Hosting With Cloudflare Transform Rules and Backblaze B2

Pat Patterson — Thu, 10 Feb 2022 12:15:55 -0000

Hi Dave - thanks for the question! I took James' post as my starting point. His code used x-bz-content-sha1, if that was in the header, otherwise x-bz-info-src_last_modified_millis, otherwise x-bz-file-id.

You're absolutely right - I could have used either of the x-bz-content-sha1 or the X-Bz-Upload-Timestamp headers, but my challenge was to replicate what James had done in his Cloudflare Worker code.

Re: Ingest data into Azure Synapse Analytics (formerly SQL DW) with StreamSets Cloud

Pat Patterson — Wed, 20 Nov 2019 19:13:00 -0000

Hi @Kevin - the destination loads data into Azure SQL Data Warehouse, which has to be running to load the data. Azure provides an automation runbook that can be leveraged to auto-pause and resume the DWH to limit costs. See https://blogs.msdn.microsof...

Re: Ingest Data into Azure Data Lake Store with StreamSets Data Collector

Pat Patterson — Wed, 17 Apr 2019 15:25:36 -0000

Sounds like you are using a Java keystore that does not contain the correct root certificate for Azure. Feel free to go deeper via one of our community channels: https://streamsets.com/comm...

Re: Installing StreamSets Data Collector on Amazon Web Services EC2

Pat Patterson — Wed, 13 Mar 2019 17:22:21 -0000

You're most welcome, @stelios ioannides - feel free to follow up via one of our community channels: https://streamsets.com/comm...

Re: Installing StreamSets Data Collector on Amazon Web Services EC2

Pat Patterson — Wed, 13 Mar 2019 10:46:41 -0000

You can install on Ubuntu on bare metal, in a VM, on AWS, anywhere: https://streamsets.com/docu...

Re: Transform Data in StreamSets Data Collector

Pat Patterson — Thu, 07 Mar 2019 15:56:42 -0000

Hi Amin - I replied to your question on the Google Group: https://groups.google.com/a...

Re: Using StreamSets to Ingest Salesforce Data for Analysis

Pat Patterson — Wed, 27 Feb 2019 16:42:23 -0000

You're most welcome - feel free to ask more questions via our community channels: https://streamsets.com/comm...

Re: Using StreamSets to Ingest Salesforce Data for Analysis

Pat Patterson — Wed, 27 Feb 2019 16:37:55 -0000

We don't currently have a Marketing Cloud connector. You could look at connecting to the Marketing Cloud REST API via HTTP.