Disqus - Latest Comments for Electrum

Re: Opinionated Database Access in Java

David Phillips — Tue, 09 Apr 2019 03:08:11 -0000

Have you looked at JDBI? http://jdbi.org/

Re: Performance Evaluation of SQL-on-Hadoop Systems using the TPC-DS Benchmark

David Phillips — Sat, 09 Feb 2019 01:48:19 -0000

What specific failures did you see for Presto at higher concurrency levels? We designed it from the beginning to be a multi-tenant system that could scale to high concurrency levels. Many production systems run constantly with 100+ concurrent queries.

(Even if you can run a huge number of queries concurrently, it's often better to configure the system with a lower limit. Find the concurrency level at which you can maximize resource usage, then set the limit somewhere above that. For example, if you can use all the CPU with 5 concurrent queries, set the limit to 10, rather than 100. The total throughput will be the same, but the latency per query will be lower. Queries above the limit will wait in the queue.)

The sink.max-buffer-size setting looks way too large. The default is 32MB. If the queries in the concurrency test are failing due to memory limits, this might be the cause.

Re: How the JVM compares your strings using the craziest x86 instruction you've never heard of

David Phillips — Tue, 06 Sep 2016 19:02:18 -0000

The JVM uses UTF-16 character arrays with length (not null-terminated), so strings can contain nulls (character value zero).

Re: Testing private methods: easier than you think! - Blog - Axel Fontaine - Entrepreneur, Architect, Developer, Trainer, Public Speaker, Author

David Phillips — Sat, 16 Jan 2016 19:21:36 -0000

You can also use an annotation rather than a comment for documentation purposes. Guava has @VisibleForTesting: http://docs.guava-libraries...

This allows IDEs or other tools to automatically flag usages of the method outside of test code. IntelliJ IDEA has an inspection "Test-only class or method call in production code" for this.

Re: Broken by Design: MongoDB Fault Tolerance

David Phillips — Thu, 10 Apr 2014 13:33:25 -0000

You typically run Oracle on an expensive storage appliance or SAN, or at least a local RAID setup. This is very different than typical web scale-out systems that use single (and often cheap) disks with no redundancy.

Re: Simplifying interactive, realtime, and advanced analytics

David Phillips — Tue, 31 Dec 2013 04:17:41 -0000

None of the current Presto developers run Windows, but contributions in that area (or any other) are welcome!

Re: Simplifying interactive, realtime, and advanced analytics

David Phillips — Wed, 06 Nov 2013 21:10:50 -0000

Great article, Ben. Now that Presto is open source, you might want to update the article to link to the website: http://prestodb.io/

Re: In Clauses

David Phillips — Wed, 01 Feb 2012 03:37:13 -0000

Very cool! Jax used a combination of the two with Oracle: use templating for fewer than 100 items, otherwise use arrays. He found that this provided optimal performance, as arrays had too much overhead for small lists.

Re: Edge Rails.info :: The Skinny on Scopes (Formerly named_scope)

David Phillips — Thu, 06 Jan 2011 20:17:48 -0000

CURRENT_TIMESTAMP is ANSI SQL-92 and is supported by every database.

Re: Amazon S3 data corruption

David Phillips — Mon, 22 Sep 2008 12:54:59 -0000

``To ensure data is not corrupted over the network, use the Content-MD5 header. When you use the Content-MD5 header, Amazon S3 checks the object against the provided MD5 value. If they do not match, Amazon S3 returns an error. Additionally, you can calculate the MD5 while putting an object to Amazon S3 and compare the returned Etag to the calculated MD5 value.''

http://docs.amazonwebservic...

Re: Tarsnap beta testers wanted

David Phillips — Mon, 22 Sep 2008 12:12:40 -0000

Note that individual S3 uploads are atomic if you set the Content-MD5 header.