Disqus - Latest Comments for yaroslavvb

Re: Matrices as Tensor Network Diagrams

Yaroslav Bulatov — Fri, 12 Nov 2021 17:39:02 -0000

BTW, there's could be something special about the tensor network that's not completely captured by looking at the graphical model corresponding to its line graph. There's a fast algorithm for contracting planar tensor networks, no equivalent for graphical models is known -- Jakes-Schauer, J., D. Anekstein, and P. Wocjan. 2019. “Carving-Width and Contraction Trees for Tensor Networks.” arXiv [cs.DM]. arXiv. https://doi.org/10.1016/j.j....

Re: What About Extra Virgin Olive Oil?

Yaroslav Bulatov — Mon, 31 May 2021 19:53:38 -0000

The first study is interventional so that's good. To summarize, adding "extra virgin olive oil" to Mediterranean diet was slightly better than adding extra nuts, but not statistically significant (57% reduction vs 55% reduction in risk), whereas going for control "low-fat" diet was 9% increase in risk. TLDR; replacing some nuts with a tea-spoon of extra-virgin olive oil doesn't seem to impact CVD mortality whereas going from "low-fat" to Mediterranean diet has a huge effect

Re: IGraph/M: a Mathematica interface for igraph

Yaroslav Bulatov — Sun, 29 Dec 2019 22:31:14 -0000

I see, btw is there anything relating to treewidth in igraph? (ie, finding tree decomposition)

Re: IGraph/M: a Mathematica interface for igraph

Yaroslav Bulatov — Sun, 29 Dec 2019 12:51:49 -0000

Just tried and it worked out of the box, great to see it's being maintained. One minor nit, the initial message prints "It can now be loaded using the command Get["IGraphM`"]". If I copy paste this, I get "Get[\" IGraphM` \"]". Maybe this last line could be emitted as a code cell that user could run directly

Re: Distributed TensorFlow - A Gentle Introduction

Yaroslav Bulatov — Mon, 27 Nov 2017 14:24:30 -0000

Nice overview!

Regarding worker leaving, it should be no problem if the worker permanently leaves AS LONG as the other worker doesn't restart. If it does restart, the first session.run call will hang since it sets up the cluster and needs all the workers to be available. The solution to this is to use "sparse job config" -- use dictionary of worker->ip mapping for necessary workers only. This way any worker not in this list can be down without affecting current worker. In a Parameter Server environment, workers don't need to know about other workers.

For failure tolerance, it's a bit annoying, but you have to recreate session each time there's any error and wait until things are OK (session created successfully and tf.report_unininitialized_variables gives empty list). So if a parameter server restarts, this causes session.run to crash in all the workers which go into the waiting loop. The chief worker has a similar loop, except it only tries to create session and then call initialization op. Eventually initialization op succeeds, workers stop waiting and training continues. I have a simpler that implements failure robustness for a set of workers adding 1's to a central parameter server here -- https://github.com/diux-dev...

Re: Tensorflow I Love You, But You're Bringing Me Down

Yaroslav Bulatov — Sun, 11 Jun 2017 14:50:49 -0000

GraphDef issue is similar to issue of "compiled vs interpreted". Compiled programs run faster at the expense of being harder to debug and longer iteration cycle. You need GraphDef in order to be able to optimize the program. But a lot of engineering work is needed to bring the ease of use back in. I'm not sure Google is best-positioned to make a good high-level neural net library. Applications are somewhat different and incentives aren't there. I've seen TensorPack gaining in popularity and it's not made by Google

Re: Fisher Information and the Hessian of Log Likelihood

Yaroslav Bulatov — Wed, 22 Feb 2017 16:36:21 -0000

BTW, there's a mistake in derivation, which gets fixed by another mistake in the last line, when you use product rule to compute derivative, you use Di,jpθ(x)/p(x) for the first term, but it should be Di,jp(x) . (ps, this was the first result I found when searching for derivation of the connection, by searching for fisher hessian)

Re: Calculus on Computational Graphs: Backpropagation

Yaroslav Bulatov — Sat, 23 Apr 2016 11:12:49 -0000

There's another cool algebraic view: for f(g(h(...))) the derivative is F*G*H where * is matmul and F,G,H are Jacobian matrices. If you have many inputs and one output, f is R^n->R^1, then your last matrix is skinny and tall, then Matrix Chain Multiplication solution tells you to do (F G)H, which is reverse mode AD. But if you have many outputs and one input, your H is wide and short, so most efficient is to do F(G H) which is forward mode AD. But also there are cases where neither forward nor reverse mode AD are the most efficient, and those are the "other" solutions of the MCM problem

Re: New Hack: CPU/Memory process monitor for Google Chrome

Yaroslav Bulatov — Sat, 30 Aug 2014 17:30:48 -0000

Doesn't seem to work on the latest from Stable channel, stuck at "Loading"

Re: Updated List of Datasets & Video Lectures

Yaroslav Bulatov — Wed, 05 Aug 2009 19:16:45 -0000

Hey, I've just put together another digit OCR dataset. It's 20k digit crops taken from natural scene photographs, and I believe this dataset is more challenging than MNIST http://yaroslavvb.blogspot....