diff options
| -rw-r--r-- | source/_posts/2015-02-10-monitor-progress-of-your-unix-pipes-with-pv.md | 6 |
1 files changed, 3 insertions, 3 deletions
diff --git a/source/_posts/2015-02-10-monitor-progress-of-your-unix-pipes-with-pv.md b/source/_posts/2015-02-10-monitor-progress-of-your-unix-pipes-with-pv.md index 766f877d..2f58e884 100644 --- a/source/_posts/2015-02-10-monitor-progress-of-your-unix-pipes-with-pv.md +++ b/source/_posts/2015-02-10-monitor-progress-of-your-unix-pipes-with-pv.md @@ -11,9 +11,9 @@ Recently I found a very useful utility called `pv` (for "pipe viewer"). [Here](h For more info, see its home page (linked above) and [man page](http://linux.die.net/man/1/pv). -Why is it useful? Well, pretty obvious if you are in the right audience. For me, one particularly important use case is with `openssl sha1`. I deal with videos on a daily basis, and back up all of them to OneDrive (ever since OneDrive went unlimited). To ensure integrity of transfer (in future downloads), I append the first seven digits of each video to its filename. This should be more than enough to reveal any error in transfer expect for active attacks. One additional advantage is that I can now have multiple versions of a same show, event, or whatever and don't have to worry about naming conflicts (and don't have to artifically say `-ver1`, `-ver2`, etc.). This little merit turns out to be huge and saves me a lot of trouble, since naming things is intrincally hard: +Why is it useful? Well, pretty obvious if you are in the right audience. For me, one particularly important use case is with `openssl sha1`. I deal with videos on a daily basis, and back up all of them to OneDrive (ever since OneDrive went unlimited). To ensure integrity of transfer (in future downloads), I append the first seven digits of each video to its filename. This should be more than enough to reveal any error in transfer except for active attacks. One additional advantage is that I can now have multiple versions of a same show, event, or whatever and don't have to worry about naming conflicts (and don't have to artificially say `-ver1`, `-ver2`, etc.). This little merit turns out to be huge and saves me a lot of trouble, since naming things is intrinsically hard: -> There are only three hard things concurreny, in computer science: cache invalidation, naming things, and off-by-one errors. +> There are only three hard things concurrency, in computer science: cache invalidation, naming things, and off-by-one errors. (I learned this beefed up version of two hard things only recently.) Well, too much digression. So SHA-1 sum is useful. (By the way, I learned in my crypto class that SHA-1 is broken as a collision-resistant hash function — not HMAC, which doesn't assume collision-resistance — and SHA-256 should be used instead. However, I'm not protecting against active attacks — I won't be able to without a shared secret key anyway — so the faster SHA-1 is good for my purpose.) But at the same time, SHA-1 is slow. Maybe what's actually slow is my HDD. Whatever the bottleneck, generating a SHA-1 digest for a 10 GB video file isn't fun at all; it's even more of a torture when there's no progress bar and ETA. But hopelessly waiting has become a thing of the past with the advent (well, discovery in my case) of `pv`. Now I have nice and informative progress bars, which reduces the anxiety of waiting by an order of magnitude. @@ -57,4 +57,4 @@ By the way, I also found a project called `cv` (Coreutils Viewer), which is [off > ... a Tiny, Dirty, Linux-Only C command that looks for coreutils basic commands (cp, mv, dd, tar, gzip/gunzip, cat, etc.) currently running on your system and displays the percentage of copied data. -. I'll look into it when I have time, but it from its description, it seems to be limited to coreutils, and OS X support might not be too awesome (at this point). +I'll look into it when I have time, but it from its description, it seems to be limited to coreutils, and OS X support might not be too awesome (at this point). |