Thursday, September 1, 2016

[PostgreSQL] How to verify if replication catch up with master change

[Run on master] Check master location with
SELECT pg_current_xlog_location();
 [Run on replica] Check replica replayed location with
SELECT pg_last_xlog_replay_location();
[Run on master] Check replica status from master
SELECT client_addr, state, sent_location, write_location, flush_location, replay_location FROM pg_stat_replication;
*write, flush, replay are status of replica

[Run on replica] Check time from the latest reply
SELECT now() - pg_last_xact_replay_timestamp() AS replication_delay;
Credit:

Wednesday, August 24, 2016

Shorthand for Git upstream (tracked) remote branch

It's @{u}.

For example, if you are on master branch, and master branch is tracked with origin/master. You can use git merge @{u} instead of git merge origin/master.

Reference: https://git-scm.com/book/en/v2/Git-Branching-Remote-Branches#upstream-shorthand-k9UotnHmIO

Tuesday, August 23, 2016

[Solved] An obscure error in case somebody else gets it too

I fixed an error below with `lein clean`. Hope this post can save your time as it used mine.
#error {
  :cause clojure.core$seq_QMARK___5848
    :via
    [{:type clojure.lang.Compiler$CompilerException
      :message java.lang.NoClassDefFoundError: clojure/core$seq_QMARK___5848, compiling:(clj_gatling/report.clj:1:1)
        :at [clojure.lang.Compiler load Compiler.java 7415]}
  {:type java.lang.NoClassDefFoundError
    :message clojure/core$seq_QMARK___5848
      :at [clojure.core.cache$lru_cache_factory invokeStatic cache.clj 581]}
    {:type java.lang.ClassNotFoundException
      :message clojure.core$seq_QMARK___5848
        :at [java.net.URLClassLoader findClass URLClassLoader.java 381]}]
        :trace
        [[java.net.URLClassLoader findClass URLClassLoader.java 381]
        [java.lang.ClassLoader loadClass ClassLoader.java 424]
        [sun.misc.Launcher$AppClassLoader loadClass Launcher.java 331]
        [java.lang.ClassLoader loadClass ClassLoader.java 357]
        [clojure.core.cache$lru_cache_factory invokeStatic cache.clj 581]
        [clojure.core.memoize$lru$fn__2765 invoke memoize.clj 334]
        [clojure.lang.AFn applyToHelper AFn.java 160]
        [clojure.lang.AFn applyTo AFn.java 144]
        [clojure.core$apply invokeStatic core.clj 659]
        [clojure.core.memoize$build_memoizer invokeStatic memoize.clj 152]
        [clojure.core.memoize$lru invokeStatic memoize.clj 332]
        [clojure.core.memoize$lru invoke memoize.clj 292]
        [clojure.core.memoize$lru invokeStatic memoize.clj 328]
        [clojure.core.memoize$lru invoke memoize.clj 292]
        [clojure.tools.analyzer.jvm.utils__init load nil 79]
        [clojure.tools.analyzer.jvm.utils__init nil -1]
        [java.lang.Class forName0 Class.java -2]
        [java.lang.Class forName Class.java 348]
        [clojure.lang.RT classForName RT.java 2183]
        [clojure.lang.RT classForName RT.java 2192]
        [clojure.lang.RT loadClassForName RT.java 2211]
        [clojure.lang.RT load RT.java 445]
        [clojure.lang.RT load RT.java 421]
        [clojure.core$load$fn__7645 invoke core.clj 6008]
        [clojure.core$load invokeStatic core.clj 6007]
        [clojure.core$load doInvoke core.clj 5991]
        [clojure.lang.RestFn invoke RestFn.java 408]
        [clojure.core$load_one invokeStatic core.clj 5812]
        [clojure.core$load_one invoke core.clj 5807]
        [clojure.core$load_lib$fn__7590 invoke core.clj 5852]
        [clojure.core$load_lib invokeStatic core.clj 5851]
        [clojure.core$load_lib doInvoke core.clj 5832]
        [clojure.lang.RestFn applyTo RestFn.java 142]
        [clojure.core$apply invokeStatic core.clj 659]
        [clojure.core$load_libs invokeStatic core.clj 5889]
        [clojure.core$load_libs doInvoke core.clj 5873]
        [clojure.lang.RestFn applyTo RestFn.java 137]
        [clojure.core$apply invokeStatic core.clj 659]
        [clojure.core$require invokeStatic core.clj 5911]
        [clojure.tools.analyzer.jvm$loading__7118__auto____2196 invoke jvm.clj 9]
        [clojure.tools.analyzer.jvm__init load nil 9]
        [clojure.tools.analyzer.jvm__init nil -1]
        [java.lang.Class forName0 Class.java -2]
        [java.lang.Class forName Class.java 348]
        [clojure.lang.RT classForName RT.java 2183]
        [clojure.lang.RT classForName RT.java 2192]
        [clojure.lang.RT loadClassForName RT.java 2211]
        [clojure.lang.RT load RT.java 445]
        [clojure.lang.RT load RT.java 421]
        [clojure.core$load$fn__7645 invoke core.clj 6008]
        [clojure.core$load invokeStatic core.clj 6007]
        [clojure.core$load doInvoke core.clj 5991]
        [clojure.lang.RestFn invoke RestFn.java 408]
        [clojure.core$load_one invokeStatic core.clj 5812]
        [clojure.core$load_one invoke core.clj 5807]
        [clojure.core$load_lib$fn__7590 invoke core.clj 5852]
        [clojure.core$load_lib invokeStatic core.clj 5851]
        [clojure.core$load_lib doInvoke core.clj 5832]
        [clojure.lang.RestFn applyTo RestFn.java 142]
        [clojure.core$apply invokeStatic core.clj 659]
        [clojure.core$load_libs invokeStatic core.clj 5889]
        [clojure.core$load_libs doInvoke core.clj 5873]
        [clojure.lang.RestFn applyTo RestFn.java 137]
        [clojure.core$apply invokeStatic core.clj 659]
        [clojure.core$require invokeStatic core.clj 5911]
        [clojure.core.async.impl.ioc_macros$loading__7118__auto____1596 invoke ioc_macros.clj 12]
        [clojure.core.async.impl.ioc_macros__init load nil 12]
        [clojure.core.async.impl.ioc_macros__init nil -1]
        [java.lang.Class forName0 Class.java -2]
        [java.lang.Class forName Class.java 348]
        [clojure.lang.RT classForName RT.java 2183]
        [clojure.lang.RT classForName RT.java 2192]
        [clojure.lang.RT loadClassForName RT.java 2211]
        [clojure.lang.RT load RT.java 445]
        [clojure.lang.RT load RT.java 421]
        [clojure.core$load$fn__7645 invoke core.clj 6008]
        [clojure.core$load invokeStatic core.clj 6007]
        [clojure.core$load doInvoke core.clj 5991]
        [clojure.lang.RestFn invoke RestFn.java 408]
        [clojure.core$load_one invokeStatic core.clj 5812]
        [clojure.core$load_one invoke core.clj 5807]
        [clojure.core$load_lib$fn__7590 invoke core.clj 5852]
        [clojure.core$load_lib invokeStatic core.clj 5851]
        [clojure.core$load_lib doInvoke core.clj 5832]
        [clojure.lang.RestFn applyTo RestFn.java 142]
        [clojure.core$apply invokeStatic core.clj 659]
        [clojure.core$load_libs invokeStatic core.clj 5889]
        [clojure.core$load_libs doInvoke core.clj 5873]
        [clojure.lang.RestFn applyTo RestFn.java 137]
        [clojure.core$apply invokeStatic core.clj 659]
        [clojure.core$require invokeStatic core.clj 5911]
        [clojure.core.async$loading__7118__auto____1152 invoke async.clj 9]
        [clojure.core.async__init load nil 9]
        [clojure.core.async__init nil -1]
        [java.lang.Class forName0 Class.java -2]
        [java.lang.Class forName Class.java 348]
        [clojure.lang.RT classForName RT.java 2183]
        [clojure.lang.RT classForName RT.java 2192]
        [clojure.lang.RT loadClassForName RT.java 2211]
        [clojure.lang.RT load RT.java 445]
        [clojure.lang.RT load RT.java 421]
        [clojure.core$load$fn__7645 invoke core.clj 6008]
        [clojure.core$load invokeStatic core.clj 6007]
        [clojure.core$load doInvoke core.clj 5991]
        [clojure.lang.RestFn invoke RestFn.java 408]
        [clojure.core$load_one invokeStatic core.clj 5812]
        [clojure.core$load_one invoke core.clj 5807]
        [clojure.core$load_lib$fn__7590 invoke core.clj 5852]
        [clojure.core$load_lib invokeStatic core.clj 5851]
        [clojure.core$load_lib doInvoke core.clj 5832]
        [clojure.lang.RestFn applyTo RestFn.java 142]
        [clojure.core$apply invokeStatic core.clj 659]
        [clojure.core$load_libs invokeStatic core.clj 5889]
        [clojure.core$load_libs doInvoke core.clj 5873]
        [clojure.lang.RestFn applyTo RestFn.java 137]
        [clojure.core$apply invokeStatic core.clj 659]
        [clojure.core$require invokeStatic core.clj 5911]
        [clojure.core$require doInvoke core.clj 5911]
        [clojure.lang.RestFn invoke RestFn.java 436]
        [clj_gatling.report$eval47$loading__7531__auto____48 invoke report.clj 1]
        [clj_gatling.report$eval47 invokeStatic report.clj 1]
        [clj_gatling.report$eval47 invoke report.clj 1]
        [clojure.lang.Compiler eval Compiler.java 6951]
        [clojure.lang.Compiler eval Compiler.java 6940]
        [clojure.lang.Compiler load Compiler.java 7403]
        [clojure.lang.RT loadResourceScript RT.java 374]
        [clojure.lang.RT loadResourceScript RT.java 365]
        [clojure.lang.RT load RT.java 455]
        [clojure.lang.RT load RT.java 421]
        [clojure.core$load$fn__7645 invoke core.clj 6008]
        [clojure.core$load invokeStatic core.clj 6007]
        [clojure.core$load doInvoke core.clj 5991]
        [clojure.lang.RestFn invoke RestFn.java 408]
        [clojure.core$load_one invokeStatic core.clj 5812]
        [clojure.core$load_one invoke core.clj 5807]
        [clojure.core$load_lib$fn__7590 invoke core.clj 5852]
        [clojure.core$load_lib invokeStatic core.clj 5851]
        [clojure.core$load_lib doInvoke core.clj 5832]
        [clojure.lang.RestFn applyTo RestFn.java 142]
        [clojure.core$apply invokeStatic core.clj 659]
        [clojure.core$load_libs invokeStatic core.clj 5889]
        [clojure.core$load_libs doInvoke core.clj 5873]
        [clojure.lang.RestFn applyTo RestFn.java 137]
        [clojure.core$apply invokeStatic core.clj 659]
        [clojure.core$require invokeStatic core.clj 5911]
        [clojure.core$require doInvoke core.clj 5911]
        [clojure.lang.RestFn invoke RestFn.java 457]
        [clj_gatling.core$eval30$loading__7531__auto____31 invoke core.clj 1]
        [clj_gatling.core$eval30 invokeStatic core.clj 1]
        [clj_gatling.core$eval30 invoke core.clj 1]
        [clojure.lang.Compiler eval Compiler.java 6951]
        [clojure.lang.Compiler eval Compiler.java 6940]
        [clojure.lang.Compiler load Compiler.java 7403]
        [clojure.lang.RT loadResourceScript RT.java 374]
        [clojure.lang.RT loadResourceScript RT.java 365]
        [clojure.lang.RT load RT.java 455]
        [clojure.lang.RT load RT.java 421]
        [clojure.core$load$fn__7645 invoke core.clj 6008]
        [clojure.core$load invokeStatic core.clj 6007]
        [clojure.core$load doInvoke core.clj 5991]
        [clojure.lang.RestFn invoke RestFn.java 408]
        [clojure.core$load_one invokeStatic core.clj 5812]
        [clojure.core$load_one invoke core.clj 5807]
        [clojure.core$load_lib$fn__7590 invoke core.clj 5852]
        [clojure.core$load_lib invokeStatic core.clj 5851]
        [clojure.core$load_lib doInvoke core.clj 5832]
        [clojure.lang.RestFn applyTo RestFn.java 142]
        [clojure.core$apply invokeStatic core.clj 659]
        [clojure.core$load_libs invokeStatic core.clj 5889]
        [clojure.core$load_libs doInvoke core.clj 5873]
        [clojure.lang.RestFn applyTo RestFn.java 137]
        [clojure.core$apply invokeStatic core.clj 659]
        [clojure.core$require invokeStatic core.clj 5911]
        [clojure.core$require doInvoke core.clj 5911]
        [clojure.lang.RestFn invoke RestFn.java 551]
        [army.core$eval24$loading__7531__auto____25 invoke core.clj 1]
        [army.core$eval24 invokeStatic core.clj 1]
        [army.core$eval24 invoke core.clj 1]
        [clojure.lang.Compiler eval Compiler.java 6951]
        [clojure.lang.Compiler eval Compiler.java 6940]
        [clojure.lang.Compiler load Compiler.java 7403]
        [clojure.lang.RT loadResourceScript RT.java 374]
        [clojure.lang.RT loadResourceScript RT.java 365]
        [clojure.lang.RT load RT.java 455]
        [clojure.lang.RT load RT.java 421]
        [clojure.core$load$fn__7645 invoke core.clj 6008]
        [clojure.core$load invokeStatic core.clj 6007]
        [clojure.core$load doInvoke core.clj 5991]
        [clojure.lang.RestFn invoke RestFn.java 408]
        [clojure.core$load_one invokeStatic core.clj 5812]
        [clojure.core$load_one invoke core.clj 5807]
        [clojure.core$load_lib$fn__7590 invoke core.clj 5852]
        [clojure.core$load_lib invokeStatic core.clj 5851]
        [clojure.core$load_lib doInvoke core.clj 5832]
        [clojure.lang.RestFn applyTo RestFn.java 142]
        [clojure.core$apply invokeStatic core.clj 659]
        [clojure.core$load_libs invokeStatic core.clj 5889]
        [clojure.core$load_libs doInvoke core.clj 5873]
        [clojure.lang.RestFn applyTo RestFn.java 137]
        [clojure.core$apply invokeStatic core.clj 659]
        [clojure.core$require invokeStatic core.clj 5911]
        [clojure.core$require doInvoke core.clj 5911]
        [clojure.lang.RestFn invoke RestFn.java 408]
        [user$eval15 invokeStatic form-init7134710474813340838.clj 1]
        [user$eval15 invoke form-init7134710474813340838.clj 1]
        [clojure.lang.Compiler eval Compiler.java 6951]
        [clojure.lang.Compiler eval Compiler.java 6940]
        [clojure.lang.Compiler eval Compiler.java 6940]
        [clojure.lang.Compiler load Compiler.java 7403]
        [clojure.lang.Compiler loadFile Compiler.java 7341]
        [clojure.main$load_script invokeStatic main.clj 276]
        [clojure.main$init_opt invokeStatic main.clj 278]
        [clojure.main$init_opt invoke main.clj 278]
        [clojure.main$initialize invokeStatic main.clj 309]
        [clojure.main$null_opt invokeStatic main.clj 343]
        [clojure.main$null_opt invoke main.clj 340]
        [clojure.main$main invokeStatic main.clj 422]
        [clojure.main$main doInvoke main.clj 385]
        [clojure.lang.RestFn applyTo RestFn.java 137]
        [clojure.lang.Var applyTo Var.java 700]
        [clojure.main main main.java 37]]}

Tuesday, August 16, 2016

Tar create/extract inside a specific directory

Create
tar -cf output.tar -C /path/to/directory .
The dot at the end is important

Extract
tar -xf input.tar -C /path/to/directory

Sunday, July 10, 2016

Why I'm writing in Clojure and Ruby

Sandy Metz. Around the 35th minute of http://bikeshed.fm/70

I came from Smalltalk, and I segued to Java a little bit, hated every minute of it, and then fell saved by Ruby. And so I have spent the last 20 some years writing code in languages that had enormous faith in my ability to do the right thing. And along with that enormous faith in my ability to do the right thing, they gave me the freedom of everything.
...

(My summary: She talked to a guy after a conference who asked her how using dynamic language could possibly work, and she answered)

You just trust people. And he said they are not trustworthy. And I said well, they shouldn't write Ruby, they should write Java. I really prefer languages that allow me to shoot myself in the foot if I do but give me the power to do that.
-----

  • Has enormous faith in my ability to do the right thing
  • Give me the freedom of everything
  • You just trust people

These are exactly why I'm writing in Clojure and Ruby.

Sunday, June 26, 2016

How to indent in Facebook post/comment on OS X


  1. Put your cursor on the text box of the post/comment you want to indent
  2. Go to the menubar at the top of the screen
  3. Edit > Emoji & Symbols
  4. On Characters window, click setting icon. It's a gear nearly the top-left of the window, close to 3 color buttons.
  5. Choose Customize Lists...
  6. Choose Code Tables > Unicode and Done
  7. Go to Unicode section at the end of the most left panel of Characters window
  8. Look for the character number 3000. It's called "IDEOGRAPHIC SPACE"
  9. Double click on that character to insert it to your text box. You can insert it as many characters as you want.
  10. Save your post/comment. This empty space should help you indent your text properly.
  11. You can add this character to Favorites section by click "Add to Favorites" button under the thumbnail of the character in the right most panel of the Characters window.
  12. Or you can just go to "Frequently Used" section and the character should already be there.

Saturday, June 4, 2016

How to comment in Clojure and my use case of each approach

There are 3 ways to comment code in Clojure. I personally use all 3 of them depending on the situation.

; [semicolon]
It's similar to // or # in the other languages. Source code from the point you put it until the end of line won't be executed.

I use it for a quick short documentation of the code or when I want to try remove a line quickly. It's the most easy to use one because all the IDEs and text editors support it out of the box.

(comment (expr)) [comment macro]
Use it to wrap expression you don't want it to be run. It's more convenient to use when using with tool to help maintaining balance of the parentheses such as paredit or parinfer.

I use this one a lot in while I'm doing interactive development via repl. I always put the code to execute function the I'm implementing in the same file with the function definition so that I can quickly test out when I change something. I wrap that code with comment macro so that I won't be execute automatically by Clojure. When I want, I send only the inside code to the repl to see what's the output of the function.

#_ reader macro
It comments out an expression that follows it, from opening to closing parens

This one I've just learnt about it recently. It comes in handy when I don't want to care about where is the end of the expression. I just put #_ and thing will behave as I expected. It's great for multiline comment, similar to the comment macro with a less keystrokes.


ClojureBridge is done a better explanation than me

Friday, May 20, 2016

Workaround for creating chart from messages with non-numeric fields in Graylog

As we all know, Graylog doesn't support creating a chart on non-numeric value at the moment. There is a neat workaround suggested by Drew Miranda that can fulfil this functionality. I'll try to rephrase it step by step.

  1. Go to Graylog UI page
  2. Make a search for the field and value that you want to represent as a chart
  3. Around the bottom left corner of search page, click all fields
  4. Find timestamp field and generate chart from it
  5. Around top right of the chart, Customize > Value > Total
That's it for a single value chart. But if you want a chart to contain multiple values, keeps going
  1. Redo 2-5 again for different fields
  2. Click and hold on hamburger menu button at the top right of each, drag to merge them
That's neat right?

Thursday, May 19, 2016

Add local registry mirror to docker daemon on systemd ubuntu (after 15.04)

After docker installation, follow this steps

  • Create directory /etc/systemd/system/docker.service.d if not exist
  • Create mirror.conf file inside that directory. The name "mirror" can be changed to anything.
  • In the file, add this lines
[Service]
ExecStart=
ExecStart=/usr/bin/docker daemon -H fd:// --registry-mirror=http://registry-host:registry-port
  • registry-host is hostname/ip of your local registry mirror
  • registry-port is port of your local registry mirror
  • Flush the change with $ sudo systemctl daemon-reload
  • Restart docker with $ sudo systemctl restart docker
  • Check if the change is successfully applied with $ systemctl status docker,  your config should appear somewhere in the command output.
Tested on Docker 1.11, Ubuntu 15.10 & 16.04

If you want more explanations about these commands, please refer to follow links. Thank you for all of them for providing me a direction to solve the issue and also come up with this post.

Thursday, May 12, 2016

Delete unused (orphaned) Docker volume

Since Docker 1.9, there's an easy way to find unused volumes and delete them.

To list unused volumes, use this command

$ docker volume ls -qf dangling=true

To delete them, use

$ docker volume rm $(docker volume ls -qf dangling=true)

Credit: I copied the whole thing from this SO answer

Tuesday, May 10, 2016

[Fixed] WARN Not all child routes satisfy compojure.api.routing/Routing. {:path nil, :method nil}, invalid child routes:

I had this error in my client app that uses compojure-api. The cause of this problem is that I put the static assets routing configuration in the wrong place. We put (route/resources "/") inside defapi which is a place for api routing configurations. So to fix this, we create a defroutes for static assets routing config and merge this defroutes with defapi before giving it to webserver as a ring handler. At the end, the code look something like this.

(ns my.namespace
  (:require [compojure.api.sweet :refer [defapi]]
            [compojure.core :refer [defroutes routes]]
            [compojure.route :as route]))

(defapi app
  ...)

(defroutes site
  ...
  (route/resources "/"))


(defn handler []
  (route site app))

Tuesday, April 26, 2016

Set daemon config for Docker for Mac

Just follow what describe in this link How to run a insecure-registry

Capture the screen in case the link is gone in the future


Unix, use output of the previous command

Not a direct solution but it suits my need.
Use !! to expand the previous command and evaluate it with $() or ``
For example;

$ which pinata  
/usr/local/bin/pinata 
$ ls -lh `!!`
... 
Credit: http://stackoverflow.com/a/5955674/1032449 

Thursday, February 25, 2016

Interesting use case example for `are` macro in Clojure test

I found an interesting use case for are in Clojure test.

You might have ever written a test for your data conversion logic. E.g.
  • Converting data pulled from third-party web service into your internal schema.
  • Converting data fetched from your database to more mobile friendly schema before sending out.
This is an example snippet of test for such a scenario.

convert-test1 shows the style I had been writing until recently. convert-test2 shows how using are making the test better.

Pros:
  • No more duplication of field values between input and output.
  • No more duplication between output schema and code under test schema.
  • It can be use as a document showing clearly how each individual field is mapped.
Cons:
An error message in convert-test1 is better. It shows clearly which field is wrong while convert-test2 throw out the whole problematic result data.

Updated: I have retested, are does report problem for an individual case as well.

Wednesday, February 10, 2016

[ClojureScript] Run all tests in multiple namespaces in REPL

From ClojureScript wiki, it says
If you have many test namespaces it's idiomatic to create a test runner namespace which imports all of your test namespaces and then invokes run-tests
I personally couldn't get it to work. Though I have managed to make run-all-tests works. This is an example

(ns myns.test.runner
  (:require [cljs.test :refer-macros [run-all-tests]]
            [myns.test.ns1]
            [myns.test.ns2]))

(run-all-tests #"myns.test.*")

[DataScript][Solved] Error: Expected number or lookup ref for entity id, got

When you are trying to use Map specification of Datomic's pull api and you got this error
Error: Expected number or lookup ref for entity id, got ....
This happens because you didn't specify schema for that data attribute. Schema is optional on DataScript but it is required in this case. Add this line to your database, the issue should be fixed
{:your-attribute1 {:db/valueType :db.type/ref}
 :your-attribute2 {:db/valueType :db.type/ref}}

Config number of file descriptors on Linux

Check file descriptor limit for a process. Vary depending on owner of the process $ cat /proc/{process-id}/limits Check current number of...