qコマンドいんすこ

参考文献

ぉぉぉ

CSV・TSVファイルでSQLを利用する「q」コマンド  

ぉぉぉ

CSVデータに対してSQLのwhere、order by、group by、joinができる `q` コマンド(導入編)  

ぉぉぉ

UNIX/Linux環境でcsv/tsvにクエリを投げることができるqが便利すぎて生きるのが辛い  

事前確認

コード表示

[root@eac201c4324b ~]# which py*
which: no py* in (/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin)
[root@eac201c4324b ~]# find /usr{/,/local/}bin -name "*py*"
/usr/bin/objcopy
/usr/bin/python2.7
/usr/bin/python2
/usr/bin/pydoc
/usr/bin/python
[root@eac201c4324b ~]# curl -s http://harelba.github.io/q/install.html | grep rpm_download_link | awk -F\" '{print $4}'
https://github.com/harelba/packages-for-q/raw/master/rpms/q-text-as-data-1.7.1-1.noarch.rpm
[root@eac201c4324b ~]# yum list installed | grep q-text-as-data-1.7.1-1.noarch.rpm
[root@eac201c4324b ~]# curl -s http://harelba.github.io/q/install.html | grep rpm_download_link | awk -F\" '{print $4}'
https://github.com/harelba/packages-for-q/raw/master/rpms/q-text-as-data-1.7.1-1.noarch.rpm
[root@eac201c4324b ~]# yum list installed | grep q-text-as-data-1.7.1-1.noarch.rpm
[root@eac201c4324b ~]# yum search q-text-as-data-1.7.1-1.noarch.rpm
Loaded plugins: fastestmirror, ovl
Loading mirror speeds from cached hostfile
 * base: ftp-srv2.kddilabs.jp
 * epel: fedora.cs.nctu.edu.tw
 * extras: ftp-srv2.kddilabs.jp
 * updates: ftp-srv2.kddilabs.jp
Warning: No matches found for: q-text-as-data-1.7.1-1.noarch.rpm
No matches found
[root@eac201c4324b ~]# which q
which: no q in (/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin)
[root@eac201c4324b ~]# find /usr{/,/local/}bin -name "q"
[root@eac201c4324b ~]# curl -s http://harelba.github.io/q/install.html | grep rpm_download_link | awk -F\" '{print $4}' | xargs rpm -ihv
Retrieving https://github.com/harelba/packages-for-q/raw/master/rpms/q-text-as-data-1.7.1-1.noarch.rpm
Preparing...                          ################################# [100%]
Updating / installing...
   1:q-text-as-data-1.7.1-1.el6       ################################# [100%]
[root@eac201c4324b ~]# which q
/usr/bin/q
[root@eac201c4324b ~]# find /usr{/,/local/}bin -name "q"
/usr/bin/q
[root@eac201c4324b ~]# q --version
q version 1.7.1
Copyright (C) 2012-2017 Harel Ben-Attia (harelba@gmail.com, @harelba on twitter)
http://harelba.github.io/q/

[root@eac201c4324b ~]# head /usr/bin/q
#!/usr/bin/env python

#   Copyright (C) 2012-2014 Harel Ben-Attia
#
#   This program is free software; you can redistribute it and/or modify
#   it under the terms of the GNU General Public License as published by
#   the Free Software Foundation; either version 3, or (at your option)
#   any later version.
#
#   This program is distributed in the hope that it will be useful,
[root@eac201c4324b ~]# q
q version 1.7.1
Copyright (C) 2012-2017 Harel Ben-Attia (harelba@gmail.com, @harelba on twitter)
http://harelba.github.io/q/

Must provide at least one query in the command line, or through a file with the -q parameter

どうさかくにん

コード表示

[root@eac201c4324b ~]# q
q version 1.7.1
Copyright (C) 2012-2017 Harel Ben-Attia (harelba@gmail.com, @harelba on twitter)
http://harelba.github.io/q/

Must provide at least one query in the command line, or through a file with the -q parameter
[root@eac201c4324b ~]# time seq 1 1000 | q "select avg(c1),sum(c1) from -"
Warning: column count is one - did you provide the correct delimiter?
500.5 500500

real	0m0.054s
user	0m0.029s
sys	0m0.012s
[root@eac201c4324b ~]# time seq 1 1000000 | q "select avg(c1),sum(c1) from -"
Warning: column count is one - did you provide the correct delimiter?
500000.5 500000500000

real	0m3.953s
user	0m3.923s
sys	0m0.057s
[root@eac201c4324b ~]# time seq 1 100000000 | q "select avg(c1),sum(c1) from -"
Warning: column count is one - did you provide the correct delimiter?
^CInterrupted


real	1m42.797s
user	1m42.436s
sys	0m1.206s

あしたあそぼ

ねる。