標準仕様だと ダウンロードしてもいいのですが 必要な td-agent を拡張する gem プラグインもインストールしたいのです。そこで github td-agent で提供されているソースから 必要な gem （td-agent のプラグイン）を全部入れた td-agent の rpm パッケージを作ってみます。
omnibus で、ruby と、td-agent をまるごとパッケージ化します。 yum で標準的に追加できる ruby とは別に、 /opt/td-agent 以下に td-agent だけで使用する ruby 環境が構築できます。
README.md にあるように ruby 2.1 以上ならいいみたいで、
We'll assume you have Ruby 2.1 and Bundler installed. First ensure all required gems are installed and ready to use:
しかしながら、 2.2.3 でもコンパイルできたときもあるけど、再現性がなかった。
td-agent の git log をみると 2016/6/6 の記述で、 ruby 2.1.10 で動作テストしたような記述があり、そのひとつあとの fluentd v0.12.26 に追従させたタグ 5a2235f34236c6f9186578b7fa3908640519afb5 でチェックアウトし、ruby 2.1.10 でパッケージを作ってみることにします。このタグは omnibus のバージョンが 5.4 に決め打ちされているのであとから再現できるのかなと思います（omnibus のバージョンもコロコロかわるので決め打ちされていないと、パッケージ作成が成功しない可能性が高い）。
いまのところ omnibus-software のバージョンが最新でも成功するが、バージョンアップされるとうまくいかない可能性もある、エラーがでる場合は、次のタグに固定するといい。
chef/omnibus-software dee0513e5234d5e42e9d0bdac90b1ba2d7876e7a
なお、ruby やら、 omunibus やら、omnibus-software の変更でよくでてくる現象は次の通りです。
Fork
edit Could not find rake-10.1.0
[Builder: fluentd-ui] I | Build fluentd-ui: 95.0097s
The following shell command exited with status 1:
$ BUNDLE_GEMFILE=Gemfile.production /opt/td-agent/embedded/bin/rake build
Output:
(nothing)
Error:
rake aborted!
/opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/bundler-1.10.6/lib/bundler/spec_set.rb:92:in `block in materialize': Could not find rake-10.1.0 in any of the sources (Bundler::GemNotFound)
td-agent を rpm パッケージでいれていたら削除してください。
$ sudo yum install http://ftp-srv2.kddilabs.jp/Linux/distributions/fedora/epel/7/x86_64/e/epel-release-7-8.noarch.rpm
$ sudo yum install gecode gecode-devel fakeroot $ sudo yum install -y openssl-devel readline-devel zlib-devel git $ sudo yum groupinstall 'Development tools'
$ git clone https://github.com/sstephenson/rbenv.git ~/.rbenv $ mkdir ~/.rbenv/plugins $ git clone https://github.com/sstephenson/ruby-build.git ~/.rbenv/plugins/ruby-build
export PATH=$HOME/.rbenv/bin:$PATH eval "$(rbenv init -)"
$ source ~/.bashrc
$ rbenv install 2.1.10 Downloading ruby-2.1.10.tar.bz2... -> https://cache.ruby-lang.org/pub/ruby/2.2/ruby-2.1.10.tar.bz2 Installing ruby-2.1.10... Installed ruby-2.1.10 to /home/yamachan/.rbenv/versions/2.1.10
$ git clone https://github.com/treasure-data/omnibus-td-agent.git $ cd omnibus-td-agent
[omnibus-td-agent]$ git checkout --force 5a2235f34236c6f9186578b7fa3908640519afb5
[omnibus-td-agent]$ rbenv local 2.1.10 [omnibus-td-agent]$ ruby -v ruby 2.1.10p492 (2016-04-01 revision 54464) [x86_64-linux] [omnibus-td-agent]$ rbenv exec gem install bundler
chef/omnibus-software dee0513e5234d5e42e9d0bdac90b1ba2d7876e7a
$ sudo mkdir -p /opt/td-agent /var/cache/omnibus
$ sudo chown yamachan:yamachan /opt/td-agent # 各ユーザー名で
$ sudo chown yamachan:yamachan /var/cache/omnibus # 各ユーザー名で
$ cd omnibus-td-agent
[omnibus-td-agent]$ bundle install --binstubs
[omnibus-td-agent]$ bin/gem_downloader core_gems.rb
[omnibus-td-agent]$ bin/gem_downloader plugin_gems.rb
[omnibus-td-agent]$ bin/gem_downloader ui_gems.rb
[omnibus-td-agent]$ bin/omnibus build td-agent2
略
[Packager::RPM] I | Creating .rpm file
[Project: td-agent] I | Building version manifest
[Compressor] I | No compressor defined for `rhel'.
[Project: td-agent] I | Building version manifest
[omnibus-td-agent]$ ls -la pkg
total 60488
drwxrwxr-x 2 yamachan yamachan 124 Aug 7 18:36 .
drwxrwxr-x 16 yamachan yamachan 4096 Aug 8 10:48 ..
-rw-rw-r-- 1 yamachan yamachan 61910902 Aug 8 10:45 td-agent-2.3.1-0.el7.x86_64.rpm
-rw-rw-r-- 1 yamachan yamachan 13686 Aug 8 10:45 td-agent-2.3.1-0.el7.x86_64.rpm.metadata.json
-rw-rw-r-- 1 yamachan yamachan 6906 Aug 8 10:45 version-manifest.json
現象:
[omnibus-td-agent]$ bin/omnibus build td-agent2
[CLI] I | Using config from 'omnibus.rb'
Attempting to apply the patch `ruby-2_1_3-no-mkmf.patch', but it was not found at any of the
following locations:
/home/yamachan/.rbenv/versions/2.1.10/lib/ruby/gems/2.1.0/bundler/gems/omnibus-software-46b5677a209d/config/patches/ruby/ruby-2_1_3-no-mkmf.patch
[omnibus-td-agent]$ wget https://raw.githubusercontent.com/habitat-sh/core-plans/master/ruby/patches/ruby-2_1_3-no-mkmf.patch [omnibus-td-agent]$ mv ruby-2_1_3-no-mkmf.patch /home/yamachan/.rbenv/versions/2.1.10/lib/ruby/gems/2.1.0/bundler/gems/omnibus-software-46b5677a209d/config/patches/ruby/
[omnibus-td-agent]$ git config --global user.email "yamachan@piwikjapan.org" [omnibus-td-agent]$ git config --global user.name "YAMAMOTO Takashi"
現象:
[NetFetcher: zlib] I | Downloading from `http://iweb.dl.sourceforge.net/project/libpng/zlib/1.2.8/zlib-1.2.8.tar.gz'
略
[NetFetcher: zlib] I | Retrying failed download due to 404 Not Found (5 retries left)...
[NetFetcher: zlib] I | Retrying failed download due to 404 Not Found (4 retries left)... 0% (0 KB/sec)
[NetFetcher: zlib] I | Retrying failed download due to 404 Not Found (3 retries left)... 0% (0 KB/sec)
略
[NetFetcher: zlib] I | Retrying failed download due to 404 Not Found (2 retries left)...
略
[NetFetcher: zlib] I | Retrying failed download due to 404 Not Found (1 retries left)...
[NetFetcher: zlib] E | Download failed - OpenURI::HTTPError! 0% (0 KB/sec)
/home/yamachan/.rbenv/versions/2.1.10/lib/ruby/2.1.0/open-uri.rb:353:in `open_http': 404 Not Found (OpenURI::HTTPError)
[omnibus-td-agent]$ wget http://zlib.net/zlib-1.2.8.tar.gz [omnibus-td-agent]$ mv zlib-1.2.8.tar.gz /var/cache/omnibus/cache/
$ sudo yum install GeoIP-devel GeoIP GeoIP-update
$ sudo yum install mariadb-server mariadb-devel
download "fluent-plugin-norikra", "0.4.2" download "fluent-plugin-elasticsearch", "1.5.0" download "fluent-plugin-record-reformer", "0.8.1" download "fluent-plugin-uri_decoder", "0.2.0" download "fluent-plugin-geoip", "0.6.1" download "fluent-plugin-mail", "0.2.4" download "fluent-plugin-mysql-replicator", "0.5.2"
[omnibus-td-agent]$ bin/gem_downloader plugin_gems.rb [omnibus-td-agent]$ bin/omnibus build td-agent2
現象:
[omnibus-td-agent]$ bin/omnibus build td-agent2
略
[HealthCheck] I | Running health on td-agent
[HealthCheck] E | Failed!
[HealthCheck] E | The following libraries have unsafe or unmet dependencies:
--> /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/geoip-c-0.9.1/ext/geoip/geoip.so
--> /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/geoip-c-0.9.1/lib/geoip.so
--> /opt/td-agent/embedded/lib/ruby/gems/2.1.0/extensions/x86_64-linux/2.1.0/geoip-c-0.9.1/geoip.so
[HealthCheck] E | The following binaries have unsafe or unmet dependencies:
[HealthCheck] E | The following libraries cannot be guaranteed to be on target systems:
--> /lib64/libGeoIP.so.1 (0x00007fdbf1c54000)
--> /lib64/libGeoIP.so.1 (0x00007f15825b5000)
--> /lib64/libGeoIP.so.1 (0x00007fed88568000)
$ sudo updatedb $ locate health_check.rb /home/yamachan/.rbenv/versions/2.1.10/lib/ruby/gems/2.1.0/gems/omnibus-5.4.0/lib/omnibus/health_check.rb
[omnibus]$ diff -u health_check.rb.orig health_check.rb
--- health_check.rb.orig 2016-08-07 21:06:39.611277359 +0900
+++ health_check.rb 2016-08-07 21:21:58.474090296 +0900
@@ -43,6 +43,18 @@
/libutil\.so/,
/linux-vdso.+/,
/linux-gate\.so/,
+ /libGeoIP\.so/,
+ /libmysqlclient\.so/,
+ /libcom_err\.so/,
+ /libcrypto\.so/,
+ /libgssapi_krb5\.so/,
+ /libk5crypto\.so/,
+ /libkeyutils\.so/,
+ /libkrb5\.so/,
+ /libkrb5support\.so/,
+ /libpcre\.so/,
+ /libselinux\.so/,
+ /libssl\.so/,
].freeze
ARCH_WHITELIST_LIBS = [
$ sudo yum install GeoIP-update $ sudo geoipupdate $ sudo rm -rf /opt/td-agent $ sudo yum install redhat-lsb-core $ cd omnibus-td-agent [omnibus-td-agent]$ sudo yum install pkg/td-agent-2.3.1-0.el7.x86_64.rpm
$ sudo yum install GeoIP GeoIP-update $ sudo geoipupdate $ sudo yum install redhat-lsb-core $ sudo yum install td-agent-2.3.1-0.el7.x86_64.rpm
[PageInfo]
LastUpdate: 2016-10-05 11:14:52, ModifiedBy: selfnavi
[Permissions]
view:all, edit:login users, delete/config:members