Yasuhiro Horimoto 2018-12-19 12:02:28 +0900 (Wed, 19 Dec 2018) Revision: 09f99595e6a248a62c19db82886e2892b0793f92 https://github.com/groonga/groonga/commit/09f99595e6a248a62c19db82886e2892b0793f92 Message: doc: add explain about simple case Copied files: doc/source/example/reference/tokenizers/token-delimit-delimiter-option-multiple-delimiters.log (from doc/source/example/reference/tokenizers/token-delimit-delimiter-option.log) Modified files: doc/locale/ja/LC_MESSAGES/reference.po doc/source/example/reference/tokenizers/token-delimit-delimiter-option.log doc/source/reference/tokenizers.rst Modified: doc/locale/ja/LC_MESSAGES/reference.po (+21 -7) =================================================================== --- doc/locale/ja/LC_MESSAGES/reference.po 2018-12-19 11:33:59 +0900 (e0c668694) +++ doc/locale/ja/LC_MESSAGES/reference.po 2018-12-19 12:02:28 +0900 (f41727b79) @@ -27269,16 +27269,22 @@ msgstr "" "``TokenDelimit`` はオプションを指定することもできます。 ``TokenDelimit`` は " "``delimiter`` オプションと ``pattern`` オプションを持っています。" +msgid "``delimiter`` option can split token with a specified characters." +msgstr "``delimiter`` オプションは、指定した文字でトークンに分割できます。" + msgid "" -"``delimiter`` option can split token with a specified characters. " -"``delimiter`` option can also specify multiple delimiters." +"For example, ``Hello,World`` is tokenized to ``Hello`` and ``World`` with " +"``delimiter`` option as below." msgstr "" -"``delimiter`` オプションは、指定した文字でトークンに分割できます。" -"``delimiter`` オプションは、複数の区切り文字を指定することもできます。" +"例えば、以下のように ``Hello,World`` は、``Hello`` と ``World`` にトークナイ" +"ズされます。" + +msgid "``delimiter`` option can also specify multiple delimiters." +msgstr "``delimiter`` オプションは、複数の区切り文字を指定することもできます。" msgid "" -"For example, ``Hello, World`` is tokenize to ``Hello`` and ``World`` with " -"``delimiter`` option as below." +"For example, ``Hello, World`` is tokenized to ``Hello`` and ``World``. ``,`` " +"and `` `` are delimiters in below example." msgstr "" "例えば、以下のように ``Hello, World`` は、``Hello`` と ``World`` にトークナイ" "ズされます。" @@ -27291,7 +27297,7 @@ msgstr "" "ションを使うことで、不要な空白を除去することができます。" msgid "" -"For example, ``This is a pen. This is an apple`` is tokenize to ``This is a " +"For example, ``This is a pen. This is an apple`` is tokenized to ``This is a " "pen`` and ``This is an apple`` with ``pattern`` option as below." msgstr "" "例えば、以下のように ``pattern`` オプションによって、 ``This is a pen. This " @@ -27981,3 +27987,11 @@ msgstr "" msgid "``window_sum``" msgstr "" + +#~ msgid "" +#~ "For example, ``This is a pen. This is an apple`` is tokenize to ``This is " +#~ "a pen`` and ``This is an apple`` with ``pattern`` option as below." +#~ msgstr "" +#~ "例えば、以下のように ``pattern`` オプションによって、 ``This is a pen. " +#~ "This is an apple`` は ``This is a pen`` と ``This is an apple`` にトークナ" +#~ "イズされます。" Copied: doc/source/example/reference/tokenizers/token-delimit-delimiter-option-multiple-delimiters.log (+0 -0) 100% =================================================================== Modified: doc/source/example/reference/tokenizers/token-delimit-delimiter-option.log (+1 -1) =================================================================== --- doc/source/example/reference/tokenizers/token-delimit-delimiter-option.log 2018-12-19 11:33:59 +0900 (0782b24ef) +++ doc/source/example/reference/tokenizers/token-delimit-delimiter-option.log 2018-12-19 12:02:28 +0900 (ffa850df8) @@ -1,6 +1,6 @@ Execution example:: - tokenize 'TokenDelimit("delimiter", ",", "delimiter", " ")' "Hello, World" + tokenize 'TokenDelimit("delimiter", ",")' "Hello,World" # [ # [ # 0, Modified: doc/source/reference/tokenizers.rst (+12 -3) =================================================================== --- doc/source/reference/tokenizers.rst 2018-12-19 11:33:59 +0900 (2fc150fe7) +++ doc/source/reference/tokenizers.rst 2018-12-19 12:02:28 +0900 (78a132c1f) @@ -433,19 +433,28 @@ Here is an example of ``TokenDelimit``: ``TokenDelimit`` has ``delimiter`` option and ``pattern`` option. ``delimiter`` option can split token with a specified characters. -``delimiter`` option can also specify multiple delimiters. -For example, ``Hello, World`` is tokenize to ``Hello`` and ``World`` +For example, ``Hello,World`` is tokenized to ``Hello`` and ``World`` with ``delimiter`` option as below. .. groonga-command .. include:: ../example/reference/tokenizers/token-delimit-delimiter-option.log +.. tokenize 'TokenDelimit("delimiter", ",")' "Hello,World" + + +``delimiter`` option can also specify multiple delimiters. + +For example, ``Hello, World`` is tokenized to ``Hello`` and ``World``. +``,`` and `` `` are delimiters in below example. + +.. groonga-command +.. include:: ../example/reference/tokenizers/token-delimit-delimiter-option-multiple-delimiters.log .. tokenize 'TokenDelimit("delimiter", ",", "delimiter", " ")' "Hello, World" ``pattern`` option can split token with a regular expression. You can except needless space by ``pattern`` option. -For example, ``This is a pen. This is an apple`` is tokenize to ``This is a pen`` and +For example, ``This is a pen. This is an apple`` is tokenized to ``This is a pen`` and ``This is an apple`` with ``pattern`` option as below. Normally, when ``This is a pen. This is an apple.`` is splitted by ``.``, -------------- next part -------------- An HTML attachment was scrubbed... URL: <https://lists.osdn.me/mailman/archives/groonga-commit/attachments/20181219/9aded906/attachment-0001.html>
