2013-04-03 15:55  sloot

	* [r15925] src/Makefile.am: increment current library version

2013-04-03 14:36  sloot

	* [r15918] tests/test.nl.tok.V: adapt to new tokenisation rules

2013-04-03 13:57  sloot

	* [r15910] include/ucto/tokenize.h, src/tokenize.cxx, src/ucto.cxx,
	  src/unicode.cxx: changed folia include path

2013-04-03 13:03  sloot

	* [r15905] NEWS, configure.ac: NEWS

2013-04-02 16:09  sloot

	* [r15885] configure.ac: we need the most recent ticcutile

2013-04-02 15:51  sloot

	* [r15875] include/ucto/tokenize.h, src/tokenize.cxx, src/ucto.cxx,
	  src/unicode.cxx: modified folia include path

2013-04-02 08:55  sloot

	* [r15851] include/ucto/Makefile.am, src/tst.sh: tags

2013-03-25 16:09  sloot

	* [r15810] src/tokenize.cxx: fixed WORD-WITHSUFFIX

2013-03-25 14:38  sloot

	* [r15809] src/tokenize.cxx: more quote hacking

2013-03-25 14:20  sloot

	* [r15807] tests/testquotes.ok, tests/testslash.ok: some rule neams
	  have changed

2013-03-25 14:18  sloot

	* [r15806] config/tokconfig-nl: small edits

2013-03-25 13:45  sloot

	* [r15805] config/tokconfig-nl, src/tokenize.cxx,
	  tests/testquotes.ok, tests/testslash.ok: some rexexp patches for
	  words like A50, vanessa23 etc.
	  adaptes tests

2013-03-25 12:19  sloot

	* [r15803] config/tokconfig-nl: more quotes for '40 etc.

2013-03-25 11:20  sloot

	* [r15802] config/tokconfig-nl: small fix for prime's/quotes

2013-03-21 14:03  sloot

	* [r15797] src/tokenize.cxx: added small hack to appen 'dangling'
	  quotes to the previous sentence.
	  This avoids sentences with only a ' or a ". (extend to exotic
	  quotest too?)

2013-03-06 13:50  sloot

	* [r15774] src/ucto.cxx: be sure to take "current" as the default
	  class for text searching

2013-03-06 13:49  sloot

	* [r15773] tests/testfoliain.ok: oesp.

2013-03-06 12:23  mvgompel

	* [r15769] config/tokconfig-de, config/tokconfig-en,
	  config/tokconfig-es, config/tokconfig-fr, config/tokconfig-it:
	  fixed ABBREVIATION rule for other languages

2013-03-06 10:40  sloot

	* [r15766] docs/ucto.1: Updated.

2013-03-06 10:28  sloot

	* [r15765] include/ucto/tokenize.h, src/tokenize.cxx, src/ucto.cxx:
	  added a --textclass option

2013-03-06 10:28  sloot

	* [r15764] tests/folia5.xml, tests/testfoliain,
	  tests/testfoliain.ok, tests/testoption1.ok, tests/testusage.ok:
	  added test for the new --textclass option

2013-02-21 14:55  sloot

	* [r15693] src/tokenize.cxx: added the passthru fix for FoLia docs
	  also elsewhere

2013-02-18 13:42  sloot

	* [r15683] src/tokenize.cxx: passtru mode was handled incorrectly
	  for FoLiA input

2013-02-18 11:12  sloot

	* [r15682] src/tokenize.cxx: make sure that we tokenize existing
	  FoLiA in our own set!

2013-02-14 12:28  mvgompel

	* [r15675] config/Makefile.am: added to makefile

2013-02-14 12:26  mvgompel

	* [r15674] config/es.abr, config/tokconfig-de, config/tokconfig-en,
	  config/tokconfig-es, config/tokconfig-fr: updated foreign
	  tokenisation configurations

2013-02-07 13:11  sloot

	* [r15648] src/tokenize.cxx: some debugging lines silenced

2013-02-07 11:19  sloot

	* [r15646] tests/bracket.nl.tok.V, tests/bracket.nl.txt: added a
	  test for the EOS in brackets problem.

2013-02-07 11:11  sloot

	* [r15645] src/tokenize.cxx, tests/test.py: added some trickery to
	  transfer EOS markers to next token if it is
	  a closing bracket. EXPERIMENTAL

2013-01-10 11:03  sloot

	* [r15587] tests/testone: improved test

2013-01-10 10:52  sloot

	* [r15586] tests/utf16bom.nl, tests/utf8bom.nl: we need this files
	  too

2013-01-10 10:48  sloot

	* [r15585] include/ucto/tokenize.h, src/tokenize.cxx: now we handle
	  (and use) BOM markers.

2013-01-10 10:48  sloot

	* [r15584] tests/testencoding2, tests/testencoding2.ok: added a
	  test for the BOM

2013-01-07 14:54  sloot

	* [r15571] include/ucto/tokenize.h, include/ucto/unicode.h,
	  src/tokenize.cxx, src/ucto.cxx, src/unicode.cxx: Bump year

2012-12-06 16:10  mvgompel

	* [r15532] docs/ucto.1: wrong man

2012-12-06 14:43  sloot

	* [r15529] config/nl_afk.abr: removed 'al' beacuse it isn't an
	  abrivation, ans it IS a word!

2012-12-05 22:23  sloot

	* [r15528] src/tokenize.cxx, tests/testfolia.ok,
	  tests/testfolia2.ok, tests/testfoliain.ok, tests/testslash.ok:
	  use datetime='now()' in declarations. And test that too

2012-12-05 16:16  sloot

	* [r15522] src/tokenize.cxx: add datetime attribute to
	  token-annotation declaration

2012-12-05 14:08  sloot

	* [r15520] src/tokenize.cxx: only add a set default definition when
	  there is something to do

2012-12-05 14:07  sloot

	* [r15519] config/tokconfig-nl-twitter: removed odd typo

2012-12-05 14:04  sloot

	* [r15518] config/Makefile.am, config/nl_afk, config/nl_afk.abr:
	  fixed

2012-11-30 19:13  mvgompel

	* [r15490] config/Makefile.am: added to config data

2012-11-30 19:12  mvgompel

	* [r15489] config/nl_afk, config/tokconfig-nl-twitter: adding
	  forgotten file

2012-11-28 10:52  sloot

	* [r15480] config/tokconfig-nl-sonarchat,
	  config/tokconfig-nl-twitter: use the nl_afk abbreviations file

2012-11-28 10:29  sloot

	* [r15479] config/tokconfig-nl, include/ucto/tokenize.h,
	  src/tokenize.cxx: implemented %include for ABBREVIATIONS

2012-11-28 10:06  mvgompel

	* [r15478] docs/ucto_manual.pdf, docs/ucto_manual.tex: docupdate

2012-11-26 22:38  mvgompel

	* [r15472] docs/ucto_manual.pdf, docs/ucto_manual.tex: updated
	  after comments

2012-11-26 13:17  mvgompel

	* [r15468] docs/ucto_manual.bib, docs/ucto_manual.pdf,
	  docs/ucto_manual.tex: Added documentation (finally)

2012-11-01 14:51  sloot

	* [r15397] tests/reverse-smiley.nl.tok.V,
	  tests/reverse-smiley.nl.txt, tests/smileys.nl.tok.V,
	  tests/smileys.nl.txt: added a test for reversed smileys.
	  addapted the smiley testbtoo.

2012-11-01 14:43  sloot

	* [r15396] config/smiley.rule: attempt to support 'repetitive'
	  smileys like this :)))))

2012-11-01 14:13  sloot

	* [r15395] config/tokconfig-nl-sonarchat,
	  config/tokconfig-nl-twitter: added REVERSE-SMILEY rule

2012-11-01 14:09  sloot

	* [r15394] config/smiley.rule, config/tokconfig-nl: attempt to
	  define a rule for reverse smileys {-8.

2012-11-01 09:33  sloot

	* [r15387] config/tokconfig-nl-twitter: get in line with standard
	  tokconfig-nl

2012-10-31 17:04  sloot

	* [r15386] tests/testall, tests/testtwitter, tests/testtwitter.ok,
	  tests/twitter.txt: added a test

2012-10-30 12:36  mvgompel

	* [r15361] bootstrap: bootstrap fix (matched failed on automake
	  1.11.6 with Ubuntu 12.10)

2012-10-30 09:56  sloot

	* [r15355] src/tokenize.cxx: adapted to changed FoLiA stylesheet
	  handling

2012-10-11 09:42  sloot

	* [r15278] tests/testfolia.ok, tests/testfolia2.ok,
	  tests/testfoliain.ok, tests/testslash.ok: adapted to improved
	  libfolia output (explicit UTF-8 encoding)

2012-10-10 12:45  sloot

	* [r15275] include/ucto/tokenize.h, include/ucto/unicode.h:
	  safeguarded the safeguards

2012-09-04 14:23  sloot

	* [r15158] configure.ac: unneeded check

2012-08-08 09:04  sloot

	* [r15062] COPYING, configure.ac, include/ucto/tokenize.h,
	  src/tokenize.cxx: use ticcutils in TiCC namespace

2012-08-06 13:27  sloot

	* [r15027] tests/test.es.txt, tests/test.es.txt.disabled,
	  tests/test.fr.txt, tests/test.fr.txt.disabled, tests/test.pt.txt,
	  tests/test.pt.txt.disabled: teporary disabled until Proycon does
	  his job

2012-08-06 12:54  sloot

	* [r15026] config/tokconfig-en: improved english tokenization rules

2012-08-06 12:53  sloot

	* [r15025] tests/test.en.tok.V: accaptable result

2012-08-06 10:20  sloot

	* [r15024] config/tokconfig-de: V.S. en d.h. gaan nu goed.
	  a.d.Donau misschien niet

2012-08-06 09:45  sloot

	* [r15023] tests/test.de.tok.V: ik denk dat dit dichter tegen de
	  waarheid zit.
	  Nu nog zorgen dat het er uit komt :{

2012-08-01 16:21  sloot

	* [r15022] tests/test.de.tok.V: new

2012-08-01 16:10  sloot

	* [r15021] config/tokconfig-de, src/tokenize.cxx: attempt to fix
	  .de

2012-08-01 13:21  sloot

	* [r15020] include/ucto/tokenize.h: keep it private

2012-08-01 12:54  sloot

	* [r15019] include/ucto/tokenize.h, src/tokenize.cxx: save
	  intermediate refactored state

2012-07-31 10:44  sloot

	* [r15016] src/tokenize.cxx: some cleanup

2012-07-31 10:02  sloot

	* [r15015] include/ucto/tokenize.h, src/tokenize.cxx: fixed eos
	  problems. '-s' option seems to work now.

2012-07-31 10:01  sloot

	* [r15014] tests/testall: test the new test

2012-07-31 10:00  sloot

	* [r15013] tests/eos.txt, tests/testoption-s,
	  tests/testoption-s.ok, tests/utt.txt: added a new test

2012-07-30 15:59  sloot

	* [r15012] include/ucto/tokenize.h, src/tokenize.cxx: better info.

2012-07-30 15:42  sloot

	* [r15011] src/tokenize.cxx: typo, and some refactoring.

2012-07-30 13:44  sloot

	* [r15010] include/ucto/tokenize.h, src/tokenize.cxx: small cleanup

2012-07-30 13:28  sloot

	* [r15009] include/ucto/tokenize.h, src/tokenize.cxx: further
	  refactoring

2012-07-23 14:57  sloot

	* [r15005] include/ucto/tokenize.h, src/tokenize.cxx:
	  re-re-re-factored :P

2012-07-23 14:13  sloot

	* [r15004] src/tokenize.cxx: one small step…

2012-07-23 13:09  sloot

	* [r15003] include/ucto/tokenize.h, src/tokenize.cxx: more cleanup.

2012-07-23 09:55  sloot

	* [r15001] include/ucto/tokenize.h, src/tokenize.cxx: some more
	  refactoring, and added some comment

2012-07-18 15:20  sloot

	* [r15000] src/tokenize.cxx: ok, more next week

2012-07-18 14:48  sloot

	* [r14999] include/ucto/tokenize.h, src/tokenize.cxx: next refactor
	  step

2012-07-18 13:31  sloot

	* [r14998] include/ucto/tokenize.h, src/tokenize.cxx: hmm.
	  progress…

2012-07-18 12:59  sloot

	* [r14997] src/tokenize.cxx: save intermediate result during
	  cleanup

2012-07-18 12:33  sloot

	* [r14996] include/ucto/tokenize.h, src/tokenize.cxx: refactoring:
	  removed parameters without any visable changes in behaviour.

2012-07-18 08:44  fkarsdorp

	* [r14995] tests/quotetest_folgert8.nl.tok.V: Correct the test
	  case.

2012-07-18 08:25  sloot

	* [r14994] AUTHORS: Honour Folgert

2012-07-17 15:36  sloot

	* [r14993] include/ucto/tokenize.h, src/tokenize.cxx: indentation
	  and such

2012-07-17 15:13  sloot

	* [r14992] include/ucto/tokenize.h, src/tokenize.cxx: some renaming
	  for my own mental health

2012-07-17 14:40  sloot

	* [r14991] src/tokenize.cxx: remove unused functionm

2012-07-17 14:26  sloot

	* [r14990] include/ucto/tokenize.h, src/tokenize.cxx: started some
	  refactoring

2012-07-17 12:25  sloot

	* [r14989] tests/folia2.txt, tests/testall, tests/testfolia2,
	  tests/testfolia2.ok: added a complex folia test. asume the result
	  is OK for now.

2012-07-16 15:26  sloot

	* [r14988] tests/bug0083.nl.tok.V: I think this more in line with
	  wat we want.
	  still doubtfull quotation

2012-07-16 15:22  sloot

	* [r14987] src/tokenize.cxx: attempt to fix bug94 (fixes
	  quotetest_folgert7 too??)

2012-07-16 15:00  sloot

	* [r14986] tests/quotetest_folgert7.nl.tok.V: I think the correct
	  file wasn't correct at all. Folgert??

2012-07-10 13:44  sloot

	* [r14972] src/tokenize.cxx: fixed debugging lines

2012-07-05 13:50  sloot

	* [r14962] NEWS: updated NEWS

2012-07-05 13:47  sloot

	* [r14961] src/tokenize.cxx: fixed folia id's

2012-07-03 12:54  mvgompel

	* [r14947] config/tokconfig-fy: partial revert

2012-07-03 12:53  mvgompel

	* [r14946] config/tokconfig-fy: fix

2012-07-03 12:48  mvgompel

	* [r14945] config/tokconfig-fy: update

2012-06-13 16:05  sloot

	* [r14861] tests/bug0094.nl.tok.V, tests/bug0094.nl.txt,
	  tests/quotetest_folgert.nl.tok.V, tests/quotetest_folgert.nl.txt,
	  tests/quotetest_folgert2.nl.tok.V,
	  tests/quotetest_folgert2.nl.txt,
	  tests/quotetest_folgert3.nl.tok.V,
	  tests/quotetest_folgert3.nl.txt,
	  tests/quotetest_folgert4.nl.tok.V,
	  tests/quotetest_folgert4.nl.txt,
	  tests/quotetest_folgert5.nl.tok.V,
	  tests/quotetest_folgert5.nl.txt,
	  tests/quotetest_folgert6.nl.tok.V,
	  tests/quotetest_folgert6.nl.txt,
	  tests/quotetest_folgert7.nl.tok.V,
	  tests/quotetest_folgert7.nl.txt,
	  tests/quotetest_folgert8.nl.tok.V,
	  tests/quotetest_folgert8.nl.txt, tests/testquotes,
	  tests/testquotes.ok: properties

2012-06-13 16:03  sloot

	* [r14860] tests/bug0094.nl.tok.V, tests/bug0094.nl.txt: new bug
	  found

2012-06-11 13:01  sloot

	* [r14847] tests/bug0051.nl.tok.V, tests/bug0052.nl.tok.V,
	  tests/bug0065b.nl.tok.V, tests/quotetest_folgert.nl.tok.V,
	  tests/quotetest_folgert2.nl.tok.V,
	  tests/quotetest_folgert3.nl.tok.V,
	  tests/quotetest_folgert4.nl.tok.V,
	  tests/quotetest_folgert5.nl.tok.V,
	  tests/quotetest_folgert6.nl.tok.V,
	  tests/quotetest_folgert7.nl.tok.V,
	  tests/quotetest_folgert8.nl.tok.V,
	  tests/quotetest_multiline.nl.tok.V,
	  tests/quotetest_multisentence.nl.tok.V,
	  tests/quotetest_nested2.nl.tok.V,
	  tests/quotetest_onesentence.nl.tok.V, tests/test.nl.tok.V,
	  tests/testquotes.ok: addapted to changed verbose output

2012-06-11 12:50  sloot

	* [r14846] tests/bug0083.nl.tok.V: ok, we are more happy now, but
	  there is still a BEGINOFSENTENCE missing I think

2012-06-11 12:42  sloot

	* [r14845] src/tokenize.cxx: fixed indentation of the source
	  also always put an extra newline after ENDOFSENNTENCE in verbose
	  mode

2012-06-11 11:12  sloot

	* [r14844] tests/nu.nl.tok.V: dit is al meer wat we willen

2012-06-11 11:04  sloot

	* [r14843] tests/test.py: reversed parameters to diff.
	  now we do 'diff <what we got> <what we want>"
	  This is more in line with other tests

2012-06-04 11:25  fkarsdorp

	* [r14833] COPYING, config/tokconfig-nl, src/tokenize.cxx,
	  tests/quotetest_folgert5.nl.tok.V,
	  tests/quotetest_folgert5.nl.txt,
	  tests/quotetest_folgert6.nl.tok.V,
	  tests/quotetest_folgert6.nl.txt,
	  tests/quotetest_folgert7.nl.tok.V,
	  tests/quotetest_folgert7.nl.txt,
	  tests/quotetest_folgert8.nl.tok.V,
	  tests/quotetest_folgert8.nl.txt: Fix nested quoted sentences;
	  added testcases; small additions to tokconfig-nl

2012-05-31 13:56  fkarsdorp

	* [r14826] include/ucto/tokenize.h: update header, u_isquote

2012-05-31 13:56  fkarsdorp

	* [r14825] src/tokenize.cxx: getting used to svn...

2012-05-31 12:55  fkarsdorp

	* [r14824] tests/quotetest_folgert4.nl.tok.V,
	  tests/quotetest_folgert4.nl.txt: small fix in quote detection,
	  testcase added.

2012-05-30 15:11  sloot

	* [r14823] configure.ac, src/Makefile.am: exterminated
	  AC_DEFINE_DIR here too

2012-05-30 13:50  mvgompel

	* [r14820] NEWS: updated

2012-05-30 13:48  mvgompel

	* [r14819] src/tokenize.cxx, tests/bug0065b.nl.tok.V,
	  tests/quotetest_folgert.nl.tok.V, tests/quotetest_folgert.nl.txt,
	  tests/quotetest_folgert2.nl.tok.V,
	  tests/quotetest_folgert2.nl.txt,
	  tests/quotetest_folgert3.nl.tok.V,
	  tests/quotetest_folgert3.nl.txt, tests/test.py: Folgert fixed
	  quote detection, extra tests added

2012-05-22 13:34  sloot

	* [r14777] src/tokenize.cxx: handle ' just like " in
	  quote-detection

2012-05-22 13:34  sloot

	* [r14776] config/standard-quotes.quote: updated comment

2012-05-22 13:33  sloot

	* [r14775] tests/testquotes.ok: let's call this OK, for now.

2012-05-22 13:32  sloot

	* [r14774] tests/qtest.nl, tests/qtest.spec, tests/testall,
	  tests/testquotes: added tests for quotes

2012-05-07 14:52  sloot

	* [r14721] config/tokconfig-nl: attempt to fix 'time' problem

2012-05-07 14:47  sloot

	* [r14720] config/smiley.rule, tests/smileys.nl.tok.V: fixed smiley

2012-05-07 14:40  sloot

	* [r14719] tests/datetime.nl.tok.V, tests/datetime.nl.txt,
	  tests/smileys.nl.tok.V, tests/smileys.nl.txt, tests/testall,
	  tests/testconf2.ok: added tests (still fail!)

2012-04-26 15:57  sloot

	* [r14703] tests/slashes.xml, tests/testfolia.ok,
	  tests/testfoliain.ok, tests/testslash.ok: adapted to new folia
	  version

2012-03-29 09:04  sloot

	* [r14587] configure.ac: bump version after release

2012-03-29 08:48  sloot

	* [r14585] NEWS, configure.ac: get ready for release

2012-03-19 10:54  sloot

	* [r14472] src/ucto.cxx: numb change

2012-03-19 10:54  sloot

	* [r14471] tests/testfolia.ok, tests/testfoliain.ok,
	  tests/testslash.ok: newer libfolia version

2012-03-09 10:17  mvgompel

	* [r14432] config/Makefile.am, config/tokconfig-fy: added frisian
	  tokenisation (untested yet)

2012-02-29 11:37  sloot

	* [r14356] configure.ac: Bumped version after release

2012-02-27 15:55  sloot

	* [r14342] NEWS, configure.ac: News!

2012-02-27 10:38  sloot

	* [r14336] src/tokenize.cxx: bit of cleanup

2012-02-23 13:23  sloot

	* [r14322] src/tokenize.cxx: formatting

2012-02-21 17:24  sloot

	* [r14304] tests/Makefile.am, tests/bug0083.nl.tok.V,
	  tests/bug0083.nl.txt, tests/folia1.xml, tests/folia2.xml,
	  tests/folia3.xml, tests/folia4.xml,
	  tests/quotetest_nested2.nl.tok.V, tests/quotetest_nested2.nl.txt,
	  tests/slashes.xml, tests/smileys.nl.tok.V, tests/smileys.nl.txt,
	  tests/test.de.txt, tests/test.en.txt, tests/test.es.txt,
	  tests/test.fr.txt, tests/test.pt.txt, tests/testall,
	  tests/testfoliain, tests/testfoliain.ok, tests/testslash,
	  tests/testslash.ok, tests/tst.cfg, tests/tst.ok, tests/tst.txt:
	  added a test for wierd \\\ problems

2012-02-21 16:05  sloot

	* [r14302] config/tokconfig-nl, src/tokenize.cxx: replaced all ?i
	  stuff in regular expressions.
	  in an attempt to avoid ICU bug

2012-02-21 12:24  mvgompel

	* [r14292] config/Makefile.am, config/tokconfig-nl-sonarchat: added
	  for sonar chat tokenisation

2012-02-21 10:48  sloot

	* [r14287] tests/testfoliain.ok: maarten mangled the test

2012-02-21 10:33  mvgompel

	* [r14286] tests/folia1.xml, tests/folia2.xml, tests/folia3.xml,
	  tests/folia4.xml, tests/testfoliain, tests/testfoliain.ok: test
	  update

2012-02-21 10:18  mvgompel

	* [r14285] docs/ucto_manual.tex, include/ucto/tokenize.h,
	  src/tokenize.cxx: folia input fixes (regarding event tag)

2012-02-21 10:14  sloot

	* [r14284] tests/testfolia.ok, tests/testfoliain.ok: adapted to
	  folia 0.8

2012-02-08 20:48  mvgompel

	* [r14223] docs/ucto_manual.tex: typo

2012-02-08 20:33  mvgompel

	* [r14222] docs/ucto_manual.tex: added beginnings of ucto manual

2012-01-31 14:49  sloot

	* [r14159] NEWS, src/ucto.cxx, tests/testoption1.ok,
	  tests/testusage.ok: added a -X and an --id= option. this is more
	  in line with frog.
	  -x <id> still works

2012-01-30 16:36  sloot

	* [r14145] config/tokconfig-nl, src/tokenize.cxx: added 'escape'
	  possibility to config files
	  now we can read:
	  [ATTACHEDSUFFIXES]
	  \['`’]s
	  \['`’]t
	  \['`’]n
	  
	  which means:
	  [ATTACHEDSUFFIXES]
	  ['`’]s
	  ['`’]t
	  ['`’]n

2012-01-30 15:47  sloot

	* [r14144] tests/folia1.xml, tests/folia2.xml, tests/folia3.xml,
	  tests/testfolia.ok, tests/testfoliain.ok: fixed tests. libfolia
	  is strict on <div> nodes (because FoliA.pdf demands it)

2012-01-30 11:54  mvgompel

	* [r14134] config/tokconfig-fr, config/tokconfig-nl,
	  config/tokconfig-nl-twitter: support for unicode apostrope 0x2019

2012-01-25 15:12  sloot

	* [r14109] include/ucto/tokenize.h, src/tokenize.cxx: Make
	  debugging output go to theErrlog generally

2012-01-12 16:49  sloot

	* [r13991] ucto.spec: was moved to Timbl-dev/rpm

2012-01-10 17:22  sloot

	* [r13944] configure.ac: cleanup configure mess

2012-01-10 15:27  sloot

	* [r13934] configure.ac: Bumped version after Release

2012-01-09 16:13  sloot

	* [r13918] NEWS, config/Makefile.am, config/smiley.rule,
	  config/tokconfig-nl, config/tokconfig-nl-twitter: put SMILEY in a
	  separate .rule file.
	  It is language independent.

2012-01-09 15:55  sloot

	* [r13917] config/tokconfig-nl: added a much more powerfull SMILEY
	  rule.
	  Which also works with older ICU cversions!

2012-01-09 15:54  sloot

	* [r13916] src/tokenize.cxx: Even better error message

2012-01-09 15:53  sloot

	* [r13915] tests/testconf2.ok, tests/testfolia.ok,
	  tests/testfoliain.ok: adapted to small changes in output

2012-01-09 14:30  sloot

	* [r13914] src/tokenize.cxx: slighly better error message

2012-01-02 16:32  sloot

	* [r13842] include/ucto/tokenize.h, include/ucto/unicode.h,
	  src/tokenize.cxx, src/ucto.cxx, src/unicode.cxx: next year

2011-12-21 11:28  sloot

	* [r13762] configure.ac: Bumped version after release

2011-12-21 11:04  sloot

	* [r13756] NEWS, configure.ac: bumped version
	  somehow 0.4.8 was already released

2011-12-14 11:10  sloot

	* [r13698] src/ucto.cxx: be more informative

2011-12-06 16:44  sloot

	* [r13652] src/tokenize.cxx: libfolia API is a moving target

2011-12-06 15:06  sloot

	* [r13648] configure.ac, include/ucto/tokenize.h, src/tokenize.cxx:
	  adapted to major change in libfolia: AbstractElement ==>
	  FoliaElement

2011-12-06 15:05  sloot

	* [r13647] tests/testfolia.ok, tests/testfoliain.ok: adapted to new
	  folia

2011-11-28 15:31  sloot

	* [r13613] tests/testfoliain.ok: test results changed

2011-11-28 15:26  sloot

	* [r13612] tests/folia1.xml: stray "

2011-11-28 15:17  sloot

	* [r13611] tests/folia1.xml, tests/folia2.xml, tests/folia3.xml:
	  replaced \n wilt real newline character &#xA;

2011-11-28 15:06  sloot

	* [r13610] src/tokenize.cxx, src/ucto.cxx: attempt to fix -F. still
	  uncertain

2011-11-28 15:05  sloot

	* [r13609] tests/folia.xml, tests/folia1.xml, tests/folia2.xml,
	  tests/folia3.xml, tests/testfoliain, tests/testfoliain.ok: added
	  and changed -F tests
	  Still not sure if this is wahat we want

2011-11-28 14:46  mvgompel

	* [r13608] config/Makefile.am, config/tokconfig-nl-twitter: Added
	  specicial tokeniser configuration for Twitter (dutch)

2011-11-28 14:39  mvgompel

	* [r13607] config/tokconfig-nl, config/url.rule,
	  tests/smileys.nl.tok.V, tests/smileys.nl.txt,
	  tests/urls.nl.tok.V, tests/urls.nl.txt: Added URL-DOMAIN and
	  SMILEY detection (enabled only in tokconfig-nl for now)

2011-11-28 13:27  sloot

	* [r13605] tests/folia.xml, tests/testall, tests/testfoliain,
	  tests/testfoliain.ok: added simple test for the _F FoLiA XML
	  input option

2011-11-28 09:53  sloot

	* [r13601] docs/ucto.1, src/ucto.cxx: small change in usag()
	  Enhanced man page

2011-11-28 09:52  sloot

	* [r13600] tests/testoption1.ok, tests/testusage.ok: again output
	  was changed

2011-11-28 09:48  mvgompel

	* [r13599] src/tokenize.cxx: fix

2011-11-28 09:47  mvgompel

	* [r13598] src/tokenize.cxx: fixes

2011-11-28 09:37  sloot

	* [r13597] tests/testoption1.ok, tests/testusage.ok: output has
	  changed, so adapt test

2011-11-28 09:22  sloot

	* [r13596] src/tokenize.cxx: removed 2 compiler warnings
	  2 others still there: Proycon, what were you thinking?

2011-11-26 19:39  mvgompel

	* [r13587] include/ucto/tokenize.h, src/tokenize.cxx: completed
	  FoLiA input support (needs more testing though)

2011-11-26 10:42  mvgompel

	* [r13586] src/tokenize.cxx: tokenisation of sentences in existing
	  FoLiA now works

2011-11-25 22:17  mvgompel

	* [r13585] src/tokenize.cxx: fixes for FoLiA input support

2011-11-25 17:24  mvgompel

	* [r13583] include/ucto/tokenize.h, src/tokenize.cxx, src/ucto.cxx:
	  Attempting to add FoLiA read support to ucto.. not entirely
	  finished and tested yet!

2011-11-09 14:06  sloot

	* [r13536] src/tokenize.cxx: added some hacks to avoid a nasty ICU
	  bug.
	  see http://bugs.icu-project.org/trac/ticket/8824

2011-11-02 16:31  sloot

	* [r13511] configure.ac: fix

2011-11-02 16:22  sloot

	* [r13510] src/Makefile.am, src/tst.sh: more lost stuff

2011-11-02 16:16  sloot

	* [r13509] Makefile.am, tests/Makefile.am, tests/bug0083.nl.tok.V,
	  tests/bug0083.nl.txt, tests/test.de.txt, tests/test.en.txt,
	  tests/test.es.txt, tests/test.fr.txt, tests/test.pt.txt,
	  tests/tst.cfg, tests/tst.ok, tests/tst.txt: hmm lost a lot of
	  stuff in the merge :{

2011-11-02 15:44  sloot

	* [r13508] ., AUTHORS, Makefile.am, NEWS, config/Makefile.am,
	  config/tokconfig-en, config/tokconfig-es, config/tokconfig-fr,
	  config/tokconfig-generic, configure.ac, docs/ucto.1,
	  include/ucto/tokenize.h, include/ucto/unicode.h, src/Makefile.am,
	  src/tokenize.cxx, src/tst.sh, src/ucto.cxx, src/unicode.cxx,
	  tests, tests/Makefile.am, tests/abbreviations.nl.tok.V,
	  tests/abbreviations.nl.txt, tests/bug0048.nl.tok.V,
	  tests/bug0048.nl.txt, tests/bug0051.nl.tok.V,
	  tests/bug0051.nl.txt, tests/bug0052.nl.tok.V,
	  tests/bug0052.nl.txt, tests/bug0054.nl.tok.V,
	  tests/bug0054.nl.txt, tests/bug0063.nl.tok.V,
	  tests/bug0063.nl.txt, tests/bug0065.nl.tok.V,
	  tests/bug0065.nl.txt, tests/bug0065b.nl.tok.V,
	  tests/bug0065b.nl.txt, tests/bug0065c.nl.tok.V,
	  tests/bug0065c.nl.txt, tests/bug0067.nl.tok.V,
	  tests/bug0067.nl.txt, tests/bug0075.nl.tok.V,
	  tests/bug0075.nl.txt, tests/bug0083.nl.tok.V,
	  tests/bug0083.nl.txt, tests/datetime.nl.tok.V,
	  tests/datetime.nl.txt, tests/encoding.nl.tok.V,
	  tests/encoding.nl.txt, tests/encoding2.nl, tests/folia.txt,
	  tests/genitive.nl.tok.V, tests/genitive.nl.txt,
	  tests/invoer.rule, tests/ligaturen.nl.tok.V,
	  tests/ligaturen.nl.txt, tests/multisentenceoneline.nl.tok.V,
	  tests/multisentenceoneline.nl.txt, tests/munten.nl.tok.V,
	  tests/munten.nl.txt, tests/normalisation.nl.tok.V,
	  tests/normalisation.nl.txt, tests/nu.nl.tok.V, tests/nu.nl.txt,
	  tests/onesentencemultiline.nl.tok.V,
	  tests/onesentencemultiline.nl.txt, tests/parentheses.nl.tok.V,
	  tests/parentheses.nl.txt, tests/partest_crlf.nl.tok.V,
	  tests/partest_crlf.nl.txt, tests/passthru.txt,
	  tests/passthru2.txt, tests/qtest.nl, tests/qtest.nl.tok,
	  tests/qtest.nl.tok.V, tests/qtest.spec, tests/qtest.spec.tok,
	  tests/qtest.spec.tok.V, tests/quoteproblem.txt,
	  tests/quotetest_multiline.nl.tok.V,
	  tests/quotetest_multiline.nl.txt,
	  tests/quotetest_multisentence.nl.tok.V,
	  tests/quotetest_multisentence.nl.txt,
	  tests/quotetest_nested.nl.tok.V, tests/quotetest_nested.nl.txt,
	  tests/quotetest_nested2.nl.tok.V, tests/quotetest_nested2.nl.txt,
	  tests/quotetest_nosentence.nl.tok.V,
	  tests/quotetest_nosentence.nl.txt,
	  tests/quotetest_onesentence.nl.tok.V,
	  tests/quotetest_onesentence.nl.txt, tests/small.cfg,
	  tests/small.txt, tests/stuk1.cfg, tests/stuk2.cfg,
	  tests/test.de.txt, tests/test.en.txt, tests/test.es.txt,
	  tests/test.fr.txt, tests/test.pt.txt, tests/test.py,
	  tests/testencoding2, tests/testencoding2.ok, tests/testfolia,
	  tests/testfolia.ok, tests/testinclude, tests/testinclude.ok,
	  tests/testnormalisation, tests/testnormalisation.ok,
	  tests/testone, tests/testoption1.ok, tests/testoption2,
	  tests/testoption2.ok, tests/testpassthru, tests/testpassthru.ok,
	  tests/testusage.ok, tests/tst.cfg, tests/tst.ok, tests/tst.txt,
	  tests/units.nl.tok.V, tests/units.nl.txt, tests/urls.nl.tok.V,
	  tests/urls.nl.txt, ucto.pc.in: Merged the folia branch into ucto.
	  ucto now always requires libfolia

2011-10-25 14:21  sloot

	* [r13462] src/tokenize.cxx: some layout changed and slack removed

2011-10-25 12:47  sloot

	* [r13459] NEWS, src/tokenize.cxx: small changes

2011-10-24 13:41  sloot

	* [r13449] src/tokenize.cxx: made compiler happy.
	  Removed dead code

2011-10-04 08:31  mvgompel

	* [r13366] config/tokconfig-es: support for spanish ¿ and ¡

2011-10-04 08:07  mvgompel

	* [r13365] config/tokconfig-es: update

2011-09-30 10:29  mvgompel

	* [r13349] config/Makefile.am: update

2011-09-30 10:27  mvgompel

	* [r13348] config/tokconfig-generic: generic tokeniser
	  configuration, not for any particular language (not recommended)

2011-09-29 19:38  mvgompel

	* [r13347] config/tokconfig-en: English tokenisation: wo n't , do
	  n't

2011-09-29 19:25  mvgompel

	* [r13346] config/tokconfig-es: Added spanish configuration

2011-09-29 17:31  mvgompel

	* [r13345] config/tokconfig-fr: Added french abbreviations

2011-09-29 17:19  mvgompel

	* [r13344] config/tokconfig-fr: updates to french configuration

2011-09-29 16:12  mvgompel

	* [r13342] config/tokconfig-en, tests/test.en.txt: update English
	  configuration

2011-09-29 15:45  mvgompel

	* [r13341] tests/test.de.txt: correction

2011-09-29 15:45  mvgompel

	* [r13340] tests/test.de.txt: correction

2011-09-29 15:43  mvgompel

	* [r13339] tests/test.de.txt: generic german test

2011-09-29 15:12  mvgompel

	* [r13336] tests/test.pt.txt: correction

2011-09-29 15:11  mvgompel

	* [r13335] tests/test.pt.txt: corrections

2011-09-29 15:07  mvgompel

	* [r13334] tests/test.pt.txt: Generic portuguese test

2011-09-29 14:53  mvgompel

	* [r13333] tests/test.es.txt: Generic Spanish test

2011-09-29 14:36  mvgompel

	* [r13332] tests/test.fr.txt: more french

2011-09-29 14:29  mvgompel

	* [r13331] tests/test.fr.txt: Initial generic french test.

2011-09-29 14:06  mvgompel

	* [r13330] tests/test.en.txt: generic English test

2011-09-15 13:51  sloot

	* [r13189] tests/Makefile.am: last small oesp

2011-09-15 13:46  sloot

	* [r13188] Makefile.am, configure.ac, src/Makefile.am, src/tst.sh,
	  tests, tests/Makefile.am, tests/tst.cfg, tests/tst.ok,
	  tests/tst.txt: attempt to provide 'make check' and 'make
	  distcheck'

2011-08-31 08:23  sloot

	* [r13043] configure.ac: Bumped version after Release

2011-08-23 14:17  sloot

	* [r12956] tests/bug0083.nl.tok.V: attempt for more precise desired
	  version\

2011-08-23 14:09  sloot

	* [r12955] tests/bug0083.nl.tok.V, tests/bug0083.nl.txt: added an
	  other quoteing problem and the disered output

2011-08-23 12:33  sloot

	* [r12950] configure.ac: bumped version after release

2011-08-23 12:32  sloot

	* [r12949] NEWS: release!

2011-08-23 10:07  sloot

	* [r12942] src/tokenize.cxx: the API should have the same defaults
	  ofcourse

2011-08-23 09:37  sloot

	* [r12940] tests/test.py, tests/testfolia, tests/testoption1.ok,
	  tests/testusage.ok: adapted test to changed QuoteDetection
	  regime.
	  hardcoded -Q in test.py, arghh.

2011-08-23 08:52  sloot

	* [r12938] NEWS, docs/ucto.1, src/tokenize.cxx, src/ucto.cxx:
	  Reversed working of -Q
	  Updated man
	  more NEWS

2011-08-22 14:01  sloot

	* [r12931] include/ucto/tokenize.h, src/tokenize.cxx, src/ucto.cxx:
	  applied same changes to readsettings an init as in the libfolia
	  branch

2011-08-22 14:00  sloot

	* [r12930] tests/testconf1.ok, tests/testconf2.ok,
	  tests/testfiles1.ok, tests/testinclude.ok, tests/testlanguage.ok:
	  outpu is changed a little bit

2011-08-18 16:01  mvgompel

	* [r12909] NEWS, src/tokenize.cxx, tests/quoteproblem.txt,
	  tests/quotetest_nested2.nl.tok.V, tests/quotetest_nested2.nl.txt:
	  Fixed nested quote issue (unbalanced quote/sentence issue) and
	  build in extra protections. Incl. test

2011-07-28 10:35  mvgompel

	* [r12749] tests/abbreviations.nl.tok.V, tests/bug0048.nl.tok.V,
	  tests/bug0051.nl.tok.V, tests/bug0052.nl.tok.V,
	  tests/bug0054.nl.tok.V, tests/bug0063.nl.tok.V,
	  tests/bug0065.nl.tok.V, tests/bug0065b.nl.tok.V,
	  tests/bug0065c.nl.tok.V, tests/bug0067.nl.tok.V,
	  tests/bug0075.nl.tok.V, tests/datetime.nl.tok.V,
	  tests/encoding.nl.tok.V, tests/genitive.nl.tok.V,
	  tests/ligaturen.nl.tok.V, tests/multisentenceoneline.nl.tok.V,
	  tests/normalisation.nl.tok.V, tests/nu.nl.tok.V,
	  tests/onesentencemultiline.nl.tok.V, tests/parentheses.nl.tok.V,
	  tests/quotetest_multiline.nl.tok.V,
	  tests/quotetest_multisentence.nl.tok.V,
	  tests/quotetest_nested.nl.tok.V,
	  tests/quotetest_nosentence.nl.tok.V,
	  tests/quotetest_onesentence.nl.tok.V, tests/test.nl.tok.V,
	  tests/urls.nl.tok.V: fixed ucto tests \o/

2011-06-07 14:59  mvgompel

	* [r10447] tests/quoteproblem.txt: added

2011-05-24 13:42  sloot

	* [r10238] NEWS: updated NEWS

2011-05-24 13:33  sloot

	* [r10237] tests/testall, tests/testfolia.ok, tests/testone,
	  tests/testoption1.ok, tests/testpassthru, tests/testpassthru.ok,
	  tests/testusage.ok: added and improved tests for passthru
	  updated a few .ok files.
	  
	  I hope proycon fixes all .V.tok files

2011-05-24 13:17  sloot

	* [r10236] src/tokenize.cxx, src/ucto.cxx: fixed encoding problem
	  with passthru mode.
	  Added --passthru option to usage()

2011-05-24 09:24  sloot

	* [r10223] src/tokenize.cxx: attempt to handle Windows™ files with
	  \r\n.
	  Just strip the last \r

2011-05-24 08:38  sloot

	* [r10220] tests/partest_crlf.nl.txt: converted the file to CRLF
	  and set the svn:eol-style property to CRLF

2011-05-24 07:47  mvgompel

	* [r10214] tests/partest_crlf.nl.tok.V, tests/partest_crlf.nl.txt:
	  added test

2011-05-23 21:40  mvgompel

	* [r10209] src/tokenize.cxx: I think I fixed bug77 (the NOSPACE
	  bug), but now all tests fail (understandably), they will have to
	  be checked thoroughly checked and updated.

2011-05-23 07:49  sloot

	* [r10156] tests/folia.txt, tests/testall, tests/testfolia,
	  tests/testfolia.ok: added a simple FoLiA test

2011-05-20 07:42  mvgompel

	* [r10121] NEWS, src/tokenize.cxx: Fix: proper XML entities in
	  FoLiA output

2011-05-17 07:59  sloot

	* [r10015] configure.ac: bumped version after release

2011-05-17 07:53  sloot

	* [r10013] NEWS: set date for release

2011-05-16 10:16  sloot

	* [r9980] NEWS: updated NEWS too

2011-05-16 10:15  sloot

	* [r9979] src/tokenize.cxx: avoid crash when no new tokens are
	  found because of emptyp words
	  (this might happen on Mac and DOS files with \r line separators )

2011-05-11 15:08  sloot

	* [r9871] src/tokenize.cxx: ok, this is the ugly code :{

2011-05-11 14:59  sloot

	* [r9867] tests/testpassthru.ok: sort of fixed <utt> handling in
	  passthru
	  code gets ugglier every day ;{

2011-05-11 14:55  sloot

	* [r9866] tests/passthru2.txt, tests/testpassthru,
	  tests/testpassthru.ok: added a better passthru test

2011-05-11 13:16  sloot

	* [r9863] NEWS: more News

2011-05-11 13:11  sloot

	* [r9862] src/tokenize.cxx, src/ucto.cxx: small change in
	  configfile handling.
	  Now the messages to stdout are correct.

2011-05-11 13:08  sloot

	* [r9861] tests/testconf1.ok, tests/testconf2.ok,
	  tests/testinclude.ok: results ar slighlty different after small
	  code reshuffle

2011-05-11 08:37  sloot

	* [r9855] tests/invoer.rule, tests/small.cfg, tests/small.txt,
	  tests/testall, tests/testinclude, tests/testinclude.ok: added a
	  test for inclusion

2011-05-11 08:36  sloot

	* [r9854] tests/testconf1.ok: small change after bug fix

2011-05-10 15:46  sloot

	* [r9853] src/tokenize.cxx: fixed include file problem. I think. :)
	  When a configfile with a (absolute/relative) path is provided (-c
	  option)
	  the include files will be searched in the same dir as that config
	  file

2011-05-10 10:32  sloot

	* [r9845] tests/bug0072.nl.tok.V, tests/bug0072.nl.txt,
	  tests/bug0075.nl.tok.V, tests/bug0075.nl.txt: wrong name for this
	  bug

2011-05-10 08:38  sloot

	* [r9844] src/tokenize.cxx: KNOWN Abbreviations are case sensitive
	  now. Fixes bug 72

2011-05-10 08:19  sloot

	* [r9843] tests/bug0072.nl.tok.V, tests/bug0072.nl.txt: added bug72

2011-04-27 16:50  mvgompel

	* [r9732] configure.ac: version bump after release

2011-04-27 16:38  mvgompel

	* [r9730] NEWS: updated NEWS

2011-04-27 16:38  mvgompel

	* [r9729] src/tokenize.cxx: Implemented sentenceperlineinput
	  support directly in DetectSentenceBounds and PassTru (previously
	  unsupported)

2011-04-19 15:15  sloot

	* [r9591] include/ucto/tokenize.h, src/tokenize.cxx: also %include
	  for [EOSMARKERS]

2011-04-19 15:15  sloot

	* [r9590] config/Makefile.am, config/exotic-eos.eos,
	  config/exotic-quotes.quote, config/ligatures.filter,
	  config/standard-eos.eos, config/standard-quotes.quote,
	  config/tokconfig-de, config/tokconfig-nl: added common files for
	  inclusion in config

2011-04-19 14:54  sloot

	* [r9589] config/Makefile.am, config/tokconfig-nl,
	  include/ucto/tokenize.h, src/tokenize.cxx: now [FILTER] and
	  [QUOTES] also honour %include requests in the config
	  For [FILTER] the file extension MUST be .filter
	  For [QUOTES] the file extension MUST be .quote

2011-04-19 13:55  sloot

	* [r9582] config/tokconfig-nl, src/tokenize.cxx: fixed bug 67.
	  Now it is possible to add an empty replacement in the [FILTER}
	  block

2011-04-19 13:12  sloot

	* [r9581] tests/bug0067.nl.tok.V, tests/bug0067.nl.txt: added a new
	  bug test

2011-04-07 14:38  mvgompel

	* [r9350] src/tokenize.cxx: no schemalocation for FoLiA

2011-04-07 09:35  sloot

	* [r9313] include/ucto/tokenize.h, src/tokenize.cxx: arghh

2011-04-07 09:18  sloot

	* [r9311] include/ucto/tokenize.h: hmpff.

2011-04-07 09:07  sloot

	* [r9310] NEWS, include/ucto/tokenize.h, src/ucto.cxx: added
	  functions to display version info.
	  Updated NEWS

2011-04-07 08:42  sloot

	* [r9309] src/ucto.cxx: return conforming exit status.

2011-04-04 12:08  sloot

	* [r9271] configure.ac: bumped version after release

2011-04-04 12:08  sloot

	* [r9270] NEWS: news!

2011-04-04 09:47  mvgompel

	* [r9268] src/tokenize.cxx, tests/bug0065.nl.tok.V,
	  tests/bug0065b.nl.tok.V, tests/bug0065b.nl.txt,
	  tests/bug0065c.nl.tok.V, tests/bug0065c.nl.txt: fixed bug0065

2011-04-04 08:09  mvgompel

	* [r9267] tests/bug0065.nl.txt: new bug

2011-03-31 15:57  sloot

	* [r9261] NEWS, src/tokenize.cxx, src/unicode.cxx: Ok, now ALL
	  unicode strings are output as UTF8.
	  Just to be sure.

2011-03-31 14:56  sloot

	* [r9258] src/tokenize.cxx: Attempt to fix output encoding
	  problems. We want it to be UTF8.

2011-03-31 12:33  sloot

	* [r9240] moved ucto to its final destination

2011-03-19 17:13  joostvb

	* [r8975] new upstream checksum

2011-03-19 17:11  joostvb

	* [r8974] new upstream, new homepage

2011-03-19 10:49  joostvb

	* [r8970] start 0.4.4 development

2011-03-19 10:44  joostvb

	* [r8969] release 0.4.3

2011-03-19 10:44  joostvb

	* [r8968] record changes

2011-03-17 15:48  sloot

	* [r8936] fixed --passthru problem

2011-03-17 15:48  sloot

	* [r8935] test now works

2011-03-17 09:31  sloot

	* [r8896] bumped version after release

2011-03-16 13:41  sloot

	* [r8877] passes -pedantic

2011-03-16 12:59  sloot

	* [r8871] some cleanup
	  bumped version

2011-03-14 11:12  sloot

	* [r8843] added (hidden) --passthru option

2011-03-14 11:12  sloot

	* [r8842] added a passthru test.
	  has t0 be tested :)

2011-03-14 10:07  sloot

	* [r8839] initialize your variables, please.

2011-03-14 10:01  sloot

	* [r8838] make compiler more happy

2011-03-14 09:16  sloot

	* [r8837] added description

2011-03-14 09:02  sloot

	* [r8836] smal update

2011-03-12 15:48  joostvb

	* [r8811] new upstream

2011-03-11 16:53  mvgompel

	* [r8796] fixed passthruline (skip=t) bug

2011-03-08 16:01  sloot

	* [r8751] fixed regexp and error messag

2011-03-08 15:54  sloot

	* [r8750] added the possiblity to ste the order of RULES in the
	  config file

2011-03-08 15:52  sloot

	* [r8749] strange a space has left the building

2011-03-08 13:47  mvgompel

	* [r8748] fix

2011-03-08 13:47  mvgompel

	* [r8747] override

2011-03-08 13:42  mvgompel

	* [r8746] fix bug0063

2011-03-08 12:59  sloot

	* [r8745] added a test for bug63
	  Not sure about the 'correct' solution

2011-03-08 10:02  sloot

	* [r8742] updated man page

2011-03-07 16:05  mvgompel

	* [r8726] FoLiA XSL has to be local unfortunately

2011-03-07 14:42  mvgompel

	* [r8721] fix

2011-03-07 14:38  mvgompel

	* [r8720] fix

2011-03-04 13:26  mvgompel

	* [r8687] added logo

2011-03-04 11:44  mvgompel

	* [r8683] fix

2011-03-03 15:23  mvgompel

	* [r8678] oops

2011-03-03 15:22  mvgompel

	* [r8677] no body in FoLiA anymore

2011-03-01 13:59  sloot

	* [r8637] bumped version. seem a good moment for 0.4

2011-03-01 13:43  sloot

	* [r8636] these tests give a different outcome now.

2011-02-28 15:36  mvgompel

	* [r8620] oops: the sequel

2011-02-28 15:35  mvgompel

	* [r8619] oops

2011-02-28 15:32  mvgompel

	* [r8618] sentence per line input and output: two modes

2011-02-28 15:11  mvgompel

	* [r8617] Fixed bug 54

2011-02-28 13:46  mvgompel

	* [r8615] fixes

2011-02-28 13:46  mvgompel

	* [r8614] FoLiA improvement

2011-02-27 15:34  joostvb

	* [r8593]

2011-02-27 15:06  joostvb

	* [r8592] first shot at ucto rpm

2011-02-27 10:35  joostvb

	* [r8576] split, in order to be able to build frog

2011-02-27 08:45  joostvb

	* [r8565] shlibs stuff. ship it

2011-02-26 15:31  joostvb

	* [r8563] first shot at building ucto for fink/macosx

2011-02-14 13:35  sloot

	* [r8318] added experimental code to use the -n option ( output one
	  sentence per line) also to process the input one sentence per
	  line

2011-02-14 13:18  sloot

	* [r8317] testcase for bug0054

2011-02-12 18:09  joostvb

	* [r8299] 0.3.6 is released. start 0.3.7

2011-02-12 18:04  joostvb

	* [r8298] ship it

2011-02-12 18:00  joostvb

	* [r8297] record more changes

2011-02-12 17:42  joostvb

	* [r8296] record some changes

2011-02-10 16:59  sloot

	* [r8291] missing file added

2011-02-10 16:33  sloot

	* [r8289] added a test for the -e option

2011-02-10 16:18  sloot

	* [r8288] ok, accept any encoding string, now.
	  Throw up when not known.

2011-02-10 16:03  sloot

	* [r8286] adapted tests to the reality

2011-02-10 15:43  sloot

	* [r8283] added possibility to set the inputEncoding
	  breaks ucto user interface!

2011-02-10 11:51  mvgompel

	* [r8280] version bump

2011-02-10 11:51  mvgompel

	* [r8279] Fixed quote problems bug0052, bug0051

2011-02-09 12:27  mvgompel

	* [r8274] debug fix

2011-02-09 12:04  mvgompel

	* [r8273] fixed tests

2011-02-09 11:55  mvgompel

	* [r8272] added more debug

2011-02-09 11:43  mvgompel

	* [r8271] updated error messages

2011-02-09 11:08  sloot

	* [r8270] better error message

2011-02-08 17:14  sloot

	* [r8264] more useful naming

2011-02-08 17:07  sloot

	* [r8263] ok, this is what is meant.

2011-02-08 16:43  sloot

	* [r8259] make the test fail again. this is dead wrong

2011-02-08 16:27  sloot

	* [r8258] added a demo for bug52

2011-02-08 10:27  sloot

	* [r8251] fixed bug51. Not realy happy

2011-02-08 10:24  sloot

	* [r8250] this is what we want

2011-02-08 09:39  sloot

	* [r8249] added a demo of bug51

2011-02-07 16:17  sloot

	* [r8243] New test. OK file is NOT OK :)

2011-02-07 16:09  sloot

	* [r8242] cleanup

2011-02-07 14:13  sloot

	* [r8237] bit of cleanup

2011-02-03 15:31  sloot

	* [r8216] fix memory leak

2011-02-03 15:24  sloot

	* [r8215] - testall now tests all.
	  - hacked around in python without real knowledge to support
	  Valgrind testing

2011-02-03 13:00  sloot

	* [r8212] removed strange stray '\'

2011-02-03 12:55  sloot

	* [r8211] last "elegant" solution was dead wrong (but passed all
	  tests)
	  this one is much better (and passes all tests)
	  Maybe the tests are wrong?

2011-02-03 11:00  sloot

	* [r8206] more elegant ;)

2011-02-03 10:57  sloot

	* [r8205] .

2011-02-03 10:00  sloot

	* [r8204] added a test for invalid combination of options

2011-02-03 09:59  sloot

	* [r8203] make sure -L and -c don't intermix

2011-02-02 15:42  sloot

	* [r8196] props set

2011-02-02 14:14  sloot

	* [r8186] ID ==> Id

2011-02-02 14:12  sloot

	* [r8184] tag

2011-02-02 14:11  sloot

	* [r8183] tags

2011-02-02 14:08  sloot

	* [r8182] more copyright

2011-02-02 14:05  sloot

	* [r8181] added copyright notice

2011-02-02 11:53  sloot

	* [r8171] we want these fuction in the std namespace, i think

2011-02-02 11:35  sloot

	* [r8170] more clean implementation. hide normalization in a Class

2011-02-02 11:34  sloot

	* [r8169] changed spelling

2011-02-02 10:17  sloot

	* [r8168] put some usefull unicode stuff in separate module

2011-02-01 15:41  sloot

	* [r8155] this one was missing

2011-02-01 15:34  mvgompel

	* [r8154] nog een poging

2011-02-01 15:30  mvgompel

	* [r8153] fix

2011-02-01 15:24  mvgompel

	* [r8152] back geport

2011-02-01 15:12  sloot

	* [r8151] oesp

2011-02-01 14:26  sloot

	* [r8148] renamed. use same number as in Bugzilla:
	  https://ilk.uvt.nl/bugzilla3//show_bug.cgi?id=48

2011-02-01 14:15  sloot

	* [r8146] Ok, it is a bit harsh, but strip all path info, to make
	  the tests worl "everywhere"

2011-02-01 11:42  sloot

	* [r8136] testresults have changed

2011-02-01 11:11  sloot

	* [r8135] better naming

2011-02-01 11:09  sloot

	* [r8134] added a test for several Normalisation types

2011-02-01 10:45  sloot

	* [r8133] now actualy use -e values for normalisation

2011-02-01 10:25  sloot

	* [r8132] -f option now works

2011-02-01 10:25  sloot

	* [r8131] better test

2011-01-31 16:52  sloot

	* [r8126] cleaned up non-dutch eosmarker
	  attempt to add 0x0a (\n) as eosmarker failed

2011-01-31 15:43  sloot

	* [r8123] new test added (.V is empty, to gat a nice diff)

2011-01-31 15:12  sloot

	* [r8122] test more

2011-01-31 15:04  sloot

	* [r8121] this is the desired result (NFC)

2011-01-31 15:01  sloot

	* [r8120] more to-the-point naming

2011-01-31 14:47  sloot

	* [r8119] added support for ligature filtering and Unicode
	  normalizing.
	  a bit rough still

2011-01-31 13:40  sloot

	* [r8117] now uses ucto-icu.pc

2011-01-31 10:46  sloot

	* [r8091] to please Maarten even more

2011-01-31 10:45  sloot

	* [r8090] zo beter dan Maarten?

2011-01-31 10:32  sloot

	* [r8086] our first officila bug!

2011-01-27 16:28  joostvb

	* [r8060] cosmetics

2011-01-27 16:18  sloot

	* [r8059] props set

2011-01-27 16:14  joostvb

	* [r8058] release 0.3.4, record changes

2011-01-27 16:13  sloot

	* [r8057] props

2011-01-27 16:08  sloot

	* [r8056] propset

2011-01-27 16:08  sloot

	* [r8055] propset

2011-01-27 16:06  sloot

	* [r8054] more tests
	  forgot some files

2011-01-27 16:06  joostvb

	* [r8053] rename icu.pc to ucto-icu.pc: be sure we wont suffer from
	  filename clashes in the future

2011-01-27 16:01  sloot

	* [r8052] more tests added

2011-01-27 16:00  mvgompel

	* [r8051] Updates in FoLiA support

2011-01-27 15:50  sloot

	* [r8050] better?

2011-01-27 15:46  sloot

	* [r8049] more tests

2011-01-27 15:38  sloot

	* [r8047] added some scripts to test illegal parameters etc.

2011-01-27 13:51  sloot

	* [r8043] een interessante test. De .V is NU wel ok, maar ik vind
	  dat ie fout moet worden ;)

2011-01-27 13:44  sloot

	* [r8042] added a test with combining vs precombined characters

2011-01-27 12:23  joostvb

	* [r8040] release 0.3.3

2011-01-27 12:18  joostvb

	* [r8039] record changes and releases

2011-01-27 11:57  joostvb

	* [r8038] tags

2011-01-27 11:57  joostvb

	* [r8037] honor --sysconfdir in ./configure call

2011-01-27 11:56  joostvb

	* [r8036] tags

2011-01-27 11:45  sloot

	* [r8034] added missing output operator declaration

2011-01-27 11:13  sloot

	* [r8030] bumped version after "release" of 0.3.1

2011-01-26 17:17  sloot

	* [r8019] install makefile too

2011-01-26 17:08  sloot

	* [r8018] fixed bug, bumped version, added Makefile and such

2011-01-18 15:34  sloot

	* [r7866] some formatting, comment and such

2011-01-18 15:09  sloot

	* [r7865] leaner ande meaner

2011-01-18 15:04  sloot

	* [r7864] refactoring detectSentenceBounds()
	  tricky!

2011-01-18 14:35  sloot

	* [r7863] much simpler. No limit to length of "opening quote"
	  string.

2011-01-18 12:46  sloot

	* [r7854] added << operator for TokenRole (and use is)

2011-01-18 12:37  sloot

	* [r7853] get rid of unused function

2011-01-18 12:34  sloot

	* [r7852] made clear() private.

2011-01-18 12:21  sloot

	* [r7851] make stream parameters a bit more C++ like

2011-01-18 10:57  sloot

	* [r7849] cleaned up TokenRole stuff

2011-01-18 10:32  sloot

	* [r7848] refactoring...

2011-01-18 10:05  sloot

	* [r7847] some refactoring going on

2011-01-13 16:46  sloot

	* [r7831] comment added

2011-01-13 16:45  sloot

	* [r7830] all configs include the same URL and E-MAIL rules now

2011-01-13 16:34  sloot

	* [r7829] minor version bump

2011-01-13 16:33  sloot

	* [r7828] ignore sysconfdir when filename contains a '/' ( so for
	  ../rel/ati/ve and /abs/olute/path )

2011-01-13 16:27  sloot

	* [r7827] barf louder when no config found

2011-01-13 16:22  sloot

	* [r7826] nu wel

2011-01-13 16:20  sloot

	* [r7825] uninclude

2011-01-13 15:53  sloot

	* [r7824] e-mail rule created

2011-01-13 15:48  sloot

	* [r7823] added simpel include mechanism. Not very robust.
	  Dont forget to add .rule files to config/Makefile.am !

2011-01-13 14:57  mvgompel

	* [r7822] fix

2011-01-13 14:41  mvgompel

	* [r7821] fixes and test update

2011-01-13 14:32  mvgompel

	* [r7820] added detectQuoteBounds

2011-01-13 14:24  sloot

	* [r7819] adde a few more sensible default

2011-01-13 14:23  sloot

	* [r7818] added a file to store useful tips for the upcoming manual

2011-01-13 14:10  mvgompel

	* [r7817] added test text from NU.nl

2011-01-13 14:07  mvgompel

	* [r7816] renamed

2011-01-13 14:03  mvgompel

	* [r7815] Added lots of tests

2011-01-13 13:52  sloot

	* [r7814] overthrown

2011-01-13 13:41  sloot

	* [r7813] less exit, more throw

2011-01-13 13:22  sloot

	* [r7811] added a setter voor the Error logfile
	  otherwise logging/debugging is impossible when ucto is included
	  in a daemon
	  (which as we all now, closes stderr)
	  This was the sole reason for having theErrLog anyway

2011-01-13 12:04  mvgompel

	* [r7810] fix

2011-01-13 12:02  mvgompel

	* [r7809] test wrapper done

2011-01-13 12:02  mvgompel

	* [r7808] added quote tests, updated test script

2011-01-13 12:02  mvgompel

	* [r7807] fix

2011-01-13 12:01  sloot

	* [r7806] use our own exceptions

2011-01-13 11:44  mvgompel

	* [r7805] fixes

2011-01-13 11:44  mvgompel

	* [r7804] added test wrapper

2011-01-13 11:38  sloot

	* [r7803] beterst

2011-01-13 11:29  sloot

	* [r7802] beterder

2011-01-13 11:26  sloot

	* [r7801] throw more sensible exceptions

2011-01-12 15:52  sloot

	* [r7798] propset

2011-01-12 15:43  sloot

	* [r7797] quotation separator is now whitespace (spaces and/or
	  tabs)

2011-01-12 14:49  sloot

	* [r7796] added changed testresults

2011-01-12 14:48  sloot

	* [r7795] better handle the 'multiple opening, single closing'
	  quotation problem
	  added test

2011-01-12 14:24  sloot

	* [r7794] the test was changed. results are not OK, I think.
	  (nested quotes)

2011-01-12 14:22  sloot

	* [r7793] first attempt to read QUOTES from config file
	  syntax undecided yet
	  what about reasonable defaults?

2011-01-12 13:37  sloot

	* [r7792] started enhancin quote handling

2011-01-12 13:29  mvgompel

	* [r7791] small adaptation to FoLiA format

2011-01-11 17:32  sloot

	* [r7788] test.nl now seems to work
	  added something naughty to qtest.spec

2011-01-11 17:18  mvgompel

	* [r7787] trailing space fix

2011-01-11 16:45  sloot

	* [r7786] first step in more clean quote handling

2011-01-11 15:59  sloot

	* [r7785] added a simple test for 'special' quotes.

2011-01-11 15:27  sloot

	* [r7784] somewaht better results, but still strange!

2011-01-11 15:01  mvgompel

	* [r7783] fix

2011-01-11 14:38  mvgompel

	* [r7781] oops, forgot )

2011-01-11 14:32  mvgompel

	* [r7780] omgekeerd is beter denk ik

2011-01-11 14:32  mvgompel

	* [r7779] fixes

2011-01-11 14:29  sloot

	* [r7778] added test for quotation

2011-01-11 14:24  mvgompel

	* [r7777] Important bugfixes!

2011-01-11 13:26  mvgompel

	* [r7776] extra debug level

2011-01-11 13:02  sloot

	* [r7775] created a tests directory.
	  added tokenized ans Verbose Tokenized output for comparision

2011-01-11 11:59  sloot

	* [r7774] fixed all warnings.
	  Not very smart yet.

2011-01-11 09:16  sloot

	* [r7773] updatet Man page

2011-01-10 18:03  mvgompel

	* [r7771] removed some obsolete things

2011-01-10 18:01  mvgompel

	* [r7770] fixes, quote output in FoLiA works now

2011-01-10 17:45  mvgompel

	* [r7769] fixes

2011-01-10 17:33  mvgompel

	* [r7768] bugfixes

2011-01-10 17:15  mvgompel

	* [r7767] improved debug mode + bugfixes

2011-01-10 15:44  mvgompel

	* [r7765] Added a new passthru mode for pre-tokenised text

2011-01-10 14:58  mvgompel

	* [r7762] oops 2, removed debug line

2011-01-10 14:57  mvgompel

	* [r7761] oops, fixed segfault

2011-01-10 14:29  mvgompel

	* [r7760] major refactoring. Improved buffering, less unnecessary
	  storing of token/sentence vectors in memory. Improved quote
	  support.

2011-01-10 11:42  mvgompel

	* [r7759] set BOS in sentences in quotes

2011-01-10 11:36  mvgompel

	* [r7758] fixed EOS detection before punctuation

2011-01-10 11:12  mvgompel

	* [r7756] working on proper output with quote support

2011-01-07 17:49  mvgompel

	* [r7754] updated some options

2011-01-07 15:54  mvgompel

	* [r7753] version bump

2011-01-07 15:49  mvgompel

	* [r7752] Ucto now remembers if a token was spaced or not in the
	  original. Enabling ucto to recontruct the original text exactly.

2011-01-07 14:04  mvgompel

	* [r7749] Added quote detection support

2011-01-06 18:25  mvgompel

	* [r7746] update?

2011-01-06 18:04  mvgompel

	* [r7745] more debug update

2011-01-06 18:02  mvgompel

	* [r7744] debug updates

2011-01-06 17:52  mvgompel

	* [r7743] updated help

2011-01-06 17:49  mvgompel

	* [r7742] more sensible debug messages

2011-01-06 17:35  mvgompel

	* [r7741] I think that all changes merit a version bump now

2011-01-06 17:35  mvgompel

	* [r7740] added my testset

2011-01-06 16:57  mvgompel

	* [r7739] important fixes in paragraph/sentence handling

2011-01-06 15:57  mvgompel

	* [r7738] some fixes in sentence/paragraph detection (not enough
	  yet)

2011-01-06 15:31  sloot

	* [r7737] now we can debug init() too

2011-01-06 15:28  sloot

	* [r7736] fixing

2011-01-06 15:04  sloot

	* [r7735] refuse to run when inputfile is bad

2011-01-06 14:58  mvgompel

	* [r7734] consistency updates

2011-01-06 14:51  mvgompel

	* [r7733] set docid to string, assuming utf-8

2011-01-06 14:46  mvgompel

	* [r7732] paragraph fix in XML output

2011-01-06 14:29  mvgompel

	* [r7731] fixed beginofsentence bug

2011-01-06 13:55  mvgompel

	* [r7730] merge fix

2011-01-06 13:50  mvgompel

	* [r7729] Added preliminary FoLiA XML output support in ucto

2011-01-06 13:50  mvgompel

	* [r7728] fix

2011-01-06 13:12  sloot

	* [r7727] added a simple 'man' page

2011-01-06 12:53  sloot

	* [r7726] terrible and destructive bug!

2011-01-06 12:52  sloot

	* [r7725] to do

2011-01-06 11:42  sloot

	* [r7724] added al -p switch to disable paragraph detection
	  how do I test that?

2011-01-06 11:14  sloot

	* [r7723] -d option now works

2011-01-05 15:40  mvgompel

	* [r7716] version update

2011-01-05 15:30  mvgompel

	* [r7714] Updated version

2011-01-05 15:07  pberck

	* [r7713] Small fix.

2011-01-05 15:05  mvgompel

	* [r7712] Big API overhaul

2011-01-05 13:43  sloot

	* [r7707] fix missing ChangeLog message while bootstrapping

2011-01-05 13:36  pberck

	* [r7706] Renamed Swedish tokconfig to tokconfig-sv according to
	  ISO-639-1.

2011-01-05 13:31  pberck

	* [r7705] First attempt at Swedish tokconfig added.

2010-12-27 12:22  mvgompel

	* [r7615] typo

2010-12-27 12:22  mvgompel

	* [r7614] updated

2010-12-27 12:22  mvgompel

	* [r7613] upaded README

2010-12-25 13:05  joostvb

	* [r7605] use sysconfdir variable, not hardcoded etc/

2010-12-25 12:59  joostvb

	* [r7604] do not install changelog, packagers take care of that

2010-12-25 12:23  joostvb

	* [r7600] release 0.0.1 before releasing 1.0

2010-12-25 12:18  joostvb

	* [r7599] ChangeLog is autogenerated, see bootstrap

2010-12-16 10:23  sloot

	* [r7443] '==" is illegal, but works on GNU system

2010-12-16 09:18  sloot

	* [r7440] adapting pberck's MacOSX adaptations

2010-12-16 09:18  sloot

	* [r7439] hmm

2010-12-16 06:50  pberck

	* [r7435] Fix for compiling on OS X.

2010-12-15 21:19  antalb

	* [r7434] changed Tokenizer to ucto

2010-12-15 17:10  sloot

	* [r7433] now the EOSMARKERS are realy fixed, i hope.

2010-12-15 16:00  sloot

	* [r7430] hmm, less const, otherwise FrogNG chokes

2010-12-15 15:54  sloot

	* [r7429] oesp

2010-12-15 15:53  sloot

	* [r7428] used a more clean UCI function to extract unicode numbers
	  to real characters

2010-12-15 15:49  sloot

	* [r7427] fixed other languages too
	  al lot of duplication, can't we do better?

2010-12-15 15:44  sloot

	* [r7425] fixed dutch too

2010-12-15 15:40  sloot

	* [r7422] two characters were wrong
	  some unicode entries were non-standard

2010-12-15 14:58  mvgompel

	* [r7418] renamed Tokenizer --> ucto

2010-12-15 14:19  sloot

	* [r7413] more const clean

2010-12-15 14:02  sloot

	* [r7412] more C++ like

2010-12-15 13:49  sloot

	* [r7411] some reshuffling. make local things more local

2010-12-15 11:57  sloot

	* [r7408] hmpfff, wrong messages

2010-12-15 11:42  sloot

	* [r7406] automagicly create an icu.pc file.
	  ICU doesn't provide one (it will do so starting with 4.6)

2010-12-14 16:59  sloot

	* [r7398] added more output fields

2010-12-14 13:58  sloot

	* [r7394] another attempt to fix it for MacOSX too

2010-12-13 21:28  mvgompel

	* [r7386] Added default configuration if no -c is specified, also
	  added -L shortcut

2010-12-13 21:00  mvgompel

	* [r7385] Added features + fixes

2010-12-13 18:20  mvgompel

	* [r7380] preparing for extra functions

2010-12-13 17:14  sloot

	* [r7379] reverted change. Mac only supports archaic ICU versions

2010-12-13 17:00  sloot

	* [r7378] attempt. but does it compile on a mac?

2010-12-13 16:49  sloot

	* [r7377] unused fuction removed

2010-12-13 16:40  sloot

	* [r7376] more standard ICU solution

2010-12-13 15:53  sloot

	* [r7367] merged with some new stuff from FrogNG

2010-12-13 15:07  sloot

	* [r7364] properties set

2010-12-13 14:59  sloot

	* [r7363] build a pkgconfig file

2010-12-13 13:20  sloot

	* [r7351] added

2010-12-13 13:16  sloot

	* [r7350] build an use libucto.so

2010-12-13 12:02  sloot

	* [r7346] did some cherry picking in unicode_utils.* and included
	  them in tokenize.*

2010-12-09 16:45  sloot

	* [r7260] oeps i did it again

2010-12-09 16:44  sloot

	* [r7258] oeps³

2010-12-09 16:42  sloot

	* [r7257] oeps²

2010-12-09 16:40  sloot

	* [r7256] oesp

2010-12-09 16:37  sloot

	* [r7255] let's be verbose

2010-12-09 16:09  sloot

	* [r7254] needed for build system

2010-12-09 16:07  sloot

	* [r7253] it works!
	  inclusive a small demo

2010-12-09 14:21  sloot

	* [r7250] more essential files

2010-12-09 14:16  sloot

	* [r7249] add some files

2010-12-09 14:14  sloot

	* [r7248] created basic dir structure

2010-12-09 11:59  sloot

	* [r7242] new dir

