Install IRSTLM

Install and setup notes

1. Download here

(sudo) bash regenerate-makefiles.sh (use bash or sh)
./configure (--enable-shared if you want to use the python wrapper) --prefix=XXXXX
make
make install
echo 'export IRSTLM=/PATH/' >> ~/.bashrc

1. TEST: Download sample here

(sudo) tlm -tr="gunzip -c test-a.gz" -n=3 -lm=wb -te=test

python wrapper

• Distributed by Dr.Madnani
• edit Makefile (for your PATH environment)

(e.g.)
IRSTLM_LIBS=$[[HOME]]/tools/irstlm-5.80.03/src/.libs
IRSTLM_INC=$HOME/tools/irstlm-5.80.03/src
PYTHON_INC=$HOME/.[[pythonz]]/pythons/CPython-2.7.5/include/python2.7

• make
• add environment info in the .bashrc

(e.g.)
export IRSTLM=$HOME/tools/irstlm-5.80.03
export PATH=$IRSTLM/src:$IRSTLM/src/.libs/:$PATH
export LD_LIBRARY_PATH=$HOME/tools/irstlm-5.80.03/src/.libs/:$LD_LIBRARY_PATH

• Test with Test.py

LM with Google 1T web corpus (LDC2006T13)

• Basic instruction (from pg.25)

(e.g.)
build-sublm.pl --size 3 --ngrams "gunzip -c 3gms/*.gz" --sublm LM.000 --witten-bell
merge-sublm.pl --size 3 --sublm LM -lm g_3grams_LM.gz
compile-lm g_3grams_LM.gz g_3grams_LM.blm
(if you get the error: "lt-compile-lm: lmtable.h:247: virtual double lmtable::setlogOOVpenalty(int): Assertion `dub > dict->size()' failed.")
compile-lm -dub=100000000 g_3grams_LM g_3grams_LM.blm (make the -dub option bigger)