Moses

Z MetaCentrum
Přejít na: navigace, hledání


Description

Moses is a statistical machine translation system that allows you to automatically train translation models for any language pair. All you need is a collection of translated texts (parallel corpus). An efficient search algorithm finds quickly the highest probability translation among the exponential number of choices.

Availability

Available modules:

  • moses-2.1.1: Moses version 2.1.0, 64-bit, compiled with KenLM language model toolkit and distributed computing (MPI) support, includes GIZA++ and IRSTLM model tools
  • moses-1.0: Moses version 1.0, 64-bit, compiled with KenLM language model toolkit and distributed computing (MPI) support, includes GIZA++ and IRSTLM model tools
  • moses-1.0beta: Moses version 1.0beta (release 20120926), 64-bit, compiled with KenLM language model toolkit

Use

1. Running the application -- Interactive mode:

  • ask the scheduler for an interactive job having desired number of nodes/processors (nodes and ppn attributes) reserved
    • Note.: Moses version 1.0beta (module moses-1.0beta) is compiled without the MPI support => ask just for a single node (the others won't be used by Moses)
$ qsub -I -l nodes=X:ppn=Y -l mem=Zg
Note: Do not forget to apppropriately set the amount of requested memory (mem attribute) and/or another job requirements.
  • load the Moses module
$ module add moses-1.0  # moses-1.0beta
  • prepare the configuration and the input files and run the computation
$ moses -f config_file.ini  <input_file >output_file  # 1-node computation
$ mpirun moses -f config_file.ini  <input_file >output_file  # MPI version
$ moses_chart -f config_file.ini  <input_file >output_file  # 1-node computation
$ mpirun moses_chart -f config_file.ini  <input_file >output_file  # MPI version

2. Running the application -- Batch mode:

  • prepare the job description script -- use a general skeleton supplemented by the following lines:
...
# load the Moses module
module add moses-1.0  # moses-1.0beta

# run the computation
moses -f config_file.ini  <input_file >output_file  # 1-node computation
# mpirun moses -f config_file.ini  <input_file >output_file  # MPI version
# moses_chart -f config_file.ini  <input_file >output_file  # 1-node computation
# mpirun moses_chart -f config_file.ini  <input_file >output_file  # MPI version
...
  • pass the job description file to the scheduler together with (at least) the requested number of nodes/processors and requested amount of memory
$ qsub -l nodes=X:ppn=Y -l mem=Zg mydescriptionfile.sh
  • Note.: Moses version 1.0beta (module moses-1.0beta) is compiled without the MPI support => ask just for a single node (the others won't be used by Moses)

Documentation

http://www.statmt.org/moses/manual/manual.pdf

License

open-source under the terms of the GNU Lesser General Public License

Supported platforms

amd64

Program administrator

Tom Rebok meta@cesnet.cz

Homepage

http://www.statmt.org/moses