Joy's Research

• Publications
• Talks
• Course related projects and talks
• Software/Tools/Demos
  • SALM: A Toolkit of Empirical Natural Language Processing using Suffix Array (including the Suffix Array Language Model)
  • PanDoRA: a Phase-based Statistical Machine Translation System for Hand-held Devices (PDA)
  • Chinese Word Segmenter
  • Confidence interval of MT evaluation
• Research Results (internal access from *.is.cs.cmu.ed and *.mt.cs.cmu.edu; password protected from other domain)
• Other Research
• Interesting facts about language
• LTI MT Wiki Page
• ISL SMT Wiki Page

Talks

• Presentation at ACL05, Workshop on Building and Parallel Text: Data-driven Machine Translation and Beyond (WPT-05), Competitive Grouping in Integrated Segmentation and Alignment Model, [PPT], July 2005
• Presentation at EAMT 2005, An Efficient Phrase-to-phrase Alignment Model for Arbitrarily Long Phrases and Large Corpora [PDF], [PPT] May 2005.
• Presentation at MT-lunch,  Mining Translations for Key Phrases from Web Corpora, (PPT), April 2005.
• Presentation at TMI 2004, Measuring Confidence Intervals for MT Evaluation Metrics(PPT), (PDF), Oct 2004, Baltimore, MD
• Talk on "TIDES MT Workshop 2004", Measuring Confidence Intervals for MT Evaluation Metrics (PPT). (PDF). June 2004, DC. review
• CMU SMT group report (bootstrapping for MT evaluation statistical significance test; Online String Search using Suffix Array; Incremental Learning Machine Translation) . Dec 2003 (internal access only)
• LTI Student Research Symposium "An Integrated Phrase Segmentation/Alignment Algorithm for Statistical Machine Translation" (Sep12, 2003)
• CMU SMT Workshop: "Version Management with CVS" (Feb 20. 2003)
• CMU SMT Workshop: "Automatic Evaluation for Machine Translation Systems" (Feb 17, 2003)
• CMU SMT Workshop: "Tools Used for SMT" (Feb 14, 2003)
• Talk on "ABC---A Phrase to Phrase Alignment Method" in the Statistical Machine Translation Workshop (July 09, 2002)
• Presentation on MT Summit VIII: "Preprocessing of Bilingual Corpora for Mandarin-English EBMT" (PPT) (Sep 18, 200)
• Talk on current status of Chinese-English EBMT (PPT), full version (Aug23, 2001)
• Talk on "Adapting EBMT to Chinese" (PPT) (Jan 26, 2001)
• Talk on improved Chinese segmentation for EBMT (PPT) (Sep 8, 2000)

Tools

• Bi-directional Maximum Word Frequency Chinese Word Segmenter:  lrsegmenter.perl (written in perl5), Java implementation of lrSegmenter by Chenmin Liang
• You can use your own word frequency list, or use the one provided by LDC. LDC-word-freq-list (43959 entries, 358K byte)
• Given a word list and enough unsegmented corpus, one can use programs based on the suffix array indexing to assign frequencies to the words (as character n-grams). Here is a 100K Chinese word list with frequencies obtained using the CollectNgramFreqCount program over the LDC Chinese Gigaword corpus: Mandarin.fre.100K.cmu (107,830 entries, 1.1MB>
  • lrSegmenter also works for other Asian languages, such as Korean. Download Korean word frequency list here.(25,251 entries, 246K byte). Another Korean word frequency list can be found at www.dataparksearch.org
• Word segmentation evaluation kit (You are welcome to evaluate your segmenter here)
• Bootstrapping for NIST/Bleu: measuring the confidence limit and statistical significance for automatic machine translation evaluation metrics

  • Download the Latest Scripts (Last update: April 27, 2005):

    • generateLog-v11.pl
    • bootstrapSingle-v11.2.pl
    • bootstrapCompare-v11.2.pl

  • Related paper for citation:

    • Ying Zhang, Stephan Vogel, "Measuring Confidence Intervals for the Machine Translation Evaluation Metrics," In the Proceedings of The 10th International Conference on Theoretical and Methodological Issues in Machine Translation (TMI 2004), Baltimore, MD USA, October 4-6, 2004. PDF

    • Ying Zhang, Stephan Vogel, Alex Waibel, "Interpreting Bleu/NIST scores: How much improvement do we need to have a better system?," In the Proceedings of LREC 2004 Lisbon, Portugal, May 2004, PDF 

• Corrected NIST MTeval v11a-cmufix.pl
• Corrected NIST MTeval v11a-cmufix_b.pl (Mar 13, 2005)
• Integrated Segmentation and Alignment model for Statistical Phrase Alignment (source code for download)
• SALM: A Toolkit of Empirical Natural Language Processing using Suffix Array (including the Suffix Array Language Model)

  ---

Demo

• PanDoRA: a Phase-based Statistical Machine Translation System for Hand-held Devices (PDA)

Other Archives

• Interviewed by MT News International (Issue No.29)
• CVS Manual for RADD-MT project
• All about the Student's t-test

Course Projects and Talks

11-721: Grammars and Lexicons, Statistical dictionary for Tetun-English
11-731: Machine Translation, Design of Multi-lingual Machine Translation for Real-time Broadcast Captioning
11-732: Machine Translation Self-paced Lab,  "Flow Network Models for Sub-Sentential Alignment" (PPT)  PS
11-745: Advanced Statistical Learning Seminar, "Linear Methods for Classification" (PPT)
11-745: Advanced Statistical Learning Seminar, "Linear Methods for Classification Part II" (PPT)
11-745: Advanced Statistical Learning Seminar, "Additive Models, Trees and Related Methods" (PPT)
11-745: Advanced Statistical Learning Seminar, "Additive Models, Trees and Related Methods Part II" (PPT)
11-733: Multilingual Speech-to-Speech Translation Seminar

• Presentation: "Survey of Speech-to-speech translation systems: who are the players" (PPT) 
• Fact Table: speech to speech translation (HTML)
• Final Report: "Survey of Current Speech Translation Research" (PDF)
  

Personal Research

Automated statistical dictionary building

Interesting Facts about Languages

• Can you read this?
• Can you understand this?

email: joy@cs.cmu.edu

View My Stats

Last updated: Feb 04, 2007