#+TITLE: Notes for Reuse of Free Resources in Machine Translation between Nynorsk and Bokmål
#+OPTIONS: skip:nil num:nil author:nil 
#+EMAIL: unhammer at gmail dot com
#+LANGUAGE: en
#+TAGS: ROTETE(r)
#+SEQ_TODO: SKRIV FERDIG
#+EXPORT_EXCLUDE_TAGS: ROTETE


* FERDIG Although the authors argue that the MT developed can be used both for the translation from Nynorsk to Bokmål, and the other way around, it is only evaluated in one direction. 
It would be nice to evaluate the translator in the other direction, since the number of structural transfer rules is quite different (33 versus 8).

* FERDIG In the introduction, it is not clear whether the system is able to translate in both directions or only from nn to nb.
==> BETWEEN

* FERDIG In section 1, you mention that the system is aimed at being acceptable for post-editing, it would be useful to mention the possible usages of MT systems (assimilation and dissemination) and say that your system is aimed at being useful for dissemination purposes, which always requires post-editing.
==> NO USE FOR A GISTING SYSTEM

* SKRIV Also it would be nice to know the amount of effort needed to develop the system; this would help other developers in deciding to build a new language pair for Apertium by re-using existing resources and semi-automatically building some of them.
  - Note taken on [2009-10-13 Tue 13:54] \\
    I tried to indicate that in the bidix section at least.. but I guess I
    could mention that I at least only worked 12 weeks full time, or
    something like that. -k

* FERDIG Finally, the paper is well written and easy to follow. The authors should avoid the use of contractions such as "doesn't" and use the more formal way "does not".

* FERDIG Although Corbí-Bellot el al. (2005) provide the first description of Apertium, it is better explained by Armentano et al. (2006) on their paper "Open-source Portuguese-Spanish machine translation".

* FERDIG In section 2, you argue that the dictionaries are compiled into reversible finite-state transducers (FST); note that the fact that FSTs are reversible is irrelevant, because no tool in Apertium reverses them.
  - Note taken on [2009-10-13 Tue 13:44] \\
     Do we have space to explicate this? Or is it ok to just remove "reversible"?

It is true that Apertium dictionaries are used both for analysis and
generation; however, Apertium uses two different FSTs, one for
analysis and another for generation. Monolingual dictionaries are
compiled twice: to obtained a FST for analysis, and to obtained a FST
for generation.


* SKRIV The Apertium pipeline description should be improved so as to allow readers not knowing Apertium to understand how it works.
==> I am not sure that it can be done any better (especially not given the
    space constraint.

* FERDIG When Constrain Grammar (CG) are described (sec. 2.2) it is not clear whether you use the Apertium tagger after CG. Later in the paper it becomes clear that the tagger is used; anyway, I think you should say something here.
  - Note taken on [2009-10-13 Tue 13:59] \\
    Hmm. We do say "This language pair differs from most of the other
    Apertium pairs in using a Constraint Grammar (CG) module3 as a
    pre-disambiguator (before the HMM).", but I guess we could be more explicit.
==> I fail to see that it could be more explicit than that.

* FERDIG In section 3.1, please rewrite sentence "..., a > 100,000 lemma ....".

* FERDIG In section 3.3, please explain the meaning of each tag you use in the examples. What do "subst.appell", "subst.prop", "n" and "np" mean?

* FERDIG Sánchez-Martínez et al. (2008) describes a method to estimate the HMM parameters, not a more advanced statistical model; the model whose parameters are estimated is still an HMM. Please rewrite that sentence to clarify this.

* FERDIG Also mention that you have used the Baum-Welch algorithm (you say you trained the tagger in an unsupervised way) to train the HMM-based tagger and how many iteration of this algorithm you have done.

* FERDIG In section 3.4. how many times have you added your bilingual dictionary to the corpus before running Giza++? If it was n times, how did you decide the value of n?

* FERDIG Sometimes you refer to the entries in the bilingual dictionary as "transfer entries" which may be confusing, specially when speaking about the use of ReTraTos.

* FERDIG In section 3.5, example (1), please provide the language, using ISO-639 codes if you want, and make clear which is the source language and the target language. Same thing applies to rest of examples.
==> Footnote changed.

** FERDIG In section 3.5, footnote nr. 10 (now 11) should be moved to the end of the sentence.
   CLOSED: [2009-10-23 Fri 21:06]

* FERDIG Explain what is a V2 language.
 - Note taken on [2009-10-13 Tue 16:30] \\
    I just removed the parenthesis. Not important, is it?
    ==> interested readers will know.

* FERDIG Explain what is coreference chaining.

* FERDIG The second paragraph in section 2.2 is confusing.
==> Now more specific reference to CG in the second paragraph.

** FERDIG The text says: "CG is also the only grammar-based parsing method to give parsing results comparable to statistical parsers. Where statistical parsers have been shown to have a ceiling under 97% ..."  I assume that the authors intended to write "taggers" instead of "parsers" (no statistical parser obtains an accuracy of 97%!). The footnote that follows indeed supports my assumption because all the papers listed (Leech, Brants, Brill) are papers about PoS tagging.
 - Note taken on [2009-10-13 Tue 14:38] \\
    Well, it is a confusing area since CG sort of mixes tagging with
    parsing; and syntactic parsing results may improve tagging
    results... But we don't really have space to mention this.
** FERDIG Then the text continues: "...Voutilainen and Heikkila (1994) and Bick (2000) both cite accuracy results above 99%, for English and Portuguese." One cannot directly compare the tagging accuracy of taggers that perform full disambiguation (like say TnT) to constraint grammar taggers which do not always remove all spurious analysis. Here the authors are comparing accuracy vs. recall. This important difference should be noted in the text.
   CLOSED: [2009-10-23 Fri 21:06]
   - Note taken on [2009-10-13 Tue 14:40] \\
     Bick does mention this difference... must re-read.
====> must look into this!

    «When forcing the parser into full disambiguation, where all
    words - with the exception of the rare cases of true ambiguity -
    end up with one reading only, recall and precision will obviously
    assume identical values, and one can regard the recall/precision
    figure as a direct measure for the parser's performance, which is
    why I will henceforth use the more general term correctness to
    mean recall/precision at 100% disambiguation.

    During the project period I have done some such correctness
    evaluation on unknown texts, too. These test runs, while being
    fairly small, consistently suggest a correctness rate of over 99%
    for morphology and part of speech, when analysing unknown
    unrestricted text.» \citep[p.~187--188]{bick2000palavras}

However: 

    «Of all words, 94.5% became unambiguous, and 99.8% (4,840) of all
    words retained the correct morphological analysis.»
    \citet[p.~9]{voutilainen1994engcg}



* FERDIG In section 3.5, "... as the syntactic analysis of the OBT is still not incorporated into Apertium". This suggest to the reader that incorporating syntactic analysis (parsing structures) is possible (this issue is again stated in section 5). A discussion related to this is needed.
 - Note taken on [2009-10-13 Tue 15:34] \\
    We don't really have space for more discussion... :-/


* FERDIG About the evaluation (section 4.2), you provide the WER and the percentage of unknown words, of these unknown words, how many are free rides? You should compare the percentage of unknown words that are not free rides to the percentage of unknown words in the other MT system.


* FERDIG In section 2.1, the individual modules of the Apertium pipeline are mentioned: morphological analysis, PoS tagging, transfer module and de-/reformatting. For the novice reader it is not clear where generation fits into the pipeline.

* FERDIG In the evaluation part, both BLEU and WER are used.  Readers not familiar with these criteria can refer to the BLEU paper cited but no reference is cited with regard to WER (excluding the Perl program used). If no paper is cited regarding WER then at least WER should be explained in a few words.

* FERDIG In section 4.3, in the discussion about CG analysis, it is stated that "223 superfluous readings had not been removed (remained undisambiguated) without causing errors in the final translation".  This is confusing because, according to the earlier discussion, an HMM tagger is used to fully disambiguate the source language.  Does this mean that the HMM tagger was able to remove all these superfluous readings and therefore it (the HMM tagger) is the one to thank for no translation errors in the system in this case?

* FERDIG In section 5, I find the discussion about compounds not clear. How can a compound translation be analysed as two nouns? Isn't a Norwegian compound a single noun?
 - Note taken on [2009-10-13 Tue 15:04] \\
    By recursion, yes... changed "compounding is very productive"
    into "words combine very productively into compunds" etc.

* FERDIG Why is it "easy to restrict the analysis of compounds to those for which we can expect good translation"?
 - Note taken on [2009-10-13 Tue 15:06] \\
    This is explained after the period of that sentence. Changed the
    period into a colon.

* FERDIG When the term "machine translation" is used the first time in the introduction, the abbreviation "MT" should follow in parenthesis (MT is used first as an abbreviation in section 2.2).

* FERDIG The Corbí-Bellot et al. reference is redundant in section 2.1, because it is already cited in the introduction.

* FERDIG The abbreviation "eg." is used in various places throughout the paper.  I prefer the standard "e.g."
  - Note taken on [2009-10-13 Tue 13:51] \\
     wow, we had 17 e.g.'s.

* FERDIG In section 2.2, a missing "(" in front of "tagged both as a past and present tense verb)"
 - Note taken on [2009-10-13 Tue 15:14] \\
    It's there, read the whole sentence.

* FERDIG In section 2.2, "The last reading is never removed, although we may end up with several readings ...".  Shouldn't "although" be "and"?

* FERDIG In section 5, there is a missing period in the first sentence.

* FERDIG In the reference list, the Corbí-Bellot reference presumably has "Spain" instead of "spain".
  - Note taken on [2009-10-13 Tue 13:49] \\
     Armentano 2006 used instead



