Selected controversial TREC10 evaluations of Webclopedia
Question 991: What is cryogenics?
Qtargets: Q-DEFINITION
Qargs: cryogenics
- Cryogenics - engineering at extremely low temperatures - is a serious business, however, with a range of applications in space and at a more down-to-earth level. [Score: 17211.0; 08/04/92; FT923-9701]
- three other answers
- Late for Dinner (Castle Rock/Nelson). Shooting in New Mexico and L.A. "Buckaroo Banzai" director W. D. Richter co-produces and directs this sci-fi comedy examining the travails of two regular guys who accidentally get freeze-dried in a cryogenics lab and emerge 30 years later. [Score: 12.9; 03/04/90; LA030490-0024]
First answer judged incorrect, last one correct and supported.
Webster entry for `cryogenics': the branch of physics that deals with very low temperatures
Scoring: Webclopedia highly favors the first answer because (1) it contains cryogenics (2) it finds a reliable apposition (3) definition candidate overlaps with WordNet gloss ("the branch of physics that studies the phenomena that occur at very low temperatures")
Scoring: Webclopedia gets a very low score for the last answer (in fallback mode): (1) it contains cryogenics (2) there are a couple of words around it with decent information content
Compare: What is physics? ... two guys who accidentally get freeze-dried in a physics lab ...
Compare: What is cryogenics? ... two guys who accidentally get lost in a cryogenics lab ...
Compare: Last answer (get freeze-dried ...) is judged supported, but answer to question 900 is judged unsupported.
Question 900: George Bush purchased a small interest in which baseball team?
Qtargets: S-NP, S-NOUN
- George W. Bush, the president's son, and the group he is leading will buy the Texas Rangers baseball team. [AP890318-0015]
Judged correct, but unsupported.
List question 2: Name 32 countries Pope John Paul II has visited.
Qtargets: I-EN-PROPER-COUNTRY
Qargs: John, type
Qplural: 32
- Pope John Paul II will visit Communist Hungary in September 1991 on his first trip to an East European country other than his native Poland, the state-run MTI news agency says. [AP890322-0003]
- Pope John Paul II will visit the Muslim country of Senegal in February, President Abdou Diouf announced Wednesday.; [SJMN91-06179215]
- Pope John Paul II will visit the West African country of Guinea-Bissau in January and tour a leper colony during a two-day stay, the Portuguese news agency Lusa reported Monday. [LA092689-0101]
- Pope John Paul II will visit Cuba in December, a Vatican spokesman said Sunday.
[Score: 733.1121; 04/22/90; AP900422-0018]
- Pope John Paul II will visit Madagascar, Zambia, Malawi and the French Indian Ocean island of Reunion from April 28 to May 6, the Vatican announced Friday. [LA021289-0127]
- Pope John Paul II will visit Madagascar, Zambia, Malawi and the French Indian Ocean island of Reunion from April 28 to May 6, the Vatican announced Friday. [LA021289-0127]
- Pope John Paul II will visit Madagascar, Zambia, Malawi and the French Indian Ocean island of Reunion from April 28 to May 6, the Vatican announced Friday. [LA021289-0127]
- ...
All answers above judged unsupported.
Fact: Thanksgiving will be on November 22 this year.
Wishful thinking: Our brave Iraqi troops will crush the Americans in the mother of all battles.
Judged supported: What is cryogenics? ... two regular guys who accidentally get freeze-dried in a cryogenics lab and emerge 30 years later ...
Question 1272: What does CPR stand for?
Qtargets: Q-ABBREVIATION-EXPANSION
Qargs: CPR
- CRA, the Australian resources group, yesterday announced it had agreed a 'farm-in' deal with Carpenter Pacific Resources (CPR), under which the latter can earn a 49 per cent interest in CRA's Hidden Valley gold project near Wau, in Papua New Guinea, for K35m (USDollars 36.9m). [FT942-15812]
- Though paramedics performed CPR, cardio-pulmonary resuscitation, doctors were unable to start the boy's heart once he arrived at the hospital's emergency room. [AP900116-0177]
First answer judged incorrect.
Question 1262: What is the Moulin Rouge?
Qtargets: Q-DEFINITION
Qargs: Moulin Rouge, organization
- The French Regency, said to have once been a residence of Elvis Presley and -- at another time -- of Howard Hughes, is on an acre and has gardens, a swimming pool, city views, two master suites, a ballroom and a room from the set of the 1952 film "Moulin Rouge," in which Gabor and Jose Ferrer starred. [LA090990-0229]
- So, it's no surprise he was greeted by applause at the Moulin Rouge, the famous nightclub. [AP890724-0040]
Both judged incorrect.
Google: Most top choices returned by Google identify Moulin Rouge as a movie.
Google: Large number of web pages identify the Moulin Rouge as a nightclub in Paris.
Question 1300: What is carbon dioxide?
Qtargets: Q-DEFINITION
Qargs: carbon dioxide|dioxide
- The crucial point is what to do about rising emissions of carbon dioxide, the principal atmospheric gas among several that trap heat like the glass panes of a greenhouse.; [SJMN91-06254099]
- In the past, U.S. officials have resisted a worldwide ceiling on emissions of carbon dioxide, the gas that scientists believe causes 60 percent of the ``greenhouse effect.'' [AP900828-0027]
- According to Energy Paper 59*, published last week, the UK will emit around 10 per cent less carbon dioxide, one of the main greenhouse gases, than was feared two years earlier in Energy Paper 58. [FT924-13745]
All judged incorrect.
WordNet: a heavy odorless colorless gas formed during respiration and by the decomposition of organic substances
Webster: a colorless, odorless, incombustible gas, CO2, present in the atmosphere and ...
(smaller font after 50 bytes)
Question 1256: What is the only artery that carries blue blood from the heart to the lungs?
Qtargets: S-NP, S-NOUN
Qargs: artery
- At Babies Hospital a more complete examination of his heart showed it had inverted ventricles, the pumping chambers, with no connection to the pulmonary artery that carries blood to the lungs. [AP900723-0190]
Judged correct, but unsupported.
Question 1387: What year did the Titanic start on its journey?
Qtargets: C-TEMP-LOC-WITH-YEAR, I-ENUM-CARDINAL (0.5)
- Seventy-seven years ago, on April 14, 1912, the British luxury liner Titanic hit an iceberg just south of Newfoundland on its maiden voyage. [LA062189-0045]
Judged unsupported.
Note: Answer sentence contains `on its maiden voyage'.