Comments (1)
Aspect-basedなPDSに関して調査した研究。
たとえば、Wikipediaのクジラに関するページでは、biological taxonomy, physical dimensions, popular cultureのように、様々なアスペクトからテキストが記述されている。ユーザモデルは各アスペクトに対する嗜好の度合いで表され、それに従い生成される要約に含まれる各種アスペクトに関する情報の量が変化する。
UserStudyの結果、アスペクトベースなユーザモデルとよりfitした、擬似的なユーザモデルから生成された要約の方が、ユーザの要約に対するratingが上昇していくことを示した。
また、要約の圧縮率に応じて、ユーザのratingが変化し、originalの長さ>長めの要約>短い要約の順にratingが有意に高かった。要約が長すぎても、あるいは短すぎてもあまり良い評価は得られない(しかしながら、長すぎる要約は実はそこまで嫌いではないことをratingは示唆している)。
Genericな要約とPersonalizedな要約のfaitufulnessをスコアリングしてもらった結果、Genericな要約の方が若干高いスコアに。しかしながら有意差はない。実際、平均して83%のsentenceはGenericとPersonalizedでoverlapしている。faitufulnessの観点から、GenericとPersonalizedな要約の間に有意差はないことを示した。
museum等で応用することを検討
from paper_notes.
Related Issues (20)
- Prometheus 2: An Open Source Language Model Specialized in Evaluating Other Language Models, Seungone Kim+, N/A, arXiv'24
- A Careful Examination of Large Language Model Performance on Grade School Arithmetic, Hugh Zhang+, N/A, arXiv'24
- Distillation Matters: Empowering Sequential Recommenders to Match the Performance of Large Language Model, Yu Cui+, N/A, arXiv'24
- In-Context Learning with Long-Context Models: An In-Depth Exploration, Amanda Bertsch+, N/A, arXiv'24
- Benchmarking Large Language Models for News Summarization, Tianyi Zhang+, N/A, arXiv'23 HOT 1
- Can Large Language Models Be an Alternative to Human Evaluations?, Cheng-Han Chiang+, N/A, arXiv'23
- The Perils of Using Mechanical Turk to Evaluate Open-Ended Text Generation, Marzena Karpinska+, N/A, EMNLP'21 HOT 2
- ReFT: Representation Finetuning for Language Models, Zhengxuan Wu+, N/A, arXiv'24 HOT 1
- Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations?, Zorik Gekhman+, N/A, arXiv'24 HOT 1
- Mistral 7B, Albert Q. Jiang+, N/A, arXiv'23 HOT 2
- RoFormer: Enhanced Transformer with Rotary Position Embedding, Jianlin Su+, N/A, arXiv'21 HOT 1
- GLU Variants Improve Transformer, Noam Shazeer, N/A, arXiv'20 HOT 1
- COMET: A Neural Framework for MT Evaluation, Ricardo Rei+, N/A, arXiv'20
- Multi-Dimensional Evaluation of Text Summarization with In-Context Learning, Sameer Jain+, N/A, arXiv'23 HOT 1
- Automated Evaluation of Personalized Text Generation using Large Language Models, Yaqing Wang+, N/A, arXiv'23
- ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debate, Chi-Min Chan+, N/A, arXiv'23
- FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation, Sewon Min+, N/A, arXiv'23
- T5Score: Discriminative Fine-tuning of Generative Evaluation Metrics, Yiwei Qin+, N/A, arXiv'22
- Using and Evaluating User Directed Summaries to Improve Information Access
- The Identification of Important Concepts in Highly Structured Technical Papers, Paice+, 1993
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from paper_notes.