Authors: Obeida ElJundi, Wissam Antoun, Nour El Droubi, Hazem Hajj, Wassim El-Hajj, Khaled Shaban
Paper: https://www.aclweb.org/anthology/W19-4608
Source Code Repository: https://github.com/aub-mind/hULMonA
Project: http://oma-project.com/
Introduction
Recent state-of-the-art models in NLP (e.g., BERT, GPT, ULMFiT) utilize transfer learning by pre-training a language model on large corpus and then fine-tuning it on any downstream task. We developed the first Arabic specific universal language model, hULMonA, that can be fine-tuned for almost any Arabic text classification task. We evaluated hULMonA on Sentiment Analysis and achieved state-of-the-art on 4 Arabic datasets. hULMonA consists of three main stages:
1. General domain hULMonA pretraining
To capture the various properties of the Arabic language, we train the SOTA-ish language model AWD-LSTM on all Arabic Wikipedia.
This step is time-consuming, but it should be done only once. We publish our pre-trained model, and it is available in the models directory. To check the implementation details, or to pre-train your own LM, check build_arabic_language_model.ipynb
2. Target task hULMonA fine-tuning
The target task data (e.g., Twitter) will likely come from a different distribution than the general-domain data (Wikipedia). Therefore, fine-tuning the pretrained general-domain LM on the target task data is necessary for the LM to adapt to the new textual properties (e.g., dialects).
To fine-tune the pre-trained hULMonA on your own dataset, please check fine_tune_LM.ipynb
3. Target task classification
Finally, for downstream task classification, we augment the fine-tuned hULMonA with two fully connected layers with ReLU and Softmax activations respectively. Implementation details can be found here: fine_tune_LM.ipynb
How do I cite hULMonA?
Please cite this paper:
@inproceedings{eljundi2019hulmona,
title={hULMonA: The Universal Language Model in Arabic},
author={ElJundi, Obeida and Antoun, Wissam and El Droubi, Nour and Hajj, Hazem and El-Hajj, Wassim and Shaban, Khaled},
booktitle={Proceedings of the Fourth Arabic Natural Language Processing Workshop},
pages={68--77},
year={2019}
}
Contact information
For help, issues, or personal communication related to using hULMonA, please contact Obeida ElJundi (oae15@mail.aub.edu), Wissam Antoun (wfa07@mail.aub.edu), or Nour El Droubi (ngd02@mail.aub.edu)
Gemwin là cổng game trực tuyến với đa dạng trò chơi, bảo mật cao và trải nghiệm cá cược mượt mà, an toàn.
Website: https://gemwinpro.app/
Thank you for sharing excellent information. Your website is very cool. I am impressed by the info that you?ve on this website. It reveals how nicely you understand this subject. Bookmarked this web page, will come back for more articles. Learnician
Sweet web site , super style and design , really clean and use pleasant. quantum healing
Your article has sparked a conversation that I believe is important and necessary.สมัคร หวย หุ้น เกาหลี
Can I simply say what a relief to seek out somebody who really is aware of what theyre speaking about on the internet. You undoubtedly know the best way to bring a problem to mild and make it important. Extra people need to learn this and perceive this aspect of the story. I cant believe youre not more widespread since you positively have the gift. olxtoto
You have brought up a very superb points, appreciate it for the post. surgical tech certification programs florida
There are bargain dvd players that are sold in our area. i think they are generic low cost dvd players” Online surgical tech program
I must express my gratitude for your kindness supporting men who must have help on your concern. Your special commitment to passing the solution across ended up being wonderfully advantageous and have continually helped many people much like me to get to their aims. Your personal helpful help implies a great deal a person like me and much more to my office workers. Best wishes; from all of us. olxtoto resmi
Unquestionably believe that which you said. Your favorite justification appeared to be on the web the simplest thing to be aware of. I say to you, I definitely get annoyed while people consider worries that they just don’t know about. You managed to hit the nail upon the top and defined out the whole thing without having side-effects , people could take a signal. Will likely be back to get more. Thanks แทงบอลออนไลน์
I always visit new blog everyday and i found your blog.;”\””` olxtoto login
What a fantabulous post this has been. Never seen this kind of useful post. I am grateful to you and expect more number of posts like these. Thank you very much. price airdrop
laptop bags using natural leather is my choice because natural leather is much softer and looks better- 카지노커뮤니티순위
I’m speechless. This is a very good blog and really attractive too. Nice work! That’s now not in reality so much coming from an beginner publisher like me, but it’s all I may just say after diving into your posts. Nice grammar and vocabulary. No longer like other blogs. You truly understand what you?re talking about too. Such a lot that you made me wish to discover more. Your weblog has transform a stepping stone for me, my friend. 4140 çelik
I am usually to blogging and i genuinely appreciate your content regularly. This great article has really peaks my interest. My goal is to bookmark your web site and keep checking for brand new details. 리니지프리서버 프리베네
leather bags look really nice specially those natural leather bags, ; Battle Creek lawn maintenance services
you’ve got a terrific blog right here! would you like to make some invite posts on my weblog? sell a small business
OMG okay
Interesting blog! Is your theme custom made or did you download it from somewhere? A theme like yours with a few simple tweeks would really make my blog jump out. Please let me know where you got your design. With thanks elram gavrieli