Authors: Obeida ElJundi, Wissam Antoun, Nour El Droubi, Hazem Hajj, Wassim El-Hajj, Khaled Shaban
Paper: https://www.aclweb.org/anthology/W19-4608
Source Code Repository: https://github.com/aub-mind/hULMonA
Project: http://oma-project.com/
Introduction
Recent state-of-the-art models in NLP (e.g., BERT, GPT, ULMFiT) utilize transfer learning by pre-training a language model on large corpus and then fine-tuning it on any downstream task. We developed the first Arabic specific universal language model, hULMonA, that can be fine-tuned for almost any Arabic text classification task. We evaluated hULMonA on Sentiment Analysis and achieved state-of-the-art on 4 Arabic datasets. hULMonA consists of three main stages:
1. General domain hULMonA pretraining
To capture the various properties of the Arabic language, we train the SOTA-ish language model AWD-LSTM on all Arabic Wikipedia.
This step is time-consuming, but it should be done only once. We publish our pre-trained model, and it is available in the models directory. To check the implementation details, or to pre-train your own LM, check build_arabic_language_model.ipynb
2. Target task hULMonA fine-tuning
The target task data (e.g., Twitter) will likely come from a different distribution than the general-domain data (Wikipedia). Therefore, fine-tuning the pretrained general-domain LM on the target task data is necessary for the LM to adapt to the new textual properties (e.g., dialects).
To fine-tune the pre-trained hULMonA on your own dataset, please check fine_tune_LM.ipynb
3. Target task classification
Finally, for downstream task classification, we augment the fine-tuned hULMonA with two fully connected layers with ReLU and Softmax activations respectively. Implementation details can be found here: fine_tune_LM.ipynb
How do I cite hULMonA?
Please cite this paper:
@inproceedings{eljundi2019hulmona,
title={hULMonA: The Universal Language Model in Arabic},
author={ElJundi, Obeida and Antoun, Wissam and El Droubi, Nour and Hajj, Hazem and El-Hajj, Wassim and Shaban, Khaled},
booktitle={Proceedings of the Fourth Arabic Natural Language Processing Workshop},
pages={68--77},
year={2019}
}
Contact information
For help, issues, or personal communication related to using hULMonA, please contact Obeida ElJundi (oae15@mail.aub.edu), Wissam Antoun (wfa07@mail.aub.edu), or Nour El Droubi (ngd02@mail.aub.edu)
Great post. I was checking continuously this blog and I am impressed! Extremely helpful info particularly the last part I care for such info a lot. I was looking for this certain info for a long time. Thank you and best of luck. rtp slot
You’re really good. I accept your opinion It’s amazing what articles you come up with. ebet ลิงค์รับทรัพย์
Im creating articles upon cancers, which can be some tips i was initially undertaking study for when My partner and i observed your blog post submit. There are usually countless health problems available that people are generally being affected by, I had no idea! This can be strong details, many thanks. Email Marketing Best Practices
I am glad that it turned out so well and I hope it will continue in the future because it is so worthwhile and meaningful to the community. 借錢
Cool stuff you have got and you keep update all of us. togel online
Your article has sparked a conversation that I believe is important and necessary.
ku bet
You should join in a tournament first of the greatest blogs on the web. I will recommend this web site! Plafond schilderen Limburg
You have brought up a very fantastic points , thankyou for the post. معدات الصحة واللياقة
This website is usually a walk-through you discover the info it suited you about it and didn’t know who need to. Glimpse here, and you’ll definitely discover it. IFA Guildford
The development of hULMonA, an Arabic-specific universal language model, showcases the power of transfer learning in adapting models to different tasks. Just like how hULMonA needs to be fine-tuned on task-specific data to improve its performance, products like rubber matting are designed to meet the unique requirements of various environments. Whether it’s for gyms, workshops, or outdoor spaces, Rubber Matting needs to be tailored to ensure durability, flexibility, and functionality, much like how NLP models must be adjusted for specific use cases. This fine-tuning process in both fields ensures optimal performance and results.
Thanks for thoroughly. It’s an eye-opener! safeway direct2hr
Your article content is being very much interested, I am very impressed with your post. I hope to receive more great posts Panda Express Feedback
Great article. I completely agree with your points.
Thanks for sharing
Also visit my webpage sports betting
I have read your article, it is very informative and helpful for me. I admire the valuable information you offer in your articles. Thanks for posting it.. Official Site
This was an amazing read! You’ve inspired me to learn more about this topic. Google
How many times will I read your article? It opened my eyes to the world. There is always new knowledge inserted. ดาวน์โหลด wm
I really appreciate the depth of knowledge you bring to your posts. So helpful! Facebook
Such a fantastic post! Can’t wait to read more from you. Twitter
If you need help with the AZMVDNow process or want to get on-call assistance by dialing the AZMVDNow phone number, then this page is for you; it has clear step-by-step instructions and essential contact information. AZ MVD
This was incredibly helpful. I’m definitely going to share this with others. Instagram
So many great ideas here, I’m excited to try them out! Twitch
I am not very great with English but I find this very leisurely to translate. https://gidonlinehd.me/movies/novogodnie-filmy
I appreciate the thoughtfulness and care with which you approached your subject matter. nổ hũ win
I love how you explain things with such clarity. Great job! Blogger
Your posts always leave me inspired. Can’t wait for the next one! Tumblr
Great things you’ve always shared with us. Just keep writing this kind of posts.The time which was wasted in traveling for tuition now it can be used for studies.Thanks Lintas88
This is exactly the type of content I’ve been looking for. Great work! ChatGPT
I admire your ability to distill complex ideas into clear and concise prose.
sòng bạc online
Wow very amazing! This is such a thorough and well-explained post. I appreciate how you broke everything down step by step. Its wonderfull
PANDORA88
Howdy just wanted to give you a brief heads up and let you know a few of the pictures aren’t loading correctly. I’m not sure why but I think its a linking issue. I’ve tried it in two different internet browsers and both show the same outcome. Redecanais hd
I love how you included examples—it makes everything easier to understand! Claude
This is such a great resource that you are providing and you give it away for free. I love seeing blog that understand the value. Im glad to have found this post as its such an interesting one! I am always on the lookout for quality posts and articles so i suppose im lucky to have found this! I hope you will be adding more in the future… kawi 777
This is such a well-thought-out post. Thanks for sharing your knowledge. YouTube
Excellent article and easy to realize explanation. How do I go about getting permission to post element of the content in my upcoming newsletter? Giving proper credit to you the author and link to the site would not be a problem. fsa hsa store eligible cervical neck pillow
I am speechless. This is a superb weblog and very engaging too. Nice work! That’s now not actually so much coming from an newbie publisher like me, but it’s all I could say after diving into your posts. Nice grammar and vocabulary. No longer like other blogs. You in point of fact realize what you?re talking approximately too. Such a lot that you simply made me want to discover more. Your blog has develop into a stepping stone for me, my friend. garage door opener installation
This is such an exciting development for Arabic NLP! It’s inspiring to see how hULMonA can be applied to various tasks like sentiment analysis. As someone involved in gaming app projects, like Orion Stars, I can see how such language models could help enhance chat functionalities or content classification within apps. It’s great to know these resources are becoming more accessible.
I am very enjoyed for this blog. Its an informative topic. It help me very much to solve some problems. Its opportunity are so fantastic and working style so speedy. situs slot gacor hari ini
Great article! It’s always insightful to read about advancements in security technology. AI Cloud Analytics Cameras in Menifee are a game-changer for monitoring and enhancing security in both residential and commercial areas. It’s exciting to see how AI is improving surveillance systems. AI Cloud Analytics Cameras in Menifee.
Thanks for maintaining such a high-quality site. หวย ยี่ กี ruay
I keep listening to the news bulletin lecture about receiving free online grant applications so I have been looking around for the most excellent site to get one. Could you advise me please, where could i get some? nagawon
By And Large I do not post on blogs, but I should to say that this post really forced me to do so. Genuinely nice article! 360-Degree Rotating Stainless Steel Survival Chainsaw
There are a handful of intriguing points in time in this post but I do not know if they all center to heart. There may be some validity but I’m going to take hold opinion until I explore it further. Very good post , thanks therefore we want much more! Included with FeedBurner also Cómo hacer un bonsái paso a paso
Wow! what an idea ! What a concept ! Lovely . Incredible. Traveling with electronics: adapter needs
Helpful information. Fortunate me I discovered your site by accident, and I’m stunned why this twist of fate didn’t came about earlier! I bookmarked it. 100% cotton underwear women
I like this info shown and it has given me some sort of commitment to succeed for some reason, so thank you. Free Soccer Predictions
Thanks for creating a resource that’s both useful and engaging. หวย ลาว สมัคร
When seeking assistance with laptop repair services in your vicinity, it is essential to explore local options that can provide timely and effective solutions. Many repair shops specialize in diagnosing and fixing a variety of issues, from hardware malfunctions to software problems.
By searching for laptop repair near me services, you can ensure that you receive personalized attention and support, often with the convenience of same-day service. This proximity not only allows for quicker turnaround times but also facilitates easier communication with technicians who can address your specific concerns.
I was reading some of your blog posts on this website and I think this website is rattling instructive! Keep putting up. Inmobiliaria Zaragoza
When I open up your RSS feed it gives me a bunch of garbled text, is the malfunction on my side? placas temporales texas
VIN88 là một nhà cái uy tín tại Việt Nam, cung cấp đa dạng các dịch vụ cá cược thể thao, game giải trí và các trò chơi trực tuyến khác…
Website: https://vin88pro.app/