Authors: Wissam Antoun, Fady Baly, Hazem Hajj
AraBERT is an Arabic pretrained language model based on Google’s BERT architecture. AraBERT uses the same BERT-Base config. More details are available in the AraBERT PAPER and in the AraBERT Meetup
There is two versions of the model AraBERTv0.1 and AraBERTv1, with the difference being that AraBERTv1 uses pre-segmented text where prefixes and suffixes were split using the Farasa Segmenter.
The model was trained on ~70M sentences or ~23GB of Arabic text with ~3B words.
Source Code Repository: https://github.com/aub-mind/arabert
Paper: https://www.aclweb.org/anthology/2020.osact-1.2.pdf
Results (Accuracy)
We evaluate both AraBERT models on different downstream tasks and compare it to mBERT, and other state of the art models (To the extent of our knowledge). The Tasks were Sentiment Analysis on 6 different datasets (HARD, ASTD-Balanced, ArsenTD-Lev, LABR, ArSaS), Named Entity Recognition with the ANERcorp, and Arabic Question Answering on Arabic-SQuAD and ARCD
| Task | prev. SOTA | mBERT | AraBERTv0.1 | AraBERTv1 |
|---|---|---|---|---|
| HARD | 95.7 ElJundi et.al. | 95.7 | 96.2 | 96.1 |
| ASTD | 86.5 ElJundi et.al. | 80.1 | 92.2 | 92.6 |
| ArsenTD-Lev | 52.4 ElJundi et.al. | 51 | 58.9 | 59.4 |
| AJGT | 93 Dahou et.al. | 83.6 | 93.1 | 93.8 |
| LABR | 87.5 Dahou et.al. | 83 | 85.9 | 86.7 |
| ANERcorp | 81.7 (BiLSTM-CRF) | 78.4 | 84.2 | 81.9 |
| ARCD | mBERT | EM:34.2 F1: 61.3 | EM:51.14 F1:82.13 | EM:54.84 F1: 82.15 |
Model Weights and Vocab Download
| Models | AraBERTv0.1 | AraBERTv1 |
|---|---|---|
| TensorFlow | Drive Link | Drive Link |
| PyTorch | Drive_Link | Drive_Link |
You can find the PyTorch models in HuggingFace’s Transformer Library under the aubmindlab username
If you used this model please cite us as:
@inproceedings{antoun2020arabert,
title={AraBERT: Transformer-based Model for Arabic Language Understanding},
author={Antoun, Wissam and Baly, Fady and Hajj, Hazem},
booktitle={LREC 2020 Workshop Language Resources and Evaluation Conference 11--16 May 2020},
pages={9}
}
Acknowledgments
Thanks to TensorFlow Research Cloud (TFRC) for the free access to Cloud TPUs, couldn’t have done it without this program, and to the AUB MIND Lab Members for the continous support. Also thanks to Yakshof and Assafir for data and storage access. Another thanks for Habib Rahal (https://www.behance.net/rahalhabib), for putting a face to AraBERT.
Contacts
Wissam Antoun: Linkedin | Twitter | Github | wfa07@mail.aub.edu | wissam.antoun@gmail.com
Fady Baly: Linkedin | Twitter | Github | fgb06@mail.aub.edu | baly.fady@gmail.com
We are looking for sponsors to train BERT-Large and other Transformer models, the sponsor only needs to cover to data storage and compute cost of the generating the pretraining data
The live streaming feature for sports betting is a great addition, allowing players to watch matches in real-time while placing bets. 8KBET
The site provides useful betting guides and tutorials for newcomers, helping them understand how to make informed wagers. 8KBET
I haven’t any word to appreciate this post…..Really i am impressed from this post….the person who create this post it was a great human..thanks for shared this with us. bandar togel online
J88 mang đến một thế giới giải trí đỉnh cao với kho game đa dạng, giao diện hiện đại và tỷ lệ trả thưởng hấp dẫn. Đây là điểm đến lý tưởng cho những ai yêu thích cá cược trực tuyến, hứa hẹn trải nghiệm tuyệt vời cho mọi người chơi. Đăng ký ngay hôm nay để nhận 100% tiền thưởng chào mừng và tận hưởng những phút giây kịch tính cùng J88! Thông tin chi tiết: https://j88j.app/
J88 mang đến một thế giới giải trí đỉnh cao với kho game đa dạng, giao diện hiện đại và tỷ lệ trả thưởng hấp dẫn. Đây là điểm đến lý tưởng cho những ai yêu thích cá cược trực tuyến, hứa hẹn trải nghiệm tuyệt vời cho mọi người chơi. Đăng ký ngay hôm nay để nhận 100% tiền thưởng chào mừng và tận hưởng những phút giây kịch tính cùng J88! Thông tin chi tiết: https://j88j.app/
I was surfing the Internet for information and came across your blog. I am impressed by the information you have on this blog. It shows how well you understand this subject. سایت شرط بندی
Thank you for some other informative website. The place else may just I get that kind of information written in such a perfect method? I have a venture that I am simply now running on, and I’ve been at the glance out for such info. 3xbet ทางเข้า
Very good written article. It will be supportive to anyone who utilizes it, including me. Keep doing what you are doing – can’r wait to read more posts. 3xbet เข้าสู่ระบบ
Your content is nothing short of brilliant in many ways. I think this is engaging and eye-opening material. Thank you so much for caring about your content and your readers. 3xbet
One of the main attractions of Loto188 is its extensive collection of engaging activities. Users can participate in various interactive experiences that provide both entertainment and opportunities for rewards. The platform ensures real-time updates and transparency, making it a reliable choice for users. Many appreciate the convenience of accessing these experiences online, eliminating the need for physical participation. Additionally, Loto188 frequently introduces new features to keep the experience fresh and exciting for its users. loto188
SIU88 là nền tảng cá cược trực tuyến đẳng cấp, nơi hội tụ hàng loạt trò chơi hấp dẫn như cá cược thể thao,game bài đổi thưởng và slot game. Với giao diện mượt mà, bảo mật tối đa và tỷ lệ cược hấp dẫn, SIU88 cam kết mang đến trải nghiệm giải trí đỉnh cao. Đăng ký ngay hôm nay để nhận thưởng chào mừng siêu khủng và tận hưởng niềm vui bất tận cùng SIU88! Chi tiết tại: https://siu88.agency/
78WIN là địa chỉ cá cược trực tuyến đáng tin cậy với đa dạng trò chơi như cá độ thể thao, trực tuyến, xổ số và game bài đổi thưởng. Hệ thống bảo mật tiên tiến, giao diện hiện đại cùng dịch vụ chăm sóc khách hàng 24/7 giúp 78WIN trở thành lựa chọn hàng đầu của hàng triệu người chơi. Đăng ký ngay để nhận thưởng 100% và tham gia vào sân chơi đầy kịch tính tại 78WIN! Chi tiết tại: https://78winclub.asia/
Loto188 also provides a comprehensive section for those interested in competitive and strategic activities. Users can explore a variety of options that cater to different interests and preferences. With real-time updates and interactive elements, users can engage with the platform dynamically. Loto188 offers special promotions and bonuses tailored to enhance the user experience, making it even more appealing. loto188
This article was written by a real thinking writer.I agree many of the with the solid points made by the writer 먹튀사이트
88NN mang đến trải nghiệm cá cược online chuyên nghiệp với hàng loạt trò chơi hấp dẫn như thể thao, lô đề, bắn cá và slot game. Nhà cái sở hữu tỷ lệ cược hấp dẫn, giao dịch nhanh chóng và chương trình khuyến mãi cực khủng. Hãy tham gia ngay hôm nay để tận hưởng ưu đãi hấp dẫn cùng sân chơi đẳng cấp tại 88NN! Chi tiết tại: https://88nn.institute/
Hello, this weekend is good for me, since this time i am reading this enormous informative article here at my home. ทีเด็ดบอล69
This is a great post. I like this topic.This site has lots of advantage.I found many interesting things from this site. It helps me in many ways.Thanks for posting this again 먹튀사이트
I’m sure your publish and internet site is incredibly constructed هزینه ی نصب کاغذ دیواری
WW88 là một thương hiệu cá cược trực tuyến uy tín, được cấp phép bởi PAGCOR, đảm bảo tính minh bạch và an toàn tuyệt đối. Với hệ thống bảo mật hiện đại, giao diện thân thiện và kho trò chơi đa dạng như thể thao, slot game, bắn cá, WW88 mang đến trải nghiệm giải trí đỉnh cao. Đăng ký ngay để tận hưởng khuyến mãi hấp dẫn và tham gia vào sân chơi cá cược chuyên nghiệp tại WW88! Chi tiết tại: https://ww88.theater/
bet88 – là điểm đến lý tưởng cho những ai đam mê cá cược trực tuyến. Với kho game đa dạng từ thể thao, slot game đến bắn cá, cùng tỷ lệ thưởng hấp dẫn và giao dịch siêu tốc, Bet88 mang đến trải nghiệm giải trí chuyên nghiệp, minh bạch. Tham gia ngay để tận hưởng đẳng cấp cá cược hàng đầu!
Customer support is another area where Loto188 excels, providing assistance through multiple channels, including live chat, email, and phone support. The customer service team is available 24/7 to address user inquiries, resolve technical issues, and provide guidance on various aspects of the platform. Many users commend the platform for its responsive and professional support team, which plays a crucial role in maintaining user satisfaction and loyalty. loto188
I simply want to tell you that I am new to weblog and definitely liked this blog site. Very likely I’m going to bookmark your blog . You absolutely have wonderful stories. Cheers for sharing with us your blog 카지노사이트
Customer support is another area where Loto188 excels, providing assistance through multiple channels, including live chat, email, and phone support. The customer service team is available 24/7 to address user inquiries, resolve technical issues, and provide guidance on various aspects of the platform. Many users commend the platform for its responsive and professional support team, which plays a crucial role in maintaining user satisfaction and loyalty. loto188
Wow, wonderful blog layout! How lengthy have you ever been running a blog for? you make running a blog glance easy. The total look of your web site is magnificent, well the content material! 探偵
Hello to every body, it’s my first pay a quick visit of this weblog;
this web site consists of remarkable and truly excellent material in favor of visitors.
my blog post login dewatogel
Customer support is another area where Loto188 excels, providing assistance through multiple channels, including live chat, email, and phone support. The customer service team is available 24/7 to address user inquiries, resolve technical issues, and provide guidance on various aspects of the platform. Many users commend the platform for its responsive and professional support team, which plays a crucial role in maintaining user satisfaction and loyalty. loto188
اكتشف تشكيلة مميزة من الأشمغة والأقمشة الرجالية الفاخرة والعطور في متجر شماغنا نقدم لكم منتجات ذات جودة عالية لتكملوا إطلالتكم بأسلوب عصري وأنيق تسوق الآن واستمتع بأفضل الأسعار والعروض الحصرية
Technical assistance are hired to help those from non invasive fields such as the fund community, or those who need professional knowledge 카지노사이트
Within this web page, you’ll see that appearance, when i highly recommend people study that examine. how to say kiolopobgofit
Due to this website, you’ll see your bank account, recall to plod through that facts. bolly44u
Book the Best Toronto Wedding DJs for an Unforgettable Reception
The right music can set the tone for your entire wedding day. It’s not just about the songs you love, but about creating a soundscape that reflects the love and joy you and your partner share. If you’re looking to hire a DJ in Toronto for your wedding, look no further. Toronto DJ’s are experts at creating custom playlists that will keep your guests entertained and your dance floor packed.
I am able to provides you with the target In this article understand the best way to practice it the right way. Understand in addition to generate anything beneficial. techinsiderz.com gadgets
Jika Anda mencari pengalaman bermain yang seru dan menguntungkan, nus toto adalah pilihan terbaik. Platform ini menawarkan berbagai permainan menarik dengan sistem yang aman dan terpercaya, sehingga para pemain dapat menikmati hiburan tanpa khawatir. Dengan tampilan yang user-friendly dan proses transaksi yang cepat, nus toto menjadi favorit bagi banyak penggemar taruhan online. Bergabung sekarang dan rasakan sensasi bermain dengan peluang menang yang lebih besar!
Property details for Royal Parks Shinagawa. We are currently running a campaign with “zero key money” and “special rebates, with up to 3.0 months of free rent”! This is a tower apartment property. This is a luxury rental apartment near JR Yamanote Line Shinagawa Station. Leave it to Gooroom to help you find your ideal room. Visit Here: ロイヤルパークス品川
Within this web page, you’ll see that appearance, when i highly recommend people study that examine. 5starsstocks.com defense
Good information. Lucky me I discovered your site
by chance (stumbleupon). I’ve saved as a favorite for
later!
My web page :: plinko game online real money
Ultra 88 adalah salah satu platform terbaik yang pernah saya gunakan. Selain mudah digunakan, tampilannya yang modern dan responsif membuat pengalaman saya semakin menyenangkan. Tidak hanya itu, layanan pelanggan yang cepat tanggap juga menjadi nilai tambah yang luar biasa Ultra88 Gacor
I should insist seldom of which it is amazing! The blog is usually info likewise generally fabricate wonderful entitys. money6x.com save money
Ones texts within this theme usually are accurate, observe how When i had written this page is very top notch. coyyn.com digital money
Beaver states that When i in addition have like fascination, you possibly can understand the page in this article: crypto30x.com bitcoin price
I am able to provides you with the target In this article understand the best way to practice it the right way. Understand in addition to generate anything beneficial. crypto30x.com tnt
AI is transforming digital marketing by optimizing campaigns, enhancing personalization, and driving data-driven decisions. Discover the Best AI Tools for Digital Marketing to stay ahead in the evolving landscape.
If you’re passionate about skincare and beauty, earning an esthetician degree can open doors to a rewarding career in the beauty industry. This degree provides in-depth training in skincare treatments, facials, makeup application, and advanced techniques to enhance clients’ appearance and confidence. With an esthetician degree, you can work in luxury spas, dermatology clinics, or even start your own beauty business. Investing in professional education ensures you gain the skills and certification needed to succeed in this growing field.
If you’re passionate about beauty and skincare, La’ James International College is the perfect place to start your journey. With top-tier esthetics programs, hands-on training, and experienced instructors, La’ James International College provides the skills and knowledge needed to excel in the beauty industry. Whether you’re looking to become a licensed esthetician or expand your expertise in skincare treatments, this institution offers a comprehensive education to help you succeed in your career.
“Wikifeet: Exploring the Fascination with Celebrity Feet” takes an in-depth look at the phenomenon of celebrity foot fetishes, with a focus on the popular website
WikiFeet. The platform allows users to upload and rate pictures of celebrities’ feet, sparking a blend of fascination and curiosity within online communities. This exploration delves into the psychology behind foot fetishes, the cultural significance of celebrity worship, and how digital spaces have become a place for niche interests to thrive. By examining the appeal of celebrity feet, the piece uncovers the broader dynamics of online fandoms, voyeurism, and the intersection of celebrity culture with internet subcultures.
Very great post. I simply stumbled upon your weblog and wanted
to mention that I have really loved browsing your blog posts.
In any case I’ll be subscribing on your rss feed and
I am hoping you write again soon!
Feel free to surf to my web page – купить ручку паркер
I truly thank you for the profitable information on this awesome subject and anticipate more incredible posts. Much obliged for getting a charge out of this excellence article with me. I am valuing it all that much! Anticipating another awesome article. Good fortunes to the creator! All the best! Ibercaja empresas
HCOOCH CH2 H2O Uses Benefits and Role in Daily Life. March 13, 2025 by saqi … dousisnews.com is an informational blog covering the latest News
HCOOCH CH2 H2O
Hi88 – Cổng game giải trí hàng đầu, mang đến trải nghiệm cá cược đỉnh cao với đa dạng trò chơi hấp dẫn và dịch vụ uy tín.
Beluga99