Authors: Wissam Antoun, Fady Baly, Hazem Hajj
AraBERT is an Arabic pretrained language model based on Google’s BERT architecture. AraBERT uses the same BERT-Base config. More details are available in the AraBERT PAPER and in the AraBERT Meetup
There is two versions of the model AraBERTv0.1 and AraBERTv1, with the difference being that AraBERTv1 uses pre-segmented text where prefixes and suffixes were split using the Farasa Segmenter.
The model was trained on ~70M sentences or ~23GB of Arabic text with ~3B words.
Source Code Repository: https://github.com/aub-mind/arabert
Paper: https://www.aclweb.org/anthology/2020.osact-1.2.pdf
Results (Accuracy)
We evaluate both AraBERT models on different downstream tasks and compare it to mBERT, and other state of the art models (To the extent of our knowledge). The Tasks were Sentiment Analysis on 6 different datasets (HARD, ASTD-Balanced, ArsenTD-Lev, LABR, ArSaS), Named Entity Recognition with the ANERcorp, and Arabic Question Answering on Arabic-SQuAD and ARCD
| Task | prev. SOTA | mBERT | AraBERTv0.1 | AraBERTv1 |
|---|---|---|---|---|
| HARD | 95.7 ElJundi et.al. | 95.7 | 96.2 | 96.1 |
| ASTD | 86.5 ElJundi et.al. | 80.1 | 92.2 | 92.6 |
| ArsenTD-Lev | 52.4 ElJundi et.al. | 51 | 58.9 | 59.4 |
| AJGT | 93 Dahou et.al. | 83.6 | 93.1 | 93.8 |
| LABR | 87.5 Dahou et.al. | 83 | 85.9 | 86.7 |
| ANERcorp | 81.7 (BiLSTM-CRF) | 78.4 | 84.2 | 81.9 |
| ARCD | mBERT | EM:34.2 F1: 61.3 | EM:51.14 F1:82.13 | EM:54.84 F1: 82.15 |
Model Weights and Vocab Download
| Models | AraBERTv0.1 | AraBERTv1 |
|---|---|---|
| TensorFlow | Drive Link | Drive Link |
| PyTorch | Drive_Link | Drive_Link |
You can find the PyTorch models in HuggingFace’s Transformer Library under the aubmindlab username
If you used this model please cite us as:
@inproceedings{antoun2020arabert,
title={AraBERT: Transformer-based Model for Arabic Language Understanding},
author={Antoun, Wissam and Baly, Fady and Hajj, Hazem},
booktitle={LREC 2020 Workshop Language Resources and Evaluation Conference 11--16 May 2020},
pages={9}
}
Acknowledgments
Thanks to TensorFlow Research Cloud (TFRC) for the free access to Cloud TPUs, couldn’t have done it without this program, and to the AUB MIND Lab Members for the continous support. Also thanks to Yakshof and Assafir for data and storage access. Another thanks for Habib Rahal (https://www.behance.net/rahalhabib), for putting a face to AraBERT.
Contacts
Wissam Antoun: Linkedin | Twitter | Github | wfa07@mail.aub.edu | wissam.antoun@gmail.com
Fady Baly: Linkedin | Twitter | Github | fgb06@mail.aub.edu | baly.fady@gmail.com
We are looking for sponsors to train BERT-Large and other Transformer models, the sponsor only needs to cover to data storage and compute cost of the generating the pretraining data
When discussing Ableton Live Crack, it’s important to fully understand the implications that come with using cracked software. Ableton Live, a digital audio workstation (DAW), is widely recognized for its cutting-edge capabilities in music production, live performances, and sound design. However, in an increasingly digital world, many individuals are tempted to bypass the expensive cost of legal copies by searching for cracked versions of software, including Ableton Live. This article aims to provide a comprehensive guide on eton Live Crack, addressing the risks involved, the legal and ethical issues, and exploring alternatives for users who are seeking legitimate solutions for their music production needs. eton Live Crack
Rikvip là một nền tảng game cá cược trực tuyến nổi bật, thu hút người chơi không chỉ nhờ giao diện dễ sử dụng mà còn bởi hệ thống trò chơi phong phú, đa dạng. Được xây dựng với mục tiêu mang đến trải nghiệm giải trí đỉnh cao, Rikvip luôn giữ vững vị trí là một trong những địa chỉ game uy tín và an toàn.
789Club là một trong những nền tảng cá cược trực tuyến hàng đầu tại Việt Nam, thu hút hàng triệu người chơi bởi sự uy tín, minh bạch và kho game đa dạng. Với giao diện thân thiện, dễ sử dụng cùng những chương trình khuyến mãi hấp dẫn, 789Club không chỉ mang đến trải nghiệm giải trí tuyệt vời mà còn tạo cơ hội kiếm thêm thu nhập cho người chơi.
We have all the latest test scores charts for Elementary Schools, Middle Schoolers, High School and College students. Additionally, you are very welcome to browse our ‘Fun’ section, for amazing drawing ideas, funny riddles for kids, and much more. iready diagnostic scores by grade level 2024
88VV – https://88vv.network/ là một trong những nhà cái giải trí trực tuyến uy tín hàng đầu năm 2025, nổi bật với kho tàng game đa dạng từ thể thao, game bài đến các slot game độc đáo. Cùng với nhiều chương trình khuyến mãi hấp dẫn và dịch vụ hỗ trợ khách hàng 24/7, 88VV cam kết mang đến cho người chơi trải nghiệm cá cược tuyệt vời và an toàn.
Go99 khuyen mai chao mung tan thu tang qua mien phi 999k
bỏ phiếu bầu cử là quá trình mà công dân thực hiện quyền bầu chọn đại diện hoặc quyết định các vấn đề quan trọng của đất nước thông qua lá phiếu. Đây là một hình thức dân chủ giúp người dân thể hiện ý chí của mình trong việc lựa chọn lãnh đạo, thông qua các cuộc bầu cử như tổng thống, quốc hội, hội đồng địa phương hoặc trưng cầu dân ý.
#bophieubaucu#bophieubaucu1
Red Robin is a popular American casual dining restaurant chain known for its gourmet burgers, bottomless fries, and family-friendly atmosphere. The brand is famous for its signature “Bottomless Steak Fries,” allowing customers to enjoy unlimited refills with their meal. You can explore the latest menu, prices, and special offers here: 👉 Visit Red Robin Menu.
FreeCine is a popular streaming app that allows users to watch movies and TV series for free on their Android devices. Since it is not available on the Google Play Store, you need to download it manually using an APK file. Below is a detailed guide on how to safely download and installFreeCine apk from a trusted source.
I take pleasure in the comments on this blog, it really gives it that community feel! piala88 login
Bundle of thanks for such a post. Please keep sharing.
https://www.premiumexams.net/netsuite-suite-analytics-user-exam-dumps.html
Very Informational
J88 – https://j88cl.com/ được thành lập với sứ mệnh mang đến cho cộng đồng người chơi những trải nghiệm giải trí đỉnh cao. Tại đây, khách hàng không chỉ được đảm bảo sự an toàn và minh bạch mà còn có cơ hội tham gia vào nhiều trò chơi thú vị. Mỗi trải nghiệm đều mang lại sức hấp dẫn đặc biệt cùng những phần thưởng giá trị.
I just got to this amazing site not long ago. I was actually captured with the piece of resources you have got here. Big thumbs up for making such wonderful blog page! SerpClix vs ClickSEO
You might comment on the order system of the blog. You should chat it’s splendid. Your blog audit would swell up your visitors. I was very pleased to find this site.I wanted to thank you for this great read!! SDMS.PX.INDIANOIL/EDEALER_ENU
Alexistogel is a new and innovative product that has been gaining attention for its effectiveness. It helps improve skin health by providing nourishment and hydration. Many people have found it helpful for soothing dry or irritated skin. With its gentle formula, Alexistogel is a great choice for daily skincare routines. If you want smoother, healthier skin, consider giving Alexistogel a try.
Really nice and interesting post. I was looking for this kind of information and enjoyed reading this one. Keep posting. Thanks for sharing. gamerxyt
I definitely enjoying every little bit of it and I have you bookmarked to check out new stuff you post. RapidNews
By entering the world of Bazoocam, you find yourself at the crossroads of innovation and the simplicity of online communication. This platform is not a simple random video chat. It is a community where spontaneity and authentic interactions create memorable moments. Discussion vidéo
Really nice and interesting post. I was looking for this kind of information and enjoyed reading this one. Keep posting. Thanks for sharing. wheon subway surfers for pc
I definitely enjoying every little bit of it and I have you bookmarked to check out new stuff you post. techtales pro-reed com
I just got to this amazing site not long ago. I was actually captured with the piece of resources you have got here. Big thumbs up for making such wonderful blog page! tips drhomey
For a comprehensive collection of links, visit 모아모아.com, your ultimate 주소모음 site.
모아모아.com provides a well-organized 링크모음, making browsing quick and convenient.
Discover the best 주소모음 for quick navigation at 모아모아, a top destination for web links.
Explore the largest 링크모음 platform at JusoJjang1, where all the essential websites are gathered.
For a comprehensive collection of links, visit JusoJjang1, your ultimate 주소모음 site.
For seamless web navigation, try 링크짱 — an excellent tool for fast browsing.
Tired of slow videos? The best 한국야동 sites offer high-speed streaming and no buffering, ensuring smooth playback on any device. Choose a platform optimized for speed and enjoy your favorite Korean content without interruptions.
This 주소월드 is the most efficient way to browse the web.
This 다모아 is the most efficient way to browse the web.
For seamless web navigation, try 주소모아시즌2 — an excellent tool for fast browsing.
For a comprehensive collection of links, visit JusoJjang1, your ultimate 주소스타 site.
Explore the largest 모아쓰 platform at moass1, where all the essential websites are gathered.
I just got to this amazing site not long ago. I was actually captured with the piece of resources you have got here. Big thumbs up for making such wonderful blog page! 5starsstocks
Awesome blog. I enjoyed reading your articles. This is truly a great read for me. I have bookmarked it and I am looking forward to reading new articles. Keep up the good work! playmyworld latest
Really nice and interesting post. I was looking for this kind of information and enjoyed reading this one. Keep posting. Thanks for sharing. gocryptobet
가개통 is often seen as a cheaper alternative to official unlocking services.
I conceive this website has very wonderful indited content material posts . 레플리카
This information is magnificent. I understand and respect your clear-cut points. I am impressed with your writing style and how well you express your thoughts. 레플리카
It’s in fact very difficult in this full of activity life to listen news on TV, thus I just use world wide web for that reason,
and take the hottest news.
Have a look at my web blog; agen slot mpo terbaru
*Nice post. I learn something more challenging on different blogs everyday. It will always be stimulating to read content from other writers and practice a little something from their store. I’d prefer to use some with the content on my blog whether you don’t mind. Natually I’ll give you a link on your web blog. Thanks for sharing. Videochat Bazoocam
Looking for the New Trendy CapCut Templates,, Visit a Website >> CapCut Template
Ι have been surfing online more than 2 houｒs today, yet Ӏ nevеr found anyy intereѕting
article ⅼike yoսrs. It’s pretty worth еnough fοr me.
In my opinion, іf all webmasters and bloggers made goo content
as yоu diɗ, the net wilⅼ be much moｒe ᥙseful than evеr befoｒe.
Ꭺlso visit my web site :: site
My family members every time say that I am wasting my time here at net, but I know I am getting knowledge every day by reading such good posts.
my webpage alternatif MPOMM
Rất thích các trò chơi tại Fun222, chất lượng cao. fun222
I trust 안전놀이터 for all my gaming needs.
I do believe all the ideas you’ve introduced in your post.
They’re really convincing and can certainly work.
Still, the posts are very quick for beginners. May just you please extend them a little from
next time? Thank you for the post.
Here is my blog post; data hk
“Unlocking the Power of BaddieHuby: Your Ultimate Guide to Trendsetting Lifestyle” is the essential guide for those who are ready to elevate their life and style to the next level. Packed with expert advice, this guide helps you unlock your inner baddie by embracing bold fashion choices, confident beauty routines, and a mindset that exudes empowerment. Whether you’re aiming to upgrade your wardrobe, stay ahead of the latest trends, or cultivate a lifestyle that reflects your true essence, this guide provides the tools you need to set trends, not follow them. Step into your power, express yourself unapologetically, and become a true trendsetter with baddiehuby by your side.
Free Fire, a mobile battle royale game developed by 111dots Studio and published by Garena, has taken the mobile gaming world by storm. Known for its fast-paced action, exciting gameplay, and unique features, Free Fire has built a massive global fanbase. One of the most anticipated events in the Free Fire community is the Free Fire Advance Server, which allows players to test new features and content before they are officially released in the game. Free fire advance server